navidrome/utils
Deluan Quintão 3e55886195
feat: add optional natural sort order for names and titles (#6015)
* feat: add optional natural sort order for names and titles

Album, artist, song and playlist lists sort with a plain text comparison, so
names containing numbers come out as "Foo 1, Foo 10, Foo 2" instead of
"Foo 1, Foo 2, Foo 10" (issue #4554).

Adds an EnableNaturalSorting option, default off, that switches those sorts to
a NATSORT collation registered on every connection and backed by
natural.CompareFold. natural.Compare gained an ASCII case-folding variant
because it replaces 'collate nocase': sort_* columns hold raw tag values, so
without folding they would order uppercase before lowercase.

Applying the collation only inside mapSortOrder would have missed the default
configuration entirely, since that mapper runs only when PreferSortTags is on.
setSortMappings now also rewrites the order_* columns when natural sorting is
enabled on its own. Sorts over plain text columns that are not order_* columns
(playlist.name, album.name, media_file.title, playlist_tracks title) are
wrapped explicitly, and qualified with their table because 'user' is joined and
also has a 'name' column.

The option defaults to off because the collation cannot use the existing
indexes: measured on a synthetic 110k album library, the first page of an
album-by-name listing goes from 0.03ms to 14ms. Indexing the expression was
rejected outright - an index declared with a custom collation makes the whole
database unreadable to any tool that does not register it, including the
sqlite3 CLI, which fails even on 'select count(*)' and 'pragma integrity_check'.

* refactor: fold the two sort-order mappers into one

mapSortOrder and mapNaturalOrder shared the same regex and loop, differing only
in the expression they substituted, and setSortMappings picked between them with
a two-case switch. mapSortOrder now selects the column shape itself and defers to
collatedSort for the collation, so the 'collate' clause is emitted in one place
and the caller only has to decide whether any mapping is needed at all.

The mapper tests were three near-identical cases that each hard-coded one flag
combination; they are now a DescribeTable covering all four combinations of
PreferSortTags and EnableNaturalSorting, which the previous set did not. The
album sorting specs collapse the same way. Behavior is unchanged.

* fix: leave plain sort columns alone when natural sorting is off

collatedSort wrapped its column unconditionally, so the tiebreakers added for
plain text columns picked up 'collate nocase' even with EnableNaturalSorting
off. media_file.title, the playlist_tracks alias of it, and user.user_name are
all declared without a collation, so a default install would have silently
switched those tiebreaks from binary to case-insensitive ordering. Only
playlist.name was already NOCASE and genuinely unaffected.

The helper is now naturalSort and returns the column untouched unless the option
is on, so the default path keeps the collation each column was declared with.
sortCollation had a single remaining caller and folded into mapSortOrder.

Tests: the CompareFold table body was a verbatim copy of the Compare one, so
both now go through one expectOrder helper, and the album sorting specs inline
two single-use closures.

* fix(natural): defer the leading-zero tie-break to keep ordering transitive

Compare applied the padding difference between numerically equal digit runs only
when one side ended at the digit boundary, and ignored it mid-string. That made
the relation intransitive: CompareFold("1","1a") < 0 and CompareFold("1a","01a")
== 0, yet CompareFold("1","01a") > 0.

SQLite requires a collating function to be transitive and leaves ORDER BY
undefined otherwise, so registering this as NATSORT was not safe. Reproduced with
the real driver on three artist names that occur in practice - "3", "3 doors
down" and "03 greedo" - where paging one row at a time returned "03 greedo"
twice and dropped "3" entirely.

The padding difference is now carried as a tie-break that is applied only when
the strings are otherwise equal, which restores transitivity while keeping the
documented intent (a01 < a1, a0 < a00). Three existing entries changed: each
asserted that two distinct strings compare equal, which was the same defect seen
from the other side.

Found by the Codex review on #6015.
2026-08-23 14:31:37 -04:00
..
cache fix(cache): write the completion marker before closing the cache writer (#5927) 2026-08-10 14:10:05 -04:00
chrono refactor: multiple syntax updates for Go 1.26 2026-05-19 18:02:36 -03:00
gg feat(subsonic): add structured sidecar lyrics support with OpenSubsonic v2 karaoke cues and agent layers (#5076) 2026-06-19 12:00:58 -04:00
gravatar feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
hasher refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
ioutils fix: handle UTF BOM in lyrics and playlist files (#4637) 2025-10-31 09:07:23 -04:00
jsoncommentstrip refactor(jsoncommentstrip): replace go-jsoncommentstrip with custom JSON comment stripping 2026-03-14 10:18:56 -04:00
merge Upgrade Go to 1.23 (#3190) 2024-08-19 17:47:54 -04:00
nanoid refactor(nanoid): replace gonanoid with custom nanoid implementation for ID generation 2026-03-13 21:06:26 -04:00
natural feat: add optional natural sort order for names and titles (#6015) 2026-08-23 14:31:37 -04:00
number chore(deps): remove direct dependency on golang.org/x/exp 2025-12-31 17:03:44 -05:00
pl chore: upgrade golangci-lint to 2.11 and fix lint issues 2026-03-06 19:23:47 -05:00
random refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
req refactor(server): drop redundant error return from req.Strings parsing (#5812) 2026-07-18 19:30:04 -04:00
run refactor: rename chain package to run and update references 2025-06-14 17:19:06 -04:00
shellquote refactor(shellquote): replace go-shellquote with custom shell quoting implementation 2026-03-14 10:23:45 -04:00
singleton refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
slice perf(subsonic): speed up artist search3 deep-offset pagination (#5620) 2026-06-16 21:47:15 -04:00
str fix(search): artists with atomic non-ASCII names unfindable after FTS5 migration (#5703) 2026-07-02 12:53:10 -04:00
context.go Refactored agents calling into its own struct 2021-06-08 17:00:02 -04:00
context_test.go Upgrade Ginkgo to V2 2022-07-26 16:53:17 -04:00
encrypt.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
encrypt_test.go Upgrade Ginkgo to V2 2022-07-26 16:53:17 -04:00
files.go feat(playlist): support #EXTALBUMARTURL directive and sidecar images (#5131) 2026-03-02 11:39:59 -05:00
files_test.go ci: run Go tests on Windows (#5380) 2026-04-19 13:16:47 -04:00
index_group_parser.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
index_group_parser_test.go Upgrade Ginkgo to V2 2022-07-26 16:53:17 -04:00
limiter.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
time.go feat(smartplaylist): per-playlist refreshDelay for stable daily/weekly playlists (#5790) 2026-07-16 22:20:35 -04:00
time_test.go feat(smartplaylist): per-playlist refreshDelay for stable daily/weekly playlists (#5790) 2026-07-16 22:20:35 -04:00
utils_suite_test.go Upgrade Ginkgo to V2 2022-07-26 16:53:17 -04:00