navidrome/persistence
Deluan Quintão 385e75e9a9
perf(db): skip annotation join in CountAll when unused (#5694)
* perf(persistence): skip annotation join in CountAll when unused

The Native API list endpoints (/api/song, /api/album, /api/artist) issue a
pagination count on every request via rest.GetAll. CountAll unconditionally
added a LEFT JOIN on the annotation table to a count(distinct id) query. The
join's columns are stripped by count(), but the distinct-over-join forced
SQLite to stream and dedup every row, making the count dominate the request
time on large libraries (e.g. ~190ms cold for 95k songs, seconds for
non-admin users behind the library subquery).

Gate the annotation join: only add it when a filter actually references an
annotation column. The need is detected by rendering the query to SQL and
matching annotation column names as whole words, which covers both named
filters (starred, has_rating) and raw squirrel filters. The column set is
derived from model.Annotations so it tracks schema changes; average_rating is
excluded because it lives on the base table, and word-boundary matching keeps
it from matching the annotation column rating.

Counts are unchanged; only the query plan changes. Unfiltered song counts drop
from ~37ms to ~4ms (warm) on a 95k-song library.

* test(persistence): make annotation-join detection case-insensitive

Address review feedback: SQLite column names are case-insensitive, so a raw
filter using e.g. "RATING" would previously evade the case-sensitive column
regex and wrongly drop the annotation join. Add the (?i) flag (average_rating
stays excluded — the underscore still prevents a word boundary before rating)
and cover it with mixed-case tests. Also make the starred-count assertion an
exact value instead of a range.
2026-06-30 22:50:43 -04:00
..
e2e fix(smartplaylist): support isMissing/isPresent operators on ReplayGain fields (#5585) 2026-06-10 21:12:31 -04:00
album_repository.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
album_repository_test.go fix(subsonic): make "recently added" order reproducible and consistent with RecentlyAddedByModTime (#5678) 2026-06-29 16:18:29 -04:00
artist_repository.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
artist_repository_test.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
collation_test.go fix(ui): make playlist name sorting case-insensitive (#4845) 2026-01-05 19:05:11 -05:00
criteria_sql.go perf(smartplaylist): use annotation index for playcount/rating/loved filters (#5662) 2026-06-24 22:26:41 -04:00
criteria_sql_benchmark_test.go perf(smartplaylists): merge negated artist/tag rules into one NOT EXISTS 2026-06-14 10:47:11 -04:00
criteria_sql_test.go perf(smartplaylist): use annotation index for playcount/rating/loved filters (#5662) 2026-06-24 22:26:41 -04:00
export_test.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
folder_repository.go fix(scanner): import playlists skipped when no admin existed yet (#5609) 2026-06-14 13:39:16 -04:00
folder_repository_test.go fix(scanner): import playlists skipped when no admin existed yet (#5609) 2026-06-14 13:39:16 -04:00
genre_repository.go refactor: multiple syntax updates for Go 1.26 2026-05-19 18:02:36 -03:00
genre_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
helpers.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
helpers_test.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
library_repository.go perf(subsonic): speed up artist search3 deep-offset pagination (#5620) 2026-06-16 21:47:15 -04:00
library_repository_test.go perf(subsonic): speed up artist search3 deep-offset pagination (#5620) 2026-06-16 21:47:15 -04:00
mediafile_repository.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
mediafile_repository_test.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
persistence.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
persistence_suite_test.go refactor: multiple syntax updates for Go 1.26 2026-05-19 18:02:36 -03:00
persistence_test.go fix(server): play queue should not return empty entries for deleted tracks 2024-09-20 11:22:37 -04:00
player_repository.go fix(share): enforce per-user ownership on share reads 2026-06-05 15:50:59 -04:00
player_repository_test.go fix(share): enforce per-user ownership on share reads 2026-06-05 15:50:59 -04:00
playlist_repository.go fix(playlists): allow toggling auto-import and avoid unnecessary artwork reloads (#5421) 2026-04-27 12:20:27 -04:00
playlist_repository_test.go refactor: centralize criteria sort parsing and extract smart playlist logic (#5415) 2026-04-26 14:49:59 -04:00
playlist_track_repository.go refactor: move playlist business logic from repositories to service layer (#5027) 2026-02-21 19:57:13 -05:00
playqueue_repository.go refactor: multiple syntax updates for Go 1.26 2026-05-19 18:02:36 -03:00
playqueue_repository_test.go fix(server): ensure single record per user by reusing existing playqueue ID 2025-06-11 17:26:13 -04:00
plugin_cleanup.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
plugin_cleanup_test.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
plugin_repository.go fix(plugins): clear plugin errors on startup to allow retrying 2026-03-02 08:56:56 -05:00
plugin_repository_test.go fix(plugins): clear plugin errors on startup to allow retrying 2026-03-02 08:56:56 -05:00
property_repository.go Replace beego/orm with dbx (#2693) 2023-12-09 13:52:17 -05:00
property_repository_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
radio_repository.go feat(ui): add cover art support for internet radio stations (#5229) 2026-03-18 18:57:33 -04:00
radio_repository_test.go refactor: more warnings clean up 2026-05-20 17:43:12 -03:00
scrobble_buffer_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
scrobble_buffer_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
scrobble_repository.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
scrobble_repository_test.go feat(server): track scrobble/linstens history (#4770) 2025-12-06 11:07:18 -05:00
share_repository.go fix(playlist/share): apply user library access to import and sharing paths (#5640) 2026-06-23 18:53:51 -04:00
share_repository_test.go fix(playlist/share): apply user library access to import and sharing paths (#5640) 2026-06-23 18:53:51 -04:00
smart_playlist_repository.go refactor: centralize criteria sort parsing and extract smart playlist logic (#5415) 2026-04-26 14:49:59 -04:00
smart_playlist_repository_test.go fix(smartplaylists): coerce string booleans in smart playlist rules (#5450) 2026-05-01 19:21:48 -04:00
sql_annotations.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
sql_annotations_test.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
sql_base_repository.go fix(share): enforce per-user ownership on share reads 2026-06-05 15:50:59 -04:00
sql_base_repository_test.go fix(server): headless library access improvements (#4362) 2025-07-20 15:58:21 -04:00
sql_bookmarks.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
sql_bookmarks_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
sql_participations.go fix(scanner): remove stale role associations when artist role changes. Fix #4242 2025-12-16 06:38:50 -05:00
sql_restful.go fix(nativeapi): make /api/song path filter work and use startsWith (#5566) 2026-06-05 14:00:08 -04:00
sql_restful_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search.go perf(subsonic): speed up artist search3 deep-offset pagination (#5620) 2026-06-16 21:47:15 -04:00
sql_search_fts.go fix(opensubsonic): make search3 empty-query pagination fast at large offsets (#5601) 2026-06-12 15:53:37 -04:00
sql_search_fts_test.go fix(search): normalization for non-NFKD Unicode letters (ø, æ, œ, ß) (#5413) 2026-04-25 20:27:38 -04:00
sql_search_like.go chore: go fix 2026-05-28 22:13:05 -03:00
sql_search_like_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_tags.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
tag_library_filtering_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
tag_repository.go feat(scanner): improve error messages for cleanup operations in annotations, bookmarks, and tags 2025-11-20 09:27:42 -05:00
tag_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
transcoding_repository.go fix: restrict transcoding config reads to admins (#5564) 2026-06-04 23:07:13 -04:00
transcoding_repository_test.go fix: restrict transcoding config reads to admins (#5564) 2026-06-04 23:07:13 -04:00
user_props_repository.go Replace beego/orm with dbx (#2693) 2023-12-09 13:52:17 -05:00
user_repository.go fix(nativeapi): make /api/song path filter work and use startsWith (#5566) 2026-06-05 14:00:08 -04:00
user_repository_test.go fix(nativeapi): make /api/song path filter work and use startsWith (#5566) 2026-06-05 14:00:08 -04:00