remote/navidrome
1
0
Fork 0
mirror of https://github.com/navidrome/navidrome.git synced 2026-10-11 03:47:18 +02:00
Code Issues Projects Releases Packages Wiki Activity
navidrome/persistence
History
Exact
Deluan 1c83f36ddf perf(persistence): make album list pages index-served 
The two album list requests reported in discussion #5929 (a 110k-album,
two-library instance) were dominated by work that scaled with the whole
library instead of the page:

- Every native API list request also runs a count for the X-Total-Count
  header. album had no index covering library_id, so that count scanned the
  entire table on every request, dragging each row's JSON blobs through the
  page cache. count(distinct id) additionally built a temp b-tree over the
  full result set even when the query had no join that could duplicate rows.
- A random sort cannot be served by an index, so SQLite evaluated SEEDEDRAND
  for every matching row and pushed each one, with its library and annotation
  joins and full column projection, through a sorter that kept only 36 rows.

Add an album(id, library_id) index so library-filtered counts are served by a
covering index scan, and make the shared count() helper use count(*) when the
rendered query has no join, keeping count(distinct id) wherever a join could
fan out rows. id leads the index deliberately: with library_id leading, the
planner drives sorted list queries off it and sorts an entire library rather
than walking a sort-satisfying index, and fresh statistics do not change that
choice.

Resolve random pages by id before hydrating them, so the sort pass runs on
that covering index with no joins and only the page itself is materialized.
Pages larger than 1000 keep the single-query path. The seeded shuffle is
unchanged: the id query carries the caller's options, so SEEDEDRAND, explicit
seeds, and the reseed-only-at-offset-0 rule all still apply, and the page is
reordered in Go to preserve it.

Also make the album name sort a total order by appending album.id, and widen
album_order_album_name to (order_album_name, order_album_artist_name, id) to
serve it. Without a tiebreaker, equal names made page order plan-dependent,
which can skip or duplicate rows across paginated requests.
2026-08-10 15:41:22 -04:00
..
e2e feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
album_repository.go perf(persistence): make album list pages index-served 2026-08-10 15:41:22 -04:00
album_repository_test.go perf(persistence): make album list pages index-served 2026-08-10 15:41:22 -04:00
artist_repository.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artist_repository_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artwork_hydration.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artwork_hydration_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artwork_queue_repository.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artwork_queue_repository_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artwork_repository.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
artwork_repository_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
collation_test.go perf(db): index media_file album/artist sort orders (#5706) 2026-07-03 08:58:55 -04:00
criteria_sql.go feat(smartplaylist): add album-level fields for sorting and filtering (#5899) 2026-08-06 10:59:50 -04:00
criteria_sql_benchmark_test.go perf(smartplaylists): merge negated artist/tag rules into one NOT EXISTS 2026-06-14 10:47:11 -04:00
criteria_sql_test.go feat(smartplaylist): add album-level fields for sorting and filtering (#5899) 2026-08-06 10:59:50 -04:00
export_test.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
folder_repository.go fix(jellyfin): stream collection responses to prevent OOM on large libraries (#5783) 2026-07-15 14:07:00 -04:00
folder_repository_test.go fix(jellyfin): stream collection responses to prevent OOM on large libraries (#5783) 2026-07-15 14:07:00 -04:00
genre_repository.go refactor: multiple syntax updates for Go 1.26 2026-05-19 18:02:36 -03:00
genre_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
helpers.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
helpers_test.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
library_repository.go perf(db): keep query planner statistics trustworthy with full ANALYZE (#5740) 2026-07-13 12:04:29 -04:00
library_repository_test.go perf(subsonic): speed up artist search3 deep-offset pagination (#5620) 2026-06-16 21:47:15 -04:00
mediafile_repository.go perf(persistence): use *_artists join tables for artist participant filters (#5930) 2026-08-10 11:42:27 -04:00
mediafile_repository_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
persistence.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
persistence_suite_test.go fix(jellyfin): stream collection responses to prevent OOM on large libraries (#5783) 2026-07-15 14:07:00 -04:00
persistence_test.go fix(server): play queue should not return empty entries for deleted tracks 2024-09-20 11:22:37 -04:00
player_repository.go fix(share): enforce per-user ownership on share reads 2026-06-05 15:50:59 -04:00
player_repository_test.go fix(share): enforce per-user ownership on share reads 2026-06-05 15:50:59 -04:00
playlist_repository.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
playlist_repository_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
playlist_track_repository.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
playlist_track_repository_test.go fix(jellyfin): close the unbounded playlist and search paths left by #5783 (#5784) 2026-07-15 18:09:59 -04:00
playqueue_repository.go refactor: multiple syntax updates for Go 1.26 2026-05-19 18:02:36 -03:00
playqueue_repository_test.go fix(server): ensure single record per user by reusing existing playqueue ID 2025-06-11 17:26:13 -04:00
plugin_cleanup.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
plugin_cleanup_test.go feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
plugin_repository.go fix(plugins): clear plugin errors on startup to allow retrying 2026-03-02 08:56:56 -05:00
plugin_repository_test.go fix(plugins): clear plugin errors on startup to allow retrying 2026-03-02 08:56:56 -05:00
property_repository.go Replace beego/orm with dbx (#2693) 2023-12-09 13:52:17 -05:00
property_repository_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
radio_repository.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
radio_repository_test.go feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
scrobble_buffer_repository.go fix(plugins): discard buffered scrobbles when a plugin is removed (#5737) 2026-07-08 12:37:17 -04:00
scrobble_buffer_repository_test.go fix(plugins): discard buffered scrobbles when a plugin is removed (#5737) 2026-07-08 12:37:17 -04:00
scrobble_repository.go feat(server): add scrobble history Native API (#5761) 2026-07-13 11:32:03 -04:00
scrobble_repository_test.go feat(server): add scrobble history Native API (#5761) 2026-07-13 11:32:03 -04:00
share_repository.go fix(playlist/share): apply user library access to import and sharing paths (#5640) 2026-06-23 18:53:51 -04:00
share_repository_test.go fix(playlist/share): apply user library access to import and sharing paths (#5640) 2026-06-23 18:53:51 -04:00
smart_playlist_repository.go feat(smartplaylist): add album-level fields for sorting and filtering (#5899) 2026-08-06 10:59:50 -04:00
smart_playlist_repository_test.go feat(smartplaylist): add album-level fields for sorting and filtering (#5899) 2026-08-06 10:59:50 -04:00
sort_index_coverage_test.go perf(db): index media_file album/artist sort orders (#5706) 2026-07-03 08:58:55 -04:00
sql_annotations.go feat(playlists): per-user starred/rating annotations (backend) (#5749) 2026-07-14 07:38:25 -04:00
sql_annotations_test.go perf(db): skip annotation join in CountAll when unused (#5694) 2026-06-30 22:50:43 -04:00
sql_base_repository.go perf(persistence): make album list pages index-served 2026-08-10 15:41:22 -04:00
sql_base_repository_test.go perf(persistence): make album list pages index-served 2026-08-10 15:41:22 -04:00
sql_bookmarks.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
sql_bookmarks_test.go revert: separation of write and read DBs 2024-11-19 18:41:50 -05:00
sql_participations.go perf(persistence): use *_artists join tables for artist participant filters (#5930) 2026-08-10 11:42:27 -04:00
sql_restful.go fix(nativeapi): make /api/song path filter work and use startsWith (#5566) 2026-06-05 14:00:08 -04:00
sql_restful_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search.go perf(subsonic): speed up artist search3 deep-offset pagination (#5620) 2026-06-16 21:47:15 -04:00
sql_search_fts.go feat(search): rank exact matches above prefix matches (#5704) 2026-07-02 15:51:03 -04:00
sql_search_fts_test.go feat(search): rank exact matches above prefix matches (#5704) 2026-07-02 15:51:03 -04:00
sql_search_like.go chore: go fix 2026-05-28 22:13:05 -03:00
sql_search_like_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_search_test.go feat(subsonic): sort search3 results by relevance (#5086) 2026-02-23 08:51:54 -05:00
sql_tags.go feat(jellyfin): experimental Jellyfin Music API support (#5730) 2026-07-14 11:46:37 -04:00
tag_library_filtering_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
tag_repository.go feat(jellyfin): filter items by year and record label (#5817) 2026-07-19 12:39:31 -04:00
tag_repository_test.go refactor: run Go modernize (#5002) 2026-02-08 09:57:30 -05:00
transcoding_repository.go fix: restrict transcoding config reads to admins (#5564) 2026-06-04 23:07:13 -04:00
transcoding_repository_test.go fix: restrict transcoding config reads to admins (#5564) 2026-06-04 23:07:13 -04:00
user_props_repository.go Replace beego/orm with dbx (#2693) 2023-12-09 13:52:17 -05:00
user_repository.go fix(nativeapi): make /api/song path filter work and use startsWith (#5566) 2026-06-05 14:00:08 -04:00
user_repository_test.go fix(nativeapi): make /api/song path filter work and use startsWith (#5566) 2026-06-05 14:00:08 -04:00