mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-13 12:57:10 +02:00
5ec6e6a8d4* fix(subsonic): make search3 empty-query pagination fast at large offsets Empty-query search3 (used by clients like Symfonium to sync the whole library) degraded linearly with songOffset: the offset optimization in optimizePagination keeps the original query's LEFT JOINs (annotation, bookmark, library) inside its rowid NOT IN subquery, making it as slow as plain OFFSET (~5s per page at offset 900K on a 920K-track library). Rewrite the empty-query branch of doSearch to use the same two-phase approach as the FTS search: Phase 1 paginates rowids on the bare main table, which SQLite satisfies with a covering index at any offset; Phase 2 hydrates only the page's rows with all JOINs. The Phase 2 hydration logic is extracted into hydrateRowidPage, now shared with ftsSearch.execute. Also replace the media_file_missing index with a composite covering index on (missing, library_id), so Phase 1 stays covering for non-admin users, whose queries include a library_id filter. The composite serves all missing-only lookups via its prefix. With a 920K-track / 85K-album test library, search3 empty-query responses are now flat (~0.1s) at every offset, for both admin and non-admin users (previously 3-5s at offsets above 600K). * refactor(persistence): share search Phase 1 contract and dedup junction fan-out Extract the Phase 1 query assembly that was duplicated between the FTS search and the empty-query search into executeTwoPhase: both paths now supply only their strategy-specific FROM/JOINs and ORDER BY, while the shared contract (missing filter, library access, options.Filters, and Max/Offset semantics) lives in one place. Also fix a pagination integrity bug: the artist library filter joins the library_artist junction table, so an artist present in multiple libraries produced duplicate rowids in Phase 1, corrupting offset-based pagination (short pages and repeated artists during full-library syncs). Phase 1 now applies DISTINCT whenever a junction-based LibraryFilter is set. DISTINCT is used instead of GROUP BY because bm25() cannot be evaluated in a grouped query; plain-filter tables (media_file, album) skip the dedup so their Phase 1 keeps the streaming covering-index plan. This also fixes the same duplication in the pre-existing FTS search path. * fix(persistence): pin artist search Phase 1 join order with CROSS JOIN search3 always filters artists by library (library_artist.library_id IN ...), and with the junction JOIN in the search Phase 1 rowid query SQLite chose to drive from library_artist, sorting every junction row with a temp b-tree on each page — a flat ~200ms penalty per request at 405K artists, even at offset 0 (the previous code avoided this by accident: its GROUP BY artist.id pinned an artist-driven plan). Use CROSS JOIN (SQLite's explicit join-order override) in a search-only variant of the artist library filter, keeping artist as the outer table so Phase 1 streams rowids in artist.id order from the primary key index and LIMIT/OFFSET short-circuits. The DISTINCT dedup stays and costs nothing under the streaming plan. Other artist queries keep the planner's freedom. With 405K artists, empty-query artist search is now 0.07s at offset 0 and 0.25s at offset 399K end-to-end (was 0.31s/0.34s before this fix, and up to 1.2s on master at deep offsets). Artist FTS text search is unaffected. |
||
|---|---|---|
| .. | ||
| e2e | ||
| album_repository.go | ||
| album_repository_test.go | ||
| artist_repository.go | ||
| artist_repository_test.go | ||
| collation_test.go | ||
| criteria_sql.go | ||
| criteria_sql_benchmark_test.go | ||
| criteria_sql_test.go | ||
| export_test.go | ||
| folder_repository.go | ||
| folder_repository_test.go | ||
| genre_repository.go | ||
| genre_repository_test.go | ||
| helpers.go | ||
| helpers_test.go | ||
| library_repository.go | ||
| library_repository_test.go | ||
| mediafile_repository.go | ||
| mediafile_repository_test.go | ||
| persistence.go | ||
| persistence_suite_test.go | ||
| persistence_test.go | ||
| player_repository.go | ||
| player_repository_test.go | ||
| playlist_repository.go | ||
| playlist_repository_test.go | ||
| playlist_track_repository.go | ||
| playqueue_repository.go | ||
| playqueue_repository_test.go | ||
| plugin_cleanup.go | ||
| plugin_cleanup_test.go | ||
| plugin_repository.go | ||
| plugin_repository_test.go | ||
| property_repository.go | ||
| property_repository_test.go | ||
| radio_repository.go | ||
| radio_repository_test.go | ||
| scrobble_buffer_repository.go | ||
| scrobble_buffer_repository_test.go | ||
| scrobble_repository.go | ||
| scrobble_repository_test.go | ||
| share_repository.go | ||
| share_repository_test.go | ||
| smart_playlist_repository.go | ||
| smart_playlist_repository_test.go | ||
| sql_annotations.go | ||
| sql_annotations_test.go | ||
| sql_base_repository.go | ||
| sql_base_repository_test.go | ||
| sql_bookmarks.go | ||
| sql_bookmarks_test.go | ||
| sql_participations.go | ||
| sql_restful.go | ||
| sql_restful_test.go | ||
| sql_search.go | ||
| sql_search_fts.go | ||
| sql_search_fts_test.go | ||
| sql_search_like.go | ||
| sql_search_like_test.go | ||
| sql_search_test.go | ||
| sql_tags.go | ||
| tag_library_filtering_test.go | ||
| tag_repository.go | ||
| tag_repository_test.go | ||
| transcoding_repository.go | ||
| transcoding_repository_test.go | ||
| user_props_repository.go | ||
| user_repository.go | ||
| user_repository_test.go | ||