remote/navidrome
1
0
Fork 0
mirror of https://github.com/navidrome/navidrome.git synced 2026-11-05 17:58:37 +01:00
Code Issues Projects Releases Packages Wiki Activity
navidrome/db/migrations
History
Deluan 1c83f36ddf perf(persistence): make album list pages index-served 
The two album list requests reported in discussion #5929 (a 110k-album,
two-library instance) were dominated by work that scaled with the whole
library instead of the page:

- Every native API list request also runs a count for the X-Total-Count
  header. album had no index covering library_id, so that count scanned the
  entire table on every request, dragging each row's JSON blobs through the
  page cache. count(distinct id) additionally built a temp b-tree over the
  full result set even when the query had no join that could duplicate rows.
- A random sort cannot be served by an index, so SQLite evaluated SEEDEDRAND
  for every matching row and pushed each one, with its library and annotation
  joins and full column projection, through a sorter that kept only 36 rows.

Add an album(id, library_id) index so library-filtered counts are served by a
covering index scan, and make the shared count() helper use count(*) when the
rendered query has no join, keeping count(distinct id) wherever a join could
fan out rows. id leads the index deliberately: with library_id leading, the
planner drives sorted list queries off it and sorts an entire library rather
than walking a sort-satisfying index, and fresh statistics do not change that
choice.

Resolve random pages by id before hydrating them, so the sort pass runs on
that covering index with no joins and only the page itself is materialized.
Pages larger than 1000 keep the single-query path. The seeded shuffle is
unchanged: the id query carries the caller's options, so SEEDEDRAND, explicit
seeds, and the reseed-only-at-offset-0 rule all still apply, and the page is
reordered in Go to preserve it.

Also make the album name sort a total order by appending album.id, and widen
album_order_album_name to (order_album_name, order_album_artist_name, id) to
serve it. Without a tiebreaker, equal names made page order plan-dependent,
which can skip or duplicate rows across paginated requests.
2026-08-10 15:41:22 -04:00
..
20200130083147_create_schema.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200131183653_standardize_item_type.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200208222418_add_defaults_to_annotations.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200220143731_change_duration_to_float.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200310171621_enable_search_by_albumartist.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200310181627_add_transcoding_and_player_tables.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200319211049_merge_search_into_main_tables.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200325185135_add_album_artist_id.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200326090707_fix_album_artists_importing.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200327193744_add_year_range_to_album.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200404214704_add_indexes.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200409002249_enable_search_by_tracks_artists.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200411164603_add_created_and_updated_fields_to_playlists.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200418110522_reindex_to_fix_album_years.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200419222708_reindex_to_change_full_text_search.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200423204116_add_sort_fields.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200508093059_add_artist_song_count.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200512104202_add_disc_subtitle.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200516140647_add_playlist_tracks_table.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200608153717_referential_integrity.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200706231659_add_default_transcodings.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200710211442_add_playlist_path.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200731095603_create_play_queues_table.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200801101355_create_bookmark_table.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20200819111809_drop_email_unique_constraint.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201003111749_add_starred_at_index.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201010162350_add_album_size.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201012210022_add_artist_playlist_size.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201021085410_add_mbids.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201021093209_add_media_file_indexes.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201021135455_add_media_file_artist_index.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201030162009_add_artist_info_table.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201110205344_add_comments_and_lyrics.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201128100726_add_real-path_option.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20201213124814_add_all_artist_ids_to_album.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210322132848_add_timestamp_indexes.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210418232815_fix_album_comments.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210430212322_add_bpm_metadata.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210530121921_create_shares_table.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210601231734_update_share_fieldnames.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210616150710_encrypt_all_passwords.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210619231716_drop_player_name_unique_constraint.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210623155401_add_user_prefs_player_scrobbler_enabled.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210625223901_add_referential_integrity_to_user_props.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210626213026_add_scrobble_buffer.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210715151153_add_genre_tables.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20210821212604_add_mediafile_channels.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20211008205505_add_smart_playlist.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20211023184825_add_order_title_to_media_file.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20211026191915_unescape_lyrics_and_comments.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20211029213200_add_userid_to_playlist.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20211102215414_add_alphabetical_by_artist_index.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20211105162746_remove_invalid_artist_ids.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20220724231849_add_musicbrainz_release_track_id.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20221219112733_add_album_image_paths.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20221219140528_remove_cover_art_id.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230112111457_add_album_paths.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230114121537_touch_playlists.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230115103212_create_internet_radio.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230117155559_add_replaygain_metadata.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230117180400_add_album_info.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230119152657_recreate_share_table.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230202143713_change_path_list_separator.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230209181414_change_image_files_list_separator.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230310222612_add_download_to_share.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230404104309_empty_sql_migration.sql Fix migration package name mismatch 2024-05-08 19:54:48 -04:00
20230515184510_add_release_date.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20230616214944_rename_musicbrainz_recording_id.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20231208182311_add_discs_to_album.go Fix migration package name mismatch 2024-05-08 19:54:48 -04:00
20231209211223_alter_lyric_column.go fix(scanner): stop logging expected lyrics sniff misses as warnings (#5702) 2026-07-02 09:46:57 -04:00
20240122223340_add_default_values_to_null_columns.go.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20240426202913_add_id_to_scrobble_buffer.go Fix migration package name mismatch 2024-05-08 19:54:48 -04:00
20240511210036_add_sample_rate.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20240511220020_add_library_table.go feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) 2025-02-19 20:35:17 -05:00
20240629152843_remove_annotation_id.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20240802044339_player_use_user_id_over_username.go Use userId in player, other fixes (#3182) 2024-08-03 13:37:21 -04:00
20241020003138_add_sort_tags_index.sql perf(server): add index for sort tags. 2024-10-19 20:46:54 -04:00
20241024125533_fix_sort_tags_collation_and_index.sql fix(server): fix case-insensitive sort order and add indexes to improve performance (#3425) 2024-10-26 14:06:34 -04:00
20241026183640_support_new_scanner.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20250522013904_share_user_id_on_delete.sql fix(db): allow deleting users that have shares (#4098) 2025-05-21 22:16:10 -04:00
20250611010101_playqueue_current_to_index.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20250701010101_add_folder_hash.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20250701010102_add_annotation_user_foreign_key.sql fix(db): add user foreign key constraint to annotation table (#4211) 2025-06-13 17:27:57 -04:00
20250701010103_add_library_stats.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20250701010104_make_replaygain_fields_nullable.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20250701010105_remove_dangling_items.sql fix(db): clear dangling music from BFR upgrade (#4262) 2025-06-28 18:43:11 -04:00
20250701010106_add_participant_stats_to_all_artists.sql feat(server): expose main credit stat to reflect only album artist | artist credit (#4268) 2025-06-28 19:00:13 -04:00
20250701010107_add_mbid_indexes.sql feat: MBID search functionality for albums, artists and songs (#4286) 2025-06-30 17:11:54 -04:00
20250701010108_add_multi_library_support.go feat: Multi-library support (#4181) 2025-07-18 18:41:12 -04:00
20250823142158_make_playqueue_position_int.sql fix(db): make playqueue position field an integer (#4481) 2025-11-06 14:41:09 -05:00
20251109010105_add_annotation_rating_date.sql feat: add Rated At field - #4653 (#4660) 2025-11-24 23:18:05 -05:00
20251206013022_create_scrobbles_table.sql feat(server): track scrobble/linstens history (#4770) 2025-12-06 11:07:18 -05:00
20260104203627_playlist_case_insensitive_name.sql fix(ui): make playlist name sorting case-insensitive (#4845) 2026-01-05 19:05:11 -05:00
20260106000620_create_plugin_table.sql feat(plugins): New Plugin System with multi-language PDK support (#4833) 2026-01-14 19:22:48 -05:00
20260117201522_add_avg_rating_column.sql feat(subsonic): Add avgRating from subsonic spec (#4900) 2026-01-18 17:42:42 -05:00
20260220173400_add_fts5_search.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20260228020813_add_plugin_allow_write_access.sql feat(plugins): allow mounting library directories as read-write (#5122) 2026-02-28 10:59:13 -05:00
20260228172956_add_playlist_image_file.go feat(playlist): add custom playlist cover art upload (#5110) 2026-03-01 14:07:18 -05:00
20260302021413_rename_playlist_image_fields.go feat(playlist): support #EXTALBUMARTURL directive and sidecar images (#5131) 2026-03-02 11:39:59 -05:00
20260307175815_add_codec_and_update_transcodings.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20260309120007_fix_probe_data_null.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20260309203355_ensure_default_transcodings.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20260310113858_fix_aac_transcode_command.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20260315233131_add_artist_uploaded_image.go feat: add artist image uploads and image-folder artwork source (#5198) 2026-03-15 22:19:55 -04:00
20260316000000_normalize_timestamps.sql fix(db): normalize timestamps and fix recently added album sorting (#5176) 2026-03-16 07:55:22 -04:00
20260318182414_add_radio_uploaded_image.go feat(ui): add cover art support for internet radio stations (#5229) 2026-03-18 18:57:33 -04:00
20260405124200_fix_schema_inconsistencies.sql fix(db): resolve schema inconsistencies in library_artist and scrobble_buffer tables (#5047) 2026-04-05 12:56:32 -04:00
20260410201914_fix_zero_album_created_at.sql fix(subsonic): always emit required created field on AlbumID3 (#5340) 2026-04-10 19:29:20 -04:00
20260513173954_move_ss_before_input.go refactor(migrations): propagate context.Context through all DB calls 2026-06-18 09:58:43 -04:00
20260520211813_add_media_file_artists_composite_index.sql fix(smartplaylists): optimize smart playlist performance for role and tag criteria (#5515) 2026-05-22 18:00:13 -03:00
20260612171826_add_media_file_missing_library_id_index.sql fix(opensubsonic): make search3 empty-query pagination fast at large offsets (#5601) 2026-06-12 15:53:37 -04:00
20260612222838_make_bpm_bitdepth_nullable.sql feat(smartplaylist): extend isMissing/isPresent to bpm, bitDepth and many text fields (#5603) 2026-06-13 13:15:20 -04:00
20260618120509_add_metadata_to_default_transcodings.go fix(transcoding): preserve source metadata when transcoding downloads (#5628) 2026-06-18 09:58:43 -04:00
20260629123100_recently_added_plain_indexes.sql fix(subsonic): make "recently added" order reproducible and consistent with RecentlyAddedByModTime (#5678) 2026-06-29 16:18:29 -04:00
20260702152457_backfill_artist_search_normalized.go fix(search): artists with atomic non-ASCII names unfindable after FTS5 migration (#5703) 2026-07-02 12:53:10 -04:00
20260703013908_optimize_media_file_sort_indexes.sql perf(db): index media_file album/artist sort orders (#5706) 2026-07-03 08:58:55 -04:00
20260712211040_add_primary_key_and_update_index_for_scrobbles.sql feat(server): add scrobble history Native API (#5761) 2026-07-13 11:32:03 -04:00
20260714120000_add_playlist_average_rating.sql feat(playlists): per-user starred/rating annotations (backend) (#5749) 2026-07-14 07:38:25 -04:00
20260714123822_add_media_file_title_sort_covering_index.sql feat(jellyfin): experimental Jellyfin Music API support (#5730) 2026-07-14 11:46:37 -04:00
20260719005427_add_album_replaygain.go fix(db): make album ReplayGain backfill scale to large libraries 2026-07-18 22:59:53 -04:00
20260720015443_uniform_canonical_ids.go refactor(db): migrate all ids to a uniform canonical 128-bit base62 encoding (#5824) 2026-08-02 12:58:53 -04:00
20260722023032_add_artwork_tables.sql feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847) 2026-08-09 15:03:27 -04:00
20260810143445_reorder_album_artists_unique_constraint.sql perf(persistence): use *_artists join tables for artist participant filters (#5930) 2026-08-10 11:42:27 -04:00
20260810171732_add_album_library_id_index.sql perf(persistence): make album list pages index-served 2026-08-10 15:41:22 -04:00
id_canonical_test.go refactor(db): migrate all ids to a uniform canonical 128-bit base62 encoding (#5824) 2026-08-02 12:58:53 -04:00
id_columns_guard_test.go refactor(db): migrate all ids to a uniform canonical 128-bit base62 encoding (#5824) 2026-08-02 12:58:53 -04:00
migration.go perf(db): keep query planner statistics trustworthy with full ANALYZE (#5740) 2026-07-13 12:04:29 -04:00
migrations_suite_test.go refactor(db): migrate all ids to a uniform canonical 128-bit base62 encoding (#5824) 2026-08-02 12:58:53 -04:00
uniform_canonical_ids_test.go refactor(db): migrate all ids to a uniform canonical 128-bit base62 encoding (#5824) 2026-08-02 12:58:53 -04:00