mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-09 02:47:29 +02:00
54de0dbc52* build: add sqlite_fts5 build tag to enable FTS5 support
* feat: add SearchBackend config option (default: fts)
* feat: add buildFTS5Query for safe FTS5 query preprocessing
* feat: add FTS5 search backend with config toggle, refactor legacy search
- Add searchExprFunc type and getSearchExpr() for backend selection
- Rename fullTextExpr to legacySearchExpr
- Add ftsSearchExpr using FTS5 MATCH subquery
- Update fullTextFilter in sql_restful.go to use configured backend
* feat: add FTS5 migration with virtual tables, triggers, and search_participants
Creates FTS5 virtual tables for media_file, album, and artist with
unicode61 tokenizer and diacritic folding. Adds search_participants
column, populates from JSON, and sets up INSERT/UPDATE/DELETE triggers.
* feat: populate search_participants in PostMapArgs for FTS5 indexing
* test: add FTS5 search integration tests
* fix: exclude FTS5 virtual tables from e2e DB restore
The restoreDB function iterates all tables in sqlite_master and
runs DELETE + INSERT to reset state. FTS5 contentless virtual tables
cannot be directly deleted from. Since triggers handle FTS5 sync
automatically, simply skip tables matching *_fts and *_fts_* patterns.
* build: add compile-time guard for sqlite_fts5 build tag
Same pattern as netgo: compilation fails with a clear error if
the sqlite_fts5 build tag is missing.
* build: add sqlite_fts5 tag to reflex dev server config
* build: extract GO_BUILD_TAGS variable in Makefile to avoid duplication
* fix: strip leading * from FTS5 queries to prevent "unknown special query" error
* feat: auto-append prefix wildcard to FTS5 search tokens for broader matching
Every plain search token now gets a trailing * appended (e.g., "love" becomes
"love*"), so searching for "love" also matches "lovelace", "lovely", etc.
Quoted phrases are preserved as exact matches without wildcards. Results are
ordered alphabetically by name/title, so shorter exact matches naturally
appear first.
* fix: clarify comments about FTS5 operator neutralization
The comments said "strip" but the code lowercases operators to
neutralize them (FTS5 operators are case-sensitive). Updated comments
to accurately describe the behavior.
* fix: use fmt.Sprintf for FTS5 phrase placeholders
The previous encoding used rune('0'+index) which silently breaks with
10+ quoted phrases. Use fmt.Sprintf for arbitrary index support.
* fix: validate and normalize SearchBackend config option
Normalize the value to lowercase and fall back to "fts" with a log
warning for unrecognized values. This prevents silent misconfiguration
from typos like "FTS", "Legacy", or "fts5".
* refactor: improve documentation for build tags and FTS5 requirements
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor: convert FTS5 query and search backend normalization tests to DescribeTable format
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: add sqlite_fts5 build tag to golangci configuration
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add UISearchDebounceMs configuration option and update related components
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: fall back to legacy search when SearchFullString is enabled
FTS5 is token-based and cannot match substrings within words, so
getSearchExpr now returns legacySearchExpr when SearchFullString
is true, regardless of SearchBackend setting.
* fix: add sqlite_fts5 build tag to CI pipeline and Dockerfile
* fix: add WHEN clauses to FTS5 AFTER UPDATE triggers
Added WHEN clauses to the media_file_fts_au, album_fts_au, and
artist_fts_au triggers so they only fire when FTS-indexed columns
actually change. Previously, every row update (e.g., play count, rating,
starred status) triggered an unnecessary delete+insert cycle in the FTS
shadow tables. The WHEN clauses use IS NOT for NULL-safe comparison of
each indexed column, avoiding FTS index churn for non-indexed updates.
* feat: add SearchBackend configuration option to data and insights components
Signed-off-by: Deluan <deluan@navidrome.org>
* fix: enhance input sanitization for FTS5 by stripping additional punctuation and special characters
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add search_normalized column for punctuated name search (R.E.M., AC/DC)
Add index-time normalization and query-time single-letter collapsing to
fix FTS5 search for punctuated names. A new search_normalized column
stores concatenated forms of punctuated words (e.g., "R.E.M." → "REM",
"AC/DC" → "ACDC") and is indexed in FTS5 tables. At query time, runs of
consecutive single letters (from dot-stripping) are collapsed into OR
expressions like ("R E M" OR REM*) to match both the original tokens and
the normalized form. This enables searching by "R.E.M.", "REM", "AC/DC",
"ACDC", "A-ha", or "Aha" and finding the correct results.
* refactor: simplify isSingleUnicodeLetter to avoid []rune allocation
Use utf8.DecodeRuneInString to check for a single Unicode letter
instead of converting the entire string to a []rune slice.
* feat: define ftsSearchColumns for flexible FTS5 search column inclusion
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: update collapseSingleLetterRuns to return quoted phrases for abbreviations
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: implement extractPunctuatedWords to handle artist/album names with embedded punctuation
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor: punctuated word handling to improve processing of artist/album names
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: add CJK support for search queries with LIKE filters
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: enhance FTS5 search by adding album version support and CJK handling
Signed-off-by: Deluan <deluan@navidrome.org>
* refactor: search configuration to use structured options
Signed-off-by: Deluan <deluan@navidrome.org>
* feat: enhance search functionality to support punctuation-only queries and update related tests
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
|
||
|---|---|---|
| .. | ||
| 20200130083147_create_schema.go | ||
| 20200131183653_standardize_item_type.go | ||
| 20200208222418_add_defaults_to_annotations.go | ||
| 20200220143731_change_duration_to_float.go | ||
| 20200310171621_enable_search_by_albumartist.go | ||
| 20200310181627_add_transcoding_and_player_tables.go | ||
| 20200319211049_merge_search_into_main_tables.go | ||
| 20200325185135_add_album_artist_id.go | ||
| 20200326090707_fix_album_artists_importing.go | ||
| 20200327193744_add_year_range_to_album.go | ||
| 20200404214704_add_indexes.go | ||
| 20200409002249_enable_search_by_tracks_artists.go | ||
| 20200411164603_add_created_and_updated_fields_to_playlists.go | ||
| 20200418110522_reindex_to_fix_album_years.go | ||
| 20200419222708_reindex_to_change_full_text_search.go | ||
| 20200423204116_add_sort_fields.go | ||
| 20200508093059_add_artist_song_count.go | ||
| 20200512104202_add_disc_subtitle.go | ||
| 20200516140647_add_playlist_tracks_table.go | ||
| 20200608153717_referential_integrity.go | ||
| 20200706231659_add_default_transcodings.go | ||
| 20200710211442_add_playlist_path.go | ||
| 20200731095603_create_play_queues_table.go | ||
| 20200801101355_create_bookmark_table.go | ||
| 20200819111809_drop_email_unique_constraint.go | ||
| 20201003111749_add_starred_at_index.go | ||
| 20201010162350_add_album_size.go | ||
| 20201012210022_add_artist_playlist_size.go | ||
| 20201021085410_add_mbids.go | ||
| 20201021093209_add_media_file_indexes.go | ||
| 20201021135455_add_media_file_artist_index.go | ||
| 20201030162009_add_artist_info_table.go | ||
| 20201110205344_add_comments_and_lyrics.go | ||
| 20201128100726_add_real-path_option.go | ||
| 20201213124814_add_all_artist_ids_to_album.go | ||
| 20210322132848_add_timestamp_indexes.go | ||
| 20210418232815_fix_album_comments.go | ||
| 20210430212322_add_bpm_metadata.go | ||
| 20210530121921_create_shares_table.go | ||
| 20210601231734_update_share_fieldnames.go | ||
| 20210616150710_encrypt_all_passwords.go | ||
| 20210619231716_drop_player_name_unique_constraint.go | ||
| 20210623155401_add_user_prefs_player_scrobbler_enabled.go | ||
| 20210625223901_add_referential_integrity_to_user_props.go | ||
| 20210626213026_add_scrobble_buffer.go | ||
| 20210715151153_add_genre_tables.go | ||
| 20210821212604_add_mediafile_channels.go | ||
| 20211008205505_add_smart_playlist.go | ||
| 20211023184825_add_order_title_to_media_file.go | ||
| 20211026191915_unescape_lyrics_and_comments.go | ||
| 20211029213200_add_userid_to_playlist.go | ||
| 20211102215414_add_alphabetical_by_artist_index.go | ||
| 20211105162746_remove_invalid_artist_ids.go | ||
| 20220724231849_add_musicbrainz_release_track_id.go | ||
| 20221219112733_add_album_image_paths.go | ||
| 20221219140528_remove_cover_art_id.go | ||
| 20230112111457_add_album_paths.go | ||
| 20230114121537_touch_playlists.go | ||
| 20230115103212_create_internet_radio.go | ||
| 20230117155559_add_replaygain_metadata.go | ||
| 20230117180400_add_album_info.go | ||
| 20230119152657_recreate_share_table.go | ||
| 20230202143713_change_path_list_separator.go | ||
| 20230209181414_change_image_files_list_separator.go | ||
| 20230310222612_add_download_to_share.go | ||
| 20230404104309_empty_sql_migration.sql | ||
| 20230515184510_add_release_date.go | ||
| 20230616214944_rename_musicbrainz_recording_id.go | ||
| 20231208182311_add_discs_to_album.go | ||
| 20231209211223_alter_lyric_column.go | ||
| 20240122223340_add_default_values_to_null_columns.go.go | ||
| 20240426202913_add_id_to_scrobble_buffer.go | ||
| 20240511210036_add_sample_rate.go | ||
| 20240511220020_add_library_table.go | ||
| 20240629152843_remove_annotation_id.go | ||
| 20240802044339_player_use_user_id_over_username.go | ||
| 20241020003138_add_sort_tags_index.sql | ||
| 20241024125533_fix_sort_tags_collation_and_index.sql | ||
| 20241026183640_support_new_scanner.go | ||
| 20250522013904_share_user_id_on_delete.sql | ||
| 20250611010101_playqueue_current_to_index.go | ||
| 20250701010101_add_folder_hash.go | ||
| 20250701010102_add_annotation_user_foreign_key.sql | ||
| 20250701010103_add_library_stats.go | ||
| 20250701010104_make_replaygain_fields_nullable.go | ||
| 20250701010105_remove_dangling_items.sql | ||
| 20250701010106_add_participant_stats_to_all_artists.sql | ||
| 20250701010107_add_mbid_indexes.sql | ||
| 20250701010108_add_multi_library_support.go | ||
| 20250823142158_make_playqueue_position_int.sql | ||
| 20251109010105_add_annotation_rating_date.sql | ||
| 20251206013022_create_scrobbles_table.sql | ||
| 20260104203627_playlist_case_insensitive_name.sql | ||
| 20260106000620_create_plugin_table.sql | ||
| 20260117201522_add_avg_rating_column.sql | ||
| 20260220173400_add_fts5_search.go | ||
| migration.go | ||