mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-11 11:57:12 +02:00
25 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Deluan Quintão
|
b293b96256 |
refactor(persistence): stateless repositories with per-call context (#6149)
* refactor(persistence): adopt generic deluan/rest repository API Pin deluan/rest to the refactor branch. REST-facing repository methods take a context and return typed values. Drop DataStore.Resource and ResourceRepository; the native API names typed repositories directly through a per-request adapter that later commits remove. * refactor(persistence): base repository helpers take a context * refactor(persistence): LibraryRepository takes a context per call * refactor(persistence): PropertyRepository takes a context per call * refactor(persistence): UserPropsRepository takes a context per call * refactor(persistence): TranscodingRepository takes a context per call * refactor(persistence): ShareRepository takes a context per call * refactor(persistence): PlayerRepository takes a context per call * refactor(persistence): RadioRepository takes a context per call * refactor(persistence): PlayQueueRepository takes a context per call * refactor(persistence): Tag and Genre repositories take a context per call * refactor(persistence): PluginRepository takes a context per call * refactor(persistence): Scrobble repositories take a context per call * refactor(persistence): FolderRepository takes a context per call * refactor(persistence): Artwork repositories take a context per call * refactor(persistence): UserRepository takes a context per call * refactor(persistence): ArtistRepository takes a context per call ReadAll no longer rewrites the shared sort mappings for the role filter; it works on a per-call copy. * test(persistence): assert artist role sort sanitization in ReadAll * refactor(persistence): AlbumRepository takes a context per call * test(persistence): pass the test context to album repository helpers * refactor(persistence): MediaFileRepository takes a context per call * refactor(persistence): Playlist repositories take a context per call * refactor(persistence): build all repositories once per store * refactor(core): REST repository wrappers are built once * refactor(persistence): repositories are stateless Remove the context field from the base repository and the per-request REST adapter. Enable the containedctx linter so no repository can hold a request context again. * chore(lint): skip containedctx in test files * refactor: share simplifications from the stateless repositories sweep Add deleteOwnedAll on sqlRepository and use it in player/share Delete to remove the duplicated bulk-delete loop; have Share.Repository() return model.ShareRepository so subsonic sharing.go drops its repeated type assertions. * chore(core): assert REST wrappers implement Persistable * chore: reformat imports * perf(persistence): build repositories on first use Each transaction store used to construct all 21 repositories up front, paying for filter and sort mapping setup the block never touched. Fields are now sync.OnceValue thunks, so a store only builds what it uses. * fix(persistence): clean plugin references per deleted user A bulk user delete that fails on a later id had already removed the earlier rows but skipped their plugin cleanup. Cleanup now runs right after each successful delete. * fix(core): unload disabled plugins even when a user delete fails A bulk delete can fail on a later id after earlier users were removed and their plugins auto-disabled. The wrapper returned before unloading, leaving those plugins running until the next successful delete or a restart. * chore(deps): pin deluan/rest to v1.0.1 Replaces the pseudo-version of the refactor branch with the tagged release. REST error messages now name the bare type (Artist, not model.Artist). * test: use the spec context instead of context.Background() Replace the context.Background()/context.TODO() calls this branch added to tests with the spec's ctx, GinkgoT().Context(), or t/b.Context(), so repository calls are bound to the running spec's lifetime. * test: declare the spec context once per Describe Set ctx from GinkgoT().Context() first in each top-level BeforeEach and reuse it, building user contexts on top of it instead of repeating inline calls. |
||
|
Deluan Quintão
|
ee6dd1bc03 |
fix(scanner): stop DB lock starvation during scans on slow storage (#6201)
* fix(artwork): pause the artwork worker while a scan is running The artwork worker added in 0.64 writes to the database continuously, including while a scan runs. On slow storage the scanner holds the write lock for many seconds per folder, so the two writers keep timing each other out: artwork writes fail with "database is locked", and a single busy timeout on the scanner side aborts the whole scan. The worker now stops dispatching queue items while scanner.IsScanning reports true, including mid-batch, and resumes on the next poll after the scan ends. Artwork requests are unaffected, since they serve local art without the worker. * fix(db): run ANALYZE one index at a time so writers are not starved A full ANALYZE is a single write transaction, so every other write waits for it to finish and fails after the 15s busy timeout. On slow NAS storage it was measured taking over 26 minutes. The analysis now runs ANALYZE per index (per table for unindexed and WITHOUT ROWID tables), which produces the same sqlite_stat1 rows as a full ANALYZE, and pauses briefly between steps (up to 150ms, just above SQLite's longest busy-handler sleep) so waiting writers get the lock. * fix(scanner): ignore Synology @eaDir metadata folders Synology creates an @eaDir folder next to media files, holding one subfolder per file with generated thumbnails. The scanner and watcher treated them as regular folders, which on one reported library added tens of thousands of extra folders to every scan. * fix(db): analyze tables with only partial indexes as a whole A partial index does not record the table's row count, so a table whose only indexes are partial needs a table-level ANALYZE to get the sqlite_stat1 row a full ANALYZE would write. Navidrome's schema has no such table today, but the stepped analysis should match a full ANALYZE for any schema a future migration creates. * fix(scanner): retry busy folder saves and stop phase 1 on a fatal error On slow storage, a single SQLITE_BUSY while saving a folder aborted the whole scan, even when another writer held the lock only briefly. The folder save now runs as a retryable unit: on a busy error it waits (5s, 10s, 15s) and reruns the transaction, up to three times, before failing. Side effects that do not survive a rollback (the album ID map consumed by persistAlbum, the artwork queue items, the image-change record) are rebuilt per attempt or recorded only after a successful commit. When a folder save does fail, phase 1 used to keep walking the library and reading tags for every remaining folder, discarding the results, before reporting the error; a reporter saw 40 silent minutes. The walk now stops as soon as the save fails, and the walker honors cancellation instead of blocking on its channel. Because an early stop leaves folders unvisited, phase 1 no longer marks unvisited folders missing when the phase failed; the resumed scan handles them. * refactor(persistence): move busy retry into DataStore.WithTxRetry The scanner retried its folder save itself, which meant it had to know SQLite error codes. WithTxRetry now owns that policy: it reruns the block in a fresh transaction on SQLITE_BUSY, up to three times with growing delays, and runs it only once when already inside a transaction, since the outer transaction would still hold the lock. The block receives the context to use, and attempts that will be retried carry a marker so a busy statement in them is logged as a warning; only the final attempt logs errors. The scanner's inner error logs are folded into wrapped errors, so a recovered retry no longer prints error-level lines, and the folder path travels in the log context. * fix(persistence): join the enclosing transaction in a nested WithTxRetry Called on a store that is already inside a transaction, WithTxRetry went through WithTx, which opens a second, independent transaction on another connection. That transaction waits on the lock the outer one holds and fails with SQLITE_BUSY, and if it does succeed the outer transaction cannot roll it back. It now runs the block on the enclosing transaction, which owns the lock, the commit and the rollback. Found by a Codex (gpt-6-sol) review. * fix(scanner): retry the remaining scan writes on a busy database Every write step after phase 1 still aborted the whole scan on a single SQLITE_BUSY: phase 1 finalize, phase 2 moves and purge, phase 3 album saves and play count refreshes, the deferred playlist import flag, library ScanBegin, GC, the missing-artwork enqueue, tag counts, and the final library update. They now go through WithTxRetry. The phase 2 move had to be made rerun-safe first: it changed the target track's ID inside the transaction, so a rerun would have deleted the moved track itself, and it marked album annotations as handled even when the transaction rolled back. It now works on a copy per attempt and records the annotation reassignment only after a commit. Artist.RefreshStats is left alone: it updates artists in batches outside a transaction, and one transaction around all of them would hold the write lock for the whole refresh on slow storage. Phase 4 playlist imports go through the playlist service and are left for a follow-up. * fix(scanner): claim the album before moving its annotations The rerun-safe moveMatched checked processedAlbumAnnotations before its transaction and marked the album only after the commit. Phase 2 runs same-library and cross-library moves in separate pipeline stages, so two moves into one album could both pass the check; the second would reassign annotations again and overwrite the album's created_at. The album is now claimed under the lock before the transaction, as the old code effectively did, and the claim is released if the move fails so a later move can still reassign. Found by a Codex (gpt-6-sol) review. * fix(artwork): keep artwork housekeeping from writing during scans The artwork worker already pauses while a scan runs, but its housekeeping jobs did not: the hourly missing-artwork recheck (a bulk INSERT ... SELECT over albums and artists), the startup run of the same recheck, and the daily prune all kept competing with the scanner for the write lock. They now run through LockForMaintenance, like the scheduled DB analysis: they skip while a scan is running and keep a scan from starting until they finish. Skipping the recheck loses nothing, since each scan with changes queues missing artwork at its end. * refactor(scanner): log retried step errors once, from the caller Blocks passed to WithTxRetry still logged their own errors at error level on every attempt, so a busy error that a retry absorbed printed several error lines (GC printed three). They now return wrapped errors and the callers, which already log them, report the final outcome once. Also: drop a leftover variable in phase 1 finalize, check the walk context once, stop repeating the folder field that is already in the log context, stop shadowing finalize's err in phase 3, and format the WithTxRetry scope the same way as WithTx. * test(scanner): make the scanner suite's temp DB cleanup best effort Which DB file the process-wide DB handle opens depends on which spec touches it first. When the Scanner container wins the random order, its temp DB stays open until db.Close after RunSpecs, and on Windows removing the temp dir fails with 'being used by another process'. Ginkgo pins that on the container's last spec, which is now one of the busy-database specs. The sibling suites skip Windows for the same reason; this one now removes its temp dir on a best-effort basis instead, so it keeps running there. |
||
|
Deluan Quintão
|
9198bde34a |
test(scanner): fix Windows flake in the quick-scan artist image spec (#6093)
The spec asserted on artistID("Kraftwerk") and intermittently found zero artists
on Windows. The scan did import the artist; it was then made invisible.
RefreshStats selects touched artists with a strict artist.updated_at >
library.last_scan_at (persistence/artist_repository.go:466). Windows' wall clock
has ~15ms granularity, so a new artist written by a quick scan can land in the
same tick as the previous scan's last_scan_at and be excluded. Its
library_artist.stats then stays at the '{}' default and the unscoped cleanup
DELETE removes the row, after which selectArtist's INNER JOIN on library_artist
hides the artist from GetAll.
Backdate last_scan_at before the scan so the comparison is unambiguous, matching
the fix already applied to the search_normalized spec below it.
|
||
|
Deluan Quintão
|
5b758fc20c |
fix(artwork): re-resolve artwork when image files change on disk (#5965)
* fix(artwork): re-resolve artwork when image files change on disk An image-only folder change (replaced, added, or deleted cover/artist images, with no audio files touched) was detected by the scanner but never reached the artwork queue, so clients kept seeing the old coverArt hash until something else forced a re-resolution. Phase 1 now diffs each changed folder's image list and imagesUpdatedAt against the previously persisted folder row, and at the end of the phase bulk-enqueues re-resolution for the affected entities: albums with tracks in the folder or its direct children (covering disc subfolder layouts), and, when an artist-pattern image is involved, artists with albums under the folder's subtree, mirroring the artist resolver's upward search. The artist mapping mirrors the resolver's sole-album-artist album selection. New repository helpers keep the mapping set-based and light: folder GetAllIDs, media_file GetAlbumIDsByFolder (distinct, indexed by folder_id), and album GetSoleAlbumArtistIDs. * refactor: simplify the image-change artwork enqueue after review Load the previous folder image state through the existing GetFolderUpdateInfo bulk pre-pass instead of a per-folder SELECT inside the persist transaction, and skip the diff for new folders, whose artwork the scanner already enqueues inline. Move the artist-image classification into core/artwork (IsArtistImageFile) so the scanner shares the resolver's ArtistArtPriority token grammar instead of re-parsing it (the copy mistreated image-folder as a filename glob). Move the folder-subtree query into the folder repository (GetSubtreeIDs) with LIKE escaping and expression-tree batching, share the sole-album-artist predicate between the resolver and the album repository (model.SoleAlbumArtistFilter), extract a chunked single-column query helper, and deduplicate the ArtworkQueueItem literals behind scanArtworkItem. * fix(persistence): keep slash-form paths in GetSubtreeIDs subtree predicates The scanner hands GetSubtreeIDs io/fs slash-form paths, but filepath.Clean rewrites them with backslashes on Windows while folder.path is stored with forward slashes, so the descendant predicates matched nothing and nested artist folders were never re-enqueued there. Normalize with path.Clean, like HasAudioOutsideFolders does, and cover a nested path in the repo test. * refactor(persistence): move the sole-album-artist rule into the album repository SQLizer filters belong in the persistence package, not model. The rule becomes an unexported filter shared by GetSoleAlbumArtistIDs and a new GetBySoleAlbumArtist repository method, which the artist artwork resolver now calls instead of building the squirrel filter itself. * perf(scanner): resolve image-change artists in one query over album.folder_ids The artist half of the image-change enqueue walked folder subtree IDs, then media_file rows, then album rows, marshalling thousands of bound IDs through the driver on each hop. Matching albums by their own folder_ids instead is one statement, and folder_ids is the same source the artist resolver uses to compute an artist's folders. Benchmarked against a copy of the production DB (97k tracks, 10k folders, 7k albums): 87ms +/-196% -> 17.4ms +/-8%, 7.1MB -> 172KB, 103k -> 1.5k allocs. The subtree predicate becomes a shared folderSubtreeFilter, so Folder GetSubtreeIDs and Album GetSoleAlbumArtistIDs are no longer needed. * fix(scanner): persist ancestor folders discovered by a quick scan A quick scan skipped any new folder with no files of its own, so an artist folder holding only album subfolders never got a row. Adding artist.jpg to it later then produced no artwork enqueue: the entry was new, so the image diff was skipped, and it has no tracks, so nothing was enqueued inline either. Skip only genuinely empty new folders, matching what a full scan already persists. This also fixes artist artwork resolving as absent for artists first imported by a quick scan, since the resolver's folder climb needs that row. Also normalizes the selective-scan preload paths with path.Clean, so its descendant predicates match the stored slash-form paths on Windows. * fix(persistence): chunk subtree paths and match artist globs by basename Two regressions from earlier commits on this branch. Collapsing the subtree query into a single statement dropped the chunking the old GetSubtreeIDs had: each path expands into 3 OR terms and SQLite rejects an expression tree deeper than 1000, measured at 166 paths. A library with more artist-image folders than that (the prod copy has 158) would fail the whole collect, dropping the album items with it, so the scanner now keeps them when the artist query fails. The artist-image classifier compared whole tokens after stripping album/, so a directory-bearing glob like images/artist.* never matched the basenames the scanner has. Match on path.Base, which is what album/artist.* already reduced to; the resolver climbs parent folders, so an exact prefix is not knowable here and a conservative match is the right failure direction. * refactor(persistence): halve the repository surface this PR adds Research on the four new repository methods found two were avoidable. GetAlbumIDsByFolder now expands the changed folders to their direct children in its own subquery, so Folder.GetAllIDs has no callers and is deleted, one round trip per scan disappears, and the previously unchunked id/parent_id IN lists are covered by the existing chunking. GetBySoleAlbumArtist becomes an exported SoleAlbumArtistFilter, matching the ParticipantIDFilter precedent for sharing a Sqlizer with core/, so the rule still lives in persistence but AlbumRepository gains nothing and the mock shim that ignored the artist filter is gone. Also drops queryAllSliceChunked, now callerless, in favour of the file-local slices.Chunk convention used by the sibling folder queries. Rejected on measurement: matching the album path by album.folder_ids is exactly equivalent (13975 pairs, zero difference) but has no index, so it scans every album and runs 5-200x slower than the media_file route. * refactor(persistence): stop reading the deprecated album_artist_id column Both artist lookups this PR touches now go through participation, matching the precedent in core/archiver.go and share_repository.go. SoleAlbumArtistFilter uses ParticipantIDFilter, which is also faster: the album_artists unique constraint is a covering index for it, while the old column needed album_artist_album_id plus a row fetch. GetSoleAlbumArtistIDsInSubtrees reads the sole artist out of the participants JSON it already parses for the sole-artist check, rather than joining back to album_artists, which measured ~1.6x slower on a prod-sized copy. Verified equivalent on that copy: 6828 sole-artist albums and 1088 subtree artists resolve identically via the column, the join and the JSON. The tests now set a deliberately wrong album_artist_id so they fail if either query starts reading it again. * docs: trim comments that carry rationale belonging in commit messages Five comments had grown past the budget with benchmark numbers, rejected alternatives, and a duplicate of the constant's own explanation. * refactor(scanner): move the image-change enqueue into phase_1_folders The three functions were methods on phaseFolders, so they belong with the type; phase_1_image_changes.go also read like a fifth phase, which it wasn't. * refactor(scanner): extract the image-change collector into its own type phaseFolders no longer owns the per-library map and the mapping methods; it records into a collector and asks it to enqueue once. The collector keeps the library alongside the folders, so enqueue needs only ctx and the datastore. * refactor(scanner): simplify enqueue method by removing redundant datastore parameter Signed-off-by: Deluan <deluan@navidrome.org> * docs(scanner): drop the stale zero-value claim on imageChangeCollector The collector now takes its datastore at construction, so the zero value is no longer usable. * fix(scanner): pin the persist stage to concurrency 1 and guard the collector The stage relied on go-pipeline defaulting to one worker; stating it at the stage makes the constraint visible where someone would change it. The collector takes a mutex too, so the type is safe on its own terms rather than by configuration. --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
944ca3100f |
feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847)
* feat(artwork): add artwork, item_artwork and artwork_queue tables * feat(artwork): add artwork models, repository interfaces and mocks * feat(artwork): implement artwork repository * feat(artwork): implement item_artwork repository with batched hydration * feat(artwork): implement artwork_queue repository * feat(artwork): add content-addressed originals store * feat(artwork): add artwork prune (orphan cleanup) * fix(artwork): never sweep files on transient DB errors during prune * refactor(artwork): fold originals package into core/artwork as ImageStore * refactor(artwork): merge item artwork state into ArtworkRepository * fix(artwork): chunk unbounded IN clauses and restore interface docs * refactor(artwork): apply simplify-pass cleanups Internal item_artwork sqlRepository helper, toSQLArgs upserts, batched queue enqueue, EnqueueStaleAbsent moved to queue repo, snapshot-based prune sweep, mock/real semantics aligned. * fix(artwork): address review findings on prune/sweep races and mock fidelity Sweep now honors an mtime grace window (in-flight acquisitions and temp files), reacquired orphans reset the prune grace window, and the queue mock implements real stale-absent semantics. * fix(artwork): atomic orphan deletion and timestamp semantics from review DeleteOrphans re-checks age+references at delete time, PutItemArtwork defaults attempted_at, queue mock timestamps mirror SQL. * fix(artwork): guard orphan file removal with the prune grace window Duplicate ImageStore writes refresh the file mtime and Remove skips files newer than the cutoff, so overlapping acquisitions cannot lose their store files to a concurrent prune. * fix(artwork): rewrite vanished duplicates and sweep stale mime variants Write falls through to a real write when the liveness touch fails, and sweep retention now matches the recorded mime's extension so obsolete variants are reclaimed. * fix(artwork): index artwork_queue in dequeue order The previous leading retry_at range column forced a temp B-tree sort of the whole eligible set on every DequeueBatch; ordering the index by (priority DESC, enqueued_at) lets scans stop after the batch size. * fix(artwork): honor the orphan cutoff in the repository mock The mock's DeleteOrphans now applies createdBefore like the SQL implementation, and a new spec covers a freshly reacquired row surviving prune. * fix(artwork): reject malformed hashes in ImageStore operations Known-absent states carry an empty hash and malformed persisted hashes could panic path sharding or inject separators; Write/Open/Remove now return an error for anything but 16 lowercase hex chars. * fix(artwork): mock PutImage refreshes created_at like the SQL repository Prune specs now age fixtures directly instead of seeding stale timestamps through the upsert. * fix(artwork): store backing-file provenance per item, not per hash * feat(artwork): import blurhash encoder from #5797 * feat(artwork): add worker-side artwork resolvers * fix(artwork): propagate playlist tile failures and dedupe external step * feat(artwork): add acquisition processor Resolves one queue item end to end: hash/dedup, decode + 128px thumbnail blurhash, place bytes (store vs source file), and persist found/absent/ failed state for the worker (Task 4) to act on. * style(artwork): tighten processor comments to budget * feat(artwork): add acquisition worker service * feat(artwork): enqueue artwork resolution from scan and CRUD paths * feat(artwork): artwork backfill, fingerprint re-resolution and scheduled jobs * test(artwork): leak/soak coverage and deferred assertions * fix(artwork): propagate transient artist image errors to the worker callGetImage swallowed all agent errors, so an agent outage surfaced as ErrNotFound and the worker settled artist artwork as a definitive absent (and reset the breaker). Add an additive ArtistImageResult path that returns the underlying agent error on transient failure while keeping ArtistImage byte-identical for existing callers; the worker's artist external step uses it via fromArtistExternalResult. * fix(artwork): resolve full playlist source chain resolvePlaylist only built the generated grid, dropping the uploaded-image, sidecar and ExternalImageURL sources the old reader_playlist.go chain serves. Port the full chain before the grid fallback: uploaded (upload), sidecar (folder), and ExternalImageURL routed through extGate with the same extError semantics as the other external steps. Also rewires the artist external step onto ArtistImageResult. * fix(artwork): purge dangling queue rows and guard concurrent re-enqueues Queue rows for deleted entities failed forever (Get -> ErrNotFound -> failed -> capped retries, unbounded). Add ArtworkQueueRepository.PurgeDangling, called from Prune next to the item_artwork purge. Separately, the found/absent path unconditionally deleted the dequeued row, erasing a concurrent scan re-enqueue; switch to DeleteIfUnchanged, which deletes only while retry_at still matches the dequeued value (verified retry_at is the column an Enqueue upsert resets). * style(artwork): fix comment accuracy and budget; fingerprint ArtistImageFolder Correct the inverted workerDeps.extGate comment, trim over-budget doc comments, and add conf.Server.ArtistImageFolder to the resolution fingerprint so an image-folder change re-resolves artist artwork. * fix(artwork): treat missing local playlist cover as definitive, not transient A playlist ExternalImageURL pointing at a local file that fails to open was routed through extError, causing failed/48h-retry loops that burn a rate limiter token forever instead of falling through to the generated grid. * refactor(artwork): deduplicate purge loop, backfill table, and extGate alias * fix(artwork): cap resolved image reads A user-editable ExternalImageURL can point at an arbitrarily large endpoint; a fast server could make the worker buffer hundreds of MB inside the 5s HTTP timeout. Bound the read to a fixed 20MB cap (no config knob) via io.LimitReader and fail the item if it is exceeded. * fix(artwork): retry higher-priority external art after fallback hit With CoverArtPriority="external,cover.jpg", a transient external failure followed by a folder hit dropped the external error: the worker recorded found and deleted the queue row, so the configured higher-priority external art was never retried. Carry extError onto the fallback resolution and add an outcomeFoundStale that persists+serves the art but reschedules via MarkFailed, giving the external source another chance. When external later answers definitively-not-found, the hit is not stale and the row is deleted. * fix(artwork): treat playlist cover URL 404 as definitive miss The playlist ExternalImageURL step used sources.go's fromURL, which maps any non-200 to a generic error, so a stale URL returning 404/410 was classified transient: infinite backoff plus it counted toward the circuit breaker, blocking valid external work. Add a local fetch in resolve.go that maps 404/410 to model.ErrNotFound (definitive) while keeping other non-200s transient. sources.go is left untouched. * test(artwork): move soak test into the Ginkgo suite * test(artwork): make leak and permission tests pass on linux goleak now ignores notify's nonrecursive-tree goroutines (linux uses inotify, which spawns dispatch+internal instead of darwin's recursive dispatch), and the read-only-dir prune spec skips under root, where permission bits cannot make Remove fail. * fix(artwork): reject decompression-bomb dimensions before decoding * fix(artwork): keep fresh re-enqueues ahead of stale failure backoff * fix(artwork): include M3U external art flag in the config fingerprint * fix(artwork): resolve private playlists with an admin context * test(artwork): convert non-synctest timing tests to Ginkgo specs TestArtworkBackoffSchedule and TestArtworkWorkerRunNoLeak needed no real *testing.T (no synctest), so move them into worker_test.go as Ginkgo specs. TestArtworkBreakerHalfOpen stays plain since testing/synctest requires a real *testing.T, matching core/scrobbler's precedent. * fix(artwork): store backing-file provenance per item, not per hash * fix(artwork): apply image limits to playlist tile decoding decodeTile ran image.Decode on every sampled album's resolved bytes before processItem's maxImageBytes/maxImagePixels guards applied, letting an oversized or decompression-bomb tile fully decode unbounded. Enforce both caps inside decodeTile itself. * refactor(artwork): reuse auth.WithAdminUser and dedupe image cap guards * perf(artwork): fetch only IDs for backfill enumeration Backfill enumerated every album, artist, playlist and radio via GetAll and mapped out just the ID. GetAll materializes full entities (library joins, participant/stats/tags JSON, annotation, artwork hydration), so on a large library it loaded tens of thousands of heavy structs only to read one field each — spiking transient RSS to ~1GB during the one-time upgrade backfill, a memory risk on small NAS/Pi hardware. Add GetAllIDs to the album, artist, playlist and radio repositories: it reuses each repo's base row-set filter (library visibility, artist content join, playlist userFilter) but projects only id, skipping the heavy columns and post-processing. A per-repo parity test asserts GetAllIDs returns exactly the same id set as GetAll. Verified on a 727MB / 29k-artist production DB copy: peak RSS during backfill dropped from ~1012MB to ~89MB, file descriptors flat, same 36,138 items enqueued. * feat(artwork): promote worker concurrency and external rate to real configs The artwork worker's drain speed was governed by two hidden Dev flags, DevArtworkWorkerConcurrency and DevArtworkExternalRPS, both defaulting to 2. On a large library's one-time backfill the external rate limiter is the real ceiling: every art-less item waits on it before the (rate-limited) external lookup, so the drain crawls at ~RPS items/sec while local-art items are unaffected. Promote both to documented, supported options: ArtworkWorkerConcurrency (default 4) sets local-resolution parallelism, ArtworkExternalMaxRPS (default 2, 0 = unlimited) caps external-agent lookups to stay polite to Last.fm/Deezer/etc. Operators can now trade first-backfill speed against external-API rate limits. The old Dev names still map for backward compat. * fix(deezer): never return empty-image-id placeholder pictures * feat(agents): enumerate enabled image-retriever agents per capability * feat(artwork): worker fetches agent images directly with per-agent rate limits and breakers * fix(artwork): treat agent not-found as breaker success * feat(model): content-hash artwork id suffix and hydratable per-entity image state * feat(persistence): hydrate artwork hash and absence onto entity pages * feat(artwork): resolve media_file embedded art in the worker, invalidate on rescan * feat(artwork): broadcast refresh events when artwork lands * fix(artwork): broadcast refresh for stale-found artwork too * feat(artwork): state-backed serving path with provisional read-through * feat(server): serve artwork from persisted state with content-hash caching * feat(subsonic): content-hash coverArt ids, omit artwork on known-absent * refactor(artwork): delete the legacy reader chain, cache warmer, and provider image methods * feat(artwork): precache on acquisition, bump on upload/radio changes, manual re-resolve API * test(artwork): end-to-end coverage for the serving cutover * chore(artwork): generic 500 bodies on refresh endpoint, trim stale test comments * fix(artwork): request read-through must not reset the failure backoff The provisional read-through and dangling re-enqueue used Enqueue, whose upsert resets retry_at, so any browse of an unresolved entity that was backing off after an external failure made it immediately eligible again — defeating the exponential backoff during a provider outage. Add EnqueueBump, which raises priority but leaves an existing row's retry_at intact, and route the serving path through it. Scan and manual re-resolve keep Enqueue's reset (a detected change wants immediate retry). * fix(artwork): keep an eligible track's cover requestable when its album is absent An embedded-eligible track with no resolved item_artwork row inherited the album's ImageAbsent, so when the album resolved absent (e.g. CoverArtPriority without 'embedded') the track's coverArt was omitted permanently — the client never requested it, so the lazy mediafile path never resolved it — even though the serving path would extract and serve the track's own embedded art. Hydration now never copies the album's absence onto an eligible-but-unresolved track. * fix(artwork): validate each agent image URL before picking the largest bestImageURL selected the largest by size and only then parsed it, so a malformed largest URL (e.g. a bad percent-escape) returned nil and shadowed a valid smaller candidate, contradicting the documented skip-unparseable behavior. Parse per candidate and compare sizes only among URLs that parse. * fix(artwork): fall back to disc art, not the album, for multi-disc tracks serveMediaFile delegated an absent/ineligible track straight to AlbumCoverArtID, skipping the disc-specific lookup that MediaFile.CoverArtID (and the deleted legacy reader) use. On multi-disc albums with per-disc images that served the album cover instead of the configured disc artwork. Delegate through DiscCoverArtID. * fix(artwork): enqueue uploaded artwork only after the filename is persisted SetImage cleared state and enqueued the bump before the caller stored the new filename, so a worker drain in that window could resolve against the old (already deleted) file and settle absent, leaving the upload unused until a later scan. Move the invalidate+enqueue into EnqueueArtwork, which each caller now invokes after the entity Put. * fix(artwork): keep multi-disc tracks requestable when the album is absent Round-1's hydration fix still copied the album's known-absent onto a non-eligible (or own-absent) track, but MediaFile.CoverArtID routes a multi-disc track to disc art, which resolves provisionally and is never known-absent. Marking it absent made Subsonic omit coverArt so clients never requested a valid disc image. Only mark a single-disc track absent, and only when its own art won't resolve. * fix(artwork): serve a local playlist ExternalImageURL as a file-backed reference A local ExternalImageURL was resolved through the external step and labelled external, so placeBytes copied it into the content-addressed store and dropped its path/mtime — replacing the file never tripped the staleness check. Classify local references as file-backed (resolved in place, even on the request path) and keep store-backed behaviour only for http(s) URLs. * fix(artwork): requeue playlist cover when its track set changes A generated-grid cover went stale after track mutations: nothing re-resolved the playlist's artwork, and the request path deliberately never rebuilds the grid, so serveEntity kept returning the old grid hash indefinitely. Enqueue pl artwork from refreshCounters (the choke point for every track-set change); no clear, so the old cover keeps serving until the worker rebuilds. * fix(artwork): open library-backed artwork through its on-disk root A library configured with a file:// path stored absRoot as the raw URI, so Abs produced strings like file:/music/cover.jpg that os.Open/os.Stat reject — folder, upload and embedded art were treated as dangling on every request, looping forever. Normalize a file:// path to its parsed OS path (the same root os.DirFS uses); non-local schemes are left unchanged (out of scope, per the artwork-musicfs TODO). * fix(artwork): only use disc resolution for multi-disc albums DiscCoverArtID returns a dc- id for any track with DiscNumber>0, so serveDisc ran the full DiscArtPriority chain even for single-disc albums, where a stray disc*/ embedded image could shadow higher-priority album art. Gate disc resolution on the album having more than one disc, matching the legacy reader; single-disc tracks serve album art directly. * fix(artwork): invalidate artwork when an uploaded image is deleted Deleting an artist/radio/playlist upload cleared the filename but left the found item_artwork row and its hash, so lists kept advertising the deleted cover's hash-suffixed immutable URL and clients could display it indefinitely. Call EnqueueArtwork after the delete-side Put, symmetric with upload, so the state is cleared and re-resolved to the next source (or absent). * fix(artwork): restore synthetic-artist guard and unicode normalization in agent lookups Moving agent calls into the worker bypassed two behaviors of the aggregate provider: Agents.GetArtistImages' guard for Unknown/Various Artists (a direct retriever call could assign an unrelated image to a synthetic artist), and auxAlbum/auxArtist.Name's DevPreserveUnicodeInExternalCalls normalization (records with typographic quotes/dashes missed exact-name searches). Re-apply both before enumerating retrievers. * fix(artwork): enforce entity visibility on the Subsonic getCoverArt path serveEntity reads persisted item_artwork by id, bypassing the library and private- playlist filters that the legacy entity-load applied. On the authenticated Subsonic path a user could fetch artwork for an inaccessible album or someone else's private playlist by guessing an id. getCoverArt now resolves the underlying entity through the request-scoped (filtered) repositories and serves the placeholder when it is not visible, so existence isn't leaked and the always-an-image invariant holds. The public share (JWT-authorized) and Jellyfin (admin) paths are intentionally untouched. * fix(artwork): version the artwork ETag with the served representation The ETag was the pixel hash of the original image, so a CoverArtQuality or EnableWebPEncoding change altered the resized bytes without changing the ETag — revalidating clients got a spurious 304 and kept the old encoding. Resized responses now carry a representation ETag (hash + size + square + encode settings) used for the ETag header and If-None-Match, while the immutable decision stays on the pixel hash (URLs remain pixel-identity per the spec, so hash-suffixed clients keep zero-request caching). Full-size originals fall back to the pixel hash as before. * fix(artwork): don't stamp the album hash onto multi-disc tracks The hydration fallback assigned a found album hash to every fallback track, but a multi-disc track's CoverArtID emits a dc- id served from disc-specific art whose hash is unknown at hydration time. Advertising dc-..._<albumHash> gave clients a content- version that never changes when the disc image does, breaking id-based refresh. Only stamp the album hash for single-disc tracks (DiscNumber == 0); multi-disc tracks stay unhashed and rely on the correct ETag returned by the served response. * fix(artwork): enqueue new empty playlists by id, and refresh on absent outcomes Two worker/enqueue fixes from review: - playlistRepository.Put assigned the generated id to the caller's Playlist but passed the stale copy (empty id) to refreshCounters, enqueueing a pl|"" row the worker failed until the daily dangling purge while the real playlist went unresolved. Set the id on the copy before enqueueing. - The drain refresh batch only included found/foundStale, so a cover removed by a scan (found -> absent) never notified clients, leaving the old immutable image displayed. Broadcast absent outcomes too; precache still only warms found/foundStale. * fix(artwork): honor disabled per-track art at serve time; use nanosecond mtime provenance Two serving-correctness fixes from review: - serveMediaFile served a persisted mf embedded image even after EnableMediaFileCoverArt was turned off (the setting isn't in the config fingerprint, so found rows aren't reprocessed). Direct mf- URLs now honor the setting at serve time and fall back to disc/album art. - The file-backed staleness check compared whole-second mtimes, so a same-second content replacement (two writes in one second, or timestamp-preserving tools) could serve different bytes under the old hash + immutable policy. RefMtime is now unix-nanoseconds (no schema change; int64 column), detecting sub-second changes where the filesystem records them. * fix(artwork): preserve the drive when normalizing Windows file:// library paths url.Parse puts the volume of file://C:/Music in Host, not Path, so localOSRoot dropped it and returned /Music — os.Open/os.Stat then failed and folder/embedded art on Windows looped as dangling. Rejoin the host volume, matching core/storage/local's newLocalStorage. * fix(artwork): clamp negative sizes to full-size; convert imghttp test to Ginkgo - A negative size (Subsonic size / Jellyfin maxwidth accept signed ints) reached resizeStaticImage, where the square path builds image.NewNRGBA(Rect(0,0,size,size)) — a giant rectangle that panics/OOMs. Clamp size<0 to 0 (full-size) at the Service entry. Positive sizes were already clamped to the original. - imghttp used a plain func Test with a table; convert to a Ginkgo DescribeTable with the suite entry point in imghttp_suite_test.go (AGENTS.md test-framework requirement). * test(artwork): use renamed ArtworkWorkerConcurrency in e2e tests * feat(artwork): carry blurhash through item image hydration * feat(nativeapi): expose artwork hash, absence and blurhash * feat(artwork): hydrate the parent album's artwork state onto tracks * test(artwork): add hydrateArtwork regression guard for AlbumImage wiring Drives hydrateArtwork itself (not applyItemImage directly) over tracks that take each of the loop's continue branches, so a future edit moving the AlbumImage fill below a continue would fail loudly instead of passing silently. * feat(jellyfin): version album and artist image tags by content hash * fix(jellyfin): trim primaryImageTag comment to why-only, within budget * feat(jellyfin): emit real blurhashes and drop the synthesized fallback * feat(ui): version cover art urls by content hash and skip absent art * feat(ui): add BlurHashCanvas placeholder component * fix(ui): clear stale blurhash pixels and assert the draw path in tests Clear the canvas before each decode attempt so a hash change that fails to decode doesn't leave the previous frame's pixels on screen once this wires into a list that recycles items. Also strengthen the specs to assert createImageData/putImageData were actually invoked (and with what), instead of only checking that a <canvas> element exists. * feat(ui): show the blurhash while an album cover loads * fix(artwork): hydrate cursor streams via an id pre-pass The album, artist and playlist GetCursor built their own select and never called hydrateArtwork, so every Jellyfin list endpoint (all six stream via GetCursor) emitted entity-id image tags and no blurhash. Only GetAll hydrated, which is why Subsonic and the native API were unaffected. Each cursor now resolves its ordered/filtered/paginated id set with the cheap id-only GetAllIDs query, then streams those ids in chunks through the repo's existing GetAll, which already hydrates and applies the full select. Max/Offset are consumed by the pre-pass alone; the chunk query carries only the caller's filters, Sort and Order. This also removes a pre-existing deep-pagination cost: keeping OFFSET out of the joined query makes the pre-pass a covering index scan instead of paying the library and annotation joins for every skipped row. Benchmarked on a synthetic 100k-album DB with the real schema, page=500 at offset 90,000: 3.9ms via the id pre-pass, 52.5ms for the current shape, 192.7ms for a naive join. An unpaginated full stream costs ~24% more, which is the trade. GetAllIDs gains the annotation join whenever the caller's filters or sort reference an annotation column (same gate CountAll uses), otherwise Filters=IsFavorite and SortBy=PlayCount would fail in the pre-pass. The playlist pre-pass repeats GetAll's columns so ORDER BY keeps resolving to playlist.name rather than the joined user.name. * fix(jellyfin): hydrate artwork on the song cursor Jellyfin's listSongs streamed media files via GetCursor, which never hydrates artwork, so songs emitted entity-id image tags and no blurhash. media_file now uses the same id pre-pass as the other three cursors (album/artist/playlist), for consistency, but on a separate method, GetCursorWithArtwork: GetCursor itself must stay untouched, since it's also the scanner's hot path and the scanner never reads artwork. Measured on 1,000,000 tracks, the pre-pass over all ids costs +41.8 MB heap and +298 ms versus GetCursor's bounded +0.0 MB. The Jellyfin path is paginated, though, so in practice it only ever pre-passes a page's worth of ids, not the full library, and doesn't pay that cost. * feat(jellyfin): emit a song's own cover art when it differs from the album's Real Jellyfin fills ImageTags from each item's own images before falling back to the parent album, and Finamp checks imageTags.Primary before AlbumId. Our mapper read only the album's image, so a track with distinct embedded art silently showed the album cover. Emit exactly one entry under ImageBlurHashes.Primary: Go marshals map[string]string in sorted key order rather than insertion order, so a second entry could pair the wrong blurhash with the image imageId resolves to, and Finamp pins that pairing in its cache for 365 days. * test(persistence): scope the GetCursorWithArtwork full-stream spec to tie-free ids The fixture has title ties (e.g. three "Antenna" tracks), so the unscoped positional comparison against GetAll only passed because SQLite's tie order happened to coincide between the full scan and the pre-pass's id IN (...) fetch. Scope it to onlySongs like the sibling ordering specs already do. * refactor(artwork): route song own-art through primaryImageTag; align chunk size Cleanups surfaced by /simplify: the song mapper's own-art branch reimplemented primaryImageTag's tag+blurhash-map construction (and its one-entry invariant) — route it through the helper so that invariant lives in one place. Tie artworkChunkSize to a whole multiple of artworkBatchSize so a cursor page re-chunks into even hydration batches. Hoist a duplicated imageLoading && blurHash boolean in the album grid. * fix(ui): serve the placeholder for known-absent art instead of a broken icon getCoverArtUrl returned '' for an imageAbsent record, so <img src={undefined}> rendered as the browser's broken-image icon on every absent cover. The server already serves a proper placeholder for absent art, so build the url and let it render. * feat(ui): show the blurhash as the loading placeholder across cover surfaces Add a shared CoverImage component (useImageUrl blob cache + blurhash + fade) and render the blurhash while a cover loads on the list thumbnails (CoverArtAvatar, radio) and the artist/album/playlist detail pages. The detail pages now go through CoverImage instead of a plain CardMedia, so their images come from the in-memory blob cache and survive React remounts without re-fetching. BlurHashCanvas gains an optional style prop. * refactor(ui): unify list cover surfaces onto the shared CoverImage component Route the album grid, CoverArtAvatar (artist/playlist lists) and the radio list's cover field through CoverImage instead of each carrying its own useImageUrl + blurhash-overlay wiring. CoverImage gains a default object-fit: cover. Radio keeps its uploaded-image gate and the generic radio placeholder for stations with no art. * fix(ui): address CoverImage review findings Restructure CoverImage so the size/shape lives on the root and the blurhash + image are absolute fills: the <img> mounts only once its blob is ready, so an unresolved cover never flashes a broken <img>. Add a fit prop (default cover) so album/playlist detail keep their letterbox instead of being cropped by a hardcoded object-fit. Remove the orphaned coverLoading styles and an unused subsonic import; add a CoverImage unit test. * perf(ui): only refetch already-loaded records on SSE refresh The artwork worker broadcasts a RefreshResource event per resolved chunk, carrying every id in the chunk. useResourceRefresh was doing a getMany for all of them, so any open list/detail page fetched hundreds of artists it was not displaying. Filter the event ids to records already in the store; the rest load fresh (with their new artwork) when navigated to. * refactor(artwork): scale worker concurrency with CPU count ArtworkWorkerConcurrency now defaults to max(2, NumCPU()/2) instead of a fixed 4, mirroring MaxOpenConns: local resolution scales with the host but stays at half the SQLite pool so it never starves the scanner/UI. External RPS stays a fixed 2 — it gates third-party API calls and is bounded by their tolerance, not the host, so it must not scale with CPUs. Also drop the DevArtworkWorkerConcurrency/DevArtworkExternalRPS deprecated aliases: those names were never released, so there is nothing to migrate. * feat(artwork): re-queue an absent cover when its page is viewed serveEntity now schedules a Bump recheck for an entity whose art was recorded absent, so viewing a missing cover re-triggers resolution (e.g. after an external source that was down during the scan comes back), matching the request-time bump that already covers never-resolved entities. Throttled by attempted_at against requestRecheckAge (1h) so repeatedly opening a genuinely-absent page can't hammer external services. EnqueueBump preserves an existing failed-state backoff via MAX(priority,...) and inserts a fresh, immediately-eligible recheck for a settled-absent row (whose queue row was already deleted). * refactor(artwork): move ImageUploadService to artwork.Uploader Relocate the image-upload service from core to core/artwork as artwork.Uploader, co-locating it with the resolver/worker/serving that own the artwork state it invalidates. MaxImageUploadSize moves too — its only callers are the two image-upload handlers — which lets core/image_upload.go be deleted entirely. Extract the shared "clear resolved state + re-queue at Bump" invalidation into artwork.Refresh and fold nativeapi's refreshArtwork handler onto it, removing the duplicated DeleteForItem+Enqueue block that had drifted into three places. The wire provider moves from core's set to artwork's; the playlists.ImageUploadService binding moves to the top-level injector so core/artwork stays unaware of playlists. Behavior is unchanged. * refactor(artwork): thread model.Kind through the artwork API Entity-level artwork queries now take a typed model.Kind instead of a bare prefix string. GetItemArtwork, DeleteForItem(s), GetInfoForItems, EnqueueStaleAbsent, hydrateItemImages, enqueueBackfillKind and artwork.Refresh convert to the prefix string only at the two real boundaries: the SQL item_kind column (kind.Prefix() inside each repo) and external string inputs (a new model.ParseKind for the nativeapi URL param, which also validates it). The Backfill/stale-absent kind slices, the resolve.go dispatch switch, and the kind→resource / kind→table lookup maps now use the Kind vars directly. The queue lifecycle methods (MarkFailed/Delete*) keep string kinds — they operate on a dequeued item's raw ItemKind column, which stays a string field, always populated via kind.Prefix(). Removes every bare "al"/"ar"/… prefix literal from non-test code (27 -> 0); behavior is unchanged. * tune(artwork): drop backoff base from 5m to 15s The exponential retry (base × 4^attempts, cap 48h) started at 5 minutes, so a single transient failure — a timeout under load, an external blip — parked a cover for 5 minutes even though a retry seconds later would have resolved it. Start at 15s instead: transient failures recover almost immediately (15s → 1m → 4m → 16m …), while persistent failures still escalate to the 48h cap (now at the 8th attempt instead of the 5th). * tune(artwork): 5s backoff base + 12h give-up, drop the cap Retry backoff now starts at 5s (was 15s) so a transient failure recovers on essentially the next drain, and jitter widens to ±40% so a wave of correlated failures doesn't re-clump into one poll. Add a 12h give-up budget measured from enqueued_at: once the next backoff would land past it, the worker stops retrying instead of grinding at a cap forever. A bare failure settles absent (handed to the 24h stale-absent sweep, and still recoverable on a page view); a found-stale keeps its already-served art. The budget bounds the tail, so the separate 48h backoffCap is removed. * fix(lastfm): match album.getInfo on name+artist only, not MBID Last.fm's album.getInfo by MBID is unreliable: a correct MBID can return a different album, or none. Observed with black midi's "7-eleven" (whose correct MBID returned a FLEETWOOD release) and both missing The Chats albums (one MBID 404s, the other resolves to a different self-titled release). The worker then recorded covers absent — or would fetch the wrong art — even though the correct cover is on Last.fm by name+artist. Stop passing the MBID to album.getInfo; query by name+artist only, which also drops the now-dead error-6 MBID-retry fallback. The low-level client keeps its MBID support for other callers; only the album lookup changes. * fix(lastfm): return agents.ErrNotFound on error 6 (not found) Last.fm returns error 6 for a missing artist/album — a definitive negative — but the agent returned the raw *lastFMError, so the artwork worker treated every not-found as a real fault: it counted toward the per-source circuit breaker (5 in a row opens it, fast-failing all Last.fm calls including valid ones) and was retried as a transient error instead of settling absent. On a first scan of a library with many artists Last.fm lacks, this stalled valid cover lookups and left entities churning in backoff. Translate error 6 to the shared agents.ErrNotFound at the agent boundary (callAlbumGetInfo / callArtistGetInfo), matching how the Deezer agent maps its client's not-found, and log it at Debug instead of Error — which also removes the not-found log spam. * feat(artwork): log external image-lookup failures at debug The worker's res.reader==nil && extError branch returned outcomeFailed with no log, so a failing external cover lookup (agent error, dead image URL, download timeout) was undiagnosable. Log the agent, entity, and underlying error at the fetch site where it's in hand — this surfaced a Last.fm album.getInfo returning an image URL that itself 404s. * fix(artwork): treat a 404/410 image URL as not-found, not a transient fault An agent (notably Last.fm's album.getInfo) can advertise a cover URL that is itself dead — a 404. sources.go's fromURL returned a generic error for any non-200, so a dead URL was treated as a transient failure: it churned in backoff and counted toward the circuit breaker, stalling valid lookups. Map 404/410 to model.ErrNotFound in fromURL so a dead URL settles absent, and collapse the near-identical fetchPlaylistImageURL (which already did this for M3U covers) into it. * feat(artwork): make artwork re-resolution targeted, not blunt Two gaps in when the pipeline re-resolves artwork: The recheck job only requeued absent-state rows (hash=''), so an entity that was never processed — added between scans, or on a server with the scanner disabled — had no periodic safety net and stayed without artwork indefinitely. Add EnqueueMissing(kind): a SQL set-difference enqueueing entities with no item_artwork row at Recheck priority (ON CONFLICT DO NOTHING, so it never disturbs a queued row). Run it once at startup and hourly alongside the stale-absent recheck. Rename staleAbsentKinds -> recheckKinds accordingly. Conversely, the config fingerprint included consts.Version, which embeds the git SHA and so changed on every build, re-enqueueing every entity in the library (~34k here) and re-querying external agents at the configured RPS for anything without local art. Replace it with an explicit artworkEpoch constant, bumped deliberately when resolution semantics change. The cases that motivated the version input — absent art becoming available — are already covered by the stale-absent and missing-row rechecks; only a corrected wrong-pick needs the epoch. A test guards against reintroducing the version. * test(artwork): restore resolution edge-case e2e coverage The serving cutover removed the album/disc/artist/mediafile/playlist/radio e2e specs that documented the folder-selection rules and guarded the #5376/#5456/ #5451/#5457 regressions; nothing replaced them, so compareImageFiles and the parent-fallback logic were left untested. Restore them driving the real pipeline: a real scanner populates the folder graph from an in-memory library, the real Worker drains the queue, and the real Service serves. Folder-backed art is file-backed (served via os.Open, which the in-memory FS can't satisfy) so its selection is asserted on the persisted state row; store-backed and real-disk sources are asserted byte-for-byte. Single-disc disc resolution now serves album art directly, so only multi-disc disc scenarios are ported. * fix(artwork): run disc resolution for single-disc albums too |
||
|
Deluan Quintão
|
116a440718 |
test(scanner): fix flaky Windows search_normalized rescan test (#5758)
The 'repopulates a stale search_normalized on a full rescan' spec runs two full scans back-to-back. Whether the second scan refreshes the unchanged artist depends on folderEntry.isOutdated(), which compares folder.updated_at (written during the first scan) against the second scan's library.last_scan_started_at using a strict time.Before(). Both are time.Now() values captured milliseconds apart. On Linux's fine-grained clock they are always distinct, so the test passes. On Windows the coarse wall-clock granularity frequently makes the two timestamps land in the same tick and compare equal, so Before() returns false, the folder is treated as up-to-date and skipped, the artist is never re-persisted, and search_normalized stays empty -- failing the assertion intermittently across unrelated PRs. Backdate the folder's updated_at an hour before the second scan so the comparison is unambiguous on every platform. This is a test-only timing artifact (real rescans never run milliseconds apart on an unchanged library), so no production code changes are needed. |
||
|
Deluan Quintão
|
427d4b9bce |
fix(search): artists with atomic non-ASCII names unfindable after FTS5 migration (#5703)
* fix(scanner): update artist search_normalized when rescanning The FTS5 migration back-fills artist.search_normalized with a SQL punctuation-strip approximation, relying on the next scan to compute the precise value in Go (normalizeForFTS transliterates atomic letters like Ø/æ/ß that FTS5's remove_diacritics cannot fold). But the scanner persisted artists with an explicit column list that omitted search_normalized, so not even a full scan ever repaired it: an artist migrated from a pre-FTS database (e.g. "GØGGS") stayed unfindable by any ASCII search, while their albums and songs, which are saved with all columns, were fixed by a full scan. Add search_normalized to the column list so a full scan re-indexes the artist via the artist_fts trigger. * refactor(persistence): move normalizeForFTS to utils/str Export it as str.NormalizeForFTS so the upcoming migration can reuse the exact index-time normalization. Migrations cannot import the persistence package (persistence -> db -> db/migrations would be an import cycle). * fix(persistence): backfill artist search_normalized via migration Recompute artist.search_normalized with the precise Go normalization for databases migrated from pre-FTS5 versions, where the SQL back-fill could not transliterate atomic letters (Ø/æ/ß) and the scanner never rewrote the column. Only changed rows are updated, so the artist_fts update trigger re-indexes exactly the affected artists, making artists like GØGGS or MØ findable again without requiring a full scan. * refactor(persistence): share FTS punctuation-strip regex via utils/str Index-time normalization (NormalizeForFTS) and query-time processing (buildFTS5Query/ftsQueryDegraded) must produce matching tokens, so keep the punctuation-strip pattern in a single exported symbol instead of two identical private copies that could drift. Also document that derived columns computed in dbArtist.PostMapArgs must be listed in the scanner's artist Put, which is how search_normalized went stale in the first place. * chore(migrations): announce artist search backfill in the log Match the FTS5 migration's notice() pattern so startup isn't silent while the backfill runs on large libraries. * docs: tighten comments added in this branch * docs: describe FTSPunctStrip by what it matches, not one replacement |
||
|
Deluan Quintão
|
838ceee26d |
perf(subsonic): speed up artist search3 deep-offset pagination (#5620)
* perf(subsonic): speed up artist search3 deep-offset pagination
Empty-query and FTS artist search (search3/search2) paginated via a
CROSS JOIN library_artist + DISTINCT in Phase 1 purely for library access
control. The DISTINCT forced a temp b-tree over the whole junction table on
every page, making deep offsets O(offset): ~200ms at offset 299k on 300k
artists.
Replace it with a join-free EXISTS predicate keyed on artist.id, backed by a
new covering index on library_artist(artist_id, library_id). EXISTS keeps
artist as the ordered driver and never fans out rowids, so Phase 1 stays a
plain ordered scan that LIMIT/OFFSET can short-circuit. Admin, headless, and
all-libraries users skip the filter entirely (the dominant case) for a flat
ordered walk over the primary key.
Measured on a 300k-artist / 1M-song library: admin/all-libs pagination is
~4.5-5.4x faster at depth (~180ms to ~33ms at offset 400k); restricted
subset users keep correct, gap-free pages while also getting faster.
The narrowing artist filter is applied at the subsonic layer only when the
request targets a strict subset of the user's libraries, so the common case
(and the admin fast-path) is never burdened with a redundant predicate.
* fix(subsonic): narrow artist search by library set, not count
narrowsArtistLibraries decided whether to add the subsonic-layer artist
narrowing filter by comparing len(requested) < len(accessible). musicFolderId
is not deduplicated, so duplicate IDs inflated the requested count: a user
requesting ?musicFolderId=1&musicFolderId=1&musicFolderId=2 against three
accessible libraries produced len([1,1,2])==3, which is not < 3, so the filter
was skipped and the user saw artists from the third library too.
Compare as set membership instead: the request narrows iff some accessible
library is absent from it (requested is always a subset of accessible, validated
upstream by selectedMusicFolderIds). This is immune to duplicate IDs. Add a
regression test that fails against the old length-based check.
Also consolidate the repeated EXISTS/no-DISTINCT/O(page) rationale that the
prior commit spread across five sites down to a single authoritative comment on
ArtistLibraryFilter, with the call sites referencing it.
* perf(subsonic): drop redundant library_artist covering index
The migration added an index on library_artist(artist_id, library_id) on the
theory that the restricted-subset artist-search EXISTS needed it to seek by
artist_id. Benchmarking on a 405k-artist / 5-library dataset showed no benefit:
the EXISTS subquery constrains both columns (artist_id = and library_id IN), so
SQLite already resolves it as a covering-index seek on the existing
(library_id, artist_id) UNIQUE autoindex. With the new index present the planner
still picks the autoindex and ignores it.
Drop the migration and correct the comment. Removing ~11MB of dead index plus
its write-amplification on every library_artist insert/delete, for zero query
gain.
* fix(scanner): mark artists missing when they lose their last library
Artist search Phase 1 filters on artist.missing and Phase 2 inner-joins
library_artist, so a non-missing artist with no library_artist row (an orphan)
takes a pagination slot in Phase 1 and then vanishes in Phase 2, shortening the
page and shifting deep offsets. The admin/headless search fast-path walks artist
unfiltered, so it is fully exposed to this.
Two paths created such orphans without updating artist.missing:
- RefreshStats deletes library_artist rows whose stats are '{}' (artist lost all
content in a library) after every scan. This is the common source.
- Library deletion cascades away the library's library_artist rows.
Mark newly-orphaned artists missing at both sources, so the shared
'missing = false' search filter excludes them immediately instead of waiting for
a later scan. In RefreshStats the update only runs when the cleanup actually
removed rows (the only way a new orphan can appear), so steady-state scans pay
nothing; measured ~160ms on 300k artists only when orphans can exist.
* refactor(subsonic): address review feedback on artist search filter
Code-review follow-ups to the artist search pagination change:
- ArtistLibraryFilter: short-circuit to a constant-false predicate when no
library IDs are given, avoiding a degenerate empty IN () subquery.
- ArtistLibraryFilter: add an inner LIMIT 1 to the correlated EXISTS so SQLite
cannot flatten it into a fan-out join (an artist in multiple of the user's
libraries would otherwise yield duplicate rowids and corrupt pagination).
- narrowsArtistLibraries: compare accessible-vs-requested as a set lookup
instead of slices.Contains in a loop.
- searchConfig.LibraryFilter: document that a join-free filter is now a
correctness requirement (DISTINCT was removed), not just a performance one.
* docs: trim verbose comments in artist search/orphan code
Condense the over-explained comments added in this PR to the essential 'why',
removing repeated cross-references and restatements of the adjacent code.
* fix(scanner): heal pre-existing orphan artists on full refresh
The orphan-marking added to RefreshStats only ran when its empty-stats cleanup
deleted rows, so it reconciled newly-created orphans but not ones already left
in the database by older versions (whose library_artist row was deleted before
this fix existed). Such legacy orphans would surface in the admin/headless search
fast-path as short/gappy pages.
Also run the orphan-marking on a full refresh (allArtists), so a full scan — which
upgrades commonly trigger and users can run manually — reconciles the backlog. No
migration needed; the runtime fixes prevent recurrence.
* perf(subsonic): extend artist search fast-path to all-library users
applyLibraryFilterToSearchQuery only skipped the library filter for admin and
headless processes. A regular (non-admin) user who can access every library has
the same result set as an admin, but was still given the EXISTS filter — an
O(offset) cost for a predicate that matches every non-missing artist anyway.
Skip the filter for them too, using a cheap library CountAll() (a count over the
tiny library table) compared against the user's library count. On any error it
falls back to the filtered path, which is correct, just slower.
* fix(scanner): log error as trailing arg, not explicit error key
Signed-off-by: Deluan <deluan@navidrome.org>
* test(scanner): e2e guard for orphan artists under PurgeMissing
Adds an end-to-end scanner test for the orphan-artist invariant fixed in
RefreshStats: with Scanner.PurgeMissing enabled, removing all of an artist's
files hard-deletes them, cascades away their media_file_artists rows, and
RefreshStats then drops the artist's emptied library_artist row. The test
asserts no non-missing artist is left without a library_artist row. Verified it
fails without the RefreshStats orphan-marking and passes with it.
* test(scanner): assert the orphaned artist is marked missing
The orphan e2e test only checked the aggregate no-orphan invariant
(orphanCount == 0), which a fully-deleted artist or an un-cleaned row would also
satisfy — so it could pass without exercising the fix. Assert Pink Floyd's row
specifically: missing=false before, missing=true after, and absent from the
non-missing results. Verified it fails without the RefreshStats orphan-marking.
* test(scanner): drop misleading non-missing-list assertion for orphan
GetAll has no default missing filter, but selectArtist inner-joins library_artist,
so an orphaned artist (no junction row) is excluded from the results whether or
not it is marked missing. The Not(ContainElement) check therefore passed for the
wrong reason. The direct floydMissing() == 1 query is the assertion that actually
validates the missing flag; keep that plus the orphan-count invariant and an
over-marking guard on The Beatles.
* test(scanner): document why orphan check reads the artist row directly
Clarify that GetAll cannot observe the orphan: selectArtist inner-joins
library_artist, so an artist with no junction row is excluded from results
whether or not it is marked missing. Asserting on GetAll would pass even without
the fix, so the test reads the artist row directly to check the missing flag.
* test(scanner): return descriptive artist state for clearer failures
floydState returns PRESENT/MISSING/NOT_FOUND instead of 0/1/-1, so a failure
reads '<string>: PRESENT to equal MISSING' rather than '0 to equal 1'.
* refactor(subsonic): keep artist library scoping in the repository
The search endpoint built a persistence-layer EXISTS predicate
(persistence.ArtistLibraryFilter) and injected it into artistOpts.Filters — the
only place the subsonic package reached into persistence, leaking a storage
detail up two layers.
Pass the same Eq{"library_id": ids} filter used for albums and songs, and let
the artist repository translate it to the join-free library_artist predicate
(scopeSearchToLibraries), where the junction knowledge belongs. The subset-vs-
fast-path decision moves there too, so narrowsArtistLibraries and the persistence
import are gone from the subsonic layer. Behavior is unchanged; coverage for the
translation moves to artist_repository_test.
* refactor(persistence): extract canonical markOrphansMissing helper
The 'mark non-missing artists with no library_artist row as missing' invariant
was hand-written as SQL in two places (RefreshStats and libraryRepository.Delete),
in two slightly different dialects (not exists vs id not in). Extract a single
artistRepository.markOrphansMissing method next to markMissing and call it from
both sites, so the invariant has one definition.
* fix(persistence): apply scoped library filter in both search phases
Two bugs from moving the artist library-scoping into the repository:
- Search() scoped opts.Filters for Phase 1 but still passed the original
(unscoped) options to selectArtist, so Phase 2 re-applied the raw
Eq{library_id} against the wrong columns and a restricted user's search
returned nothing. Pass the scoped opts to both phases.
- scopeSearchToLibraries dropped the filter unconditionally for admins, so an
admin explicitly narrowing via musicFolderId (e.g. search3?musicFolderId=2)
leaked content from other libraries. Compare the request against the user's
visible library set (all libraries for admin/headless), narrowing whenever it
is a strict subset.
Both regressions were caught by the server/e2e multi-library suite.
* fix(core): delete library and reconcile orphans in one transaction
libraryRepository.Delete runs the FK-cascade delete and the orphaned-artist
reconciliation (markOrphansMissing) as two writes on r.db. Called directly they
autocommit separately, so an interruption between them could leave non-missing
artists with no library_artist row — the orphan state the artist search
fast-path forbids. Wrap the deletion in ds.WithTx at the core wrapper so both
writes commit atomically; the watcher/scanner/broker side-effects stay
post-commit.
* refactor(persistence): unify artist search library scoping into one filter
Phase 1 previously applied two overlapping library predicates: cfg.LibraryFilter
(scoped to the user's libraries) AND options.Filters (the requested subset),
producing two correlated EXISTS subqueries per rowid even though the request is
always a subset of the user's libraries. And the 'does this user see everything'
decision was implemented twice (userHasAllLibraries via CountAll vs
scopeSearchToLibraries via set-membership), with applyLibraryFilterToSearchQuery
as a third scoping path.
Resolve the effective library scope once in Search() via searchScope (intersect
the requested set with the user's visible libraries; nil = fast-path), clear
opts.Filters, and realize that single scope as the only Phase-1 LibraryFilter.
The visibility logic is now one pipeline: requestedLibraryIDs + visibleLibraryIDs
+ userSeesAllLibraries. Behavior unchanged; one EXISTS instead of two on the hot
path, one source of truth for library visibility.
* fix(persistence): harden artist search against malformed library_id filter
Search consumed only an Eq{"library_id": []int} filter; an Eq whose library_id
value wasn't []int slipped through unconsumed and would reach Phase 1's bare
artist table (no library_id column) → SQL error. Recognize any Eq carrying a
library_id key (isLibraryIDFilter) and always consume it, falling back to the
user's visible scope for a malformed value. Non-library filters are still left
in place for doSearch.
* refactor(persistence): trim redundant comments and unexport artist library filter
The artist-search-pagination work left dense explanatory comments, with the
join-free / LIMIT-1 anti-flatten rationale and the orphan-artist mechanics each
restated in several places. Consolidate each rationale into one canonical home
(artistLibraryFilter for the EXISTS/LIMIT-1 trick, markOrphansMissing for the
orphan lifecycle) and have the other sites reference it instead of repeating it.
Also unexport ArtistLibraryFilter to artistLibraryFilter: its only caller is
searchCfg in the same package and no test references it, so it never needed to
be part of the package's exported surface.
Comments only plus the rename; no behavior change.
* refactor: add slice.ToSet and use it for the artist search subset check
searchScope's subset test compared the requested libraries against the visible
set with a nested slices.Contains, which is O(visible * requested). On an instance
with many libraries (e.g. 100 libraries, a user granted 99) and an explicit
musicFolderId request, that is ~9.8k comparisons; with a set it is ~200.
Add a small reusable slice.ToSet helper (a slice -> map[T]struct{} set, collapsing
duplicates) and use it to make the membership lookups O(1), restoring O(n+m) without
the throwaway struct{}{} literal that an inline ToMap would need. No behavior change.
* refactor(artist): move artistLibraryFilter to artist_repository
Signed-off-by: Deluan <deluan@navidrome.org>
---------
Signed-off-by: Deluan <deluan@navidrome.org>
|
||
|
Deluan Quintão
|
64c8d3f4c5 |
ci: run Go tests on Windows (#5380)
* ci(windows): add skeleton go-windows job (compile-only smoke test)
* ci(windows): fix comment to reference Task 7 not Task 6
* ci(windows): harden PATH visibility and set explicit bash shell
* ci(windows): enable full go test suite and ndpgen check
* test(gotaglib): skip Unix-only permission tests on Windows
* test(lyrics): skip Windows-incompatible tests
* test(utils): skip Windows-incompatible tests
* test(mpv): skip Windows-incompatible playback tests
Skip 3 subprocess-execution tests that rely on Unix-style mpv
invocation; .bat output includes \r-terminated lines that break
argument parsing (#TBD-mpv-windows).
* test(storage): skip Windows-incompatible tests
Skip relative-path test where filepath.Join uses backslash but the
storage implementation returns a forward-slash URL path
(#TBD-path-sep-storage).
* test(storage/local): skip Windows-incompatible tests
Skip 13 tests that fail because url.Parse("file://" + windowsPath)
treats the drive letter colon as an invalid port; also skip the
Windows drive-letter path test that exposes a backslash vs
forward-slash normalisation bug (#TBD-path-sep-storage-local).
* test(playlists): skip Windows-incompatible tests
* test(model): skip Windows-incompatible tests
* test(model/metadata): skip Windows-incompatible tests
* test(core): skip Windows-incompatible tests
AbsolutePath uses filepath.Join which produces OS-native path separators;
skip the assertion test on Windows until the production code is fixed
(#TBD-path-sep-core).
* test(artwork): skip Windows-incompatible tests
Artwork readers produce OS-native path separators on Windows while tests
assert forward-slash paths; skip 11 affected tests pending a fix in
production code (#TBD-path-sep-artwork).
* test(persistence): skip Windows-incompatible tests
Skip flaky timestamp comparison (#TBD-flake-persistence) and path-separator
real-bugs (#TBD-path-sep-persistence) in FolderRepository.GetFolderUpdateInfo
which uses filepath.Clean/os.PathSeparator converting stored forward-slash paths
to backslashes on Windows.
* test(scanner): skip Windows-incompatible tests
Skip symlink tests (Unix-assumption), ndignore path-separator bugs
(#TBD-path-sep-scanner) in processLibraryEvents/resolveFolderPath where
filepath.Rel/filepath.Split return backslash paths incompatible with fs.FS
forward-slash expectations, error message mismatch on Windows, and file
format upgrade detection (#TBD-path-sep-scanner).
* test(plugins): skip Windows-incompatible tests
Add //go:build !windows tags to test files that reference the suite
bootstrap (testManager, testdataDir, createTestManager) which is only
compiled on non-Windows. Add a Windows-only suite stub that skips all
specs via BeforeEach to prevent [build failed] on Windows CI.
* test(server): skip Windows-incompatible tests
Skip createUnixSocketFile tests that rely on Unix file permission bits
(chmod/fchmod) which are not supported on Windows.
* test(nativeapi): skip Windows-incompatible tests
Skip the i18n JSON validation test that uses filepath.Join to build
embedded-FS paths; filepath.Join produces backslashes on Windows which
breaks fs.Open (embedded FS always uses forward slashes).
* test(e2e): skip Windows-incompatible tests
On Windows, SQLite holds file locks that prevent the Ginkgo TempDir
DeferCleanup from deleting the DB file. Register an explicit db.Close
DeferCleanup (LIFO before TempDir cleanup) on Windows so the file lock
is released before the temp directory is removed.
* test(windows): fix e2e AfterSuite and skip remaining scanner path test
* test(scanner): skip another Windows path-sep test (#TBD-path-sep-scanner)
* test(subsonic): skip timing-flaky test on Windows (#TBD-flake-time-resolution-subsonic)
* test(scanner): skip 'detects file moved to different folder' on Windows
* test(scanner): consolidate 'Library changes' Windows skips into BeforeEach
* test(scanner): close DB before TempDir cleanup to fix Windows file lock
* test(scanner): skip ScanFolders suite on Windows instead of closing shared DB
* ci: retrigger for Windows soak run 2/3
* ci: retrigger for Windows soak run 3/3
* ci: retrigger for Windows soak run 3/3 (take 2)
* test(scanner): skip Multi-Library suite on Windows (SQLite file lock)
* ci(windows): promote go-windows to blocking status check
* test(plugins): run platform-neutral specs on Windows, drop blanket Skip
* test(windows): make tests cross-platform instead of skipping
- subsonic: back-date submissionTime baseline by 1s so
BeTemporally(">") passes under millisecond clock resolution
- persistence: sleep briefly between Put calls so UpdatedAt is
strictly after CreatedAt on low-resolution clocks
- utils/files: close tempFile before os.Remove so the test works on
Windows (where an open handle holds a file lock)
- tests.TempFile: close the handle before returning; metadata tests
no longer leak the open file into Ginkgo's TempDir cleanup
Resolves Copilot review comments on #5380.
* test(tests): add SkipOnWindows helper to reduce boilerplate
Introduces tests.SkipOnWindows(reason) that wraps the 3-line
runtime.GOOS guard pattern used in every Windows-skipped spec.
* test(adapters): use tests.SkipOnWindows helper
* test(core): use tests.SkipOnWindows helper
* test(model): use tests.SkipOnWindows helper
* test(persistence): use tests.SkipOnWindows helper
* test(scanner): use tests.SkipOnWindows helper
* test(server): use tests.SkipOnWindows helper
* test(plugins): run pure-Go unit tests on Windows
config_validation_test, manager_loader_test, and migrate_test have no
WASM/exec dependencies and don't rely on the make-built test plugins
from plugins_suite_test.go. Let them run on Windows too.
|
||
|
Deluan Quintão
|
ab8a58157a |
feat: add artist image uploads and image-folder artwork source (#5198)
* feat: add shared ImageUploadService for entity image management * feat: add UploadedImage field and methods to Artist model * feat: add uploaded_image column to artist table * feat: add ArtistImageFolder config option * refactor: wire ImageUploadService and delegate playlist file ops to it Wire ImageUploadService into the DI container and refactor the playlist service to delegate image file operations (SetImage/RemoveImage) to the shared ImageUploadService, removing duplicated file I/O logic. A local ImageUploadService interface is defined in core/playlists to avoid an import cycle between core and core/playlists. * feat: artist artwork reader checks uploaded image first * feat: add image-folder priority source for artist artwork * feat: cache key invalidation for image-folder and uploaded images * refactor: extract shared image upload HTTP helpers * feat: add artist image upload/delete API endpoints * refactor: playlist handlers use shared image upload helpers * feat: add shared ImageUploadOverlay component * feat: add i18n keys for artist image upload * feat: add image upload overlay to artist detail pages * refactor: playlist details uses shared ImageUploadOverlay component * fix: add gosec nolint directive for ParseMultipartForm * refactor: deduplicate image upload code and optimize dir scanning - Remove dead ImageFilename methods from Artist and Playlist models (production code uses core.imageFilename exclusively) - Extract shared uploadedImagePath helper in model/image.go - Extract findImageInArtistFolder to deduplicate dir-scanning logic between fromArtistImageFolder and getArtistImageFolderModTime - Fix fileInputRef in useCallback dependency array * fix: include artist UpdatedAt in artwork cache key Without this, uploading or deleting an artist image would not invalidate the cached artwork because the cache key was only based on album folder timestamps, not the artist's own UpdatedAt field. * feat: add Portuguese translations for artist image upload * refactor: use shared i18n keys for cover art upload messages Move cover art upload/remove translations from per-entity sections (artist, playlist) to a shared top-level "message" section, avoiding duplication across entity types and translation files. * refactor: move cover art i18n keys to shared message section for all languages * refactor: simplify image upload code and eliminate redundancies Extracted duplicate image loading/lightbox state logic from DesktopArtistDetails and MobileArtistDetails into a shared useArtistImageState hook. Moved entity type constants to the consts package and replaced raw string literals throughout model, core, and nativeapi packages. Exported model.UploadedImagePath and reused it in core/image_upload.go to consolidate path construction. Cached the ArtistImageFolder lookup result in artistReader to eliminate a redundant os.ReadDir call on every artwork request. Signed-off-by: Deluan <deluan@navidrome.org> * style: fix prettier formatting in ImageUploadOverlay * fix: address code review feedback on image upload error handling - RemoveImage now returns errors instead of swallowing them - Artist handlers distinguish not-found from other DB errors - Defer multipart temp file cleanup after parsing * fix: enforce hard request size limit with MaxBytesReader for image uploads Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
7ad2907719 |
refactor: move playlist business logic from repositories to service layer (#5027)
* refactor: move playlist business logic from repositories to core.Playlists service Move authorization, permission checks, and orchestration logic from playlist repositories to the core.Playlists service, following the existing pattern used by core.Share and core.Library. Changes: - Expand core.Playlists interface with read, mutation, track management, and REST adapter methods - Add playlistRepositoryWrapper for REST Save/Update/Delete with permission checks (follows Share/Library pattern) - Simplify persistence/playlist_repository.go: remove isWritable(), auth checks from Delete()/Put()/updatePlaylist() - Simplify persistence/playlist_track_repository.go: remove isTracksEditable() and permission checks from Add/Delete/Reorder - Update Subsonic API handlers to route through service - Update Native API handlers to accept core.Playlists instead of model.DataStore * test: add coverage for playlist service methods and REST wrapper Add 30 new tests covering the service methods added during the playlist refactoring: - Delete: owner, admin, denied, not found - Create: new playlist, replace tracks, admin bypass, denied, not found - AddTracks: owner, admin, denied, smart playlist, not found - RemoveTracks: owner, smart playlist denied, non-owner denied - ReorderTrack: owner, smart playlist denied - NewRepository wrapper: Save (owner assignment, ID clearing), Update (owner, admin, denied, ownership change, not found), Delete (delegation with permission checks) Expand mockedPlaylistRepo with Get, Delete, Tracks, GetWithTracks, and rest.Persistable methods. Add mockedPlaylistTrackRepo for track operation verification. * fix: add authorization check to playlist Update method Added ownership verification to the Subsonic Update endpoint in the playlist service layer. The authorization check was present in the old repository code but was not carried over during the refactoring to the service layer, allowing any authenticated user to modify playlists they don't own via the Subsonic API. Also added corresponding tests for the Update method's permission logic. * refactor: improve playlist permission checks and error handling, add e2e tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename core.Playlists to playlists package and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename playlists_internal_test.go to parse_m3u_test.go and update tests; add new parse_nsp.go and rest_adapter.go files Signed-off-by: Deluan <deluan@navidrome.org> * fix: block track mutations on smart playlists in Create and Update Create now rejects replacing tracks on smart playlists (pre-existing gap). Update now uses checkTracksEditable instead of checkWritable when track changes are requested, restoring the protection that was removed from the repository layer during the refactoring. Metadata-only updates on smart playlists remain allowed. * test: add smart playlist protection tests to ensure readonly behavior and mutation restrictions * refactor: optimize track removal and renumbering in playlists Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement track reordering in playlists with SQL updates Signed-off-by: Deluan <deluan@navidrome.org> * refactor: wrap track deletion and reordering in transactions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused getTracks method from playlistTrackRepository Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize playlist track renumbering with CTE-based UPDATE Replace the DELETE + re-INSERT renumbering strategy with a two-step UPDATE approach using a materialized CTE and ROW_NUMBER() window function. The previous approach (SELECT all IDs, DELETE all tracks, re-INSERT in chunks of 200) required 13 SQL operations for a 2000-track playlist. The new approach uses just 2 UPDATEs: first negating all IDs to clear the positive space, then assigning sequential positions via UPDATE...FROM with a CTE. This avoids the UNIQUE constraint violations that affected the original correlated subquery while reducing per-delete request time from ~110ms to ~12ms on a 2000-track playlist. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename New function to NewPlaylists for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update mock playlist repository and tests for consistency Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
8c80be56da |
fix(scanner): ensure FullScanInProgress reflects current scan request during interrupted scans
Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
28d5299ffc |
feat(scanner): implement selective folder scanning and file system watcher improvements (#4674)
* feat: Add selective folder scanning capability Implement targeted scanning of specific library/folder pairs without full recursion. This enables efficient rescanning of individual folders when changes are detected, significantly reducing scan time for large libraries. Key changes: - Add ScanTarget struct and ScanFolders API to Scanner interface - Implement CLI flag --targets for specifying libraryID:folderPath pairs - Add FolderRepository.GetByPaths() for batch folder info retrieval - Create loadSpecificFolders() for non-recursive directory loading - Scope GC operations to affected libraries only (with TODO for full impl) - Add comprehensive tests for selective scanning behavior The selective scan: - Only processes specified folders (no subdirectory recursion) - Maintains library isolation - Runs full maintenance pipeline scoped to affected libraries - Supports both full and quick scan modes Examples: navidrome scan --targets "1:Music/Rock,1:Music/Jazz" navidrome scan --full --targets "2:Classical" * feat(folder): replace GetByPaths with GetFolderUpdateInfo for improved folder updates retrieval Signed-off-by: Deluan <deluan@navidrome.org> * test: update parseTargets test to handle folder names with spaces Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): remove unused LibraryPath struct and update GC logging message Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): enhance external scanner to support target-specific scanning Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify scanner methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement folder scanning notifications with deduplication Signed-off-by: Deluan <deluan@navidrome.org> * refactor(watcher): add resolveFolderPath function for testability Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement path ignoring based on .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): implement IgnoreChecker for managing .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(ignore_checker): rename scanner to lineScanner for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance ScanTarget struct with String method for better target representation Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): validate library ID to prevent negative values Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify GC method by removing library ID parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): update folder scanning to include all descendants of specified folders Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): allow selective scan in the /startScan endpoint Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update CallScan to handle specific library/folder pairs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline scanning logic by removing scanAll method Signed-off-by: Deluan <deluan@navidrome.org> * test: enhance mockScanner for thread safety and improve test reliability Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move scanner.ScanTarget to model.ScanTarget Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move scanner types to model,implement MockScanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update scanner interface and implementations to use model.Scanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder_repository): normalize target path handling by using filepath.Clean Signed-off-by: Deluan <deluan@navidrome.org> * test(folder_repository): add comprehensive tests for folder retrieval and child exclusion Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify selective scan logic using slice.Filter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline phase folder and album creation by removing unnecessary library parameter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move initialization logic from phase_1 to the scanner itself Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): rename selective scan test file to scanner_selective_test.go Signed-off-by: Deluan <deluan@navidrome.org> * feat(configuration): add DevSelectiveWatcher configuration option Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): enhance .ndignore handling for folder deletions and file changes Signed-off-by: Deluan <deluan@navidrome.org> * docs(scanner): comments Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance walkDirTree to support target folder scanning Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner, watcher): handle errors when pushing ignore patterns for folders Signed-off-by: Deluan <deluan@navidrome.org> * Update scanner/phase_1_folders.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor(scanner): replace parseTargets function with direct call to scanner.ParseTargets Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): add tests for ScanBegin and ScanEnd functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(library): update PRAGMA optimize to check table sizes without ANALYZE Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): refactor tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add selective scan options and update translations Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add quick and full scan options for individual libraries Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add Scan buttonsto the LibraryList Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): update scanning parameters from 'path' to 'target' for selective scans. * refactor(scan): move ParseTargets function to model package * test(scan): suppress unused return value from SetUserLibraries in tests * feat(gc): enhance garbage collection to support selective library purging Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): prevent race condition when scanning deleted folders When the watcher detects changes in a folder that gets deleted before the scanner runs (due to the 10-second delay), the scanner was prematurely removing these folders from the tracking map, preventing them from being marked as missing. The issue occurred because `newFolderEntry` was calling `popLastUpdate` before verifying the folder actually exists on the filesystem. Changes: - Move fs.Stat check before newFolderEntry creation in loadDir to ensure deleted folders remain in lastUpdates for finalize() to handle - Add early existence check in walkDirTree to skip non-existent target folders with a warning log - Add unit test verifying non-existent folders aren't removed from lastUpdates prematurely - Add integration test for deleted folder scenario with ScanFolders Fixes the issue where deleting entire folders (e.g., /music/AC_DC) wouldn't mark tracks as missing when using selective folder scanning. * refactor(scan): streamline folder entry creation and update handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): add '@Recycle' (QNAP) to ignored directories list Signed-off-by: Deluan <deluan@navidrome.org> * fix(log): improve thread safety in logging level management * test(scan): move unit tests for ParseTargets function Signed-off-by: Deluan <deluan@navidrome.org> * review Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: deluan <deluan.quintao@mechanical-orchard.com> |
||
|
Deluan Quintão
|
00c83af170 |
feat: Multi-library support (#4181)
* feat(database): add user_library table and library access methods Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # tests/mock_library_repo.go * feat(database): enhance user retrieval with library associations Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement library management and user-library association endpoints Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): restrict access to library and config endpoints to admin users Signed-off-by: Deluan <deluan@navidrome.org> * refactor(library): implement library management service and update API routes Signed-off-by: Deluan <deluan@navidrome.org> * feat(database): add library filtering to album, folder, and media file queries Signed-off-by: Deluan <deluan@navidrome.org> * refactor library service to use REST repository pattern and remove CRUD operations Signed-off-by: Deluan <deluan@navidrome.org> * add total_duration column to library and update user_library table Signed-off-by: Deluan <deluan@navidrome.org> * fix migration file name Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add library management features including create, edit, delete, and list functionalities - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance library validation and management with path checks and normalization - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): improve library path validation and error handling - WIP Signed-off-by: Deluan <deluan@navidrome.org> * use utils/formatBytes Signed-off-by: Deluan <deluan@navidrome.org> * simplify DeleteLibraryButton.jsx Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance validation messages and error handling for library paths Signed-off-by: Deluan <deluan@navidrome.org> * lint Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): add tests for multi-library scanning and validation Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): improve handling of filesystem errors and ensure warnings are returned Signed-off-by: Deluan <deluan@navidrome.org> * feat(controller): add function to retrieve the most recent scan time across all libraries Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add additional fields and restructure LibraryEdit component for enhanced statistics display Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance LibraryCreate and LibraryEdit components with additional props and styling Signed-off-by: Deluan <deluan@navidrome.org> * feat(mediafile): add LibraryName field and update queries to include library name Signed-off-by: Deluan <deluan@navidrome.org> * feat(missingfiles): add library filter and display in MissingFilesList component Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): implement scanner interface for triggering library scans on create/update Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # cmd/wire_gen.go # cmd/wire_injectors.go # Conflicts: # cmd/wire_gen.go # Conflicts: # cmd/wire_gen.go # cmd/wire_injectors.go * feat(library): trigger scan after successful library deletion to clean up orphaned data Signed-off-by: Deluan <deluan@navidrome.org> * rename migration file for user library table to maintain versioning order Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move scan triggering logic into a helper method for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add library path and name fields to album and mediafile models Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add/remove watchers on demand, not only when server starts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline library handling by using state-libraries for consistency Signed-off-by: Deluan <deluan@navidrome.org> * fix: track processed libraries by updating state with scan timestamps Signed-off-by: Deluan <deluan@navidrome.org> * prepend libraryID for track and album PIDs Signed-off-by: Deluan <deluan@navidrome.org> * feat(repository): apply library filtering in CountAll methods for albums, folders, and media files Signed-off-by: Deluan <deluan@navidrome.org> * feat(user): add library selection for user creation and editing Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): implement library selection functionality with reducer and UI component Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # .github/copilot-instructions.md # Conflicts: # .gitignore * feat(library): add tests for LibrarySelector and library selection hooks Signed-off-by: Deluan <deluan@navidrome.org> * test: add unit tests for file utility functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add library ID filtering for album resources Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): streamline library ID filtering in repositories and update resource filtering logic Signed-off-by: Deluan <deluan@navidrome.org> * fix(repository): add table name handling in filter functions for SQL queries Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add refresh functionality on LibrarySelector close Signed-off-by: Deluan <deluan@navidrome.org> * feat(artist): add library ID filtering for artists in repository and update resource filtering logic Signed-off-by: Deluan <deluan@navidrome.org> # Conflicts: # persistence/artist_repository.go * Add library_id field support for smart playlists - Add library_id field to smart playlist criteria system - Supports Is and IsNot operators for filtering by library ID - Includes comprehensive test coverage for single values and lists - Enables creation of library-specific smart playlists * feat(subsonic): implement user-specific library access in GetMusicFolders Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): enhance LibrarySelectionField to extract library IDs from record Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): update GetIndexes and GetArtists method to support library ID filtering Signed-off-by: Deluan <deluan@navidrome.org> * fix: ensure LibrarySelector dropdown refreshes on button close Added refresh() call when closing the dropdown via button click to maintain consistency with the ClickAwayListener behavior. This ensures the UI updates properly regardless of how the dropdown is closed, fixing an inconsistent refresh behavior between different closing methods. The fix tracks the previous open state and calls refresh() only when the dropdown was open and is being closed by the button click. * refactor: simplify getUserAccessibleLibraries function and update related tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance selectedMusicFolderIds function to handle valid music folder IDs and improve fallback logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor: change ArtistRepository.GetIndex to accept multiple library IDs Updated the GetIndex method signature to accept a slice of library IDs instead of a single ID, enabling support for filtering artists across multiple libraries simultaneously. Changes include: - Modified ArtistRepository interface in model/artist.go - Updated implementation in persistence/artist_repository.go with improved library filtering logic - Refactored Subsonic API browsing.go to use new selectedMusicFolderIds helper - Added comprehensive test coverage for multiple library scenarios - Updated mock repository implementation for testing This change improves flexibility for multi-library operations while maintaining backward compatibility through the selectedMusicFolderIds helper function. * feat: add library access validation to selectedMusicFolderIds Enhanced the selectedMusicFolderIds function to validate musicFolderId parameters against the user's accessible libraries. Invalid library IDs (those the user doesn't have access to) are now silently filtered out, improving security by preventing users from accessing libraries they don't have permission for. Changes include: - Added validation logic to check musicFolderId parameters against user's accessible libraries - Added slices package import for efficient validation - Enhanced function documentation to clarify validation behavior - Added comprehensive test cases covering validation scenarios - Maintains backward compatibility with existing behavior * feat: implement multi-library support for GetAlbumList and GetAlbumList2 endpoints - Enhanced selectedMusicFolderIds helper to validate and filter library IDs - Added ApplyLibraryFilter function in filter/filters.go for library filtering - Updated getAlbumList to support musicFolderId parameter filtering - Added comprehensive tests for multi-library functionality - Supports single and multiple musicFolderId values - Falls back to all accessible libraries when no musicFolderId provided - Validates library access permissions for user security * feat: implement multi-library support for GetRandomSongs, GetSongsByGenre, GetStarred, and GetStarred2 - Added multi-library filtering to GetRandomSongs endpoint using musicFolderId parameter - Added multi-library filtering to GetSongsByGenre endpoint using musicFolderId parameter - Enhanced GetStarred and GetStarred2 to filter artists, albums, and songs by library - Added Options field to MockMediaFileRepo and MockArtistRepo for test compatibility - Added comprehensive Ginkgo/Gomega tests for all new multi-library functionality - All tests verify proper SQL filter generation and library access validation - Supports single/multiple musicFolderId values with fallback to all accessible libraries * refactor: optimize starred items queries with parallel execution and fix test isolation Refactored starred items functionality by extracting common logic into getStarredItems() method that executes artist, album, and media file queries in parallel for better performance. This eliminates code duplication between GetStarred and GetStarred2 methods while improving response times through concurrent database queries using run.Parallel(). Also fixed test isolation issues by adding missing auth.Init(ds) call in album lists test setup. This resolves nil pointer dereference errors in GetStarred and GetStarred2 tests when run independently. * fix: add ApplyArtistLibraryFilter to filter artists by associated music folders Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library access methods to User model Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement library access filtering for artist queries based on user permissions Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance artist library filtering based on user permissions and optimize library ID retrieval Signed-off-by: Deluan <deluan@navidrome.org> * fix: return error when any musicFolderId is invalid or inaccessible Changed behavior from silently filtering invalid library IDs to returning ErrorDataNotFound (code 70) when any provided musicFolderId parameter is invalid or the user doesn't have access to it. The error message includes the specific library number for better debugging. This affects album/song list endpoints (getAlbumList, getRandomSongs, getSongsByGenre, getStarred) to provide consistent error handling across all Subsonic API endpoints. Updated corresponding tests to expect errors instead of silent filtering. * feat: add musicFolderId parameter support to Search2 and Search3 endpoints Implemented musicFolderId parameter support for Subsonic API Search2 and Search3 endpoints, completing multi-library functionality across all Subsonic endpoints. Key changes: - Added musicFolderId parameter handling to Search2 and Search3 endpoints - Updated search logic to filter results by specified library or all accessible libraries when parameter not provided - Added proper error handling for invalid/inaccessible musicFolderId values - Refactored SearchableRepository interface to support library filtering with variadic QueryOptions - Updated repository implementations (Album, Artist, MediaFile) to handle library filtering in search operations - Added comprehensive test coverage with robust assertions verifying library filtering works correctly - Enhanced mock repositories to capture QueryOptions for test validation Signed-off-by: Deluan <deluan@navidrome.org> * feat: refresh LibraryList on scan end Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow editing name of main library Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement SendBroadcastMessage method for event broadcasting Signed-off-by: Deluan <deluan@navidrome.org> * feat: add event broadcasting for library creation, update, and deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: add useRefreshOnEvents hook for custom refresh logic on event changes Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance library management with refresh event broadcasting Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace AddUserLibrary and RemoveUserLibrary with SetUserLibraries for better library management Signed-off-by: Deluan <deluan@navidrome.org> * chore: remove commented-out genre repository code from persistence tests * feat: enhance library selection with master checkbox functionality Added a master checkbox to the SelectLibraryInput component, allowing users to select or deselect all libraries at once. This improves user experience by simplifying the selection process when multiple libraries are available. Additionally, updated translations in the en.json file to include a new message for selecting all libraries, ensuring consistency in user interface messaging. Signed-off-by: Deluan <deluan@navidrome.org> * feat: add default library assignment for new users Introduced a new column `default_new_users` in the library table to facilitate automatic assignment of default libraries to new regular users. When a new user is created, they will now be assigned to libraries marked as default, enhancing user experience by ensuring they have immediate access to essential resources. Additionally, updated the user repository logic to handle this new functionality and modified the user creation validation to reflect that library selection is optional for non-admin users. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct updated_at assignment in library repository Signed-off-by: Deluan <deluan@navidrome.org> * fix: improve cache buffering logic Refactored the cache buffering logic to ensure thread safety when checking the buffer length Signed-off-by: Deluan <deluan@navidrome.org> * fix formating Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement per-library artist statistics with automatic aggregation Implemented comprehensive multi-library support for artist statistics that automatically aggregates stats from user-accessible libraries. This fundamental change moves artist statistics from global scope to per-library granularity while maintaining backward compatibility and transparent operation. Key changes include: - Migrated artist statistics from global artist.stats to per-library library_artist.stats - Added automatic library filtering and aggregation in existing Get/GetAll methods - Updated role-based filtering to work with per-library statistics storage - Enhanced statistics calculation to process and store stats per library - Implemented user permission-aware aggregation that respects library access control - Added comprehensive test coverage for library filtering and restricted user access - Created helper functions to ensure proper library associations in tests This enables users to see statistics that accurately reflect only the content from libraries they have access to, providing proper multi-tenant behavior while maintaining the existing API surface and UI functionality. Signed-off-by: Deluan <deluan@navidrome.org> * feat: add multi-library support with per-library tag statistics - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: genre and tag repositories. add comprehensive tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add multi-library support to tag repository system Implemented comprehensive library filtering for tag repositories to support the multi-library feature. This change ensures that users only see tags from libraries they have access to, while admin users can see all tags. Key changes: - Enhanced TagRepository.Add() method to accept libraryID parameter for proper library association - Updated baseTagRepository to implement library-aware queries with proper joins - Added library_tag table integration for per-library tag statistics - Implemented user permission-based filtering through user_library associations - Added comprehensive test coverage for library filtering scenarios - Updated UI data provider to include tag filtering by selected libraries - Modified scanner to pass library ID when adding tags during folder processing The implementation maintains backward compatibility while providing proper isolation between libraries for tag-based operations like genres and other metadata tags. * refactor: simplify artist repository library filtering Removed conditional admin logic from applyLibraryFilterToArtistQuery method and unified the library filtering approach to match the tag repository pattern. The method now always uses the same SQL join structure regardless of user role, with admin access handled automatically through user_library associations. Added artistLibraryIdFilter function to properly qualify library_id column references and prevent SQL ambiguity errors when multiple tables contain library_id columns. This ensures the filter targets library_artist.library_id specifically rather than causing ambiguous column name conflicts. * fix: resolve LibrarySelectionField validation error for non-admin users Fixed validation error 'At least one library must be selected for non-admin users' that appeared even when libraries were selected. The issue was caused by a data format mismatch between backend and frontend. The backend sends user data with libraries as an array of objects, but the LibrarySelectionField component expects libraryIds as an array of IDs. Added data transformation in the data provider's getOne method to automatically convert libraries array to libraryIds format when fetching user records. Also extracted validation logic into a separate userValidation module for better code organization and added comprehensive test coverage to prevent similar issues. * refactor: remove unused library access functions and related tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename search_test.go to searching_test.go for consistency Signed-off-by: Deluan <deluan@navidrome.org> * fix: add user context to scrobble buffer getParticipants call Added user context handling to scrobbleBufferRepository.Next method to resolve SQL error 'no such column: library_artist.library_id' when processing scrobble entries in multi-library environments. The artist repository now requires user context for proper library filtering, so we fetch the user and temporarily inject it into the context before calling getParticipants. This ensures background scrobbling operations work correctly with multi-library support. * feat: add cross-library move detection for scanner Implemented cross-library move detection for the scanner phase 2 to properly handle files moved between libraries. This prevents users from losing play counts, ratings, and other metadata when moving files across library boundaries. Changes include: - Added MediaFileRepository methods for two-tier matching: FindRecentFilesByMBZTrackID (primary) and FindRecentFilesByProperties (fallback) - Extended scanner phase 2 pipeline with processCrossLibraryMoves stage that processes files unmatched within their library - Implemented findCrossLibraryMatch with MusicBrainz Release Track ID priority and intrinsic properties fallback - Updated producer logic to handle missing tracks without matches, ensuring cross-library processing - Updated tests to reflect new producer behavior and cross-library functionality The implementation uses existing moveMatched function for unified move operations, automatically preserving all user data through database foreign key relationships. Cross-library moves are detected using the same Equals() and IsEquivalent() matching logic as within-library moves for consistency. Signed-off-by: Deluan <deluan@navidrome.org> * feat: add album annotation reassignment for cross-library moves Implemented album annotation reassignment functionality for the scanner's missing tracks phase. When tracks move between libraries and change album IDs, the system now properly reassigns album annotations (starred status, ratings) from the old album to the new album. This prevents loss of user annotations when tracks are moved across library boundaries. The implementation includes: - Thread-safe annotation reassignment using mutex protection - Duplicate reassignment prevention through processed album tracking - Graceful error handling that doesn't fail the entire move operation - Comprehensive test coverage for various scenarios including error conditions This enhancement ensures data integrity and user experience continuity during cross-library media file movements. * fix: address PR review comments for multi-library support Fixed several issues identified in PR review: - Removed unnecessary artist stats initialization check since the map is already initialized in PostScan() - Improved code clarity in user repository by extracting isNewUser variable to avoid checking count == 0 twice - Fixed library selection logic to properly handle initial library state and prevent overriding user selections These changes address code quality and logic issues identified during the multi-library support PR review. * feat: add automatic playlist statistics refreshing Implemented automatic playlist statistics (duration, size, song count) refreshing when tracks are modified. Added new refreshStats() method to recalculate statistics from playlist tracks, and SetTracks() method to update tracks and refresh statistics atomically. Modified all track manipulation methods (RemoveTracks, AddTracks, AddMediaFiles) to automatically refresh statistics. Updated playlist repository to use the new SetTracks method for consistent statistics handling. * refactor: rename AddTracks to AddMediaFilesByID for clarity Renamed the AddTracks method to AddMediaFilesByID throughout the codebase to better reflect its purpose of adding media files to a playlist by their IDs. This change improves code readability and makes the method name more descriptive of its actual functionality. Updated all references in playlist model, tests, core playlist logic, and Subsonic API handlers to use the new method name. * refactor: consolidate user context access in persistence layer Removed duplicate helper functions userId() and isAdmin() from sql_base_repository.go and consolidated all user context access to use loggedUser(r.ctx).ID and loggedUser(r.ctx).IsAdmin consistently across the persistence layer. This change eliminates code duplication and provides a single, consistent pattern for accessing user context information in repository methods. All functionality remains unchanged - this is purely a code cleanup refactoring. * refactor: eliminate MockLibraryService duplication using embedded struct - Replace 235-line MockLibraryService with 40-line embedded struct pattern - Enhance MockLibraryRepo with service-layer methods (192→310 lines) - Maintain full compatibility with existing tests - All 72 nativeapi specs pass with proper error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: cleanup Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
b63630fa6e |
fix(scanner) artist stats not refreshing during quick scan and after missing file deletion (#4269)
* Fix artist not being marked as touched during quick scans When a new album is added during quick scans, artists were not being marked as 'touched' due to media files having older modification times than the scan completion time. Changes: - Add 'updated_at' to artist Put() columns in scanner to ensure timestamp is set when artists are processed - Simplify RefreshStats query to check artist.updated_at directly instead of complex media file joins - Artists from new albums now properly get refreshed in later phases This fixes the issue where newly added albums would have incomplete artist information after quick scans. * fix(missing): refresh artist stats in background after deleting missing files Signed-off-by: Deluan <deluan@navidrome.org> * fix(request): add InternalAuth to user context Signed-off-by: Deluan <deluan@navidrome.org> * Add comprehensive test for artist stats update during quick scans - Add test that verifies artist statistics are correctly updated when new files are added during incremental scans - Test ensures both overall stats (AlbumCount, SongCount) and role-specific stats are properly refreshed - Validates fix for artist stats not being refreshed during quick scans when new albums are added - Uses real artist repository instead of mock to verify actual stats calculation behavior --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
e5e2d860ef |
fix(scanner): ensure full scans update the DB (#4252)
* fix: ensure full scan refreshes all artist stats After PR #4059, full scans were not forcing a refresh of all artists. This change ensures that during full scans, all artist stats are refreshed instead of only those with recently updated media files. Changes: - Set changesDetected=true at start of full scans to ensure maintenance operations run - Add allArtists parameter to RefreshStats() method - Pass fullScan state to RefreshStats to control refresh scope - Update mock repository to match new interface Fixes #4246 Related to PR #4059 * fix: add tests for full and incremental scans Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
1e4c759d93 |
test: fix flaky scanner tests by setting maximum open connections to 1
Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Copilot
|
992c78376c |
feat(scanner): add Scanner.PurgeMissing configuration option (#4107)
* Initial plan for issue * Add Scanner.PurgeMissing configuration option Co-authored-by: deluan <331353+deluan@users.noreply.github.com> * Remove GC call from phaseMissingTracks.purgeMissing method Co-authored-by: deluan <331353+deluan@users.noreply.github.com> * Address PR comments for Scanner.PurgeMissing feature Co-authored-by: deluan <331353+deluan@users.noreply.github.com> * Address PR comments and add DeleteAllMissing method Co-authored-by: deluan <331353+deluan@users.noreply.github.com> * refactor(scanner): simplify purgeMissing logic and improve error handling Signed-off-by: Deluan <deluan@navidrome.org> * fix configuration test Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: deluan <331353+deluan@users.noreply.github.com> Co-authored-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
c795bcfcf7 |
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709)
* fix(server): more race conditions when updating artist/album from external sources Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394 Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): pass configfile option to child process Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): resume interrupted fullScans Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): remove old scanner code Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): rename old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): move old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update Go to 1.23.4 Signed-off-by: Deluan <deluan@navidrome.org> * fix: logs Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): Signed-off-by: Deluan <deluan@navidrome.org> * fix: log level Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add config for scanner watcher Signed-off-by: Deluan <deluan@navidrome.org> * refactor: children playlists Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace `interface{}` with `any` Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlists with genres Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow any tags in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist names in playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlist's sort by tags Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use generic JSONArray for OS arrays Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use https in test Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add releaseTypes to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add recordLabels to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): rename JSONArray to Array Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to Child Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): do not pre-populate smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement a simplified version of ArtistID3. See https://github.com/opensubsonic/open-subsonic-api/discussions/120 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to album child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add contributors to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add albumArtists to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayArtist and displayAlbumArtist Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayComposer to Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add roles to ArtistID3 Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): use " • " separator for displayComposer Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): respect `PreferSortTags` config option Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize purging non-unused tags Signed-off-by: Deluan <deluan@navidrome.org> * refactor: don't run 'refresh artist stats' concurrently with other transactions Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix: log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Scanner.ScanOnStartup config option, default true Signed-off-by: Deluan <deluan@navidrome.org> * feat: better json parsing error msg when importing NSPs Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update album's imported_time when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle interrupted scans and full scans after migrations Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `analyze` when migration requires a full rescan Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `PRAGMA optimize` at the end of the scan Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update artist's updated_at when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat: handle multiple artists and roles in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): dim missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * fix: album missing logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: error encoding in gob Signed-off-by: Deluan <deluan@navidrome.org> * feat: separate warnings from errors Signed-off-by: Deluan <deluan@navidrome.org> * fix: mark albums as missing if they were contained in a deleted folder Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add participant names to media_file and album tables Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use participations in criteria, instead of m2m relationship Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename participations to participants Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to album child Signed-off-by: Deluan <deluan@navidrome.org> * fix: albumartist role case Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): run scanner as an external process by default Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): show albumArtist names Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): dim out missing albums Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * fix(server): scrobble buffer mapping. fix #3583 Signed-off-by: Deluan <deluan@navidrome.org> * refactor: more participations renaming Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling Signed-off-by: Deluan <deluan@navidrome.org> * feat: send release_group_mbid to listenbrainz Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement OpenSubsonic explicitStatus field (#3597) * feat: implement OpenSubsonic explicitStatus field * fix(subsonic): fix failing snapshot tests * refactor: create helper for setting explicitStatus * fix: store smaller values for explicit-status on database * test: ToAlbum explicitStatus * refactor: rename explicitStatus helper function --------- Co-authored-by: Deluan Quintão <deluan@navidrome.org> * fix: handle album and track tags in the DB based on the mappings.yaml file Signed-off-by: Deluan <deluan@navidrome.org> * save similar artists as JSONB Signed-off-by: Deluan <deluan@navidrome.org> * fix: getAlbumList byGenre Signed-off-by: Deluan <deluan@navidrome.org> * detect changes in PID configuration Signed-off-by: Deluan <deluan@navidrome.org> * set default album PID to legacy_pid Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * fix SIGSEGV Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't lose album stars/ratings when migrating Signed-off-by: Deluan <deluan@navidrome.org> * store full PID conf in properties Signed-off-by: Deluan <deluan@navidrome.org> * fix: keep album annotations when changing PID.Album config Signed-off-by: Deluan <deluan@navidrome.org> * fix: reassign album annotations Signed-off-by: Deluan <deluan@navidrome.org> * feat: use (display) albumArtist and add links to each artist Signed-off-by: Deluan <deluan@navidrome.org> * fix: not showing albums by albumartist Signed-off-by: Deluan <deluan@navidrome.org> * fix: error msgs Signed-off-by: Deluan <deluan@navidrome.org> * fix: hide PID from Native API Signed-off-by: Deluan <deluan@navidrome.org> * fix: album cover art resolution Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim participant names Signed-off-by: Deluan <deluan@navidrome.org> * fix: reduce watcher log spam Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic when initializing the watcher Signed-off-by: Deluan <deluan@navidrome.org> * fix: various artists Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't store empty lyrics in the DB Signed-off-by: Deluan <deluan@navidrome.org> * remove unused methods Signed-off-by: Deluan <deluan@navidrome.org> * drop full_text indexes, as they are not being used by SQLite Signed-off-by: Deluan <deluan@navidrome.org> * keep album created_at when upgrading Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null pointer Signed-off-by: Deluan <deluan@navidrome.org> * fix: album artwork cache Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't expose missing files in Subsonic API Signed-off-by: Deluan <deluan@navidrome.org> * refactor: searchable interface Signed-off-by: Deluan <deluan@navidrome.org> * fix: filter out missing items from subsonic search * fix: filter out missing items from playlists * fix: filter out missing items from shares Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add filter by artist role Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): only return albumartists in getIndexes and getArtists endpoints Signed-off-by: Deluan <deluan@navidrome.org> * sort roles alphabetically Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist playcounts Signed-off-by: Deluan <deluan@navidrome.org> * change default Album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * fix albumartist link when it does not match any albumartists values Signed-off-by: Deluan <deluan@navidrome.org> * fix `Ignoring filter not whitelisted` (role) message Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim any names/titles being imported Signed-off-by: Deluan <deluan@navidrome.org> * remove unused genre code Signed-off-by: Deluan <deluan@navidrome.org> * serialize calls to Last.fm's getArtist Signed-off-by: Deluan <deluan@navidrome.org> xxx Signed-off-by: Deluan <deluan@navidrome.org> * add counters to genres Signed-off-by: Deluan <deluan@navidrome.org> * nit: fix migration `notice` message Signed-off-by: Deluan <deluan@navidrome.org> * optimize similar artists query Signed-off-by: Deluan <deluan@navidrome.org> * fix: last.fm.getInfo when mbid does not exist Signed-off-by: Deluan <deluan@navidrome.org> * ui only show missing items for admins Signed-off-by: Deluan <deluan@navidrome.org> * don't allow interaction with missing items Signed-off-by: Deluan <deluan@navidrome.org> * Add Missing Files view (WIP) Signed-off-by: Deluan <deluan@navidrome.org> * refactor: merged tag_counts into tag table Signed-off-by: Deluan <deluan@navidrome.org> * add option to completely disable automatic scanner Signed-off-by: Deluan <deluan@navidrome.org> * add delete missing files functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: playlists not showing for regular users Signed-off-by: Deluan <deluan@navidrome.org> * reduce updateLastAccess frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * reduce update player frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * add timeout when updating player Signed-off-by: Deluan <deluan@navidrome.org> * remove dead code Signed-off-by: Deluan <deluan@navidrome.org> * fix duplicated roles in stats Signed-off-by: Deluan <deluan@navidrome.org> * add `; ` to artist splitters Signed-off-by: Deluan <deluan@navidrome.org> * fix stats query Signed-off-by: Deluan <deluan@navidrome.org> * more logs Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * add record label filter Signed-off-by: Deluan <deluan@navidrome.org> * add release type filter Signed-off-by: Deluan <deluan@navidrome.org> * fix purgeUnused tags Signed-off-by: Deluan <deluan@navidrome.org> * add grouping filter to albums Signed-off-by: Deluan <deluan@navidrome.org> * allow any album tags to be used in as filters in the API Signed-off-by: Deluan <deluan@navidrome.org> * remove empty tags from album info Signed-off-by: Deluan <deluan@navidrome.org> * comments in the migration Signed-off-by: Deluan <deluan@navidrome.org> * fix: Cannot read properties of undefined Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling (#3640) Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove duplicated tag values Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't ignore the taglib folder! Signed-off-by: Deluan <deluan@navidrome.org> * feat: show track subtitle tag Signed-off-by: Deluan <deluan@navidrome.org> * fix: show artists stats based on selected role Signed-off-by: Deluan <deluan@navidrome.org> * fix: inspect Signed-off-by: Deluan <deluan@navidrome.org> * add media type to album info/filters Signed-off-by: Deluan <deluan@navidrome.org> * fix: change format of subtitle in the UI Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in Subsonic API and search Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in UI's player Signed-off-by: Deluan <deluan@navidrome.org> * fix: split strings should be case-insensitive Signed-off-by: Deluan <deluan@navidrome.org> * disable ScanSchedule Signed-off-by: Deluan <deluan@navidrome.org> * increase default sessiontimeout Signed-off-by: Deluan <deluan@navidrome.org> * add sqlite command line tool to docker image Signed-off-by: Deluan <deluan@navidrome.org> * fix: resources override Signed-off-by: Deluan <deluan@navidrome.org> * fix: album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * change migration to mark current artists as albumArtists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Allow filtering on multiple genres (#3679) * feat(ui): Allow filtering on multiple genres Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> * add multi-genre filter in Album list Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> * add more multi-valued tag filters to Album and Song views Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): unselect missing files after removing Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): song filter Signed-off-by: Deluan <deluan@navidrome.org> * fix sharing tracks. fix #3687 Signed-off-by: Deluan <deluan@navidrome.org> * use rowids when using search for sync (ex: Symfonium) Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients for search Signed-off-by: Deluan <deluan@navidrome.org> * add libraryPath to Native API /songs endpoint Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add album version Signed-off-by: Deluan <deluan@navidrome.org> * made all tags lowercase as they are case-insensitive anyways. Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Show full paths, extended properties for album/song (#3691) * feat(ui): Show full paths, extended properties for album/song - uses library path + os separator + path - show participants (album/song) and tags (song) - make album/participant clickable in show info * add source to path * fix pathSeparator in UI Signed-off-by: Deluan <deluan@navidrome.org> * fix local artist artwork (#3695) Signed-off-by: Deluan <deluan@navidrome.org> * fix: parse vorbis performers Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean function into smaller functions Signed-off-by: Deluan <deluan@navidrome.org> * fix translations for en and pt Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow performers without instrument/subrole Signed-off-by: Deluan <deluan@navidrome.org> * refactor: metadata clean function again Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize split function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split function is now a method of TagConf Signed-off-by: Deluan <deluan@navidrome.org> * fix: humanize Artist total size Signed-off-by: Deluan <deluan@navidrome.org> * add album version to album details Signed-off-by: Deluan <deluan@navidrome.org> * don't display album-level tags in SongInfo Signed-off-by: Deluan <deluan@navidrome.org> * fix genre clicking in Album Page Signed-off-by: Deluan <deluan@navidrome.org> * don't use mbids in Last.fm api calls. From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248: With MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo { artist: { name: "Bee Gees", mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810", url: "https://www.last.fm/music/Bee+Gees", } ``` Without MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo { artist: { name: "Van Morrison", mbid: "a41ac10f-0a56-4672-9161-b83f9b223559", url: "https://www.last.fm/music/Van+Morrison", } ``` Signed-off-by: Deluan <deluan@navidrome.org> * better logging for when the artist folder is not found Signed-off-by: Deluan <deluan@navidrome.org> * fix various issues with artist image resolution Signed-off-by: Deluan <deluan@navidrome.org> * hide "Additional Tags" header if there are none. Signed-off-by: Deluan <deluan@navidrome.org> * simplify tag rendering Signed-off-by: Deluan <deluan@navidrome.org> * enhance logging for artist folder detection Signed-off-by: Deluan <deluan@navidrome.org> * make folderID consistent for relative and absolute folderPaths Signed-off-by: Deluan <deluan@navidrome.org> * handle more folder paths scenarios Signed-off-by: Deluan <deluan@navidrome.org> * filter out other roles when SubsonicArtistParticipations = true Signed-off-by: Deluan <deluan@navidrome.org> * fix "Cannot read properties of undefined" Signed-off-by: Deluan <deluan@navidrome.org> * fix lyrics and comments being truncated (#3701) * fix lyrics and comments being truncated * specifically test for lyrics and comment length * reorder assertions Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * fix(server): Expose library_path for playlist (#3705) Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic) * fix BFR on Windows (#3704) * fix potential reflected cross-site scripting vulnerability Signed-off-by: Deluan <deluan@navidrome.org> * hack to make it work on Windows * ignore windows executables * try fixing the pipeline Signed-off-by: Deluan <deluan@navidrome.org> * allow MusicFolder in other drives * move windows local drive logic to local storage implementation --------- Signed-off-by: Deluan <deluan@navidrome.org> * increase pagination sizes for missing files Signed-off-by: Deluan <deluan@navidrome.org> * reduce level of "already scanning" watcher log message Signed-off-by: Deluan <deluan@navidrome.org> * only count folders with audio files in it See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930 Signed-off-by: Deluan <deluan@navidrome.org> * add album version and catalog number to search Signed-off-by: Deluan <deluan@navidrome.org> * add `organization` alias for `recordlabel` Signed-off-by: Deluan <deluan@navidrome.org> * remove mbid from Last.fm agent Signed-off-by: Deluan <deluan@navidrome.org> * feat: support inspect in ui (#3726) * inspect in ui * address round 1 * add catalogNum to AlbumInfo Signed-off-by: Deluan <deluan@navidrome.org> * remove dependency on metadata_old (deprecated) package Signed-off-by: Deluan <deluan@navidrome.org> * add `RawTags` to model Signed-off-by: Deluan <deluan@navidrome.org> * support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698) * parse standard roles, vorbis/m4a work for now * fix djmixer * working roles, use DJ-mix * add performers to file * map mbids * add a few more tests * add test Signed-off-by: Deluan <deluan@navidrome.org> * try to simplify the performers logic Signed-off-by: Deluan <deluan@navidrome.org> * stylistic changes --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * remove param mutation Signed-off-by: Deluan <deluan@navidrome.org> * run automated SQLite optimizations Signed-off-by: Deluan <deluan@navidrome.org> * fix playlists import/export on Windows * fix import playlists * fix export playlists * better handling of Windows volumes Signed-off-by: Deluan <deluan@navidrome.org> * handle more album ID reassignments Signed-off-by: Deluan <deluan@navidrome.org> * allow adding/overriding tags in the config file Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): Fix playlist track id, handle missing tracks better (#3734) - Use `mediaFileId` instead of `id` for playlist tracks - Only fetch if the file is not missing - If extractor fails to get the file, also error (rather than panic) * optimize DB after each scan. Signed-off-by: Deluan <deluan@navidrome.org> * remove sortable from AlbumSongs columns Signed-off-by: Deluan <deluan@navidrome.org> * simplify query to get missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * mark Scanner.Extractor as deprecated Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Caio Cotts <caio@cotts.com.br> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com> |
||
|
Deluan
|
4b70daf143 | Converted scanner to a proper Import "class", to facilitate testing | ||
|
Deluan
|
4843ccb46c | go fmt | ||
|
Deluan
|
272a499c7e | Refactoring to a cleaner architecture | ||
|
Deluan
|
c64a0f8242 | Implemented proper index grouping | ||
|
Deluan
|
54460d5860 | Empty tests should be skipped | ||
|
Deluan
|
1c47b7b8ef | Added empty tests in each package, to get the more precise coverage percentage |