Compare commits

..

87 commits

Author SHA1 Message Date
Deluan
a998a329e6 test(artwork): use renamed ArtworkWorkerConcurrency in e2e tests 2026-07-23 14:09:39 -04:00
Deluan
42fd263cdf fix(artwork): clamp negative sizes to full-size; convert imghttp test to Ginkgo
- A negative size (Subsonic size / Jellyfin maxwidth accept signed ints) reached
  resizeStaticImage, where the square path builds image.NewNRGBA(Rect(0,0,size,size))
  — a giant rectangle that panics/OOMs. Clamp size<0 to 0 (full-size) at the Service
  entry. Positive sizes were already clamped to the original.
- imghttp used a plain func Test with a table; convert to a Ginkgo DescribeTable with
  the suite entry point in imghttp_suite_test.go (AGENTS.md test-framework requirement).
2026-07-23 14:08:08 -04:00
Deluan
f92507b152 fix(artwork): preserve the drive when normalizing Windows file:// library paths
url.Parse puts the volume of file://C:/Music in Host, not Path, so localOSRoot dropped
it and returned /Music — os.Open/os.Stat then failed and folder/embedded art on Windows
looped as dangling. Rejoin the host volume, matching core/storage/local's newLocalStorage.
2026-07-23 14:08:08 -04:00
Deluan
aba7ed925c fix(artwork): honor disabled per-track art at serve time; use nanosecond mtime provenance
Two serving-correctness fixes from review:
- serveMediaFile served a persisted mf embedded image even after EnableMediaFileCoverArt
  was turned off (the setting isn't in the config fingerprint, so found rows aren't
  reprocessed). Direct mf- URLs now honor the setting at serve time and fall back to
  disc/album art.
- The file-backed staleness check compared whole-second mtimes, so a same-second content
  replacement (two writes in one second, or timestamp-preserving tools) could serve
  different bytes under the old hash + immutable policy. RefMtime is now unix-nanoseconds
  (no schema change; int64 column), detecting sub-second changes where the filesystem
  records them.
2026-07-23 14:08:08 -04:00
Deluan
aa6c0b1f17 fix(artwork): enqueue new empty playlists by id, and refresh on absent outcomes
Two worker/enqueue fixes from review:
- playlistRepository.Put assigned the generated id to the caller's Playlist but passed
  the stale copy (empty id) to refreshCounters, enqueueing a pl|"" row the worker
  failed until the daily dangling purge while the real playlist went unresolved. Set
  the id on the copy before enqueueing.
- The drain refresh batch only included found/foundStale, so a cover removed by a scan
  (found -> absent) never notified clients, leaving the old immutable image displayed.
  Broadcast absent outcomes too; precache still only warms found/foundStale.
2026-07-23 14:08:08 -04:00
Deluan
e9c15d7fcb fix(artwork): don't stamp the album hash onto multi-disc tracks
The hydration fallback assigned a found album hash to every fallback track, but a
multi-disc track's CoverArtID emits a dc- id served from disc-specific art whose hash
is unknown at hydration time. Advertising dc-..._<albumHash> gave clients a content-
version that never changes when the disc image does, breaking id-based refresh. Only
stamp the album hash for single-disc tracks (DiscNumber == 0); multi-disc tracks stay
unhashed and rely on the correct ETag returned by the served response.
2026-07-23 14:08:08 -04:00
Deluan
3eaa21229d fix(artwork): version the artwork ETag with the served representation
The ETag was the pixel hash of the original image, so a CoverArtQuality or
EnableWebPEncoding change altered the resized bytes without changing the ETag —
revalidating clients got a spurious 304 and kept the old encoding. Resized responses
now carry a representation ETag (hash + size + square + encode settings) used for the
ETag header and If-None-Match, while the immutable decision stays on the pixel hash
(URLs remain pixel-identity per the spec, so hash-suffixed clients keep zero-request
caching). Full-size originals fall back to the pixel hash as before.
2026-07-23 14:08:08 -04:00
Deluan
9dd306eb10 fix(artwork): enforce entity visibility on the Subsonic getCoverArt path
serveEntity reads persisted item_artwork by id, bypassing the library and private-
playlist filters that the legacy entity-load applied. On the authenticated Subsonic
path a user could fetch artwork for an inaccessible album or someone else's private
playlist by guessing an id. getCoverArt now resolves the underlying entity through
the request-scoped (filtered) repositories and serves the placeholder when it is not
visible, so existence isn't leaked and the always-an-image invariant holds. The
public share (JWT-authorized) and Jellyfin (admin) paths are intentionally untouched.
2026-07-23 14:08:08 -04:00
Deluan
8d715ba2fa fix(artwork): restore synthetic-artist guard and unicode normalization in agent lookups
Moving agent calls into the worker bypassed two behaviors of the aggregate provider:
Agents.GetArtistImages' guard for Unknown/Various Artists (a direct retriever call
could assign an unrelated image to a synthetic artist), and auxAlbum/auxArtist.Name's
DevPreserveUnicodeInExternalCalls normalization (records with typographic quotes/dashes
missed exact-name searches). Re-apply both before enumerating retrievers.
2026-07-23 14:08:08 -04:00
Deluan
50ada9ad29 fix(artwork): invalidate artwork when an uploaded image is deleted
Deleting an artist/radio/playlist upload cleared the filename but left the found
item_artwork row and its hash, so lists kept advertising the deleted cover's
hash-suffixed immutable URL and clients could display it indefinitely. Call
EnqueueArtwork after the delete-side Put, symmetric with upload, so the state is
cleared and re-resolved to the next source (or absent).
2026-07-23 14:08:08 -04:00
Deluan
ed4178a6a9 fix(artwork): only use disc resolution for multi-disc albums
DiscCoverArtID returns a dc- id for any track with DiscNumber>0, so serveDisc ran
the full DiscArtPriority chain even for single-disc albums, where a stray disc*/
embedded image could shadow higher-priority album art. Gate disc resolution on the
album having more than one disc, matching the legacy reader; single-disc tracks
serve album art directly.
2026-07-23 14:08:08 -04:00
Deluan
ce06599288 fix(artwork): open library-backed artwork through its on-disk root
A library configured with a file:// path stored absRoot as the raw URI, so Abs
produced strings like file:/music/cover.jpg that os.Open/os.Stat reject — folder,
upload and embedded art were treated as dangling on every request, looping forever.
Normalize a file:// path to its parsed OS path (the same root os.DirFS uses);
non-local schemes are left unchanged (out of scope, per the artwork-musicfs TODO).
2026-07-23 14:08:08 -04:00
Deluan
f016192eec fix(artwork): requeue playlist cover when its track set changes
A generated-grid cover went stale after track mutations: nothing re-resolved the
playlist's artwork, and the request path deliberately never rebuilds the grid, so
serveEntity kept returning the old grid hash indefinitely. Enqueue pl artwork from
refreshCounters (the choke point for every track-set change); no clear, so the old
cover keeps serving until the worker rebuilds.
2026-07-23 14:08:08 -04:00
Deluan
cfca4a2433 fix(artwork): serve a local playlist ExternalImageURL as a file-backed reference
A local ExternalImageURL was resolved through the external step and labelled
external, so placeBytes copied it into the content-addressed store and dropped its
path/mtime — replacing the file never tripped the staleness check. Classify local
references as file-backed (resolved in place, even on the request path) and keep
store-backed behaviour only for http(s) URLs.
2026-07-23 14:08:08 -04:00
Deluan
49039fab47 fix(artwork): keep multi-disc tracks requestable when the album is absent
Round-1's hydration fix still copied the album's known-absent onto a non-eligible
(or own-absent) track, but MediaFile.CoverArtID routes a multi-disc track to disc
art, which resolves provisionally and is never known-absent. Marking it absent made
Subsonic omit coverArt so clients never requested a valid disc image. Only mark a
single-disc track absent, and only when its own art won't resolve.
2026-07-23 14:08:08 -04:00
Deluan
66d3d23149 fix(artwork): enqueue uploaded artwork only after the filename is persisted
SetImage cleared state and enqueued the bump before the caller stored the new
filename, so a worker drain in that window could resolve against the old (already
deleted) file and settle absent, leaving the upload unused until a later scan. Move
the invalidate+enqueue into EnqueueArtwork, which each caller now invokes after the
entity Put.
2026-07-23 14:08:08 -04:00
Deluan
f4e14e9c1a fix(artwork): fall back to disc art, not the album, for multi-disc tracks
serveMediaFile delegated an absent/ineligible track straight to AlbumCoverArtID,
skipping the disc-specific lookup that MediaFile.CoverArtID (and the deleted legacy
reader) use. On multi-disc albums with per-disc images that served the album cover
instead of the configured disc artwork. Delegate through DiscCoverArtID.
2026-07-23 14:08:08 -04:00
Deluan
d15d85ad0c fix(artwork): validate each agent image URL before picking the largest
bestImageURL selected the largest by size and only then parsed it, so a malformed
largest URL (e.g. a bad percent-escape) returned nil and shadowed a valid smaller
candidate, contradicting the documented skip-unparseable behavior. Parse per
candidate and compare sizes only among URLs that parse.
2026-07-23 14:08:08 -04:00
Deluan
0781c4a9b2 fix(artwork): keep an eligible track's cover requestable when its album is absent
An embedded-eligible track with no resolved item_artwork row inherited the album's
ImageAbsent, so when the album resolved absent (e.g. CoverArtPriority without
'embedded') the track's coverArt was omitted permanently — the client never
requested it, so the lazy mediafile path never resolved it — even though the
serving path would extract and serve the track's own embedded art. Hydration now
never copies the album's absence onto an eligible-but-unresolved track.
2026-07-23 14:08:08 -04:00
Deluan
ca4220b029 fix(artwork): request read-through must not reset the failure backoff
The provisional read-through and dangling re-enqueue used Enqueue, whose upsert
resets retry_at, so any browse of an unresolved entity that was backing off after
an external failure made it immediately eligible again — defeating the exponential
backoff during a provider outage. Add EnqueueBump, which raises priority but leaves
an existing row's retry_at intact, and route the serving path through it. Scan and
manual re-resolve keep Enqueue's reset (a detected change wants immediate retry).
2026-07-23 14:08:08 -04:00
Deluan
c2d7ae773c chore(artwork): generic 500 bodies on refresh endpoint, trim stale test comments 2026-07-23 14:08:08 -04:00
Deluan
dfd4bec270 test(artwork): end-to-end coverage for the serving cutover 2026-07-23 14:08:08 -04:00
Deluan
0d1df1648e feat(artwork): precache on acquisition, bump on upload/radio changes, manual re-resolve API 2026-07-23 14:08:08 -04:00
Deluan
937e58e5fb refactor(artwork): delete the legacy reader chain, cache warmer, and provider image methods 2026-07-23 14:08:08 -04:00
Deluan
8fea7efa50 feat(subsonic): content-hash coverArt ids, omit artwork on known-absent 2026-07-23 14:08:08 -04:00
Deluan
25b32f9706 feat(server): serve artwork from persisted state with content-hash caching 2026-07-23 14:08:08 -04:00
Deluan
313998fd65 feat(artwork): state-backed serving path with provisional read-through 2026-07-23 14:08:08 -04:00
Deluan
3a9dadfe34 fix(artwork): broadcast refresh for stale-found artwork too 2026-07-23 14:08:08 -04:00
Deluan
05ba549843 feat(artwork): broadcast refresh events when artwork lands 2026-07-23 14:08:08 -04:00
Deluan
5179691811 feat(artwork): resolve media_file embedded art in the worker, invalidate on rescan 2026-07-23 14:07:52 -04:00
Deluan
2ab1323b28 feat(persistence): hydrate artwork hash and absence onto entity pages 2026-07-23 14:07:52 -04:00
Deluan
3b7cbf41dd feat(model): content-hash artwork id suffix and hydratable per-entity image state 2026-07-23 14:07:52 -04:00
Deluan
39e939686b fix(artwork): treat agent not-found as breaker success 2026-07-23 14:07:52 -04:00
Deluan
190c291e61 feat(artwork): worker fetches agent images directly with per-agent rate limits and breakers 2026-07-23 14:07:52 -04:00
Deluan
b7f94f6727 feat(agents): enumerate enabled image-retriever agents per capability 2026-07-23 14:06:30 -04:00
Deluan
f34a386137 fix(deezer): never return empty-image-id placeholder pictures 2026-07-23 14:06:30 -04:00
Deluan
fc55e8bf16 feat(artwork): promote worker concurrency and external rate to real configs
The artwork worker's drain speed was governed by two hidden Dev flags,
DevArtworkWorkerConcurrency and DevArtworkExternalRPS, both defaulting to 2.
On a large library's one-time backfill the external rate limiter is the real
ceiling: every art-less item waits on it before the (rate-limited) external
lookup, so the drain crawls at ~RPS items/sec while local-art items are
unaffected.

Promote both to documented, supported options: ArtworkWorkerConcurrency
(default 4) sets local-resolution parallelism, ArtworkExternalMaxRPS
(default 2, 0 = unlimited) caps external-agent lookups to stay polite to
Last.fm/Deezer/etc. Operators can now trade first-backfill speed against
external-API rate limits. The old Dev names still map for backward compat.
2026-07-23 14:05:10 -04:00
Deluan
9ce51cf575 perf(artwork): fetch only IDs for backfill enumeration
Backfill enumerated every album, artist, playlist and radio via GetAll
and mapped out just the ID. GetAll materializes full entities (library
joins, participant/stats/tags JSON, annotation, artwork hydration), so on
a large library it loaded tens of thousands of heavy structs only to read
one field each — spiking transient RSS to ~1GB during the one-time
upgrade backfill, a memory risk on small NAS/Pi hardware.

Add GetAllIDs to the album, artist, playlist and radio repositories: it
reuses each repo's base row-set filter (library visibility, artist
content join, playlist userFilter) but projects only id, skipping the
heavy columns and post-processing. A per-repo parity test asserts
GetAllIDs returns exactly the same id set as GetAll.

Verified on a 727MB / 29k-artist production DB copy: peak RSS during
backfill dropped from ~1012MB to ~89MB, file descriptors flat, same
36,138 items enqueued.
2026-07-23 13:20:22 -04:00
Deluan
b172ce4296 refactor(artwork): reuse auth.WithAdminUser and dedupe image cap guards 2026-07-22 18:02:54 -04:00
Deluan
bba0eab3a5 fix(artwork): apply image limits to playlist tile decoding
decodeTile ran image.Decode on every sampled album's resolved bytes
before processItem's maxImageBytes/maxImagePixels guards applied,
letting an oversized or decompression-bomb tile fully decode
unbounded. Enforce both caps inside decodeTile itself.
2026-07-22 17:09:22 -04:00
Deluan
f614850ff0 fix(artwork): store backing-file provenance per item, not per hash 2026-07-22 15:54:08 -04:00
Deluan
ba2290af6d test(artwork): convert non-synctest timing tests to Ginkgo specs
TestArtworkBackoffSchedule and TestArtworkWorkerRunNoLeak needed no real
*testing.T (no synctest), so move them into worker_test.go as Ginkgo
specs. TestArtworkBreakerHalfOpen stays plain since testing/synctest
requires a real *testing.T, matching core/scrobbler's precedent.
2026-07-22 15:53:32 -04:00
Deluan
55608b2d20 fix(artwork): resolve private playlists with an admin context 2026-07-22 15:53:32 -04:00
Deluan
7713a6d6b2 fix(artwork): include M3U external art flag in the config fingerprint 2026-07-22 15:53:32 -04:00
Deluan
bc30ce67c6 fix(artwork): keep fresh re-enqueues ahead of stale failure backoff 2026-07-22 15:53:32 -04:00
Deluan
d6434b9929 fix(artwork): reject decompression-bomb dimensions before decoding 2026-07-22 15:53:32 -04:00
Deluan
b3526c0fba test(artwork): make leak and permission tests pass on linux
goleak now ignores notify's nonrecursive-tree goroutines (linux uses
inotify, which spawns dispatch+internal instead of darwin's recursive
dispatch), and the read-only-dir prune spec skips under root, where
permission bits cannot make Remove fail.
2026-07-22 15:53:32 -04:00
Deluan
3d32157403 test(artwork): move soak test into the Ginkgo suite 2026-07-22 15:53:32 -04:00
Deluan
5482784bfc fix(artwork): treat playlist cover URL 404 as definitive miss
The playlist ExternalImageURL step used sources.go's fromURL, which maps any
non-200 to a generic error, so a stale URL returning 404/410 was classified
transient: infinite backoff plus it counted toward the circuit breaker,
blocking valid external work. Add a local fetch in resolve.go that maps
404/410 to model.ErrNotFound (definitive) while keeping other non-200s
transient. sources.go is left untouched.
2026-07-22 15:53:32 -04:00
Deluan
6afcb93a9b fix(artwork): retry higher-priority external art after fallback hit
With CoverArtPriority="external,cover.jpg", a transient external failure
followed by a folder hit dropped the external error: the worker recorded
found and deleted the queue row, so the configured higher-priority external
art was never retried. Carry extError onto the fallback resolution and add
an outcomeFoundStale that persists+serves the art but reschedules via
MarkFailed, giving the external source another chance. When external later
answers definitively-not-found, the hit is not stale and the row is deleted.
2026-07-22 15:53:32 -04:00
Deluan
67f6d8aee8 fix(artwork): cap resolved image reads
A user-editable ExternalImageURL can point at an arbitrarily large endpoint;
a fast server could make the worker buffer hundreds of MB inside the 5s HTTP
timeout. Bound the read to a fixed 20MB cap (no config knob) via io.LimitReader
and fail the item if it is exceeded.
2026-07-22 15:53:32 -04:00
Deluan
87095fab08 refactor(artwork): deduplicate purge loop, backfill table, and extGate alias 2026-07-22 15:53:32 -04:00
Deluan
c57496d50d fix(artwork): treat missing local playlist cover as definitive, not transient
A playlist ExternalImageURL pointing at a local file that fails to open was
routed through extError, causing failed/48h-retry loops that burn a rate
limiter token forever instead of falling through to the generated grid.
2026-07-22 15:53:32 -04:00
Deluan
0fbbd01357 style(artwork): fix comment accuracy and budget; fingerprint ArtistImageFolder
Correct the inverted workerDeps.extGate comment, trim over-budget doc comments, and add conf.Server.ArtistImageFolder to the resolution fingerprint so an image-folder change re-resolves artist artwork.
2026-07-22 15:53:32 -04:00
Deluan
bab9b5cd3a fix(artwork): purge dangling queue rows and guard concurrent re-enqueues
Queue rows for deleted entities failed forever (Get -> ErrNotFound -> failed -> capped retries, unbounded). Add ArtworkQueueRepository.PurgeDangling, called from Prune next to the item_artwork purge. Separately, the found/absent path unconditionally deleted the dequeued row, erasing a concurrent scan re-enqueue; switch to DeleteIfUnchanged, which deletes only while retry_at still matches the dequeued value (verified retry_at is the column an Enqueue upsert resets).
2026-07-22 15:53:32 -04:00
Deluan
454fd24833 fix(artwork): resolve full playlist source chain
resolvePlaylist only built the generated grid, dropping the uploaded-image, sidecar and ExternalImageURL sources the old reader_playlist.go chain serves. Port the full chain before the grid fallback: uploaded (upload), sidecar (folder), and ExternalImageURL routed through extGate with the same extError semantics as the other external steps. Also rewires the artist external step onto ArtistImageResult.
2026-07-22 15:53:32 -04:00
Deluan
c01e9b3184 fix(artwork): propagate transient artist image errors to the worker
callGetImage swallowed all agent errors, so an agent outage surfaced as ErrNotFound and the worker settled artist artwork as a definitive absent (and reset the breaker). Add an additive ArtistImageResult path that returns the underlying agent error on transient failure while keeping ArtistImage byte-identical for existing callers; the worker's artist external step uses it via fromArtistExternalResult.
2026-07-22 15:53:32 -04:00
Deluan
1ed8ebf9b0 test(artwork): leak/soak coverage and deferred assertions 2026-07-22 15:53:32 -04:00
Deluan
ad38cd1d58 feat(artwork): artwork backfill, fingerprint re-resolution and scheduled jobs 2026-07-22 15:53:32 -04:00
Deluan
25a05fd017 feat(artwork): enqueue artwork resolution from scan and CRUD paths 2026-07-22 15:53:32 -04:00
Deluan
57c64e386a feat(artwork): add acquisition worker service 2026-07-22 15:53:32 -04:00
Deluan
d6fc829f84 style(artwork): tighten processor comments to budget 2026-07-22 15:53:32 -04:00
Deluan
e0655dc882 feat(artwork): add acquisition processor
Resolves one queue item end to end: hash/dedup, decode + 128px thumbnail
blurhash, place bytes (store vs source file), and persist found/absent/
failed state for the worker (Task 4) to act on.
2026-07-22 15:53:32 -04:00
Deluan
608db503a7 fix(artwork): propagate playlist tile failures and dedupe external step 2026-07-22 15:53:32 -04:00
Deluan
967de74bf7 feat(artwork): add worker-side artwork resolvers 2026-07-22 15:53:32 -04:00
Deluan
2efa697e52 feat(artwork): import blurhash encoder from #5797 2026-07-22 15:53:32 -04:00
Deluan
1f818e7633 fix(artwork): store backing-file provenance per item, not per hash 2026-07-22 15:52:54 -04:00
Deluan
c04c8ee02a fix(artwork): mock PutImage refreshes created_at like the SQL repository
Prune specs now age fixtures directly instead of seeding stale timestamps through the upsert.
2026-07-22 00:58:22 -04:00
Deluan
ebbe533c6a fix(artwork): reject malformed hashes in ImageStore operations
Known-absent states carry an empty hash and malformed persisted hashes could panic path sharding or inject separators; Write/Open/Remove now return an error for anything but 16 lowercase hex chars.
2026-07-22 00:51:13 -04:00
Deluan
0147cc59b1 fix(artwork): honor the orphan cutoff in the repository mock
The mock's DeleteOrphans now applies createdBefore like the SQL implementation, and a new spec covers a freshly reacquired row surviving prune.
2026-07-22 00:42:42 -04:00
Deluan
034cd17498 fix(artwork): index artwork_queue in dequeue order
The previous leading retry_at range column forced a temp B-tree sort of the whole eligible set on every DequeueBatch; ordering the index by (priority DESC, enqueued_at) lets scans stop after the batch size.
2026-07-22 00:36:22 -04:00
Deluan
8147f7c40b fix(artwork): rewrite vanished duplicates and sweep stale mime variants
Write falls through to a real write when the liveness touch fails, and sweep retention now matches the recorded mime's extension so obsolete variants are reclaimed.
2026-07-22 00:28:51 -04:00
Deluan
bf614e66ad fix(artwork): guard orphan file removal with the prune grace window
Duplicate ImageStore writes refresh the file mtime and Remove skips files newer than the cutoff, so overlapping acquisitions cannot lose their store files to a concurrent prune.
2026-07-22 00:19:45 -04:00
Deluan
623b7d6a6c fix(artwork): atomic orphan deletion and timestamp semantics from review
DeleteOrphans re-checks age+references at delete time, PutItemArtwork defaults attempted_at, queue mock timestamps mirror SQL.
2026-07-22 00:11:15 -04:00
Deluan
6f7f9c6463 fix(artwork): address review findings on prune/sweep races and mock fidelity
Sweep now honors an mtime grace window (in-flight acquisitions and temp files), reacquired orphans reset the prune grace window, and the queue mock implements real stale-absent semantics.
2026-07-21 23:57:58 -04:00
Deluan
8fd7ef19f3 refactor(artwork): apply simplify-pass cleanups
Internal item_artwork sqlRepository helper, toSQLArgs upserts, batched queue enqueue, EnqueueStaleAbsent moved to queue repo, snapshot-based prune sweep, mock/real semantics aligned.
2026-07-21 23:37:05 -04:00
Deluan
1041e45ca7 fix(artwork): chunk unbounded IN clauses and restore interface docs 2026-07-21 23:24:42 -04:00
Deluan
4f835437a9 refactor(artwork): merge item artwork state into ArtworkRepository 2026-07-21 23:14:01 -04:00
Deluan
b16ef725c9 refactor(artwork): fold originals package into core/artwork as ImageStore 2026-07-21 23:09:31 -04:00
Deluan
3e7685adc2 fix(artwork): never sweep files on transient DB errors during prune 2026-07-21 23:02:47 -04:00
Deluan
db16b3de9a feat(artwork): add artwork prune (orphan cleanup) 2026-07-21 22:58:20 -04:00
Deluan
b72597821a feat(artwork): add content-addressed originals store 2026-07-21 22:53:00 -04:00
Deluan
fcff9c63e7 feat(artwork): implement artwork_queue repository 2026-07-21 22:45:08 -04:00
Deluan
14dd57052e feat(artwork): implement item_artwork repository with batched hydration 2026-07-21 22:45:03 -04:00
Deluan
f926539c04 feat(artwork): implement artwork repository 2026-07-21 22:44:24 -04:00
Deluan
6cce65f759 feat(artwork): add artwork models, repository interfaces and mocks 2026-07-21 22:35:34 -04:00
Deluan
7aacb01f4f feat(artwork): add artwork, item_artwork and artwork_queue tables 2026-07-21 22:31:03 -04:00
980 changed files with 15744 additions and 57263 deletions

View file

@ -4,7 +4,7 @@
"dockerfile": "Dockerfile", "dockerfile": "Dockerfile",
"args": { "args": {
// Update the VARIANT arg to pick a version of Go: 1, 1.15, 1.14 // Update the VARIANT arg to pick a version of Go: 1, 1.15, 1.14
"VARIANT": "1.27", "VARIANT": "1.26",
// Options // Options
"INSTALL_NODE": "true", "INSTALL_NODE": "true",
"NODE_VERSION": "v24" "NODE_VERSION": "v24"

6
.github/FUNDING.yml vendored
View file

@ -1,10 +1,10 @@
# These are supported funding model platforms # These are supported funding model platforms
ko_fi: deluan
github: deluan github: deluan
open_collective: navidrome
liberapay: deluan
patreon: # Replace with a single Patreon username patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: deluan
liberapay: deluan
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
issuehunt: # Replace with a single IssueHunt username issuehunt: # Replace with a single IssueHunt username

View file

@ -68,11 +68,6 @@ runs:
- name: Set up Docker Buildx - name: Set up Docker Buildx
id: buildx id: buildx
uses: docker/setup-buildx-action@v4 uses: docker/setup-buildx-action@v4
with:
# Runner IPs are shared, so anonymous base image pulls get rate-limited.
buildkitd-config-inline: |
[registry."docker.io"]
mirrors = ["mirror.gcr.io"]
- name: Extract metadata for Docker image - name: Extract metadata for Docker image
id: meta id: meta

View file

@ -1,60 +0,0 @@
name: Report coverage on PR
on:
workflow_run:
workflows: ['Pipeline: Test, Lint, Build']
types: [completed]
jobs:
comment:
name: Comment coverage report
if: github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion == 'success'
runs-on: ubuntu-latest
permissions:
contents: read
actions: read
pull-requests: write
env:
COVERAGE_COMMENT: 'true'
steps:
# Only the config, from the base branch: this job holds a write token, so
# it must never check out the fork.
- name: Check out the octocov config
uses: actions/checkout@v7
with:
sparse-checkout: .octocov.yml
sparse-checkout-cone-mode: false
persist-credentials: false
# Into a subdirectory. A pull_request run executes the fork's own copy of
# pipeline.yml, so every file in here is attacker-controlled.
- uses: actions/download-artifact@v8
with:
name: octocov-pr
path: untrusted
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}
- name: Verify the artifact and take the coverage profile
id: pr
env:
GH_TOKEN: ${{ github.token }}
HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
run: |
number=$(head -c 20 untrusted/pr_number | tr -d '[:space:]')
case "$number" in ''|*[!0-9]*)
echo "::error::artifact pr_number is not a number"; exit 1;;
esac
sha=$(gh api "repos/$GITHUB_REPOSITORY/pulls/$number" --jq .head.sha)
if [ "$sha" != "$HEAD_SHA" ]; then
echo "::error::artifact claims PR #$number, but its head $sha is not $HEAD_SHA"; exit 1
fi
cp untrusted/coverage.out coverage.out
echo "number=$number" >> "$GITHUB_OUTPUT"
- uses: k1LoW/octocov-action@v1
env:
# A workflow_run job looks like a push to the default branch. Point
# octocov back at the pull request and at the run that produced it.
GITHUB_PULL_REQUEST_NUMBER: ${{ steps.pr.outputs.number }}
OCTOCOV_GITHUB_REF: refs/pull/${{ steps.pr.outputs.number }}/merge
OCTOCOV_GITHUB_SHA: ${{ github.event.workflow_run.head_sha }}
OCTOCOV_GITHUB_RUN_ID: ${{ github.event.workflow_run.id }}

View file

@ -34,19 +34,16 @@ jobs:
} }
const {data: {artifacts}} = await github.rest.actions.listWorkflowRunArtifacts({owner, repo, run_id}); const {data: {artifacts}} = await github.rest.actions.listWorkflowRunArtifacts({owner, repo, run_id});
const downloadable = artifacts.filter((art) => !art.name.startsWith('octocov-')); if (!artifacts.length) {
if (!downloadable.length) {
return core.error(`No artifacts found`); return core.error(`No artifacts found`);
} }
const header = `Download the artifacts for this pull request:`; let body = `Download the artifacts for this pull request:\n`;
let body = `${header}\n`; for (const art of artifacts) {
for (const art of downloadable) {
body += `\n* [${art.name}.zip](https://nightly.link/${owner}/${repo}/actions/artifacts/${art.id}.zip)`; body += `\n* [${art.name}.zip](https://nightly.link/${owner}/${repo}/actions/artifacts/${art.id}.zip)`;
} }
const {data: comments} = await github.rest.issues.listComments({repo, owner, issue_number}); const {data: comments} = await github.rest.issues.listComments({repo, owner, issue_number});
// Match on the body too: octocov also comments as github-actions[bot]. const existing_comment = comments.find((c) => c.user.login === 'github-actions[bot]');
const existing_comment = comments.find((c) => c.user.login === 'github-actions[bot]' && c.body.startsWith(header));
if (existing_comment) { if (existing_comment) {
core.info(`Updating comment ${existing_comment.id}`); core.info(`Updating comment ${existing_comment.id}`);
await github.rest.issues.updateComment({repo, owner, comment_id: existing_comment.id, body}); await github.rest.issues.updateComment({repo, owner, comment_id: existing_comment.id, body});

View file

@ -68,16 +68,10 @@ jobs:
with: with:
go-version-file: go.mod go-version-file: go.mod
# Keep CI on the same version `make lint` installs, so a clean local run
# cannot turn red in CI just because a new golangci-lint was released.
- name: Resolve golangci-lint version
id: golangci-version
run: echo "version=$(grep '^GOLANGCI_LINT_VERSION' Makefile | cut -d ' ' -f 3)" >> "$GITHUB_OUTPUT"
- name: golangci-lint - name: golangci-lint
uses: golangci/golangci-lint-action@v9 uses: golangci/golangci-lint-action@v9
with: with:
version: ${{ steps.golangci-version.outputs.version }} version: latest
problem-matchers: true problem-matchers: true
args: --timeout 2m args: --timeout 2m
@ -92,23 +86,8 @@ jobs:
exit 1 exit 1
fi fi
- name: Resolve OpenAPI tool versions
id: api-tools
run: echo "key=$(grep -E '^(VACUUM|OAPI_CODEGEN|OASDIFF)_VERSION' Makefile | tr -d ' \n')" >> "$GITHUB_OUTPUT"
- name: Cache OpenAPI tools
uses: actions/cache@v6
with:
path: bin
key: api-tools-${{ runner.os }}-${{ steps.api-tools.outputs.key }}
- name: Lint OpenAPI spec
run: make api-lint
- name: Run go generate - name: Run go generate
run: | run: go generate ./...
make api-gen
go generate ./...
- name: Verify no changes from go generate - name: Verify no changes from go generate
run: | run: |
git status --porcelain git status --porcelain
@ -117,12 +96,6 @@ jobs:
exit 1 exit 1
fi fi
- name: Check for breaking OpenAPI changes
if: github.event_name == 'pull_request'
run: |
git fetch --no-tags --depth=1 origin ${{ github.event.pull_request.base.sha }}
make api-diff API_DIFF_BASE=${{ github.event.pull_request.base.sha }}
validate-migrations: validate-migrations:
name: Validate DB migrations name: Validate DB migrations
runs-on: ubuntu-latest runs-on: ubuntu-latest
@ -158,10 +131,8 @@ jobs:
- name: Download dependencies - name: Download dependencies
run: go mod download run: go mod download
# Name must stay unique across the workflow: octocov matches step names - name: Test
# by name across every job, and waits for each match to finish. run: go test -shuffle=on -tags netgo,sqlite_fts5 -race ./... -v
- name: Test with coverage
run: go test -shuffle=on -tags netgo,sqlite_fts5 -race -v -covermode=atomic -coverprofile=coverage.out $(go list ./... | grep -v '/plugins$')
- name: Test ndpgen - name: Test ndpgen
run: | run: |
@ -170,84 +141,6 @@ jobs:
go build -o ndpgen . go build -o ndpgen .
./ndpgen --help ./ndpgen --help
- name: Upload coverage profile
uses: actions/upload-artifact@v7
with:
name: octocov-go
path: coverage.out
if-no-files-found: error
go-plugins:
name: Test Go plugins
runs-on: ubuntu-latest
steps:
- name: Check out code into the Go module directory
uses: actions/checkout@v7
- uses: actions/setup-go@v6
id: setup-go
with:
go-version-file: go.mod
# Without this, the suite recompiles every test plugin WASM module,
# which dominates its runtime under -race.
- name: Cache the WASM compilation cache
uses: actions/cache@v6
with:
path: plugins/testdata/.wazero-cache
key: wazero-${{ runner.os }}-go${{ steps.setup-go.outputs.go-version }}-${{ hashFiles('plugins/testdata/*/*.go', 'plugins/testdata/*/go.*', 'plugins/pdk/go/**/*.go', 'plugins/pdk/go/go.*') }}
restore-keys: wazero-${{ runner.os }}-
- name: Test plugins
run: go tool ginkgo -p -race -tags netgo,sqlite_fts5 --cover --covermode=atomic --coverprofile=coverage.out --output-dir=. ./plugins/
- name: Upload coverage profile
uses: actions/upload-artifact@v7
with:
name: octocov-plugins
path: coverage.out
if-no-files-found: error
coverage:
name: Report coverage
runs-on: ubuntu-latest
needs: [go, go-plugins]
permissions:
contents: read
actions: write
env:
COVERAGE_COMMENT: 'false'
steps:
- uses: actions/checkout@v7
- uses: actions/download-artifact@v8
with:
pattern: octocov-*
# Merge here rather than letting octocov do it: octocov reports statement
# coverage for a single profile, but switches to line counting for several.
- name: Merge coverage profiles
run: |
echo "mode: atomic" > coverage.out
awk 'FNR==1 && /^mode:/ {next} {k=$1" "$2; c[k]+=$3} END {for (k in c) print k, c[k]}' \
octocov-*/coverage.out | sort >> coverage.out
- uses: k1LoW/octocov-action@v1
- name: Save the PR number for the comment workflow
if: github.event_name == 'pull_request'
run: echo "${{ github.event.pull_request.number }}" > pr_number
- name: Upload the merged profile for the comment workflow
if: github.event_name == 'pull_request'
uses: actions/upload-artifact@v7
with:
name: octocov-pr
path: |
coverage.out
pr_number
if-no-files-found: error
go-windows: go-windows:
name: Test Go code (Windows) name: Test Go code (Windows)
runs-on: windows-2022 runs-on: windows-2022
@ -314,12 +207,12 @@ jobs:
run: go test -shuffle=on -tags netgo,sqlite_fts5 ./... -v run: go test -shuffle=on -tags netgo,sqlite_fts5 ./... -v
- name: Test ndpgen - name: Test ndpgen
shell: bash shell: pwsh
run: | run: |
cd plugins/cmd/ndpgen cd plugins\cmd\ndpgen
go test -shuffle=on -v go test -shuffle=on -v
go build -o ndpgen.exe . go build -o ndpgen.exe .
./ndpgen.exe --help .\ndpgen.exe --help
js: js:
name: Test JS code name: Test JS code
@ -385,7 +278,7 @@ jobs:
build: build:
name: Build name: Build
needs: [js, go, go-plugins, go-windows, go-lint, i18n-lint, git-version, check-push-enabled, validate-migrations] needs: [js, go, go-windows, go-lint, i18n-lint, git-version, check-push-enabled, validate-migrations]
strategy: strategy:
matrix: matrix:
platform: [ linux/amd64, linux/arm64, linux/arm/v5, linux/arm/v6, linux/arm/v7, linux/386, linux/riscv64, darwin/amd64, darwin/arm64, windows/amd64, windows/386 ] platform: [ linux/amd64, linux/arm64, linux/arm/v5, linux/arm/v6, linux/arm/v7, linux/386, linux/riscv64, darwin/amd64, darwin/arm64, windows/amd64, windows/386 ]
@ -570,7 +463,7 @@ jobs:
env: env:
GH_TOKEN: ${{ github.token }} GH_TOKEN: ${{ github.token }}
run: | run: |
for artifact in $(gh api repos/${{ github.repository }}/actions/runs/${{ github.run_id }}/artifacts | jq -r '.artifacts[] | select(.name | startswith("digests-")) | .id'); do for artifact in $(gh api repos/${{ github.repository }}/actions/artifacts | jq -r '.artifacts[] | select(.name | startswith("digests-")) | .id'); do
gh api --method DELETE repos/${{ github.repository }}/actions/artifacts/$artifact gh api --method DELETE repos/${{ github.repository }}/actions/artifacts/$artifact
done done

8
.gitignore vendored
View file

@ -40,11 +40,3 @@ openspec/
.agents .agents
go.work* go.work*
.worktrees/ .worktrees/
.playwright-mcp/
# Temp benchmark files
zz_*_test.go
# wazero compilation cache for the plugins test suite
/plugins/testdata/.wazero-cache/
/plugins/testdata/*.stage/

View file

@ -9,7 +9,6 @@ linters:
- asciicheck - asciicheck
- bidichk - bidichk
- bodyclose - bodyclose
- containedctx
- copyloopvar - copyloopvar
- dogsled - dogsled
- durationcheck - durationcheck
@ -28,9 +27,6 @@ linters:
disable: disable:
- staticcheck - staticcheck
settings: settings:
errcheck:
exclude-functions:
- (*github.com/zeebo/xxh3.Hasher).Write
gocritic: gocritic:
disable-all: true disable-all: true
enabled-checks: enabled-checks:
@ -58,9 +54,6 @@ linters:
- gosec - gosec
path: _test\.go path: _test\.go
text: "G703" text: "G703"
- path: _test\.go
linters:
- containedctx
- path-except: 'db/migrations/' - path-except: 'db/migrations/'
linters: linters:
- forbidigo - forbidigo
@ -76,7 +69,6 @@ linters:
- examples$ - examples$
- node_modules - node_modules
- _gen\.go$ - _gen\.go$
- .worktrees
formatters: formatters:
exclusions: exclusions:
generated: lax generated: lax

View file

@ -1,44 +0,0 @@
# Code coverage reporting for pull requests. See https://github.com/k1LoW/octocov
# The 30s default is not enough: scanning this repo's artifacts for the baseline
# eats most of it, leaving none for the report upload.
timeout: 5m
coverage:
# A single pre-merged profile: octocov reports statements for one path, but
# switches to line counting when it merges several itself.
paths:
- coverage.out
# Not code under test: tests/ holds the mocks and helpers, *_gen.go is generated.
# Both patterns need the '**/' prefix: the comment workflow has no source tree,
# so octocov cannot shorten the profile's import paths to repo-relative ones.
exclude:
- '**/tests/**'
- '**/*_gen.go'
codeToTestRatio:
# Needs the pull request's own source, which the comment workflow must not
# check out: it holds a write token.
if: env.COVERAGE_COMMENT != 'true'
code:
- '**/*.go'
- '!**/*_test.go'
- '!**/*_gen.go'
test:
- '**/*_test.go'
testExecutionTime:
if: true
steps:
- Test with coverage
- Test plugins
diff:
datastores:
- artifact://${GITHUB_REPOSITORY}
comment:
# Only the 'Report coverage on PR' workflow sets this: a pull_request run from
# a fork gets a read-only token, so commenting from here 403s.
if: env.COVERAGE_COMMENT == 'true'
updatePrevious: true
summary:
if: true
report:
if: is_default_branch
datastores:
- artifact://${GITHUB_REPOSITORY}

View file

@ -2,7 +2,7 @@ FROM --platform=$BUILDPLATFORM ghcr.io/crazy-max/osxcross:14.5-debian AS osxcros
######################################################################################################################## ########################################################################################################################
### Build xx (original image: tonistiigi/xx) ### Build xx (original image: tonistiigi/xx)
FROM --platform=$BUILDPLATFORM alpine:3.22 AS xx-build FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/alpine:3.20 AS xx-build
# v1.9.0 # v1.9.0
ENV XX_VERSION=a5592eab7a57895e8d385394ff12241bc65ecd50 ENV XX_VERSION=a5592eab7a57895e8d385394ff12241bc65ecd50
@ -26,7 +26,7 @@ COPY --from=xx-build /out/ /usr/bin/
######################################################################################################################## ########################################################################################################################
### Build Navidrome UI ### Build Navidrome UI
FROM --platform=$BUILDPLATFORM node:lts-alpine AS ui FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/node:lts-alpine AS ui
WORKDIR /app WORKDIR /app
# Install node dependencies # Install node dependencies
@ -43,7 +43,7 @@ COPY --from=ui /build /build
######################################################################################################################## ########################################################################################################################
### Build Navidrome binary for Docker image (dynamic musl, enables native libwebp via dlopen) ### Build Navidrome binary for Docker image (dynamic musl, enables native libwebp via dlopen)
FROM --platform=$BUILDPLATFORM golang:1.27-alpine AS build-alpine FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/golang:1.26-alpine AS build-alpine
COPY --from=xx / / COPY --from=xx / /
ARG TARGETPLATFORM ARG TARGETPLATFORM
@ -85,7 +85,7 @@ EOT
######################################################################################################################## ########################################################################################################################
### Build Navidrome binary for standalone distribution (static glibc, cross-compiled) ### Build Navidrome binary for standalone distribution (static glibc, cross-compiled)
FROM --platform=$BUILDPLATFORM golang:1.27-trixie AS base FROM --platform=$BUILDPLATFORM public.ecr.aws/docker/library/golang:1.26-trixie AS base
RUN apt-get update && apt-get install -y clang lld RUN apt-get update && apt-get install -y clang lld
COPY --from=xx / / COPY --from=xx / /
WORKDIR /workspace WORKDIR /workspace
@ -152,52 +152,19 @@ RUN xx-verify --static /out/navidrome*
FROM scratch AS binary FROM scratch AS binary
COPY --from=build /out / COPY --from=build /out /
########################################################################################################################
### Build no-op stubs for mpv's video-output libraries
# mpv links libEGL/libgbm for video output only; Navidrome drives it headless, for audio.
# Real mesa pulls in LLVM + gallium (+218MB uncompressed), so ship stubs it never calls.
FROM --platform=$BUILDPLATFORM alpine:3.22 AS mpv-stubs
COPY --from=xx / /
RUN apk add --no-cache clang lld binutils mesa-egl mesa-gbm
ARG TARGETPLATFORM
RUN xx-apk add --no-cache musl-dev
RUN <<EOT
set -e
mkdir -p /out
for so in libEGL.so.1 libgbm.so.1; do
readelf -sW /usr/lib/$so \
| awk '$5 == "GLOBAL" && $7 != "UND" { print $8 }' \
| sed 's/@.*//' \
| grep -vE '^(_init|_fini|_edata|_end|__bss_start|_GLOBAL_OFFSET_TABLE_)$' \
| sort -u \
| awk '{ print "void " $1 "(void) {}" }' > /tmp/stub.c
test -s /tmp/stub.c
xx-clang -shared -nostdlib -fPIC -Wl,-soname,$so -o /out/$so /tmp/stub.c
xx-verify /out/$so
done
EOT
######################################################################################################################## ########################################################################################################################
### Build Final Image ### Build Final Image
FROM alpine:3.22 AS final FROM public.ecr.aws/docker/library/alpine:3.20 AS final
LABEL maintainer="deluan@navidrome.org" LABEL maintainer="deluan@navidrome.org"
LABEL org.opencontainers.image.source="https://github.com/navidrome/navidrome" LABEL org.opencontainers.image.source="https://github.com/navidrome/navidrome"
# Install runtime dependencies # Install runtime dependencies
# - libwebp + symlinks: enables native WebP encoding via purego/dlopen # - libwebp + symlinks: enables native WebP encoding via purego/dlopen
# The mesa/LLVM stack mpv pulls in for video output is dropped in this same layer, RUN apk add -U --no-cache ffmpeg mpv sqlite libwebp libwebpdemux libwebpmux && \
# otherwise the deleted bytes still ship in the image.
RUN apk add -U --no-cache curl ffmpeg mpv sqlite libwebp libwebpdemux libwebpmux && \
for lib in libwebp libwebpdemux libwebpmux; do \ for lib in libwebp libwebpdemux libwebpmux; do \
target=$(ls /usr/lib/$lib.so.* 2>/dev/null | head -1) && \ target=$(ls /usr/lib/$lib.so.* 2>/dev/null | head -1) && \
[ -n "$target" ] && ln -sf "$target" /usr/lib/$lib.so; \ [ -n "$target" ] && ln -sf "$target" /usr/lib/$lib.so; \
done && \ done
rm -rf /usr/lib/gallium-pipe /usr/lib/dri \
/usr/lib/libEGL.so* /usr/lib/libgbm.so* /usr/lib/libgallium*.so /usr/lib/libLLVM.so* \
/usr/lib/libGL.so* /usr/lib/libGLESv2.so* /usr/lib/libglapi.so*
COPY --from=mpv-stubs /out/ /usr/lib/
RUN mpv --no-video --ao=null --version > /dev/null
# Copy navidrome binary (musl build for Docker, enables native libwebp) # Copy navidrome binary (musl build for Docker, enables native libwebp)
COPY --from=build-alpine /out/navidrome /app/ COPY --from=build-alpine /out/navidrome /app/

View file

@ -20,11 +20,7 @@ IMAGE_PLATFORMS ?= $(shell echo $(SUPPORTED_PLATFORMS) | tr ',' '\n' | grep "lin
PLATFORMS ?= $(SUPPORTED_PLATFORMS) PLATFORMS ?= $(SUPPORTED_PLATFORMS)
DOCKER_TAG ?= deluan/navidrome:develop DOCKER_TAG ?= deluan/navidrome:develop
GOLANGCI_LINT_VERSION ?= v2.14.0 GOLANGCI_LINT_VERSION ?= v2.12.0
VACUUM_VERSION ?= v0.30.6
OAPI_CODEGEN_VERSION ?= v2.8.0
OASDIFF_VERSION ?= v1.32.1
API_DIFF_BASE ?= origin/master
UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*") UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*")
@ -96,45 +92,6 @@ install-golangci-lint: ##@Development Install golangci-lint if not present
fi fi
.PHONY: install-golangci-lint .PHONY: install-golangci-lint
install-api-tools: ##@Development Install OpenAPI tools (vacuum, oapi-codegen, oasdiff) into ./bin
@STAMP=bin/.api-tools-$(VACUUM_VERSION)-$(OAPI_CODEGEN_VERSION)-$(OASDIFF_VERSION); \
if [ ! -f $$STAMP ] || [ ! -x bin/vacuum ] || [ ! -x bin/oapi-codegen ] || [ ! -x bin/oasdiff ]; then \
echo "Installing OpenAPI tools..."; \
GOBIN=$(CURDIR)/bin go install github.com/daveshanley/vacuum@$(VACUUM_VERSION) && \
GOBIN=$(CURDIR)/bin go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@$(OAPI_CODEGEN_VERSION) && \
GOBIN=$(CURDIR)/bin go install github.com/oasdiff/oasdiff@$(OASDIFF_VERSION) && \
rm -f bin/.api-tools-* && touch $$STAMP; \
fi
.PHONY: install-api-tools
api-lint: install-api-tools ##@Development Lint the OpenAPI spec
./bin/vacuum lint -r api/.vacuum.yaml -d -q -b --fail-severity error api/openapi/openapi.yaml
.PHONY: api-lint
api-bundle: install-api-tools ##@Development Bundle the multi-file OpenAPI spec into api/bundled
./bin/vacuum bundle -q --composed -p api/openapi api/openapi/openapi.yaml api/bundled/openapi.yaml
./bin/vacuum bundle -q --composed --format json -p api/openapi api/openapi/openapi.yaml api/bundled/openapi.json
.PHONY: api-bundle
api-gen: api-bundle ##@Development Generate the API v1 server code from the bundled spec
./bin/oapi-codegen -config server/apiv1/oapi-codegen.yaml api/bundled/openapi.json
.PHONY: api-gen
api-diff: api-bundle ##@Development Fail on breaking OpenAPI changes against the merge-base with $(API_DIFF_BASE)
@git rev-parse --verify --quiet $(API_DIFF_BASE)^{commit} >/dev/null || { echo "Base ref $(API_DIFF_BASE) not found; set API_DIFF_BASE"; exit 1; }; \
BASE="$$(git merge-base HEAD $(API_DIFF_BASE) 2>/dev/null)"; \
if [ -z "$$BASE" ]; then \
echo "No merge-base with $(API_DIFF_BASE); falling back to its tip"; \
BASE=$(API_DIFF_BASE); \
fi; \
if git cat-file -e $$BASE:api/bundled/openapi.json 2>/dev/null; then \
git show $$BASE:api/bundled/openapi.json > $(CURDIR)/bin/api-base.json && \
./bin/oasdiff breaking $(CURDIR)/bin/api-base.json api/bundled/openapi.json --fail-on ERR --severity-levels api/.oasdiff-levels.txt; \
else \
echo "No bundled spec at $$BASE; skipping breaking-change check"; \
fi
.PHONY: api-diff
lint: install-golangci-lint ##@Development Lint Go code lint: install-golangci-lint ##@Development Lint Go code
PATH=./bin:$$PATH golangci-lint run --timeout 5m PATH=./bin:$$PATH golangci-lint run --timeout 5m
.PHONY: lint .PHONY: lint
@ -154,7 +111,7 @@ wire: check_go_env ##@Development Update Dependency Injection
go tool wire gen -tags="$$(echo '$(GO_BUILD_TAGS)' | tr ',' ' ')" ./... go tool wire gen -tags="$$(echo '$(GO_BUILD_TAGS)' | tr ',' ' ')" ./...
.PHONY: wire .PHONY: wire
gen: check_go_env api-gen ##@Development Run go generate for code generation gen: check_go_env ##@Development Run go generate for code generation
go generate ./... go generate ./...
cd plugins/cmd/ndpgen && go run . -shared-types -input=../../types -output=../../pdk -go -rust cd plugins/cmd/ndpgen && go run . -shared-types -input=../../types -output=../../pdk -go -rust
cd plugins/cmd/ndpgen && go run . -host-wrappers -input=../../host -package=host -shared=../../types cd plugins/cmd/ndpgen && go run . -host-wrappers -input=../../host -package=host -shared=../../types

View file

@ -35,18 +35,11 @@ See instructions on the [project's website](https://www.navidrome.org/docs/insta
## Cloud Hosting ## Cloud Hosting
Several cloud hosting providers partner with us to offer [officially supported, cloud-hosted solutions](https://www.navidrome.org/docs/installation/managed). If you sign up with any of these providers, a share of the revenue funds the development of Navidrome at no additional cost for you. [PikaPods](https://www.pikapods.com) has partnered with us to offer you an
[officially supported, cloud-hosted solution](https://www.navidrome.org/docs/installation/managed/#pikapods).
<a href="https://www.pikapods.com/pods?run=navidrome"><img src="resources/hosting/pikapods.svg" alt="Run on PikaPods" width="200" height="40"></a> A share of the revenue helps fund the development of Navidrome at no additional cost for you.
<br>
<a href="https://zenith.hosting/host/navidrome?ref=navidrome"><img src="resources/hosting/zenith.svg" alt="Deploy with Zenith" width="200" height="40"></a>
<br>
<a href="https://store.elfhosted.com/product/navidrome/"><img src="resources/hosting/elfhosted.svg" alt="Deploy on ElfHosted" width="200" height="40"></a>
[![PikaPods](https://www.pikapods.com/static/run-button.svg)](https://www.pikapods.com/pods?run=navidrome)
## Features ## Features

View file

@ -13,26 +13,15 @@ import (
"strings" "strings"
"github.com/microcosm-cc/bluemonday" "github.com/microcosm-cc/bluemonday"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
) )
const apiBaseURL = "https://api.deezer.com" const apiBaseURL = "https://api.deezer.com"
const authBaseURL = "https://auth.deezer.com" const authBaseURL = "https://auth.deezer.com"
// errCodeQuota is Deezer's "Quota limit exceeded"; it arrives in the body, with HTTP 200 var (
// and no rate-limit headers, so the body code is the only signal. ErrNotFound = errors.New("deezer: not found")
const errCodeQuota = 4 )
type deezerError struct {
Type string `json:"type"`
Message string `json:"message"`
Code int `json:"code"`
}
func (e *deezerError) Error() string {
return fmt.Sprintf("deezer error(%d): %s", e.Code, e.Message)
}
type httpDoer interface { type httpDoer interface {
Do(req *http.Request) (*http.Response, error) Do(req *http.Request) (*http.Response, error)
@ -67,7 +56,7 @@ func (c *client) searchArtists(ctx context.Context, name string, limit int) ([]A
} }
if len(results.Data) == 0 { if len(results.Data) == 0 {
return nil, agents.ErrNotFound return nil, ErrNotFound
} }
return results.Data, nil return results.Data, nil
} }
@ -85,31 +74,20 @@ func (c *client) makeRequest(req *http.Request, response any) error {
return err return err
} }
// Checked before the status: a throttled request still answers 200, and decoding its body
// into a result type yields an empty one, which reads as "nothing found".
if err := parseBodyError(data); err != nil {
return err
}
if resp.StatusCode != 200 { if resp.StatusCode != 200 {
return fmt.Errorf("deezer http status: (%d)", resp.StatusCode) return c.parseError(data)
} }
return json.Unmarshal(data, response) return json.Unmarshal(data, response)
} }
// parseBodyError returns the error Deezer reported in the body, or nil when it reported none. func (c *client) parseError(data []byte) error {
func parseBodyError(data []byte) error { var deezerError Error
var body errorResponse err := json.Unmarshal(data, &deezerError)
// Discarded: a payload that is not an error object leaves Error nil, which is the "none" answer. if err != nil {
_ = json.Unmarshal(data, &body) return err
switch {
case body.Error == nil:
return nil
case body.Error.Code == errCodeQuota:
return errors.Join(body.Error, agents.ErrRetryLater)
default:
return body.Error
} }
return fmt.Errorf("deezer error(%d): %s", deezerError.Error.Code, deezerError.Error.Message)
} }
func (c *client) getRelatedArtists(ctx context.Context, artistID int) ([]Artist, error) { func (c *client) getRelatedArtists(ctx context.Context, artistID int) ([]Artist, error) {

View file

@ -2,14 +2,12 @@ package deezer
import ( import (
"bytes" "bytes"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"os" "os"
"time" "time"
"github.com/navidrome/navidrome/core/agents"
. "github.com/onsi/ginkgo/v2" . "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
) )
@ -43,37 +41,7 @@ var _ = Describe("client", func() {
}) })
_, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20) _, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20)
Expect(err).To(MatchError(agents.ErrNotFound)) Expect(err).To(MatchError(ErrNotFound))
})
// Deezer answers 200 with no rate-limit headers when throttling, so this body is the only signal.
It("reports an exhausted quota as a retryable error, not as a missing artist", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
Body: io.NopCloser(bytes.NewBufferString(
`{"error":{"type":"Exception","message":"Quota limit exceeded","code":4}}`)),
})
_, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20)
Expect(err).To(HaveOccurred())
Expect(err).ToNot(MatchError(agents.ErrNotFound),
"a throttled lookup would otherwise settle the artist as having no image")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
Expect(err.Error()).To(ContainSubstring("Quota limit exceeded"))
})
It("reports a non-quota body error as a plain error", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
Body: io.NopCloser(bytes.NewBufferString(
`{"error":{"type":"Exception","message":"Invalid query","code":100}}`)),
})
_, err := client.searchArtists(GinkgoT().Context(), "Michael Jackson", 20)
Expect(err).To(HaveOccurred())
Expect(err).ToNot(MatchError(agents.ErrNotFound))
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeFalse(),
"only a throttle asks the caller to come back later")
}) })
}) })

View file

@ -5,6 +5,7 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"net/http"
"slices" "slices"
"strings" "strings"
@ -14,7 +15,6 @@ import (
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache" "github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/slice" "github.com/navidrome/navidrome/utils/slice"
) )
@ -36,7 +36,9 @@ func deezerConstructor(dataStore model.DataStore) agents.Interface {
dataStore: dataStore, dataStore: dataStore,
languages: conf.Server.Deezer.Languages, languages: conf.Server.Deezer.Languages,
} }
httpClient := httpclient.New(consts.DefaultHttpClientTimeOut) httpClient := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
cachedHttpClient := cache.NewHTTPClient(httpClient, consts.DefaultHttpClientTimeOut) cachedHttpClient := cache.NewHTTPClient(httpClient, consts.DefaultHttpClientTimeOut)
agent.client = newClient(cachedHttpClient) agent.client = newClient(cachedHttpClient)
return agent return agent
@ -91,6 +93,9 @@ func isPlaceholderPicture(url string) bool {
func (s *deezerAgent) searchArtist(ctx context.Context, name string) (*Artist, error) { func (s *deezerAgent) searchArtist(ctx context.Context, name string) (*Artist, error) {
artists, err := s.client.searchArtists(ctx, name, deezerArtistSearchLimit) artists, err := s.client.searchArtists(ctx, name, deezerArtistSearchLimit)
if errors.Is(err, ErrNotFound) || len(artists) == 0 {
return nil, agents.ErrNotFound
}
if err != nil { if err != nil {
return nil, err return nil, err
} }

View file

@ -3,7 +3,6 @@ package deezer
import ( import (
"bytes" "bytes"
"context" "context"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
@ -81,22 +80,6 @@ var _ = Describe("deezerAgent", func() {
Expect(artist.ID).To(Equal(2)) Expect(artist.ID).To(Equal(2))
}) })
// The artwork worker settles an artist as "no image" on agents.ErrNotFound, so a throttled
// lookup reaching that here would record a permanent absence.
It("surfaces an exhausted quota instead of reporting the artist as not found", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200,
Body: io.NopCloser(bytes.NewBufferString(
`{"error":{"type":"Exception","message":"Quota limit exceeded","code":4}}`)),
})
_, err := agent.searchArtist(ctx, "Queen")
Expect(err).To(HaveOccurred())
Expect(err).ToNot(MatchError(agents.ErrNotFound))
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
})
It("returns ErrNotFound when no result matches the name exactly", func() { It("returns ErrNotFound when no result matches the name exactly", func() {
httpClient.mock("https://api.deezer.com/search/artist", http.Response{ httpClient.mock("https://api.deezer.com/search/artist", http.Response{
StatusCode: 200, StatusCode: 200,

View file

@ -22,8 +22,12 @@ type Artist struct {
Type string `json:"type"` Type string `json:"type"`
} }
type errorResponse struct { type Error struct {
Error *deezerError `json:"error"` Error struct {
Type string `json:"type"`
Message string `json:"message"`
Code int `json:"code"`
} `json:"error"`
} }
type RelatedArtists struct { type RelatedArtists struct {

View file

@ -26,7 +26,7 @@ var _ = Describe("Responses", func() {
Describe("Error", func() { Describe("Error", func() {
It("parses the error response correctly", func() { It("parses the error response correctly", func() {
var errorResp errorResponse var errorResp Error
body := []byte(`{"error":{"type":"MissingParameterException","message":"Missing parameters: q","code":501}}`) body := []byte(`{"error":{"type":"MissingParameterException","message":"Missing parameters: q","code":501}}`)
err := json.Unmarshal(body, &errorResp) err := json.Unmarshal(body, &errorResp)
Expect(err).To(BeNil()) Expect(err).To(BeNil())

View file

@ -90,7 +90,7 @@ var _ = Describe("Extractor", func() {
info.FileInfo = testFileInfo{FileInfo: fileInfo} info.FileInfo = testFileInfo{FileInfo: fileInfo}
metadata := metadata.New(path, info) metadata := metadata.New(path, info)
return new(metadata.ToMediaFile(model.Library{ID: 1}, "folderID")) return new(metadata.ToMediaFile(1, "folderID"))
} }
BeforeEach(func() { BeforeEach(func() {

View file

@ -18,7 +18,6 @@ import (
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache" "github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/httpclient"
"golang.org/x/net/html" "golang.org/x/net/html"
) )
@ -60,7 +59,9 @@ func lastFMConstructor(ds model.DataStore) *lastfmAgent {
secret: conf.Server.LastFM.Secret, secret: conf.Server.LastFM.Secret,
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty}, sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
} }
hc := httpclient.New(consts.DefaultHttpClientTimeOut) hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
chc := cache.NewHTTPClient(hc, consts.DefaultHttpClientTimeOut) chc := cache.NewHTTPClient(hc, consts.DefaultHttpClientTimeOut)
l.httpClient = chc l.httpClient = chc
l.client = newClient(l.apiKey, l.secret, chc) l.client = newClient(l.apiKey, l.secret, chc)
@ -92,7 +93,7 @@ func (l *lastfmAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid strin
var resp agents.AlbumInfo var resp agents.AlbumInfo
for _, lang := range l.languages { for _, lang := range l.languages {
var err error var err error
a, err = l.callAlbumGetInfo(ctx, name, artist, lang) a, err = l.callAlbumGetInfo(ctx, name, artist, mbid, lang)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -113,7 +114,7 @@ func (l *lastfmAgent) GetAlbumInfo(ctx context.Context, name, artist, mbid strin
} }
func (l *lastfmAgent) GetAlbumImages(ctx context.Context, name, artist, mbid string) ([]agents.ExternalImage, error) { func (l *lastfmAgent) GetAlbumImages(ctx context.Context, name, artist, mbid string) ([]agents.ExternalImage, error) {
a, err := l.callAlbumGetInfo(ctx, name, artist, l.languages[0]) a, err := l.callAlbumGetInfo(ctx, name, artist, mbid, l.languages[0])
if err != nil { if err != nil {
return nil, err return nil, err
} }
@ -241,10 +242,6 @@ func (l *lastfmAgent) GetSimilarSongsByTrack(ctx context.Context, id, name, arti
var ( var (
artistOpenGraphQuery = cascadia.MustCompile(`html > head > meta[property="og:image"]`) artistOpenGraphQuery = cascadia.MustCompile(`html > head > meta[property="og:image"]`)
artistIgnoredImage = "2a96cbd8b46e442fc41c2b86b821562f" // Last.fm artist placeholder image name artistIgnoredImage = "2a96cbd8b46e442fc41c2b86b821562f" // Last.fm artist placeholder image name
// Not a RetryLaterError on purpose: parking the agent would also stall its API-backed
// methods, which the page block does not affect.
errNoArtistPage = errors.New("no artist image in Last.fm page")
) )
func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string) ([]agents.ExternalImage, error) { func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string) ([]agents.ExternalImage, error) {
@ -271,9 +268,7 @@ func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string)
var res []agents.ExternalImage var res []agents.ExternalImage
n := cascadia.Query(node, artistOpenGraphQuery) n := cascadia.Query(node, artistOpenGraphQuery)
if n == nil { if n == nil {
// A real artist page always has og:image; its absence means a bot challenge or a redesign. return res, nil
log.Warn(ctx, "Last.fm did not return a usable artist page", "name", name, "url", a.URL)
return nil, errNoArtistPage
} }
for _, attr := range n.Attr { for _, attr := range n.Attr {
if attr.Key != "content" { if attr.Key != "content" {
@ -291,18 +286,22 @@ func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string)
return res, nil return res, nil
} }
// callAlbumGetInfo matches on name+artist only. Last.fm's album.getInfo by MBID is unreliable — func (l *lastfmAgent) callAlbumGetInfo(ctx context.Context, name, artist, mbid string, lang string) (*Album, error) {
// a correct MBID can return a different album (or none) — so the MBID is deliberately not passed. a, err := l.client.albumGetInfo(ctx, name, artist, mbid, lang)
func (l *lastfmAgent) callAlbumGetInfo(ctx context.Context, name, artist, lang string) (*Album, error) { var lfErr *lastFMError
a, err := l.client.albumGetInfo(ctx, name, artist, "", lang) isLastFMError := errors.As(err, &lfErr)
if mbid != "" && (isLastFMError && lfErr.Code == 6) {
log.Debug(ctx, "LastFM/album.getInfo could not find album by mbid, trying again", "album", name, "mbid", mbid)
return l.callAlbumGetInfo(ctx, name, artist, "", lang)
}
if err != nil { if err != nil {
if lfErr, ok := errors.AsType[*lastFMError](err); ok && lfErr.Code == 6 { if isLastFMError && lfErr.Code == 6 {
// A not-found is a definitive absence, not a fault: return the shared sentinel so the log.Debug(ctx, "Album not found", "album", name, "mbid", mbid, err)
// artwork worker's breaker/transient checks don't retry it, and log it at Debug. } else {
log.Debug(ctx, "Album not found in Last.fm", "album", name, "artist", artist) log.Error(ctx, "Error calling LastFM/album.getInfo", "album", name, "mbid", mbid, err)
return nil, agents.ErrNotFound
} }
log.Error(ctx, "Error calling LastFM/album.getInfo", "album", name, "artist", artist, err)
return nil, err return nil, err
} }
return a, nil return a, nil
@ -314,12 +313,6 @@ func (l *lastfmAgent) callArtistGetInfo(ctx context.Context, name string, lang s
a, err := l.client.artistGetInfo(ctx, name, lang) a, err := l.client.artistGetInfo(ctx, name, lang)
if err != nil { if err != nil {
if lfErr, ok := errors.AsType[*lastFMError](err); ok && lfErr.Code == 6 {
// A not-found is a definitive absence, not a fault: return the shared sentinel so it
// doesn't trip the artwork worker's breaker, and log at Debug instead of Error.
log.Debug(ctx, "Artist not found in Last.fm", "artist", name)
return nil, agents.ErrNotFound
}
log.Error(ctx, "Error calling LastFM/artist.getInfo", "artist", name, err) log.Error(ctx, "Error calling LastFM/artist.getInfo", "artist", name, err)
return nil, err return nil, err
} }
@ -411,8 +404,7 @@ func (l *lastfmAgent) Scrobble(ctx context.Context, userId string, s scrobbler.S
log.Warn(ctx, "Last.fm client.scrobble returned error", "track", s.Title, err) log.Warn(ctx, "Last.fm client.scrobble returned error", "track", s.Title, err)
return errors.Join(err, scrobbler.ErrRetryLater) return errors.Join(err, scrobbler.ErrRetryLater)
} }
// 11: service offline; 16: temporarily unavailable. Rate limiting is mapped by the client. if lfErr.Code == 11 || lfErr.Code == 16 {
if lfErr.Code == 11 || lfErr.Code == 16 || errors.Is(err, scrobbler.ErrRetryLater) {
return errors.Join(err, scrobbler.ErrRetryLater) return errors.Join(err, scrobbler.ErrRetryLater)
} }
return errors.Join(err, scrobbler.ErrUnrecoverable) return errors.Join(err, scrobbler.ErrUnrecoverable)

View file

@ -100,15 +100,6 @@ var _ = Describe("lastfmAgent", func() {
Expect(httpClient.RequestCount).To(Equal(1)) Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2")) Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("U2"))
}) })
It("returns ErrRetryLater on error 29 (rate limit exceeded)", func() {
httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`{"error":29,"message":"Rate limit exceeded"}`)),
StatusCode: 200,
}
_, err := agent.GetArtistBiography(ctx, "123", "U2", "")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
})
}) })
Describe("Language Fallback", func() { Describe("Language Fallback", func() {
@ -357,7 +348,7 @@ var _ = Describe("lastfmAgent", func() {
var httpClient *tests.FakeHttpClient var httpClient *tests.FakeHttpClient
var track *model.MediaFile var track *model.MediaFile
BeforeEach(func() { BeforeEach(func() {
_ = ds.UserProps().Put(ctx, "user-1", sessionKeyProperty, "SK-1") _ = ds.UserProps(ctx).Put("user-1", sessionKeyProperty, "SK-1")
httpClient = &tests.FakeHttpClient{} httpClient = &tests.FakeHttpClient{}
client := newClient("API_KEY", "SECRET", httpClient) client := newClient("API_KEY", "SECRET", httpClient)
agent = lastFMConstructor(ds) agent = lastFMConstructor(ds)
@ -506,16 +497,6 @@ var _ = Describe("lastfmAgent", func() {
Expect(err).To(MatchError(scrobbler.ErrRetryLater)) Expect(err).To(MatchError(scrobbler.ErrRetryLater))
}) })
It("returns ErrRetryLater on error 29 (rate limit exceeded)", func() {
httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`{"error":29,"message":"Rate limit exceeded"}`)),
StatusCode: 200,
}
err := agent.Scrobble(ctx, "user-1", scrobbler.Scrobble{MediaFile: *track, TimeStamp: time.Now()})
Expect(errors.Is(err, scrobbler.ErrRetryLater)).To(BeTrue())
})
It("returns ErrRetryLater on http errors", func() { It("returns ErrRetryLater on http errors", func() {
httpClient.Res = http.Response{ httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`internal server error`)), Body: io.NopCloser(bytes.NewBufferString(`internal server error`)),
@ -558,10 +539,7 @@ var _ = Describe("lastfmAgent", func() {
URL: "https://www.last.fm/music/Cher/Believe", URL: "https://www.last.fm/music/Cher/Believe",
})) }))
Expect(httpClient.RequestCount).To(Equal(1)) Expect(httpClient.RequestCount).To(Equal(1))
// MBID is deliberately not sent — album.getInfo matches on name+artist only. Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("03c91c40-49a6-44a7-90e7-a700edf97a62"))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
Expect(httpClient.SavedRequest.URL.Query().Get("album")).To(Equal("Believe"))
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("Cher"))
}) })
It("returns empty images if no images are available", func() { It("returns empty images if no images are available", func() {
@ -580,7 +558,7 @@ var _ = Describe("lastfmAgent", func() {
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234") _, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1)) Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty()) Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
}) })
It("returns an error if Last.fm call returns an error", func() { It("returns an error if Last.fm call returns an error", func() {
@ -588,17 +566,23 @@ var _ = Describe("lastfmAgent", func() {
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234") _, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
Expect(httpClient.RequestCount).To(Equal(1)) Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty()) Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(Equal("mbid-1234"))
}) })
It("returns an error when Last.fm returns an error 6 (album not found)", func() { It("returns an error if Last.fm call returns an error 6 and mbid is empty", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200} httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234") _, err := agent.GetAlbumInfo(ctx, "123", "U2", "")
Expect(err).To(HaveOccurred()) Expect(err).To(HaveOccurred())
// A definitive not-found must satisfy the sentinel, or the artwork worker retries it.
Expect(errors.Is(err, agents.ErrNotFound)).To(BeTrue())
Expect(httpClient.RequestCount).To(Equal(1)) Expect(httpClient.RequestCount).To(Equal(1))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty()) })
Context("MBID non existent in Last.fm", func() {
It("calls again when last.fm returns an error 6", func() {
httpClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, _ = agent.GetAlbumInfo(ctx, "123", "U2", "mbid-1234")
Expect(httpClient.RequestCount).To(Equal(2))
Expect(httpClient.SavedRequest.URL.Query().Get("mbid")).To(BeEmpty())
})
}) })
}) })
@ -629,13 +613,6 @@ var _ = Describe("lastfmAgent", func() {
Expect(images[0].URL).To(Equal("https://lastfm.freetls.fastly.net/i/u/ar0/818148bf682d429dc21b59a73ef6f68e.png")) Expect(images[0].URL).To(Equal("https://lastfm.freetls.fastly.net/i/u/ar0/818148bf682d429dc21b59a73ef6f68e.png"))
}) })
It("maps a Last.fm error 6 (artist not found) to the shared not-found sentinel", func() {
apiClient.Res = http.Response{Body: io.NopCloser(bytes.NewBufferString(lastfmError6)), StatusCode: 200}
_, err := agent.GetArtistImages(ctx, "123", "Nonexistent Artist", "")
// Not a fault: runs of missing artists must not trip the worker's circuit breaker.
Expect(errors.Is(err, agents.ErrNotFound)).To(BeTrue())
})
It("returns empty list if image is the ignored default image", func() { It("returns empty list if image is the ignored default image", func() {
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json") fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200} apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
@ -648,41 +625,18 @@ var _ = Describe("lastfmAgent", func() {
Expect(images).To(BeEmpty()) Expect(images).To(BeEmpty())
}) })
It("errors when the page has no meta tags", func() { It("returns empty list if page has no meta tags", func() {
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json") fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200} apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.no_meta.html") fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.no_meta.html")
httpClient.Res = http.Response{Body: fScraper, StatusCode: 200} httpClient.Res = http.Response{Body: fScraper, StatusCode: 200}
_, err := agent.GetArtistImages(ctx, "123", "U2", "")
Expect(err).To(MatchError(errNoArtistPage))
})
It("errors when Last.fm serves a bot challenge page", func() {
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.challenge.html")
httpClient.Res = http.Response{Body: fScraper, StatusCode: 200}
images, err := agent.GetArtistImages(ctx, "123", "U2", "") images, err := agent.GetArtistImages(ctx, "123", "U2", "")
Expect(err).To(MatchError(errNoArtistPage)) Expect(err).ToNot(HaveOccurred())
Expect(images).To(BeEmpty()) Expect(images).To(BeEmpty())
}) })
It("does not park the agent: the failure is not a retry-later", func() {
// A RetryLaterError would cool down the agent's API-backed methods too.
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.challenge.html")
httpClient.Res = http.Response{Body: fScraper, StatusCode: 200}
_, err := agent.GetArtistImages(ctx, "123", "U2", "")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeFalse())
})
It("returns error if API call fails", func() { It("returns error if API call fails", func() {
apiClient.Err = errors.New("api error") apiClient.Err = errors.New("api error")
_, err := agent.GetArtistImages(ctx, "123", "U2", "") _, err := agent.GetArtistImages(ctx, "123", "U2", "")

View file

@ -18,7 +18,6 @@ import (
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request" "github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/server" "github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/req" "github.com/navidrome/navidrome/utils/req"
) )
@ -42,7 +41,9 @@ func NewRouter(ds model.DataStore) *Router {
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty}, sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
} }
r.Handler = r.routes() r.Handler = r.routes()
hc := httpclient.New(consts.DefaultHttpClientTimeOut) hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
r.client = newClient(r.apiKey, r.secret, hc) r.client = newClient(r.apiKey, r.secret, hc)
return r return r
} }
@ -132,7 +133,7 @@ func (s *Router) callback(w http.ResponseWriter, r *http.Request) {
func (s *Router) fetchSessionKey(ctx context.Context, uid, token string) error { func (s *Router) fetchSessionKey(ctx context.Context, uid, token string) error {
sessionKey, err := s.client.getSession(ctx, token) sessionKey, err := s.client.getSession(ctx, token)
if err != nil { if err != nil {
log.Error(ctx, "Could not fetch LastFM session key", "userId", uid, log.Error(ctx, "Could not fetch LastFM session key", "userId", uid, "token", token,
"requestId", middleware.GetReqID(ctx), err) "requestId", middleware.GetReqID(ctx), err)
return err return err
} }

View file

@ -50,7 +50,7 @@ var _ = Describe("auth_router", func() {
}) })
storedSessionKey := func(userID string) string { storedSessionKey := func(userID string) string {
key, _ := userProps.Get(GinkgoT().Context(), userID, sessionKeyProperty) key, _ := userProps.Get(userID, sessionKeyProperty)
return key return key
} }
@ -214,14 +214,5 @@ var _ = Describe("auth_router", func() {
_, err = verifyLinkToken(nonExpiringToken) _, err = verifyLinkToken(nonExpiringToken)
Expect(err).To(MatchError("link token missing expiration")) Expect(err).To(MatchError("link token missing expiration"))
}) })
It("rejects a Jellyfin access token", func() {
usr := &model.User{ID: "u1", UserName: "johndoe"}
tokenStr, err := auth.CreateAPIToken(usr, auth.AudienceJellyfin)
Expect(err).ToNot(HaveOccurred())
_, err = verifyLinkToken(tokenStr)
Expect(err).To(HaveOccurred())
})
}) })
}) })

View file

@ -5,7 +5,6 @@ import (
"crypto/md5" "crypto/md5"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"net/http" "net/http"
"net/url" "net/url"
@ -15,15 +14,11 @@ import (
"strings" "strings"
"time" "time"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
) )
const ( const (
apiBaseUrl = "https://ws.audioscrobbler.com/2.0/" apiBaseUrl = "https://ws.audioscrobbler.com/2.0/"
// errCodeRateLimit is Last.fm's "rate limit exceeded"; it arrives in the body, with HTTP 200
// and no rate-limit headers, so the body code is the only signal.
errCodeRateLimit = 29
) )
type lastFMError struct { type lastFMError struct {
@ -49,12 +44,6 @@ type client struct {
hc httpDoer hc httpDoer
} }
// escapePlus works around Last.fm decoding artist.* and track.* params twice, turning "+" into a space.
// album.getInfo decodes only once, so it must not use this.
func escapePlus(s string) string {
return strings.ReplaceAll(s, "+", "%2B")
}
func (c *client) albumGetInfo(ctx context.Context, name string, artist string, mbid string, lang string) (*Album, error) { func (c *client) albumGetInfo(ctx context.Context, name string, artist string, mbid string, lang string) (*Album, error) {
params := url.Values{} params := url.Values{}
params.Add("method", "album.getInfo") params.Add("method", "album.getInfo")
@ -72,7 +61,7 @@ func (c *client) albumGetInfo(ctx context.Context, name string, artist string, m
func (c *client) artistGetInfo(ctx context.Context, name string, lang string) (*Artist, error) { func (c *client) artistGetInfo(ctx context.Context, name string, lang string) (*Artist, error) {
params := url.Values{} params := url.Values{}
params.Add("method", "artist.getInfo") params.Add("method", "artist.getInfo")
params.Add("artist", escapePlus(name)) params.Add("artist", name)
params.Add("lang", lang) params.Add("lang", lang)
response, err := c.makeRequest(ctx, http.MethodGet, params, false) response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil { if err != nil {
@ -84,7 +73,7 @@ func (c *client) artistGetInfo(ctx context.Context, name string, lang string) (*
func (c *client) artistGetSimilar(ctx context.Context, name string, limit int) (*SimilarArtists, error) { func (c *client) artistGetSimilar(ctx context.Context, name string, limit int) (*SimilarArtists, error) {
params := url.Values{} params := url.Values{}
params.Add("method", "artist.getSimilar") params.Add("method", "artist.getSimilar")
params.Add("artist", escapePlus(name)) params.Add("artist", name)
params.Add("limit", strconv.Itoa(limit)) params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false) response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil { if err != nil {
@ -96,7 +85,7 @@ func (c *client) artistGetSimilar(ctx context.Context, name string, limit int) (
func (c *client) artistGetTopTracks(ctx context.Context, name string, limit int) (*TopTracks, error) { func (c *client) artistGetTopTracks(ctx context.Context, name string, limit int) (*TopTracks, error) {
params := url.Values{} params := url.Values{}
params.Add("method", "artist.getTopTracks") params.Add("method", "artist.getTopTracks")
params.Add("artist", escapePlus(name)) params.Add("artist", name)
params.Add("limit", strconv.Itoa(limit)) params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false) response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil { if err != nil {
@ -108,8 +97,8 @@ func (c *client) artistGetTopTracks(ctx context.Context, name string, limit int)
func (c *client) trackGetSimilar(ctx context.Context, name, artist string, limit int) (*SimilarTracks, error) { func (c *client) trackGetSimilar(ctx context.Context, name, artist string, limit int) (*SimilarTracks, error) {
params := url.Values{} params := url.Values{}
params.Add("method", "track.getSimilar") params.Add("method", "track.getSimilar")
params.Add("track", escapePlus(name)) params.Add("track", name)
params.Add("artist", escapePlus(artist)) params.Add("artist", artist)
params.Add("limit", strconv.Itoa(limit)) params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false) response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil { if err != nil {
@ -236,11 +225,7 @@ func (c *client) makeRequest(ctx context.Context, method string, params url.Valu
return nil, jsonErr return nil, jsonErr
} }
if response.Error != 0 { if response.Error != 0 {
var err error = &lastFMError{Code: response.Error, Message: response.Message} return &response, &lastFMError{Code: response.Error, Message: response.Message}
if response.Error == errCodeRateLimit {
err = errors.Join(err, &agents.RetryLaterError{})
}
return &response, err
} }
return &response, nil return &response, nil

View file

@ -35,15 +35,6 @@ var _ = Describe("client", func() {
Expect(album.Name).To(Equal("Believe")) Expect(album.Name).To(Equal("Believe"))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?album=Believe&api_key=API_KEY&artist=U2&format=json&lang=pt&mbid=mbid-1234&method=album.getInfo")) Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?album=Believe&api_key=API_KEY&artist=U2&format=json&lang=pt&mbid=mbid-1234&method=album.getInfo"))
}) })
It("does not double-encode plus signs", func() {
f, _ := os.Open("tests/fixtures/lastfm.album.getinfo.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.albumGetInfo(context.Background(), "Lungs", "Florence + the Machine", "", "en")
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("Florence + the Machine"))
})
}) })
Describe("artistGetInfo", func() { Describe("artistGetInfo", func() {
@ -57,15 +48,6 @@ var _ = Describe("client", func() {
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&lang=pt&method=artist.getInfo")) Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&lang=pt&method=artist.getInfo"))
}) })
It("double-encodes plus signs in the artist name", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.artistGetInfo(context.Background(), "Florence + the Machine", "en")
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("Florence %2B the Machine"))
})
It("fails if Last.fm returns an http status != 200", func() { It("fails if Last.fm returns an http status != 200", func() {
httpClient.Res = http.Response{ httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`Internal Server Error`)), Body: io.NopCloser(bytes.NewBufferString(`Internal Server Error`)),
@ -125,15 +107,6 @@ var _ = Describe("client", func() {
Expect(len(similar.Artists)).To(Equal(2)) Expect(len(similar.Artists)).To(Equal(2))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getSimilar")) Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getSimilar"))
}) })
It("double-encodes plus signs in the artist name", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getsimilar.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.artistGetSimilar(context.Background(), "+44", 2)
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("%2B44"))
})
}) })
Describe("artistGetTopTracks", func() { Describe("artistGetTopTracks", func() {
@ -146,15 +119,6 @@ var _ = Describe("client", func() {
Expect(len(top.Track)).To(Equal(2)) Expect(len(top.Track)).To(Equal(2))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getTopTracks")) Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getTopTracks"))
}) })
It("double-encodes plus signs in the artist name", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.gettoptracks.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.artistGetTopTracks(context.Background(), "C+C Music Factory", 2)
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("C%2BC Music Factory"))
})
}) })
Describe("trackGetSimilar", func() { Describe("trackGetSimilar", func() {
@ -171,17 +135,6 @@ var _ = Describe("client", func() {
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=Depeche+Mode&format=json&limit=5&method=track.getSimilar&track=Just+Can%27t+Get+Enough")) Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=Depeche+Mode&format=json&limit=5&method=track.getSimilar&track=Just+Can%27t+Get+Enough"))
}) })
It("double-encodes plus signs in the track and artist names", func() {
f, _ := os.Open("tests/fixtures/lastfm.track.getsimilar.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.trackGetSimilar(context.Background(), "1+1", "Queen + Paul Rodgers", 5)
Expect(err).ToNot(HaveOccurred())
query := httpClient.SavedRequest.URL.Query()
Expect(query.Get("track")).To(Equal("1%2B1"))
Expect(query.Get("artist")).To(Equal("Queen %2B Paul Rodgers"))
})
It("returns empty list when no similar tracks found", func() { It("returns empty list when no similar tracks found", func() {
f, _ := os.Open("tests/fixtures/lastfm.track.getsimilar.unknown.json") f, _ := os.Open("tests/fixtures/lastfm.track.getsimilar.unknown.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200} httpClient.Res = http.Response{Body: f, StatusCode: 200}

View file

@ -3,6 +3,7 @@ package listenbrainz
import ( import (
"context" "context"
"errors" "errors"
"net/http"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/consts"
@ -11,7 +12,6 @@ import (
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/cache" "github.com/navidrome/navidrome/utils/cache"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/slice" "github.com/navidrome/navidrome/utils/slice"
) )
@ -33,7 +33,9 @@ func listenBrainzConstructor(ds model.DataStore) *listenBrainzAgent {
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty}, sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
baseURL: conf.Server.ListenBrainz.BaseURL, baseURL: conf.Server.ListenBrainz.BaseURL,
} }
hc := httpclient.New(consts.DefaultHttpClientTimeOut) hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
chc := cache.NewHTTPClient(hc, consts.DefaultHttpClientTimeOut) chc := cache.NewHTTPClient(hc, consts.DefaultHttpClientTimeOut)
l.client = newClient(l.baseURL, chc) l.client = newClient(l.baseURL, chc)
return l return l

View file

@ -30,7 +30,7 @@ var _ = Describe("listenBrainzAgent", func() {
BeforeEach(func() { BeforeEach(func() {
ds = &tests.MockDataStore{} ds = &tests.MockDataStore{}
ctx = context.Background() ctx = context.Background()
_ = ds.UserProps().Put(ctx, "user-1", sessionKeyProperty, "SK-1") _ = ds.UserProps(ctx).Put("user-1", sessionKeyProperty, "SK-1")
httpClient = &tests.FakeHttpClient{} httpClient = &tests.FakeHttpClient{}
agent = listenBrainzConstructor(ds) agent = listenBrainzConstructor(ds)
agent.client = newClient("http://localhost:8080", httpClient) agent.client = newClient("http://localhost:8080", httpClient)
@ -164,19 +164,6 @@ var _ = Describe("listenBrainzAgent", func() {
err := agent.Scrobble(ctx, "user-1", sc) err := agent.Scrobble(ctx, "user-1", sc)
Expect(err).To(MatchError(scrobbler.ErrUnrecoverable)) Expect(err).To(MatchError(scrobbler.ErrUnrecoverable))
}) })
It("keeps a 429 scrobble for retry and carries the delay", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"7"}},
Body: io.NopCloser(bytes.NewBufferString(`{"code":429,"error":"rate limited"}`)),
}
err := agent.Scrobble(ctx, "user-1", scrobbler.Scrobble{MediaFile: *track, TimeStamp: time.Now()})
Expect(errors.Is(err, scrobbler.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(7 * time.Second))
})
}) })
Describe("GetArtistUrl", func() { Describe("GetArtistUrl", func() {

View file

@ -16,7 +16,6 @@ import (
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request" "github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/server" "github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/utils/httpclient"
) )
type sessionKeysRepo interface { type sessionKeysRepo interface {
@ -38,7 +37,9 @@ func NewRouter(ds model.DataStore) *Router {
sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty}, sessionKeys: &agents.SessionKeys{DataStore: ds, KeyName: sessionKeyProperty},
} }
r.Handler = r.routes() r.Handler = r.routes()
hc := httpclient.New(consts.DefaultHttpClientTimeOut) hc := &http.Client{
Timeout: consts.DefaultHttpClientTimeOut,
}
r.client = newClient(conf.Server.ListenBrainz.BaseURL, hc) r.client = newClient(conf.Server.ListenBrainz.BaseURL, hc)
return r return r
} }

View file

@ -13,7 +13,6 @@ import (
"slices" "slices"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
) )
@ -22,12 +21,6 @@ const (
labsBase = "https://labs.api.listenbrainz.org/" labsBase = "https://labs.api.listenbrainz.org/"
) )
// retryLaterErr reads the wait ListenBrainz asked for. It sends X-RateLimit-Reset-In
// (delta-seconds) on every response, including the 429, and never Retry-After.
func retryLaterErr(h http.Header) *agents.RetryLaterError {
return &agents.RetryLaterError{RetryIn: agents.ParseRetryIn(h.Get("X-RateLimit-Reset-In"))}
}
var ( var (
ErrorNotFound = errors.New("listenbrainz: not found") ErrorNotFound = errors.New("listenbrainz: not found")
) )
@ -181,9 +174,6 @@ func (c *client) makeAuthenticatedRequest(ctx context.Context, method string, en
} }
defer resp.Body.Close() defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
return nil, retryLaterErr(resp.Header)
}
decoder := json.NewDecoder(resp.Body) decoder := json.NewDecoder(resp.Body)
var response listenBrainzResponse var response listenBrainzResponse
@ -195,10 +185,6 @@ func (c *client) makeAuthenticatedRequest(ctx context.Context, method string, en
return nil, jsonErr return nil, jsonErr
} }
if response.Code != 0 && response.Code != 200 { if response.Code != 0 && response.Code != 200 {
// LB also reports rate limiting as a body code, not only as an HTTP status.
if response.Code == http.StatusTooManyRequests {
return &response, retryLaterErr(resp.Header)
}
return &response, &listenBrainzError{Code: response.Code, Message: response.Error} return &response, &listenBrainzError{Code: response.Code, Message: response.Error}
} }
@ -225,9 +211,6 @@ func (c *client) makeGenericRequest(ctx context.Context, method string, endpoint
// On a 200 code, there is no code. Decode using using error message if it exists // On a 200 code, there is no code. Decode using using error message if it exists
if resp.StatusCode != 200 { if resp.StatusCode != 200 {
defer resp.Body.Close() defer resp.Body.Close()
if resp.StatusCode == http.StatusTooManyRequests {
return nil, retryLaterErr(resp.Header)
}
decoder := json.NewDecoder(resp.Body) decoder := json.NewDecoder(resp.Body)
var lbzError lbzHttpError var lbzError lbzHttpError

View file

@ -4,17 +4,13 @@ import (
"bytes" "bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"os" "os"
"strings"
"time"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/tests" "github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2" . "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
@ -465,73 +461,4 @@ var _ = Describe("client", func() {
})) }))
}) })
}) })
Describe("rate limiting", func() {
It("returns RetryLaterError with the header delay on 429", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"3"}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"You have exceeded your rate limit."}`)),
}
_, err := client.validateToken(context.Background(), "token")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(3 * time.Second))
})
It("returns RetryLaterError with zero delay when no header is present", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"rate limited"}`)),
}
_, err := client.validateToken(context.Background(), "token")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, _ := errors.AsType[*agents.RetryLaterError](err)
Expect(retry.RetryIn).To(BeZero())
})
DescribeTable("caps absurd header values at one hour",
func(header string) {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{header}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"rate limited"}`)),
}
_, err := client.validateToken(context.Background(), "token")
retry, _ := errors.AsType[*agents.RetryLaterError](err)
Expect(retry.RetryIn).To(Equal(time.Hour))
},
Entry("a large value", "999999"),
Entry("a huge value", "99999999999"),
// Scaling this to nanoseconds before capping wraps past 2^64, landing on ~0.29s.
Entry("a value that overflows int64 nanoseconds", "18446744074"),
)
It("maps a body-level 429 sent with a non-429 status", func() {
httpClient.Res = http.Response{
StatusCode: 200,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"7"}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"You have exceeded your rate limit."}`)),
}
_, err := client.validateToken(context.Background(), "token")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(7 * time.Second))
})
It("returns RetryLaterError on a 429 from makeGenericRequest", func() {
httpClient.Res = http.Response{
StatusCode: 429,
Header: http.Header{"X-Ratelimit-Reset-In": []string{"5"}},
Body: io.NopCloser(strings.NewReader(`{"code":429,"error":"rate limited"}`)),
}
_, err := client.getArtistUrl(context.Background(), "1")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue())
retry, ok := errors.AsType[*agents.RetryLaterError](err)
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(5 * time.Second))
})
})
}) })

View file

@ -1 +0,0 @@
response-property-enum-value-added INFO

View file

@ -1,155 +0,0 @@
extends: [[spectral:oas, recommended]]
rules:
# vacuum's `enumeration` function mis-resolves hyphenated `then.field` names,
# so the value check below targets `x-module` via `given` instead.
nd-operation-x-module-required:
description: Every operation belongs to exactly one capability module.
severity: error
given: $.paths[*][get,put,post,delete,patch]
then:
field: x-module
function: truthy
nd-operation-x-module:
description: Every operation's capability module is one of the known values.
severity: error
given: $.paths[*][get,put,post,delete,patch]['x-module']
then:
function: enumeration
functionOptions:
values:
- core
- streaming
- download
- artwork
- lyrics
- transcoding
- annotations
- playback
- queue
- custom-tags
- grouping
- playlists
- smart-playlists
- sync
- events
- jukebox
- sharing
- radio
- admin
nd-operation-stability-level-required:
description: Every operation declares its stability level, which the breaking-change gate relies on.
severity: error
given: $.paths[*][get,put,post,delete,patch]
then:
field: x-stability-level
function: truthy
nd-operation-stability-level:
description: Every operation's stability level is alpha, beta, or stable.
severity: error
given: $.paths[*][get,put,post,delete,patch]['x-stability-level']
then:
function: enumeration
functionOptions:
values:
- alpha
- beta
- stable
nd-operation-required-fields:
description: Operations need a stable operationId, summary, description and tags.
severity: error
given: $.paths[*][get,put,post,delete,patch]
then:
- field: operationId
function: truthy
- field: summary
function: truthy
- field: description
function: truthy
- field: tags
function: truthy
# Our schemas live in path/response files, not root components, so this
# walks every resolved `properties` map in the document via `$..` instead.
nd-schema-property-descriptions:
description: Every schema property is documented.
severity: error
given: $..properties[*]
then:
field: description
function: truthy
# patternProperties covers the full 4xx/5xx range; needs an explicit
# `properties` entry too, or `additionalProperties: false` rejects it.
nd-error-responses-are-problems:
description: 4xx and 5xx responses use application/problem+json.
severity: error
given: $.paths[*][*].responses
then:
function: schema
functionOptions:
forceValidationOnCurrentNode: true
schema:
type: object
patternProperties:
"^[45][0-9][0-9]$":
type: object
required: [content]
properties:
content:
type: object
properties:
application/problem+json: {}
required: [application/problem+json]
additionalProperties: false
# Same filter limitation applies here: "is this a list endpoint" is expressed
# as a JSON Schema if/then on the operation object instead of a `given` filter.
nd-list-endpoints-paginate:
description: List endpoints declare the shared offset and limit parameters.
severity: error
given: $.paths[*].get
then:
function: schema
functionOptions:
forceValidationOnCurrentNode: true
schema:
type: object
if:
required: [responses]
properties:
responses:
type: object
required: ['200']
properties:
'200':
type: object
required: [content]
properties:
content:
type: object
required: [application/json]
properties:
application/json:
type: object
required: [schema]
properties:
schema:
type: object
required: [properties]
properties:
properties:
type: object
required: [items]
then:
required: [parameters]
properties:
parameters:
type: array
allOf:
- contains:
type: object
properties:
name:
const: offset
- contains:
type: object
properties:
name:
const: limit

View file

@ -1,262 +0,0 @@
{
"openapi": "3.0.3",
"info": {
"title": "Navidrome API",
"version": "1.0.0",
"description": "Navidrome API v1. Spec-first, additive within v1. Clients discover implemented\ncapability modules through `GET /server` and never sniff versions.\n\nEnums are open: new values may be added to any enum within v1. Clients must\naccept values they do not recognise instead of failing.\n\nEvery operation declares `x-stability-level`: `alpha` operations may change or\ndisappear without notice, `beta` and `stable` operations only change additively.\nA level is only ever raised, never lowered.\n\n`HEAD` is accepted wherever `GET` is. A `405` response lists the allowed methods\nin its `Allow` header.\n",
"license": {
"name": "GPL-3.0",
"url": "https://www.gnu.org/licenses/gpl-3.0.html"
}
},
"servers": [
{
"url": "/api/v1"
}
],
"tags": [
{
"name": "server",
"description": "Server discovery and the published OpenAPI document."
}
],
"paths": {
"/server": {
"get": {
"operationId": "getServerInfo",
"x-module": "core",
"x-stability-level": "alpha",
"tags": [
"server"
],
"summary": "Describe the server",
"description": "Returns the public server description. No authentication required.\nAuthenticated requests will additionally receive the implemented capability modules\nonce authentication is available.\n",
"responses": {
"200": {
"description": "Server description.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ServerInfo"
}
}
}
},
"500": {
"$ref": "#/components/responses/InternalError"
}
}
}
},
"/openapi.json": {
"get": {
"operationId": "getOpenAPISpecJSON",
"x-module": "core",
"x-stability-level": "alpha",
"tags": [
"server"
],
"summary": "Get the OpenAPI document (JSON)",
"description": "The bundled OpenAPI document of the running server version. Supports ETag revalidation.",
"responses": {
"200": {
"description": "The OpenAPI document.",
"headers": {
"ETag": {
"$ref": "#/components/headers/ETag"
}
},
"content": {
"application/json": {
"schema": {
"type": "object",
"description": "OpenAPI 3.0 document."
}
}
}
},
"304": {
"$ref": "#/components/responses/NotModified"
}
}
}
},
"/openapi.yaml": {
"get": {
"operationId": "getOpenAPISpecYAML",
"x-module": "core",
"x-stability-level": "alpha",
"tags": [
"server"
],
"summary": "Get the OpenAPI document (YAML)",
"description": "The bundled OpenAPI document of the running server version. Supports ETag revalidation.",
"responses": {
"200": {
"description": "The OpenAPI document.",
"headers": {
"ETag": {
"$ref": "#/components/headers/ETag"
}
},
"content": {
"application/yaml": {
"schema": {
"type": "object",
"description": "OpenAPI 3.0 document."
}
}
}
},
"304": {
"$ref": "#/components/responses/NotModified"
}
}
}
}
},
"components": {
"securitySchemes": {
"bearerAuth": {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": "Short-lived access token minted from a device grant. Not yet applied to any operation."
}
},
"schemas": {
"ServerInfo": {
"type": "object",
"description": "Public server description. Everything an add-server screen needs before login.",
"required": [
"name",
"serverVersion",
"specVersion",
"setupRequired",
"loginMethods"
],
"properties": {
"name": {
"type": "string",
"description": "Human-readable server product name."
},
"serverVersion": {
"type": "string",
"description": "Version of the running server build."
},
"specVersion": {
"type": "string",
"description": "Version of the OpenAPI document this server implements."
},
"setupRequired": {
"type": "boolean",
"description": "True until the first admin user has been created."
},
"loginMethods": {
"type": "array",
"description": "Login methods this server accepts. New methods may be added; clients ignore values they do not recognise.",
"items": {
"type": "string",
"enum": [
"password"
]
}
}
}
},
"Problem": {
"type": "object",
"description": "RFC 9457 problem details, returned for every 4xx and 5xx response.",
"required": [
"title",
"status",
"code"
],
"properties": {
"type": {
"type": "string",
"description": "URI reference identifying the problem type. Omitted while the problem carries no semantics\nbeyond its HTTP status code, which RFC 9457 defines as `about:blank`. Problems with their\nown semantics get their own URI; switch on `code` instead.\n"
},
"title": {
"type": "string",
"description": "Short human-readable summary, the same for all occurrences of this problem type."
},
"status": {
"type": "integer",
"description": "HTTP status code of this response."
},
"detail": {
"type": "string",
"description": "Human-readable explanation specific to this occurrence. Omitted for internal errors."
},
"code": {
"type": "string",
"description": "Machine-readable error code, and the value clients switch on. New codes may be added.",
"enum": [
"validation",
"unauthorized",
"forbidden",
"not_found",
"method_not_allowed",
"unavailable",
"internal"
]
},
"errors": {
"type": "array",
"description": "Per-field failures. Present only when `code` is `validation`.",
"items": {
"$ref": "#/components/schemas/ValidationError"
}
}
}
},
"ValidationError": {
"type": "object",
"description": "One field-level validation failure.",
"required": [
"field",
"message"
],
"properties": {
"field": {
"type": "string",
"description": "Name of the offending query parameter, path parameter, or body field (dotted for nested)."
},
"message": {
"type": "string",
"description": "Why the value was rejected."
}
}
}
},
"responses": {
"InternalError": {
"description": "Unexpected server failure. Details are in the server log.",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/Problem"
}
}
}
},
"NotModified": {
"description": "Not modified.",
"headers": {
"ETag": {
"$ref": "#/components/headers/ETag"
}
}
}
},
"headers": {
"ETag": {
"description": "Entity tag for `If-None-Match` revalidation.",
"schema": {
"type": "string"
}
}
}
}
}

View file

@ -1,194 +0,0 @@
openapi: 3.0.3
info:
title: Navidrome API
version: 1.0.0
description: |
Navidrome API v1. Spec-first, additive within v1. Clients discover implemented
capability modules through `GET /server` and never sniff versions.
Enums are open: new values may be added to any enum within v1. Clients must
accept values they do not recognise instead of failing.
Every operation declares `x-stability-level`: `alpha` operations may change or
disappear without notice, `beta` and `stable` operations only change additively.
A level is only ever raised, never lowered.
`HEAD` is accepted wherever `GET` is. A `405` response lists the allowed methods
in its `Allow` header.
license:
name: GPL-3.0
url: https://www.gnu.org/licenses/gpl-3.0.html
servers:
- url: /api/v1
tags:
- name: server
description: Server discovery and the published OpenAPI document.
paths:
/server:
get:
operationId: getServerInfo
x-module: core
x-stability-level: alpha
tags: [server]
summary: Describe the server
description: |
Returns the public server description. No authentication required.
Authenticated requests will additionally receive the implemented capability modules
once authentication is available.
responses:
'200':
description: Server description.
content:
application/json:
schema:
$ref: '#/components/schemas/ServerInfo'
'500':
$ref: '#/components/responses/InternalError'
/openapi.json:
get:
operationId: getOpenAPISpecJSON
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (JSON)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: '#/components/headers/ETag'
content:
application/json:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: '#/components/responses/NotModified'
/openapi.yaml:
get:
operationId: getOpenAPISpecYAML
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (YAML)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: '#/components/headers/ETag'
content:
application/yaml:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: '#/components/responses/NotModified'
components:
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: Short-lived access token minted from a device grant. Not yet applied to any operation.
schemas:
ServerInfo:
type: object
description: Public server description. Everything an add-server screen needs before login.
required:
- name
- serverVersion
- specVersion
- setupRequired
- loginMethods
properties:
name:
type: string
description: Human-readable server product name.
serverVersion:
type: string
description: Version of the running server build.
specVersion:
type: string
description: Version of the OpenAPI document this server implements.
setupRequired:
type: boolean
description: True until the first admin user has been created.
loginMethods:
type: array
description: Login methods this server accepts. New methods may be added; clients ignore values they do not recognise.
items:
type: string
enum:
- password
Problem:
type: object
description: RFC 9457 problem details, returned for every 4xx and 5xx response.
required:
- title
- status
- code
properties:
type:
type: string
description: |
URI reference identifying the problem type. Omitted while the problem carries no semantics
beyond its HTTP status code, which RFC 9457 defines as `about:blank`. Problems with their
own semantics get their own URI; switch on `code` instead.
title:
type: string
description: Short human-readable summary, the same for all occurrences of this problem type.
status:
type: integer
description: HTTP status code of this response.
detail:
type: string
description: Human-readable explanation specific to this occurrence. Omitted for internal errors.
code:
type: string
description: Machine-readable error code, and the value clients switch on. New codes may be added.
enum:
- validation
- unauthorized
- forbidden
- not_found
- method_not_allowed
- unavailable
- internal
errors:
type: array
description: Per-field failures. Present only when `code` is `validation`.
items:
$ref: '#/components/schemas/ValidationError'
ValidationError:
type: object
description: One field-level validation failure.
required:
- field
- message
properties:
field:
type: string
description: Name of the offending query parameter, path parameter, or body field (dotted for nested).
message:
type: string
description: Why the value was rejected.
responses:
InternalError:
description: Unexpected server failure. Details are in the server log.
content:
application/problem+json:
schema:
$ref: '#/components/schemas/Problem'
NotModified:
description: Not modified.
headers:
ETag:
$ref: '#/components/headers/ETag'
headers:
ETag:
description: Entity tag for `If-None-Match` revalidation.
schema:
type: string

View file

@ -1,35 +0,0 @@
package api
import (
_ "embed"
"encoding/json"
"sync"
)
//go:embed bundled/openapi.json
var specJSON []byte
//go:embed bundled/openapi.yaml
var specYAML []byte
func SpecJSON() []byte {
return specJSON
}
func SpecYAML() []byte {
return specYAML
}
var specVersion = sync.OnceValue(func() string {
var doc struct {
Info struct {
Version string `json:"version"`
} `json:"info"`
}
_ = json.Unmarshal(SpecJSON(), &doc)
return doc.Info.Version
})
func SpecVersion() string {
return specVersion()
}

View file

@ -1,37 +0,0 @@
package api_test
import (
"os"
"github.com/getkin/kin-openapi/openapi3"
"github.com/navidrome/navidrome/api"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"gopkg.in/yaml.v3"
)
var _ = Describe("Bundled spec", func() {
It("embeds a valid OpenAPI 3 document", func() {
doc, err := openapi3.NewLoader().LoadFromData(api.SpecJSON())
Expect(err).ToNot(HaveOccurred())
Expect(doc.Validate(GinkgoT().Context())).To(Succeed())
Expect(doc.Paths.Find("/server")).ToNot(BeNil())
})
It("embeds the YAML variant", func() {
var doc map[string]any
Expect(yaml.Unmarshal(api.SpecYAML(), &doc)).To(Succeed())
Expect(doc).To(HaveKey("paths"))
})
It("reports the version from the bundle, matching the source root document", func() {
src, err := os.ReadFile("api/openapi/openapi.yaml")
Expect(err).ToNot(HaveOccurred())
var root struct {
Info struct{ Version string } `yaml:"info"`
}
Expect(yaml.Unmarshal(src, &root)).To(Succeed())
Expect(api.SpecVersion()).To(Equal(root.Info.Version))
Expect(api.SpecVersion()).ToNot(BeEmpty())
})
})

View file

@ -1,3 +0,0 @@
description: Entity tag for `If-None-Match` revalidation.
schema:
type: string

View file

@ -1,9 +0,0 @@
name: limit
in: query
description: Maximum number of items to return.
required: false
schema:
type: integer
minimum: 1
maximum: 2000
default: 100

View file

@ -1,8 +0,0 @@
name: offset
in: query
description: Zero-based index of the first item to return.
required: false
schema:
type: integer
minimum: 0
default: 0

View file

@ -1,5 +0,0 @@
description: The request is malformed or fails validation.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,5 +0,0 @@
description: The caller is authenticated but not allowed to do this.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,5 +0,0 @@
description: Unexpected server failure. Details are in the server log.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,5 +0,0 @@
description: No such resource or endpoint.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,4 +0,0 @@
description: Not modified.
headers:
ETag:
$ref: ../headers/ETag.yaml

View file

@ -1,5 +0,0 @@
description: Missing, invalid, or expired credentials.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,13 +0,0 @@
type: object
description: Pagination metadata carried by every list response.
required: [total, offset, limit]
properties:
total:
type: integer
description: Total number of items matching the request, ignoring pagination.
offset:
type: integer
description: Zero-based index of the first returned item.
limit:
type: integer
description: Maximum number of items in this page.

View file

@ -1,35 +0,0 @@
type: object
description: RFC 9457 problem details, returned for every 4xx and 5xx response.
required: [title, status, code]
properties:
type:
type: string
description: |
URI reference identifying the problem type. Omitted while the problem carries no semantics
beyond its HTTP status code, which RFC 9457 defines as `about:blank`. Problems with their
own semantics get their own URI; switch on `code` instead.
title:
type: string
description: Short human-readable summary, the same for all occurrences of this problem type.
status:
type: integer
description: HTTP status code of this response.
detail:
type: string
description: Human-readable explanation specific to this occurrence. Omitted for internal errors.
code:
type: string
description: Machine-readable error code, and the value clients switch on. New codes may be added.
enum:
- validation
- unauthorized
- forbidden
- not_found
- method_not_allowed
- unavailable
- internal
errors:
type: array
description: Per-field failures. Present only when `code` is `validation`.
items:
$ref: ./ValidationError.yaml

View file

@ -1,22 +0,0 @@
type: object
description: Public server description. Everything an add-server screen needs before login.
required: [name, serverVersion, specVersion, setupRequired, loginMethods]
properties:
name:
type: string
description: Human-readable server product name.
serverVersion:
type: string
description: Version of the running server build.
specVersion:
type: string
description: Version of the OpenAPI document this server implements.
setupRequired:
type: boolean
description: True until the first admin user has been created.
loginMethods:
type: array
description: Login methods this server accepts. New methods may be added; clients ignore values they do not recognise.
items:
type: string
enum: [password]

View file

@ -1,10 +0,0 @@
type: object
description: One field-level validation failure.
required: [field, message]
properties:
field:
type: string
description: Name of the offending query parameter, path parameter, or body field (dotted for nested).
message:
type: string
description: Why the value was rejected.

View file

@ -1,39 +0,0 @@
openapi: 3.0.3
info:
title: Navidrome API
version: 1.0.0
description: |
Navidrome API v1. Spec-first, additive within v1. Clients discover implemented
capability modules through `GET /server` and never sniff versions.
Enums are open: new values may be added to any enum within v1. Clients must
accept values they do not recognise instead of failing.
Every operation declares `x-stability-level`: `alpha` operations may change or
disappear without notice, `beta` and `stable` operations only change additively.
A level is only ever raised, never lowered.
`HEAD` is accepted wherever `GET` is. A `405` response lists the allowed methods
in its `Allow` header.
license:
name: GPL-3.0
url: https://www.gnu.org/licenses/gpl-3.0.html
servers:
- url: /api/v1
tags:
- name: server
description: Server discovery and the published OpenAPI document.
paths:
/server:
$ref: ./paths/server.yaml
/openapi.json:
$ref: ./paths/openapi.yaml#/json
/openapi.yaml:
$ref: ./paths/openapi.yaml#/yaml
components:
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: Short-lived access token minted from a device grant. Not yet applied to any operation.

View file

@ -1,42 +0,0 @@
json:
get:
operationId: getOpenAPISpecJSON
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (JSON)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: ../components/headers/ETag.yaml
content:
application/json:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: ../components/responses/NotModified.yaml
yaml:
get:
operationId: getOpenAPISpecYAML
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (YAML)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: ../components/headers/ETag.yaml
content:
application/yaml:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: ../components/responses/NotModified.yaml

View file

@ -1,19 +0,0 @@
get:
operationId: getServerInfo
x-module: core
x-stability-level: alpha
tags: [server]
summary: Describe the server
description: |
Returns the public server description. No authentication required.
Authenticated requests will additionally receive the implemented capability modules
once authentication is available.
responses:
'200':
description: Server description.
content:
application/json:
schema:
$ref: ../components/schemas/ServerInfo.yaml
'500':
$ref: ../components/responses/InternalError.yaml

File diff suppressed because it is too large Load diff

File diff suppressed because it is too large Load diff

View file

@ -2,7 +2,9 @@ package cmd
import ( import (
"context" "context"
"path/filepath" "fmt"
"os"
"strings"
"time" "time"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
@ -29,7 +31,7 @@ func init() {
pruneCmd.Flags().BoolVarP(&force, "force", "f", false, "bypass warning when backup count is zero") pruneCmd.Flags().BoolVarP(&force, "force", "f", false, "bypass warning when backup count is zero")
backupRoot.AddCommand(pruneCmd) backupRoot.AddCommand(pruneCmd)
restoreCommand.Flags().StringVarP(&restorePath, "backup-file", "b", "", "file name of the backup database to restore (resolved against the backup directory unless it is an absolute path)") restoreCommand.Flags().StringVarP(&restorePath, "backup-file", "b", "", "path of backup database to restore")
restoreCommand.Flags().BoolVarP(&force, "force", "f", false, "bypass restore warning") restoreCommand.Flags().BoolVarP(&force, "force", "f", false, "bypass restore warning")
_ = restoreCommand.MarkFlagRequired("backup-file") _ = restoreCommand.MarkFlagRequired("backup-file")
backupRoot.AddCommand(restoreCommand) backupRoot.AddCommand(restoreCommand)
@ -76,12 +78,24 @@ func runBackup(ctx context.Context) {
conf.Server.Backup.Path = conf.NewDir(backupDir) conf.Server.Backup.Path = conf.NewDir(backupDir)
} }
requireExistingDB() idx := strings.LastIndex(conf.Server.DbPath, "?")
var path string
if idx == -1 {
path = conf.Server.DbPath
} else {
path = conf.Server.DbPath[:idx]
}
if _, err := os.Stat(path); os.IsNotExist(err) {
log.Fatal("No existing database", "path", path)
return
}
start := time.Now() start := time.Now()
path, err := db.Backup(ctx) path, err := db.Backup(ctx)
if err != nil { if err != nil {
log.Fatal("Error backing up database", "backupPath", conf.Server.Backup.Path, err) log.Fatal("Error backing up database", "backup path", conf.Server.BasePath, err)
} }
elapsed := time.Since(start) elapsed := time.Since(start)
@ -97,17 +111,36 @@ func runPrune(ctx context.Context) {
conf.Server.Backup.Count = backupCount conf.Server.Backup.Count = backupCount
} }
if conf.Server.Backup.Count == 0 && !force && !confirmYES("Warning: pruning ALL backups") { if conf.Server.Backup.Count == 0 && !force {
log.Warn("Prune cancelled") fmt.Println("Warning: pruning ALL backups")
return fmt.Printf("Please enter YES (all caps) to continue: ")
var input string
_, err := fmt.Scanln(&input)
if input != "YES" || err != nil {
log.Warn("Prune cancelled")
return
}
} }
requireExistingDB() idx := strings.LastIndex(conf.Server.DbPath, "?")
var path string
if idx == -1 {
path = conf.Server.DbPath
} else {
path = conf.Server.DbPath[:idx]
}
if _, err := os.Stat(path); os.IsNotExist(err) {
log.Fatal("No existing database", "path", path)
return
}
start := time.Now() start := time.Now()
count, err := db.Prune(ctx) count, err := db.Prune(ctx)
if err != nil { if err != nil {
log.Fatal("Error pruning database", "backupPath", conf.Server.Backup.Path, err) log.Fatal("Error pruning up database", "backup path", conf.Server.BasePath, err)
} }
elapsed := time.Since(start) elapsed := time.Since(start)
@ -116,29 +149,36 @@ func runPrune(ctx context.Context) {
} }
func runRestore(ctx context.Context) { func runRestore(ctx context.Context) {
requireExistingDB() idx := strings.LastIndex(conf.Server.DbPath, "?")
var path string
// A relative --backup-file is resolved against Backup.Path, the same folder if idx == -1 {
// `backup create` writes to. Without this, the value was treated as relative path = conf.Server.DbPath
// to the working directory, where the file does not exist. } else {
if !filepath.IsAbs(restorePath) { path = conf.Server.DbPath[:idx]
backupPath, err := conf.Server.Backup.Path.Path()
if err != nil {
log.Fatal("Backup directory not available", "backupPath", conf.Server.Backup.Path, err)
return
}
restorePath = filepath.Join(backupPath, restorePath)
} }
if !force && !confirmYES("Warning: restoring the Navidrome database should only be done offline, especially if your backup is very old.") { if _, err := os.Stat(path); os.IsNotExist(err) {
log.Warn("Restore cancelled") log.Fatal("No existing database", "path", path)
return return
} }
if !force {
fmt.Println("Warning: restoring the Navidrome database should only be done offline, especially if your backup is very old.")
fmt.Printf("Please enter YES (all caps) to continue: ")
var input string
_, err := fmt.Scanln(&input)
if input != "YES" || err != nil {
log.Warn("Restore cancelled")
return
}
}
start := time.Now() start := time.Now()
err := db.Restore(ctx, restorePath) err := db.Restore(ctx, restorePath)
if err != nil { if err != nil {
log.Fatal("Error restoring database", "backupFile", restorePath, err) log.Fatal("Error restoring database", "backup path", conf.Server.BasePath, err)
} }
elapsed := time.Since(start) elapsed := time.Since(start)

View file

@ -1,99 +0,0 @@
package cmd
import (
"context"
"database/sql"
"fmt"
"io"
"os"
"github.com/navidrome/navidrome/db"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(doctorCmd)
}
var doctorCmd = &cobra.Command{
Use: "doctor",
Short: "Check your Navidrome installation for problems",
Long: "Run read-only health checks and report what was found. Checks the database for " +
"corruption and foreign key violations, and reports whether 'navidrome search rebuild' " +
"can fix what it finds. This command never alters your data",
Run: func(cmd *cobra.Command, _ []string) {
runDoctor(cmd.Context())
},
}
func runDoctor(ctx context.Context) {
requireExistingDB()
healthy := doctor(ctx, db.Db(), os.Stdout)
db.Close(ctx)
if !healthy {
os.Exit(1)
}
}
const recoveryAdvice = "Restore a backup (navidrome backup restore), or try SQLite's '.recover' command."
func printFindings(out io.Writer, check, noun string, items []string) {
fmt.Fprintf(out, "%s reported %d %s:\n", check, len(items), noun)
for _, item := range items {
fmt.Fprintln(out, " "+item)
}
}
func doctor(ctx context.Context, database *sql.DB, out io.Writer) bool {
healthy := true
fmt.Fprintln(out, "Checking database integrity...")
issues, truncated, err := db.IntegrityCheck(ctx, database)
switch {
case err != nil:
fmt.Fprintln(out, "The integrity check could not complete: "+err.Error())
fmt.Fprintln(out, recoveryAdvice)
return false
case len(issues) == 0:
fmt.Fprintln(out, "Integrity check passed.")
default:
healthy = false
printFindings(out, "Integrity check", "issue(s)", issues)
switch {
case truncated:
fmt.Fprintln(out, "The integrity check stopped at its limit, so the damage may reach further than listed.")
fmt.Fprintln(out, recoveryAdvice)
case db.IsFTSCorruptionOnly(issues):
fmt.Fprintln(out, "Corruption is limited to the search index. Run 'navidrome search rebuild' to fix it.")
default:
fmt.Fprintln(out, "Corruption is not limited to the search index, and cannot be repaired automatically.")
fmt.Fprintln(out, recoveryAdvice)
}
}
fmt.Fprintln(out, "Checking foreign keys...")
violations, err := db.ForeignKeyCheck(ctx, database)
switch {
case err != nil:
healthy = false
fmt.Fprintln(out, "The foreign key check could not complete: "+err.Error())
case len(violations) == 0:
fmt.Fprintln(out, "Foreign key check passed.")
default:
healthy = false
lines := make([]string, 0, len(violations))
for _, v := range violations {
lines = append(lines,
fmt.Sprintf("%s: %d row(s) reference missing rows in %s", v.Table, v.Count, v.Parent))
}
printFindings(out, "Foreign key check", "violation(s)", lines)
fmt.Fprintln(out, "These are orphaned rows, not corruption. 'navidrome scan -f' clears some of them "+
"in library data; the rest have to be removed by hand.")
}
if healthy {
fmt.Fprintln(out, "Database is healthy.")
}
return healthy
}

View file

@ -1,124 +0,0 @@
package cmd
import (
"context"
"database/sql"
"os"
"path/filepath"
"strings"
"github.com/navidrome/navidrome/db"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("doctor", func() {
var (
ctx context.Context
dbPath string
database *sql.DB
out *strings.Builder
reopen func()
)
// A file-backed DB so specs can corrupt raw pages; a table named like a real FTS
// search table so IsFTSCorruptionOnly matches, plus a parent/child pair for FK checks.
BeforeEach(func() {
ctx = context.Background()
dbPath = filepath.Join(GinkgoT().TempDir(), "doctor.db")
reopen = func() {
var err error
database, err = sql.Open(db.Dialect, dbPath)
Expect(err).ToNot(HaveOccurred())
database.SetMaxOpenConns(1)
}
reopen()
DeferCleanup(func() { _ = database.Close() })
for _, stmt := range []string{
`create virtual table media_file_fts using fts5(title, content='', content_rowid='rowid')`,
`insert into media_file_fts(rowid, title) values (1, 'teenage lobotomy'), (2, 'rockaway beach')`,
`create table library(id integer primary key)`,
`create table media_file(id integer primary key, library_id integer references library(id))`,
} {
_, err := database.ExecContext(ctx, stmt)
Expect(err).ToNot(HaveOccurred())
}
out = &strings.Builder{}
})
It("reports a healthy database", func() {
Expect(doctor(ctx, database, out)).To(BeTrue())
Expect(out.String()).To(ContainSubstring("Database is healthy."))
})
It("points to 'search rebuild' when corruption is limited to the search index", func() {
_, err := database.ExecContext(ctx,
`update media_file_fts_data set block = x'deadbeefdeadbeef' where id > 1`)
Expect(err).ToNot(HaveOccurred())
Expect(doctor(ctx, database, out)).To(BeFalse())
Expect(out.String()).To(ContainSubstring("navidrome search rebuild"))
})
It("points to a backup restore when corruption is not limited to the search index", func() {
_, err := database.ExecContext(ctx,
`insert into library(id)
with recursive s(x) as (select 1 union all select x+1 from s where x < 200)
select x from s`)
Expect(err).ToNot(HaveOccurred())
var rootPage, pageSize int64
Expect(database.QueryRowContext(ctx,
`select rootpage from sqlite_master where name = 'library'`).Scan(&rootPage)).To(Succeed())
Expect(database.QueryRowContext(ctx, `pragma page_size`).Scan(&pageSize)).To(Succeed())
Expect(database.Close()).To(Succeed())
f, err := os.OpenFile(dbPath, os.O_WRONLY, 0600)
Expect(err).ToNot(HaveOccurred())
_, err = f.WriteAt([]byte{0xde, 0xad, 0xbe, 0xef, 0xde, 0xad, 0xbe, 0xef}, (rootPage-1)*pageSize+40)
Expect(err).ToNot(HaveOccurred())
Expect(f.Close()).To(Succeed())
reopen()
Expect(doctor(ctx, database, out)).To(BeFalse())
Expect(out.String()).To(ContainSubstring("backup restore"))
Expect(out.String()).ToNot(ContainSubstring("search rebuild"))
})
It("reports foreign key violations", func() {
_, err := database.ExecContext(ctx, `pragma foreign_keys = off`)
Expect(err).ToNot(HaveOccurred())
_, err = database.ExecContext(ctx, `insert into media_file(id, library_id) values (1, 999)`)
Expect(err).ToNot(HaveOccurred())
Expect(doctor(ctx, database, out)).To(BeFalse())
Expect(out.String()).To(ContainSubstring("Foreign key check reported"))
Expect(out.String()).To(ContainSubstring("media_file"))
Expect(out.String()).To(ContainSubstring("navidrome scan -f"))
// GC never touches player, share or playqueue, so don't promise a full cleanup.
Expect(out.String()).To(ContainSubstring("removed by hand"))
})
// Every issue names an FTS-like index, so IsFTSCorruptionOnly alone would send the
// user to 'search rebuild', but the pragma stopped at its limit without saying so.
It("does not blame the search index when the issue list is truncated", func() {
for _, stmt := range []string{
`create table t(a, b)`,
`with recursive s(x) as (select 1 union all select x+1 from s where x < 300)
insert into t select x, x + 10000 from s`,
`create index media_file_fts_probe on t(a)`,
`pragma writable_schema=on`,
`update sqlite_master set sql = 'CREATE INDEX media_file_fts_probe ON t(b)'
where name = 'media_file_fts_probe'`,
} {
_, err := database.ExecContext(ctx, stmt)
Expect(err).ToNot(HaveOccurred())
}
Expect(database.Close()).To(Succeed())
reopen()
Expect(doctor(ctx, database, out)).To(BeFalse())
Expect(out.String()).ToNot(ContainSubstring("search rebuild"))
Expect(out.String()).To(ContainSubstring("backup restore"))
})
})

View file

@ -1,17 +1,13 @@
package cmd package cmd
import ( import (
"context"
"encoding/json" "encoding/json"
"fmt" "fmt"
"path/filepath"
"strings" "strings"
"github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/pelletier/go-toml/v2" "github.com/pelletier/go-toml/v2"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"gopkg.in/yaml.v3" "gopkg.in/yaml.v3"
@ -32,7 +28,7 @@ var inspectCmd = &cobra.Command{
Long: "Show file tags as seen by Navidrome", Long: "Show file tags as seen by Navidrome",
Args: cobra.MinimumNArgs(1), Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
runInspector(cmd.Context(), args) runInspector(args)
}, },
} }
@ -59,24 +55,18 @@ func prettyMarshal(v any) ([]byte, error) {
return []byte(res.String()), nil return []byte(res.String()), nil
} }
func runInspector(ctx context.Context, args []string) { func runInspector(args []string) {
marshal := marshalers[format] marshal := marshalers[format]
if marshal == nil { if marshal == nil {
log.Fatal("Invalid format", "format", format) log.Fatal("Invalid format", "format", format)
} }
libs := loadLibraries(ctx)
matcher := model.NewLibraryMatcher(libs)
var out []core.InspectOutput var out []core.InspectOutput
for _, filePath := range args { for _, filePath := range args {
if !model.IsAudioFile(filePath) { if !model.IsAudioFile(filePath) {
log.Warn("Not an audio file", "file", filePath) log.Warn("Not an audio file", "file", filePath)
continue continue
} }
lib, ok := libraryForFile(matcher, filePath) output, err := core.Inspect(filePath, 1, "")
if !ok && len(libs) > 0 {
log.Warn("File is not in any library, using the global PID config", "file", filePath)
}
output, err := core.Inspect(filePath, lib, "")
if err != nil { if err != nil {
log.Warn("Unable to process file", "file", filePath, "error", err) log.Warn("Unable to process file", "file", filePath, "error", err)
continue continue
@ -87,33 +77,3 @@ func runInspector(ctx context.Context, args []string) {
data, _ := marshal(out) data, _ := marshal(out)
fmt.Println(string(data)) fmt.Println(string(data))
} }
// loadLibraries reads the libraries, so each file gets its library's PID config. It never creates a DB.
func loadLibraries(ctx context.Context) model.Libraries {
if dbFile, ok := existingDBFile(); !ok {
log.Warn(ctx, "No database found, using the global PID config", "path", dbFile)
return nil
}
defer db.Init(ctx)()
libs, err := persistence.New(db.Db()).Library().GetAll(ctx)
if err != nil {
log.Warn(ctx, "Could not load libraries, using the global PID config", err)
return nil
}
for i := range libs {
if absPath, err := filepath.Abs(libs[i].Path); err == nil {
libs[i].Path = absPath
}
}
return libs
}
// libraryForFile falls back to the default library with no overrides, which uses the global PID config.
func libraryForFile(matcher *model.LibraryMatcher, filePath string) (model.Library, bool) {
if absPath, err := filepath.Abs(filePath); err == nil {
if lib, ok := matcher.FindLibrary(absPath); ok {
return lib, true
}
}
return model.Library{ID: model.DefaultLibraryID}, false
}

View file

@ -1,61 +0,0 @@
package cmd
import (
"os"
"path/filepath"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("inspect", func() {
Describe("libraryForFile", func() {
var matcher *model.LibraryMatcher
var root string
BeforeEach(func() {
root = GinkgoT().TempDir()
cwd, err := os.Getwd()
Expect(err).ToNot(HaveOccurred())
matcher = model.NewLibraryMatcher(model.Libraries{
{ID: 1, Path: filepath.Join(root, "music")},
{ID: 2, Path: filepath.Join(cwd, "loose"), PIDAlbum: "folder"},
})
})
It("returns the library that contains an absolute path", func() {
lib, ok := libraryForFile(matcher, filepath.Join(root, "music", "album", "track.mp3"))
Expect(ok).To(BeTrue())
Expect(lib.ID).To(Equal(1))
})
It("resolves a relative path against the working directory", func() {
lib, ok := libraryForFile(matcher, filepath.Join("loose", "track.mp3"))
Expect(ok).To(BeTrue())
Expect(lib.PIDAlbum).To(Equal("folder"))
})
It("falls back to the default library without overrides", func() {
lib, ok := libraryForFile(matcher, filepath.Join(root, "elsewhere", "track.mp3"))
Expect(ok).To(BeFalse())
Expect(lib).To(Equal(model.Library{ID: model.DefaultLibraryID}))
})
})
Describe("loadLibraries", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
It("does not create a database when there is none", func() {
dbFile := filepath.Join(GinkgoT().TempDir(), "navidrome.db")
conf.Server.DbPath = dbFile + "?_journal_mode=WAL"
Expect(loadLibraries(GinkgoT().Context())).To(BeNil())
Expect(dbFile).ToNot(BeAnExistingFile())
})
})
})

View file

@ -1,171 +0,0 @@
package cmd
import (
"bufio"
"context"
"encoding/csv"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"strconv"
"strings"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/slice"
"github.com/spf13/cobra"
)
var missingListFormat string
func init() {
missingListCmd.Flags().StringVarP(&missingListFormat, "format", "f", "csv", "output format [supported values: csv, json]")
missingCmd.AddCommand(missingListCmd)
missingCmd.AddCommand(missingFixCmd)
rootCmd.AddCommand(missingCmd)
}
var (
missingCmd = &cobra.Command{
Use: "missing",
Short: "Manage missing files",
Long: "List files marked as missing and remap them onto existing files",
}
missingListCmd = &cobra.Command{
Use: "list",
Short: "List missing files",
Run: func(cmd *cobra.Command, _ []string) {
runMissingList(cmd.Context())
},
}
missingFixCmd = &cobra.Command{
Use: "fix <missing path|id> <target path|id>",
Short: "Remap a missing file onto an existing file",
Long: "Remap a file marked as missing onto an existing (non-missing) file, the same way\n" +
"the scanner reconciles moved or renamed files. Each argument may be a media file ID,\n" +
"a library-relative path, or a libraryID:path pair.",
Args: cobra.ExactArgs(2),
Run: func(cmd *cobra.Command, args []string) {
runMissingFix(cmd.Context(), args[0], args[1])
},
}
)
type displayMissingFile struct {
ID string `json:"id"`
LibraryID int `json:"libraryId"`
Title string `json:"title"`
Album string `json:"album"`
Artist string `json:"artist"`
Path string `json:"path"`
}
func runMissingList(ctx context.Context) {
if missingListFormat != "csv" && missingListFormat != "json" {
log.Fatal("Invalid output format. Must be one of csv, json", "format", missingListFormat)
}
ds, ctx := getAdminContext(ctx)
mfs, err := ds.MediaFile().GetCursor(ctx, model.QueryOptions{
Filters: squirrel.Eq{"missing": true},
Sort: "path",
})
if err == nil {
err = writeMissingList(os.Stdout, missingListFormat, mfs)
}
if err != nil {
log.Fatal(ctx, "Failed to retrieve missing files", err)
}
}
// writeMissingList streams the cursor so a library with many missing files doesn't get loaded into memory
func writeMissingList(w io.Writer, format string, mfs model.MediaFileCursor) error {
if format == "json" {
bw := bufio.NewWriter(w)
_, _ = io.WriteString(bw, "[")
sep := ""
for mf, err := range mfs {
if err != nil {
return err
}
j, _ := json.Marshal(displayMissingFile{ID: mf.ID, LibraryID: mf.LibraryID, Title: mf.Title, Album: mf.Album, Artist: mf.Artist, Path: mf.Path})
_, _ = fmt.Fprintf(bw, "%s%s", sep, j)
sep = ","
}
_, _ = io.WriteString(bw, "]\n")
return bw.Flush()
}
cw := csv.NewWriter(w)
_ = cw.Write([]string{"id", "library id", "title", "album", "artist", "path"})
for mf, err := range mfs {
if err != nil {
return err
}
_ = cw.Write([]string{mf.ID, strconv.Itoa(mf.LibraryID), mf.Title, mf.Album, mf.Artist, mf.Path})
}
cw.Flush()
return cw.Error()
}
func runMissingFix(ctx context.Context, missingRef, targetRef string) {
ds, ctx := getAdminContext(ctx)
missing := resolveMediaFile(ctx, ds, missingRef)
target := resolveMediaFile(ctx, ds, targetRef)
if err := core.NewMaintenance(ds).RemapMissingFile(ctx, missing.ID, target.ID); err != nil {
log.Fatal(ctx, "Failed to remap missing file", "missing", missing.Path, "target", target.Path, err)
}
fmt.Printf("Remapped %q onto %q\n", missing.Path, target.Path)
}
// resolveMediaFile looks up a media file by ID first, then by path (optionally libraryID:path).
func resolveMediaFile(ctx context.Context, ds model.DataStore, ref string) *model.MediaFile {
mf, err := ds.MediaFile().Get(ctx, ref)
if err == nil {
return mf
}
if !errors.Is(err, model.ErrNotFound) {
log.Fatal(ctx, "Error looking up media file", "ref", ref, err)
}
mfs, err := ds.MediaFile().FindByPaths(ctx, []string{ref})
if err != nil {
log.Fatal(ctx, "Error looking up media file by path", "ref", ref, err)
}
if len(mfs) == 0 {
log.Fatal(ctx, "No media file found", "ref", ref)
}
mfs = preferQualified(ref, mfs)
if len(mfs) > 1 {
log.Fatal(ctx, "Path matches multiple files; disambiguate with an ID or libraryID:path", "ref", ref, "matches", len(mfs))
}
return &mfs[0]
}
// preferQualified resolves the ambiguity FindByPaths creates by searching a "libraryID:path"
// reference both ways: an explicit library wins over a file literally named like one.
func preferQualified(ref string, mfs model.MediaFiles) model.MediaFiles {
id, path, ok := strings.Cut(ref, ":")
if !ok {
return mfs
}
libraryID, err := strconv.Atoi(id)
if err != nil {
return mfs
}
qualified := slice.Filter(mfs, func(mf model.MediaFile) bool {
return mf.LibraryID == libraryID && strings.EqualFold(mf.Path, path)
})
if len(qualified) == 0 {
return mfs
}
return qualified
}

View file

@ -1,81 +0,0 @@
package cmd
import (
"errors"
"strings"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("writeMissingList", func() {
cursor := func(err error, mfs ...model.MediaFile) model.MediaFileCursor {
return func(yield func(model.MediaFile, error) bool) {
for _, mf := range mfs {
if !yield(mf, nil) {
return
}
}
if err != nil {
yield(model.MediaFile{}, err)
}
}
}
song := model.MediaFile{ID: "1", LibraryID: 1, Path: "Bach: Goldberg/01.mp3", Title: "Aria", Album: "Goldberg", Artist: "Bach"}
It("writes csv with a header, quoting as needed", func() {
var out strings.Builder
Expect(writeMissingList(&out, "csv", cursor(nil, song))).To(Succeed())
Expect(out.String()).To(Equal("id,library id,title,album,artist,path\n1,1,Aria,Goldberg,Bach,Bach: Goldberg/01.mp3\n"))
})
It("writes a json array", func() {
var out strings.Builder
Expect(writeMissingList(&out, "json", cursor(nil, song, song))).To(Succeed())
Expect(out.String()).To(MatchJSON(`[
{"id":"1","libraryId":1,"path":"Bach: Goldberg/01.mp3","title":"Aria","album":"Goldberg","artist":"Bach"},
{"id":"1","libraryId":1,"path":"Bach: Goldberg/01.mp3","title":"Aria","album":"Goldberg","artist":"Bach"}
]`))
})
It("writes an empty json array when nothing is missing", func() {
var out strings.Builder
Expect(writeMissingList(&out, "json", cursor(nil))).To(Succeed())
Expect(out.String()).To(MatchJSON(`[]`))
})
It("returns the cursor's error", func() {
var out strings.Builder
Expect(writeMissingList(&out, "csv", cursor(errors.New("boom"), song))).To(MatchError("boom"))
})
})
var _ = Describe("preferQualified", func() {
target := model.MediaFile{ID: "want", LibraryID: 1, Path: "foo.mp3"}
decoy := model.MediaFile{ID: "decoy", LibraryID: 1, Path: "1:foo.mp3"}
It("picks the library-qualified match over a literal path that looks like one", func() {
Expect(preferQualified("1:foo.mp3", model.MediaFiles{target, decoy})).To(Equal(model.MediaFiles{target}))
})
It("picks the named library when the same path exists in two", func() {
other := model.MediaFile{ID: "other", LibraryID: 2, Path: "foo.mp3"}
Expect(preferQualified("1:foo.mp3", model.MediaFiles{target, other})).To(Equal(model.MediaFiles{target}))
})
It("leaves an unqualified reference ambiguous", func() {
both := model.MediaFiles{target, {ID: "other", LibraryID: 2, Path: "foo.mp3"}}
Expect(preferQualified("foo.mp3", both)).To(Equal(both))
})
It("leaves it alone when the prefix is not a library id", func() {
both := model.MediaFiles{decoy, {ID: "other", LibraryID: 2, Path: "1:foo.mp3"}}
Expect(preferQualified("x:foo.mp3", both)).To(Equal(both))
})
It("leaves it alone when no candidate matches the qualified form", func() {
both := model.MediaFiles{decoy, {ID: "other", LibraryID: 2, Path: "1:foo.mp3"}}
Expect(preferQualified("9:nope.mp3", both)).To(Equal(both))
})
})

View file

@ -6,14 +6,13 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io"
"os" "os"
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings" "strings"
"github.com/Masterminds/squirrel" "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
@ -109,7 +108,7 @@ func fetchPlaylists(ctx context.Context, ds model.DataStore, sort string) model.
} }
options.Filters = squirrel.Eq{"owner_id": user.ID} options.Filters = squirrel.Eq{"owner_id": user.ID}
} }
pls, err := ds.Playlist().GetAll(ctx, options) pls, err := ds.Playlist(ctx).GetAll(options)
if err != nil { if err != nil {
log.Fatal(ctx, "Failed to retrieve playlists", err) log.Fatal(ctx, "Failed to retrieve playlists", err)
} }
@ -117,17 +116,17 @@ func fetchPlaylists(ctx context.Context, ds model.DataStore, sort string) model.
} }
func findPlaylist(ctx context.Context, ds model.DataStore, nameOrID string) *model.Playlist { func findPlaylist(ctx context.Context, ds model.DataStore, nameOrID string) *model.Playlist {
playlist, err := ds.Playlist().GetWithTracks(ctx, nameOrID, true, false) playlist, err := ds.Playlist(ctx).GetWithTracks(nameOrID, true, false)
if err != nil && !errors.Is(err, model.ErrNotFound) { if err != nil && !errors.Is(err, model.ErrNotFound) {
log.Fatal("Error retrieving playlist", "name", nameOrID, err) log.Fatal("Error retrieving playlist", "name", nameOrID, err)
} }
if errors.Is(err, model.ErrNotFound) { if errors.Is(err, model.ErrNotFound) {
playlists, err := ds.Playlist().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"playlist.name": nameOrID}}) playlists, err := ds.Playlist(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"playlist.name": nameOrID}})
if err != nil { if err != nil {
log.Fatal("Error retrieving playlist", "name", nameOrID, err) log.Fatal("Error retrieving playlist", "name", nameOrID, err)
} }
if len(playlists) > 0 { if len(playlists) > 0 {
playlist, err = ds.Playlist().GetWithTracks(ctx, playlists[0].ID, true, false) playlist, err = ds.Playlist(ctx).GetWithTracks(playlists[0].ID, true, false)
if err != nil { if err != nil {
log.Fatal("Error retrieving playlist", "name", nameOrID, err) log.Fatal("Error retrieving playlist", "name", nameOrID, err)
} }
@ -142,16 +141,14 @@ func findPlaylist(ctx context.Context, ds model.DataStore, nameOrID string) *mod
func runExporter(ctx context.Context) { func runExporter(ctx context.Context) {
ds, ctx := getAdminContext(ctx) ds, ctx := getAdminContext(ctx)
playlist := findPlaylist(ctx, ds, playlistID) playlist := findPlaylist(ctx, ds, playlistID)
writePlaylist(playlist.ToM3U8(), os.Stdout, outputFile) pls := playlist.ToM3U8()
} if outputFile == "-" || outputFile == "" {
println(pls)
func writePlaylist(m3u string, out io.Writer, file string) {
if file == "" || file == "-" {
fmt.Fprint(out, m3u)
return return
} }
if err := os.WriteFile(file, []byte(m3u), 0600); err != nil { err := os.WriteFile(outputFile, []byte(pls), 0600)
log.Fatal("Error writing to the output file", "file", file, err) if err != nil {
log.Fatal("Error writing to the output file", "file", outputFile, err)
} }
} }
@ -160,7 +157,7 @@ func runExport(ctx context.Context) {
if playlistID != "" && outputFile == "" { if playlistID != "" && outputFile == "" {
playlist := findPlaylist(ctx, ds, playlistID) playlist := findPlaylist(ctx, ds, playlistID)
writePlaylist(playlist.ToM3U8(), os.Stdout, outputFile) println(playlist.ToM3U8())
return return
} }
@ -194,7 +191,7 @@ func runExport(ctx context.Context) {
exported := 0 exported := 0
for _, pls := range allPls { for _, pls := range allPls {
plsWithTracks, err := ds.Playlist().GetWithTracks(ctx, pls.ID, true, false) plsWithTracks, err := ds.Playlist(ctx).GetWithTracks(pls.ID, true, false)
if err != nil { if err != nil {
log.Error("Error loading playlist tracks", "playlist", pls.Name, err) log.Error("Error loading playlist tracks", "playlist", pls.Name, err)
continue continue
@ -263,7 +260,7 @@ func runImport(ctx context.Context, files []string) {
ctx = request.WithUser(ctx, *user) ctx = request.WithUser(ctx, *user)
} }
pls := playlists.NewPlaylists(ds, artwork.NewUploader(ds)) pls := playlists.NewPlaylists(ds, core.NewImageUploadService(ds))
for _, file := range files { for _, file := range files {
absPath, err := filepath.Abs(file) absPath, err := filepath.Abs(file)

View file

@ -1,35 +0,0 @@
package cmd
import (
"fmt"
"os"
"path/filepath"
"strings"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("writePlaylist", func() {
const m3u = "#EXTM3U\n#PLAYLIST:DJ Wave\n#EXTINF:364,Bel Canto - Dreaming Girl\n"
plsFile := filepath.Join(os.TempDir(), fmt.Sprintf("navidrome-pls-%d.m3u8", os.Getpid()))
BeforeEach(func() {
DeferCleanup(func() { _ = os.Remove(plsFile) })
})
DescribeTable("writes the playlist to exactly one destination",
func(file, wantStream, wantFile string) {
var out strings.Builder
writePlaylist(m3u, &out, file)
written, _ := os.ReadFile(plsFile)
Expect(out.String()).To(Equal(wantStream))
Expect(string(written)).To(Equal(wantFile))
},
Entry("no file name writes to the stream", "", m3u, ""),
Entry("a dash writes to the stream", "-", m3u, ""),
Entry("a path writes to the file", plsFile, "", m3u),
)
})

View file

@ -9,6 +9,7 @@ import (
"os" "os"
"strconv" "strconv"
"strings" "strings"
"text/tabwriter"
"time" "time"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
@ -243,7 +244,7 @@ func runPluginInfo(ctx context.Context, arg string) {
} }
requirePluginsEnabled(ctx) requirePluginsEnabled(ctx)
ds, ctx := getAdminContext(ctx) ds, ctx := getAdminContext(ctx)
p, err := ds.Plugin().Get(ctx, arg) p, err := ds.Plugin(ctx).Get(arg)
if err != nil { if err != nil {
log.Fatal(ctx, "Plugin not found", "id", arg, err) log.Fatal(ctx, "Plugin not found", "id", arg, err)
} }
@ -264,7 +265,7 @@ func runPluginValidate(ctx context.Context, arg string) {
} }
requirePluginsEnabled(ctx) requirePluginsEnabled(ctx)
ds, ctx := getAdminContext(ctx) ds, ctx := getAdminContext(ctx)
p, err := ds.Plugin().Get(ctx, arg) p, err := ds.Plugin(ctx).Get(arg)
if err != nil { if err != nil {
log.Fatal(ctx, "Plugin not found", "id", arg, err) log.Fatal(ctx, "Plugin not found", "id", arg, err)
} }
@ -313,7 +314,7 @@ func formatPluginList(list model.Plugins, format string) (string, error) {
return sb.String(), w.Error() return sb.String(), w.Error()
case "table": case "table":
var sb strings.Builder var sb strings.Builder
w := newTabWriter(&sb) w := tabwriter.NewWriter(&sb, 0, 4, 2, ' ', 0)
fmt.Fprintln(w, "ID\tNAME\tVERSION\tENABLED\tLAST ERROR") fmt.Fprintln(w, "ID\tNAME\tVERSION\tENABLED\tLAST ERROR")
for _, p := range list { for _, p := range list {
name, version := manifestSummary(p) name, version := manifestSummary(p)
@ -329,7 +330,7 @@ func formatPluginList(list model.Plugins, format string) (string, error) {
func runPluginList(ctx context.Context) { func runPluginList(ctx context.Context) {
requirePluginsEnabled(ctx) requirePluginsEnabled(ctx)
ds, ctx := getAdminContext(ctx) ds, ctx := getAdminContext(ctx)
list, err := ds.Plugin().GetAll(ctx) list, err := ds.Plugin(ctx).GetAll()
if err != nil { if err != nil {
log.Fatal(ctx, "Failed to list plugins", err) log.Fatal(ctx, "Failed to list plugins", err)
} }
@ -372,7 +373,7 @@ var pluginEditCmd = &cobra.Command{
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
requirePluginsEnabled(cmd.Context()) requirePluginsEnabled(cmd.Context())
ds, ctx := getAdminContext(cmd.Context()) ds, ctx := getAdminContext(cmd.Context())
cur, err := ds.Plugin().Get(ctx, args[0]) cur, err := ds.Plugin(ctx).Get(args[0])
if err != nil { if err != nil {
log.Fatal(ctx, "Plugin not found", "id", args[0], err) log.Fatal(ctx, "Plugin not found", "id", args[0], err)
} }

View file

@ -2,7 +2,6 @@ package cmd
import ( import (
"context" "context"
"net/http"
"os" "os"
"os/signal" "os/signal"
"strings" "strings"
@ -44,9 +43,7 @@ Complete documentation is available at https://www.navidrome.org/docs`,
preRun() preRun()
}, },
Run: func(cmd *cobra.Command, args []string) { Run: func(cmd *cobra.Command, args []string) {
if err := runNavidrome(cmd.Context()); err != nil { runNavidrome(cmd.Context())
log.Fatal("Fatal error in Navidrome. Aborting", err)
}
}, },
PostRun: func(cmd *cobra.Command, args []string) { PostRun: func(cmd *cobra.Command, args []string) {
postRun() postRun()
@ -78,17 +75,16 @@ func postRun() {
} }
// runNavidrome is the main entry point for the Navidrome server. It starts all the services and blocks. // runNavidrome is the main entry point for the Navidrome server. It starts all the services and blocks.
// If any of the services returns an error, it stops the others and returns that error, so the caller can // If any of the services returns an error, it will log it and exit. If the process receives a signal to exit,
// exit with a non-zero code. If the context is cancelled (a signal or a service stop), it returns nil. // it will cancel the context and exit gracefully.
func runNavidrome(parentCtx context.Context) error { func runNavidrome(ctx context.Context) {
defer db.Init(parentCtx)() defer db.Init(ctx)()
g, ctx := errgroup.WithContext(parentCtx) g, ctx := errgroup.WithContext(ctx)
g.Go(startServer(ctx)) g.Go(startServer(ctx))
g.Go(startSignaller(ctx)) g.Go(startSignaller(ctx))
g.Go(startScheduler(ctx)) g.Go(startScheduler(ctx))
g.Go(startPlaybackServer(ctx)) g.Go(startPlaybackServer(ctx))
g.Go(startJellyfinDiscovery(ctx))
g.Go(schedulePeriodicBackup(ctx)) g.Go(schedulePeriodicBackup(ctx))
g.Go(startInsightsCollector(ctx)) g.Go(startInsightsCollector(ctx))
g.Go(scheduleDBAnalyzer(ctx)) g.Go(scheduleDBAnalyzer(ctx))
@ -104,11 +100,9 @@ func runNavidrome(parentCtx context.Context) error {
log.Warn(ctx, "Automatic Scanning is DISABLED") log.Warn(ctx, "Automatic Scanning is DISABLED")
} }
// Errors caused by a normal shutdown are not failures if err := g.Wait(); err != nil {
if err := g.Wait(); err != nil && parentCtx.Err() == nil { log.Error("Fatal error in Navidrome. Aborting", err)
return err
} }
return nil
} }
// mainContext returns a context that is cancelled when the process receives a signal to exit. // mainContext returns a context that is cancelled when the process receives a signal to exit.
@ -137,9 +131,6 @@ func startServer(ctx context.Context) func() error {
if conf.Server.Jellyfin.Enabled { if conf.Server.Jellyfin.Enabled {
a.MountRouter("Jellyfin API", consts.URLPathJellyfinAPI, CreateJellyfinAPIRouter(ctx)) a.MountRouter("Jellyfin API", consts.URLPathJellyfinAPI, CreateJellyfinAPIRouter(ctx))
} }
if conf.Server.DevAPIv1 {
a.MountRouter("API v1", consts.URLPathAPIv1, CreateAPIv1Router(ctx))
}
if conf.Server.Prometheus.Enabled { if conf.Server.Prometheus.Enabled {
p := CreatePrometheus() p := CreatePrometheus()
// blocking call because takes <100ms but useful if fails // blocking call because takes <100ms but useful if fails
@ -147,7 +138,7 @@ func startServer(ctx context.Context) func() error {
a.MountRouter("Prometheus metrics", conf.Server.Prometheus.MetricsPath, p.GetHandler()) a.MountRouter("Prometheus metrics", conf.Server.Prometheus.MetricsPath, p.GetHandler())
} }
if conf.Server.DevEnableProfiler { if conf.Server.DevEnableProfiler {
a.MountRouter("Profiling", "/debug", profilerHandler()) a.MountRouter("Profiling", "/debug", middleware.Profiler())
} }
if strings.HasPrefix(conf.Server.UILoginBackgroundURL, "/") { if strings.HasPrefix(conf.Server.UILoginBackgroundURL, "/") {
a.MountRouter("Background images", conf.Server.UILoginBackgroundURL, backgrounds.NewHandler()) a.MountRouter("Background images", conf.Server.UILoginBackgroundURL, backgrounds.NewHandler())
@ -156,14 +147,6 @@ func startServer(ctx context.Context) func() error {
} }
} }
// profilerHandler returns the pprof handler. net/http/pprof resolves the profile
// name from the raw request path, so the BasePath has to come off first.
func profilerHandler() http.Handler {
// A trailing or root slash would make StripPrefix drop the leading slash chi needs.
basePath := strings.TrimRight(conf.Server.BasePath, "/")
return http.StripPrefix(basePath, middleware.Profiler())
}
// schedulePeriodicScan schedules a periodic scan of the music library, if configured. // schedulePeriodicScan schedules a periodic scan of the music library, if configured.
func schedulePeriodicScan(ctx context.Context) func() error { func schedulePeriodicScan(ctx context.Context) func() error {
return func() error { return func() error {
@ -190,50 +173,46 @@ func schedulePeriodicScan(ctx context.Context) func() error {
} }
} }
// librariesWithChangedPID returns the names of the libraries whose effective PID config differs from func pidHashChanged(ds model.DataStore) (bool, error) {
// the one used by their last finished scan pidAlbum, err := ds.Property(context.Background()).DefaultGet(consts.PIDAlbumKey, "")
func librariesWithChangedPID(ctx context.Context, ds model.DataStore) ([]string, error) {
libs, err := ds.Library().GetAll(ctx)
if err != nil { if err != nil {
return nil, err return false, err
} }
var names []string pidTrack, err := ds.Property(context.Background()).DefaultGet(consts.PIDTrackKey, "")
for _, lib := range libs { if err != nil {
if lib.PIDChanged() { return false, err
names = append(names, lib.Name)
}
} }
return names, nil return !strings.EqualFold(pidAlbum, conf.Server.PID.Album) || !strings.EqualFold(pidTrack, conf.Server.PID.Track), nil
} }
// runInitialScan runs an initial scan of the music library if needed. // runInitialScan runs an initial scan of the music library if needed.
func runInitialScan(ctx context.Context) func() error { func runInitialScan(ctx context.Context) func() error {
return func() error { return func() error {
ds := CreateDataStore() ds := CreateDataStore()
fullScanRequired, err := ds.Property().DefaultGet(ctx, consts.FullScanAfterMigrationFlagKey, "0") fullScanRequired, err := ds.Property(ctx).DefaultGet(consts.FullScanAfterMigrationFlagKey, "0")
if err != nil { if err != nil {
return err return err
} }
inProgress, err := ds.Library().ScanInProgress(ctx) inProgress, err := ds.Library(ctx).ScanInProgress()
if err != nil { if err != nil {
return err return err
} }
pidChangedLibs, err := librariesWithChangedPID(ctx, ds) pidHasChanged, err := pidHashChanged(ds)
if err != nil { if err != nil {
return err return err
} }
scanOnStartup := conf.Server.Scanner.Enabled && conf.Server.Scanner.ScanOnStartup scanOnStartup := conf.Server.Scanner.Enabled && conf.Server.Scanner.ScanOnStartup
scanNeeded := scanOnStartup || inProgress || fullScanRequired == "1" || len(pidChangedLibs) > 0 scanNeeded := scanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged
time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan
if scanNeeded { if scanNeeded {
s := CreateScanner(ctx) s := CreateScanner(ctx)
switch { switch {
case fullScanRequired == "1": case fullScanRequired == "1":
log.Warn(ctx, "Full scan required after migration") log.Warn(ctx, "Full scan required after migration")
_ = ds.Property().Delete(ctx, consts.FullScanAfterMigrationFlagKey) _ = ds.Property(ctx).Delete(consts.FullScanAfterMigrationFlagKey)
case len(pidChangedLibs) > 0: case pidHasChanged:
// Includes never-scanned libraries. The scanner rescans in full only the ones that need it log.Warn(ctx, "PID config changed, performing full scan")
log.Warn(ctx, "Libraries with a new or changed PID config, scanning", "libraries", pidChangedLibs) fullScanRequired = "1"
case inProgress: case inProgress:
log.Warn(ctx, "Resuming interrupted scan") log.Warn(ctx, "Resuming interrupted scan")
default: default:
@ -355,18 +334,6 @@ func startInsightsCollector(ctx context.Context) func() error {
} }
} }
// startJellyfinDiscovery never returns an error: a discovery failure must not stop the server.
func startJellyfinDiscovery(ctx context.Context) func() error {
return func() error {
if !conf.Server.Jellyfin.Enabled || !conf.Server.Jellyfin.AutoDiscovery {
log.Debug("Jellyfin auto-discovery is DISABLED")
return nil
}
CreateJellyfinDiscovery().Serve(ctx)
return nil
}
}
// startPlaybackServer starts the Navidrome playback server, if configured. // startPlaybackServer starts the Navidrome playback server, if configured.
// It is responsible for the Jukebox functionality // It is responsible for the Jukebox functionality
func startPlaybackServer(ctx context.Context) func() error { func startPlaybackServer(ctx context.Context) func() error {
@ -386,50 +353,50 @@ func startPlaybackServer(ctx context.Context) func() error {
func startArtworkWorker(ctx context.Context, worker *artwork.Worker) func() error { func startArtworkWorker(ctx context.Context, worker *artwork.Worker) func() error {
return func() error { return func() error {
log.Info(ctx, "Starting artwork worker") log.Info(ctx, "Starting artwork worker")
// The scanner writes to the DB for its whole run; competing for the write lock makes both fail.
worker.PauseWhile(scanner.IsScanning)
return worker.Run(ctx) return worker.Run(ctx)
} }
} }
// outsideScan runs a DB maintenance job unless a scan is running, and keeps a scan from starting // scheduleArtworkHousekeeping runs the startup fingerprint backfill and registers the
// until it ends; both write to the DB, and competing for the lock can make either fail. // recurring stale-absent recheck and prune jobs. Scan-triggered prune lands in a later phase.
func outsideScan(ctx context.Context, job string, run func(context.Context) error) {
release, ok := scanner.LockForMaintenance()
if !ok {
log.Debug(ctx, "Skipping "+job+" because a scan is in progress")
return
}
defer release()
if err := run(ctx); err != nil {
log.Error(ctx, "Error running "+job, err)
}
}
// scheduleArtworkHousekeeping registers the recurring missing-state and prune jobs, and
// reports an artwork config change without acting on it.
func scheduleArtworkHousekeeping(ctx context.Context, worker *artwork.Worker) func() error { func scheduleArtworkHousekeeping(ctx context.Context, worker *artwork.Worker) func() error {
return func() error { return func() error {
ds := CreateDataStore()
schedulerInstance := scheduler.GetInstance() schedulerInstance := scheduler.GetInstance()
if _, err := schedulerInstance.Add(consts.ArtworkEnqueueMissingSchedule, func() { if _, err := schedulerInstance.Add(consts.ArtworkStaleAbsentRecheckSchedule, func() {
outsideScan(ctx, "artwork missing-state recheck", worker.EnqueueMissingAll) if err := artwork.EnqueueStaleAbsentAll(ctx, ds); err != nil {
log.Error(ctx, "Error enqueueing stale artwork rechecks", err)
}
}); err != nil { }); err != nil {
log.Error(ctx, "Error scheduling artwork missing-state recheck", err) log.Error(ctx, "Error scheduling artwork stale-absent recheck", err)
} }
if _, err := schedulerInstance.Add(consts.ArtworkPruneSchedule, func() { if _, err := schedulerInstance.Add(consts.ArtworkPruneSchedule, func() {
outsideScan(ctx, "artwork prune", worker.RunPrune) if err := worker.RunPrune(ctx); err != nil {
log.Error(ctx, "Error running artwork prune", err)
}
}); err != nil { }); err != nil {
log.Error(ctx, "Error scheduling artwork prune", err) log.Error(ctx, "Error scheduling artwork prune", err)
} }
// Also run the missing-row recheck once at startup so a never-scanned entity is picked up backfilled, err := artwork.Backfill(ctx, ds)
// immediately, not only on the next hourly tick (e.g. after enabling the feature). if err != nil {
outsideScan(ctx, "artwork missing-state recheck", worker.EnqueueMissingAll) log.Error(ctx, "Error running artwork backfill", err)
return nil
if err := worker.ReconcileConfig(ctx); err != nil { }
log.Error(ctx, "Error checking the artwork config fingerprint", err) if !backfilled {
return nil
}
log.Info(ctx, "Artwork backfill enqueued, scheduling a follow-up prune")
timer := time.NewTimer(consts.ArtworkPostBackfillPruneDelay)
defer timer.Stop()
select {
case <-timer.C:
if err := worker.RunPrune(ctx); err != nil {
log.Error(ctx, "Error running post-backfill artwork prune", err)
}
case <-ctx.Done():
} }
return nil return nil
} }
@ -485,7 +452,7 @@ func init() {
rootCmd.Flags().String("albumplaycountmode", viper.GetString("albumplaycountmode"), "how to compute playcount for albums. absolute (default) or normalized") rootCmd.Flags().String("albumplaycountmode", viper.GetString("albumplaycountmode"), "how to compute playcount for albums. absolute (default) or normalized")
rootCmd.Flags().Bool("autoimportplaylists", viper.GetBool("autoimportplaylists"), "enable/disable .m3u playlist auto-import`") rootCmd.Flags().Bool("autoimportplaylists", viper.GetBool("autoimportplaylists"), "enable/disable .m3u playlist auto-import`")
rootCmd.Flags().Bool("prometheus.enabled", viper.GetBool("prometheus.enabled"), "enable/disable prometheus metrics endpoint") rootCmd.Flags().Bool("prometheus.enabled", viper.GetBool("prometheus.enabled"), "enable/disable prometheus metrics endpoint`")
rootCmd.Flags().String("prometheus.metricspath", viper.GetString("prometheus.metricspath"), "http endpoint for prometheus metrics") rootCmd.Flags().String("prometheus.metricspath", viper.GetString("prometheus.metricspath"), "http endpoint for prometheus metrics")
_ = viper.BindPFlag("address", rootCmd.Flags().Lookup("address")) _ = viper.BindPFlag("address", rootCmd.Flags().Lookup("address"))

View file

@ -1,76 +0,0 @@
package cmd
import (
"errors"
"net/http"
"net/http/httptest"
"path"
"runtime/pprof"
"github.com/go-chi/chi/v5"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = pprof.NewProfile("nd-profiler-test")
var _ = Describe("profilerHandler", func() {
// Mirrors how server.MountRouter mounts the handler.
mount := func() http.Handler {
router := chi.NewRouter()
router.Mount(path.Join(conf.Server.BasePath, "/debug"), profilerHandler())
return router
}
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
DescribeTable("serves a named profile",
func(basePath string) {
conf.Server.BasePath = basePath
w := httptest.NewRecorder()
target := path.Join(basePath, "/debug/pprof/nd-profiler-test") + "?debug=1"
mount().ServeHTTP(w, httptest.NewRequest(http.MethodGet, target, nil))
Expect(w.Code).To(Equal(http.StatusOK))
Expect(w.Body.String()).To(HavePrefix("nd-profiler-test profile: total 0"))
},
Entry("without a BasePath", ""),
Entry("with a BasePath", "/music"),
Entry("with a root BasePath", "/"),
Entry("with a trailing-slash BasePath", "/music/"),
)
})
var _ = Describe("librariesWithChangedPID", func() {
var ds *tests.MockDataStore
var libs *tests.MockLibraryRepo
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
libs = &tests.MockLibraryRepo{}
ds = &tests.MockDataStore{MockedLibrary: libs}
})
It("returns only the libraries whose PID config changed", func() {
pid := model.Library{}.EffectivePID()
libs.SetData(model.Libraries{
{ID: 1, Name: "Same", ScannedPIDAlbum: pid.Album, ScannedPIDTrack: pid.Track},
{ID: 2, Name: "Changed", PIDAlbum: "folder", ScannedPIDAlbum: pid.Album, ScannedPIDTrack: pid.Track},
{ID: 3, Name: "Never scanned"},
})
Expect(librariesWithChangedPID(GinkgoT().Context(), ds)).To(ConsistOf("Changed", "Never scanned"))
})
It("returns the error from the repository", func() {
libs.Err = errors.New("db down")
_, err := librariesWithChangedPID(GinkgoT().Context(), ds)
Expect(err).To(MatchError("db down"))
})
})

View file

@ -9,7 +9,7 @@ import (
"os" "os"
"strings" "strings"
"github.com/navidrome/navidrome/core/artwork" "github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
@ -82,7 +82,7 @@ func runScanner(ctx context.Context) {
sqlDB := db.Db() sqlDB := db.Db()
defer db.Db().Close() defer db.Db().Close()
ds := persistence.New(sqlDB) ds := persistence.New(sqlDB)
pls := playlists.NewPlaylists(ds, artwork.NewUploader(ds)) pls := playlists.NewPlaylists(ds, core.NewImageUploadService(ds))
// Parse targets from command line or file // Parse targets from command line or file
var scanTargets []model.ScanTarget var scanTargets []model.ScanTarget

View file

@ -1,55 +0,0 @@
package cmd
import (
"context"
"fmt"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log"
"github.com/spf13/cobra"
)
var searchRebuildForce bool
func init() {
rootCmd.AddCommand(searchRoot)
searchRebuildCmd.Flags().BoolVarP(&searchRebuildForce, "force", "f", false, "bypass rebuild confirmation")
searchRoot.AddCommand(searchRebuildCmd)
}
var (
searchRoot = &cobra.Command{
Use: "search",
Short: "Search index maintenance",
}
searchRebuildCmd = &cobra.Command{
Use: "rebuild",
Short: "Rebuild the full-text search index",
Long: "Drop and rebuild the full-text search index from the library data. Fixes a corrupted " +
"or desynced search index without any data loss. Note that 'navidrome doctor' detects a " +
"corrupted index, but cannot tell when the index has merely drifted out of sync with the " +
"library. This must be done offline",
Run: func(cmd *cobra.Command, _ []string) {
runSearchRebuild(cmd.Context())
},
}
)
func runSearchRebuild(ctx context.Context) {
requireExistingDB()
if !searchRebuildForce && !confirmYES("This will rebuild the search index. Make sure Navidrome is not running.") {
log.Warn("Rebuild cancelled")
return
}
fmt.Println("Rebuilding the search index...")
err := db.RebuildFTS(ctx, db.Db())
db.Close(ctx)
if err != nil {
log.Fatal("Error rebuilding the search index", err)
}
fmt.Println("Search index rebuilt successfully.")
}

View file

@ -44,7 +44,7 @@ var svcCmd = &cobra.Command{
} }
type svcControl struct { type svcControl struct {
ctx context.Context //nolint:containedctx // service lifecycle ctx, cancelled by Stop ctx context.Context
cancel context.CancelFunc cancel context.CancelFunc
done chan struct{} done chan struct{}
} }
@ -53,13 +53,8 @@ func (p *svcControl) Start(service.Service) error {
p.done = make(chan struct{}) p.done = make(chan struct{})
p.ctx, p.cancel = context.WithCancel(context.Background()) p.ctx, p.cancel = context.WithCancel(context.Background())
go func() { go func() {
err := runNavidrome(p.ctx) runNavidrome(p.ctx)
close(p.done) close(p.done)
// service.Run() only returns when it gets a stop request, so exit here to let the
// service manager see the failure and restart the service
if err != nil {
log.Fatal("Fatal error in Navidrome. Aborting", err)
}
}() }()
return nil return nil
} }
@ -79,7 +74,7 @@ func (p *svcControl) Stop(service.Service) error {
var svcInstance = sync.OnceValue(func() service.Service { var svcInstance = sync.OnceValue(func() service.Service {
options := make(service.KeyValue) options := make(service.KeyValue)
options["Restart"] = "on-failure" options["Restart"] = "on-failure"
options["SuccessExitStatus"] = "SIGKILL" options["SuccessExitStatus"] = "1 2 8 SIGKILL"
options["UserService"] = false options["UserService"] = false
options["LogDirectory"] = conf.Server.DataFolder.String() options["LogDirectory"] = conf.Server.DataFolder.String()
options["SystemdScript"] = systemdScript options["SystemdScript"] = systemdScript

View file

@ -183,7 +183,7 @@ func runCreateUser(ctx context.Context) {
ds, ctx := getAdminContext(ctx) ds, ctx := getAdminContext(ctx)
err := ds.WithTx(func(tx model.DataStore) error { err := ds.WithTx(func(tx model.DataStore) error {
existingUser, err := tx.User().FindByUsername(ctx, userID) existingUser, err := tx.User(ctx).FindByUsername(userID)
if existingUser != nil { if existingUser != nil {
return fmt.Errorf("existing user '%s'", userID) return fmt.Errorf("existing user '%s'", userID)
} }
@ -193,7 +193,7 @@ func runCreateUser(ctx context.Context) {
} }
if len(libraryIds) > 0 && !setAdmin { if len(libraryIds) > 0 && !setAdmin {
user.Libraries, err = tx.Library().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}}) user.Libraries, err = tx.Library(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}})
if err != nil { if err != nil {
return err return err
} }
@ -202,13 +202,13 @@ func runCreateUser(ctx context.Context) {
return libraryError(user.Libraries) return libraryError(user.Libraries)
} }
} else { } else {
user.Libraries, err = tx.Library().GetAll(ctx) user.Libraries, err = tx.Library(ctx).GetAll()
if err != nil { if err != nil {
return err return err
} }
} }
err = tx.User().Put(ctx, &user) err = tx.User(ctx).Put(&user)
if err != nil { if err != nil {
return err return err
} }
@ -218,7 +218,7 @@ func runCreateUser(ctx context.Context) {
updatedIds[idx] = lib.ID updatedIds[idx] = lib.ID
} }
err = tx.User().SetUserLibraries(ctx, user.ID, updatedIds) err = tx.User(ctx).SetUserLibraries(user.ID, updatedIds)
return err return err
}) })
@ -236,7 +236,7 @@ func runDeleteUser(ctx context.Context) {
var user *model.User var user *model.User
err = ds.WithTx(func(tx model.DataStore) error { err = ds.WithTx(func(tx model.DataStore) error {
count, err := tx.User().CountAll(ctx) count, err := tx.User(ctx).CountAll()
if err != nil { if err != nil {
return err return err
} }
@ -250,7 +250,7 @@ func runDeleteUser(ctx context.Context) {
return err return err
} }
return tx.User().Delete(ctx, user.ID) return tx.User(ctx).Delete(user.ID)
}) })
if err != nil { if err != nil {
@ -276,7 +276,7 @@ func runUserEdit(ctx context.Context) {
} }
if len(libraryIds) > 0 && !setAdmin { if len(libraryIds) > 0 && !setAdmin {
libraries, err := tx.Library().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}}) libraries, err := tx.Library(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}})
if err != nil { if err != nil {
return err return err
@ -291,7 +291,7 @@ func runUserEdit(ctx context.Context) {
} }
if setAdmin && !user.IsAdmin { if setAdmin && !user.IsAdmin {
libraries, err := tx.Library().GetAll(ctx) libraries, err := tx.Library(ctx).GetAll()
if err != nil { if err != nil {
return err return err
} }
@ -337,7 +337,7 @@ func runUserEdit(ctx context.Context) {
return nil return nil
} }
err := tx.User().Put(ctx, user) err := tx.User(ctx).Put(user)
if err != nil { if err != nil {
return err return err
} }
@ -348,7 +348,7 @@ func runUserEdit(ctx context.Context) {
updatedIds[idx] = lib.ID updatedIds[idx] = lib.ID
} }
err := tx.User().SetUserLibraries(ctx, user.ID, updatedIds) err := tx.User(ctx).SetUserLibraries(user.ID, updatedIds)
if err != nil { if err != nil {
return err return err
} }
@ -393,11 +393,13 @@ func runUserList(ctx context.Context) {
ds, ctx := getAdminContext(ctx) ds, ctx := getAdminContext(ctx)
userList, err := ds.User().ReadAll(ctx) users, err := ds.User(ctx).ReadAll()
if err != nil { if err != nil {
log.Fatal(ctx, "Failed to retrieve users", err) log.Fatal(ctx, "Failed to retrieve users", err)
} }
userList := users.(model.Users)
if outputFormat == "csv" { if outputFormat == "csv" {
w := csv.NewWriter(os.Stdout) w := csv.NewWriter(os.Stdout)
_ = w.Write([]string{ _ = w.Write([]string{

View file

@ -4,12 +4,7 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"io"
"os"
"strings"
"text/tabwriter"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/core/auth" "github.com/navidrome/navidrome/core/auth"
"github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
@ -18,33 +13,6 @@ import (
"github.com/navidrome/navidrome/persistence" "github.com/navidrome/navidrome/persistence"
) )
// existingDBFile returns the database file (DbPath minus DSN params), and whether it exists.
func existingDBFile() (string, bool) {
path, _, _ := strings.Cut(conf.Server.DbPath, "?")
_, err := os.Stat(path)
return path, err == nil
}
// requireExistingDB aborts the command when the database file does not exist.
func requireExistingDB() {
if path, ok := existingDBFile(); !ok {
log.Fatal("No existing database", "path", path)
}
}
func confirmYES(warning string) bool {
fmt.Println(warning)
fmt.Printf("Please enter YES (all caps) to continue: ")
var input string
_, err := fmt.Scanln(&input)
return input == "YES" && err == nil
}
// newTabWriter keeps every CLI table on the same column settings.
func newTabWriter(out io.Writer) *tabwriter.Writer {
return tabwriter.NewWriter(out, 0, 4, 2, ' ', 0)
}
func getAdminContext(ctx context.Context) (model.DataStore, context.Context) { func getAdminContext(ctx context.Context) (model.DataStore, context.Context) {
sqlDB := db.Db() sqlDB := db.Db()
ds := persistence.New(sqlDB) ds := persistence.New(sqlDB)
@ -57,14 +25,14 @@ func getAdminContext(ctx context.Context) (model.DataStore, context.Context) {
} }
func getUser(ctx context.Context, id string, ds model.DataStore) (*model.User, error) { func getUser(ctx context.Context, id string, ds model.DataStore) (*model.User, error) {
user, err := ds.User().FindByUsername(ctx, id) user, err := ds.User(ctx).FindByUsername(id)
if err != nil && !errors.Is(err, model.ErrNotFound) { if err != nil && !errors.Is(err, model.ErrNotFound) {
return nil, fmt.Errorf("finding user by name: %w", err) return nil, fmt.Errorf("finding user by name: %w", err)
} }
if errors.Is(err, model.ErrNotFound) { if errors.Is(err, model.ErrNotFound) {
user, err = ds.User().Get(ctx, id) user, err = ds.User(ctx).Get(id)
if err != nil { if err != nil {
return nil, fmt.Errorf("finding user by id: %w", err) return nil, fmt.Errorf("finding user by id: %w", err)
} }

View file

@ -21,7 +21,6 @@ import (
"github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/playlists" "github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/quickconnect"
"github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/core/sonic" "github.com/navidrome/navidrome/core/sonic"
"github.com/navidrome/navidrome/core/stream" "github.com/navidrome/navidrome/core/stream"
@ -31,7 +30,6 @@ import (
"github.com/navidrome/navidrome/plugins" "github.com/navidrome/navidrome/plugins"
"github.com/navidrome/navidrome/scanner" "github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server" "github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/server/apiv1"
"github.com/navidrome/navidrome/server/events" "github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/server/jellyfin" "github.com/navidrome/navidrome/server/jellyfin"
"github.com/navidrome/navidrome/server/nativeapi" "github.com/navidrome/navidrome/server/nativeapi"
@ -67,22 +65,18 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
share := core.NewShare(dataStore) share := core.NewShare(dataStore)
uploader := artwork.NewUploader(dataStore) imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader) playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
insights := metrics.GetInstance(dataStore) insights := metrics.GetInstance(dataStore)
broker := events.GetBroker() broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore) metricsMetrics := metrics.GetPrometheusInstance(dataStore)
modelScanner := scanner.GetInstance(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics) modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
watcher := scanner.GetWatcher(dataStore, modelScanner) watcher := scanner.GetWatcher(dataStore, modelScanner)
manager := plugins.GetManager(dataStore, broker, metricsMetrics) manager := plugins.GetManager(dataStore, broker, metricsMetrics)
library := core.NewLibrary(dataStore, modelScanner, watcher, broker, manager) library := core.NewLibrary(dataStore, modelScanner, watcher, broker, manager)
user := core.NewUser(dataStore, manager) user := core.NewUser(dataStore, manager)
maintenance := core.NewMaintenance(dataStore) maintenance := core.NewMaintenance(dataStore)
agentsAgents := agents.GetAgents(dataStore, manager) router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager, imageUploadService)
matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker)
quickConnect := quickconnect.GetInstance()
router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager, uploader, provider, quickConnect)
return router return router
} }
@ -90,29 +84,29 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache() fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore() imageStore := artwork.ProvideImageStore()
fFmpeg := ffmpeg.New() fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg) service := artwork.NewService(dataStore, fileCache, imageStore, fFmpeg)
transcodingCache := stream.GetTranscodingCache() transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
share := core.NewShare(dataStore) share := core.NewShare(dataStore)
archiver := core.NewArchiver(mediaStreamer, transcodeDecider, dataStore, share, artworkArtwork) archiver := core.NewArchiver(mediaStreamer, dataStore, share)
players := core.NewPlayers(dataStore) players := core.NewPlayers(dataStore)
broker := events.GetBroker() broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore) metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics) manager := plugins.GetManager(dataStore, broker, metricsMetrics)
agentsAgents := agents.GetAgents(dataStore, manager) agentsAgents := agents.GetAgents(dataStore, manager)
matcherMatcher := matcher.New(dataStore) matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker) provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher)
uploader := artwork.NewUploader(dataStore) imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader) playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
modelScanner := scanner.GetInstance(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics) modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager) playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
playbackServer := playback.GetInstance(dataStore) playbackServer := playback.GetInstance(dataStore)
lyricsLyrics := lyrics.NewLyrics(dataStore, manager) lyricsLyrics := lyrics.NewLyrics(dataStore, manager)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
sonicSonic := sonic.New(dataStore, manager, matcherMatcher) sonicSonic := sonic.New(dataStore, manager, matcherMatcher)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics, transcodeDecider, sonicSonic) router := subsonic.New(dataStore, service, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics, transcodeDecider, sonicSonic)
return router return router
} }
@ -120,9 +114,9 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache() fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore() imageStore := artwork.ProvideImageStore()
fFmpeg := ffmpeg.New() fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg) service := artwork.NewService(dataStore, fileCache, imageStore, fFmpeg)
transcodingCache := stream.GetTranscodingCache() transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg) transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
@ -131,22 +125,14 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
metricsMetrics := metrics.GetPrometheusInstance(dataStore) metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics) manager := plugins.GetManager(dataStore, broker, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager) playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
uploader := artwork.NewUploader(dataStore) imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader) playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
agentsAgents := agents.GetAgents(dataStore, manager) agentsAgents := agents.GetAgents(dataStore, manager)
matcherMatcher := matcher.New(dataStore) matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker) provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher)
sonicSonic := sonic.New(dataStore, manager, matcherMatcher) sonicSonic := sonic.New(dataStore, manager, matcherMatcher)
lyricsLyrics := lyrics.NewLyrics(dataStore, manager) lyricsLyrics := lyrics.NewLyrics(dataStore, manager)
quickConnect := quickconnect.GetInstance() router := jellyfin.New(dataStore, service, mediaStreamer, transcodeDecider, players, playTracker, playlistsPlaylists, provider, sonicSonic, lyricsLyrics, broker)
router := jellyfin.New(dataStore, artworkArtwork, mediaStreamer, transcodeDecider, players, playTracker, playlistsPlaylists, provider, sonicSonic, lyricsLyrics, broker, quickConnect)
return router
}
func CreateAPIv1Router(ctx context.Context) *apiv1.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
router := apiv1.New(dataStore)
return router return router
} }
@ -154,15 +140,14 @@ func CreatePublicRouter() *public.Router {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
fileCache := artwork.GetImageCache() fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore() imageStore := artwork.ProvideImageStore()
fFmpeg := ffmpeg.New() fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg) service := artwork.NewService(dataStore, fileCache, imageStore, fFmpeg)
transcodingCache := stream.GetTranscodingCache() transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache) mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
share := core.NewShare(dataStore) share := core.NewShare(dataStore)
archiver := core.NewArchiver(mediaStreamer, transcodeDecider, dataStore, share, artworkArtwork) archiver := core.NewArchiver(mediaStreamer, dataStore, share)
router := public.New(dataStore, artworkArtwork, mediaStreamer, transcodeDecider, share, archiver) router := public.New(dataStore, service, mediaStreamer, share, archiver)
return router return router
} }
@ -180,13 +165,6 @@ func CreateListenBrainzRouter() *listenbrainz.Router {
return router return router
} }
func CreateJellyfinDiscovery() *jellyfin.Discovery {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
discovery := jellyfin.NewDiscovery(dataStore)
return discovery
}
func CreateInsights() metrics.Insights { func CreateInsights() metrics.Insights {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
@ -205,10 +183,10 @@ func CreateScanner(ctx context.Context) model.Scanner {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
broker := events.GetBroker() broker := events.GetBroker()
uploader := artwork.NewUploader(dataStore) imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader) playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
metricsMetrics := metrics.GetPrometheusInstance(dataStore) metricsMetrics := metrics.GetPrometheusInstance(dataStore)
modelScanner := scanner.GetInstance(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics) modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
return modelScanner return modelScanner
} }
@ -216,10 +194,10 @@ func CreateScanWatcher(ctx context.Context) scanner.Watcher {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
broker := events.GetBroker() broker := events.GetBroker()
uploader := artwork.NewUploader(dataStore) imageUploadService := core.NewImageUploadService(dataStore)
playlistsPlaylists := playlists.NewPlaylists(dataStore, uploader) playlistsPlaylists := playlists.NewPlaylists(dataStore, imageUploadService)
metricsMetrics := metrics.GetPrometheusInstance(dataStore) metricsMetrics := metrics.GetPrometheusInstance(dataStore)
modelScanner := scanner.GetInstance(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics) modelScanner := scanner.New(ctx, dataStore, broker, playlistsPlaylists, metricsMetrics)
watcher := scanner.GetWatcher(dataStore, modelScanner) watcher := scanner.GetWatcher(dataStore, modelScanner)
return watcher return watcher
} }
@ -234,7 +212,7 @@ func GetPlaybackServer() playback.PlaybackServer {
func CreateArtworkWorker() *artwork.Worker { func CreateArtworkWorker() *artwork.Worker {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
imageStore := artwork.GetImageStore() imageStore := artwork.ProvideImageStore()
broker := events.GetBroker() broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore) metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics) manager := plugins.GetManager(dataStore, broker, metricsMetrics)
@ -245,18 +223,6 @@ func CreateArtworkWorker() *artwork.Worker {
return worker return worker
} }
func CreateArtworkResolver(trace *artwork.ChainTrace, live bool) *artwork.TracingResolver {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
agentsAgents := agents.GetAgents(dataStore, manager)
fFmpeg := ffmpeg.New()
tracingResolver := artwork.NewTracingResolver(dataStore, agentsAgents, fFmpeg, trace, live)
return tracingResolver
}
func getPluginManager() *plugins.Manager { func getPluginManager() *plugins.Manager {
sqlDB := db.Db() sqlDB := db.Db()
dataStore := persistence.New(sqlDB) dataStore := persistence.New(sqlDB)
@ -268,7 +234,7 @@ func getPluginManager() *plugins.Manager {
// wire_injectors.go: // wire_injectors.go:
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, jellyfin.New, jellyfin.NewDiscovery, apiv1.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.GetInstance, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, sonic.New, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.Engine), new(*sonic.Sonic)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher)), wire.Bind(new(playlists.ImageUploadService), new(artwork.Uploader))) var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, jellyfin.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.New, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, sonic.New, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.Engine), new(*sonic.Sonic)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher)))
func GetPluginManager(ctx context.Context) *plugins.Manager { func GetPluginManager(ctx context.Context) *plugins.Manager {
manager := getPluginManager() manager := getPluginManager()

View file

@ -14,7 +14,6 @@ import (
"github.com/navidrome/navidrome/core/lyrics" "github.com/navidrome/navidrome/core/lyrics"
"github.com/navidrome/navidrome/core/metrics" "github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback" "github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/scrobbler" "github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/core/sonic" "github.com/navidrome/navidrome/core/sonic"
"github.com/navidrome/navidrome/db" "github.com/navidrome/navidrome/db"
@ -23,7 +22,6 @@ import (
"github.com/navidrome/navidrome/plugins" "github.com/navidrome/navidrome/plugins"
"github.com/navidrome/navidrome/scanner" "github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server" "github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/server/apiv1"
"github.com/navidrome/navidrome/server/events" "github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/server/jellyfin" "github.com/navidrome/navidrome/server/jellyfin"
"github.com/navidrome/navidrome/server/nativeapi" "github.com/navidrome/navidrome/server/nativeapi"
@ -37,15 +35,13 @@ var allProviders = wire.NewSet(
server.New, server.New,
subsonic.New, subsonic.New,
jellyfin.New, jellyfin.New,
jellyfin.NewDiscovery,
apiv1.New,
nativeapi.New, nativeapi.New,
public.New, public.New,
persistence.New, persistence.New,
lastfm.NewRouter, lastfm.NewRouter,
listenbrainz.NewRouter, listenbrainz.NewRouter,
events.GetBroker, events.GetBroker,
scanner.GetInstance, scanner.New,
scanner.GetWatcher, scanner.GetWatcher,
metrics.GetPrometheusInstance, metrics.GetPrometheusInstance,
db.Db, db.Db,
@ -60,7 +56,6 @@ var allProviders = wire.NewSet(
wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)),
wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)),
wire.Bind(new(core.Watcher), new(scanner.Watcher)), wire.Bind(new(core.Watcher), new(scanner.Watcher)),
wire.Bind(new(playlists.ImageUploadService), new(artwork.Uploader)),
) )
func CreateDataStore() model.DataStore { func CreateDataStore() model.DataStore {
@ -93,12 +88,6 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
)) ))
} }
func CreateAPIv1Router(ctx context.Context) *apiv1.Router {
panic(wire.Build(
allProviders,
))
}
func CreatePublicRouter() *public.Router { func CreatePublicRouter() *public.Router {
panic(wire.Build( panic(wire.Build(
allProviders, allProviders,
@ -117,12 +106,6 @@ func CreateListenBrainzRouter() *listenbrainz.Router {
)) ))
} }
func CreateJellyfinDiscovery() *jellyfin.Discovery {
panic(wire.Build(
allProviders,
))
}
func CreateInsights() metrics.Insights { func CreateInsights() metrics.Insights {
panic(wire.Build( panic(wire.Build(
allProviders, allProviders,
@ -159,13 +142,6 @@ func CreateArtworkWorker() *artwork.Worker {
)) ))
} }
func CreateArtworkResolver(trace *artwork.ChainTrace, live bool) *artwork.TracingResolver {
panic(wire.Build(
allProviders,
artwork.NewTracingResolver,
))
}
func getPluginManager() *plugins.Manager { func getPluginManager() *plugins.Manager {
panic(wire.Build( panic(wire.Build(
allProviders, allProviders,

View file

@ -2,19 +2,14 @@ package conf
import ( import (
"cmp" "cmp"
"encoding"
"encoding/json" "encoding/json"
"fmt" "fmt"
"math"
"net/url" "net/url"
"os" "os"
"path/filepath" "path/filepath"
"reflect"
"regexp"
"runtime" "runtime"
"slices" "slices"
"strings" "strings"
"sync"
"time" "time"
"github.com/bmatcuk/doublestar/v4" "github.com/bmatcuk/doublestar/v4"
@ -26,12 +21,11 @@ import (
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/scheduler" "github.com/navidrome/navidrome/scheduler"
"github.com/navidrome/navidrome/utils/run" "github.com/navidrome/navidrome/utils/run"
"github.com/navidrome/navidrome/utils/slice"
"github.com/spf13/viper" "github.com/spf13/viper"
) )
type configOptions struct { type configOptions struct {
ConfigFile string `conf:"-"` ConfigFile string
Address string Address string
Port int Port int
UnixSocketPerm string UnixSocketPerm string
@ -63,6 +57,8 @@ type configOptions struct {
ImageCacheSize string ImageCacheSize string
AlbumPlayCountMode string AlbumPlayCountMode string
EnableArtworkPrecache bool EnableArtworkPrecache bool
ArtworkWorkerConcurrency int
ArtworkExternalMaxRPS int
AutoImportPlaylists bool AutoImportPlaylists bool
DefaultPlaylistPublicVisibility bool DefaultPlaylistPublicVisibility bool
PlaylistsPath string PlaylistsPath string
@ -73,7 +69,6 @@ type configOptions struct {
Matcher matcherOptions `json:",omitzero"` Matcher matcherOptions `json:",omitzero"`
RecentlyAddedByModTime bool RecentlyAddedByModTime bool
PreferSortTags bool PreferSortTags bool
EnableNaturalSorting bool
IgnoredArticles string IgnoredArticles string
IndexGroups string IndexGroups string
FFmpegPath string FFmpegPath string
@ -92,7 +87,6 @@ type configOptions struct {
EnableUserEditing bool EnableUserEditing bool
EnableArtworkUpload bool EnableArtworkUpload bool
MaxImageUploadSize string MaxImageUploadSize string
MaxImageSize string
EnableSharing bool EnableSharing bool
ShareURL string ShareURL string
DefaultShareExpiration time.Duration DefaultShareExpiration time.Duration
@ -147,8 +141,6 @@ type configOptions struct {
DevArtworkThrottleBacklogLimit int DevArtworkThrottleBacklogLimit int
DevArtworkThrottleBacklogTimeout time.Duration DevArtworkThrottleBacklogTimeout time.Duration
DevArtworkThrottleBuffered bool DevArtworkThrottleBuffered bool
DevArtworkWorkerConcurrency int
DevArtworkExternalMaxRPS int
DevArtistInfoTimeToLive time.Duration DevArtistInfoTimeToLive time.Duration
DevAlbumInfoTimeToLive time.Duration DevAlbumInfoTimeToLive time.Duration
DevExternalScanner bool DevExternalScanner bool
@ -161,7 +153,6 @@ type configOptions struct {
DevExternalArtistFetchMultiplier float64 DevExternalArtistFetchMultiplier float64
DevPreserveUnicodeInExternalCalls bool DevPreserveUnicodeInExternalCalls bool
DevEnableMediaFileProbe bool DevEnableMediaFileProbe bool
DevAPIv1 bool
} }
type scannerOptions struct { type scannerOptions struct {
@ -212,7 +203,7 @@ type lastfmOptions struct {
ScrobbleFirstArtistOnly bool ScrobbleFirstArtistOnly bool
// Computed values // Computed values
Languages []string `conf:"-"` // Computed from Language, split by comma Languages []string // Computed from Language, split by comma
} }
type deezerOptions struct { type deezerOptions struct {
@ -220,7 +211,7 @@ type deezerOptions struct {
Language string Language string
// Computed values // Computed values
Languages []string `conf:"-"` // Computed from Language, split by comma Languages []string // Computed from Language, split by comma
} }
type listenBrainzOptions struct { type listenBrainzOptions struct {
@ -236,8 +227,6 @@ type jellyfinOptions struct {
// ExposedPublicUsers is a comma-separated list of usernames to advertise on the unauthenticated // ExposedPublicUsers is a comma-separated list of usernames to advertise on the unauthenticated
// GET /Users/Public, so Jellyfin clients can show a login user-picker. Empty exposes no users. // GET /Users/Public, so Jellyfin clients can show a login user-picker. Empty exposes no users.
ExposedPublicUsers string ExposedPublicUsers string
AutoDiscovery bool
QuickConnect bool
// MaxConcurrentStreams bounds how many collection responses can stream at once. Each holds a DB // MaxConcurrentStreams bounds how many collection responses can stream at once. Each holds a DB
// cursor — and its pooled connection — for the whole client-paced response, so without a bound // cursor — and its pooled connection — for the whole client-paced response, so without a bound
// enough slow clients would take the entire pool and stall the scanner, scrobbles and the UI. // enough slow clients would take the entire pool and stall the scanner, scrobbles and the UI.
@ -318,12 +307,6 @@ var currentGOOS = func() string {
return runtime.GOOS return runtime.GOOS
} }
// TLSEnabled reports whether the server serves HTTPS. Both halves are required,
// so callers cannot infer it from the certificate alone.
func (c *configOptions) TLSEnabled() bool {
return c.TLSCert != "" && c.TLSKey != ""
}
var ( var (
Server = &configOptions{} Server = &configOptions{}
hooks []func() hooks []func()
@ -354,23 +337,19 @@ func LoadFromFile(confFile string) {
Load(true) Load(true)
} }
func durationNonNegativeOrDefault(val *time.Duration, original time.Duration) {
if val.Nanoseconds() < 0 {
log.Warn("Duration is a negative value. Using default value", "value", *val, "default", original)
*val = original
}
}
func Load(noConfigDump bool) { func Load(noConfigDump bool) {
parseIniFileConfiguration() parseIniFileConfiguration()
remapEnvVarKeysFromConfig() remapEnvVarKeysFromConfig()
// Map deprecated options to their new names for backwards compatibility // Map deprecated options to their new names for backwards compatibility
for _, o := range deprecatedOptions { mapDeprecatedOption("ReverseProxyWhitelist", "ExtAuth.TrustedSources")
if o.replacement != "" { mapDeprecatedOption("ReverseProxyUserHeader", "ExtAuth.UserHeader")
mapDeprecatedOption(o.name, o.replacement) mapDeprecatedOption("HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions")
} mapDeprecatedOption("CoverJpegQuality", "CoverArtQuality")
} mapDeprecatedOption("SimilarSongsMatchThreshold", "Matcher.FuzzyThreshold")
mapDeprecatedOption("EnableTranscodingCancellation", "Transcoding.EnableCancellation")
mapDeprecatedOption("DevArtworkWorkerConcurrency", "ArtworkWorkerConcurrency")
mapDeprecatedOption("DevArtworkExternalRPS", "ArtworkExternalMaxRPS")
err := viper.Unmarshal(&Server, viper.DecodeHook( err := viper.Unmarshal(&Server, viper.DecodeHook(
mapstructure.ComposeDecodeHookFunc( mapstructure.ComposeDecodeHookFunc(
@ -410,7 +389,7 @@ func Load(noConfigDump bool) {
if mkErr := os.MkdirAll(filepath.Dir(Server.LogFile), os.ModePerm); mkErr != nil { if mkErr := os.MkdirAll(filepath.Dir(Server.LogFile), os.ModePerm); mkErr != nil {
logFatal(fmt.Sprintf("Error creating log file directory: %s", mkErr.Error())) logFatal(fmt.Sprintf("Error creating log file directory: %s", mkErr.Error()))
} }
out, err = os.OpenFile(Server.LogFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600) out, err = os.OpenFile(Server.LogFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil { if err != nil {
logFatal(fmt.Sprintf("Error opening log file %s: %s", Server.LogFile, err.Error())) logFatal(fmt.Sprintf("Error opening log file %s: %s", Server.LogFile, err.Error()))
} }
@ -428,34 +407,12 @@ func Load(noConfigDump bool) {
log.SetLogSourceLine(Server.DevLogSourceLine) log.SetLogSourceLine(Server.DevLogSourceLine)
log.SetRedacting(Server.EnableLogRedacting) log.SetRedacting(Server.EnableLogRedacting)
durationNonNegativeOrDefault(&Server.SessionTimeout, consts.DefaultSessionTimeout)
durationNonNegativeOrDefault(&Server.SmartPlaylistRefreshDelay, consts.DefaultSmartRefresh)
durationNonNegativeOrDefault(&Server.DefaultShareExpiration, consts.DefaultShareExpiration)
durationNonNegativeOrDefault(&Server.UIPlaybackReportInterval, consts.DefaultUIPlaybackReportInterval)
durationNonNegativeOrDefault(&Server.AuthWindowLength, consts.DefaultAuthWindowLength)
durationNonNegativeOrDefault(&Server.Scanner.WatcherWait, consts.DefaultWatcherWait)
durationNonNegativeOrDefault(&Server.DevActivityPanelUpdateRate, consts.DefaultActivityPanelUpdateRate)
durationNonNegativeOrDefault(&Server.DevArtworkThrottleBacklogTimeout, consts.RequestThrottleBacklogTimeout)
durationNonNegativeOrDefault(&Server.DevArtistInfoTimeToLive, consts.ArtistInfoTimeToLive)
durationNonNegativeOrDefault(&Server.DevAlbumInfoTimeToLive, consts.AlbumInfoTimeToLive)
durationNonNegativeOrDefault(&Server.DevInsightsInitialDelay, consts.InsightsInitialDelay)
durationNonNegativeOrDefault(&Server.DevPluginCompilationTimeout, consts.DefaultPluginCompilationTimeout)
// Log deprecated, removed and unknown options
for _, o := range deprecatedOptions {
logDeprecatedOptions(o.name, o.replacement)
}
logRemovedOptions(removedOptions...)
logUnknownOptions()
err = run.Sequentially( err = run.Sequentially(
validateScanSchedule, validateScanSchedule,
validateBackupSchedule, validateBackupSchedule,
validatePlaylistsPath, validatePlaylistsPath,
validatePurgeMissingOption, validatePurgeMissingOption,
validateByteSize("MaxImageUploadSize", Server.MaxImageUploadSize), validateMaxImageUploadSize,
validateByteSize("MaxImageSize", Server.MaxImageSize),
validateURL("ExtAuth.LogoutURL", Server.ExtAuth.LogoutURL), validateURL("ExtAuth.LogoutURL", Server.ExtAuth.LogoutURL),
) )
if err != nil { if err != nil {
@ -508,6 +465,21 @@ func Load(noConfigDump bool) {
// Parse Deezer.Language into Languages slice (comma-separated, with fallback to DefaultInfoLanguage) // Parse Deezer.Language into Languages slice (comma-separated, with fallback to DefaultInfoLanguage)
Server.Deezer.Languages = parseLanguages(Server.Deezer.Language) Server.Deezer.Languages = parseLanguages(Server.Deezer.Language)
// Deprecated options
logDeprecatedOptions("Scanner.GenreSeparators", "")
logDeprecatedOptions("Scanner.GroupAlbumReleases", "")
logDeprecatedOptions("DevEnableBufferedScrobble", "") // Deprecated: Buffered scrobbling is now always enabled and this option is ignored
logDeprecatedOptions("SearchFullString", "Search.FullString")
logDeprecatedOptions("ReverseProxyWhitelist", "ExtAuth.TrustedSources")
logDeprecatedOptions("ReverseProxyUserHeader", "ExtAuth.UserHeader")
logDeprecatedOptions("HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions")
logDeprecatedOptions("CoverJpegQuality", "CoverArtQuality")
logDeprecatedOptions("SimilarSongsMatchThreshold", "Matcher.FuzzyThreshold")
logDeprecatedOptions("EnableTranscodingCancellation", "Transcoding.EnableCancellation")
// Removed options
logRemovedOptions("Spotify.ID", "Spotify.Secret")
// Validate other options // Validate other options
if Server.UICoverArtSize < 200 || Server.UICoverArtSize > 1200 { if Server.UICoverArtSize < 200 || Server.UICoverArtSize > 1200 {
newValue := max(200, min(1200, Server.UICoverArtSize)) newValue := max(200, min(1200, Server.UICoverArtSize))
@ -515,45 +487,15 @@ func Load(noConfigDump bool) {
Server.UICoverArtSize = newValue Server.UICoverArtSize = newValue
} }
if Server.Scanner.Extractor != consts.DefaultScannerExtractor {
log.Warn("Invalid Scanner.Extractor, using default", "value", Server.Scanner.Extractor, "default", consts.DefaultScannerExtractor)
Server.Scanner.Extractor = consts.DefaultScannerExtractor
}
// Floor MaxImageSize at MaxImageUploadSize so accepted uploads can always be read back.
imgSize, _ := humanize.ParseBytes(Server.MaxImageSize)
uploadSize, _ := humanize.ParseBytes(Server.MaxImageUploadSize)
if imgSize < uploadSize {
log.Warn("MaxImageSize must be at least MaxImageUploadSize, raising", "value", Server.MaxImageSize, "newValue", Server.MaxImageUploadSize)
Server.MaxImageSize = Server.MaxImageUploadSize
}
// Call init hooks // Call init hooks
for _, hook := range hooks { for _, hook := range hooks {
hook() hook()
} }
} }
// deprecatedOptions still work, but will be removed in a future release. An empty
// replacement means the option is now ignored.
var deprecatedOptions = []struct{ name, replacement string }{
{"Scanner.GenreSeparators", ""},
{"Scanner.GroupAlbumReleases", ""},
{"DevEnableBufferedScrobble", ""},
{"SearchFullString", "Search.FullString"},
{"ReverseProxyWhitelist", "ExtAuth.TrustedSources"},
{"ReverseProxyUserHeader", "ExtAuth.UserHeader"},
{"HTTPSecurityHeaders.CustomFrameOptionsValue", "HTTPHeaders.FrameOptions"},
{"CoverJpegQuality", "CoverArtQuality"},
{"SimilarSongsMatchThreshold", "Matcher.FuzzyThreshold"},
{"EnableTranscodingCancellation", "Transcoding.EnableCancellation"},
}
var removedOptions = []string{"Spotify.ID", "Spotify.Secret"}
func logDeprecatedOptions(oldName, newName string) { func logDeprecatedOptions(oldName, newName string) {
envVar := envVarName(oldName) envVar := "ND_" + strings.ToUpper(strings.ReplaceAll(oldName, ".", "_"))
newEnvVar := envVarName(newName) newEnvVar := "ND_" + strings.ToUpper(strings.ReplaceAll(newName, ".", "_"))
logWarning := func(oldName, newName string) { logWarning := func(oldName, newName string) {
if newName != "" { if newName != "" {
log.Warn(fmt.Sprintf("Option '%s' is deprecated and will be ignored in a future release. Please use the new '%s'", oldName, newName)) log.Warn(fmt.Sprintf("Option '%s' is deprecated and will be ignored in a future release. Please use the new '%s'", oldName, newName))
@ -573,7 +515,7 @@ func logDeprecatedOptions(oldName, newName string) {
// not available anymore // not available anymore
func logRemovedOptions(options ...string) { func logRemovedOptions(options ...string) {
for _, option := range options { for _, option := range options {
envVar := envVarName(option) envVar := "ND_" + strings.ToUpper(strings.ReplaceAll(option, ".", "_"))
logWarning := func(option string) { logWarning := func(option string) {
log.Warn(fmt.Sprintf("Option '%s' is not available anymore and will be ignored. Please remove it from your config", option)) log.Warn(fmt.Sprintf("Option '%s' is not available anymore and will be ignored. Please remove it from your config", option))
} }
@ -594,193 +536,35 @@ func remapEnvVarKeysFromConfig() {
continue continue
} }
stripped := strings.TrimPrefix(key, "nd_") stripped := strings.TrimPrefix(key, "nd_")
canonicalKey := ndKeyToCanonical(key) canonicalKey := strings.ReplaceAll(stripped, "_", ".")
displayNDKey := "ND_" + strings.ToUpper(stripped) displayNDKey := "ND_" + strings.ToUpper(stripped)
canonicalName := canonicalOptionName(canonicalKey) displayCanonical := toPascalCase(canonicalKey)
if viper.InConfig(canonicalKey) { if viper.InConfig(canonicalKey) {
logFatal(fmt.Sprintf( logFatal(fmt.Sprintf(
"Config file contains both '%s' and '%s'. Remove the ND_-prefixed version. "+ "Config file contains both '%s' and '%s'. Remove the ND_-prefixed version. "+
"The 'ND_' prefix is only needed for environment variables, not config file keys.", "The 'ND_' prefix is only needed for environment variables, not config file keys.",
displayNDKey, cmp.Or(canonicalName, toPascalCase(canonicalKey)), displayNDKey, displayCanonical,
)) ))
return return
} }
viper.Set(canonicalKey, viper.Get(key)) viper.Set(canonicalKey, viper.Get(key))
// Unknown keys get no advice here, logUnknownOptions reports them instead _, _ = fmt.Fprintf(os.Stderr, "WARNING: Config key '%s' uses environment variable naming. Use '%s' instead. "+
if canonicalName != "" { "The 'ND_' prefix is only needed for environment variables.\n",
_, _ = fmt.Fprintf(os.Stderr, "WARNING: Config key '%s' uses environment variable naming. Use '%s' instead. "+ displayNDKey, displayCanonical,
"The 'ND_' prefix is only needed for environment variables.\n", )
displayNDKey, canonicalName,
)
}
} }
} }
// mapDeprecatedOption is used to provide backwards compatibility for deprecated options. It should be called after // mapDeprecatedOption is used to provide backwards compatibility for deprecated options. It should be called after
// the config has been read by viper, but before unmarshalling it into the Config struct. // the config has been read by viper, but before unmarshalling it into the Config struct.
func mapDeprecatedOption(legacyName, newName string) { func mapDeprecatedOption(legacyName, newName string) {
// viper.Set outranks the config file, so an explicit replacement must win over the legacy value if viper.IsSet(legacyName) {
if viper.IsSet(legacyName) && !explicitlySet(newName) {
viper.Set(newName, viper.Get(legacyName)) viper.Set(newName, viper.Get(legacyName))
} }
} }
// explicitlySet reports whether the user provided the option, ignoring defaults,
// which viper.IsSet counts as set. The ND_ spelling is also accepted in the config
// file, and remapEnvVarKeysFromConfig has already moved it out of InConfig's reach.
func explicitlySet(name string) bool {
envVar := envVarName(name)
return viper.InConfig(name) || os.Getenv(envVar) != "" || viper.InConfig(strings.ToLower(envVar))
}
func envVarName(option string) string {
if option == "" {
return ""
}
return "ND_" + strings.ToUpper(strings.ReplaceAll(option, ".", "_"))
}
func logUnknownOptions() {
for _, key := range unknownConfigKeys() {
msg := fmt.Sprintf("Option '%s' is not recognized and will be ignored", key)
if matches := suggestOptions(key); len(matches) > 0 {
msg += fmt.Sprintf(". Did you mean '%s'?", strings.Join(matches, "' or '"))
}
log.Warn(msg)
}
}
// suggestOptions returns the known options sharing the last segment with key,
// catching options written outside their section.
func suggestOptions(key string) []string {
key = strings.ToLower(key)
leaf := leafKey(key)
canonical, _ := configKeys()
var matches []string
for known, name := range canonical {
// Removed options are known only so they get their own warning, never suggest them
if known != key && leafKey(known) == leaf && !slices.Contains(removedOptions, name) {
matches = append(matches, name)
}
}
slices.Sort(matches)
return matches
}
func leafKey(key string) string {
return key[strings.LastIndex(key, ".")+1:]
}
// unknownConfigKeys returns config file keys that don't match any known option, so
// typos and options written outside their section don't fail silently.
func unknownConfigKeys() []string {
// INI files keep the original [default] section alongside the merged one
skipDefault := strings.EqualFold(filepath.Ext(viper.ConfigFileUsed()), ".ini")
var unknown []string
for _, key := range viper.AllKeys() {
if !viper.InConfig(key) || canonicalOptionName(key) != "" {
continue
}
if skipDefault && strings.HasPrefix(key, "default.") {
continue
}
// Only ND_-prefixed keys that remapEnvVarKeysFromConfig could resolve are valid
if strings.HasPrefix(key, "nd_") && canonicalOptionName(ndKeyToCanonical(key)) != "" {
continue
}
unknown = append(unknown, key)
}
slices.Sort(unknown)
return asWrittenInConfigFile(unknown)
}
func ndKeyToCanonical(key string) string {
return strings.ReplaceAll(strings.TrimPrefix(key, "nd_"), "_", ".")
}
// canonicalOptionName returns the documented spelling of a known option key, or ""
// if it matches no option. Subkeys of free-form maps have no fixed spelling.
func canonicalOptionName(key string) string {
keys, prefixes := configKeys()
if name, ok := keys[key]; ok {
return name
}
if slices.ContainsFunc(prefixes, func(p string) bool { return strings.HasPrefix(key, p) }) {
return toPascalCase(key)
}
return ""
}
// asWrittenInConfigFile restores the casing the keys have in the config file, as
// viper lowercases every key it loads.
func asWrittenInConfigFile(keys []string) []string {
if len(keys) == 0 {
return nil
}
data, err := os.ReadFile(viper.ConfigFileUsed())
if err != nil {
return keys
}
casing := map[string]string{}
for _, match := range configFileKeyRx.FindAllStringSubmatch(string(data), -1) {
for segment := range strings.SplitSeq(match[1], ".") {
lower := strings.ToLower(segment)
casing[lower] = cmp.Or(casing[lower], segment)
}
}
return slice.Map(keys, func(key string) string {
segments := strings.Split(key, ".")
for i, s := range segments {
segments[i] = cmp.Or(casing[s], s)
}
return strings.Join(segments, ".")
})
}
// Matches keys and section headers in all supported config formats.
var configFileKeyRx = regexp.MustCompile(`(?m)^\s*\[?\s*"?([\w.]+)"?\s*[]=:]`)
// configKeys maps every accepted option name, lowercased, to its canonical spelling,
// plus the prefixes of free-form map options (Tags, DevLogLevels).
var configKeys = sync.OnceValues(func() (map[string]string, []string) {
keys := map[string]string{}
var prefixes []string
var collect func(t reflect.Type, prefix string)
collect = func(t reflect.Type, prefix string) {
for field := range t.Fields() {
// `conf:"-"` marks values computed during Load, not settable in the config
if !field.IsExported() || field.Tag.Get("conf") == "-" {
continue
}
name := prefix + field.Name
if field.Type.Kind() == reflect.Struct && !reflect.PointerTo(field.Type).Implements(textUnmarshalerType) {
collect(field.Type, name+".")
continue
}
lower := strings.ToLower(name)
keys[lower] = name
if field.Type.Kind() == reflect.Map {
prefixes = append(prefixes, lower+".")
}
}
}
collect(reflect.TypeFor[configOptions](), "")
for _, o := range deprecatedOptions {
keys[strings.ToLower(o.name)] = o.name
}
for _, o := range removedOptions {
keys[strings.ToLower(o)] = o
}
return keys, prefixes
})
var textUnmarshalerType = reflect.TypeFor[encoding.TextUnmarshaler]()
// parseIniFileConfiguration is used to parse the config file when it is in INI format. For INI files, it // parseIniFileConfiguration is used to parse the config file when it is in INI format. For INI files, it
// would require a nested structure, so instead we unmarshal it to a map and then merge the nested [default] // would require a nested structure, so instead we unmarshal it to a map and then merge the nested [default]
// section into the root level. // section into the root level.
@ -853,20 +637,11 @@ func validatePurgeMissingOption() error {
return nil return nil
} }
func validateByteSize(name, value string) func() error { func validateMaxImageUploadSize() error {
return func() error { if _, err := humanize.ParseBytes(Server.MaxImageUploadSize); err != nil {
size, err := humanize.ParseBytes(value) return fmt.Errorf("invalid MaxImageUploadSize %q: use values like '10MB', '1GB', or raw bytes like '10485760': %w", Server.MaxImageUploadSize, err)
if err != nil {
return fmt.Errorf("invalid %s %q: use values like '10MB', '1GB', or raw bytes like '10485760': %w", name, value, err)
}
if size == 0 {
return fmt.Errorf("invalid %s %q: must be greater than zero", name, value)
}
if size > math.MaxInt64 {
return fmt.Errorf("invalid %s %q: value is too large", name, value)
}
return nil
} }
return nil
} }
func validateEnforceNonRootUser() error { func validateEnforceNonRootUser() error {
@ -996,7 +771,7 @@ func setViperDefaults() {
viper.SetDefault("autoimportplaylists", true) viper.SetDefault("autoimportplaylists", true)
viper.SetDefault("defaultplaylistpublicvisibility", false) viper.SetDefault("defaultplaylistpublicvisibility", false)
viper.SetDefault("playlistspath", "") viper.SetDefault("playlistspath", "")
viper.SetDefault("smartPlaylistRefreshDelay", consts.DefaultSmartRefresh) viper.SetDefault("smartPlaylistRefreshDelay", 5*time.Second)
viper.SetDefault("enabledownloads", true) viper.SetDefault("enabledownloads", true)
viper.SetDefault("enableexternalservices", true) viper.SetDefault("enableexternalservices", true)
viper.SetDefault("enablem3uexternalalbumart", false) viper.SetDefault("enablem3uexternalalbumart", false)
@ -1009,7 +784,6 @@ func setViperDefaults() {
viper.SetDefault("matcher.fuzzythreshold", 85) viper.SetDefault("matcher.fuzzythreshold", 85)
viper.SetDefault("recentlyaddedbymodtime", false) viper.SetDefault("recentlyaddedbymodtime", false)
viper.SetDefault("prefersorttags", false) viper.SetDefault("prefersorttags", false)
viper.SetDefault("enablenaturalsorting", false)
viper.SetDefault("ignoredarticles", "The El La Los Las Le Les Os As O A") viper.SetDefault("ignoredarticles", "The El La Los Las Le Les Os As O A")
viper.SetDefault("indexgroups", "A B C D E F G H I J K L M N O P Q R S T U V W X-Z(XYZ) [Unknown]([)") viper.SetDefault("indexgroups", "A B C D E F G H I J K L M N O P Q R S T U V W X-Z(XYZ) [Unknown]([)")
viper.SetDefault("ffmpegpath", "") viper.SetDefault("ffmpegpath", "")
@ -1037,17 +811,16 @@ func setViperDefaults() {
viper.SetDefault("uiplaybackreportinterval", consts.DefaultUIPlaybackReportInterval) viper.SetDefault("uiplaybackreportinterval", consts.DefaultUIPlaybackReportInterval)
viper.SetDefault("enableartworkupload", true) viper.SetDefault("enableartworkupload", true)
viper.SetDefault("maximageuploadsize", consts.DefaultMaxImageUploadSize) viper.SetDefault("maximageuploadsize", consts.DefaultMaxImageUploadSize)
viper.SetDefault("maximagesize", consts.DefaultMaxImageSize)
viper.SetDefault("enablesharing", true) viper.SetDefault("enablesharing", true)
viper.SetDefault("shareurl", "") viper.SetDefault("shareurl", "")
viper.SetDefault("defaultshareexpiration", consts.DefaultShareExpiration) viper.SetDefault("defaultshareexpiration", 8760*time.Hour)
viper.SetDefault("defaultdownloadableshare", false) viper.SetDefault("defaultdownloadableshare", false)
viper.SetDefault("gatrackingid", "") viper.SetDefault("gatrackingid", "")
viper.SetDefault("enableinsightscollector", true) viper.SetDefault("enableinsightscollector", true)
viper.SetDefault("enablescheduleddbanalyze", true) viper.SetDefault("enablescheduleddbanalyze", true)
viper.SetDefault("enablelogredacting", true) viper.SetDefault("enablelogredacting", true)
viper.SetDefault("authrequestlimit", 5) viper.SetDefault("authrequestlimit", 5)
viper.SetDefault("authwindowlength", consts.DefaultAuthWindowLength) viper.SetDefault("authwindowlength", 20*time.Second)
viper.SetDefault("passwordencryptionkey", "") viper.SetDefault("passwordencryptionkey", "")
viper.SetDefault("extauth.userheader", "Remote-User") viper.SetDefault("extauth.userheader", "Remote-User")
viper.SetDefault("extauth.trustedsources", "") viper.SetDefault("extauth.trustedsources", "")
@ -1095,8 +868,6 @@ func setViperDefaults() {
viper.SetDefault("listenbrainz.trackalgorithm", consts.DefaultListenBrainzTrackAlgorithm) viper.SetDefault("listenbrainz.trackalgorithm", consts.DefaultListenBrainzTrackAlgorithm)
viper.SetDefault("jellyfin.enabled", false) viper.SetDefault("jellyfin.enabled", false)
viper.SetDefault("jellyfin.servername", "") viper.SetDefault("jellyfin.servername", "")
viper.SetDefault("jellyfin.autodiscovery", false)
viper.SetDefault("jellyfin.quickconnect", true)
viper.SetDefault("enablescrobblehistory", true) viper.SetDefault("enablescrobblehistory", true)
viper.SetDefault("httpheaders.frameoptions", "DENY") viper.SetDefault("httpheaders.frameoptions", "DENY")
viper.SetDefault("backup.path", "") viper.SetDefault("backup.path", "")
@ -1125,7 +896,6 @@ func setViperDefaults() {
viper.SetDefault("devshowartistpage", true) viper.SetDefault("devshowartistpage", true)
viper.SetDefault("devuishowconfig", true) viper.SetDefault("devuishowconfig", true)
viper.SetDefault("devneweventstream", true) viper.SetDefault("devneweventstream", true)
viper.SetDefault("devapiv1", false)
viper.SetDefault("devoffsetoptimize", 50000) viper.SetDefault("devoffsetoptimize", 50000)
// Half the pool: streams may take up to this many connections, leaving the rest for the scanner, // Half the pool: streams may take up to this many connections, leaving the rest for the scanner,
// scrobbles and the UI. See MaxOpenConns. // scrobbles and the UI. See MaxOpenConns.
@ -1134,12 +904,8 @@ func setViperDefaults() {
viper.SetDefault("devartworkthrottlebackloglimit", consts.RequestThrottleBacklogLimit) viper.SetDefault("devartworkthrottlebackloglimit", consts.RequestThrottleBacklogLimit)
viper.SetDefault("devartworkthrottlebacklogtimeout", consts.RequestThrottleBacklogTimeout) viper.SetDefault("devartworkthrottlebacklogtimeout", consts.RequestThrottleBacklogTimeout)
viper.SetDefault("devartworkthrottlebuffered", true) viper.SetDefault("devartworkthrottlebuffered", true)
// Half the CPU count (min 2), so local resolution scales with the host but stays under the viper.SetDefault("artworkworkerconcurrency", 4)
// SQLite pool (MaxOpenConns) — leaving connections for the scanner, scrobbles and the UI. viper.SetDefault("artworkexternalmaxrps", 2)
viper.SetDefault("devartworkworkerconcurrency", max(2, runtime.NumCPU()/2))
// External RPS gates outbound calls to third-party services (per service); it is bounded by
// their tolerance, not the host, so it stays a small constant regardless of CPU count.
viper.SetDefault("devartworkexternalmaxrps", 2)
viper.SetDefault("devartistinfotimetolive", consts.ArtistInfoTimeToLive) viper.SetDefault("devartistinfotimetolive", consts.ArtistInfoTimeToLive)
viper.SetDefault("devalbuminfotimetolive", consts.AlbumInfoTimeToLive) viper.SetDefault("devalbuminfotimetolive", consts.AlbumInfoTimeToLive)
viper.SetDefault("devexternalscanner", true) viper.SetDefault("devexternalscanner", true)

View file

@ -1,18 +1,12 @@
package conf_test package conf_test
import ( import (
"bytes"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"runtime"
"testing" "testing"
"time"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/log"
. "github.com/onsi/ginkgo/v2" . "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
"github.com/spf13/viper" "github.com/spf13/viper"
@ -184,123 +178,6 @@ var _ = Describe("Configuration", func() {
}) })
}) })
Describe("unknownConfigKeys", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
It("reports misplaced and misspelled options, as spelled in the config file", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_unknown_keys.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(ConsistOf(
"ArtistSplitExceptions", "EnableDownlods", "Whatever.Foo",
))
})
DescribeTable("recovers the original casing in all supported formats",
func(file string) {
conf.InitConfig(filepath.Join("testdata", file), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(ConsistOf("NotAnOption"))
},
Entry("TOML", "cfg_unknown_casing.toml"),
Entry("YAML", "cfg_unknown_casing.yaml"),
Entry("JSON", "cfg_unknown_casing.json"),
Entry("INI", "cfg_unknown_casing.ini"),
)
It("does not report valid, deprecated or free-form keys", func() {
conf.InitConfig(filepath.Join("testdata", "cfg.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
It("does not report the [default] section of INI files", func() {
conf.InitConfig(filepath.Join("testdata", "cfg.ini"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
DescribeTable("SuggestOptions",
func(key string, expected []string) {
Expect(conf.SuggestOptions(key)).To(Equal(expected))
},
Entry("suggests the section of a misplaced option", "artistsplitexceptions",
[]string{"Scanner.ArtistSplitExceptions"}),
Entry("suggests the section of a misplaced nested option", "backup.fuzzythreshold",
[]string{"Matcher.FuzzyThreshold"}),
Entry("suggests every section defining the option", "schedule",
[]string{"Backup.Schedule", "Scanner.Schedule"}),
Entry("suggests nothing for a typo", "enabledownlods", nil),
)
It("does not report ND_-prefixed keys, as they are remapped", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_nd_keys.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
It("reports ND_-prefixed keys that remap to no known option", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_nd_bogus.toml"), false)
conf.Load(true)
Expect(conf.UnknownConfigKeys()).To(ConsistOf("ND_TOTALLY_BOGUS_OPTION"))
Expect(conf.Server.Scanner.Schedule).To(Equal("@every 1h"))
})
It("migrates every deprecated option that has a replacement", func() {
conf.InitConfig(filepath.Join("testdata", "cfg_deprecated_search.toml"), false)
conf.Load(true)
Expect(conf.Server.Search.FullString).To(BeTrue())
Expect(conf.UnknownConfigKeys()).To(BeEmpty())
})
It("warns about each unrecognized option at startup", func() {
var logBuf bytes.Buffer
log.SetOutput(&logBuf)
DeferCleanup(func() { log.SetOutput(GinkgoWriter) })
conf.InitConfig(filepath.Join("testdata", "cfg_warning_output.toml"), false)
conf.Load(true)
Expect(logBuf.String()).To(ContainSubstring(
"Option 'ArtistSplitExceptions' is not recognized and will be ignored. " +
"Did you mean 'Scanner.ArtistSplitExceptions'?"))
Expect(logBuf.String()).To(ContainSubstring(
"Option 'EnableDownlods' is not recognized and will be ignored"))
Expect(logBuf.String()).ToNot(ContainSubstring("ArtistJoiner"))
})
Context("with runtime-computed and removed options in the config", func() {
BeforeEach(func() {
conf.InitConfig(filepath.Join("testdata", "cfg_runtime_fields.toml"), false)
conf.Load(true)
})
It("reports values computed during Load, which the config cannot set", func() {
Expect(conf.UnknownConfigKeys()).To(ContainElements("ConfigFile", "LastFM.Languages"))
})
It("never suggests a removed option", func() {
Expect(conf.SuggestOptions("id")).To(BeEmpty())
})
It("keeps an explicit replacement over the deprecated value", func() {
Expect(conf.Server.Search.FullString).To(BeFalse())
})
})
})
Describe("logFatal", func() { Describe("logFatal", func() {
var invalidPath string var invalidPath string
BeforeEach(func() { BeforeEach(func() {
@ -330,21 +207,6 @@ var _ = Describe("Configuration", func() {
}).To(PanicWith(ContainSubstring("Error creating log file directory"))) }).To(PanicWith(ContainSubstring("Error creating log file directory")))
}) })
It("creates the log file readable only by the owner", func() {
if runtime.GOOS == "windows" {
Skip("file modes are not enforced on Windows")
}
logFile := filepath.Join(GinkgoT().TempDir(), "navidrome.log")
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("logfile", logFile)
DeferCleanup(log.SetOutput, os.Stderr)
conf.Load(true)
info, err := os.Stat(logFile)
Expect(err).ToNot(HaveOccurred())
Expect(info.Mode().Perm()).To(Equal(os.FileMode(0600)))
})
It("is called when BaseURL is invalid", func() { It("is called when BaseURL is invalid", func() {
viper.SetDefault("datafolder", GinkgoT().TempDir()) viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("baseurl", "://invalid") viper.SetDefault("baseurl", "://invalid")
@ -355,10 +217,19 @@ var _ = Describe("Configuration", func() {
}) })
Describe("ValidateByteSize", func() { Describe("ValidateMaxImageUploadSize", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
DescribeTable("accepts valid size values", DescribeTable("accepts valid size values",
func(input string) { func(input string) {
Expect(conf.ValidateByteSize("MaxImageSize", input)()).To(Succeed()) conf.Server.MaxImageUploadSize = input
Expect(conf.ValidateMaxImageUploadSize()).To(Succeed())
}, },
Entry("megabytes", "10MB"), Entry("megabytes", "10MB"),
Entry("gigabytes", "1GB"), Entry("gigabytes", "1GB"),
@ -369,60 +240,14 @@ var _ = Describe("Configuration", func() {
DescribeTable("rejects invalid size values", DescribeTable("rejects invalid size values",
func(input string) { func(input string) {
Expect(conf.ValidateByteSize("MaxImageSize", input)()).To(MatchError(ContainSubstring("invalid MaxImageSize"))) conf.Server.MaxImageUploadSize = input
Expect(conf.ValidateMaxImageUploadSize()).To(MatchError(ContainSubstring("invalid MaxImageUploadSize")))
}, },
Entry("garbage string", "not-a-size"), Entry("garbage string", "not-a-size"),
Entry("negative-looking", "-10MB"), Entry("negative-looking", "-10MB"),
Entry("zero", "0"),
Entry("zero with unit", "0MB"),
Entry("overflows int64", "9223372036854775808"),
) )
}) })
Describe("MaxImageSize floor", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
It("is raised to MaxImageUploadSize when configured lower", func() {
viper.SetDefault("maximagesize", "5MB")
viper.SetDefault("maximageuploadsize", "50MB")
conf.Load(true)
Expect(conf.Server.MaxImageSize).To(Equal("50MB"))
})
It("keeps a larger MaxImageSize unchanged", func() {
viper.SetDefault("maximagesize", "30MB")
conf.Load(true)
Expect(conf.Server.MaxImageSize).To(Equal("30MB"))
})
})
Describe("Scanner.Extractor", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
It("falls back to taglib for an unknown extractor", func() {
viper.SetDefault("scanner.extractor", "ffmpeg")
conf.Load(true)
Expect(conf.Server.Scanner.Extractor).To(Equal("taglib"))
})
It("keeps taglib", func() {
conf.Load(true)
Expect(conf.Server.Scanner.Extractor).To(Equal("taglib"))
})
})
Describe("EnforceNonRootUser", func() { Describe("EnforceNonRootUser", func() {
It("defaults to false", func() { It("defaults to false", func() {
conf.Load(true) conf.Load(true)
@ -492,73 +317,4 @@ var _ = Describe("Configuration", func() {
Entry("INI format", "ini"), Entry("INI format", "ini"),
Entry("JSON format", "json"), Entry("JSON format", "json"),
) )
It("should use default values for negative duration fields", func() {
filename := filepath.Join("testdata", "invalid_duration.toml")
conf.InitConfig(filename, false)
conf.Load(true)
server := conf.Server
Expect(server.SessionTimeout).To(Equal(consts.DefaultSessionTimeout))
Expect(server.SmartPlaylistRefreshDelay).To(Equal(consts.DefaultSmartRefresh))
Expect(server.DefaultShareExpiration).To(Equal(consts.DefaultShareExpiration))
Expect(server.UIPlaybackReportInterval).To(Equal(consts.DefaultUIPlaybackReportInterval))
Expect(server.AuthWindowLength).To(Equal(consts.DefaultAuthWindowLength))
Expect(server.Scanner.WatcherWait).To(Equal(consts.DefaultWatcherWait))
Expect(server.DevActivityPanelUpdateRate).To(Equal(consts.DefaultActivityPanelUpdateRate))
Expect(server.DevArtworkThrottleBacklogTimeout).To(Equal(consts.RequestThrottleBacklogTimeout))
Expect(server.DevArtistInfoTimeToLive).To(Equal(consts.ArtistInfoTimeToLive))
Expect(server.DevAlbumInfoTimeToLive).To(Equal(consts.AlbumInfoTimeToLive))
Expect(server.DevInsightsInitialDelay).To(Equal(consts.InsightsInitialDelay))
Expect(server.DevPluginCompilationTimeout).To(Equal(consts.DefaultPluginCompilationTimeout))
})
It("should use parsed values for duration fields", func() {
conf.InitConfig(filepath.Join("testdata", "valid_duration.toml"), false)
conf.Load(true)
configured := 1 * time.Second
server := conf.Server
Expect(server.SessionTimeout).To(Equal(configured))
Expect(server.SmartPlaylistRefreshDelay).To(Equal(configured))
Expect(server.DefaultShareExpiration).To(Equal(configured))
Expect(server.UIPlaybackReportInterval).To(Equal(configured))
Expect(server.AuthWindowLength).To(Equal(configured))
Expect(server.Scanner.WatcherWait).To(Equal(configured))
Expect(server.DevActivityPanelUpdateRate).To(Equal(configured))
Expect(server.DevArtworkThrottleBacklogTimeout).To(Equal(configured))
Expect(server.DevArtistInfoTimeToLive).To(Equal(configured))
Expect(server.DevAlbumInfoTimeToLive).To(Equal(configured))
Expect(server.DevInsightsInitialDelay).To(Equal(configured))
Expect(server.DevPluginCompilationTimeout).To(Equal(configured))
})
})
var _ = Describe("TLSEnabled", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
It("is false when neither the certificate nor the key is set", func() {
Expect(conf.Server.TLSEnabled()).To(BeFalse())
})
It("is true when both the certificate and the key are set", func() {
conf.Server.TLSCert = "cert.pem"
conf.Server.TLSKey = "key.pem"
Expect(conf.Server.TLSEnabled()).To(BeTrue())
})
It("is false when only the certificate is set", func() {
conf.Server.TLSCert = "cert.pem"
Expect(conf.Server.TLSEnabled()).To(BeFalse())
})
It("is false when only the key is set", func() {
conf.Server.TLSKey = "key.pem"
Expect(conf.Server.TLSEnabled()).To(BeFalse())
})
}) })

View file

@ -14,7 +14,7 @@ var NormalizeSearchBackend = normalizeSearchBackend
var ToPascalCase = toPascalCase var ToPascalCase = toPascalCase
var ValidateByteSize = validateByteSize var ValidateMaxImageUploadSize = validateMaxImageUploadSize
func SetRuntimeInfoForTest(goos string, euid int) func() { func SetRuntimeInfoForTest(goos string, euid int) func() {
oldGOOS := currentGOOS oldGOOS := currentGOOS
@ -32,7 +32,3 @@ func SetLogFatal(f func(...any)) func() {
logFatal = f logFatal = f
return func() { logFatal = old } return func() { logFatal = old }
} }
var UnknownConfigKeys = unknownConfigKeys
var SuggestOptions = suggestOptions

View file

@ -1,2 +0,0 @@
MusicFolder = "/toml/music"
SearchFullString = true

View file

@ -1,3 +0,0 @@
MusicFolder = "/toml/music"
ND_TOTALLY_BOGUS_OPTION = true
ND_SCANNER_SCHEDULE = "@every 1h"

View file

@ -1,10 +0,0 @@
MusicFolder = "/toml/music"
SearchFullString = true
ConfigFile = "/somewhere/else"
ID = "oops"
[Search]
FullString = false
[LastFM]
Languages = ["pt"]

View file

@ -1,3 +0,0 @@
[default]
MusicFolder = /ini/music
NotAnOption = true

View file

@ -1,4 +0,0 @@
{
"MusicFolder": "/json/music",
"NotAnOption": true
}

View file

@ -1,2 +0,0 @@
MusicFolder = "/toml/music"
NotAnOption = true

View file

@ -1,2 +0,0 @@
MusicFolder: /yaml/music
NotAnOption: true

View file

@ -1,18 +0,0 @@
MusicFolder = "/toml/music"
# Valid option, but written at the root level instead of under Scanner
ArtistSplitExceptions = ["AC/DC", "Tyler, the creator"]
# Misspelled option
EnableDownlods = true
# Unknown section
[Whatever]
Foo = "bar"
# Valid options, must not be reported
[Scanner]
ArtistJoiner = " • "
[Tags.custom]
aliases = ["toml", "test"]

View file

@ -1,7 +0,0 @@
MusicFolder = "/toml/music"
LogLevel = "warn"
ArtistSplitExceptions = ["AC/DC"]
EnableDownlods = true
[Scanner]
ArtistJoiner = " • "

View file

@ -1,12 +0,0 @@
SessionTimeout = "-10s"
SmartPlaylistRefreshDelay = "-10s"
UIPlaybackReportInterval = "-10s"
AuthWindowLength = "-10s"
DefaultShareExpiration = "-10s"
Scanner.WatcherWait = "-10s"
DevActivityPanelUpdateRate = "-10s"
DevArtworkThrottleBacklogTimeout = "-10s"
DevArtistInfoTimeToLive = "-10s"
DevAlbumInfoTimeToLive = "-10s"
DevInsightsInitialDelay = "-10s"
DevPluginCompilationTimeout = "-10s"

View file

@ -1,12 +0,0 @@
SessionTimeout = "1s"
SmartPlaylistRefreshDelay = "1s"
UIPlaybackReportInterval = "1s"
AuthWindowLength = "1s"
DefaultShareExpiration = "1s"
Scanner.WatcherWait = "1s"
DevActivityPanelUpdateRate = "1s"
DevArtworkThrottleBacklogTimeout = "1s"
DevArtistInfoTimeToLive = "1s"
DevAlbumInfoTimeToLive = "1s"
DevInsightsInitialDelay = "1s"
DevPluginCompilationTimeout = "1s"

View file

@ -24,32 +24,26 @@ const (
LastDBAnalyzeAttemptAtKey = "LastDBAnalyzeAttemptAt" LastDBAnalyzeAttemptAtKey = "LastDBAnalyzeAttemptAt"
DBAnalyzePendingKey = "DBAnalyzePending" DBAnalyzePendingKey = "DBAnalyzePending"
DBAnalyzeFailureCountKey = "DBAnalyzeFailureCount" DBAnalyzeFailureCountKey = "DBAnalyzeFailureCount"
// ArtConfFingerprintPropertyKey is the model.PropertyRepository key the artwork config check
// compares against to detect artwork-affecting config changes across restarts.
ArtConfFingerprintPropertyKey = "ArtConfFingerprint"
UIAuthorizationHeader = "X-ND-Authorization" UIAuthorizationHeader = "X-ND-Authorization"
UIClientUniqueIDHeader = "X-ND-Client-Unique-Id" UIClientUniqueIDHeader = "X-ND-Client-Unique-Id"
JWTSecretKey = "JWTSecret" JWTSecretKey = "JWTSecret"
JWTPublicSecretKey = "JWTPublicSecret"
JWTIssuer = "ND" JWTIssuer = "ND"
DefaultSessionTimeout = 48 * time.Hour DefaultSessionTimeout = 48 * time.Hour
DefaultSmartRefresh = 5 * time.Second
DefaultShareExpiration = 8760 * time.Hour
CookieExpiry = 365 * 24 * 3600 // One year CookieExpiry = 365 * 24 * 3600 // One year
DBAnalyzeCheckSchedule = "@every 30m" DBAnalyzeCheckSchedule = "@every 30m"
DBAnalyzeMaxAge = 24 * time.Hour DBAnalyzeMaxAge = 24 * time.Hour
ArtworkEnqueueMissingSchedule = "@every 1h" ArtworkStaleAbsentRecheckSchedule = "@every 1h"
ArtworkPruneSchedule = "@daily" ArtworkPruneSchedule = "@daily"
ArtworkPostBackfillPruneDelay = 10 * time.Minute
// DefaultEncryptionKey This is the encryption key used if none is specified in the `PasswordEncryptionKey` option // DefaultEncryptionKey This is the encryption key used if none is specified in the `PasswordEncryptionKey` option
// Never ever change this! Or it will break all Navidrome installations that don't set the config option // Never ever change this! Or it will break all Navidrome installations that don't set the config option
DefaultEncryptionKey = "just for obfuscation" DefaultEncryptionKey = "just for obfuscation"
PasswordsEncryptedKey = "PasswordsEncryptedKey" PasswordsEncryptedKey = "PasswordsEncryptedKey"
PasswordAutogenPrefix = "__NAVIDROME_AUTOGEN__" //nolint:gosec PasswordAutogenPrefix = "__NAVIDROME_AUTOGEN__" //nolint:gosec
APIKeyPrefix = "nds_"
DevInitialUserName = "admin" DevInitialUserName = "admin"
DevInitialName = "Dev Admin" DevInitialName = "Dev Admin"
@ -60,7 +54,6 @@ const (
URLPathPublic = "/share" URLPathPublic = "/share"
URLPathPublicImages = URLPathPublic + "/img" URLPathPublicImages = URLPathPublic + "/img"
URLPathJellyfinAPI = "/jellyfin" URLPathJellyfinAPI = "/jellyfin"
URLPathAPIv1 = "/api/v1"
// JellyfinServerIDKey is the Property key for the stable, persisted server Id reported by the // JellyfinServerIDKey is the Property key for the stable, persisted server Id reported by the
// Jellyfin API. Jellyfin clients cache this value, so it must survive process restarts. // Jellyfin API. Jellyfin clients cache this value, so it must survive process restarts.
@ -75,7 +68,6 @@ const (
DefaultUILoginBackgroundURLOffline = "data:image/png;base64," + DefaultUILoginBackgroundOffline DefaultUILoginBackgroundURLOffline = "data:image/png;base64," + DefaultUILoginBackgroundOffline
DefaultMaxSidebarPlaylists = 100 DefaultMaxSidebarPlaylists = 100
DefaultAuthWindowLength = 20 * time.Second
RequestThrottleBacklogLimit = 100 RequestThrottleBacklogLimit = 100
RequestThrottleBacklogTimeout = time.Minute RequestThrottleBacklogTimeout = time.Minute
@ -91,9 +83,6 @@ const (
I18nFolder = "i18n" I18nFolder = "i18n"
ScanIgnoreFile = ".ndignore" ScanIgnoreFile = ".ndignore"
ArtworkFolder = "artwork" ArtworkFolder = "artwork"
// HashedArtworkFolder is a subtree of ArtworkFolder, kept apart from the name-addressed
// upload folders beside it so Prune's sweep never reaches them.
HashedArtworkFolder = "hashed"
PlaceholderArtistArt = "artist-placeholder.webp" PlaceholderArtistArt = "artist-placeholder.webp"
PlaceholderAlbumArt = "album-placeholder.webp" PlaceholderAlbumArt = "album-placeholder.webp"
@ -111,15 +100,11 @@ const (
DefaultScannerExtractor = "taglib" DefaultScannerExtractor = "taglib"
DefaultWatcherWait = 5 * time.Second DefaultWatcherWait = 5 * time.Second
Zwsp = string('\u200b') Zwsp = string('\u200b')
DefaultActivityPanelUpdateRate = 300 * time.Millisecond
DefaultPluginCompilationTimeout = time.Minute
) )
const ( const (
DefaultUICoverArtSize = 300 DefaultUICoverArtSize = 300
DefaultMaxImageUploadSize = "10MB" DefaultMaxImageUploadSize = "10MB"
DefaultMaxImageSize = "20MB"
) )
// Prometheus options // Prometheus options
@ -156,6 +141,8 @@ const (
//DefaultAlbumPID = "album_legacy" //DefaultAlbumPID = "album_legacy"
DefaultAlbumPID = "musicbrainz_albumid|albumartistid,album,albumversion,releasedate" DefaultAlbumPID = "musicbrainz_albumid|albumartistid,album,albumversion,releasedate"
DefaultTrackPID = "musicbrainz_trackid|albumid,discnumber,tracknumber,title" DefaultTrackPID = "musicbrainz_trackid|albumid,discnumber,tracknumber,title"
PIDAlbumKey = "PIDAlbum"
PIDTrackKey = "PIDTrack"
) )
const ( const (
@ -206,7 +193,7 @@ var (
} }
) )
var HTTPUserAgent = "Navidrome/" + Version + " - https://github.com/navidrome" var HTTPUserAgent = "Navidrome" + "/" + Version
var ( var (
VariousArtists = "Various Artists" VariousArtists = "Various Artists"

View file

@ -2,7 +2,7 @@
name=$RC_SVCNAME name=$RC_SVCNAME
command="/opt/navidrome/${RC_SVCNAME}" command="/opt/navidrome/${RC_SVCNAME}"
command_args="--datafolder /opt/navidrome" command_args="-datafolder /opt/navidrome"
command_user="${RC_SVCNAME}" command_user="${RC_SVCNAME}"
pidfile="/var/run/${RC_SVCNAME}.pid" pidfile="/var/run/${RC_SVCNAME}.pid"
output_log="/opt/navidrome/${RC_SVCNAME}.log" output_log="/opt/navidrome/${RC_SVCNAME}.log"

View file

@ -36,7 +36,7 @@ RestrictNamespaces=yes
RestrictRealtime=yes RestrictRealtime=yes
SystemCallFilter=@system-service SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources SystemCallFilter=~@privileged @resources
SystemCallFilter=setrlimit mbind SystemCallFilter=setrlimit
SystemCallArchitectures=native SystemCallArchitectures=native
UMask=0066 UMask=0066

View file

@ -1,13 +1,9 @@
package agents package agents
import ( import (
"cmp"
"context" "context"
"errors"
"maps"
"slices" "slices"
"strings" "strings"
"sync"
"time" "time"
"github.com/navidrome/navidrome/conf" "github.com/navidrome/navidrome/conf"
@ -26,43 +22,11 @@ type PluginLoader interface {
LoadMediaAgent(name string) (Interface, bool) LoadMediaAgent(name string) (Interface, bool)
} }
// agentCooldown is the default cooldown duration for an agent that returns a RetryLaterError without a specific
// RetryIn duration.
const agentCooldown = time.Minute
// errUnsupported marks an agent that does not implement the requested method: it never ran,
// so it neither answered nor throttled.
var errUnsupported = errors.New("agent does not support this method")
// Agents is a meta-agent that aggregates multiple built-in and plugin agents. It tries each enabled agent in order // Agents is a meta-agent that aggregates multiple built-in and plugin agents. It tries each enabled agent in order
// until one returns valid data. // until one returns valid data.
type Agents struct { type Agents struct {
ds model.DataStore ds model.DataStore
pluginLoader PluginLoader pluginLoader PluginLoader
cooldowns cooldowns
}
// cooldowns remembers, across dispatches, which agents asked to be left alone and until when.
type cooldowns struct {
mu sync.RWMutex
until map[string]time.Time
}
func (c *cooldowns) active(name string) bool {
c.mu.RLock()
defer c.mu.RUnlock()
return time.Now().Before(c.until[name])
}
// park keeps whichever deadline is later, so a call still in flight when a longer cooldown
// starts cannot cut it short when it finally answers.
func (c *cooldowns) park(name string, d time.Duration) {
until := time.Now().Add(d)
c.mu.Lock()
defer c.mu.Unlock()
if until.After(c.until[name]) {
c.until[name] = until
}
} }
// GetAgents returns the singleton instance of Agents // GetAgents returns the singleton instance of Agents
@ -77,7 +41,6 @@ func createAgents(ds model.DataStore, pluginLoader PluginLoader) *Agents {
return &Agents{ return &Agents{
ds: ds, ds: ds,
pluginLoader: pluginLoader, pluginLoader: pluginLoader,
cooldowns: cooldowns{until: map[string]time.Time{}},
} }
} }
@ -127,19 +90,12 @@ func (a *Agents) getEnabledAgentNames() []enabledAgent {
} else if isPlugin { } else if isPlugin {
validAgents = append(validAgents, enabledAgent{name: name, isPlugin: true}) validAgents = append(validAgents, enabledAgent{name: name, isPlugin: true})
} else { } else {
log.Debug("Unknown agent ignored", "name", name, "available", availableAgentNames(availablePlugins)) log.Debug("Unknown agent ignored", "name", name)
} }
} }
return validAgents return validAgents
} }
// availableAgentNames returns every name accepted by the Agents config option.
func availableAgentNames(plugins []string) []string {
names := append(slices.Collect(maps.Keys(Map)), plugins...)
slices.Sort(names)
return names
}
func (a *Agents) getAgent(ea enabledAgent) Interface { func (a *Agents) getAgent(ea enabledAgent) Interface {
if ea.isPlugin { if ea.isPlugin {
// Try to load WASM plugin agent (if plugin loader is available) // Try to load WASM plugin agent (if plugin loader is available)
@ -185,7 +141,11 @@ type AlbumImageAgent struct {
func (a *Agents) ArtistImageAgents() []ArtistImageAgent { func (a *Agents) ArtistImageAgents() []ArtistImageAgent {
var result []ArtistImageAgent var result []ArtistImageAgent
for _, ea := range a.getEnabledAgentNames() { for _, ea := range a.getEnabledAgentNames() {
if retriever, ok := a.getAgent(ea).(ArtistImageRetriever); ok { ag := a.getAgent(ea)
if ag == nil {
continue
}
if retriever, ok := ag.(ArtistImageRetriever); ok {
result = append(result, ArtistImageAgent{Name: ea.name, Retriever: retriever}) result = append(result, ArtistImageAgent{Name: ea.name, Retriever: retriever})
} }
} }
@ -197,7 +157,11 @@ func (a *Agents) ArtistImageAgents() []ArtistImageAgent {
func (a *Agents) AlbumImageAgents() []AlbumImageAgent { func (a *Agents) AlbumImageAgents() []AlbumImageAgent {
var result []AlbumImageAgent var result []AlbumImageAgent
for _, ea := range a.getEnabledAgentNames() { for _, ea := range a.getEnabledAgentNames() {
if retriever, ok := a.getAgent(ea).(AlbumImageRetriever); ok { ag := a.getAgent(ea)
if ag == nil {
continue
}
if retriever, ok := ag.(AlbumImageRetriever); ok {
result = append(result, AlbumImageAgent{Name: ea.name, Retriever: retriever}) result = append(result, AlbumImageAgent{Name: ea.name, Retriever: retriever})
} }
} }
@ -215,7 +179,7 @@ func (a *Agents) GetArtistMBID(ctx context.Context, id string, name string) (str
return callAgentMethod(ctx, a, "GetArtistMBID", func(ag Interface) (string, error) { return callAgentMethod(ctx, a, "GetArtistMBID", func(ag Interface) (string, error) {
retriever, ok := ag.(ArtistMBIDRetriever) retriever, ok := ag.(ArtistMBIDRetriever)
if !ok { if !ok {
return "", errUnsupported return "", ErrNotFound
} }
return retriever.GetArtistMBID(ctx, id, name) return retriever.GetArtistMBID(ctx, id, name)
}) })
@ -232,7 +196,7 @@ func (a *Agents) GetArtistURL(ctx context.Context, id, name, mbid string) (strin
return callAgentMethod(ctx, a, "GetArtistURL", func(ag Interface) (string, error) { return callAgentMethod(ctx, a, "GetArtistURL", func(ag Interface) (string, error) {
retriever, ok := ag.(ArtistURLRetriever) retriever, ok := ag.(ArtistURLRetriever)
if !ok { if !ok {
return "", errUnsupported return "", ErrNotFound
} }
return retriever.GetArtistURL(ctx, id, name, mbid) return retriever.GetArtistURL(ctx, id, name, mbid)
}) })
@ -249,7 +213,7 @@ func (a *Agents) GetArtistBiography(ctx context.Context, id, name, mbid string)
return callAgentMethod(ctx, a, "GetArtistBiography", func(ag Interface) (string, error) { return callAgentMethod(ctx, a, "GetArtistBiography", func(ag Interface) (string, error) {
retriever, ok := ag.(ArtistBiographyRetriever) retriever, ok := ag.(ArtistBiographyRetriever)
if !ok { if !ok {
return "", errUnsupported return "", ErrNotFound
} }
return retriever.GetArtistBiography(ctx, id, name, mbid) return retriever.GetArtistBiography(ctx, id, name, mbid)
}) })
@ -268,11 +232,7 @@ func (a *Agents) GetSimilarArtists(ctx context.Context, id, name, mbid string, l
overLimit := int(float64(limit) * conf.Server.DevExternalArtistFetchMultiplier) overLimit := int(float64(limit) * conf.Server.DevExternalArtistFetchMultiplier)
start := time.Now() start := time.Now()
attempts := newAttempts(&a.cooldowns)
for _, enabledAgent := range a.getEnabledAgentNames() { for _, enabledAgent := range a.getEnabledAgentNames() {
if attempts.skip(enabledAgent.name) {
continue
}
ag := a.getAgent(enabledAgent) ag := a.getAgent(enabledAgent)
if ag == nil { if ag == nil {
continue continue
@ -285,7 +245,6 @@ func (a *Agents) GetSimilarArtists(ctx context.Context, id, name, mbid string, l
continue continue
} }
similar, err := retriever.GetSimilarArtists(ctx, id, name, mbid, overLimit) similar, err := retriever.GetSimilarArtists(ctx, id, name, mbid, overLimit)
attempts.record(enabledAgent.name, err)
if len(similar) > 0 && err == nil { if len(similar) > 0 && err == nil {
if log.IsGreaterOrEqualTo(log.LevelTrace) { if log.IsGreaterOrEqualTo(log.LevelTrace) {
log.Debug(ctx, "Got Similar Artists", "agent", ag.AgentName(), "artist", name, "similar", similar, "elapsed", time.Since(start)) log.Debug(ctx, "Got Similar Artists", "agent", ag.AgentName(), "artist", name, "similar", similar, "elapsed", time.Since(start))
@ -295,7 +254,7 @@ func (a *Agents) GetSimilarArtists(ctx context.Context, id, name, mbid string, l
return similar, err return similar, err
} }
} }
return nil, attempts.noResultErr() return nil, ErrNotFound
} }
func (a *Agents) GetArtistImages(ctx context.Context, id, name, mbid string) ([]ExternalImage, error) { func (a *Agents) GetArtistImages(ctx context.Context, id, name, mbid string) ([]ExternalImage, error) {
@ -309,7 +268,7 @@ func (a *Agents) GetArtistImages(ctx context.Context, id, name, mbid string) ([]
return callAgentSliceMethod(ctx, a, "GetArtistImages", func(ag Interface) ([]ExternalImage, error) { return callAgentSliceMethod(ctx, a, "GetArtistImages", func(ag Interface) ([]ExternalImage, error) {
retriever, ok := ag.(ArtistImageRetriever) retriever, ok := ag.(ArtistImageRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetArtistImages(ctx, id, name, mbid) return retriever.GetArtistImages(ctx, id, name, mbid)
}) })
@ -330,7 +289,7 @@ func (a *Agents) GetArtistTopSongs(ctx context.Context, id, artistName, mbid str
return callAgentSliceMethod(ctx, a, "GetArtistTopSongs", func(ag Interface) ([]Song, error) { return callAgentSliceMethod(ctx, a, "GetArtistTopSongs", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(ArtistTopSongsRetriever) retriever, ok := ag.(ArtistTopSongsRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetArtistTopSongs(ctx, id, artistName, mbid, overLimit) return retriever.GetArtistTopSongs(ctx, id, artistName, mbid, overLimit)
}) })
@ -344,7 +303,7 @@ func (a *Agents) GetAlbumInfo(ctx context.Context, name, artist, mbid string) (*
return callAgentMethod(ctx, a, "GetAlbumInfo", func(ag Interface) (*AlbumInfo, error) { return callAgentMethod(ctx, a, "GetAlbumInfo", func(ag Interface) (*AlbumInfo, error) {
retriever, ok := ag.(AlbumInfoRetriever) retriever, ok := ag.(AlbumInfoRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetAlbumInfo(ctx, name, artist, mbid) return retriever.GetAlbumInfo(ctx, name, artist, mbid)
}) })
@ -358,7 +317,7 @@ func (a *Agents) GetAlbumImages(ctx context.Context, name, artist, mbid string)
return callAgentSliceMethod(ctx, a, "GetAlbumImages", func(ag Interface) ([]ExternalImage, error) { return callAgentSliceMethod(ctx, a, "GetAlbumImages", func(ag Interface) ([]ExternalImage, error) {
retriever, ok := ag.(AlbumImageRetriever) retriever, ok := ag.(AlbumImageRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetAlbumImages(ctx, name, artist, mbid) return retriever.GetAlbumImages(ctx, name, artist, mbid)
}) })
@ -369,7 +328,7 @@ func (a *Agents) GetSimilarSongsByTrack(ctx context.Context, id, name, artist, m
return callAgentSliceMethod(ctx, a, "GetSimilarSongsByTrack", func(ag Interface) ([]Song, error) { return callAgentSliceMethod(ctx, a, "GetSimilarSongsByTrack", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(SimilarSongsByTrackRetriever) retriever, ok := ag.(SimilarSongsByTrackRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetSimilarSongsByTrack(ctx, id, name, artist, mbid, count) return retriever.GetSimilarSongsByTrack(ctx, id, name, artist, mbid, count)
}) })
@ -380,7 +339,7 @@ func (a *Agents) GetSimilarSongsByAlbum(ctx context.Context, id, name, artist, m
return callAgentSliceMethod(ctx, a, "GetSimilarSongsByAlbum", func(ag Interface) ([]Song, error) { return callAgentSliceMethod(ctx, a, "GetSimilarSongsByAlbum", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(SimilarSongsByAlbumRetriever) retriever, ok := ag.(SimilarSongsByAlbumRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetSimilarSongsByAlbum(ctx, id, name, artist, mbid, count) return retriever.GetSimilarSongsByAlbum(ctx, id, name, artist, mbid, count)
}) })
@ -398,61 +357,16 @@ func (a *Agents) GetSimilarSongsByArtist(ctx context.Context, id, name, mbid str
return callAgentSliceMethod(ctx, a, "GetSimilarSongsByArtist", func(ag Interface) ([]Song, error) { return callAgentSliceMethod(ctx, a, "GetSimilarSongsByArtist", func(ag Interface) ([]Song, error) {
retriever, ok := ag.(SimilarSongsByArtistRetriever) retriever, ok := ag.(SimilarSongsByArtistRetriever)
if !ok { if !ok {
return nil, errUnsupported return nil, ErrNotFound
} }
return retriever.GetSimilarSongsByArtist(ctx, id, name, mbid, count) return retriever.GetSimilarSongsByArtist(ctx, id, name, mbid, count)
}) })
} }
// agentAttempts tallies what the enabled agents did in one dispatch. func callAgentMethod[T comparable](ctx context.Context, agents *Agents, methodName string, fn func(Interface) (T, error)) (T, error) {
type agentAttempts struct {
cooldowns *cooldowns
throttled bool
answered bool
}
func newAttempts(c *cooldowns) agentAttempts {
return agentAttempts{cooldowns: c}
}
// skip reports whether name is still cooling down, counting it as throttled for this dispatch.
func (t *agentAttempts) skip(name string) bool {
if !t.cooldowns.active(name) {
return false
}
t.throttled = true
return true
}
// record files one agent's outcome, parking it when it asked to be retried later.
func (t *agentAttempts) record(name string, err error) {
switch retry, isRetryLater := errors.AsType[*RetryLaterError](err); {
case errors.Is(err, errUnsupported):
case isRetryLater:
t.cooldowns.park(name, cmp.Or(retry.RetryIn, agentCooldown))
t.throttled = true
default:
t.answered = true
}
}
// noResultErr tells a retryable empty dispatch (nobody answered) from a definitive miss.
func (t *agentAttempts) noResultErr() error {
if t.throttled && !t.answered {
return ErrRetryLater
}
return ErrNotFound
}
// callAgent tries each enabled agent in order until found reports a usable result.
func callAgent[T any](ctx context.Context, agents *Agents, methodName string, fn func(Interface) (T, error), found func(T) bool) (T, error) {
var zero T var zero T
start := time.Now() start := time.Now()
attempts := newAttempts(&agents.cooldowns)
for _, enabledAgent := range agents.getEnabledAgentNames() { for _, enabledAgent := range agents.getEnabledAgentNames() {
if attempts.skip(enabledAgent.name) {
continue
}
ag := agents.getAgent(enabledAgent) ag := agents.getAgent(enabledAgent)
if ag == nil { if ag == nil {
continue continue
@ -461,29 +375,41 @@ func callAgent[T any](ctx context.Context, agents *Agents, methodName string, fn
break break
} }
result, err := fn(ag) result, err := fn(ag)
attempts.record(enabledAgent.name, err)
if err != nil { if err != nil {
log.Trace(ctx, "Agent method call error", "method", methodName, "agent", ag.AgentName(), "error", err) log.Trace(ctx, "Agent method call error", "method", methodName, "agent", ag.AgentName(), "error", err)
continue continue
} }
if found(result) { if result != zero {
log.Debug(ctx, "Got result", "method", methodName, "agent", ag.AgentName(), "elapsed", time.Since(start)) log.Debug(ctx, "Got result", "method", methodName, "agent", ag.AgentName(), "elapsed", time.Since(start))
return result, nil return result, nil
} }
} }
return zero, attempts.noResultErr() return zero, ErrNotFound
}
func callAgentMethod[T comparable](ctx context.Context, agents *Agents, methodName string, fn func(Interface) (T, error)) (T, error) {
return callAgent(ctx, agents, methodName, fn, func(result T) bool {
var zero T
return result != zero
})
} }
func callAgentSliceMethod[T any](ctx context.Context, agents *Agents, methodName string, fn func(Interface) ([]T, error)) ([]T, error) { func callAgentSliceMethod[T any](ctx context.Context, agents *Agents, methodName string, fn func(Interface) ([]T, error)) ([]T, error) {
return callAgent(ctx, agents, methodName, fn, func(results []T) bool { return len(results) > 0 }) start := time.Now()
for _, enabledAgent := range agents.getEnabledAgentNames() {
ag := agents.getAgent(enabledAgent)
if ag == nil {
continue
}
if utils.IsCtxDone(ctx) {
break
}
results, err := fn(ag)
if err != nil {
log.Trace(ctx, "Agent method call error", "method", methodName, "agent", ag.AgentName(), "error", err)
continue
}
if len(results) > 0 {
log.Debug(ctx, "Got results", "method", methodName, "agent", ag.AgentName(), "count", len(results), "elapsed", time.Since(start))
return results, nil
}
}
return nil, ErrNotFound
} }
var _ Interface = (*Agents)(nil) var _ Interface = (*Agents)(nil)

View file

@ -3,8 +3,6 @@ package agents
import ( import (
"context" "context"
"errors" "errors"
"slices"
"time"
"github.com/navidrome/navidrome/conf/configtest" "github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/consts" "github.com/navidrome/navidrome/consts"
@ -16,29 +14,6 @@ import (
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
) )
var _ = Describe("cooldowns", func() {
// Calls to one agent overlap, so a short cooldown can land after a long one started.
It("keeps the longer deadline when a shorter park lands after it", func() {
c := cooldowns{until: map[string]time.Time{}}
c.park("fake", time.Hour)
c.park("fake", time.Millisecond)
time.Sleep(10 * time.Millisecond)
Expect(c.active("fake")).To(BeTrue())
})
It("extends the deadline when the later park is longer", func() {
c := cooldowns{until: map[string]time.Time{}}
c.park("fake", time.Millisecond)
c.park("fake", time.Hour)
time.Sleep(10 * time.Millisecond)
Expect(c.active("fake")).To(BeTrue())
})
})
var _ = Describe("Agents", func() { var _ = Describe("Agents", func() {
var ctx context.Context var ctx context.Context
var cancel context.CancelFunc var cancel context.CancelFunc
@ -59,10 +34,10 @@ var _ = Describe("Agents", func() {
}) })
It("calls the placeholder GetArtistImages", func() { It("calls the placeholder GetArtistImages", func() {
mfRepo.SetData(model.MediaFiles{{ID: "1", Title: "One"}, {ID: "2", Title: "Two"}}) mfRepo.SetData(model.MediaFiles{{ID: "1", Title: "One", MbzReleaseTrackID: "111"}, {ID: "2", Title: "Two", MbzReleaseTrackID: "222"}})
songs, err := ag.GetArtistTopSongs(ctx, "123", "John Doe", "mb123", 2) songs, err := ag.GetArtistTopSongs(ctx, "123", "John Doe", "mb123", 2)
Expect(err).ToNot(HaveOccurred()) Expect(err).ToNot(HaveOccurred())
Expect(songs).To(ConsistOf([]Song{{ID: "1", Name: "One"}, {ID: "2", Name: "Two"}})) Expect(songs).To(ConsistOf([]Song{{Name: "One", MBID: "111"}, {Name: "Two", MBID: "222"}}))
}) })
}) })
@ -92,22 +67,6 @@ var _ = Describe("Agents", func() {
Expect(ags).ToNot(ContainElement("disabled")) Expect(ags).ToNot(ContainElement("disabled"))
}) })
Describe("availableAgentNames", func() {
It("combines built-in agents with the given plugins", func() {
names := availableAgentNames([]string{"apple-music"})
Expect(names).To(ContainElements("apple-music", LocalAgentName, "fake", "empty"))
})
It("returns the names sorted", func() {
names := availableAgentNames([]string{"zz-plugin", "aa-plugin"})
Expect(slices.IsSorted(names)).To(BeTrue())
})
It("works when there are no plugins", func() {
Expect(availableAgentNames(nil)).To(ContainElement(LocalAgentName))
})
})
Describe("GetArtistMBID", func() { Describe("GetArtistMBID", func() {
It("returns on first match", func() { It("returns on first match", func() {
Expect(ag.GetArtistMBID(ctx, "123", "test")).To(Equal("mbid")) Expect(ag.GetArtistMBID(ctx, "123", "test")).To(Equal("mbid"))
@ -201,102 +160,6 @@ var _ = Describe("Agents", func() {
}) })
}) })
Describe("cooldown", func() {
It("skips an agent that returned RetryLaterError until the deadline", func() {
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
// Immediately after: agent is skipped, not called
mock.Err = nil
calls := mock.Calls
_, err = ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(mock.Calls).To(Equal(calls))
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
})
// Providers that throttle without saying for how long (Last.fm sends no delay at all)
// must still be parked, or the aggregate keeps calling them on every request.
It("parks an agent that asked to be retried without a delay", func() {
mock.Err = ErrRetryLater
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
mock.Err = nil
calls := mock.Calls
_, err = ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(mock.Calls).To(Equal(calls), "the default cooldown must outlast the request")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
})
It("calls the agent again once the cooldown expires", func() {
mock.Err = &RetryLaterError{RetryIn: 10 * time.Millisecond}
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
mock.Err = nil
Eventually(func() (string, error) {
return ag.GetArtistBiography(ctx, "id", "name", "mbid")
}, 5*time.Second, 10*time.Millisecond).Should(Equal("bio"))
})
It("returns ErrNotFound, not ErrRetryLater, when agents failed for other reasons", func() {
mock.Err = errors.New("boom")
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
})
// ErrRetryLater tells the caller "nobody answered, do not cache this". A definitive
// answer from any other agent is an answer, throttled peer or not.
It("returns ErrNotFound when another agent answered with a definitive miss", func() {
other := &mockAgent{Err: ErrNotFound}
Register("fake2", func(model.DataStore) Interface { return other })
conf.Server.Agents = "fake,fake2"
ag = createAgents(ds, nil)
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
// The cooldown was still recorded for the throttled agent
calls := mock.Calls
_, _ = ag.GetArtistBiography(ctx, "id", "name", "mbid")
Expect(mock.Calls).To(Equal(calls))
})
It("returns ErrNotFound when another agent answered with an empty slice", func() {
empty := &testImageAgent{Name: "emptyImages"}
Register("emptyImages", func(model.DataStore) Interface { return empty })
conf.Server.Agents = "fake,emptyImages"
ag = createAgents(ds, nil)
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetArtistImages(ctx, "123", "test", "mb123")
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
})
It("returns ErrRetryLater from GetSimilarArtists when only cooling agents remain", func() {
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetSimilarArtists(ctx, "123", "test", "mb123", 2)
Expect(errors.Is(err, ErrRetryLater)).To(BeTrue())
})
It("returns ErrNotFound from GetSimilarArtists when another agent answered", func() {
other := &mockAgent{Err: ErrNotFound}
Register("fake2", func(model.DataStore) Interface { return other })
conf.Server.Agents = "fake,fake2"
ag = createAgents(ds, nil)
mock.Err = &RetryLaterError{RetryIn: time.Hour}
_, err := ag.GetSimilarArtists(ctx, "123", "test", "mb123", 2)
Expect(errors.Is(err, ErrNotFound)).To(BeTrue())
Expect(errors.Is(err, ErrRetryLater)).To(BeFalse())
})
})
Describe("GetArtistImages", func() { Describe("GetArtistImages", func() {
It("returns on first match", func() { It("returns on first match", func() {
Expect(ag.GetArtistImages(ctx, "123", "test", "mb123")).To(Equal([]ExternalImage{{ Expect(ag.GetArtistImages(ctx, "123", "test", "mb123")).To(Equal([]ExternalImage{{
@ -560,9 +423,8 @@ var _ = Describe("Agents", func() {
}) })
type mockAgent struct { type mockAgent struct {
Args []any Args []any
Err error Err error
Calls int
} }
func (a *mockAgent) AgentName() string { func (a *mockAgent) AgentName() string {
@ -587,7 +449,6 @@ func (a *mockAgent) GetArtistURL(_ context.Context, id, name, mbid string) (stri
func (a *mockAgent) GetArtistBiography(_ context.Context, id, name, mbid string) (string, error) { func (a *mockAgent) GetArtistBiography(_ context.Context, id, name, mbid string) (string, error) {
a.Args = []any{id, name, mbid} a.Args = []any{id, name, mbid}
a.Calls++
if a.Err != nil { if a.Err != nil {
return "", a.Err return "", a.Err
} }

View file

@ -3,9 +3,6 @@ package agents
import ( import (
"context" "context"
"errors" "errors"
"fmt"
"strconv"
"time"
"github.com/gohugoio/hashstructure" "github.com/gohugoio/hashstructure"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
@ -55,49 +52,9 @@ func (s Song) Equals(other Song) bool {
return h1 == h2 return h1 == h2
} }
// ErrNotFound means the provider answered and had nothing. Return the underlying error var (
// for a fault instead, or callers that back off on faults will treat it as definitive. ErrNotFound = errors.New("not found")
var ErrNotFound = errors.New("not found") )
// ErrRetryLater is the zero-delay RetryLaterError: the provider is temporarily unavailable
// or throttling us, but did not say for how long. Both errors.Is(err, ErrRetryLater) and
// errors.AsType[*RetryLaterError] match it and every delay-carrying variant.
// Treat it as immutable; build a new RetryLaterError to name a delay.
var ErrRetryLater = &RetryLaterError{}
// RetryLaterError asks callers to back off, optionally for the delay the provider requested.
type RetryLaterError struct {
RetryIn time.Duration
}
func (e *RetryLaterError) Error() string {
if e.RetryIn > 0 {
return fmt.Sprintf("retry later (in %s)", e.RetryIn)
}
return "retry later"
}
func (e *RetryLaterError) Is(target error) bool {
_, ok := target.(*RetryLaterError)
return ok
}
// MaxRetryIn caps a delay parsed from a provider, so a bogus value cannot park it indefinitely.
const MaxRetryIn = time.Hour
const maxRetryInSeconds = int(MaxRetryIn / time.Second)
// ParseRetryIn reads a provider's delay given in seconds, from a header or a plugin token.
// Anything unparseable or non-positive means unspecified.
func ParseRetryIn(seconds string) time.Duration {
// Clamp in seconds: scaling first would wrap a huge value past int64 nanoseconds,
// turning "wait an age" into a fraction of a second. Parse at a fixed width so the
// cap holds on the 32-bit targets we ship, where a plain Atoi would overflow first.
secs, err := strconv.ParseInt(seconds, 10, 64)
if err != nil || secs <= 0 {
return 0
}
return time.Duration(min(secs, int64(maxRetryInSeconds))) * time.Second
}
// AlbumInfoRetriever provides album info (no images) // AlbumInfoRetriever provides album info (no images)
type AlbumInfoRetriever interface { type AlbumInfoRetriever interface {

View file

@ -1,42 +1,27 @@
package agents_test package agents
import ( import (
"errors"
"fmt"
"time"
"github.com/navidrome/navidrome/core/agents"
"github.com/navidrome/navidrome/core/scrobbler"
. "github.com/onsi/ginkgo/v2" . "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
) )
var _ = Describe("RetryLaterError", func() { var _ = Describe("Song.Equals", func() {
It("matches the ErrRetryLater sentinel via errors.Is", func() { base := Song{ID: "1", Name: "S", Artists: []Artist{{ID: "x", Name: "A"}}}
err := &agents.RetryLaterError{RetryIn: 30 * time.Second} It("true for identical songs incl Artists", func() {
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue()) Expect(base.Equals(base)).To(BeTrue())
}) })
It("false when Artists differ", func() {
It("matches through errors.Join and wrapping", func() { other := base
err := fmt.Errorf("calling LB: %w", errors.Join(errors.New("http 429"), &agents.RetryLaterError{})) other.Artists = []Artist{{ID: "y", Name: "B"}}
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeTrue()) Expect(base.Equals(other)).To(BeFalse())
}) })
It("false when a scalar differs", func() {
It("exposes the delay through the wrapped error", func() { other := base
err := errors.Join(errors.New("http 429"), &agents.RetryLaterError{RetryIn: 42 * time.Second}) other.Name = "T"
retry, ok := errors.AsType[*agents.RetryLaterError](err) Expect(base.Equals(other)).To(BeFalse())
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(Equal(42 * time.Second))
}) })
It("true when both have empty Artists and equal scalars", func() {
It("matches the sentinel too, reporting no delay", func() { a := Song{ID: "1", Name: "S"}
retry, ok := errors.AsType[*agents.RetryLaterError](agents.ErrRetryLater) Expect(a.Equals(a)).To(BeTrue())
Expect(ok).To(BeTrue())
Expect(retry.RetryIn).To(BeZero())
})
It("is the same sentinel as scrobbler.ErrRetryLater", func() {
Expect(errors.Is(scrobbler.ErrRetryLater, agents.ErrRetryLater)).To(BeTrue())
Expect(errors.Is(&agents.RetryLaterError{}, scrobbler.ErrRetryLater)).To(BeTrue())
}) })
}) })

View file

@ -5,8 +5,6 @@ import (
"github.com/Masterminds/squirrel" "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/utils/slice"
) )
const LocalAgentName = "local" const LocalAgentName = "local"
@ -24,7 +22,7 @@ func (p *localAgent) AgentName() string {
} }
func (p *localAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]Song, error) { func (p *localAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]Song, error) {
top, err := p.ds.MediaFile().GetAll(ctx, model.QueryOptions{ top, err := p.ds.MediaFile(ctx).GetAll(model.QueryOptions{
Sort: "playCount", Sort: "playCount",
Order: "desc", Order: "desc",
Max: count, Max: count,
@ -39,51 +37,14 @@ func (p *localAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid
if err != nil { if err != nil {
return nil, err return nil, err
} }
return songsFrom(top), nil var result []Song
} for _, s := range top {
result = append(result, Song{
func (p *localAgent) GetSimilarSongsByTrack(ctx context.Context, id, name, artist, mbid string, count int) ([]Song, error) { Name: s.Title,
seed, err := p.ds.MediaFile().Get(ctx, id) MBID: s.MbzReleaseTrackID,
if err != nil { })
return nil, err
} }
// Tag ids derive from (name, value), so the seed's genre ids need no extra query. return result, nil
genreIDs := slice.Map(seed.Tags.Flatten(model.TagGenre), func(t model.Tag) string { return t.ID })
if len(genreIDs) == 0 {
return nil, nil
}
// Ask for extra so we can drop the seed itself and still fill the count.
candidates, err := p.ds.MediaFile().GetRandom(ctx, model.QueryOptions{
Filters: squirrel.And{
persistence.SongGenres.ByID(genreIDs),
squirrel.Eq{"missing": false},
},
Max: count + 1,
})
if err != nil {
return nil, err
}
filtered := make(model.MediaFiles, 0, len(candidates))
for _, s := range candidates {
if s.ID == id {
continue
}
filtered = append(filtered, s)
if len(filtered) >= count {
break
}
}
return songsFrom(filtered), nil
}
func songsFrom(mfs model.MediaFiles) []Song {
if len(mfs) == 0 {
return nil
}
return slice.Map(mfs, func(mf model.MediaFile) Song {
return Song{ID: mf.ID, Name: mf.Title}
})
} }
func init() { func init() {

View file

@ -1,96 +0,0 @@
package agents
import (
"context"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/slice"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("localAgent GetSimilarSongsByTrack", func() {
var ds *tests.MockDataStore
var mfRepo *tests.MockMediaFileRepo
var agent *localAgent
var ctx context.Context
BeforeEach(func() {
ctx = context.Background()
mfRepo = &tests.MockMediaFileRepo{}
ds = &tests.MockDataStore{MockedMediaFile: mfRepo}
agent = &localAgent{ds: ds}
})
It("excludes the seed track from its own similars", func() {
seed := model.MediaFile{ID: "seed-1", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
related := model.MediaFile{ID: "rel-1", Title: "Related", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
// SetData keys by ID; a duplicate "seed-1" entry would clobber the real seed.
mfRepo.SetData(model.MediaFiles{seed, related})
songs, err := agent.GetSimilarSongsByTrack(ctx, "seed-1", "Seed", "", "", 10)
Expect(err).ToNot(HaveOccurred())
names := slice.Map(songs, func(s Song) string { return s.Name })
Expect(names).ToNot(ContainElement("Seed"))
})
// The mock ignores QueryOptions.Filters, so assert the predicate itself: otherwise this spec
// would pass just as well with no genre filter at all.
It("queries the indexed genre join for the seed's own genres, skipping missing files", func() {
rock := model.NewTag(model.TagGenre, "Rock")
seed := model.MediaFile{ID: "seed-4", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
mfRepo.SetData(model.MediaFiles{seed})
_, err := agent.GetSimilarSongsByTrack(ctx, "seed-4", "Seed", "", "", 10)
Expect(err).ToNot(HaveOccurred())
sql, args, sqlErr := mfRepo.Options.Filters.ToSql()
Expect(sqlErr).ToNot(HaveOccurred())
Expect(sql).To(ContainSubstring("media_file_tags"), "must use the indexed join, not a json_tree scan")
Expect(sql).To(ContainSubstring("missing"))
Expect(args).To(ContainElement(false), "must exclude missing files, not select them")
Expect(args).To(ContainElement(rock.ID), "must filter on the seed's own genre tag id")
Expect(args).ToNot(ContainElement(model.NewTag(model.TagGenre, "Jazz").ID))
})
It("returns the library id so the matcher can resolve the song", func() {
seed := model.MediaFile{ID: "seed-3", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
// Without the id the matcher falls through to its MBID/title phases and resolves nothing,
// so the local fallback silently returns an empty mix.
related := model.MediaFile{ID: "rel-3", Title: "Related", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
mfRepo.SetData(model.MediaFiles{seed, related})
songs, err := agent.GetSimilarSongsByTrack(ctx, "seed-3", "Seed", "", "", 10)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(ContainElement(Song{ID: "rel-3", Name: "Related"}))
})
It("asks for one extra candidate so dropping the seed still fills the count", func() {
// The mock returns rows sorted by id, so the seed comes first and would consume the only
// slot if the query did not over-fetch.
seed := model.MediaFile{ID: "a-seed", Title: "Seed", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
related := model.MediaFile{ID: "b-rel", Title: "Related", Tags: model.Tags{model.TagGenre: []string{"Rock"}}}
mfRepo.SetData(model.MediaFiles{seed, related})
songs, err := agent.GetSimilarSongsByTrack(ctx, "a-seed", "Seed", "", "", 1)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(HaveLen(1))
Expect(songs[0].Name).To(Equal("Related"))
})
It("returns nil when the seed track has no genres", func() {
seed := model.MediaFile{ID: "seed-2", Title: "NoGenre"}
mfRepo.SetData(model.MediaFiles{seed})
songs, err := agent.GetSimilarSongsByTrack(ctx, "seed-2", "NoGenre", "", "", 10)
Expect(err).ToNot(HaveOccurred())
Expect(songs).To(BeEmpty())
// Without the early return an empty tag filter would scan the whole library.
Expect(mfRepo.Options).To(Equal(model.QueryOptions{}), "must not query at all")
})
})

View file

@ -13,13 +13,13 @@ type SessionKeys struct {
} }
func (sk *SessionKeys) Put(ctx context.Context, userId, sessionKey string) error { func (sk *SessionKeys) Put(ctx context.Context, userId, sessionKey string) error {
return sk.DataStore.UserProps().Put(ctx, userId, sk.KeyName, sessionKey) return sk.DataStore.UserProps(ctx).Put(userId, sk.KeyName, sessionKey)
} }
func (sk *SessionKeys) Get(ctx context.Context, userId string) (string, error) { func (sk *SessionKeys) Get(ctx context.Context, userId string) (string, error) {
return sk.DataStore.UserProps().Get(ctx, userId, sk.KeyName) return sk.DataStore.UserProps(ctx).Get(userId, sk.KeyName)
} }
func (sk *SessionKeys) Delete(ctx context.Context, userId string) error { func (sk *SessionKeys) Delete(ctx context.Context, userId string) error {
return sk.DataStore.UserProps().Delete(ctx, userId, sk.KeyName) return sk.DataStore.UserProps(ctx).Delete(userId, sk.KeyName)
} }

View file

@ -1,27 +0,0 @@
package agents
import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("Song.Equals", func() {
base := Song{ID: "1", Name: "S", Artists: []Artist{{ID: "x", Name: "A"}}}
It("true for identical songs incl Artists", func() {
Expect(base.Equals(base)).To(BeTrue())
})
It("false when Artists differ", func() {
other := base
other.Artists = []Artist{{ID: "y", Name: "B"}}
Expect(base.Equals(other)).To(BeFalse())
})
It("false when a scalar differs", func() {
other := base
other.Name = "T"
Expect(base.Equals(other)).To(BeFalse())
})
It("true when both have empty Artists and equal scalars", func() {
a := Song{ID: "1", Name: "S"}
Expect(a.Equals(a)).To(BeTrue())
})
})

View file

@ -2,86 +2,66 @@ package core
import ( import (
"archive/zip" "archive/zip"
"cmp"
"context" "context"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"net/http"
"os" "os"
"path"
"path/filepath" "path/filepath"
"strconv"
"strings" "strings"
"time"
"github.com/Masterminds/squirrel" "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/stream" "github.com/navidrome/navidrome/core/stream"
"github.com/navidrome/navidrome/log" "github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/utils/slice" "github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/utils/str" "github.com/navidrome/navidrome/utils/str"
) )
const archiveCoverArtSize = 500
type Archiver interface { type Archiver interface {
ZipAlbum(ctx context.Context, id string, format string, bitrate int, w io.Writer) error ZipAlbum(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
ZipArtist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error ZipArtist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
ZipShare(ctx context.Context, s *model.Share, w io.Writer) error ZipShare(ctx context.Context, id string, w io.Writer) error
ZipPlaylist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error ZipPlaylist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
} }
func NewArchiver(ms stream.MediaStreamer, decider stream.TranscodeDecider, ds model.DataStore, shares Share, artwork artwork.Artwork) Archiver { func NewArchiver(ms stream.MediaStreamer, ds model.DataStore, shares Share) Archiver {
return &archiver{ds: ds, ms: ms, decider: decider, shares: shares, artwork: artwork} return &archiver{ds: ds, ms: ms, shares: shares}
} }
type archiver struct { type archiver struct {
ds model.DataStore ds model.DataStore
ms stream.MediaStreamer ms stream.MediaStreamer
decider stream.TranscodeDecider shares Share
shares Share
artwork artwork.Artwork
} }
func (a *archiver) ZipAlbum(ctx context.Context, id string, format string, bitrate int, out io.Writer) error { func (a *archiver) ZipAlbum(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
return a.zipAlbums(ctx, id, format, bitrate, out, squirrel.Eq{"album_id": id}, model.ArtworkID{}) return a.zipAlbums(ctx, id, format, bitrate, out, squirrel.Eq{"album_id": id})
} }
func (a *archiver) ZipArtist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error { func (a *archiver) ZipArtist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
// Match by album-artist participation, not the deprecated album_artist_id return a.zipAlbums(ctx, id, format, bitrate, out, squirrel.Eq{"album_artist_id": id})
// column (first album artist only), so co-album-artists are included too.
filter := squirrel.And{
persistence.ParticipantIDFilter("media_file", id, model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}
return a.zipAlbums(ctx, id, format, bitrate, out, filter, model.Artist{ID: id}.CoverArtID())
} }
// rootArt, when set, is added to the archive root. func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitrate int, out io.Writer, filters squirrel.Sqlizer) error {
func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitrate int, out io.Writer, filters squirrel.Sqlizer, rootArt model.ArtworkID) error { mfs, err := a.ds.MediaFile(ctx).GetAll(model.QueryOptions{Filters: filters, Sort: "album"})
mfs, err := a.ds.MediaFile().GetAll(ctx, model.QueryOptions{Filters: filters, Sort: "album"})
if err != nil { if err != nil {
log.Error(ctx, "Error loading mediafiles from artist", "id", id, err) log.Error(ctx, "Error loading mediafiles from artist", "id", id, err)
return err return err
} }
z := createZipWriter(out, format, bitrate) z := createZipWriter(out, format, bitrate)
albums := slice.GroupOrdered(mfs, func(mf model.MediaFile) string { return mf.AlbumID }) albums := slice.Group(mfs, func(mf model.MediaFile) string {
folders := albumFolders(albums) return mf.AlbumID
})
for _, album := range albums { for _, album := range albums {
discs := slice.Group(album, func(mf model.MediaFile) int { return mf.DiscNumber }) discs := slice.Group(album, func(mf model.MediaFile) int { return mf.DiscNumber })
isMultiDisc := len(discs) > 1 isMultiDisc := len(discs) > 1
folder := folders[album[0].AlbumID] log.Debug(ctx, "Zipping album", "name", album[0].Album, "artist", album[0].AlbumArtist,
log.Debug(ctx, "Zipping album", "name", album[0].Album, "artist", album[0].AlbumArtist, "folder", folder,
"format", format, "bitrate", bitrate, "isMultiDisc", isMultiDisc, "numTracks", len(album)) "format", format, "bitrate", bitrate, "isMultiDisc", isMultiDisc, "numTracks", len(album))
for _, mf := range album { for _, mf := range album {
req := a.resolveRequest(ctx, &mf, format, bitrate) file := a.albumFilename(mf, format, isMultiDisc)
file := a.albumFilename(mf, req.Format, isMultiDisc, folder) if addErr := a.addFileToZip(ctx, z, mf, format, bitrate, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
if addErr := a.addFileToZip(ctx, z, mf, req, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
// Stop iterating: continuing would just rack up more // Stop iterating: continuing would just rack up more
// rejections from the limiter. Close finalises whatever // rejections from the limiter. Close finalises whatever
// tracks were already written; the rejected one is not // tracks were already written; the rejected one is not
@ -91,10 +71,7 @@ func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitr
return addErr return addErr
} }
} }
// After the tracks, so a slow artwork lookup doesn't delay the first bytes.
a.addCoverArtToZip(ctx, z, album[0].AlbumCoverArtID(), folder)
} }
a.addCoverArtToZip(ctx, z, rootArt, "")
err = z.Close() err = z.Close()
if err != nil { if err != nil {
log.Error(ctx, "Error closing zip file", "id", id, err) log.Error(ctx, "Error closing zip file", "id", id, err)
@ -112,61 +89,7 @@ func createZipWriter(out io.Writer, format string, bitrate int) *zip.Writer {
return z return z
} }
// Tried in order; the first one whose values are distinct across the clashing albums wins. func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc bool) string {
// One album may have an empty value: it keeps the plain name, which the others can't clash with.
var albumDisambiguators = []func(model.MediaFile) string{
func(mf model.MediaFile) string { return mf.Tags.First(model.TagAlbumVersion) },
func(mf model.MediaFile) string {
// Reissues share Year (often the original's) but not ReleaseYear.
if y := cmp.Or(mf.ReleaseYear, mf.Year); y != 0 {
return strconv.Itoa(y)
}
return ""
},
func(mf model.MediaFile) string { return mf.MbzAlbumType },
func(mf model.MediaFile) string { return mf.Tags.First(model.TagRecordLabel) },
func(mf model.MediaFile) string { return mf.CatalogNum },
func(mf model.MediaFile) string { return mf.AlbumID[:min(6, len(mf.AlbumID))] },
func(mf model.MediaFile) string { return mf.AlbumID },
}
// albumFolders maps each album id to its zip folder. Albums whose names sanitize to the
// same folder get a " [suffix]" from the first disambiguator that tells them all apart.
func albumFolders(albums [][]model.MediaFile) map[string]string {
byName := map[string][]model.MediaFile{}
for _, album := range albums {
name := str.SanitizeFilename(album[0].FullAlbumName())
byName[name] = append(byName[name], album[0])
}
folders := make(map[string]string, len(albums))
for name, group := range byName {
if len(group) == 1 {
folders[group[0].AlbumID] = name
continue
}
fields:
for _, field := range albumDisambiguators {
ids := make(map[string]string, len(group)) // suffix -> album id
for _, mf := range group {
s := str.SanitizeFilename(field(mf))
if _, dup := ids[s]; dup {
continue fields
}
ids[s] = mf.AlbumID
}
for s, id := range ids {
folders[id] = name
if s != "" {
folders[id] = fmt.Sprintf("%s [%s]", name, s)
}
}
break
}
}
return folders
}
func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc bool, folder string) string {
_, file := filepath.Split(mf.Path) _, file := filepath.Split(mf.Path)
if format != "raw" { if format != "raw" {
file = strings.TrimSuffix(file, mf.Suffix) + format file = strings.TrimSuffix(file, mf.Suffix) + format
@ -174,41 +97,39 @@ func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc
if isMultiDisc { if isMultiDisc {
file = fmt.Sprintf("Disc %02d/%s", mf.DiscNumber, file) file = fmt.Sprintf("Disc %02d/%s", mf.DiscNumber, file)
} }
return fmt.Sprintf("%s/%s", folder, file) return fmt.Sprintf("%s/%s", str.SanitizeFilename(mf.Album), file)
} }
// ZipShare takes an already-loaded share: Share.Load records a visit, so func (a *archiver) ZipShare(ctx context.Context, id string, out io.Writer) error {
// loading it again here would count every download twice. s, err := a.shares.Load(ctx, id)
func (a *archiver) ZipShare(ctx context.Context, s *model.Share, out io.Writer) error { if err != nil {
return err
}
if !s.Downloadable { if !s.Downloadable {
return model.ErrNotAuthorized return model.ErrNotAuthorized
} }
log.Debug(ctx, "Zipping share", "name", s.ID, "format", s.Format, "bitrate", s.MaxBitRate, "numTracks", len(s.Tracks)) log.Debug(ctx, "Zipping share", "name", s.ID, "format", s.Format, "bitrate", s.MaxBitRate, "numTracks", len(s.Tracks))
// The share is the authorization (as in the public image handler): an anonymous lookup would return a.zipMediaFiles(ctx, id, s.ID, s.Format, s.MaxBitRate, out, s.Tracks, false)
// hide a private playlist. Only the cover read is elevated.
coverCtx := request.WithUser(ctx, model.User{IsAdmin: true})
return a.zipMediaFiles(ctx, s.ID, s.ID, s.Format, s.MaxBitRate, out, s.Tracks, coverCtx, s.CoverArtID(), false)
} }
func (a *archiver) ZipPlaylist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error { func (a *archiver) ZipPlaylist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
pls, err := a.ds.Playlist().GetWithTracks(ctx, id, true, false) pls, err := a.ds.Playlist(ctx).GetWithTracks(id, true, false)
if err != nil { if err != nil {
log.Error(ctx, "Error loading mediafiles from playlist", "id", id, err) log.Error(ctx, "Error loading mediafiles from playlist", "id", id, err)
return err return err
} }
mfs := pls.MediaFiles() mfs := pls.MediaFiles()
log.Debug(ctx, "Zipping playlist", "name", pls.Name, "format", format, "bitrate", bitrate, "numTracks", len(mfs)) log.Debug(ctx, "Zipping playlist", "name", pls.Name, "format", format, "bitrate", bitrate, "numTracks", len(mfs))
return a.zipMediaFiles(ctx, id, pls.Name, format, bitrate, out, mfs, ctx, pls.CoverArtID(), true) return a.zipMediaFiles(ctx, id, pls.Name, format, bitrate, out, mfs, true)
} }
func (a *archiver) zipMediaFiles(ctx context.Context, id, name string, format string, bitrate int, out io.Writer, mfs model.MediaFiles, coverCtx context.Context, coverArt model.ArtworkID, addM3U bool) error { func (a *archiver) zipMediaFiles(ctx context.Context, id, name string, format string, bitrate int, out io.Writer, mfs model.MediaFiles, addM3U bool) error {
z := createZipWriter(out, format, bitrate) z := createZipWriter(out, format, bitrate)
zippedMfs := make(model.MediaFiles, len(mfs)) zippedMfs := make(model.MediaFiles, len(mfs))
for idx, mf := range mfs { for idx, mf := range mfs {
req := a.resolveRequest(ctx, &mf, format, bitrate) file := a.playlistFilename(mf, format, idx)
file := a.playlistFilename(mf, req.Format, idx) if addErr := a.addFileToZip(ctx, z, mf, format, bitrate, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
if addErr := a.addFileToZip(ctx, z, mf, req, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
// Abort the whole archive: continuing would silently emit // Abort the whole archive: continuing would silently emit
// empty zip entries since the headers are already written. // empty zip entries since the headers are already written.
_ = z.Close() _ = z.Close()
@ -217,7 +138,6 @@ func (a *archiver) zipMediaFiles(ctx context.Context, id, name string, format st
mf.Path = file mf.Path = file
zippedMfs[idx] = mf zippedMfs[idx] = mf
} }
a.addCoverArtToZip(coverCtx, z, coverArt, "")
// Add M3U file if requested // Add M3U file if requested
if addM3U && len(zippedMfs) > 0 { if addM3U && len(zippedMfs) > 0 {
@ -254,14 +174,7 @@ func (a *archiver) playlistFilename(mf model.MediaFile, format string, idx int)
return fmt.Sprintf("%02d - %s - %s.%s", idx+1, str.SanitizeFilename(mf.Artist), str.SanitizeFilename(mf.Title), ext) return fmt.Sprintf("%02d - %s - %s.%s", idx+1, str.SanitizeFilename(mf.Artist), str.SanitizeFilename(mf.Title), ext)
} }
func (a *archiver) resolveRequest(ctx context.Context, mf *model.MediaFile, format string, bitrate int) stream.Request { func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.MediaFile, format string, bitrate int, filename string) error {
if format == "" || format == "raw" {
return stream.Request{Format: "raw"}
}
return a.decider.ResolveRequest(ctx, mf, format, bitrate, 0)
}
func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.MediaFile, req stream.Request, filename string) error {
path := mf.AbsolutePath() path := mf.AbsolutePath()
// Open the source before writing the zip entry header so a rejection // Open the source before writing the zip entry header so a rejection
@ -269,13 +182,13 @@ func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.Med
// archive. // archive.
var r io.ReadCloser var r io.ReadCloser
var err error var err error
if req.Format != "raw" { if format != "raw" && format != "" {
r, err = a.ms.NewStream(ctx, &mf, req) r, err = a.ms.NewStream(ctx, &mf, stream.Request{Format: format, BitRate: bitrate})
} else { } else {
r, err = os.Open(path) r, err = os.Open(path)
} }
if err != nil { if err != nil {
log.Error(ctx, "Error opening file for zipping", "file", path, "format", req.Format, err) log.Error(ctx, "Error opening file for zipping", "file", path, "format", format, err)
return err return err
} }
defer func() { defer func() {
@ -302,61 +215,3 @@ func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.Med
return nil return nil
} }
// addCoverArtToZip adds the cover as dir/folder.<ext>. Errors are logged, never returned.
func (a *archiver) addCoverArtToZip(ctx context.Context, z *zip.Writer, artID model.ArtworkID, dir string) {
if artID.ID == "" {
return
}
// Buffered so a failed read leaves no empty entry.
data, err := a.readCoverArt(ctx, artID)
if errors.Is(err, artwork.ErrUnavailable) || errors.Is(err, model.ErrNotFound) {
log.Debug(ctx, "No cover art to add to zip", "artID", artID)
return
}
if err != nil {
log.Warn(ctx, "Error reading cover art for zipping", "artID", artID, err)
return
}
ext := coverArtExtension(data)
if ext == "" {
log.Warn(ctx, "Unknown cover art image type, not adding it to zip", "artID", artID)
return
}
w, err := z.CreateHeader(&zip.FileHeader{
Name: path.Join(dir, "folder."+ext),
Modified: time.Now(),
Method: zip.Store,
})
if err != nil {
log.Warn(ctx, "Error creating cover art zip entry", "artID", artID, err)
return
}
if _, err = w.Write(data); err != nil {
log.Warn(ctx, "Error zipping cover art", "artID", artID, err)
}
}
func (a *archiver) readCoverArt(ctx context.Context, artID model.ArtworkID) ([]byte, error) {
img, err := a.artwork.Get(ctx, artID, archiveCoverArtSize, false)
if err != nil {
return nil, err
}
defer img.Close()
return io.ReadAll(img)
}
// Resizing may re-encode the image, so the type comes from its bytes.
func coverArtExtension(data []byte) string {
switch http.DetectContentType(data) {
case "image/jpeg":
return "jpg"
case "image/png":
return "png"
case "image/webp":
return "webp"
case "image/gif":
return "gif"
}
return ""
}

View file

@ -4,19 +4,13 @@ import (
"archive/zip" "archive/zip"
"bytes" "bytes"
"context" "context"
"errors"
"io" "io"
"strings" "strings"
"github.com/Masterminds/squirrel" "github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core" "github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/stream" "github.com/navidrome/navidrome/core/stream"
"github.com/navidrome/navidrome/model" "github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/persistence"
. "github.com/onsi/ginkgo/v2" . "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega" . "github.com/onsi/gomega"
"github.com/stretchr/testify/mock" "github.com/stretchr/testify/mock"
@ -26,19 +20,15 @@ var _ = Describe("Archiver", func() {
var ( var (
arch core.Archiver arch core.Archiver
ms *mockMediaStreamer ms *mockMediaStreamer
dc *fakeDecider
ds *mockDataStore ds *mockDataStore
sh *mockShare sh *mockShare
ca *mockCoverArt
) )
BeforeEach(func() { BeforeEach(func() {
ms = &mockMediaStreamer{} ms = &mockMediaStreamer{}
dc = &fakeDecider{}
sh = &mockShare{} sh = &mockShare{}
ds = &mockDataStore{} ds = &mockDataStore{}
ca = &mockCoverArt{images: map[string][]byte{}} arch = core.NewArchiver(ms, ds, sh)
arch = core.NewArchiver(ms, dc, ds, sh, ca)
}) })
Context("ZipAlbum", func() { Context("ZipAlbum", func() {
@ -54,7 +44,7 @@ var _ = Describe("Archiver", func() {
Sort: "album", Sort: "album",
}}).Return(mfs, nil) }}).Return(mfs, nil)
ds.On("MediaFile").Return(mfRepo) ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(3) ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(3)
out := new(bytes.Buffer) out := new(bytes.Buffer)
@ -68,23 +58,6 @@ var _ = Describe("Archiver", func() {
Expect(zr.File[0].Name).To(Equal("Album_Promo/01 - track1.mp3")) Expect(zr.File[0].Name).To(Equal("Album_Promo/01 - track1.mp3"))
Expect(zr.File[1].Name).To(Equal("Album_Promo/02 - track2.mp3")) Expect(zr.File[1].Name).To(Equal("Album_Promo/02 - track2.mp3"))
}) })
It("streams the request resolved by the transcode decider and names the entry after its format", func() {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", mock.Anything).Return(model.MediaFiles{{Path: "test_data/01 - track1.flac", Suffix: "flac", AlbumID: "1"}}, nil)
ds.On("MediaFile").Return(mfRepo)
resolved := stream.Request{Format: "opus", BitRate: 128, SampleRate: 48000, Channels: 2}
dc.resolved = &resolved
ms.On("NewStream", mock.Anything, mock.Anything, resolved).Return(io.NopCloser(strings.NewReader("test")), nil).Once()
out := new(bytes.Buffer)
Expect(arch.ZipAlbum(GinkgoT().Context(), "1", "mp3", 128, out)).To(Succeed())
ms.AssertExpectations(GinkgoT())
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).ToNot(HaveOccurred())
Expect(zr.File[0].Name).To(HaveSuffix("01 - track1.opus"))
})
}) })
Context("ZipArtist", func() { Context("ZipArtist", func() {
@ -96,14 +69,11 @@ var _ = Describe("Archiver", func() {
mfRepo := &mockMediaFileRepository{} mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", []model.QueryOptions{{ mfRepo.On("GetAll", []model.QueryOptions{{
Filters: squirrel.And{ Filters: squirrel.Eq{"album_artist_id": "1"},
persistence.ParticipantIDFilter("media_file", "1", model.RoleAlbumArtist), Sort: "album",
squirrel.Eq{"missing": false},
},
Sort: "album",
}}).Return(mfs, nil) }}).Return(mfs, nil)
ds.On("MediaFile").Return(mfRepo) ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
out := new(bytes.Buffer) out := new(bytes.Buffer)
@ -117,140 +87,6 @@ var _ = Describe("Archiver", func() {
Expect(zr.File[0].Name).To(Equal("Album 1/01 - track1.mp3")) Expect(zr.File[0].Name).To(Equal("Album 1/01 - track1.mp3"))
Expect(zr.File[1].Name).To(Equal("Album 1/02 - track2.mp3")) Expect(zr.File[1].Name).To(Equal("Album 1/02 - track2.mp3"))
}) })
When("albums that share a name", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
// zipArtistEntries zips the given tracks as artist "1" and returns the entry names in zip order.
zipArtistEntries := func(mfs model.MediaFiles) []string {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", mock.Anything).Return(mfs, nil)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
out := new(bytes.Buffer)
Expect(arch.ZipArtist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).To(BeNil())
names := make([]string, len(zr.File))
for i, f := range zr.File {
names[i] = f.Name
}
return names
}
It("keeps the albums in query order", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "3", Album: "Album C"},
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album A"},
{Path: "a/02.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album A"},
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Album B"},
})
Expect(names).To(Equal([]string{"Album C/01.mp3", "Album A/01.mp3", "Album A/02.mp3", "Album B/01.mp3"}))
})
It("suffixes the year when it tells the albums apart", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01 - Intro.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01 - Intro.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005},
})
Expect(names).To(Equal([]string{"Greatest Hits [2001]/01 - Intro.mp3", "Greatest Hits [2005]/01 - Intro.mp3"}))
})
It("prefers the release year, so reissues of the same original are told apart", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 1996, ReleaseYear: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 1996, ReleaseYear: 2011},
{Path: "c/01.mp3", Suffix: "mp3", AlbumID: "3", Album: "Greatest Hits", Year: 1996},
})
Expect(names).To(Equal([]string{"Greatest Hits [2001]/01.mp3", "Greatest Hits [2011]/01.mp3", "Greatest Hits [1996]/01.mp3"}))
})
It("names the folder after the full album name", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001,
Tags: model.Tags{model.TagAlbumVersion: {"Original"}}},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005,
Tags: model.Tags{model.TagAlbumVersion: {"CD/Digital"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits (Original)/01.mp3", "Greatest Hits (CD_Digital)/01.mp3"}))
})
It("prefers the album version over the year when it is not part of the name", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001,
Tags: model.Tags{model.TagAlbumVersion: {"Original"}}},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005,
Tags: model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits [Original]/01.mp3", "Greatest Hits [Deluxe Edition]/01.mp3"}))
})
It("leaves the one album without the field unsuffixed", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005,
Tags: model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits/01.mp3", "Greatest Hits [Deluxe Edition]/01.mp3"}))
})
It("skips a field that is empty on more than one album", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005},
{Path: "c/01.mp3", Suffix: "mp3", AlbumID: "3", Album: "Greatest Hits", Year: 2010,
Tags: model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits [2001]/01.mp3", "Greatest Hits [2005]/01.mp3", "Greatest Hits [2010]/01.mp3"}))
})
It("skips a field that is the same on every album", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Live", Year: 2001, MbzAlbumType: "album", CatalogNum: "CAT-1"},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Live", Year: 2001, MbzAlbumType: "album", CatalogNum: "CAT-2"},
})
Expect(names).To(Equal([]string{"Live [CAT-1]/01.mp3", "Live [CAT-2]/01.mp3"}))
})
It("falls back to the album id when nothing differs", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "0123456789abcdef", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "fedcba9876543210", Album: "Greatest Hits", Year: 2001},
})
Expect(names).To(Equal([]string{"Greatest Hits [012345]/01.mp3", "Greatest Hits [fedcba]/01.mp3"}))
})
It("treats names that sanitize to the same folder as a clash", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "A/B", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: `A\B`, Year: 2005},
})
Expect(names).To(Equal([]string{"A_B [2001]/01.mp3", "A_B [2005]/01.mp3"}))
})
It("sanitizes the suffix", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Hits", Tags: model.Tags{model.TagAlbumVersion: {"Vinyl"}}},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Hits", Tags: model.Tags{model.TagAlbumVersion: {"CD/Digital"}}},
})
Expect(names).To(Equal([]string{"Hits [Vinyl]/01.mp3", "Hits [CD_Digital]/01.mp3"}))
})
It("leaves the folder name alone when only one album has it", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Other", Year: 2005},
})
Expect(names).To(Equal([]string{"Greatest Hits/01.mp3", "Other/01.mp3"}))
})
})
}) })
Context("when the transcode limiter rejects a file", func() { Context("when the transcode limiter rejects a file", func() {
@ -265,7 +101,7 @@ var _ = Describe("Archiver", func() {
Filters: squirrel.Eq{"album_id": "1"}, Filters: squirrel.Eq{"album_id": "1"},
Sort: "album", Sort: "album",
}}).Return(mfs, nil) }}).Return(mfs, nil)
ds.On("MediaFile").Return(mfRepo) ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}). ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).
Return(nil, stream.ErrTooManyTranscodes).Once() Return(nil, stream.ErrTooManyTranscodes).Once()
@ -294,16 +130,13 @@ var _ = Describe("Archiver", func() {
Tracks: mfs, Tracks: mfs,
} }
sh.On("Load", mock.Anything, "1").Return(share, nil)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
out := new(bytes.Buffer) out := new(bytes.Buffer)
err := arch.ZipShare(context.Background(), share, out) err := arch.ZipShare(context.Background(), "1", out)
Expect(err).To(BeNil()) Expect(err).To(BeNil())
// Share.Load records a visit; re-loading here would double-count
// every download.
sh.AssertNotCalled(GinkgoT(), "Load", mock.Anything, mock.Anything)
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len())) zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).To(BeNil()) Expect(err).To(BeNil())
@ -315,30 +148,6 @@ var _ = Describe("Archiver", func() {
}) })
Context("ZipPlaylist", func() { Context("ZipPlaylist", func() {
It("names the entries and the M3U lines after the resolved format", func() {
pls := &model.Playlist{ID: "1", Name: "Test Playlist", Tracks: []model.PlaylistTrack{
{MediaFile: model.MediaFile{Path: "test_data/01 - track1.flac", Suffix: "flac", Artist: "Artist 1", Title: "track1"}},
}}
plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil)
ds.On("Playlist").Return(plRepo)
dc.resolved = &stream.Request{Format: "opus", BitRate: 128}
ms.On("NewStream", mock.Anything, mock.Anything, *dc.resolved).Return(io.NopCloser(strings.NewReader("test")), nil)
out := new(bytes.Buffer)
Expect(arch.ZipPlaylist(GinkgoT().Context(), "1", "mp3", 128, out)).To(Succeed())
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).ToNot(HaveOccurred())
Expect(zr.File[0].Name).To(Equal("01 - Artist 1 - track1.opus"))
m3u, err := zr.File[1].Open()
Expect(err).ToNot(HaveOccurred())
defer m3u.Close()
content, err := io.ReadAll(m3u)
Expect(err).ToNot(HaveOccurred())
Expect(string(content)).To(ContainSubstring("01 - Artist 1 - track1.opus"))
})
It("zips a playlist correctly", func() { It("zips a playlist correctly", func() {
tracks := []model.PlaylistTrack{ tracks := []model.PlaylistTrack{
{MediaFile: model.MediaFile{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album 1", DiscNumber: 1, Artist: "AC/DC", Title: "track1"}}, {MediaFile: model.MediaFile{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album 1", DiscNumber: 1, Artist: "AC/DC", Title: "track1"}},
@ -353,7 +162,7 @@ var _ = Describe("Archiver", func() {
plRepo := &mockPlaylistRepository{} plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil) plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil)
ds.On("Playlist").Return(plRepo) ds.On("Playlist", mock.Anything).Return(plRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2) ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
out := new(bytes.Buffer) out := new(bytes.Buffer)
@ -380,195 +189,24 @@ var _ = Describe("Archiver", func() {
Expect(string(m3uContent)).To(Equal(expectedM3U)) Expect(string(m3uContent)).To(Equal(expectedM3U))
}) })
}) })
Context("cover art", func() {
var (
jpegData = []byte("\xff\xd8\xff\xe0 fake jpeg")
pngData = []byte("\x89PNG\x0d\x0a\x1a\x0a fake png")
)
mockAlbumTracks := func(filter squirrel.Sqlizer, mfs model.MediaFiles) {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", []model.QueryOptions{{Filters: filter, Sort: "album"}}).Return(mfs, nil)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
}
It("adds the album cover to the album folder", func() {
ca.images["al-1"] = jpegData
mockAlbumTracks(squirrel.Eq{"album_id": "1"}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album/Promo", DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipAlbum(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(2))
Expect(files).To(HaveKeyWithValue("Album_Promo/folder.jpg", jpegData))
Expect(ca.requests).To(ConsistOf(coverRequest{id: "al-1", size: 500, square: false}))
})
It("adds the artist image to the root and each album cover to its folder", func() {
ca.images["ar-1"] = pngData
ca.images["al-1"] = jpegData
ca.images["al-2"] = jpegData
mockAlbumTracks(squirrel.And{
persistence.ParticipantIDFilter("media_file", "1", model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album 1", DiscNumber: 1},
{Path: "test_data/02 - track2.mp3", Suffix: "mp3", AlbumID: "2", Album: "Album 2", DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipArtist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(5))
Expect(files).To(HaveKeyWithValue("folder.png", pngData))
Expect(files).To(HaveKeyWithValue("Album 1/folder.jpg", jpegData))
Expect(files).To(HaveKeyWithValue("Album 2/folder.jpg", jpegData))
})
It("puts each same-named album's cover in that album's own folder", func() {
ca.images["al-1"] = jpegData
ca.images["al-2"] = pngData
mockAlbumTracks(squirrel.And{
persistence.ParticipantIDFilter("media_file", "1", model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001, DiscNumber: 1},
{Path: "test_data/02 - track2.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005, DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipArtist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveKeyWithValue("Greatest Hits [2001]/folder.jpg", jpegData))
Expect(files).To(HaveKeyWithValue("Greatest Hits [2005]/folder.png", pngData))
})
It("adds the playlist cover to the root", func() {
ca.images["pl-1"] = jpegData
plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true, false).Return(&model.Playlist{
ID: "1",
Name: "Test Playlist",
Tracks: []model.PlaylistTrack{
{MediaFile: model.MediaFile{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Artist: "Artist 1", Title: "track1"}},
},
}, nil)
ds.On("Playlist", mock.Anything).Return(plRepo)
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
out := new(bytes.Buffer)
Expect(arch.ZipPlaylist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(3))
Expect(files).To(HaveKeyWithValue("folder.jpg", jpegData))
Expect(files).To(HaveKey("Test Playlist.m3u"))
})
It("adds the shared item's cover to the root, even for a private playlist", func() {
ca.images["pl-10"] = jpegData
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
share := &model.Share{
ID: "1",
Downloadable: true,
Format: "mp3",
MaxBitRate: 128,
ResourceType: "playlist",
ResourceIDs: "10",
Tracks: model.MediaFiles{
{ID: "1", Path: "test_data/01 - track1.mp3", Suffix: "mp3", Artist: "Artist 1", Title: "track1"},
},
}
out := new(bytes.Buffer)
Expect(arch.ZipShare(context.Background(), share, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(2))
Expect(files).To(HaveKeyWithValue("folder.jpg", jpegData))
Expect(ca.requests).To(ConsistOf(coverRequest{id: "pl-10", size: 500, square: false, admin: true}))
})
It("still builds the archive when the cover cannot be read", func() {
ca.err = errors.New("boom")
mockAlbumTracks(squirrel.Eq{"album_id": "1"}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album", DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipAlbum(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(1))
Expect(files).To(HaveKey("Album/01 - track1.mp3"))
})
})
}) })
func readZip(out *bytes.Buffer) map[string][]byte {
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).ToNot(HaveOccurred())
files := make(map[string][]byte, len(zr.File))
for _, f := range zr.File {
r, err := f.Open()
Expect(err).ToNot(HaveOccurred())
data, err := io.ReadAll(r)
Expect(err).ToNot(HaveOccurred())
_ = r.Close()
files[f.Name] = data
}
return files
}
type coverRequest struct {
id string
size int
square bool
admin bool
}
type mockCoverArt struct {
artwork.Artwork
images map[string][]byte
err error
requests []coverRequest
}
func (m *mockCoverArt) Get(ctx context.Context, artID model.ArtworkID, size int, square bool) (*artwork.Image, error) {
user, _ := request.UserFrom(ctx)
m.requests = append(m.requests, coverRequest{id: artID.String(), size: size, square: square, admin: user.IsAdmin})
if m.err != nil {
return nil, m.err
}
data, ok := m.images[artID.String()]
if !ok {
return nil, artwork.ErrUnavailable
}
return &artwork.Image{ReadCloser: io.NopCloser(bytes.NewReader(data))}, nil
}
type mockDataStore struct { type mockDataStore struct {
mock.Mock mock.Mock
model.DataStore model.DataStore
} }
func (m *mockDataStore) MediaFile() model.MediaFileRepository { func (m *mockDataStore) MediaFile(ctx context.Context) model.MediaFileRepository {
args := m.Called() args := m.Called(ctx)
return args.Get(0).(model.MediaFileRepository) return args.Get(0).(model.MediaFileRepository)
} }
func (m *mockDataStore) Playlist() model.PlaylistRepository { func (m *mockDataStore) Playlist(ctx context.Context) model.PlaylistRepository {
args := m.Called() args := m.Called(ctx)
return args.Get(0).(model.PlaylistRepository) return args.Get(0).(model.PlaylistRepository)
} }
func (m *mockDataStore) Library() model.LibraryRepository { func (m *mockDataStore) Library(context.Context) model.LibraryRepository {
return &mockLibraryRepository{} return &mockLibraryRepository{}
} }
@ -577,7 +215,7 @@ type mockLibraryRepository struct {
model.LibraryRepository model.LibraryRepository
} }
func (m *mockLibraryRepository) GetPath(_ context.Context, id int) (string, error) { func (m *mockLibraryRepository) GetPath(id int) (string, error) {
return "/music", nil return "/music", nil
} }
@ -586,7 +224,7 @@ type mockMediaFileRepository struct {
model.MediaFileRepository model.MediaFileRepository
} }
func (m *mockMediaFileRepository) GetAll(ctx context.Context, options ...model.QueryOptions) (model.MediaFiles, error) { func (m *mockMediaFileRepository) GetAll(options ...model.QueryOptions) (model.MediaFiles, error) {
args := m.Called(options) args := m.Called(options)
return args.Get(0).(model.MediaFiles), args.Error(1) return args.Get(0).(model.MediaFiles), args.Error(1)
} }
@ -596,7 +234,7 @@ type mockPlaylistRepository struct {
model.PlaylistRepository model.PlaylistRepository
} }
func (m *mockPlaylistRepository) GetWithTracks(_ context.Context, id string, refreshSmartPlaylists, includeMissing bool) (*model.Playlist, error) { func (m *mockPlaylistRepository) GetWithTracks(id string, refreshSmartPlaylists, includeMissing bool) (*model.Playlist, error) {
args := m.Called(id, refreshSmartPlaylists, includeMissing) args := m.Called(id, refreshSmartPlaylists, includeMissing)
return args.Get(0).(*model.Playlist), args.Error(1) return args.Get(0).(*model.Playlist), args.Error(1)
} }
@ -614,19 +252,6 @@ func (m *mockMediaStreamer) NewStream(ctx context.Context, mf *model.MediaFile,
return &stream.Stream{ReadCloser: args.Get(0).(io.ReadCloser)}, nil return &stream.Stream{ReadCloser: args.Get(0).(io.ReadCloser)}, nil
} }
// fakeDecider echoes the legacy format/bitrate unless a resolved request is set.
type fakeDecider struct {
stream.TranscodeDecider
resolved *stream.Request
}
func (f *fakeDecider) ResolveRequest(_ context.Context, _ *model.MediaFile, format string, bitRate int, offset int) stream.Request {
if f.resolved != nil {
return *f.resolved
}
return stream.Request{Format: format, BitRate: bitRate, Offset: offset}
}
type mockShare struct { type mockShare struct {
mock.Mock mock.Mock
core.Share core.Share

Some files were not shown because too many files have changed in this diff Show more