mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-27 02:37:21 +01:00
29 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Deluan Quintão
|
237276efcd |
fix(artwork): block private and loopback addresses in remote image fetches (#6181)
* fix(artwork): block private and loopback addresses in remote image fetches fromURL fetched any URL with a plain HTTP client, and two untrusted inputs reach it. A playlist can set #EXTALBUMARTURL to an http(s) URL, which the artwork worker later fetches when EnableM3UExternalAlbumArt is on, so any user who can import a playlist controls the target. Metadata agents, including WASM plugins without the http permission, return image URLs that the core fetches too. Either path could make the server request loopback, LAN or link-local addresses and store the response as artwork that is served back. Add httpclient.NewExternal, which dials through a net.Dialer Control hook that rejects private, loopback, link-local and unspecified addresses. The check runs at dial time on the resolved IP, so DNS names, redirects and DNS rebinding are covered. fromURL now uses one shared client built with it and treats a refused address as a definitive miss, so the item settles absent instead of retrying and tripping the agent's circuit breaker. httpclient.New is unchanged for the other callers. The IP classification moves from plugins to the new utils/netguard package, shared by the plugin host client and the new constructor. The artwork test suite swaps in a client that allows loopback so existing specs can keep using httptest servers; the fromURL specs use the production client to assert the refusal. * fix(httpclient): keep dialing a configured proxy in the guarded client The guard runs on the resolved address, and with HTTP_PROXY set that address is the proxy, not the image host. A proxy on a private address would have had every remote artwork fetch refused, and a refusal settles the item as absent, so covers would silently disappear for those setups. Dial the configured proxy endpoint directly and keep the guard for every other dial. A proxy relays the request itself, so it is the operator's egress policy, the same one every other httpclient.New caller already goes through. * fix(httpclient): exempt only the hop that actually goes through the proxy The exemption matched any dial to a configured proxy's address, but net/http never proxies loopback targets, so a URL aimed at a loopback proxy was dialed directly and skipped the guard. That let an image URL reach that one address. Tag each request with the proxy it resolves to and exempt a dial only when it is that hop. Redirects re-enter the RoundTripper, so every hop is tagged on its own. |
||
|
Deluan Quintão
|
8b4125267e |
fix(artwork): only use image files as local artwork sources (#6180)
* fix(playlists): limit local cover paths to images in owner's libraries A local #EXTALBUMARTURL path (absolute or file://) was only checked against the union of all libraries. The artwork resolver then opened it with no further check and served the bytes as the playlist cover, undecoded. Any user who can upload an M3U could read any file under any library root, including libraries they were not granted, through getCoverArt (GHSA-vwq6-xrw5-phpg). resolveImageURL now requires an image extension, and for uploaded playlists (no folder) the library holding the cover must pass the owner's HasLibraryAccess. Scanner and CLI imports keep the all-libraries check, since those files are admin-controlled. resolveLocalFile, used by every file-backed artwork source, now ignores paths without an image extension, which covers playlists stored before this fix that were not resolved yet. openOriginal refuses a stored file-backed row whose path is not an image, so the existing dangling path re-resolves it and the playlist falls back to the generated grid. No migration is needed. * fix(artwork): skip non-image files matched by folder cover patterns Album and disc folder sources opened any file in the folder's image list that matched a cover pattern, without checking its extension. openOriginal now refuses to serve file-backed rows whose path is not an image, so a stored row like that would be refused, re-resolved to the same file, and refused again on every view. The list comes from the scanner, which only records image files, but a database scanned where the OS mime table knows more image types than the serving process could still reach this. Both fromExternalFile variants now skip matches that are not image files, so the album falls back to its next source instead. Also correct the parser comment: a playlist without a folder can come from an API upload or from a CLI import of a file outside all libraries. * fix(artwork): check stored source type before using the resize cache The image-extension check for file-backed rows ran inside openOriginal, which the resize cache skips on a hit. Before the fix, a resized request for a playlist pointing at a non-image file cached the raw bytes, because a failed resize falls back to the original data. After the upgrade the same request still hit that entry and returned the file. serveHash now refuses a file-backed row whose path is not an image before calling serveSource, so both full-size and resized requests go through dangling and re-resolve the item. The stale cache entry is keyed by the old hash and is no longer reachable once the row changes. * test: register mime_types.yaml in test binaries Artwork resolution now skips candidates that are not image files, and model.IsImageFile answers from the process mime table. The server registers the extra image types from resources/mime_types.yaml through a conf hook, but a test binary only does that if it links conf/mime, so the artwork e2e suite fell back to the host table: .jxl resolves on macOS and Linux and does not on Windows, where the #5950 cover spec then found no source. tests.Init now imports conf/mime for its side effect, so every suite that loads the test config sees the same image types as the server. * fix(artwork): drop the image-file guard from the disc art reader The guard was added to both fromExternalFile variants, but disc artwork keeps no state row and is never queued, so it cannot hit the refuse-and-re-resolve loop the guard exists to prevent. The only case where it can fire is a real image whose extension this process's mime table does not know, and there it drops a disc cover that used to work. The album variant keeps the guard, since those resolutions are stored and re-served. |
||
|
Deluan Quintão
|
9ff0058620 |
fix: assorted scanner, plugin, and server fixes from the Go 1.27 work (#6050)
* fix(plugins): stop the cache janitor when a plugin cache is dropped
newCacheService started a ttlcache janitor goroutine that only stopped via the
explicit Close() path, so a cache service that was discarded without being closed
leaked its janitor for the process lifetime. It now registers the same
runtime.AddCleanup safety net that utils/cache.simpleCache already uses.
* fix(scanner): stop splitting multi-byte characters when truncating tags
sanitize() capped tag values with a byte slice, so a value whose limit falls in
the middle of a multi-byte character was stored as invalid UTF-8. defaultMaxTagLength
is 1024, which is not a multiple of 3, so any sufficiently long CJK title hit this.
Only trailing invalid bytes are trimmed, leaving bad bytes elsewhere in the value
untouched.
* fix(scanner): store MusicBrainz ids in their canonical form
uuid.Parse accepts a UUID wrapped in any two bytes, as well as braced and urn:
forms, but sanitize() returned the raw string. A tag like {<mbid>} or a quoted
value was therefore persisted with its wrapper into the mbz_* columns, where the
exact-match MBID search can never find it. The parsed value is now stored, which
also lowercases uppercase ids and adds the dashes to unhyphenated ones.
* fix(plugins): parse IPv6 hosts correctly in the websocket allowlist
isHostAllowed cut the host at the last colon, which mangles an IPv6 literal:
"[::1]:8080" became "[::1]" and "[::1]" became "[:". A plugin manifest could
therefore never allow an IPv6 host. It now uses net.SplitHostPort, falling back to
unwrapping the brackets when there is no port.
* fix(server): serve pprof profiles when a BaseURL is configured
net/http/pprof's Index resolves the profile name by trimming "/debug/pprof/" from
the raw request path, which never matches once MountRouter prepends the BasePath.
Requests for any profile without an explicit chi route fell through to the index
page, returning HTML with a 200 instead of the profile. The handler now strips the
BasePath first.
* test(scanner): run the goroutine leak check unconditionally
The scanner suite's goleak check only ran when the GOLEAK env var was set, so it
never ran in CI and could not catch a regression. It passes with the existing
ignore list, verified over repeated runs, so the gate is removed.
* fix(server): close the background image body on a non-200 response
serveImage returned early on an unexpected status code without closing the response
body, pinning the connection until the 5s client timeout. The nolint:bodyclose
above the request suppressed the linter that would have caught it, and its
justification only holds on the success path, where the body is handed to the
CachedStream wrapper.
* test(scanner): repair BenchmarkScan so it can actually run
The benchmark failed three ways before reaching its first iteration: it reused a
shared temp DB and tried to repoint the default library, it never loaded the config
defaults so the scanner got a concurrency of 0, and it lacked the notify ignore that
the suite already carries. tests.Init now takes a testing.TB so a benchmark can load
the test config the same way the suites do.
* refactor(artwork): drop the unused sourceFunc Stringer
sourceFunc.String derived a label from the closure's symbol name via reflection, but
nothing called it: the trace output builds its candidate labels from explicit strings.
Whole-program analysis confirms it is unreachable, and dropping it removes a
reflection-based dependency on compiler closure-naming details.
* refactor(plugins): reuse extractHostname in the websocket allowlist
The IPv6 host parsing added for isHostAllowed duplicated extractHostname, which
already lives in the same package and backs the HTTP client's identical allowlist
check. Two copies of a security-relevant parser can drift, so the websocket service
now calls the existing helper. The port-stripping specs move into the URL Validation
block that already covered them.
* perf(scanner): bound the tag truncation trim to a partial rune
The trim loop dropped every trailing byte that failed to decode, so a value ending
in a long run of invalid bytes was walked one byte at a time: a 1 MiB lyrics tag
measured 2.58ms against 45ns for a normal cut. A partial rune is at most 3 trailing
bytes, so the loop is capped there, which also stops it consuming a pre-existing
invalid run.
* test: tighten the tests added with the Go 1.27 bugfixes
Drop the testItem stub in favour of the package's own cacheKey, register the pprof
test profile once at package scope, and replace the hand-rolled goroutine settle
loop with Eventually. Also corrects a comment that credited a TestMain the scanner
suite does not have.
* test(scanner): ignore notify's nonrecursive-tree goroutines on Linux
The goroutine leak check only ignored the recursive tree (macOS/FSEvents).
Linux CI uses inotify, whose nonrecursive tree leaks dispatch and internal
goroutines after Stop(), failing the check.
* fix(scanner): avoid a truncation panic when MaxLength is 1 or 2
A value of only UTF-8 continuation bytes drained the partial-rune loop to
empty, then sliced value[:-1] and panicked. Break when DecodeLastRune returns
size 0 (empty string) by testing size != 1 instead of size > 1.
* fix: address Codex review on the pprof base path and scan benchmark
- profilerHandler: treat a root BasePath ("/") as no prefix, so http.StripPrefix
keeps the leading slash chi needs; without this the profiler 404s when BaseURL
is "/". Cover the root case in the test.
- BenchmarkScan: make it run regardless of test/benchmark ordering. Add
singleton.DeleteInstance so a fresh DB is opened after TestScanner closes the
shared one, guard driver registration with sync.Once so the rebuild does not
re-Register, and ignore the Ginkgo interrupt-handler and Linux notify
goroutines the preceding suite leaves behind.
* fix: address Codex round 2 on BasePath trailing slash and benchmark DB cleanup
- profilerHandler: trim all trailing slashes (TrimRight), not just a bare "/", so
a BaseURL like "/music/" strips correctly instead of 404ing. Cover it in the test.
- BenchmarkScan: keep and defer db.Init's closer so the DB is closed before
b.TempDir cleanup, which otherwise cannot delete the open SQLite/WAL files on Windows.
|
||
|
Deluan
|
a9962ebe5d |
refactor(artwork): use the shared httpclient for image downloads
Same behavior: httpclient.New sets the Navidrome User-Agent via its transport, so the manual header is no longer needed. |
||
|
Deluan Quintão
|
dc40bcaf80 |
feat(cli): add an artwork command group for diagnosing and re-driving artwork (#5957)
* feat(artwork): add a resolution chain trace collector
* feat(artwork): trace the local priority chain
* fix(artwork): record priority candidates the chain never evaluated
* refactor(artwork): report never-evaluated candidates as skipped
* feat(artwork): trace external agents at the gate seam
* feat(artwork): add repository queries to enqueue by current source
* feat(artwork): expose a tracing resolver for the CLI
* feat(artwork): read a single queue row by item
The explain CLI must report whether an item is queued, at what priority and when it
retries; the queue repository could only be drained in eligibility batches, which
cannot see a row that is still backing off.
* feat(cli): add artwork explain
Prints why an item has the artwork it has: the stored state, its queue row, the
governing config, the resolver's priority-chain walk and the verdict. Offline by
default so a diagnostic run cannot add load to an external provider; --live asks
the agents for real. Playlists and radios do not walk a priority chain, so they
report that instead of an empty chain table.
* fix(artwork): trace an external tier that never reaches an agent
A configured 'external' token vanished from the chain when no enabled agent provided
images for that entity type, and for synthetic artists, leaving the trace unable to
say whether the tier was even considered.
* fix(cli): never state an artwork outcome the walk did not observe
A transient external failure traced as 'error' fell through to 'not resolved', which
is the most common state behind a missing-artwork report. It is now indeterminate, and
an offline win that a skipped higher-priority external candidate could have taken says
so instead of naming a winner the live chain might not pick.
* feat(cli): add artwork refresh
* feat(cli): add artwork reprocess
Bulk re-enqueues artwork by kind and/or by the source an item currently
resolves from, previewing the matched count and confirming before queueing.
The preview counts with CountBySource (rows matched) and reports separately
what EnqueueBySource inserted: its DO NOTHING conflict policy leaves an
already-queued row untouched, so the two numbers differ and the output must
not claim the skipped rows were re-queued.
An unknown --source is rejected against the sources present in item_artwork,
rather than silently matching nothing and printing a reassuring 0.
* fix(cli): cover the reprocess selection rule and validate sources table-wide
The reconciliation that makes --source alone target every kind was only
exercised through runReprocess, which no test calls: mutating it to
`all := reprocessAll` left the suite green. It is now reprocessSelectsAll,
covered for all three selectors.
Scoping source validation to the selected kinds made the same well-formed
filter valid or invalid depending on which other kinds were selected, and its
error read the same for a typo as for a source that simply does not apply to
the chosen kind. Validation is now table-wide: a typo still aborts, while a
valid-but-inapplicable source falls through to "Nothing matches".
Also: the prompt now counts only the kinds that reach an external agent as
external cost, and --dry-run on an empty selection reports a dry run.
* fix(cli): cover the reprocess --yes guard and preview the external cost
Mutating the --yes check to `if true` left the suite green, so the one bypass
of the confirmation was unverified. The choice is now reprocessConfirm(yes, in),
covered in both directions.
The external estimate only reached the operator through the prompt, which
--dry-run skips — hiding the number in the one mode that exists to show it
before committing. The preview now carries it, and the prompt drops the clause
when no lookup will be made.
An empty selection says so again under --dry-run.
* feat(artwork): add read-only queue and absent counters
Both are needed by the artwork status CLI: a queue breakdown by kind and priority, and
the absent totals split against the recheck cutoff.
* feat(cli): add artwork status
Reports the queue, where artwork currently resolves from, absent counts against the 24h
recheck window, and the stored config fingerprint versus the current one — the line that
turns 'why is my server re-resolving everything?' into one command.
fingerprint() and staleAbsentAge are exported so the CLI reports the values backfill
itself compares, instead of a second copy of the formula that can silently drift.
* fix(cli): lead the artwork status backfill line with the queued backlog
By the time anyone runs a diagnostic, backfill has usually already stored the new
fingerprint, so 'up to date' was printed while thousands of items churned through external
providers. The backlog is the finding; the fingerprint is context.
Also echoes the config inputs the fingerprint covers, so a change can be traced to the
setting that caused it, and pins the rendered rows: the Absent values, the queue TOTAL and
a queue-scoped kind/priority pair were all unasserted, so kindName and priorityName were
effectively untested. FingerprintInputs is now the single listing ConfigFingerprint hashes;
a pinned hash proves the value did not change.
* refactor(artwork): export the trace outcome vocabulary
The CLI hardcoded the outcome literals and the "external:" prefix, so renaming a
constant's value in core/artwork left cmd compiling and the suite green while
`artwork explain` silently degraded its verdict.
Renaming a value now fails the golden vocabulary test in core/artwork and the
explainResult tests in cmd.
* fix(cli): keep the re-enqueue warning when a backfill is already running
A stale stored fingerprint with items already queued is the worst state the
system can be in: a second full re-enqueue is pending on top of the one running.
The line carried the weakest wording of the three, and was untested.
* refactor(artwork): drop the unreachable breaker branch from the tracing gate
--live wires the tracing gate straight to passthroughGate, so errBreakerOpen can
never reach it; the test only passed by injecting a fake gate.
* refactor(artwork): delete the never-emitted not-reached outcome
Candidates after the winner are lower priority and say nothing about why a source
won; the ones that matter sit above it and are already recorded.
* refactor(artwork): make the trace nil-safe in one place only
add already handles a nil trace, so record's own guard was dead; Steps was the
odd one out and would panic where every other method tolerates nil.
* refactor(artwork): export the trace types directly
ChainTrace and TraceStep were unexported types re-exported through aliases,
which existed only so the CLI had a name to refer to them by. The types are
public API — Resolver.Steps returns []TraceStep and the CLI constructs a
ChainTrace — so name them that way and drop the indirection.
Encapsulation is unchanged: add, mu and steps stay unexported, so only this
package can write a step.
* refactor(cli): simplify parseArtworkKind with slices.Contains
Replaces a nested loop and a manual append with slices.Contains and the
repo's slice.Map helper. Same behaviour, same error message.
* fix(cli): print the absent artwork source under the name --source accepts
`artwork explain` rendered the stored empty source as "(absent)", while
`artwork reprocess --source` only accepts "absent", so pasting what explain
printed straight back into reprocess was rejected as an unknown source.
* refactor(artwork): own the kind list and the chain predicate in the package
Export RecheckKinds and add WalksPriorityChain so the CLI stops keeping its
own copies of both, and unexport externalCandidate, which nothing outside the
package consumes.
* refactor(cli): drop the artwork command's duplicated state and formatting
Reuse artwork.RecheckKinds and artwork.WalksPriorityChain, extract
newTabWriter and externalEstimate, fold reprocessSelectsAll into
selectedKinds, and derive the queue total and the walks-chain flag instead of
carrying them in the report structs.
* test(persistence): drop two artwork-queue specs that cannot fail
One seeded hash and source together and then asserted the two counts agree,
so its setup guaranteed the result; the other repeated the count-does-not-
enqueue property already covered by the CountBySource spec.
* refactor(artwork): rename Resolver to TracingResolver for clarity
* fix(cli): count playlists in the artwork reprocess external estimate
The estimate used WalksPriorityChain, which is true only for artist and album,
so a playlist-only reprocess reported "External lookups: none" and the
confirmation prompt dropped the external-cost warning. Playlists do reach the
network: through the m3u ExternalImageURL fetch when EnableM3UExternalAlbumArt
is on, and — verified by test — through the generated grid, whose tiles resolve
album art via the full album priority chain.
Adds artwork.MayFetchExternal, a config-aware predicate for "can this kind's
resolver reach the network", and uses it for the estimate. WalksPriorityChain
keeps its separate job of deciding whether explain prints a chain block.
* fix(cli): estimate artwork reprocess external lookups per agent, not per item
The reprocess prompt billed one external lookup per externally-capable item.
fetchArtistImage/fetchAlbumImage try every enabled image agent and stop early
only on a hit, and resolvePlaylist can fetch the m3u image and then resolve up
to four sampled albums for the grid, each walking the album agents again. The
number the operator confirmed could understate real provider traffic several
fold, in the prompt whose whole job is to stop a provider flood.
ExternalLookupsPerItem now multiplies by the visible image-agent count and adds
the playlist grid factor. It stays a floor: the CLI never calls Manager.Start(),
so the plugin registry is empty and plugin-provided agents are dropped by
getEnabledAgentNames. On an install with 5 agents of which 3 are plugins the
count is well under the truth, so the wording is now "at least N" rather than
"up to N" — a zero visible count still bills one lookup for the same reason.
Fixing the plugin visibility is out of scope: Manager.Start() needs a Subsonic
router and writes to the DB via syncPlugins, breaking this command group's
read-only guarantee.
* fix(cli): state the artwork reprocess estimate as an estimate, not a bound
Neither bound is true. A ceiling is false because plugin agents are invisible to
a CLI that never starts the plugin manager, and a floor is false because a local
hit ends the walk before any agent is asked and a hit on the first agent skips
the rest. "at least N" traded one wrong claim for another.
The line now names its blind spots instead:
External lookups: ~340 estimated (plugin agents not counted; local hits may
need fewer).
The same line is reused in the confirmation prompt, and the zero case still
reads "External lookups: none." with the prompt dropping the clause entirely.
The count itself is unchanged.
* fix(cli): account for every configured agent in artwork explain
The Agents: line printed the raw config while the Chain only showed the agents the CLI could
construct, with nothing explaining the gap: plugin agents are never registered in a CLI that does
not start the plugin manager, and a built-in without credentials returns nil. Three of five agents
could vanish, including ones ranked above the one shown.
Also treat a live external error before the winning hit like the already-handled would-try case:
the resolver serves such a hit provisionally and retries later, so the verdict is indeterminate.
The Result line is still not qualified when an unavailable agent might have won; that needs agent
ranking, and is left to the follow-up that makes the CLI load plugin agents for real.
* fix(cli): do not call an external artwork win indeterminate
explainResult qualified the verdict whenever an external OutcomeError
appeared before the winning hit. When a later external agent returns an
image, fetchArtistImage/fetchAlbumImage discard the earlier error, so
extError is false: the worker settles the item and schedules no retry.
Telling the operator it may resolve differently on a retry was wrong.
The warning is only correct when a lower-priority local source won while
an external error was recorded, which is the case that carries extError.
* fix(cli): accept --source absent when nothing is currently absent
validateSources checks the requested sources against the ones item_artwork
actually uses, to catch a typo. The reserved empty source (spelled 'absent' on
the CLI) is a valid filter even when it matches nothing, so a scheduled
'artwork reprocess --source absent --yes' stopped working the moment the
library finished resolving. Treat it as intrinsically valid and let the
existing zero-match path report it.
* feat(artwork): explain disc and media file artwork from the CLI
`artwork explain` rejected `dc` and `mf` because it validated against RecheckKinds,
the list of kinds the backfill revisits. Those are different questions: a kind with no
recheck path still has artwork someone can report as wrong.
Disc artwork now walks DiscArtPriority under a trace, so explain reports which entry won
and why the others lost, including entries that map to no source at all (external is
unsupported, a disc with no subtitle, an album folder with no images). Media file artwork
traces its single embedded candidate, separating "EnableMediaFileCoverArt is off" from
"the track has no embedded art" — stored state cannot tell those apart.
Each command now validates against the kinds it can actually serve: explain takes all six,
refresh takes artwork.RefreshableKinds (which nativeapi now shares instead of keeping its
own copy), reprocess still takes RecheckKinds. Disc artwork stays out of refresh: the
worker cannot resolve it, so the queue row would be rejected on every drain.
WalksPriorityChain becomes Explainable, and ResolveArtist/ResolveAlbum collapse into
Resolve(kind, id).
* refactor(artwork): one disc-artwork walk for serving and explain
resolveDisc duplicated the loop selectImageReader already ran: try each source in
priority order, take the first that yields an image. The serving path and the CLI
diverged on two details as a result — only selectImageReader checked ctx between
candidates and logged each attempt.
Both now call discArtworkReader.selectImage, which takes the chainState the CLI already
uses for the other kinds. The serving path passes an untraced one, whose nil trace makes
recording a no-op. selectImageReader had no other caller and is gone.
The disc tests move from fromDiscArtPriority to discCandidates, so they assert the skip
reason for an entry that maps to no source rather than that it silently vanished, and
cancellation mid-walk is now covered.
* fix(artwork): reject a nil reader in the resize cache instead of panicking
resizedItem.Reader closes what open() hands back, so an open() that reports "no image"
as (nil, nil) rather than an error takes the request down with a nil-pointer panic. Every
caller returns an error today, and no test covered it: the resolution e2e harness stubs
the resize reader out entirely, so no e2e path reaches this code at all.
Guard it and cover Reader directly.
* refactor(artwork): move the keeps-state fact into core, drop a redundant guard
keepsArtworkState lived in package cmd and re-derived by hand what RefreshableKinds
already encodes: the same five-of-six kinds. It is now artwork.KeepsState, beside the
list, with a test pinning the two together — nothing else stopped them drifting, and a
drift would have explain report stored state for a kind that keeps none.
serveDisc's closure also hand-rolled a nil-reader error that both consumers of open()
now produce themselves: serveSource for a full-size request, resizedItem.Reader for a
resized one.
* fix(artwork): route disc candidates through the shared resolvers
openCandidate ran its own source loop and threw the error away, so a disc track that
exists but cannot be parsed traced as "miss" — indistinguishable from a track with no
embedded art. fromTag and fromFFmpegTag already report that case as errSourceUnreadable;
only this loop was discarding it. Telling those two apart is what the trace is for.
Candidates now carry a resolve func instead of raw sources: embedded goes to
resolveEmbedded, and the folder-backed entries to resolveFolderSource, extracted from
resolveFolderFile so both callers classify an unopenable file the same way. openCandidate
and its absolute-path special case go away with it.
Disc's own fromExternalFile and fromDiscSubtitle still swallow open errors, so folder
candidates cannot report unreadable yet; that is a change to their error contracts.
* fix(artwork): report an unreadable local candidate as indeterminate
processor.acquire treats resolution.localError exactly as it treats extError: a fault is
not a definitive "no image", so it retries instead of settling absent. explainResult
qualified only the external case, so a chain that ended on an unreadable local candidate
printed "not resolved" — the one verdict that says the walk was conclusive.
The qualification belongs only to the unresolved branch. chainState.try stamps extErr onto
a hit and deliberately drops localErr, so an unreadable step followed by a hit is settled
as found and must not carry a warning; a test pins that.
Found by Codex on
|
||
|
Deluan Quintão
|
944ca3100f |
feat(artwork): new artwork pipeline with background resolution and Low Quality Image Placeholders (#5847)
* feat(artwork): add artwork, item_artwork and artwork_queue tables * feat(artwork): add artwork models, repository interfaces and mocks * feat(artwork): implement artwork repository * feat(artwork): implement item_artwork repository with batched hydration * feat(artwork): implement artwork_queue repository * feat(artwork): add content-addressed originals store * feat(artwork): add artwork prune (orphan cleanup) * fix(artwork): never sweep files on transient DB errors during prune * refactor(artwork): fold originals package into core/artwork as ImageStore * refactor(artwork): merge item artwork state into ArtworkRepository * fix(artwork): chunk unbounded IN clauses and restore interface docs * refactor(artwork): apply simplify-pass cleanups Internal item_artwork sqlRepository helper, toSQLArgs upserts, batched queue enqueue, EnqueueStaleAbsent moved to queue repo, snapshot-based prune sweep, mock/real semantics aligned. * fix(artwork): address review findings on prune/sweep races and mock fidelity Sweep now honors an mtime grace window (in-flight acquisitions and temp files), reacquired orphans reset the prune grace window, and the queue mock implements real stale-absent semantics. * fix(artwork): atomic orphan deletion and timestamp semantics from review DeleteOrphans re-checks age+references at delete time, PutItemArtwork defaults attempted_at, queue mock timestamps mirror SQL. * fix(artwork): guard orphan file removal with the prune grace window Duplicate ImageStore writes refresh the file mtime and Remove skips files newer than the cutoff, so overlapping acquisitions cannot lose their store files to a concurrent prune. * fix(artwork): rewrite vanished duplicates and sweep stale mime variants Write falls through to a real write when the liveness touch fails, and sweep retention now matches the recorded mime's extension so obsolete variants are reclaimed. * fix(artwork): index artwork_queue in dequeue order The previous leading retry_at range column forced a temp B-tree sort of the whole eligible set on every DequeueBatch; ordering the index by (priority DESC, enqueued_at) lets scans stop after the batch size. * fix(artwork): honor the orphan cutoff in the repository mock The mock's DeleteOrphans now applies createdBefore like the SQL implementation, and a new spec covers a freshly reacquired row surviving prune. * fix(artwork): reject malformed hashes in ImageStore operations Known-absent states carry an empty hash and malformed persisted hashes could panic path sharding or inject separators; Write/Open/Remove now return an error for anything but 16 lowercase hex chars. * fix(artwork): mock PutImage refreshes created_at like the SQL repository Prune specs now age fixtures directly instead of seeding stale timestamps through the upsert. * fix(artwork): store backing-file provenance per item, not per hash * feat(artwork): import blurhash encoder from #5797 * feat(artwork): add worker-side artwork resolvers * fix(artwork): propagate playlist tile failures and dedupe external step * feat(artwork): add acquisition processor Resolves one queue item end to end: hash/dedup, decode + 128px thumbnail blurhash, place bytes (store vs source file), and persist found/absent/ failed state for the worker (Task 4) to act on. * style(artwork): tighten processor comments to budget * feat(artwork): add acquisition worker service * feat(artwork): enqueue artwork resolution from scan and CRUD paths * feat(artwork): artwork backfill, fingerprint re-resolution and scheduled jobs * test(artwork): leak/soak coverage and deferred assertions * fix(artwork): propagate transient artist image errors to the worker callGetImage swallowed all agent errors, so an agent outage surfaced as ErrNotFound and the worker settled artist artwork as a definitive absent (and reset the breaker). Add an additive ArtistImageResult path that returns the underlying agent error on transient failure while keeping ArtistImage byte-identical for existing callers; the worker's artist external step uses it via fromArtistExternalResult. * fix(artwork): resolve full playlist source chain resolvePlaylist only built the generated grid, dropping the uploaded-image, sidecar and ExternalImageURL sources the old reader_playlist.go chain serves. Port the full chain before the grid fallback: uploaded (upload), sidecar (folder), and ExternalImageURL routed through extGate with the same extError semantics as the other external steps. Also rewires the artist external step onto ArtistImageResult. * fix(artwork): purge dangling queue rows and guard concurrent re-enqueues Queue rows for deleted entities failed forever (Get -> ErrNotFound -> failed -> capped retries, unbounded). Add ArtworkQueueRepository.PurgeDangling, called from Prune next to the item_artwork purge. Separately, the found/absent path unconditionally deleted the dequeued row, erasing a concurrent scan re-enqueue; switch to DeleteIfUnchanged, which deletes only while retry_at still matches the dequeued value (verified retry_at is the column an Enqueue upsert resets). * style(artwork): fix comment accuracy and budget; fingerprint ArtistImageFolder Correct the inverted workerDeps.extGate comment, trim over-budget doc comments, and add conf.Server.ArtistImageFolder to the resolution fingerprint so an image-folder change re-resolves artist artwork. * fix(artwork): treat missing local playlist cover as definitive, not transient A playlist ExternalImageURL pointing at a local file that fails to open was routed through extError, causing failed/48h-retry loops that burn a rate limiter token forever instead of falling through to the generated grid. * refactor(artwork): deduplicate purge loop, backfill table, and extGate alias * fix(artwork): cap resolved image reads A user-editable ExternalImageURL can point at an arbitrarily large endpoint; a fast server could make the worker buffer hundreds of MB inside the 5s HTTP timeout. Bound the read to a fixed 20MB cap (no config knob) via io.LimitReader and fail the item if it is exceeded. * fix(artwork): retry higher-priority external art after fallback hit With CoverArtPriority="external,cover.jpg", a transient external failure followed by a folder hit dropped the external error: the worker recorded found and deleted the queue row, so the configured higher-priority external art was never retried. Carry extError onto the fallback resolution and add an outcomeFoundStale that persists+serves the art but reschedules via MarkFailed, giving the external source another chance. When external later answers definitively-not-found, the hit is not stale and the row is deleted. * fix(artwork): treat playlist cover URL 404 as definitive miss The playlist ExternalImageURL step used sources.go's fromURL, which maps any non-200 to a generic error, so a stale URL returning 404/410 was classified transient: infinite backoff plus it counted toward the circuit breaker, blocking valid external work. Add a local fetch in resolve.go that maps 404/410 to model.ErrNotFound (definitive) while keeping other non-200s transient. sources.go is left untouched. * test(artwork): move soak test into the Ginkgo suite * test(artwork): make leak and permission tests pass on linux goleak now ignores notify's nonrecursive-tree goroutines (linux uses inotify, which spawns dispatch+internal instead of darwin's recursive dispatch), and the read-only-dir prune spec skips under root, where permission bits cannot make Remove fail. * fix(artwork): reject decompression-bomb dimensions before decoding * fix(artwork): keep fresh re-enqueues ahead of stale failure backoff * fix(artwork): include M3U external art flag in the config fingerprint * fix(artwork): resolve private playlists with an admin context * test(artwork): convert non-synctest timing tests to Ginkgo specs TestArtworkBackoffSchedule and TestArtworkWorkerRunNoLeak needed no real *testing.T (no synctest), so move them into worker_test.go as Ginkgo specs. TestArtworkBreakerHalfOpen stays plain since testing/synctest requires a real *testing.T, matching core/scrobbler's precedent. * fix(artwork): store backing-file provenance per item, not per hash * fix(artwork): apply image limits to playlist tile decoding decodeTile ran image.Decode on every sampled album's resolved bytes before processItem's maxImageBytes/maxImagePixels guards applied, letting an oversized or decompression-bomb tile fully decode unbounded. Enforce both caps inside decodeTile itself. * refactor(artwork): reuse auth.WithAdminUser and dedupe image cap guards * perf(artwork): fetch only IDs for backfill enumeration Backfill enumerated every album, artist, playlist and radio via GetAll and mapped out just the ID. GetAll materializes full entities (library joins, participant/stats/tags JSON, annotation, artwork hydration), so on a large library it loaded tens of thousands of heavy structs only to read one field each — spiking transient RSS to ~1GB during the one-time upgrade backfill, a memory risk on small NAS/Pi hardware. Add GetAllIDs to the album, artist, playlist and radio repositories: it reuses each repo's base row-set filter (library visibility, artist content join, playlist userFilter) but projects only id, skipping the heavy columns and post-processing. A per-repo parity test asserts GetAllIDs returns exactly the same id set as GetAll. Verified on a 727MB / 29k-artist production DB copy: peak RSS during backfill dropped from ~1012MB to ~89MB, file descriptors flat, same 36,138 items enqueued. * feat(artwork): promote worker concurrency and external rate to real configs The artwork worker's drain speed was governed by two hidden Dev flags, DevArtworkWorkerConcurrency and DevArtworkExternalRPS, both defaulting to 2. On a large library's one-time backfill the external rate limiter is the real ceiling: every art-less item waits on it before the (rate-limited) external lookup, so the drain crawls at ~RPS items/sec while local-art items are unaffected. Promote both to documented, supported options: ArtworkWorkerConcurrency (default 4) sets local-resolution parallelism, ArtworkExternalMaxRPS (default 2, 0 = unlimited) caps external-agent lookups to stay polite to Last.fm/Deezer/etc. Operators can now trade first-backfill speed against external-API rate limits. The old Dev names still map for backward compat. * fix(deezer): never return empty-image-id placeholder pictures * feat(agents): enumerate enabled image-retriever agents per capability * feat(artwork): worker fetches agent images directly with per-agent rate limits and breakers * fix(artwork): treat agent not-found as breaker success * feat(model): content-hash artwork id suffix and hydratable per-entity image state * feat(persistence): hydrate artwork hash and absence onto entity pages * feat(artwork): resolve media_file embedded art in the worker, invalidate on rescan * feat(artwork): broadcast refresh events when artwork lands * fix(artwork): broadcast refresh for stale-found artwork too * feat(artwork): state-backed serving path with provisional read-through * feat(server): serve artwork from persisted state with content-hash caching * feat(subsonic): content-hash coverArt ids, omit artwork on known-absent * refactor(artwork): delete the legacy reader chain, cache warmer, and provider image methods * feat(artwork): precache on acquisition, bump on upload/radio changes, manual re-resolve API * test(artwork): end-to-end coverage for the serving cutover * chore(artwork): generic 500 bodies on refresh endpoint, trim stale test comments * fix(artwork): request read-through must not reset the failure backoff The provisional read-through and dangling re-enqueue used Enqueue, whose upsert resets retry_at, so any browse of an unresolved entity that was backing off after an external failure made it immediately eligible again — defeating the exponential backoff during a provider outage. Add EnqueueBump, which raises priority but leaves an existing row's retry_at intact, and route the serving path through it. Scan and manual re-resolve keep Enqueue's reset (a detected change wants immediate retry). * fix(artwork): keep an eligible track's cover requestable when its album is absent An embedded-eligible track with no resolved item_artwork row inherited the album's ImageAbsent, so when the album resolved absent (e.g. CoverArtPriority without 'embedded') the track's coverArt was omitted permanently — the client never requested it, so the lazy mediafile path never resolved it — even though the serving path would extract and serve the track's own embedded art. Hydration now never copies the album's absence onto an eligible-but-unresolved track. * fix(artwork): validate each agent image URL before picking the largest bestImageURL selected the largest by size and only then parsed it, so a malformed largest URL (e.g. a bad percent-escape) returned nil and shadowed a valid smaller candidate, contradicting the documented skip-unparseable behavior. Parse per candidate and compare sizes only among URLs that parse. * fix(artwork): fall back to disc art, not the album, for multi-disc tracks serveMediaFile delegated an absent/ineligible track straight to AlbumCoverArtID, skipping the disc-specific lookup that MediaFile.CoverArtID (and the deleted legacy reader) use. On multi-disc albums with per-disc images that served the album cover instead of the configured disc artwork. Delegate through DiscCoverArtID. * fix(artwork): enqueue uploaded artwork only after the filename is persisted SetImage cleared state and enqueued the bump before the caller stored the new filename, so a worker drain in that window could resolve against the old (already deleted) file and settle absent, leaving the upload unused until a later scan. Move the invalidate+enqueue into EnqueueArtwork, which each caller now invokes after the entity Put. * fix(artwork): keep multi-disc tracks requestable when the album is absent Round-1's hydration fix still copied the album's known-absent onto a non-eligible (or own-absent) track, but MediaFile.CoverArtID routes a multi-disc track to disc art, which resolves provisionally and is never known-absent. Marking it absent made Subsonic omit coverArt so clients never requested a valid disc image. Only mark a single-disc track absent, and only when its own art won't resolve. * fix(artwork): serve a local playlist ExternalImageURL as a file-backed reference A local ExternalImageURL was resolved through the external step and labelled external, so placeBytes copied it into the content-addressed store and dropped its path/mtime — replacing the file never tripped the staleness check. Classify local references as file-backed (resolved in place, even on the request path) and keep store-backed behaviour only for http(s) URLs. * fix(artwork): requeue playlist cover when its track set changes A generated-grid cover went stale after track mutations: nothing re-resolved the playlist's artwork, and the request path deliberately never rebuilds the grid, so serveEntity kept returning the old grid hash indefinitely. Enqueue pl artwork from refreshCounters (the choke point for every track-set change); no clear, so the old cover keeps serving until the worker rebuilds. * fix(artwork): open library-backed artwork through its on-disk root A library configured with a file:// path stored absRoot as the raw URI, so Abs produced strings like file:/music/cover.jpg that os.Open/os.Stat reject — folder, upload and embedded art were treated as dangling on every request, looping forever. Normalize a file:// path to its parsed OS path (the same root os.DirFS uses); non-local schemes are left unchanged (out of scope, per the artwork-musicfs TODO). * fix(artwork): only use disc resolution for multi-disc albums DiscCoverArtID returns a dc- id for any track with DiscNumber>0, so serveDisc ran the full DiscArtPriority chain even for single-disc albums, where a stray disc*/ embedded image could shadow higher-priority album art. Gate disc resolution on the album having more than one disc, matching the legacy reader; single-disc tracks serve album art directly. * fix(artwork): invalidate artwork when an uploaded image is deleted Deleting an artist/radio/playlist upload cleared the filename but left the found item_artwork row and its hash, so lists kept advertising the deleted cover's hash-suffixed immutable URL and clients could display it indefinitely. Call EnqueueArtwork after the delete-side Put, symmetric with upload, so the state is cleared and re-resolved to the next source (or absent). * fix(artwork): restore synthetic-artist guard and unicode normalization in agent lookups Moving agent calls into the worker bypassed two behaviors of the aggregate provider: Agents.GetArtistImages' guard for Unknown/Various Artists (a direct retriever call could assign an unrelated image to a synthetic artist), and auxAlbum/auxArtist.Name's DevPreserveUnicodeInExternalCalls normalization (records with typographic quotes/dashes missed exact-name searches). Re-apply both before enumerating retrievers. * fix(artwork): enforce entity visibility on the Subsonic getCoverArt path serveEntity reads persisted item_artwork by id, bypassing the library and private- playlist filters that the legacy entity-load applied. On the authenticated Subsonic path a user could fetch artwork for an inaccessible album or someone else's private playlist by guessing an id. getCoverArt now resolves the underlying entity through the request-scoped (filtered) repositories and serves the placeholder when it is not visible, so existence isn't leaked and the always-an-image invariant holds. The public share (JWT-authorized) and Jellyfin (admin) paths are intentionally untouched. * fix(artwork): version the artwork ETag with the served representation The ETag was the pixel hash of the original image, so a CoverArtQuality or EnableWebPEncoding change altered the resized bytes without changing the ETag — revalidating clients got a spurious 304 and kept the old encoding. Resized responses now carry a representation ETag (hash + size + square + encode settings) used for the ETag header and If-None-Match, while the immutable decision stays on the pixel hash (URLs remain pixel-identity per the spec, so hash-suffixed clients keep zero-request caching). Full-size originals fall back to the pixel hash as before. * fix(artwork): don't stamp the album hash onto multi-disc tracks The hydration fallback assigned a found album hash to every fallback track, but a multi-disc track's CoverArtID emits a dc- id served from disc-specific art whose hash is unknown at hydration time. Advertising dc-..._<albumHash> gave clients a content- version that never changes when the disc image does, breaking id-based refresh. Only stamp the album hash for single-disc tracks (DiscNumber == 0); multi-disc tracks stay unhashed and rely on the correct ETag returned by the served response. * fix(artwork): enqueue new empty playlists by id, and refresh on absent outcomes Two worker/enqueue fixes from review: - playlistRepository.Put assigned the generated id to the caller's Playlist but passed the stale copy (empty id) to refreshCounters, enqueueing a pl|"" row the worker failed until the daily dangling purge while the real playlist went unresolved. Set the id on the copy before enqueueing. - The drain refresh batch only included found/foundStale, so a cover removed by a scan (found -> absent) never notified clients, leaving the old immutable image displayed. Broadcast absent outcomes too; precache still only warms found/foundStale. * fix(artwork): honor disabled per-track art at serve time; use nanosecond mtime provenance Two serving-correctness fixes from review: - serveMediaFile served a persisted mf embedded image even after EnableMediaFileCoverArt was turned off (the setting isn't in the config fingerprint, so found rows aren't reprocessed). Direct mf- URLs now honor the setting at serve time and fall back to disc/album art. - The file-backed staleness check compared whole-second mtimes, so a same-second content replacement (two writes in one second, or timestamp-preserving tools) could serve different bytes under the old hash + immutable policy. RefMtime is now unix-nanoseconds (no schema change; int64 column), detecting sub-second changes where the filesystem records them. * fix(artwork): preserve the drive when normalizing Windows file:// library paths url.Parse puts the volume of file://C:/Music in Host, not Path, so localOSRoot dropped it and returned /Music — os.Open/os.Stat then failed and folder/embedded art on Windows looped as dangling. Rejoin the host volume, matching core/storage/local's newLocalStorage. * fix(artwork): clamp negative sizes to full-size; convert imghttp test to Ginkgo - A negative size (Subsonic size / Jellyfin maxwidth accept signed ints) reached resizeStaticImage, where the square path builds image.NewNRGBA(Rect(0,0,size,size)) — a giant rectangle that panics/OOMs. Clamp size<0 to 0 (full-size) at the Service entry. Positive sizes were already clamped to the original. - imghttp used a plain func Test with a table; convert to a Ginkgo DescribeTable with the suite entry point in imghttp_suite_test.go (AGENTS.md test-framework requirement). * test(artwork): use renamed ArtworkWorkerConcurrency in e2e tests * feat(artwork): carry blurhash through item image hydration * feat(nativeapi): expose artwork hash, absence and blurhash * feat(artwork): hydrate the parent album's artwork state onto tracks * test(artwork): add hydrateArtwork regression guard for AlbumImage wiring Drives hydrateArtwork itself (not applyItemImage directly) over tracks that take each of the loop's continue branches, so a future edit moving the AlbumImage fill below a continue would fail loudly instead of passing silently. * feat(jellyfin): version album and artist image tags by content hash * fix(jellyfin): trim primaryImageTag comment to why-only, within budget * feat(jellyfin): emit real blurhashes and drop the synthesized fallback * feat(ui): version cover art urls by content hash and skip absent art * feat(ui): add BlurHashCanvas placeholder component * fix(ui): clear stale blurhash pixels and assert the draw path in tests Clear the canvas before each decode attempt so a hash change that fails to decode doesn't leave the previous frame's pixels on screen once this wires into a list that recycles items. Also strengthen the specs to assert createImageData/putImageData were actually invoked (and with what), instead of only checking that a <canvas> element exists. * feat(ui): show the blurhash while an album cover loads * fix(artwork): hydrate cursor streams via an id pre-pass The album, artist and playlist GetCursor built their own select and never called hydrateArtwork, so every Jellyfin list endpoint (all six stream via GetCursor) emitted entity-id image tags and no blurhash. Only GetAll hydrated, which is why Subsonic and the native API were unaffected. Each cursor now resolves its ordered/filtered/paginated id set with the cheap id-only GetAllIDs query, then streams those ids in chunks through the repo's existing GetAll, which already hydrates and applies the full select. Max/Offset are consumed by the pre-pass alone; the chunk query carries only the caller's filters, Sort and Order. This also removes a pre-existing deep-pagination cost: keeping OFFSET out of the joined query makes the pre-pass a covering index scan instead of paying the library and annotation joins for every skipped row. Benchmarked on a synthetic 100k-album DB with the real schema, page=500 at offset 90,000: 3.9ms via the id pre-pass, 52.5ms for the current shape, 192.7ms for a naive join. An unpaginated full stream costs ~24% more, which is the trade. GetAllIDs gains the annotation join whenever the caller's filters or sort reference an annotation column (same gate CountAll uses), otherwise Filters=IsFavorite and SortBy=PlayCount would fail in the pre-pass. The playlist pre-pass repeats GetAll's columns so ORDER BY keeps resolving to playlist.name rather than the joined user.name. * fix(jellyfin): hydrate artwork on the song cursor Jellyfin's listSongs streamed media files via GetCursor, which never hydrates artwork, so songs emitted entity-id image tags and no blurhash. media_file now uses the same id pre-pass as the other three cursors (album/artist/playlist), for consistency, but on a separate method, GetCursorWithArtwork: GetCursor itself must stay untouched, since it's also the scanner's hot path and the scanner never reads artwork. Measured on 1,000,000 tracks, the pre-pass over all ids costs +41.8 MB heap and +298 ms versus GetCursor's bounded +0.0 MB. The Jellyfin path is paginated, though, so in practice it only ever pre-passes a page's worth of ids, not the full library, and doesn't pay that cost. * feat(jellyfin): emit a song's own cover art when it differs from the album's Real Jellyfin fills ImageTags from each item's own images before falling back to the parent album, and Finamp checks imageTags.Primary before AlbumId. Our mapper read only the album's image, so a track with distinct embedded art silently showed the album cover. Emit exactly one entry under ImageBlurHashes.Primary: Go marshals map[string]string in sorted key order rather than insertion order, so a second entry could pair the wrong blurhash with the image imageId resolves to, and Finamp pins that pairing in its cache for 365 days. * test(persistence): scope the GetCursorWithArtwork full-stream spec to tie-free ids The fixture has title ties (e.g. three "Antenna" tracks), so the unscoped positional comparison against GetAll only passed because SQLite's tie order happened to coincide between the full scan and the pre-pass's id IN (...) fetch. Scope it to onlySongs like the sibling ordering specs already do. * refactor(artwork): route song own-art through primaryImageTag; align chunk size Cleanups surfaced by /simplify: the song mapper's own-art branch reimplemented primaryImageTag's tag+blurhash-map construction (and its one-entry invariant) — route it through the helper so that invariant lives in one place. Tie artworkChunkSize to a whole multiple of artworkBatchSize so a cursor page re-chunks into even hydration batches. Hoist a duplicated imageLoading && blurHash boolean in the album grid. * fix(ui): serve the placeholder for known-absent art instead of a broken icon getCoverArtUrl returned '' for an imageAbsent record, so <img src={undefined}> rendered as the browser's broken-image icon on every absent cover. The server already serves a proper placeholder for absent art, so build the url and let it render. * feat(ui): show the blurhash as the loading placeholder across cover surfaces Add a shared CoverImage component (useImageUrl blob cache + blurhash + fade) and render the blurhash while a cover loads on the list thumbnails (CoverArtAvatar, radio) and the artist/album/playlist detail pages. The detail pages now go through CoverImage instead of a plain CardMedia, so their images come from the in-memory blob cache and survive React remounts without re-fetching. BlurHashCanvas gains an optional style prop. * refactor(ui): unify list cover surfaces onto the shared CoverImage component Route the album grid, CoverArtAvatar (artist/playlist lists) and the radio list's cover field through CoverImage instead of each carrying its own useImageUrl + blurhash-overlay wiring. CoverImage gains a default object-fit: cover. Radio keeps its uploaded-image gate and the generic radio placeholder for stations with no art. * fix(ui): address CoverImage review findings Restructure CoverImage so the size/shape lives on the root and the blurhash + image are absolute fills: the <img> mounts only once its blob is ready, so an unresolved cover never flashes a broken <img>. Add a fit prop (default cover) so album/playlist detail keep their letterbox instead of being cropped by a hardcoded object-fit. Remove the orphaned coverLoading styles and an unused subsonic import; add a CoverImage unit test. * perf(ui): only refetch already-loaded records on SSE refresh The artwork worker broadcasts a RefreshResource event per resolved chunk, carrying every id in the chunk. useResourceRefresh was doing a getMany for all of them, so any open list/detail page fetched hundreds of artists it was not displaying. Filter the event ids to records already in the store; the rest load fresh (with their new artwork) when navigated to. * refactor(artwork): scale worker concurrency with CPU count ArtworkWorkerConcurrency now defaults to max(2, NumCPU()/2) instead of a fixed 4, mirroring MaxOpenConns: local resolution scales with the host but stays at half the SQLite pool so it never starves the scanner/UI. External RPS stays a fixed 2 — it gates third-party API calls and is bounded by their tolerance, not the host, so it must not scale with CPUs. Also drop the DevArtworkWorkerConcurrency/DevArtworkExternalRPS deprecated aliases: those names were never released, so there is nothing to migrate. * feat(artwork): re-queue an absent cover when its page is viewed serveEntity now schedules a Bump recheck for an entity whose art was recorded absent, so viewing a missing cover re-triggers resolution (e.g. after an external source that was down during the scan comes back), matching the request-time bump that already covers never-resolved entities. Throttled by attempted_at against requestRecheckAge (1h) so repeatedly opening a genuinely-absent page can't hammer external services. EnqueueBump preserves an existing failed-state backoff via MAX(priority,...) and inserts a fresh, immediately-eligible recheck for a settled-absent row (whose queue row was already deleted). * refactor(artwork): move ImageUploadService to artwork.Uploader Relocate the image-upload service from core to core/artwork as artwork.Uploader, co-locating it with the resolver/worker/serving that own the artwork state it invalidates. MaxImageUploadSize moves too — its only callers are the two image-upload handlers — which lets core/image_upload.go be deleted entirely. Extract the shared "clear resolved state + re-queue at Bump" invalidation into artwork.Refresh and fold nativeapi's refreshArtwork handler onto it, removing the duplicated DeleteForItem+Enqueue block that had drifted into three places. The wire provider moves from core's set to artwork's; the playlists.ImageUploadService binding moves to the top-level injector so core/artwork stays unaware of playlists. Behavior is unchanged. * refactor(artwork): thread model.Kind through the artwork API Entity-level artwork queries now take a typed model.Kind instead of a bare prefix string. GetItemArtwork, DeleteForItem(s), GetInfoForItems, EnqueueStaleAbsent, hydrateItemImages, enqueueBackfillKind and artwork.Refresh convert to the prefix string only at the two real boundaries: the SQL item_kind column (kind.Prefix() inside each repo) and external string inputs (a new model.ParseKind for the nativeapi URL param, which also validates it). The Backfill/stale-absent kind slices, the resolve.go dispatch switch, and the kind→resource / kind→table lookup maps now use the Kind vars directly. The queue lifecycle methods (MarkFailed/Delete*) keep string kinds — they operate on a dequeued item's raw ItemKind column, which stays a string field, always populated via kind.Prefix(). Removes every bare "al"/"ar"/… prefix literal from non-test code (27 -> 0); behavior is unchanged. * tune(artwork): drop backoff base from 5m to 15s The exponential retry (base × 4^attempts, cap 48h) started at 5 minutes, so a single transient failure — a timeout under load, an external blip — parked a cover for 5 minutes even though a retry seconds later would have resolved it. Start at 15s instead: transient failures recover almost immediately (15s → 1m → 4m → 16m …), while persistent failures still escalate to the 48h cap (now at the 8th attempt instead of the 5th). * tune(artwork): 5s backoff base + 12h give-up, drop the cap Retry backoff now starts at 5s (was 15s) so a transient failure recovers on essentially the next drain, and jitter widens to ±40% so a wave of correlated failures doesn't re-clump into one poll. Add a 12h give-up budget measured from enqueued_at: once the next backoff would land past it, the worker stops retrying instead of grinding at a cap forever. A bare failure settles absent (handed to the 24h stale-absent sweep, and still recoverable on a page view); a found-stale keeps its already-served art. The budget bounds the tail, so the separate 48h backoffCap is removed. * fix(lastfm): match album.getInfo on name+artist only, not MBID Last.fm's album.getInfo by MBID is unreliable: a correct MBID can return a different album, or none. Observed with black midi's "7-eleven" (whose correct MBID returned a FLEETWOOD release) and both missing The Chats albums (one MBID 404s, the other resolves to a different self-titled release). The worker then recorded covers absent — or would fetch the wrong art — even though the correct cover is on Last.fm by name+artist. Stop passing the MBID to album.getInfo; query by name+artist only, which also drops the now-dead error-6 MBID-retry fallback. The low-level client keeps its MBID support for other callers; only the album lookup changes. * fix(lastfm): return agents.ErrNotFound on error 6 (not found) Last.fm returns error 6 for a missing artist/album — a definitive negative — but the agent returned the raw *lastFMError, so the artwork worker treated every not-found as a real fault: it counted toward the per-source circuit breaker (5 in a row opens it, fast-failing all Last.fm calls including valid ones) and was retried as a transient error instead of settling absent. On a first scan of a library with many artists Last.fm lacks, this stalled valid cover lookups and left entities churning in backoff. Translate error 6 to the shared agents.ErrNotFound at the agent boundary (callAlbumGetInfo / callArtistGetInfo), matching how the Deezer agent maps its client's not-found, and log it at Debug instead of Error — which also removes the not-found log spam. * feat(artwork): log external image-lookup failures at debug The worker's res.reader==nil && extError branch returned outcomeFailed with no log, so a failing external cover lookup (agent error, dead image URL, download timeout) was undiagnosable. Log the agent, entity, and underlying error at the fetch site where it's in hand — this surfaced a Last.fm album.getInfo returning an image URL that itself 404s. * fix(artwork): treat a 404/410 image URL as not-found, not a transient fault An agent (notably Last.fm's album.getInfo) can advertise a cover URL that is itself dead — a 404. sources.go's fromURL returned a generic error for any non-200, so a dead URL was treated as a transient failure: it churned in backoff and counted toward the circuit breaker, stalling valid lookups. Map 404/410 to model.ErrNotFound in fromURL so a dead URL settles absent, and collapse the near-identical fetchPlaylistImageURL (which already did this for M3U covers) into it. * feat(artwork): make artwork re-resolution targeted, not blunt Two gaps in when the pipeline re-resolves artwork: The recheck job only requeued absent-state rows (hash=''), so an entity that was never processed — added between scans, or on a server with the scanner disabled — had no periodic safety net and stayed without artwork indefinitely. Add EnqueueMissing(kind): a SQL set-difference enqueueing entities with no item_artwork row at Recheck priority (ON CONFLICT DO NOTHING, so it never disturbs a queued row). Run it once at startup and hourly alongside the stale-absent recheck. Rename staleAbsentKinds -> recheckKinds accordingly. Conversely, the config fingerprint included consts.Version, which embeds the git SHA and so changed on every build, re-enqueueing every entity in the library (~34k here) and re-querying external agents at the configured RPS for anything without local art. Replace it with an explicit artworkEpoch constant, bumped deliberately when resolution semantics change. The cases that motivated the version input — absent art becoming available — are already covered by the stale-absent and missing-row rechecks; only a corrected wrong-pick needs the epoch. A test guards against reintroducing the version. * test(artwork): restore resolution edge-case e2e coverage The serving cutover removed the album/disc/artist/mediafile/playlist/radio e2e specs that documented the folder-selection rules and guarded the #5376/#5456/ #5451/#5457 regressions; nothing replaced them, so compareImageFiles and the parent-fallback logic were left untested. Restore them driving the real pipeline: a real scanner populates the folder graph from an in-memory library, the real Worker drains the queue, and the real Service serves. Folder-backed art is file-backed (served via os.Open, which the in-memory FS can't satisfy) so its selection is asserted on the persisted state row; store-backed and real-disk sources are asserted byte-for-byte. Single-disc disc resolution now serves album art directly, so only multi-disc disc scenarios are ported. * fix(artwork): run disc resolution for single-disc albums too |
||
|
Deluan Quintão
|
5d1c1157b5 |
refactor(artwork): migrate readers to storage.MusicFS and add e2e suite (#5379)
* test(artwork): add e2e suite documenting album/disc resolution Adds core/artwork/e2e/ with a real-tempdir + scanner harness that exercises artwork resolution end-to-end. Covers album and disc kinds; pending (PIt) cases document two known bugs in reader_album.go for regression-guard flipping once they are fixed. * refactor(artwork): add libraryFS helper to resolve MusicFS for a library * test(artwork): tighten libraryFS test isolation and add scheme-error case * test(artwork): update libraryFS test description to match implementation * refactor(artwork): convert fromExternalFile to use fs.FS Add a temporary fromExternalFileAbs shim so existing absolute-path callers still compile; the shim is removed once all readers are migrated. * refactor(artwork): make fromExternalFileAbs a thin delegator Introduce a minimal osDirectFS adapter so the shim no longer duplicates the matching loop. Both will be removed in Task 9. * refactor(artwork): convert fromTag to taglib.OpenStream over fs.FS Add a temporary fromTagAbs shim so existing absolute-path callers still compile; removed in Task 9. Reuses the osDirectFS adapter from Task 2. * refactor(artwork): defer fs.File close until after taglib reads finish Mirror the lifetime pattern used by adapters/gotaglib/gotaglib.go: keep the underlying fs.File open until taglib.File is closed, and pass WithFilename so format detection doesn't rely on content sniffing. * docs(artwork): note ffmpeg's path-based API limitation * refactor(artwork): migrate album reader to MusicFS - Add libFS (storage.MusicFS) field to albumArtworkReader; resolved once at construction time via libraryFS() - Switch fromCoverArtPriority from abs-path shims to FS-based fromTag/fromExternalFile; only fromFFmpegTag retains absolute path - Build imgFiles as library-relative forward-slash paths in loadAlbumFoldersPaths using path.Join(f.Path, f.Name, img) - Guard embedAbs so that an empty EmbedArtPath never produces a non-empty absolute path (prevents accidental ffmpeg invocation) - Register testfile:// storage scheme in artwork test suite to provide an os.DirFS-backed MusicFS without requiring the taglib extractor - Update test assertions from filepath.FromSlash(abs) to bare forward-slash relative strings * fix(artwork): use path package in compareImageFiles for forward-slash relative paths * refactor(artwork): migrate disc reader to MusicFS Replace os.Open absolute-path access with libFS.Open on library-relative forward-slash paths. Rename discFolders→discFoldersRel, split firstTrackPath into firstTrackRelPath (for fromTag) and firstTrackAbsPath (for fromFFmpegTag), and switch path.Dir/Base/Ext for forward-slash safety. * refactor(artwork): build discFoldersRel directly and guard empty first track * refactor(artwork): migrate mediafile reader to MusicFS * refactor(artwork): migrate artist album-art lookup to MusicFS * refactor(artwork): remove temporary path-based shims All readers now use the FS-based fromTag and fromExternalFile directly, so the absolute-path adapters and the osDirectFS helper that backed them can go away. * test(artwork): rewrite e2e suite to use storagetest.FakeFS Switches from real-tempdir + local storage to FakeFS via the storage registry. Adds a proper multi-disc scenario using the disc tag, which previously required curated MP3 fixtures we did not have. * test(artwork): use maps.Copy in trackFile tag merge Lint cleanup: replace the manual map-copy loop flagged by mapsloop. * test(artwork): reuse tests.MockFFmpeg in e2e harness Replace the hand-rolled noopFFmpeg stub with tests.NewMockFFmpeg, which already satisfies the full ffmpeg.FFmpeg interface and won't drift when new methods are added. Also tie imageBytes to imageFile so they cannot silently disagree on the on-disk encoding. * test(artwork): add e2e scenarios from artwork documentation Covers the behaviors documented at https://www.navidrome.org/docs/usage/library/artwork/: - Album: folder.*/front.* fallbacks and priority order with cover.*. - Disc: cd*.* match, cover.* inside disc folder, DiscArtPriority="" skip path, the documented multi-disc layout, and the discsubtitle keyword. - MediaFile: disc-level fallback for multi-disc tracks and album-level fallback for single-disc tracks (doc section "MediaFiles" items 2-3). - Artist: album/artist.* lookup via libFS (passes). The artist-folder branch is XIt-marked because fromArtistFolder still calls os.DirFS directly on an absolute path and can't read from a FakeFS-backed library — migrating that to storage.MusicFS is a follow-up. Signed-off-by: Deluan <deluan@navidrome.org> * refactor(artwork): scope artist folder traversal to library root Route fromArtistFolder reads through storage.MusicFS and bound the parent-directory walk at the library root. This keeps artwork resolution scoped to the configured library and unblocks FakeFS-backed e2e scenarios that depend on the artist folder. Also consolidate the libraryFS + core.AbsolutePath pairing (used by three readers) into a single libraryFSAndRoot helper. * test(artwork): add ASCII file-tree diagrams to e2e scenarios Each It/PIt block now shows the on-disk layout it exercises, with arrows indicating which file wins (or should win, for the known-bug PIt cases). Makes scenarios readable at a glance without having to parse the MapFS map. * test(artwork): add e2e tests for playlist and radio artwork resolution Signed-off-by: Deluan <deluan@navidrome.org> * test(artwork): enhance e2e tests with real MP3 fixtures for embedded artwork Signed-off-by: Deluan <deluan@navidrome.org> * test(ffmpeg): add support for animated WebP encoder detection and fallback handling Signed-off-by: Deluan <deluan@navidrome.org> * test(artwork): cover additional edge cases in e2e suite Add high-value scenarios uncovered by the existing specs: - Album: three-way basename tie (unsuffixed wins), unknown pattern in CoverArtPriority is skipped, embedded-first with no embedded art falls through. - Disc: discsubtitle with no matching image falls through. - Artist: ArtistArtPriority can reach images via album/<pattern>. - Playlist: generates a 2x2 tiled cover from album art when the playlist has no uploaded/sidecar/external image. New helper realPNG() produces real taglib/image-decodable bytes so the tiled-cover test can exercise the generator's decode + compose path. * test(artwork): refactor image upload logic in e2e tests for consistency Signed-off-by: Deluan <deluan@navidrome.org> * test(ffmpeg): simplify animated WebP encoder check by removing context parameter Signed-off-by: Deluan <deluan@navidrome.org> * fix(artwork): normalize rel path for fs.Glob on Windows filepath.Rel returns backslash-separated paths on Windows, but fs.Glob and path.Join require forward slashes. Convert with filepath.ToSlash after computing the relative path and use path.Dir for the parent walk so the artist-folder lookup works cross-platform. * fix(ffmpeg): retry animated WebP probe on transient failure The probe previously used the caller's request context inside sync.Once, so a single cancelled first request would permanently disable animated WebP for the rest of the process. Switch to a mutex + probed flag, use a fresh background context with its own timeout, and only cache the result when the probe actually succeeds. * test(ffmpeg): reset ffOnce so ConvertAnimatedImage test is order-independent The ConvertAnimatedImage stand-in test sets ffmpegPath directly but does not reset ffOnce. If ffmpegCmd() has not been called earlier in the test process, the next call inside hasAnimatedWebPEncoder runs ffOnce.Do and re-resolves the real ffmpeg binary, overwriting the stand-in and breaking the test. Reset ffOnce and conf.Server.FFmpegPath alongside the other globals to pin resolution to the stand-in. * test(artwork): unblock Windows CI — forward-slash fs paths and suite-level DB lifetime The internal artwork test planted a Windows absolute path (backslashes) into Folder.Path and then fed it through libFS.Open, which fs.ValidPath rejects. Rooting the testfile library at the temp dir directly and using filepath.ToSlash keeps the path model library-relative and forward-slash, matching production. The e2e suite opened a per-spec DB in a per-spec TempDir, but the go-sqlite3 singleton kept the file open across specs. Ginkgo's per-spec TempDir cleanup then tried to unlink a file still held by that handle — fine on POSIX, fails on Windows. Moving the DB to a suite-level tempdir and closing it in AfterSuite avoids the race. * test(artwork): keep Windows drive letters intact in testfile library URLs url.Parse on `testfile://C:/path` reads `C` as the host and the path loses the drive letter, so Windows libFS lookups go to `/path` and fail. testFileLibPath now prepends a `/` when the OS path has no leading slash, and the testfile constructor strips that extra slash back off before handing the path to os.Stat / os.DirFS. * refactor(artwork): consolidate libFS + root into libraryView helper Collapses the per-reader libFS/libPath/rootFolder/firstTrackAbsPath fields into a single libraryView{FS, absRoot} with an Abs(rel) method. Also folds the two library lookups (ds.Library.Get + core.AbsolutePath) into one, and uses mf.Path directly instead of stripping libRoot off an absolute path. * refactor(ffmpeg): replace hasAnimatedWebPEncoder with encoderProbe for state management Signed-off-by: Deluan <deluan@navidrome.org> * fix: escape artist folder names in artwork glob Escape glob metacharacters in the library-relative artist folder path before composing the fs.Glob pattern for artist image lookup. This preserves literal folder names such as Artist [Live] while keeping the configured filename pattern behavior unchanged, and adds a regression test for bracketed artist folders. Signed-off-by: Deluan <deluan@navidrome.org> * fix(artwork): correct test path assertions after MusicFS migration Source functions (fromTag, fromExternalFile) now return forward-slash fs.FS-relative paths, so test assertions should compare against plain forward-slash strings, not filepath.FromSlash(). The artistArtPriority test needs filepath.FromSlash() on the suffix because findImageInFolder returns OS-native absolute paths via filepath.Join. * fix(artwork): normalize path separators in artistArtPriority assertion The two table entries exercise different code paths: entry 1 goes through fromArtistFolder (returns OS-native paths via filepath.Join), while entry 2 goes through fromExternalFile (returns forward-slash fs.FS paths). Using filepath.FromSlash on the expected value only works for entry 1. Normalize the actual path to forward slashes with filepath.ToSlash so a single HaveSuffix assertion works for both code paths on all platforms. --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
420d2c8e5a |
fix(artwork): validate ffmpeg pipe before returning in cover art fallback
ffmpeg.ExtractImage returns a pipe-based reader immediately, before ffmpeg finishes processing. When the audio file has no embedded image stream (e.g. a plain MP3), ffmpeg exits with an error that closes the pipe asynchronously. The selectImageReader function saw the non-nil reader as a success and returned it instead of falling through to the next source in the chain (album art). This caused getCoverArt to return an error response for tracks on albums where the disc artwork reader was invoked but no embedded art existed. Fixed by reading one byte from the pipe to validate the stream delivers data before returning it. If the read fails, the reader is closed and nil is returned, allowing the fallback chain to continue to album artwork. Closes #5265 |
||
|
Deluan
|
1ce561cc8e |
refactor(server): remove legacy embedded coverart logic
Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
5fa8356b31 |
chore(deps): bump golangci-lint to v2.10.0 and suppress new gosec false positives
Bump golangci-lint from v2.9.0 to v2.10.0, which includes a newer gosec with additional taint-analysis rules (G117, G703, G704, G705) and a stricter G101 check. Added inline //nolint:gosec comments to suppress 21 false positives across 19 files: struct fields flagged as secrets (G117), w.Write calls flagged as XSS (G705), HTTP client calls flagged as SSRF (G704), os.Stat/os.ReadFile/os.Remove flagged as path traversal (G703), and a sort mapping flagged as hardcoded credentials (G101). Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
66474fc9f4 |
feat: add support for reading embedded images using taglib by default
Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
03a45753e9 |
feat(plugins): New Plugin System with multi-language PDK support (#4833)
* chore(plugins): remove the old plugins system implementation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement new plugin system with using Extism Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add capability detection for plugins based on exported functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): standardize variable names and remove superfluous wrapper functions Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): improve error handling and logging in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement plugin function call helper and refactor MetadataAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): race condition in plugin manager * tests(plugins): change BeforeEach to BeforeAll in MetadataAgent tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): optimize tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): more optimizations Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): ignore goroutine leaks from notify library in tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Wikimedia plugin for Navidrome to fetch artist metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin logging and set User-Agent header Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement scrobbler plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate logs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): clean up manifest struct and improve plugin loading logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add metadata agent and scrobbler schemas for bootstrapping plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(hostgen): add hostgen tool for generating Extism host function wrappers - Implemented hostgen tool to generate wrappers from annotated Go interfaces. - Added command-line flags for input/output directories and package name. - Introduced parsing and code generation logic for host services. - Created test data for various service interfaces and expected generated code. - Added documentation for host services and annotations for code generation. - Implemented SubsonicAPI service with corresponding generated code. * feat(subsonicapi): update Call method to return JSON string response Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement SubsonicAPI host function integration with permissions Signed-off-by: Deluan <deluan@navidrome.org> * fix(generator): error-only methods in response handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate client wrappers for host functions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(generator): remove error handling for response.Error in client templates Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add Scheduler service interface with host function wrappers for scheduling tasks * feat(plugins): add WASI build constraints to client wrapper templates, to avoid lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler service with one-time and recurring scheduling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor(manifest): remove unused ConfigPermission from permissions schema Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add scheduler callback schema and implementation for plugins Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): streamline scheduling logic and remove unused callback tracking Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): add Close method for resource cleanup on plugin unload Signed-off-by: Deluan <deluan@navidrome.org> * docs(scheduler): clarify SchedulerCallback requirement for scheduling functions Signed-off-by: Deluan <deluan@navidrome.org> * fix: update wasm build rule to include all Go files in the directory Signed-off-by: Deluan <deluan@navidrome.org> * feat: rewrite the wikimedia plugin using the XTP CLI Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): replace uuid with id.NewRandom for schedule ID generation Signed-off-by: Deluan <deluan@navidrome.org> * refactor: capabilities registration Signed-off-by: Deluan <deluan@navidrome.org> * test: add scheduler service isolation test for plugin instances Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin manager initialization and encapsulate logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add WebSocket service definitions for plugin communication Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WebSocket service for plugin integration and connection management Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Crypto Ticker example plugin for real-time cryptocurrency price updates via Coinbase WebSocket API Also add the lifecycle capability Signed-off-by: Deluan <deluan@navidrome.org> * fix: use context.Background() in invokeCallback for scheduled tasks Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin.create() to plugin.instance() Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginInstance to plugin for consistency across the codebase Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify schedule cloning in Close method and enhance plugin cleanup error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Artwork service for generating artwork URLs in Navidrome plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: moved public URL builders to avoid import cycles Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cache service for in-memory TTL-based caching in plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Discord Rich Presence example plugin for Navidrome integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor: host function wrappers to use structured request and response types - Updated the host function signatures in `nd_host_artwork.go`, `nd_host_scheduler.go`, `nd_host_subsonicapi.go`, and `nd_host_websocket.go` to accept a single parameter for JSON requests. - Introduced structured request and response types for various cache operations in `nd_host_cache.go`. - Modified cache functions to marshal requests to JSON and unmarshal responses, improving error handling and code clarity. - Removed redundant memory allocation for string parameters in favor of JSON marshaling. - Enhanced error handling in WebSocket and cache operations to return structured error responses. * refactor: error handling in various plugins to convert response.Error to Go errors - Updated error handling in `nd_host_scheduler.go`, `nd_host_websocket.go`, `nd_host_artwork.go`, `nd_host_cache.go`, and `nd_host_subsonicapi.go` to convert string errors from responses into Go errors. - Removed redundant error checks in test data plugins for cleaner code. - Ensured consistent error handling across all plugins to improve reliability and maintainability. * refactor: rename fake plugins to test plugins for clarity in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add help target to Makefile for plugin usage instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cover Art Archive plugin as an example of Python plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Makefile and README to clarify Go plugin usage Signed-off-by: Deluan <deluan@navidrome.org> * feat: include plugin capabilities in loading log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add trace logging for plugin availability and error handling in agents Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Now Playing Logger plugin to showcase calling host functions from Python plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: generate Python client wrappers for various host services Signed-off-by: Deluan <deluan@navidrome.org> * feat: add generated host function wrappers for Scheduler and SubsonicAPI services Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Python plugin documentation and usage instructions for host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Webhook Scrobbler plugin in Rust to send HTTP notifications on scrobble events Signed-off-by: Deluan <deluan@navidrome.org> * feat: enable parallel loading of plugins during startup Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include WebSocket callback schema in plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin watcher with improved logging and debounce duration adjustment Signed-off-by: Deluan <deluan@navidrome.org> * add trace message for plugin recompiles Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin cache purging functionality Signed-off-by: Deluan <deluan@navidrome.org> * test: move purgeCacheBySize unit tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin repository and database support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin management routes and middleware Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): implement plugin synchronization with database for add, update, and remove actions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add PluginList and PluginShow components with plugin management functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): optimize plugin change detection Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins UI): improve PluginList structure Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): enhance PluginShow with author, website, and permissions display Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): refactor to use MUI and RA components Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add error handling for plugin enable/disable actions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): inject PluginManager into native API Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update GetManager to accept DataStore parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add subsonicRouter to Manager and refactor host service registration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): enhance debug logging for plugin actions and recompile logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): break manager.go into smaller, focused files Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline error handling and improve plugin retrieval logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update newWebSocketService to use WebSocketPermission for allowed hosts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): introduce ToggleEnabledSwitch for managing plugin enable/disable state Signed-off-by: Deluan <deluan@navidrome.org> * docs: update READMEs Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add Library service for metadata access and filesystem integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Library Inspector plugin for periodic library inspection and file size logging Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect JSON configuration format for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix(build): update target to wasm32-wasip1 for improved WASI support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement configuration management UI with key-value pairs support Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): adjust grid layout in InfoRow component for improved responsiveness Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename ErrorIndicator to EnabledOrErrorField and enhance error handling logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(i18n): add Portuguese translations for plugin management and notifications Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for .ndp plugin packages and update build process Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README for .ndp plugin packaging and installation instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement KVStore service for persistent key-value storage Signed-off-by: Deluan <deluan@navidrome.org> * docs: enhance README with Extism plugin development resources and recommendations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate event broker into plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update config handling in PluginShow to track last record state Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Rust host function library and example implementation of Discord Rich Presence plugin in Rust Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Rust lib.rs file to expose host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update JSON field names to camelCase for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reduce cyclomatic complexity by refactoring main function Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance Rust code generation with typed struct support and improved type handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Go client library with host function wrappers and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Go client stubs for non-WASM platforms Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update client template file names for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add initial implementation of the Navidrome Plugin Development Kit code generator - Pahse 1 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 (2) Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 4 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 5 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consistent naming/types across PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline plugin function signatures and error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update scrobbler interface to return errors directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * test: make all test plugins use the PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize and sort type definitions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update error handling for methods to return errors directly Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update function signatures to return values directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update request/response types to use private naming conventions Signed-off-by: Deluan <deluan@navidrome.org> * build: mark .wasm files as intermediate for cleanup after building .ndp Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consolidate PDK module path and update Go version to 1.25 Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Rust PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize Rust output structure to follow standard conventions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update Discord Rich Presence and Library Inspector plugins to use nd-pdk for service calls and implement lifecycle management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update macro names for websocket and metadata registration to improve clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename scheduler callback methods for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update export wrappers to use `//go:wasmexport` for WebAssembly compatibility Signed-off-by: Deluan <deluan@navidrome.org> * docs: update plugin registration docs Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): generate host wrappers Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): conditionally run goleak checks based on CI environment Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect changes in plugin import paths Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update plugin instance creation to accept context for cancellation support Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update return types in metadata interfaces to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): enhance type handling for Rust and XTP output in capability generation Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update IsAuthorized method to return boolean instead of response object Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add unit tests for rustOutputType and isPrimitiveRustType functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement XTP JSONSchema validation for generated schemas Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update response types in testMetadataAgent methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * docs: update Go and Rust plugin developer sections for clarity Signed-off-by: Deluan <deluan@navidrome.org> * docs: correct example link for library inspector in README Signed-off-by: Deluan <deluan@navidrome.org> * docs: clarify artwork URL generation capabilities in service descriptions Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include Rust PDK crate information for plugin developers Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle URL parsing errors and use atomic upsert in plugin repository Added proper error handling for url.Parse calls in PublicURL and AbsoluteURL functions. When parsing fails, PublicURL now falls back to AbsoluteURL, and AbsoluteURL logs the error and returns an empty string, preventing malformed URLs from being generated. Replaced the non-atomic UPDATE-then-INSERT pattern in plugin repository Put method with a single atomic INSERT ... ON CONFLICT statement. This eliminates potential race conditions and improves consistency with the upsert pattern already used in host_kvstore.go. * feat: implement mock service instances for non-WASM builds using testify/mock Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Discord RPC struct to encapsulate WebSocket logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for experimental WebAssembly threads Signed-off-by: Deluan <deluan@navidrome.org> * feat: add PDK abstraction layer with mock support for non-WASM builds Signed-off-by: Deluan <deluan@navidrome.org> * feat: add unit tests for Discord plugin and RPC functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: update return types in minimalPlugin and wikimediaPlugin methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix: context cancellation and implement WebSocket callback timeout for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: conditionally include error handling in generated client code templates Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ConfigService for plugin configuration management Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance plugin manager to support metrics recording Signed-off-by: Deluan <deluan@navidrome.org> * refactor: make MockPDK private Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update interface types to use 'any' in plugin repository methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename List method to Keys for clarity in configuration management Signed-off-by: Deluan <deluan@navidrome.org> * test: add ndpgen plugin tests in the pipeline and update Makefile Signed-off-by: Deluan <deluan@navidrome.org> * feat: add users permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * refactor: streamline users integration tests and enhance plugin user management Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove UserID from scrobbler request structure Signed-off-by: Deluan <deluan@navidrome.org> * test: add integration tests for UsersService enable gate behavior Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user permissions for SubsonicAPI and scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: show proper error in the UI when enabling a plugin fails Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * feat: add user permission for processing scrobbles in Discord Rich Presence plugin Signed-off-by: Deluan <deluan@navidrome.org> * fix: implement dynamic loading for buffered scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add GetAdmins method to retrieve admin users from the plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Portuguese translations for user and library permissions Signed-off-by: Deluan <deluan@navidrome.org> * reorder migrations Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove unnecessary bulkActionButtons prop from PluginList component * feat: add manual plugin rescan functionality and corresponding UI action Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user/library and plugin management integration with cleanup on deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace core mock services with test-specific implementations to avoid import cycles * feat: add ID fields to Artist and Song structs and enhance track loading logic by prioritizing ID matches Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin permissions from allowedHosts to requiredHosts for better clarity and consistency * feat: refactor plugin host permissions to use RequiredHosts directly for improved clarity * fix: don't record metrics for plugin calls that aren't implemented at all Signed-off-by: Deluan <deluan@navidrome.org> * fix: enhance connection management with improved error handling and cleanup logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: introduce ArtistRef struct for better artist representation and update track metadata handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: update user configuration handling to use user key prefix for improved clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance ConfigCard input fields with multiline support and vertical resizing Signed-off-by: Deluan <deluan@navidrome.org> * fix: rust plugin compilation error Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement IsOptionPattern method for better return type handling in Rust PDK generation Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
bytesingsong
|
b69a7652b9 |
chore: fix some typos in comment and logs (#4333)
Signed-off-by: bytesingsong <bytesing@icloud.com> |
||
|
Deluan Quintão
|
58367afaea |
refactor: external_metadata -> external.Provider (#3903)
* tests for TopSongs Signed-off-by: Deluan <deluan@navidrome.org> * convert to Ginkgo Signed-off-by: Deluan <deluan@navidrome.org> * consolidate tests Signed-off-by: Deluan <deluan@navidrome.org> * rename external metadata -wip Signed-off-by: Deluan <deluan@navidrome.org> * rename external metadata to extdata.Provider Signed-off-by: Deluan <deluan@navidrome.org> * refactor tests - wip Signed-off-by: Deluan <deluan@navidrome.org> * refactor test helpers Signed-off-by: Deluan <deluan@navidrome.org> * remove reflection Signed-off-by: Deluan <deluan@navidrome.org> * use mock.Mock Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * fix Signed-off-by: Deluan <deluan@navidrome.org> * receive Agents interface in Provider constructor Signed-off-by: Deluan <deluan@navidrome.org> * use mock for Agents Signed-off-by: Deluan <deluan@navidrome.org> * tests for SimilarSongs Signed-off-by: Deluan <deluan@navidrome.org> * remove duplication Signed-off-by: Deluan <deluan@navidrome.org> * ArtistImage tests Signed-off-by: Deluan <deluan@navidrome.org> * AlbumImage tests Signed-off-by: Deluan <deluan@navidrome.org> * fix provider error handling Signed-off-by: Deluan <deluan@navidrome.org> * UpdateAlbumInfo tests - wip Signed-off-by: Deluan <deluan@navidrome.org> * UpdateAlbumInfo tests - wip Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * UpdateArtistInfo tests - wip Signed-off-by: Deluan <deluan@navidrome.org> * clean up Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * fix test descriptions Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename extdata package to external Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
c795bcfcf7 |
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709)
* fix(server): more race conditions when updating artist/album from external sources Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394 Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): pass configfile option to child process Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): resume interrupted fullScans Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): remove old scanner code Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): rename old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): move old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update Go to 1.23.4 Signed-off-by: Deluan <deluan@navidrome.org> * fix: logs Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): Signed-off-by: Deluan <deluan@navidrome.org> * fix: log level Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add config for scanner watcher Signed-off-by: Deluan <deluan@navidrome.org> * refactor: children playlists Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace `interface{}` with `any` Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlists with genres Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow any tags in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist names in playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlist's sort by tags Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use generic JSONArray for OS arrays Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use https in test Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add releaseTypes to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add recordLabels to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): rename JSONArray to Array Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to Child Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): do not pre-populate smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement a simplified version of ArtistID3. See https://github.com/opensubsonic/open-subsonic-api/discussions/120 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to album child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add contributors to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add albumArtists to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayArtist and displayAlbumArtist Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayComposer to Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add roles to ArtistID3 Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): use " • " separator for displayComposer Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): respect `PreferSortTags` config option Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize purging non-unused tags Signed-off-by: Deluan <deluan@navidrome.org> * refactor: don't run 'refresh artist stats' concurrently with other transactions Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix: log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Scanner.ScanOnStartup config option, default true Signed-off-by: Deluan <deluan@navidrome.org> * feat: better json parsing error msg when importing NSPs Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update album's imported_time when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle interrupted scans and full scans after migrations Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `analyze` when migration requires a full rescan Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `PRAGMA optimize` at the end of the scan Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update artist's updated_at when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat: handle multiple artists and roles in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): dim missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * fix: album missing logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: error encoding in gob Signed-off-by: Deluan <deluan@navidrome.org> * feat: separate warnings from errors Signed-off-by: Deluan <deluan@navidrome.org> * fix: mark albums as missing if they were contained in a deleted folder Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add participant names to media_file and album tables Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use participations in criteria, instead of m2m relationship Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename participations to participants Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to album child Signed-off-by: Deluan <deluan@navidrome.org> * fix: albumartist role case Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): run scanner as an external process by default Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): show albumArtist names Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): dim out missing albums Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * fix(server): scrobble buffer mapping. fix #3583 Signed-off-by: Deluan <deluan@navidrome.org> * refactor: more participations renaming Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling Signed-off-by: Deluan <deluan@navidrome.org> * feat: send release_group_mbid to listenbrainz Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement OpenSubsonic explicitStatus field (#3597) * feat: implement OpenSubsonic explicitStatus field * fix(subsonic): fix failing snapshot tests * refactor: create helper for setting explicitStatus * fix: store smaller values for explicit-status on database * test: ToAlbum explicitStatus * refactor: rename explicitStatus helper function --------- Co-authored-by: Deluan Quintão <deluan@navidrome.org> * fix: handle album and track tags in the DB based on the mappings.yaml file Signed-off-by: Deluan <deluan@navidrome.org> * save similar artists as JSONB Signed-off-by: Deluan <deluan@navidrome.org> * fix: getAlbumList byGenre Signed-off-by: Deluan <deluan@navidrome.org> * detect changes in PID configuration Signed-off-by: Deluan <deluan@navidrome.org> * set default album PID to legacy_pid Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * fix SIGSEGV Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't lose album stars/ratings when migrating Signed-off-by: Deluan <deluan@navidrome.org> * store full PID conf in properties Signed-off-by: Deluan <deluan@navidrome.org> * fix: keep album annotations when changing PID.Album config Signed-off-by: Deluan <deluan@navidrome.org> * fix: reassign album annotations Signed-off-by: Deluan <deluan@navidrome.org> * feat: use (display) albumArtist and add links to each artist Signed-off-by: Deluan <deluan@navidrome.org> * fix: not showing albums by albumartist Signed-off-by: Deluan <deluan@navidrome.org> * fix: error msgs Signed-off-by: Deluan <deluan@navidrome.org> * fix: hide PID from Native API Signed-off-by: Deluan <deluan@navidrome.org> * fix: album cover art resolution Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim participant names Signed-off-by: Deluan <deluan@navidrome.org> * fix: reduce watcher log spam Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic when initializing the watcher Signed-off-by: Deluan <deluan@navidrome.org> * fix: various artists Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't store empty lyrics in the DB Signed-off-by: Deluan <deluan@navidrome.org> * remove unused methods Signed-off-by: Deluan <deluan@navidrome.org> * drop full_text indexes, as they are not being used by SQLite Signed-off-by: Deluan <deluan@navidrome.org> * keep album created_at when upgrading Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null pointer Signed-off-by: Deluan <deluan@navidrome.org> * fix: album artwork cache Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't expose missing files in Subsonic API Signed-off-by: Deluan <deluan@navidrome.org> * refactor: searchable interface Signed-off-by: Deluan <deluan@navidrome.org> * fix: filter out missing items from subsonic search * fix: filter out missing items from playlists * fix: filter out missing items from shares Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add filter by artist role Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): only return albumartists in getIndexes and getArtists endpoints Signed-off-by: Deluan <deluan@navidrome.org> * sort roles alphabetically Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist playcounts Signed-off-by: Deluan <deluan@navidrome.org> * change default Album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * fix albumartist link when it does not match any albumartists values Signed-off-by: Deluan <deluan@navidrome.org> * fix `Ignoring filter not whitelisted` (role) message Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim any names/titles being imported Signed-off-by: Deluan <deluan@navidrome.org> * remove unused genre code Signed-off-by: Deluan <deluan@navidrome.org> * serialize calls to Last.fm's getArtist Signed-off-by: Deluan <deluan@navidrome.org> xxx Signed-off-by: Deluan <deluan@navidrome.org> * add counters to genres Signed-off-by: Deluan <deluan@navidrome.org> * nit: fix migration `notice` message Signed-off-by: Deluan <deluan@navidrome.org> * optimize similar artists query Signed-off-by: Deluan <deluan@navidrome.org> * fix: last.fm.getInfo when mbid does not exist Signed-off-by: Deluan <deluan@navidrome.org> * ui only show missing items for admins Signed-off-by: Deluan <deluan@navidrome.org> * don't allow interaction with missing items Signed-off-by: Deluan <deluan@navidrome.org> * Add Missing Files view (WIP) Signed-off-by: Deluan <deluan@navidrome.org> * refactor: merged tag_counts into tag table Signed-off-by: Deluan <deluan@navidrome.org> * add option to completely disable automatic scanner Signed-off-by: Deluan <deluan@navidrome.org> * add delete missing files functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: playlists not showing for regular users Signed-off-by: Deluan <deluan@navidrome.org> * reduce updateLastAccess frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * reduce update player frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * add timeout when updating player Signed-off-by: Deluan <deluan@navidrome.org> * remove dead code Signed-off-by: Deluan <deluan@navidrome.org> * fix duplicated roles in stats Signed-off-by: Deluan <deluan@navidrome.org> * add `; ` to artist splitters Signed-off-by: Deluan <deluan@navidrome.org> * fix stats query Signed-off-by: Deluan <deluan@navidrome.org> * more logs Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * add record label filter Signed-off-by: Deluan <deluan@navidrome.org> * add release type filter Signed-off-by: Deluan <deluan@navidrome.org> * fix purgeUnused tags Signed-off-by: Deluan <deluan@navidrome.org> * add grouping filter to albums Signed-off-by: Deluan <deluan@navidrome.org> * allow any album tags to be used in as filters in the API Signed-off-by: Deluan <deluan@navidrome.org> * remove empty tags from album info Signed-off-by: Deluan <deluan@navidrome.org> * comments in the migration Signed-off-by: Deluan <deluan@navidrome.org> * fix: Cannot read properties of undefined Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling (#3640) Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove duplicated tag values Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't ignore the taglib folder! Signed-off-by: Deluan <deluan@navidrome.org> * feat: show track subtitle tag Signed-off-by: Deluan <deluan@navidrome.org> * fix: show artists stats based on selected role Signed-off-by: Deluan <deluan@navidrome.org> * fix: inspect Signed-off-by: Deluan <deluan@navidrome.org> * add media type to album info/filters Signed-off-by: Deluan <deluan@navidrome.org> * fix: change format of subtitle in the UI Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in Subsonic API and search Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in UI's player Signed-off-by: Deluan <deluan@navidrome.org> * fix: split strings should be case-insensitive Signed-off-by: Deluan <deluan@navidrome.org> * disable ScanSchedule Signed-off-by: Deluan <deluan@navidrome.org> * increase default sessiontimeout Signed-off-by: Deluan <deluan@navidrome.org> * add sqlite command line tool to docker image Signed-off-by: Deluan <deluan@navidrome.org> * fix: resources override Signed-off-by: Deluan <deluan@navidrome.org> * fix: album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * change migration to mark current artists as albumArtists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Allow filtering on multiple genres (#3679) * feat(ui): Allow filtering on multiple genres Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> * add multi-genre filter in Album list Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> * add more multi-valued tag filters to Album and Song views Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): unselect missing files after removing Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): song filter Signed-off-by: Deluan <deluan@navidrome.org> * fix sharing tracks. fix #3687 Signed-off-by: Deluan <deluan@navidrome.org> * use rowids when using search for sync (ex: Symfonium) Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients for search Signed-off-by: Deluan <deluan@navidrome.org> * add libraryPath to Native API /songs endpoint Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add album version Signed-off-by: Deluan <deluan@navidrome.org> * made all tags lowercase as they are case-insensitive anyways. Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Show full paths, extended properties for album/song (#3691) * feat(ui): Show full paths, extended properties for album/song - uses library path + os separator + path - show participants (album/song) and tags (song) - make album/participant clickable in show info * add source to path * fix pathSeparator in UI Signed-off-by: Deluan <deluan@navidrome.org> * fix local artist artwork (#3695) Signed-off-by: Deluan <deluan@navidrome.org> * fix: parse vorbis performers Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean function into smaller functions Signed-off-by: Deluan <deluan@navidrome.org> * fix translations for en and pt Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow performers without instrument/subrole Signed-off-by: Deluan <deluan@navidrome.org> * refactor: metadata clean function again Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize split function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split function is now a method of TagConf Signed-off-by: Deluan <deluan@navidrome.org> * fix: humanize Artist total size Signed-off-by: Deluan <deluan@navidrome.org> * add album version to album details Signed-off-by: Deluan <deluan@navidrome.org> * don't display album-level tags in SongInfo Signed-off-by: Deluan <deluan@navidrome.org> * fix genre clicking in Album Page Signed-off-by: Deluan <deluan@navidrome.org> * don't use mbids in Last.fm api calls. From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248: With MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo { artist: { name: "Bee Gees", mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810", url: "https://www.last.fm/music/Bee+Gees", } ``` Without MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo { artist: { name: "Van Morrison", mbid: "a41ac10f-0a56-4672-9161-b83f9b223559", url: "https://www.last.fm/music/Van+Morrison", } ``` Signed-off-by: Deluan <deluan@navidrome.org> * better logging for when the artist folder is not found Signed-off-by: Deluan <deluan@navidrome.org> * fix various issues with artist image resolution Signed-off-by: Deluan <deluan@navidrome.org> * hide "Additional Tags" header if there are none. Signed-off-by: Deluan <deluan@navidrome.org> * simplify tag rendering Signed-off-by: Deluan <deluan@navidrome.org> * enhance logging for artist folder detection Signed-off-by: Deluan <deluan@navidrome.org> * make folderID consistent for relative and absolute folderPaths Signed-off-by: Deluan <deluan@navidrome.org> * handle more folder paths scenarios Signed-off-by: Deluan <deluan@navidrome.org> * filter out other roles when SubsonicArtistParticipations = true Signed-off-by: Deluan <deluan@navidrome.org> * fix "Cannot read properties of undefined" Signed-off-by: Deluan <deluan@navidrome.org> * fix lyrics and comments being truncated (#3701) * fix lyrics and comments being truncated * specifically test for lyrics and comment length * reorder assertions Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * fix(server): Expose library_path for playlist (#3705) Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic) * fix BFR on Windows (#3704) * fix potential reflected cross-site scripting vulnerability Signed-off-by: Deluan <deluan@navidrome.org> * hack to make it work on Windows * ignore windows executables * try fixing the pipeline Signed-off-by: Deluan <deluan@navidrome.org> * allow MusicFolder in other drives * move windows local drive logic to local storage implementation --------- Signed-off-by: Deluan <deluan@navidrome.org> * increase pagination sizes for missing files Signed-off-by: Deluan <deluan@navidrome.org> * reduce level of "already scanning" watcher log message Signed-off-by: Deluan <deluan@navidrome.org> * only count folders with audio files in it See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930 Signed-off-by: Deluan <deluan@navidrome.org> * add album version and catalog number to search Signed-off-by: Deluan <deluan@navidrome.org> * add `organization` alias for `recordlabel` Signed-off-by: Deluan <deluan@navidrome.org> * remove mbid from Last.fm agent Signed-off-by: Deluan <deluan@navidrome.org> * feat: support inspect in ui (#3726) * inspect in ui * address round 1 * add catalogNum to AlbumInfo Signed-off-by: Deluan <deluan@navidrome.org> * remove dependency on metadata_old (deprecated) package Signed-off-by: Deluan <deluan@navidrome.org> * add `RawTags` to model Signed-off-by: Deluan <deluan@navidrome.org> * support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698) * parse standard roles, vorbis/m4a work for now * fix djmixer * working roles, use DJ-mix * add performers to file * map mbids * add a few more tests * add test Signed-off-by: Deluan <deluan@navidrome.org> * try to simplify the performers logic Signed-off-by: Deluan <deluan@navidrome.org> * stylistic changes --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * remove param mutation Signed-off-by: Deluan <deluan@navidrome.org> * run automated SQLite optimizations Signed-off-by: Deluan <deluan@navidrome.org> * fix playlists import/export on Windows * fix import playlists * fix export playlists * better handling of Windows volumes Signed-off-by: Deluan <deluan@navidrome.org> * handle more album ID reassignments Signed-off-by: Deluan <deluan@navidrome.org> * allow adding/overriding tags in the config file Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): Fix playlist track id, handle missing tracks better (#3734) - Use `mediaFileId` instead of `id` for playlist tracks - Only fetch if the file is not missing - If extractor fails to get the file, also error (rather than panic) * optimize DB after each scan. Signed-off-by: Deluan <deluan@navidrome.org> * remove sortable from AlbumSongs columns Signed-off-by: Deluan <deluan@navidrome.org> * simplify query to get missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * mark Scanner.Extractor as deprecated Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Caio Cotts <caio@cotts.com.br> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com> |
||
|
Deluan
|
2d4f483812 |
refactor: remove unnecessary intermediate buffer for ffmpeg image extraction
Signed-off-by: Deluan <deluan@navidrome.org> |
||
|
Deluan Quintão
|
3fda7445b0 |
fix(server): try to find proper embedded front cover - only for vorbis comments for now (#3348)
* fix(artwork): get the first image from vorbis comments, not the last. fixes #3254 This uses a fork for now. * fix(artwork): prioritize getting embedded types that are listed as "front" covers * fix: cleanup |
||
|
Deluan
|
e1ba152a38 | Reduce noise in logs when pre-caching artwork | ||
|
Caio Cotts
|
0488fb92cb |
Fix image stuttering (#3035)
* Fix image stuttering. * Fix docker publishing for PRs * Write tests for new square parameter. * Simplify code for createImage. --------- Co-authored-by: Deluan Quintão <deluan@navidrome.org> |
||
|
Deluan
|
9b81aa4403 |
Fix artwork resolution when paths contains :. Fix #2137
|
||
|
Deluan
|
d8e794317f |
Return 404 when artwork is not available in /share/img endpoint
|
||
|
Kendall Garner
|
93adda66d9 |
Get album info (when available) from Last.fm, add getAlbumInfo endpoint (#2061)
* lastfm album.getInfo, getAlbuminfo(2) endpoints * ... for description and reduce not found log level * address first comments * return all images * Update migration timestamp * Handle a few edge cases * Add CoverArtPriority option to retrieve albumart from external sources * Make agents methods more descriptive * Use Last.fm name consistently Co-authored-by: Deluan <deluan@navidrome.org> |
||
|
Deluan
|
5564f00838 | Some refactor, log message changes | ||
|
Deluan
|
c90468b895 |
Find artist.* image in Artist folder
|
||
|
Deluan
|
bf461473ef | Add local agent, only for images | ||
|
Deluan
|
bc09de6640 | Better error handling | ||
|
Deluan
|
949331ed24 | GetCoverArt generates a tiled (2x2) image for playlists | ||
|
Deluan
|
92ddae4a65 | Created dedicated artwork readers | ||
|
Deluan
|
c1c4645501 | Move artwork handling to its own package |