navidrome/core/agents/interfaces.go

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

178 lines
5.4 KiB
Go
Raw Permalink Normal View History

2021-02-07 16:46:15 -05:00
package agents
import (
"context"
"errors"
fix(scrobbler): back off when a provider asks us to, instead of retrying per play (#6028) * feat(agents): retry-later error type with optional server delay Add agents.ErrRetryLater and agents.RetryLaterError, which carries the delay requested by an external service (e.g. ListenBrainz's X-RateLimit-Reset-In). scrobbler.ErrRetryLater becomes an alias of the new sentinel, so existing errors.Is checks and the plugin error-string protocol keep working unchanged. Groundwork for honoring server-requested retry delays across scrobbling, metadata agents and artwork. Song.Equals tests moved to song_test.go to enable external test package. * fix(scrobbler): honor backoff window and server-requested retry delay ListenBrainz 429s were decoded into a typed error that classified as unrecoverable, silently discarding the scrobble (a JSON-bodied 429 was measured live). The client now maps any 429 to agents.RetryLaterError, carrying X-RateLimit-Reset-In when present (capped at 1h). Last.fm error 29 (rate limit) is now retryable like 11/16. The buffer's drain loop no longer lets wake signals bypass an active backoff window - new plays enqueue but drain only when the window closes - and the wait honors the server delay via max(backoff, retryIn). * feat(agents): skip cooling-down agents in aggregate calls When an agent reports retry-later, remember a per-agent cooldown deadline (the server-requested delay, or 1 minute when unspecified) and skip that agent in all aggregate metadata calls until it passes. A round that found no data but skipped or saw a throttled agent returns ErrRetryLater instead of ErrNotFound, so callers cannot mistake rate limiting for a definitive 'no data' answer. * feat(artwork): honor server-requested retry delay when rescheduling When an external image lookup fails with a retry-later error carrying a delay (e.g. a 429 with X-RateLimit-Reset-In), the chain trace carries the largest such hint back to the worker, which reschedules the item at max(exponential backoff, server delay) instead of backoff alone. * feat(plugins): retry-later with optional delay for scrobbler and agent plugins Scrobbler plugins can now return scrobbler(retry_later:N) to request a retry in N seconds (capped at 1h); the bare token keeps its old meaning. Metadata-agent plugins, which had no error vocabulary at all, gain the parallel agent(retry_later[:N]) token, mapped to agents.RetryLaterError so the aggregate's cooldown and the artwork worker honor plugin throttling the same way as built-in agents. * fix: address whole-branch review findings for retry-later handling Narrow the aggregate's throttled rule to the spec sentence: core.Agents returns ErrRetryLater only when no agent answered at all (all skipped-cooling or retry-later). An agent that does not implement the called method now returns an internal errUnsupported instead of ErrNotFound, so it counts as "did not run" — without that, the always-appended local agent would answer for biography, URL and images and make ErrRetryLater unreachable. Wire the consequence in core/external: a throttled round no longer stamps ExternalInfoUpdatedAt (artist and album), so the empty result is not cached for the TTL, and TopSongs maps ErrRetryLater to the same empty-200 the not-found path already produced instead of a new client-facing error. Move the Last.fm code-29 mapping into the client's central error construction so every metadata path produces RetryLaterError, and map ListenBrainz's body-level code 429 (sent with a non-429 HTTP status) the same way. Clamp server- and plugin-requested delays in seconds before scaling to a Duration, in all three parse sites: a header of 18446744074 wrapped past 2^64 and came out as a 0.29s delay. Also: extract the artwork worker's reschedule computation into retryDelay() and cover both it and the trace RetryIn wiring with tests; collapse the double regex call in mapScrobblerError; drop capabilities.ScrobblerErrorRetryLaterIn (ndpgen never emits funcs, so plugin authors could not reach it); regenerate the PDKs so MetadataAgentError reaches the Go and Rust SDKs; de-flake the cooldown tests (long RetryIn for the skip case, separate expiry spec); and cover the max() retry-delay aggregation across users in the scrobble buffer. * refactor: dedupe retry-later parsing and simplify error collection - Add agents.NewRetryLater and agents.RetryLaterFromSeconds, with a single 1h cap, replacing the parse+clamp+multiply logic and the maxRetryInSeconds constant duplicated across listenbrainz, plugins and the agent adapter. - Move HTTP header parsing to httpclient.RetryAfter, so the transport layer owns it and stays domain-agnostic; drop retryInFromHeaders from the ListenBrainz client. Covered by a new Ginkgo table in that package. - Collapse the two near-identical plugin retry_later regexes into one parseRetryLater(prefix, msg) shared by the agent and scrobbler adapters. - Fold the duplicated noteRetryIn snippet from fetchArtistImage and fetchAlbumImage into recordAgent, which already branched on the same isTransientExternal condition. - Replace the atomic.Bool + note() closure in populateArtistInfo with errgroup's own error collection; the group carries no context, so a returned error does not cancel its siblings. - Reuse recoveringScrobbler for the per-user delay test instead of a third double, and switch fakeScrobbler's mutex-guarded error to the atomic.Pointer idiom already used in the same package. * refactor(listenbrainz): keep rate-limit header parsing in the adapter The X-RateLimit-Reset-In header is ListenBrainz's own convention, not a shared one: Last.fm sends no rate-limit headers at all and reports its limit as a body code, and no other integration in tree sends Retry-After. A parser in utils/httpclient implied a uniformity across services that does not exist, so it moves back next to the only client that can know which header its service sends. * refactor(agents): collapse the retry-later sentinel and error into one type ErrRetryLater is now the zero-delay RetryLaterError rather than a separate errors.New value, so errors.Is and errors.AsType both match the sentinel and every delay-carrying variant. That removes the trap where a bare sentinel silently skipped the AsType path, and lets every consumer read the delay off the error directly: the RetryIn accessor and the two constructors are gone, with the policy cap applied where untrusted input is parsed. * refactor(agents): split the cooldown store from the per-dispatch tally The cooldown map and mutex become a cooldowns value with active/park, holding no knowledge of errors; agentAttempts records one dispatch's outcomes and owns the classification that noteAgentError used to hide behind a bool. The three dispatch loops now touch a single object: skip folds the cooldown check and the throttled flag into one call, so the store never appears in the loops. * refactor(agents): share one dispatch loop between the agent call helpers callAgentMethod and callAgentSliceMethod ran identical loops, differing only in how they test a result for emptiness: a slice cannot be compared against its zero value, so the two could not share a constraint. Both now delegate to callAgent, which takes that test as a parameter. Keeping the loop in one place matters more than the lines saved: it holds the cooldown skip, the attempt recording and the empty-dispatch verdict, and a fix applied to one copy but not the other would be silent. * test: cover the two retry-later paths a mutation could break silently Both gaps were proven, not guessed: making the artwork worker pass 0 instead of the collected hint left all 386 specs green, and replacing the default agent cooldown with 0 left the agents suite green. The worker test drives a throttled image agent through drain and asserts the persisted retry_at, and the cooldown test parks an agent that asked to be retried without naming a delay, which is what Last.fm does on every rate limit. * refactor(artwork): carry the external failure as an error, not a flag plus a trace field The retry delay was riding on ChainTrace, a diagnostic that gets persisted, while the very same signal — an external source faulted — already travelled by value as resolution.extError. That was two mechanisms for one idea, and it put control-flow state inside a serializable trace. resolution.extError and chainState.extErr become the error itself, so a caller checks err != nil for the fault and errors.AsType for the delay the provider asked for. The agent loops return that error last, per convention, and longerRetry keeps whichever failure wants the longer wait. ChainTrace goes back to holding only steps and no longer imports core/agents. * fix(artwork): check the resolve error before reading its resolution Reading res.extError before the err check was safe only because every error path in resolve returns a bare resolution{}; a future path returning a partly-filled one would have been read silently. The failure path now returns no delay explicitly. * test(artwork): assert the delay acquire reports, not just its downstream effect acquire's retry delay was only covered through the worker's persisted retry_at, one layer away from where the value is computed. Both outcomes are now pinned at the processor: a plain failure asks for nothing, a throttled provider's delay is passed through. * refactor: share the retry-seconds parse and drop the backoff deadline arithmetic The clamp-before-scaling invariant lived in two parsers and was independently re-tested in three files with the same magic number; a fix applied to one copy would have left the others wrapping a huge value down to a fraction of a second. It moves to agents.ParseRetryIn. The buffer tracked an absolute retryDeadline only to re-arm a timer that was already armed for the same instant; a backingOff flag says the same thing without the arithmetic. The plugin token regex now carries its capability in the pattern instead of capturing and comparing, so another capability's token in the same message cannot mask it. resolution.extError becomes extErr, matching its chainState counterpart. * fix(agents): keep the longer cooldown when parks overlap Calls to one agent overlap, so a short cooldown could land after a long one started and cut it short. park now keeps whichever deadline is later, matching the rule longerRetry already applies on the artwork side. No in-tree provider can currently produce two different delays for the same agent, so this is hardening rather than a fix for observed behaviour. * fix(agents): parse the retry delay at a fixed width strconv.Atoi parses into the native int, so on the 32-bit targets we ship (linux/386, windows/386, three ARM variants) a delay above MaxInt32 seconds overflowed and became unspecified instead of being capped. No provider sends a 68-year delay, so this is not user-visible, but the overflow tests asserted the cap and would have failed on those architectures, where tests never run. * fix(plugins): anchor the retry_later regex to a word boundary Prevents a superstring like useragent(retry_later) from matching the agent capability token.
2026-08-29 17:28:29 -04:00
"fmt"
"strconv"
"time"
feat(matcher): match similar/top songs by multiple artists (#5668) * feat(matcher): add Song.Artists (agents.Artist) and field-wise song dedup * refactor(matcher): make song equality an agents.Song.Equals method via hashstructure Move the sameSong free function from core/matcher into an Equals method on agents.Song, following the model.MediaFile/Album.Equals convention. Uses strict hashstructure hashing (nil opts, no IgnoreZeroValue) to preserve the original whole-value equality contract. Tests moved to core/agents. * feat(matcher): match by multiple artists with overlap ranking and artist-ID fast-path * refactor(matcher): rank artist overlap and specificity above the preferred-track flag Identity signals (specificityLevel, artistOverlap) now outrank the taste signal (preferredMatch) in betterThan. A starred/4-star track that is a worse identity match no longer beats a more specific or higher-overlap track. PreferStarred still breaks ties when specificity and overlap are equal. * fix(matcher): score artist-MBID specificity against all credited artists, not just the last sanitizedTrack.artistMBID (string) replaced with artistMBIDs (map[string]struct{}) so bucketTracks collects all credited owned MBIDs per query instead of last-write-wins. computeSpecificityLevel tests set membership, letting each of a collaboration's MBID-bearing artists reach the proper specificity level (4/5) independently. * feat(plugins): carry multiple artists (with IDs) through SongRef conversions * feat(plugins): regenerate schemas and PDK wrappers for multi-artist SongRef * refactor(matcher): tidy bucketTracks accumulator and artist resolution Replace bucketTracks' two parallel per-track maps (overlapByQuery/mbidsByQuery) with a named queryAccum struct (F2). Collapse resolveArtists' four hand-mutated parallel maps into a pendingArtist slice with derived nameToQueries/mbidToQueries maps (F1). Replace the own() method on resolvedArtists with a package-level addToSet helper that drops the method/receiver indirection (F3). * docs(matcher): trim comments that restate the code * fix(plugins): use Vec::is_empty for slice fields in generated Rust PDK * fix(matcher): treat a resolved artist ID as an identity match for specificity * docs(matcher): reflect artist-ID identity in the specificity ladder
2026-06-26 14:46:09 -04:00
"github.com/gohugoio/hashstructure"
"github.com/navidrome/navidrome/model"
2021-02-07 16:46:15 -05:00
)
type Constructor func(ds model.DataStore) Interface
2021-02-07 16:46:15 -05:00
2021-02-08 10:14:29 -05:00
type Interface interface {
AgentName() string
}
2021-02-07 16:46:15 -05:00
feat(plugins): experimental support for plugins (#3998) * feat(plugins): add minimal test agent plugin with API definitions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add plugin manager with auto-registration and unique agent names Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator. * test: add Ginkgo suite and test for plugin manager auto-registration Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support. * test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement persistent compilation cache for WASM agent plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmAgent to improve resource management Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance logging for wasmAgent and plugin manager operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement HttpService for handling HTTP requests in WASM plugins Also add a sample Wikimedia plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): standardize error handling in wasmAgent and MinimalAgent Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean up wikimedia plugin code Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy. * refactor: use unified SPARQLResult struct and parser for SPARQL responses Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability. * feat(plugins): improve error handling in HTTP request processing Signed-off-by: Deluan <deluan@navidrome.org> * fix: background plugin compilation, logging, and race safety Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass. * perf: implement true lazy loading for agents Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass. * fix: ensure wasm plugin instances are closed via runtime.AddCleanup Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup. * refactor: break down long functions in plugin manager and agent Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go. Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename wasmAgent to wasmArtistAgent Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename MinimalAgent for artist metadata service Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin pool creation with custom runtime and precompilation support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement generic plugin pool and agent factory for improved service handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize plugin management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): improve function signatures for clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement background precompilation for plugins and agent factory creation Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): include instanceID in logging for better traceability Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add tests for plugin pre-compilation and agent factory synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add minimal album test agent plugin for AlbumMetadataService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename fake artist and album test agent plugins for metadata services Signed-off-by: Deluan <deluan@navidrome.org> * feat(makefile): add Makefile for building plugin WASM binaries Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized. * feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove wasm module pooling it was causing issues with the GC and the Close methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename metadata service files to adapter naming convention Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle nil values in data redaction process Signed-off-by: Deluan <deluan@navidrome.org> * fix: add timeout for plugin compilation to prevent indefinite blocking Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ScrobblerService plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify generalization Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin management by improving scanning and loading mechanisms Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin creation functions to return specific interfaces for better type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement MediaMetadataService with combined artist and album methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve MediaMetadataService plugin implementation and testing structure Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add tests for Adapter Media Agent and improve plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance agent management with plugin loading and caching Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update agent discovery logic to include only local agent when no config is specified Signed-off-by: Deluan <deluan@navidrome.org> * refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct file extension filter in goimports command Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use defer to unlock the mutex Signed-off-by: Deluan <deluan@navidrome.org> * chore: move Cover Art Archive AlbumMetadataService plugins to an example folder Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle errors when creating media metadata and scrobbler service plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: increase compilation timeout to one minute Signed-off-by: Deluan <deluan@navidrome.org> * feat: add configurable plugin compilation timeout Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin scrobbler support in PlayTracker Signed-off-by: Deluan <deluan@navidrome.org> * feat: add context management and Stop method to buffered scrobbler Signed-off-by: Deluan <deluan@navidrome.org> * feat: add username field to scrobbler requests and update logging Signed-off-by: Deluan <deluan@navidrome.org> * fix: data race in test Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename http proto files to host and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused plugin registration methods from manager Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin manifests and implement plugin management commands Signed-off-by: Deluan <deluan@navidrome.org> * Update utils/files.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip") Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * feat: add plugin dev workflow support Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(README): update documentation to include TimerCallbackService and its functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add InitService with OnInit method and initialization tracking - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add tests for InitService and plugin initialization tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): expand documentation on plugin system implementation and architecture Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): redirect plugins' stderr to logs Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add safe accessor methods for TimerService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add plugin-specific configuration support in InitRequest and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add TimerCallbackService plugin adapter and integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename services for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add mutex for configuration access and clone plugin config Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): remove configtest dependency to prevent data races in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): correctly pass error to done function in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename service types to capabilities for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename InitService to LifecycleManagement for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): fix instance ID logging in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add config service for plugin-specific configuration management Signed-off-by: Deluan <deluan@navidrome.org> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat(crontab): implement crontab service for scheduling and canceling jobs Signed-off-by: Deluan <deluan@navidrome.org> * fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit) Squashed commits: [325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService Signed-off-by: Deluan <deluan@navidrome.org> * fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): consolidate job scheduling logic into a single helper function Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename GetInstance method to Instantiate for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add WebSocket service for handling connections and messages Signed-off-by: Deluan <deluan@navidrome.org> * feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): enhance connection management and callback handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): only create one adapter instance for each adapter/capability pair Signed-off-by: Deluan <deluan@navidrome.org> * fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): refactor WebSocket service integration and improve error logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): add SchedulerCallback support and improve reconnection logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: test panic Signed-off-by: Deluan <deluan@navidrome.org> * docs: add crypto-ticker plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): add error field to response messages for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(cache): implement CacheService with string, int, float, and byte operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo Signed-off-by: Deluan <deluan@navidrome.org> * refactor(cache): simplify cache service implementation and remove unnecessary synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): add build step for test plugins in the test suite Signed-off-by: Deluan <deluan@navidrome.org> * wip Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(rpc): enhance activity image processing and improve error handling in Discord integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(discord): enhance activity state with artist list and add large text asset Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(artwork): implement ArtworkService for retrieving artwork URLs Signed-off-by: Deluan <deluan@navidrome.org> * Add playback position to scrobble NowPlaying (#4089) * test(playtracker): cover playback position * address review comment Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix merge Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unnecessary check for empty slice in Map function Signed-off-by: Deluan <deluan@navidrome.org> * fix: update reflex.conf to include .wasm file extension Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): normalize attribute strings and add edge case tests for PID calculation Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458 Signed-off-by: Deluan <deluan@navidrome.org> * test(ui): fix warnings (#4187) * fix(ui): address test warnings * ignore lint error in test Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * refactor(server): optimize top songs lookup (#4189) * optimize top songs lookup * Optimize title matching queries * refactor: simplify top songs matching * improve error handling and logging in track loading functions Signed-off-by: Deluan <deluan@navidrome.org> * test: add cases for fallback to title matching and combined MBID/title matching Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): playlist details overflow in spotify-based themes (#4184) * test: ensure playlist details width * fix(test): simplify expectation for minWidth in NDPlaylistDetails Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): test all themes Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update TagLib to version 2.1 (#4185) * chore: update cross-taglib * fix(taglib): add logging for TagLib version Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * test: verify agents fallback (#4191) * build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version. This is to reduce the image size, as we don't really need the latest. Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(runtime): implement pooled WASM runtime and module for better instance management Signed-off-by: Deluan <deluan@navidrome.org> * fix(discord-plugin): adjust timer delay calculation for track completion Signed-off-by: Deluan <deluan@navidrome.org> * resolve PR comments Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement cache cleanup by size functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(manager): return error from getCompilationCache and handle it in ScanPlugins Signed-off-by: Deluan <deluan@navidrome.org> * fix possible rce condition Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): update README to include Cache and Artwork services Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add permissions support for host services in custom runtime - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add permissions field to plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * test(permissions): implement permission validation and testing for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add unauthorized_plugin to test permission enforcement - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): add Plugin Permission System section to README - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add detailed reasons for permissions in plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement granular HTTP permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unexport all plugins package private symbols Signed-off-by: Deluan <deluan@navidrome.org> * update docs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin_lifecycle_manager Signed-off-by: Deluan <deluan@navidrome.org> * docs: add discord-rich-presence plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for PATCH, HEAD, and OPTIONS HTTP methods Signed-off-by: Deluan <deluan@navidrome.org> * feat: use folder names as unique identifiers for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: read config just once, to avoid data race in tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginName to pluginID for consistency across services Signed-off-by: Deluan <deluan@navidrome.org> * fix: use symlink name instead of folder name for plugin registration Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin output format to include ID and enhance README with symlink usage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement shared plugin discovery function to streamline plugin scanning and error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: show plugin permissions in `plugin info` Signed-off-by: Deluan <deluan@navidrome.org> * feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement typed permissions for plugins to enhance permission handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor plugin permissions to use typed schema and improve validation - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused JSON schema validation for plugin manifests Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused fields from PluginPackage struct in package.go Signed-off-by: Deluan <deluan@navidrome.org> * feat: update file permissions in tests and remove unused permission parsing function Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor test plugin creation to use typed permissions and remove legacy helper Signed-off-by: Deluan <deluan@navidrome.org> * feat: add website field to plugin manifests and update test cases Signed-off-by: Deluan <deluan@navidrome.org> * refactor: permission schema to use basePermission structure for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance host service management by adding permission checks for each service Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reorganize code files Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify custom runtime creation by removing compilation cache parameter Signed-off-by: Deluan <deluan@navidrome.org> * doc: add WebSocketService and update ConfigService for plugin-specific configuration Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WASM loading optimization to enhance plugin instance creation speed Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename custom runtime functions and update related tests for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin structure with compilation handling and error reporting Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve logging and context tracing in runtime and wasm base plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance runtime management with scoped runtime and caching improvements Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement EnsureCompiled method for improved plugin compilation handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement cached module management with TTL for improved performance Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace map with sync.Map Signed-off-by: Deluan <deluan@navidrome.org> * refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance image processing with fallback mechanism for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * docs: review test plugins readme Signed-off-by: Deluan <deluan@navidrome.org> * feat: set default timeout for HTTP client to 10 seconds Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance wasm instance pool with concurrency limits and timeout settings Signed-off-by: Deluan <deluan@navidrome.org> * feat(discordrp): implement caching for processed image URLs with configurable TTL Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00
// AlbumInfo contains album metadata (no images)
type AlbumInfo struct {
Name string
MBID string
Description string
URL string
}
2021-02-07 16:46:15 -05:00
type Artist struct {
ID string
2021-02-07 16:46:15 -05:00
Name string
MBID string
}
type ExternalImage struct {
2021-02-07 16:46:15 -05:00
URL string
Size int
}
2021-02-08 15:53:07 -05:00
type Song struct {
feat(listenbrainz): match collaboration top-songs via all credited artist MBIDs (#5670) * refactor(agents): drop single Artist fields from Song, keep only Artists Song now represents credited artists solely via the Artists slice; the single Artist/ArtistMBID fields and the ArtistList passthrough are removed. Equals continues to hash the whole value. * refactor(lastfm,listenbrainz): build Song.Artists in built-in agents Last.fm and ListenBrainz now populate the Artists slice directly. For ListenBrainz top songs, all credited artist MBIDs are mapped (the combined display name on the first credit plus MBID-only collaborators) instead of keeping only the first MBID, feeding the matcher's per-MBID specificity. * refactor(external): set Song.Artists in top-songs enrichment getMatchingTopSongs now seeds an Artists entry from the known artist when a song carries none, replacing the single Artist/ArtistMBID writes. * refactor(plugins): fold single-artist SongRef into Song.Artists SongRef keeps its single Artist/ArtistMBID fields as part of the plugin wire contract; songRefToAgentSong now folds them into a one-element Artists list when a plugin sends no artists array. * refactor(matcher): read Song.Artists directly groupQueries consumes s.Artists now that ArtistList is gone; test inputs build the Artists slice. * fix(matcher): keep MBID-only artists as identity signals Review follow-up: an artist credited only by MBID (empty ID and name) was dropped before resolution in four places, defeating the multi-MBID matching path this PR adds. - matcher.groupQueries: treat a non-empty MBID as a usable artist signal - external.getMatchingTopSongs: backfill the primary credit's name/MBID when the agent left them empty - plugins.songRefToAgentSong: fold a single-artist SongRef when only ArtistMBID is set (not just when Artist name is set) - listenbrainz.topSongArtists: return nil instead of an empty-name placeholder when neither name nor MBIDs are present * fix(external): only backfill top-song artist onto an unnamed credit Review follow-up (codex P2): the previous backfill stamped the queried artist's MBID onto Artists[0] whenever it was empty, even when that credit already named a different (e.g. featured) artist — producing a mismatched name+MBID pair that could mis-rank matches. Now only an unnamed first credit is filled (it is, by construction, the queried artist); an already-named credit is left untouched. Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2026-06-26 17:06:14 -04:00
ID string
Name string
MBID string
ISRC string
Artists []Artist
Album string
AlbumMBID string
Duration uint32 // Duration in milliseconds, 0 means unknown
2021-02-07 16:46:15 -05:00
}
feat(matcher): match similar/top songs by multiple artists (#5668) * feat(matcher): add Song.Artists (agents.Artist) and field-wise song dedup * refactor(matcher): make song equality an agents.Song.Equals method via hashstructure Move the sameSong free function from core/matcher into an Equals method on agents.Song, following the model.MediaFile/Album.Equals convention. Uses strict hashstructure hashing (nil opts, no IgnoreZeroValue) to preserve the original whole-value equality contract. Tests moved to core/agents. * feat(matcher): match by multiple artists with overlap ranking and artist-ID fast-path * refactor(matcher): rank artist overlap and specificity above the preferred-track flag Identity signals (specificityLevel, artistOverlap) now outrank the taste signal (preferredMatch) in betterThan. A starred/4-star track that is a worse identity match no longer beats a more specific or higher-overlap track. PreferStarred still breaks ties when specificity and overlap are equal. * fix(matcher): score artist-MBID specificity against all credited artists, not just the last sanitizedTrack.artistMBID (string) replaced with artistMBIDs (map[string]struct{}) so bucketTracks collects all credited owned MBIDs per query instead of last-write-wins. computeSpecificityLevel tests set membership, letting each of a collaboration's MBID-bearing artists reach the proper specificity level (4/5) independently. * feat(plugins): carry multiple artists (with IDs) through SongRef conversions * feat(plugins): regenerate schemas and PDK wrappers for multi-artist SongRef * refactor(matcher): tidy bucketTracks accumulator and artist resolution Replace bucketTracks' two parallel per-track maps (overlapByQuery/mbidsByQuery) with a named queryAccum struct (F2). Collapse resolveArtists' four hand-mutated parallel maps into a pendingArtist slice with derived nameToQueries/mbidToQueries maps (F1). Replace the own() method on resolvedArtists with a package-level addToSet helper that drops the method/receiver indirection (F3). * docs(matcher): trim comments that restate the code * fix(plugins): use Vec::is_empty for slice fields in generated Rust PDK * fix(matcher): treat a resolved artist ID as an identity match for specificity * docs(matcher): reflect artist-ID identity in the specificity ladder
2026-06-26 14:46:09 -04:00
// Equals reports strict whole-value equality, used to dedup identical input songs. It hashes
// rather than comparing with ==, which the Artists slice makes illegal.
func (s Song) Equals(other Song) bool {
h1, _ := hashstructure.Hash(s, nil)
h2, _ := hashstructure.Hash(other, nil)
return h1 == h2
}
fix(scrobbler): back off when a provider asks us to, instead of retrying per play (#6028) * feat(agents): retry-later error type with optional server delay Add agents.ErrRetryLater and agents.RetryLaterError, which carries the delay requested by an external service (e.g. ListenBrainz's X-RateLimit-Reset-In). scrobbler.ErrRetryLater becomes an alias of the new sentinel, so existing errors.Is checks and the plugin error-string protocol keep working unchanged. Groundwork for honoring server-requested retry delays across scrobbling, metadata agents and artwork. Song.Equals tests moved to song_test.go to enable external test package. * fix(scrobbler): honor backoff window and server-requested retry delay ListenBrainz 429s were decoded into a typed error that classified as unrecoverable, silently discarding the scrobble (a JSON-bodied 429 was measured live). The client now maps any 429 to agents.RetryLaterError, carrying X-RateLimit-Reset-In when present (capped at 1h). Last.fm error 29 (rate limit) is now retryable like 11/16. The buffer's drain loop no longer lets wake signals bypass an active backoff window - new plays enqueue but drain only when the window closes - and the wait honors the server delay via max(backoff, retryIn). * feat(agents): skip cooling-down agents in aggregate calls When an agent reports retry-later, remember a per-agent cooldown deadline (the server-requested delay, or 1 minute when unspecified) and skip that agent in all aggregate metadata calls until it passes. A round that found no data but skipped or saw a throttled agent returns ErrRetryLater instead of ErrNotFound, so callers cannot mistake rate limiting for a definitive 'no data' answer. * feat(artwork): honor server-requested retry delay when rescheduling When an external image lookup fails with a retry-later error carrying a delay (e.g. a 429 with X-RateLimit-Reset-In), the chain trace carries the largest such hint back to the worker, which reschedules the item at max(exponential backoff, server delay) instead of backoff alone. * feat(plugins): retry-later with optional delay for scrobbler and agent plugins Scrobbler plugins can now return scrobbler(retry_later:N) to request a retry in N seconds (capped at 1h); the bare token keeps its old meaning. Metadata-agent plugins, which had no error vocabulary at all, gain the parallel agent(retry_later[:N]) token, mapped to agents.RetryLaterError so the aggregate's cooldown and the artwork worker honor plugin throttling the same way as built-in agents. * fix: address whole-branch review findings for retry-later handling Narrow the aggregate's throttled rule to the spec sentence: core.Agents returns ErrRetryLater only when no agent answered at all (all skipped-cooling or retry-later). An agent that does not implement the called method now returns an internal errUnsupported instead of ErrNotFound, so it counts as "did not run" — without that, the always-appended local agent would answer for biography, URL and images and make ErrRetryLater unreachable. Wire the consequence in core/external: a throttled round no longer stamps ExternalInfoUpdatedAt (artist and album), so the empty result is not cached for the TTL, and TopSongs maps ErrRetryLater to the same empty-200 the not-found path already produced instead of a new client-facing error. Move the Last.fm code-29 mapping into the client's central error construction so every metadata path produces RetryLaterError, and map ListenBrainz's body-level code 429 (sent with a non-429 HTTP status) the same way. Clamp server- and plugin-requested delays in seconds before scaling to a Duration, in all three parse sites: a header of 18446744074 wrapped past 2^64 and came out as a 0.29s delay. Also: extract the artwork worker's reschedule computation into retryDelay() and cover both it and the trace RetryIn wiring with tests; collapse the double regex call in mapScrobblerError; drop capabilities.ScrobblerErrorRetryLaterIn (ndpgen never emits funcs, so plugin authors could not reach it); regenerate the PDKs so MetadataAgentError reaches the Go and Rust SDKs; de-flake the cooldown tests (long RetryIn for the skip case, separate expiry spec); and cover the max() retry-delay aggregation across users in the scrobble buffer. * refactor: dedupe retry-later parsing and simplify error collection - Add agents.NewRetryLater and agents.RetryLaterFromSeconds, with a single 1h cap, replacing the parse+clamp+multiply logic and the maxRetryInSeconds constant duplicated across listenbrainz, plugins and the agent adapter. - Move HTTP header parsing to httpclient.RetryAfter, so the transport layer owns it and stays domain-agnostic; drop retryInFromHeaders from the ListenBrainz client. Covered by a new Ginkgo table in that package. - Collapse the two near-identical plugin retry_later regexes into one parseRetryLater(prefix, msg) shared by the agent and scrobbler adapters. - Fold the duplicated noteRetryIn snippet from fetchArtistImage and fetchAlbumImage into recordAgent, which already branched on the same isTransientExternal condition. - Replace the atomic.Bool + note() closure in populateArtistInfo with errgroup's own error collection; the group carries no context, so a returned error does not cancel its siblings. - Reuse recoveringScrobbler for the per-user delay test instead of a third double, and switch fakeScrobbler's mutex-guarded error to the atomic.Pointer idiom already used in the same package. * refactor(listenbrainz): keep rate-limit header parsing in the adapter The X-RateLimit-Reset-In header is ListenBrainz's own convention, not a shared one: Last.fm sends no rate-limit headers at all and reports its limit as a body code, and no other integration in tree sends Retry-After. A parser in utils/httpclient implied a uniformity across services that does not exist, so it moves back next to the only client that can know which header its service sends. * refactor(agents): collapse the retry-later sentinel and error into one type ErrRetryLater is now the zero-delay RetryLaterError rather than a separate errors.New value, so errors.Is and errors.AsType both match the sentinel and every delay-carrying variant. That removes the trap where a bare sentinel silently skipped the AsType path, and lets every consumer read the delay off the error directly: the RetryIn accessor and the two constructors are gone, with the policy cap applied where untrusted input is parsed. * refactor(agents): split the cooldown store from the per-dispatch tally The cooldown map and mutex become a cooldowns value with active/park, holding no knowledge of errors; agentAttempts records one dispatch's outcomes and owns the classification that noteAgentError used to hide behind a bool. The three dispatch loops now touch a single object: skip folds the cooldown check and the throttled flag into one call, so the store never appears in the loops. * refactor(agents): share one dispatch loop between the agent call helpers callAgentMethod and callAgentSliceMethod ran identical loops, differing only in how they test a result for emptiness: a slice cannot be compared against its zero value, so the two could not share a constraint. Both now delegate to callAgent, which takes that test as a parameter. Keeping the loop in one place matters more than the lines saved: it holds the cooldown skip, the attempt recording and the empty-dispatch verdict, and a fix applied to one copy but not the other would be silent. * test: cover the two retry-later paths a mutation could break silently Both gaps were proven, not guessed: making the artwork worker pass 0 instead of the collected hint left all 386 specs green, and replacing the default agent cooldown with 0 left the agents suite green. The worker test drives a throttled image agent through drain and asserts the persisted retry_at, and the cooldown test parks an agent that asked to be retried without naming a delay, which is what Last.fm does on every rate limit. * refactor(artwork): carry the external failure as an error, not a flag plus a trace field The retry delay was riding on ChainTrace, a diagnostic that gets persisted, while the very same signal — an external source faulted — already travelled by value as resolution.extError. That was two mechanisms for one idea, and it put control-flow state inside a serializable trace. resolution.extError and chainState.extErr become the error itself, so a caller checks err != nil for the fault and errors.AsType for the delay the provider asked for. The agent loops return that error last, per convention, and longerRetry keeps whichever failure wants the longer wait. ChainTrace goes back to holding only steps and no longer imports core/agents. * fix(artwork): check the resolve error before reading its resolution Reading res.extError before the err check was safe only because every error path in resolve returns a bare resolution{}; a future path returning a partly-filled one would have been read silently. The failure path now returns no delay explicitly. * test(artwork): assert the delay acquire reports, not just its downstream effect acquire's retry delay was only covered through the worker's persisted retry_at, one layer away from where the value is computed. Both outcomes are now pinned at the processor: a plain failure asks for nothing, a throttled provider's delay is passed through. * refactor: share the retry-seconds parse and drop the backoff deadline arithmetic The clamp-before-scaling invariant lived in two parsers and was independently re-tested in three files with the same magic number; a fix applied to one copy would have left the others wrapping a huge value down to a fraction of a second. It moves to agents.ParseRetryIn. The buffer tracked an absolute retryDeadline only to re-arm a timer that was already armed for the same instant; a backingOff flag says the same thing without the arithmetic. The plugin token regex now carries its capability in the pattern instead of capturing and comparing, so another capability's token in the same message cannot mask it. resolution.extError becomes extErr, matching its chainState counterpart. * fix(agents): keep the longer cooldown when parks overlap Calls to one agent overlap, so a short cooldown could land after a long one started and cut it short. park now keeps whichever deadline is later, matching the rule longerRetry already applies on the artwork side. No in-tree provider can currently produce two different delays for the same agent, so this is hardening rather than a fix for observed behaviour. * fix(agents): parse the retry delay at a fixed width strconv.Atoi parses into the native int, so on the 32-bit targets we ship (linux/386, windows/386, three ARM variants) a delay above MaxInt32 seconds overflowed and became unspecified instead of being capped. No provider sends a 68-year delay, so this is not user-visible, but the overflow tests asserted the cap and would have failed on those architectures, where tests never run. * fix(plugins): anchor the retry_later regex to a word boundary Prevents a superstring like useragent(retry_later) from matching the agent capability token.
2026-08-29 17:28:29 -04:00
// ErrNotFound means the provider answered and had nothing. Return the underlying error
// for a fault instead, or callers that back off on faults will treat it as definitive.
var ErrNotFound = errors.New("not found")
// ErrRetryLater is the zero-delay RetryLaterError: the provider is temporarily unavailable
// or throttling us, but did not say for how long. Both errors.Is(err, ErrRetryLater) and
// errors.AsType[*RetryLaterError] match it and every delay-carrying variant.
// Treat it as immutable; build a new RetryLaterError to name a delay.
var ErrRetryLater = &RetryLaterError{}
// RetryLaterError asks callers to back off, optionally for the delay the provider requested.
type RetryLaterError struct {
RetryIn time.Duration
}
func (e *RetryLaterError) Error() string {
if e.RetryIn > 0 {
return fmt.Sprintf("retry later (in %s)", e.RetryIn)
}
return "retry later"
}
func (e *RetryLaterError) Is(target error) bool {
_, ok := target.(*RetryLaterError)
return ok
}
// MaxRetryIn caps a delay parsed from a provider, so a bogus value cannot park it indefinitely.
const MaxRetryIn = time.Hour
const maxRetryInSeconds = int(MaxRetryIn / time.Second)
// ParseRetryIn reads a provider's delay given in seconds, from a header or a plugin token.
// Anything unparseable or non-positive means unspecified.
func ParseRetryIn(seconds string) time.Duration {
// Clamp in seconds: scaling first would wrap a huge value past int64 nanoseconds,
// turning "wait an age" into a fraction of a second. Parse at a fixed width so the
// cap holds on the 32-bit targets we ship, where a plain Atoi would overflow first.
secs, err := strconv.ParseInt(seconds, 10, 64)
if err != nil || secs <= 0 {
return 0
}
return time.Duration(min(secs, int64(maxRetryInSeconds))) * time.Second
}
2021-02-07 16:46:15 -05:00
feat(plugins): experimental support for plugins (#3998) * feat(plugins): add minimal test agent plugin with API definitions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add plugin manager with auto-registration and unique agent names Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator. * test: add Ginkgo suite and test for plugin manager auto-registration Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support. * test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement persistent compilation cache for WASM agent plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmAgent to improve resource management Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance logging for wasmAgent and plugin manager operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement HttpService for handling HTTP requests in WASM plugins Also add a sample Wikimedia plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): standardize error handling in wasmAgent and MinimalAgent Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean up wikimedia plugin code Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy. * refactor: use unified SPARQLResult struct and parser for SPARQL responses Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability. * feat(plugins): improve error handling in HTTP request processing Signed-off-by: Deluan <deluan@navidrome.org> * fix: background plugin compilation, logging, and race safety Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass. * perf: implement true lazy loading for agents Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass. * fix: ensure wasm plugin instances are closed via runtime.AddCleanup Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup. * refactor: break down long functions in plugin manager and agent Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go. Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename wasmAgent to wasmArtistAgent Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename MinimalAgent for artist metadata service Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin pool creation with custom runtime and precompilation support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement generic plugin pool and agent factory for improved service handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize plugin management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): improve function signatures for clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement background precompilation for plugins and agent factory creation Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): include instanceID in logging for better traceability Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add tests for plugin pre-compilation and agent factory synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add minimal album test agent plugin for AlbumMetadataService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename fake artist and album test agent plugins for metadata services Signed-off-by: Deluan <deluan@navidrome.org> * feat(makefile): add Makefile for building plugin WASM binaries Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized. * feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove wasm module pooling it was causing issues with the GC and the Close methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename metadata service files to adapter naming convention Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle nil values in data redaction process Signed-off-by: Deluan <deluan@navidrome.org> * fix: add timeout for plugin compilation to prevent indefinite blocking Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ScrobblerService plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify generalization Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin management by improving scanning and loading mechanisms Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin creation functions to return specific interfaces for better type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement MediaMetadataService with combined artist and album methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve MediaMetadataService plugin implementation and testing structure Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add tests for Adapter Media Agent and improve plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance agent management with plugin loading and caching Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update agent discovery logic to include only local agent when no config is specified Signed-off-by: Deluan <deluan@navidrome.org> * refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct file extension filter in goimports command Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use defer to unlock the mutex Signed-off-by: Deluan <deluan@navidrome.org> * chore: move Cover Art Archive AlbumMetadataService plugins to an example folder Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle errors when creating media metadata and scrobbler service plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: increase compilation timeout to one minute Signed-off-by: Deluan <deluan@navidrome.org> * feat: add configurable plugin compilation timeout Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin scrobbler support in PlayTracker Signed-off-by: Deluan <deluan@navidrome.org> * feat: add context management and Stop method to buffered scrobbler Signed-off-by: Deluan <deluan@navidrome.org> * feat: add username field to scrobbler requests and update logging Signed-off-by: Deluan <deluan@navidrome.org> * fix: data race in test Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename http proto files to host and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused plugin registration methods from manager Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin manifests and implement plugin management commands Signed-off-by: Deluan <deluan@navidrome.org> * Update utils/files.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip") Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * feat: add plugin dev workflow support Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(README): update documentation to include TimerCallbackService and its functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add InitService with OnInit method and initialization tracking - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add tests for InitService and plugin initialization tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): expand documentation on plugin system implementation and architecture Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): redirect plugins' stderr to logs Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add safe accessor methods for TimerService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add plugin-specific configuration support in InitRequest and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add TimerCallbackService plugin adapter and integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename services for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add mutex for configuration access and clone plugin config Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): remove configtest dependency to prevent data races in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): correctly pass error to done function in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename service types to capabilities for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename InitService to LifecycleManagement for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): fix instance ID logging in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add config service for plugin-specific configuration management Signed-off-by: Deluan <deluan@navidrome.org> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat(crontab): implement crontab service for scheduling and canceling jobs Signed-off-by: Deluan <deluan@navidrome.org> * fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit) Squashed commits: [325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService Signed-off-by: Deluan <deluan@navidrome.org> * fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): consolidate job scheduling logic into a single helper function Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename GetInstance method to Instantiate for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add WebSocket service for handling connections and messages Signed-off-by: Deluan <deluan@navidrome.org> * feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): enhance connection management and callback handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): only create one adapter instance for each adapter/capability pair Signed-off-by: Deluan <deluan@navidrome.org> * fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): refactor WebSocket service integration and improve error logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): add SchedulerCallback support and improve reconnection logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: test panic Signed-off-by: Deluan <deluan@navidrome.org> * docs: add crypto-ticker plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): add error field to response messages for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(cache): implement CacheService with string, int, float, and byte operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo Signed-off-by: Deluan <deluan@navidrome.org> * refactor(cache): simplify cache service implementation and remove unnecessary synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): add build step for test plugins in the test suite Signed-off-by: Deluan <deluan@navidrome.org> * wip Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(rpc): enhance activity image processing and improve error handling in Discord integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(discord): enhance activity state with artist list and add large text asset Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(artwork): implement ArtworkService for retrieving artwork URLs Signed-off-by: Deluan <deluan@navidrome.org> * Add playback position to scrobble NowPlaying (#4089) * test(playtracker): cover playback position * address review comment Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix merge Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unnecessary check for empty slice in Map function Signed-off-by: Deluan <deluan@navidrome.org> * fix: update reflex.conf to include .wasm file extension Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): normalize attribute strings and add edge case tests for PID calculation Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458 Signed-off-by: Deluan <deluan@navidrome.org> * test(ui): fix warnings (#4187) * fix(ui): address test warnings * ignore lint error in test Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * refactor(server): optimize top songs lookup (#4189) * optimize top songs lookup * Optimize title matching queries * refactor: simplify top songs matching * improve error handling and logging in track loading functions Signed-off-by: Deluan <deluan@navidrome.org> * test: add cases for fallback to title matching and combined MBID/title matching Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): playlist details overflow in spotify-based themes (#4184) * test: ensure playlist details width * fix(test): simplify expectation for minWidth in NDPlaylistDetails Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): test all themes Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update TagLib to version 2.1 (#4185) * chore: update cross-taglib * fix(taglib): add logging for TagLib version Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * test: verify agents fallback (#4191) * build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version. This is to reduce the image size, as we don't really need the latest. Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(runtime): implement pooled WASM runtime and module for better instance management Signed-off-by: Deluan <deluan@navidrome.org> * fix(discord-plugin): adjust timer delay calculation for track completion Signed-off-by: Deluan <deluan@navidrome.org> * resolve PR comments Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement cache cleanup by size functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(manager): return error from getCompilationCache and handle it in ScanPlugins Signed-off-by: Deluan <deluan@navidrome.org> * fix possible rce condition Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): update README to include Cache and Artwork services Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add permissions support for host services in custom runtime - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add permissions field to plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * test(permissions): implement permission validation and testing for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add unauthorized_plugin to test permission enforcement - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): add Plugin Permission System section to README - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add detailed reasons for permissions in plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement granular HTTP permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unexport all plugins package private symbols Signed-off-by: Deluan <deluan@navidrome.org> * update docs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin_lifecycle_manager Signed-off-by: Deluan <deluan@navidrome.org> * docs: add discord-rich-presence plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for PATCH, HEAD, and OPTIONS HTTP methods Signed-off-by: Deluan <deluan@navidrome.org> * feat: use folder names as unique identifiers for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: read config just once, to avoid data race in tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginName to pluginID for consistency across services Signed-off-by: Deluan <deluan@navidrome.org> * fix: use symlink name instead of folder name for plugin registration Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin output format to include ID and enhance README with symlink usage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement shared plugin discovery function to streamline plugin scanning and error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: show plugin permissions in `plugin info` Signed-off-by: Deluan <deluan@navidrome.org> * feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement typed permissions for plugins to enhance permission handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor plugin permissions to use typed schema and improve validation - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused JSON schema validation for plugin manifests Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused fields from PluginPackage struct in package.go Signed-off-by: Deluan <deluan@navidrome.org> * feat: update file permissions in tests and remove unused permission parsing function Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor test plugin creation to use typed permissions and remove legacy helper Signed-off-by: Deluan <deluan@navidrome.org> * feat: add website field to plugin manifests and update test cases Signed-off-by: Deluan <deluan@navidrome.org> * refactor: permission schema to use basePermission structure for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance host service management by adding permission checks for each service Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reorganize code files Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify custom runtime creation by removing compilation cache parameter Signed-off-by: Deluan <deluan@navidrome.org> * doc: add WebSocketService and update ConfigService for plugin-specific configuration Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WASM loading optimization to enhance plugin instance creation speed Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename custom runtime functions and update related tests for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin structure with compilation handling and error reporting Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve logging and context tracing in runtime and wasm base plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance runtime management with scoped runtime and caching improvements Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement EnsureCompiled method for improved plugin compilation handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement cached module management with TTL for improved performance Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace map with sync.Map Signed-off-by: Deluan <deluan@navidrome.org> * refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance image processing with fallback mechanism for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * docs: review test plugins readme Signed-off-by: Deluan <deluan@navidrome.org> * feat: set default timeout for HTTP client to 10 seconds Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance wasm instance pool with concurrency limits and timeout settings Signed-off-by: Deluan <deluan@navidrome.org> * feat(discordrp): implement caching for processed image URLs with configurable TTL Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00
// AlbumInfoRetriever provides album info (no images)
type AlbumInfoRetriever interface {
GetAlbumInfo(ctx context.Context, name, artist, mbid string) (*AlbumInfo, error)
}
feat(plugins): experimental support for plugins (#3998) * feat(plugins): add minimal test agent plugin with API definitions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add plugin manager with auto-registration and unique agent names Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator. * test: add Ginkgo suite and test for plugin manager auto-registration Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support. * test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement persistent compilation cache for WASM agent plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmAgent to improve resource management Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance logging for wasmAgent and plugin manager operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement HttpService for handling HTTP requests in WASM plugins Also add a sample Wikimedia plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): standardize error handling in wasmAgent and MinimalAgent Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean up wikimedia plugin code Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy. * refactor: use unified SPARQLResult struct and parser for SPARQL responses Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability. * feat(plugins): improve error handling in HTTP request processing Signed-off-by: Deluan <deluan@navidrome.org> * fix: background plugin compilation, logging, and race safety Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass. * perf: implement true lazy loading for agents Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass. * fix: ensure wasm plugin instances are closed via runtime.AddCleanup Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup. * refactor: break down long functions in plugin manager and agent Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go. Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename wasmAgent to wasmArtistAgent Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename MinimalAgent for artist metadata service Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin pool creation with custom runtime and precompilation support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement generic plugin pool and agent factory for improved service handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize plugin management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): improve function signatures for clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement background precompilation for plugins and agent factory creation Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): include instanceID in logging for better traceability Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add tests for plugin pre-compilation and agent factory synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add minimal album test agent plugin for AlbumMetadataService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename fake artist and album test agent plugins for metadata services Signed-off-by: Deluan <deluan@navidrome.org> * feat(makefile): add Makefile for building plugin WASM binaries Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized. * feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove wasm module pooling it was causing issues with the GC and the Close methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename metadata service files to adapter naming convention Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle nil values in data redaction process Signed-off-by: Deluan <deluan@navidrome.org> * fix: add timeout for plugin compilation to prevent indefinite blocking Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ScrobblerService plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify generalization Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin management by improving scanning and loading mechanisms Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin creation functions to return specific interfaces for better type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement MediaMetadataService with combined artist and album methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve MediaMetadataService plugin implementation and testing structure Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add tests for Adapter Media Agent and improve plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance agent management with plugin loading and caching Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update agent discovery logic to include only local agent when no config is specified Signed-off-by: Deluan <deluan@navidrome.org> * refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct file extension filter in goimports command Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use defer to unlock the mutex Signed-off-by: Deluan <deluan@navidrome.org> * chore: move Cover Art Archive AlbumMetadataService plugins to an example folder Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle errors when creating media metadata and scrobbler service plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: increase compilation timeout to one minute Signed-off-by: Deluan <deluan@navidrome.org> * feat: add configurable plugin compilation timeout Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin scrobbler support in PlayTracker Signed-off-by: Deluan <deluan@navidrome.org> * feat: add context management and Stop method to buffered scrobbler Signed-off-by: Deluan <deluan@navidrome.org> * feat: add username field to scrobbler requests and update logging Signed-off-by: Deluan <deluan@navidrome.org> * fix: data race in test Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename http proto files to host and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused plugin registration methods from manager Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin manifests and implement plugin management commands Signed-off-by: Deluan <deluan@navidrome.org> * Update utils/files.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip") Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * feat: add plugin dev workflow support Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(README): update documentation to include TimerCallbackService and its functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add InitService with OnInit method and initialization tracking - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add tests for InitService and plugin initialization tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): expand documentation on plugin system implementation and architecture Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): redirect plugins' stderr to logs Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add safe accessor methods for TimerService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add plugin-specific configuration support in InitRequest and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add TimerCallbackService plugin adapter and integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename services for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add mutex for configuration access and clone plugin config Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): remove configtest dependency to prevent data races in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): correctly pass error to done function in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename service types to capabilities for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename InitService to LifecycleManagement for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): fix instance ID logging in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add config service for plugin-specific configuration management Signed-off-by: Deluan <deluan@navidrome.org> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat(crontab): implement crontab service for scheduling and canceling jobs Signed-off-by: Deluan <deluan@navidrome.org> * fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit) Squashed commits: [325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService Signed-off-by: Deluan <deluan@navidrome.org> * fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): consolidate job scheduling logic into a single helper function Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename GetInstance method to Instantiate for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add WebSocket service for handling connections and messages Signed-off-by: Deluan <deluan@navidrome.org> * feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): enhance connection management and callback handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): only create one adapter instance for each adapter/capability pair Signed-off-by: Deluan <deluan@navidrome.org> * fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): refactor WebSocket service integration and improve error logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): add SchedulerCallback support and improve reconnection logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: test panic Signed-off-by: Deluan <deluan@navidrome.org> * docs: add crypto-ticker plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): add error field to response messages for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(cache): implement CacheService with string, int, float, and byte operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo Signed-off-by: Deluan <deluan@navidrome.org> * refactor(cache): simplify cache service implementation and remove unnecessary synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): add build step for test plugins in the test suite Signed-off-by: Deluan <deluan@navidrome.org> * wip Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(rpc): enhance activity image processing and improve error handling in Discord integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(discord): enhance activity state with artist list and add large text asset Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(artwork): implement ArtworkService for retrieving artwork URLs Signed-off-by: Deluan <deluan@navidrome.org> * Add playback position to scrobble NowPlaying (#4089) * test(playtracker): cover playback position * address review comment Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix merge Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unnecessary check for empty slice in Map function Signed-off-by: Deluan <deluan@navidrome.org> * fix: update reflex.conf to include .wasm file extension Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): normalize attribute strings and add edge case tests for PID calculation Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458 Signed-off-by: Deluan <deluan@navidrome.org> * test(ui): fix warnings (#4187) * fix(ui): address test warnings * ignore lint error in test Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * refactor(server): optimize top songs lookup (#4189) * optimize top songs lookup * Optimize title matching queries * refactor: simplify top songs matching * improve error handling and logging in track loading functions Signed-off-by: Deluan <deluan@navidrome.org> * test: add cases for fallback to title matching and combined MBID/title matching Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): playlist details overflow in spotify-based themes (#4184) * test: ensure playlist details width * fix(test): simplify expectation for minWidth in NDPlaylistDetails Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): test all themes Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update TagLib to version 2.1 (#4185) * chore: update cross-taglib * fix(taglib): add logging for TagLib version Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * test: verify agents fallback (#4191) * build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version. This is to reduce the image size, as we don't really need the latest. Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(runtime): implement pooled WASM runtime and module for better instance management Signed-off-by: Deluan <deluan@navidrome.org> * fix(discord-plugin): adjust timer delay calculation for track completion Signed-off-by: Deluan <deluan@navidrome.org> * resolve PR comments Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement cache cleanup by size functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(manager): return error from getCompilationCache and handle it in ScanPlugins Signed-off-by: Deluan <deluan@navidrome.org> * fix possible rce condition Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): update README to include Cache and Artwork services Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add permissions support for host services in custom runtime - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add permissions field to plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * test(permissions): implement permission validation and testing for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add unauthorized_plugin to test permission enforcement - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): add Plugin Permission System section to README - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add detailed reasons for permissions in plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement granular HTTP permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unexport all plugins package private symbols Signed-off-by: Deluan <deluan@navidrome.org> * update docs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin_lifecycle_manager Signed-off-by: Deluan <deluan@navidrome.org> * docs: add discord-rich-presence plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for PATCH, HEAD, and OPTIONS HTTP methods Signed-off-by: Deluan <deluan@navidrome.org> * feat: use folder names as unique identifiers for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: read config just once, to avoid data race in tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginName to pluginID for consistency across services Signed-off-by: Deluan <deluan@navidrome.org> * fix: use symlink name instead of folder name for plugin registration Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin output format to include ID and enhance README with symlink usage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement shared plugin discovery function to streamline plugin scanning and error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: show plugin permissions in `plugin info` Signed-off-by: Deluan <deluan@navidrome.org> * feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement typed permissions for plugins to enhance permission handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor plugin permissions to use typed schema and improve validation - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused JSON schema validation for plugin manifests Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused fields from PluginPackage struct in package.go Signed-off-by: Deluan <deluan@navidrome.org> * feat: update file permissions in tests and remove unused permission parsing function Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor test plugin creation to use typed permissions and remove legacy helper Signed-off-by: Deluan <deluan@navidrome.org> * feat: add website field to plugin manifests and update test cases Signed-off-by: Deluan <deluan@navidrome.org> * refactor: permission schema to use basePermission structure for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance host service management by adding permission checks for each service Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reorganize code files Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify custom runtime creation by removing compilation cache parameter Signed-off-by: Deluan <deluan@navidrome.org> * doc: add WebSocketService and update ConfigService for plugin-specific configuration Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WASM loading optimization to enhance plugin instance creation speed Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename custom runtime functions and update related tests for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin structure with compilation handling and error reporting Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve logging and context tracing in runtime and wasm base plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance runtime management with scoped runtime and caching improvements Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement EnsureCompiled method for improved plugin compilation handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement cached module management with TTL for improved performance Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace map with sync.Map Signed-off-by: Deluan <deluan@navidrome.org> * refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance image processing with fallback mechanism for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * docs: review test plugins readme Signed-off-by: Deluan <deluan@navidrome.org> * feat: set default timeout for HTTP client to 10 seconds Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance wasm instance pool with concurrency limits and timeout settings Signed-off-by: Deluan <deluan@navidrome.org> * feat(discordrp): implement caching for processed image URLs with configurable TTL Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00
// AlbumImageRetriever provides album images
type AlbumImageRetriever interface {
GetAlbumImages(ctx context.Context, name, artist, mbid string) ([]ExternalImage, error)
}
2021-02-07 16:46:15 -05:00
type ArtistMBIDRetriever interface {
GetArtistMBID(ctx context.Context, id string, name string) (string, error)
2021-02-07 16:46:15 -05:00
}
type ArtistURLRetriever interface {
GetArtistURL(ctx context.Context, id, name, mbid string) (string, error)
2021-02-07 16:46:15 -05:00
}
type ArtistBiographyRetriever interface {
GetArtistBiography(ctx context.Context, id, name, mbid string) (string, error)
2021-02-07 16:46:15 -05:00
}
type ArtistSimilarRetriever interface {
GetSimilarArtists(ctx context.Context, id, name, mbid string, limit int) ([]Artist, error)
2021-02-07 16:46:15 -05:00
}
type ArtistImageRetriever interface {
GetArtistImages(ctx context.Context, id, name, mbid string) ([]ExternalImage, error)
2021-02-07 16:46:15 -05:00
}
type ArtistTopSongsRetriever interface {
GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]Song, error)
2021-02-07 16:46:15 -05:00
}
feat: add similar songs functionality in agents, and Instant Mix (song-based) to UI (#4919) * refactor: rename ArtistRadio to SimilarSongs for clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement GetSimilarSongsByTrack and related functionality for song similarity retrieval Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance GetSimilarSongsByTrack to include artist and album details and update tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance song matching by implementing title and artist filtering in loadTracksByTitleAndArtist Signed-off-by: Deluan <deluan@navidrome.org> * test: add unit tests for song matching functionality in provider Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract song matching functionality into its own file Signed-off-by: Deluan <deluan@navidrome.org> * docs: clarify similarSongsFallback function description in provider.go Signed-off-by: Deluan <deluan@navidrome.org> * refactor: initialize result slice for songs with capacity based on response length Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify agent method calls for retrieving images and similar songs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify agent method calls for retrieving images and similar songs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove outdated comments in GetSimilarSongs methods Signed-off-by: Deluan <deluan@navidrome.org> * fix: use composite key for song matches to handle duplicates by title and artist Signed-off-by: Deluan <deluan@navidrome.org> * refactor: consolidate expectations setup for similar songs tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add instant mix action to song context menu and update translations Signed-off-by: Deluan <deluan@navidrome.org> * fix(provider): handle unknown entity types in GetSimilarSongs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move playSimilar action to playbackActions and streamline song processing Signed-off-by: Deluan <deluan@navidrome.org> * format Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance instant mix functionality with loading notification and shuffle option Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement fuzzy matching for similar songs based on configurable threshold Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement track matching with multiple specificity levels Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance track matching by implementing unified scoring with specificity levels Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance deezer top tracks result with album Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance track matching with fuzzy album similarity for improved scoring Signed-off-by: Deluan <deluan@navidrome.org> * docs: document multi-phase song matching algorithm with detailed scoring and prioritization Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2026-01-25 16:16:43 -05:00
// SimilarSongsByTrackRetriever provides similar songs based on a specific track
type SimilarSongsByTrackRetriever interface {
// GetSimilarSongsByTrack returns songs similar to the given track.
// Parameters:
// - id: local mediafile ID
// - name: track title
// - artist: artist name
// - mbid: MusicBrainz recording ID (may be empty)
// - count: maximum number of results
GetSimilarSongsByTrack(ctx context.Context, id, name, artist, mbid string, count int) ([]Song, error)
}
// SimilarSongsByAlbumRetriever provides similar songs based on an album
type SimilarSongsByAlbumRetriever interface {
// GetSimilarSongsByAlbum returns songs similar to tracks on the given album.
// Parameters:
// - id: local album ID
// - name: album name
// - artist: album artist name
// - mbid: MusicBrainz release ID (may be empty)
// - count: maximum number of results
GetSimilarSongsByAlbum(ctx context.Context, id, name, artist, mbid string, count int) ([]Song, error)
}
// SimilarSongsByArtistRetriever provides similar songs based on an artist
type SimilarSongsByArtistRetriever interface {
// GetSimilarSongsByArtist returns songs similar to the artist's catalog.
// Parameters:
// - id: local artist ID
// - name: artist name
// - mbid: MusicBrainz artist ID (may be empty)
// - count: maximum number of results
GetSimilarSongsByArtist(ctx context.Context, id, name, mbid string, count int) ([]Song, error)
}
2021-02-07 16:46:15 -05:00
var Map map[string]Constructor
func Register(name string, init Constructor) {
if Map == nil {
Map = make(map[string]Constructor)
}
Map[name] = init
}