3 commits

Author	SHA1	Message	Date
Deluan Quintão	b293b96256	refactor(persistence): stateless repositories with per-call context (#6149) ... * refactor(persistence): adopt generic deluan/rest repository API Pin deluan/rest to the refactor branch. REST-facing repository methods take a context and return typed values. Drop DataStore.Resource and ResourceRepository; the native API names typed repositories directly through a per-request adapter that later commits remove. * refactor(persistence): base repository helpers take a context * refactor(persistence): LibraryRepository takes a context per call * refactor(persistence): PropertyRepository takes a context per call * refactor(persistence): UserPropsRepository takes a context per call * refactor(persistence): TranscodingRepository takes a context per call * refactor(persistence): ShareRepository takes a context per call * refactor(persistence): PlayerRepository takes a context per call * refactor(persistence): RadioRepository takes a context per call * refactor(persistence): PlayQueueRepository takes a context per call * refactor(persistence): Tag and Genre repositories take a context per call * refactor(persistence): PluginRepository takes a context per call * refactor(persistence): Scrobble repositories take a context per call * refactor(persistence): FolderRepository takes a context per call * refactor(persistence): Artwork repositories take a context per call * refactor(persistence): UserRepository takes a context per call * refactor(persistence): ArtistRepository takes a context per call ReadAll no longer rewrites the shared sort mappings for the role filter; it works on a per-call copy. * test(persistence): assert artist role sort sanitization in ReadAll * refactor(persistence): AlbumRepository takes a context per call * test(persistence): pass the test context to album repository helpers * refactor(persistence): MediaFileRepository takes a context per call * refactor(persistence): Playlist repositories take a context per call * refactor(persistence): build all repositories once per store * refactor(core): REST repository wrappers are built once * refactor(persistence): repositories are stateless Remove the context field from the base repository and the per-request REST adapter. Enable the containedctx linter so no repository can hold a request context again. * chore(lint): skip containedctx in test files * refactor: share simplifications from the stateless repositories sweep Add deleteOwnedAll on sqlRepository and use it in player/share Delete to remove the duplicated bulk-delete loop; have Share.Repository() return model.ShareRepository so subsonic sharing.go drops its repeated type assertions. * chore(core): assert REST wrappers implement Persistable * chore: reformat imports * perf(persistence): build repositories on first use Each transaction store used to construct all 21 repositories up front, paying for filter and sort mapping setup the block never touched. Fields are now sync.OnceValue thunks, so a store only builds what it uses. * fix(persistence): clean plugin references per deleted user A bulk user delete that fails on a later id had already removed the earlier rows but skipped their plugin cleanup. Cleanup now runs right after each successful delete. * fix(core): unload disabled plugins even when a user delete fails A bulk delete can fail on a later id after earlier users were removed and their plugins auto-disabled. The wrapper returned before unloading, leaving those plugins running until the next successful delete or a restart. * chore(deps): pin deluan/rest to v1.0.1 Replaces the pseudo-version of the refactor branch with the tagged release. REST error messages now name the bare type (Artist, not model.Artist). * test: use the spec context instead of context.Background() Replace the context.Background()/context.TODO() calls this branch added to tests with the spec's ctx, GinkgoT().Context(), or t/b.Context(), so repository calls are bound to the running spec's lifetime. * test: declare the spec context once per Describe Set ctx from GinkgoT().Context() first in each top-level BeforeEach and reuse it, building user contexts on top of it instead of repeating inline calls.	2026-09-25 18:06:10 -04:00
Deluan Quintão	c9385fbb6b	test(plugins): build test plugins in Go instead of shelling out to make (#6060) ... * test(plugins): build test plugins in Go instead of shelling out to make The plugins suite built its .ndp test packages by running `make -C plugins/testdata`, which needs make and zip on the PATH. That is the reason the 26 WASM-dependent spec files are tagged //go:build !windows. buildTestPlugins now does the same work in Go: the same mtime check make performed, `GOOS=wasip1 GOARCH=wasm go build` per plugin, and archive/zip for the package. TinyGo was already optional and unused in CI, so nothing is lost there. The first plugin builds on its own so the shared wasip1 stdlib and PDK objects land in the build cache before the rest fan out: on a cold cache that is 2.2s against 3.4s for the sequential make and 7.3s for an unrestrained fan-out. Packaging moved into a writeNdp helper shared with createTestPackage, which was already writing the same two-entry archive. Entries are written in a fixed order, so the .ndp bytes are now reproducible; the loader hashes those bytes, and `zip` also stored file mtimes, so the previous packages differed on every rebuild. The Makefile is unchanged and still works for building the plugins by hand. Removing the !windows tags is a separate step, once CI is green here. * test(plugins): run the WASM plugin specs on Windows With the test plugins now built in Go, nothing in the suite needs a Unix toolchain, so the //go:build !windows tags come off all 25 spec files. The Windows CI job runs `go test ./...`, so it picks the suite up with no workflow change. plugins_suite_windows_test.go existed only to bootstrap the handful of specs that compiled on Windows; plugins_suite_test.go now serves both. * test(plugins): skip the planted-symlink spec where symlinks need privileges os.Symlink needs an elevated token or Developer Mode on Windows, so the unconditional Expect(...).To(Succeed()) would fail for contributors running the suite on an ordinary Windows box. The elevated GitHub runner hides this. The equivalent spec in sandbox_fs_internal_test.go already attempts the symlink and skips on error; this does the same, keeping the pin live everywhere it can run, including Windows CI. * fix(ci): stop the Windows ndpgen test failing silently The ndpgen suite builds its helper binary to %TEMP%\ndpgen-test, and Windows will not exec a file without an executable extension, so the "supports verbose mode" spec has been failing there. Nobody noticed because the Test ndpgen step ran under pwsh, which carries on after a non-zero exit and takes the step's status from the last command, so the job stayed green with a FAIL line in its log. Add the .exe suffix, and run the step under bash like the Linux job does, so a failure in any of its three commands fails the job.	2026-08-31 21:42:10 -04:00
Deluan Quintão	f580d3ea76	feat(plugins): expose the song Matcher as a host service (#5643) ... * feat(plugins): add public Track and Artist DTOs for host services * feat(plugins): add Matcher host-service interface and MatchSong DTO * feat(plugins): generate Matcher host wrappers, PDK clients, and matcher permission * feat(plugins): implement Matcher host service and MediaFile-to-Track converter Also fixes an ndpgen bug where ParseDirectory parsed each host-service file in isolation, so a service method referencing a struct defined in another file of the same package (host.Track in track.go) could not be resolved. ParseDirectory now collects package-wide structs in a first pass, mirroring ParseCapabilities; PDK clients regenerated cleanly via make gen. * feat(plugins): register Matcher host service in the manager * test(plugins): add Matcher host service integration test plugin * refactor(plugins): simplify matcher converter and parser file collection - toTrack: use gg.V for nil-able field derefs and slice.Map for genres/ participants, removing the repeated nil-guard blocks and inner loop - manager_loader: drop the redundant ds==nil guard (loadEnabledPlugins already gates a nil DataStore), matching the other service entries - ndpgen parser: extract collectGoFiles, shared by ParseDirectory and ParseCapabilities instead of duplicating the file-filter loop * fix(plugins): keep nullable Track numerics as pointers ReplayGain values, BitDepth, and BPM are nullable in model.MediaFile, and 0 is a valid measured ReplayGain value. Flattening them to value types with omitempty made a real 0 indistinguishable from absent. Model them as *float64 /*int32 so plugins can tell 'no data' from a measured 0. Regenerated PDK clients; converter passes the model pointers through (RG) or maps *int->*int32 (BitDepth/BPM). * refactor(plugins): trim redundant pass labels in ndpgen ParseDirectory The function doc already explains the two-pass approach; the inline labels restated it. Reduce to bare waypoints. * fix(plugins): gate Track.Path on library filesystem permission MatchSongs copied mf.Path into every result unconditionally, letting a plugin with only the matcher permission enumerate on-disk file paths by matching known songs. Gate Path behind library.filesystem, matching the Library host service. toTrack is now a method carrying the permission flag. * refactor(plugins): align MatchSong JSON casing and parse Go files once - MatchSong: artistMBID/albumMBID JSON tags -> artistMbid/albumMbid so the Go wire format matches the Rust SDK's camelCase serialization (cross-SDK fix) - MatchSongs doc reworded to language-neutral 'empty (absent)' so generated Rust/Python client docs no longer say Go-specific 'nil' - ndpgen: parse each package file once (parseGoFiles) and reuse the ASTs across both passes in ParseDirectory and ParseCapabilities, instead of re-parsing * refactor(plugins): use shared types for Matcher host service Move the Matcher host service onto the shared plugins/types package instead of the host-local MatchSong and Track structs. MatchSongs now takes []types.SongRef and returns []*types.Track, dropping host.MatchSong and moving host.Track (with its host.Artist dependency collapsed onto types.ArtistRef) into plugins/types. ArtistRef gains SortName and SubRole so it can back a track's Participants. SongRef gains a millisecond-precision DurationMs field that supersedes the now deprecated seconds-based Duration, with DurationInMs() resolving the effective value and SetDurationMs() keeping both fields in sync when populating a SongRef to send to a plugin. The ndpgen host-wrapper template only ever imported context, json and extism, so a host service referencing the shared types package produced uncompilable code. Emit the plugins/types import when the service references shared types directly (gated on the existing Service.ImportsSharedTypes), matching the client template, and cover it with GenerateHost tests. This removes the need for host-local re-export aliases. Regenerated the Go/Rust/Python PDK and capability schemas accordingly. * test(plugins): cover SongRef duration and artist conversion Add unit coverage for the new SongRef behavior: SetDurationMs populating both DurationMs and the deprecated seconds field, and the SongRef-to-agents.Song conversion preferring DurationMs over Duration and the Artists list over the scalar Artist/ArtistMBID. Extract the inline SongRef-to-agents.Song closure in MatchSongs into a named toAgentSong function so the conversion can be asserted directly rather than only through the opaque matcher. The end-to-end wire shape of the moved types is already validated by the existing MatcherService integration test, so no new WASM-boundary test is needed. * fix(plugins): harden and unify SongRef-to-agents.Song duration conversion Address findings from a code review of the matcher host service: - DurationInMs now clamps a negative deprecated-seconds value to 0 instead of converting it through uint32, which previously wrapped a value like -1s into a ~49-day duration that corrupted the matcher's duration-proximity tiebreaker. - Replace the unused SetDurationMs(uint32) with SetDuration(seconds float32), which takes the unit callers actually hold (model.MediaFile.Duration is float32 seconds) and centralizes the seconds-to-ms conversion. Wire it into mediaFileToSongRef so outbound SongRefs carry both duration fields in sync. - Make the metadata-agent path use DurationInMs() so every consumer of the shared SongRef honors the DurationMs-over-Duration precedence contract; a plugin sending only DurationMs no longer loses its duration on that path. - Collapse the matcher's duplicate toAgentSong/agentArtists helpers into the existing songRefToAgentSong converter, so there is a single SongRef-to-Song mapping. Tests narrowed to the duration cases, with artist precedence still covered in metadata_agent_test.go. * feat(plugins): allow Matcher host service to scope a match to a user Add an options struct to the Matcher host service so a plugin can run a match as a specific user. When MatchOptions.Username is set, the match is run in that user's context: their favourites and ratings inform the matcher's tiebreaker, and the returned tracks carry that user's per-user annotations (Starred, StarredAt, Rating, PlayCount, PlayDate, added to types.Track). An empty username preserves the previous unscoped behaviour. Cross-user access is gated by the same allowedUsers/allUsers permission the Users and SubsonicAPI host services use: an unknown username, or one the plugin is not permitted to act as, returns an error. User-library access applies automatically once the user is in context (applyLibraryFilter). Independently, results are now restricted to the libraries the plugin itself may access via the precomputed libraryAccess set, dropping any matched track outside that set (the input index stays unmatched) — this applies even without a username and even for an admin-scoped user. core/matcher is unchanged: it already loads and uses annotations and applies user-library filtering from context, so the feature works by deriving the request context and post-filtering by plugin library access in the host adapter. The new opts parameter and the Track annotation fields are propagated to all PDK clients (Go/Rust/Python) by make gen. * fix(plugins): correct Matcher library scope and unify user-access checks Address findings from a code review of the user-scoped Matcher host service: - The plugin-library post-filter previously dropped every match for a plugin that holds only the matcher permission, because library config is tied to the Library permission and a matcher-only plugin has none (empty allowedLibraries, AllLibraries=false). Gate the filter on whether the plugin actually declared the Library permission: matcher-only plugins are no longer library-restricted, while plugins that opt into a library scope are enforced as before. The per-user library filter (applyLibraryFilter) still applies whenever a non-admin user is scoped. - resolveUser collapsed every FindByUsername error (including transient DB failures) into a misleading "not found". Extract a shared userAccess type (alongside libraryAccess) whose resolve() distinguishes model.ErrNotFound from a real backend error and authorizes the user against the allowed set. The Matcher service now uses it, and host_subsonicapi shares the same userAccess type for its permission check (preserving its existing error messages), removing a third divergent copy of the resolve-and-authorize logic. - Document in the matcher tests that the mock MediaFileRepo returns annotations unconditionally, so the unit tests cover the adapter's scoped-flag gating and access checks but not the SQL per-user join. Add tests for the library-permission gating and for surfacing a backend error instead of masking it as not-found. * fix(plugins): require a library scope for Matcher, fail closed Reverse the permissive default introduced when fixing the library post-filter: a Matcher plugin now must be granted a library scope (all libraries, or at least one specific library) and MatchSongs rejects the request with "no libraries configured" when it has none, instead of either silently matching nothing or defaulting to every library. This mirrors how the SubsonicAPI host service requires a user scope (checkPermissions errors with "no users configured" when none is set): the check is a runtime guard via libraryAccess.configured(), needs no manifest changes, and keeps the failure loud rather than silent. The per-match library post-filter then always applies, and the restrictLibraries flag added in the previous commit is removed. * fix(plugins): require library permission for matcher; guard nil user Close the gap where a plugin declaring only the matcher permission loaded successfully but failed every MatchSongs call with "no libraries configured", with no way for an admin to grant a library scope (the library-config UI is gated on the library permission). Add a cross-field manifest rule, mirroring the existing "subsonicapi requires users" rule, so the matcher permission requires the library permission to be declared. A matcher plugin therefore also surfaces the library-config panel and is subject to the existing load/enable-time library configuration gate, making the fail-closed library check reachable and fixable rather than a silent dead end. The test plugin manifest now declares the library permission accordingly. Also restore a defensive nil-user guard in userAccess.resolve: if a DataStore's FindByUsername ever returns (nil, nil) instead of model.ErrNotFound, return a clean "not found" error rather than dereferencing a nil *model.User. * feat(plugins): expose track AverageRating in Matcher results Add AverageRating to the Matcher's Track DTO. Unlike the per-user annotations (Starred, Rating, PlayCount, ...), AverageRating is an aggregate stored on the track itself and is loaded regardless of the request user, so it is populated unconditionally rather than gated on a scoped username. Propagated to the PDK types by make gen. Signed-off-by: Deluan <deluan@navidrome.org> * style(plugins): trim verbose comments in matcher host service Condense the over-long explanatory comments added across the matcher host service to one-liners that state the why, and simplify the ptrInt32/unixPtr helpers to Go 1.26's new(value). No behavior change. * refactor(plugins): pass userAccess into newSubsonicAPIService Move newUserAccess construction to the loader call site so the SubsonicAPI service constructor takes a userAccess value directly, matching newMatcherService. Pure refactor: the service already stored a userAccess internally, so behavior and error messages are unchanged. * fix(plugins): regenerate PDK and drop omitempty from AverageRating Re-run make gen so the generated PDK doc comments match the source comment trimmed in an earlier commit (the source was simplified but the PDK was not regenerated, leaving the committed files stale — a 'generated files up to date' hazard). Also drop omitempty from Track.AverageRating: it is always set (0 when unrated), so it should be present in the payload like the other always-set fields (BirthTime/CreatedAt/UpdatedAt), not dropped at zero. Tag change propagated to the PDK by the same regeneration. * fix(plugins): reject user-scoped match before lookup when plugin has no user scope A matcher plugin requires the library permission but not the users permission, so a matcher-only plugin always has an empty user scope (allUsers=false, no allowed users). MatchSongs still ran FindByUsername for any opts.Username before checking authorization and returned distinguishable errors ('user X not found' vs 'not allowed to act as user X'), letting such a plugin enumerate account names from the error text. Guard userAccess.resolve to reject with a single fixed error before the lookup when the plugin has no user scope, mirroring how the SubsonicAPI service short-circuits with 'no users configured'. The unscoped match path (no username) is unaffected, so matcher-only plugins still match normally. * fix(plugins): run unscoped matcher as admin, not the inherited request user A matcher host call can arrive on a context that already carries a request user (e.g. a plugin capability invoked while serving that user's request — extism propagates the call context into host functions). With no opts.Username, MatchSongs passed that context straight through, so the media-file repository applied the caller's library filter and per-user annotation ranking to an explicitly unscoped match. Set the user context explicitly: a username scopes to that user (overriding any inherited one), and an unscoped match runs under adminContext so only the plugin's own library scope constrains results. Adds tests using a context-capturing DataStore to assert the user the matcher resolves in both cases. * chore(plugins): drop the generated Python matcher PDK The Python plugin PDK is no longer supported (ndpgen generates only Go and Rust clients), so remove the stale generated nd_host_matcher.py rather than leave a client that drifts from the host interface. * docs(plugins): deprecate SongRef.Artist/ArtistMBID in favor of Artists Mark the scalar single-artist fields deprecated; Artists (the ArtistRef list) is the preferred way to supply artist data and already takes precedence for matching. Propagated to the PDK and capability schemas by make gen. * refactor(plugins): flatten Track.Participants and add Role to ArtistRef Change Track.Participants from map[role][]ArtistRef to a flat []ArtistRef, and give ArtistRef a Role field (the participation category: artist/composer/performer/...) alongside SubRole (a specialization within a role, e.g. the instrument for a performer). In the flat list each entry now self-describes its role rather than relying on a map key, matching how SongRef.Artists is already a flat list; the converter tags each entry with its role and emits them in a stable role order. Propagated to the PDK and capability schemas by make gen. --------- Signed-off-by: Deluan <deluan@navidrome.org>	2026-07-01 11:19:15 -04:00