navidrome/log/log.go

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

379 lines
8.5 KiB
Go
Raw Permalink Normal View History

2020-01-08 20:45:07 -05:00
package log
import (
"cmp"
2020-01-08 20:45:07 -05:00
"context"
"errors"
"fmt"
feat(build): MSI installer improvements (#3376) * feat(build): add a make target to build a msi installer locally * Testing wrapping the executable in cmd * build(ci): build msis in parallel * feat(server): add LogFile config option * Revert "Testing wrapping the executable in cmd" This reverts commit be29592254cb903fd4904f2f50d4d1a860795d33. * Adding --log-file for service executable * feat(ini): wip * feat(ini): parse nested ini section * fix(conf): fix fatal error messages * Now navidrome supports INI, we can use the built-in msi ini system and not require the VBScript to convert it into toml * File needs to be called .ini to be parsed as an INI and correct filename needs to be passed to the service * fix(msi): build msi locally * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): Makefile * fix(msi): more clean up * fix(log): convert LF to CRLF on Windows * fix(msi): config filename should be case-insensitive * fix(msi): make it a little more idiomatic * Including the latest windows release of ffmpeg into the msi as built by https://www.gyan.dev/ffmpeg/builds/ (linked to on the official ffmpeg source) * This should version independent * Need bash expansion for the * to work * This will run twice, once for x86 and once for x64, I'll make it cache the executable for now as it'll be quicker * Silencing wget * Add ffmpeg path to the config so Navidrome knows where to find it * refactor: download ffmpeg from our repository * When going back from the "Are you ready to install?" it should go back to the Settings dialogue that you just came from * fix: comments --------- Co-authored-by: Deluan <deluan@navidrome.org>
2024-10-22 23:32:56 +00:00
"io"
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) * fix(server): more race conditions when updating artist/album from external sources Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394 Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): pass configfile option to child process Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): resume interrupted fullScans Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): remove old scanner code Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): rename old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): move old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update Go to 1.23.4 Signed-off-by: Deluan <deluan@navidrome.org> * fix: logs Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): Signed-off-by: Deluan <deluan@navidrome.org> * fix: log level Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add config for scanner watcher Signed-off-by: Deluan <deluan@navidrome.org> * refactor: children playlists Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace `interface{}` with `any` Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlists with genres Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow any tags in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist names in playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlist's sort by tags Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use generic JSONArray for OS arrays Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use https in test Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add releaseTypes to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add recordLabels to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): rename JSONArray to Array Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to Child Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): do not pre-populate smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement a simplified version of ArtistID3. See https://github.com/opensubsonic/open-subsonic-api/discussions/120 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to album child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add contributors to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add albumArtists to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayArtist and displayAlbumArtist Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayComposer to Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add roles to ArtistID3 Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): use " • " separator for displayComposer Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): respect `PreferSortTags` config option Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize purging non-unused tags Signed-off-by: Deluan <deluan@navidrome.org> * refactor: don't run 'refresh artist stats' concurrently with other transactions Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix: log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Scanner.ScanOnStartup config option, default true Signed-off-by: Deluan <deluan@navidrome.org> * feat: better json parsing error msg when importing NSPs Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update album's imported_time when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle interrupted scans and full scans after migrations Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `analyze` when migration requires a full rescan Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `PRAGMA optimize` at the end of the scan Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update artist's updated_at when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat: handle multiple artists and roles in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): dim missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * fix: album missing logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: error encoding in gob Signed-off-by: Deluan <deluan@navidrome.org> * feat: separate warnings from errors Signed-off-by: Deluan <deluan@navidrome.org> * fix: mark albums as missing if they were contained in a deleted folder Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add participant names to media_file and album tables Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use participations in criteria, instead of m2m relationship Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename participations to participants Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to album child Signed-off-by: Deluan <deluan@navidrome.org> * fix: albumartist role case Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): run scanner as an external process by default Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): show albumArtist names Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): dim out missing albums Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * fix(server): scrobble buffer mapping. fix #3583 Signed-off-by: Deluan <deluan@navidrome.org> * refactor: more participations renaming Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling Signed-off-by: Deluan <deluan@navidrome.org> * feat: send release_group_mbid to listenbrainz Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement OpenSubsonic explicitStatus field (#3597) * feat: implement OpenSubsonic explicitStatus field * fix(subsonic): fix failing snapshot tests * refactor: create helper for setting explicitStatus * fix: store smaller values for explicit-status on database * test: ToAlbum explicitStatus * refactor: rename explicitStatus helper function --------- Co-authored-by: Deluan Quintão <deluan@navidrome.org> * fix: handle album and track tags in the DB based on the mappings.yaml file Signed-off-by: Deluan <deluan@navidrome.org> * save similar artists as JSONB Signed-off-by: Deluan <deluan@navidrome.org> * fix: getAlbumList byGenre Signed-off-by: Deluan <deluan@navidrome.org> * detect changes in PID configuration Signed-off-by: Deluan <deluan@navidrome.org> * set default album PID to legacy_pid Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * fix SIGSEGV Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't lose album stars/ratings when migrating Signed-off-by: Deluan <deluan@navidrome.org> * store full PID conf in properties Signed-off-by: Deluan <deluan@navidrome.org> * fix: keep album annotations when changing PID.Album config Signed-off-by: Deluan <deluan@navidrome.org> * fix: reassign album annotations Signed-off-by: Deluan <deluan@navidrome.org> * feat: use (display) albumArtist and add links to each artist Signed-off-by: Deluan <deluan@navidrome.org> * fix: not showing albums by albumartist Signed-off-by: Deluan <deluan@navidrome.org> * fix: error msgs Signed-off-by: Deluan <deluan@navidrome.org> * fix: hide PID from Native API Signed-off-by: Deluan <deluan@navidrome.org> * fix: album cover art resolution Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim participant names Signed-off-by: Deluan <deluan@navidrome.org> * fix: reduce watcher log spam Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic when initializing the watcher Signed-off-by: Deluan <deluan@navidrome.org> * fix: various artists Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't store empty lyrics in the DB Signed-off-by: Deluan <deluan@navidrome.org> * remove unused methods Signed-off-by: Deluan <deluan@navidrome.org> * drop full_text indexes, as they are not being used by SQLite Signed-off-by: Deluan <deluan@navidrome.org> * keep album created_at when upgrading Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null pointer Signed-off-by: Deluan <deluan@navidrome.org> * fix: album artwork cache Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't expose missing files in Subsonic API Signed-off-by: Deluan <deluan@navidrome.org> * refactor: searchable interface Signed-off-by: Deluan <deluan@navidrome.org> * fix: filter out missing items from subsonic search * fix: filter out missing items from playlists * fix: filter out missing items from shares Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add filter by artist role Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): only return albumartists in getIndexes and getArtists endpoints Signed-off-by: Deluan <deluan@navidrome.org> * sort roles alphabetically Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist playcounts Signed-off-by: Deluan <deluan@navidrome.org> * change default Album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * fix albumartist link when it does not match any albumartists values Signed-off-by: Deluan <deluan@navidrome.org> * fix `Ignoring filter not whitelisted` (role) message Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim any names/titles being imported Signed-off-by: Deluan <deluan@navidrome.org> * remove unused genre code Signed-off-by: Deluan <deluan@navidrome.org> * serialize calls to Last.fm's getArtist Signed-off-by: Deluan <deluan@navidrome.org> xxx Signed-off-by: Deluan <deluan@navidrome.org> * add counters to genres Signed-off-by: Deluan <deluan@navidrome.org> * nit: fix migration `notice` message Signed-off-by: Deluan <deluan@navidrome.org> * optimize similar artists query Signed-off-by: Deluan <deluan@navidrome.org> * fix: last.fm.getInfo when mbid does not exist Signed-off-by: Deluan <deluan@navidrome.org> * ui only show missing items for admins Signed-off-by: Deluan <deluan@navidrome.org> * don't allow interaction with missing items Signed-off-by: Deluan <deluan@navidrome.org> * Add Missing Files view (WIP) Signed-off-by: Deluan <deluan@navidrome.org> * refactor: merged tag_counts into tag table Signed-off-by: Deluan <deluan@navidrome.org> * add option to completely disable automatic scanner Signed-off-by: Deluan <deluan@navidrome.org> * add delete missing files functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: playlists not showing for regular users Signed-off-by: Deluan <deluan@navidrome.org> * reduce updateLastAccess frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * reduce update player frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * add timeout when updating player Signed-off-by: Deluan <deluan@navidrome.org> * remove dead code Signed-off-by: Deluan <deluan@navidrome.org> * fix duplicated roles in stats Signed-off-by: Deluan <deluan@navidrome.org> * add `; ` to artist splitters Signed-off-by: Deluan <deluan@navidrome.org> * fix stats query Signed-off-by: Deluan <deluan@navidrome.org> * more logs Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * add record label filter Signed-off-by: Deluan <deluan@navidrome.org> * add release type filter Signed-off-by: Deluan <deluan@navidrome.org> * fix purgeUnused tags Signed-off-by: Deluan <deluan@navidrome.org> * add grouping filter to albums Signed-off-by: Deluan <deluan@navidrome.org> * allow any album tags to be used in as filters in the API Signed-off-by: Deluan <deluan@navidrome.org> * remove empty tags from album info Signed-off-by: Deluan <deluan@navidrome.org> * comments in the migration Signed-off-by: Deluan <deluan@navidrome.org> * fix: Cannot read properties of undefined Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling (#3640) Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove duplicated tag values Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't ignore the taglib folder! Signed-off-by: Deluan <deluan@navidrome.org> * feat: show track subtitle tag Signed-off-by: Deluan <deluan@navidrome.org> * fix: show artists stats based on selected role Signed-off-by: Deluan <deluan@navidrome.org> * fix: inspect Signed-off-by: Deluan <deluan@navidrome.org> * add media type to album info/filters Signed-off-by: Deluan <deluan@navidrome.org> * fix: change format of subtitle in the UI Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in Subsonic API and search Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in UI's player Signed-off-by: Deluan <deluan@navidrome.org> * fix: split strings should be case-insensitive Signed-off-by: Deluan <deluan@navidrome.org> * disable ScanSchedule Signed-off-by: Deluan <deluan@navidrome.org> * increase default sessiontimeout Signed-off-by: Deluan <deluan@navidrome.org> * add sqlite command line tool to docker image Signed-off-by: Deluan <deluan@navidrome.org> * fix: resources override Signed-off-by: Deluan <deluan@navidrome.org> * fix: album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * change migration to mark current artists as albumArtists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Allow filtering on multiple genres (#3679) * feat(ui): Allow filtering on multiple genres Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> * add multi-genre filter in Album list Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> * add more multi-valued tag filters to Album and Song views Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): unselect missing files after removing Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): song filter Signed-off-by: Deluan <deluan@navidrome.org> * fix sharing tracks. fix #3687 Signed-off-by: Deluan <deluan@navidrome.org> * use rowids when using search for sync (ex: Symfonium) Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients for search Signed-off-by: Deluan <deluan@navidrome.org> * add libraryPath to Native API /songs endpoint Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add album version Signed-off-by: Deluan <deluan@navidrome.org> * made all tags lowercase as they are case-insensitive anyways. Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Show full paths, extended properties for album/song (#3691) * feat(ui): Show full paths, extended properties for album/song - uses library path + os separator + path - show participants (album/song) and tags (song) - make album/participant clickable in show info * add source to path * fix pathSeparator in UI Signed-off-by: Deluan <deluan@navidrome.org> * fix local artist artwork (#3695) Signed-off-by: Deluan <deluan@navidrome.org> * fix: parse vorbis performers Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean function into smaller functions Signed-off-by: Deluan <deluan@navidrome.org> * fix translations for en and pt Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow performers without instrument/subrole Signed-off-by: Deluan <deluan@navidrome.org> * refactor: metadata clean function again Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize split function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split function is now a method of TagConf Signed-off-by: Deluan <deluan@navidrome.org> * fix: humanize Artist total size Signed-off-by: Deluan <deluan@navidrome.org> * add album version to album details Signed-off-by: Deluan <deluan@navidrome.org> * don't display album-level tags in SongInfo Signed-off-by: Deluan <deluan@navidrome.org> * fix genre clicking in Album Page Signed-off-by: Deluan <deluan@navidrome.org> * don't use mbids in Last.fm api calls. From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248: With MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo { artist: { name: "Bee Gees", mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810", url: "https://www.last.fm/music/Bee+Gees", } ``` Without MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo { artist: { name: "Van Morrison", mbid: "a41ac10f-0a56-4672-9161-b83f9b223559", url: "https://www.last.fm/music/Van+Morrison", } ``` Signed-off-by: Deluan <deluan@navidrome.org> * better logging for when the artist folder is not found Signed-off-by: Deluan <deluan@navidrome.org> * fix various issues with artist image resolution Signed-off-by: Deluan <deluan@navidrome.org> * hide "Additional Tags" header if there are none. Signed-off-by: Deluan <deluan@navidrome.org> * simplify tag rendering Signed-off-by: Deluan <deluan@navidrome.org> * enhance logging for artist folder detection Signed-off-by: Deluan <deluan@navidrome.org> * make folderID consistent for relative and absolute folderPaths Signed-off-by: Deluan <deluan@navidrome.org> * handle more folder paths scenarios Signed-off-by: Deluan <deluan@navidrome.org> * filter out other roles when SubsonicArtistParticipations = true Signed-off-by: Deluan <deluan@navidrome.org> * fix "Cannot read properties of undefined" Signed-off-by: Deluan <deluan@navidrome.org> * fix lyrics and comments being truncated (#3701) * fix lyrics and comments being truncated * specifically test for lyrics and comment length * reorder assertions Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * fix(server): Expose library_path for playlist (#3705) Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic) * fix BFR on Windows (#3704) * fix potential reflected cross-site scripting vulnerability Signed-off-by: Deluan <deluan@navidrome.org> * hack to make it work on Windows * ignore windows executables * try fixing the pipeline Signed-off-by: Deluan <deluan@navidrome.org> * allow MusicFolder in other drives * move windows local drive logic to local storage implementation --------- Signed-off-by: Deluan <deluan@navidrome.org> * increase pagination sizes for missing files Signed-off-by: Deluan <deluan@navidrome.org> * reduce level of "already scanning" watcher log message Signed-off-by: Deluan <deluan@navidrome.org> * only count folders with audio files in it See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930 Signed-off-by: Deluan <deluan@navidrome.org> * add album version and catalog number to search Signed-off-by: Deluan <deluan@navidrome.org> * add `organization` alias for `recordlabel` Signed-off-by: Deluan <deluan@navidrome.org> * remove mbid from Last.fm agent Signed-off-by: Deluan <deluan@navidrome.org> * feat: support inspect in ui (#3726) * inspect in ui * address round 1 * add catalogNum to AlbumInfo Signed-off-by: Deluan <deluan@navidrome.org> * remove dependency on metadata_old (deprecated) package Signed-off-by: Deluan <deluan@navidrome.org> * add `RawTags` to model Signed-off-by: Deluan <deluan@navidrome.org> * support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698) * parse standard roles, vorbis/m4a work for now * fix djmixer * working roles, use DJ-mix * add performers to file * map mbids * add a few more tests * add test Signed-off-by: Deluan <deluan@navidrome.org> * try to simplify the performers logic Signed-off-by: Deluan <deluan@navidrome.org> * stylistic changes --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * remove param mutation Signed-off-by: Deluan <deluan@navidrome.org> * run automated SQLite optimizations Signed-off-by: Deluan <deluan@navidrome.org> * fix playlists import/export on Windows * fix import playlists * fix export playlists * better handling of Windows volumes Signed-off-by: Deluan <deluan@navidrome.org> * handle more album ID reassignments Signed-off-by: Deluan <deluan@navidrome.org> * allow adding/overriding tags in the config file Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): Fix playlist track id, handle missing tracks better (#3734) - Use `mediaFileId` instead of `id` for playlist tracks - Only fetch if the file is not missing - If extractor fails to get the file, also error (rather than panic) * optimize DB after each scan. Signed-off-by: Deluan <deluan@navidrome.org> * remove sortable from AlbumSongs columns Signed-off-by: Deluan <deluan@navidrome.org> * simplify query to get missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * mark Scanner.Extractor as deprecated Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Caio Cotts <caio@cotts.com.br> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-02-19 17:35:17 -08:00
"iter"
2020-01-08 20:45:07 -05:00
"net/http"
"os"
"runtime"
"slices"
2020-01-08 20:45:07 -05:00
"strings"
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
"sync"
"sync/atomic"
2021-02-03 13:04:20 -05:00
"time"
2020-01-08 20:45:07 -05:00
"github.com/sirupsen/logrus"
)
2024-08-21 12:15:25 -04:00
type Level uint32
2020-01-08 20:45:07 -05:00
2026-02-08 08:57:30 -06:00
type LevelFunc = func(ctx any, msg any, keyValuePairs ...any)
2020-01-08 20:45:07 -05:00
var redacted = &Hook{
AcceptedLevels: logrus.AllLevels,
RedactionList: []string{
// Keys from the config
"(ApiKey:[\\s]*\")[\\w]*",
"(Secret:[\\s]*\")[\\w]*",
"(PasswordEncryptionKey:[\\s]*\")[^\"]*",
"(UserHeader:[\\s]*\")[^\"]*",
"(TrustedSources:[\\s]*\")[^\"]*",
2023-03-21 11:16:00 -04:00
"(MetricsPath:[\\s]*\")[^\"]*",
"(DevAutoCreateAdminPassword:[\\s]*\")[^\"]*",
"(DevAutoLoginUsername:[\\s]*\")[^\"]*",
// Prometheus.Password. Any character is allowed, so skip escaped quotes in the value
`(Password:[\s]*")(?:[^"\\]|\\.)*`,
// UI appConfig
"(subsonicToken:)[\\w]+(\\s)",
"(subsonicSalt:)[\\w]+(\\s)",
"(token:)[^\\s]+",
// Subsonic query params
"([^\\w]t=)[\\w]+",
"([^\\w]s=)[^&]+",
"([^\\w]p=)[^&]+",
"([^\\w]jwt=)[^&]+",
feat(jellyfin): experimental Jellyfin Music API support (#5730) * feat(sharing): enable sharing by default Flip the EnableSharing default from false to true so new installations have the sharing feature available out of the box. Users can still disable it via the EnableSharing config option. The native API only registers the /share route when sharing is enabled, so the nativeapi tests that build the router without wiring a share service now explicitly disable sharing in their setup to avoid registering a route backed by a nil service. * feat(jellyfin): add config flag and URL path constant Adds the disabled-by-default Server.Jellyfin config option (Enabled, ServerName) and consts.URLPathJellyfinAPI, following the existing LastFM/ListenBrainz patterns. Later tasks will use these to mount the Jellyfin-compatible API router. * fix(jellyfin): default Jellyfin ServerName to "Navidrome" * feat(jellyfin): package skeleton with System handshake endpoints Adds server/jellyfin: the Router (mirrors server/subsonic and server/public), its Wire-friendly New(...) constructor, a chi routes() table, and the ok() JSON response helper. Implements the unauthenticated handshake surface Jellyfin clients probe first: GET /System/Info/Public, GET+POST /System/Ping, and GET /QuickConnect/Enabled (quick connect is unsupported, so it always reports disabled). The public info payload's Id must be stable across restarts (Jellyfin clients cache ServerId), so it reuses the get-or-create Property pattern already used for InsightsID in core/metrics/insights.go: fetch consts.JellyfinServerIDKey from the Property repository, generating and persisting a new UUID on first read. A dedicated key (rather than the existing InsightsID) keeps the anonymous telemetry identifier from being exposed on this unauthenticated endpoint. * test(jellyfin): cover ping and quickConnectEnabled handlers Both were added alongside the System/Info/Public handshake endpoint but had no direct test coverage. * fix(jellyfin): memoize stable server Id and cover get-or-create path * feat(jellyfin): wire and mount the router behind Jellyfin.Enabled Add jellyfin.New to the shared Wire provider set and a CreateJellyfinAPIRouter injector, then mount the router at /jellyfin in startServer(), gated by conf.Server.Jellyfin.Enabled, mirroring the existing LastFM/ListenBrainz router mounts. * feat(jellyfin): add Jellyfin DTOs and model mappers Adds BaseItemDto, QueryResult, UserItemDataDto, UserDto, AuthenticationResult, MediaSourceInfo, PlaybackInfoResponse, NameGuidPair and SessionInfo DTOs to server/jellyfin/dto, plus model-to-DTO mappers for songs, albums, artists and genres that later browse/stream/write endpoints will consume. * test(jellyfin): cover GenreToBaseItem mapper * feat(jellyfin): advertise Jellyfin-compatible version, brand ServerName with Navidrome version * fix(jellyfin): map LastPlayedDate and omit zero track/disc numbers * feat(jellyfin): authentication middleware and AuthenticateByName login * fix(jellyfin): reject empty login password, wire ServerName, add negative auth tests * refactor(jellyfin): use new(x) builtin instead of intPtr helper * feat(jellyfin): user views and current-user endpoints * feat(jellyfin): Items query engine, item detail, and latest Adds the /Items universal query endpoint, dispatching by IncludeItemTypes over albums/artists/songs/genres with ParentId, SearchTerm, Filters=IsFavorite, SortBy/SortOrder and StartIndex/Limit support, plus GET /Items/{itemId} and /Users/{userId}/Items/Latest. Reuses server/subsonic/filter builders (by artist/album/starred) instead of hand-rolled squirrel filters, and resolves sort keys per item type since each repository maps sort names to different real/aliased columns. Also adds the missing CountAll to tests.MockArtistRepo, exposed by this task (the mock embedded a nil ArtistRepository for it and would panic on use). * refactor(server): extract shared query filter builders to server/filter Moves server/subsonic/filter to server/filter so server/jellyfin can use the shared query-option builders without importing server/subsonic, enforcing the rule that no API package imports another API's package. * feat(jellyfin): full multi-library support with per-user access scoping Replace the single hardcoded "music" UserView with one CollectionFolder view per library the user can access, and scope every /Items browse query (albums, songs, artists, /Latest) to those libraries via server/filter's ApplyLibraryFilter/ApplyArtistLibraryFilter. ParentId is now disambiguated: a numeric value the user has access to is treated as a library scope (browsing a UserView), otherwise it falls through as an entity id (artist/album), which safely matches nothing rather than leaking another library's content. getItem now 404s when fetching an album or song outside the user's accessible libraries; artists are skipped (they can span multiple libraries) with a TODO. Genres remain unscoped since they're global tags, not per-library entities. * test(jellyfin): cover admin library-access path and drop nil test contexts * feat(jellyfin): Artists and Genres endpoints Add /Artists, /Artists/AlbumArtists, /Genres and /MusicGenres. Artists listing is library-scoped (defaulting to the user's accessible libraries, narrowed by an accessible ParentId), delegating access control to listArtists/ApplyArtistLibraryFilter. Genres are global and unscoped, matching the ML decision already made for listGenres. * refactor(jellyfin): share resolveLibraryScope between items and artists * feat(jellyfin): item image endpoint via artwork service * fix(jellyfin): let net/http sniff image Content-Type instead of forcing jpeg * feat(jellyfin): audio streaming and PlaybackInfo with library access control * feat(jellyfin): favorites and rating write-back with library access control Adds POST/DELETE handlers for /Users/{userId}/FavoriteItems/{itemId} and /Users/{userId}/Items/{itemId}/Rating. A shared resolveAnnotated helper probes album/artist/media file (mirroring getItem's order) and 404s before writing if the user lacks access to the album/song's library; artists are exempt since they span multiple libraries. Ratings are halved coming in (Jellyfin 0-10 -> Navidrome 0-5) to match the doubling in dto.UserData. Also teaches MockMediaFileRepo and MockArtistRepo's SetStar/SetRating (and MockAlbumRepo's, previously a no-op) to actually mutate the backing data so write-back can be asserted in tests. * feat(jellyfin): playback reporting and scrobbling Add /Sessions/Playing[/Progress|/Stopped] and /Sessions/Capabilities[/Full] handlers, backed by core/scrobbler.PlayTracker. A new withPlayer middleware resolves/registers a model.Player from the Emby DeviceId header (used directly as the stable player id, unlike Subsonic's cookie fallback) and injects it into the request context for scrobbling. The Stopped report calls ReportPlayback with IgnoreScrobble to end the now-playing session without double-counting, then calls Submit as the single source of the play-count increment and external scrobble. * feat(jellyfin): playlist read and write-back Adds POST /Playlists, GET/POST/DELETE /Playlists/{id}/Items. Tags each playlist item with PlaylistItemId (the entry's position within the playlist) so DELETE .../Items?EntryIds=... can remove a specific occurrence by the id core/playlists.RemoveTracks actually expects, rather than the song id used everywhere else. * test(jellyfin): cover empty Ids/EntryIds in playlist add/remove * feat(jellyfin): unknown-route logging, generic 500s, rating clamp, docs Hardening pass ahead of real client testing: unmatched routes and unsupported methods now return a logged, JSON 404 instead of chi's default plain-text response, so a missing endpoint a client needs is easy to spot in the logs. Internal errors (ffmpeg output, file paths, etc.) no longer leak into 500 response bodies -- a shared internalError helper logs the real error server-side and always returns a generic message. Inbound Jellyfin ratings are clamped to 0-10 before being halved into Navidrome's 0-5 scale, and /System/Ping now replies with a bare plain-text body as real Jellyfin servers do. Adds a README with an enable/curl walkthrough and known limitations. * docs(jellyfin): clarify public image endpoint and accessibleLibraryIDs comments * fix(jellyfin): case-insensitive path routing for Jellyfin client compatibility * refactor(server): extract case-insensitive path routing to a shared helper * fix(jellyfin): return User Policy and Configuration so Finamp completes login * fix(jellyfin): support Playlist type, multi-type Items queries, and PlayCount/DatePlayed sort Real Finamp requests break against three /Items query engine bugs: Playlist requests fell through to albums, multi-type IncludeItemTypes (e.g. Finamp's favorites screen) only returned the first requested type, and comma-separated SortBy lists (e.g. "DateCreated,SortName") were matched as one opaque string so they always fell through to the repo default. Also extends tests/mock_playlist_repo.go with SetData/GetAll so playlist listing is testable like the other mock repos. * feat(jellyfin): implement /socket WebSocket for real-time client sessions * fix(jellyfin): resolve library-view ids in getItem so clients can load the library * fix(jellyfin): serve direct file at /Items/{id}/File and accept ApiKey query param * fix(jellyfin): resolve playlist ids in getItem * fix(jellyfin): read lowercase ids/entryIds playlist params and add playlist-users endpoints * fix(jellyfin): include MediaSources with Size/Bitrate on tracks so clients show download size * fix(jellyfin): populate all required MediaSourceInfo bool/array fields to match Jellyfin * fix(jellyfin): hex-encode item ids at the API boundary for Jellyfin client compatibility Finamp (and presumably other clients) parses ids as radix-16, but Navidrome's base62 nanoids aren't valid hex and crash its queue packing. Hex-encode every id emitted by the Jellyfin API and decode every id received, keeping the transform reversible and stateless at the boundary rather than touching any model id. * fix(jellyfin): populate MediaStreams with the audio stream so clients can size/transcode * fix(jellyfin): support Ids batch-fetch in /Items so clients can fetch items by id * fix(jellyfin): do not disable Jellyfin server when disabling external services * fix(jellyfin): emit per-image ImageBlurHashes so clients de-dupe images and stop warning * fix(jellyfin): support playlist cover upload/delete and display via item image endpoints * feat(jellyfin): implement GET /Playlists/{id} for playlist visibility Finamp's playlist edit screen calls GET /Playlists/{id} to read the OpenAccess (public visibility) flag; we only had the sub-routes, so it 404'd and the edit screen failed to load. Return the Jellyfin PlaylistDto shape (OpenAccess from Public, empty Shares, media item ids). * fix(jellyfin): display playlist cover art Uploaded playlist covers never showed in clients for two reasons: the playlist BaseItemDto advertised no Primary ImageTag (so clients didn't know to fetch a cover), and the public image endpoint resolved artwork under the request's anonymous context, so a private playlist failed its visibility filter and fell back to the placeholder. Advertise the Primary image tag/blurhash on playlists, and resolve artwork under an elevated context (as core/artwork's cache warmer does). * fix(jellyfin): expand album/artist/playlist ids when building playlists Jellyfin clients (Finamp) send container ids — an album, artist or playlist — in a playlist's Ids list and expect the server to expand each into its child tracks. core/playlists only understands media file ids, so creating or adding with an album id silently produced an empty playlist. Expand container ids to their tracks (in order) before create/add; bare song ids still pass through. Adds filter.SongsByArtistID. * fix(jellyfin): support playlist deletion via DELETE /Items/{id} Finamp deletes a playlist with DELETE /Items/{id}, which we didn't route, so deletion silently failed. Implement it via core/playlists.Delete (which enforces ownership and removes the cover file). Only playlists are deletable through this API; non-playlist ids return 404, non-owners 403. * test(jellyfin): add e2e suite harness + smoke tests * test(jellyfin): e2e for system, auth, routing, browsing, annotations * test(jellyfin): e2e for playlists (CRUD, expansion, cover) and item images * test(jellyfin): e2e for streaming, sessions, and multi-user access control * fix(jellyfin): artist search 500 (library filter leaked into FTS query) getArtists/listArtists reused the browse-path filters (notMissing + ApplyArtistLibraryFilter) for the search path, but artist Search expects a sole Eq{library_id} filter it can consume as a scope — artists have no library_id column, so the compound/join filter leaked into the FTS query and 500'd. Every Finamp artist search failed (the filter is applied even unscoped, since admins resolve to all library ids). Build search filters separately. Adds e2e search coverage. * feat(jellyfin): implement POST /Playlists/{id} to update name, visibility, tracks Finamp edits a playlist (make public, rename, reorder) via POST /Playlists/{id}, which we didn't route, so every edit 404'd. Implement it: Ids present -> replace track list (Create with existing id, preserving name); otherwise update Name/IsPublic via core/playlists.Update. Adds a shared playlistError helper (403/404/500) reused by deleteItem, plus e2e coverage. * docs(jellyfin): update README for playlists, images, id encoding, and e2e * fix(jellyfin): default album track listing to track order Browsing an album's tracks (Items?ParentId=<albumId>&IncludeItemTypes=Audio) took only SongsByAlbum's filters and dropped its Sort, so tracks came back in arbitrary order. Default opts.Sort to the album (disc+track) order when browsing an album without an explicit SortBy — matching Subsonic's GetAlbum and real Jellyfin. An explicit SortBy still wins. * fix(jellyfin): filter items by AlbumArtistIds/ArtistIds An artist's page in Finamp sends ParentId=<libraryId> (scoping) plus AlbumArtistIds/ArtistIds/contributingArtistIds for the artist itself, but queryItems only honored ParentId, so an artist's albums and tracks came back unfiltered (every artist's content). Parse the artist-id params and apply AlbumsByArtistID (albums) / SongsByArtistID (tracks). Adds e2e coverage. * fix(jellyfin): only count a play past the scrobble threshold reportPlaybackStopped set IgnoreScrobble and force-submitted a play on every Stopped report, so a briefly-played track (e.g. an immediate skip) was marked played. Finamp sends Stopped on every track switch, so the threshold must be applied server-side. Let ReportPlayback's StateStopped logic decide (play + scrobble only past 50% of the track / 4-minute cap) instead. Subsonic differs because there the client gates submission. * fix(jellyfin): sort album tracks by track number when client sends IndexNumber SortBy Finamp's album view requests SortBy=ParentIndexNumber,IndexNumber,SortName (disc, track, name), but applySort didn't recognize ParentIndexNumber or IndexNumber and fell through to SortName, sorting tracks alphabetically by title. Map both keys to the album (disc+track) sort. The reversed-title fixture lets the e2e tell track order from title order. * fix(jellyfin): honor the isFavorite query param for favorites filtering Finamp's artist 'Favourite tracks' widget requests favorites via the standalone isFavorite=true query param, not Filters=IsFavorite, so the filter was ignored and non-favorited tracks were returned. Detect both forms. (The widget's reshuffling is Finamp's explicit SortBy=Random.) * feat(jellyfin): emit DateCreated (Date Added) on items BaseItemDto had no DateCreated, so clients showed 'No Date Added' and had nothing to sort 'Recently Added' by. Emit it as ISO 8601 from each entity's CreatedAt (the same field the recently_added sort uses) for songs, albums and artists. * fix(jellyfin): set ArtistItems/AlbumArtists on songs (Now Playing artist) SongToBaseItem only set Artists (names) and AlbumArtist, not the structured ArtistItems/AlbumArtists. Finamp's Now Playing screen reads ArtistItems and shows 'Unknown Artist' when it's absent. Populate both (track artist and album artist) as name+id pairs, mirroring AlbumToBaseItem. * docs(jellyfin): note synthetic blurhash as a follow-up Document that ImageBlurHashes are derived from the item id (a solid-color placeholder), not computed from the cover art like real Jellyfin, and outline what a proper implementation would take. * docs(jellyfin): note WebSocket events and favourited playlists as follow-ups * fix(jellyfin): filter the Artists page by role (album artist vs performer) /Artists and /Artists/AlbumArtists both called the same role-agnostic handler, so Navidrome's per-role artist entries (composers, arrangers, performers) all showed as Album Artists, and the two tabs were identical. Filter /Artists/AlbumArtists to RoleAlbumArtist and /Artists to RoleArtist (and the MusicArtist browse to album artists), via filter.ArtistsByRole. Verified live: Beatles Singles' album-artists dropped 138->5, composers excluded, performers (Billy Preston) show only under /Artists. * fix(jellyfin): sort playlists by name (missing Playlist sort mapping) applySort had no sortColumnsByType entry for the Playlist type, so SortBy=SortName was ignored and the Playlists screen showed them in the repo's default order. Map SortName/Name -> name (as Subsonic does) and DateCreated -> created_at. Verified live: case-insensitive alphabetical. * fix(jellyfin): read query params case-insensitively (Jellify support) Jellify (and the official Jellyfin TypeScript SDK) send query params in camelCase (parentId, albumArtistIds, artistIds, includeItemTypes), where Finamp sends PascalCase. Our handlers read fixed-case keys, so every Jellify filter/sort/paging param was silently dropped: - an artist's page listed albums and tracks from all artists - opening an album listed every album instead of its tracks Real Jellyfin binds query params case-insensitively (ASP.NET model binding), so add a normalizeQueryKeys middleware that folds every query key to lowercase once, and read params by their lowercase name. This also removes the scattered dual-case hacks (Ids/ids, IsFavorite, ContributingArtistIds, api_key/ApiKey, queryParam) that let this bug class through. Also infer the child type from an album parent: Jellify browses an album with only parentId (no IncludeItemTypes), and Jellyfin infers Audio from the parent; without it we fell back to listing all albums. Verified live against Finamp+Jellify and with 4 new e2e specs replaying Jellify's exact request shapes. * fix(jellyfin): advertise LocalAddress in the public system info handshake Jellify (and other @jellyfin/sdk clients) that connect by raw address fall back to HTTP when TLS isn't available, then adopt the handshake's LocalAddress as their server base URL. We never populated it, so Jellify's SDK `api` object was undefined and sign-in crashed with "Cannot read property 'configuration' of undefined" (getUserApi(api!) with an undefined api). Over an HTTPS connection Jellify uses connectionType=hostname and never reads LocalAddress, which is why it worked while an HTTPS proxy was in front. Populate LocalAddress from the request — scheme + host (honoring X-Forwarded-*), plus the /jellyfin mount path — matching real Jellyfin, which always sends it. Export server.ServerAddress for the resolution. * fix(jellyfin): separate "Featured On" from an artist's own discography Jellify's artist page fetches the discography via albumArtistIds and the "Featured On" section via contributingArtistIds, relying on the server to return disjoint sets. We collapsed albumArtistIds/artistIds/ contributingArtistIds into one AlbumsByArtistID filter, so an artist's own albums appeared in both sections. Add AlbumsByContributingArtistID — albums where the artist is a track artist but NOT the album artist — matching Jellyfin's ContributingArtistIds (in Artists, not in AlbumArtists), and route contributingArtistIds to it. * fix(jellyfin): accept a bare Authorization token for audio streaming Jellify's native player (react-native-nitro-player) authenticates the audio stream by setting a bare Authorization header carrying the raw access token ({ AUTHORIZATION: api.accessToken }), not the "MediaBrowser ... Token=" scheme parseEmbyAuth understands. tokenFromRequest didn't recognize it, so every /Audio/{id}/stream request from the native player 401'd and no audio played (the JS client still optimistically posted playback progress, masking it). Accept a bare (or "Bearer <token>") Authorization header, as real Jellyfin does. Verified live: the native player's exact request now streams 200. * fix(jellyfin): embed a self-authenticating stream URL in PlaybackInfo Jellify's native audio player (react-native-nitro-player / ExoPlayer) fetches the stream without forwarding any auth — captured request headers were only Connection/Icy-Metadata/Accept-Encoding/User-Agent, no Authorization and no api_key — so every /Audio/{id}/stream request 401'd and nothing played (the JS client still optimistically posted progress, masking it). Real Jellyfin returns stream URLs with the token embedded; we returned none, so Jellify fell back to a token-less DirectPlay URL. Populate MediaSources[0].TranscodingUrl with /Audio/{id}/universal?api_key=<caller token>, which Jellify's player uses verbatim. Direct-play clients (Finamp builds its own /Items/{id}/File?ApiKey URL) ignore the field, so they're unaffected. * feat(jellyfin): serve per-item UserData (GET /UserItems/{id}/UserData) Jellify fetches this per item to render played/favourite indicators; we 404'd it. Resolve the item via the existing getItem resolver (which loads the caller's annotations and enforces the same library-access gate) and return its UserData, falling back to an empty-but-valid object for items without annotations (e.g. playlists). Also registers the legacy /Users/{userId}/Items/{itemId}/UserData spelling. * refactor(jellyfin): drop unused bare-Authorization token support Added mid-troubleshooting on the theory that Jellify's native player sends a bare Authorization header, but the debug capture showed it sends no auth header at all — the real fix was embedding api_key in PlaybackInfo's TranscodingUrl. No client reaches this path (Jellify JS uses the MediaBrowser scheme, Finamp uses X-Emby-Token, the native player uses the api_key URL), so remove it and its tests per YAGNI. Effectively reverts 4f8ac1e0. * feat(jellyfin): implement Similar endpoints via the external provider Add GET /Artists/{id}/Similar (related artists) and GET /Items/{id}/Similar (similar songs for a track, similar albums for an album, related artists for an artist), sourced from the same external.Provider (Last.fm etc.) that powers Subsonic's getArtistInfo2/getSimilarSongs. Only library-present artists are returned so each is navigable; provider errors and unknown ids degrade to an empty 200 result, so clients (Jellify) stop hammering these with 404 retries. Injects external.Provider into the Router (regenerated via make wire). * perf(jellyfin): make Similar endpoints non-blocking (bounded quick wait) Each /Similar request fetched from Last.fm synchronously (500ms-1.4s), and Jellify requests Similar for many items at once on the home/artist screens, so the whole screen stalled — a home pull-to-refresh dragged for seconds. Run the external lookup on a background context and return within a 500ms quick wait: a cached artist resolves instantly, a cold one returns empty now while the lookup finishes caching in the background, so a later load is fast and populated. Verified live: cold calls bounded at ~500ms (was up to 1.4s), warm calls 3-8ms. * fix(jellyfin): answer ManualPlaylistsFolder so the home stops stalling Jellify resolves its "playlists library" via IncludeItemTypes= ManualPlaylistsFolder, then lists playlists with ParentId set to that folder's id. We didn't recognize the type, so parseTypes fell back to MusicAlbum and returned the album list. Jellify's query then found no item with CollectionType=playlists and resolved undefined — which React Query rejects, retrying it in a backoff loop that stalled the home pull-to-refresh for ~5s (every response was fast server-side; the delay was the client's retries). Return a synthetic "playlists" folder (CollectionType=playlists) for the ManualPlaylistsFolder query, resolve ParentId=<that folder> to the user's playlists, and give playlists a Path under "data" (Jellify drops playlists whose Path lacks it). Adds a Path field to BaseItemDto. * docs(jellyfin): note lyrics and InstantMix/sonic-similarity as follow-ups * fix(jellyfin): genre paging params and same-key casing collisions getGenres read StartIndex/Limit in PascalCase, which normalizeQueryKeys had already folded to lowercase, so genre paging was silently ignored; totals now come from the full (small) genre list instead of the page length. normalizeQueryKeys now merges values when two casings of a key collide, instead of nondeterministically keeping one. * fix(jellyfin): round ratings to the nearest star instead of truncating Rating is a nullable double 0-10 in Jellyfin's contract. Truncating integer division stored 9 as 4 stars, and both Rating=1 and fractional values (which failed integer parsing) became 0 — silently deleting the rating. Parse as float, round, and floor nonzero input at one star. * fix(jellyfin): apply Name/IsPublic sent together with a track replacement Jellyfin's UpdatePlaylist applies every provided field, but the Ids branch returned early, silently discarding a rename or visibility change sent in the same body (core/playlists.Create with an existing id ignores the name). * fix(jellyfin): don't rotate the stored server id on transient DB errors serverID treated any Property.Get error as "no id yet" and persisted a fresh UUID over JellyfinServerID, with sync.Once pinning it for the process lifetime — a busy DB or canceled request context on the first request would break every client's cached ServerId. Only ErrNotFound mints a new id now, and failures yield an uncached temporary value so the next request retries. * fix(jellyfin): report real search totals instead of page length or unfiltered counts Artist search returned TotalRecordCount = len(page), so clients stopped after the first page; album/song search counted via CountAll, which can't see the search term, so clients paged through phantom results. The repos' Search API has no match count, so fetch one row beyond the page: offset+len is exact on the last page and a strictly growing lower bound before it — paging clients terminate exactly at the last match. * fix(jellyfin): browse playlists via the generic /Items path and resolve the playlists folder by id A typeless /Items?ParentId=<playlistId> (legal in real Jellyfin, used by generic clients) fell through to the MusicAlbum default and returned an empty list; it now returns the playlist's tracks, paginated, with visibility enforced by GetWithTracks. resolveItemByID also answers the synthetic playlists-folder id the server itself advertises instead of 404ing it. * perf(jellyfin): cap per-type queries in multi-type /Items requests The multi-type merge path queried each type with a zero-value QueryOptions — no LIMIT in SQL — materializing every matching row (each with embedded MediaSources) just to slice out one page in memory. Each type now fetches at most StartIndex+Limit rows, the worst case one type can contribute to the merged window; totals still come from CountAll. * fix(jellyfin): dedupe and bound the background Similar fetches awaitSimilar spawned a detached, deadline-free goroutine per request; clients re-polling after the empty quick-wait response piled up duplicate provider chains (no singleflight anywhere below) racing writes on the same artist row. Identical in-flight requests now share one fetch — keyed per user, since the mapped items embed the user's annotations — and the background context gets a one-minute deadline so a hung provider can't hold goroutines forever. * fix(jellyfin): gate private playlist covers on the public image route The unauthenticated image endpoint elevated every request to an admin context, so anyone who knew or guessed a playlist id could fetch another user's private uploaded cover. Library artwork still resolves elevated (Jellyfin clients fetch images without auth headers), but playlist covers are now served only when the playlist is public or the request's optional token identifies its owner or an admin; everyone else gets the placeholder. * fix(log): redact full api_key values, including JWTs The api_key pattern matched only word characters, stopping at a JWT's first '.' — the Jellyfin API embeds the session JWT as api_key in TranscodingUrl, so request logs kept its payload and signature, enough to reconstruct a replayable token by prepending the constant header. Match to the next query separator instead, like the sibling s=/p=/jwt= patterns. * fix(jellyfin): record LastLoginAt on Jellyfin logins authenticateByName re-implements credential validation and skipped the UpdateLastLoginAt call the web UI's validateLogin makes, so users who only log in via Jellyfin clients showed a never/stale Last Login in the admin UI. * perf(jellyfin): batch song resolution in /Items?ids= and playlist expansion Both paths probed up to four repositories per client-supplied id. Songs — the common case — now resolve via chunked media_file.id IN queries (same pattern as playqueue's loadTracks); only the residue pays the container probes. * fix(jellyfin): wait for the real Similar result instead of answering a cacheable empty list The 500ms quick wait returned an empty 200 for any cold lookup — indistinguishable from "no similar items exist", so clients cached the wrong answer until an app restart. With fetches deduplicated, wait up to the agents' HTTP timeout for the actual result; only a hung provider now yields the empty fallback, and its fetch still warms the cache in the background. Also makes the dedup test deterministic (the old one raced its release channel). * refactor(tests): extract the shared e2e harness into tests/harness The Subsonic and Jellyfin e2e suites each carried their own copy of the golden-DB lifecycle (boot, seed users/library, scan, WAL snapshot), the ATTACH-DATABASE restore, fixture-FS registration, and the SpyStreamer / NoopFFmpeg doubles. Those now live in one importable package (same pattern as core/storage/storagetest); fixture libraries and request helpers stay per-suite since they encode each API's test expectations. * docs(jellyfin): make comments concise Compress the narrative comments accumulated during live client testing into short why-only notes; keep the client quirks, security rationales and gotchas, drop the exposition. Comments-only change (net -141 lines). * fix(tests): silence gosec taint false-positive in harness snapshot write The write moved from a _test.go file (which gosec skips) into the importable harness package; the path derives from GinkgoT().TempDir(). * fix(jellyfin): route the current /UserFavoriteItems favorite endpoint @jellyfin/sdk 0.13.0 (used by Jellify) posts favorites to POST/DELETE /UserFavoriteItems/{itemId}, while we only routed the legacy /Users/{userId}/FavoriteItems/{itemId} (Finamp). Jellify favorites 404'd ("Failed to add favourite"). Route both spellings to the same handlers. * feat(jellyfin): honor Fields on /Items and add missing conformance fields Match real Jellyfin's response shape: gate MediaSources (and MediaStreams) behind Fields=MediaSources instead of always embedding them — clients that need Size request it, as Finamp does — which cuts a 46-track artist response from ~87KB to a fraction. Also emit the always-present fields Jellyfin sets: ServerId (stamped centrally in ok), LocationType, HasLyrics, and SortName (when Fields=SortName). PlaybackInfo still carries MediaSources (its purpose). * fix(jellyfin): address code review security and correctness findings Applies the reviewed findings from PR #5730: - Gate similar songs/albums on the caller's library access, so the external provider can't surface metadata from libraries the user cannot see. Also clamp the client-supplied limit before it sizes any allocation or provider fetch (CodeQL user-controlled allocation). - Prepend the /jellyfin mount prefix to the PlaybackInfo TranscodingUrl, so a client resolving it as an absolute host path still reaches the mounted router. - Recognize WebP and GIF magic numbers on raw cover uploads, matching the formats Navidrome already supports. - Bound the playlist cover upload: honor EnableArtworkUpload for non-admins and cap the body with MaxBytesReader/MaxImageUploadSize, mirroring the native image endpoint. - Propagate non-not-found repository errors from resolveAnnotated as 500 instead of silently returning 404. - Normalize the literal prefix of mixed literal.param path segments (e.g. STREAM.mp3) so case-insensitive routing reaches stream.{container}. - Rate-limit POST /Users/AuthenticateByName with the same per-IP limiter as /auth/login when AuthRequestLimit is set. - Guard against a nil user in authenticateByName. * fix(jellyfin): correct playlist track browsing and multi-id edits Fixes three playlist issues, two found testing against Jellify and one from the PR review: - Resolve a playlist ParentId to its tracks even when the client sends IncludeItemTypes=Audio. Jellify opens a playlist with ParentId=<playlist>&IncludeItemTypes=Audio; the id was routed through listSongs as an album id, returning an empty list. - Read repeated id query params (ids=X&ids=Y), not just the first value. Jellify's @jellyfin/sdk serializes id arrays as repeated params, so adding an album (which it expands client-side into many ids) only added the first track. Applies to both add and remove; the comma-separated form other clients use still works. - Let an explicit empty Ids array clear a playlist. Ids is now a pointer so an omitted field still means 'leave unchanged', while an empty list clears the tracks (via RemoveTracks, since the repository skips track writes for an empty list). * fix(jellyfin): register clients as players on any authenticated request Jellyfin clients did not appear in the players list. Unlike Subsonic, whose getPlayer middleware runs on every authenticated endpoint, the Jellyfin router only registered a player on the /Sessions/Playing reports, so browsing or streaming never created one. Apply withPlayer to the whole authenticated group, mirroring Subsonic, so the calling device registers (and scrobbling has a player) as soon as it makes any authenticated request. Two follow-ups found while testing: - Skip registration when the request carries no client/device info (no X-Emby-Authorization, e.g. the /socket handshake that auths via ?api_key= only), which otherwise created a junk player named ' []'. - URL-decode the X-Emby-Authorization field values. Jellify's @jellyfin/sdk percent-encodes them (Device='Pixel%208%20Pro') while Finamp sends them raw, so the player name showed as 'Jellify [Pixel%208%20Pro]'. * refactor(jellyfin): move case-insensitive routing into the jellyfin package The case-insensitive path normalization lived in the server package but was only ever used by the Jellyfin router (its whole purpose is that Jellyfin clients route case-insensitively while chi does not). Move it into server/jellyfin and unexport it, so it sits with its only caller and no longer needs to be exported across a package boundary. * docs(jellyfin): document player registration, playlist and image behavior Updates the package README for the behavior added/fixed this round: - New 'Players and sessions' section: any authenticated request now registers the device as a player (like Subsonic), with the Client [Device] naming, URL-decoding of the Emby auth fields, and the /socket skip that avoids a nameless player. - Authentication: note AuthenticateByName is rate-limited per IP. - Playlists: repeated vs comma-separated id params, and that an explicit empty Ids clears the playlist while an omitted Ids leaves it untouched. - Cover art: WebP/GIF magic-number detection plus the MaxImageUploadSize and EnableArtworkUpload gates. - Endpoints table: add the /Similar and /UserFavoriteItems / /UserItems/.../UserData routes that were already served but unlisted. * feat(jellyfin): expose configured users on the login user-picker Adds Jellyfin.ExposedPublicUsers, a comma-separated allowlist of usernames that GET /Users/Public advertises so Jellyfin clients (Finamp, Jellify) can show a login user-picker instead of a blank username field. The endpoint is unauthenticated, so it defaults to exposing no users and never lists the full user table: only the admin-configured names are returned, resolved live per request (a name that doesn't exist is skipped and logged). Each entry is a minimal DTO (Name, Id) with no Policy/Configuration, so admin status isn't leaked pre-login, and no avatar since Navidrome has no per-user profile images. * style(jellyfin): trim redundant comments Remove comments that restated the code or duplicated an explanation already given nearby, keeping the ones that capture non-obvious rationale (client-specific quirks, gotchas). Comment-only change; no behavior difference. * perf(db): keep query planner statistics trustworthy with full ANALYZE PRAGMA optimize's internal ANALYZE runs with a limited analysis budget (~2000 rows) that writes wrong sqlite_stat1 entries for low-cardinality indexes: on a 96K-track library it claimed (missing, library_id) narrows to ~2000 rows when it matches the whole table. The planner then prefers that index over the sort index and falls back to a full-table temp B-tree sort per request, turning paginated song listings into multi-second queries (reproduced at 5.5s on real hardware; ~90x slower than with correct stats). Every index-creating migration re-triggered the poisoning via the post-migration optimize, and the daily optimizer could re-trigger it on large library changes. Setting analysis_limit on the connection does not help: optimize ignores it. Run a plain full ANALYZE instead: after migrations with schema changes, and in db.Optimize (daily schedule and scan-end). Stats are stored in the database file, so one connection suffices and the per-connection pool loop is gone. The Optimize call at shutdown is removed: stats are maintained at migration/scan/daily points, and an ANALYZE during shutdown only delays it and races container stop timeouts. * perf(db): add covering index for title-sorted song listings Deep pagination over songs sorted by title (WHERE missing/library_id, ORDER BY order_title LIMIT/OFFSET - the shape Jellyfin clients use to enumerate the library, and non-admin native/Subsonic song lists share) walked media_file_order_title and fetched the table row for every skipped entry just to evaluate the filter and the annotation/bookmark join keys: offset+limit random reads, seconds per page on cold spinning disks. The new (missing, library_id, order_title, id) index makes the offset skip fully index-resident: filter columns and the join key come from the index, and only the emitted page touches table rows. Measured on a 96K-track library: offset 50000 drops from ~6.5s (poisoned stats) / ~100ms (good stats, warm) to 24ms, and cold deep pages on NAS hardware from ~7s to ~0.5s. * feat(jellyfin): support transcoding via HLS playlist and server-forced player format - withPlayer now propagates the player's configured transcoding into the request context (like Subsonic's getPlayer), so a format forced in Settings > Players applies to the Jellyfin stream endpoints - new GET /Audio/{itemId}/main.m3u8, the endpoint Finamp plays through when its transcoding setting is enabled: a single-segment HLS VOD playlist pointing at the existing progressive transcode endpoint - streamAudio: treat audioBitRate as bits/sec (Jellyfin convention) and fall back to audioCodec as target format when no container is given * fix(jellyfin): honor GenreIds when browsing genre albums and tracks Finamp's genre screen sends ParentId=<libraryId> plus GenreIds=<genreId>, but /Items ignored the param, so every genre returned the whole library. - filter.ByGenreID delegates to persistence.TagIDFilter (exported, was tagIDFilter), the same mechanism behind the native API's genre_id filter - id lists are read via queryIDs, covering both spellings clients use (comma-separated and repeated params); /Items?ids= gains the repeated form too * feat(jellyfin): filter album artists by GenreIds Finamp's artist tab sends GenreIds to /Artists/AlbumArtists when a genre filter is active; the param was ignored, returning all artists. filter.ArtistsByGenreID matches artists credited as album artist on an album with the genre, via a non-correlated semi-join over album participants (86ms on a 29K-artist library; the correlated EXISTS form takes 11 minutes). Applied on the browse path of /Artists* and /Items?IncludeItemTypes=MusicArtist. The performers variant (/Artists?GenreIds=) still ignores the filter: it needs the same semi-join against media_file.participants, unmeasured on large libraries. * fix(jellyfin): version playlist image tag so clients refresh uploaded covers Uploads were stored and served correctly, but Finamp kept showing the old cover: it caches covers keyed by blurHash (its imageId is the item id, which never changes), and both our image tag and synthetic blurhash were derived from the playlist id alone. The tag is now <id>-<UpdatedAt millis hex> (SetImage/RemoveImage go through a full Put, which bumps UpdatedAt), and the blurhash derives from the tag, so every cover change rotates both. UpdatedAt over-invalidates (any playlist edit busts the cover cache), which only costs a refetch; hashing the actual image remains the proper long-term tag. An e2e test guards the whole chain, since a partial Put(pls, cols...) would silently stop bumping UpdatedAt. * fix(jellyfin): align cover upload limit and validation with the native endpoint Cover uploads of large photos failed with a silent 400: the MaxImageUploadSize cap was applied to the wire body, which Jellyfin clients base64-encode (4/3 inflation), so the effective raw-image limit was only ~7.5MB — and Finamp uploads picked photos uncompressed. Align with the native endpoint: - the limit caps the decoded image; the read cap allows for base64 inflation - validate by decoding (image.DecodeConfig) and take the storage extension from the real format instead of the Content-Type header, which clients get wrong (Finamp falls back to image/jpeg); a HEIC or corrupt file is now rejected instead of stored as a broken cover - rejected uploads log the reason (size/limit/decode error); diagnosing this from production logs previously required guesswork * fix(jellyfin): optimize SongsByArtistID filter to improve performance at library scale Signed-off-by: Deluan <deluan@navidrome.org> * fix(jellyfin): sort tracks by release year for SortBy=PremiereDate Finamp's "Latest Releases" artist section sends SortBy=PremiereDate,Album,... descending; PremiereDate wasn't in the Audio sort map, so applySort fell through to Album and the view came back in reverse album-name order (a 2002 remix album first, the 2013 release last). Map premieredate/productionyear to the year sort key, matching the ProductionYear the DTO exposes for songs and the existing MusicAlbum mapping (premieredate -> max_year). * feat(jellyfin): expose PremiereDate on tracks and albums Finamp's "Latest Releases" artist/genre sections re-sort the merged server responses client-side by PremiereDate and keep the top 5. Without the field every comparison returns equal and Dart's unstable sort leaves the picks in arbitrary order — a 2007 remix could lead the list even with the server sorting by year correctly. Serialize PremiereDate as ISO 8601 from the date tag (padding partial "2007"/"2007-02" values so DateTime.tryParse accepts them), falling back to the year; omitted when neither exists. * feat(jellyfin): resolve Finamp-truncated item ids (saved queue restore) Finamp persists its play queue by packing every item id into exactly 16 bytes (packIds assumes Jellyfin's 32-hex GUID ids), so our longer ids come back truncated after an app restart and "Failed to restore queue" loops forever: the /Items?ids= batch resolves nothing. Navidrome ids can't be made GUID-shaped (nanoid ids can exceed 128 bits), so compensate server-side: a 16-char id — a length no Navidrome id family uses — is resolved by unique-prefix range scan, with ambiguity failing safe. The ids= batch echoes the id as requested (Finamp matches restored items by its stored ids), and stream, image, item, user-data, favorite, rating and playback-report endpoints accept truncated ids transparently. The proper fix belongs upstream in Finamp's packIds; documented in the README so this layer can be removed once that ships. * feat(jellyfin): implement Items/{id}/InstantMix Finamp requests an instant mix on every track tap when its "start instant mix for individual tracks" setting is on (plus the long-press menus); the 404 made those taps fail with an error and play nothing. A track seed returns itself first — Finamp plays exactly what comes back — followed by the external provider's similar songs, capped at the requested limit and filtered to the caller's libraries. Container seeds (artist/album) return the provider's similar-songs blend. Provider errors and unknown seeds degrade to seed-only/empty results instead of 404s, reusing the Similar endpoints' bounded-wait singleflight (with a distinct cache key, since mixes and similar lists answer different shapes). Sonic-similarity backing stays a follow-up (see README). * style(jellyfin): trim wordy comments * refactor(jellyfin): apply cleanup review findings - batch truncated-id resolution in /Items?ids=: one chunked range query for all media-file prefixes instead of a query per id (a restored queue sends hundreds of truncated ids) - resolve truncated ids on the /Similar endpoints too, matching the neighboring InstantMix; document which entry points don't resolve - hoist maxImageUploadSize to core, deleting the byte-identical copies in nativeapi and jellyfin (tests moved to core) - drop the unused type parameter on premiereDate * fix(jellyfin): never drop the instant mix seed on a slow provider The seed track was built inside the awaited provider fetch, so when the external agent was slow or unreachable the request hit the 10s wait and answered a fully empty mix — Finamp then played nothing on tap, even though the seed needs no provider at all (seen live: Last.fm unreachable from the server, responseSize=49). Build the seed outside the await: only the similar-songs tail is fetched and bounded, and a timeout now degrades to a seed-only mix. Also folds instantMixForSong into getInstantMix, since the tail is exactly similarSongs. * fix(jellyfin): prefer the recommended Authorization scheme when picking a token Jellyfin's authorization guidance deprecates X-Emby-Token, X-MediaBrowser-Token, X-Emby-Authorization and api_key; the Authorization MediaBrowser scheme is the recommended form. All spellings stay accepted, but when a client sends several, the recommended one now wins. Adds coverage for the canonical Authorization header, which no test exercised directly. * refactor(jellyfin): rename parseEmbyAuth to parseMediaBrowserAuth The scheme is named MediaBrowser; the old name evoked the deprecated X-Emby-* spellings even though the function also parses the recommended Authorization header. * fix(jellyfin): prefer the Authorization header over X-Emby-Authorization The recommended header now wins when both carry MediaBrowser data — but only when it actually parses as MediaBrowser: a reverse proxy may inject Basic/Digest credentials into Authorization while the client sends the deprecated header, and those must not swallow the client's auth. * fix(jellyfin): require the MediaBrowser scheme when parsing auth headers The parser extracted key="value" pairs from any Authorization value; a foreign scheme whose parameters happened to use our field names would have been misread as client auth. Validate the scheme word instead (case-insensitively, per HTTP), accepting the legacy "Emby" spelling like real Jellyfin. Replaces the any-recognized-field heuristic for detecting proxy-injected Basic/Digest credentials. * Revert "perf(db): keep query planner statistics trustworthy with full ANALYZE" This reverts commit 118563e1053196f0e2e42dd4bee54e39a04ab145. The planner-statistics work now lives in its own PR (#5740); this branch keeps only the covering-index migration. * fix(db): renumber jellyfin covering-index migration after master's latest * fix(db): renumber Jellyfin covering-index migration * docs(jellyfin): document playlist annotations * refactor(log): clarify comments on external services query params * refactor(e2e): move Subsonic e2e suite to server/subsonic/e2e All files in server/e2e were Subsonic tests, so relocate the package under server/subsonic/e2e to sit alongside the API it exercises. The package name stays 'e2e'; only doc/comment references are updated. * refactor(filter): consolidate genre filters and unexport tagIDFilter Route filterByGenre, ByGenreID, and ArtistsByGenreID through a single genreTagFilter helper so the EXISTS json_tree(tags,"$.genre") predicate lives in one place. With server/filter no longer using it, persistence's TagIDFilter is only referenced within the package, so unexport it. --------- Signed-off-by: Deluan <deluan@navidrome.org>
2026-07-14 11:46:37 -04:00
// External services query params. Values can be JWTs (dots, dashes), so match everything up
// to the next query separator or whitespace, not just word chars. A [\w]+ class would stop
feat(jellyfin): non-expiring, audience-scoped tokens revocable by password change (#6013) * feat(auth): add per-user token_epoch column and bump method * feat(auth): add aud and ep claims, omitted when zero * feat(auth): add CreateAPIToken for non-expiring, audience-scoped tokens * feat(auth): add CheckClaims for epoch and audience validation * feat(jellyfin): issue non-expiring, jellyfin-scoped access tokens * fix(subsonic): reject API-scoped and revoked tokens on the jwt path * fix(server): reject API-scoped and revoked tokens on the native API * fix(server): pin the token-subject guard and stop leaking test config Adds a regression spec for the DevAutoLogin/ExtAuth guard in tokenAllowed, switches its comparison to case-insensitive to match the user lookup's own COLLATE NOCASE semantics, and restores Subsonic JWT test config after each spec instead of leaking SessionTimeout. * feat(request): add a token epoch holder for handler-to-middleware signalling * refactor(server): write the refreshed JWT header after the handler runs * feat(auth): revoke all tokens for a user when their password changes * fix(server): restore Unwrap on the JWT refresh writer so SSE write deadlines apply * test(auth): pin that non-session tokens reject API access tokens * test(jellyfin): pin token scoping and epoch revocation end to end Exercises auth.CreateAPIToken and CheckClaims against the real Jellyfin router and SQLite DB: the minted token has no exp and is aud-scoped to jellyfin, and bumping token_epoch through the real UserRepository revokes an already-issued token on the next protected request. * test(nativeapi): pin the token-epoch handoff through a real password-change request Drive a self password change through the real Authenticator/JWTRefresher chain and a real SQLite-backed userRepository, so the epoch handoff between Put and the refreshed-token writer is verified end to end, not as two separately-tested halves. Also fix tokenAllowed to read the enriched ctx it was given instead of r.Context(), so its warning log carries the username. * refactor(server): drop tokenAllowed's now-unused request parameter Finding-2 already moved every use to ctx; r was dead weight. Also note in the new nativeapi test why it must stay the package's only real-DB spec: db.Db() is a process-wide singleton its cleanup closes for good. * refactor(auth): remove duplication in claim decoding and token minting * refactor(auth): group aud with the standard JWT claims * refactor(auth): read aud with the standard-claim accessor pattern * fix(log): redact every api_key spelling the Jellyfin API accepts * fix(auth): bind session tokens to the user id, not just the username * fix(auth): return the token epoch from the same atomic increment * fix(auth): bump the token epoch in the same statement as the password write * chore(auth): trim comments to the why-only budget
2026-08-22 20:36:24 -04:00
// at a JWT's first '.' and leak its payload and signature. Case-insensitive with an
// optional underscore: the API accepts api_key, apikey and ApiKey alike.
"(?i)([^\\w]api_?key=)[^&\\s]+",
// Sensitive request headers, logged as a JSON blob at trace level and never matched by the
// query-param patterns above. Blank the whole value array; values may hold escaped quotes.
`(?i)("(?:Authorization|X-Emby-Token|X-MediaBrowser-Token|X-Nd-Authorization)":\[")[^\]]*("\])`,
},
}
2020-01-08 20:45:07 -05:00
const (
LevelFatal = Level(logrus.FatalLevel)
LevelError = Level(logrus.ErrorLevel)
LevelWarn = Level(logrus.WarnLevel)
LevelInfo = Level(logrus.InfoLevel)
LevelDebug = Level(logrus.DebugLevel)
LevelTrace = Level(logrus.TraceLevel)
2020-01-08 20:45:07 -05:00
)
2020-05-13 16:49:55 -04:00
type contextKey string
const loggerCtxKey = contextKey("logger")
type levelPath struct {
path string
level Level
}
2020-01-08 20:45:07 -05:00
var (
currentLevel atomic.Uint32
hasLogLevelOverrides atomic.Bool
loggerMu sync.RWMutex
defaultLogger = logrus.New()
logSourceLine = false
rootPath string
logLevels []levelPath
2020-01-08 20:45:07 -05:00
)
// SetLevel sets the global log level used by the simple logger.
func SetLevel(l Level) {
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.Lock()
currentLevel.Store(uint32(l))
defaultLogger.Level = logrus.TraceLevel
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.Unlock()
2020-01-08 20:45:07 -05:00
logrus.SetLevel(logrus.Level(l))
}
func SetLevelString(l string) {
feat(plugins): New Plugin System with multi-language PDK support (#4833) * chore(plugins): remove the old plugins system implementation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement new plugin system with using Extism Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add capability detection for plugins based on exported functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): standardize variable names and remove superfluous wrapper functions Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): improve error handling and logging in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement plugin function call helper and refactor MetadataAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): race condition in plugin manager * tests(plugins): change BeforeEach to BeforeAll in MetadataAgent tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): optimize tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): more optimizations Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): ignore goroutine leaks from notify library in tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Wikimedia plugin for Navidrome to fetch artist metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin logging and set User-Agent header Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement scrobbler plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate logs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): clean up manifest struct and improve plugin loading logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add metadata agent and scrobbler schemas for bootstrapping plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(hostgen): add hostgen tool for generating Extism host function wrappers - Implemented hostgen tool to generate wrappers from annotated Go interfaces. - Added command-line flags for input/output directories and package name. - Introduced parsing and code generation logic for host services. - Created test data for various service interfaces and expected generated code. - Added documentation for host services and annotations for code generation. - Implemented SubsonicAPI service with corresponding generated code. * feat(subsonicapi): update Call method to return JSON string response Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement SubsonicAPI host function integration with permissions Signed-off-by: Deluan <deluan@navidrome.org> * fix(generator): error-only methods in response handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate client wrappers for host functions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(generator): remove error handling for response.Error in client templates Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add Scheduler service interface with host function wrappers for scheduling tasks * feat(plugins): add WASI build constraints to client wrapper templates, to avoid lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler service with one-time and recurring scheduling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor(manifest): remove unused ConfigPermission from permissions schema Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add scheduler callback schema and implementation for plugins Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): streamline scheduling logic and remove unused callback tracking Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): add Close method for resource cleanup on plugin unload Signed-off-by: Deluan <deluan@navidrome.org> * docs(scheduler): clarify SchedulerCallback requirement for scheduling functions Signed-off-by: Deluan <deluan@navidrome.org> * fix: update wasm build rule to include all Go files in the directory Signed-off-by: Deluan <deluan@navidrome.org> * feat: rewrite the wikimedia plugin using the XTP CLI Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): replace uuid with id.NewRandom for schedule ID generation Signed-off-by: Deluan <deluan@navidrome.org> * refactor: capabilities registration Signed-off-by: Deluan <deluan@navidrome.org> * test: add scheduler service isolation test for plugin instances Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin manager initialization and encapsulate logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add WebSocket service definitions for plugin communication Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WebSocket service for plugin integration and connection management Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Crypto Ticker example plugin for real-time cryptocurrency price updates via Coinbase WebSocket API Also add the lifecycle capability Signed-off-by: Deluan <deluan@navidrome.org> * fix: use context.Background() in invokeCallback for scheduled tasks Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin.create() to plugin.instance() Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginInstance to plugin for consistency across the codebase Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify schedule cloning in Close method and enhance plugin cleanup error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Artwork service for generating artwork URLs in Navidrome plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: moved public URL builders to avoid import cycles Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cache service for in-memory TTL-based caching in plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Discord Rich Presence example plugin for Navidrome integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor: host function wrappers to use structured request and response types - Updated the host function signatures in `nd_host_artwork.go`, `nd_host_scheduler.go`, `nd_host_subsonicapi.go`, and `nd_host_websocket.go` to accept a single parameter for JSON requests. - Introduced structured request and response types for various cache operations in `nd_host_cache.go`. - Modified cache functions to marshal requests to JSON and unmarshal responses, improving error handling and code clarity. - Removed redundant memory allocation for string parameters in favor of JSON marshaling. - Enhanced error handling in WebSocket and cache operations to return structured error responses. * refactor: error handling in various plugins to convert response.Error to Go errors - Updated error handling in `nd_host_scheduler.go`, `nd_host_websocket.go`, `nd_host_artwork.go`, `nd_host_cache.go`, and `nd_host_subsonicapi.go` to convert string errors from responses into Go errors. - Removed redundant error checks in test data plugins for cleaner code. - Ensured consistent error handling across all plugins to improve reliability and maintainability. * refactor: rename fake plugins to test plugins for clarity in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add help target to Makefile for plugin usage instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cover Art Archive plugin as an example of Python plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Makefile and README to clarify Go plugin usage Signed-off-by: Deluan <deluan@navidrome.org> * feat: include plugin capabilities in loading log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add trace logging for plugin availability and error handling in agents Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Now Playing Logger plugin to showcase calling host functions from Python plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: generate Python client wrappers for various host services Signed-off-by: Deluan <deluan@navidrome.org> * feat: add generated host function wrappers for Scheduler and SubsonicAPI services Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Python plugin documentation and usage instructions for host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Webhook Scrobbler plugin in Rust to send HTTP notifications on scrobble events Signed-off-by: Deluan <deluan@navidrome.org> * feat: enable parallel loading of plugins during startup Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include WebSocket callback schema in plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin watcher with improved logging and debounce duration adjustment Signed-off-by: Deluan <deluan@navidrome.org> * add trace message for plugin recompiles Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin cache purging functionality Signed-off-by: Deluan <deluan@navidrome.org> * test: move purgeCacheBySize unit tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin repository and database support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin management routes and middleware Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): implement plugin synchronization with database for add, update, and remove actions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add PluginList and PluginShow components with plugin management functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): optimize plugin change detection Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins UI): improve PluginList structure Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): enhance PluginShow with author, website, and permissions display Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): refactor to use MUI and RA components Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add error handling for plugin enable/disable actions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): inject PluginManager into native API Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update GetManager to accept DataStore parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add subsonicRouter to Manager and refactor host service registration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): enhance debug logging for plugin actions and recompile logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): break manager.go into smaller, focused files Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline error handling and improve plugin retrieval logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update newWebSocketService to use WebSocketPermission for allowed hosts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): introduce ToggleEnabledSwitch for managing plugin enable/disable state Signed-off-by: Deluan <deluan@navidrome.org> * docs: update READMEs Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add Library service for metadata access and filesystem integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Library Inspector plugin for periodic library inspection and file size logging Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect JSON configuration format for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix(build): update target to wasm32-wasip1 for improved WASI support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement configuration management UI with key-value pairs support Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): adjust grid layout in InfoRow component for improved responsiveness Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename ErrorIndicator to EnabledOrErrorField and enhance error handling logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(i18n): add Portuguese translations for plugin management and notifications Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for .ndp plugin packages and update build process Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README for .ndp plugin packaging and installation instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement KVStore service for persistent key-value storage Signed-off-by: Deluan <deluan@navidrome.org> * docs: enhance README with Extism plugin development resources and recommendations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate event broker into plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update config handling in PluginShow to track last record state Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Rust host function library and example implementation of Discord Rich Presence plugin in Rust Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Rust lib.rs file to expose host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update JSON field names to camelCase for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reduce cyclomatic complexity by refactoring main function Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance Rust code generation with typed struct support and improved type handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Go client library with host function wrappers and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Go client stubs for non-WASM platforms Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update client template file names for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add initial implementation of the Navidrome Plugin Development Kit code generator - Pahse 1 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 (2) Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 4 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 5 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consistent naming/types across PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline plugin function signatures and error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update scrobbler interface to return errors directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * test: make all test plugins use the PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize and sort type definitions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update error handling for methods to return errors directly Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update function signatures to return values directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update request/response types to use private naming conventions Signed-off-by: Deluan <deluan@navidrome.org> * build: mark .wasm files as intermediate for cleanup after building .ndp Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consolidate PDK module path and update Go version to 1.25 Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Rust PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize Rust output structure to follow standard conventions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update Discord Rich Presence and Library Inspector plugins to use nd-pdk for service calls and implement lifecycle management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update macro names for websocket and metadata registration to improve clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename scheduler callback methods for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update export wrappers to use `//go:wasmexport` for WebAssembly compatibility Signed-off-by: Deluan <deluan@navidrome.org> * docs: update plugin registration docs Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): generate host wrappers Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): conditionally run goleak checks based on CI environment Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect changes in plugin import paths Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update plugin instance creation to accept context for cancellation support Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update return types in metadata interfaces to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): enhance type handling for Rust and XTP output in capability generation Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update IsAuthorized method to return boolean instead of response object Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add unit tests for rustOutputType and isPrimitiveRustType functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement XTP JSONSchema validation for generated schemas Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update response types in testMetadataAgent methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * docs: update Go and Rust plugin developer sections for clarity Signed-off-by: Deluan <deluan@navidrome.org> * docs: correct example link for library inspector in README Signed-off-by: Deluan <deluan@navidrome.org> * docs: clarify artwork URL generation capabilities in service descriptions Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include Rust PDK crate information for plugin developers Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle URL parsing errors and use atomic upsert in plugin repository Added proper error handling for url.Parse calls in PublicURL and AbsoluteURL functions. When parsing fails, PublicURL now falls back to AbsoluteURL, and AbsoluteURL logs the error and returns an empty string, preventing malformed URLs from being generated. Replaced the non-atomic UPDATE-then-INSERT pattern in plugin repository Put method with a single atomic INSERT ... ON CONFLICT statement. This eliminates potential race conditions and improves consistency with the upsert pattern already used in host_kvstore.go. * feat: implement mock service instances for non-WASM builds using testify/mock Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Discord RPC struct to encapsulate WebSocket logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for experimental WebAssembly threads Signed-off-by: Deluan <deluan@navidrome.org> * feat: add PDK abstraction layer with mock support for non-WASM builds Signed-off-by: Deluan <deluan@navidrome.org> * feat: add unit tests for Discord plugin and RPC functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: update return types in minimalPlugin and wikimediaPlugin methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix: context cancellation and implement WebSocket callback timeout for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: conditionally include error handling in generated client code templates Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ConfigService for plugin configuration management Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance plugin manager to support metrics recording Signed-off-by: Deluan <deluan@navidrome.org> * refactor: make MockPDK private Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update interface types to use 'any' in plugin repository methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename List method to Keys for clarity in configuration management Signed-off-by: Deluan <deluan@navidrome.org> * test: add ndpgen plugin tests in the pipeline and update Makefile Signed-off-by: Deluan <deluan@navidrome.org> * feat: add users permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * refactor: streamline users integration tests and enhance plugin user management Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove UserID from scrobbler request structure Signed-off-by: Deluan <deluan@navidrome.org> * test: add integration tests for UsersService enable gate behavior Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user permissions for SubsonicAPI and scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: show proper error in the UI when enabling a plugin fails Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * feat: add user permission for processing scrobbles in Discord Rich Presence plugin Signed-off-by: Deluan <deluan@navidrome.org> * fix: implement dynamic loading for buffered scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add GetAdmins method to retrieve admin users from the plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Portuguese translations for user and library permissions Signed-off-by: Deluan <deluan@navidrome.org> * reorder migrations Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove unnecessary bulkActionButtons prop from PluginList component * feat: add manual plugin rescan functionality and corresponding UI action Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user/library and plugin management integration with cleanup on deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace core mock services with test-specific implementations to avoid import cycles * feat: add ID fields to Artist and Song structs and enhance track loading logic by prioritizing ID matches Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin permissions from allowedHosts to requiredHosts for better clarity and consistency * feat: refactor plugin host permissions to use RequiredHosts directly for improved clarity * fix: don't record metrics for plugin calls that aren't implemented at all Signed-off-by: Deluan <deluan@navidrome.org> * fix: enhance connection management with improved error handling and cleanup logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: introduce ArtistRef struct for better artist representation and update track metadata handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: update user configuration handling to use user key prefix for improved clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance ConfigCard input fields with multiline support and vertical resizing Signed-off-by: Deluan <deluan@navidrome.org> * fix: rust plugin compilation error Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement IsOptionPattern method for better return type handling in Rust PDK generation Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2026-01-14 19:22:48 -05:00
level := ParseLogLevel(l)
SetLevel(level)
}
feat(plugins): New Plugin System with multi-language PDK support (#4833) * chore(plugins): remove the old plugins system implementation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement new plugin system with using Extism Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add capability detection for plugins based on exported functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): standardize variable names and remove superfluous wrapper functions Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): improve error handling and logging in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement plugin function call helper and refactor MetadataAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): race condition in plugin manager * tests(plugins): change BeforeEach to BeforeAll in MetadataAgent tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): optimize tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): more optimizations Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): ignore goroutine leaks from notify library in tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Wikimedia plugin for Navidrome to fetch artist metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin logging and set User-Agent header Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement scrobbler plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate logs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): clean up manifest struct and improve plugin loading logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add metadata agent and scrobbler schemas for bootstrapping plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(hostgen): add hostgen tool for generating Extism host function wrappers - Implemented hostgen tool to generate wrappers from annotated Go interfaces. - Added command-line flags for input/output directories and package name. - Introduced parsing and code generation logic for host services. - Created test data for various service interfaces and expected generated code. - Added documentation for host services and annotations for code generation. - Implemented SubsonicAPI service with corresponding generated code. * feat(subsonicapi): update Call method to return JSON string response Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement SubsonicAPI host function integration with permissions Signed-off-by: Deluan <deluan@navidrome.org> * fix(generator): error-only methods in response handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate client wrappers for host functions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(generator): remove error handling for response.Error in client templates Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add Scheduler service interface with host function wrappers for scheduling tasks * feat(plugins): add WASI build constraints to client wrapper templates, to avoid lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler service with one-time and recurring scheduling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor(manifest): remove unused ConfigPermission from permissions schema Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add scheduler callback schema and implementation for plugins Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): streamline scheduling logic and remove unused callback tracking Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): add Close method for resource cleanup on plugin unload Signed-off-by: Deluan <deluan@navidrome.org> * docs(scheduler): clarify SchedulerCallback requirement for scheduling functions Signed-off-by: Deluan <deluan@navidrome.org> * fix: update wasm build rule to include all Go files in the directory Signed-off-by: Deluan <deluan@navidrome.org> * feat: rewrite the wikimedia plugin using the XTP CLI Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): replace uuid with id.NewRandom for schedule ID generation Signed-off-by: Deluan <deluan@navidrome.org> * refactor: capabilities registration Signed-off-by: Deluan <deluan@navidrome.org> * test: add scheduler service isolation test for plugin instances Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin manager initialization and encapsulate logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add WebSocket service definitions for plugin communication Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WebSocket service for plugin integration and connection management Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Crypto Ticker example plugin for real-time cryptocurrency price updates via Coinbase WebSocket API Also add the lifecycle capability Signed-off-by: Deluan <deluan@navidrome.org> * fix: use context.Background() in invokeCallback for scheduled tasks Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin.create() to plugin.instance() Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginInstance to plugin for consistency across the codebase Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify schedule cloning in Close method and enhance plugin cleanup error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Artwork service for generating artwork URLs in Navidrome plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: moved public URL builders to avoid import cycles Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cache service for in-memory TTL-based caching in plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Discord Rich Presence example plugin for Navidrome integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor: host function wrappers to use structured request and response types - Updated the host function signatures in `nd_host_artwork.go`, `nd_host_scheduler.go`, `nd_host_subsonicapi.go`, and `nd_host_websocket.go` to accept a single parameter for JSON requests. - Introduced structured request and response types for various cache operations in `nd_host_cache.go`. - Modified cache functions to marshal requests to JSON and unmarshal responses, improving error handling and code clarity. - Removed redundant memory allocation for string parameters in favor of JSON marshaling. - Enhanced error handling in WebSocket and cache operations to return structured error responses. * refactor: error handling in various plugins to convert response.Error to Go errors - Updated error handling in `nd_host_scheduler.go`, `nd_host_websocket.go`, `nd_host_artwork.go`, `nd_host_cache.go`, and `nd_host_subsonicapi.go` to convert string errors from responses into Go errors. - Removed redundant error checks in test data plugins for cleaner code. - Ensured consistent error handling across all plugins to improve reliability and maintainability. * refactor: rename fake plugins to test plugins for clarity in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add help target to Makefile for plugin usage instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cover Art Archive plugin as an example of Python plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Makefile and README to clarify Go plugin usage Signed-off-by: Deluan <deluan@navidrome.org> * feat: include plugin capabilities in loading log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add trace logging for plugin availability and error handling in agents Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Now Playing Logger plugin to showcase calling host functions from Python plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: generate Python client wrappers for various host services Signed-off-by: Deluan <deluan@navidrome.org> * feat: add generated host function wrappers for Scheduler and SubsonicAPI services Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Python plugin documentation and usage instructions for host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Webhook Scrobbler plugin in Rust to send HTTP notifications on scrobble events Signed-off-by: Deluan <deluan@navidrome.org> * feat: enable parallel loading of plugins during startup Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include WebSocket callback schema in plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin watcher with improved logging and debounce duration adjustment Signed-off-by: Deluan <deluan@navidrome.org> * add trace message for plugin recompiles Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin cache purging functionality Signed-off-by: Deluan <deluan@navidrome.org> * test: move purgeCacheBySize unit tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin repository and database support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin management routes and middleware Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): implement plugin synchronization with database for add, update, and remove actions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add PluginList and PluginShow components with plugin management functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): optimize plugin change detection Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins UI): improve PluginList structure Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): enhance PluginShow with author, website, and permissions display Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): refactor to use MUI and RA components Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add error handling for plugin enable/disable actions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): inject PluginManager into native API Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update GetManager to accept DataStore parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add subsonicRouter to Manager and refactor host service registration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): enhance debug logging for plugin actions and recompile logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): break manager.go into smaller, focused files Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline error handling and improve plugin retrieval logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update newWebSocketService to use WebSocketPermission for allowed hosts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): introduce ToggleEnabledSwitch for managing plugin enable/disable state Signed-off-by: Deluan <deluan@navidrome.org> * docs: update READMEs Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add Library service for metadata access and filesystem integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Library Inspector plugin for periodic library inspection and file size logging Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect JSON configuration format for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix(build): update target to wasm32-wasip1 for improved WASI support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement configuration management UI with key-value pairs support Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): adjust grid layout in InfoRow component for improved responsiveness Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename ErrorIndicator to EnabledOrErrorField and enhance error handling logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(i18n): add Portuguese translations for plugin management and notifications Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for .ndp plugin packages and update build process Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README for .ndp plugin packaging and installation instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement KVStore service for persistent key-value storage Signed-off-by: Deluan <deluan@navidrome.org> * docs: enhance README with Extism plugin development resources and recommendations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate event broker into plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update config handling in PluginShow to track last record state Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Rust host function library and example implementation of Discord Rich Presence plugin in Rust Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Rust lib.rs file to expose host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update JSON field names to camelCase for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reduce cyclomatic complexity by refactoring main function Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance Rust code generation with typed struct support and improved type handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Go client library with host function wrappers and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Go client stubs for non-WASM platforms Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update client template file names for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add initial implementation of the Navidrome Plugin Development Kit code generator - Pahse 1 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 (2) Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 4 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 5 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consistent naming/types across PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline plugin function signatures and error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update scrobbler interface to return errors directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * test: make all test plugins use the PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize and sort type definitions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update error handling for methods to return errors directly Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update function signatures to return values directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update request/response types to use private naming conventions Signed-off-by: Deluan <deluan@navidrome.org> * build: mark .wasm files as intermediate for cleanup after building .ndp Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consolidate PDK module path and update Go version to 1.25 Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Rust PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize Rust output structure to follow standard conventions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update Discord Rich Presence and Library Inspector plugins to use nd-pdk for service calls and implement lifecycle management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update macro names for websocket and metadata registration to improve clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename scheduler callback methods for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update export wrappers to use `//go:wasmexport` for WebAssembly compatibility Signed-off-by: Deluan <deluan@navidrome.org> * docs: update plugin registration docs Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): generate host wrappers Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): conditionally run goleak checks based on CI environment Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect changes in plugin import paths Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update plugin instance creation to accept context for cancellation support Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update return types in metadata interfaces to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): enhance type handling for Rust and XTP output in capability generation Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update IsAuthorized method to return boolean instead of response object Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add unit tests for rustOutputType and isPrimitiveRustType functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement XTP JSONSchema validation for generated schemas Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update response types in testMetadataAgent methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * docs: update Go and Rust plugin developer sections for clarity Signed-off-by: Deluan <deluan@navidrome.org> * docs: correct example link for library inspector in README Signed-off-by: Deluan <deluan@navidrome.org> * docs: clarify artwork URL generation capabilities in service descriptions Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include Rust PDK crate information for plugin developers Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle URL parsing errors and use atomic upsert in plugin repository Added proper error handling for url.Parse calls in PublicURL and AbsoluteURL functions. When parsing fails, PublicURL now falls back to AbsoluteURL, and AbsoluteURL logs the error and returns an empty string, preventing malformed URLs from being generated. Replaced the non-atomic UPDATE-then-INSERT pattern in plugin repository Put method with a single atomic INSERT ... ON CONFLICT statement. This eliminates potential race conditions and improves consistency with the upsert pattern already used in host_kvstore.go. * feat: implement mock service instances for non-WASM builds using testify/mock Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Discord RPC struct to encapsulate WebSocket logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for experimental WebAssembly threads Signed-off-by: Deluan <deluan@navidrome.org> * feat: add PDK abstraction layer with mock support for non-WASM builds Signed-off-by: Deluan <deluan@navidrome.org> * feat: add unit tests for Discord plugin and RPC functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: update return types in minimalPlugin and wikimediaPlugin methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix: context cancellation and implement WebSocket callback timeout for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: conditionally include error handling in generated client code templates Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ConfigService for plugin configuration management Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance plugin manager to support metrics recording Signed-off-by: Deluan <deluan@navidrome.org> * refactor: make MockPDK private Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update interface types to use 'any' in plugin repository methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename List method to Keys for clarity in configuration management Signed-off-by: Deluan <deluan@navidrome.org> * test: add ndpgen plugin tests in the pipeline and update Makefile Signed-off-by: Deluan <deluan@navidrome.org> * feat: add users permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * refactor: streamline users integration tests and enhance plugin user management Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove UserID from scrobbler request structure Signed-off-by: Deluan <deluan@navidrome.org> * test: add integration tests for UsersService enable gate behavior Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user permissions for SubsonicAPI and scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: show proper error in the UI when enabling a plugin fails Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * feat: add user permission for processing scrobbles in Discord Rich Presence plugin Signed-off-by: Deluan <deluan@navidrome.org> * fix: implement dynamic loading for buffered scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add GetAdmins method to retrieve admin users from the plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Portuguese translations for user and library permissions Signed-off-by: Deluan <deluan@navidrome.org> * reorder migrations Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove unnecessary bulkActionButtons prop from PluginList component * feat: add manual plugin rescan functionality and corresponding UI action Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user/library and plugin management integration with cleanup on deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace core mock services with test-specific implementations to avoid import cycles * feat: add ID fields to Artist and Song structs and enhance track loading logic by prioritizing ID matches Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin permissions from allowedHosts to requiredHosts for better clarity and consistency * feat: refactor plugin host permissions to use RequiredHosts directly for improved clarity * fix: don't record metrics for plugin calls that aren't implemented at all Signed-off-by: Deluan <deluan@navidrome.org> * fix: enhance connection management with improved error handling and cleanup logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: introduce ArtistRef struct for better artist representation and update track metadata handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: update user configuration handling to use user key prefix for improved clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance ConfigCard input fields with multiline support and vertical resizing Signed-off-by: Deluan <deluan@navidrome.org> * fix: rust plugin compilation error Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement IsOptionPattern method for better return type handling in Rust PDK generation Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2026-01-14 19:22:48 -05:00
func ParseLogLevel(l string) Level {
envLevel := strings.ToLower(l)
var level Level
switch envLevel {
case "fatal":
level = LevelFatal
case "error":
level = LevelError
case "warn":
level = LevelWarn
case "debug":
level = LevelDebug
case "trace":
level = LevelTrace
default:
level = LevelInfo
}
return level
}
// SetLogLevels sets the log levels for specific paths in the codebase.
func SetLogLevels(levels map[string]string) {
feat(scanner): implement selective folder scanning and file system watcher improvements (#4674) * feat: Add selective folder scanning capability Implement targeted scanning of specific library/folder pairs without full recursion. This enables efficient rescanning of individual folders when changes are detected, significantly reducing scan time for large libraries. Key changes: - Add ScanTarget struct and ScanFolders API to Scanner interface - Implement CLI flag --targets for specifying libraryID:folderPath pairs - Add FolderRepository.GetByPaths() for batch folder info retrieval - Create loadSpecificFolders() for non-recursive directory loading - Scope GC operations to affected libraries only (with TODO for full impl) - Add comprehensive tests for selective scanning behavior The selective scan: - Only processes specified folders (no subdirectory recursion) - Maintains library isolation - Runs full maintenance pipeline scoped to affected libraries - Supports both full and quick scan modes Examples: navidrome scan --targets "1:Music/Rock,1:Music/Jazz" navidrome scan --full --targets "2:Classical" * feat(folder): replace GetByPaths with GetFolderUpdateInfo for improved folder updates retrieval Signed-off-by: Deluan <deluan@navidrome.org> * test: update parseTargets test to handle folder names with spaces Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): remove unused LibraryPath struct and update GC logging message Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): enhance external scanner to support target-specific scanning Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify scanner methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement folder scanning notifications with deduplication Signed-off-by: Deluan <deluan@navidrome.org> * refactor(watcher): add resolveFolderPath function for testability Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement path ignoring based on .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): implement IgnoreChecker for managing .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(ignore_checker): rename scanner to lineScanner for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance ScanTarget struct with String method for better target representation Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): validate library ID to prevent negative values Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify GC method by removing library ID parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): update folder scanning to include all descendants of specified folders Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): allow selective scan in the /startScan endpoint Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update CallScan to handle specific library/folder pairs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline scanning logic by removing scanAll method Signed-off-by: Deluan <deluan@navidrome.org> * test: enhance mockScanner for thread safety and improve test reliability Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move scanner.ScanTarget to model.ScanTarget Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move scanner types to model,implement MockScanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update scanner interface and implementations to use model.Scanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder_repository): normalize target path handling by using filepath.Clean Signed-off-by: Deluan <deluan@navidrome.org> * test(folder_repository): add comprehensive tests for folder retrieval and child exclusion Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify selective scan logic using slice.Filter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline phase folder and album creation by removing unnecessary library parameter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move initialization logic from phase_1 to the scanner itself Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): rename selective scan test file to scanner_selective_test.go Signed-off-by: Deluan <deluan@navidrome.org> * feat(configuration): add DevSelectiveWatcher configuration option Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): enhance .ndignore handling for folder deletions and file changes Signed-off-by: Deluan <deluan@navidrome.org> * docs(scanner): comments Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance walkDirTree to support target folder scanning Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner, watcher): handle errors when pushing ignore patterns for folders Signed-off-by: Deluan <deluan@navidrome.org> * Update scanner/phase_1_folders.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor(scanner): replace parseTargets function with direct call to scanner.ParseTargets Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): add tests for ScanBegin and ScanEnd functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(library): update PRAGMA optimize to check table sizes without ANALYZE Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): refactor tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add selective scan options and update translations Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add quick and full scan options for individual libraries Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add Scan buttonsto the LibraryList Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): update scanning parameters from 'path' to 'target' for selective scans. * refactor(scan): move ParseTargets function to model package * test(scan): suppress unused return value from SetUserLibraries in tests * feat(gc): enhance garbage collection to support selective library purging Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): prevent race condition when scanning deleted folders When the watcher detects changes in a folder that gets deleted before the scanner runs (due to the 10-second delay), the scanner was prematurely removing these folders from the tracking map, preventing them from being marked as missing. The issue occurred because `newFolderEntry` was calling `popLastUpdate` before verifying the folder actually exists on the filesystem. Changes: - Move fs.Stat check before newFolderEntry creation in loadDir to ensure deleted folders remain in lastUpdates for finalize() to handle - Add early existence check in walkDirTree to skip non-existent target folders with a warning log - Add unit test verifying non-existent folders aren't removed from lastUpdates prematurely - Add integration test for deleted folder scenario with ScanFolders Fixes the issue where deleting entire folders (e.g., /music/AC_DC) wouldn't mark tracks as missing when using selective folder scanning. * refactor(scan): streamline folder entry creation and update handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): add '@Recycle' (QNAP) to ignored directories list Signed-off-by: Deluan <deluan@navidrome.org> * fix(log): improve thread safety in logging level management * test(scan): move unit tests for ParseTargets function Signed-off-by: Deluan <deluan@navidrome.org> * review Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: deluan <deluan.quintao@mechanical-orchard.com>
2025-11-14 22:15:43 -05:00
loggerMu.Lock()
defer loggerMu.Unlock()
2024-06-16 22:25:25 -04:00
logLevels = nil
for k, v := range levels {
feat(plugins): New Plugin System with multi-language PDK support (#4833) * chore(plugins): remove the old plugins system implementation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement new plugin system with using Extism Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add capability detection for plugins based on exported functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add auto-reload functionality for plugins with file watcher support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): standardize variable names and remove superfluous wrapper functions Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): improve error handling and logging in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement plugin function call helper and refactor MetadataAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): race condition in plugin manager * tests(plugins): change BeforeEach to BeforeAll in MetadataAgent tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): optimize tests Signed-off-by: Deluan <deluan@navidrome.org> * tests(plugins): more optimizations Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): ignore goroutine leaks from notify library in tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Wikimedia plugin for Navidrome to fetch artist metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin logging and set User-Agent header Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement scrobbler plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate logs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): clean up manifest struct and improve plugin loading logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add metadata agent and scrobbler schemas for bootstrapping plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(hostgen): add hostgen tool for generating Extism host function wrappers - Implemented hostgen tool to generate wrappers from annotated Go interfaces. - Added command-line flags for input/output directories and package name. - Introduced parsing and code generation logic for host services. - Created test data for various service interfaces and expected generated code. - Added documentation for host services and annotations for code generation. - Implemented SubsonicAPI service with corresponding generated code. * feat(subsonicapi): update Call method to return JSON string response Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement SubsonicAPI host function integration with permissions Signed-off-by: Deluan <deluan@navidrome.org> * fix(generator): error-only methods in response handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate client wrappers for host functions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(generator): remove error handling for response.Error in client templates Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add Scheduler service interface with host function wrappers for scheduling tasks * feat(plugins): add WASI build constraints to client wrapper templates, to avoid lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler service with one-time and recurring scheduling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor(manifest): remove unused ConfigPermission from permissions schema Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): add scheduler callback schema and implementation for plugins Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): streamline scheduling logic and remove unused callback tracking Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): add Close method for resource cleanup on plugin unload Signed-off-by: Deluan <deluan@navidrome.org> * docs(scheduler): clarify SchedulerCallback requirement for scheduling functions Signed-off-by: Deluan <deluan@navidrome.org> * fix: update wasm build rule to include all Go files in the directory Signed-off-by: Deluan <deluan@navidrome.org> * feat: rewrite the wikimedia plugin using the XTP CLI Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): replace uuid with id.NewRandom for schedule ID generation Signed-off-by: Deluan <deluan@navidrome.org> * refactor: capabilities registration Signed-off-by: Deluan <deluan@navidrome.org> * test: add scheduler service isolation test for plugin instances Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin manager initialization and encapsulate logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add WebSocket service definitions for plugin communication Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WebSocket service for plugin integration and connection management Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Crypto Ticker example plugin for real-time cryptocurrency price updates via Coinbase WebSocket API Also add the lifecycle capability Signed-off-by: Deluan <deluan@navidrome.org> * fix: use context.Background() in invokeCallback for scheduled tasks Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin.create() to plugin.instance() Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginInstance to plugin for consistency across the codebase Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify schedule cloning in Close method and enhance plugin cleanup error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Artwork service for generating artwork URLs in Navidrome plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor: moved public URL builders to avoid import cycles Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cache service for in-memory TTL-based caching in plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Discord Rich Presence example plugin for Navidrome integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor: host function wrappers to use structured request and response types - Updated the host function signatures in `nd_host_artwork.go`, `nd_host_scheduler.go`, `nd_host_subsonicapi.go`, and `nd_host_websocket.go` to accept a single parameter for JSON requests. - Introduced structured request and response types for various cache operations in `nd_host_cache.go`. - Modified cache functions to marshal requests to JSON and unmarshal responses, improving error handling and code clarity. - Removed redundant memory allocation for string parameters in favor of JSON marshaling. - Enhanced error handling in WebSocket and cache operations to return structured error responses. * refactor: error handling in various plugins to convert response.Error to Go errors - Updated error handling in `nd_host_scheduler.go`, `nd_host_websocket.go`, `nd_host_artwork.go`, `nd_host_cache.go`, and `nd_host_subsonicapi.go` to convert string errors from responses into Go errors. - Removed redundant error checks in test data plugins for cleaner code. - Ensured consistent error handling across all plugins to improve reliability and maintainability. * refactor: rename fake plugins to test plugins for clarity in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * feat: add help target to Makefile for plugin usage instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Cover Art Archive plugin as an example of Python plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Makefile and README to clarify Go plugin usage Signed-off-by: Deluan <deluan@navidrome.org> * feat: include plugin capabilities in loading log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add trace logging for plugin availability and error handling in agents Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Now Playing Logger plugin to showcase calling host functions from Python plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: generate Python client wrappers for various host services Signed-off-by: Deluan <deluan@navidrome.org> * feat: add generated host function wrappers for Scheduler and SubsonicAPI services Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Python plugin documentation and usage instructions for host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Webhook Scrobbler plugin in Rust to send HTTP notifications on scrobble events Signed-off-by: Deluan <deluan@navidrome.org> * feat: enable parallel loading of plugins during startup Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include WebSocket callback schema in plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin watcher with improved logging and debounce duration adjustment Signed-off-by: Deluan <deluan@navidrome.org> * add trace message for plugin recompiles Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin cache purging functionality Signed-off-by: Deluan <deluan@navidrome.org> * test: move purgeCacheBySize unit tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin repository and database support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add plugin management routes and middleware Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): implement plugin synchronization with database for add, update, and remove actions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add PluginList and PluginShow components with plugin management functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): optimize plugin change detection Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins UI): improve PluginList structure Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): enhance PluginShow with author, website, and permissions display Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): refactor to use MUI and RA components Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins UI): add error handling for plugin enable/disable actions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): inject PluginManager into native API Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update GetManager to accept DataStore parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add subsonicRouter to Manager and refactor host service registration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): enhance debug logging for plugin actions and recompile logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): break manager.go into smaller, focused files Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline error handling and improve plugin retrieval logic Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update newWebSocketService to use WebSocketPermission for allowed hosts Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): introduce ToggleEnabledSwitch for managing plugin enable/disable state Signed-off-by: Deluan <deluan@navidrome.org> * docs: update READMEs Signed-off-by: Deluan <deluan@navidrome.org> * feat(library): add Library service for metadata access and filesystem integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Library Inspector plugin for periodic library inspection and file size logging Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect JSON configuration format for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix(build): update target to wasm32-wasip1 for improved WASI support Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement configuration management UI with key-value pairs support Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): adjust grid layout in InfoRow component for improved responsiveness Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename ErrorIndicator to EnabledOrErrorField and enhance error handling logic Signed-off-by: Deluan <deluan@navidrome.org> * feat(i18n): add Portuguese translations for plugin management and notifications Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for .ndp plugin packages and update build process Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README for .ndp plugin packaging and installation instructions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement KVStore service for persistent key-value storage Signed-off-by: Deluan <deluan@navidrome.org> * docs: enhance README with Extism plugin development resources and recommendations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): integrate event broker into plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update config handling in PluginShow to track last record state Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Rust host function library and example implementation of Discord Rich Presence plugin in Rust Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Rust lib.rs file to expose host function wrappers Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update JSON field names to camelCase for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reduce cyclomatic complexity by refactoring main function Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance Rust code generation with typed struct support and improved type handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add Go client library with host function wrappers and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): generate Go client stubs for non-WASM platforms Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): update client template file names for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add initial implementation of the Navidrome Plugin Development Kit code generator - Pahse 1 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 2 (2) Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 4 Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implementation of the Navidrome Plugin Development Kit with generated client wrappers and service interfaces - Phase 5 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consistent naming/types across PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): streamline plugin function signatures and error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update scrobbler interface to return errors directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * test: make all test plugins use the PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize and sort type definitions for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update error handling for methods to return errors directly Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update function signatures to return values directly instead of response structs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update request/response types to use private naming conventions Signed-off-by: Deluan <deluan@navidrome.org> * build: mark .wasm files as intermediate for cleanup after building .ndp Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): consolidate PDK module path and update Go version to 1.25 Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement Rust PDK Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize Rust output structure to follow standard conventions Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update Discord Rich Presence and Library Inspector plugins to use nd-pdk for service calls and implement lifecycle management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update macro names for websocket and metadata registration to improve clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename scheduler callback methods for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update export wrappers to use `//go:wasmexport` for WebAssembly compatibility Signed-off-by: Deluan <deluan@navidrome.org> * docs: update plugin registration docs Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): generate host wrappers Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): conditionally run goleak checks based on CI environment Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to reflect changes in plugin import paths Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update plugin instance creation to accept context for cancellation support Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update return types in metadata interfaces to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): enhance type handling for Rust and XTP output in capability generation Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update IsAuthorized method to return boolean instead of response object Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add unit tests for rustOutputType and isPrimitiveRustType functions Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement XTP JSONSchema validation for generated schemas Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): update response types in testMetadataAgent methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * docs: update Go and Rust plugin developer sections for clarity Signed-off-by: Deluan <deluan@navidrome.org> * docs: correct example link for library inspector in README Signed-off-by: Deluan <deluan@navidrome.org> * docs: clarify artwork URL generation capabilities in service descriptions Signed-off-by: Deluan <deluan@navidrome.org> * docs: update README to include Rust PDK crate information for plugin developers Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle URL parsing errors and use atomic upsert in plugin repository Added proper error handling for url.Parse calls in PublicURL and AbsoluteURL functions. When parsing fails, PublicURL now falls back to AbsoluteURL, and AbsoluteURL logs the error and returns an empty string, preventing malformed URLs from being generated. Replaced the non-atomic UPDATE-then-INSERT pattern in plugin repository Put method with a single atomic INSERT ... ON CONFLICT statement. This eliminates potential race conditions and improves consistency with the upsert pattern already used in host_kvstore.go. * feat: implement mock service instances for non-WASM builds using testify/mock Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Discord RPC struct to encapsulate WebSocket logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for experimental WebAssembly threads Signed-off-by: Deluan <deluan@navidrome.org> * feat: add PDK abstraction layer with mock support for non-WASM builds Signed-off-by: Deluan <deluan@navidrome.org> * feat: add unit tests for Discord plugin and RPC functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: update return types in minimalPlugin and wikimediaPlugin methods to use pointers Signed-off-by: Deluan <deluan@navidrome.org> * fix: context cancellation and implement WebSocket callback timeout for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: conditionally include error handling in generated client code templates Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ConfigService for plugin configuration management Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance plugin manager to support metrics recording Signed-off-by: Deluan <deluan@navidrome.org> * refactor: make MockPDK private Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update interface types to use 'any' in plugin repository methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename List method to Keys for clarity in configuration management Signed-off-by: Deluan <deluan@navidrome.org> * test: add ndpgen plugin tests in the pipeline and update Makefile Signed-off-by: Deluan <deluan@navidrome.org> * feat: add users permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * refactor: streamline users integration tests and enhance plugin user management Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove UserID from scrobbler request structure Signed-off-by: Deluan <deluan@navidrome.org> * test: add integration tests for UsersService enable gate behavior Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user permissions for SubsonicAPI and scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: show proper error in the UI when enabling a plugin fails Signed-off-by: Deluan <deluan@navidrome.org> * feat: add library permission management to plugin system Signed-off-by: Deluan <deluan@navidrome.org> * feat: add user permission for processing scrobbles in Discord Rich Presence plugin Signed-off-by: Deluan <deluan@navidrome.org> * fix: implement dynamic loading for buffered scrobbler plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat: add GetAdmins method to retrieve admin users from the plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat: update Portuguese translations for user and library permissions Signed-off-by: Deluan <deluan@navidrome.org> * reorder migrations Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove unnecessary bulkActionButtons prop from PluginList component * feat: add manual plugin rescan functionality and corresponding UI action Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement user/library and plugin management integration with cleanup on deletion Signed-off-by: Deluan <deluan@navidrome.org> * feat: replace core mock services with test-specific implementations to avoid import cycles * feat: add ID fields to Artist and Song structs and enhance track loading logic by prioritizing ID matches Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin permissions from allowedHosts to requiredHosts for better clarity and consistency * feat: refactor plugin host permissions to use RequiredHosts directly for improved clarity * fix: don't record metrics for plugin calls that aren't implemented at all Signed-off-by: Deluan <deluan@navidrome.org> * fix: enhance connection management with improved error handling and cleanup logic Signed-off-by: Deluan <deluan@navidrome.org> * feat: introduce ArtistRef struct for better artist representation and update track metadata handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: update user configuration handling to use user key prefix for improved clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance ConfigCard input fields with multiline support and vertical resizing Signed-off-by: Deluan <deluan@navidrome.org> * fix: rust plugin compilation error Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement IsOptionPattern method for better return type handling in Rust PDK generation Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2026-01-14 19:22:48 -05:00
logLevels = append(logLevels, levelPath{path: k, level: ParseLogLevel(v)})
}
slices.SortFunc(logLevels, func(a, b levelPath) int {
return cmp.Compare(b.path, a.path)
})
hasLogLevelOverrides.Store(len(logLevels) != 0)
}
func SetLogSourceLine(enabled bool) {
logSourceLine = enabled
}
func SetRedacting(enabled bool) {
if enabled {
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.Lock()
defer loggerMu.Unlock()
defaultLogger.AddHook(redacted)
}
}
feat(build): MSI installer improvements (#3376) * feat(build): add a make target to build a msi installer locally * Testing wrapping the executable in cmd * build(ci): build msis in parallel * feat(server): add LogFile config option * Revert "Testing wrapping the executable in cmd" This reverts commit be29592254cb903fd4904f2f50d4d1a860795d33. * Adding --log-file for service executable * feat(ini): wip * feat(ini): parse nested ini section * fix(conf): fix fatal error messages * Now navidrome supports INI, we can use the built-in msi ini system and not require the VBScript to convert it into toml * File needs to be called .ini to be parsed as an INI and correct filename needs to be passed to the service * fix(msi): build msi locally * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): Makefile * fix(msi): more clean up * fix(log): convert LF to CRLF on Windows * fix(msi): config filename should be case-insensitive * fix(msi): make it a little more idiomatic * Including the latest windows release of ffmpeg into the msi as built by https://www.gyan.dev/ffmpeg/builds/ (linked to on the official ffmpeg source) * This should version independent * Need bash expansion for the * to work * This will run twice, once for x86 and once for x64, I'll make it cache the executable for now as it'll be quicker * Silencing wget * Add ffmpeg path to the config so Navidrome knows where to find it * refactor: download ffmpeg from our repository * When going back from the "Are you ready to install?" it should go back to the Settings dialogue that you just came from * fix: comments --------- Co-authored-by: Deluan <deluan@navidrome.org>
2024-10-22 23:32:56 +00:00
func SetOutput(w io.Writer) {
if runtime.GOOS == "windows" {
w = CRLFWriter(w)
}
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.Lock()
defer loggerMu.Unlock()
feat(build): MSI installer improvements (#3376) * feat(build): add a make target to build a msi installer locally * Testing wrapping the executable in cmd * build(ci): build msis in parallel * feat(server): add LogFile config option * Revert "Testing wrapping the executable in cmd" This reverts commit be29592254cb903fd4904f2f50d4d1a860795d33. * Adding --log-file for service executable * feat(ini): wip * feat(ini): parse nested ini section * fix(conf): fix fatal error messages * Now navidrome supports INI, we can use the built-in msi ini system and not require the VBScript to convert it into toml * File needs to be called .ini to be parsed as an INI and correct filename needs to be passed to the service * fix(msi): build msi locally * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): pipeline * fix(msi): Makefile * fix(msi): more clean up * fix(log): convert LF to CRLF on Windows * fix(msi): config filename should be case-insensitive * fix(msi): make it a little more idiomatic * Including the latest windows release of ffmpeg into the msi as built by https://www.gyan.dev/ffmpeg/builds/ (linked to on the official ffmpeg source) * This should version independent * Need bash expansion for the * to work * This will run twice, once for x86 and once for x64, I'll make it cache the executable for now as it'll be quicker * Silencing wget * Add ffmpeg path to the config so Navidrome knows where to find it * refactor: download ffmpeg from our repository * When going back from the "Are you ready to install?" it should go back to the Settings dialogue that you just came from * fix: comments --------- Co-authored-by: Deluan <deluan@navidrome.org>
2024-10-22 23:32:56 +00:00
defaultLogger.SetOutput(w)
}
// EnableJournalFormat wraps the current logger formatter with syslog
// priority prefixes for systemd-journald. Only call this when output
// goes to stderr and JOURNAL_STREAM is set.
func EnableJournalFormat() {
loggerMu.Lock()
defer loggerMu.Unlock()
defaultLogger.Formatter = &journalFormatter{inner: defaultLogger.Formatter}
}
// Redact applies redaction to a single string
func Redact(msg string) string {
r, _ := redacted.redact(msg)
return r
}
2026-02-08 08:57:30 -06:00
func NewContext(ctx context.Context, keyValuePairs ...any) context.Context {
2020-01-08 20:45:07 -05:00
if ctx == nil {
ctx = context.Background()
}
logger, ok := ctx.Value(loggerCtxKey).(*logrus.Entry)
if !ok {
logger = createNewLogger()
}
logger = addFields(logger, keyValuePairs)
2020-05-13 16:49:55 -04:00
ctx = context.WithValue(ctx, loggerCtxKey, logger)
2020-01-08 20:45:07 -05:00
return ctx
}
fix(scanner): stop logging expected lyrics sniff misses as warnings (#5702) * fix(scanner): stop logging expected lyrics sniff misses as warnings During a scan, embedded lyrics are parsed with an empty suffix, which puts ParseLyrics into content-sniffing mode: it tries the TTML, SRT and Lyricsfile YAML parsers in turn before falling back to plain text. Every plain-text or LRC lyric therefore fails the structured probes on its way to the fallback, and each failure was logged at warning level with no indication of which file triggered it, flooding the scan log with benign "Error parsing lyrics, falling back to plain text" messages. A probe rejecting content it does not own during sniffing is expected control flow, so it is now logged at trace instead. A parse failure under an explicitly requested suffix (e.g. a malformed .yaml/.srt/.ttml sidecar) still warns, since the user declared that format. ParseLyrics gains ctx and path parameters so any warning names the offending file and carries request context where available; all call sites are updated accordingly. Also fixes a test-isolation bug in the new logging spec: the BeforeEach swapped the process-global default logger via SetDefaultLogger but only restored the log level on cleanup, leaking the null logger and its hook into later specs in the shared model suite. * test: use spec-scoped contexts instead of context.Background in lyrics tests Replace context.Background() with GinkgoT().Context() (and b.Context() in the parse benchmarks) across the lyrics-related tests, so contexts are cancelled when each spec ends. The embeddedLyrics fixture in core/lyrics is now a hand-written literal like its sibling fixtures, removing the construction-time ParseLyrics call that could not use a spec-scoped context. * refactor(model): attach lyrics parse log attribution via context Narrow ParseLyrics back to (ctx, suffix, lang, contents), dropping the path parameter added by the previous commit. Attribution now uses the codebase's existing idiom: callers that know the source attach it with log.NewContext (e.g. "file" for the media file or sidecar), and the plugin adapter tags both the plugin name and the track, fixing probe-miss logs that misattributed plugin-returned content to the file's own tags. This removes three adjacent string parameters that were easy to swap silently, and the "" placeholder most call sites had to pass. Also hardens the logging spec from the previous commit: the null test logger is now swapped in before raising the level (SetLevel forces the current default logger to trace, so the old order left the null logger at info and trace entries never reached the hook), the sniff test now asserts probe misses are observable at trace with file attribution instead of only asserting the absence of warnings, and cleanup restores the actual previous logger — via a new return value on log.SetDefaultLogger — instead of a bare logrus.New() that would discard hooks configured on the process-wide logger. * refactor(lyrics): hoist attributed log contexts out of loops Address review feedback on #5702: build the log-attributed context once per operation instead of per iteration, and reuse it on the surrounding log calls so the error/trace lines around ParseLyrics carry the same attribution fields. In fromExternalFile the sidecar path now rides the context for all log lines in the function, replacing the repeated explicit "path" field. * style(model): pass lyrics parse errors as final log arguments Per the project logging convention, errors go as the last argument (the log package normalizes them via its error case) instead of a keyed "error" pair, which stores the raw error value and bypasses that handling. Flagged by review on #5702; the keyed form was inherited from the original warning line.
2026-07-02 09:46:57 -04:00
// SetDefaultLogger swaps the process-wide logger and returns the previous one,
// so tests can restore the original (with its hooks and formatter) on cleanup.
func SetDefaultLogger(l *logrus.Logger) *logrus.Logger {
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.Lock()
defer loggerMu.Unlock()
fix(scanner): stop logging expected lyrics sniff misses as warnings (#5702) * fix(scanner): stop logging expected lyrics sniff misses as warnings During a scan, embedded lyrics are parsed with an empty suffix, which puts ParseLyrics into content-sniffing mode: it tries the TTML, SRT and Lyricsfile YAML parsers in turn before falling back to plain text. Every plain-text or LRC lyric therefore fails the structured probes on its way to the fallback, and each failure was logged at warning level with no indication of which file triggered it, flooding the scan log with benign "Error parsing lyrics, falling back to plain text" messages. A probe rejecting content it does not own during sniffing is expected control flow, so it is now logged at trace instead. A parse failure under an explicitly requested suffix (e.g. a malformed .yaml/.srt/.ttml sidecar) still warns, since the user declared that format. ParseLyrics gains ctx and path parameters so any warning names the offending file and carries request context where available; all call sites are updated accordingly. Also fixes a test-isolation bug in the new logging spec: the BeforeEach swapped the process-global default logger via SetDefaultLogger but only restored the log level on cleanup, leaking the null logger and its hook into later specs in the shared model suite. * test: use spec-scoped contexts instead of context.Background in lyrics tests Replace context.Background() with GinkgoT().Context() (and b.Context() in the parse benchmarks) across the lyrics-related tests, so contexts are cancelled when each spec ends. The embeddedLyrics fixture in core/lyrics is now a hand-written literal like its sibling fixtures, removing the construction-time ParseLyrics call that could not use a spec-scoped context. * refactor(model): attach lyrics parse log attribution via context Narrow ParseLyrics back to (ctx, suffix, lang, contents), dropping the path parameter added by the previous commit. Attribution now uses the codebase's existing idiom: callers that know the source attach it with log.NewContext (e.g. "file" for the media file or sidecar), and the plugin adapter tags both the plugin name and the track, fixing probe-miss logs that misattributed plugin-returned content to the file's own tags. This removes three adjacent string parameters that were easy to swap silently, and the "" placeholder most call sites had to pass. Also hardens the logging spec from the previous commit: the null test logger is now swapped in before raising the level (SetLevel forces the current default logger to trace, so the old order left the null logger at info and trace entries never reached the hook), the sniff test now asserts probe misses are observable at trace with file attribution instead of only asserting the absence of warnings, and cleanup restores the actual previous logger — via a new return value on log.SetDefaultLogger — instead of a bare logrus.New() that would discard hooks configured on the process-wide logger. * refactor(lyrics): hoist attributed log contexts out of loops Address review feedback on #5702: build the log-attributed context once per operation instead of per iteration, and reuse it on the surrounding log calls so the error/trace lines around ParseLyrics carry the same attribution fields. In fromExternalFile the sidecar path now rides the context for all log lines in the function, replacing the repeated explicit "path" field. * style(model): pass lyrics parse errors as final log arguments Per the project logging convention, errors go as the last argument (the log package normalizes them via its error case) instead of a keyed "error" pair, which stores the raw error value and bypasses that handling. Flagged by review on #5702; the keyed form was inherited from the original warning line.
2026-07-02 09:46:57 -04:00
prev := defaultLogger
2020-01-08 20:45:07 -05:00
defaultLogger = l
fix(scanner): stop logging expected lyrics sniff misses as warnings (#5702) * fix(scanner): stop logging expected lyrics sniff misses as warnings During a scan, embedded lyrics are parsed with an empty suffix, which puts ParseLyrics into content-sniffing mode: it tries the TTML, SRT and Lyricsfile YAML parsers in turn before falling back to plain text. Every plain-text or LRC lyric therefore fails the structured probes on its way to the fallback, and each failure was logged at warning level with no indication of which file triggered it, flooding the scan log with benign "Error parsing lyrics, falling back to plain text" messages. A probe rejecting content it does not own during sniffing is expected control flow, so it is now logged at trace instead. A parse failure under an explicitly requested suffix (e.g. a malformed .yaml/.srt/.ttml sidecar) still warns, since the user declared that format. ParseLyrics gains ctx and path parameters so any warning names the offending file and carries request context where available; all call sites are updated accordingly. Also fixes a test-isolation bug in the new logging spec: the BeforeEach swapped the process-global default logger via SetDefaultLogger but only restored the log level on cleanup, leaking the null logger and its hook into later specs in the shared model suite. * test: use spec-scoped contexts instead of context.Background in lyrics tests Replace context.Background() with GinkgoT().Context() (and b.Context() in the parse benchmarks) across the lyrics-related tests, so contexts are cancelled when each spec ends. The embeddedLyrics fixture in core/lyrics is now a hand-written literal like its sibling fixtures, removing the construction-time ParseLyrics call that could not use a spec-scoped context. * refactor(model): attach lyrics parse log attribution via context Narrow ParseLyrics back to (ctx, suffix, lang, contents), dropping the path parameter added by the previous commit. Attribution now uses the codebase's existing idiom: callers that know the source attach it with log.NewContext (e.g. "file" for the media file or sidecar), and the plugin adapter tags both the plugin name and the track, fixing probe-miss logs that misattributed plugin-returned content to the file's own tags. This removes three adjacent string parameters that were easy to swap silently, and the "" placeholder most call sites had to pass. Also hardens the logging spec from the previous commit: the null test logger is now swapped in before raising the level (SetLevel forces the current default logger to trace, so the old order left the null logger at info and trace entries never reached the hook), the sniff test now asserts probe misses are observable at trace with file attribution instead of only asserting the absence of warnings, and cleanup restores the actual previous logger — via a new return value on log.SetDefaultLogger — instead of a bare logrus.New() that would discard hooks configured on the process-wide logger. * refactor(lyrics): hoist attributed log contexts out of loops Address review feedback on #5702: build the log-attributed context once per operation instead of per iteration, and reuse it on the surrounding log calls so the error/trace lines around ParseLyrics carry the same attribution fields. In fromExternalFile the sidecar path now rides the context for all log lines in the function, replacing the repeated explicit "path" field. * style(model): pass lyrics parse errors as final log arguments Per the project logging convention, errors go as the last argument (the log package normalizes them via its error case) instead of a keyed "error" pair, which stores the raw error value and bypasses that handling. Flagged by review on #5702; the keyed form was inherited from the original warning line.
2026-07-02 09:46:57 -04:00
return prev
2020-01-08 20:45:07 -05:00
}
func CurrentLevel() Level {
return Level(currentLevel.Load())
2020-01-08 20:45:07 -05:00
}
// IsGreaterOrEqualTo returns true if the caller's current log level is equal or greater than the provided level.
func IsGreaterOrEqualTo(level Level) bool {
2024-06-16 22:25:25 -04:00
return shouldLog(level, 2)
}
2026-02-08 08:57:30 -06:00
func Fatal(args ...any) {
log(LevelFatal, args...)
os.Exit(1)
}
2026-02-08 08:57:30 -06:00
func Error(args ...any) {
log(LevelError, args...)
2020-01-08 20:45:07 -05:00
}
2026-02-08 08:57:30 -06:00
func Warn(args ...any) {
log(LevelWarn, args...)
2020-01-08 20:45:07 -05:00
}
2026-02-08 08:57:30 -06:00
func Info(args ...any) {
log(LevelInfo, args...)
2020-01-08 20:45:07 -05:00
}
2026-02-08 08:57:30 -06:00
func Debug(args ...any) {
log(LevelDebug, args...)
2020-01-08 20:45:07 -05:00
}
2026-02-08 08:57:30 -06:00
func Trace(args ...any) {
log(LevelTrace, args...)
}
2026-02-08 08:57:30 -06:00
func Log(level Level, args ...any) {
log(level, args...)
}
func log(level Level, args ...any) {
2024-06-16 22:25:25 -04:00
if !shouldLog(level, 3) {
2020-01-10 13:11:24 -05:00
return
}
2020-01-10 13:11:24 -05:00
logger, msg := parseArgs(args)
logger.Log(logrus.Level(level), msg)
}
feat(plugins): experimental support for plugins (#3998) * feat(plugins): add minimal test agent plugin with API definitions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add plugin manager with auto-registration and unique agent names Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator. * test: add Ginkgo suite and test for plugin manager auto-registration Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support. * test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement persistent compilation cache for WASM agent plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmAgent to improve resource management Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance logging for wasmAgent and plugin manager operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement HttpService for handling HTTP requests in WASM plugins Also add a sample Wikimedia plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): standardize error handling in wasmAgent and MinimalAgent Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean up wikimedia plugin code Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy. * refactor: use unified SPARQLResult struct and parser for SPARQL responses Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability. * feat(plugins): improve error handling in HTTP request processing Signed-off-by: Deluan <deluan@navidrome.org> * fix: background plugin compilation, logging, and race safety Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass. * perf: implement true lazy loading for agents Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass. * fix: ensure wasm plugin instances are closed via runtime.AddCleanup Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup. * refactor: break down long functions in plugin manager and agent Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go. Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename wasmAgent to wasmArtistAgent Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename MinimalAgent for artist metadata service Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin pool creation with custom runtime and precompilation support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement generic plugin pool and agent factory for improved service handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize plugin management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): improve function signatures for clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement background precompilation for plugins and agent factory creation Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): include instanceID in logging for better traceability Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add tests for plugin pre-compilation and agent factory synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add minimal album test agent plugin for AlbumMetadataService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename fake artist and album test agent plugins for metadata services Signed-off-by: Deluan <deluan@navidrome.org> * feat(makefile): add Makefile for building plugin WASM binaries Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized. * feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove wasm module pooling it was causing issues with the GC and the Close methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename metadata service files to adapter naming convention Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle nil values in data redaction process Signed-off-by: Deluan <deluan@navidrome.org> * fix: add timeout for plugin compilation to prevent indefinite blocking Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ScrobblerService plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify generalization Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin management by improving scanning and loading mechanisms Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin creation functions to return specific interfaces for better type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement MediaMetadataService with combined artist and album methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve MediaMetadataService plugin implementation and testing structure Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add tests for Adapter Media Agent and improve plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance agent management with plugin loading and caching Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update agent discovery logic to include only local agent when no config is specified Signed-off-by: Deluan <deluan@navidrome.org> * refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct file extension filter in goimports command Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use defer to unlock the mutex Signed-off-by: Deluan <deluan@navidrome.org> * chore: move Cover Art Archive AlbumMetadataService plugins to an example folder Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle errors when creating media metadata and scrobbler service plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: increase compilation timeout to one minute Signed-off-by: Deluan <deluan@navidrome.org> * feat: add configurable plugin compilation timeout Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin scrobbler support in PlayTracker Signed-off-by: Deluan <deluan@navidrome.org> * feat: add context management and Stop method to buffered scrobbler Signed-off-by: Deluan <deluan@navidrome.org> * feat: add username field to scrobbler requests and update logging Signed-off-by: Deluan <deluan@navidrome.org> * fix: data race in test Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename http proto files to host and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused plugin registration methods from manager Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin manifests and implement plugin management commands Signed-off-by: Deluan <deluan@navidrome.org> * Update utils/files.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip") Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * feat: add plugin dev workflow support Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(README): update documentation to include TimerCallbackService and its functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add InitService with OnInit method and initialization tracking - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add tests for InitService and plugin initialization tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): expand documentation on plugin system implementation and architecture Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): redirect plugins' stderr to logs Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add safe accessor methods for TimerService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add plugin-specific configuration support in InitRequest and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add TimerCallbackService plugin adapter and integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename services for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add mutex for configuration access and clone plugin config Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): remove configtest dependency to prevent data races in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): correctly pass error to done function in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename service types to capabilities for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename InitService to LifecycleManagement for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): fix instance ID logging in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add config service for plugin-specific configuration management Signed-off-by: Deluan <deluan@navidrome.org> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat(crontab): implement crontab service for scheduling and canceling jobs Signed-off-by: Deluan <deluan@navidrome.org> * fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit) Squashed commits: [325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService Signed-off-by: Deluan <deluan@navidrome.org> * fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): consolidate job scheduling logic into a single helper function Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename GetInstance method to Instantiate for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add WebSocket service for handling connections and messages Signed-off-by: Deluan <deluan@navidrome.org> * feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): enhance connection management and callback handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): only create one adapter instance for each adapter/capability pair Signed-off-by: Deluan <deluan@navidrome.org> * fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): refactor WebSocket service integration and improve error logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): add SchedulerCallback support and improve reconnection logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: test panic Signed-off-by: Deluan <deluan@navidrome.org> * docs: add crypto-ticker plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): add error field to response messages for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(cache): implement CacheService with string, int, float, and byte operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo Signed-off-by: Deluan <deluan@navidrome.org> * refactor(cache): simplify cache service implementation and remove unnecessary synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): add build step for test plugins in the test suite Signed-off-by: Deluan <deluan@navidrome.org> * wip Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(rpc): enhance activity image processing and improve error handling in Discord integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(discord): enhance activity state with artist list and add large text asset Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(artwork): implement ArtworkService for retrieving artwork URLs Signed-off-by: Deluan <deluan@navidrome.org> * Add playback position to scrobble NowPlaying (#4089) * test(playtracker): cover playback position * address review comment Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix merge Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unnecessary check for empty slice in Map function Signed-off-by: Deluan <deluan@navidrome.org> * fix: update reflex.conf to include .wasm file extension Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): normalize attribute strings and add edge case tests for PID calculation Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458 Signed-off-by: Deluan <deluan@navidrome.org> * test(ui): fix warnings (#4187) * fix(ui): address test warnings * ignore lint error in test Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * refactor(server): optimize top songs lookup (#4189) * optimize top songs lookup * Optimize title matching queries * refactor: simplify top songs matching * improve error handling and logging in track loading functions Signed-off-by: Deluan <deluan@navidrome.org> * test: add cases for fallback to title matching and combined MBID/title matching Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): playlist details overflow in spotify-based themes (#4184) * test: ensure playlist details width * fix(test): simplify expectation for minWidth in NDPlaylistDetails Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): test all themes Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update TagLib to version 2.1 (#4185) * chore: update cross-taglib * fix(taglib): add logging for TagLib version Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * test: verify agents fallback (#4191) * build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version. This is to reduce the image size, as we don't really need the latest. Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(runtime): implement pooled WASM runtime and module for better instance management Signed-off-by: Deluan <deluan@navidrome.org> * fix(discord-plugin): adjust timer delay calculation for track completion Signed-off-by: Deluan <deluan@navidrome.org> * resolve PR comments Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement cache cleanup by size functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(manager): return error from getCompilationCache and handle it in ScanPlugins Signed-off-by: Deluan <deluan@navidrome.org> * fix possible rce condition Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): update README to include Cache and Artwork services Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add permissions support for host services in custom runtime - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add permissions field to plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * test(permissions): implement permission validation and testing for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add unauthorized_plugin to test permission enforcement - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): add Plugin Permission System section to README - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add detailed reasons for permissions in plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement granular HTTP permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unexport all plugins package private symbols Signed-off-by: Deluan <deluan@navidrome.org> * update docs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin_lifecycle_manager Signed-off-by: Deluan <deluan@navidrome.org> * docs: add discord-rich-presence plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for PATCH, HEAD, and OPTIONS HTTP methods Signed-off-by: Deluan <deluan@navidrome.org> * feat: use folder names as unique identifiers for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: read config just once, to avoid data race in tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginName to pluginID for consistency across services Signed-off-by: Deluan <deluan@navidrome.org> * fix: use symlink name instead of folder name for plugin registration Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin output format to include ID and enhance README with symlink usage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement shared plugin discovery function to streamline plugin scanning and error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: show plugin permissions in `plugin info` Signed-off-by: Deluan <deluan@navidrome.org> * feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement typed permissions for plugins to enhance permission handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor plugin permissions to use typed schema and improve validation - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused JSON schema validation for plugin manifests Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused fields from PluginPackage struct in package.go Signed-off-by: Deluan <deluan@navidrome.org> * feat: update file permissions in tests and remove unused permission parsing function Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor test plugin creation to use typed permissions and remove legacy helper Signed-off-by: Deluan <deluan@navidrome.org> * feat: add website field to plugin manifests and update test cases Signed-off-by: Deluan <deluan@navidrome.org> * refactor: permission schema to use basePermission structure for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance host service management by adding permission checks for each service Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reorganize code files Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify custom runtime creation by removing compilation cache parameter Signed-off-by: Deluan <deluan@navidrome.org> * doc: add WebSocketService and update ConfigService for plugin-specific configuration Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WASM loading optimization to enhance plugin instance creation speed Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename custom runtime functions and update related tests for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin structure with compilation handling and error reporting Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve logging and context tracing in runtime and wasm base plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance runtime management with scoped runtime and caching improvements Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement EnsureCompiled method for improved plugin compilation handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement cached module management with TTL for improved performance Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace map with sync.Map Signed-off-by: Deluan <deluan@navidrome.org> * refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance image processing with fallback mechanism for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * docs: review test plugins readme Signed-off-by: Deluan <deluan@navidrome.org> * feat: set default timeout for HTTP client to 10 seconds Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance wasm instance pool with concurrency limits and timeout settings Signed-off-by: Deluan <deluan@navidrome.org> * feat(discordrp): implement caching for processed image URLs with configurable TTL Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00
func Writer() io.Writer {
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.RLock()
defer loggerMu.RUnlock()
feat(plugins): experimental support for plugins (#3998) * feat(plugins): add minimal test agent plugin with API definitions Signed-off-by: Deluan <deluan@navidrome.org> * feat: add plugin manager with auto-registration and unique agent names Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator. * test: add Ginkgo suite and test for plugin manager auto-registration Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support. * test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement persistent compilation cache for WASM agent plugins Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmAgent to improve resource management Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance logging for wasmAgent and plugin manager operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement HttpService for handling HTTP requests in WASM plugins Also add a sample Wikimedia plugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): standardize error handling in wasmAgent and MinimalAgent Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean up wikimedia plugin code Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy. * refactor: use unified SPARQLResult struct and parser for SPARQL responses Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability. * feat(plugins): improve error handling in HTTP request processing Signed-off-by: Deluan <deluan@navidrome.org> * fix: background plugin compilation, logging, and race safety Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass. * perf: implement true lazy loading for agents Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass. * fix: ensure wasm plugin instances are closed via runtime.AddCleanup Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup. * refactor: break down long functions in plugin manager and agent Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go. Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename wasmAgent to wasmArtistAgent Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename MinimalAgent for artist metadata service Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): enhance plugin pool creation with custom runtime and precompilation support Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): implement generic plugin pool and agent factory for improved service handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): reorganize plugin management Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): improve function signatures for clarity and consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement background precompilation for plugins and agent factory creation Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): include instanceID in logging for better traceability Signed-off-by: Deluan <deluan@navidrome.org> * test(plugins): add tests for plugin pre-compilation and agent factory synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add minimal album test agent plugin for AlbumMetadataService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): rename fake artist and album test agent plugins for metadata services Signed-off-by: Deluan <deluan@navidrome.org> * feat(makefile): add Makefile for building plugin WASM binaries Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized. * feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove wasm module pooling it was causing issues with the GC and the Close methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename metadata service files to adapter naming convention Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unify album and artist method calls by introducing callMethod function Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle nil values in data redaction process Signed-off-by: Deluan <deluan@navidrome.org> * fix: add timeout for plugin compilation to prevent indefinite blocking Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement ScrobblerService plugin with authorization and scrobbling capabilities Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify generalization Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin management by improving scanning and loading mechanisms Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update plugin creation functions to return specific interfaces for better type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement MediaMetadataService with combined artist and album methods Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve MediaMetadataService plugin implementation and testing structure Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add tests for Adapter Media Agent and improve plugin documentation Signed-off-by: Deluan <deluan@navidrome.org> * docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance agent management with plugin loading and caching Signed-off-by: Deluan <deluan@navidrome.org> * refactor: update agent discovery logic to include only local agent when no config is specified Signed-off-by: Deluan <deluan@navidrome.org> * refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met. Signed-off-by: Deluan <deluan@navidrome.org> * fix: correct file extension filter in goimports command Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use defer to unlock the mutex Signed-off-by: Deluan <deluan@navidrome.org> * chore: move Cover Art Archive AlbumMetadataService plugins to an example folder Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle errors when creating media metadata and scrobbler service plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: increase compilation timeout to one minute Signed-off-by: Deluan <deluan@navidrome.org> * feat: add configurable plugin compilation timeout Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement plugin scrobbler support in PlayTracker Signed-off-by: Deluan <deluan@navidrome.org> * feat: add context management and Stop method to buffered scrobbler Signed-off-by: Deluan <deluan@navidrome.org> * feat: add username field to scrobbler requests and update logging Signed-off-by: Deluan <deluan@navidrome.org> * fix: data race in test Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename http proto files to host and update references Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unused plugin registration methods from manager Signed-off-by: Deluan <deluan@navidrome.org> * feat: extend plugin manifests and implement plugin management commands Signed-off-by: Deluan <deluan@navidrome.org> * Update utils/files.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip") Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> * feat: add plugin dev workflow support Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely. Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement timer service with register and cancel functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: lint errors Signed-off-by: Deluan <deluan@navidrome.org> * feat(README): update documentation to include TimerCallbackService and its functionality Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add InitService with OnInit method and initialization tracking - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add tests for InitService and plugin initialization tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): expand documentation on plugin system implementation and architecture Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): redirect plugins' stderr to logs Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add safe accessor methods for TimerService Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add plugin-specific configuration support in InitRequest and documentation Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add TimerCallbackService plugin adapter and integration Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename services for consistency and clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add mutex for configuration access and clone plugin config Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): remove configtest dependency to prevent data races in integration tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size Signed-off-by: Deluan <deluan@navidrome.org> * fix(plugins): correctly pass error to done function in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename service types to capabilities for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename InitService to LifecycleManagement for consistency Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): fix instance ID logging in wasmBasePlugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): make timers be isolated per plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add config service for plugin-specific configuration management Signed-off-by: Deluan <deluan@navidrome.org> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update plugins/manager.go Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat(crontab): implement crontab service for scheduling and canceling jobs Signed-off-by: Deluan <deluan@navidrome.org> * fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit) Squashed commits: [325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService Signed-off-by: Deluan <deluan@navidrome.org> * fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scheduler): consolidate job scheduling logic into a single helper function Signed-off-by: Deluan <deluan@navidrome.org> * refactor(plugin): rename GetInstance method to Instantiate for clarity Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add WebSocket service for handling connections and messages Signed-off-by: Deluan <deluan@navidrome.org> * feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): enhance connection management and callback handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): only create one adapter instance for each adapter/capability pair Signed-off-by: Deluan <deluan@navidrome.org> * fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): refactor WebSocket service integration and improve error logging Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugin): add SchedulerCallback support and improve reconnection logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: test panic Signed-off-by: Deluan <deluan@navidrome.org> * docs: add crypto-ticker plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(websocket): add error field to response messages for better error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(cache): implement CacheService with string, int, float, and byte operations Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo Signed-off-by: Deluan <deluan@navidrome.org> * refactor(cache): simplify cache service implementation and remove unnecessary synchronization Signed-off-by: Deluan <deluan@navidrome.org> * feat(tests): add build step for test plugins in the test suite Signed-off-by: Deluan <deluan@navidrome.org> * wip Signed-off-by: Deluan <deluan@navidrome.org> * feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(rpc): enhance activity image processing and improve error handling in Discord integration Signed-off-by: Deluan <deluan@navidrome.org> * feat(discord): enhance activity state with artist list and add large text asset Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(artwork): implement ArtworkService for retrieving artwork URLs Signed-off-by: Deluan <deluan@navidrome.org> * Add playback position to scrobble NowPlaying (#4089) * test(playtracker): cover playback position * address review comment Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix merge Signed-off-by: Deluan <deluan@navidrome.org> * refactor: remove unnecessary check for empty slice in Map function Signed-off-by: Deluan <deluan@navidrome.org> * fix: update reflex.conf to include .wasm file extension Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): normalize attribute strings and add edge case tests for PID calculation Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458 Signed-off-by: Deluan <deluan@navidrome.org> * test(ui): fix warnings (#4187) * fix(ui): address test warnings * ignore lint error in test Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * refactor(server): optimize top songs lookup (#4189) * optimize top songs lookup * Optimize title matching queries * refactor: simplify top songs matching * improve error handling and logging in track loading functions Signed-off-by: Deluan <deluan@navidrome.org> * test: add cases for fallback to title matching and combined MBID/title matching Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): playlist details overflow in spotify-based themes (#4184) * test: ensure playlist details width * fix(test): simplify expectation for minWidth in NDPlaylistDetails Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): test all themes Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update TagLib to version 2.1 (#4185) * chore: update cross-taglib * fix(taglib): add logging for TagLib version Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> * test: verify agents fallback (#4191) * build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version. This is to reduce the image size, as we don't really need the latest. Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(runtime): implement pooled WASM runtime and module for better instance management Signed-off-by: Deluan <deluan@navidrome.org> * fix(discord-plugin): adjust timer delay calculation for track completion Signed-off-by: Deluan <deluan@navidrome.org> * resolve PR comments Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): implement cache cleanup by size functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(manager): return error from getCompilationCache and handle it in ScanPlugins Signed-off-by: Deluan <deluan@navidrome.org> * fix possible rce condition Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): update README to include Cache and Artwork services Signed-off-by: Deluan <deluan@navidrome.org> * feat(manager): add permissions support for host services in custom runtime - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add permissions field to plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * test(permissions): implement permission validation and testing for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(plugins): add unauthorized_plugin to test permission enforcement - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(docs): add Plugin Permission System section to README - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(manifest): add detailed reasons for permissions in plugin manifests - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement granular HTTP permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP Signed-off-by: Deluan <deluan@navidrome.org> * refactor Signed-off-by: Deluan <deluan@navidrome.org> * refactor: unexport all plugins package private symbols Signed-off-by: Deluan <deluan@navidrome.org> * update docs Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename plugin_lifecycle_manager Signed-off-by: Deluan <deluan@navidrome.org> * docs: add discord-rich-presence plugin example to README Signed-off-by: Deluan <deluan@navidrome.org> * feat: add support for PATCH, HEAD, and OPTIONS HTTP methods Signed-off-by: Deluan <deluan@navidrome.org> * feat: use folder names as unique identifiers for plugins Signed-off-by: Deluan <deluan@navidrome.org> * fix: read config just once, to avoid data race in tests Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename pluginName to pluginID for consistency across services Signed-off-by: Deluan <deluan@navidrome.org> * fix: use symlink name instead of folder name for plugin registration Signed-off-by: Deluan <deluan@navidrome.org> * feat: update plugin output format to include ID and enhance README with symlink usage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement shared plugin discovery function to streamline plugin scanning and error handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: show plugin permissions in `plugin info` Signed-off-by: Deluan <deluan@navidrome.org> * feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement typed permissions for plugins to enhance permission handling Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor plugin permissions to use typed schema and improve validation - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused JSON schema validation for plugin manifests Signed-off-by: Deluan <deluan@navidrome.org> * feat: remove unused fields from PluginPackage struct in package.go Signed-off-by: Deluan <deluan@navidrome.org> * feat: update file permissions in tests and remove unused permission parsing function Signed-off-by: Deluan <deluan@navidrome.org> * feat: refactor test plugin creation to use typed permissions and remove legacy helper Signed-off-by: Deluan <deluan@navidrome.org> * feat: add website field to plugin manifests and update test cases Signed-off-by: Deluan <deluan@navidrome.org> * refactor: permission schema to use basePermission structure for consistency Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance host service management by adding permission checks for each service Signed-off-by: Deluan <deluan@navidrome.org> * refactor: reorganize code files Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify custom runtime creation by removing compilation cache parameter Signed-off-by: Deluan <deluan@navidrome.org> * doc: add WebSocketService and update ConfigService for plugin-specific configuration Signed-off-by: Deluan <deluan@navidrome.org> * feat: implement WASM loading optimization to enhance plugin instance creation speed Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename custom runtime functions and update related tests for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance plugin structure with compilation handling and error reporting Signed-off-by: Deluan <deluan@navidrome.org> * refactor: improve logging and context tracing in runtime and wasm base plugin Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance runtime management with scoped runtime and caching improvements Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement EnsureCompiled method for improved plugin compilation handling Signed-off-by: Deluan <deluan@navidrome.org> * refactor: implement cached module management with TTL for improved performance Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace map with sync.Map Signed-off-by: Deluan <deluan@navidrome.org> * refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness Signed-off-by: Deluan <deluan@navidrome.org> * refactor: enhance image processing with fallback mechanism for improved error handling Signed-off-by: Deluan <deluan@navidrome.org> * docs: review test plugins readme Signed-off-by: Deluan <deluan@navidrome.org> * feat: set default timeout for HTTP client to 10 seconds Signed-off-by: Deluan <deluan@navidrome.org> * feat: enhance wasm instance pool with concurrency limits and timeout settings Signed-off-by: Deluan <deluan@navidrome.org> * feat(discordrp): implement caching for processed image URLs with configurable TTL Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00
return defaultLogger.Writer()
}
2024-06-16 22:25:25 -04:00
func shouldLog(requiredLevel Level, skip int) bool {
level := Level(currentLevel.Load())
feat(scanner): implement selective folder scanning and file system watcher improvements (#4674) * feat: Add selective folder scanning capability Implement targeted scanning of specific library/folder pairs without full recursion. This enables efficient rescanning of individual folders when changes are detected, significantly reducing scan time for large libraries. Key changes: - Add ScanTarget struct and ScanFolders API to Scanner interface - Implement CLI flag --targets for specifying libraryID:folderPath pairs - Add FolderRepository.GetByPaths() for batch folder info retrieval - Create loadSpecificFolders() for non-recursive directory loading - Scope GC operations to affected libraries only (with TODO for full impl) - Add comprehensive tests for selective scanning behavior The selective scan: - Only processes specified folders (no subdirectory recursion) - Maintains library isolation - Runs full maintenance pipeline scoped to affected libraries - Supports both full and quick scan modes Examples: navidrome scan --targets "1:Music/Rock,1:Music/Jazz" navidrome scan --full --targets "2:Classical" * feat(folder): replace GetByPaths with GetFolderUpdateInfo for improved folder updates retrieval Signed-off-by: Deluan <deluan@navidrome.org> * test: update parseTargets test to handle folder names with spaces Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): remove unused LibraryPath struct and update GC logging message Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): enhance external scanner to support target-specific scanning Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify scanner methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement folder scanning notifications with deduplication Signed-off-by: Deluan <deluan@navidrome.org> * refactor(watcher): add resolveFolderPath function for testability Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement path ignoring based on .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): implement IgnoreChecker for managing .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(ignore_checker): rename scanner to lineScanner for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance ScanTarget struct with String method for better target representation Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): validate library ID to prevent negative values Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify GC method by removing library ID parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): update folder scanning to include all descendants of specified folders Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): allow selective scan in the /startScan endpoint Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update CallScan to handle specific library/folder pairs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline scanning logic by removing scanAll method Signed-off-by: Deluan <deluan@navidrome.org> * test: enhance mockScanner for thread safety and improve test reliability Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move scanner.ScanTarget to model.ScanTarget Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move scanner types to model,implement MockScanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update scanner interface and implementations to use model.Scanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder_repository): normalize target path handling by using filepath.Clean Signed-off-by: Deluan <deluan@navidrome.org> * test(folder_repository): add comprehensive tests for folder retrieval and child exclusion Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify selective scan logic using slice.Filter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline phase folder and album creation by removing unnecessary library parameter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move initialization logic from phase_1 to the scanner itself Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): rename selective scan test file to scanner_selective_test.go Signed-off-by: Deluan <deluan@navidrome.org> * feat(configuration): add DevSelectiveWatcher configuration option Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): enhance .ndignore handling for folder deletions and file changes Signed-off-by: Deluan <deluan@navidrome.org> * docs(scanner): comments Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance walkDirTree to support target folder scanning Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner, watcher): handle errors when pushing ignore patterns for folders Signed-off-by: Deluan <deluan@navidrome.org> * Update scanner/phase_1_folders.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor(scanner): replace parseTargets function with direct call to scanner.ParseTargets Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): add tests for ScanBegin and ScanEnd functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(library): update PRAGMA optimize to check table sizes without ANALYZE Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): refactor tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add selective scan options and update translations Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add quick and full scan options for individual libraries Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add Scan buttonsto the LibraryList Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): update scanning parameters from 'path' to 'target' for selective scans. * refactor(scan): move ParseTargets function to model package * test(scan): suppress unused return value from SetUserLibraries in tests * feat(gc): enhance garbage collection to support selective library purging Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): prevent race condition when scanning deleted folders When the watcher detects changes in a folder that gets deleted before the scanner runs (due to the 10-second delay), the scanner was prematurely removing these folders from the tracking map, preventing them from being marked as missing. The issue occurred because `newFolderEntry` was calling `popLastUpdate` before verifying the folder actually exists on the filesystem. Changes: - Move fs.Stat check before newFolderEntry creation in loadDir to ensure deleted folders remain in lastUpdates for finalize() to handle - Add early existence check in walkDirTree to skip non-existent target folders with a warning log - Add unit test verifying non-existent folders aren't removed from lastUpdates prematurely - Add integration test for deleted folder scenario with ScanFolders Fixes the issue where deleting entire folders (e.g., /music/AC_DC) wouldn't mark tracks as missing when using selective folder scanning. * refactor(scan): streamline folder entry creation and update handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): add '@Recycle' (QNAP) to ignored directories list Signed-off-by: Deluan <deluan@navidrome.org> * fix(log): improve thread safety in logging level management * test(scan): move unit tests for ParseTargets function Signed-off-by: Deluan <deluan@navidrome.org> * review Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: deluan <deluan.quintao@mechanical-orchard.com>
2025-11-14 22:15:43 -05:00
if level >= requiredLevel {
return true
}
if !hasLogLevelOverrides.Load() {
return false
}
loggerMu.RLock()
levels := logLevels
loggerMu.RUnlock()
2024-06-16 22:25:25 -04:00
_, file, _, ok := runtime.Caller(skip)
if !ok {
return false
}
file = strings.TrimPrefix(file, rootPath)
feat(scanner): implement selective folder scanning and file system watcher improvements (#4674) * feat: Add selective folder scanning capability Implement targeted scanning of specific library/folder pairs without full recursion. This enables efficient rescanning of individual folders when changes are detected, significantly reducing scan time for large libraries. Key changes: - Add ScanTarget struct and ScanFolders API to Scanner interface - Implement CLI flag --targets for specifying libraryID:folderPath pairs - Add FolderRepository.GetByPaths() for batch folder info retrieval - Create loadSpecificFolders() for non-recursive directory loading - Scope GC operations to affected libraries only (with TODO for full impl) - Add comprehensive tests for selective scanning behavior The selective scan: - Only processes specified folders (no subdirectory recursion) - Maintains library isolation - Runs full maintenance pipeline scoped to affected libraries - Supports both full and quick scan modes Examples: navidrome scan --targets "1:Music/Rock,1:Music/Jazz" navidrome scan --full --targets "2:Classical" * feat(folder): replace GetByPaths with GetFolderUpdateInfo for improved folder updates retrieval Signed-off-by: Deluan <deluan@navidrome.org> * test: update parseTargets test to handle folder names with spaces Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): remove unused LibraryPath struct and update GC logging message Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder): enhance external scanner to support target-specific scanning Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify scanner methods Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement folder scanning notifications with deduplication Signed-off-by: Deluan <deluan@navidrome.org> * refactor(watcher): add resolveFolderPath function for testability Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): implement path ignoring based on .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): implement IgnoreChecker for managing .ndignore patterns Signed-off-by: Deluan <deluan@navidrome.org> * refactor(ignore_checker): rename scanner to lineScanner for clarity Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance ScanTarget struct with String method for better target representation Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): validate library ID to prevent negative values Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify GC method by removing library ID parameter Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): update folder scanning to include all descendants of specified folders Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): allow selective scan in the /startScan endpoint Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update CallScan to handle specific library/folder pairs Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline scanning logic by removing scanAll method Signed-off-by: Deluan <deluan@navidrome.org> * test: enhance mockScanner for thread safety and improve test reliability Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move scanner.ScanTarget to model.ScanTarget Signed-off-by: Deluan <deluan@navidrome.org> * refactor: move scanner types to model,implement MockScanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): update scanner interface and implementations to use model.Scanner Signed-off-by: Deluan <deluan@navidrome.org> * refactor(folder_repository): normalize target path handling by using filepath.Clean Signed-off-by: Deluan <deluan@navidrome.org> * test(folder_repository): add comprehensive tests for folder retrieval and child exclusion Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): simplify selective scan logic using slice.Filter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): streamline phase folder and album creation by removing unnecessary library parameter Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): move initialization logic from phase_1 to the scanner itself Signed-off-by: Deluan <deluan@navidrome.org> * refactor(tests): rename selective scan test file to scanner_selective_test.go Signed-off-by: Deluan <deluan@navidrome.org> * feat(configuration): add DevSelectiveWatcher configuration option Signed-off-by: Deluan <deluan@navidrome.org> * feat(watcher): enhance .ndignore handling for folder deletions and file changes Signed-off-by: Deluan <deluan@navidrome.org> * docs(scanner): comments Signed-off-by: Deluan <deluan@navidrome.org> * refactor(scanner): enhance walkDirTree to support target folder scanning Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner, watcher): handle errors when pushing ignore patterns for folders Signed-off-by: Deluan <deluan@navidrome.org> * Update scanner/phase_1_folders.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor(scanner): replace parseTargets function with direct call to scanner.ParseTargets Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): add tests for ScanBegin and ScanEnd functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix(library): update PRAGMA optimize to check table sizes without ANALYZE Signed-off-by: Deluan <deluan@navidrome.org> * test(scanner): refactor tests Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add selective scan options and update translations Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add quick and full scan options for individual libraries Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add Scan buttonsto the LibraryList Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): update scanning parameters from 'path' to 'target' for selective scans. * refactor(scan): move ParseTargets function to model package * test(scan): suppress unused return value from SetUserLibraries in tests * feat(gc): enhance garbage collection to support selective library purging Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): prevent race condition when scanning deleted folders When the watcher detects changes in a folder that gets deleted before the scanner runs (due to the 10-second delay), the scanner was prematurely removing these folders from the tracking map, preventing them from being marked as missing. The issue occurred because `newFolderEntry` was calling `popLastUpdate` before verifying the folder actually exists on the filesystem. Changes: - Move fs.Stat check before newFolderEntry creation in loadDir to ensure deleted folders remain in lastUpdates for finalize() to handle - Add early existence check in walkDirTree to skip non-existent target folders with a warning log - Add unit test verifying non-existent folders aren't removed from lastUpdates prematurely - Add integration test for deleted folder scenario with ScanFolders Fixes the issue where deleting entire folders (e.g., /music/AC_DC) wouldn't mark tracks as missing when using selective folder scanning. * refactor(scan): streamline folder entry creation and update handling Signed-off-by: Deluan <deluan@navidrome.org> * feat(scan): add '@Recycle' (QNAP) to ignored directories list Signed-off-by: Deluan <deluan@navidrome.org> * fix(log): improve thread safety in logging level management * test(scan): move unit tests for ParseTargets function Signed-off-by: Deluan <deluan@navidrome.org> * review Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: deluan <deluan.quintao@mechanical-orchard.com>
2025-11-14 22:15:43 -05:00
for _, lp := range levels {
if strings.HasPrefix(file, lp.path) {
return lp.level >= requiredLevel
}
}
return false
2020-01-10 13:11:24 -05:00
}
2026-02-08 08:57:30 -06:00
func parseArgs(args []any) (*logrus.Entry, string) {
2020-01-08 20:45:07 -05:00
var l *logrus.Entry
var err error
if args[0] == nil {
l = createNewLogger()
args = args[1:]
} else {
l, err = extractLogger(args[0])
if err != nil {
l = createNewLogger()
} else {
args = args[1:]
}
}
if len(args) > 1 {
kvPairs := args[1:]
l = addFields(l, kvPairs)
}
if logSourceLine {
_, file, line, ok := runtime.Caller(3)
if !ok {
file = "???"
line = 0
}
//_, filename := path.Split(file)
//l = l.WithField("filename", filename).WithField("line", line)
l = l.WithField(" source", fmt.Sprintf("file://%s:%d", file, line))
}
2020-01-08 20:45:07 -05:00
switch msg := args[0].(type) {
case error:
return l, msg.Error()
case string:
return l, msg
}
2020-01-08 20:45:07 -05:00
return l, ""
}
2026-02-08 08:57:30 -06:00
func addFields(logger *logrus.Entry, keyValuePairs []any) *logrus.Entry {
2020-01-08 20:45:07 -05:00
for i := 0; i < len(keyValuePairs); i += 2 {
switch name := keyValuePairs[i].(type) {
case error:
logger = logger.WithField("error", name.Error())
case string:
if i+1 >= len(keyValuePairs) {
logger = logger.WithField(name, "!!!!Invalid number of arguments in log call!!!!")
} else {
2021-02-03 13:04:20 -05:00
switch v := keyValuePairs[i+1].(type) {
case time.Duration:
logger = logger.WithField(name, ShortDur(v))
2022-12-20 12:55:40 -05:00
case fmt.Stringer:
logger = logger.WithField(name, StringerValue(v))
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709) * fix(server): more race conditions when updating artist/album from external sources Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394 Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): pass configfile option to child process Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): resume interrupted fullScans Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): remove old scanner code Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): rename old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): move old metadata package Signed-off-by: Deluan <deluan@navidrome.org> * fix: tests Signed-off-by: Deluan <deluan@navidrome.org> * chore(deps): update Go to 1.23.4 Signed-off-by: Deluan <deluan@navidrome.org> * fix: logs Signed-off-by: Deluan <deluan@navidrome.org> * fix(test): Signed-off-by: Deluan <deluan@navidrome.org> * fix: log level Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add config for scanner watcher Signed-off-by: Deluan <deluan@navidrome.org> * refactor: children playlists Signed-off-by: Deluan <deluan@navidrome.org> * refactor: replace `interface{}` with `any` Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlists with genres Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow any tags in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist names in playlists Signed-off-by: Deluan <deluan@navidrome.org> * fix: smart playlist's sort by tags Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use generic JSONArray for OS arrays Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): use https in test Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add releaseTypes to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add recordLabels to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): rename JSONArray to Array Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to AlbumID3 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to Child Signed-off-by: Deluan <deluan@navidrome.org> * fix(scanner): do not pre-populate smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement a simplified version of ArtistID3. See https://github.com/opensubsonic/open-subsonic-api/discussions/120 Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add artists to album child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add contributors to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add albumArtists to mediafile Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayArtist and displayAlbumArtist Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add displayComposer to Child Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add roles to ArtistID3 Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): use " • " separator for displayComposer Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * fix(subsonic): respect `PreferSortTags` config option Signed-off-by: Deluan <deluan@navidrome.org> * refactor(subsonic): Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize purging non-unused tags Signed-off-by: Deluan <deluan@navidrome.org> * refactor: don't run 'refresh artist stats' concurrently with other transactions Signed-off-by: Deluan <deluan@navidrome.org> * refactor: Signed-off-by: Deluan <deluan@navidrome.org> * fix: log message Signed-off-by: Deluan <deluan@navidrome.org> * feat: add Scanner.ScanOnStartup config option, default true Signed-off-by: Deluan <deluan@navidrome.org> * feat: better json parsing error msg when importing NSPs Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update album's imported_time when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * fix: handle interrupted scans and full scans after migrations Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `analyze` when migration requires a full rescan Signed-off-by: Deluan <deluan@navidrome.org> * feat: run `PRAGMA optimize` at the end of the scan Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't update artist's updated_at when updating external_metadata Signed-off-by: Deluan <deluan@navidrome.org> * feat: handle multiple artists and roles in smart playlists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): dim missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * fix: album missing logic Signed-off-by: Deluan <deluan@navidrome.org> * fix: error encoding in gob Signed-off-by: Deluan <deluan@navidrome.org> * feat: separate warnings from errors Signed-off-by: Deluan <deluan@navidrome.org> * fix: mark albums as missing if they were contained in a deleted folder Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add participant names to media_file and album tables Signed-off-by: Deluan <deluan@navidrome.org> * refactor: use participations in criteria, instead of m2m relationship Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename participations to participants Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add moods to album child Signed-off-by: Deluan <deluan@navidrome.org> * fix: albumartist role case Signed-off-by: Deluan <deluan@navidrome.org> * feat(scanner): run scanner as an external process by default Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): show albumArtist names Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): dim out missing albums Signed-off-by: Deluan <deluan@navidrome.org> * fix: flaky test Signed-off-by: Deluan <deluan@navidrome.org> * fix(server): scrobble buffer mapping. fix #3583 Signed-off-by: Deluan <deluan@navidrome.org> * refactor: more participations renaming Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling Signed-off-by: Deluan <deluan@navidrome.org> * feat: send release_group_mbid to listenbrainz Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): implement OpenSubsonic explicitStatus field (#3597) * feat: implement OpenSubsonic explicitStatus field * fix(subsonic): fix failing snapshot tests * refactor: create helper for setting explicitStatus * fix: store smaller values for explicit-status on database * test: ToAlbum explicitStatus * refactor: rename explicitStatus helper function --------- Co-authored-by: Deluan Quintão <deluan@navidrome.org> * fix: handle album and track tags in the DB based on the mappings.yaml file Signed-off-by: Deluan <deluan@navidrome.org> * save similar artists as JSONB Signed-off-by: Deluan <deluan@navidrome.org> * fix: getAlbumList byGenre Signed-off-by: Deluan <deluan@navidrome.org> * detect changes in PID configuration Signed-off-by: Deluan <deluan@navidrome.org> * set default album PID to legacy_pid Signed-off-by: Deluan <deluan@navidrome.org> * fix tests Signed-off-by: Deluan <deluan@navidrome.org> * fix SIGSEGV Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't lose album stars/ratings when migrating Signed-off-by: Deluan <deluan@navidrome.org> * store full PID conf in properties Signed-off-by: Deluan <deluan@navidrome.org> * fix: keep album annotations when changing PID.Album config Signed-off-by: Deluan <deluan@navidrome.org> * fix: reassign album annotations Signed-off-by: Deluan <deluan@navidrome.org> * feat: use (display) albumArtist and add links to each artist Signed-off-by: Deluan <deluan@navidrome.org> * fix: not showing albums by albumartist Signed-off-by: Deluan <deluan@navidrome.org> * fix: error msgs Signed-off-by: Deluan <deluan@navidrome.org> * fix: hide PID from Native API Signed-off-by: Deluan <deluan@navidrome.org> * fix: album cover art resolution Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim participant names Signed-off-by: Deluan <deluan@navidrome.org> * fix: reduce watcher log spam Signed-off-by: Deluan <deluan@navidrome.org> * fix: panic when initializing the watcher Signed-off-by: Deluan <deluan@navidrome.org> * fix: various artists Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't store empty lyrics in the DB Signed-off-by: Deluan <deluan@navidrome.org> * remove unused methods Signed-off-by: Deluan <deluan@navidrome.org> * drop full_text indexes, as they are not being used by SQLite Signed-off-by: Deluan <deluan@navidrome.org> * keep album created_at when upgrading Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): null pointer Signed-off-by: Deluan <deluan@navidrome.org> * fix: album artwork cache Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't expose missing files in Subsonic API Signed-off-by: Deluan <deluan@navidrome.org> * refactor: searchable interface Signed-off-by: Deluan <deluan@navidrome.org> * fix: filter out missing items from subsonic search * fix: filter out missing items from playlists * fix: filter out missing items from shares Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): add filter by artist role Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): only return albumartists in getIndexes and getArtists endpoints Signed-off-by: Deluan <deluan@navidrome.org> * sort roles alphabetically Signed-off-by: Deluan <deluan@navidrome.org> * fix: artist playcounts Signed-off-by: Deluan <deluan@navidrome.org> * change default Album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * fix albumartist link when it does not match any albumartists values Signed-off-by: Deluan <deluan@navidrome.org> * fix `Ignoring filter not whitelisted` (role) message Signed-off-by: Deluan <deluan@navidrome.org> * fix: trim any names/titles being imported Signed-off-by: Deluan <deluan@navidrome.org> * remove unused genre code Signed-off-by: Deluan <deluan@navidrome.org> * serialize calls to Last.fm's getArtist Signed-off-by: Deluan <deluan@navidrome.org> xxx Signed-off-by: Deluan <deluan@navidrome.org> * add counters to genres Signed-off-by: Deluan <deluan@navidrome.org> * nit: fix migration `notice` message Signed-off-by: Deluan <deluan@navidrome.org> * optimize similar artists query Signed-off-by: Deluan <deluan@navidrome.org> * fix: last.fm.getInfo when mbid does not exist Signed-off-by: Deluan <deluan@navidrome.org> * ui only show missing items for admins Signed-off-by: Deluan <deluan@navidrome.org> * don't allow interaction with missing items Signed-off-by: Deluan <deluan@navidrome.org> * Add Missing Files view (WIP) Signed-off-by: Deluan <deluan@navidrome.org> * refactor: merged tag_counts into tag table Signed-off-by: Deluan <deluan@navidrome.org> * add option to completely disable automatic scanner Signed-off-by: Deluan <deluan@navidrome.org> * add delete missing files functionality Signed-off-by: Deluan <deluan@navidrome.org> * fix: playlists not showing for regular users Signed-off-by: Deluan <deluan@navidrome.org> * reduce updateLastAccess frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * reduce update player frequency to once every minute Signed-off-by: Deluan <deluan@navidrome.org> * add timeout when updating player Signed-off-by: Deluan <deluan@navidrome.org> * remove dead code Signed-off-by: Deluan <deluan@navidrome.org> * fix duplicated roles in stats Signed-off-by: Deluan <deluan@navidrome.org> * add `; ` to artist splitters Signed-off-by: Deluan <deluan@navidrome.org> * fix stats query Signed-off-by: Deluan <deluan@navidrome.org> * more logs Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP Signed-off-by: Deluan <deluan@navidrome.org> * add record label filter Signed-off-by: Deluan <deluan@navidrome.org> * add release type filter Signed-off-by: Deluan <deluan@navidrome.org> * fix purgeUnused tags Signed-off-by: Deluan <deluan@navidrome.org> * add grouping filter to albums Signed-off-by: Deluan <deluan@navidrome.org> * allow any album tags to be used in as filters in the API Signed-off-by: Deluan <deluan@navidrome.org> * remove empty tags from album info Signed-off-by: Deluan <deluan@navidrome.org> * comments in the migration Signed-off-by: Deluan <deluan@navidrome.org> * fix: Cannot read properties of undefined Signed-off-by: Deluan <deluan@navidrome.org> * fix: listenbrainz scrobbling (#3640) Signed-off-by: Deluan <deluan@navidrome.org> * fix: remove duplicated tag values Signed-off-by: Deluan <deluan@navidrome.org> * fix: don't ignore the taglib folder! Signed-off-by: Deluan <deluan@navidrome.org> * feat: show track subtitle tag Signed-off-by: Deluan <deluan@navidrome.org> * fix: show artists stats based on selected role Signed-off-by: Deluan <deluan@navidrome.org> * fix: inspect Signed-off-by: Deluan <deluan@navidrome.org> * add media type to album info/filters Signed-off-by: Deluan <deluan@navidrome.org> * fix: change format of subtitle in the UI Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in Subsonic API and search Signed-off-by: Deluan <deluan@navidrome.org> * fix: subtitle in UI's player Signed-off-by: Deluan <deluan@navidrome.org> * fix: split strings should be case-insensitive Signed-off-by: Deluan <deluan@navidrome.org> * disable ScanSchedule Signed-off-by: Deluan <deluan@navidrome.org> * increase default sessiontimeout Signed-off-by: Deluan <deluan@navidrome.org> * add sqlite command line tool to docker image Signed-off-by: Deluan <deluan@navidrome.org> * fix: resources override Signed-off-by: Deluan <deluan@navidrome.org> * fix: album PID conf Signed-off-by: Deluan <deluan@navidrome.org> * change migration to mark current artists as albumArtists Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Allow filtering on multiple genres (#3679) * feat(ui): Allow filtering on multiple genres Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> * add multi-genre filter in Album list Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> * add more multi-valued tag filters to Album and Song views Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): unselect missing files after removing Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): song filter Signed-off-by: Deluan <deluan@navidrome.org> * fix sharing tracks. fix #3687 Signed-off-by: Deluan <deluan@navidrome.org> * use rowids when using search for sync (ex: Symfonium) Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients Signed-off-by: Deluan <deluan@navidrome.org> * fix "Report Real Paths" option for subsonic clients for search Signed-off-by: Deluan <deluan@navidrome.org> * add libraryPath to Native API /songs endpoint Signed-off-by: Deluan <deluan@navidrome.org> * feat(subsonic): add album version Signed-off-by: Deluan <deluan@navidrome.org> * made all tags lowercase as they are case-insensitive anyways. Signed-off-by: Deluan <deluan@navidrome.org> * feat(ui): Show full paths, extended properties for album/song (#3691) * feat(ui): Show full paths, extended properties for album/song - uses library path + os separator + path - show participants (album/song) and tags (song) - make album/participant clickable in show info * add source to path * fix pathSeparator in UI Signed-off-by: Deluan <deluan@navidrome.org> * fix local artist artwork (#3695) Signed-off-by: Deluan <deluan@navidrome.org> * fix: parse vorbis performers Signed-off-by: Deluan <deluan@navidrome.org> * refactor: clean function into smaller functions Signed-off-by: Deluan <deluan@navidrome.org> * fix translations for en and pt Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * add trace log to show annotations reassignment Signed-off-by: Deluan <deluan@navidrome.org> * fix: allow performers without instrument/subrole Signed-off-by: Deluan <deluan@navidrome.org> * refactor: metadata clean function again Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize split function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: split function is now a method of TagConf Signed-off-by: Deluan <deluan@navidrome.org> * fix: humanize Artist total size Signed-off-by: Deluan <deluan@navidrome.org> * add album version to album details Signed-off-by: Deluan <deluan@navidrome.org> * don't display album-level tags in SongInfo Signed-off-by: Deluan <deluan@navidrome.org> * fix genre clicking in Album Page Signed-off-by: Deluan <deluan@navidrome.org> * don't use mbids in Last.fm api calls. From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248: With MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo { artist: { name: "Bee Gees", mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810", url: "https://www.last.fm/music/Bee+Gees", } ``` Without MBID: ``` GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo { artist: { name: "Van Morrison", mbid: "a41ac10f-0a56-4672-9161-b83f9b223559", url: "https://www.last.fm/music/Van+Morrison", } ``` Signed-off-by: Deluan <deluan@navidrome.org> * better logging for when the artist folder is not found Signed-off-by: Deluan <deluan@navidrome.org> * fix various issues with artist image resolution Signed-off-by: Deluan <deluan@navidrome.org> * hide "Additional Tags" header if there are none. Signed-off-by: Deluan <deluan@navidrome.org> * simplify tag rendering Signed-off-by: Deluan <deluan@navidrome.org> * enhance logging for artist folder detection Signed-off-by: Deluan <deluan@navidrome.org> * make folderID consistent for relative and absolute folderPaths Signed-off-by: Deluan <deluan@navidrome.org> * handle more folder paths scenarios Signed-off-by: Deluan <deluan@navidrome.org> * filter out other roles when SubsonicArtistParticipations = true Signed-off-by: Deluan <deluan@navidrome.org> * fix "Cannot read properties of undefined" Signed-off-by: Deluan <deluan@navidrome.org> * fix lyrics and comments being truncated (#3701) * fix lyrics and comments being truncated * specifically test for lyrics and comment length * reorder assertions Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * fix(server): Expose library_path for playlist (#3705) Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic) * fix BFR on Windows (#3704) * fix potential reflected cross-site scripting vulnerability Signed-off-by: Deluan <deluan@navidrome.org> * hack to make it work on Windows * ignore windows executables * try fixing the pipeline Signed-off-by: Deluan <deluan@navidrome.org> * allow MusicFolder in other drives * move windows local drive logic to local storage implementation --------- Signed-off-by: Deluan <deluan@navidrome.org> * increase pagination sizes for missing files Signed-off-by: Deluan <deluan@navidrome.org> * reduce level of "already scanning" watcher log message Signed-off-by: Deluan <deluan@navidrome.org> * only count folders with audio files in it See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930 Signed-off-by: Deluan <deluan@navidrome.org> * add album version and catalog number to search Signed-off-by: Deluan <deluan@navidrome.org> * add `organization` alias for `recordlabel` Signed-off-by: Deluan <deluan@navidrome.org> * remove mbid from Last.fm agent Signed-off-by: Deluan <deluan@navidrome.org> * feat: support inspect in ui (#3726) * inspect in ui * address round 1 * add catalogNum to AlbumInfo Signed-off-by: Deluan <deluan@navidrome.org> * remove dependency on metadata_old (deprecated) package Signed-off-by: Deluan <deluan@navidrome.org> * add `RawTags` to model Signed-off-by: Deluan <deluan@navidrome.org> * support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698) * parse standard roles, vorbis/m4a work for now * fix djmixer * working roles, use DJ-mix * add performers to file * map mbids * add a few more tests * add test Signed-off-by: Deluan <deluan@navidrome.org> * try to simplify the performers logic Signed-off-by: Deluan <deluan@navidrome.org> * stylistic changes --------- Signed-off-by: Deluan <deluan@navidrome.org> Co-authored-by: Deluan <deluan@navidrome.org> * remove param mutation Signed-off-by: Deluan <deluan@navidrome.org> * run automated SQLite optimizations Signed-off-by: Deluan <deluan@navidrome.org> * fix playlists import/export on Windows * fix import playlists * fix export playlists * better handling of Windows volumes Signed-off-by: Deluan <deluan@navidrome.org> * handle more album ID reassignments Signed-off-by: Deluan <deluan@navidrome.org> * allow adding/overriding tags in the config file Signed-off-by: Deluan <deluan@navidrome.org> * fix(ui): Fix playlist track id, handle missing tracks better (#3734) - Use `mediaFileId` instead of `id` for playlist tracks - Only fetch if the file is not missing - If extractor fails to get the file, also error (rather than panic) * optimize DB after each scan. Signed-off-by: Deluan <deluan@navidrome.org> * remove sortable from AlbumSongs columns Signed-off-by: Deluan <deluan@navidrome.org> * simplify query to get missing tracks Signed-off-by: Deluan <deluan@navidrome.org> * mark Scanner.Extractor as deprecated Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org> Signed-off-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Caio Cotts <caio@cotts.com.br> Co-authored-by: Henrik Nordvik <henrikno@gmail.com> Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-02-19 17:35:17 -08:00
case iter.Seq[string]:
logger = logger.WithField(name, formatSeq(v))
case []string:
logger = logger.WithField(name, formatSlice(v))
2021-02-03 13:04:20 -05:00
default:
logger = logger.WithField(name, v)
}
}
2020-01-08 20:45:07 -05:00
}
}
return logger
}
2026-02-08 08:57:30 -06:00
func extractLogger(ctx any) (*logrus.Entry, error) {
2020-01-08 20:45:07 -05:00
switch ctx := ctx.(type) {
case *logrus.Entry:
return ctx, nil
case context.Context:
2020-05-13 16:49:55 -04:00
logger := ctx.Value(loggerCtxKey)
2020-01-08 20:45:07 -05:00
if logger != nil {
return logger.(*logrus.Entry), nil
}
return extractLogger(NewContext(ctx))
2020-01-08 20:45:07 -05:00
case *http.Request:
return extractLogger(ctx.Context())
}
return nil, errors.New("no logger found")
}
func createNewLogger() *logrus.Entry {
//logrus.SetFormatter(&logrus.TextFormatter{ForceColors: true, DisableTimestamp: false, FullTimestamp: true})
2020-01-08 20:45:07 -05:00
//l.Formatter = &logrus.TextFormatter{ForceColors: true, DisableTimestamp: false, FullTimestamp: true}
fix: album statistics not updating after deleting missing files (#4668) * feat: add album refresh functionality after deleting missing files Implemented RefreshAlbums method in AlbumRepository to recalculate album attributes (size, duration, song count) from their constituent media files. This method processes albums in batches to maintain efficiency with large datasets. Added integration in deleteMissingFiles to automatically refresh affected albums in the background after deleting missing media files, ensuring album statistics remain accurate. Includes comprehensive test coverage for various scenarios including single/multiple albums, empty batches, and large batch processing. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: extract missing files deletion into reusable service layer Extracted inline deletion logic from server/nativeapi/missing.go into a new core.MissingFiles service interface and implementation. This provides better separation of concerns and testability. The MissingFiles service handles: - Deletion of specific or all missing files via transaction - Garbage collection after deletion - Extraction of affected album IDs from missing files - Background refresh of artist and album statistics The deleteMissingFiles HTTP handler now simply delegates to the service, removing 70+ lines of inline logic. All deletion, transaction, and stat refresh logic is now centralized in core/missing_files.go. Updated dependency injection to provide MissingFiles service to the native API router. Renamed receiver variable from 'n' to 'api' throughout native_api.go for consistency. * refactor: consolidate maintenance operations into unified service Consolidate MissingFiles and RefreshAlbums functionality into a new Maintenance service. This refactoring: - Creates core.Maintenance interface combining DeleteMissingFiles, DeleteAllMissingFiles, and RefreshAlbums methods - Moves RefreshAlbums logic from AlbumRepository persistence layer to core Maintenance service - Removes MissingFiles interface and moves its implementation to maintenanceService - Updates all references in wire providers, native API router, and handlers - Removes RefreshAlbums interface method from AlbumRepository model - Improves separation of concerns by centralizing maintenance operations in the core domain This change provides a cleaner API and better organization of maintenance-related database operations. * refactor: remove MissingFiles interface and update references Remove obsolete MissingFiles interface and its references: - Delete core/missing_files.go and core/missing_files_test.go - Remove RefreshAlbums method from AlbumRepository interface and implementation - Remove RefreshAlbums tests from AlbumRepository test suite - Update wire providers to use NewMaintenance instead of NewMissingFiles - Update native API router to use Maintenance service - Update missing.go handler to use Maintenance interface All functionality is now consolidated in the core.Maintenance service. Signed-off-by: Deluan <deluan@navidrome.org> * refactor: rename RefreshAlbums to refreshAlbums and update related calls Signed-off-by: Deluan <deluan@navidrome.org> * refactor: optimize album refresh logic and improve test coverage Signed-off-by: Deluan <deluan@navidrome.org> * refactor: simplify logging setup in tests with reusable LogHook function Signed-off-by: Deluan <deluan@navidrome.org> * refactor: add synchronization to logger and maintenance service for thread safety Signed-off-by: Deluan <deluan@navidrome.org> --------- Signed-off-by: Deluan <deluan@navidrome.org>
2025-11-08 20:11:00 -05:00
loggerMu.RLock()
defer loggerMu.RUnlock()
2020-01-08 20:45:07 -05:00
logger := logrus.NewEntry(defaultLogger)
return logger
}
func init() {
defaultLogger.Level = logrus.TraceLevel
_, file, _, ok := runtime.Caller(0)
if !ok {
return
}
rootPath = strings.TrimSuffix(file, "log/log.go")
}