Compare commits

..

4 commits

Author SHA1 Message Date
Deluan
1bfe5accc8 refactor(ui): only re-fetch no-store artwork stand-ins
Refresh events also name albums and songs for plays, stars and ratings, so
versioning every cached cover re-downloaded final, immutable images on each of
them. useImageUrl now marks a no-store response as a stand-in and re-fetches only
those entries, on remount as well as on a version change; final images are never
re-fetched, and cache entries no longer carry a version.

useArtworkRefresh now selects a primitive, so an event naming other items does
not re-render every mounted cover.
2026-09-14 21:45:46 -04:00
Deluan
2cc2ab1160 fix(artwork): reload animated covers in the UI once converted
A resized animated GIF is first served as a static stand-in while the conversion
runs in the background. The converted image is cached under the same URL, since the
cover hash identifies the original upload, so the UI kept painting the stand-in
from its blob cache until a full page reload.

When a background conversion is stored, the artwork service now broadcasts a
refreshResource event for the item (the service receives the event broker via
wire). In the UI, useArtworkRefresh returns when the last event named a record,
ignoring wildcards so scans do not refetch every cover, and useImageUrl takes that
as a version: a changed version re-fetches the cached URL, keeps showing the old
blob until the new one arrives, and keeps it if the re-fetch fails.
2026-09-14 21:31:14 -04:00
Deluan
c0da9fcd93 refactor(artwork): defer animated GIF conversion via a loader error
Replace cache.Transient with an artwork-local *deferredAnimation error returned
by resizedItem.Reader. The file cache already stores nothing when its loader
fails, so the generic cache no longer needs a transient mode. The error carries
the GIF bytes and the static stand-in, replacing the deferAnimated callback.

The cache's loader-error path now cancels the entry writer, so readers that
joined the miss fail with the loader's error instead of reading an empty entry,
and the removal no longer blocks on them.

The background job resizes the bytes it already holds instead of re-reading
them, orOriginal centralizes the serve-the-original fallback, and warmCache
shares the store-and-drain step with the worker's precache.
2026-09-14 20:14:13 -04:00
Deluan
bc16cafb91 fix(artwork): convert animated GIF covers in the background
Resized animated GIF covers were converted to animated WebP by ffmpeg inside
the image request, bound to its 10s timeout. On slow CPUs large GIFs never
finished: ffmpeg was killed, nothing was cached, and every request repeated
the work. With ffmpeg 5.1 the conversion also failed mid-stream when reading
the GIF from a pipe, after the stream was already handed to the client, so
there was no fallback.

Requests now get a static stand-in right away, served with no-store and no
validators, while the conversion runs in the background. A single
process-wide slot bounds these conversions, since they outlive the request
throttle; when it is busy nothing is queued and a later request retries.
The result, or a static fallback if ffmpeg fails, is cached under the same
key. ffmpeg output is now buffered so mid-stream failures fall back to a
static resize. The file cache gains cache.Transient for loader results that
must be served but not stored. Without an available cache, conversion stays
inline.
2026-09-14 19:36:50 -04:00
589 changed files with 9720 additions and 20129 deletions

View file

@ -92,23 +92,8 @@ jobs:
exit 1
fi
- name: Resolve OpenAPI tool versions
id: api-tools
run: echo "key=$(grep -E '^(VACUUM|OAPI_CODEGEN|OASDIFF)_VERSION' Makefile | tr -d ' \n')" >> "$GITHUB_OUTPUT"
- name: Cache OpenAPI tools
uses: actions/cache@v6
with:
path: bin
key: api-tools-${{ runner.os }}-${{ steps.api-tools.outputs.key }}
- name: Lint OpenAPI spec
run: make api-lint
- name: Run go generate
run: |
make api-gen
go generate ./...
run: go generate ./...
- name: Verify no changes from go generate
run: |
git status --porcelain
@ -117,12 +102,6 @@ jobs:
exit 1
fi
- name: Check for breaking OpenAPI changes
if: github.event_name == 'pull_request'
run: |
git fetch --no-tags --depth=1 origin ${{ github.event.pull_request.base.sha }}
make api-diff API_DIFF_BASE=${{ github.event.pull_request.base.sha }}
validate-migrations:
name: Validate DB migrations
runs-on: ubuntu-latest
@ -570,7 +549,7 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
for artifact in $(gh api repos/${{ github.repository }}/actions/runs/${{ github.run_id }}/artifacts | jq -r '.artifacts[] | select(.name | startswith("digests-")) | .id'); do
for artifact in $(gh api repos/${{ github.repository }}/actions/artifacts | jq -r '.artifacts[] | select(.name | startswith("digests-")) | .id'); do
gh api --method DELETE repos/${{ github.repository }}/actions/artifacts/$artifact
done

View file

@ -9,7 +9,6 @@ linters:
- asciicheck
- bidichk
- bodyclose
- containedctx
- copyloopvar
- dogsled
- durationcheck
@ -58,9 +57,6 @@ linters:
- gosec
path: _test\.go
text: "G703"
- path: _test\.go
linters:
- containedctx
- path-except: 'db/migrations/'
linters:
- forbidigo

View file

@ -20,11 +20,7 @@ IMAGE_PLATFORMS ?= $(shell echo $(SUPPORTED_PLATFORMS) | tr ',' '\n' | grep "lin
PLATFORMS ?= $(SUPPORTED_PLATFORMS)
DOCKER_TAG ?= deluan/navidrome:develop
GOLANGCI_LINT_VERSION ?= v2.14.0
VACUUM_VERSION ?= v0.30.6
OAPI_CODEGEN_VERSION ?= v2.8.0
OASDIFF_VERSION ?= v1.32.1
API_DIFF_BASE ?= origin/master
GOLANGCI_LINT_VERSION ?= v2.13.2
UI_SRC_FILES := $(shell find ui -type f -not -path "ui/build/*" -not -path "ui/node_modules/*")
@ -96,45 +92,6 @@ install-golangci-lint: ##@Development Install golangci-lint if not present
fi
.PHONY: install-golangci-lint
install-api-tools: ##@Development Install OpenAPI tools (vacuum, oapi-codegen, oasdiff) into ./bin
@STAMP=bin/.api-tools-$(VACUUM_VERSION)-$(OAPI_CODEGEN_VERSION)-$(OASDIFF_VERSION); \
if [ ! -f $$STAMP ] || [ ! -x bin/vacuum ] || [ ! -x bin/oapi-codegen ] || [ ! -x bin/oasdiff ]; then \
echo "Installing OpenAPI tools..."; \
GOBIN=$(CURDIR)/bin go install github.com/daveshanley/vacuum@$(VACUUM_VERSION) && \
GOBIN=$(CURDIR)/bin go install github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen@$(OAPI_CODEGEN_VERSION) && \
GOBIN=$(CURDIR)/bin go install github.com/oasdiff/oasdiff@$(OASDIFF_VERSION) && \
rm -f bin/.api-tools-* && touch $$STAMP; \
fi
.PHONY: install-api-tools
api-lint: install-api-tools ##@Development Lint the OpenAPI spec
./bin/vacuum lint -r api/.vacuum.yaml -d -q -b --fail-severity error api/openapi/openapi.yaml
.PHONY: api-lint
api-bundle: install-api-tools ##@Development Bundle the multi-file OpenAPI spec into api/bundled
./bin/vacuum bundle -q --composed -p api/openapi api/openapi/openapi.yaml api/bundled/openapi.yaml
./bin/vacuum bundle -q --composed --format json -p api/openapi api/openapi/openapi.yaml api/bundled/openapi.json
.PHONY: api-bundle
api-gen: api-bundle ##@Development Generate the API v1 server code from the bundled spec
./bin/oapi-codegen -config server/apiv1/oapi-codegen.yaml api/bundled/openapi.json
.PHONY: api-gen
api-diff: api-bundle ##@Development Fail on breaking OpenAPI changes against the merge-base with $(API_DIFF_BASE)
@git rev-parse --verify --quiet $(API_DIFF_BASE)^{commit} >/dev/null || { echo "Base ref $(API_DIFF_BASE) not found; set API_DIFF_BASE"; exit 1; }; \
BASE="$$(git merge-base HEAD $(API_DIFF_BASE) 2>/dev/null)"; \
if [ -z "$$BASE" ]; then \
echo "No merge-base with $(API_DIFF_BASE); falling back to its tip"; \
BASE=$(API_DIFF_BASE); \
fi; \
if git cat-file -e $$BASE:api/bundled/openapi.json 2>/dev/null; then \
git show $$BASE:api/bundled/openapi.json > $(CURDIR)/bin/api-base.json && \
./bin/oasdiff breaking $(CURDIR)/bin/api-base.json api/bundled/openapi.json --fail-on ERR --severity-levels api/.oasdiff-levels.txt; \
else \
echo "No bundled spec at $$BASE; skipping breaking-change check"; \
fi
.PHONY: api-diff
lint: install-golangci-lint ##@Development Lint Go code
PATH=./bin:$$PATH golangci-lint run --timeout 5m
.PHONY: lint
@ -154,7 +111,7 @@ wire: check_go_env ##@Development Update Dependency Injection
go tool wire gen -tags="$$(echo '$(GO_BUILD_TAGS)' | tr ',' ' ')" ./...
.PHONY: wire
gen: check_go_env api-gen ##@Development Run go generate for code generation
gen: check_go_env ##@Development Run go generate for code generation
go generate ./...
cd plugins/cmd/ndpgen && go run . -shared-types -input=../../types -output=../../pdk -go -rust
cd plugins/cmd/ndpgen && go run . -host-wrappers -input=../../host -package=host -shared=../../types

View file

@ -90,7 +90,7 @@ var _ = Describe("Extractor", func() {
info.FileInfo = testFileInfo{FileInfo: fileInfo}
metadata := metadata.New(path, info)
return new(metadata.ToMediaFile(model.Library{ID: 1}, "folderID"))
return new(metadata.ToMediaFile(1, "folderID"))
}
BeforeEach(func() {

View file

@ -241,10 +241,6 @@ func (l *lastfmAgent) GetSimilarSongsByTrack(ctx context.Context, id, name, arti
var (
artistOpenGraphQuery = cascadia.MustCompile(`html > head > meta[property="og:image"]`)
artistIgnoredImage = "2a96cbd8b46e442fc41c2b86b821562f" // Last.fm artist placeholder image name
// Not a RetryLaterError on purpose: parking the agent would also stall its API-backed
// methods, which the page block does not affect.
errNoArtistPage = errors.New("no artist image in Last.fm page")
)
func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string) ([]agents.ExternalImage, error) {
@ -271,9 +267,7 @@ func (l *lastfmAgent) GetArtistImages(ctx context.Context, _, name, mbid string)
var res []agents.ExternalImage
n := cascadia.Query(node, artistOpenGraphQuery)
if n == nil {
// A real artist page always has og:image; its absence means a bot challenge or a redesign.
log.Warn(ctx, "Last.fm did not return a usable artist page", "name", name, "url", a.URL)
return nil, errNoArtistPage
return res, nil
}
for _, attr := range n.Attr {
if attr.Key != "content" {

View file

@ -357,7 +357,7 @@ var _ = Describe("lastfmAgent", func() {
var httpClient *tests.FakeHttpClient
var track *model.MediaFile
BeforeEach(func() {
_ = ds.UserProps().Put(ctx, "user-1", sessionKeyProperty, "SK-1")
_ = ds.UserProps(ctx).Put("user-1", sessionKeyProperty, "SK-1")
httpClient = &tests.FakeHttpClient{}
client := newClient("API_KEY", "SECRET", httpClient)
agent = lastFMConstructor(ds)
@ -648,41 +648,18 @@ var _ = Describe("lastfmAgent", func() {
Expect(images).To(BeEmpty())
})
It("errors when the page has no meta tags", func() {
It("returns empty list if page has no meta tags", func() {
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.no_meta.html")
httpClient.Res = http.Response{Body: fScraper, StatusCode: 200}
_, err := agent.GetArtistImages(ctx, "123", "U2", "")
Expect(err).To(MatchError(errNoArtistPage))
})
It("errors when Last.fm serves a bot challenge page", func() {
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.challenge.html")
httpClient.Res = http.Response{Body: fScraper, StatusCode: 200}
images, err := agent.GetArtistImages(ctx, "123", "U2", "")
Expect(err).To(MatchError(errNoArtistPage))
Expect(err).ToNot(HaveOccurred())
Expect(images).To(BeEmpty())
})
It("does not park the agent: the failure is not a retry-later", func() {
// A RetryLaterError would cool down the agent's API-backed methods too.
fApi, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
apiClient.Res = http.Response{Body: fApi, StatusCode: 200}
fScraper, _ := os.Open("tests/fixtures/lastfm.artist.page.challenge.html")
httpClient.Res = http.Response{Body: fScraper, StatusCode: 200}
_, err := agent.GetArtistImages(ctx, "123", "U2", "")
Expect(errors.Is(err, agents.ErrRetryLater)).To(BeFalse())
})
It("returns error if API call fails", func() {
apiClient.Err = errors.New("api error")
_, err := agent.GetArtistImages(ctx, "123", "U2", "")

View file

@ -50,7 +50,7 @@ var _ = Describe("auth_router", func() {
})
storedSessionKey := func(userID string) string {
key, _ := userProps.Get(GinkgoT().Context(), userID, sessionKeyProperty)
key, _ := userProps.Get(userID, sessionKeyProperty)
return key
}

View file

@ -49,12 +49,6 @@ type client struct {
hc httpDoer
}
// escapePlus works around Last.fm decoding artist.* and track.* params twice, turning "+" into a space.
// album.getInfo decodes only once, so it must not use this.
func escapePlus(s string) string {
return strings.ReplaceAll(s, "+", "%2B")
}
func (c *client) albumGetInfo(ctx context.Context, name string, artist string, mbid string, lang string) (*Album, error) {
params := url.Values{}
params.Add("method", "album.getInfo")
@ -72,7 +66,7 @@ func (c *client) albumGetInfo(ctx context.Context, name string, artist string, m
func (c *client) artistGetInfo(ctx context.Context, name string, lang string) (*Artist, error) {
params := url.Values{}
params.Add("method", "artist.getInfo")
params.Add("artist", escapePlus(name))
params.Add("artist", name)
params.Add("lang", lang)
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {
@ -84,7 +78,7 @@ func (c *client) artistGetInfo(ctx context.Context, name string, lang string) (*
func (c *client) artistGetSimilar(ctx context.Context, name string, limit int) (*SimilarArtists, error) {
params := url.Values{}
params.Add("method", "artist.getSimilar")
params.Add("artist", escapePlus(name))
params.Add("artist", name)
params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {
@ -96,7 +90,7 @@ func (c *client) artistGetSimilar(ctx context.Context, name string, limit int) (
func (c *client) artistGetTopTracks(ctx context.Context, name string, limit int) (*TopTracks, error) {
params := url.Values{}
params.Add("method", "artist.getTopTracks")
params.Add("artist", escapePlus(name))
params.Add("artist", name)
params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {
@ -108,8 +102,8 @@ func (c *client) artistGetTopTracks(ctx context.Context, name string, limit int)
func (c *client) trackGetSimilar(ctx context.Context, name, artist string, limit int) (*SimilarTracks, error) {
params := url.Values{}
params.Add("method", "track.getSimilar")
params.Add("track", escapePlus(name))
params.Add("artist", escapePlus(artist))
params.Add("track", name)
params.Add("artist", artist)
params.Add("limit", strconv.Itoa(limit))
response, err := c.makeRequest(ctx, http.MethodGet, params, false)
if err != nil {

View file

@ -35,15 +35,6 @@ var _ = Describe("client", func() {
Expect(album.Name).To(Equal("Believe"))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?album=Believe&api_key=API_KEY&artist=U2&format=json&lang=pt&mbid=mbid-1234&method=album.getInfo"))
})
It("does not double-encode plus signs", func() {
f, _ := os.Open("tests/fixtures/lastfm.album.getinfo.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.albumGetInfo(context.Background(), "Lungs", "Florence + the Machine", "", "en")
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("Florence + the Machine"))
})
})
Describe("artistGetInfo", func() {
@ -57,15 +48,6 @@ var _ = Describe("client", func() {
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&lang=pt&method=artist.getInfo"))
})
It("double-encodes plus signs in the artist name", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getinfo.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.artistGetInfo(context.Background(), "Florence + the Machine", "en")
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("Florence %2B the Machine"))
})
It("fails if Last.fm returns an http status != 200", func() {
httpClient.Res = http.Response{
Body: io.NopCloser(bytes.NewBufferString(`Internal Server Error`)),
@ -125,15 +107,6 @@ var _ = Describe("client", func() {
Expect(len(similar.Artists)).To(Equal(2))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getSimilar"))
})
It("double-encodes plus signs in the artist name", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.getsimilar.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.artistGetSimilar(context.Background(), "+44", 2)
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("%2B44"))
})
})
Describe("artistGetTopTracks", func() {
@ -146,15 +119,6 @@ var _ = Describe("client", func() {
Expect(len(top.Track)).To(Equal(2))
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=U2&format=json&limit=2&method=artist.getTopTracks"))
})
It("double-encodes plus signs in the artist name", func() {
f, _ := os.Open("tests/fixtures/lastfm.artist.gettoptracks.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.artistGetTopTracks(context.Background(), "C+C Music Factory", 2)
Expect(err).ToNot(HaveOccurred())
Expect(httpClient.SavedRequest.URL.Query().Get("artist")).To(Equal("C%2BC Music Factory"))
})
})
Describe("trackGetSimilar", func() {
@ -171,17 +135,6 @@ var _ = Describe("client", func() {
Expect(httpClient.SavedRequest.URL.String()).To(Equal(apiBaseUrl + "?api_key=API_KEY&artist=Depeche+Mode&format=json&limit=5&method=track.getSimilar&track=Just+Can%27t+Get+Enough"))
})
It("double-encodes plus signs in the track and artist names", func() {
f, _ := os.Open("tests/fixtures/lastfm.track.getsimilar.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}
_, err := client.trackGetSimilar(context.Background(), "1+1", "Queen + Paul Rodgers", 5)
Expect(err).ToNot(HaveOccurred())
query := httpClient.SavedRequest.URL.Query()
Expect(query.Get("track")).To(Equal("1%2B1"))
Expect(query.Get("artist")).To(Equal("Queen %2B Paul Rodgers"))
})
It("returns empty list when no similar tracks found", func() {
f, _ := os.Open("tests/fixtures/lastfm.track.getsimilar.unknown.json")
httpClient.Res = http.Response{Body: f, StatusCode: 200}

View file

@ -30,7 +30,7 @@ var _ = Describe("listenBrainzAgent", func() {
BeforeEach(func() {
ds = &tests.MockDataStore{}
ctx = context.Background()
_ = ds.UserProps().Put(ctx, "user-1", sessionKeyProperty, "SK-1")
_ = ds.UserProps(ctx).Put("user-1", sessionKeyProperty, "SK-1")
httpClient = &tests.FakeHttpClient{}
agent = listenBrainzConstructor(ds)
agent.client = newClient("http://localhost:8080", httpClient)

View file

@ -1 +0,0 @@
response-property-enum-value-added INFO

View file

@ -1,155 +0,0 @@
extends: [[spectral:oas, recommended]]
rules:
# vacuum's `enumeration` function mis-resolves hyphenated `then.field` names,
# so the value check below targets `x-module` via `given` instead.
nd-operation-x-module-required:
description: Every operation belongs to exactly one capability module.
severity: error
given: $.paths[*][get,put,post,delete,patch]
then:
field: x-module
function: truthy
nd-operation-x-module:
description: Every operation's capability module is one of the known values.
severity: error
given: $.paths[*][get,put,post,delete,patch]['x-module']
then:
function: enumeration
functionOptions:
values:
- core
- streaming
- download
- artwork
- lyrics
- transcoding
- annotations
- playback
- queue
- custom-tags
- grouping
- playlists
- smart-playlists
- sync
- events
- jukebox
- sharing
- radio
- admin
nd-operation-stability-level-required:
description: Every operation declares its stability level, which the breaking-change gate relies on.
severity: error
given: $.paths[*][get,put,post,delete,patch]
then:
field: x-stability-level
function: truthy
nd-operation-stability-level:
description: Every operation's stability level is alpha, beta, or stable.
severity: error
given: $.paths[*][get,put,post,delete,patch]['x-stability-level']
then:
function: enumeration
functionOptions:
values:
- alpha
- beta
- stable
nd-operation-required-fields:
description: Operations need a stable operationId, summary, description and tags.
severity: error
given: $.paths[*][get,put,post,delete,patch]
then:
- field: operationId
function: truthy
- field: summary
function: truthy
- field: description
function: truthy
- field: tags
function: truthy
# Our schemas live in path/response files, not root components, so this
# walks every resolved `properties` map in the document via `$..` instead.
nd-schema-property-descriptions:
description: Every schema property is documented.
severity: error
given: $..properties[*]
then:
field: description
function: truthy
# patternProperties covers the full 4xx/5xx range; needs an explicit
# `properties` entry too, or `additionalProperties: false` rejects it.
nd-error-responses-are-problems:
description: 4xx and 5xx responses use application/problem+json.
severity: error
given: $.paths[*][*].responses
then:
function: schema
functionOptions:
forceValidationOnCurrentNode: true
schema:
type: object
patternProperties:
"^[45][0-9][0-9]$":
type: object
required: [content]
properties:
content:
type: object
properties:
application/problem+json: {}
required: [application/problem+json]
additionalProperties: false
# Same filter limitation applies here: "is this a list endpoint" is expressed
# as a JSON Schema if/then on the operation object instead of a `given` filter.
nd-list-endpoints-paginate:
description: List endpoints declare the shared offset and limit parameters.
severity: error
given: $.paths[*].get
then:
function: schema
functionOptions:
forceValidationOnCurrentNode: true
schema:
type: object
if:
required: [responses]
properties:
responses:
type: object
required: ['200']
properties:
'200':
type: object
required: [content]
properties:
content:
type: object
required: [application/json]
properties:
application/json:
type: object
required: [schema]
properties:
schema:
type: object
required: [properties]
properties:
properties:
type: object
required: [items]
then:
required: [parameters]
properties:
parameters:
type: array
allOf:
- contains:
type: object
properties:
name:
const: offset
- contains:
type: object
properties:
name:
const: limit

View file

@ -1,262 +0,0 @@
{
"openapi": "3.0.3",
"info": {
"title": "Navidrome API",
"version": "1.0.0",
"description": "Navidrome API v1. Spec-first, additive within v1. Clients discover implemented\ncapability modules through `GET /server` and never sniff versions.\n\nEnums are open: new values may be added to any enum within v1. Clients must\naccept values they do not recognise instead of failing.\n\nEvery operation declares `x-stability-level`: `alpha` operations may change or\ndisappear without notice, `beta` and `stable` operations only change additively.\nA level is only ever raised, never lowered.\n\n`HEAD` is accepted wherever `GET` is. A `405` response lists the allowed methods\nin its `Allow` header.\n",
"license": {
"name": "GPL-3.0",
"url": "https://www.gnu.org/licenses/gpl-3.0.html"
}
},
"servers": [
{
"url": "/api/v1"
}
],
"tags": [
{
"name": "server",
"description": "Server discovery and the published OpenAPI document."
}
],
"paths": {
"/server": {
"get": {
"operationId": "getServerInfo",
"x-module": "core",
"x-stability-level": "alpha",
"tags": [
"server"
],
"summary": "Describe the server",
"description": "Returns the public server description. No authentication required.\nAuthenticated requests will additionally receive the implemented capability modules\nonce authentication is available.\n",
"responses": {
"200": {
"description": "Server description.",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/ServerInfo"
}
}
}
},
"500": {
"$ref": "#/components/responses/InternalError"
}
}
}
},
"/openapi.json": {
"get": {
"operationId": "getOpenAPISpecJSON",
"x-module": "core",
"x-stability-level": "alpha",
"tags": [
"server"
],
"summary": "Get the OpenAPI document (JSON)",
"description": "The bundled OpenAPI document of the running server version. Supports ETag revalidation.",
"responses": {
"200": {
"description": "The OpenAPI document.",
"headers": {
"ETag": {
"$ref": "#/components/headers/ETag"
}
},
"content": {
"application/json": {
"schema": {
"type": "object",
"description": "OpenAPI 3.0 document."
}
}
}
},
"304": {
"$ref": "#/components/responses/NotModified"
}
}
}
},
"/openapi.yaml": {
"get": {
"operationId": "getOpenAPISpecYAML",
"x-module": "core",
"x-stability-level": "alpha",
"tags": [
"server"
],
"summary": "Get the OpenAPI document (YAML)",
"description": "The bundled OpenAPI document of the running server version. Supports ETag revalidation.",
"responses": {
"200": {
"description": "The OpenAPI document.",
"headers": {
"ETag": {
"$ref": "#/components/headers/ETag"
}
},
"content": {
"application/yaml": {
"schema": {
"type": "object",
"description": "OpenAPI 3.0 document."
}
}
}
},
"304": {
"$ref": "#/components/responses/NotModified"
}
}
}
}
},
"components": {
"securitySchemes": {
"bearerAuth": {
"type": "http",
"scheme": "bearer",
"bearerFormat": "JWT",
"description": "Short-lived access token minted from a device grant. Not yet applied to any operation."
}
},
"schemas": {
"ServerInfo": {
"type": "object",
"description": "Public server description. Everything an add-server screen needs before login.",
"required": [
"name",
"serverVersion",
"specVersion",
"setupRequired",
"loginMethods"
],
"properties": {
"name": {
"type": "string",
"description": "Human-readable server product name."
},
"serverVersion": {
"type": "string",
"description": "Version of the running server build."
},
"specVersion": {
"type": "string",
"description": "Version of the OpenAPI document this server implements."
},
"setupRequired": {
"type": "boolean",
"description": "True until the first admin user has been created."
},
"loginMethods": {
"type": "array",
"description": "Login methods this server accepts. New methods may be added; clients ignore values they do not recognise.",
"items": {
"type": "string",
"enum": [
"password"
]
}
}
}
},
"Problem": {
"type": "object",
"description": "RFC 9457 problem details, returned for every 4xx and 5xx response.",
"required": [
"title",
"status",
"code"
],
"properties": {
"type": {
"type": "string",
"description": "URI reference identifying the problem type. Omitted while the problem carries no semantics\nbeyond its HTTP status code, which RFC 9457 defines as `about:blank`. Problems with their\nown semantics get their own URI; switch on `code` instead.\n"
},
"title": {
"type": "string",
"description": "Short human-readable summary, the same for all occurrences of this problem type."
},
"status": {
"type": "integer",
"description": "HTTP status code of this response."
},
"detail": {
"type": "string",
"description": "Human-readable explanation specific to this occurrence. Omitted for internal errors."
},
"code": {
"type": "string",
"description": "Machine-readable error code, and the value clients switch on. New codes may be added.",
"enum": [
"validation",
"unauthorized",
"forbidden",
"not_found",
"method_not_allowed",
"unavailable",
"internal"
]
},
"errors": {
"type": "array",
"description": "Per-field failures. Present only when `code` is `validation`.",
"items": {
"$ref": "#/components/schemas/ValidationError"
}
}
}
},
"ValidationError": {
"type": "object",
"description": "One field-level validation failure.",
"required": [
"field",
"message"
],
"properties": {
"field": {
"type": "string",
"description": "Name of the offending query parameter, path parameter, or body field (dotted for nested)."
},
"message": {
"type": "string",
"description": "Why the value was rejected."
}
}
}
},
"responses": {
"InternalError": {
"description": "Unexpected server failure. Details are in the server log.",
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/Problem"
}
}
}
},
"NotModified": {
"description": "Not modified.",
"headers": {
"ETag": {
"$ref": "#/components/headers/ETag"
}
}
}
},
"headers": {
"ETag": {
"description": "Entity tag for `If-None-Match` revalidation.",
"schema": {
"type": "string"
}
}
}
}
}

View file

@ -1,194 +0,0 @@
openapi: 3.0.3
info:
title: Navidrome API
version: 1.0.0
description: |
Navidrome API v1. Spec-first, additive within v1. Clients discover implemented
capability modules through `GET /server` and never sniff versions.
Enums are open: new values may be added to any enum within v1. Clients must
accept values they do not recognise instead of failing.
Every operation declares `x-stability-level`: `alpha` operations may change or
disappear without notice, `beta` and `stable` operations only change additively.
A level is only ever raised, never lowered.
`HEAD` is accepted wherever `GET` is. A `405` response lists the allowed methods
in its `Allow` header.
license:
name: GPL-3.0
url: https://www.gnu.org/licenses/gpl-3.0.html
servers:
- url: /api/v1
tags:
- name: server
description: Server discovery and the published OpenAPI document.
paths:
/server:
get:
operationId: getServerInfo
x-module: core
x-stability-level: alpha
tags: [server]
summary: Describe the server
description: |
Returns the public server description. No authentication required.
Authenticated requests will additionally receive the implemented capability modules
once authentication is available.
responses:
'200':
description: Server description.
content:
application/json:
schema:
$ref: '#/components/schemas/ServerInfo'
'500':
$ref: '#/components/responses/InternalError'
/openapi.json:
get:
operationId: getOpenAPISpecJSON
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (JSON)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: '#/components/headers/ETag'
content:
application/json:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: '#/components/responses/NotModified'
/openapi.yaml:
get:
operationId: getOpenAPISpecYAML
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (YAML)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: '#/components/headers/ETag'
content:
application/yaml:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: '#/components/responses/NotModified'
components:
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: Short-lived access token minted from a device grant. Not yet applied to any operation.
schemas:
ServerInfo:
type: object
description: Public server description. Everything an add-server screen needs before login.
required:
- name
- serverVersion
- specVersion
- setupRequired
- loginMethods
properties:
name:
type: string
description: Human-readable server product name.
serverVersion:
type: string
description: Version of the running server build.
specVersion:
type: string
description: Version of the OpenAPI document this server implements.
setupRequired:
type: boolean
description: True until the first admin user has been created.
loginMethods:
type: array
description: Login methods this server accepts. New methods may be added; clients ignore values they do not recognise.
items:
type: string
enum:
- password
Problem:
type: object
description: RFC 9457 problem details, returned for every 4xx and 5xx response.
required:
- title
- status
- code
properties:
type:
type: string
description: |
URI reference identifying the problem type. Omitted while the problem carries no semantics
beyond its HTTP status code, which RFC 9457 defines as `about:blank`. Problems with their
own semantics get their own URI; switch on `code` instead.
title:
type: string
description: Short human-readable summary, the same for all occurrences of this problem type.
status:
type: integer
description: HTTP status code of this response.
detail:
type: string
description: Human-readable explanation specific to this occurrence. Omitted for internal errors.
code:
type: string
description: Machine-readable error code, and the value clients switch on. New codes may be added.
enum:
- validation
- unauthorized
- forbidden
- not_found
- method_not_allowed
- unavailable
- internal
errors:
type: array
description: Per-field failures. Present only when `code` is `validation`.
items:
$ref: '#/components/schemas/ValidationError'
ValidationError:
type: object
description: One field-level validation failure.
required:
- field
- message
properties:
field:
type: string
description: Name of the offending query parameter, path parameter, or body field (dotted for nested).
message:
type: string
description: Why the value was rejected.
responses:
InternalError:
description: Unexpected server failure. Details are in the server log.
content:
application/problem+json:
schema:
$ref: '#/components/schemas/Problem'
NotModified:
description: Not modified.
headers:
ETag:
$ref: '#/components/headers/ETag'
headers:
ETag:
description: Entity tag for `If-None-Match` revalidation.
schema:
type: string

View file

@ -1,35 +0,0 @@
package api
import (
_ "embed"
"encoding/json"
"sync"
)
//go:embed bundled/openapi.json
var specJSON []byte
//go:embed bundled/openapi.yaml
var specYAML []byte
func SpecJSON() []byte {
return specJSON
}
func SpecYAML() []byte {
return specYAML
}
var specVersion = sync.OnceValue(func() string {
var doc struct {
Info struct {
Version string `json:"version"`
} `json:"info"`
}
_ = json.Unmarshal(SpecJSON(), &doc)
return doc.Info.Version
})
func SpecVersion() string {
return specVersion()
}

View file

@ -1,37 +0,0 @@
package api_test
import (
"os"
"github.com/getkin/kin-openapi/openapi3"
"github.com/navidrome/navidrome/api"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"gopkg.in/yaml.v3"
)
var _ = Describe("Bundled spec", func() {
It("embeds a valid OpenAPI 3 document", func() {
doc, err := openapi3.NewLoader().LoadFromData(api.SpecJSON())
Expect(err).ToNot(HaveOccurred())
Expect(doc.Validate(GinkgoT().Context())).To(Succeed())
Expect(doc.Paths.Find("/server")).ToNot(BeNil())
})
It("embeds the YAML variant", func() {
var doc map[string]any
Expect(yaml.Unmarshal(api.SpecYAML(), &doc)).To(Succeed())
Expect(doc).To(HaveKey("paths"))
})
It("reports the version from the bundle, matching the source root document", func() {
src, err := os.ReadFile("api/openapi/openapi.yaml")
Expect(err).ToNot(HaveOccurred())
var root struct {
Info struct{ Version string } `yaml:"info"`
}
Expect(yaml.Unmarshal(src, &root)).To(Succeed())
Expect(api.SpecVersion()).To(Equal(root.Info.Version))
Expect(api.SpecVersion()).ToNot(BeEmpty())
})
})

View file

@ -1,3 +0,0 @@
description: Entity tag for `If-None-Match` revalidation.
schema:
type: string

View file

@ -1,9 +0,0 @@
name: limit
in: query
description: Maximum number of items to return.
required: false
schema:
type: integer
minimum: 1
maximum: 2000
default: 100

View file

@ -1,8 +0,0 @@
name: offset
in: query
description: Zero-based index of the first item to return.
required: false
schema:
type: integer
minimum: 0
default: 0

View file

@ -1,5 +0,0 @@
description: The request is malformed or fails validation.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,5 +0,0 @@
description: The caller is authenticated but not allowed to do this.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,5 +0,0 @@
description: Unexpected server failure. Details are in the server log.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,5 +0,0 @@
description: No such resource or endpoint.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,4 +0,0 @@
description: Not modified.
headers:
ETag:
$ref: ../headers/ETag.yaml

View file

@ -1,5 +0,0 @@
description: Missing, invalid, or expired credentials.
content:
application/problem+json:
schema:
$ref: ../schemas/Problem.yaml

View file

@ -1,13 +0,0 @@
type: object
description: Pagination metadata carried by every list response.
required: [total, offset, limit]
properties:
total:
type: integer
description: Total number of items matching the request, ignoring pagination.
offset:
type: integer
description: Zero-based index of the first returned item.
limit:
type: integer
description: Maximum number of items in this page.

View file

@ -1,35 +0,0 @@
type: object
description: RFC 9457 problem details, returned for every 4xx and 5xx response.
required: [title, status, code]
properties:
type:
type: string
description: |
URI reference identifying the problem type. Omitted while the problem carries no semantics
beyond its HTTP status code, which RFC 9457 defines as `about:blank`. Problems with their
own semantics get their own URI; switch on `code` instead.
title:
type: string
description: Short human-readable summary, the same for all occurrences of this problem type.
status:
type: integer
description: HTTP status code of this response.
detail:
type: string
description: Human-readable explanation specific to this occurrence. Omitted for internal errors.
code:
type: string
description: Machine-readable error code, and the value clients switch on. New codes may be added.
enum:
- validation
- unauthorized
- forbidden
- not_found
- method_not_allowed
- unavailable
- internal
errors:
type: array
description: Per-field failures. Present only when `code` is `validation`.
items:
$ref: ./ValidationError.yaml

View file

@ -1,22 +0,0 @@
type: object
description: Public server description. Everything an add-server screen needs before login.
required: [name, serverVersion, specVersion, setupRequired, loginMethods]
properties:
name:
type: string
description: Human-readable server product name.
serverVersion:
type: string
description: Version of the running server build.
specVersion:
type: string
description: Version of the OpenAPI document this server implements.
setupRequired:
type: boolean
description: True until the first admin user has been created.
loginMethods:
type: array
description: Login methods this server accepts. New methods may be added; clients ignore values they do not recognise.
items:
type: string
enum: [password]

View file

@ -1,10 +0,0 @@
type: object
description: One field-level validation failure.
required: [field, message]
properties:
field:
type: string
description: Name of the offending query parameter, path parameter, or body field (dotted for nested).
message:
type: string
description: Why the value was rejected.

View file

@ -1,39 +0,0 @@
openapi: 3.0.3
info:
title: Navidrome API
version: 1.0.0
description: |
Navidrome API v1. Spec-first, additive within v1. Clients discover implemented
capability modules through `GET /server` and never sniff versions.
Enums are open: new values may be added to any enum within v1. Clients must
accept values they do not recognise instead of failing.
Every operation declares `x-stability-level`: `alpha` operations may change or
disappear without notice, `beta` and `stable` operations only change additively.
A level is only ever raised, never lowered.
`HEAD` is accepted wherever `GET` is. A `405` response lists the allowed methods
in its `Allow` header.
license:
name: GPL-3.0
url: https://www.gnu.org/licenses/gpl-3.0.html
servers:
- url: /api/v1
tags:
- name: server
description: Server discovery and the published OpenAPI document.
paths:
/server:
$ref: ./paths/server.yaml
/openapi.json:
$ref: ./paths/openapi.yaml#/json
/openapi.yaml:
$ref: ./paths/openapi.yaml#/yaml
components:
securitySchemes:
bearerAuth:
type: http
scheme: bearer
bearerFormat: JWT
description: Short-lived access token minted from a device grant. Not yet applied to any operation.

View file

@ -1,42 +0,0 @@
json:
get:
operationId: getOpenAPISpecJSON
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (JSON)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: ../components/headers/ETag.yaml
content:
application/json:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: ../components/responses/NotModified.yaml
yaml:
get:
operationId: getOpenAPISpecYAML
x-module: core
x-stability-level: alpha
tags: [server]
summary: Get the OpenAPI document (YAML)
description: The bundled OpenAPI document of the running server version. Supports ETag revalidation.
responses:
'200':
description: The OpenAPI document.
headers:
ETag:
$ref: ../components/headers/ETag.yaml
content:
application/yaml:
schema:
type: object
description: OpenAPI 3.0 document.
'304':
$ref: ../components/responses/NotModified.yaml

View file

@ -1,19 +0,0 @@
get:
operationId: getServerInfo
x-module: core
x-stability-level: alpha
tags: [server]
summary: Describe the server
description: |
Returns the public server description. No authentication required.
Authenticated requests will additionally receive the implemented capability modules
once authentication is available.
responses:
'200':
description: Server description.
content:
application/json:
schema:
$ref: ../components/schemas/ServerInfo.yaml
'500':
$ref: ../components/responses/InternalError.yaml

View file

@ -174,28 +174,28 @@ func queueTotal(stats []model.ArtworkQueueStat) int64 {
}
func collectStatus(ctx context.Context, ds model.DataStore) (statusReport, error) {
q := ds.ArtworkQueue()
q := ds.ArtworkQueue(ctx)
var rep statusReport
var err error
if rep.queue, err = q.CountQueued(ctx, nil, nil); err != nil {
if rep.queue, err = q.CountQueued(nil, nil); err != nil {
return rep, fmt.Errorf("breaking the artwork queue down by kind: %w", err)
}
for _, k := range artwork.ReprocessKinds {
sources, err := q.SourcesInUse(ctx, k)
sources, err := q.SourcesInUse(k)
if err != nil {
return rep, fmt.Errorf("listing the sources in use by %s artwork: %w", k, err)
}
slices.Sort(sources)
for _, s := range sources {
n, err := q.CountBySource(ctx, k, []string{s})
n, err := q.CountBySource(k, []string{s})
if err != nil {
return rep, fmt.Errorf("counting %s artwork resolved from %s: %w", k, displaySource(s), err)
}
rep.sources = append(rep.sources, sourceCount{kind: k, source: s, count: n})
// An absent state is exactly a row with no source, so it needs no second query.
if s == "" {
failed, err := q.CountBySource(ctx, k, []string{model.ArtworkSourceFailed})
failed, err := q.CountBySource(k, []string{model.ArtworkSourceFailed})
if err != nil {
return rep, fmt.Errorf("counting failed %s artwork: %w", k, err)
}
@ -205,7 +205,7 @@ func collectStatus(ctx context.Context, ds model.DataStore) (statusReport, error
}
rep.current, rep.inputs = artwork.ConfigFingerprint(), artwork.FingerprintInputs()
if rep.stored, err = ds.Property().DefaultGet(ctx, consts.ArtConfFingerprintPropertyKey, ""); err != nil {
if rep.stored, err = ds.Property(ctx).DefaultGet(consts.ArtConfFingerprintPropertyKey, ""); err != nil {
return rep, fmt.Errorf("reading the stored artwork fingerprint: %w", err)
}
return rep, nil
@ -442,13 +442,13 @@ func promptConfirm(in io.Reader, verb string) confirmFunc {
// validateSources rejects a typo'd source: matching nothing silently reads as "nothing to do" when
// it means the filter was wrong. Checked table-wide, so a filter is never a typo for one --kind only.
func validateSources(ctx context.Context, q model.ArtworkQueueRepository, sources []string) error {
func validateSources(q model.ArtworkQueueRepository, sources []string) error {
if len(sources) == 0 {
return nil
}
var inUse []string
for _, k := range artwork.ReprocessKinds {
found, err := q.SourcesInUse(ctx, k)
found, err := q.SourcesInUse(k)
if err != nil {
return fmt.Errorf("listing the sources in use by %s artwork: %w", k, err)
}
@ -475,8 +475,8 @@ func validateSources(ctx context.Context, q model.ArtworkQueueRepository, source
// actually inserted; the two differ because an already-queued row is left untouched.
func reprocessArtwork(ctx context.Context, ds model.DataStore, kinds []model.Kind, sources []string,
imageAgents artwork.ImageAgentCount, dryRun bool, confirm confirmFunc, out io.Writer) error {
q := ds.ArtworkQueue()
if err := validateSources(ctx, q, sources); err != nil {
q := ds.ArtworkQueue(ctx)
if err := validateSources(q, sources); err != nil {
return err
}
@ -495,7 +495,7 @@ func reprocessArtwork(ctx context.Context, ds model.DataStore, kinds []model.Kin
matched := make([]int64, len(kinds))
var total, external int64
for i, k := range kinds {
n, err := q.CountBySource(ctx, k, sources)
n, err := q.CountBySource(k, sources)
if err != nil {
return fmt.Errorf("counting %s artwork: %w", k, err)
}
@ -523,7 +523,7 @@ func reprocessArtwork(ctx context.Context, ds model.DataStore, kinds []model.Kin
if matched[i] == 0 {
continue
}
n, err := q.EnqueueBySource(ctx, k, sources, model.ArtworkPriorityRecheck)
n, err := q.EnqueueBySource(k, sources, model.ArtworkPriorityRecheck)
if err != nil {
return fmt.Errorf("queueing %s artwork: %w", k, err)
}
@ -590,8 +590,8 @@ func parseAll[T comparable](values []string, parse func(string) (T, error)) ([]T
func cancelArtwork(ctx context.Context, ds model.DataStore, kinds []model.Kind, priorities []int,
dryRun bool, confirm confirmFunc, out io.Writer) error {
q := ds.ArtworkQueue()
matched, err := q.CountQueued(ctx, kinds, priorities)
q := ds.ArtworkQueue(ctx)
matched, err := q.CountQueued(kinds, priorities)
if err != nil {
return fmt.Errorf("counting queued artwork: %w", err)
}
@ -612,7 +612,7 @@ func cancelArtwork(ctx context.Context, ds model.DataStore, kinds []model.Kind,
return nil
}
cancelled, err := q.PurgeQueued(ctx, kinds, priorities)
cancelled, err := q.PurgeQueued(kinds, priorities)
if err != nil {
return fmt.Errorf("cancelling queued artwork: %w", err)
}
@ -984,11 +984,11 @@ func runExplain(ctx context.Context, args []string) {
}
rep := explainReport{kind: kind, id: id, name: name}
if artwork.KeepsState(kind) {
rep.stored, err = ds.Artwork().GetItemArtwork(ctx, kind, id, model.ImageTypePrimary)
rep.stored, err = ds.Artwork(ctx).GetItemArtwork(kind, id, model.ImageTypePrimary)
if err != nil && !errors.Is(err, model.ErrNotFound) {
log.Fatal(ctx, "Failed to read artwork state", "kind", kind, "id", id, err)
}
rep.queued, err = ds.ArtworkQueue().Get(ctx, kind, id, model.ImageTypePrimary)
rep.queued, err = ds.ArtworkQueue(ctx).Get(kind, id, model.ImageTypePrimary)
if err != nil && !errors.Is(err, model.ErrNotFound) {
log.Fatal(ctx, "Failed to read the artwork queue", "kind", kind, "id", id, err)
}

View file

@ -491,17 +491,17 @@ var _ = Describe("explain/reprocess source round trip", func() {
It("names the absent state as reprocess --source accepts it", func() {
ds := &tests.MockDataStore{}
art := ds.Artwork().(*tests.MockArtworkRepo)
Expect(art.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: model.KindArtistArtwork.Prefix(),
art := ds.Artwork(ctx).(*tests.MockArtworkRepo)
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: model.KindArtistArtwork.Prefix(),
ItemID: "ar-1", ImageType: model.ImageTypePrimary})).To(Succeed())
shown := storedSource(formatExplain(explainReport{kind: model.KindArtistArtwork, id: "ar-1",
stored: &model.ItemArtwork{AttemptedAt: time.Now()}}))
q := ds.ArtworkQueue()
Expect(validateSources(ctx, q, repositorySources([]string{shown}))).To(Succeed(),
q := ds.ArtworkQueue(ctx)
Expect(validateSources(q, repositorySources([]string{shown}))).To(Succeed(),
"explain's spelling of a source must be pasteable into --source")
Expect(validateSources(ctx, q, repositorySources([]string{"(" + shown + ")"}))).ToNot(Succeed(),
Expect(validateSources(q, repositorySources([]string{"(" + shown + ")"}))).ToNot(Succeed(),
"a parenthesised name would be rejected, so explain must not print one")
})
})
@ -573,7 +573,7 @@ var _ = Describe("reprocessArtwork", func() {
decline := func(io.Writer, int64, int64) bool { return false }
put := func(kind model.Kind, id, source string) {
Expect(art.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: kind.Prefix(), ItemID: id,
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: kind.Prefix(), ItemID: id,
ImageType: model.ImageTypePrimary, Hash: "h" + id, Source: source})).To(Succeed())
}
@ -584,8 +584,8 @@ var _ = Describe("reprocessArtwork", func() {
conf.Server.EnableM3UExternalAlbumArt = false
imageAgents = artwork.ImageAgentCount{Artist: 1, Album: 1}
ds = &tests.MockDataStore{}
art = ds.Artwork().(*tests.MockArtworkRepo)
queue = ds.ArtworkQueue().(*tests.MockArtworkQueueRepo)
art = ds.Artwork(ctx).(*tests.MockArtworkRepo)
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
out.Reset()
put(model.KindArtistArtwork, "ar-1", "external:deezer")
put(model.KindArtistArtwork, "ar-2", "")
@ -601,19 +601,19 @@ var _ = Describe("reprocessArtwork", func() {
Expect(out.String()).To(ContainSubstring("album"))
Expect(out.String()).To(ContainSubstring("TOTAL"))
Expect(out.String()).To(ContainSubstring("Dry run"))
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
})
It("queues nothing when the operator declines", func() {
Expect(reprocessArtwork(ctx, ds, kinds, nil, imageAgents, false, decline, &out)).To(Succeed())
Expect(out.String()).To(ContainSubstring("Aborted"))
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
})
DescribeTable("records the applied config only for a run that leaves nothing on the old one",
func(selected []model.Kind, sources []string, dryRun, applied bool) {
Expect(ds.Property().Put(ctx, consts.ArtConfFingerprintPropertyKey, "stale-fingerprint")).To(Succeed())
Expect(ds.Property(ctx).Put(consts.ArtConfFingerprintPropertyKey, "stale-fingerprint")).To(Succeed())
Expect(reprocessArtwork(ctx, ds, selected, sources, imageAgents, dryRun, accept, &out)).To(Succeed())
@ -621,7 +621,7 @@ var _ = Describe("reprocessArtwork", func() {
if applied {
want = artwork.ConfigFingerprint()
}
Expect(ds.Property().Get(ctx, consts.ArtConfFingerprintPropertyKey)).To(Equal(want))
Expect(ds.Property(ctx).Get(consts.ArtConfFingerprintPropertyKey)).To(Equal(want))
},
Entry("every kind, unfiltered", artwork.ReprocessKinds, nil, false, true),
Entry("every kind, but nothing matched", artwork.ReprocessKinds, []string{}, false, true),
@ -633,14 +633,14 @@ var _ = Describe("reprocessArtwork", func() {
It("queues the matching items at recheck priority, leaving their artwork state alone", func() {
Expect(reprocessArtwork(ctx, ds, kinds, []string{"external:deezer"}, imageAgents, false, accept, &out)).To(Succeed())
Expect(queue.Count(ctx)).To(Equal(int64(2)))
queued, err := queue.Get(ctx, model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
Expect(queue.Count()).To(Equal(int64(2)))
queued, err := queue.Get(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(queued.Priority).To(Equal(model.ArtworkPriorityRecheck))
_, err = queue.Get(ctx, model.KindAlbumArtwork, "al-2", model.ImageTypePrimary)
_, err = queue.Get(model.KindAlbumArtwork, "al-2", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "a non-matching source must not be queued")
stored, err := art.GetItemArtwork(ctx, model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
stored, err := art.GetItemArtwork(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(stored.Hash).To(Equal("hal-1"), "bulk reprocessing must not blank the current artwork")
})
@ -648,20 +648,20 @@ var _ = Describe("reprocessArtwork", func() {
It("targets the absent state", func() {
Expect(reprocessArtwork(ctx, ds, kinds, []string{""}, imageAgents, false, accept, &out)).To(Succeed())
Expect(queue.Count(ctx)).To(Equal(int64(1)))
_, err := queue.Get(ctx, model.KindArtistArtwork, "ar-2", model.ImageTypePrimary)
Expect(queue.Count()).To(Equal(int64(1)))
_, err := queue.Get(model.KindArtistArtwork, "ar-2", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
})
It("reports matched and queued separately when part of the set is already queued", func() {
Expect(queue.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-1",
Expect(queue.Enqueue(model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-1",
ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBump})).To(Succeed())
Expect(reprocessArtwork(ctx, ds, kinds, []string{"external:deezer"}, imageAgents, false, accept, &out)).To(Succeed())
Expect(out.String()).To(ContainSubstring("Queued 1 of 2 matched items"))
Expect(out.String()).To(ContainSubstring("Already queued, left unchanged: 1"))
queued, err := queue.Get(ctx, model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
queued, err := queue.Get(model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(queued.Priority).To(Equal(model.ArtworkPriorityBump),
"an already-queued row keeps its priority and backoff")
@ -675,7 +675,7 @@ var _ = Describe("reprocessArtwork", func() {
}, &out)).To(Succeed())
Expect(out.String()).To(ContainSubstring("Nothing"))
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
})
It("reports an empty selection as a dry run when one was asked for", func() {
@ -767,7 +767,7 @@ var _ = Describe("reprocessArtwork", func() {
Expect(err.Error()).To(ContainSubstring("external:deezer"))
Expect(err.Error()).To(ContainSubstring("folder"))
Expect(err.Error()).To(ContainSubstring("absent"), "the empty source prints under its user-facing name")
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
})
It("accepts the absent filter with nothing absent, still rejecting a typo", func() {
@ -777,7 +777,7 @@ var _ = Describe("reprocessArtwork", func() {
imageAgents, false, accept, &out)).To(Succeed(),
"a reserved source must stay valid once the library has none of it")
Expect(out.String()).To(ContainSubstring("Nothing matches"))
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
Expect(reprocessArtwork(ctx, ds, kinds, repositorySources([]string{"absnt"}),
imageAgents, true, accept, &out)).ToNot(Succeed(), "a typo must still be rejected")
@ -797,7 +797,7 @@ var _ = Describe("reprocessArtwork", func() {
Expect(out.String()).To(ContainSubstring("Nothing matches"),
"a well-formed filter must not be reported as a typo because of the kinds selected")
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
})
})
@ -816,19 +816,19 @@ var _ = Describe("collectStatus", func() {
BeforeEach(func() {
ds = &tests.MockDataStore{}
art = ds.Artwork().(*tests.MockArtworkRepo)
queue = ds.ArtworkQueue().(*tests.MockArtworkQueueRepo)
art = ds.Artwork(ctx).(*tests.MockArtworkRepo)
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
put := func(kind model.Kind, id, source, hash string, attempted time.Time) {
Expect(art.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: kind.Prefix(), ItemID: id,
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: kind.Prefix(), ItemID: id,
ImageType: model.ImageTypePrimary, Source: source, Hash: hash, AttemptedAt: attempted})).To(Succeed())
}
put(model.KindArtistArtwork, "ar-1", "external:deezer", "h1", time.Now())
put(model.KindArtistArtwork, "ar-2", "", "", time.Now().Add(-24*time.Hour))
// ar-3 is absent because it gave up, so the two absent artists split across the columns.
Expect(art.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "ar", ItemID: "ar-3",
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: "ar", ItemID: "ar-3",
ImageType: model.ImageTypePrimary, LastFailure: "[]", AttemptedAt: time.Now()})).To(Succeed())
put(model.KindAlbumArtwork, "al-1", "folder", "h2", time.Now())
Expect(queue.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-9",
Expect(queue.Enqueue(model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-9",
ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBackfill})).To(Succeed())
})
@ -848,7 +848,7 @@ var _ = Describe("collectStatus", func() {
})
It("compares the stored fingerprint against the current one", func() {
Expect(ds.Property().Put(ctx, consts.ArtConfFingerprintPropertyKey, "old-fingerprint")).To(Succeed())
Expect(ds.Property(ctx).Put(consts.ArtConfFingerprintPropertyKey, "old-fingerprint")).To(Succeed())
rep, err := collectStatus(ctx, ds)
Expect(err).ToNot(HaveOccurred())
@ -860,7 +860,7 @@ var _ = Describe("collectStatus", func() {
It("queues nothing", func() {
_, err := collectStatus(ctx, ds)
Expect(err).ToNot(HaveOccurred())
Expect(queue.Count(ctx)).To(Equal(int64(1)), "status must not enqueue anything")
Expect(queue.Count()).To(Equal(int64(1)), "status must not enqueue anything")
})
})
@ -973,21 +973,21 @@ var _ = Describe("refreshItems", func() {
albums := tests.CreateMockAlbumRepo()
albums.SetData(model.Albums{{ID: "al-1"}, {ID: "al-3"}})
ds = &tests.MockDataStore{MockedAlbum: albums}
art = ds.Artwork().(*tests.MockArtworkRepo)
queue = ds.ArtworkQueue().(*tests.MockArtworkQueueRepo)
art = ds.Artwork(ctx).(*tests.MockArtworkRepo)
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
out.Reset()
})
It("clears the stored state and queues each id at Bump priority", func() {
Expect(art.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: model.KindAlbumArtwork.Prefix(),
Expect(art.PutItemArtwork(&model.ItemArtwork{ItemKind: model.KindAlbumArtwork.Prefix(),
ItemID: "al-1", ImageType: model.ImageTypePrimary, Hash: "abc123"})).To(Succeed())
Expect(refreshItems(ctx, ds, []model.ArtworkID{
{Kind: model.KindAlbumArtwork, ID: "al-1"}, {Kind: model.KindAlbumArtwork, ID: "al-3"}}, &out)).To(BeZero())
_, err := art.GetItemArtwork(ctx, model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
_, err := art.GetItemArtwork(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
queued, err := queue.Get(ctx, model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
queued, err := queue.Get(model.KindAlbumArtwork, "al-1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(queued.Priority).To(Equal(model.ArtworkPriorityBump))
Expect(out.String()).To(Equal("al/al-1: queued\nal/al-3: queued\n"))
@ -996,7 +996,7 @@ var _ = Describe("refreshItems", func() {
It("skips an id that does not exist instead of queuing it", func() {
Expect(refreshItems(ctx, ds, []model.ArtworkID{{Kind: model.KindAlbumArtwork, ID: "al-2"}}, &out)).To(Equal(1))
_, err := queue.Get(ctx, model.KindAlbumArtwork, "al-2", model.ImageTypePrimary)
_, err := queue.Get(model.KindAlbumArtwork, "al-2", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "a typo must not leave an orphan queue row")
Expect(out.String()).To(BeEmpty())
})
@ -1145,9 +1145,9 @@ var _ = Describe("cancelArtwork", func() {
BeforeEach(func() {
ds = &tests.MockDataStore{}
queue = ds.ArtworkQueue().(*tests.MockArtworkQueueRepo)
queue = ds.ArtworkQueue(ctx).(*tests.MockArtworkQueueRepo)
out.Reset()
Expect(queue.Enqueue(ctx,
Expect(queue.Enqueue(
model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-1", ImageType: model.ImageTypePrimary,
Priority: model.ArtworkPriorityBackfill},
model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar-2", ImageType: model.ImageTypePrimary,
@ -1164,28 +1164,28 @@ var _ = Describe("cancelArtwork", func() {
Expect(out.String()).To(ContainSubstring("backfill"))
Expect(out.String()).To(ContainSubstring("TOTAL"))
Expect(out.String()).To(ContainSubstring("Dry run"))
Expect(queue.Count(ctx)).To(BeNumerically("==", 3))
Expect(queue.Count()).To(BeNumerically("==", 3))
})
It("cancels nothing when the operator declines", func() {
Expect(cancelArtwork(ctx, ds, nil, nil, false, decline, &out)).To(Succeed())
Expect(out.String()).To(ContainSubstring("Aborted"))
Expect(queue.Count(ctx)).To(BeNumerically("==", 3))
Expect(queue.Count()).To(BeNumerically("==", 3))
})
It("deletes the selected rows and leaves the rest queued", func() {
Expect(cancelArtwork(ctx, ds, nil, []int{model.ArtworkPriorityBackfill}, false, accept, &out)).To(Succeed())
Expect(queue.Count(ctx)).To(BeNumerically("==", 1))
_, err := queue.Get(ctx, model.KindArtistArtwork, "ar-2", model.ImageTypePrimary)
Expect(queue.Count()).To(BeNumerically("==", 1))
_, err := queue.Get(model.KindArtistArtwork, "ar-2", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred(), "a non-matching priority must stay queued")
Expect(out.String()).To(ContainSubstring("Cancelled 2 of 2 matched items."))
})
It("cancels every kind and priority when neither filter is given", func() {
Expect(cancelArtwork(ctx, ds, nil, nil, false, accept, &out)).To(Succeed())
Expect(queue.Count(ctx)).To(BeZero())
Expect(queue.Count()).To(BeZero())
})
It("stops at a selection that matches nothing instead of prompting", func() {
@ -1196,7 +1196,7 @@ var _ = Describe("cancelArtwork", func() {
Expect(cancelArtwork(ctx, ds, []model.Kind{model.KindPlaylistArtwork}, nil, false, refuse, &out)).To(Succeed())
Expect(out.String()).To(ContainSubstring("Nothing matches this selection."))
Expect(queue.Count(ctx)).To(BeNumerically("==", 3))
Expect(queue.Count()).To(BeNumerically("==", 3))
})
It("reports a queue read failure instead of reporting nothing to cancel", func() {

View file

@ -1,17 +1,13 @@
package cmd
import (
"context"
"encoding/json"
"fmt"
"path/filepath"
"strings"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/db"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/persistence"
"github.com/pelletier/go-toml/v2"
"github.com/spf13/cobra"
"gopkg.in/yaml.v3"
@ -32,7 +28,7 @@ var inspectCmd = &cobra.Command{
Long: "Show file tags as seen by Navidrome",
Args: cobra.MinimumNArgs(1),
Run: func(cmd *cobra.Command, args []string) {
runInspector(cmd.Context(), args)
runInspector(args)
},
}
@ -59,24 +55,18 @@ func prettyMarshal(v any) ([]byte, error) {
return []byte(res.String()), nil
}
func runInspector(ctx context.Context, args []string) {
func runInspector(args []string) {
marshal := marshalers[format]
if marshal == nil {
log.Fatal("Invalid format", "format", format)
}
libs := loadLibraries(ctx)
matcher := model.NewLibraryMatcher(libs)
var out []core.InspectOutput
for _, filePath := range args {
if !model.IsAudioFile(filePath) {
log.Warn("Not an audio file", "file", filePath)
continue
}
lib, ok := libraryForFile(matcher, filePath)
if !ok && len(libs) > 0 {
log.Warn("File is not in any library, using the global PID config", "file", filePath)
}
output, err := core.Inspect(filePath, lib, "")
output, err := core.Inspect(filePath, 1, "")
if err != nil {
log.Warn("Unable to process file", "file", filePath, "error", err)
continue
@ -87,33 +77,3 @@ func runInspector(ctx context.Context, args []string) {
data, _ := marshal(out)
fmt.Println(string(data))
}
// loadLibraries reads the libraries, so each file gets its library's PID config. It never creates a DB.
func loadLibraries(ctx context.Context) model.Libraries {
if dbFile, ok := existingDBFile(); !ok {
log.Warn(ctx, "No database found, using the global PID config", "path", dbFile)
return nil
}
defer db.Init(ctx)()
libs, err := persistence.New(db.Db()).Library().GetAll(ctx)
if err != nil {
log.Warn(ctx, "Could not load libraries, using the global PID config", err)
return nil
}
for i := range libs {
if absPath, err := filepath.Abs(libs[i].Path); err == nil {
libs[i].Path = absPath
}
}
return libs
}
// libraryForFile falls back to the default library with no overrides, which uses the global PID config.
func libraryForFile(matcher *model.LibraryMatcher, filePath string) (model.Library, bool) {
if absPath, err := filepath.Abs(filePath); err == nil {
if lib, ok := matcher.FindLibrary(absPath); ok {
return lib, true
}
}
return model.Library{ID: model.DefaultLibraryID}, false
}

View file

@ -1,61 +0,0 @@
package cmd
import (
"os"
"path/filepath"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/model"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("inspect", func() {
Describe("libraryForFile", func() {
var matcher *model.LibraryMatcher
var root string
BeforeEach(func() {
root = GinkgoT().TempDir()
cwd, err := os.Getwd()
Expect(err).ToNot(HaveOccurred())
matcher = model.NewLibraryMatcher(model.Libraries{
{ID: 1, Path: filepath.Join(root, "music")},
{ID: 2, Path: filepath.Join(cwd, "loose"), PIDAlbum: "folder"},
})
})
It("returns the library that contains an absolute path", func() {
lib, ok := libraryForFile(matcher, filepath.Join(root, "music", "album", "track.mp3"))
Expect(ok).To(BeTrue())
Expect(lib.ID).To(Equal(1))
})
It("resolves a relative path against the working directory", func() {
lib, ok := libraryForFile(matcher, filepath.Join("loose", "track.mp3"))
Expect(ok).To(BeTrue())
Expect(lib.PIDAlbum).To(Equal("folder"))
})
It("falls back to the default library without overrides", func() {
lib, ok := libraryForFile(matcher, filepath.Join(root, "elsewhere", "track.mp3"))
Expect(ok).To(BeFalse())
Expect(lib).To(Equal(model.Library{ID: model.DefaultLibraryID}))
})
})
Describe("loadLibraries", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
It("does not create a database when there is none", func() {
dbFile := filepath.Join(GinkgoT().TempDir(), "navidrome.db")
conf.Server.DbPath = dbFile + "?_journal_mode=WAL"
Expect(loadLibraries(GinkgoT().Context())).To(BeNil())
Expect(dbFile).ToNot(BeAnExistingFile())
})
})
})

View file

@ -72,7 +72,7 @@ func runMissingList(ctx context.Context) {
}
ds, ctx := getAdminContext(ctx)
mfs, err := ds.MediaFile().GetCursor(ctx, model.QueryOptions{
mfs, err := ds.MediaFile(ctx).GetCursor(model.QueryOptions{
Filters: squirrel.Eq{"missing": true},
Sort: "path",
})
@ -128,7 +128,7 @@ func runMissingFix(ctx context.Context, missingRef, targetRef string) {
// resolveMediaFile looks up a media file by ID first, then by path (optionally libraryID:path).
func resolveMediaFile(ctx context.Context, ds model.DataStore, ref string) *model.MediaFile {
mf, err := ds.MediaFile().Get(ctx, ref)
mf, err := ds.MediaFile(ctx).Get(ref)
if err == nil {
return mf
}
@ -136,7 +136,7 @@ func resolveMediaFile(ctx context.Context, ds model.DataStore, ref string) *mode
log.Fatal(ctx, "Error looking up media file", "ref", ref, err)
}
mfs, err := ds.MediaFile().FindByPaths(ctx, []string{ref})
mfs, err := ds.MediaFile(ctx).FindByPaths([]string{ref})
if err != nil {
log.Fatal(ctx, "Error looking up media file by path", "ref", ref, err)
}

View file

@ -109,7 +109,7 @@ func fetchPlaylists(ctx context.Context, ds model.DataStore, sort string) model.
}
options.Filters = squirrel.Eq{"owner_id": user.ID}
}
pls, err := ds.Playlist().GetAll(ctx, options)
pls, err := ds.Playlist(ctx).GetAll(options)
if err != nil {
log.Fatal(ctx, "Failed to retrieve playlists", err)
}
@ -117,17 +117,17 @@ func fetchPlaylists(ctx context.Context, ds model.DataStore, sort string) model.
}
func findPlaylist(ctx context.Context, ds model.DataStore, nameOrID string) *model.Playlist {
playlist, err := ds.Playlist().GetWithTracks(ctx, nameOrID, true, false)
playlist, err := ds.Playlist(ctx).GetWithTracks(nameOrID, true, false)
if err != nil && !errors.Is(err, model.ErrNotFound) {
log.Fatal("Error retrieving playlist", "name", nameOrID, err)
}
if errors.Is(err, model.ErrNotFound) {
playlists, err := ds.Playlist().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"playlist.name": nameOrID}})
playlists, err := ds.Playlist(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"playlist.name": nameOrID}})
if err != nil {
log.Fatal("Error retrieving playlist", "name", nameOrID, err)
}
if len(playlists) > 0 {
playlist, err = ds.Playlist().GetWithTracks(ctx, playlists[0].ID, true, false)
playlist, err = ds.Playlist(ctx).GetWithTracks(playlists[0].ID, true, false)
if err != nil {
log.Fatal("Error retrieving playlist", "name", nameOrID, err)
}
@ -194,7 +194,7 @@ func runExport(ctx context.Context) {
exported := 0
for _, pls := range allPls {
plsWithTracks, err := ds.Playlist().GetWithTracks(ctx, pls.ID, true, false)
plsWithTracks, err := ds.Playlist(ctx).GetWithTracks(pls.ID, true, false)
if err != nil {
log.Error("Error loading playlist tracks", "playlist", pls.Name, err)
continue

View file

@ -243,7 +243,7 @@ func runPluginInfo(ctx context.Context, arg string) {
}
requirePluginsEnabled(ctx)
ds, ctx := getAdminContext(ctx)
p, err := ds.Plugin().Get(ctx, arg)
p, err := ds.Plugin(ctx).Get(arg)
if err != nil {
log.Fatal(ctx, "Plugin not found", "id", arg, err)
}
@ -264,7 +264,7 @@ func runPluginValidate(ctx context.Context, arg string) {
}
requirePluginsEnabled(ctx)
ds, ctx := getAdminContext(ctx)
p, err := ds.Plugin().Get(ctx, arg)
p, err := ds.Plugin(ctx).Get(arg)
if err != nil {
log.Fatal(ctx, "Plugin not found", "id", arg, err)
}
@ -329,7 +329,7 @@ func formatPluginList(list model.Plugins, format string) (string, error) {
func runPluginList(ctx context.Context) {
requirePluginsEnabled(ctx)
ds, ctx := getAdminContext(ctx)
list, err := ds.Plugin().GetAll(ctx)
list, err := ds.Plugin(ctx).GetAll()
if err != nil {
log.Fatal(ctx, "Failed to list plugins", err)
}
@ -372,7 +372,7 @@ var pluginEditCmd = &cobra.Command{
Run: func(cmd *cobra.Command, args []string) {
requirePluginsEnabled(cmd.Context())
ds, ctx := getAdminContext(cmd.Context())
cur, err := ds.Plugin().Get(ctx, args[0])
cur, err := ds.Plugin(ctx).Get(args[0])
if err != nil {
log.Fatal(ctx, "Plugin not found", "id", args[0], err)
}

View file

@ -44,9 +44,7 @@ Complete documentation is available at https://www.navidrome.org/docs`,
preRun()
},
Run: func(cmd *cobra.Command, args []string) {
if err := runNavidrome(cmd.Context()); err != nil {
log.Fatal("Fatal error in Navidrome. Aborting", err)
}
runNavidrome(cmd.Context())
},
PostRun: func(cmd *cobra.Command, args []string) {
postRun()
@ -78,17 +76,16 @@ func postRun() {
}
// runNavidrome is the main entry point for the Navidrome server. It starts all the services and blocks.
// If any of the services returns an error, it stops the others and returns that error, so the caller can
// exit with a non-zero code. If the context is cancelled (a signal or a service stop), it returns nil.
func runNavidrome(parentCtx context.Context) error {
defer db.Init(parentCtx)()
// If any of the services returns an error, it will log it and exit. If the process receives a signal to exit,
// it will cancel the context and exit gracefully.
func runNavidrome(ctx context.Context) {
defer db.Init(ctx)()
g, ctx := errgroup.WithContext(parentCtx)
g, ctx := errgroup.WithContext(ctx)
g.Go(startServer(ctx))
g.Go(startSignaller(ctx))
g.Go(startScheduler(ctx))
g.Go(startPlaybackServer(ctx))
g.Go(startJellyfinDiscovery(ctx))
g.Go(schedulePeriodicBackup(ctx))
g.Go(startInsightsCollector(ctx))
g.Go(scheduleDBAnalyzer(ctx))
@ -104,11 +101,9 @@ func runNavidrome(parentCtx context.Context) error {
log.Warn(ctx, "Automatic Scanning is DISABLED")
}
// Errors caused by a normal shutdown are not failures
if err := g.Wait(); err != nil && parentCtx.Err() == nil {
return err
if err := g.Wait(); err != nil {
log.Error("Fatal error in Navidrome. Aborting", err)
}
return nil
}
// mainContext returns a context that is cancelled when the process receives a signal to exit.
@ -137,9 +132,6 @@ func startServer(ctx context.Context) func() error {
if conf.Server.Jellyfin.Enabled {
a.MountRouter("Jellyfin API", consts.URLPathJellyfinAPI, CreateJellyfinAPIRouter(ctx))
}
if conf.Server.DevAPIv1 {
a.MountRouter("API v1", consts.URLPathAPIv1, CreateAPIv1Router(ctx))
}
if conf.Server.Prometheus.Enabled {
p := CreatePrometheus()
// blocking call because takes <100ms but useful if fails
@ -190,50 +182,46 @@ func schedulePeriodicScan(ctx context.Context) func() error {
}
}
// librariesWithChangedPID returns the names of the libraries whose effective PID config differs from
// the one used by their last finished scan
func librariesWithChangedPID(ctx context.Context, ds model.DataStore) ([]string, error) {
libs, err := ds.Library().GetAll(ctx)
func pidHashChanged(ds model.DataStore) (bool, error) {
pidAlbum, err := ds.Property(context.Background()).DefaultGet(consts.PIDAlbumKey, "")
if err != nil {
return nil, err
return false, err
}
var names []string
for _, lib := range libs {
if lib.PIDChanged() {
names = append(names, lib.Name)
}
pidTrack, err := ds.Property(context.Background()).DefaultGet(consts.PIDTrackKey, "")
if err != nil {
return false, err
}
return names, nil
return !strings.EqualFold(pidAlbum, conf.Server.PID.Album) || !strings.EqualFold(pidTrack, conf.Server.PID.Track), nil
}
// runInitialScan runs an initial scan of the music library if needed.
func runInitialScan(ctx context.Context) func() error {
return func() error {
ds := CreateDataStore()
fullScanRequired, err := ds.Property().DefaultGet(ctx, consts.FullScanAfterMigrationFlagKey, "0")
fullScanRequired, err := ds.Property(ctx).DefaultGet(consts.FullScanAfterMigrationFlagKey, "0")
if err != nil {
return err
}
inProgress, err := ds.Library().ScanInProgress(ctx)
inProgress, err := ds.Library(ctx).ScanInProgress()
if err != nil {
return err
}
pidChangedLibs, err := librariesWithChangedPID(ctx, ds)
pidHasChanged, err := pidHashChanged(ds)
if err != nil {
return err
}
scanOnStartup := conf.Server.Scanner.Enabled && conf.Server.Scanner.ScanOnStartup
scanNeeded := scanOnStartup || inProgress || fullScanRequired == "1" || len(pidChangedLibs) > 0
scanNeeded := scanOnStartup || inProgress || fullScanRequired == "1" || pidHasChanged
time.Sleep(2 * time.Second) // Wait 2 seconds before the initial scan
if scanNeeded {
s := CreateScanner(ctx)
switch {
case fullScanRequired == "1":
log.Warn(ctx, "Full scan required after migration")
_ = ds.Property().Delete(ctx, consts.FullScanAfterMigrationFlagKey)
case len(pidChangedLibs) > 0:
// Includes never-scanned libraries. The scanner rescans in full only the ones that need it
log.Warn(ctx, "Libraries with a new or changed PID config, scanning", "libraries", pidChangedLibs)
_ = ds.Property(ctx).Delete(consts.FullScanAfterMigrationFlagKey)
case pidHasChanged:
log.Warn(ctx, "PID config changed, performing full scan")
fullScanRequired = "1"
case inProgress:
log.Warn(ctx, "Resuming interrupted scan")
default:
@ -355,18 +343,6 @@ func startInsightsCollector(ctx context.Context) func() error {
}
}
// startJellyfinDiscovery never returns an error: a discovery failure must not stop the server.
func startJellyfinDiscovery(ctx context.Context) func() error {
return func() error {
if !conf.Server.Jellyfin.Enabled || !conf.Server.Jellyfin.AutoDiscovery {
log.Debug("Jellyfin auto-discovery is DISABLED")
return nil
}
CreateJellyfinDiscovery().Serve(ctx)
return nil
}
}
// startPlaybackServer starts the Navidrome playback server, if configured.
// It is responsible for the Jukebox functionality
func startPlaybackServer(ctx context.Context) func() error {
@ -386,26 +362,10 @@ func startPlaybackServer(ctx context.Context) func() error {
func startArtworkWorker(ctx context.Context, worker *artwork.Worker) func() error {
return func() error {
log.Info(ctx, "Starting artwork worker")
// The scanner writes to the DB for its whole run; competing for the write lock makes both fail.
worker.PauseWhile(scanner.IsScanning)
return worker.Run(ctx)
}
}
// outsideScan runs a DB maintenance job unless a scan is running, and keeps a scan from starting
// until it ends; both write to the DB, and competing for the lock can make either fail.
func outsideScan(ctx context.Context, job string, run func(context.Context) error) {
release, ok := scanner.LockForMaintenance()
if !ok {
log.Debug(ctx, "Skipping "+job+" because a scan is in progress")
return
}
defer release()
if err := run(ctx); err != nil {
log.Error(ctx, "Error running "+job, err)
}
}
// scheduleArtworkHousekeeping registers the recurring missing-state and prune jobs, and
// reports an artwork config change without acting on it.
func scheduleArtworkHousekeeping(ctx context.Context, worker *artwork.Worker) func() error {
@ -413,20 +373,26 @@ func scheduleArtworkHousekeeping(ctx context.Context, worker *artwork.Worker) fu
schedulerInstance := scheduler.GetInstance()
if _, err := schedulerInstance.Add(consts.ArtworkEnqueueMissingSchedule, func() {
outsideScan(ctx, "artwork missing-state recheck", worker.EnqueueMissingAll)
if err := worker.EnqueueMissingAll(ctx); err != nil {
log.Error(ctx, "Error enqueueing missing artwork rechecks", err)
}
}); err != nil {
log.Error(ctx, "Error scheduling artwork missing-state recheck", err)
}
if _, err := schedulerInstance.Add(consts.ArtworkPruneSchedule, func() {
outsideScan(ctx, "artwork prune", worker.RunPrune)
if err := worker.RunPrune(ctx); err != nil {
log.Error(ctx, "Error running artwork prune", err)
}
}); err != nil {
log.Error(ctx, "Error scheduling artwork prune", err)
}
// Also run the missing-row recheck once at startup so a never-scanned entity is picked up
// immediately, not only on the next hourly tick (e.g. after enabling the feature).
outsideScan(ctx, "artwork missing-state recheck", worker.EnqueueMissingAll)
if err := worker.EnqueueMissingAll(ctx); err != nil {
log.Error(ctx, "Error enqueueing missing artwork rechecks", err)
}
if err := worker.ReconcileConfig(ctx); err != nil {
log.Error(ctx, "Error checking the artwork config fingerprint", err)

View file

@ -1,7 +1,6 @@
package cmd
import (
"errors"
"net/http"
"net/http/httptest"
"path"
@ -10,8 +9,6 @@ import (
"github.com/go-chi/chi/v5"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
@ -47,30 +44,3 @@ var _ = Describe("profilerHandler", func() {
Entry("with a trailing-slash BasePath", "/music/"),
)
})
var _ = Describe("librariesWithChangedPID", func() {
var ds *tests.MockDataStore
var libs *tests.MockLibraryRepo
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
libs = &tests.MockLibraryRepo{}
ds = &tests.MockDataStore{MockedLibrary: libs}
})
It("returns only the libraries whose PID config changed", func() {
pid := model.Library{}.EffectivePID()
libs.SetData(model.Libraries{
{ID: 1, Name: "Same", ScannedPIDAlbum: pid.Album, ScannedPIDTrack: pid.Track},
{ID: 2, Name: "Changed", PIDAlbum: "folder", ScannedPIDAlbum: pid.Album, ScannedPIDTrack: pid.Track},
{ID: 3, Name: "Never scanned"},
})
Expect(librariesWithChangedPID(GinkgoT().Context(), ds)).To(ConsistOf("Changed", "Never scanned"))
})
It("returns the error from the repository", func() {
libs.Err = errors.New("db down")
_, err := librariesWithChangedPID(GinkgoT().Context(), ds)
Expect(err).To(MatchError("db down"))
})
})

View file

@ -44,7 +44,7 @@ var svcCmd = &cobra.Command{
}
type svcControl struct {
ctx context.Context //nolint:containedctx // service lifecycle ctx, cancelled by Stop
ctx context.Context
cancel context.CancelFunc
done chan struct{}
}
@ -53,13 +53,8 @@ func (p *svcControl) Start(service.Service) error {
p.done = make(chan struct{})
p.ctx, p.cancel = context.WithCancel(context.Background())
go func() {
err := runNavidrome(p.ctx)
runNavidrome(p.ctx)
close(p.done)
// service.Run() only returns when it gets a stop request, so exit here to let the
// service manager see the failure and restart the service
if err != nil {
log.Fatal("Fatal error in Navidrome. Aborting", err)
}
}()
return nil
}
@ -79,7 +74,7 @@ func (p *svcControl) Stop(service.Service) error {
var svcInstance = sync.OnceValue(func() service.Service {
options := make(service.KeyValue)
options["Restart"] = "on-failure"
options["SuccessExitStatus"] = "SIGKILL"
options["SuccessExitStatus"] = "1 2 8 SIGKILL"
options["UserService"] = false
options["LogDirectory"] = conf.Server.DataFolder.String()
options["SystemdScript"] = systemdScript

View file

@ -183,7 +183,7 @@ func runCreateUser(ctx context.Context) {
ds, ctx := getAdminContext(ctx)
err := ds.WithTx(func(tx model.DataStore) error {
existingUser, err := tx.User().FindByUsername(ctx, userID)
existingUser, err := tx.User(ctx).FindByUsername(userID)
if existingUser != nil {
return fmt.Errorf("existing user '%s'", userID)
}
@ -193,7 +193,7 @@ func runCreateUser(ctx context.Context) {
}
if len(libraryIds) > 0 && !setAdmin {
user.Libraries, err = tx.Library().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}})
user.Libraries, err = tx.Library(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}})
if err != nil {
return err
}
@ -202,13 +202,13 @@ func runCreateUser(ctx context.Context) {
return libraryError(user.Libraries)
}
} else {
user.Libraries, err = tx.Library().GetAll(ctx)
user.Libraries, err = tx.Library(ctx).GetAll()
if err != nil {
return err
}
}
err = tx.User().Put(ctx, &user)
err = tx.User(ctx).Put(&user)
if err != nil {
return err
}
@ -218,7 +218,7 @@ func runCreateUser(ctx context.Context) {
updatedIds[idx] = lib.ID
}
err = tx.User().SetUserLibraries(ctx, user.ID, updatedIds)
err = tx.User(ctx).SetUserLibraries(user.ID, updatedIds)
return err
})
@ -236,7 +236,7 @@ func runDeleteUser(ctx context.Context) {
var user *model.User
err = ds.WithTx(func(tx model.DataStore) error {
count, err := tx.User().CountAll(ctx)
count, err := tx.User(ctx).CountAll()
if err != nil {
return err
}
@ -250,7 +250,7 @@ func runDeleteUser(ctx context.Context) {
return err
}
return tx.User().Delete(ctx, user.ID)
return tx.User(ctx).Delete(user.ID)
})
if err != nil {
@ -276,7 +276,7 @@ func runUserEdit(ctx context.Context) {
}
if len(libraryIds) > 0 && !setAdmin {
libraries, err := tx.Library().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}})
libraries, err := tx.Library(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"id": libraryIds}})
if err != nil {
return err
@ -291,7 +291,7 @@ func runUserEdit(ctx context.Context) {
}
if setAdmin && !user.IsAdmin {
libraries, err := tx.Library().GetAll(ctx)
libraries, err := tx.Library(ctx).GetAll()
if err != nil {
return err
}
@ -337,7 +337,7 @@ func runUserEdit(ctx context.Context) {
return nil
}
err := tx.User().Put(ctx, user)
err := tx.User(ctx).Put(user)
if err != nil {
return err
}
@ -348,7 +348,7 @@ func runUserEdit(ctx context.Context) {
updatedIds[idx] = lib.ID
}
err := tx.User().SetUserLibraries(ctx, user.ID, updatedIds)
err := tx.User(ctx).SetUserLibraries(user.ID, updatedIds)
if err != nil {
return err
}
@ -393,11 +393,13 @@ func runUserList(ctx context.Context) {
ds, ctx := getAdminContext(ctx)
userList, err := ds.User().ReadAll(ctx)
users, err := ds.User(ctx).ReadAll()
if err != nil {
log.Fatal(ctx, "Failed to retrieve users", err)
}
userList := users.(model.Users)
if outputFormat == "csv" {
w := csv.NewWriter(os.Stdout)
_ = w.Write([]string{

View file

@ -18,16 +18,11 @@ import (
"github.com/navidrome/navidrome/persistence"
)
// existingDBFile returns the database file (DbPath minus DSN params), and whether it exists.
func existingDBFile() (string, bool) {
path, _, _ := strings.Cut(conf.Server.DbPath, "?")
_, err := os.Stat(path)
return path, err == nil
}
// requireExistingDB aborts the command when the database file does not exist.
// requireExistingDB aborts the command when the database file (DbPath minus DSN
// params) does not exist.
func requireExistingDB() {
if path, ok := existingDBFile(); !ok {
path, _, _ := strings.Cut(conf.Server.DbPath, "?")
if _, err := os.Stat(path); os.IsNotExist(err) {
log.Fatal("No existing database", "path", path)
}
}
@ -57,14 +52,14 @@ func getAdminContext(ctx context.Context) (model.DataStore, context.Context) {
}
func getUser(ctx context.Context, id string, ds model.DataStore) (*model.User, error) {
user, err := ds.User().FindByUsername(ctx, id)
user, err := ds.User(ctx).FindByUsername(id)
if err != nil && !errors.Is(err, model.ErrNotFound) {
return nil, fmt.Errorf("finding user by name: %w", err)
}
if errors.Is(err, model.ErrNotFound) {
user, err = ds.User().Get(ctx, id)
user, err = ds.User(ctx).Get(id)
if err != nil {
return nil, fmt.Errorf("finding user by id: %w", err)
}

View file

@ -21,7 +21,6 @@ import (
"github.com/navidrome/navidrome/core/metrics"
"github.com/navidrome/navidrome/core/playback"
"github.com/navidrome/navidrome/core/playlists"
"github.com/navidrome/navidrome/core/quickconnect"
"github.com/navidrome/navidrome/core/scrobbler"
"github.com/navidrome/navidrome/core/sonic"
"github.com/navidrome/navidrome/core/stream"
@ -31,7 +30,6 @@ import (
"github.com/navidrome/navidrome/plugins"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/server/apiv1"
"github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/server/jellyfin"
"github.com/navidrome/navidrome/server/nativeapi"
@ -81,8 +79,7 @@ func CreateNativeAPIRouter(ctx context.Context) *nativeapi.Router {
agentsAgents := agents.GetAgents(dataStore, manager)
matcherMatcher := matcher.New(dataStore)
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker)
quickConnect := quickconnect.GetInstance()
router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager, uploader, provider, quickConnect)
router := nativeapi.New(dataStore, share, playlistsPlaylists, insights, library, user, maintenance, manager, uploader, provider)
return router
}
@ -92,14 +89,13 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore()
fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg)
broker := events.GetBroker()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg, broker)
transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
share := core.NewShare(dataStore)
archiver := core.NewArchiver(mediaStreamer, transcodeDecider, dataStore, share, artworkArtwork)
archiver := core.NewArchiver(mediaStreamer, dataStore, share)
players := core.NewPlayers(dataStore)
broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
agentsAgents := agents.GetAgents(dataStore, manager)
@ -111,6 +107,7 @@ func CreateSubsonicAPIRouter(ctx context.Context) *subsonic.Router {
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
playbackServer := playback.GetInstance(dataStore)
lyricsLyrics := lyrics.NewLyrics(dataStore, manager)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
sonicSonic := sonic.New(dataStore, manager, matcherMatcher)
router := subsonic.New(dataStore, artworkArtwork, mediaStreamer, archiver, players, provider, modelScanner, broker, playlistsPlaylists, playTracker, share, playbackServer, metricsMetrics, lyricsLyrics, transcodeDecider, sonicSonic)
return router
@ -122,12 +119,12 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore()
fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg)
broker := events.GetBroker()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg, broker)
transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
players := core.NewPlayers(dataStore)
broker := events.GetBroker()
metricsMetrics := metrics.GetPrometheusInstance(dataStore)
manager := plugins.GetManager(dataStore, broker, metricsMetrics)
playTracker := scrobbler.GetPlayTracker(dataStore, broker, manager)
@ -138,15 +135,7 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
provider := external.NewProvider(dataStore, agentsAgents, matcherMatcher, broker)
sonicSonic := sonic.New(dataStore, manager, matcherMatcher)
lyricsLyrics := lyrics.NewLyrics(dataStore, manager)
quickConnect := quickconnect.GetInstance()
router := jellyfin.New(dataStore, artworkArtwork, mediaStreamer, transcodeDecider, players, playTracker, playlistsPlaylists, provider, sonicSonic, lyricsLyrics, broker, quickConnect)
return router
}
func CreateAPIv1Router(ctx context.Context) *apiv1.Router {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
router := apiv1.New(dataStore)
router := jellyfin.New(dataStore, artworkArtwork, mediaStreamer, transcodeDecider, players, playTracker, playlistsPlaylists, provider, sonicSonic, lyricsLyrics, broker)
return router
}
@ -156,13 +145,13 @@ func CreatePublicRouter() *public.Router {
fileCache := artwork.GetImageCache()
imageStore := artwork.GetImageStore()
fFmpeg := ffmpeg.New()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg)
broker := events.GetBroker()
artworkArtwork := artwork.NewArtwork(dataStore, fileCache, imageStore, fFmpeg, broker)
transcodingCache := stream.GetTranscodingCache()
mediaStreamer := stream.NewMediaStreamer(dataStore, fFmpeg, transcodingCache)
transcodeDecider := stream.NewTranscodeDecider(dataStore, fFmpeg)
share := core.NewShare(dataStore)
archiver := core.NewArchiver(mediaStreamer, transcodeDecider, dataStore, share, artworkArtwork)
router := public.New(dataStore, artworkArtwork, mediaStreamer, transcodeDecider, share, archiver)
archiver := core.NewArchiver(mediaStreamer, dataStore, share)
router := public.New(dataStore, artworkArtwork, mediaStreamer, share, archiver)
return router
}
@ -180,13 +169,6 @@ func CreateListenBrainzRouter() *listenbrainz.Router {
return router
}
func CreateJellyfinDiscovery() *jellyfin.Discovery {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
discovery := jellyfin.NewDiscovery(dataStore)
return discovery
}
func CreateInsights() metrics.Insights {
sqlDB := db.Db()
dataStore := persistence.New(sqlDB)
@ -268,7 +250,7 @@ func getPluginManager() *plugins.Manager {
// wire_injectors.go:
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, jellyfin.New, jellyfin.NewDiscovery, apiv1.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.GetInstance, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, sonic.New, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.Engine), new(*sonic.Sonic)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher)), wire.Bind(new(playlists.ImageUploadService), new(artwork.Uploader)))
var allProviders = wire.NewSet(core.Set, artwork.Set, server.New, subsonic.New, jellyfin.New, nativeapi.New, public.New, persistence.New, lastfm.NewRouter, listenbrainz.NewRouter, events.GetBroker, scanner.GetInstance, scanner.GetWatcher, metrics.GetPrometheusInstance, db.Db, plugins.GetManager, sonic.New, wire.Bind(new(agents.PluginLoader), new(*plugins.Manager)), wire.Bind(new(scrobbler.PluginLoader), new(*plugins.Manager)), wire.Bind(new(lyrics.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.PluginLoader), new(*plugins.Manager)), wire.Bind(new(sonic.Engine), new(*sonic.Sonic)), wire.Bind(new(nativeapi.PluginManager), new(*plugins.Manager)), wire.Bind(new(core.PluginUnloader), new(*plugins.Manager)), wire.Bind(new(plugins.PluginMetricsRecorder), new(metrics.Metrics)), wire.Bind(new(core.Watcher), new(scanner.Watcher)), wire.Bind(new(playlists.ImageUploadService), new(artwork.Uploader)))
func GetPluginManager(ctx context.Context) *plugins.Manager {
manager := getPluginManager()

View file

@ -23,7 +23,6 @@ import (
"github.com/navidrome/navidrome/plugins"
"github.com/navidrome/navidrome/scanner"
"github.com/navidrome/navidrome/server"
"github.com/navidrome/navidrome/server/apiv1"
"github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/server/jellyfin"
"github.com/navidrome/navidrome/server/nativeapi"
@ -37,8 +36,6 @@ var allProviders = wire.NewSet(
server.New,
subsonic.New,
jellyfin.New,
jellyfin.NewDiscovery,
apiv1.New,
nativeapi.New,
public.New,
persistence.New,
@ -93,12 +90,6 @@ func CreateJellyfinAPIRouter(ctx context.Context) *jellyfin.Router {
))
}
func CreateAPIv1Router(ctx context.Context) *apiv1.Router {
panic(wire.Build(
allProviders,
))
}
func CreatePublicRouter() *public.Router {
panic(wire.Build(
allProviders,
@ -117,12 +108,6 @@ func CreateListenBrainzRouter() *listenbrainz.Router {
))
}
func CreateJellyfinDiscovery() *jellyfin.Discovery {
panic(wire.Build(
allProviders,
))
}
func CreateInsights() metrics.Insights {
panic(wire.Build(
allProviders,

View file

@ -161,7 +161,6 @@ type configOptions struct {
DevExternalArtistFetchMultiplier float64
DevPreserveUnicodeInExternalCalls bool
DevEnableMediaFileProbe bool
DevAPIv1 bool
}
type scannerOptions struct {
@ -236,8 +235,6 @@ type jellyfinOptions struct {
// ExposedPublicUsers is a comma-separated list of usernames to advertise on the unauthenticated
// GET /Users/Public, so Jellyfin clients can show a login user-picker. Empty exposes no users.
ExposedPublicUsers string
AutoDiscovery bool
QuickConnect bool
// MaxConcurrentStreams bounds how many collection responses can stream at once. Each holds a DB
// cursor — and its pooled connection — for the whole client-paced response, so without a bound
// enough slow clients would take the entire pool and stall the scanner, scrobbles and the UI.
@ -515,11 +512,6 @@ func Load(noConfigDump bool) {
Server.UICoverArtSize = newValue
}
if Server.Scanner.Extractor != consts.DefaultScannerExtractor {
log.Warn("Invalid Scanner.Extractor, using default", "value", Server.Scanner.Extractor, "default", consts.DefaultScannerExtractor)
Server.Scanner.Extractor = consts.DefaultScannerExtractor
}
// Floor MaxImageSize at MaxImageUploadSize so accepted uploads can always be read back.
imgSize, _ := humanize.ParseBytes(Server.MaxImageSize)
uploadSize, _ := humanize.ParseBytes(Server.MaxImageUploadSize)
@ -1095,8 +1087,6 @@ func setViperDefaults() {
viper.SetDefault("listenbrainz.trackalgorithm", consts.DefaultListenBrainzTrackAlgorithm)
viper.SetDefault("jellyfin.enabled", false)
viper.SetDefault("jellyfin.servername", "")
viper.SetDefault("jellyfin.autodiscovery", false)
viper.SetDefault("jellyfin.quickconnect", true)
viper.SetDefault("enablescrobblehistory", true)
viper.SetDefault("httpheaders.frameoptions", "DENY")
viper.SetDefault("backup.path", "")
@ -1125,7 +1115,6 @@ func setViperDefaults() {
viper.SetDefault("devshowartistpage", true)
viper.SetDefault("devuishowconfig", true)
viper.SetDefault("devneweventstream", true)
viper.SetDefault("devapiv1", false)
viper.SetDefault("devoffsetoptimize", 50000)
// Half the pool: streams may take up to this many connections, leaving the rest for the scanner,
// scrobbles and the UI. See MaxOpenConns.

View file

@ -402,27 +402,6 @@ var _ = Describe("Configuration", func() {
})
})
Describe("Scanner.Extractor", func() {
BeforeEach(func() {
viper.Reset()
conf.SetViperDefaults()
viper.SetDefault("datafolder", GinkgoT().TempDir())
viper.SetDefault("loglevel", "error")
conf.ResetConf()
})
It("falls back to taglib for an unknown extractor", func() {
viper.SetDefault("scanner.extractor", "ffmpeg")
conf.Load(true)
Expect(conf.Server.Scanner.Extractor).To(Equal("taglib"))
})
It("keeps taglib", func() {
conf.Load(true)
Expect(conf.Server.Scanner.Extractor).To(Equal("taglib"))
})
})
Describe("EnforceNonRootUser", func() {
It("defaults to false", func() {
conf.Load(true)

View file

@ -49,7 +49,6 @@ const (
DefaultEncryptionKey = "just for obfuscation"
PasswordsEncryptedKey = "PasswordsEncryptedKey"
PasswordAutogenPrefix = "__NAVIDROME_AUTOGEN__" //nolint:gosec
APIKeyPrefix = "nds_"
DevInitialUserName = "admin"
DevInitialName = "Dev Admin"
@ -60,7 +59,6 @@ const (
URLPathPublic = "/share"
URLPathPublicImages = URLPathPublic + "/img"
URLPathJellyfinAPI = "/jellyfin"
URLPathAPIv1 = "/api/v1"
// JellyfinServerIDKey is the Property key for the stable, persisted server Id reported by the
// Jellyfin API. Jellyfin clients cache this value, so it must survive process restarts.
@ -156,6 +154,8 @@ const (
//DefaultAlbumPID = "album_legacy"
DefaultAlbumPID = "musicbrainz_albumid|albumartistid,album,albumversion,releasedate"
DefaultTrackPID = "musicbrainz_trackid|albumid,discnumber,tracknumber,title"
PIDAlbumKey = "PIDAlbum"
PIDTrackKey = "PIDTrack"
)
const (

View file

@ -36,7 +36,7 @@ RestrictNamespaces=yes
RestrictRealtime=yes
SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources
SystemCallFilter=setrlimit mbind
SystemCallFilter=setrlimit
SystemCallArchitectures=native
UMask=0066

View file

@ -24,7 +24,7 @@ func (p *localAgent) AgentName() string {
}
func (p *localAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid string, count int) ([]Song, error) {
top, err := p.ds.MediaFile().GetAll(ctx, model.QueryOptions{
top, err := p.ds.MediaFile(ctx).GetAll(model.QueryOptions{
Sort: "playCount",
Order: "desc",
Max: count,
@ -43,7 +43,7 @@ func (p *localAgent) GetArtistTopSongs(ctx context.Context, id, artistName, mbid
}
func (p *localAgent) GetSimilarSongsByTrack(ctx context.Context, id, name, artist, mbid string, count int) ([]Song, error) {
seed, err := p.ds.MediaFile().Get(ctx, id)
seed, err := p.ds.MediaFile(ctx).Get(id)
if err != nil {
return nil, err
}
@ -53,7 +53,7 @@ func (p *localAgent) GetSimilarSongsByTrack(ctx context.Context, id, name, artis
return nil, nil
}
// Ask for extra so we can drop the seed itself and still fill the count.
candidates, err := p.ds.MediaFile().GetRandom(ctx, model.QueryOptions{
candidates, err := p.ds.MediaFile(ctx).GetRandom(model.QueryOptions{
Filters: squirrel.And{
persistence.SongGenres.ByID(genreIDs),
squirrel.Eq{"missing": false},

View file

@ -13,13 +13,13 @@ type SessionKeys struct {
}
func (sk *SessionKeys) Put(ctx context.Context, userId, sessionKey string) error {
return sk.DataStore.UserProps().Put(ctx, userId, sk.KeyName, sessionKey)
return sk.DataStore.UserProps(ctx).Put(userId, sk.KeyName, sessionKey)
}
func (sk *SessionKeys) Get(ctx context.Context, userId string) (string, error) {
return sk.DataStore.UserProps().Get(ctx, userId, sk.KeyName)
return sk.DataStore.UserProps(ctx).Get(userId, sk.KeyName)
}
func (sk *SessionKeys) Delete(ctx context.Context, userId string) error {
return sk.DataStore.UserProps().Delete(ctx, userId, sk.KeyName)
return sk.DataStore.UserProps(ctx).Delete(userId, sk.KeyName)
}

View file

@ -2,32 +2,23 @@ package core
import (
"archive/zip"
"cmp"
"context"
"errors"
"fmt"
"io"
"net/http"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/stream"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/persistence"
"github.com/navidrome/navidrome/utils/slice"
"github.com/navidrome/navidrome/utils/str"
)
const archiveCoverArtSize = 500
type Archiver interface {
ZipAlbum(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
ZipArtist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
@ -35,20 +26,18 @@ type Archiver interface {
ZipPlaylist(ctx context.Context, id string, format string, bitrate int, w io.Writer) error
}
func NewArchiver(ms stream.MediaStreamer, decider stream.TranscodeDecider, ds model.DataStore, shares Share, artwork artwork.Artwork) Archiver {
return &archiver{ds: ds, ms: ms, decider: decider, shares: shares, artwork: artwork}
func NewArchiver(ms stream.MediaStreamer, ds model.DataStore, shares Share) Archiver {
return &archiver{ds: ds, ms: ms, shares: shares}
}
type archiver struct {
ds model.DataStore
ms stream.MediaStreamer
decider stream.TranscodeDecider
shares Share
artwork artwork.Artwork
ds model.DataStore
ms stream.MediaStreamer
shares Share
}
func (a *archiver) ZipAlbum(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
return a.zipAlbums(ctx, id, format, bitrate, out, squirrel.Eq{"album_id": id}, model.ArtworkID{})
return a.zipAlbums(ctx, id, format, bitrate, out, squirrel.Eq{"album_id": id})
}
func (a *archiver) ZipArtist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
@ -58,30 +47,28 @@ func (a *archiver) ZipArtist(ctx context.Context, id string, format string, bitr
persistence.ParticipantIDFilter("media_file", id, model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}
return a.zipAlbums(ctx, id, format, bitrate, out, filter, model.Artist{ID: id}.CoverArtID())
return a.zipAlbums(ctx, id, format, bitrate, out, filter)
}
// rootArt, when set, is added to the archive root.
func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitrate int, out io.Writer, filters squirrel.Sqlizer, rootArt model.ArtworkID) error {
mfs, err := a.ds.MediaFile().GetAll(ctx, model.QueryOptions{Filters: filters, Sort: "album"})
func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitrate int, out io.Writer, filters squirrel.Sqlizer) error {
mfs, err := a.ds.MediaFile(ctx).GetAll(model.QueryOptions{Filters: filters, Sort: "album"})
if err != nil {
log.Error(ctx, "Error loading mediafiles from artist", "id", id, err)
return err
}
z := createZipWriter(out, format, bitrate)
albums := slice.GroupOrdered(mfs, func(mf model.MediaFile) string { return mf.AlbumID })
folders := albumFolders(albums)
albums := slice.Group(mfs, func(mf model.MediaFile) string {
return mf.AlbumID
})
for _, album := range albums {
discs := slice.Group(album, func(mf model.MediaFile) int { return mf.DiscNumber })
isMultiDisc := len(discs) > 1
folder := folders[album[0].AlbumID]
log.Debug(ctx, "Zipping album", "name", album[0].Album, "artist", album[0].AlbumArtist, "folder", folder,
log.Debug(ctx, "Zipping album", "name", album[0].Album, "artist", album[0].AlbumArtist,
"format", format, "bitrate", bitrate, "isMultiDisc", isMultiDisc, "numTracks", len(album))
for _, mf := range album {
req := a.resolveRequest(ctx, &mf, format, bitrate)
file := a.albumFilename(mf, req.Format, isMultiDisc, folder)
if addErr := a.addFileToZip(ctx, z, mf, req, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
file := a.albumFilename(mf, format, isMultiDisc)
if addErr := a.addFileToZip(ctx, z, mf, format, bitrate, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
// Stop iterating: continuing would just rack up more
// rejections from the limiter. Close finalises whatever
// tracks were already written; the rejected one is not
@ -91,10 +78,7 @@ func (a *archiver) zipAlbums(ctx context.Context, id string, format string, bitr
return addErr
}
}
// After the tracks, so a slow artwork lookup doesn't delay the first bytes.
a.addCoverArtToZip(ctx, z, album[0].AlbumCoverArtID(), folder)
}
a.addCoverArtToZip(ctx, z, rootArt, "")
err = z.Close()
if err != nil {
log.Error(ctx, "Error closing zip file", "id", id, err)
@ -112,61 +96,7 @@ func createZipWriter(out io.Writer, format string, bitrate int) *zip.Writer {
return z
}
// Tried in order; the first one whose values are distinct across the clashing albums wins.
// One album may have an empty value: it keeps the plain name, which the others can't clash with.
var albumDisambiguators = []func(model.MediaFile) string{
func(mf model.MediaFile) string { return mf.Tags.First(model.TagAlbumVersion) },
func(mf model.MediaFile) string {
// Reissues share Year (often the original's) but not ReleaseYear.
if y := cmp.Or(mf.ReleaseYear, mf.Year); y != 0 {
return strconv.Itoa(y)
}
return ""
},
func(mf model.MediaFile) string { return mf.MbzAlbumType },
func(mf model.MediaFile) string { return mf.Tags.First(model.TagRecordLabel) },
func(mf model.MediaFile) string { return mf.CatalogNum },
func(mf model.MediaFile) string { return mf.AlbumID[:min(6, len(mf.AlbumID))] },
func(mf model.MediaFile) string { return mf.AlbumID },
}
// albumFolders maps each album id to its zip folder. Albums whose names sanitize to the
// same folder get a " [suffix]" from the first disambiguator that tells them all apart.
func albumFolders(albums [][]model.MediaFile) map[string]string {
byName := map[string][]model.MediaFile{}
for _, album := range albums {
name := str.SanitizeFilename(album[0].FullAlbumName())
byName[name] = append(byName[name], album[0])
}
folders := make(map[string]string, len(albums))
for name, group := range byName {
if len(group) == 1 {
folders[group[0].AlbumID] = name
continue
}
fields:
for _, field := range albumDisambiguators {
ids := make(map[string]string, len(group)) // suffix -> album id
for _, mf := range group {
s := str.SanitizeFilename(field(mf))
if _, dup := ids[s]; dup {
continue fields
}
ids[s] = mf.AlbumID
}
for s, id := range ids {
folders[id] = name
if s != "" {
folders[id] = fmt.Sprintf("%s [%s]", name, s)
}
}
break
}
}
return folders
}
func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc bool, folder string) string {
func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc bool) string {
_, file := filepath.Split(mf.Path)
if format != "raw" {
file = strings.TrimSuffix(file, mf.Suffix) + format
@ -174,7 +104,7 @@ func (a *archiver) albumFilename(mf model.MediaFile, format string, isMultiDisc
if isMultiDisc {
file = fmt.Sprintf("Disc %02d/%s", mf.DiscNumber, file)
}
return fmt.Sprintf("%s/%s", folder, file)
return fmt.Sprintf("%s/%s", str.SanitizeFilename(mf.Album), file)
}
// ZipShare takes an already-loaded share: Share.Load records a visit, so
@ -184,31 +114,27 @@ func (a *archiver) ZipShare(ctx context.Context, s *model.Share, out io.Writer)
return model.ErrNotAuthorized
}
log.Debug(ctx, "Zipping share", "name", s.ID, "format", s.Format, "bitrate", s.MaxBitRate, "numTracks", len(s.Tracks))
// The share is the authorization (as in the public image handler): an anonymous lookup would
// hide a private playlist. Only the cover read is elevated.
coverCtx := request.WithUser(ctx, model.User{IsAdmin: true})
return a.zipMediaFiles(ctx, s.ID, s.ID, s.Format, s.MaxBitRate, out, s.Tracks, coverCtx, s.CoverArtID(), false)
return a.zipMediaFiles(ctx, s.ID, s.ID, s.Format, s.MaxBitRate, out, s.Tracks, false)
}
func (a *archiver) ZipPlaylist(ctx context.Context, id string, format string, bitrate int, out io.Writer) error {
pls, err := a.ds.Playlist().GetWithTracks(ctx, id, true, false)
pls, err := a.ds.Playlist(ctx).GetWithTracks(id, true, false)
if err != nil {
log.Error(ctx, "Error loading mediafiles from playlist", "id", id, err)
return err
}
mfs := pls.MediaFiles()
log.Debug(ctx, "Zipping playlist", "name", pls.Name, "format", format, "bitrate", bitrate, "numTracks", len(mfs))
return a.zipMediaFiles(ctx, id, pls.Name, format, bitrate, out, mfs, ctx, pls.CoverArtID(), true)
return a.zipMediaFiles(ctx, id, pls.Name, format, bitrate, out, mfs, true)
}
func (a *archiver) zipMediaFiles(ctx context.Context, id, name string, format string, bitrate int, out io.Writer, mfs model.MediaFiles, coverCtx context.Context, coverArt model.ArtworkID, addM3U bool) error {
func (a *archiver) zipMediaFiles(ctx context.Context, id, name string, format string, bitrate int, out io.Writer, mfs model.MediaFiles, addM3U bool) error {
z := createZipWriter(out, format, bitrate)
zippedMfs := make(model.MediaFiles, len(mfs))
for idx, mf := range mfs {
req := a.resolveRequest(ctx, &mf, format, bitrate)
file := a.playlistFilename(mf, req.Format, idx)
if addErr := a.addFileToZip(ctx, z, mf, req, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
file := a.playlistFilename(mf, format, idx)
if addErr := a.addFileToZip(ctx, z, mf, format, bitrate, file); errors.Is(addErr, stream.ErrTooManyTranscodes) {
// Abort the whole archive: continuing would silently emit
// empty zip entries since the headers are already written.
_ = z.Close()
@ -217,7 +143,6 @@ func (a *archiver) zipMediaFiles(ctx context.Context, id, name string, format st
mf.Path = file
zippedMfs[idx] = mf
}
a.addCoverArtToZip(coverCtx, z, coverArt, "")
// Add M3U file if requested
if addM3U && len(zippedMfs) > 0 {
@ -254,14 +179,7 @@ func (a *archiver) playlistFilename(mf model.MediaFile, format string, idx int)
return fmt.Sprintf("%02d - %s - %s.%s", idx+1, str.SanitizeFilename(mf.Artist), str.SanitizeFilename(mf.Title), ext)
}
func (a *archiver) resolveRequest(ctx context.Context, mf *model.MediaFile, format string, bitrate int) stream.Request {
if format == "" || format == "raw" {
return stream.Request{Format: "raw"}
}
return a.decider.ResolveRequest(ctx, mf, format, bitrate, 0)
}
func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.MediaFile, req stream.Request, filename string) error {
func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.MediaFile, format string, bitrate int, filename string) error {
path := mf.AbsolutePath()
// Open the source before writing the zip entry header so a rejection
@ -269,13 +187,13 @@ func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.Med
// archive.
var r io.ReadCloser
var err error
if req.Format != "raw" {
r, err = a.ms.NewStream(ctx, &mf, req)
if format != "raw" && format != "" {
r, err = a.ms.NewStream(ctx, &mf, stream.Request{Format: format, BitRate: bitrate})
} else {
r, err = os.Open(path)
}
if err != nil {
log.Error(ctx, "Error opening file for zipping", "file", path, "format", req.Format, err)
log.Error(ctx, "Error opening file for zipping", "file", path, "format", format, err)
return err
}
defer func() {
@ -302,61 +220,3 @@ func (a *archiver) addFileToZip(ctx context.Context, z *zip.Writer, mf model.Med
return nil
}
// addCoverArtToZip adds the cover as dir/folder.<ext>. Errors are logged, never returned.
func (a *archiver) addCoverArtToZip(ctx context.Context, z *zip.Writer, artID model.ArtworkID, dir string) {
if artID.ID == "" {
return
}
// Buffered so a failed read leaves no empty entry.
data, err := a.readCoverArt(ctx, artID)
if errors.Is(err, artwork.ErrUnavailable) || errors.Is(err, model.ErrNotFound) {
log.Debug(ctx, "No cover art to add to zip", "artID", artID)
return
}
if err != nil {
log.Warn(ctx, "Error reading cover art for zipping", "artID", artID, err)
return
}
ext := coverArtExtension(data)
if ext == "" {
log.Warn(ctx, "Unknown cover art image type, not adding it to zip", "artID", artID)
return
}
w, err := z.CreateHeader(&zip.FileHeader{
Name: path.Join(dir, "folder."+ext),
Modified: time.Now(),
Method: zip.Store,
})
if err != nil {
log.Warn(ctx, "Error creating cover art zip entry", "artID", artID, err)
return
}
if _, err = w.Write(data); err != nil {
log.Warn(ctx, "Error zipping cover art", "artID", artID, err)
}
}
func (a *archiver) readCoverArt(ctx context.Context, artID model.ArtworkID) ([]byte, error) {
img, err := a.artwork.Get(ctx, artID, archiveCoverArtSize, false)
if err != nil {
return nil, err
}
defer img.Close()
return io.ReadAll(img)
}
// Resizing may re-encode the image, so the type comes from its bytes.
func coverArtExtension(data []byte) string {
switch http.DetectContentType(data) {
case "image/jpeg":
return "jpg"
case "image/png":
return "png"
case "image/webp":
return "webp"
case "image/gif":
return "gif"
}
return ""
}

View file

@ -4,18 +4,13 @@ import (
"archive/zip"
"bytes"
"context"
"errors"
"io"
"strings"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/core/artwork"
"github.com/navidrome/navidrome/core/stream"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/request"
"github.com/navidrome/navidrome/persistence"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
@ -26,19 +21,15 @@ var _ = Describe("Archiver", func() {
var (
arch core.Archiver
ms *mockMediaStreamer
dc *fakeDecider
ds *mockDataStore
sh *mockShare
ca *mockCoverArt
)
BeforeEach(func() {
ms = &mockMediaStreamer{}
dc = &fakeDecider{}
sh = &mockShare{}
ds = &mockDataStore{}
ca = &mockCoverArt{images: map[string][]byte{}}
arch = core.NewArchiver(ms, dc, ds, sh, ca)
arch = core.NewArchiver(ms, ds, sh)
})
Context("ZipAlbum", func() {
@ -54,7 +45,7 @@ var _ = Describe("Archiver", func() {
Sort: "album",
}}).Return(mfs, nil)
ds.On("MediaFile").Return(mfRepo)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(3)
out := new(bytes.Buffer)
@ -68,23 +59,6 @@ var _ = Describe("Archiver", func() {
Expect(zr.File[0].Name).To(Equal("Album_Promo/01 - track1.mp3"))
Expect(zr.File[1].Name).To(Equal("Album_Promo/02 - track2.mp3"))
})
It("streams the request resolved by the transcode decider and names the entry after its format", func() {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", mock.Anything).Return(model.MediaFiles{{Path: "test_data/01 - track1.flac", Suffix: "flac", AlbumID: "1"}}, nil)
ds.On("MediaFile").Return(mfRepo)
resolved := stream.Request{Format: "opus", BitRate: 128, SampleRate: 48000, Channels: 2}
dc.resolved = &resolved
ms.On("NewStream", mock.Anything, mock.Anything, resolved).Return(io.NopCloser(strings.NewReader("test")), nil).Once()
out := new(bytes.Buffer)
Expect(arch.ZipAlbum(GinkgoT().Context(), "1", "mp3", 128, out)).To(Succeed())
ms.AssertExpectations(GinkgoT())
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).ToNot(HaveOccurred())
Expect(zr.File[0].Name).To(HaveSuffix("01 - track1.opus"))
})
})
Context("ZipArtist", func() {
@ -103,7 +77,7 @@ var _ = Describe("Archiver", func() {
Sort: "album",
}}).Return(mfs, nil)
ds.On("MediaFile").Return(mfRepo)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
out := new(bytes.Buffer)
@ -117,140 +91,6 @@ var _ = Describe("Archiver", func() {
Expect(zr.File[0].Name).To(Equal("Album 1/01 - track1.mp3"))
Expect(zr.File[1].Name).To(Equal("Album 1/02 - track2.mp3"))
})
When("albums that share a name", func() {
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
})
// zipArtistEntries zips the given tracks as artist "1" and returns the entry names in zip order.
zipArtistEntries := func(mfs model.MediaFiles) []string {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", mock.Anything).Return(mfs, nil)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
out := new(bytes.Buffer)
Expect(arch.ZipArtist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).To(BeNil())
names := make([]string, len(zr.File))
for i, f := range zr.File {
names[i] = f.Name
}
return names
}
It("keeps the albums in query order", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "3", Album: "Album C"},
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album A"},
{Path: "a/02.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album A"},
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Album B"},
})
Expect(names).To(Equal([]string{"Album C/01.mp3", "Album A/01.mp3", "Album A/02.mp3", "Album B/01.mp3"}))
})
It("suffixes the year when it tells the albums apart", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01 - Intro.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01 - Intro.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005},
})
Expect(names).To(Equal([]string{"Greatest Hits [2001]/01 - Intro.mp3", "Greatest Hits [2005]/01 - Intro.mp3"}))
})
It("prefers the release year, so reissues of the same original are told apart", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 1996, ReleaseYear: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 1996, ReleaseYear: 2011},
{Path: "c/01.mp3", Suffix: "mp3", AlbumID: "3", Album: "Greatest Hits", Year: 1996},
})
Expect(names).To(Equal([]string{"Greatest Hits [2001]/01.mp3", "Greatest Hits [2011]/01.mp3", "Greatest Hits [1996]/01.mp3"}))
})
It("names the folder after the full album name", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001,
Tags: model.Tags{model.TagAlbumVersion: {"Original"}}},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005,
Tags: model.Tags{model.TagAlbumVersion: {"CD/Digital"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits (Original)/01.mp3", "Greatest Hits (CD_Digital)/01.mp3"}))
})
It("prefers the album version over the year when it is not part of the name", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001,
Tags: model.Tags{model.TagAlbumVersion: {"Original"}}},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005,
Tags: model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits [Original]/01.mp3", "Greatest Hits [Deluxe Edition]/01.mp3"}))
})
It("leaves the one album without the field unsuffixed", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005,
Tags: model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits/01.mp3", "Greatest Hits [Deluxe Edition]/01.mp3"}))
})
It("skips a field that is empty on more than one album", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005},
{Path: "c/01.mp3", Suffix: "mp3", AlbumID: "3", Album: "Greatest Hits", Year: 2010,
Tags: model.Tags{model.TagAlbumVersion: {"Deluxe Edition"}}},
})
Expect(names).To(Equal([]string{"Greatest Hits [2001]/01.mp3", "Greatest Hits [2005]/01.mp3", "Greatest Hits [2010]/01.mp3"}))
})
It("skips a field that is the same on every album", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Live", Year: 2001, MbzAlbumType: "album", CatalogNum: "CAT-1"},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Live", Year: 2001, MbzAlbumType: "album", CatalogNum: "CAT-2"},
})
Expect(names).To(Equal([]string{"Live [CAT-1]/01.mp3", "Live [CAT-2]/01.mp3"}))
})
It("falls back to the album id when nothing differs", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "0123456789abcdef", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "fedcba9876543210", Album: "Greatest Hits", Year: 2001},
})
Expect(names).To(Equal([]string{"Greatest Hits [012345]/01.mp3", "Greatest Hits [fedcba]/01.mp3"}))
})
It("treats names that sanitize to the same folder as a clash", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "A/B", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: `A\B`, Year: 2005},
})
Expect(names).To(Equal([]string{"A_B [2001]/01.mp3", "A_B [2005]/01.mp3"}))
})
It("sanitizes the suffix", func() {
conf.Server.Subsonic.AppendAlbumVersion = false
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Hits", Tags: model.Tags{model.TagAlbumVersion: {"Vinyl"}}},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Hits", Tags: model.Tags{model.TagAlbumVersion: {"CD/Digital"}}},
})
Expect(names).To(Equal([]string{"Hits [Vinyl]/01.mp3", "Hits [CD_Digital]/01.mp3"}))
})
It("leaves the folder name alone when only one album has it", func() {
names := zipArtistEntries(model.MediaFiles{
{Path: "a/01.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001},
{Path: "b/01.mp3", Suffix: "mp3", AlbumID: "2", Album: "Other", Year: 2005},
})
Expect(names).To(Equal([]string{"Greatest Hits/01.mp3", "Other/01.mp3"}))
})
})
})
Context("when the transcode limiter rejects a file", func() {
@ -265,7 +105,7 @@ var _ = Describe("Archiver", func() {
Filters: squirrel.Eq{"album_id": "1"},
Sort: "album",
}}).Return(mfs, nil)
ds.On("MediaFile").Return(mfRepo)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).
Return(nil, stream.ErrTooManyTranscodes).Once()
@ -315,30 +155,6 @@ var _ = Describe("Archiver", func() {
})
Context("ZipPlaylist", func() {
It("names the entries and the M3U lines after the resolved format", func() {
pls := &model.Playlist{ID: "1", Name: "Test Playlist", Tracks: []model.PlaylistTrack{
{MediaFile: model.MediaFile{Path: "test_data/01 - track1.flac", Suffix: "flac", Artist: "Artist 1", Title: "track1"}},
}}
plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil)
ds.On("Playlist").Return(plRepo)
dc.resolved = &stream.Request{Format: "opus", BitRate: 128}
ms.On("NewStream", mock.Anything, mock.Anything, *dc.resolved).Return(io.NopCloser(strings.NewReader("test")), nil)
out := new(bytes.Buffer)
Expect(arch.ZipPlaylist(GinkgoT().Context(), "1", "mp3", 128, out)).To(Succeed())
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).ToNot(HaveOccurred())
Expect(zr.File[0].Name).To(Equal("01 - Artist 1 - track1.opus"))
m3u, err := zr.File[1].Open()
Expect(err).ToNot(HaveOccurred())
defer m3u.Close()
content, err := io.ReadAll(m3u)
Expect(err).ToNot(HaveOccurred())
Expect(string(content)).To(ContainSubstring("01 - Artist 1 - track1.opus"))
})
It("zips a playlist correctly", func() {
tracks := []model.PlaylistTrack{
{MediaFile: model.MediaFile{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album 1", DiscNumber: 1, Artist: "AC/DC", Title: "track1"}},
@ -353,7 +169,7 @@ var _ = Describe("Archiver", func() {
plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true, false).Return(pls, nil)
ds.On("Playlist").Return(plRepo)
ds.On("Playlist", mock.Anything).Return(plRepo)
ms.On("NewStream", mock.Anything, mock.Anything, stream.Request{Format: "mp3", BitRate: 128}).Return(io.NopCloser(strings.NewReader("test")), nil).Times(2)
out := new(bytes.Buffer)
@ -380,195 +196,24 @@ var _ = Describe("Archiver", func() {
Expect(string(m3uContent)).To(Equal(expectedM3U))
})
})
Context("cover art", func() {
var (
jpegData = []byte("\xff\xd8\xff\xe0 fake jpeg")
pngData = []byte("\x89PNG\x0d\x0a\x1a\x0a fake png")
)
mockAlbumTracks := func(filter squirrel.Sqlizer, mfs model.MediaFiles) {
mfRepo := &mockMediaFileRepository{}
mfRepo.On("GetAll", []model.QueryOptions{{Filters: filter, Sort: "album"}}).Return(mfs, nil)
ds.On("MediaFile", mock.Anything).Return(mfRepo)
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
}
It("adds the album cover to the album folder", func() {
ca.images["al-1"] = jpegData
mockAlbumTracks(squirrel.Eq{"album_id": "1"}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album/Promo", DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipAlbum(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(2))
Expect(files).To(HaveKeyWithValue("Album_Promo/folder.jpg", jpegData))
Expect(ca.requests).To(ConsistOf(coverRequest{id: "al-1", size: 500, square: false}))
})
It("adds the artist image to the root and each album cover to its folder", func() {
ca.images["ar-1"] = pngData
ca.images["al-1"] = jpegData
ca.images["al-2"] = jpegData
mockAlbumTracks(squirrel.And{
persistence.ParticipantIDFilter("media_file", "1", model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album 1", DiscNumber: 1},
{Path: "test_data/02 - track2.mp3", Suffix: "mp3", AlbumID: "2", Album: "Album 2", DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipArtist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(5))
Expect(files).To(HaveKeyWithValue("folder.png", pngData))
Expect(files).To(HaveKeyWithValue("Album 1/folder.jpg", jpegData))
Expect(files).To(HaveKeyWithValue("Album 2/folder.jpg", jpegData))
})
It("puts each same-named album's cover in that album's own folder", func() {
ca.images["al-1"] = jpegData
ca.images["al-2"] = pngData
mockAlbumTracks(squirrel.And{
persistence.ParticipantIDFilter("media_file", "1", model.RoleAlbumArtist),
squirrel.Eq{"missing": false},
}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Greatest Hits", Year: 2001, DiscNumber: 1},
{Path: "test_data/02 - track2.mp3", Suffix: "mp3", AlbumID: "2", Album: "Greatest Hits", Year: 2005, DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipArtist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveKeyWithValue("Greatest Hits [2001]/folder.jpg", jpegData))
Expect(files).To(HaveKeyWithValue("Greatest Hits [2005]/folder.png", pngData))
})
It("adds the playlist cover to the root", func() {
ca.images["pl-1"] = jpegData
plRepo := &mockPlaylistRepository{}
plRepo.On("GetWithTracks", "1", true, false).Return(&model.Playlist{
ID: "1",
Name: "Test Playlist",
Tracks: []model.PlaylistTrack{
{MediaFile: model.MediaFile{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Artist: "Artist 1", Title: "track1"}},
},
}, nil)
ds.On("Playlist", mock.Anything).Return(plRepo)
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
out := new(bytes.Buffer)
Expect(arch.ZipPlaylist(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(3))
Expect(files).To(HaveKeyWithValue("folder.jpg", jpegData))
Expect(files).To(HaveKey("Test Playlist.m3u"))
})
It("adds the shared item's cover to the root, even for a private playlist", func() {
ca.images["pl-10"] = jpegData
ms.On("NewStream", mock.Anything, mock.Anything, mock.Anything).Return(io.NopCloser(strings.NewReader("test")), nil)
share := &model.Share{
ID: "1",
Downloadable: true,
Format: "mp3",
MaxBitRate: 128,
ResourceType: "playlist",
ResourceIDs: "10",
Tracks: model.MediaFiles{
{ID: "1", Path: "test_data/01 - track1.mp3", Suffix: "mp3", Artist: "Artist 1", Title: "track1"},
},
}
out := new(bytes.Buffer)
Expect(arch.ZipShare(context.Background(), share, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(2))
Expect(files).To(HaveKeyWithValue("folder.jpg", jpegData))
Expect(ca.requests).To(ConsistOf(coverRequest{id: "pl-10", size: 500, square: false, admin: true}))
})
It("still builds the archive when the cover cannot be read", func() {
ca.err = errors.New("boom")
mockAlbumTracks(squirrel.Eq{"album_id": "1"}, model.MediaFiles{
{Path: "test_data/01 - track1.mp3", Suffix: "mp3", AlbumID: "1", Album: "Album", DiscNumber: 1},
})
out := new(bytes.Buffer)
Expect(arch.ZipAlbum(context.Background(), "1", "mp3", 128, out)).To(Succeed())
files := readZip(out)
Expect(files).To(HaveLen(1))
Expect(files).To(HaveKey("Album/01 - track1.mp3"))
})
})
})
func readZip(out *bytes.Buffer) map[string][]byte {
zr, err := zip.NewReader(bytes.NewReader(out.Bytes()), int64(out.Len()))
Expect(err).ToNot(HaveOccurred())
files := make(map[string][]byte, len(zr.File))
for _, f := range zr.File {
r, err := f.Open()
Expect(err).ToNot(HaveOccurred())
data, err := io.ReadAll(r)
Expect(err).ToNot(HaveOccurred())
_ = r.Close()
files[f.Name] = data
}
return files
}
type coverRequest struct {
id string
size int
square bool
admin bool
}
type mockCoverArt struct {
artwork.Artwork
images map[string][]byte
err error
requests []coverRequest
}
func (m *mockCoverArt) Get(ctx context.Context, artID model.ArtworkID, size int, square bool) (*artwork.Image, error) {
user, _ := request.UserFrom(ctx)
m.requests = append(m.requests, coverRequest{id: artID.String(), size: size, square: square, admin: user.IsAdmin})
if m.err != nil {
return nil, m.err
}
data, ok := m.images[artID.String()]
if !ok {
return nil, artwork.ErrUnavailable
}
return &artwork.Image{ReadCloser: io.NopCloser(bytes.NewReader(data))}, nil
}
type mockDataStore struct {
mock.Mock
model.DataStore
}
func (m *mockDataStore) MediaFile() model.MediaFileRepository {
args := m.Called()
func (m *mockDataStore) MediaFile(ctx context.Context) model.MediaFileRepository {
args := m.Called(ctx)
return args.Get(0).(model.MediaFileRepository)
}
func (m *mockDataStore) Playlist() model.PlaylistRepository {
args := m.Called()
func (m *mockDataStore) Playlist(ctx context.Context) model.PlaylistRepository {
args := m.Called(ctx)
return args.Get(0).(model.PlaylistRepository)
}
func (m *mockDataStore) Library() model.LibraryRepository {
func (m *mockDataStore) Library(context.Context) model.LibraryRepository {
return &mockLibraryRepository{}
}
@ -577,7 +222,7 @@ type mockLibraryRepository struct {
model.LibraryRepository
}
func (m *mockLibraryRepository) GetPath(_ context.Context, id int) (string, error) {
func (m *mockLibraryRepository) GetPath(id int) (string, error) {
return "/music", nil
}
@ -586,7 +231,7 @@ type mockMediaFileRepository struct {
model.MediaFileRepository
}
func (m *mockMediaFileRepository) GetAll(ctx context.Context, options ...model.QueryOptions) (model.MediaFiles, error) {
func (m *mockMediaFileRepository) GetAll(options ...model.QueryOptions) (model.MediaFiles, error) {
args := m.Called(options)
return args.Get(0).(model.MediaFiles), args.Error(1)
}
@ -596,7 +241,7 @@ type mockPlaylistRepository struct {
model.PlaylistRepository
}
func (m *mockPlaylistRepository) GetWithTracks(_ context.Context, id string, refreshSmartPlaylists, includeMissing bool) (*model.Playlist, error) {
func (m *mockPlaylistRepository) GetWithTracks(id string, refreshSmartPlaylists, includeMissing bool) (*model.Playlist, error) {
args := m.Called(id, refreshSmartPlaylists, includeMissing)
return args.Get(0).(*model.Playlist), args.Error(1)
}
@ -614,19 +259,6 @@ func (m *mockMediaStreamer) NewStream(ctx context.Context, mf *model.MediaFile,
return &stream.Stream{ReadCloser: args.Get(0).(io.ReadCloser)}, nil
}
// fakeDecider echoes the legacy format/bitrate unless a resolved request is set.
type fakeDecider struct {
stream.TranscodeDecider
resolved *stream.Request
}
func (f *fakeDecider) ResolveRequest(_ context.Context, _ *model.MediaFile, format string, bitRate int, offset int) stream.Request {
if f.resolved != nil {
return *f.resolved
}
return stream.Request{Format: format, BitRate: bitRate, Offset: offset}
}
type mockShare struct {
mock.Mock
core.Share

View file

@ -16,6 +16,7 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/resources"
"github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/utils/cache"
)
@ -31,6 +32,7 @@ type Image struct {
ETag string // representation validator; "" means Hash applies (full-size original)
LastUpdated time.Time
Placeholder bool
Transient bool // stand-in while the real representation is produced in the background
}
// representationTag varies with dimensions and encode settings, so a config change invalidates
@ -48,8 +50,8 @@ type Artwork interface {
GetOrPlaceholder(ctx context.Context, id string, size int, square bool) (*Image, error)
}
func NewArtwork(ds model.DataStore, cache cache.FileCache, store *ImageStore, ffm ffmpeg.FFmpeg) Artwork {
return &service{ds: ds, cache: cache, store: store, ffmpeg: ffm}
func NewArtwork(ds model.DataStore, cache cache.FileCache, store *ImageStore, ffm ffmpeg.FFmpeg, broker events.Broker) Artwork {
return &service{ds: ds, cache: cache, store: store, ffmpeg: ffm, broker: broker}
}
// entityExists reports whether the entity an artwork id points at is still there: state rows
@ -59,21 +61,21 @@ func entityExists(ctx context.Context, ds model.DataStore, artID model.ArtworkID
var err error
switch artID.Kind {
case model.KindArtistArtwork:
found, err = ds.Artist().Exists(ctx, artID.ID)
found, err = ds.Artist(ctx).Exists(artID.ID)
case model.KindAlbumArtwork:
found, err = ds.Album().Exists(ctx, artID.ID)
found, err = ds.Album(ctx).Exists(artID.ID)
case model.KindMediaFileArtwork:
found, err = ds.MediaFile().Exists(ctx, artID.ID)
found, err = ds.MediaFile(ctx).Exists(artID.ID)
case model.KindPlaylistArtwork:
found, err = ds.Playlist().Exists(ctx, artID.ID)
found, err = ds.Playlist(ctx).Exists(artID.ID)
case model.KindRadioArtwork:
found, err = ds.Radio().Exists(ctx, artID.ID)
found, err = ds.Radio(ctx).Exists(artID.ID)
case model.KindDiscArtwork:
albumID, _, perr := model.ParseDiscArtworkID(artID.ID)
if perr != nil {
return false
}
found, err = ds.Album().Exists(ctx, albumID)
found, err = ds.Album(ctx).Exists(albumID)
default:
return false
}
@ -85,6 +87,7 @@ type service struct {
cache cache.FileCache
store *ImageStore
ffmpeg ffmpeg.FFmpeg
broker events.Broker
}
func (s *service) GetOrPlaceholder(ctx context.Context, id string, size int, square bool) (*Image, error) {
@ -119,7 +122,7 @@ func (s *service) Get(ctx context.Context, artID model.ArtworkID, size int, squa
}
func (s *service) serveEntity(ctx context.Context, artID model.ArtworkID, size int, square bool) (*Image, error) {
ia, err := s.ds.Artwork().GetItemArtwork(ctx, artID.Kind, artID.ID, model.ImageTypePrimary)
ia, err := s.ds.Artwork(ctx).GetItemArtwork(artID.Kind, artID.ID, model.ImageTypePrimary)
switch {
case errors.Is(err, model.ErrNotFound):
return s.provisional(ctx, artID, size, square)
@ -135,7 +138,7 @@ func (s *service) serveEntity(ctx context.Context, artID model.ArtworkID, size i
// serveSource is the one place bytes become an Image. hash is the pixel identity ("" for disc art)
// and doubles as the full-size validator, so an ETag is only needed when resized or hash is "".
func (s *service) serveSource(ctx context.Context, key, hash string, lastUpdate time.Time,
func (s *service) serveSource(ctx context.Context, artID model.ArtworkID, key, hash string, lastUpdate time.Time,
size int, square bool, open func() (io.ReadCloser, error),
) (*Image, error) {
if size == 0 && !square {
@ -152,15 +155,55 @@ func (s *service) serveSource(ctx context.Context, key, hash string, lastUpdate
}
return img, nil
}
stream, err := s.cache.Get(ctx, &resizedItem{
hash: key, size: size, square: square, ffmpeg: s.ffmpeg, open: open,
})
// Without a cache a background conversion could never be served, so convert inline instead.
item := &resizedItem{hash: key, size: size, square: square, ffmpeg: s.ffmpeg, open: open,
deferAnimated: s.cache.Available(ctx)}
stream, err := s.cache.Get(ctx, item)
if deferred, ok := errors.AsType[*deferredAnimation](err); ok {
s.convertInBackground(artID, item, deferred.data)
return &Image{ReadCloser: io.NopCloser(deferred.standIn), Hash: hash, LastUpdated: lastUpdate, Transient: true}, nil
}
if err != nil {
return nil, err
}
return &Image{ReadCloser: stream, Hash: hash, ETag: representationTag(key, size, square), LastUpdated: lastUpdate}, nil
}
// animConvertSlot bounds animated conversions process-wide: they outlive their request, so no
// request throttle limits them.
var animConvertSlot = make(chan struct{}, 1)
const animConvertTimeout = time.Minute
// convertInBackground caches item's conversion of data. When the slot is busy it does nothing,
// and a later request for the key tries again.
func (s *service) convertInBackground(artID model.ArtworkID, item *resizedItem, data []byte) {
select {
case animConvertSlot <- struct{}{}:
default:
return
}
key, size, square := item.Key(), item.size, item.square
go func() {
defer func() { <-animConvertSlot }()
ctx, cancel := context.WithTimeout(context.Background(), animConvertTimeout)
defer cancel()
start := time.Now()
resized, _, err := resizeImageData(ctx, s.ffmpeg, data, size, square)
out, _ := io.ReadAll(orOriginal(data, resized, err))
// Converting before touching the cache keeps the entry from blocking readers for the whole conversion.
if err := warmCache(ctx, s.cache, &storedItem{key: key, data: out}); err != nil {
log.Warn(ctx, "Artwork: Could not cache animated image", "key", key, err)
return
}
log.Debug(ctx, "Artwork: Converted animated image", "key", key, "bytes", len(out), "elapsed", time.Since(start))
// The image URL is unchanged, so UIs holding the stand-in only reload it when told.
if res, ok := artworkKindToResource[artID.Kind]; ok {
s.broker.SendBroadcastMessage(ctx, (&events.RefreshResource{}).With(res, artID.ID))
}
}()
}
// serveHash serves the bytes of a found state row. A mismatch/open error is dangling, but a
// cancelled request is not: it must not enqueue a re-resolution.
func (s *service) serveHash(ctx context.Context, artID model.ArtworkID, ia *model.ItemArtwork, size int, square bool) (*Image, error) {
@ -168,19 +211,14 @@ func (s *service) serveHash(ctx context.Context, artID model.ArtworkID, ia *mode
if !entityExists(ctx, s.ds, artID) {
return nil, ErrUnavailable
}
// Checked here, not in openOriginal: a resize-cache hit never opens the source.
if isFileBacked(ia.Source) && !model.IsImageFile(ia.SourcePath) {
log.Warn(ctx, "Artwork: Stored source is not an image file, re-resolving", "artID", artID, "path", ia.SourcePath)
return s.dangling(ctx, artID)
}
art, err := s.ds.Artwork().GetImage(ctx, ia.Hash)
art, err := s.ds.Artwork(ctx).GetImage(ia.Hash)
if err != nil {
if errors.Is(err, model.ErrNotFound) {
return s.dangling(ctx, artID)
}
return nil, err
}
img, err := s.serveSource(ctx, ia.Hash, ia.Hash, ia.UpdatedAt, size, square,
img, err := s.serveSource(ctx, artID, ia.Hash, ia.Hash, ia.UpdatedAt, size, square,
func() (io.ReadCloser, error) { return openOriginal(ia, art.Mime, s.store) })
if err != nil {
if errors.Is(err, context.Canceled) {
@ -238,11 +276,11 @@ func (s *service) provisional(ctx context.Context, artID model.ArtworkID, size i
s.enqueue(ctx, artID, model.ArtworkPriorityBump)
log.Debug(ctx, "Artwork: Provisional read-through, no state row yet", "artID", artID,
"source", res.source, "hit", res.reader != nil)
return s.serveResolution(ctx, res, size, square)
return s.serveResolution(ctx, artID, res, size, square)
}
// serveResolution turns a local resolution's bytes into a servable Image (byte-hash only, no decode).
func (s *service) serveResolution(ctx context.Context, res resolution, size int, square bool) (*Image, error) {
func (s *service) serveResolution(ctx context.Context, artID model.ArtworkID, res resolution, size int, square bool) (*Image, error) {
if res.reader == nil {
return nil, ErrUnavailable
}
@ -256,7 +294,7 @@ func (s *service) serveResolution(ctx context.Context, res resolution, size int,
return nil, ErrUnavailable
}
// Keyed by the byte-hash, so the entry lines up with the worker's eventual store entry.
return s.serveSource(ctx, hash, hash, unixMtime(res.refMtime), size, square,
return s.serveSource(ctx, artID, hash, hash, unixMtime(res.refMtime), size, square,
func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(data)), nil })
}
@ -264,13 +302,13 @@ func (s *service) serveMediaFile(ctx context.Context, artID model.ArtworkID, siz
// The setting is not in the config fingerprint, so honor it at serve time: a direct mf- URL
// must fall back to disc/album instead of serving stale persisted embedded art.
if !conf.Server.EnableMediaFileCoverArt {
mf, err := s.ds.MediaFile().Get(ctx, artID.ID)
mf, err := s.ds.MediaFile(ctx).Get(artID.ID)
if err != nil {
return nil, err
}
return s.Get(ctx, mf.DiscCoverArtID(), size, square)
}
ia, err := s.ds.Artwork().GetItemArtwork(ctx, model.KindMediaFileArtwork, artID.ID, model.ImageTypePrimary)
ia, err := s.ds.Artwork(ctx).GetItemArtwork(model.KindMediaFileArtwork, artID.ID, model.ImageTypePrimary)
switch {
case err == nil && ia.Hash != "":
return s.serveHash(ctx, artID, ia, size, square)
@ -283,7 +321,7 @@ func (s *service) serveMediaFile(ctx context.Context, artID model.ArtworkID, siz
}
noRow := errors.Is(err, model.ErrNotFound)
mf, err := s.ds.MediaFile().Get(ctx, artID.ID)
mf, err := s.ds.MediaFile(ctx).Get(artID.ID)
if err != nil {
return nil, err
}
@ -306,7 +344,7 @@ func (s *service) provisionalEmbedded(ctx context.Context, artID model.ArtworkID
// Eligible but unextractable: fall back the way CoverArtID does, not to a placeholder.
return s.Get(ctx, mf.DiscCoverArtID(), size, square)
}
return s.serveResolution(ctx, res, size, square)
return s.serveResolution(ctx, artID, res, size, square)
}
// serveDisc reads disc art through with no state row and no enqueue, falling back to the album cover.
@ -324,7 +362,7 @@ func (s *service) serveDisc(ctx context.Context, artID model.ArtworkID, size int
// Disc art has no state row, hence no content hash: keying on id, album mtime and
// DiscArtPriority lets a warm cache answer without running the chain or touching the disk.
key := fmt.Sprintf("%s|%d|%s", artID.ID, dr.cacheTime().UnixNano(), conf.Server.DiscArtPriority)
img, err := s.serveSource(ctx, key, "", dr.cacheTime(), size, square, selectImage)
img, err := s.serveSource(ctx, artID, key, "", dr.cacheTime(), size, square, selectImage)
if err != nil {
if errors.Is(err, context.Canceled) {
return nil, err
@ -342,7 +380,7 @@ func (s *service) dangling(ctx context.Context, artID model.ArtworkID) (*Image,
}
func (s *service) enqueue(ctx context.Context, artID model.ArtworkID, priority int) {
err := s.ds.ArtworkQueue().EnqueuePreservingBackoff(ctx, model.ArtworkQueueItem{
err := s.ds.ArtworkQueue(ctx).EnqueuePreservingBackoff(model.ArtworkQueueItem{
ItemKind: artID.Kind.Prefix(),
ItemID: artID.ID,
ImageType: model.ImageTypePrimary,

View file

@ -1,23 +1,19 @@
package artwork
import (
"context"
"io/fs"
"net/netip"
"net/url"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"time"
"github.com/navidrome/navidrome/core/storage"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/model/metadata"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/httpclient"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"go.uber.org/goleak"
@ -41,14 +37,6 @@ func TestArtwork(t *testing.T) {
RunSpecs(t, "Artwork Suite")
}
// productionImageClient keeps the guarded client for the specs that assert it refuses loopback.
var productionImageClient = remoteImageClient
// httptest servers listen on loopback, which the production client refuses.
var _ = BeforeSuite(func() {
remoteImageClient = httpclient.NewExternal(5*time.Second, netip.MustParsePrefix("127.0.0.0/8"), netip.MustParsePrefix("::1/128"))
})
// osDirFS wraps os.DirFS as a storage.MusicFS for integration tests.
type osDirFS struct{ fs.FS }
@ -109,15 +97,15 @@ type fakeFolderRepo struct {
otherAudioErr error
}
func (f *fakeFolderRepo) GetAll(context.Context, ...model.QueryOptions) ([]model.Folder, error) {
func (f *fakeFolderRepo) GetAll(...model.QueryOptions) ([]model.Folder, error) {
return f.result, f.err
}
func (f *fakeFolderRepo) HasAudioOutsideFolders(context.Context, model.Folder, []string) (bool, error) {
func (f *fakeFolderRepo) HasAudioOutsideFolders(model.Folder, []string) (bool, error) {
return f.hasOtherAudio, f.otherAudioErr
}
func (f *fakeFolderRepo) Get(context.Context, string) (*model.Folder, error) {
func (f *fakeFolderRepo) Get(string) (*model.Folder, error) {
f.getCallCount++
if f.getErr != nil {
return nil, f.getErr

View file

@ -3,10 +3,12 @@ package artwork
import (
"bytes"
"context"
"errors"
"image"
"io"
"os"
"path/filepath"
"sync"
"time"
"github.com/navidrome/navidrome/conf"
@ -14,6 +16,7 @@ import (
"github.com/navidrome/navidrome/consts"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/resources"
"github.com/navidrome/navidrome/server/events"
"github.com/navidrome/navidrome/tests"
"github.com/navidrome/navidrome/utils/cache"
. "github.com/onsi/ginkgo/v2"
@ -45,8 +48,8 @@ var _ = Describe("Artwork", func() {
hash, err := hashImage(bytes.NewReader(imgBytes))
Expect(err).ToNot(HaveOccurred())
Expect(store.Write(hash, "image/jpeg", bytes.NewReader(imgBytes))).To(Succeed())
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: hash, Mime: "image/jpeg"})).To(Succeed())
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: kind, ItemID: id, Hash: hash, Source: "external"})).To(Succeed())
Expect(artRepo.PutImage(&model.Artwork{Hash: hash, Mime: "image/jpeg"})).To(Succeed())
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: kind, ItemID: id, Hash: hash, Source: "external"})).To(Succeed())
seedEntity(kind, id)
return hash
}
@ -56,9 +59,9 @@ var _ = Describe("Artwork", func() {
GinkgoHelper()
switch kind {
case "al":
Expect(albumRepo.Put(ctx, &model.Album{ID: id, Name: "Album"})).To(Succeed())
Expect(albumRepo.Put(&model.Album{ID: id, Name: "Album"})).To(Succeed())
case "mf":
Expect(mfRepo.Put(ctx, &model.MediaFile{ID: id})).To(Succeed())
Expect(mfRepo.Put(&model.MediaFile{ID: id})).To(Succeed())
}
}
@ -107,7 +110,7 @@ var _ = Describe("Artwork", func() {
return arg.(artworkReader).Reader(ctx)
})
Eventually(func() bool { return imgCache.Available(ctx) }, 10*time.Second).Should(BeTrue())
svc = NewArtwork(ds, imgCache, store, ffm)
svc = NewArtwork(ds, imgCache, store, ffm, events.NoopBroker())
})
Describe("found state", func() {
@ -147,9 +150,9 @@ var _ = Describe("Artwork", func() {
imgPath := filepath.Join(dir, "cover.jpg")
Expect(os.WriteFile(imgPath, coverBytes, 0600)).To(Succeed())
mtime := fileMtime(imgPath)
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: "aaaaaaaaaaaaaaaa", Mime: "image/jpeg"})).To(Succeed())
Expect(artRepo.PutImage(&model.Artwork{Hash: "aaaaaaaaaaaaaaaa", Mime: "image/jpeg"})).To(Succeed())
seedEntity("al", "al2")
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "al", ItemID: "al2", Hash: "aaaaaaaaaaaaaaaa",
Source: "folder", SourcePath: imgPath, RefMtime: mtime,
})).To(Succeed())
@ -159,60 +162,13 @@ var _ = Describe("Artwork", func() {
Expect(readAll(img)).To(Equal(coverBytes))
})
It("treats a file-backed row pointing at a non-image file as dangling", func() {
dir := GinkgoT().TempDir()
secretPath := filepath.Join(dir, "config.ini")
Expect(os.WriteFile(secretPath, []byte("password=secret"), 0600)).To(Succeed())
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: "dddddddddddddddd", Mime: "image/jpeg"})).To(Succeed())
seedEntity("al", "alni")
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
ItemKind: "al", ItemID: "alni", Hash: "dddddddddddddddd",
Source: "folder", SourcePath: secretPath, RefMtime: fileMtime(secretPath),
})).To(Succeed())
_, err := svc.Get(ctx, model.MustParseArtworkID("al-alni"), 0, false)
Expect(err).To(MatchError(ErrUnavailable))
Expect(queueRepo.Data[primaryKey("al", "alni")].Priority).To(Equal(model.ArtworkPriorityScan))
})
It("refuses a non-image file-backed row even when a resized copy is already cached", func() {
secret := []byte("password=secret")
dir := GinkgoT().TempDir()
secretPath := filepath.Join(dir, "config.ini")
Expect(os.WriteFile(secretPath, secret, 0600)).To(Succeed())
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: "eeeeeeeeeeeeeeee", Mime: "image/jpeg"})).To(Succeed())
seedEntity("al", "alnic")
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
ItemKind: "al", ItemID: "alnic", Hash: "eeeeeeeeeeeeeeee",
Source: "folder", SourcePath: secretPath, RefMtime: fileMtime(secretPath),
})).To(Succeed())
// Older versions cached the raw bytes when the resize failed.
seed := func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(secret)), nil }
stream, err := imgCache.Get(ctx, &resizedItem{hash: "eeeeeeeeeeeeeeee", size: 100, open: seed, ffmpeg: ffm})
Expect(err).ToNot(HaveOccurred())
Expect(io.ReadAll(stream)).To(Equal(secret))
Expect(stream.Close()).To(Succeed())
Eventually(func(g Gomega) {
s, err := imgCache.Get(ctx, &resizedItem{hash: "eeeeeeeeeeeeeeee", size: 100, ffmpeg: ffm,
open: func() (io.ReadCloser, error) { return nil, os.ErrNotExist }})
g.Expect(err).ToNot(HaveOccurred())
g.Expect(s.Cached).To(BeTrue())
_ = s.Close()
}).Should(Succeed())
_, err = svc.Get(ctx, model.MustParseArtworkID("al-alnic"), 100, false)
Expect(err).To(MatchError(ErrUnavailable))
Expect(queueRepo.Data[primaryKey("al", "alnic")].Priority).To(Equal(model.ArtworkPriorityScan))
})
It("treats a full-size mtime mismatch as dangling: unavailable, re-enqueued at Scan, state untouched", func() {
dir := GinkgoT().TempDir()
imgPath := filepath.Join(dir, "cover.jpg")
Expect(os.WriteFile(imgPath, coverBytes, 0600)).To(Succeed())
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: "bbbbbbbbbbbbbbbb", Mime: "image/jpeg"})).To(Succeed())
Expect(artRepo.PutImage(&model.Artwork{Hash: "bbbbbbbbbbbbbbbb", Mime: "image/jpeg"})).To(Succeed())
seedEntity("al", "al3")
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "al", ItemID: "al3", Hash: "bbbbbbbbbbbbbbbb",
Source: "folder", SourcePath: imgPath, RefMtime: fileMtime(imgPath) + 999,
})).To(Succeed())
@ -220,7 +176,7 @@ var _ = Describe("Artwork", func() {
_, err := svc.Get(ctx, model.MustParseArtworkID("al-al3"), 0, false)
Expect(err).To(MatchError(ErrUnavailable))
Expect(queueRepo.Data[primaryKey("al", "al3")].Priority).To(Equal(model.ArtworkPriorityScan))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al3", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al3", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).To(Equal("bbbbbbbbbbbbbbbb"))
})
@ -229,9 +185,9 @@ var _ = Describe("Artwork", func() {
dir := GinkgoT().TempDir()
imgPath := filepath.Join(dir, "cover.jpg")
Expect(os.WriteFile(imgPath, coverBytes, 0600)).To(Succeed())
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: "cccccccccccccccc", Mime: "image/jpeg"})).To(Succeed())
Expect(artRepo.PutImage(&model.Artwork{Hash: "cccccccccccccccc", Mime: "image/jpeg"})).To(Succeed())
seedEntity("al", "al3b")
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "al", ItemID: "al3b", Hash: "cccccccccccccccc",
Source: "folder", SourcePath: imgPath, RefMtime: fileMtime(imgPath) + 999,
})).To(Succeed())
@ -252,7 +208,7 @@ var _ = Describe("Artwork", func() {
})
It("never re-enqueues an absent state on view, however old", func() {
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "al", ItemID: "al4", AttemptedAt: time.Now().Add(-365 * 24 * time.Hour),
})).To(Succeed())
@ -272,7 +228,7 @@ var _ = Describe("Artwork", func() {
Expect(readAll(img)).To(Equal(coverBytes))
Expect(queueRepo.Data[primaryKey("al", "al5")].Priority).To(Equal(model.ArtworkPriorityBump))
_, err = artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al5", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindAlbumArtwork, "al5", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -283,7 +239,7 @@ var _ = Describe("Artwork", func() {
_, err := svc.Get(ctx, model.MustParseArtworkID("al-al6"), 0, false)
Expect(err).To(MatchError(ErrUnavailable))
Expect(queueRepo.Data[primaryKey("al", "al6")].Priority).To(Equal(model.ArtworkPriorityBump))
_, err = artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al6", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindAlbumArtwork, "al6", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
})
@ -310,7 +266,7 @@ var _ = Describe("Artwork", func() {
It("delegates to the album when the track's state is absent", func() {
seedFoundStore("al", "albm", coverBytes)
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "mf", ItemID: "mf2"})).To(Succeed())
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: "mf", ItemID: "mf2"})).To(Succeed())
mfRepo.SetData(model.MediaFiles{{ID: "mf2", AlbumID: "albm"}})
img, err := svc.Get(ctx, model.MustParseArtworkID("mf-mf2"), 0, false)
@ -343,7 +299,7 @@ var _ = Describe("Artwork", func() {
Expect(err).ToNot(HaveOccurred())
Expect(len(readAll(img))).To(BeNumerically(">", 0))
Expect(queueRepo.Data[primaryKey("mf", "mf4")].Priority).To(Equal(model.ArtworkPriorityBump))
_, err = artRepo.GetItemArtwork(ctx, model.KindMediaFileArtwork, "mf4", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mf4", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -469,6 +425,95 @@ var _ = Describe("Artwork", func() {
})
})
Describe("animated GIF", func() {
var (
gifBytes []byte
fake *animFFmpeg
broker *fakeEventBroker
)
get := func(id string) *Image {
GinkgoHelper()
img, err := svc.Get(ctx, model.MustParseArtworkID(id), 100, false)
Expect(err).ToNot(HaveOccurred())
return img
}
waitForConversions := func() {
GinkgoHelper()
// Taking the slot, not len(), gives the race detector a happens-before with the conversion.
Eventually(func() bool {
select {
case animConvertSlot <- struct{}{}:
<-animConvertSlot
return true
default:
return false
}
}).Should(BeTrue())
}
BeforeEach(func() {
gifBytes = createAnimatedGIF(3)
fake = &animFFmpeg{MockFFmpeg: tests.NewMockFFmpeg(""), out: []byte("animated-webp")}
broker = &fakeEventBroker{}
svc = NewArtwork(ds, imgCache, store, fake, broker)
seedFoundStore("al", "al1", gifBytes)
DeferCleanup(waitForConversions)
})
It("serves a transient stand-in, then the conversion from the cache", func() {
img := get("al-al1")
Expect(img.Transient).To(BeTrue())
Expect(readAll(img)).To(Equal(gifBytes))
waitForConversions()
img = get("al-al1")
Expect(img.Transient).To(BeFalse())
Expect(readAll(img)).To(Equal([]byte("animated-webp")))
Expect(fake.calls.Load()).To(Equal(int32(1)))
})
It("tells the UI to reload the item once its conversion is cached", func() {
readAll(get("al-al1"))
waitForConversions()
sent := broker.getEvents()
Expect(sent).To(HaveLen(1))
Expect(sent[0].Data(sent[0])).To(MatchJSON(`{"album":["al1"]}`))
})
It("runs one conversion at a time and does not queue the others", func() {
fake.release = make(chan struct{})
release := sync.OnceFunc(func() { close(fake.release) })
DeferCleanup(release)
seedFoundStore("al", "al2", createAnimatedGIF(4))
readAll(get("al-al1"))
Eventually(fake.calls.Load).Should(Equal(int32(1)))
img := get("al-al2")
Expect(img.Transient).To(BeTrue())
readAll(img)
Consistently(fake.calls.Load, "100ms").Should(Equal(int32(1)))
release()
waitForConversions()
img = get("al-al2")
Expect(img.Transient).To(BeTrue())
readAll(img)
})
It("caches the static fallback when the conversion fails, so it is not retried", func() {
fake.Error = errors.New("pipe:0: Input/output error")
readAll(get("al-al1"))
waitForConversions()
img := get("al-al1")
Expect(img.Transient).To(BeFalse())
Expect(readAll(img)).To(Equal(gifBytes))
Expect(fake.calls.Load()).To(Equal(int32(1)))
})
})
Describe("GetOrPlaceholder", func() {
It("accepts a raw entity id and serves its cover art", func() {
albumRepo.SetData(model.Albums{{ID: "rawal", Name: "Album"}})
@ -494,7 +539,7 @@ var _ = Describe("Artwork", func() {
})
It("falls back to the artist placeholder for an absent artist", func() {
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "ar", ItemID: "arph"})).To(Succeed())
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: "ar", ItemID: "arph"})).To(Succeed())
img, err := svc.GetOrPlaceholder(ctx, "ar-arph", 300, false)
Expect(err).ToNot(HaveOccurred())
@ -535,7 +580,7 @@ var _ = Describe("EntityExists", func() {
artistRepo := tests.CreateMockArtistRepo()
artistRepo.SetData(model.Artists{{ID: "ar1"}})
radioRepo := tests.CreateMockedRadioRepo()
Expect(radioRepo.Put(ctx, &model.Radio{ID: "ra1", Name: "R"})).To(Succeed())
Expect(radioRepo.Put(&model.Radio{ID: "ra1", Name: "R"})).To(Succeed())
ds = &tests.MockDataStore{MockedAlbum: albumRepo, MockedArtist: artistRepo, MockedRadio: radioRepo}
})

View file

@ -45,7 +45,7 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
return nil, fmt.Errorf("invalid disc artwork id '%s': %w", artID.ID, err)
}
al, err := ds.Album().Get(ctx, albumID)
al, err := ds.Album(ctx).Get(albumID)
if err != nil {
return nil, err
}
@ -61,7 +61,7 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
}
// Query mediafiles for this album + disc to find folder associations and first track
mfs, err := ds.MediaFile().GetAll(ctx, model.QueryOptions{
mfs, err := ds.MediaFile(ctx).GetAll(model.QueryOptions{
Sort: "track_number",
Order: "ASC",
Filters: squirrel.Eq{"album_id": albumID, "disc_number": discNumber},
@ -88,7 +88,7 @@ func newDiscArtworkReader(ctx context.Context, ds model.DataStore, artID model.A
// Resolve folder IDs to library-relative paths
discFoldersRel := make(map[string]bool)
if len(folderIDs) > 0 {
folders, err := ds.Folder().GetAll(ctx, model.QueryOptions{
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"folder.id": folderIDs},
})
if err != nil {

View file

@ -44,20 +44,20 @@ var _ = Describe("Acquisition → serve loop", func() {
itemFound := func(kind model.Kind, id string) func() bool {
return func() bool {
ia, err := artRepo.GetItemArtwork(ctx, kind, id, model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(kind, id, model.ImageTypePrimary)
return err == nil && ia.Hash != ""
}
}
itemAbsent := func(kind model.Kind, id string) func() bool {
return func() bool {
ia, err := artRepo.GetItemArtwork(ctx, kind, id, model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(kind, id, model.ImageTypePrimary)
return err == nil && ia.Hash == ""
}
}
// Enqueues the way the serving paths do, so the drain is driven by a plain queue row.
bump := func(kind, id string) {
GinkgoHelper()
Expect(ds.ArtworkQueue().EnqueuePreservingBackoff(ctx, model.ArtworkQueueItem{
Expect(ds.ArtworkQueue(ctx).EnqueuePreservingBackoff(model.ArtworkQueueItem{
ItemKind: kind, ItemID: id, ImageType: model.ImageTypePrimary,
Priority: model.ArtworkPriorityBump,
})).To(Succeed())
@ -108,7 +108,7 @@ var _ = Describe("Acquisition → serve loop", func() {
})
Eventually(func() bool { return imgCache.Available(ctx) }, 10*time.Second).Should(BeTrue())
svc = artwork.NewArtwork(ds, imgCache, store, ffm)
svc = artwork.NewArtwork(ds, imgCache, store, ffm, events.NoopBroker())
worker = artwork.NewWorker(ds, store, agents.GetAgents(ds, nil), ffm, events.NoopBroker(), imgCache)
})
@ -141,7 +141,7 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("al", "al1")
runWorkerUntil(ctx, worker, itemFound(model.KindAlbumArtwork, "al1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"))
@ -158,7 +158,7 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("ar", "ar1")
runWorkerUntil(ctx, worker, itemFound(model.KindArtistArtwork, "ar1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindArtistArtwork, "ar1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindArtistArtwork, "ar1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("upload"))
@ -175,14 +175,14 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("pl", "pl1")
runWorkerUntil(ctx, worker, itemFound(model.KindPlaylistArtwork, "pl1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindPlaylistArtwork, "pl1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindPlaylistArtwork, "pl1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("generated"))
img, err := svc.Get(ctx, model.MustParseArtworkID("pl-pl1"), 0, false)
Expect(err).ToNot(HaveOccurred())
Expect(img.Hash).To(Equal(ia.Hash))
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Mime).To(Equal("image/png"))
Expect(len(readAll(img))).To(BeNumerically(">", 0))
@ -194,7 +194,7 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound(model.KindRadioArtwork, "ra1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("upload"))
@ -216,12 +216,12 @@ var _ = Describe("Acquisition → serve loop", func() {
provisionalBytes := readAll(provisional)
Expect(len(provisionalBytes)).To(BeNumerically(">", 0))
_, err = artRepo.GetItemArtwork(ctx, model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "provisional serving must not write a state row")
// The provisional read enqueued a Bump; drain it.
runWorkerUntil(ctx, worker, itemFound(model.KindMediaFileArtwork, "mf1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("embedded"))
Expect(ia.Hash).To(Equal(provisional.Hash))
@ -237,9 +237,9 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("al", "al1")
runWorkerUntil(ctx, worker, itemFound(model.KindAlbumArtwork, "al1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Mime).To(Equal("image/jpeg"))
Expect(art.Width).To(BeNumerically(">", 0))
@ -259,9 +259,9 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound(model.KindRadioArtwork, "ra1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Mime).To(Equal("image/gif"))
Expect(art.Width).To(BeNumerically("==", 4))
@ -279,9 +279,9 @@ var _ = Describe("Acquisition → serve loop", func() {
return itemFound(model.KindAlbumArtwork, "al1")() && itemFound(model.KindAlbumArtwork, "al2")()
})
ia1, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
ia1, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
ia2, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al2", model.ImageTypePrimary)
ia2, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al2", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia1.Hash).To(Equal(ia2.Hash), "identical bytes must share one content hash")
Expect(readAll(mustGet(svc.Get(ctx, model.MustParseArtworkID("al-al2"), 0, false)))).To(Equal(coverBytes))
@ -293,7 +293,7 @@ var _ = Describe("Acquisition → serve loop", func() {
bump("ra", "ra1")
runWorkerUntil(ctx, worker, itemFound(model.KindRadioArtwork, "ra1"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
staleHash := ia.Hash
@ -308,7 +308,7 @@ var _ = Describe("Acquisition → serve loop", func() {
// That failed read enqueued a re-resolution.
runWorkerUntil(ctx, worker, func() bool {
cur, gerr := artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
cur, gerr := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
return gerr == nil && cur.Hash != "" && cur.Hash != staleHash
})
img, err := svc.Get(ctx, model.MustParseArtworkID("ra-ra1"), 0, false)

View file

@ -90,28 +90,6 @@ var _ = Describe("Artist artwork resolution", func() {
})
})
When("the artist's only album folder has no images of its own", func() {
// Artist/
// ├── backdrop1.jpg
// ├── folder.jpg ← matched by folder.*
// ├── logo.png
// └── Album/
// ├── 01 - Track.mp3
// └── 02 - Track.mp3
It("resolves the artist folder, not the library root", func() {
conf.Server.ArtistArtPriority = "folder.*, artist.*, album/artist.*"
setLayout(fstest.MapFS{
"Artist/Album/01 - Track.mp3": trackFile(1, "Track 1", map[string]any{"albumartist": "Artist", "album": "Album"}),
"Artist/Album/02 - Track.mp3": trackFile(2, "Track 2", map[string]any{"albumartist": "Artist", "album": "Album"}),
"Artist/backdrop1.jpg": smallPNG("backdrop"),
"Artist/folder.jpg": smallPNG("artist-folder"),
"Artist/logo.png": smallPNG("logo"),
})
scan()
expectArtistFolder(soleArtist(), "Artist/folder.jpg")
})
})
When("the artist's only album has its tracks in disc subfolders", func() {
// Artist/
// ├── artist.jpg ← wins (artist.* before album/artist.*)
@ -201,7 +179,7 @@ var _ = Describe("Artist artwork resolution", func() {
uploaded := ar.ID + "_upload.jpg"
writeUploadedImage(consts.EntityArtist, uploaded, pngBytes("artist-uploaded"))
ar.UploadedImage = uploaded
Expect(rds.Artist().Put(rctx, &ar)).To(Succeed())
Expect(rds.Artist(rctx).Put(&ar)).To(Succeed())
ia := acquire(model.KindArtistArtwork, ar.ID)
Expect(ia.Source).To(Equal("upload"))
@ -279,7 +257,7 @@ var _ = Describe("Artist artwork resolution", func() {
func soleArtist() model.Artist {
GinkgoHelper()
artists, err := rds.Artist().GetAll(rctx, model.QueryOptions{
artists, err := rds.Artist(rctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"artist.name": "Artist"},
})
Expect(err).ToNot(HaveOccurred())

View file

@ -56,8 +56,7 @@ func runWorkerUntil(ctx context.Context, worker *artwork.Worker, until func() bo
runCtx, cancel := context.WithCancel(ctx)
done := make(chan error, 1)
go func() { done <- worker.Run(runCtx) }()
// Long enough for one retry (3-7s backoff, 5s poll tick).
Eventually(until, 15*time.Second, 10*time.Millisecond).Should(BeTrue())
Eventually(until, 5*time.Second, 10*time.Millisecond).Should(BeTrue())
cancel()
Eventually(done, 2*time.Second).Should(Receive(BeNil()))
}
@ -67,17 +66,13 @@ type fakeFolderRepo struct {
result []model.Folder
}
func (f *fakeFolderRepo) GetAll(context.Context, ...model.QueryOptions) ([]model.Folder, error) {
return f.result, nil
}
func (f *fakeFolderRepo) GetAll(...model.QueryOptions) ([]model.Folder, error) { return f.result, nil }
func (f *fakeFolderRepo) HasAudioOutsideFolders(context.Context, model.Folder, []string) (bool, error) {
func (f *fakeFolderRepo) HasAudioOutsideFolders(model.Folder, []string) (bool, error) {
return false, nil
}
func (f *fakeFolderRepo) Get(context.Context, string) (*model.Folder, error) {
return nil, model.ErrNotFound
}
func (f *fakeFolderRepo) Get(string) (*model.Folder, error) { return nil, model.ErrNotFound }
func writeUpload(entityType, name, srcFixture string) string {
GinkgoHelper()

View file

@ -137,7 +137,7 @@ var _ = Describe("MediaFile artwork resolution", func() {
func mediafileOn(relPath string) model.MediaFile {
GinkgoHelper()
mfs, err := rds.MediaFile().GetAll(rctx, model.QueryOptions{
mfs, err := rds.MediaFile(rctx).GetAll(model.QueryOptions{
Filters: squirrel.Like{"media_file.path": relPath},
})
Expect(err).ToNot(HaveOccurred())

View file

@ -142,13 +142,13 @@ var _ = Describe("Playlist artwork resolution", func() {
})
scan()
mfs, err := rds.MediaFile().GetAll(rctx, model.QueryOptions{})
mfs, err := rds.MediaFile(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
Expect(mfs).To(HaveLen(2))
pl := model.Playlist{ID: "pl-7", Name: "Mix", OwnerID: "admin-1"}
pl.AddMediaFilesByID([]string{mfs[0].ID, mfs[1].ID})
Expect(rds.Playlist().Put(rctx, &pl)).To(Succeed())
Expect(rds.Playlist(rctx).Put(&pl)).To(Succeed())
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Source).To(Equal("generated"))
@ -180,14 +180,14 @@ var _ = Describe("Playlist artwork resolution", func() {
setLayout(layout)
scan()
mfs, err := rds.MediaFile().GetAll(rctx, model.QueryOptions{})
mfs, err := rds.MediaFile(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
Expect(mfs).To(HaveLen(4))
ids := slice.Map(mfs, func(mf model.MediaFile) string { return mf.ID })
pl := model.Playlist{ID: "pl-8", Name: "Four", OwnerID: "admin-1"}
pl.AddMediaFilesByID(ids)
Expect(rds.Playlist().Put(rctx, &pl)).To(Succeed())
Expect(rds.Playlist(rctx).Put(&pl)).To(Succeed())
ia := acquire(model.KindPlaylistArtwork, pl.ID)
Expect(ia.Source).To(Equal("generated"))
@ -208,6 +208,6 @@ func putPlaylist(pl model.Playlist) model.Playlist {
if pl.OwnerID == "" {
pl.OwnerID = "admin-1"
}
Expect(rds.Playlist().Put(rctx, &pl)).To(Succeed())
Expect(rds.Playlist(rctx).Put(&pl)).To(Succeed())
return pl
}

View file

@ -23,7 +23,7 @@ var _ = Describe("Radio artwork resolution", func() {
It("returns the uploaded image bytes", func() {
writeUploadedImage(consts.EntityRadio, "rd-1_logo.jpg", pngBytes("radio-logo"))
rd := model.Radio{ID: "rd-1", Name: "Test Radio", StreamUrl: "https://example.com/stream", UploadedImage: "rd-1_logo.jpg"}
Expect(rds.Radio().Put(rctx, &rd)).To(Succeed())
Expect(rds.Radio(rctx).Put(&rd)).To(Succeed())
ia := acquire(model.KindRadioArtwork, rd.ID)
Expect(ia.Source).To(Equal("upload"))
@ -35,7 +35,7 @@ var _ = Describe("Radio artwork resolution", func() {
// (no files on disk — the resolver has no sources to fall back to)
It("settles absent", func() {
rd := model.Radio{ID: "rd-2", Name: "Bare Radio", StreamUrl: "https://example.com/stream"}
Expect(rds.Radio().Put(rctx, &rd)).To(Succeed())
Expect(rds.Radio(rctx).Put(&rd)).To(Succeed())
ia := acquire(model.KindRadioArtwork, rd.ID)
Expect(ia.Hash).To(BeEmpty())

View file

@ -99,11 +99,11 @@ func setupResolutionHarness() {
rds = &tests.MockDataStore{RealDS: persistence.New(db.Db())}
adminUser := model.User{ID: "admin-1", UserName: "admin", Name: "Admin", IsAdmin: true, NewPassword: "password"}
Expect(rds.User().Put(rctx, &adminUser)).To(Succeed())
Expect(rds.User(rctx).Put(&adminUser)).To(Succeed())
lib := model.Library{ID: 1, Name: "Music", Path: fakeLibPath}
Expect(rds.Library().Put(rctx, &lib)).To(Succeed())
Expect(rds.User().SetUserLibraries(rctx, adminUser.ID, []int{lib.ID})).To(Succeed())
Expect(rds.Library(rctx).Put(&lib)).To(Succeed())
Expect(rds.User(rctx).SetUserLibraries(adminUser.ID, []int{lib.ID})).To(Succeed())
loadEmbeddedFixture()
@ -119,7 +119,7 @@ func setupResolutionHarness() {
})
Eventually(func() bool { return imgCache.Available(rctx) }, 10*time.Second).Should(BeTrue())
rsvc = artwork.NewArtwork(rds, imgCache, rstore, ffm)
rsvc = artwork.NewArtwork(rds, imgCache, rstore, ffm, events.NoopBroker())
rworker = artwork.NewWorker(rds, rstore, agents.GetAgents(rds, nil), ffm, events.NoopBroker(), imgCache)
}
@ -140,13 +140,13 @@ func scan() {
func acquire(kind model.Kind, id string) model.ItemArtwork {
GinkgoHelper()
// Enqueues the way the serving paths do, so the drain is driven by a plain queue row.
Expect(rds.ArtworkQueue().EnqueuePreservingBackoff(rctx, model.ArtworkQueueItem{
Expect(rds.ArtworkQueue(rctx).EnqueuePreservingBackoff(model.ArtworkQueueItem{
ItemKind: kind.Prefix(), ItemID: id, ImageType: model.ImageTypePrimary,
Priority: model.ArtworkPriorityBump,
})).To(Succeed())
var ia *model.ItemArtwork
runResolutionWorkerUntil(func() bool {
got, err := rds.Artwork().GetItemArtwork(rctx, kind, id, model.ImageTypePrimary)
got, err := rds.Artwork(rctx).GetItemArtwork(kind, id, model.ImageTypePrimary)
if err != nil {
return false
}
@ -211,7 +211,7 @@ func expectAlbumFolderCover(al model.Album, suffix string) {
// A drain settles every ready item, so byte-level folder assertions must precede any acquire.
func requireNoStateRow(kind model.Kind, id string) {
GinkgoHelper()
_, err := rds.Artwork().GetItemArtwork(rctx, kind, id, model.ImageTypePrimary)
_, err := rds.Artwork(rctx).GetItemArtwork(kind, id, model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound),
"assert %s %q before acquiring any other entity in this spec", kind, id)
}
@ -266,7 +266,7 @@ func gridQuadrants(data []byte) [4]color.RGBA {
// Store-backed sources only (embedded/generated); file-backed ones assert on ia.SourcePath.
func storedBytes(ia model.ItemArtwork) []byte {
GinkgoHelper()
art, err := rds.Artwork().GetImage(rctx, ia.Hash)
art, err := rds.Artwork(rctx).GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
r, err := rstore.Open(ia.Hash, art.Mime)
Expect(err).ToNot(HaveOccurred())
@ -345,7 +345,7 @@ func replaceWithRealMP3(relPath string) {
func firstAlbum() model.Album {
GinkgoHelper()
albums, err := rds.Album().GetAll(rctx, model.QueryOptions{})
albums, err := rds.Album(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
Expect(albums).To(HaveLen(1), "expected exactly one album, got %d", len(albums))
return albums[0]
@ -353,7 +353,7 @@ func firstAlbum() model.Album {
func albumByName(name string) model.Album {
GinkgoHelper()
albums, err := rds.Album().GetAll(rctx, model.QueryOptions{})
albums, err := rds.Album(rctx).GetAll(model.QueryOptions{})
Expect(err).ToNot(HaveOccurred())
for _, al := range albums {
if al.Name == name {

View file

@ -36,7 +36,7 @@ func loadAlbumFoldersPaths(ctx context.Context, ds model.DataStore, album model.
}
func loadFolders(ctx context.Context, ds model.DataStore, folderIDs []string) ([]model.Folder, error) {
return ds.Folder().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderIDs, "missing": false}})
return ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderIDs, "missing": false}})
}
// folderImages collects the folders' image files, sorted so files without
@ -79,7 +79,7 @@ func albumRootParent(ctx context.Context, ds model.DataStore, folders []model.Fo
if len(folders) < 2 && anyFolderHasImages(folders) {
return nil, nil
}
parent, err := ds.Folder().Get(ctx, commonParentID)
parent, err := ds.Folder(ctx).Get(commonParentID)
if errors.Is(err, model.ErrNotFound) {
log.Warn(ctx, "Artwork: Parent folder not found for album cover art lookup", "parentID", commonParentID)
return nil, nil
@ -91,7 +91,7 @@ func albumRootParent(ctx context.Context, ds model.DataStore, folders []model.Fo
// The library root can never be an album root
return nil, nil
}
hasOtherAudio, err := ds.Folder().HasAudioOutsideFolders(ctx, *parent, folderIDs)
hasOtherAudio, err := ds.Folder(ctx).HasAudioOutsideFolders(*parent, folderIDs)
if err != nil {
return nil, err
}

View file

@ -14,6 +14,7 @@ import (
"time"
"github.com/Masterminds/squirrel"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils"
@ -168,15 +169,14 @@ func loadArtistFolder(ctx context.Context, ds model.DataStore, albums model.Albu
folderPath = filepath.Dir(folderPath)
}
// Cleaned like the album paths; Join keeps an empty path empty, Clean would return ".".
libPath, _ := ds.Library().GetPath(ctx, libID)
libPath = filepath.Join(libPath)
// TODO: Hacky, but the easiest way to get the folder ID ATM
libPath := core.AbsolutePath(ctx, ds, libID, "")
folderID := model.FolderID(model.Library{ID: libID, Path: libPath}, folderPath)
log.Trace(ctx, "Artwork: Calculating artist folder details", "folderPath", folderPath, "folderID", folderID,
"libPath", libPath, "libID", libID, "albumPaths", paths)
folders, err := ds.Folder().GetAll(ctx, model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderID, "missing": false}})
folders, err := ds.Folder(ctx).GetAll(model.QueryOptions{Filters: squirrel.Eq{"folder.id": folderID, "missing": false}})
if err != nil || len(folders) == 0 {
log.Warn(ctx, "Artwork: Could not find folder for artist", "folderPath", folderPath, "id", folderID,
"libPath", libPath, "libID", libID, err)

View file

@ -6,6 +6,7 @@ import (
"path/filepath"
"time"
"github.com/navidrome/navidrome/core"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
@ -46,11 +47,11 @@ var _ = Describe("loadArtistFolder", func() {
BeforeEach(func() {
ctx = context.Background()
DeferCleanup(stubCoreAbsolutePath())
updatedAt = time.Now().Truncate(time.Second).Add(5 * time.Minute)
repo = &fakeFolderRepo{result: []model.Folder{{ImagesUpdatedAt: updatedAt}}}
libRepo := &tests.MockLibraryRepo{}
libRepo.SetData(model.Libraries{{ID: 1, Path: filepath.FromSlash("/music")}})
ds = &tests.MockDataStore{MockedFolder: repo, MockedLibrary: libRepo}
ds = &tests.MockDataStore{MockedFolder: repo}
albums = model.Albums{{LibraryID: 1, ID: "album1", Name: "Album 1"}}
})
@ -106,3 +107,11 @@ var _ = Describe("loadArtistFolder", func() {
Expect(upd).To(BeZero())
})
})
func stubCoreAbsolutePath() func() {
original := core.AbsolutePath
core.AbsolutePath = func(context.Context, model.DataStore, int, string) string {
return filepath.FromSlash("/music")
}
return func() { core.AbsolutePath = original }
}

View file

@ -70,7 +70,7 @@ func ConfigFingerprint() string {
// resolved under it. Nothing re-resolves on its own; applying a change is an explicit reprocess.
func ReconcileConfigFingerprint(ctx context.Context, ds model.DataStore) error {
current := ConfigFingerprint()
stored, err := ds.Property().DefaultGet(ctx, consts.ArtConfFingerprintPropertyKey, "")
stored, err := ds.Property(ctx).DefaultGet(consts.ArtConfFingerprintPropertyKey, "")
if err != nil {
return err
}
@ -89,14 +89,14 @@ func ReconcileConfigFingerprint(ctx context.Context, ds model.DataStore) error {
// MarkConfigApplied records the current fingerprint as the one the library is resolved under.
func MarkConfigApplied(ctx context.Context, ds model.DataStore) error {
return ds.Property().Put(ctx, consts.ArtConfFingerprintPropertyKey, ConfigFingerprint())
return ds.Property(ctx).Put(consts.ArtConfFingerprintPropertyKey, ConfigFingerprint())
}
// enqueueMissingAll is the safety net for entities a scan never enqueued (added between scans, or scanner off).
func enqueueMissingAll(ctx context.Context, ds model.DataStore) error {
queue := ds.ArtworkQueue()
queue := ds.ArtworkQueue(ctx)
for _, kind := range ReprocessKinds {
if _, err := queue.EnqueueAllMissing(ctx, kind, model.ArtworkPriorityRecheck); err != nil {
if _, err := queue.EnqueueAllMissing(kind, model.ArtworkPriorityRecheck); err != nil {
return err
}
}
@ -108,31 +108,31 @@ func enqueueMissingAll(ctx context.Context, ds model.DataStore) error {
func ItemName(ctx context.Context, ds model.DataStore, kind model.Kind, id string) (string, error) {
switch kind {
case model.KindArtistArtwork:
ar, err := ds.Artist().Get(ctx, id)
ar, err := ds.Artist(ctx).Get(id)
if err != nil {
return "", err
}
return ar.Name, nil
case model.KindAlbumArtwork:
al, err := ds.Album().Get(ctx, id)
al, err := ds.Album(ctx).Get(id)
if err != nil {
return "", err
}
return al.Name, nil
case model.KindPlaylistArtwork:
pls, err := ds.Playlist().Get(ctx, id)
pls, err := ds.Playlist(ctx).Get(id)
if err != nil {
return "", err
}
return pls.Name, nil
case model.KindRadioArtwork:
rd, err := ds.Radio().Get(ctx, id)
rd, err := ds.Radio(ctx).Get(id)
if err != nil {
return "", err
}
return rd.Name, nil
case model.KindMediaFileArtwork:
mf, err := ds.MediaFile().Get(ctx, id)
mf, err := ds.MediaFile(ctx).Get(id)
if err != nil {
return "", err
}
@ -148,7 +148,7 @@ func discArtworkName(ctx context.Context, ds model.DataStore, id string) (string
if err != nil {
return "", err
}
al, err := ds.Album().Get(ctx, albumID)
al, err := ds.Album(ctx).Get(albumID)
if err != nil {
return "", err
}
@ -162,11 +162,11 @@ func discArtworkName(ctx context.Context, ds model.DataStore, id string) (string
// Refresh drops an item's resolved artwork state and re-queues it at Bump priority.
func Refresh(ctx context.Context, ds model.DataStore, kind model.Kind, id string) error {
if err := ds.Artwork().DeleteForItems(ctx, kind, []string{id}); err != nil {
if err := ds.Artwork(ctx).DeleteForItems(kind, []string{id}); err != nil {
return fmt.Errorf("clearing artwork state: %w", err)
}
item := model.ArtworkQueueItem{ItemKind: kind.Prefix(), ItemID: id, ImageType: model.ImageTypePrimary, Priority: model.ArtworkPriorityBump}
if err := ds.ArtworkQueue().Enqueue(ctx, item); err != nil {
if err := ds.ArtworkQueue(ctx).Enqueue(item); err != nil {
return fmt.Errorf("enqueuing artwork refresh: %w", err)
}
return nil

View file

@ -95,15 +95,15 @@ var _ = Describe("Housekeeping", func() {
It("records the current fingerprint when none was ever stored", func() {
Expect(ReconcileConfigFingerprint(ctx, ds)).To(Succeed())
Expect(propRepo.Get(ctx, consts.ArtConfFingerprintPropertyKey)).To(Equal(ConfigFingerprint()))
Expect(propRepo.Get(consts.ArtConfFingerprintPropertyKey)).To(Equal(ConfigFingerprint()))
})
It("leaves a stale fingerprint stored, so the warning survives a restart", func() {
Expect(propRepo.Put(ctx, consts.ArtConfFingerprintPropertyKey, "stale-fingerprint")).To(Succeed())
Expect(propRepo.Put(consts.ArtConfFingerprintPropertyKey, "stale-fingerprint")).To(Succeed())
Expect(ReconcileConfigFingerprint(ctx, ds)).To(Succeed())
Expect(propRepo.Get(ctx, consts.ArtConfFingerprintPropertyKey)).To(Equal("stale-fingerprint"))
Expect(propRepo.Get(consts.ArtConfFingerprintPropertyKey)).To(Equal("stale-fingerprint"))
})
})
@ -153,7 +153,7 @@ var _ = Describe("ItemName", func() {
{ID: "al-2", Name: "Sandinista!", Discs: model.Discs{2: "Side Three"}},
})
ds = &tests.MockDataStore{MockedAlbum: albumRepo}
Expect(ds.Artist().(*tests.MockArtistRepo).Put(ctx, &model.Artist{ID: "ar-1", Name: "Radiohead"})).To(Succeed())
Expect(ds.Artist(ctx).(*tests.MockArtistRepo).Put(&model.Artist{ID: "ar-1", Name: "Radiohead"})).To(Succeed())
})
It("returns the album name", func() {

View file

@ -42,6 +42,8 @@ type resizedItem struct {
square bool
ffmpeg ffmpeg.FFmpeg
open func() (io.ReadCloser, error)
// deferAnimated makes Reader return a *deferredAnimation for animated GIFs instead of converting inline.
deferAnimated bool
}
// Key is the ETag namespaced for the cache, so the validator a client holds and the entry it
@ -64,13 +66,50 @@ func (r *resizedItem) Reader(ctx context.Context) (io.ReadCloser, error) {
if err != nil {
return nil, err
}
if r.deferAnimated && isAnimatedGIF(data) && r.ffmpeg.IsAvailable() {
static, _, err := resizeStaticImage(data, r.size, r.square)
return nil, &deferredAnimation{data: data, standIn: orOriginal(data, static, err)}
}
resized, _, err := resizeImageData(ctx, r.ffmpeg, data, r.size, r.square)
if err != nil || resized == nil {
// Resize failed or image already within bounds: serve the original bytes.
return io.NopCloser(bytes.NewReader(data)), nil
}
if rc, ok := resized.(io.ReadCloser); ok {
return rc, nil
}
return io.NopCloser(resized), nil
return io.NopCloser(orOriginal(data, resized, err)), nil
}
// orOriginal serves the original bytes when the resize failed or the image was already within bounds.
func orOriginal(data []byte, resized io.Reader, err error) io.Reader {
if err != nil || resized == nil {
return bytes.NewReader(data)
}
return resized
}
// deferredAnimation is returned as an error so the cache stores nothing: it carries the GIF bytes
// to convert in the background and a static stand-in to serve meanwhile.
type deferredAnimation struct {
data []byte
standIn io.Reader
}
func (*deferredAnimation) Error() string { return "animated image conversion deferred" }
// warmCache stores item in c, returning once the entry is written.
func warmCache(ctx context.Context, c cache.FileCache, item cache.Item) error {
stream, err := c.Get(ctx, item)
if err != nil {
return err
}
defer stream.Close()
_, err = io.Copy(io.Discard, stream)
return err
}
// storedItem caches bytes produced outside the cache.
type storedItem struct {
key string
data []byte
}
func (i *storedItem) Key() string { return i.key }
func (i *storedItem) Reader(context.Context) (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(i.data)), nil
}

View file

@ -98,9 +98,6 @@ func (s *ImageStore) Write(hash, mimeType string, r io.Reader) error {
if err := tmp.Close(); err != nil {
return err
}
if err := os.Chmod(tmp.Name(), 0640); err != nil {
return err
}
return os.Rename(tmp.Name(), dst)
}

View file

@ -48,17 +48,6 @@ var _ = Describe("ImageStore", func() {
Expect(got).To(Equal(data))
})
It("writes group-readable image files", func() {
tests.SkipOnWindows("uses Unix file permission bits")
data := []byte("jpeg-bytes")
h, _ := hashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
info, err := os.Stat(store.path(h, "image/jpeg"))
Expect(err).ToNot(HaveOccurred())
Expect(info.Mode().Perm()).To(Equal(os.FileMode(0640)))
})
It("is idempotent on duplicate writes and preserves the original content", func() {
data := []byte("dup")
h, _ := hashImage(bytes.NewReader(data))

View file

@ -30,7 +30,7 @@ func (v libraryView) Abs(rel string) string {
// loadLibraryView resolves the MusicFS and absolute root path in a single
// library lookup.
func loadLibraryView(ctx context.Context, ds model.DataStore, libID int) (libraryView, error) {
lib, err := ds.Library().Get(ctx, libID)
lib, err := ds.Library(ctx).Get(libID)
if err != nil {
return libraryView{}, err
}

View file

@ -24,7 +24,7 @@ var _ = Describe("loadLibraryView", Ordered, func() {
})
It("returns a view for a library backed by registered storage", func() {
Expect(ds.Library().Put(ctx, &model.Library{ID: 1, Path: "fake:///music"})).To(Succeed())
Expect(ds.Library(ctx).Put(&model.Library{ID: 1, Path: "fake:///music"})).To(Succeed())
lib, err := loadLibraryView(ctx, ds, 1)
Expect(err).ToNot(HaveOccurred())
@ -45,7 +45,7 @@ var _ = Describe("loadLibraryView", Ordered, func() {
})
It("returns an error when the library path uses an unregistered scheme", func() {
Expect(ds.Library().Put(ctx, &model.Library{ID: 2, Path: "unsupported:///music"})).To(Succeed())
Expect(ds.Library(ctx).Put(&model.Library{ID: 2, Path: "unsupported:///music"})).To(Succeed())
_, err := loadLibraryView(ctx, ds, 2)
Expect(err).To(HaveOccurred())
})

View file

@ -82,7 +82,7 @@ type processor struct {
// acquire resolves one queue item end to end: find an image, hash/decode/
// blurhash it, place its bytes, and persist the resulting state.
func (p *processor) acquire(ctx context.Context, item model.ArtworkQueueItem) (out outcome, got *acquired, retryIn time.Duration) {
repo := p.ds.Artwork()
repo := p.ds.Artwork(ctx)
start := time.Now()
defer func() {
log.Debug(ctx, "Artwork: Acquisition finished", "kind", item.ItemKind, "id", item.ItemID,
@ -137,7 +137,7 @@ func (p *processor) acquire(ctx context.Context, item model.ArtworkQueueItem) (o
log.Trace(ctx, "Artwork: Hashed image", "kind", item.ItemKind, "id", item.ItemID,
"hash", hash, "bytes", len(data), "elapsed", time.Since(hashStart))
art, err := repo.GetImage(ctx, hash)
art, err := repo.GetImage(hash)
switch {
case err == nil && art.Width > 0:
log.Debug(ctx, "Artwork: Reusing a known image, skipping decode", "kind", item.ItemKind,
@ -195,7 +195,7 @@ func (p *processor) persist(ctx context.Context, repo model.ArtworkRepository, i
if err != nil {
return nil, fmt.Errorf("writing image store: %w", err)
}
if err := repo.PutImage(ctx, art); err != nil {
if err := repo.PutImage(art); err != nil {
return nil, fmt.Errorf("persisting artwork image: %w", err)
}
ia := &model.ItemArtwork{
@ -210,7 +210,7 @@ func (p *processor) persist(ctx context.Context, repo model.ArtworkRepository, i
Trace: traceFrom(ctx).encode(sourcePath),
}
// PutItemArtwork stamps UpdatedAt on ia, so the returned struct matches the persisted row.
if err := repo.PutItemArtwork(ctx, ia); err != nil {
if err := repo.PutItemArtwork(ia); err != nil {
return nil, fmt.Errorf("persisting item artwork state: %w", err)
}
return ia, nil
@ -218,7 +218,7 @@ func (p *processor) persist(ctx context.Context, repo model.ArtworkRepository, i
// writeAbsent records a known-absent state: every source answered definitively "no".
func writeAbsent(ctx context.Context, repo model.ArtworkRepository, item model.ArtworkQueueItem) outcome {
err := repo.PutItemArtwork(ctx, &model.ItemArtwork{
err := repo.PutItemArtwork(&model.ItemArtwork{
ItemKind: item.ItemKind,
ItemID: item.ItemID,
ImageType: item.ImageType,

View file

@ -93,14 +93,14 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al1"})
Expect(out).To(Equal(outcomeFound))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).ToNot(BeEmpty())
Expect(ia.Source).To(Equal("folder"))
Expect(filepath.ToSlash(ia.SourcePath)).To(HaveSuffix("tests/fixtures/artist/an-album/cover.jpg"))
Expect(ia.RefMtime).To(BeNumerically(">", 0))
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
// Every placeholder is derived from the one shared thumbnail, so all three land together.
Expect(art.BlurHash).ToNot(BeEmpty())
@ -156,12 +156,12 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al2"})
Expect(out).To(Equal(outcomeFound))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al2", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al2", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("embedded"))
Expect(filepath.ToSlash(ia.SourcePath)).To(HaveSuffix("tests/fixtures/artist/an-album/test.mp3"))
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.BlurHash).ToNot(BeEmpty())
@ -179,7 +179,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al3"})
Expect(out).To(Equal(outcomeAbsent))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al3", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al3", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).To(BeEmpty())
Expect(ia.Source).To(BeEmpty())
@ -200,7 +200,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al-io"})
Expect(out).To(Equal(outcomeFailed))
_, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al-io", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al-io", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "an I/O fault must not be recorded as absent")
})
@ -225,7 +225,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "ra", ItemID: "ra-io"})
Expect(out).To(Equal(outcomeFailed))
_, err := artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "ra-io", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra-io", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "an unreadable upload must not be recorded as absent")
})
@ -269,7 +269,7 @@ var _ = Describe("processor.acquire", func() {
Expect(out).To(Equal(outcomeFailed))
Expect(retryIn).To(BeZero(), "a plain failure asks for no particular delay")
_, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al4", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al4", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -299,7 +299,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alstale"})
Expect(out).To(Equal(outcomeFoundStale))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alstale", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alstale", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).ToNot(BeEmpty())
Expect(ia.Source).To(Equal("folder"))
@ -316,10 +316,10 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alU"})
Expect(out).To(Equal(outcomeFound))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alU", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alU", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"))
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(art.Width).To(BeZero())
Expect(art.BlurHash).To(BeEmpty())
@ -336,7 +336,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alE"})
Expect(out).To(Equal(outcomeFailed))
_, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alE", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alE", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -354,7 +354,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alX"})
Expect(out).To(Equal(outcomeFailed))
_, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alX", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alX", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -373,12 +373,12 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alext"})
Expect(out).To(Equal(outcomeFound))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alext", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alext", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("external:deezerFake"))
Expect(ia.Hash).ToNot(BeEmpty())
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
rc, err := store.Open(ia.Hash, art.Mime)
Expect(err).ToNot(HaveOccurred())
@ -397,7 +397,7 @@ var _ = Describe("processor.acquire", func() {
out1, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al5"})
Expect(out1).To(Equal(outcomeFound))
ia1, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al5", model.ImageTypePrimary)
ia1, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al5", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
// A re-decode instead of a hash dedup would overwrite this sentinel.
@ -407,11 +407,11 @@ var _ = Describe("processor.acquire", func() {
out2, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al6"})
Expect(out2).To(Equal(outcomeFound))
ia2, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al6", model.ImageTypePrimary)
ia2, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al6", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia2.Hash).To(Equal(ia1.Hash))
reused, err := artRepo.GetImage(ctx, ia1.Hash)
reused, err := artRepo.GetImage(ia1.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(reused.BlurHash).To(Equal("SENTINEL"))
})
@ -437,7 +437,7 @@ var _ = Describe("processor.acquire", func() {
folderRepo.result = []model.Folder{{Path: "album-a", ImageFiles: []string{"cover.jpg"}}}
outN, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alA"})
Expect(outN).To(Equal(outcomeFound))
iaA, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alA", model.ImageTypePrimary)
iaA, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alA", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(iaA.Source).To(Equal("folder"))
Expect(filepath.ToSlash(iaA.SourcePath)).To(HaveSuffix("album-a/cover.jpg"))
@ -451,19 +451,19 @@ var _ = Describe("processor.acquire", func() {
folderRepo.result = []model.Folder{{Path: "album-b", ImageFiles: []string{"cover.jpg"}}}
outN, _, _ = proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alB"})
Expect(outN).To(Equal(outcomeFound))
iaB, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alB", model.ImageTypePrimary)
iaB, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alB", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(iaB.Hash).To(Equal(iaA.Hash))
Expect(filepath.ToSlash(iaB.SourcePath)).To(HaveSuffix("album-b/cover.jpg"))
Expect(iaB.RefMtime).To(Equal(time.Unix(2000, 0).UnixNano()))
iaAafter, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alA", model.ImageTypePrimary)
iaAafter, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alA", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(filepath.ToSlash(iaAafter.SourcePath)).To(HaveSuffix("album-a/cover.jpg"))
Expect(iaAafter.RefMtime).To(Equal(time.Unix(1000, 0).UnixNano()))
Expect(artRepo.Data).To(HaveLen(1))
reused, err := artRepo.GetImage(ctx, iaA.Hash)
reused, err := artRepo.GetImage(iaA.Hash)
Expect(err).ToNot(HaveOccurred())
Expect(reused.BlurHash).To(Equal("SENTINEL"))
})
@ -483,7 +483,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "ra", ItemID: "ra1"})
Expect(out).To(Equal(outcomeFailed))
_, err := artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindRadioArtwork, "ra1", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -504,7 +504,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "ra", ItemID: "big"})
Expect(out).To(Equal(outcomeFailed))
_, err = artRepo.GetItemArtwork(ctx, model.KindRadioArtwork, "big", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindRadioArtwork, "big", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
@ -565,12 +565,12 @@ var _ = Describe("processor.acquire", func() {
hash, err := hashImage(bytes.NewReader(imgBytes))
Expect(err).ToNot(HaveOccurred())
Expect(artRepo.PutImage(ctx, &model.Artwork{Hash: hash, Mime: "application/octet-stream"})).To(Succeed())
Expect(artRepo.PutImage(&model.Artwork{Hash: hash, Mime: "application/octet-stream"})).To(Succeed())
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alM"})
Expect(out).To(Equal(outcomeFound))
upgraded, err := artRepo.GetImage(ctx, hash)
upgraded, err := artRepo.GetImage(hash)
Expect(err).ToNot(HaveOccurred())
Expect(upgraded.Width).To(BeNumerically(">", 0))
Expect(upgraded.BlurHash).ToNot(BeEmpty())
@ -590,7 +590,7 @@ var _ = Describe("processor.acquire", func() {
out, _, _ := proc.acquire(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al7"})
Expect(out).To(Equal(outcomeFailed))
_, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al7", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al7", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
})
})

View file

@ -14,9 +14,9 @@ const pruneMinAge = time.Hour
func prune(ctx context.Context, ds model.DataStore, store *ImageStore) error {
start := time.Now()
defer func() { log.Debug(ctx, "Artwork: Prune finished", "elapsed", time.Since(start)) }()
repo := ds.Artwork()
repo := ds.Artwork(ctx)
purged, err := repo.PurgeDanglingItems(ctx)
purged, err := repo.PurgeDanglingItems()
if err != nil {
return err
}
@ -25,7 +25,7 @@ func prune(ctx context.Context, ds model.DataStore, store *ImageStore) error {
}
// Queue rows for deleted entities would otherwise retry forever (Get -> not found -> failed).
queuePurged, err := ds.ArtworkQueue().PurgeDangling(ctx)
queuePurged, err := ds.ArtworkQueue(ctx).PurgeDangling()
if err != nil {
return err
}
@ -35,7 +35,7 @@ func prune(ctx context.Context, ds model.DataStore, store *ImageStore) error {
// Files younger than the grace window may belong to acquisitions whose rows aren't committed yet.
cutoff := time.Now().Add(-pruneMinAge)
orphans, err := repo.PurgeOrphans(ctx, cutoff)
orphans, err := repo.PurgeOrphans(cutoff)
if err != nil {
return err
}
@ -44,7 +44,7 @@ func prune(ctx context.Context, ds model.DataStore, store *ImageStore) error {
}
// Read after the delete, so the sweep below reclaims the files of the rows just removed.
mimes, err := repo.GetMimeByHash(ctx)
mimes, err := repo.GetMimeByHash()
if err != nil {
return err
}

View file

@ -18,20 +18,18 @@ type flakyGetArtworkRepo struct {
*tests.MockArtworkRepo
}
func (f *flakyGetArtworkRepo) GetMimeByHash(context.Context) (map[string]string, error) {
func (f *flakyGetArtworkRepo) GetMimeByHash() (map[string]string, error) {
return nil, errors.New("db locked")
}
var _ = Describe("Prune", func() {
var ctx context.Context
var ds *tests.MockDataStore
var store *ImageStore
var awRepo *tests.MockArtworkRepo
BeforeEach(func() {
ctx = GinkgoT().Context()
ds = &tests.MockDataStore{}
awRepo = ds.Artwork().(*tests.MockArtworkRepo)
awRepo = ds.Artwork(context.Background()).(*tests.MockArtworkRepo)
store = NewImageStore(GinkgoT().TempDir())
})
@ -43,9 +41,9 @@ var _ = Describe("Prune", func() {
}
It("purges dangling item_artwork state for gone entities, summed across kinds", func() {
Expect(awRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "al", ItemID: "gone-album", ImageType: model.ImageTypePrimary})).To(Succeed())
Expect(awRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "ar", ItemID: "gone-artist", ImageType: model.ImageTypePrimary})).To(Succeed())
Expect(awRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "ar", ItemID: "live-artist", ImageType: model.ImageTypePrimary})).To(Succeed())
Expect(awRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: "al", ItemID: "gone-album", ImageType: model.ImageTypePrimary})).To(Succeed())
Expect(awRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: "ar", ItemID: "gone-artist", ImageType: model.ImageTypePrimary})).To(Succeed())
Expect(awRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: "ar", ItemID: "live-artist", ImageType: model.ImageTypePrimary})).To(Succeed())
awRepo.ExistingIDs = map[string]map[string]bool{
"al": {},
"ar": {"live-artist": true},
@ -53,17 +51,17 @@ var _ = Describe("Prune", func() {
Expect(prune(context.Background(), ds, store)).To(Succeed())
_, err := awRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "gone-album", model.ImageTypePrimary)
_, err := awRepo.GetItemArtwork(model.KindAlbumArtwork, "gone-album", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
_, err = awRepo.GetItemArtwork(ctx, model.KindArtistArtwork, "gone-artist", model.ImageTypePrimary)
_, err = awRepo.GetItemArtwork(model.KindArtistArtwork, "gone-artist", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
_, err = awRepo.GetItemArtwork(ctx, model.KindArtistArtwork, "live-artist", model.ImageTypePrimary)
_, err = awRepo.GetItemArtwork(model.KindArtistArtwork, "live-artist", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
})
It("purges dangling artwork_queue rows for gone entities", func() {
queueRepo := tests.CreateMockArtworkQueueRepo()
Expect(queueRepo.Enqueue(ctx,
Expect(queueRepo.Enqueue(
model.ArtworkQueueItem{ItemKind: "al", ItemID: "gone-album", ImageType: model.ImageTypePrimary},
model.ArtworkQueueItem{ItemKind: "al", ItemID: "live-album", ImageType: model.ImageTypePrimary},
)).To(Succeed())
@ -82,17 +80,17 @@ var _ = Describe("Prune", func() {
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
old := time.Now().Add(-2 * time.Hour)
Expect(os.Chtimes(store.path(h, "image/jpeg"), old, old)).To(Succeed())
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
ageArtwork(h, old)
kept := []byte("kept-bytes")
hk, _ := hashImage(bytes.NewReader(kept))
Expect(store.Write(hk, "image/jpeg", bytes.NewReader(kept))).To(Succeed())
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: hk, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: hk, Mime: "image/jpeg"})).To(Succeed())
Expect(prune(context.Background(), ds, store)).To(Succeed())
_, err := awRepo.GetImage(ctx, h)
_, err := awRepo.GetImage(h)
Expect(err).To(MatchError(model.ErrNotFound))
_, err = store.Open(h, "image/jpeg")
Expect(os.IsNotExist(err)).To(BeTrue())
@ -105,14 +103,14 @@ var _ = Describe("Prune", func() {
data := []byte("reacquired-bytes")
h, _ := hashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
ageArtwork(h, time.Now().Add(-2*time.Hour))
Expect(awRepo.PutItemArtwork(ctx, &model.ItemArtwork{ItemKind: "al", ItemID: "a1",
Expect(awRepo.PutItemArtwork(&model.ItemArtwork{ItemKind: "al", ItemID: "a1",
ImageType: model.ImageTypePrimary, Hash: h, Source: "folder"})).To(Succeed())
Expect(prune(context.Background(), ds, store)).To(Succeed())
_, err := awRepo.GetImage(ctx, h)
_, err := awRepo.GetImage(h)
Expect(err).ToNot(HaveOccurred())
rc, err := store.Open(h, "image/jpeg")
Expect(err).ToNot(HaveOccurred())
@ -124,11 +122,11 @@ var _ = Describe("Prune", func() {
h, _ := hashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
// Reacquisition refreshed created_at, so the row is unreferenced but too young to drop.
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(prune(context.Background(), ds, store)).To(Succeed())
_, err := awRepo.GetImage(ctx, h)
_, err := awRepo.GetImage(h)
Expect(err).ToNot(HaveOccurred())
rc, err := store.Open(h, "image/jpeg")
Expect(err).ToNot(HaveOccurred())
@ -139,7 +137,7 @@ var _ = Describe("Prune", func() {
data := []byte("racing-bytes")
h, _ := hashImage(bytes.NewReader(data))
Expect(store.Write(h, "image/jpeg", bytes.NewReader(data))).To(Succeed())
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
ageArtwork(h, time.Now().Add(-2*time.Hour))
// The row is orphaned, but a concurrent acquisition just touched the file's mtime.
@ -172,7 +170,7 @@ var _ = Describe("Prune", func() {
Expect(os.Chtimes(store.path(h, "image/png"), old, old)).To(Succeed())
Expect(os.Chtimes(store.path(h, "image/jpeg"), old, old)).To(Succeed())
// The row records the current mime; the .png file is a superseded variant.
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: h, Mime: "image/jpeg"})).To(Succeed())
Expect(prune(context.Background(), ds, store)).To(Succeed())
@ -194,14 +192,14 @@ var _ = Describe("Prune", func() {
hb, _ := hashImage(bytes.NewReader(blocked))
Expect(store.Write(hb, "image/jpeg", bytes.NewReader(blocked))).To(Succeed())
Expect(os.Chtimes(store.path(hb, "image/jpeg"), old, old)).To(Succeed())
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: hb, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: hb, Mime: "image/jpeg"})).To(Succeed())
ageArtwork(hb, old)
good := []byte("good-bytes")
hg, _ := hashImage(bytes.NewReader(good))
Expect(store.Write(hg, "image/jpeg", bytes.NewReader(good))).To(Succeed())
Expect(os.Chtimes(store.path(hg, "image/jpeg"), old, old)).To(Succeed())
Expect(awRepo.PutImage(ctx, &model.Artwork{Hash: hg, Mime: "image/jpeg"})).To(Succeed())
Expect(awRepo.PutImage(&model.Artwork{Hash: hg, Mime: "image/jpeg"})).To(Succeed())
ageArtwork(hg, old)
// A read-only shard directory makes os.Remove fail (EACCES) for hb's file only.
@ -212,13 +210,13 @@ var _ = Describe("Prune", func() {
Expect(prune(context.Background(), ds, store)).To(Succeed())
_, err := awRepo.GetImage(ctx, hg)
_, err := awRepo.GetImage(hg)
Expect(err).To(MatchError(model.ErrNotFound))
_, err = store.Open(hg, "image/jpeg")
Expect(os.IsNotExist(err)).To(BeTrue())
// The row purge does not depend on file removal, so only the file survives.
_, err = awRepo.GetImage(ctx, hb)
_, err = awRepo.GetImage(hb)
Expect(err).To(MatchError(model.ErrNotFound))
rc, err := store.Open(hb, "image/jpeg")
Expect(err).ToNot(HaveOccurred())

View file

@ -3,6 +3,7 @@ package artwork
import (
"bytes"
"context"
"errors"
"fmt"
"image"
"image/draw"
@ -48,9 +49,9 @@ func resizeImageData(ctx context.Context, ffm ffmpeg.FFmpeg, data []byte, size i
if isAnimatedGIF(data) {
if ffm.IsAvailable() {
// Animated GIF: convert to animated WebP via ffmpeg (with optional resize)
r, err := ffm.ConvertAnimatedImage(ctx, bytes.NewReader(data), size, conf.Server.CoverArtQuality)
out, err := convertAnimatedGIF(ctx, ffm, data, size)
if err == nil {
return r, 0, nil
return bytes.NewReader(out), 0, nil
}
log.Warn(ctx, "Artwork: Could not convert animated GIF, falling back to static", err)
}
@ -62,6 +63,24 @@ func resizeImageData(ctx context.Context, ffm ffmpeg.FFmpeg, data []byte, size i
return resizeStaticImage(data, size, square)
}
// convertAnimatedGIF buffers the output because ffmpeg can fail after it starts streaming,
// too late for a caller holding the stream to fall back.
func convertAnimatedGIF(ctx context.Context, ffm ffmpeg.FFmpeg, data []byte, size int) ([]byte, error) {
r, err := ffm.ConvertAnimatedImage(ctx, bytes.NewReader(data), size, conf.Server.CoverArtQuality)
if err != nil {
return nil, err
}
defer r.Close()
out, err := io.ReadAll(r)
if err != nil {
return nil, err
}
if len(out) == 0 {
return nil, errors.New("ffmpeg produced no output")
}
return out, nil
}
// toFastScaleType converts types x/image/draw has no optimized scaler for (e.g. *image.NYCbCrA,
// *image.Paletted) to *image.RGBA, avoiding CatmullRom.Scale's generic per-pixel fallback.
func toFastScaleType(img image.Image) image.Image {

View file

@ -1,6 +1,17 @@
package artwork
import (
"bytes"
"context"
"errors"
"image"
"io"
"sync/atomic"
"testing/iotest"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/tests"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
@ -11,3 +22,62 @@ var _ = Describe("resizeStaticImage", func() {
Expect(err).To(MatchError(ContainSubstring("exceed pixel cap")))
})
})
var _ = Describe("resizeImageData", func() {
var gifBytes []byte
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
conf.Server.EnableWebPEncoding = false
gifBytes = createAnimatedGIF(3)
})
It("converts an animated GIF with ffmpeg", func() {
fake := &animFFmpeg{MockFFmpeg: tests.NewMockFFmpeg(""), out: []byte("animated-webp")}
r, _, err := resizeImageData(context.Background(), fake, gifBytes, 1, false)
Expect(err).ToNot(HaveOccurred())
Expect(io.ReadAll(r)).To(Equal([]byte("animated-webp")))
})
DescribeTable("falls back to a static resize when ffmpeg fails",
func(fake *animFFmpeg) {
r, _, err := resizeImageData(context.Background(), fake, gifBytes, 1, false)
Expect(err).ToNot(HaveOccurred())
data, err := io.ReadAll(r)
Expect(err).ToNot(HaveOccurred())
_, format, err := image.DecodeConfig(bytes.NewReader(data))
Expect(err).ToNot(HaveOccurred())
Expect(format).To(Equal("jpeg"))
},
Entry("before producing output", &animFFmpeg{MockFFmpeg: &tests.MockFFmpeg{Error: errors.New("no libwebp_anim")}}),
Entry("mid-stream, like ffmpeg 5.1 reading a GIF from a pipe", &animFFmpeg{MockFFmpeg: tests.NewMockFFmpeg(""), streamErr: errors.New("pipe:0: Input/output error")}),
Entry("with empty output", &animFFmpeg{MockFFmpeg: tests.NewMockFFmpeg("")}),
)
})
// animFFmpeg fakes animated conversion; a non-nil release blocks it until closed.
type animFFmpeg struct {
*tests.MockFFmpeg
calls atomic.Int32
release chan struct{}
out []byte
streamErr error
}
func (f *animFFmpeg) ConvertAnimatedImage(ctx context.Context, _ io.Reader, _ int, _ int) (io.ReadCloser, error) {
f.calls.Add(1)
if f.release != nil {
select {
case <-f.release:
case <-ctx.Done():
return nil, ctx.Err()
}
}
if f.Error != nil {
return nil, f.Error
}
if f.streamErr != nil {
return io.NopCloser(iotest.ErrReader(f.streamErr)), nil //nolint:nilerr // the stream fails, not the call
}
return io.NopCloser(bytes.NewReader(f.out)), nil
}

View file

@ -198,7 +198,7 @@ func (r *resolver) fetchExternalArtist(ctx context.Context, ar model.Artist) (io
// resolveAlbum walks conf.Server.CoverArtPriority over the folder, embedded and external sources.
func (r *resolver) resolveAlbum(ctx context.Context, albumID string) (resolution, error) {
al, err := r.ds.Album().Get(ctx, albumID)
al, err := r.ds.Album(ctx).Get(albumID)
if err != nil {
return resolution{}, err
}
@ -243,7 +243,7 @@ func (r *resolver) resolveAlbum(ctx context.Context, albumID string) (resolution
// resolveArtist tries the uploaded image first, then walks conf.Server.ArtistArtPriority.
func (r *resolver) resolveArtist(ctx context.Context, artistID string) (resolution, error) {
ar, err := r.ds.Artist().Get(ctx, artistID)
ar, err := r.ds.Artist(ctx).Get(artistID)
if err != nil {
return resolution{}, err
}
@ -259,7 +259,7 @@ func (r *resolver) resolveArtist(ctx context.Context, artistID string) (resoluti
}
// Only consider albums where the artist is the sole album artist.
als, err := r.ds.Album().GetAll(ctx, model.QueryOptions{Filters: persistence.SoleAlbumArtistFilter(artistID)})
als, err := r.ds.Album(ctx).GetAll(model.QueryOptions{Filters: persistence.SoleAlbumArtistFilter(artistID)})
if err != nil {
return resolution{}, err
}
@ -328,7 +328,7 @@ const PlaylistGridSamples = 4
// resolvePlaylist tries the uploaded image, the sidecar and ExternalImageURL, then a generated grid.
func (r *resolver) resolvePlaylist(ctx context.Context, playlistID string) (resolution, error) {
pl, err := r.ds.Playlist().Get(ctx, playlistID)
pl, err := r.ds.Playlist(ctx).Get(playlistID)
if err != nil {
return resolution{}, err
}
@ -374,11 +374,8 @@ func (r *resolver) resolvePlaylist(ctx context.Context, playlistID string) (reso
}
}
tracks := r.ds.Playlist().Tracks(ctx, pl.ID, false)
if tracks == nil {
return resolution{}, fmt.Errorf("resolvePlaylist: could not load tracks for playlist %s", pl.ID)
}
albumIDs, err := tracks.GetAlbumIDs(ctx, model.QueryOptions{Max: PlaylistGridSamples, Sort: "random()"})
albumIDs, err := r.ds.Playlist(ctx).Tracks(pl.ID, false).
GetAlbumIDs(model.QueryOptions{Max: PlaylistGridSamples, Sort: "random()"})
if err != nil {
return resolution{}, err
}
@ -431,7 +428,7 @@ func (r *resolver) resolvePlaylist(ctx context.Context, playlistID string) (reso
// resolveRadio serves only an uploaded image; there is no fallback.
func (r *resolver) resolveRadio(ctx context.Context, radioID string) (resolution, error) {
radio, err := r.ds.Radio().Get(ctx, radioID)
radio, err := r.ds.Radio(ctx).Get(radioID)
if err != nil {
return resolution{}, err
}
@ -442,7 +439,7 @@ func (r *resolver) resolveRadio(ctx context.Context, radioID string) (resolution
// resolveMediaFile resolves a track's own embedded art only, so disabled or missing cover art
// is a definitive absent.
func (r *resolver) resolveMediaFile(ctx context.Context, id string) (resolution, error) {
mf, err := r.ds.MediaFile().Get(ctx, id)
mf, err := r.ds.MediaFile(ctx).Get(id)
if err != nil {
return resolution{}, err
}
@ -575,7 +572,7 @@ func resolveArtistFolderPattern(ctx context.Context, lib libraryView, artistFold
// resolveLocalFile opens an absolute path directly. A missing path is "no source"; any other
// open failure says nothing about whether the image exists.
func resolveLocalFile(path, source string) (resolution, bool) {
if path == "" || !model.IsImageFile(path) {
if path == "" {
return resolution{}, false
}
f, err := os.Open(path)

View file

@ -498,23 +498,6 @@ var _ = Describe("resolveItem", func() {
Expect(res.refMtime).To(BeNumerically(">", 0))
})
It("never opens a local ExternalImageURL that is not an image file", func() {
folderRepo.result = nil // no grid tiles, so only the local file could produce a reader
dir := GinkgoT().TempDir()
secretPath := filepath.Join(dir, "config.ini")
Expect(os.WriteFile(secretPath, []byte("password=secret"), 0600)).To(Succeed())
plRepo := tests.CreateMockPlaylistRepo()
plRepo.SetData(model.Playlists{{ID: "plni", Name: "Playlist", ExternalImageURL: secretPath}})
plRepo.TracksRepo = &tests.MockPlaylistTrackRepo{AlbumIDs: []string{"t1"}}
ds.MockedPlaylist = plRepo
res, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "pl", ItemID: "plni"})
Expect(err).ToNot(HaveOccurred())
Expect(res.reader).To(BeNil())
Expect(res.sourcePath).ToNot(Equal(secretPath))
})
It("routes ExternalImageURL through extGate and sets extError on transient failure", func() {
conf.Server.EnableM3UExternalAlbumArt = true
folderRepo.result = nil // no grid tiles, so the external failure is what surfaces
@ -707,16 +690,6 @@ var _ = Describe("resolveItem", func() {
Expect(err).To(HaveOccurred())
Expect(res).To(Equal(resolution{}))
})
It("returns an error when the playlist tracks cannot be loaded", func() {
plRepo := tests.CreateMockPlaylistRepo()
plRepo.SetData(model.Playlists{{ID: "pl4", Name: "Playlist"}})
ds.MockedPlaylist = plRepo
res, err := newResolver(ds, ag, ffm, nil).resolve(ctx, model.ArtworkQueueItem{ItemKind: "pl", ItemID: "pl4"})
Expect(err).To(HaveOccurred())
Expect(res).To(Equal(resolution{}))
})
})
})

View file

@ -18,7 +18,6 @@ import (
"github.com/navidrome/navidrome/log"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/httpclient"
"github.com/navidrome/navidrome/utils/netguard"
"go.senan.xyz/taglib"
)
@ -38,7 +37,7 @@ func fromExternalFile(ctx context.Context, libFS fs.FS, files []string, pattern
log.Warn(ctx, "Artwork: Error matching cover art file to pattern", "pattern", pattern, "file", file)
continue
}
if !match || !model.IsImageFile(name) {
if !match {
continue
}
f, err := libFS.Open(file)
@ -151,18 +150,10 @@ type readCloser struct {
io.Closer
}
// remoteImageClient fetches URLs from playlists and agents (plugins included), so it must not reach
// internal hosts. Shared so fetches reuse connections.
var remoteImageClient = httpclient.NewExternal(5 * time.Second)
func fromURL(ctx context.Context, imageUrl *url.URL) (io.ReadCloser, string, error) {
hc := httpclient.New(5 * time.Second)
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, imageUrl.String(), nil)
resp, err := remoteImageClient.Do(req)
if errors.Is(err, netguard.ErrPrivateAddress) {
// Retrying cannot change where the URL points: settle absent instead of tripping the breaker.
log.Warn(ctx, "Artwork: Refused to fetch image from a private or loopback address", "url", imageUrl, err)
return nil, "", model.ErrNotFound
}
resp, err := hc.Do(req) //nolint:gosec
if err != nil {
return nil, "", err
}

View file

@ -5,87 +5,13 @@ import (
"errors"
"io"
"io/fs"
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"os"
"strings"
"sync/atomic"
"testing/fstest"
"time"
"github.com/navidrome/navidrome/model"
"github.com/navidrome/navidrome/utils/httpclient"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("fromURL", func() {
var (
hits atomic.Int32
target *httptest.Server
)
BeforeEach(func() {
hits.Store(0)
target = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
hits.Add(1)
_, _ = w.Write([]byte("image-bytes"))
}))
DeferCleanup(target.Close)
})
useClient := func(c *http.Client) {
prev := remoteImageClient
remoteImageClient = c
DeferCleanup(func() { remoteImageClient = prev })
}
fetch := func(rawURL string) ([]byte, error) {
u, err := url.Parse(rawURL)
Expect(err).ToNot(HaveOccurred())
r, _, err := fromURL(GinkgoT().Context(), u)
if err != nil {
return nil, err
}
defer r.Close()
return io.ReadAll(r)
}
// Stand-in for a public host: only 127.0.0.1 is allowed, so every other private address stays refused.
onlyLocalhostV4 := func() *http.Client {
return httpclient.NewExternal(5*time.Second, netip.MustParsePrefix("127.0.0.1/32"))
}
DescribeTable("refuses private and loopback targets as a definitive miss",
func(rawURL string) {
useClient(productionImageClient)
u, _ := url.Parse(target.URL)
_, err := fetch(strings.ReplaceAll(rawURL, "PORT", u.Port()))
Expect(err).To(MatchError(model.ErrNotFound))
Expect(hits.Load()).To(BeZero())
},
Entry("IPv4 loopback", "http://127.0.0.1:PORT/x"),
Entry("localhost", "http://localhost:PORT/x"),
Entry("cloud metadata", "http://169.254.169.254/"),
Entry("IPv6 loopback", "http://[::1]/"),
)
It("refuses a redirect from an allowed host to a loopback address", func() {
useClient(onlyLocalhostV4())
redirector := httptest.NewServer(http.RedirectHandler(strings.Replace(target.URL, "127.0.0.1", "127.0.0.2", 1), http.StatusFound))
DeferCleanup(redirector.Close)
_, err := fetch(redirector.URL)
Expect(err).To(MatchError(model.ErrNotFound))
Expect(hits.Load()).To(BeZero())
})
It("fetches from an allowed address", func() {
useClient(onlyLocalhostV4())
Expect(fetch(target.URL + "/cover.jpg")).To(Equal([]byte("image-bytes")))
})
})
var _ = Describe("fromExternalFile", func() {
It("opens a matching file via the library FS", func() {
fsys := fstest.MapFS{
@ -122,18 +48,6 @@ var _ = Describe("fromExternalFile", func() {
Expect(b).To(Equal([]byte("a")))
Expect(path).To(Equal("a/cover.jpg"))
})
It("skips a matching file that is not an image", func() {
fsys := fstest.MapFS{
"a/cover.ini": &fstest.MapFile{Data: []byte("password=secret")},
"a/cover.jpg": &fstest.MapFile{Data: []byte("a")},
}
f := fromExternalFile(GinkgoT().Context(), fsys, []string{"a/cover.ini", "a/cover.jpg"}, "cover.*")
r, path, err := f()
Expect(err).ToNot(HaveOccurred())
defer r.Close()
Expect(path).To(Equal("a/cover.jpg"))
})
})
var _ = Describe("fromTag", func() {

View file

@ -16,14 +16,12 @@ import (
)
var _ = Describe("Uploader", func() {
var ctx context.Context
var svc Uploader
var tmpDir string
var artRepo *tests.MockArtworkRepo
var queueRepo *tests.MockArtworkQueueRepo
BeforeEach(func() {
ctx = GinkgoT().Context()
DeferCleanup(configtest.SetupConfig())
tmpDir = GinkgoT().TempDir()
conf.Server.DataFolder = conf.NewDir(tmpDir)
@ -35,6 +33,7 @@ var _ = Describe("Uploader", func() {
Describe("SetImage", func() {
It("creates directory and saves image file", func() {
ctx := context.Background()
reader := strings.NewReader("fake image data")
filename, err := svc.SetImage(ctx, consts.EntityArtist, "ar-1", "Pink Floyd", "", reader, ".jpg")
Expect(err).ToNot(HaveOccurred())
@ -47,6 +46,7 @@ var _ = Describe("Uploader", func() {
})
It("falls back to ID-only filename when name cleans to empty", func() {
ctx := context.Background()
reader := strings.NewReader("data")
filename, err := svc.SetImage(ctx, consts.EntityPlaylist, "pl-1", "!!!", "", reader, ".png")
Expect(err).ToNot(HaveOccurred())
@ -54,6 +54,7 @@ var _ = Describe("Uploader", func() {
})
It("removes old image when replacing", func() {
ctx := context.Background()
oldDir := filepath.Join(tmpDir, "artwork", "artist")
Expect(os.MkdirAll(oldDir, 0755)).To(Succeed())
oldFile := filepath.Join(oldDir, "ar-1_old.png")
@ -69,13 +70,15 @@ var _ = Describe("Uploader", func() {
})
It("ignores missing old file without error", func() {
ctx := context.Background()
reader := strings.NewReader("data")
_, err := svc.SetImage(ctx, consts.EntityArtist, "ar-1", "Name", "/nonexistent/path.jpg", reader, ".jpg")
Expect(err).ToNot(HaveOccurred())
})
It("does not touch artwork state or the queue (that is EnqueueArtwork's job, post-Put)", func() {
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
ctx := context.Background()
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "ar", ItemID: "ar-1", Hash: "oldhash", Source: "external",
})).To(Succeed())
@ -84,24 +87,25 @@ var _ = Describe("Uploader", func() {
// SetImage only writes the file; the state row survives and nothing is queued until
// the caller has persisted the new filename and called EnqueueArtwork.
_, err = artRepo.GetItemArtwork(ctx, model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(queueRepo.DequeueBatch(ctx, 1000)).To(BeEmpty())
Expect(queueRepo.DequeueBatch(1000)).To(BeEmpty())
})
})
Describe("EnqueueArtwork", func() {
It("clears artwork state and enqueues a Bump", func() {
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
ctx := context.Background()
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "ar", ItemID: "ar-1", Hash: "oldhash", Source: "external",
})).To(Succeed())
svc.EnqueueArtwork(ctx, consts.EntityArtist, "ar-1")
_, err := artRepo.GetItemArtwork(ctx, model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindArtistArtwork, "ar-1", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound))
queued, err := queueRepo.DequeueBatch(ctx, 1000)
queued, err := queueRepo.DequeueBatch(1000)
Expect(err).ToNot(HaveOccurred())
Expect(queued).To(ContainElement(SatisfyAll(
HaveField("ItemKind", "ar"),
@ -112,12 +116,13 @@ var _ = Describe("Uploader", func() {
It("is a no-op for an unknown entity type", func() {
svc.EnqueueArtwork(context.Background(), "unknown", "x-1")
Expect(queueRepo.DequeueBatch(ctx, 1000)).To(BeEmpty())
Expect(queueRepo.DequeueBatch(1000)).To(BeEmpty())
})
})
Describe("RemoveImage", func() {
It("removes the file at the given path", func() {
ctx := context.Background()
dir := filepath.Join(tmpDir, "artwork", "artist")
Expect(os.MkdirAll(dir, 0755)).To(Succeed())
path := filepath.Join(dir, "ar-1_test.jpg")
@ -129,11 +134,13 @@ var _ = Describe("Uploader", func() {
})
It("succeeds when file does not exist", func() {
ctx := context.Background()
err := svc.RemoveImage(ctx, "/nonexistent/file.jpg")
Expect(err).ToNot(HaveOccurred())
})
It("succeeds with empty path", func() {
ctx := context.Background()
err := svc.RemoveImage(ctx, "")
Expect(err).ToNot(HaveOccurred())
})

View file

@ -4,11 +4,9 @@ import (
"bytes"
"cmp"
"context"
"fmt"
"io"
"math"
"math/rand/v2"
"runtime/debug"
"sync"
"time"
@ -47,8 +45,7 @@ type Worker struct {
broker events.Broker
pruneMu sync.RWMutex
pools []*drainPool
runCtx context.Context //nolint:containedctx // worker lifecycle ctx, set at Run
paused func() bool
runCtx context.Context
gatesMu sync.Mutex
gates map[string]*extGate
@ -62,7 +59,6 @@ func NewWorker(ds model.DataStore, store *ImageStore, ag *agents.Agents, ffmpeg
broker: broker,
pools: newDrainPools(),
runCtx: context.Background(),
paused: func() bool { return false },
gates: map[string]*extGate{},
}
w.proc.resolver = newResolver(ds, ag, ffmpeg, w.gate)
@ -94,11 +90,6 @@ var (
}
)
// PauseWhile holds off queue draining whenever paused reports true. Call it before Run.
func (w *Worker) PauseWhile(paused func() bool) {
w.paused = paused
}
// Run blocks draining the queue until ctx is cancelled.
func (w *Worker) Run(ctx context.Context) error {
w.runCtx = ctx
@ -152,12 +143,9 @@ func (w *Worker) EnqueueMissingAll(ctx context.Context) error {
}
func (w *Worker) drain(ctx context.Context, concurrency int, kinds ...string) (int, error) {
if w.paused() {
return 0, nil
}
// Dequeue well past the pool size so a slow external lookup never idles the other slots.
// DequeueBatch does not mark rows taken, so this is one query per pass, not per slot.
items, err := w.proc.ds.ArtworkQueue().DequeueBatch(ctx, max(16, 4*concurrency), kinds...)
items, err := w.proc.ds.ArtworkQueue(ctx).DequeueBatch(max(16, 4*concurrency), kinds...)
if err != nil {
return 0, err
}
@ -182,9 +170,6 @@ func (w *Worker) drain(ctx context.Context, concurrency int, kinds ...string) (i
wg.Wait()
return len(items), nil //nolint:nilerr // a cancelled drain is a clean stop, not an error
}
if w.paused() {
break
}
wg.Go(func() {
defer func() { <-sem }()
out, got := w.process(ctx, item)
@ -246,14 +231,14 @@ func (w *Worker) process(ctx context.Context, item model.ArtworkQueueItem) (outc
item.ImageType = cmp.Or(item.ImageType, model.ImageTypePrimary)
trace := &ChainTrace{}
ctx = withTrace(ctx, trace)
out, got, retryIn := w.safeAcquire(ctx, item)
out, got, retryIn := w.proc.acquire(ctx, item)
queue := w.proc.ds.ArtworkQueue()
queue := w.proc.ds.ArtworkQueue(ctx)
switch out {
case outcomeFound, outcomeAbsent:
// A scan that re-enqueued this row mid-flight reset its retry_at, so the row survives
// here and the next drain re-resolves it.
if err := queue.DeleteIfUnchanged(ctx, item.ItemKind, item.ItemID, item.ImageType, item.RetryAt); err != nil {
if err := queue.DeleteIfUnchanged(item.ItemKind, item.ItemID, item.ImageType, item.RetryAt); err != nil {
log.Warn(ctx, "Artwork: Could not delete processed queue item", "kind", item.ItemKind, "id", item.ItemID, err)
}
case outcomeFoundStale, outcomeFailed:
@ -262,7 +247,7 @@ func (w *Worker) process(ctx context.Context, item model.ArtworkQueueItem) (outc
if retryAt.Before(item.EnqueuedAt.Add(giveUpAfter)) {
// A mid-flight re-enqueue reset retry_at; stale backoff must not stomp its
// fresh, immediate eligibility.
if err := queue.MarkFailedIfUnchanged(ctx, item.ItemKind, item.ItemID, item.ImageType, item.RetryAt, retryAt, encoded); err != nil {
if err := queue.MarkFailedIfUnchanged(item.ItemKind, item.ItemID, item.ImageType, item.RetryAt, retryAt, encoded); err != nil {
log.Warn(ctx, "Artwork: Could not reschedule failed queue item", "kind", item.ItemKind, "id", item.ItemID, err)
}
log.Debug(ctx, "Artwork: Rescheduled item", "kind", item.ItemKind, "id", item.ItemID,
@ -273,7 +258,7 @@ func (w *Worker) process(ctx context.Context, item model.ArtworkQueueItem) (outc
// Art already being served is kept: exhaustion means unreachable, not removed.
settled := "kept previous state"
if out == outcomeFailed && settlesAbsentOnGiveUp(item.ItemKind) && !w.hasResolvedArtwork(ctx, item) {
writeAbsent(ctx, w.proc.ds.Artwork(), item)
writeAbsent(ctx, w.proc.ds.Artwork(ctx), item)
settled = "recorded absent"
}
// The queue row is about to go, taking the only record of the failure with it. This write is
@ -281,27 +266,13 @@ func (w *Worker) process(ctx context.Context, item model.ArtworkQueueItem) (outc
w.recordGiveUp(ctx, item, encoded)
log.Info(ctx, "Artwork: Retry budget exhausted, giving up", "kind", item.ItemKind, "id", item.ItemID,
"outcome", out, "attempts", item.Attempts+1, "budget", giveUpAfter, "settled", settled)
if err := queue.DeleteIfUnchanged(ctx, item.ItemKind, item.ItemID, item.ImageType, item.RetryAt); err != nil {
if err := queue.DeleteIfUnchanged(item.ItemKind, item.ItemID, item.ImageType, item.RetryAt); err != nil {
log.Warn(ctx, "Artwork: Could not remove exhausted queue item", "kind", item.ItemKind, "id", item.ItemID, err)
}
}
return out, got
}
// safeAcquire turns a panic into a failed attempt: the drain runs on a bare goroutine, so an
// unrecovered panic would crash the server, and the still-queued row would crash it again on restart.
func (w *Worker) safeAcquire(ctx context.Context, item model.ArtworkQueueItem) (out outcome, got *acquired, retryIn time.Duration) {
defer func() {
if r := recover(); r != nil {
log.Error(ctx, "Artwork: Panic while processing item", "kind", item.ItemKind, "id", item.ItemID,
"imageType", item.ImageType, "attempts", item.Attempts, "panic", r, "stack", string(debug.Stack()))
traceStage(ctx, "panic", fmt.Errorf("%v", r))
out, got, retryIn = outcomeFailed, nil, 0
}
}()
return w.proc.acquire(ctx, item)
}
// recordGiveUp keeps the last failure on the state row after the queue row is deleted. An item
// that never resolved has no row to update, and creating one would settle it absent.
func (w *Worker) recordGiveUp(ctx context.Context, item model.ArtworkQueueItem, trace string) {
@ -309,7 +280,7 @@ func (w *Worker) recordGiveUp(ctx context.Context, item model.ArtworkQueueItem,
if !ok {
return
}
if err := w.proc.ds.Artwork().PutLastFailure(ctx, kind, item.ItemID, item.ImageType, trace); err != nil {
if err := w.proc.ds.Artwork(ctx).PutLastFailure(kind, item.ItemID, item.ImageType, trace); err != nil {
log.Warn(ctx, "Artwork: Could not record the last failure", "kind", item.ItemKind, "id", item.ItemID, err)
}
}
@ -319,7 +290,7 @@ func (w *Worker) hasResolvedArtwork(ctx context.Context, item model.ArtworkQueue
if !ok {
return false
}
ia, err := w.proc.ds.Artwork().GetItemArtwork(ctx, kind, item.ItemID, item.ImageType)
ia, err := w.proc.ds.Artwork(ctx).GetItemArtwork(kind, item.ItemID, item.ImageType)
return err == nil && ia.Hash != ""
}
@ -339,13 +310,10 @@ func (w *Worker) precache(ctx context.Context, got *acquired) {
ffmpeg: w.ffmpeg,
open: func() (io.ReadCloser, error) { return io.NopCloser(bytes.NewReader(got.data)), nil },
}
stream, err := w.cache.Get(ctx, item)
if err != nil {
if err := warmCache(ctx, w.cache, item); err != nil {
log.Debug(ctx, "Artwork: Precache failed", "kind", got.ia.ItemKind, "id", got.ia.ItemID, err)
return
}
_, _ = io.Copy(io.Discard, stream)
_ = stream.Close()
log.Trace(ctx, "Artwork: Precached UI size", "kind", got.ia.ItemKind, "id", got.ia.ItemID,
"size", conf.Server.UICoverArtSize, "elapsed", time.Since(precacheStart))
}

View file

@ -21,12 +21,6 @@ import (
const soakCycles = 2200
var _ = Describe("Worker soak", func() {
var ctx context.Context
BeforeEach(func() {
ctx = GinkgoT().Context()
})
It("does not leak goroutines, heap, or fds over many acquisition cycles", func() {
if testing.Short() {
Skip("skipping soak test in short mode")
@ -106,9 +100,9 @@ var _ = Describe("Worker soak", func() {
// Read-back exercises the surfaces a caller would use after acquisition.
if out == outcomeFound {
kind, _ := model.ParseKind(it.ItemKind)
ia, err := artRepo.GetItemArtwork(ctx, kind, it.ItemID, model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(kind, it.ItemID, model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred(), "cycle %d: GetItemArtwork", i)
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred(), "cycle %d: GetImage", i)
rc, err := store.Open(ia.Hash, art.Mime)
switch {

View file

@ -56,8 +56,8 @@ type reenqueueOnDequeue struct {
done bool
}
func (r *reenqueueOnDequeue) DequeueBatch(ctx context.Context, n int, kinds ...string) ([]model.ArtworkQueueItem, error) {
items, err := r.MockArtworkQueueRepo.DequeueBatch(ctx, n, kinds...)
func (r *reenqueueOnDequeue) DequeueBatch(n int, kinds ...string) ([]model.ArtworkQueueItem, error) {
items, err := r.MockArtworkQueueRepo.DequeueBatch(n, kinds...)
if !r.done && len(items) > 0 {
r.done = true
for k, it := range r.Data {
@ -124,39 +124,18 @@ type visibilityPlaylistDS struct {
tracks model.PlaylistTrackRepository
}
func (v *visibilityPlaylistDS) Playlist() model.PlaylistRepository {
func (v *visibilityPlaylistDS) Playlist(ctx context.Context) model.PlaylistRepository {
repo := tests.CreateMockPlaylistRepo()
repo.TracksRepo = v.tracks
repo.SetData(model.Playlists{v.private})
return &visibilityPlaylistRepo{MockPlaylistRepo: repo}
}
type visibilityPlaylistRepo struct {
*tests.MockPlaylistRepo
}
func (v *visibilityPlaylistRepo) Get(ctx context.Context, id string) (*model.Playlist, error) {
if u, ok := request.UserFrom(ctx); !ok || !u.IsAdmin {
return nil, model.ErrNotFound
if u, ok := request.UserFrom(ctx); ok && u.IsAdmin {
repo.SetData(model.Playlists{v.private})
}
return v.MockPlaylistRepo.Get(ctx, id)
}
type panickingAlbumRepo struct {
*tests.MockAlbumRepo
panicID string
}
func (r *panickingAlbumRepo) Get(ctx context.Context, id string) (*model.Album, error) {
if id == r.panicID {
panic("boom")
}
return r.MockAlbumRepo.Get(ctx, id)
return repo
}
func adminUserRepo() *tests.MockedUserRepo {
repo := tests.CreateMockUserRepo()
Expect(repo.Put(GinkgoT().Context(), &model.User{ID: "admin", UserName: "admin", IsAdmin: true})).To(Succeed())
Expect(repo.Put(&model.User{ID: "admin", UserName: "admin", IsAdmin: true})).To(Succeed())
return repo
}
@ -178,8 +157,8 @@ var _ = Describe("Worker", func() {
)
BeforeEach(func() {
ctx = GinkgoT().Context()
DeferCleanup(configtest.SetupConfig())
ctx = context.Background()
var err error
repoRoot, err = os.Getwd()
Expect(err).ToNot(HaveOccurred())
@ -221,7 +200,7 @@ var _ = Describe("Worker", func() {
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{
{ID: "al1", Name: "Album", FolderIDs: []string{"f1"}},
})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "al", ItemID: "al1", Priority: model.ArtworkPriorityScan,
})).To(Succeed())
@ -229,11 +208,11 @@ var _ = Describe("Worker", func() {
Expect(err).ToNot(HaveOccurred())
Expect(n).To(Equal(1))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"))
count, err := queueRepo.Count(ctx)
count, err := queueRepo.Count()
Expect(err).ToNot(HaveOccurred())
Expect(count).To(BeZero(), "a found item must be deleted from the queue")
})
@ -244,7 +223,7 @@ var _ = Describe("Worker", func() {
ds.MockedMediaFile.(*tests.MockMediaFileRepo).SetData(model.MediaFiles{
{ID: "mf1", LibraryID: 0, Path: "tests/fixtures/artist/an-album/test.mp3", HasCoverArt: true},
})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "mf", ItemID: "mf1", Priority: model.ArtworkPriorityBump,
})).To(Succeed())
@ -252,12 +231,12 @@ var _ = Describe("Worker", func() {
Expect(err).ToNot(HaveOccurred())
Expect(n).To(Equal(1))
ia, err := artRepo.GetItemArtwork(ctx, model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mf1", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("embedded"))
Expect(ia.Hash).ToNot(BeEmpty())
art, err := artRepo.GetImage(ctx, ia.Hash)
art, err := artRepo.GetImage(ia.Hash)
Expect(err).ToNot(HaveOccurred())
r, err := store.Open(ia.Hash, art.Mime)
Expect(err).ToNot(HaveOccurred())
@ -266,7 +245,7 @@ var _ = Describe("Worker", func() {
Expect(err).ToNot(HaveOccurred())
Expect(data).ToNot(BeEmpty(), "embedded bytes must be written to the store")
count, err := queueRepo.Count(ctx)
count, err := queueRepo.Count()
Expect(err).ToNot(HaveOccurred())
Expect(count).To(BeZero())
})
@ -275,7 +254,7 @@ var _ = Describe("Worker", func() {
conf.Server.CoverArtPriority = "external"
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al4", Name: "Album"}})
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al4"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al4"})).To(Succeed())
n, err := w.drain(ctx, 2)
Expect(err).ToNot(HaveOccurred())
@ -286,47 +265,17 @@ var _ = Describe("Worker", func() {
Expect(it.Attempts).To(Equal(1))
Expect(it.RetryAt).To(BeTemporally(">", time.Now()))
_, err = artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al4", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindAlbumArtwork, "al4", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "a timeout must never settle on absent")
})
It("fails an item that panics, without stopping the rest of the batch", func() {
folderRepo.result = []model.Folder{{
Path: "tests/fixtures/artist/an-album",
ImageFiles: []string{"cover.jpg"},
}}
albums := tests.CreateMockAlbumRepo()
albums.SetData(model.Albums{
{ID: "alboom", Name: "Album", FolderIDs: []string{"f1"}},
{ID: "alok", Name: "Album", FolderIDs: []string{"f1"}},
})
ds.MockedAlbum = &panickingAlbumRepo{MockAlbumRepo: albums, panicID: "alboom"}
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alboom"})).To(Succeed())
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alok"})).To(Succeed())
n, err := w.drain(ctx, 1)
Expect(err).ToNot(HaveOccurred())
Expect(n).To(Equal(2))
it := findQueued(queueRepo, "al", "alboom")
Expect(it).ToNot(BeNil(), "a panicking item must be rescheduled, not dropped")
Expect(it.Attempts).To(Equal(1))
Expect(it.RetryAt).To(BeTemporally(">", time.Now()))
Expect(it.Trace).To(ContainSubstring("boom"))
Expect(findQueued(queueRepo, "al", "alok")).To(BeNil())
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alok", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"))
})
It("reschedules past the provider's requested delay when it exceeds the backoff", func() {
conf.Server.CoverArtPriority = "external"
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al9", Name: "Album"}})
// Well above backoff(0)'s jittered ceiling, so only the hint can produce this retry_at.
const askedFor = 90 * time.Minute
imageAgents(&fakeImageAgent{name: "throttledAgent", err: &agents.RetryLaterError{RetryIn: askedFor}})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al9"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al9"})).To(Succeed())
n, err := w.drain(ctx, 2)
Expect(err).ToNot(HaveOccurred())
@ -347,7 +296,7 @@ var _ = Describe("Worker", func() {
{ID: "alstale", Name: "Album", FolderIDs: []string{"f1"}},
})
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alstale"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "alstale"})).To(Succeed())
n, err := w.drain(ctx, 2)
Expect(err).ToNot(HaveOccurred())
@ -358,7 +307,7 @@ var _ = Describe("Worker", func() {
Expect(it.Attempts).To(Equal(1))
Expect(it.RetryAt).To(BeTemporally(">", time.Now()))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alstale", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alstale", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"), "the fallback art is served meanwhile")
@ -378,7 +327,7 @@ var _ = Describe("Worker", func() {
racing := &reenqueueOnDequeue{MockArtworkQueueRepo: queueRepo}
ds.MockedArtworkQueue = racing
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "al", ItemID: "al7", Priority: model.ArtworkPriorityScan,
})).To(Succeed())
@ -388,7 +337,7 @@ var _ = Describe("Worker", func() {
// The concurrent re-enqueue changed retry_at, so the found-path delete was a no-op.
Expect(findQueued(queueRepo, "al", "al7")).ToNot(BeNil())
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al7", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al7", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Source).To(Equal("folder"))
})
@ -400,7 +349,7 @@ var _ = Describe("Worker", func() {
racing := &reenqueueOnDequeue{MockArtworkQueueRepo: queueRepo}
ds.MockedArtworkQueue = racing
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al8"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al8"})).To(Succeed())
dequeued := findQueued(queueRepo, "al", "al8").RetryAt
n, err := w.drain(ctx, 1)
@ -419,7 +368,7 @@ var _ = Describe("Worker", func() {
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al9", Name: "Album"}})
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al9"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al9"})).To(Succeed())
// Age the row past the retry budget.
expireQueued(queueRepo, "al9")
@ -428,7 +377,7 @@ var _ = Describe("Worker", func() {
Expect(n).To(Equal(1))
Expect(findQueued(queueRepo, "al", "al9")).To(BeNil())
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al9", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al9", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).To(BeEmpty())
})
@ -436,13 +385,13 @@ var _ = Describe("Worker", func() {
It("keeps already-served art when the retry budget is exhausted", func() {
conf.Server.CoverArtPriority = "external"
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al10", Name: "Album"}})
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "al", ItemID: "al10", ImageType: model.ImageTypePrimary,
Hash: "cafebabe", Source: "external:lastfm",
})).To(Succeed())
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al10"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al10"})).To(Succeed())
expireQueued(queueRepo, "al10")
n, err := w.drain(ctx, 1)
@ -450,7 +399,7 @@ var _ = Describe("Worker", func() {
Expect(n).To(Equal(1))
Expect(findQueued(queueRepo, "al", "al10")).To(BeNil())
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al10", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al10", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).To(Equal("cafebabe"), "a persistent outage must not discard served art")
})
@ -460,7 +409,7 @@ var _ = Describe("Worker", func() {
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al11", Name: "Album"}})
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al11"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al11"})).To(Succeed())
_, err := w.drain(ctx, 1)
Expect(err).ToNot(HaveOccurred())
@ -481,13 +430,13 @@ var _ = Describe("Worker", func() {
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al13", Name: "Album"}})
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al13"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al13"})).To(Succeed())
expireQueued(queueRepo, "al13")
_, err := w.drain(ctx, 1)
Expect(err).ToNot(HaveOccurred())
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al13", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al13", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred(), "settling absent must create the row the failure is written to")
Expect(ia.Hash).To(BeEmpty())
Expect(DecodeTrace(ia.LastFailure, "")).ToNot(BeEmpty())
@ -496,20 +445,20 @@ var _ = Describe("Worker", func() {
It("keeps the failure on the state row after the queue row is deleted", func() {
conf.Server.CoverArtPriority = "external"
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al12", Name: "Album"}})
Expect(artRepo.PutItemArtwork(ctx, &model.ItemArtwork{
Expect(artRepo.PutItemArtwork(&model.ItemArtwork{
ItemKind: "al", ItemID: "al12", ImageType: model.ImageTypePrimary,
Hash: "cafebabe", Source: "external:lastfm",
})).To(Succeed())
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
w = NewWorker(ds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al12"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al12"})).To(Succeed())
expireQueued(queueRepo, "al12")
_, err := w.drain(ctx, 1)
Expect(err).ToNot(HaveOccurred())
Expect(findQueued(queueRepo, "al", "al12")).To(BeNil())
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al12", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al12", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(DecodeTrace(ia.LastFailure, "")).ToNot(BeEmpty(),
"the queue row is gone, so this is the only remaining record of the failure")
@ -524,7 +473,7 @@ var _ = Describe("Worker", func() {
ds.MockedMediaFile.(*tests.MockMediaFileRepo).SetData(model.MediaFiles{
{ID: "mfX", LibraryID: 0, Path: "tests/fixtures/artist/an-album/gone.mp3", HasCoverArt: true},
})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "mf", ItemID: "mfX"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "mf", ItemID: "mfX"})).To(Succeed())
expireQueued(queueRepo, "mfX")
n, err := w.drain(ctx, 1)
@ -532,7 +481,7 @@ var _ = Describe("Worker", func() {
Expect(n).To(Equal(1))
Expect(findQueued(queueRepo, "mf", "mfX")).To(BeNil(), "the row must stop retrying")
_, err = artRepo.GetItemArtwork(ctx, model.KindMediaFileArtwork, "mfX", model.ImageTypePrimary)
_, err = artRepo.GetItemArtwork(model.KindMediaFileArtwork, "mfX", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound),
"no row leaves the track unresolved, so a later view can still recover it")
// Known gap: with no row and no absent settle, there is nowhere to keep the failure.
@ -547,14 +496,14 @@ var _ = Describe("Worker", func() {
tracks: &tests.MockPlaylistTrackRepo{},
}
w = NewWorker(vds, store, ag, ffm, broker, imgCache)
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "pl", ItemID: "plPriv"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "pl", ItemID: "plPriv"})).To(Succeed())
n, err := w.drain(ctx, 1)
Expect(err).ToNot(HaveOccurred())
Expect(n).To(Equal(1))
Expect(findQueued(queueRepo, "pl", "plPriv")).To(BeNil())
ia, err := artRepo.GetItemArtwork(ctx, model.KindPlaylistArtwork, "plPriv", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindPlaylistArtwork, "plPriv", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).To(BeEmpty())
})
@ -574,9 +523,9 @@ var _ = Describe("Worker", func() {
{ID: "al1", Name: "Album 1", FolderIDs: []string{"f1"}},
{ID: "al2", Name: "Album 2", FolderIDs: []string{"f1"}},
})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al1", Priority: model.ArtworkPriorityScan})).To(Succeed())
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al2", Priority: model.ArtworkPriorityScan})).To(Succeed())
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar1", Priority: model.ArtworkPriorityScan})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al1", Priority: model.ArtworkPriorityScan})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al2", Priority: model.ArtworkPriorityScan})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "ar", ItemID: "ar1", Priority: model.ArtworkPriorityScan})).To(Succeed())
n, err := w.drain(ctx, 3)
Expect(err).ToNot(HaveOccurred())
@ -623,7 +572,7 @@ var _ = Describe("Worker", func() {
conf.Server.CoverArtPriority = "cover.*" // local-only; no folder image → absent
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "al3", Name: "Artless"}})
folderRepo.result = nil
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "al3", Priority: model.ArtworkPriorityScan})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "al3", Priority: model.ArtworkPriorityScan})).To(Succeed())
n, err := w.drain(ctx, 2)
Expect(err).ToNot(HaveOccurred())
@ -633,7 +582,7 @@ var _ = Describe("Worker", func() {
Expect(evts).To(HaveLen(1), "a removed cover must live-refresh clients so they drop it")
Expect(evts[0].(*events.RefreshResource).Data(evts[0])).To(ContainSubstring("al3"))
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "al3", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "al3", model.ImageTypePrimary)
Expect(err).ToNot(HaveOccurred())
Expect(ia.Hash).To(BeEmpty(), "the outcome was absent, not found")
})
@ -642,7 +591,7 @@ var _ = Describe("Worker", func() {
conf.Server.CoverArtPriority = "external"
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: "alx", Name: "Album"}})
imageAgents(&fakeImageAgent{name: "failAgent", err: errors.New("agent timed out")})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{ItemKind: "al", ItemID: "alx"})).To(Succeed())
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{ItemKind: "al", ItemID: "alx"})).To(Succeed())
n, err := w.drain(ctx, 2)
Expect(err).ToNot(HaveOccurred())
@ -776,7 +725,7 @@ var _ = Describe("Worker", func() {
{ID: "alpc", Name: "Album", FolderIDs: []string{"f1"}},
})
conf.Server.UICoverArtSize = 300
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "al", ItemID: "alpc", Priority: model.ArtworkPriorityScan,
})).To(Succeed())
})
@ -864,11 +813,11 @@ var _ = Describe("Worker", func() {
// Artists first, exactly as Backfill orders them.
for _, a := range artists {
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "ar", ItemID: a.ID, Priority: model.ArtworkPriorityBackfill,
})).To(Succeed())
}
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "al", ItemID: "alx", Priority: model.ArtworkPriorityBackfill,
})).To(Succeed())
@ -883,12 +832,12 @@ var _ = Describe("Worker", func() {
})
Eventually(func() bool {
ia, err := artRepo.GetItemArtwork(ctx, model.KindAlbumArtwork, "alx", model.ImageTypePrimary)
ia, err := artRepo.GetItemArtwork(model.KindAlbumArtwork, "alx", model.ImageTypePrimary)
return err == nil && ia.Hash != ""
}, 5*time.Second, 50*time.Millisecond).Should(BeTrue(),
"a blocked external pool must not hold up local artwork")
_, err := artRepo.GetItemArtwork(ctx, model.KindArtistArtwork, "arx0", model.ImageTypePrimary)
_, err := artRepo.GetItemArtwork(model.KindArtistArtwork, "arx0", model.ImageTypePrimary)
Expect(err).To(MatchError(model.ErrNotFound), "artists are still blocked, as intended")
})
})
@ -900,7 +849,7 @@ var _ = Describe("Worker", func() {
for i := range 8 {
id := fmt.Sprintf("alc%d", i)
albums = append(albums, model.Album{ID: id, Name: "Album"})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "al", ItemID: id, Priority: model.ArtworkPriorityScan,
})).To(Succeed())
}
@ -917,38 +866,10 @@ var _ = Describe("Worker", func() {
}
})
It("stops dispatching and leaves the rest queued when paused mid-batch", func() {
folderRepo.result = []model.Folder{{
Path: "tests/fixtures/artist/an-album",
ImageFiles: []string{"cover.jpg"},
}}
albums := model.Albums{}
for i := range 8 {
id := fmt.Sprintf("alp%d", i)
albums = append(albums, model.Album{ID: id, Name: "Album", FolderIDs: []string{"f1"}})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
ItemKind: "al", ItemID: id, Priority: model.ArtworkPriorityScan,
})).To(Succeed())
}
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(albums)
// Pauses as soon as the first item has left the queue.
w.PauseWhile(func() bool {
n, _ := queueRepo.Count(ctx)
return n < 8
})
_, err := w.drain(ctx, 1)
Expect(err).ToNot(HaveOccurred())
count, err := queueRepo.Count(ctx)
Expect(err).ToNot(HaveOccurred())
Expect(count).To(Equal(int64(7)), "only the item dispatched before the pause may leave the queue")
})
It("dequeues past the worker pool so one drain covers many items", func() {
for i := range 16 {
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{{ID: fmt.Sprintf("alb%d", i), Name: "Album"}})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
Expect(queueRepo.Enqueue(model.ArtworkQueueItem{
ItemKind: "al", ItemID: fmt.Sprintf("alb%d", i), Priority: model.ArtworkPriorityScan,
})).To(Succeed())
}
@ -975,30 +896,6 @@ var _ = Describe("Worker", func() {
Eventually(done, time.Second).Should(Receive(BeNil()))
})
It("does not drain the queue while paused", func() {
folderRepo.result = []model.Folder{{
Path: "tests/fixtures/artist/an-album",
ImageFiles: []string{"cover.jpg"},
}}
ds.MockedAlbum.(*tests.MockAlbumRepo).SetData(model.Albums{
{ID: "al1", Name: "Album", FolderIDs: []string{"f1"}},
})
Expect(queueRepo.Enqueue(ctx, model.ArtworkQueueItem{
ItemKind: "al", ItemID: "al1", Priority: model.ArtworkPriorityScan,
})).To(Succeed())
w.PauseWhile(func() bool { return true })
runCtx, cancel := context.WithCancel(ctx)
done := make(chan error, 1)
go func() { done <- w.Run(runCtx) }()
DeferCleanup(func() {
cancel()
Eventually(done, 2*time.Second).Should(Receive(BeNil()))
})
Consistently(func() any { return findQueued(queueRepo, "al", "al1") }, 300*time.Millisecond).ShouldNot(BeNil())
})
It("does not leak goroutines after Run exits", func() {
DeferCleanup(configtest.SetupConfig())

View file

@ -48,7 +48,7 @@ func Init(ds model.DataStore) {
}
func loadOrCreateSecret(ctx context.Context, ds model.DataStore, key string) string {
secret, err := ds.Property().Get(ctx, key)
secret, err := ds.Property(ctx).Get(key)
if err != nil || secret == "" {
log.Info(ctx, "Creating new JWT secret", "key", key)
return createNewSecret(ctx, ds, key)
@ -154,9 +154,9 @@ func CheckClaims(c Claims, usr model.User, audience string) error {
}
func WithAdminUser(ctx context.Context, ds model.DataStore) context.Context {
u, err := ds.User().FindFirstAdmin(ctx)
u, err := ds.User(ctx).FindFirstAdmin()
if err != nil {
c, err := ds.User().CountAll(ctx)
c, err := ds.User(ctx).CountAll()
if c == 0 && err == nil {
log.Debug(ctx, "No admin user yet!", err)
} else {
@ -176,7 +176,7 @@ func createNewSecret(ctx context.Context, ds model.DataStore, key string) string
log.Error(ctx, "Could not encrypt JWT secret", err)
return secret
}
if err := ds.Property().Put(ctx, key, encSecret); err != nil {
if err := ds.Property(ctx).Put(key, encSecret); err != nil {
log.Error(ctx, "Could not save JWT secret in DB", err)
}
return secret

View file

@ -19,7 +19,7 @@ func userName(ctx context.Context) string {
// BFR We should only access files through the `storage.Storage` interface. This will require changing how
// TagLib and ffmpeg access files
var AbsolutePath = func(ctx context.Context, ds model.DataStore, libId int, path string) string {
libPath, err := ds.Library().GetPath(ctx, libId)
libPath, err := ds.Library(ctx).GetPath(libId)
if err != nil {
return path
}

View file

@ -31,7 +31,7 @@ func (m *mockArtistRepo) SetData(artists model.Artists) {
}
// Get implements model.ArtistRepository.
func (m *mockArtistRepo) Get(_ context.Context, id string) (*model.Artist, error) {
func (m *mockArtistRepo) Get(id string) (*model.Artist, error) {
args := m.Called(id)
if args.Get(0) == nil {
return nil, args.Error(1)
@ -40,7 +40,7 @@ func (m *mockArtistRepo) Get(_ context.Context, id string) (*model.Artist, error
}
// GetAll implements model.ArtistRepository.
func (m *mockArtistRepo) GetAll(_ context.Context, options ...model.QueryOptions) (model.Artists, error) {
func (m *mockArtistRepo) GetAll(options ...model.QueryOptions) (model.Artists, error) {
argsSlice := make([]any, len(options))
for i, v := range options {
argsSlice[i] = v
@ -85,7 +85,7 @@ func (m *mockMediaFileRepo) SetData(mediaFiles model.MediaFiles) {
}
// Get implements model.MediaFileRepository.
func (m *mockMediaFileRepo) Get(ctx context.Context, id string) (*model.MediaFile, error) {
func (m *mockMediaFileRepo) Get(id string) (*model.MediaFile, error) {
args := m.Called(id)
if args.Get(0) == nil {
return nil, args.Error(1)
@ -94,12 +94,12 @@ func (m *mockMediaFileRepo) Get(ctx context.Context, id string) (*model.MediaFil
}
// GetAllByTags implements model.MediaFileRepository.
func (m *mockMediaFileRepo) GetAllByTags(ctx context.Context, _ model.TagName, _ []string, options ...model.QueryOptions) (model.MediaFiles, error) {
return m.GetAll(ctx, options...)
func (m *mockMediaFileRepo) GetAllByTags(_ model.TagName, _ []string, options ...model.QueryOptions) (model.MediaFiles, error) {
return m.GetAll(options...)
}
// GetAll implements model.MediaFileRepository.
func (m *mockMediaFileRepo) GetAll(ctx context.Context, options ...model.QueryOptions) (model.MediaFiles, error) {
func (m *mockMediaFileRepo) GetAll(options ...model.QueryOptions) (model.MediaFiles, error) {
argsSlice := make([]any, len(options))
for i, v := range options {
argsSlice[i] = v
@ -112,7 +112,7 @@ func (m *mockMediaFileRepo) GetAll(ctx context.Context, options ...model.QueryOp
}
// GetRandom implements model.MediaFileRepository.
func (m *mockMediaFileRepo) GetRandom(ctx context.Context, options ...model.QueryOptions) (model.MediaFiles, error) {
func (m *mockMediaFileRepo) GetRandom(options ...model.QueryOptions) (model.MediaFiles, error) {
argsSlice := make([]any, len(options))
for i, v := range options {
argsSlice[i] = v
@ -156,7 +156,7 @@ func newMockAlbumRepo() *mockAlbumRepo {
}
// Get implements model.AlbumRepository.
func (m *mockAlbumRepo) Get(_ context.Context, id string) (*model.Album, error) {
func (m *mockAlbumRepo) Get(id string) (*model.Album, error) {
args := m.Called(id)
if args.Get(0) == nil {
return nil, args.Error(1)
@ -165,7 +165,7 @@ func (m *mockAlbumRepo) Get(_ context.Context, id string) (*model.Album, error)
}
// GetAll implements model.AlbumRepository.
func (m *mockAlbumRepo) GetAll(_ context.Context, options ...model.QueryOptions) (model.Albums, error) {
func (m *mockAlbumRepo) GetAll(options ...model.QueryOptions) (model.Albums, error) {
argsSlice := make([]any, len(options))
for i, v := range options {
argsSlice[i] = v

View file

@ -182,7 +182,7 @@ func (e *provider) populateAlbumInfo(ctx context.Context, album auxAlbum) (auxAl
}
}
err = e.ds.Album().UpdateExternalInfo(ctx, &album.Album)
err = e.ds.Album(ctx).UpdateExternalInfo(&album.Album)
if err != nil {
log.Error(ctx, "Error trying to update album external information", "id", album.ID, "name", albumName,
"elapsed", time.Since(start), err)
@ -285,7 +285,7 @@ func (e *provider) populateArtistInfo(ctx context.Context, artist auxArtist) (au
if !throttled {
artist.ExternalInfoUpdatedAt = new(time.Now())
}
err := e.ds.Artist().UpdateExternalInfo(ctx, &artist.Artist)
err := e.ds.Artist(ctx).UpdateExternalInfo(&artist.Artist)
if err != nil {
log.Error(ctx, "Error trying to update artist external information", "id", artist.ID, "name", artistName,
"elapsed", time.Since(start), err)
@ -548,7 +548,7 @@ func (e *provider) loadArtistsByID(ctx context.Context, similar []agents.Artist)
if len(ids) == 0 {
return matches, nil
}
res, err := e.ds.Artist().GetAll(ctx, model.QueryOptions{
res, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"artist.id": ids},
})
if err != nil {
@ -577,7 +577,7 @@ func (e *provider) loadArtistsByMBID(ctx context.Context, similar []agents.Artis
if len(mbids) == 0 {
return matches, nil
}
res, err := e.ds.Artist().GetAll(ctx, model.QueryOptions{
res, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"mbz_artist_id": mbids},
})
if err != nil {
@ -612,7 +612,7 @@ func (e *provider) loadArtistsByName(ctx context.Context, similar []agents.Artis
clauses := slice.Map(names, func(name string) squirrel.Sqlizer {
return squirrel.Like{"artist.name": name}
})
res, err := e.ds.Artist().GetAll(ctx, model.QueryOptions{
res, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Or(clauses),
})
if err != nil {
@ -628,7 +628,7 @@ func (e *provider) loadArtistsByName(ctx context.Context, similar []agents.Artis
func (e *provider) findArtist(ctx context.Context, artistName, id string) (*auxArtist, error) {
if id != "" {
artist, err := e.ds.Artist().Get(ctx, id)
artist, err := e.ds.Artist(ctx).Get(id)
if err == nil {
return &auxArtist{Artist: *artist}, nil
}
@ -644,7 +644,7 @@ func (e *provider) findArtist(ctx context.Context, artistName, id string) (*auxA
return nil, model.ErrNotFound
}
artists, err := e.ds.Artist().GetAll(ctx, model.QueryOptions{
artists, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Like{"artist.name": artistName},
Max: 1,
})
@ -666,7 +666,7 @@ func (e *provider) loadSimilar(ctx context.Context, artist *auxArtist, count int
ids = append(ids, sa.ID)
}
similar, err := e.ds.Artist().GetAll(ctx, model.QueryOptions{
similar, err := e.ds.Artist(ctx).GetAll(model.QueryOptions{
Filters: squirrel.Eq{"artist.id": ids},
})
if err != nil {

View file

@ -71,8 +71,8 @@ var _ = Describe("Provider - RefreshInfo", func() {
ag = new(mockAgents)
broker = &fakeBroker{}
p = external.NewProvider(ds, ag, matcher.New(ds), broker)
mockArtistRepo = ds.Artist().(*tests.MockArtistRepo)
mockAlbumRepo = ds.Album().(*tests.MockAlbumRepo)
mockArtistRepo = ds.Artist(ctx).(*tests.MockArtistRepo)
mockAlbumRepo = ds.Album(ctx).(*tests.MockAlbumRepo)
})
It("repopulates an artist even when its info is fresh", func() {
@ -84,7 +84,7 @@ var _ = Describe("Provider - RefreshInfo", func() {
Expect(p.RefreshInfo(ctx, model.KindArtistArtwork, "ar-1")).To(Succeed())
saved, err := mockArtistRepo.Get(ctx, "ar-1")
saved, err := mockArtistRepo.Get("ar-1")
Expect(err).ToNot(HaveOccurred())
Expect(saved.Biography).To(Equal("Fresh Bio"))
})
@ -99,7 +99,7 @@ var _ = Describe("Provider - RefreshInfo", func() {
Expect(p.RefreshInfo(ctx, model.KindAlbumArtwork, "al-1")).To(Succeed())
saved, err := mockAlbumRepo.Get(ctx, "al-1")
saved, err := mockAlbumRepo.Get("al-1")
Expect(err).ToNot(HaveOccurred())
Expect(saved.Description).To(Equal("Fresh Notes"))
})

View file

@ -37,7 +37,7 @@ func (e *provider) SimilarSongs(ctx context.Context, id string, count int) (mode
if !errors.Is(err, model.ErrNotFound) {
return nil, err
}
genre, err := e.ds.Genre().Get(ctx, id)
genre, err := e.ds.Genre(ctx).Get(id)
if err != nil {
return nil, err
}
@ -178,13 +178,13 @@ func (e *provider) seedMix(ctx context.Context, count int, sample func() (model.
func (e *provider) samplePlaylistTracks(ctx context.Context, playlistID string, n int) (model.MediaFiles, error) {
// Refresh: a smart playlist materializes no tracks until it is evaluated, so skipping it would
// mix an empty seed set. It is a no-op for regular playlists and inside the refresh delay.
repo := e.ds.Playlist().Tracks(ctx, playlistID, true)
repo := e.ds.Playlist(ctx).Tracks(playlistID, true)
if repo == nil {
return nil, model.ErrNotFound
}
// A playlist can hold the same file at several positions, so over-fetch and dedup: a repeated
// seed wastes an agent call and can reach the mix twice through the seed fallback.
tracks, err := repo.GetAll(ctx, model.QueryOptions{
tracks, err := repo.GetAll(model.QueryOptions{
Sort: "random",
Max: n * 4,
Filters: squirrel.Eq{"missing": false},
@ -225,7 +225,7 @@ func (e *provider) sampleGenreTracks(ctx context.Context, genre *model.Genre, n
// sampleTracks returns up to n random present tracks. Seeds can end up in the mix verbatim, so
// missing files would surface as unplayable entries.
func (e *provider) sampleTracks(ctx context.Context, filter squirrel.Sqlizer, n int) (model.MediaFiles, error) {
return e.ds.MediaFile().GetRandom(ctx, model.QueryOptions{
return e.ds.MediaFile(ctx).GetRandom(model.QueryOptions{
Filters: squirrel.And{filter, squirrel.Eq{"missing": false}},
Max: n,
})

View file

@ -35,7 +35,7 @@ var _ = Describe("Provider - UpdateAlbumInfo", func() {
ds = new(tests.MockDataStore)
ag = new(mockAgents)
p = external.NewProvider(ds, ag, matcher.New(ds), &fakeBroker{})
mockAlbumRepo = ds.Album().(*tests.MockAlbumRepo)
mockAlbumRepo = ds.Album(ctx).(*tests.MockAlbumRepo)
conf.Server.DevAlbumInfoTimeToLive = 1 * time.Hour
})

View file

@ -38,7 +38,7 @@ var _ = Describe("Provider - UpdateArtistInfo", func() {
ds = new(tests.MockDataStore)
ag = new(mockAgents)
p = external.NewProvider(ds, ag, matcher.New(ds), &fakeBroker{})
mockArtistRepo = ds.Artist().(*tests.MockArtistRepo)
mockArtistRepo = ds.Artist(ctx).(*tests.MockArtistRepo)
})
It("returns error when artist is not found", func() {

View file

@ -49,6 +49,7 @@ type FFmpeg interface {
Transcode(ctx context.Context, opts TranscodeOptions) (io.ReadCloser, error)
ExtractImage(ctx context.Context, path string) (io.ReadCloser, error)
ConvertAnimatedImage(ctx context.Context, reader io.Reader, maxSize int, quality int) (io.ReadCloser, error)
Probe(ctx context.Context, files []string) (string, error)
ProbeAudioStream(ctx context.Context, filePath string) (*AudioProbeResult, error)
CmdPath() (string, error)
IsAvailable() bool
@ -67,6 +68,7 @@ var ErrAnimatedWebPUnsupported = errors.New("ffmpeg lacks libwebp_anim encoder
const (
extractImageCmd = "ffmpeg -i %s -map 0:v -map -0:V -vcodec copy -f image2pipe -"
probeCmd = "ffmpeg %s -f ffmetadata"
probeAudioStreamCmd = "ffprobe -v error -select_streams a:0 -print_format json -show_streams -show_format %s"
)
@ -147,6 +149,17 @@ func fileExists(path string) error {
return nil
}
func (e *ffmpeg) Probe(ctx context.Context, files []string) (string, error) {
if _, err := ffmpegCmd(); err != nil {
return "", err
}
args := createProbeCommand(probeCmd, files)
log.Trace(ctx, "Executing ffmpeg command", "args", args)
cmd := exec.CommandContext(ctx, args[0], args[1:]...) // #nosec
output, _ := cmd.CombinedOutput()
return string(output), nil
}
func (e *ffmpeg) ProbeAudioStream(ctx context.Context, filePath string) (*AudioProbeResult, error) {
if _, err := ffmpegCmd(); err != nil {
return nil, err
@ -580,6 +593,20 @@ func createFFmpegCommand(cmd, path string, maxBitRate, offset int) []string {
return args
}
func createProbeCommand(cmd string, inputs []string) []string {
var args []string
for _, s := range fixCmd(cmd) {
if s == "%s" {
for _, inp := range inputs {
args = append(args, "-i", inp)
}
} else {
args = append(args, s)
}
}
return args
}
func fixCmd(cmd string) []string {
split := strings.Fields(cmd)
cmdPath, _ := ffmpegCmd()

View file

@ -62,16 +62,23 @@ var _ = Describe("ffmpeg", func() {
})
})
Describe("createProbeCommand", func() {
It("creates a valid command line", func() {
args := createProbeCommand(probeCmd, []string{"/music library/one.mp3", "/music library/two.mp3"})
Expect(args).To(Equal([]string{"ffmpeg", "-i", "/music library/one.mp3", "-i", "/music library/two.mp3", "-f", "ffmetadata"}))
})
})
When("ffmpegPath is set", func() {
It("returns the correct ffmpeg path", func() {
ffmpegPath = "/usr/bin/ffmpeg"
args := createFFmpegCommand("ffmpeg -i %s -f mp3 -", "one.mp3", 0, 0)
Expect(args).To(Equal([]string{"/usr/bin/ffmpeg", "-i", "one.mp3", "-f", "mp3", "-"}))
args := createProbeCommand(probeCmd, []string{"one.mp3"})
Expect(args).To(Equal([]string{"/usr/bin/ffmpeg", "-i", "one.mp3", "-f", "ffmetadata"}))
})
It("returns the correct ffmpeg path with spaces", func() {
ffmpegPath = "/usr/bin/with spaces/ffmpeg.exe"
args := createFFmpegCommand("ffmpeg -i %s -f mp3 -", "one.mp3", 0, 0)
Expect(args).To(Equal([]string{"/usr/bin/with spaces/ffmpeg.exe", "-i", "one.mp3", "-f", "mp3", "-"}))
args := createProbeCommand(probeCmd, []string{"one.mp3"})
Expect(args).To(Equal([]string{"/usr/bin/with spaces/ffmpeg.exe", "-i", "one.mp3", "-f", "ffmetadata"}))
})
})

View file

@ -15,7 +15,7 @@ type InspectOutput struct {
MappedTags *model.MediaFile `json:"mappedTags,omitempty"`
}
func Inspect(filePath string, lib model.Library, folderId string) (*InspectOutput, error) {
func Inspect(filePath string, libraryId int, folderId string) (*InspectOutput, error) {
path, file := filepath.Split(filePath)
s, err := storage.For(path)
@ -39,22 +39,12 @@ func Inspect(filePath string, lib model.Library, folderId string) (*InspectOutpu
return nil, model.ErrNotFound
}
md := metadata.New(scannerPath(lib, filePath), tag)
md := metadata.New(path, tag)
result := &InspectOutput{
File: filePath,
RawTags: tags[file].Tags,
MappedTags: new(md.ToMediaFile(lib, folderId)),
MappedTags: new(md.ToMediaFile(libraryId, folderId)),
}
return result, nil
}
// scannerPath returns the path the scanner uses for the file (relative to its library), so
// folder-based PIDs match the DB. Files outside the library keep their absolute path.
func scannerPath(lib model.Library, filePath string) string {
absPath, err := filepath.Abs(filePath)
if err != nil || lib.Path == "" {
return filePath
}
return model.LibraryRelativePath(lib.Path, absPath)
}

Some files were not shown because too many files have changed in this diff Show more