mirror of
https://github.com/navidrome/navidrome.git
synced 2026-11-14 14:18:42 +01:00
* perf(db): keep query planner statistics trustworthy with full ANALYZE PRAGMA optimize's internal ANALYZE runs with a limited analysis budget (~2000 rows) that writes wrong sqlite_stat1 entries for low-cardinality indexes: on a 96K-track library it claimed (missing, library_id) narrows to ~2000 rows when it matches the whole table. The planner then prefers that index over the sort index and falls back to a full-table temp B-tree sort per request, turning paginated song listings into multi-second queries (reproduced at 5.5s on real hardware; ~90x slower than with correct stats). Every index-creating migration re-triggered the poisoning via the post-migration optimize, and the daily optimizer could re-trigger it on large library changes. Setting analysis_limit on the connection does not help: optimize ignores it. Run a plain full ANALYZE instead: after migrations with schema changes, and in db.Optimize (daily schedule and scan-end). Stats are stored in the database file, so one connection suffices and the per-connection pool loop is gone. The Optimize call at shutdown is removed: stats are maintained at migration/scan/daily points, and an ANALYZE during shutdown only delays it and races container stop timeouts. * perf(db): drop startup PRAGMA optimize that re-poisons planner stats The startup PRAGMA optimize=0x10002 runs SQLite's budget-limited internal ANALYZE (bit 0x02), which writes truncated sqlite_stat1 rows for low-cardinality indexes -- the exact statistics-poisoning this PR set out to eliminate. Because DevOptimizeDB defaults to true, a restart with no pending migrations would re-poison the planner until the next scan or daily Optimize. Remove it: statistics are already refreshed with a full ANALYZE after schema-changing migrations (Init) and via Optimize at scan-end and on the daily schedule, so nothing on the startup path needs to touch them. Also clarify that Optimize is a no-op unless DevOptimizeDB is enabled. * chore(db): remove the DevOptimizeDB flag and skip Optimize on quick scans The flag only gated the optimize/ANALYZE maintenance calls and there is no reason to leave planner statistics unmaintained; the guards are gone along with the flag. The scan-end Optimize now runs only after full scans — quick scans barely move the statistics, and the daily schedule covers drift. * style(scanner): drop redundant comment in runOptimize * chore(persistence): drop the no-op PRAGMA optimize from ScanEnd Mask 0x10000 only selects candidate tables by size change; without the 0x02 action bit optimize does nothing (verified: sqlite_stat1 stays stale after a 100x table growth). The scan-end statistics refresh is db.Optimize's full ANALYZE, and the expression-collation-index concern the old comment guarded against no longer applies. * fix(scanner): run the post-scan ANALYZE in the server process With the external scanner (the default), the scan pipeline runs in a subprocess, so its ANALYZE was invisible to the server: SQLite loads sqlite_stat1 into the process's shared schema cache, and an ANALYZE from another process does not refresh it — verified with the production DSN that even brand-new pool connections keep planning with the old statistics until the server restarts. An in-process ANALYZE, by contrast, is immediately visible to every pooled connection through the same shared cache. Move the full-scan Optimize from the scanner pipeline to the scan controller, which always runs in the server process. * fix(scanner): honor promoted full scans in the optimize gate A quick scan resuming an interrupted full scan is promoted inside the scanner (possibly in a subprocess); mirror the promotion in the controller so the post-scan ANALYZE isn't skipped. * refactor: apply cleanup review findings - drop forceFullRescan's inline ANALYZE: Init already runs a full ANALYZE after any migration batch with schema changes, so upgrades including a full-rescan migration analyzed the whole DB twice - resumingFullScan uses a filtered CountAll instead of fetching and scanning all libraries - document why CallScan (CLI) deliberately skips the post-scan Optimize * perf(db): make planner analysis maintenance resilient Check analysis freshness every 30 minutes and refresh statistics when the last successful run is over 24 hours old or a scan marked them pending. Persist successful analysis state, retry skipped or failed maintenance, coordinate checks with scans, and cover standalone CLI full scans. * perf(db): avoid analyzing routine quick-scan changes Reserve pending analysis for full scans, unscanned libraries, and retry state. Incremental quick scans now rely on the 24-hour freshness window instead of triggering a full ANALYZE at the next maintenance check. * fix(scan): analyze resumed full scans in CLI * fix(db): back off failed analysis retries * feat(db): allow disabling scheduled analysis * test(db): remove redundant analysis coverage * refactor(db): split ANALYZE maintenance into optimize.go and dedupe call sites - move query-planner statistics code from db.go to its own optimize.go (and matching optimize_test.go) - log ANALYZE elapsed time inside Optimize/OptimizeIfNeeded instead of repeating the timing block at every call site - drop the LastDBAnalyzeAttemptAt write on success: it is only read while failures >= 1, and every failure rewrites it first - extract runPostScanAnalysis (cmd) and anyIncludedLibrary (scanner) helpers
212 lines
7.3 KiB
Go
212 lines
7.3 KiB
Go
package consts
|
|
|
|
import (
|
|
"os"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/navidrome/navidrome/model/id"
|
|
)
|
|
|
|
const (
|
|
AppName = "navidrome"
|
|
|
|
DefaultDbPath = "navidrome.db?cache=shared&_busy_timeout=15000&_journal_mode=WAL&_foreign_keys=on&synchronous=normal"
|
|
InitialSetupFlagKey = "InitialSetup"
|
|
FullScanAfterMigrationFlagKey = "FullScanAfterMigration"
|
|
// PlaylistsImportPendingFlagKey marks that playlist import was deferred because
|
|
// no admin user existed yet; the next scan with an admin imports them.
|
|
PlaylistsImportPendingFlagKey = "PlaylistsImportPending"
|
|
LastScanErrorKey = "LastScanError"
|
|
LastScanTypeKey = "LastScanType"
|
|
LastScanStartTimeKey = "LastScanStartTime"
|
|
LastDBAnalyzeAtKey = "LastDBAnalyzeAt"
|
|
LastDBAnalyzeAttemptAtKey = "LastDBAnalyzeAttemptAt"
|
|
DBAnalyzePendingKey = "DBAnalyzePending"
|
|
DBAnalyzeFailureCountKey = "DBAnalyzeFailureCount"
|
|
|
|
UIAuthorizationHeader = "X-ND-Authorization"
|
|
UIClientUniqueIDHeader = "X-ND-Client-Unique-Id"
|
|
JWTSecretKey = "JWTSecret"
|
|
JWTIssuer = "ND"
|
|
DefaultSessionTimeout = 48 * time.Hour
|
|
CookieExpiry = 365 * 24 * 3600 // One year
|
|
|
|
DBAnalyzeCheckSchedule = "@every 30m"
|
|
DBAnalyzeMaxAge = 24 * time.Hour
|
|
|
|
// DefaultEncryptionKey This is the encryption key used if none is specified in the `PasswordEncryptionKey` option
|
|
// Never ever change this! Or it will break all Navidrome installations that don't set the config option
|
|
DefaultEncryptionKey = "just for obfuscation"
|
|
PasswordsEncryptedKey = "PasswordsEncryptedKey"
|
|
PasswordAutogenPrefix = "__NAVIDROME_AUTOGEN__" //nolint:gosec
|
|
|
|
DevInitialUserName = "admin"
|
|
DevInitialName = "Dev Admin"
|
|
|
|
URLPathUI = "/app"
|
|
URLPathNativeAPI = "/api"
|
|
URLPathSubsonicAPI = "/rest"
|
|
URLPathPublic = "/share"
|
|
URLPathPublicImages = URLPathPublic + "/img"
|
|
|
|
// DefaultUILoginBackgroundURL uses Navidrome curated background images collection,
|
|
// available at https://unsplash.com/collections/20072696/navidrome
|
|
DefaultUILoginBackgroundURL = "/backgrounds"
|
|
|
|
// DefaultUILoginBackgroundOffline Background image used in case external integrations are disabled
|
|
DefaultUILoginBackgroundOffline = "iVBORw0KGgoAAAANSUhEUgAAAMgAAADICAIAAAAiOjnJAAAABGdBTUEAALGPC/xhBQAAAiJJREFUeF7t0IEAAAAAw6D5Ux/khVBhwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDBgwIABAwYMGDDwMDDVlwABBWcSrQAAAABJRU5ErkJggg=="
|
|
DefaultUILoginBackgroundURLOffline = "data:image/png;base64," + DefaultUILoginBackgroundOffline
|
|
DefaultMaxSidebarPlaylists = 100
|
|
|
|
RequestThrottleBacklogLimit = 100
|
|
RequestThrottleBacklogTimeout = time.Minute
|
|
|
|
ServerReadHeaderTimeout = 3 * time.Second
|
|
|
|
DefaultInfoLanguage = "en"
|
|
|
|
ArtistInfoTimeToLive = 24 * time.Hour
|
|
AlbumInfoTimeToLive = 7 * 24 * time.Hour
|
|
UpdateLastAccessFrequency = time.Minute
|
|
UpdatePlayerFrequency = time.Minute
|
|
|
|
I18nFolder = "i18n"
|
|
ScanIgnoreFile = ".ndignore"
|
|
ArtworkFolder = "artwork"
|
|
|
|
PlaceholderArtistArt = "artist-placeholder.webp"
|
|
PlaceholderAlbumArt = "album-placeholder.webp"
|
|
PlaceholderAvatar = "logo-192x192.png"
|
|
DefaultUIVolume = 100
|
|
DefaultUISearchDebounceMs = 200
|
|
DefaultUIPlaybackReportInterval = time.Minute
|
|
|
|
DefaultHttpClientTimeOut = 10 * time.Second
|
|
|
|
DefaultListenBrainzBaseURL = "https://api.listenbrainz.org/1/"
|
|
DefaultListenBrainzArtistAlgorithm = "session_based_days_9000_session_300_contribution_5_threshold_15_limit_50_skip_30"
|
|
DefaultListenBrainzTrackAlgorithm = "session_based_days_9000_session_300_contribution_5_threshold_15_limit_50_skip_30"
|
|
|
|
DefaultScannerExtractor = "taglib"
|
|
DefaultWatcherWait = 5 * time.Second
|
|
Zwsp = string('\u200b')
|
|
)
|
|
|
|
const (
|
|
DefaultUICoverArtSize = 300
|
|
DefaultMaxImageUploadSize = "10MB"
|
|
)
|
|
|
|
// Prometheus options
|
|
const (
|
|
PrometheusDefaultPath = "/metrics"
|
|
PrometheusAuthUser = "navidrome"
|
|
)
|
|
|
|
// Cache options
|
|
const (
|
|
TranscodingCacheDir = "transcoding"
|
|
DefaultTranscodingCacheMaxItems = 0 // Unlimited
|
|
|
|
ImageCacheDir = "images"
|
|
DefaultImageCacheMaxItems = 0 // Unlimited
|
|
|
|
DefaultCacheSize = 100 * 1024 * 1024 // 100MB
|
|
DefaultCacheCleanUpInterval = 10 * time.Minute
|
|
)
|
|
|
|
// Entity types
|
|
const (
|
|
EntityArtist = "artist"
|
|
EntityPlaylist = "playlist"
|
|
EntityRadio = "radio"
|
|
)
|
|
|
|
const (
|
|
AlbumPlayCountModeAbsolute = "absolute"
|
|
AlbumPlayCountModeNormalized = "normalized"
|
|
)
|
|
|
|
const (
|
|
//DefaultAlbumPID = "album_legacy"
|
|
DefaultAlbumPID = "musicbrainz_albumid|albumartistid,album,albumversion,releasedate"
|
|
DefaultTrackPID = "musicbrainz_trackid|albumid,discnumber,tracknumber,title"
|
|
PIDAlbumKey = "PIDAlbum"
|
|
PIDTrackKey = "PIDTrack"
|
|
)
|
|
|
|
const (
|
|
InsightsIDKey = "InsightsID"
|
|
InsightsEndpoint = "https://insights.navidrome.org/collect"
|
|
InsightsUpdateInterval = 24 * time.Hour
|
|
InsightsInitialDelay = 30 * time.Minute
|
|
)
|
|
|
|
const (
|
|
PurgeMissingNever = "never"
|
|
PurgeMissingAlways = "always"
|
|
PurgeMissingFull = "full"
|
|
)
|
|
|
|
var (
|
|
DefaultDownsamplingFormat = "opus"
|
|
DefaultTranscodings = []struct {
|
|
Name string
|
|
TargetFormat string
|
|
DefaultBitRate int
|
|
Command string
|
|
}{
|
|
{
|
|
Name: "mp3 audio",
|
|
TargetFormat: "mp3",
|
|
DefaultBitRate: 192,
|
|
Command: "ffmpeg -ss %t -i %s -map 0:a:0 -map_metadata 0 -map_metadata 0:s:a:0 -b:a %bk -v 0 -f mp3 -",
|
|
},
|
|
{
|
|
Name: "opus audio",
|
|
TargetFormat: "opus",
|
|
DefaultBitRate: 128,
|
|
Command: "ffmpeg -ss %t -i %s -map 0:a:0 -map_metadata 0 -map_metadata 0:s:a:0 -b:a %bk -v 0 -c:a libopus -f opus -",
|
|
},
|
|
{
|
|
Name: "aac audio",
|
|
TargetFormat: "aac",
|
|
DefaultBitRate: 256,
|
|
Command: "ffmpeg -ss %t -i %s -map 0:a:0 -map_metadata 0 -map_metadata 0:s:a:0 -b:a %bk -v 0 -c:a aac -f adts -",
|
|
},
|
|
{
|
|
Name: "flac audio",
|
|
TargetFormat: "flac",
|
|
DefaultBitRate: 0,
|
|
Command: "ffmpeg -ss %t -i %s -map 0:a:0 -map_metadata 0 -map_metadata 0:s:a:0 -v 0 -c:a flac -f flac -",
|
|
},
|
|
}
|
|
)
|
|
|
|
var HTTPUserAgent = "Navidrome" + "/" + Version
|
|
|
|
var (
|
|
VariousArtists = "Various Artists"
|
|
// TODO This will be dynamic when using disambiguation
|
|
VariousArtistsID = "63sqASlAfjbGMuLP4JhnZU"
|
|
UnknownAlbum = "[Unknown Album]"
|
|
UnknownArtist = "[Unknown Artist]"
|
|
// TODO This will be dynamic when using disambiguation
|
|
UnknownArtistID = id.NewHash(strings.ToLower(UnknownArtist))
|
|
VariousArtistsMbzId = "89ad4ac3-39f7-470e-963a-56509c546377"
|
|
|
|
ArtistJoiner = " • "
|
|
)
|
|
|
|
var (
|
|
ServerStart = time.Now()
|
|
|
|
InContainer = func() bool {
|
|
// Check if the /.nddockerenv file exists
|
|
if _, err := os.Stat("/.nddockerenv"); err == nil {
|
|
return true
|
|
}
|
|
return false
|
|
}()
|
|
)
|