mirror of
https://github.com/navidrome/navidrome.git
synced 2026-10-31 07:27:58 +01:00
refactor(db): migrate all ids to a uniform canonical 128-bit base62 encoding (#5824)
* refactor(model): extract canonical 128-bit base62 id codec * feat(model): generate random ids as canonical 128-bit base62 values * feat(scanner): emit legacy PIDs in canonical base62 encoding * feat(db): add id canonicalization transform for the uniform-ids migration * feat(db): migrate all ids to canonical 128-bit base62 encoding * fix(db): canonicalize ids in junction tables and JSON columns * chore(jellyfin): update id-family notes for uniform canonical ids * test(ids): harden codec input contract and migration edge coverage * refactor(model): use log.Fatal for Encode128 contract guard per project convention * fix(db): force full rescan after id migration for legacy PID configs * test(db): guard id-column inventory against schema drift * refactor(ids): compile-time Encode128 contract and unified column rewrite helper * refactor(db): apply review feedback to id migration Filter empty strings in collectColumn's SQL, reuse a prepared statement for rewriteColumn updates, and clarify the legacy ID functions' comment now that they emit the canonical encoding. * feat(auth): split session and public-link JWT secrets, rotating sessions on id migration * test(subsonic): initialize public token secret in helpers suite The suite sets auth.TokenAuth directly instead of calling auth.Init, so the new PublicTokenAuth was nil whenever Ginkgo's spec order ran a helpers spec before any spec that calls auth.Init, panicking in publicurl.ImageURL. * refactor(db): inline canonicalID into its only consumer, the uniform-ids migration * refactor(model): rename Encode128/Decode128 to Encode/Decode With every id now exactly 128 bits, the width suffix is redundant; the package-qualified id.Encode/id.Decode carries the same information. * test(db): make the id-columns guard classify JSON columns too The guard only inspected columns named id/pid/*_id, so it could not see ids embedded in JSON. Widen it to *_ids and to every JSON column, and drive the "covered" set from a new embeddedIDColumns list instead of the inline calls in the migration. Every JSON column the schema has now carries a verdict. The four denormalized caches -- media_file/album.participants, media_file/album.tags, album.folder_ids and artist.similar_artists -- hold only artist, tag and folder ids. Those all come from id.NewHash, whose 22-char base62 encoding of a 128-bit MD5 is already in canonical range, so canonicalID is the identity on them and the migration correctly leaves them alone. A new codec test pins that invariant, since the exemptions depend on it. Verified on a copy of a 727MB/96k-track production database: canonicalizing those four columns changed zero rows, and artist, tag and folder ids were themselves unchanged by the migration (only media_file ids moved, 95108 of 96666).
This commit is contained in:
parent
b40b41584a
commit
f853ca604a
25 changed files with 957 additions and 59 deletions
|
|
@ -90,9 +90,9 @@ skipped, so it doesn't create a nameless player.
|
|||
Navidrome item ids are **hex-encoded at the API boundary** (`dto.EncodeID`/`DecodeID`): every id
|
||||
is hex-encoded on the way out and hex-decoded on the way in. This is required because some clients
|
||||
parse ids as radix-16 — Finamp's queue `packIds`, for instance, does `int.parse(chunk, radix:16)`,
|
||||
which chokes on Navidrome's base-62 nanoids (e.g. `5QFKvMsJrd57QE2Le2dKKo`). Because a raw MD5 id
|
||||
from an old migrated library is itself valid hex, correctness depends on every emit path encoding
|
||||
and every receive path decoding — see `dto/ids.go`.
|
||||
which chokes on Navidrome's base62 ids (e.g. `5QFKvMsJrd57QE2Le2dKKo`). Because a base62 id can
|
||||
itself be valid hex, correctness depends on every emit path encoding and every receive path
|
||||
decoding — see `dto/ids.go`.
|
||||
|
||||
## Multi-library behavior
|
||||
|
||||
|
|
@ -184,13 +184,13 @@ their Navidrome `ArtworkID`.
|
|||
|
||||
Real Jellyfin item ids are GUIDs — 128-bit values, always 32 hex characters. Finamp relies on that
|
||||
when persisting its play queue across restarts: `packIds()` bit-packs every id into exactly 16
|
||||
bytes. Navidrome ids are longer (nanoid ids can exceed 128 bits, so they cannot be mapped into
|
||||
GUIDs), which means Finamp silently stores only the first 16 characters of each id and asks for
|
||||
those **truncated ids** back when restoring the queue — item lookups, then streaming, images,
|
||||
favorites and playback reports for the restored tracks.
|
||||
bytes. Navidrome ids are 22-character base62 strings, not 32-hex GUIDs, which means Finamp silently
|
||||
stores only the first 16 characters of each id and asks for those **truncated ids** back when
|
||||
restoring the queue — item lookups, then streaming, images, favorites and playback reports for the
|
||||
restored tracks.
|
||||
|
||||
This API compensates server-side (`truncated_ids.go`): a 16-character id — a length no Navidrome
|
||||
id family uses — is resolved to the full id by unique-prefix lookup (an indexed range scan;
|
||||
id uses — is resolved to the full id by unique-prefix lookup (an indexed range scan;
|
||||
ambiguity is detected and fails safe). The `/Items?ids=` batch response echoes the id **as
|
||||
requested**, because Finamp matches restored items back to its stored ids, and the other item
|
||||
endpoints accept truncated ids transparently.
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ package dto
|
|||
import "encoding/hex"
|
||||
|
||||
// EncodeID renders a Navidrome id as lowercase hex; Jellyfin clients parse ids as radix-16 (e.g.
|
||||
// Finamp's queue packing) and crash on Navidrome's base62 nanoids if emitted as-is.
|
||||
// Finamp's queue packing) and crash on Navidrome's base62 ids if emitted as-is.
|
||||
func EncodeID(id string) string {
|
||||
if id == "" {
|
||||
return ""
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@ import (
|
|||
)
|
||||
|
||||
var _ = Describe("id codec", func() {
|
||||
It("round-trips a base62 nanoid through Encode/Decode", func() {
|
||||
It("round-trips a base62 id", func() {
|
||||
id := "5QFKvMsJrd57QE2Le2dKKo"
|
||||
Expect(DecodeID(EncodeID(id))).To(Equal(id))
|
||||
})
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@ import (
|
|||
)
|
||||
|
||||
// truncatedIDLen is what Finamp's saved-queue persistence cuts item ids to (16 bytes, assuming
|
||||
// Jellyfin GUIDs). No Navidrome id family is 16 chars (nanoid=22, legacy MD5=32, playlist
|
||||
// UUID=36), so the length alone identifies a truncated id. See README.
|
||||
// Jellyfin GUIDs). All Navidrome ids are 22 chars (share ids 10), so length alone flags a
|
||||
// truncated id. See README.
|
||||
//
|
||||
// Handlers taking an item id resolve it via resolveItemID/resolveItemIDs; playlist-write handlers
|
||||
// and ParentId scoping don't (a restored queue never edits playlists or browses by container id).
|
||||
|
|
|
|||
|
|
@ -68,7 +68,7 @@ func (pub *Router) handleImages(w http.ResponseWriter, r *http.Request) {
|
|||
}
|
||||
|
||||
func decodeArtworkID(tokenString string) (model.ArtworkID, error) {
|
||||
token, err := auth.TokenAuth.Decode(tokenString)
|
||||
token, err := auth.PublicTokenAuth.Decode(tokenString)
|
||||
if err != nil {
|
||||
return model.ArtworkID{}, err
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ import (
|
|||
|
||||
var _ = Describe("decodeArtworkID", func() {
|
||||
BeforeEach(func() {
|
||||
auth.TokenAuth = jwtauth.New("HS256", []byte("super secret"), nil)
|
||||
auth.PublicTokenAuth = jwtauth.New("HS256", []byte("super secret"), nil)
|
||||
})
|
||||
|
||||
It("fails to decode an invalid token", func() {
|
||||
|
|
|
|||
|
|
@ -111,8 +111,8 @@ func (pub *Router) mapShareToM3U(r *http.Request, s model.Share) *model.Share {
|
|||
// admin flag) and grants access to nothing beyond the share it belongs to; the
|
||||
// stream handler still verifies the share exists, is unexpired, and that the
|
||||
// track is actually a member of it. An attacker who can forge these tokens
|
||||
// necessarily already holds the signing secret, which also signs real user
|
||||
// sessions, so that scenario is out of scope for the share boundary specifically.
|
||||
// necessarily already holds the public-link signing secret, a full-server
|
||||
// compromise that is out of scope for the share boundary specifically.
|
||||
func encodeMediafileShare(s model.Share, id string) string {
|
||||
claims := auth.Claims{
|
||||
ID: id,
|
||||
|
|
|
|||
|
|
@ -107,7 +107,7 @@ func shareContainsTrack(share *model.Share, mediaFileID string) bool {
|
|||
// public-share capability, not an auth credential; see encodeMediafileShare for
|
||||
// why a JWT is used here.
|
||||
func decodeStreamInfo(tokenString string) (shareTrackInfo, error) {
|
||||
c, err := auth.Validate(tokenString)
|
||||
c, err := auth.ValidatePublic(tokenString)
|
||||
if err != nil {
|
||||
return shareTrackInfo{}, err
|
||||
}
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ func (m *mockStreamer) NewStream(_ context.Context, _ *model.MediaFile, r stream
|
|||
|
||||
var _ = Describe("decodeStreamInfo", func() {
|
||||
BeforeEach(func() {
|
||||
auth.TokenAuth = jwtauth.New("HS256", []byte("test-secret"), nil)
|
||||
auth.PublicTokenAuth = jwtauth.New("HS256", []byte("test-secret"), nil)
|
||||
})
|
||||
|
||||
It("decodes a valid token with all fields", func() {
|
||||
|
|
@ -81,7 +81,7 @@ var _ = Describe("decodeStreamInfo", func() {
|
|||
|
||||
var _ = Describe("encodeMediafileShare", func() {
|
||||
BeforeEach(func() {
|
||||
auth.TokenAuth = jwtauth.New("HS256", []byte("test-secret"), nil)
|
||||
auth.PublicTokenAuth = jwtauth.New("HS256", []byte("test-secret"), nil)
|
||||
})
|
||||
|
||||
It("includes the share ID in the token", func() {
|
||||
|
|
@ -113,7 +113,7 @@ var _ = Describe("handleStream", func() {
|
|||
var pub *Router
|
||||
|
||||
BeforeEach(func() {
|
||||
auth.TokenAuth = jwtauth.New("HS256", []byte("test-secret"), nil)
|
||||
auth.PublicTokenAuth = jwtauth.New("HS256", []byte("test-secret"), nil)
|
||||
ds = &tests.MockDataStore{}
|
||||
shareRepo = &tests.MockShareRepo{}
|
||||
ds.MockedShare = shareRepo
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ var _ = Describe("helpers", func() {
|
|||
BeforeEach(func() {
|
||||
DeferCleanup(configtest.SetupConfig())
|
||||
auth.TokenAuth = jwtauth.New("HS256", []byte("test secret"), nil)
|
||||
auth.PublicTokenAuth = jwtauth.New("HS256", []byte("test public secret"), nil)
|
||||
})
|
||||
|
||||
Describe("fakePath", func() {
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue