Commit graph

10 commits

Author SHA1 Message Date
Deluan
0cf80707ac fix(pglite): do not resend Terminate, which never gets a reply 2026-09-04 23:40:40 -04:00
Deluan
f4effde18e fix(pglite): resend a request the backend produced no output for
Three times during a scan on a real library, a new pool connection's
first query got no bytes back at all, which the previous commit turned
from a hang into an error. A run always emits at least a completion, so
no output means the packet was not run; the bridge now hands it over
again (twice at most) and logs it.
2026-09-04 23:32:29 -04:00
Deluan
ba16ab4158 perf(pglite): artist stats as hash joins, one statement for a full refresh; bigint library size
The stats UPDATE now joins the materialized counters (with a LEFT JOIN
so artists without files still get '{}') instead of a correlated
subquery, and a full refresh runs as a single statement: 3.3 s for 30k
artists locally versus 30 batches. Batches of 1000 remain for partial
refreshes (0.24 s each).

library.total_size was integer; a 2 TB library overflowed it at the end
of the first scan. It is bigint now, matching the Go int64.

The bridge no longer synthesizes a reply for Terminate, which never gets
one by design.
2026-09-04 23:23:50 -04:00
Deluan
75e14f14d6 fix(pglite): never leave a client without ReadyForQuery
When the backend produced no bytes for a request, or a reply without
ReadyForQuery, the bridge forwarded it as-is and the client waited for
ever while the session was free. On roy this froze the UI: a pooled
connection hung right after its handshake and every request queued in
the pool behind it. Incomplete replies now get a synthesized error plus
ReadyForQuery, and the bridge logs them, so a glitch becomes a visible
SQL error that the pool can retry.
2026-09-04 23:17:48 -04:00
Deluan
e2c8a1930a fix(pglite): do not cut slow readers, and bind playlist rules as text
The bridge gave every reply write a 5 second deadline. Scanner phase 3
streams all touched albums through a cursor and runs queries per row on
other pooled connections, which queue on the single session, so a large
library could not drain the reply in time and the bridge closed the pipe
mid-message ('unexpected EOF'). Replies now block as long as the client
needs, and client connections are closed on shutdown instead.

Playlist rules were bound as []byte, which pgx sends as bytea; the text
column then stored the hex form and the JSON parser failed on it.
2026-09-04 22:25:13 -04:00
Deluan
a528de445d fix(pglite): stop leaking every wire message's memory in the wasm module
The PGlite fork stubs MemoryContextResetAndDeleteChildren, which
PostgreSQL 17 removed, as an empty macro, so MessageContext was never
reset and each statement leaked its parse and plan state (about 10 KB).
On a real library the process grew to 15 GB during the first scan and
the host swapped itself to a crawl. The new build maps the macro to
MemoryContextReset (message_context_reset.diff); memory now stays flat
across thousands of statements. Adds PGlite.MemorySize for measuring.
2026-09-04 21:38:21 -04:00
Deluan
99c9adc69e fix(pglite): give the initial migration a valid goose timestamp
The migration validator requires a 14-digit timestamp newer than the
newest migration on master; 00000000000001 fails both checks.
2026-09-04 20:45:22 -04:00
Deluan
1fd3261f7c build(pglite): add the embedded PGlite wasm module
pglite.wasi.gz is PostgreSQL 17.5 built for WASI at -O2 without the wizer
snapshot, gzipped from 14.6 MB to 4.9 MB. db/pglite/embed.go go:embed's
it, so the pipeline cannot build the binary without it in the tree. The
recipe that produces it is in db/pglite/build/README.md.
2026-09-04 20:43:19 -04:00
Deluan
65f6bc2526 feat(pglite): run Navidrome on the embedded PostgreSQL end to end
Replace the 131 SQLite migrations with a single goose migration that creates
the whole schema in PostgreSQL (the user table becomes user_account, json
columns become jsonb, seed rows for the default library and transcodings).
Port the repositories' SQL just far enough for Navidrome to start, scan,
log in and browse artists and albums: json_agg/jsonb_* for the SQLite JSON
functions, lower() for COLLATE NOCASE, ON CONFLICT DO NOTHING, coalesce on
annotations, stricter GROUP BY, explicit casts under the simple protocol,
sequential queries in library.RefreshStats. db/ is now Postgres-only, with
backup/restore stubbed and DevExternalScanner forced off.

Embed the PGlite module like go-taglib: pglite.wasi.gz is go:embed'ed,
decompressed in memory and compiled by wazero with an on-disk compilation
cache, so the module itself is never written to disk. The cluster is
created by initdb at runtime on the first start; initdb runs in a
throwaway wasm instance and the backend in a fresh one, otherwise the next
start cannot find a valid checkpoint. Since initdb creates only template1,
a short-lived backend then runs CREATE DATABASE navidrome and the real
backend starts on it. The build recipe and patches for the -O2, no-wizer
module live in db/pglite/build.
2026-09-04 20:41:52 -04:00
Deluan
98555ca499 feat(db): spike an embedded PostgreSQL via PGlite under wazero
Adds db/pglite, a bridge that runs the PGlite WASI build of PostgreSQL 17.5
inside the Navidrome process with wazero and exposes it to database/sql
through pgx over the real wire protocol. Selected with DbPath = "pglite://<dir>".

The bridge handles what the WASI build cannot do on its own: it emulates dup2
onto stdin (wazero refuses fd_renumber on preopens), replays ParameterStatus
for connections after the first, synthesizes a missing ReadyForQuery after an
error, recovers from the trap every PG ERROR raises, and passes wire bytes
through shared wasm memory instead of files (round-trip floor 377 µs to 39 µs).
Several connections are accepted and serialized onto the single backend,
holding the session across a whole transaction and a whole handshake; a client
that disconnects mid-transaction gets a ROLLBACK.

db.go opens the pglite:// scheme, skips the SQLite-only migrations, and can
apply a translated schema from ND_PGLITE_SCHEMA. The shared count() helper now
adds its ORDER BY only for SQLite, since Postgres rejects it next to
count(distinct ...).

The wasm archive is not committed. The README explains how to build it and
lists the limits found: one session, one process (DevExternalScanner must be
off), shared session state, simple protocol only.
2026-09-04 18:29:58 -04:00