remote/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2026-10-21 16:57:03 +02:00
Code Issues Projects Releases Packages Wiki Activity

Merge spike/schema-watcher into spike/integrated, reconciling the three spikes

Browse source 
Read path: the leased pool replaces the thread-local cache. Pooled
connections record Database._conn_generation when opened; a stale one is
discarded on checkout or return, and _invalidate_connections() (called
by the SchemaWatcher on a replaced/deleted file) closes that database's
idle connections at once. Leased ones are only ever closed by their own
thread on release.

Write path: the idle-exit write thread keeps its lock protocol; its
connection lives in a _WriteConnection holder that is reopened when the
generation moves, before the next task. Post-write schema_version check
and _after_write catalog await are kept for every write API in both
modes. Non-threaded write connections are reopened on a generation change
and never cached half-prepared.

Deleted files: write connections use mode=rw once Datasette has seen the
file exist (_file_seen), so a deleted file is never silently recreated,
while a database added before its file exists is still created by its
first write.

Settings: max_open_connections (clamped to >= 4 x num_sql_threads),
connection_idle_timeout_ms (30000, read connections and write threads),
schema_watch_interval_ms, default_schema_watch (external|owned). The
spike-only knobs (connection_idle_timeout, connection_pool_wait_ms,
pool_read_connections, write_thread_idle_timeout_ms) are gone, along with
the pool's wait/disable code paths. The reaper falls back to checkout-time
reaping where threads cannot be started (Pyodide).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
This commit is contained in:
Claude 2026-10-04 04:49:04 +00:00
commit 990c8782db
No known key found for this signature in database
17 changed files with 2508 additions and 440 deletions
Download patch file Download diff file Expand all files Collapse all files

24
docs/cli-reference.rst
View file

@ -252,18 +252,18 @@ These can be passed to ``datasette serve`` using ``datasette serve --setting nam
num_sql_threads Number of threads in the thread pool for
executing SQLite queries (default=3)
max_open_connections Maximum number of pooled read connections open
across all databases - least recently used idle
connections are closed first, 0 for no limit
(default=128)
connection_idle_timeout Close pooled read connections that have been idle
for this many seconds - 0 to keep them open
(default=60)
connection_pool_wait_ms When max_open_connections is reached and every
connection is in use, wait this long for one to
be returned before opening one more (default=0)
pool_read_connections Reuse read connections between queries - turn off
to open a new connection for every query
(default=True)
across all databases (at least 4 x
num_sql_threads) - 0 for no limit (default=128)
connection_idle_timeout_ms Close read connections, and stop write threads,
that have been idle for this many milliseconds -
0 to keep them open (default=30000)
schema_watch_interval_ms How often to check external database files for
schema changes - 0 to disable polling
(default=1000)
default_schema_watch Schema watch mode for databases opened from
files: external (poll for changes made by other
processes) or owned (only Datasette changes them)
(default=external)
sql_time_limit_ms Time limit for a SQL query in milliseconds
(default=1000)
default_facet_size Number of values to return for requested facets