Read path: the leased pool replaces the thread-local cache. Pooled
connections record Database._conn_generation when opened; a stale one is
discarded on checkout or return, and _invalidate_connections() (called
by the SchemaWatcher on a replaced/deleted file) closes that database's
idle connections at once. Leased ones are only ever closed by their own
thread on release.
Write path: the idle-exit write thread keeps its lock protocol; its
connection lives in a _WriteConnection holder that is reopened when the
generation moves, before the next task. Post-write schema_version check
and _after_write catalog await are kept for every write API in both
modes. Non-threaded write connections are reopened on a generation change
and never cached half-prepared.
Deleted files: write connections use mode=rw once Datasette has seen the
file exist (_file_seen), so a deleted file is never silently recreated,
while a database added before its file exists is still created by its
first write.
Settings: max_open_connections (clamped to >= 4 x num_sql_threads),
connection_idle_timeout_ms (30000, read connections and write threads),
schema_watch_interval_ms, default_schema_watch (external|owned). The
spike-only knobs (connection_idle_timeout, connection_pool_wait_ms,
pool_read_connections, write_thread_idle_timeout_ms) are gone, along with
the pool's wait/disable code paths. The reaper falls back to checkout-time
reaping where threads cannot be started (Pyodide).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
Replace the per-thread cached read connections with a pool. Each
execute_fn()/execute() callback checks a connection out for its duration
and gets a LeasedConnection proxy that raises ConnectionLeaseError if
used after the callback returns.
New settings: max_open_connections (global LRU cap, default 128),
connection_idle_timeout (seconds, default 60, enforced by one
process-wide reaper thread plus opportunistic reaping on checkout),
connection_pool_wait_ms (soft cap: wait then exceed) and
pool_read_connections (off = fresh connection per query, a control).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
This change updates the `-s/--setting` option to `datasette serve` to allow it to be used to set arbitrarily complex nested settings in a way that is compatible with the new `-c datasette.yml` work happening in:
- #2143
It will enable things like this:
```
datasette data.db --setting plugins.datasette-ripgrep.path "/home/simon/code"
```
For the moment though it just affects [settings](https://docs.datasette.io/en/1.0a4/settings.html) - so you can do this:
```
datasette data.db --setting settings.sql_time_limit_ms 3500
```
I've also implemented a backwards compatibility mechanism, so if you use it this way (the old way):
```
datasette data.db --setting sql_time_limit_ms 3500
```
It will notice that the setting you passed is one of Datasette's core settings, and will treat that as if you said `settings.sql_time_limit_ms` instead.
