remote/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2026-11-12 13:18:30 +01:00
Code Issues Projects Releases Packages Wiki Activity

Spike: leased read-connection pool with global cap and idle reaper

Browse source 
Replace the per-thread cached read connections with a pool. Each
execute_fn()/execute() callback checks a connection out for its duration
and gets a LeasedConnection proxy that raises ConnectionLeaseError if
used after the callback returns.

New settings: max_open_connections (global LRU cap, default 128),
connection_idle_timeout (seconds, default 60, enforced by one
process-wide reaper thread plus opportunistic reaping on checkout),
connection_pool_wait_ms (soft cap: wait then exceed) and
pool_read_connections (off = fresh connection per query, a control).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
This commit is contained in:
Claude 2026-10-04 01:26:01 +00:00
commit 791ce7a3d9
No known key found for this signature in database
9 changed files with 900 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

10
tests/test_internals_database.py
View file

@ -13,6 +13,7 @@ from opentelemetry import context as otel_context_api
from datasette.app import Datasette
from datasette.database import (
ConnectionLeaseError,
Database,
DatasetteClosedError,
ExecuteWriteResult,
@ -1312,8 +1313,15 @@ async def test_database_close_is_idempotent(tmpdir):
async def test_close_releases_memory_connections(num_sql_threads, named):
ds = Datasette(memory=True, settings={"num_sql_threads": num_sql_threads})
db = ds.add_memory_database(uuid.uuid4().hex) if named else ds.get_database()
read_connection = await db.execute_fn(lambda conn: conn)
leased = await db.execute_fn(lambda conn: conn)
# Read connections are leased: the proxy is unusable once the callback
# returns, so reach the underlying connection through a cursor
read_connection = await db.execute_fn(
lambda conn: conn.execute("select 1").connection
)
write_connection = await db.execute_write_fn(lambda conn: conn)
with pytest.raises(ConnectionLeaseError):
leased.execute("select 1")
ds.close()
for conn in (read_connection, write_connection):
with pytest.raises(sqlite3.ProgrammingError, match="closed"):