mirror of
https://github.com/simonw/datasette.git
synced 2026-11-22 09:59:38 +01:00
Torture harness, and the fixes it found
tests/connection_torture.py drives one Datasette from several event loops (threads) with the smallest pool, 1ms idle timeout and 10ms schema polling through every connection operation, plus add/remove_database, scratch databases, external schema changes, replaced/deleted files, cancellation, Database.close()/Datasette.close() mid-flight and non-threaded mode. One scenario per process, so crashes and hangs are results. It checks for hangs, crashes, undocumented exceptions, per-database write order, lost writes, read-your-writes, catalog freshness, and fds/threads/the instance released after close(). tests/test_connection_torture.py runs a short fixed-seed subset of every scenario. Fixes for what it found: - A cursor that left the thread owning its connection (the cursor returned by execute_write_many()/execute_write_script(), one returned by a write callback, one rejected by execute_fn(), or one held by an exception's frames) could be freed on the event loop thread while the connection ran the same SQL again; on Python < 3.12 freeing it resets the shared cached statement, and the other query failed with "bad parameter or other API misuse". Such cursors are now closed on the owning thread. Pre-existing on main (51 failures per 1,800 multi-loop execute_write_many calls). - invoke_startup() is single-flight across event loops (a concurrent.futures future, datasette.utils.inflight) instead of _startup_sequence() holding an asyncio.Lock: a second loop got "bound to a different event loop" or hung, and two loops could run the startup hooks twice. A request made from a startup hook no longer re-runs or deadlocks startup. - The index page takes one snapshot of ds.databases: a database removed, renamed or closed mid-request, or whose file was deleted, no longer turns the page into a 500. Database.size is 0 for a deleted file. The scan-coalescing helpers move to datasette/utils/inflight.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
This commit is contained in:
parent
d7a2ce91cc
commit
da523c30e5
11 changed files with 1763 additions and 106 deletions
|
|
@ -625,7 +625,9 @@ async def test_default_schema_watch_setting(tmp_path, db_path, setting, expected
|
|||
Database(ds, path=other), name="polled", schema_watch="external"
|
||||
)
|
||||
assert polled._watch_state.mode == "external"
|
||||
owned = ds.add_database(Database(ds, path=other), name="owned", schema_watch="owned")
|
||||
owned = ds.add_database(
|
||||
Database(ds, path=other), name="owned", schema_watch="owned"
|
||||
)
|
||||
assert owned._watch_state.mode == "owned"
|
||||
ds.close()
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue