remote/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2026-10-21 00:36:59 +02:00
Code Issues Projects Releases Packages Wiki Activity

Connection budget tests for scratch databases

Browse source 
- Startup with 1000 scratch databases (threaded and num_sql_threads=0):
  Datasette() opens none of them; _startup_sequence() with a temporary
  internal database opens each exactly once (its catalog scan) with no
  pooled reads, writes or ATTACH; the index page, /-/databases.json,
  /-/api, /-/allowed, /-/jump, /-/check, /-/permissions and idle sweeps
  open none.
- Restart with a persisted internal database opens no scratch database,
  even for one written after its last scan (closing fingerprints).
- check_databases() only opens each scratch database for the catalog scan
  and does not fail on a broken scratch file.
- create, write, read, list, rename and delete touch only the scratch
  database concerned (list touches nothing); direct sqlite3.connect()
  calls are confined to the scratch directory (registry, WAL setup,
  rename checkpoint).
- docs: serve does not check scratch databases; listing pages do not open
  them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
This commit is contained in:
Claude 2026-10-04 07:13:03 +00:00
commit a509b81f2c
No known key found for this signature in database
2 changed files with 235 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/internals.rst
View file

@ -1412,7 +1412,7 @@ Use a directory just for scratch databases. Only one Datasette instance can use
The directory contains one ``<name>.db`` file per scratch database, plus a registry (``.datasette-scratch.db``) that records each database's owner, metadata, creation time and when it was last used. Scratch databases are created in WAL mode (rollback journal mode with ``--nolock``).
**Startup.** Existing scratch databases are attached when Datasette starts without opening any of them: Datasette reads the registry and lists the directory. Like any database, a scratch database gets connections and a write thread only while it is being used, and they are closed again after :ref:`setting_connection_idle_timeout_ms`. With a persistent internal database (``--internal``) and a clean shutdown, the table catalog for scratch databases is restored at startup without opening them either. With the default temporary internal database each scratch database is opened briefly, once, at startup to read its schema - as are, after a crash, the ones that were written to since their schema was last read.
**Startup.** Existing scratch databases are attached when Datasette starts without opening any of them: Datasette reads the registry and lists the directory. Like any database, a scratch database gets connections and a write thread only while it is being used, and they are closed again after :ref:`setting_connection_idle_timeout_ms`. With a persistent internal database (``--internal``) and a clean shutdown, the table catalog for scratch databases is restored at startup without opening them either. With the default temporary internal database each scratch database is opened briefly, once, at startup to read its schema - as are, after a crash, the ones that were written to since their schema was last read. ``datasette serve`` does not run its startup check on scratch databases: a scratch database whose file cannot be read is reported in the log and does not stop the server from starting. Pages that list every database, such as the index page, read scratch databases' tables from the catalog and do not open them.
**The catalog** of a scratch database's tables is kept current by its own writes (schema watch mode ``"owned"``): a table created through any of the write methods is in the catalog before the write returns. If another process writes to a scratch database file directly, Datasette will not notice schema changes.