mirror of
https://github.com/simonw/datasette.git
synced 2026-10-21 00:36:59 +02:00
Connection budget: pages, startup and sweeps only touch the databases they need
Adds tests/connection_budget.py, a helper that records which databases a block of code opens, reads, writes or ATTACHes, and tests/test_connection_budget.py which asserts budgets for ~60 endpoints, startup, sweeps and telemetry with 50 attached databases. Fixes for every all-database loop it found: - Index page: tables, views, hidden tables, columns, primary keys, FTS tables and foreign-key relationship counts come from the _internal catalog (a few queries for any number of databases). Row counts are only run when at most 20 databases need counting (COUNT_MAX_DATABASES); above that the page shows table counts plus counts already known. - Derived-table permissions: allowed_resources() no longer queries every database per page; dependency maps are computed by the SchemaWatcher from the rows it scans, cached per database, and read back from the catalog for databases restored from a persisted catalog. - catalog_tables gains a type column (PRAGMA table_list: table, virtual, shadow) so hidden shadow tables can be identified without a connection. - PaginatedResources.all() fetches pages of 1000 after the first. - /-/api and /-/debug/autocomplete use the catalog; /-/schema does one permission query and skips databases with nothing visible. - /-/versions computes SQLite details once per instance, without crossdb ATTACHes; detect_json1() caches its answer. - /-/databases only stats/hashes databases the actor can see. - Immutable table counts are computed when first needed, not at startup. - check_databases() (serve, --get) reuses the catalog scan instead of opening every database a second time, and reports "file is not a database" as a usage error. - SchemaWatcher: failed scans are retried when the file changes, not on every sweep; scan connections skip crossdb ATTACH; catalog_generation counter for caches. - The SQL write UI no longer forces a stat sweep of every database. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwkLo2rJWidpkDVT3HkvNv
This commit is contained in:
parent
87ba82de92
commit
1bc15caef7
17 changed files with 1768 additions and 232 deletions
|
|
@ -2876,7 +2876,7 @@ The internal database schema is as follows:
|
|||
database_name TEXT,
|
||||
table_name TEXT,
|
||||
rootpage INTEGER,
|
||||
sql TEXT,
|
||||
sql TEXT, type TEXT,
|
||||
PRIMARY KEY (database_name, table_name),
|
||||
FOREIGN KEY (database_name) REFERENCES catalog_databases(database_name)
|
||||
);
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue