Spans created on a worker thread resolve their parent from that thread's
ambient context, so without this every span produced below Database came
back as an unparented root, disconnected from the request that caused it.
Carrying the caller's context across each boundary is also what makes the
thread-pool wait visible: db.query covers the full round trip, the new
db.query.execute covers only the work inside the worker, and the gap
between them is the queueing the old tracer folds invisibly into one
number.
- execute_fn()'s executor.submit() and execute_isolated_fn()'s
run_in_executor() (immutable databases) now run the callable inside a
contextvars.copy_context(). A *fresh* copy per submit is required:
concurrently entering one shared Context raises "RuntimeError: cannot
enter context ... already entered".
- WriteTask carries the otel Context captured on the event loop at enqueue
time plus an enqueued_at_ns timestamp (both need __slots__ entries, or
they fail with AttributeError at runtime). _execute_writes attaches that
context right after the _SHUTDOWN check and detaches it in a finally
spanning all three execution branches - the write thread is persistent
and shared, so a leaked token would grow its context stack for every
write processed afterwards, and a wrong-token detach only logs rather
than raising.
- New spans: db.query.execute (read worker thread), db.write.queue_wait
(explicit start/end timestamps, so its duration is the real enqueue ->
dequeue wait rather than the microseconds spent building the span) and
db.write.execute (skipped in the conn_exception branch, where fn never
runs). db.query.execute honours log_sql_errors for the same reason
db.query does: facet suggestion probes with log_sql_errors=False and
would otherwise paint two red spans per text column on every table page.
- The write-thread warm-up prepare_connection is left as a documented
orphan root - no caller context exists that early.
Tests assert actual parent/child span-id relationships in a shared trace,
not just that spans exist, since an unparented root looks identical to a
correct span if you only check presence.
Note that copy_context() copies every ContextVar, not just OTel's, so
Datasette's own context vars (_skip_permission_checks,
_permission_check_cache, _in_datasette_client) now flow into worker
threads where they previously did not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
execute_isolated_fn() always opened its temporary connection with
write=True, which is not allowed for immutable databases - so APIs
that rely on it, like SQL analysis when storing a query, failed.
An immutable database can never receive writes, so there is no write
queue to block: in that case the function now opens a read-only
connection and runs it on the executor, bypassing the write thread
entirely. Mutable databases keep the existing write-thread behavior.
Also fixed a latent bug in the write thread where a connect() failure
for an isolated task would crash the thread instead of delivering the
exception back to the caller.
Closes#2768
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix for execute write returning, closes#2762
* Fix stored write returning rowcount message
* Add configurable execute_write returning limit
* Return rows/truncated from execute query if it used RETURNING
* INSERT ... RETURNING shows rows in /-/execute-write
* Skip RETURNING tests if SQLite version does not support it
Screenshot: https://github.com/simonw/datasette/issues/2762#issuecomment-4588111545
Experimental, we may need this for the upcoming canned query
work so that we can tell if a user should be able to save
a writable canned query by confirming they have the right
permissions to update the affected tables.
Refs #2735
After this commit, Database.close() sends a sentinel to the write queue so
the background write thread exits cleanly, closes cached read/write
connections, and marks the instance closed. Subsequent calls to execute*()
raise DatasetteClosedError. close() remains idempotent and one-way.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes#2683
* Add is_temp_disk option to Database for temp file-backed databases
Replace the default in-memory internal database with a temporary
file-backed database using WAL mode. This fixes concurrent read/write
locking errors that occur with named in-memory SQLite databases.
The new is_temp_disk parameter on Database creates a temp file via
tempfile.mkstemp, connects to it as a regular file-based database
with WAL mode enabled, and cleans it up on close() and via atexit.
https://claude.ai/code/session_01TteLrUjpDcARjnP1GMRqz2
* Fix flaky test_database_page test with deterministic ordering
- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst
* Update test_row_foreign_key_tables for new deterministic FK ordering
The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.
* Update test_table_names for new alphabetical ordering
The table_names() method now returns tables sorted alphabetically.
* Fix for test that fails prior to SQLite 3.37
---------
Co-authored-by: Claude <noreply@anthropic.com>
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:
- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
(was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
(fixed 71 test errors caused by fixtures not being registered)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Test for cross-database join, refs #283
* Warn if --crossdb used with more than 10 DBs, refs #283
* latest.datasette.io demo of --crossdb joins, refs #283
* Show attached databases on /_memory page, refs #283
* Documentation for cross-database queries, refs #283
* Support for generated columns, closes#1116
* Show SQLite version in pytest report header
* Use table_info() if SQLite < 3.26.0
* Cache sqlite_version() rather than re-calculate every time
* Adjust test_database_page for SQLite 3.26.0 or higher
