Datasette core is gaining OpenTelemetry spans alongside the existing
hand-rolled tracer. This commit only lays the groundwork - no span is
emitted yet.
Core takes a runtime dependency on opentelemetry-api and nothing more.
It deliberately never creates a TracerProvider, configures an exporter,
or touches sampling: that belongs to whoever runs Datasette, normally
via an opentelemetry-instrument agent. Owning a provider in core was
tried in an earlier design and produced a cross-request span leak, a
process-global provider that tests could not tear down, and a sampling
env var that silently blanked output. With no provider installed every
span is a NonRecordingSpan and costs approximately nothing.
datasette/telemetry.py exposes the module-level tracer plus
sql_attribute(), which truncates SQL to 2048 characters. On a public
instance the SQL is attacker-controlled and unbounded - someone can
paste a 10MB query into ?sql= - so it must never reach a telemetry
pipeline verbatim.
opentelemetry-sdk goes in the dev dependency group only, because the
test suite needs it to assert on spans while the package itself must
not import it. tests/test_telemetry.py enforces that by importing
datasette in a fresh interpreter and inspecting sys.modules, which
catches a lazy import inside a function body that a grep would miss.
conftest.py gains a session-scoped autouse fixture installing an SDK
provider with an InMemorySpanExporter. It has to be session-scoped
because set_tracer_provider() is effectively once-per-process - a
second call logs a warning and is ignored. SimpleSpanProcessor rather
than BatchSpanProcessor, so assertions made right after a request never
race a background export thread. The otel_spans fixture that later
tickets assert against is added here too.
test_datasette_package_never_imports_the_sdk is moved to the front of
the run. Late in a serial run the pytest process holds enough threads
that the fork half of subprocess' fork+exec segfaults the interpreter
on macOS/CPython 3.13. That reproduces with any subprocess call in that
position on an unmodified tree, so it is a pre-existing hazard rather
than something this commit introduces; the repo already moves its other
subprocess-spawning tests to the front for related reasons.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Close in-memory database connections, including reads opened on threads
- Close completed and partial file upload file handles
- Close SQLite connections and file handles owned by tests
When streaming a view as CSV with ?_stream=on, _next was being
re-read from request.args, overwriting the kwarg passed by the
streaming loop with None. This caused the pagination offset to stay
stuck on page 1 forever.
execute_write_fn(fn, block=False) is documented to return "a UUID representing
the queued task". Two things stopped that being true.
_send_to_write_thread() derived the id from uuid.uuid5(NAMESPACE_DNS,
"datasette.io"), which is deterministic, so every non-blocking write in every
database in every process returned 3f143baa-4e3d-5842-a36f-4fa2f683b72f. A
constant cannot identify a particular task. Now uuid4(). Refs #2860.
With num_sql_threads=0 there is no write thread, so execute_write_fn took the
synchronous branch and `result` was the write function's return value, normally
None. The block=False path then unpacked it unconditionally and raised
TypeError: cannot unpack non-iterable NoneType object. The non-threaded branch
now returns the same (task_id, reply_future) shape, with the future already
resolved because the write has finished, so both modes share one code path.
Refs #2859.
test_execute_write_fn_block_false only asserted isinstance(task_id, uuid.UUID),
which a constant satisfies. The new test is parametrized over threaded and
non-threaded and asserts two calls return different ids, so either regression
fails it.
* Run datasette serve startup and uvicorn on a single event loop
* Move the serve-subprocess test plumbing into a conftest fixture
* Fix datasette-litestream URL and trim marker-task test comments
* Explain why serve_with_plugins needs a subprocess and plugin files
* Apply ruff 0.16 and black fixes
* Tweaked some comments
Several internal helpers quoted table names using SQLite [bracket]
identifiers built with an f-string, e.g. PRAGMA foreign_key_list([{table}]).
Bracket quoting cannot escape a "]" character, so any table whose name
contains "]" (for example "[foo]" or "foo]") produced
"sqlite3.OperationalError: unrecognized token" - crashing schema
introspection at startup and 500-ing the table page.
Switch these call sites to the existing escape_sqlite() helper, which uses
"double quote" quoting with correct "" escaping (the same approach already
used elsewhere in the codebase and in the test suite):
- utils/internal_db.py: PRAGMA foreign_key_list / index_list
- utils/__init__.py: get_outbound_foreign_keys
- database.py: table_counts count query
- facets.py: default "select * from" SQL
Added a regression test covering table names with "]" characters.
Co-authored-by: Claude <noreply@anthropic.com>
Reverts the object envelope introduced in 1.0a36 for this endpoint -
it once again returns a top-level JSON array of plugin objects.
Closes#2842
Claude-Session: https://claude.ai/code/session_012TYc1NTBK4zEjabB3u2zqu
Co-authored-by: Claude <noreply@anthropic.com>
