Outcome of a whole-stack review with the kit visible as one system:
- otel_reset: an autouse fixture draining the span exporter and metric
reader after every test. Without it a large suite accumulates hundreds
of thousands of recorded spans in the session-scoped exporter - the
likeliest amplifier of the slow-runner CI flakes - and plugins would
inherit the same leak.
- assert_registry_covered renamed to assert_spans_covered: the old name
read as covering the whole registry, which is exactly wrong next to
assert_metrics_covered. Public API is forever; renamed before anything
ships, no alias.
- The installers now verify their provider actually took: with a
provider installed first (opentelemetry-instrument, an embedding app),
set_*_provider() is silently ignored, and fixtures would assert
against an exporter wired to nothing. They skip clearly instead.
- UPDOWN_COUNTER registry kind, mapped to Sum with monotonicity checked
both ways - a Counter must collect monotonic, an UpDownCounter must
not. Previously an UpDownCounter's kind check was silently skipped.
- The docs page now prescribes naming: scope = import package name
(underscores), signal prefix = a name you own, never bare datasette.*;
its own examples no longer teach the hyphenated outlier. Plus an
observable-gauges pattern section and a prefix-overlap note.
- assert_no_forbidden_values(): the enforcement half of the privacy
rules - plant sentinel secrets in a workload and assert they never
appear in any span name, attribute, event, status description or
metric attribute, across all scopes by default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
The five surveyed plugin plans all kept a hand-rolled metrics-vs-registry
diff because the kit's conformance helpers covered spans only. This adds
the metric side:
- metric_for() in the registry (the span_for analogue - no prefix/dynamic
machinery, metric names are static), and the attribute helpers are
documented as accepting MetricName entries.
- MetricsCollector.collect() now retains the instrumentation scope per
collected metric, so a plugin is judged against its own meter only.
- assert_metrics_conform(): every collected metric in scope is registered,
was created as the instrument kind and unit its registry entry declares
(drift between the registry entry and the meter.create_*() call was
previously caught by nothing, in core or any plugin), sets only
registered attributes, and respects values= enums - the check that makes
a metric dimension provably bounded.
- assert_metrics_covered(): every registered metric collected at least
once with every non-optional attribute seen. Both *_covered helpers now
exempt optional=True attributes, so a workload is not forced to
manufacture every error path; pin those with targeted tests instead.
- datasette.operation declares values={"read", "write"} - core dogfoods
the enum enforcement on the dimension where it matters most.
- Core's generic metric conformance tests are now calls to the kit
helpers with scope_name="datasette"; the stricter literal-pinning and
optional-attribute-coverage tests stay hand-written on purpose.
- The metric reference docs render attributes through the same helper as
spans, so *(optional)* markers and enum values now appear there too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
A survey of five plugin OTel plans (datasette-paper, -agent, -litestream,
-accounts, -cron) found every one hand-copying the same core machinery:
the registry classes, the conformance-test harness, the pytest fixtures,
the bucket boundaries and the detached-root-with-Link recipe. This makes
that machinery importable instead:
- The registry classes are documented public API. Attribute gains
values= (a closed enum the conformance helpers enforce - what makes an
attribute safe as a metric dimension); SpanName gains prefix=True for
span families like "chat {model}" whose names share a fixed prefix,
matched by span_for() after exact names. span_for()/attribute helpers
accept a spans= tuple so plugin registries can use them.
- datasette.telemetry.linked_root_span_kwargs(): the root-span-with-Link
shape for work a request caused without containing - background jobs,
scheduled ticks, block=False writes. Core's own write thread now uses
it instead of building the kwargs inline.
- datasette.telemetry_testing: the session provider fixtures, otel_spans
/ otel_metrics, a two-way registry conformance checker (including enum
and prefix handling, filtered by instrumentation scope) and an
assert_package_never_imports_sdk() guard. Core's conftest now imports
these instead of defining them, so the suite consumes the kit exactly
as a plugin's would.
- New "Telemetry for plugin authors" docs page: scope discipline,
registry usage, privacy/cardinality rules, named-callable guidance,
request_span(), the background root-with-link convention (one root per
tick, always emitted), provider-ordering facts and known caveats.
request_span() is now documented public API.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
Spans describe requests that have finished. They structurally cannot answer
"am I saturating my 3 SQL threads right now", because that is a level rather
than an event - and with num_sql_threads defaulting to 3, it is usually the
first thing worth knowing about a busy Datasette. This adds the metrics that
answer it.
Five observable gauges, computed only when something is collecting, so an
instance with no MeterProvider installed does no work for them at all:
datasette.sql.threads.limit num_sql_threads
datasette.sql.threads.queue_depth queries waiting for a free thread
datasette.sql.queries.pending in-flight reads, by db.namespace
datasette.write.queue_depth writes behind the single write thread
datasette.connections.open tracked file connections
Three instruments recorded inline, which matters because metrics survive
trace sampling and spans do not - an operator sampling 1% of traces still
gets 100% of the latency distribution:
db.client.operation.duration semconv histogram, with error.type
datasette.write.queue_wait the metric twin of the existing span
datasette.sql.queries.interrupted sql_time_limit_ms kills
The interrupted counter closes a gap the plan called out as unanswerable:
"how often are we killing queries at the limit" is a rate, and a rate cannot
be recovered from sampled spans.
Core still creates no provider of any kind, so the architecture is unchanged;
`grep -rn 'opentelemetry.sdk' datasette/` stays empty. One real difference
from tracing is worth recording: _ProxyMeter and its instruments forward to a
provider installed after they were created, whereas ProxyTracer permanently
caches the first concrete tracer it resolves. Module-level instruments are
therefore safe and the test fixture has no ordering constraint.
Live instances are tracked in a lock-guarded WeakSet so instrumenting an
instance never keeps it alive. The pool gauges carry no attribute saying
which Datasette produced them: production runs one instance per process, and
adding an id to disambiguate the test suite's hundreds of instances would buy
unbounded attribute cardinality to fix a case that does not occur. The
collision is documented instead, and the gauge callbacks are plain generator
functions so tests can assert exact values by calling them directly rather
than through the SDK's last-value aggregation.
demos/otel/metrics_demo.py fires 12 concurrent 40ms queries at a 3-thread
pool and samples the gauges mid-flight: queue_depth peaks at exactly 9, and
the duration histogram reads max=0.1695s for a query whose work is 40ms. That
gap is the queue, and it is the thing traces alone will not show you.
Also corrects the demo README's privacy section, which still claimed
parameter values are never recorded - that stopped being unconditionally true
when trace_sql_parameters landed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from 6ef0dd8c and adapted to the rebuilt phase-1 stack:
attribute names now come from telemetry_registry where entries exist, the
meter carries the instrumentation-scope version and schema URL, and the
interrupted-queries counter skips expected timeouts - callers that opted
into a deliberately short budget, like facet suggestion - matching how
those are excluded from span error status. The internals.rst reference
lands with the registry commit that follows.)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F2h9ANGZ7paWSpqs5DUAcG
- Add the missing changelog bullet for the HTTP request span - the most
user-visible signal in the stack had no entry.
- State the deliberate deviation on server.address: it is the verbatim
Host header including any :port, not the semconv address/port split.
- Deduplicate the is_recording()-vs-is_valid rationale: the middleware
fast path keeps the full telling, request_span() now points at it.
- Trim the http.route regex rationale in the docs intro to a pointer at
the attribute description, halve the httpx raw_path digression, drop
a comment that restated the call below it, and leave client-IP policy
to the privacy section instead of the user_agent description.
- Assert the routed 404 still carries http.route and an enriched span
name - route enrichment must not be gated on a successful response.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
The database instrumentation covered the four SQL-string entry points but
not the callback entry points, which are the documented way for plugins to
run arbitrary SQL - so the JSON write API's inserts and deletes, the
startup catalog scan, and every plugin built on execute_fn/execute_write_fn
were invisible to a trace, or worse, showed orphan-looking db.write.* spans
with no db.query above them.
Each callback method now opens the same db.query CLIENT span as its
SQL-string sibling, carrying a new optional datasette.callback attribute
(the callable's qualified name, captured before _wrap_fn_with_hooks() can
rename it) in place of db.query.text, which is now marked optional. A bare
execute_fn() also wraps the callback in a db.query.execute child, so the
"gap between the spans is thread-wait" story holds for plugin callbacks
too. No db.operation.name: there is no statement to take a keyword from,
and the registry says that attribute is omitted rather than guessed.
The previous bodies move to private _execute_fn()/_execute_write_fn() and
the SQL-string methods call those, so an execute() emits exactly the spans
it did before - pinned by test_execute_does_not_double_wrap. Database's own
introspection helpers stay on the public method deliberately: they are real
SQLite round trips, which lifts a table page from ~58 to ~100 (no-op) spans.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
- Remove the registry's unused prefix=True slot, its span_for() branch,
its doc-rendering case and its test - nothing in the stack sets it.
- Stop promising a "later phase" query-duration metric dimension in the
db.operation.name description; the cardinality rationale stands alone.
- Replace baked-in benchmark numbers in the telemetry module docstring
with the docs' own phrasing (below run-to-run variation).
- Compact the duplicated copy_context() and enqueue-site comments in
database.py to pointers at their canonical tellings.
- Make the "catch people out" gotchas skimmable as a bullet list and
give the changelog's "nothing is removed" line a clear antecedent.
- Add a test that a result cut short by max_returned_rows records
datasette.truncated=True - previously only ever asserted False.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
The span reference itself is generated from the registry, so this adds the
prose the generated list cannot supply: how to actually see a span, what is
deliberately never recorded, and where the instrumentation stops short.
The "how to turn it on" part is the part people get wrong. Core installs no
provider, so OTEL_TRACES_EXPORTER=console against a plain `datasette` process
emits nothing at all - that variable is read by the SDK auto-configuration
which only runs under `opentelemetry-instrument`. Documented as a warning
because it reads like a bug when you hit it. Two more measured facts get the
same treatment: the SDK's BatchSpanProcessor default schedule delay is 5000ms
(checked, not assumed - `BatchSpanProcessor._default_schedule_delay_millis()`
on opentelemetry-sdk 1.44), so nothing appears for five seconds; and without
OTEL_SERVICE_NAME the default resource reports service.name=unknown_service.
Privacy properties are stated positively rather than left implicit: SQL
truncated at 2048 characters, parameter values never recorded, no actor
identifiers, table names only from an explicit `table=` argument. The last of
those is now documented on db.execute() itself, since it is public API.
The limitations section claims only what was measured. An earlier draft said
two traces per process are orphaned by the register_output_renderer and
asgi_wrapper hooks; measuring it showed a default install emits zero spans
from either, because Datasette queries no database there - it is a plugin
that would produce the orphan. Corrected to say that.
It also deliberately does NOT say an embedder must install its provider
before Datasette's first span or get nothing. That claim is false:
ProxyTracer._tracer returns the no-op tracer without caching it when no
provider is set, so early spans are dropped and nothing is poisoned.
The telemetry.py docstring said no-op spans "cost approximately nothing".
The benchmark for this diff does not support a claim that strong - a table
page emits ~58 spans - so it now states the measurement instead: median
9.80ms to 9.98ms across 15 runs, inside a 1.4ms run-to-run spread.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The alias existed so API payloads could mirror the params key used by
queries defined in datasette.yaml, but it was undocumented and untested,
and the create endpoint is not part of the stable API. The API now only
accepts parameters - sending params is a 400 Invalid keys error. The
documented params key for queries in configuration is unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
Documents the canonical error format, the ok/envelope changes, the
array-to-object endpoint conversions, 401s for invalid tokens, the
pagination and page-size unification, removed legacy keys and formats,
and the new Response.error(), TokenInvalid, count_truncated and
unstable-marker APIs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
* Add web UI to edit and delete stored queries
Stored query pages now offer Edit and Delete actions in the query
actions menu, gated by the update-query and delete-query permissions.
- New QueryEditView (GET/POST at /<db>/<query>/-/edit) renders a
pre-filled form for editing a query's title, description, SQL and
privacy, reusing the create-query analysis UI. Changing the SQL still
requires execute-sql; metadata-only edits do not.
- QueryDeleteView gains a GET confirmation page and HTML form POST that
redirects to the query list, while keeping the existing JSON API.
- New default query_actions hook adds the Edit/Delete links for stored
(non-config, non-trusted) queries the actor is allowed to manage.
Permission semantics (already enforced by default_query_permissions_sql)
are surfaced in the UI: owners can always edit/delete their queries;
non-private queries can be edited/deleted by any actor with the relevant
permission; private queries remain owner-only.
Shared the create-query form styles into _query_form_styles.html so the
edit form can reuse them.
Animated demo: https://github.com/simonw/datasette/pull/2764#issuecomment-4655694668Closes#2760https://claude.ai/code/session_019GU9g3pZAERukLKYNa4uAL
Expand the unreleased changelog with the deny-by-default operation analysis model, SQL function handling, and the VACUUM and virtual/shadow table restrictions for user-supplied write SQL.
Clarify the /-/execute-write JSON API documentation with the same restrictions and DDL permission requirements.
