Outcome of a whole-stack review with the kit visible as one system:
- otel_reset: an autouse fixture draining the span exporter and metric
reader after every test. Without it a large suite accumulates hundreds
of thousands of recorded spans in the session-scoped exporter - the
likeliest amplifier of the slow-runner CI flakes - and plugins would
inherit the same leak.
- assert_registry_covered renamed to assert_spans_covered: the old name
read as covering the whole registry, which is exactly wrong next to
assert_metrics_covered. Public API is forever; renamed before anything
ships, no alias.
- The installers now verify their provider actually took: with a
provider installed first (opentelemetry-instrument, an embedding app),
set_*_provider() is silently ignored, and fixtures would assert
against an exporter wired to nothing. They skip clearly instead.
- UPDOWN_COUNTER registry kind, mapped to Sum with monotonicity checked
both ways - a Counter must collect monotonic, an UpDownCounter must
not. Previously an UpDownCounter's kind check was silently skipped.
- The docs page now prescribes naming: scope = import package name
(underscores), signal prefix = a name you own, never bare datasette.*;
its own examples no longer teach the hyphenated outlier. Plus an
observable-gauges pattern section and a prefix-overlap note.
- assert_no_forbidden_values(): the enforcement half of the privacy
rules - plant sentinel secrets in a workload and assert they never
appear in any span name, attribute, event, status description or
metric attribute, across all scopes by default.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
The five surveyed plugin plans all kept a hand-rolled metrics-vs-registry
diff because the kit's conformance helpers covered spans only. This adds
the metric side:
- metric_for() in the registry (the span_for analogue - no prefix/dynamic
machinery, metric names are static), and the attribute helpers are
documented as accepting MetricName entries.
- MetricsCollector.collect() now retains the instrumentation scope per
collected metric, so a plugin is judged against its own meter only.
- assert_metrics_conform(): every collected metric in scope is registered,
was created as the instrument kind and unit its registry entry declares
(drift between the registry entry and the meter.create_*() call was
previously caught by nothing, in core or any plugin), sets only
registered attributes, and respects values= enums - the check that makes
a metric dimension provably bounded.
- assert_metrics_covered(): every registered metric collected at least
once with every non-optional attribute seen. Both *_covered helpers now
exempt optional=True attributes, so a workload is not forced to
manufacture every error path; pin those with targeted tests instead.
- datasette.operation declares values={"read", "write"} - core dogfoods
the enum enforcement on the dimension where it matters most.
- Core's generic metric conformance tests are now calls to the kit
helpers with scope_name="datasette"; the stricter literal-pinning and
optional-attribute-coverage tests stay hand-written on purpose.
- The metric reference docs render attributes through the same helper as
spans, so *(optional)* markers and enum values now appear there too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
A survey of five plugin OTel plans (datasette-paper, -agent, -litestream,
-accounts, -cron) found every one hand-copying the same core machinery:
the registry classes, the conformance-test harness, the pytest fixtures,
the bucket boundaries and the detached-root-with-Link recipe. This makes
that machinery importable instead:
- The registry classes are documented public API. Attribute gains
values= (a closed enum the conformance helpers enforce - what makes an
attribute safe as a metric dimension); SpanName gains prefix=True for
span families like "chat {model}" whose names share a fixed prefix,
matched by span_for() after exact names. span_for()/attribute helpers
accept a spans= tuple so plugin registries can use them.
- datasette.telemetry.linked_root_span_kwargs(): the root-span-with-Link
shape for work a request caused without containing - background jobs,
scheduled ticks, block=False writes. Core's own write thread now uses
it instead of building the kwargs inline.
- datasette.telemetry_testing: the session provider fixtures, otel_spans
/ otel_metrics, a two-way registry conformance checker (including enum
and prefix handling, filtered by instrumentation scope) and an
assert_package_never_imports_sdk() guard. Core's conftest now imports
these instead of defining them, so the suite consumes the kit exactly
as a plugin's would.
- New "Telemetry for plugin authors" docs page: scope discipline,
registry usage, privacy/cardinality rules, named-callable guidance,
request_span(), the background root-with-link convention (one root per
tick, always emitted), provider-ordering facts and known caveats.
request_span() is now documented public API.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
