Commit graph

2 commits

Author SHA1 Message Date
Alex Garcia
2a05263ab8 Extend the kit with metric-side conformance: kind, unit and enum checks
The five surveyed plugin plans all kept a hand-rolled metrics-vs-registry
diff because the kit's conformance helpers covered spans only. This adds
the metric side:

- metric_for() in the registry (the span_for analogue - no prefix/dynamic
  machinery, metric names are static), and the attribute helpers are
  documented as accepting MetricName entries.
- MetricsCollector.collect() now retains the instrumentation scope per
  collected metric, so a plugin is judged against its own meter only.
- assert_metrics_conform(): every collected metric in scope is registered,
  was created as the instrument kind and unit its registry entry declares
  (drift between the registry entry and the meter.create_*() call was
  previously caught by nothing, in core or any plugin), sets only
  registered attributes, and respects values= enums - the check that makes
  a metric dimension provably bounded.
- assert_metrics_covered(): every registered metric collected at least
  once with every non-optional attribute seen. Both *_covered helpers now
  exempt optional=True attributes, so a workload is not forced to
  manufacture every error path; pin those with targeted tests instead.
- datasette.operation declares values={"read", "write"} - core dogfoods
  the enum enforcement on the dimension where it matters most.
- Core's generic metric conformance tests are now calls to the kit
  helpers with scope_name="datasette"; the stricter literal-pinning and
  optional-attribute-coverage tests stay hand-written on purpose.
- The metric reference docs render attributes through the same helper as
  spans, so *(optional)* markers and enum values now appear there too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-02 14:21:14 -07:00
Alex Garcia
f28db54eda Add a plugin telemetry kit: public registry API, linked_root_span_kwargs, test helpers, docs
A survey of five plugin OTel plans (datasette-paper, -agent, -litestream,
-accounts, -cron) found every one hand-copying the same core machinery:
the registry classes, the conformance-test harness, the pytest fixtures,
the bucket boundaries and the detached-root-with-Link recipe. This makes
that machinery importable instead:

- The registry classes are documented public API. Attribute gains
  values= (a closed enum the conformance helpers enforce - what makes an
  attribute safe as a metric dimension); SpanName gains prefix=True for
  span families like "chat {model}" whose names share a fixed prefix,
  matched by span_for() after exact names. span_for()/attribute helpers
  accept a spans= tuple so plugin registries can use them.
- datasette.telemetry.linked_root_span_kwargs(): the root-span-with-Link
  shape for work a request caused without containing - background jobs,
  scheduled ticks, block=False writes. Core's own write thread now uses
  it instead of building the kwargs inline.
- datasette.telemetry_testing: the session provider fixtures, otel_spans
  / otel_metrics, a two-way registry conformance checker (including enum
  and prefix handling, filtered by instrumentation scope) and an
  assert_package_never_imports_sdk() guard. Core's conftest now imports
  these instead of defining them, so the suite consumes the kit exactly
  as a plugin's would.
- New "Telemetry for plugin authors" docs page: scope discipline,
  registry usage, privacy/cardinality rules, named-callable guidance,
  request_span(), the background root-with-link convention (one root per
  tick, always emitted), provider-ordering facts and known caveats.
  request_span() is now documented public API.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-02 14:21:14 -07:00