datasette/docs/telemetry_doc.py
Alex Garcia 2a05263ab8 Extend the kit with metric-side conformance: kind, unit and enum checks
The five surveyed plugin plans all kept a hand-rolled metrics-vs-registry
diff because the kit's conformance helpers covered spans only. This adds
the metric side:

- metric_for() in the registry (the span_for analogue - no prefix/dynamic
  machinery, metric names are static), and the attribute helpers are
  documented as accepting MetricName entries.
- MetricsCollector.collect() now retains the instrumentation scope per
  collected metric, so a plugin is judged against its own meter only.
- assert_metrics_conform(): every collected metric in scope is registered,
  was created as the instrument kind and unit its registry entry declares
  (drift between the registry entry and the meter.create_*() call was
  previously caught by nothing, in core or any plugin), sets only
  registered attributes, and respects values= enums - the check that makes
  a metric dimension provably bounded.
- assert_metrics_covered(): every registered metric collected at least
  once with every non-optional attribute seen. Both *_covered helpers now
  exempt optional=True attributes, so a workload is not forced to
  manufacture every error path; pin those with targeted tests instead.
- datasette.operation declares values={"read", "write"} - core dogfoods
  the enum enforcement on the dimension where it matters most.
- Core's generic metric conformance tests are now calls to the kit
  helpers with scope_name="datasette"; the stricter literal-pinning and
  optional-attribute-coverage tests stay hand-written on purpose.
- The metric reference docs render attributes through the same helper as
  spans, so *(optional)* markers and enum values now appear there too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-02 14:21:14 -07:00

53 lines
1.9 KiB
Python

"""
Render the span reference in ``internals.rst`` from
``datasette/telemetry_registry.py``.
Driven by cog, and ``cog --check docs/*.rst`` runs in CI - so adding a span
without documenting it, or documenting one that no longer exists, is a build
failure rather than something a reader discovers later.
"""
def _attribute_lines(cog, attributes):
if not attributes:
cog.out(" No attributes.\n\n")
return
cog.out(" Attributes:\n\n")
for attribute in attributes:
suffix = " *(optional)*" if attribute.optional else ""
line = f" - ``{attribute}``{suffix} - {attribute.description}"
if attribute.values is not None:
rendered = ", ".join(f"``{value}``" for value in sorted(attribute.values))
line += f" One of: {rendered}."
cog.out(line + "\n")
cog.out("\n")
def spans(cog):
from opentelemetry.trace import SpanKind
from datasette.telemetry_registry import SPANS
cog.out("\n")
for span in SPANS:
cog.out(f"``{span}``\n")
cog.out(f" {span.description}\n\n")
# INTERNAL is the default and the overwhelming majority of spans -
# printing it on every one would be noise. Only the exceptional case,
# a real database call, is worth calling out.
if span.kind != SpanKind.INTERNAL:
cog.out(f" Kind: ``{span.kind.name}``.\n\n")
_attribute_lines(cog, span.attributes)
def metrics(cog):
from datasette.telemetry_registry import METRICS
cog.out("\n")
for metric in METRICS:
cog.out(f"``{metric}``\n")
cog.out(f" {metric.kind}, unit ``{metric.unit}``. {metric.description}\n\n")
if metric.buckets:
boundaries = ", ".join(f"``{boundary}``" for boundary in metric.buckets)
cog.out(f" Bucket boundaries: {boundaries}.\n\n")
_attribute_lines(cog, metric.attributes)