Extend the kit with metric-side conformance: kind, unit and enum checks

The five surveyed plugin plans all kept a hand-rolled metrics-vs-registry
diff because the kit's conformance helpers covered spans only. This adds
the metric side:

- metric_for() in the registry (the span_for analogue - no prefix/dynamic
  machinery, metric names are static), and the attribute helpers are
  documented as accepting MetricName entries.
- MetricsCollector.collect() now retains the instrumentation scope per
  collected metric, so a plugin is judged against its own meter only.
- assert_metrics_conform(): every collected metric in scope is registered,
  was created as the instrument kind and unit its registry entry declares
  (drift between the registry entry and the meter.create_*() call was
  previously caught by nothing, in core or any plugin), sets only
  registered attributes, and respects values= enums - the check that makes
  a metric dimension provably bounded.
- assert_metrics_covered(): every registered metric collected at least
  once with every non-optional attribute seen. Both *_covered helpers now
  exempt optional=True attributes, so a workload is not forced to
  manufacture every error path; pin those with targeted tests instead.
- datasette.operation declares values={"read", "write"} - core dogfoods
  the enum enforcement on the dimension where it matters most.
- Core's generic metric conformance tests are now calls to the kit
  helpers with scope_name="datasette"; the stricter literal-pinning and
  optional-attribute-coverage tests stay hand-written on purpose.
- The metric reference docs render attributes through the same helper as
  spans, so *(optional)* markers and enum values now appear there too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA

docs/changelog.rst

@@ -14,7 +14,7 @@ Unreleased
 - Every HTTP request now gets an OpenTelemetry ``SERVER`` span, named after the request method and matched route, carrying ``http.route``, the response status and W3C trace context extracted from inbound headers - so every database span has a request to belong to, and Datasette joins distributed traces started by a proxy or calling service. The query string is never recorded. See :ref:`internals_telemetry_requests`. (:issue:`1730`)
 - Datasette core now also emits OpenTelemetry **metrics** covering SQL thread pool saturation, per-database write queue depth, open connections, query latency and time-limit interruptions. These answer operational questions that spans structurally cannot - "am I saturating my :ref:`setting_num_sql_threads` threads?" is a level, not an event - and they survive trace sampling. As with spans, core installs no ``MeterProvider``, so there is no cost unless metrics are collected externally. See :ref:`internals_telemetry`. (:issue:`1730`)
 
-- New :ref:`plugin telemetry kit <plugin_telemetry>` for plugins that emit their own OpenTelemetry signals: the registry classes (``Attribute`` with closed-enum ``values=``, ``SpanName`` with prefix-matched families, ``MetricName``) are now documented public API, ``datasette.telemetry.linked_root_span_kwargs()`` provides the root-span-with-link shape for background work, ``datasette.telemetry.request_span()`` is documented, and ``datasette.telemetry_testing`` ships the pytest fixtures and two-way conformance checks core's own suite uses. (:issue:`1730`)
+- New :ref:`plugin telemetry kit <plugin_telemetry>` for plugins that emit their own OpenTelemetry signals: the registry classes (``Attribute`` with closed-enum ``values=``, ``SpanName`` with prefix-matched families, ``MetricName``) are now documented public API, ``datasette.telemetry.linked_root_span_kwargs()`` provides the root-span-with-link shape for background work, ``datasette.telemetry.request_span()`` is documented, and ``datasette.telemetry_testing`` ships the pytest fixtures and two-way conformance checks - for spans and metrics, including instrument kind/unit verification and enum enforcement - that core's own suite uses. (:issue:`1730`)
 
 Nothing is removed by the OpenTelemetry work: the ``?_trace=1`` query string parameter, the ``trace_debug`` setting and the :ref:`internals_tracer` module all continue to work as before.
 

docs/internals.rst

@@ -2476,8 +2476,8 @@ This reference is generated from ``datasette/telemetry_registry.py``, like the s
 
     - ``db.system`` - Always ``sqlite``.
     - ``db.namespace`` - Name of the database being queried.
-    - ``datasette.operation`` - ``read`` or ``write``.
-    - ``error.type`` - Set when the request failed: the exception class name if one escaped the application, otherwise the status code as a string for a 5xx response. A 4xx does **not** set this and does not set an error status - per semantic conventions a client error is not a server span's failure.
+    - ``datasette.operation`` - Whether the operation was a read or a write. One of: ``read``, ``write``.
+    - ``error.type`` *(optional)* - Set when the request failed: the exception class name if one escaped the application, otherwise the status code as a string for a 5xx response. A 4xx does **not** set this and does not set an error status - per semantic conventions a client error is not a server span's failure.
 
 ``datasette.write.queue_wait``
     Histogram, unit ``s``. Time each write waited in its database's write queue. The metric counterpart of the ``db.write.queue_wait`` span.

docs/plugin_telemetry.rst

@@ -163,27 +163,35 @@ Wire your registry to reality with the conformance helpers - the two directions
 .. code-block:: python
 
     from datasette.telemetry_testing import (
+        assert_metrics_conform,
+        assert_metrics_covered,
         assert_package_never_imports_sdk,
         assert_registry_covered,
         assert_spans_conform,
     )
 
-    from my_plugin.telemetry import SPANS
+    from my_plugin.telemetry import METRICS, SPANS
 
 
-    def test_conformance(otel_spans):
+    def test_conformance(otel_spans, otel_metrics):
         run_a_workload_that_exercises_everything()
         finished = otel_spans.get_finished_spans()
         # Everything emitted is registered (and enum values are legal):
         assert_spans_conform(SPANS, finished, scope_name="my-plugin")
         # Everything registered was emitted:
         assert_registry_covered(SPANS, finished, scope_name="my-plugin")
+        # Same two directions for metrics - one collect() after the workload:
+        otel_metrics.collect()
+        assert_metrics_conform(METRICS, otel_metrics, scope_name="my-plugin")
+        assert_metrics_covered(METRICS, otel_metrics, scope_name="my-plugin")
 
 
     def test_api_only_dependency():
         assert_package_never_imports_sdk("my_plugin")
 
-Always pass ``scope_name`` - the exporter also holds core's spans, and your registry should only be judged against your own.
+Always pass ``scope_name`` - the exporter and reader also hold core's signals, and your registry should only be judged against your own.
+
+The metric helpers check more than names: ``assert_metrics_conform`` asserts each instrument was created as the **kind** and **unit** its registry entry declares (the registry entry and the ``meter.create_*()`` call are separate statements, and a dashboard built on the registry's word breaks silently if they drift), and that every value on a ``values=`` enum attribute is a member - which is what makes a metric dimension *provably* bounded rather than bounded by intent. Both ``*_covered`` helpers exempt attributes marked ``optional=True`` (an ``error.type`` only present on failures should not force your workload to manufacture errors - pin those with targeted tests instead), and the metrics reader uses delta temporality, so run one broad workload followed by a single ``collect()``.
 
 .. _plugin_telemetry_caveats:
 

docs/telemetry_doc.py

@@ -50,10 +50,4 @@ def metrics(cog):
         if metric.buckets:
             boundaries = ", ".join(f"``{boundary}``" for boundary in metric.buckets)
             cog.out(f"    Bucket boundaries: {boundaries}.\n\n")
-        if metric.attributes:
-            cog.out("    Attributes:\n\n")
-            for attribute in metric.attributes:
-                cog.out(f"    - ``{attribute}`` - {attribute.description}\n")
-            cog.out("\n")
-        else:
-            cog.out("    No attributes.\n\n")
+        _attribute_lines(cog, metric.attributes)