Register metrics and give histograms bucket boundaries suited to seconds

Both histograms declared unit="s" but inherited OpenTelemetry's default
boundaries, which are tuned for milliseconds - so every SQLite query
landed in the single (0, 5] second bucket and every quantile query
returned noise.

The boundaries are the semantic conventions' recommended set for
db.client.operation.duration plus 0.0001 and 0.0005 at the bottom, since
SQLite is in-process and many real queries take tens of microseconds.

(Adapted from 024f2029: that commit assumed the metrics were already in
telemetry_registry.py, which on this lineage held spans only - so this
commit also brings the MetricName registry machinery, the registry
entries for all eight phase-3 metrics, the cog-generated Metric
reference in internals.rst, and the datasette.operation attribute. The
template and facet histograms it also touched belong to phase 5 and are
not included.)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F2h9ANGZ7paWSpqs5DUAcG

docs/internals.rst

@@ -2354,7 +2354,7 @@ Spans do not appear immediately. The SDK's default ``BatchSpanProcessor`` flushe
 
 Always set ``OTEL_SERVICE_NAME``. Without it the SDK's default resource reports a ``service.name`` of ``unknown_service``, and your traces will be filed under that instead of under a name you can search for.
 
-Setting ``OTEL_METRICS_EXPORTER=none`` and ``OTEL_LOGS_EXPORTER=none`` is worth doing unless your backend accepts those signals too - ``opentelemetry-distro`` defaults every signal to OTLP, and a traces-only backend will reject the other two noisily. Datasette itself emits no metrics and no logs through OpenTelemetry.
+Setting ``OTEL_LOGS_EXPORTER=none`` is worth doing unless your backend accepts logs too - ``opentelemetry-distro`` defaults every signal to OTLP, and a backend that does not take a signal will reject it noisily. Datasette emits no logs through OpenTelemetry; it does emit metrics (see :ref:`internals_telemetry_metrics`), so set ``OTEL_METRICS_EXPORTER=none`` only if your backend does not accept them.
 
 Span reference
 --------------
@@ -2439,6 +2439,85 @@ That is the route's compiled regular expression, not a prettified ``/{database}/
 
 .. [[[end]]]
 
+.. _internals_telemetry_metrics:
+
+Metric reference
+----------------
+
+Spans describe events; metrics describe levels and rates. "Am I saturating my :ref:`setting_num_sql_threads` threads right now?" cannot be answered by any span, because it is a level sampled at collection time - and it is usually the first thing worth knowing about a busy Datasette, since ``num_sql_threads`` defaults to ``3``. Metrics also survive trace sampling: an operator keeping 1% of traces still gets 100% of every histogram and counter below.
+
+As with spans, core emits these through the OpenTelemetry API only. Without a ``MeterProvider`` every instrument is a no-op, and the observable-gauge callbacks are never invoked at all, so an uninstrumented install pays nothing for them.
+
+Every duration histogram is in **seconds**, with explicit bucket boundaries chosen for an in-process database - OpenTelemetry's default boundaries are tuned for milliseconds and would file every SQLite query into a single bucket, making quantile queries meaningless. The boundaries are listed with each histogram because a ``histogram_quantile()`` query is only as good as the buckets underneath it.
+
+This reference is generated from ``datasette/telemetry_registry.py``, like the span reference above.
+
+.. [[[cog
+    from telemetry_doc import metrics
+    metrics(cog)
+.. ]]]
+
+``db.client.operation.duration``
+    Histogram, unit ``s``. Duration of a SQL operation. The standard OpenTelemetry semantic convention metric, and the one that survives trace sampling.
+
+    Bucket boundaries: ``0.0001``, ``0.0005``, ``0.001``, ``0.005``, ``0.01``, ``0.05``, ``0.1``, ``0.5``, ``1``, ``5``, ``10``.
+
+    Attributes:
+
+    - ``db.system`` - Always ``sqlite``.
+    - ``db.namespace`` - Name of the database being queried.
+    - ``datasette.operation`` - ``read`` or ``write``.
+    - ``error.type`` - Set when the request failed: the exception class name if one escaped the application, otherwise the status code as a string for a 5xx response. A 4xx does **not** set this and does not set an error status - per semantic conventions a client error is not a server span's failure.
+
+``datasette.write.queue_wait``
+    Histogram, unit ``s``. Time each write waited in its database's write queue. The metric counterpart of the ``db.write.queue_wait`` span.
+
+    Bucket boundaries: ``0.0001``, ``0.0005``, ``0.001``, ``0.005``, ``0.01``, ``0.05``, ``0.1``, ``0.5``, ``1``, ``5``, ``10``.
+
+    Attributes:
+
+    - ``db.namespace`` - Name of the database being queried.
+
+``datasette.sql.queries.interrupted``
+    Counter, unit ``{query}``. Queries cancelled for exceeding :ref:`setting_sql_time_limit_ms`. Worth alerting on: a rising rate means the limit is too tight or a table has outgrown its queries. A caller that opted into a deliberately shorter budget - facet suggestion, for example - is not counted, for the same reason its timeout is not a span error.
+
+    Attributes:
+
+    - ``db.namespace`` - Name of the database being queried.
+
+``datasette.sql.threads.limit``
+    Observable gauge, unit ``{thread}``. Maximum concurrent read queries - the :ref:`setting_num_sql_threads` value. Not reported when ``num_sql_threads`` is ``0``, since then queries run on the event loop and there is no pool.
+
+    No attributes.
+
+``datasette.sql.threads.queue_depth``
+    Observable gauge, unit ``{query}``. Read queries waiting for a free thread. **This is the saturation signal** - sustained above zero means requests are queueing on ``num_sql_threads``.
+
+    No attributes.
+
+``datasette.sql.queries.pending``
+    Observable gauge, unit ``{query}``. Read queries submitted to the pool and not yet complete. Summed across databases and compared against the thread limit, this is pool utilisation.
+
+    Attributes:
+
+    - ``db.namespace`` - Name of the database being queried.
+
+``datasette.write.queue_depth``
+    Observable gauge, unit ``{write}``. Writes queued behind a database's single write thread. Backpressure that raising ``num_sql_threads`` cannot relieve. Not reported for a database that has never been written to.
+
+    Attributes:
+
+    - ``db.namespace`` - Name of the database being queried.
+
+``datasette.connections.open``
+    Observable gauge, unit ``{connection}``. Open SQLite file connections currently tracked for closing.
+
+    Attributes:
+
+    - ``db.namespace`` - Name of the database being queried.
+
+.. [[[end]]]
+
 .. _internals_telemetry_requests:
 
 Requests and inbound trace context

docs/telemetry_doc.py

@@ -35,3 +35,22 @@ def spans(cog):
         if span.kind != SpanKind.INTERNAL:
             cog.out(f"    Kind: ``{span.kind.name}``.\n\n")
         _attribute_lines(cog, span.attributes)
+
+
+def metrics(cog):
+    from datasette.telemetry_registry import METRICS
+
+    cog.out("\n")
+    for metric in METRICS:
+        cog.out(f"``{metric}``\n")
+        cog.out(f"    {metric.kind}, unit ``{metric.unit}``. {metric.description}\n\n")
+        if metric.buckets:
+            boundaries = ", ".join(f"``{boundary}``" for boundary in metric.buckets)
+            cog.out(f"    Bucket boundaries: {boundaries}.\n\n")
+        if metric.attributes:
+            cog.out("    Attributes:\n\n")
+            for attribute in metric.attributes:
+                cog.out(f"    - ``{attribute}`` - {attribute.description}\n")
+            cog.out("\n")
+        else:
+            cog.out("    No attributes.\n\n")