Commit graph

1,391 commits

Author SHA1 Message Date
Alex Garcia
2e1c769ad4 Stop marking a deliberately-short query budget as a span error
Datasette has a family of callers that run a query under a tiny time limit
and treat "did not finish" as a usable answer. table_counts() is the loudest:
the homepage counts every table with a 10ms budget and stores None for the
ones that blow it. The QueryInterrupted handler on the db.query span was
unconditional, so on a two-table database that produced four ERROR spans -
two db.query and two db.query.execute - on every homepage hit. Measured on a
30MB two-table database: 4 red spans before, 0 after.

Honouring log_sql_errors here would have silenced none of it. Only the three
ArrayFacet json_type() probes pass log_sql_errors=False, and they are not the
queries that time out; table_counts() and ColumnFacet.suggest both leave it at
its True default. The signal that does separate the two cases is the budget
itself: a caller asking for less time than sql_time_limit_ms is saying the
query may not finish. Keying off that needs no new API and no changes outside
database.py. A query that runs out the instance-wide limit is still an error.

datasette.interrupted is still set in every case - it is the signal worth
having, and only the ERROR status becomes conditional. Its registry
description said the status is "also set to ERROR" full stop, which is now
wrong, and that string is published in docs/internals.rst.

The inner db.query.execute span carried the same bug through
set_status_on_exception=log_sql_errors, so its exception handling is now
explicit, matching the db.query span above it. The context manager's flags
apply to every exception type alike and this span has to tell two apart.

test_query_interrupted_sets_error_status forced its timeout with
?_timelimit=5, which is exactly the signal now reclassified as expected. It
now forces one via sql_time_limit_ms so it still tests what it was written to
test.

Also documents, at the copy_context() sites, that context propagation carries
Datasette's non-OTel ContextVars into worker threads too. Verified harmless:
nothing reads _skip_permission_checks, _permission_check_cache or
_in_datasette_client off the event loop, and Context.run() restores the
thread's previous context on return, so no value can reach the next task on
the shared pool.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 09:22:45 -07:00
Alex Garcia
c90586db06 Name every span and attribute once, in a registry the docs are built from
The span and attribute names were string literals spread across four call
sites in database.py and one in app.py, with a hand-written reference page
that would have been true only on the day it was written. That drift is not
hypothetical: an earlier iteration of this work carried a README asserting
parameter values were never recorded for two branches after that had stopped
being true.

datasette/telemetry_registry.py now holds each name once, with its
documentation. Attribute and SpanName subclass str, so a registry entry *is*
the string OpenTelemetry wants - no wrapper API over the OTel calls, no
parallel structure to keep in step, and a typo becomes an ImportError rather
than a silently misnamed attribute. docs/internals.rst renders the span
reference from it via cog, and `cog --check docs/*.rst` already runs in CI,
so the reference cannot drift from the definitions.

Nothing changes on the wire: the emitted span names and attribute keys are
byte-identical before and after, verified by diffing a dump of both.

tests/test_telemetry_registry.py exercises a real workload and compares it
against the registry in both directions - emitted-but-unregistered catches
instrumentation added without documentation, registered-but-never-emitted
catches documentation that has outlived its code. Because the call sites now
take their names from the registry, neither direction can catch a rename:
move DB_NAMESPACE to "db.namespace2" and code and registry still agree while
every dashboard breaks. So the literal names are also written out in the test
and asserted against the registry and against the wire separately. That pair
is the only comparison in the file not derived from the registry itself.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 09:22:45 -07:00
Simon Willison
2eb2571fee Release 1.0a40
Refs #1681, #1695, #1861, #2005, #2129, #2639, #2876, #2902, #2903, #2914, #2924, #2925, GHSA-h547-rmjf-5m2m
2026-09-16 16:46:51 -07:00
Simon Willison
caf238aac8 Fix for GHSA-h547-rmjf-5m2m 2026-09-16 16:43:34 -07:00
Simon Willison
d2098e9f84
Fix facet selection for explicit exact filters
`colname__exact=value` is now treated the same as `colname=value` when determining which facets are selected.

Closes #1695
2026-09-16 14:56:09 -07:00
Simon Willison
0d0bb5dd8c
Merge branch 'main' into codex/test-write-wrapper-alter-rollback 2026-09-16 14:42:20 -07:00
Simon Willison
61fd3461c9 Docs showing how to use db.atomic(), refs #2871 2026-09-16 14:41:29 -07:00
Simon Willison
6c119323d0 Keep alter-table changes rollbackable by write wrappers 2026-09-16 14:40:50 -07:00
Simon Willison
6dd5297b34
Allow extra_template_vars to resolve to None
Closes #2005
2026-09-16 10:30:11 -07:00
Simon Willison
df4c0fde0f
Make request header lookups case-insensitive
Closes #1861
2026-09-16 10:27:35 -07:00
Simon Willison
faff4c8820 Return CSV errors as plain text, closes #2129 2026-09-15 15:53:10 -07:00
Alex Garcia
4bf30aaf8e
Use $DATASETTE_INTERNAL in absence of --internal (#2174) 2026-09-15 15:39:39 -07:00
Simon Willison
27b7240f30 Fix for render_cell() pks for SQL views, refs #2639 2026-09-15 15:30:51 -07:00
Simon Willison
7c8ed1015e Changelog for #1681, #2876 2026-09-15 13:35:51 -07:00
Simon Willison
f40d838429 Changelog entry for #2902 and #2903 2026-09-15 12:58:41 -07:00
Simon Willison
ea3edcc1b9 How to use crumbs for plugin authors, closes #1902 2026-09-15 12:53:02 -07:00
nightcityblade
4fe985945d
Document customizing breadcrumbs (#2885)
Refs #1902
2026-09-15 12:45:01 -07:00
Simon Willison
ee5b5728bc Fix some broken links in docs
Refs #2912
2026-09-15 12:31:55 -07:00
Simon Willison
5827feff74 Fix old broken link in Changelog
Refs #2912
2026-09-15 12:20:47 -07:00
Aurora
d60d88488f
docs: fix dead Starlette, Uvicorn and Docker Hub links (#2912)
www.starlette.io and www.uvicorn.org no longer resolve (NXDOMAIN); both
projects now publish their documentation at starlette.dev and uvicorn.dev
respectively. The Docker Hub link was missing the /r/ path segment, which
returns a 404.

- docs/plugin_hooks.rst: www.starlette.io/middleware/ -> starlette.dev/middleware/
- docs/changelog.rst (x3): www.uvicorn.org/ -> uvicorn.dev/
- docs/changelog.rst: hub.docker.com/datasetteproject/... -> hub.docker.com/r/datasetteproject/...

All four replacement URLs verified to return HTTP 200.
2026-09-15 11:57:39 -07:00
Alex Garcia
374b194ff5
Add /-/tasks introspection endpoint for supervised background tasks (#2892)
Co-authored-by: Simon Willison <swillison@gmail.com>
2026-09-15 11:56:53 -07:00
Alex Garcia
784695aea6 Ensure startup() runs before any plugin ASGI middleware (#2891) 2026-09-15 11:22:45 -07:00
Alex Garcia
cca08d2886 Add shutdown() plugin hook with ordered graceful teardown (#2890) 2026-09-15 11:09:57 -07:00
Alex Garcia
0c31778580 Add datasette.add_background_task() with supervised launch after startup (#2889) 2026-09-15 10:55:54 -07:00
Simon Willison
1538832830 /db/table/-/count? endpoint, fixed 'count all' button
Closes #2914
2026-09-15 10:03:17 -07:00
Simon Willison
b338c6f5f6
Migrate from httpx to httpx2, closes #2879
https://claude.ai/code/session_01Xdqoneq8ddvruVZETo6rFf
2026-09-10 19:44:49 -07:00
Simon Willison
5e7cdaabbd Release 1.0a39 2026-09-10 16:52:52 -07:00
Simon Willison
92c7d4b608 Limit derived-table permissions to one source hop
Simplify the solution to 5de0c1724e - avoid contextvar.
2026-09-10 16:52:52 -07:00
Simon Willison
b97bb5f016 Reconcile write-timeout regression with the per-call limit 2026-09-10 16:52:26 -07:00
Simon Willison
e036907fc3 Reject structured row writes to virtual and shadow tables 2026-09-10 16:52:25 -07:00
Simon Willison
3f8d8417f6 Inherit source permissions for FTS vocabulary tables 2026-09-10 16:52:25 -07:00
Simon Willison
d334539a1e Deny SQLite statistics table access through a default hook 2026-09-10 16:52:25 -07:00
Simon Willison
506c4bb522 Match table permission identities using SQLite case semantics 2026-09-09 08:39:03 -07:00
Simon Willison
ac2a9a43a5 Add a default execution time limit to writes 2026-09-08 21:17:03 -07:00
Simon Willison
ceef351622 Protect personalized dynamic responses from shared caching 2026-09-08 21:16:36 -07:00
Simon Willison
6aa58bf4e5 Authorize configured full-text search targets 2026-09-08 21:16:35 -07:00
Simon Willison
6473a7ecb0 Clearly document relationship between execute-sql and facets
Refs GHSA-5fff-xcm9-q6vh

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:49:52 -07:00
Simon Willison
12b25affb5 Release 1.0a38
Refs #2868
2026-08-06 11:20:25 -07:00
Simon Willison
e889403d3b
Upgrade to ruff>=0.16.0 (#2857)
* ruff>=0.16.0

See https://astral.sh/blog/ruff-v0.16.0

* uv run ruff check . --fix --unsafe-fixes

* Ruff fixes by Claude Code Opus 5
2026-07-25 15:47:08 -07:00
Simon Willison
481df7ff6d Shorten link text in changelog 2026-07-14 09:31:28 -07:00
Simon Willison
2ffd8a860e Release 1.0a37
Refs #2831, #2832, #2841, #2842, #2843, #2846
2026-07-14 09:28:29 -07:00
Simon Willison
7f0a8b38ae
Better permission debug tools and documentation
Closes #2841
2026-07-14 08:40:07 -07:00
Simon Willison
10088dfa1d
execute_write(transaction=False) parameter, plus fix for errors inside tasks
Ensure a write inside a failing Datasette task never becomes visible. Refs #2831
2026-07-13 22:42:44 -07:00
Simon Willison
ccace40e5a
/-/plugins.json is now an array of objects again (#2843)
Reverts the object envelope introduced in 1.0a36 for this endpoint -
it once again returns a top-level JSON array of plugin objects.

Closes #2842


Claude-Session: https://claude.ai/code/session_012TYc1NTBK4zEjabB3u2zqu

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-13 21:19:04 -07:00
Simon Willison
52ae7d1b6d Release 1.0a36
Refs
#1983, #1996, #2783, #2806, #2809, #2811, #2812, #2813, #2815, #2818, #2819, #2822, #2823, #2827
2026-07-07 14:32:25 -07:00
Simon Willison
a31673c90b Changelog for #2811, #2815, #2783 2026-07-07 14:26:48 -07:00
Simon Willison
617acedd38 Remove readthedocs/actions/preview
Closes #2828
2026-07-07 14:18:10 -07:00
Simon Willison
7f37205e76 Remove Datasette Desktop from installation guide
Until I have time to fix it up and bring it back.
2026-07-07 14:04:46 -07:00
Simon Willison
a926ab392e Updated internals.rst schema using cog, refs #2827 2026-07-07 14:02:02 -07:00
Claude
b7bbde04be
Link the consistency review release note to PR #2824
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00