Commit graph

1,376 commits

Author SHA1 Message Date
Claude
6488b7a30e
Remove duplicate params key from stored query JSON objects
Every stored-query object carried the same list of parameter names
twice, as both "params" and "parameters". Output objects now carry only
"parameters", consistent with /-/query/parameters and the two analyze
endpoints (and distinct from the "params" bound-values dictionary used
by the query extra and /-/execute-write). "params" remains an accepted
input alias for query creation, update and datasette.yaml config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 16:00:25 +00:00
Claude
f3f5e891c9
Block API deletion of trusted stored queries
QueryUpdateView already rejected is_trusted queries but QueryDeleteView
did not, so an actor with delete-query could delete a config-defined
trusted query - which would then silently reappear on restart when the
config re-syncs. Both the POST endpoint and the HTML confirmation page
now return 403, matching update. datasette.remove_query() is unchanged
for internal use.

The docs already claimed this behavior ("Trusted stored queries cannot
be edited or deleted through the web interface or the JSON API") - the
code now matches them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 15:49:49 +00:00
Claude
ea9c1b1524
Return 400 for query data formats when ?sql= is missing
GET /db/-/query.json with no (or blank) ?sql= previously returned 200
with empty rows, masking caller bugs, while the .csv format returned
400 "?sql= is required" for the same request. All data formats now
return the 400; the HTML SQL editor page is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 15:39:11 +00:00
Claude
aaaffe45b8
Return 401 for invalid or expired bearer tokens
Invalid dstok_ tokens - bad signature, malformed payload, expired, or
presented while allow_signed_tokens is off - previously degraded the
request to anonymous, so clients saw a 403 permission error or worse,
a 200 with anonymous-visible data. Token handlers can now raise
TokenInvalid for tokens they recognize but reject; Datasette responds
with 401, the canonical JSON error body and a WWW-Authenticate: Bearer
error="invalid_token" header, even when a valid cookie is also present.

Bearer tokens no registered handler recognizes are still ignored, so
authentication plugins with their own token formats keep working.
TokenInvalid is exported from the datasette package for use by plugin
token handlers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 15:18:12 +00:00
Claude
b2cdc81d34
Return 400 instead of 500 for row delete write failures
Row delete previously returned 500 when the write failed (for example
a constraint violation raised by a trigger or foreign key), while row
update and every other write endpoint report the same failure class as
400. Delete now matches.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:55:41 +00:00
Claude
e8048e023f
Return 400 for write canned-query SQL failures
POST to a write canned query previously returned HTTP 200 with
{"ok": false, "message": ...} when the SQL failed to execute, so JSON
clients (and anything that trusts HTTP status) recorded success for
failed writes. SQL failures now return 400 with the canonical error
shape plus the "redirect" context key from on_error_redirect; the
QueryWriteRejected 403 branch uses the canonical shape too. Successful
executions and the HTML flash-message flow are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:35:54 +00:00
Claude
ae10a99811
Return canonical JSON error for Forbidden on JSON requests
The default forbidden() hook previously rendered an HTML error page even
for .json requests. It now returns the canonical JSON error shape with
status 403 when the request path ends in .json or the request sends an
Accept: application/json or Content-Type: application/json header. HTML
requests still get the error page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:09:10 +00:00
Claude
f091b6dab1
Filter /-/databases by view-database permission
/-/databases previously listed every attached database (including
filesystem paths and sizes) to any actor with view-instance, while the
homepage and every other endpoint filtered by view-database. The
endpoint now only lists databases the current actor is allowed to view.

JsonDataView data callbacks may now be async.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:00:42 +00:00
Claude
23ccdaeffc
Convert /-/actions.json from top-level array to object
/-/actions.json now returns {"ok": true, "actions": [...]} instead of a
bare JSON array, so the response can grow additional keys without a
breaking change. The debug_actions.html template reads data.actions,
and the endpoint is now documented in docs/introspection.rst.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:46:26 +00:00
Claude
19e54b10d4
Convert /-/databases.json from top-level array to object
/-/databases.json now returns {"ok": true, "databases": [...]} instead
of a bare JSON array, so the response can grow additional keys without
a breaking change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:43:44 +00:00
Claude
b74a8e5b12
Convert /-/plugins.json from top-level array to object
/-/plugins.json now returns {"ok": true, "plugins": [...]} instead of a
bare JSON array, so the response can grow additional keys without a
breaking change. The `datasette plugins` CLI command still outputs a
plain array.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:42:24 +00:00
Claude
089e96a437
Add "ok": true to every JSON object success response
JsonDataView now injects "ok": true into dict responses, covering
/-/versions, /-/settings, /-/config, /-/threads and /-/actor. The
homepage JSON, /-/jump, the three /-/schema endpoints, /-/allowed,
/-/rules, /-/check, POST /-/permissions and the table /-/autocomplete
endpoint set it explicitly.

The remaining top-level array endpoints (/-/plugins, /-/databases,
/-/actions) will be converted to objects in separate commits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:40:05 +00:00
Claude
0679e04bd3
Unify JSON error responses into one canonical shape
All JSON error responses now use a single format built by the new
datasette.utils.error_body() helper:

    {"ok": false, "error": "...", "errors": ["..."], "status": 400}

- error is all messages joined with '; ', errors is the full list,
  status always matches the HTTP status code
- The exception handler no longer emits the legacy title key in JSON
  (it is still available to the HTML error template)
- The permission debug endpoints (/-/allowed, /-/rules, /-/check,
  POST /-/permissions) no longer return bare {"error": ...} objects
- JSON renderer SQL errors keep their rows/truncated context keys but
  now include the canonical keys as well
- _shape=object misuse (queries or tables without primary keys) now
  returns HTTP 400 instead of 200 with an error body
- Method-not-allowed 405 responses use the canonical shape

Adds tests/test_error_shape.py covering all four previous shape
producers, updates affected tests, and documents the format in a new
'Error responses' section of docs/json_api.rst.

Implements section 1 of stable-api-recommendations.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 03:12:15 +00:00
Simon Willison
34ab85e664
Handle recursive CTEs in query analysis
Closes #2809
2026-06-29 11:36:54 -07:00
Ritesh Kewlani
463eea2bd0
Fix _col=<pk> producing duplicate column in output (#2774)
Closes #1975
2026-06-23 14:18:49 -07:00
Simon Willison
22ccd8a087 escape_sqlite() favors double quotes, closes #2795 2026-06-23 14:04:20 -07:00
Simon Willison
5eca46a4bc
Add cache-busted static asset helper (#2804)
* Add cache-busted static asset helper

Add a static() helper for Datasette, plugin, and mounted static assets that appends content-based hashes, caches hashes in production, and serves matching hashed asset URLs with immutable far-future cache headers.

Closes #2800
2026-06-23 13:44:58 -07:00
Simon Willison
0c523dda20 Remove count truncated context test 2026-06-23 12:29:21 -07:00
Simon Willison
34d9a3bf33 Use dataclasses for database table context 2026-06-23 12:11:26 -07:00
Simon Willison
59ab0c0ca0 Clarify template context metadata names 2026-06-23 11:30:30 -07:00
Simon Willison
4d031c8562 Add count_truncated template context 2026-06-22 19:47:21 -07:00
Simon Willison
49b1adba7b Merge remote-tracking branch 'origin/main' into template-context-docs
# Conflicts:
#	datasette/views/row.py
2026-06-22 18:44:37 -07:00
Simon Willison
f831352551 Avoid SQLite RETURNING for compat with older SQLite
Refs #2789
2026-06-22 14:24:34 -07:00
Simon Willison
b932d0dc78 current_unixtime and current_unixtime_ms default_expr options
Plus tweaked how alter table changing those works a bit.
2026-06-22 13:42:35 -07:00
Simon Willison
2ebae5ed71 Add rename table controls to alter table dialog
Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow.

Redirect to the renamed table URL after applying changes and cover the review text in Playwright.
2026-06-22 12:51:23 -07:00
Simon Willison
fa43aba309 Unify create and alter table modal controls
Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend.

Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.
2026-06-22 12:50:57 -07:00
Simon Willison
063b04ad83 Expose foreign key data to alter table UI
Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table.

Add API and HTML data tests for the new alter-table foreign key support.
2026-06-22 12:47:02 -07:00
Simon Willison
dada4de172 Fix for Safari select box heights
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4772241681
2026-06-22 12:35:34 -07:00
Simon Willison
4b219be8bd Alter table API can now rename tables, refs #2788
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4771774289
2026-06-22 12:09:07 -07:00
Simon Willison
87354cf94e not_null, default and default_exr support for create table API columns 2026-06-22 11:04:19 -07:00
Simon Willison
084df1fba2 Removed the alter table dry run feature
It works by doing conn.backup(memory_conn) which could use
a lot of memory for a large database.
2026-06-22 10:18:01 -07:00
Simon Willison
b02999bca6 Fix broken Playwright tests 2026-06-22 10:11:56 -07:00
Simon Willison
354780a136 Drop table button in alter dialog 2026-06-22 10:11:56 -07:00
Simon Willison
c77dad910b More robust test_datasette_https_server.sh test 2026-06-22 10:11:56 -07:00
Simon Willison
21c156dfb1 Expose foreign key targets to create table UI
- Add foreignKeyTargetsPath to create table page data
- Filter hidden tables from database-level foreign key target results
- Update JSON API docs and tests for filtered targets
2026-06-22 10:11:56 -07:00
Simon Willison
a6ef65f90d /<database>/-/foreign-key-targets API endpoint
Returns a list of tables with a single primary key, and for each one
the name of that primary key column and its SQLite type affinity.

This will be used by the create table UI to suggest foreign keys.
2026-06-22 10:11:56 -07:00
Simon Willison
2900efb32d /db/table/-/foreign-key-suggestions API
Improved version of the implementation datasette-edit-schema
2026-06-22 10:11:56 -07:00
Simon Willison
9d9a2d3ff3 Add foreign keys to alter table API
- Add add_foreign_key, drop_foreign_key, and set_foreign_keys operations.
- Validate flat fk_table and fk_column arguments with Pydantic.
- Document the API and cover inferred primary-key and validation cases.
2026-06-22 10:11:56 -07:00
Simon Willison
9766a9c087 Add foreign keys to create table API
- Add fk_table and optional fk_column support to create-table columns.
- Validate create-table requests with Pydantic while preserving existing errors.
- Document the API and cover inferred primary-key and validation cases.
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733544452
2026-06-22 10:11:56 -07:00
Simon Willison
4115213e17 Precompute action permissions for table pages
- Extract reusable helpers for database and table action permission preloading.
- Precompute those permissions before building table-page HTML data.
- Document the default table actions plugin.
2026-06-22 10:11:56 -07:00
Simon Willison
c9c79fdfc8 Isolate Unix domain socket test server paths
- Use a per-process socket path for the UDS test fixture.
- Clean up stale socket files before and after the fixture runs.
- Close the HTTP client and wait for the Datasette subprocess to exit.
2026-06-22 10:11:56 -07:00
Simon Willison
fdd1b61a3e Add alter table modal
- Register a built-in table action and expose alter-table metadata to table pages.
- Build the client-side modal for editing columns, defaults, ordering, primary keys, and custom column types.
- Add a review/apply confirmation flow with HTML and Playwright coverage.

Refs #2788
2026-06-22 10:11:56 -07:00
Simon Willison
b40665dd14 Add alter table JSON API
- Add POST /<database>/<table>/-/alter with Pydantic validation and dry-run support.
- Support add, rename, alter, drop, primary-key and reorder operations, including allow-listed default expressions.
- Document the endpoint and cover schema changes, validation, permissions, events and dry runs.

Refs #2788
2026-06-22 10:11:56 -07:00
Simon Willison
2d3c85dfc0 Add create table UI
Adds a permission-gated database action that opens a create table modal on database pages, backed by the existing create-table JSON API.

The modal starts with an id integer primary key column plus a blank text column, supports SQLite type selection, and shows custom column type controls only when the actor can set column types.

Selected custom column types are applied after table creation with follow-up set-column-type API calls. Includes styling plus HTML and Playwright coverage for the action payload and create-table flow.
2026-06-22 10:11:56 -07:00
Simon Willison
387f4dd4bc Try sqlite-utils 4.0rc1 in CI
Also output current sqlite-utils version in pytest headers

Refs https://github.com/simonw/sqlite-utils/issues/758
2026-06-21 16:46:20 -07:00
Simon Willison
ad6fe47a95 Better way of setting min height on CodeMirror
Previously there were four lines of whitespace, but that
meant users had to delete that whitespace themselves when
they started editing. Now it is four lines tall without that.
2026-06-19 22:40:37 -07:00
Simon Willison
f673e7416f
Add create table execute-write template (#2794)
Add a Create table starter template to the execute-write page, alongside the existing table-specific insert/update/delete templates.

Keep template button clicks in-page by updating CodeMirror and the URL without collapsing the template controls.

Refresh schema metadata after successful schema-changing execute-write statements so newly created tables appear immediately.
2026-06-19 17:14:33 -07:00
Simon Willison
8c700f5337 Renamed base_url parameter to ds_base_url in tests
Integrating Playwright brought in pytest-base-url which declares a
base_url fixture that was colliding with our own base_url parameters.
2026-06-16 14:25:42 -07:00
Simon Willison
3edc98a871 Playwright tests exercise the real UI now
Refs https://github.com/simonw/datasette/pull/2785#issuecomment-4723583959
2026-06-16 14:13:44 -07:00
Simon Willison
a8124172dd Remove load_edit_tools(), use a new fixture instead
Refs https://github.com/simonw/datasette/pull/2785/changes#r3423913454
2026-06-16 13:54:00 -07:00