Commit graph

1,442 commits

Author SHA1 Message Date
Claude
e8048e023f
Return 400 for write canned-query SQL failures
POST to a write canned query previously returned HTTP 200 with
{"ok": false, "message": ...} when the SQL failed to execute, so JSON
clients (and anything that trusts HTTP status) recorded success for
failed writes. SQL failures now return 400 with the canonical error
shape plus the "redirect" context key from on_error_redirect; the
QueryWriteRejected 403 branch uses the canonical shape too. Successful
executions and the HTML flash-message flow are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:35:54 +00:00
Claude
ae10a99811
Return canonical JSON error for Forbidden on JSON requests
The default forbidden() hook previously rendered an HTML error page even
for .json requests. It now returns the canonical JSON error shape with
status 403 when the request path ends in .json or the request sends an
Accept: application/json or Content-Type: application/json header. HTML
requests still get the error page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:09:10 +00:00
Claude
f091b6dab1
Filter /-/databases by view-database permission
/-/databases previously listed every attached database (including
filesystem paths and sizes) to any actor with view-instance, while the
homepage and every other endpoint filtered by view-database. The
endpoint now only lists databases the current actor is allowed to view.

JsonDataView data callbacks may now be async.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 14:00:42 +00:00
Claude
23ccdaeffc
Convert /-/actions.json from top-level array to object
/-/actions.json now returns {"ok": true, "actions": [...]} instead of a
bare JSON array, so the response can grow additional keys without a
breaking change. The debug_actions.html template reads data.actions,
and the endpoint is now documented in docs/introspection.rst.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:46:26 +00:00
Claude
19e54b10d4
Convert /-/databases.json from top-level array to object
/-/databases.json now returns {"ok": true, "databases": [...]} instead
of a bare JSON array, so the response can grow additional keys without
a breaking change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:43:44 +00:00
Claude
b74a8e5b12
Convert /-/plugins.json from top-level array to object
/-/plugins.json now returns {"ok": true, "plugins": [...]} instead of a
bare JSON array, so the response can grow additional keys without a
breaking change. The `datasette plugins` CLI command still outputs a
plain array.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:42:24 +00:00
Claude
089e96a437
Add "ok": true to every JSON object success response
JsonDataView now injects "ok": true into dict responses, covering
/-/versions, /-/settings, /-/config, /-/threads and /-/actor. The
homepage JSON, /-/jump, the three /-/schema endpoints, /-/allowed,
/-/rules, /-/check, POST /-/permissions and the table /-/autocomplete
endpoint set it explicitly.

The remaining top-level array endpoints (/-/plugins, /-/databases,
/-/actions) will be converted to objects in separate commits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:40:05 +00:00
Claude
0679e04bd3
Unify JSON error responses into one canonical shape
All JSON error responses now use a single format built by the new
datasette.utils.error_body() helper:

    {"ok": false, "error": "...", "errors": ["..."], "status": 400}

- error is all messages joined with '; ', errors is the full list,
  status always matches the HTTP status code
- The exception handler no longer emits the legacy title key in JSON
  (it is still available to the HTML error template)
- The permission debug endpoints (/-/allowed, /-/rules, /-/check,
  POST /-/permissions) no longer return bare {"error": ...} objects
- JSON renderer SQL errors keep their rows/truncated context keys but
  now include the canonical keys as well
- _shape=object misuse (queries or tables without primary keys) now
  returns HTTP 400 instead of 200 with an error body
- Method-not-allowed 405 responses use the canonical shape

Adds tests/test_error_shape.py covering all four previous shape
producers, updates affected tests, and documents the format in a new
'Error responses' section of docs/json_api.rst.

Implements section 1 of stable-api-recommendations.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 03:12:15 +00:00
Simon Willison
58c07cc264 max_post_body_bytes setting, enforced for reuest.post_body()
Closes #2823
2026-07-03 17:42:39 -07:00
Simon Willison
7ab8b644a7 Merge remote-tracking branch 'origin/main' into edit-blobs
# Conflicts:
#	docs/changelog.rst
#	tests/test_playwright.py
2026-07-03 17:25:41 -07:00
Simon Willison
81d6ee69cd Allow declared column type case to vary 2026-07-03 17:12:26 -07:00
Simon Willison
19dde1c860 Support BLOB values in row edit UI 2026-07-03 16:09:27 -07:00
Simon Willison
b476218edb Refine column_details metadata shape 2026-07-03 16:08:34 -07:00
Simon Willison
8856914be8 Add column_details JSON extra 2026-07-03 14:45:38 -07:00
Simon Willison
3b24c88e93 Support editing BLOBs through JSON APIs 2026-07-03 14:05:45 -07:00
Simon Willison
c9975c1fe1
Bulk row inserts / create table from data
Merge pull request #2813 from simonw/bulk-insert
2026-07-03 13:07:42 -07:00
Simon Willison
2bcabd8e1f Harden Playwright server startup wait
Allow the Datasette subprocess more time to become reachable under CI load.

If startup still times out, terminate the process and include stdout and stderr in the assertion so future failures have useful diagnostics.
2026-07-03 12:57:36 -07:00
Simon Willison
8f32a8f134 Fix bulk insert Playwright expectations
Keep copied spreadsheet templates aligned with the single-row insert columns while still previewing auto primary keys in bulk insert rows.

Treat invalid JSON during live bulk-insert validation as unfinished input so Firefox does not expose browser-specific parse errors while typing.
2026-07-03 12:47:22 -07:00
Simon Willison
3aea678eab Show auto primary keys in bulk insert preview
Render omitted auto integer primary key values as muted auto text in the bulk insert preview so generated IDs are easier to understand.

Expose the auto primary key flag in insert UI metadata and cover the preview behavior in Playwright.
2026-07-03 12:27:02 -07:00
Simon Willison
c864bc866d Add bulk insert upsert controls
Expose conflict handling in the bulk insert UI when pasted rows include primary key columns, and route update-or-insert selections through the existing upsert API.

Add live validation for bulk textarea column errors and Playwright coverage for upsert permissions, conflict options, and validation behavior.

Refs shttps://github.com/simonw/datasette/pull/2813#issuecomment-4878320713
2026-07-03 10:44:36 -07:00
Simon Willison
141fe194c6 Hide create-from-data UI without insert permission
Expose whether the actor can insert rows in the create-table page data, then use that flag to hide and guard the create-table-from-data path. This avoids offering a flow that fails after preview for create-table-only actors.

Refs https://github.com/simonw/datasette/pull/2813#issuecomment-4878146376
2026-07-03 09:56:09 -07:00
Simon Willison
4a39e44eb6 Preserve numeric inference for blank CSV values
Convert blank delimited cells to null when create-table-from-data inferred an integer or float column. This keeps sqlite-utils from treating mixed numeric and empty-string values as text.

Add a Playwright regression covering CSV table creation with a blank numeric value.
2026-07-03 09:53:57 -07:00
Simon Willison
4bd9d41c43 Preserve SQLite defaults in bulk insert
Omit columns that are absent from pasted bulk insert data so SQLite defaults can fire during insert. Keep preview cells blank for omitted values while preserving explicit null rendering.

Refs [https://github.com/simonw/datasette/pull/2813#issuecomment-4878063728](https://github.com/simonw/datasette/pull/2813#issuecomment-4878063728)
2026-07-03 09:45:34 -07:00
Simon Willison
2f84ab77f2
Support CREATE VIEW / DROP VIEW in execute-write-sql
New create-view and drop-view actions.

Also fix a related bug in analyze_sql_tables(): SQLite's authorizer fires a spurious SQLITE_DELETE callback against the view name when a view is dropped (the same thing it does for dropped tables), which was incorrectly surfaced as a delete-row requirement on the view. Broaden the existing drop-table-delete suppression to cover dropped views too.

Closes #2819
2026-07-02 08:56:04 -07:00
Simon Willison
b759ea5486 Mock navigator.clipboard so Playwright passes in all three browsers 2026-06-29 17:38:51 -07:00
Simon Willison
9ec42b2dad Improvements to create table from data 2026-06-29 16:02:56 -07:00
Simon Willison
e0a138ffbd Tweaks to bulp insert screen
Moved text around, added clickable 'open a file' button for mobile.

Screenshot: https://github.com/simonw/datasette/pull/2813#issuecomment-4837736664
2026-06-29 15:17:19 -07:00
Simon Willison
34ab85e664
Handle recursive CTEs in query analysis
Closes #2809
2026-06-29 11:36:54 -07:00
Simon Willison
7ae601588e Create table from data feature
Similar to insert rows from JSON/CSV/TSV
but it creates a brand new table instead.
2026-06-25 22:32:47 -07:00
Simon Willison
b7a896a803 Insert multiple rows feature in the insert dialog
Works entirely using existing JSON API.

User can paste in CSV, TSV or JSON (or drop a text file on).

They get a preview of the rows to be imported, then they
are imported in max_insert_rows batches.
2026-06-25 22:02:39 -07:00
Ritesh Kewlani
463eea2bd0
Fix _col=<pk> producing duplicate column in output (#2774)
Closes #1975
2026-06-23 14:18:49 -07:00
Simon Willison
22ccd8a087 escape_sqlite() favors double quotes, closes #2795 2026-06-23 14:04:20 -07:00
Simon Willison
5eca46a4bc
Add cache-busted static asset helper (#2804)
* Add cache-busted static asset helper

Add a static() helper for Datasette, plugin, and mounted static assets that appends content-based hashes, caches hashes in production, and serves matching hashed asset URLs with immutable far-future cache headers.

Closes #2800
2026-06-23 13:44:58 -07:00
Simon Willison
0c523dda20 Remove count truncated context test 2026-06-23 12:29:21 -07:00
Simon Willison
34d9a3bf33 Use dataclasses for database table context 2026-06-23 12:11:26 -07:00
Simon Willison
59ab0c0ca0 Clarify template context metadata names 2026-06-23 11:30:30 -07:00
Simon Willison
4d031c8562 Add count_truncated template context 2026-06-22 19:47:21 -07:00
Simon Willison
49b1adba7b Merge remote-tracking branch 'origin/main' into template-context-docs
# Conflicts:
#	datasette/views/row.py
2026-06-22 18:44:37 -07:00
Simon Willison
f831352551 Avoid SQLite RETURNING for compat with older SQLite
Refs #2789
2026-06-22 14:24:34 -07:00
Simon Willison
b932d0dc78 current_unixtime and current_unixtime_ms default_expr options
Plus tweaked how alter table changing those works a bit.
2026-06-22 13:42:35 -07:00
Simon Willison
2ebae5ed71 Add rename table controls to alter table dialog
Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow.

Redirect to the renamed table URL after applying changes and cover the review text in Playwright.
2026-06-22 12:51:23 -07:00
Simon Willison
fa43aba309 Unify create and alter table modal controls
Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend.

Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.
2026-06-22 12:50:57 -07:00
Simon Willison
063b04ad83 Expose foreign key data to alter table UI
Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table.

Add API and HTML data tests for the new alter-table foreign key support.
2026-06-22 12:47:02 -07:00
Simon Willison
dada4de172 Fix for Safari select box heights
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4772241681
2026-06-22 12:35:34 -07:00
Simon Willison
4b219be8bd Alter table API can now rename tables, refs #2788
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4771774289
2026-06-22 12:09:07 -07:00
Simon Willison
87354cf94e not_null, default and default_exr support for create table API columns 2026-06-22 11:04:19 -07:00
Simon Willison
084df1fba2 Removed the alter table dry run feature
It works by doing conn.backup(memory_conn) which could use
a lot of memory for a large database.
2026-06-22 10:18:01 -07:00
Simon Willison
b02999bca6 Fix broken Playwright tests 2026-06-22 10:11:56 -07:00
Simon Willison
354780a136 Drop table button in alter dialog 2026-06-22 10:11:56 -07:00
Simon Willison
c77dad910b More robust test_datasette_https_server.sh test 2026-06-22 10:11:56 -07:00