Commit graph

3,403 commits

Author SHA1 Message Date
Alex Garcia
343e8258fb Add datasette-sql-editor ESM primitives module
Single source of truth for the CodeMirror setup: SQLiteDialect,
createSqlEditor() (delegable history with host undo/redo forwarding,
hostChange annotation for echo suppression, submit/escape callbacks,
fixed-tooltip mode, per-editor Compartment updateSchema), and
datasetteSchema() which fetches /-/editor-schema.json and maps it to a
lang-sql SQLNamespace identical to the server-inlined shape.
cm-editor.js is now a thin consumer; rollup emits both the IIFE and an
importable ESM bundle. Submit key is Mod-Enter (Cmd on mac as before,
now also Ctrl elsewhere) plus Shift-Enter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 11:07:21 -07:00
Alex Garcia
174099b707 Update test_execute_sql schema assertions for rich completion shape
Follow-up to d12f0d2c: the test regexes the inlined schema= JS and
still asserted the old flat list-of-strings shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 11:01:38 -07:00
Alex Garcia
cc1a24fb4f Pass defaultTable to the SQL editor from table-scoped pages
The table page's 'View and edit SQL' link now carries ?_table=<name>;
QueryView validates it against the actor-visible tables/views for the
database before exposing it as default_table, so the editor completes
that table's columns unprefixed. Stored/canned queries are unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:59:23 -07:00
Alex Garcia
49f1660dcd Document DatabaseEditorSchemaView label for docs coverage test
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:59:23 -07:00
Alex Garcia
b9716d4278 Add /{database}/-/editor-schema.json endpoint for SQL editor consumers
Neutral {database, tables: [{name, view, columns: [{name, type}]}]}
shape, gated on view-database + execute-sql with no table-name leak on
403, hidden tables excluded. /-/schema.json was already taken by the
DDL endpoint, hence editor-schema.json. _editor_schema() now maps from
the shared _schema_tables() introspection helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:56:21 -07:00
Alex Garcia
7e555b01f6 SQL editor: dynamic schema updates via Compartment (editor.updateSchema)
Per-view Compartment wraps the sql() extension; window.editor.updateSchema(
{schema, defaultTable, defaultSchema}) reconfigures autocomplete live.
Declares @codemirror/state as a direct dependency since it is now
imported directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:47:47 -07:00
Alex Garcia
d12f0d2c59 Rich SQL editor completions: column types, view columns, ranking
_editor_schema() emits lang-sql Completion objects (column type as
detail, boost above keywords) and gives views their real columns via
PRAGMA table_xinfo in a self/children container labelled 'view'.
_table_columns() is unchanged for the write-template path. Note the
table_columns field in the database JSON context now carries this
richer shape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:46:55 -07:00
Alex Garcia
28d811320b Version-agnostic CodeMirror bundle filenames, rollup.config.mjs build
cm-editor-6.0.1.{js,bundle.js} -> cm-editor.{js,bundle.js}; new
npm run build:codemirror replaces the documented rollup one-liner.
Cache busting already handled by the static() template helper's
?_hash= content hash.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:44:58 -07:00
Alex Garcia
79f6ac3f47 Upgrade CodeMirror to latest 6.x, fix sql() option names
codemirror 6.0.1 -> 6.0.2, @codemirror/lang-sql 6.3.3 -> 6.10.0
(autocomplete 6.20.3, view 6.43.6, state 6.7.1).
defaultTableName/defaultSchemaName were never valid SQLConfig options;
the real names are defaultTable/defaultSchema. Also enables
caseInsensitiveIdentifiers on the SQLite dialect.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 10:39:42 -07:00
Simon Willison
db82123108 Bump a whole lot of GitHub Actions versions 1.0a36 2026-07-07 14:40:33 -07:00
Simon Willison
52ae7d1b6d Release 1.0a36
Refs
#1983, #1996, #2783, #2806, #2809, #2811, #2812, #2813, #2815, #2818, #2819, #2822, #2823, #2827
2026-07-07 14:32:25 -07:00
Simon Willison
a31673c90b Changelog for #2811, #2815, #2783 2026-07-07 14:26:48 -07:00
Simon Willison
54597f22fa A few more SQLite string fixes, refs #2783 2026-07-07 14:26:34 -07:00
JSap0914
bf3e277c98
Fix named_parameters when string literals contain comment markers (#2783)
named_parameters stripped SQL comments before string literals in
separate passes. A string literal such as '-- TODO' would be treated
as the start of a line comment, swallowing the rest of the line and
hiding any named parameters that followed it. For example:

    select * from t where note = '-- TODO' and id = :id

returned [] instead of ['id'], so the query parameter input form
would be missing the :id field.

Match comments and string literals in a single left-to-right pass so
that whichever construct starts first wins, matching how SQL is
actually tokenized.

Co-authored-by: JSap0914 <JSap0914@users.noreply.github.com>
2026-07-07 14:23:31 -07:00
Zain Dana Harper
211e70d4e1
Return 400 not 500 for wrong-arity composite-PK row URLs (#2815)
Fixes #2811

Co-authored-by: Zain Dana Harper <zain@aurora-framework.dev>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 14:19:08 -07:00
Simon Willison
617acedd38 Remove readthedocs/actions/preview
Closes #2828
2026-07-07 14:18:10 -07:00
Simon Willison
7f37205e76 Remove Datasette Desktop from installation guide
Until I have time to fix it up and bring it back.
2026-07-07 14:04:46 -07:00
Simon Willison
a926ab392e Updated internals.rst schema using cog, refs #2827 2026-07-07 14:02:02 -07:00
Simon Willison
db7ba1d30c Switch to sqlite-utils migrations for internal.db, closes #2827 2026-07-07 13:58:48 -07:00
Simon Willison
96e8b85523 Upgrade to sqlite-utils 4.0 2026-07-07 13:57:06 -07:00
Simon Willison
6f27aa112a
Test against sqlite-utils>=4.0
https://github.com/simonw/sqlite-utils/issues/769
2026-07-07 12:03:28 -07:00
Simon Willison
d2695a0c2f
Test Datasette against sqlite-utils>=4.0rc4
Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4900497417
2026-07-06 22:43:04 -07:00
Simon Willison
ebd013c6ef Bump GitHub Actions versions 2026-07-06 22:38:57 -07:00
Simon Willison
27a5be1326
Fable review of JSON API consistency and subsequent improvements
Merge PR #2824
2026-07-06 22:30:19 -07:00
Claude
b7bbde04be
Link the consistency review release note to PR #2824
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00
Claude
be25d6e3e4
Remove test_query_list_json_signals_pagination_via_next_only
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00
Claude
4a853cb10c
Use UNSTABLE_API_MESSAGE constant in tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00
Claude
57ce1a059f
Tighten unstable marker release note
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 01:13:33 +00:00
Claude
b83b12dd7a
Remove params input alias from the query create and update APIs
The alias existed so API payloads could mirror the params key used by
queries defined in datasette.yaml, but it was undocumented and untested,
and the create endpoint is not part of the stable API. The API now only
accepts parameters - sending params is a 400 Invalid keys error. The
documented params key for queries in configuration is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 01:13:33 +00:00
Claude
b23fc4ec48
Unreleased release notes for the JSON API consistency review
Documents the canonical error format, the ok/envelope changes, the
array-to-object endpoint conversions, 401s for invalid tokens, the
pagination and page-size unification, removed legacy keys and formats,
and the new Response.error(), TokenInvalid, count_truncated and
unstable-marker APIs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 00:32:29 +00:00
Claude
610c24d59a
/-/jump is a stable documented endpoint, not a debug exemption
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 00:25:09 +00:00
Claude
4874c29286
Remove the next_url extra - the key is always present
next_url became a default table JSON key alongside next, making the
extra a no-op. Requesting ?_extra=next_url now returns the standard
unknown-extra 400 error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:59:53 +00:00
Claude
8b159144a5
Add Response.error() for JSON errors in the standard format
Response.error(messages, status=400) builds a JSON error response in
Datasette's standard error format, alongside Response.json/html/text.
messages can be a single string or a list. All internal error response
construction now uses it - the private views.base._error() helper is
gone and the verbose Response.json(error_body(...), status=...) sites
are converted. error_body() remains for the cases that merge the error
keys into a larger payload (the JSON renderer, handle_exception and the
permission debug payload builders).

Since Response is public plugin API, plugins that build JSON endpoints
now have an obvious way to return errors in the canonical shape.
Documented in the internals documentation, including the guidance to
raise Forbidden/NotFound/BadRequest/DatasetteError instead when the
error should content-negotiate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:48:49 +00:00
Claude
53ccca5e15
Remove working analysis documents
existing-api.md and stable-api-recommendations.md were working
documents for the 1.0 API consistency review. Their content remains
available in this branch history; they are not intended to merge to
main.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:34:17 +00:00
Claude
0d962deb05
Plain text SQL Interrupted errors in JSON responses
The SQL time limit error embedded an HTML fragment (paragraph, textarea
and script tags) as the error string in JSON responses. DatasetteError
now accepts a plain_message which the exception handler prefers for
JSON error bodies; the HTML error page keeps the rich message with the
SQL textarea.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:34:17 +00:00
Claude
60bac9439d
Mark shape=object item done in recommendations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:21:42 +00:00
Claude
87cd695ca3
Write endpoints parse the body as JSON regardless of Content-Type
The insert, upsert, alter and set-column-type endpoints previously
required Content-Type: application/json while /-/create parsed the body
blind - and insert returned a 500 AttributeError when the header was
missing entirely. The lenient rule is now uniform: the body is always
parsed as JSON and invalid JSON is a 400. This makes curl -d and
requests data=json.dumps(...) invocations work without remembering the
header. Cross-site request forgery remains prevented by the Origin and
Sec-Fetch-Site checks in CrossOriginProtectionMiddleware, which is the
defense the strict content-type requirement was historically standing
in for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:20:47 +00:00
Claude
022eb6d3a0
Mark documented items in recommendations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:09:27 +00:00
Claude
8154f7149f
Advise plugin authors on naming secret configuration keys
/-/config redacts values for keys whose names contain secret, key,
password, token, hash or dsn. Plugins that follow that naming get
automatic redaction; plugins that don't will leak their secrets on
that endpoint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:09:10 +00:00
Claude
3322e1f528
Document the boolean query string argument grammar
Boolean arguments parsed by value_as_boolean() accept on/true/1 and
off/false/0 - state this once in the JSON API docs rather than leaving
each argument to imply its own grammar.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:08:28 +00:00
Claude
6e17c51361
Document why upsert returns 200 where insert returns 201
An upsert may update existing rows without creating anything, so it
deliberately does not claim resource creation with a 201.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:08:05 +00:00
Claude
e892c686c2
Remove has_more from query list JSON - next: null signals the end
next: null (with next_url: null) is the single end-of-results signal
across the API, keeping default response keys to a minimum. The
StoredQueryPage.has_more attribute on the documented Python API is
unchanged.

Also fixes a bug this uncovered: the query list JSON next_url pointed
at the HTML page (it was built from the query list path, dropping the
.json extension) and was a relative path where the table view next_url
is absolute. It is now built from the request path and absolute, so it
preserves the requested format and can be followed directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:02:42 +00:00
Claude
5c418efd7f
Update tests from main for canonical error shape and rows key
Two tests merged from main were written against the pre-merge response
shapes: the max_post_body_bytes 413 error now uses the canonical error
envelope, and row update with return:true responds with a rows list
rather than a singular row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 22:42:12 +00:00
Claude
194ee95ae2
Merge remote-tracking branch 'origin/main' into claude/json-api-docs-1-0-review-a3e83u
# Conflicts:
#	datasette/__init__.py
#	tests/test_api_write.py
2026-07-06 22:25:27 +00:00
Claude
f4dfd6e0f7
Fix unused variable flagged by ruff
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 22:19:35 +00:00
Simon Willison
b6f5fd5cd0 test_internal_foreign_key_references() fix for sqlite-utils 4.0rc3
Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4889529980
2026-07-05 23:04:51 -07:00
Simon Willison
9a0b78b76c Vendor setup-sqlite-version
Same solution as https://github.com/simonw/sqlite-utils/pull/775
2026-07-05 22:53:12 -07:00
Simon Willison
c833217401 Test with sqlite-utils>=4.0rc3
Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4889420844
2026-07-05 22:47:05 -07:00
Simon Willison
557e08c6ef Make binary playwright test more robust
Failed here: https://github.com/simonw/datasette/actions/runs/28746832371/job/85239037361
2026-07-05 22:46:00 -07:00
Simon Willison
6d253d10c8 Compatibility with sqlite-utils>=4.0rc2
Runs the tests in CI. Had to add a little bit of code to handle
the difference between [table] and "table" and REAL v.s FLOAT.
2026-07-05 09:09:20 -07:00