Commit graph

271 commits

Author SHA1 Message Date
Claude
4a853cb10c
Use UNSTABLE_API_MESSAGE constant in tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00
Claude
4874c29286
Remove the next_url extra - the key is always present
next_url became a default table JSON key alongside next, making the
extra a no-op. Requesting ?_extra=next_url now returns the standard
unknown-extra 400 error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:59:53 +00:00
Claude
0d962deb05
Plain text SQL Interrupted errors in JSON responses
The SQL time limit error embedded an HTML fragment (paragraph, textarea
and script tags) as the error string in JSON responses. DatasetteError
now accepts a plain_message which the exception handler prefers for
JSON error bodies; the HTML error page keeps the rich message with the
SQL textarea.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-06 23:34:17 +00:00
Claude
194ee95ae2
Merge remote-tracking branch 'origin/main' into claude/json-api-docs-1-0-review-a3e83u
# Conflicts:
#	datasette/__init__.py
#	tests/test_api_write.py
2026-07-06 22:25:27 +00:00
Claude
afa7b1ba0d
Add unstable marker to undocumented JSON endpoints
JSON endpoints that are not part of the documented API now include
"unstable": "This API is not part of Datasette's stable interface and
may change at any time" in their responses, making the stability tier
machine-readable. Applied to the homepage (/.json and /-/.json),
/db/-/queries/analyze, POST /db/-/queries/store,
/db/<query>/-/definition, /db/-/query/parameters,
/db/-/execute-write/analyze and the POST /-/permissions playground
response. The message lives in datasette.utils.UNSTABLE_API_MESSAGE.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 17:01:53 +00:00
Claude
5cb2bc6909
Homepage JSON returns databases as a list
/.json previously returned databases as an object keyed by database
name, unlike /-/databases.json and every other collection in the API.
It now returns a list of database objects. The HTML template already
consumed the list. This endpoint remains undocumented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 16:45:19 +00:00
Claude
e5e9aca871
Require permissions-debug for /-/threads
/-/threads exposes runtime internals - thread idents and asyncio task
reprs including file paths - but only required view-instance. It now
requires permissions-debug, like /-/actions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 16:22:41 +00:00
Claude
23ccdaeffc
Convert /-/actions.json from top-level array to object
/-/actions.json now returns {"ok": true, "actions": [...]} instead of a
bare JSON array, so the response can grow additional keys without a
breaking change. The debug_actions.html template reads data.actions,
and the endpoint is now documented in docs/introspection.rst.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:46:26 +00:00
Claude
19e54b10d4
Convert /-/databases.json from top-level array to object
/-/databases.json now returns {"ok": true, "databases": [...]} instead
of a bare JSON array, so the response can grow additional keys without
a breaking change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:43:44 +00:00
Claude
b74a8e5b12
Convert /-/plugins.json from top-level array to object
/-/plugins.json now returns {"ok": true, "plugins": [...]} instead of a
bare JSON array, so the response can grow additional keys without a
breaking change. The `datasette plugins` CLI command still outputs a
plain array.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:42:24 +00:00
Claude
089e96a437
Add "ok": true to every JSON object success response
JsonDataView now injects "ok": true into dict responses, covering
/-/versions, /-/settings, /-/config, /-/threads and /-/actor. The
homepage JSON, /-/jump, the three /-/schema endpoints, /-/allowed,
/-/rules, /-/check, POST /-/permissions and the table /-/autocomplete
endpoint set it explicitly.

The remaining top-level array endpoints (/-/plugins, /-/databases,
/-/actions) will be converted to objects in separate commits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 13:40:05 +00:00
Claude
0679e04bd3
Unify JSON error responses into one canonical shape
All JSON error responses now use a single format built by the new
datasette.utils.error_body() helper:

    {"ok": false, "error": "...", "errors": ["..."], "status": 400}

- error is all messages joined with '; ', errors is the full list,
  status always matches the HTTP status code
- The exception handler no longer emits the legacy title key in JSON
  (it is still available to the HTML error template)
- The permission debug endpoints (/-/allowed, /-/rules, /-/check,
  POST /-/permissions) no longer return bare {"error": ...} objects
- JSON renderer SQL errors keep their rows/truncated context keys but
  now include the canonical keys as well
- _shape=object misuse (queries or tables without primary keys) now
  returns HTTP 400 instead of 200 with an error body
- Method-not-allowed 405 responses use the canonical shape

Adds tests/test_error_shape.py covering all four previous shape
producers, updates affected tests, and documents the format in a new
'Error responses' section of docs/json_api.rst.

Implements section 1 of stable-api-recommendations.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-04 03:12:15 +00:00
Simon Willison
58c07cc264 max_post_body_bytes setting, enforced for reuest.post_body()
Closes #2823
2026-07-03 17:42:39 -07:00
Simon Willison
81d6ee69cd Allow declared column type case to vary 2026-07-03 17:12:26 -07:00
Simon Willison
b476218edb Refine column_details metadata shape 2026-07-03 16:08:34 -07:00
Simon Willison
8856914be8 Add column_details JSON extra 2026-07-03 14:45:38 -07:00
Simon Willison
f4b4506035 Remove legacy ?_extras= row parameter
The pre-1.0 ?_extras= (plural) parameter was kept for backwards
compatibility with the old row JSON API. ?_extra= is the documented
mechanism now that row pages share the extras registry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 21:49:23 -07:00
Simon Willison
4d6daa175a Add row and query JSON extras 2026-06-09 02:56:27 -07:00
wheelman
b013aa1f7f
Add CORS headers to /db?sql= query redirect (#2730)
Closes #2728
2026-05-23 21:21:13 -07:00
Simon Willison
5f39036b9b ok: true in /db.json for consistency 2026-04-15 15:44:06 -07:00
Simon Willison
9c164572d3
Add actor= parameter to datasette.client methods (#2688)
`datasette.client.get(path, actor={"id": "root"}` now makes the internal request with that actor as `request.actor` - same for the other HTTP verb methods on `datasette.client`.

Upgraded relevant tests to use the new `actor=` mechanism.
2026-04-14 18:31:57 -07:00
Claude
51e341b06a Fix test assertions broken by new fixture rows in 170f9de
The render_cell pks parameter commit added rows to compound_primary_key
(2->3 rows) and no_primary_key (201->202 rows) tables but did not
update existing tests that had hardcoded row count expectations.

https://claude.ai/code/session_01XfPSZfK57bzRRiEa7Kz5n1
2026-02-17 13:22:57 -08:00
Simon Willison
7915c46ddd
Fix flaky test_database_page test with deterministic ordering (#2628)
* Fix flaky test_database_page test with deterministic ordering

- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
  500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst

* Update test_row_foreign_key_tables for new deterministic FK ordering

The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.

* Update test_table_names for new alphabetical ordering

The table_names() method now returns tables sorted alphabetically.

* Fix for test that fails prior to SQLite 3.37

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 20:57:25 -08:00
Simon Willison
66d2a033f8 Switch to ruff and fix all lint errors, refs #2630 2026-01-23 20:43:16 -08:00
Simon Willison
6fede23a2e Only return render_coll columns that differ from default, refs #2619 2025-12-21 20:18:26 -08:00
Simon Willison
eae94dc2c3 Initial render_cell and foreign_key_tables extras for row
Closes #2619, refs #2050
2025-12-21 20:03:10 -08:00
Simon Willison
232a404743 Switch searchable_fts test table to FTS5, closes #2613 2025-12-12 22:18:35 -08:00
Simon Willison
3b4c7e1abe {"ok": true} on row API, to be consistent with table 2025-12-12 21:43:00 -08:00
Simon Willison
c76c3e6e6f facet_suggest_time_limit_ms 200ms in tests, closes #2574 2025-11-03 11:52:12 -08:00
Simon Willison
73014abe8b Improved permissions UI WIP 2025-10-30 10:28:04 -07:00
Simon Willison
b3721eaf50 Add /-/actions endpoint to list registered actions
This adds a new endpoint at /-/actions that lists all registered actions
in the permission system. The endpoint supports both JSON and HTML output.

Changes:
- Added _actions() method to Datasette class to return action list
- Added route for /-/actions with JsonDataView
- Created actions.html template for nice HTML display
- Added template parameter to JsonDataView for custom templates
- Moved respond_json_or_html from BaseView to JsonDataView
- Added test for the new endpoint

The endpoint requires view-instance permission and provides details about
each action including name, abbreviation, description, resource class,
and parent/child requirements.

Closes #2547

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:14:58 -07:00
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
82cc3d5c86 Migrate view-query permission to SQL-based system, refs #2510
This change integrates canned queries with Datasette's new SQL-based
permissions system by making the following changes:

1. **Default canned_queries plugin hook**: Added a new hookimpl in
   default_permissions.py that returns canned queries from datasette
   configuration. This extracts config-reading logic into a plugin hook,
   allowing QueryResource to discover all queries.

2. **Async resources_sql()**: Converted Resource.resources_sql() from a
   synchronous class method returning a string to an async method that
   receives the datasette instance. This allows QueryResource to call
   plugin hooks and query the database.

3. **QueryResource implementation**: Implemented QueryResource.resources_sql()
   to gather all canned queries by:
   - Querying catalog_databases for all databases
   - Calling canned_queries hooks for each database with actor=None
   - Building a UNION ALL SQL query of all (database, query_name) pairs
   - Properly escaping single quotes in resource names

4. **Simplified get_canned_queries()**: Removed config-reading logic since
   it's now handled by the default plugin hook.

5. **Added view-query to default allow**: Added "view-query" to the
   default_allow_actions set so canned queries are accessible by default.

6. **Removed xfail markers**: Removed test xfail markers from:
   - tests/test_canned_queries.py (entire module)
   - tests/test_html.py (2 tests)
   - tests/test_permissions.py (1 test)
   - tests/test_plugins.py (1 test)

All canned query tests now pass with the new permission system.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
ee1d7983ba Mark canned query tests as xfail, refs #2510, refs #2528
Canned queries are not accessible because view-query permission
has not yet been migrated to the SQL-based permission system.

Marks the following tests with xfail:
- test_config_cache_size (test_api.py)
- test_edit_sql_link_not_shown_if_user_lacks_permission (test_html.py)
- test_database_color - removes canned query path (test_html.py)
- test_hook_register_output_renderer_* (test_plugins.py - 3 tests)
- test_hook_query_actions canned query parameter (test_plugins.py)
- test_custom_query_with_unicode_characters (test_table_api.py)
- test_permissions_checked neighborhood_search (test_permissions.py)
- test_padlocks_on_database_page (test_permissions.py)

All reference issue #2510 for tracking view-query migration.
2025-10-25 15:38:07 -07:00
Simon Willison
f2485dce9c
Hide FTS tables that have content=
* Hide FTS tables that have content=, closes #2477
2025-04-16 21:44:09 -07:00
Simon Willison
53a3b3c80e
Test improvements and fixed deprecation warnings (#2464)
* `asyncio_default_fixture_loop_scope = function`
* Fix a bunch of BeautifulSoup deprecation warnings
* Fix for PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]>
* xfail for sql_time_limit tests (these can be flaky in CI)

Refs #2461
2025-02-04 14:49:52 -08:00
Simon Willison
4dff846271 simple_primary_key now uses integer id, helps close #2458 2025-02-01 21:44:53 -08:00
Simon Willison
39dfc7d7d7
Removed units functionality and Pint dependency
Closes #2400, unblocks #2320
2024-08-20 19:03:33 -07:00
Simon Willison
53a8ae1871 Applied Black, refs #2327, #2326 2024-08-15 17:16:47 -07:00
Seb Bacon
9cb5700d60
bugfix: correctly detect json1 in versions.json (#2327)
Fixes #2326
2024-08-15 13:20:26 -07:00
Alex Garcia
6d91d082e0
Hide shadow tables, don't hide virtual tables
Closes #2296
2024-08-15 13:19:22 -07:00
Alex Garcia
a23c2aee00
Introduce new /$DB/-/query endpoint, soft replaces /$DB?sql=... (#2363)
* Introduce new default /$DB/-/query endpoint
* Fix a lot of tests
* Update pyodide test to use query endpoint
* Link to /fixtures/-/query in a few places
* Documentation for QueryView

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-07-15 10:33:51 -07:00
Alex Garcia
e1bfab3fca
Move Metadata to --internal database
Refs:
- https://github.com/simonw/datasette/pull/2343
- https://github.com/simonw/datasette/issues/2341
2024-06-11 09:33:23 -07:00
Simon Willison
7818e8b9d1 Hide tables starting with an _, refs #2104 2024-03-07 00:03:42 -05:00
Simon Willison
398a92cf1e Include database in name of _execute_writes thread, closes #2265 2024-02-08 20:12:31 -08:00
Simon Willison
60c6692f68
table_config instead of table_metadata (#2257)
Table configuration that was incorrectly placed in metadata is now treated as if it was in config.

New await datasette.table_config() method.

Closes #2247
2024-02-06 21:57:09 -08:00
Simon Willison
1e901aa690 /-/config page, closes #2254 2024-02-06 12:33:46 -08:00
Simon Willison
bcc4f6bf1f
track_event() mechanism for analytics and plugins
* Closes #2240
* Documentation for event plugin hooks, refs #2240
* Include example track_event plugin in docs, refs #2240
* Tests for track_event() and register_events() hooks, refs #2240
* Initial documentation for core events, refs #2240
* Internals documentation for datasette.track_event()
2024-01-31 15:21:40 -08:00
Simon Willison
2200abfa17 Fix for flaky test_hidden_sqlite_stat1_table, closes #2179 2023-09-07 15:49:50 -07:00
Simon Willison
e34d09c6ec Don't include columns in query JSON, refs #2136 2023-08-09 12:01:59 -07:00