Commit graph

1,468 commits

Author SHA1 Message Date
Alex Garcia
0948d28554 Apply ruff 0.16 and black fixes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:30:53 -07:00
Alex Garcia
4d3530c032 Remove references to untracked local plans/ and todos/ directories
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:30:53 -07:00
Alex Garcia
3125c36802 Run plugin asgi_wrapper middleware inside the startup-arming layer
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-14 08:30:53 -07:00
Alex Garcia
e07e8f93fc Apply ruff 0.16 and black fixes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:30:02 -07:00
Alex Garcia
a628415c19 Remove references to untracked local plans/ and todos/ directories
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:30:02 -07:00
Alex Garcia
940aae1ea5 Add a shutdown() plugin hook with ordered graceful teardown
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-14 08:30:02 -07:00
Alex Garcia
0fb9b861b4 Rename crasher test helper to satisfy codespell
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:29:48 -07:00
Alex Garcia
52e410cd63 Apply ruff 0.16 and black fixes
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:29:48 -07:00
Alex Garcia
442662c55f Remove references to untracked local plans/ and todos/ directories
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012U7coQfVu8nK2R4q2mCULA
2026-09-14 08:29:48 -07:00
Alex Garcia
5277f9297b Add datasette.add_background_task() with supervised launch after startup
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-09-14 08:29:48 -07:00
Simon Willison
b338c6f5f6
Migrate from httpx to httpx2, closes #2879
https://claude.ai/code/session_01Xdqoneq8ddvruVZETo6rFf
2026-09-10 19:44:49 -07:00
Simon Willison
92c7d4b608 Limit derived-table permissions to one source hop
Simplify the solution to 5de0c1724e - avoid contextvar.
2026-09-10 16:52:52 -07:00
Simon Willison
f70edbfa60 Filter incoming foreign-key relationships by view permission 2026-09-10 16:52:52 -07:00
Simon Willison
b97bb5f016 Reconcile write-timeout regression with the per-call limit 2026-09-10 16:52:26 -07:00
Simon Willison
e036907fc3 Reject structured row writes to virtual and shadow tables 2026-09-10 16:52:25 -07:00
Simon Willison
3f8d8417f6 Inherit source permissions for FTS vocabulary tables 2026-09-10 16:52:25 -07:00
Simon Willison
d334539a1e Deny SQLite statistics table access through a default hook 2026-09-10 16:52:25 -07:00
Simon Willison
506c4bb522 Match table permission identities using SQLite case semantics 2026-09-09 08:39:03 -07:00
Alex Garcia
c6ba7b3298 Refuse API token creation from restricted actors 2026-09-09 08:38:40 -07:00
Simon Willison
ceef351622 Protect personalized dynamic responses from shared caching 2026-09-08 21:16:36 -07:00
Simon Willison
6aa58bf4e5 Authorize configured full-text search targets 2026-09-08 21:16:35 -07:00
Simon Willison
c01e95f3bd Filter foreign-key helper targets by view permission 2026-09-08 21:14:10 -07:00
Simon Willison
5de0c1724e Viewing derived table requires permission for both table and its source
Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 18:09:23 -07:00
Simon Willison
d06737b6f4 Fix CREATE VIEW analysis on Python 3.10 2026-09-08 10:49:52 -07:00
Simon Willison
3ae092896d Only allow /db/name/-/schema against tables and views
Refs GHSA-926p-cw2f-643h

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:20 -07:00
Simon Willison
5d9a74f370 Check view-table as part of /-/schema and /db/-/schema
Refs GHSA-926p-cw2f-643h

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
01bf476d51 Require view-instance permission for /-/allowed
Refs GHSA-hp2x-vx2r-6vxg

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
f8e8e65af7 actor cookie respects expire_after
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
577aeb73f0 Disallow ?_through= if user lacks view-table permission
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
c280c47424 POST /db/-/create checks table-level insert/update/alter permissions
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
4d0a2f2e84 execute-write: Check view-table for every table in a CREATE VIEW
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
bdaa8cc76c Disable extension loading once --load-extension extensions are loaded
Refs GHSA-2mvv-ffvc-q5p6

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Zain Dana Harper
7403ae68bb Give each non-blocking write a distinct task id, refs #2860, #2859
execute_write_fn(fn, block=False) is documented to return "a UUID representing
the queued task". Two things stopped that being true.

_send_to_write_thread() derived the id from uuid.uuid5(NAMESPACE_DNS,
"datasette.io"), which is deterministic, so every non-blocking write in every
database in every process returned 3f143baa-4e3d-5842-a36f-4fa2f683b72f. A
constant cannot identify a particular task. Now uuid4(). Refs #2860.

With num_sql_threads=0 there is no write thread, so execute_write_fn took the
synchronous branch and `result` was the write function's return value, normally
None. The block=False path then unpacked it unconditionally and raised
TypeError: cannot unpack non-iterable NoneType object. The non-threaded branch
now returns the same (task_id, reply_future) shape, with the future already
resolved because the write has finished, so both modes share one code path.
Refs #2859.

test_execute_write_fn_block_false only asserted isinstance(task_id, uuid.UUID),
which a constant satisfies. The new test is parametrized over threaded and
non-threaded and asserts two calls return different ids, so either regression
fails it.
2026-09-08 10:03:52 -07:00
Alex Garcia
3e018bb1b5
Run startup via ASGI lifespan instead of waiting for the first request (#2887)
* Run startup via ASGI lifespan instead of waiting for the first request
* Ensure immutable table counts still precompute when startup ran first
2026-09-01 09:39:25 -07:00
Alex Garcia
e78b8a2e6a
Run datasette serve startup and uvicorn on a single event loop (#2886)
* Run datasette serve startup and uvicorn on a single event loop
* Move the serve-subprocess test plumbing into a conftest fixture
* Fix datasette-litestream URL and trim marker-task test comments
* Explain why serve_with_plugins needs a subprocess and plugin files
* Apply ruff 0.16 and black fixes
* Tweaked some comments
2026-09-01 09:32:37 -07:00
Simon Willison
0337fba234
disable_fts() before dropping table
Closes #2874
2026-08-10 15:03:29 -07:00
Simon Willison
eb6c2b96b9 Fix for SQL injection issue in table filters, refs #2868 2026-08-06 11:19:50 -07:00
Simon Willison
e889403d3b
Upgrade to ruff>=0.16.0 (#2857)
* ruff>=0.16.0

See https://astral.sh/blog/ruff-v0.16.0

* uv run ruff check . --fix --unsafe-fixes

* Ruff fixes by Claude Code Opus 5
2026-07-25 15:47:08 -07:00
TowyTowy
591b909a4d
Escape table names with [square] brackets, refs #2431 (#2846)
Several internal helpers quoted table names using SQLite [bracket]
identifiers built with an f-string, e.g. PRAGMA foreign_key_list([{table}]).
Bracket quoting cannot escape a "]" character, so any table whose name
contains "]" (for example "[foo]" or "foo]") produced
"sqlite3.OperationalError: unrecognized token" - crashing schema
introspection at startup and 500-ing the table page.

Switch these call sites to the existing escape_sqlite() helper, which uses
"double quote" quoting with correct "" escaping (the same approach already
used elsewhere in the codebase and in the test suite):

- utils/internal_db.py: PRAGMA foreign_key_list / index_list
- utils/__init__.py: get_outbound_foreign_keys
- database.py: table_counts count query
- facets.py: default "select * from" SQL

Added a regression test covering table names with "]" characters.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-14 08:53:45 -07:00
Simon Willison
7f0a8b38ae
Better permission debug tools and documentation
Closes #2841
2026-07-14 08:40:07 -07:00
Simon Willison
10088dfa1d
execute_write(transaction=False) parameter, plus fix for errors inside tasks
Ensure a write inside a failing Datasette task never becomes visible. Refs #2831
2026-07-13 22:42:44 -07:00
Simon Willison
ccace40e5a
/-/plugins.json is now an array of objects again (#2843)
Reverts the object envelope introduced in 1.0a36 for this endpoint -
it once again returns a top-level JSON array of plugin objects.

Closes #2842


Claude-Session: https://claude.ai/code/session_012TYc1NTBK4zEjabB3u2zqu

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-13 21:19:04 -07:00
Simon Willison
54597f22fa A few more SQLite string fixes, refs #2783 2026-07-07 14:26:34 -07:00
JSap0914
bf3e277c98
Fix named_parameters when string literals contain comment markers (#2783)
named_parameters stripped SQL comments before string literals in
separate passes. A string literal such as '-- TODO' would be treated
as the start of a line comment, swallowing the rest of the line and
hiding any named parameters that followed it. For example:

    select * from t where note = '-- TODO' and id = :id

returned [] instead of ['id'], so the query parameter input form
would be missing the :id field.

Match comments and string literals in a single left-to-right pass so
that whichever construct starts first wins, matching how SQL is
actually tokenized.

Co-authored-by: JSap0914 <JSap0914@users.noreply.github.com>
2026-07-07 14:23:31 -07:00
Zain Dana Harper
211e70d4e1
Return 400 not 500 for wrong-arity composite-PK row URLs (#2815)
Fixes #2811

Co-authored-by: Zain Dana Harper <zain@aurora-framework.dev>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 14:19:08 -07:00
Simon Willison
db7ba1d30c Switch to sqlite-utils migrations for internal.db, closes #2827 2026-07-07 13:58:48 -07:00
Simon Willison
96e8b85523 Upgrade to sqlite-utils 4.0 2026-07-07 13:57:06 -07:00
Claude
be25d6e3e4
Remove test_query_list_json_signals_pagination_via_next_only
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00
Claude
4a853cb10c
Use UNSTABLE_API_MESSAGE constant in tests
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00
Claude
b83b12dd7a
Remove params input alias from the query create and update APIs
The alias existed so API payloads could mirror the params key used by
queries defined in datasette.yaml, but it was undocumented and untested,
and the create endpoint is not part of the stable API. The API now only
accepts parameters - sending params is a 400 Invalid keys error. The
documented params key for queries in configuration is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 01:13:33 +00:00