Commit graph

3,121 commits

Author SHA1 Message Date
Simon Willison
cd838daef4 Refactor tests a bit 2026-05-28 15:22:21 -07:00
Simon Willison
0b7c26c6c8 Refactored write decision tests 2026-05-28 12:09:20 -07:00
Simon Willison
17f45b884b Clarify ignored write SQL operation tests
Split the combined ignored-operation decision test into separate internal-operation and select-statement cases.

Assert the decision reason for each case instead of checking the shared base class, so the tests document why those operations are ignored.
2026-05-28 12:06:57 -07:00
Simon Willison
cbe9594a3d Use SQLiteTableType directly in SQL analysis
Remove the redundant SQLTableKind alias from the write SQL analysis model. Operation.table_kind and the analyzer cache now use the SQLite metadata classification type directly, making the source of table-kind values clearer.
2026-05-28 11:00:04 -07:00
Simon Willison
b2b20b36c5 Document write SQL analyzer restrictions
Expand the unreleased changelog with the deny-by-default operation analysis model, SQL function handling, and the VACUUM and virtual/shadow table restrictions for user-supplied write SQL.

Clarify the /-/execute-write JSON API documentation with the same restrictions and DDL permission requirements.
2026-05-28 10:24:40 -07:00
Simon Willison
51dab16149 Allow SQL functions in SQL write queries
Closes #2751
2026-05-28 10:22:28 -07:00
Simon Willison
8bd7e165f4 Refactored for code readability 2026-05-28 09:50:56 -07:00
Simon Willison
2785fd29de Fix tests I just broke 2026-05-28 09:03:10 -07:00
Simon Willison
aaf00e9ec2 Refactor hidden_table_names() to use new implemenatation
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4565727978
2026-05-28 08:42:06 -07:00
Simon Willison
bcd989f4f8 Detect and disallow insert to virtual/shadow table
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4565727978
2026-05-28 08:36:59 -07:00
Simon Willison
0c5053cdf6 Docs for /<database>/-/execute-write JSON API
Closes #2750, refs #2742
2026-05-27 17:26:50 -07:00
Simon Willison
11bddc8919 Deny VACUUM in user-authored SQL
Reject VACUUM explicitly during write-query permission analysis so arbitrary write SQL and untrusted stored write queries cannot run it, even when the actor has execute-write-sql.

Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803 (P3)
2026-05-27 17:09:27 -07:00
Simon Willison
951f5a9f30 Detect VACUUM in SQL analysis
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803
2026-05-27 16:30:21 -07:00
Simon Willison
1932f8429f Deny user-authored schema table reads in write SQL
Stop marking sqlite_master and sqlite_schema reads as internal as soon as the SQLite authorizer reports them. The later DDL-aware pass still treats schema catalog access as internal when it accompanies semantic CREATE, ALTER, or DROP operations.

This makes explicit catalog reads in write SQL fall through to the deny-by-default path as unsupported read schema operations, preventing queries from copying private table definitions into writable tables.

Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803
2026-05-27 16:14:56 -07:00
Simon Willison
03b2c66f63 Require full row mutation permissions for raw SQL
Raw SQL insert and update statements can have broader effects than their SQLite authorizer callbacks reveal. INSERT OR REPLACE and UPDATE OR REPLACE can delete conflicting rows while only surfacing insert or update operations.

Expand table insert and update operations to require insert-row, update-row, and delete-row together. Keep delete operations mapped to delete-row, and update the analysis UI/API to report and evaluate multiple required permissions for a single operation.

Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559083539
2026-05-27 15:17:22 -07:00
Simon Willison
86d0e7335f Deny unsupported write SQL operations by default
Require view-table permission for reads discovered inside write SQL analysis, including INSERT ... SELECT and CREATE TABLE ... AS SELECT.

Record additional SQLite authorizer callbacks as Operation values so unsupported functions, savepoints, virtual table DDL, and unknown callbacks are denied unless explicitly handled.
2026-05-27 14:52:52 -07:00
Simon Willison
737ff03efb Expanded analysis of SQL operations, refs #2748 2026-05-26 22:11:35 -07:00
Simon Willison
9f66cf72c1 Removed execute write SQL from query create page 2026-05-26 21:42:50 -07:00
Simon Willison
316daf9a43
Write SQL query UI, canned queries now stored in internal database
PR #2741
2026-05-26 16:54:00 -07:00
Simon Willison
b1289a73f9 stored_queries.StoredQuery dataclass 2026-05-26 16:51:00 -07:00
Simon Willison
2fde692a3e Disallow edits of dangerous decsription_html/on_success_message_sql
Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549891578
2026-05-26 16:34:48 -07:00
Simon Willison
90e19a7d58 Docs for datasette methods for queries
Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549824373
2026-05-26 16:33:36 -07:00
Simon Willison
ec438496a9 Get rid of the write/is_write dual properties 2026-05-26 16:31:07 -07:00
Simon Willison
56160e44fc Trusted queries cannot be updated using the API
Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549620486
2026-05-26 16:25:33 -07:00
Simon Willison
2eb307b8c6 Changelog updates for queries branch
Refs #2735, #2742
2026-05-26 16:10:05 -07:00
Simon Willison
3c29b002ca Do not document unstable JSON APIs for stored queries 2026-05-26 16:07:53 -07:00
Simon Willison
cef52b1ffc Break up giant views/database.py into smaller modules 2026-05-26 16:06:14 -07:00
Simon Willison
7214cc3761 Remove obsolete label 2026-05-26 15:52:44 -07:00
Simon Willison
d6de8e7520 Link to save query from /-/execute-write 2026-05-26 15:52:16 -07:00
Simon Willison
c3ceabae03 Ran Black 2026-05-26 15:51:40 -07:00
Simon Willison
58e2e3a8ab Ran cog 2026-05-26 15:43:34 -07:00
Simon Willison
1bcd99df90 Refactor code from datasette.app into datasette.stored_queries
The datasette/app.py file had grown a lot in this branch.
2026-05-26 15:42:40 -07:00
Simon Willison
e89ffa0e06 Fixed broken test caused by apply_queries_config() rename 2026-05-26 15:37:21 -07:00
Simon Willison
ca4907ab6b Make _save_queries_from_config a private method 2026-05-26 15:30:36 -07:00
Simon Willison
e2864fc895 test_stored_queries.py 2026-05-26 15:21:09 -07:00
Simon Willison
cafb6b9dbd Need is_trusted=True for the counters demo 2026-05-26 15:20:29 -07:00
Simon Willison
02a1468f1b Renamed canned queries to queries / stored queries in docs
And a few renames in code and YAML as well.
2026-05-26 15:17:51 -07:00
Simon Willison
56b14f37d5 The stored queries do not live in that DB 2026-05-26 15:16:18 -07:00
Simon Willison
2f73869c09 Document that canned_queries() has been removed 2026-05-26 15:09:48 -07:00
Simon Willison
b1029acc68 top_canned_query is now top_stored_query, closes #2747 2026-05-26 15:05:41 -07:00
Simon Willison
4bf1c4b065 Rename canned queries to queries/stored queries in docs 2026-05-26 14:54:35 -07:00
Simon Willison
0cadd07187 No need to document QueryCreateAnalyzeView 2026-05-26 14:53:31 -07:00
Simon Willison
24887004cf Rename insert-query to store-query
Also queries/insert to queries/store

Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549103663
2026-05-26 14:51:59 -07:00
Simon Willison
180a6a86fd Remove queries-plan.md
We do not need this any more. It can live
forever in Git history.
2026-05-26 14:16:10 -07:00
Simon Willison
ac6ee097dd Disallow update/delete of private queries
If a user does not own a private query they cannot update
or delete it either, even if they have global update-query.

https://github.com/simonw/datasette/pull/2741/changes#r3306417463
2026-05-26 14:10:48 -07:00
Simon Willison
024b911772 Clarifying comment
https://github.com/simonw/datasette/pull/2741/changes#r3306856046
2026-05-26 14:09:53 -07:00
Simon Willison
f7e9dbc27e Tweaked design of create query page 2026-05-26 14:02:44 -07:00
Simon Willison
5dca2dc9be Show query count on database page 2026-05-26 13:54:47 -07:00
Simon Willison
6033bf8e40 Merge branch 'main' into queries 2026-05-26 13:51:51 -07:00
Simon Willison
eb7c25c57c Major redesign of create saved query UI
https://github.com/simonw/datasette/pull/2741#issuecomment-4548707129
2026-05-26 13:48:40 -07:00