Commit graph

3,129 commits

Author SHA1 Message Date
Simon Willison
72cf476d1d Tidied up release notes ready to ship
Refs #2741, #2749
2026-05-28 20:28:24 -07:00
Simon Willison
9e377e8b90 Only show valid SQL write templates
Closes #2753

Demo: https://github.com/simonw/datasette/issues/2753#issuecomment-4570071413
2026-05-28 20:01:56 -07:00
Simon Willison
52729faa54 /<database>/-/query.json and changelog docs 2026-05-28 20:01:56 -07:00
Simon Willison
e5b6166fa3 Nicer UI around Execute Write SQL denied
Refs https://github.com/simonw/datasette/issues/2753#issuecomment-4569117665
2026-05-28 20:01:56 -07:00
Simon Willison
6a998610ee
datasette inspect now counts 10,000+ tables correctly (#2752)
Closes #2712

Refs https://github.com/simonw/datasette/pull/2721#issuecomment-4568966383
2026-05-28 15:52:51 -07:00
Simon Willison
74324cb849 Improved docs for user-facing SQL query pages
- /database-name/-/execute-write
- /-/queries
2026-05-28 15:46:27 -07:00
Simon Willison
b6e9b18990 datasette.yml can no longer set a query to private
Private means it has an owner, and the config does not let
you say who the owner is - plus configured queries should
not be possible to edit or delete in the UI so having an
owner makes even less sense.

You can still make configured queries visible to specific
people using regular view-query permissions.
2026-05-28 15:37:48 -07:00
Simon Willison
dd73eb018d
Analyze write SQL for full set of SQLite operations
PR #2749  - https://github.com/simonw/datasette/pull/2749
2026-05-28 15:25:28 -07:00
Simon Willison
cd838daef4 Refactor tests a bit 2026-05-28 15:22:21 -07:00
Simon Willison
0b7c26c6c8 Refactored write decision tests 2026-05-28 12:09:20 -07:00
Simon Willison
17f45b884b Clarify ignored write SQL operation tests
Split the combined ignored-operation decision test into separate internal-operation and select-statement cases.

Assert the decision reason for each case instead of checking the shared base class, so the tests document why those operations are ignored.
2026-05-28 12:06:57 -07:00
Simon Willison
cbe9594a3d Use SQLiteTableType directly in SQL analysis
Remove the redundant SQLTableKind alias from the write SQL analysis model. Operation.table_kind and the analyzer cache now use the SQLite metadata classification type directly, making the source of table-kind values clearer.
2026-05-28 11:00:04 -07:00
Simon Willison
b2b20b36c5 Document write SQL analyzer restrictions
Expand the unreleased changelog with the deny-by-default operation analysis model, SQL function handling, and the VACUUM and virtual/shadow table restrictions for user-supplied write SQL.

Clarify the /-/execute-write JSON API documentation with the same restrictions and DDL permission requirements.
2026-05-28 10:24:40 -07:00
Simon Willison
51dab16149 Allow SQL functions in SQL write queries
Closes #2751
2026-05-28 10:22:28 -07:00
Simon Willison
8bd7e165f4 Refactored for code readability 2026-05-28 09:50:56 -07:00
Simon Willison
2785fd29de Fix tests I just broke 2026-05-28 09:03:10 -07:00
Simon Willison
aaf00e9ec2 Refactor hidden_table_names() to use new implemenatation
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4565727978
2026-05-28 08:42:06 -07:00
Simon Willison
bcd989f4f8 Detect and disallow insert to virtual/shadow table
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4565727978
2026-05-28 08:36:59 -07:00
Simon Willison
0c5053cdf6 Docs for /<database>/-/execute-write JSON API
Closes #2750, refs #2742
2026-05-27 17:26:50 -07:00
Simon Willison
11bddc8919 Deny VACUUM in user-authored SQL
Reject VACUUM explicitly during write-query permission analysis so arbitrary write SQL and untrusted stored write queries cannot run it, even when the actor has execute-write-sql.

Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803 (P3)
2026-05-27 17:09:27 -07:00
Simon Willison
951f5a9f30 Detect VACUUM in SQL analysis
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803
2026-05-27 16:30:21 -07:00
Simon Willison
1932f8429f Deny user-authored schema table reads in write SQL
Stop marking sqlite_master and sqlite_schema reads as internal as soon as the SQLite authorizer reports them. The later DDL-aware pass still treats schema catalog access as internal when it accompanies semantic CREATE, ALTER, or DROP operations.

This makes explicit catalog reads in write SQL fall through to the deny-by-default path as unsupported read schema operations, preventing queries from copying private table definitions into writable tables.

Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803
2026-05-27 16:14:56 -07:00
Simon Willison
03b2c66f63 Require full row mutation permissions for raw SQL
Raw SQL insert and update statements can have broader effects than their SQLite authorizer callbacks reveal. INSERT OR REPLACE and UPDATE OR REPLACE can delete conflicting rows while only surfacing insert or update operations.

Expand table insert and update operations to require insert-row, update-row, and delete-row together. Keep delete operations mapped to delete-row, and update the analysis UI/API to report and evaluate multiple required permissions for a single operation.

Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559083539
2026-05-27 15:17:22 -07:00
Simon Willison
86d0e7335f Deny unsupported write SQL operations by default
Require view-table permission for reads discovered inside write SQL analysis, including INSERT ... SELECT and CREATE TABLE ... AS SELECT.

Record additional SQLite authorizer callbacks as Operation values so unsupported functions, savepoints, virtual table DDL, and unknown callbacks are denied unless explicitly handled.
2026-05-27 14:52:52 -07:00
Simon Willison
737ff03efb Expanded analysis of SQL operations, refs #2748 2026-05-26 22:11:35 -07:00
Simon Willison
9f66cf72c1 Removed execute write SQL from query create page 2026-05-26 21:42:50 -07:00
Simon Willison
316daf9a43
Write SQL query UI, canned queries now stored in internal database
PR #2741
2026-05-26 16:54:00 -07:00
Simon Willison
b1289a73f9 stored_queries.StoredQuery dataclass 2026-05-26 16:51:00 -07:00
Simon Willison
2fde692a3e Disallow edits of dangerous decsription_html/on_success_message_sql
Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549891578
2026-05-26 16:34:48 -07:00
Simon Willison
90e19a7d58 Docs for datasette methods for queries
Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549824373
2026-05-26 16:33:36 -07:00
Simon Willison
ec438496a9 Get rid of the write/is_write dual properties 2026-05-26 16:31:07 -07:00
Simon Willison
56160e44fc Trusted queries cannot be updated using the API
Refs https://github.com/simonw/datasette/pull/2741#issuecomment-4549620486
2026-05-26 16:25:33 -07:00
Simon Willison
2eb307b8c6 Changelog updates for queries branch
Refs #2735, #2742
2026-05-26 16:10:05 -07:00
Simon Willison
3c29b002ca Do not document unstable JSON APIs for stored queries 2026-05-26 16:07:53 -07:00
Simon Willison
cef52b1ffc Break up giant views/database.py into smaller modules 2026-05-26 16:06:14 -07:00
Simon Willison
7214cc3761 Remove obsolete label 2026-05-26 15:52:44 -07:00
Simon Willison
d6de8e7520 Link to save query from /-/execute-write 2026-05-26 15:52:16 -07:00
Simon Willison
c3ceabae03 Ran Black 2026-05-26 15:51:40 -07:00
Simon Willison
58e2e3a8ab Ran cog 2026-05-26 15:43:34 -07:00
Simon Willison
1bcd99df90 Refactor code from datasette.app into datasette.stored_queries
The datasette/app.py file had grown a lot in this branch.
2026-05-26 15:42:40 -07:00
Simon Willison
e89ffa0e06 Fixed broken test caused by apply_queries_config() rename 2026-05-26 15:37:21 -07:00
Simon Willison
ca4907ab6b Make _save_queries_from_config a private method 2026-05-26 15:30:36 -07:00
Simon Willison
e2864fc895 test_stored_queries.py 2026-05-26 15:21:09 -07:00
Simon Willison
cafb6b9dbd Need is_trusted=True for the counters demo 2026-05-26 15:20:29 -07:00
Simon Willison
02a1468f1b Renamed canned queries to queries / stored queries in docs
And a few renames in code and YAML as well.
2026-05-26 15:17:51 -07:00
Simon Willison
56b14f37d5 The stored queries do not live in that DB 2026-05-26 15:16:18 -07:00
Simon Willison
2f73869c09 Document that canned_queries() has been removed 2026-05-26 15:09:48 -07:00
Simon Willison
b1029acc68 top_canned_query is now top_stored_query, closes #2747 2026-05-26 15:05:41 -07:00
Simon Willison
4bf1c4b065 Rename canned queries to queries/stored queries in docs 2026-05-26 14:54:35 -07:00
Simon Willison
0cadd07187 No need to document QueryCreateAnalyzeView 2026-05-26 14:53:31 -07:00