Commit graph

3,428 commits

Author SHA1 Message Date
Simon Willison
bf348a22fc Escape LIKE metacharacters in FTS detection 2026-09-08 21:12:23 -07:00
Simon Willison
59618371e9 Validate URL before rendering column links 2026-09-08 21:11:44 -07:00
Simon Willison
5de0c1724e Viewing derived table requires permission for both table and its source
Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 18:09:23 -07:00
Simon Willison
d06737b6f4 Fix CREATE VIEW analysis on Python 3.10 2026-09-08 10:49:52 -07:00
Simon Willison
6473a7ecb0 Clearly document relationship between execute-sql and facets
Refs GHSA-5fff-xcm9-q6vh

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:49:52 -07:00
Simon Willison
f6d0f9bd38 detect_fts() now uses parameterized SQL
Refs GHSA-jcvx-2fh3-pjfp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:20 -07:00
Simon Willison
c899beaebe escape_sqlite() against column names
Refs GHSA-jcvx-2fh3-pjfp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:20 -07:00
Simon Willison
3ae092896d Only allow /db/name/-/schema against tables and views
Refs GHSA-926p-cw2f-643h

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:20 -07:00
Simon Willison
5d9a74f370 Check view-table as part of /-/schema and /db/-/schema
Refs GHSA-926p-cw2f-643h

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
01bf476d51 Require view-instance permission for /-/allowed
Refs GHSA-hp2x-vx2r-6vxg

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
4904249025 Use .textContent for column names in column chooser
Refs GHSA-hp2x-vx2r-6vxg

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
435e55ff0a Remove JSON syntax highlighting
Refs GHSA-hp2x-vx2r-6vxg

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
f8e8e65af7 actor cookie respects expire_after
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
577aeb73f0 Disallow ?_through= if user lacks view-table permission
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
c280c47424 POST /db/-/create checks table-level insert/update/alter permissions
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
4d0a2f2e84 execute-write: Check view-table for every table in a CREATE VIEW
Refs GHSA-53fc-rhfg-h7qp

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Simon Willison
c7944fc454 Skip deploy if environment variables are missing 2026-09-08 10:48:19 -07:00
Simon Willison
bdaa8cc76c Disable extension loading once --load-extension extensions are loaded
Refs GHSA-2mvv-ffvc-q5p6

Co-authored-by: Alex Garcia <15178711+asg017@users.noreply.github.com>
2026-09-08 10:48:19 -07:00
Zain Dana Harper
7403ae68bb Give each non-blocking write a distinct task id, refs #2860, #2859
execute_write_fn(fn, block=False) is documented to return "a UUID representing
the queued task". Two things stopped that being true.

_send_to_write_thread() derived the id from uuid.uuid5(NAMESPACE_DNS,
"datasette.io"), which is deterministic, so every non-blocking write in every
database in every process returned 3f143baa-4e3d-5842-a36f-4fa2f683b72f. A
constant cannot identify a particular task. Now uuid4(). Refs #2860.

With num_sql_threads=0 there is no write thread, so execute_write_fn took the
synchronous branch and `result` was the write function's return value, normally
None. The block=False path then unpacked it unconditionally and raised
TypeError: cannot unpack non-iterable NoneType object. The non-threaded branch
now returns the same (task_id, reply_future) shape, with the future already
resolved because the write has finished, so both modes share one code path.
Refs #2859.

test_execute_write_fn_block_false only asserted isinstance(task_id, uuid.UUID),
which a constant satisfies. The new test is parametrized over threaded and
non-threaded and asserts two calls return different ids, so either regression
fails it.
2026-09-08 10:03:52 -07:00
Simon Willison
bdc9731740
check-latest: true, add 3.15 to test matrix, to test RCs (#2895)
See https://simonwillison.net/2026/Sep/1/python-315-rc-2/
2026-09-01 13:37:15 -07:00
Alex Garcia
3e018bb1b5
Run startup via ASGI lifespan instead of waiting for the first request (#2887)
* Run startup via ASGI lifespan instead of waiting for the first request
* Ensure immutable table counts still precompute when startup ran first
2026-09-01 09:39:25 -07:00
Alex Garcia
e78b8a2e6a
Run datasette serve startup and uvicorn on a single event loop (#2886)
* Run datasette serve startup and uvicorn on a single event loop
* Move the serve-subprocess test plumbing into a conftest fixture
* Fix datasette-litestream URL and trim marker-task test comments
* Explain why serve_with_plugins needs a subprocess and plugin files
* Apply ruff 0.16 and black fixes
* Tweaked some comments
2026-09-01 09:32:37 -07:00
Simon Willison
0337fba234
disable_fts() before dropping table
Closes #2874
2026-08-10 15:03:29 -07:00
Simon Willison
12b25affb5 Release 1.0a38 1.0a38
Refs #2868
2026-08-06 11:20:25 -07:00
Simon Willison
eb6c2b96b9 Fix for SQL injection issue in table filters, refs #2868 2026-08-06 11:19:50 -07:00
Simon Willison
e889403d3b
Upgrade to ruff>=0.16.0 (#2857)
* ruff>=0.16.0

See https://astral.sh/blog/ruff-v0.16.0

* uv run ruff check . --fix --unsafe-fixes

* Ruff fixes by Claude Code Opus 5
2026-07-25 15:47:08 -07:00
Simon Willison
481df7ff6d Shorten link text in changelog 1.0a37 2026-07-14 09:31:28 -07:00
Simon Willison
2ffd8a860e Release 1.0a37
Refs #2831, #2832, #2841, #2842, #2843, #2846
2026-07-14 09:28:29 -07:00
Simon Willison
8b7c942d5e Major performance boost for SQL permissions, closes #2832 2026-07-14 09:18:51 -07:00
TowyTowy
591b909a4d
Escape table names with [square] brackets, refs #2431 (#2846)
Several internal helpers quoted table names using SQLite [bracket]
identifiers built with an f-string, e.g. PRAGMA foreign_key_list([{table}]).
Bracket quoting cannot escape a "]" character, so any table whose name
contains "]" (for example "[foo]" or "foo]") produced
"sqlite3.OperationalError: unrecognized token" - crashing schema
introspection at startup and 500-ing the table page.

Switch these call sites to the existing escape_sqlite() helper, which uses
"double quote" quoting with correct "" escaping (the same approach already
used elsewhere in the codebase and in the test suite):

- utils/internal_db.py: PRAGMA foreign_key_list / index_list
- utils/__init__.py: get_outbound_foreign_keys
- database.py: table_counts count query
- facets.py: default "select * from" SQL

Added a regression test covering table names with "]" characters.

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-14 08:53:45 -07:00
Simon Willison
9cfc252394
Make internal catalog refresh atomic
Refs #2831
2026-07-14 08:41:27 -07:00
Simon Willison
7f0a8b38ae
Better permission debug tools and documentation
Closes #2841
2026-07-14 08:40:07 -07:00
Simon Willison
10088dfa1d
execute_write(transaction=False) parameter, plus fix for errors inside tasks
Ensure a write inside a failing Datasette task never becomes visible. Refs #2831
2026-07-13 22:42:44 -07:00
Simon Willison
ccace40e5a
/-/plugins.json is now an array of objects again (#2843)
Reverts the object envelope introduced in 1.0a36 for this endpoint -
it once again returns a top-level JSON array of plugin objects.

Closes #2842


Claude-Session: https://claude.ai/code/session_012TYc1NTBK4zEjabB3u2zqu

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-13 21:19:04 -07:00
Simon Willison
db82123108 Bump a whole lot of GitHub Actions versions 1.0a36 2026-07-07 14:40:33 -07:00
Simon Willison
52ae7d1b6d Release 1.0a36
Refs
#1983, #1996, #2783, #2806, #2809, #2811, #2812, #2813, #2815, #2818, #2819, #2822, #2823, #2827
2026-07-07 14:32:25 -07:00
Simon Willison
a31673c90b Changelog for #2811, #2815, #2783 2026-07-07 14:26:48 -07:00
Simon Willison
54597f22fa A few more SQLite string fixes, refs #2783 2026-07-07 14:26:34 -07:00
JSap0914
bf3e277c98
Fix named_parameters when string literals contain comment markers (#2783)
named_parameters stripped SQL comments before string literals in
separate passes. A string literal such as '-- TODO' would be treated
as the start of a line comment, swallowing the rest of the line and
hiding any named parameters that followed it. For example:

    select * from t where note = '-- TODO' and id = :id

returned [] instead of ['id'], so the query parameter input form
would be missing the :id field.

Match comments and string literals in a single left-to-right pass so
that whichever construct starts first wins, matching how SQL is
actually tokenized.

Co-authored-by: JSap0914 <JSap0914@users.noreply.github.com>
2026-07-07 14:23:31 -07:00
Zain Dana Harper
211e70d4e1
Return 400 not 500 for wrong-arity composite-PK row URLs (#2815)
Fixes #2811

Co-authored-by: Zain Dana Harper <zain@aurora-framework.dev>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 14:19:08 -07:00
Simon Willison
617acedd38 Remove readthedocs/actions/preview
Closes #2828
2026-07-07 14:18:10 -07:00
Simon Willison
7f37205e76 Remove Datasette Desktop from installation guide
Until I have time to fix it up and bring it back.
2026-07-07 14:04:46 -07:00
Simon Willison
a926ab392e Updated internals.rst schema using cog, refs #2827 2026-07-07 14:02:02 -07:00
Simon Willison
db7ba1d30c Switch to sqlite-utils migrations for internal.db, closes #2827 2026-07-07 13:58:48 -07:00
Simon Willison
96e8b85523 Upgrade to sqlite-utils 4.0 2026-07-07 13:57:06 -07:00
Simon Willison
6f27aa112a
Test against sqlite-utils>=4.0
https://github.com/simonw/sqlite-utils/issues/769
2026-07-07 12:03:28 -07:00
Simon Willison
d2695a0c2f
Test Datasette against sqlite-utils>=4.0rc4
Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4900497417
2026-07-06 22:43:04 -07:00
Simon Willison
ebd013c6ef Bump GitHub Actions versions 2026-07-06 22:38:57 -07:00
Simon Willison
27a5be1326
Fable review of JSON API consistency and subsequent improvements
Merge PR #2824
2026-07-06 22:30:19 -07:00
Claude
b7bbde04be
Link the consistency review release note to PR #2824
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GrHZSypDfMnym1tM5XJAFZ
2026-07-07 05:18:54 +00:00