Commit graph

2,977 commits

Author SHA1 Message Date
Simon Willison
d72dd35378 Wire Datasette.close into ASGI lifespan shutdown
AsgiLifespan now receives an on_shutdown callback that invokes
Datasette.close(), so resources are released cleanly when the ASGI server
delivers a lifespan.shutdown message (SIGTERM / SIGINT for uvicorn).

Refs #2692

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:11:02 -07:00
Simon Willison
290f27158f Datasette.close() closes databases, shuts down executor, unlinks temp file
Datasette.close() iterates over every attached Database (including the
internal database), calls Database.close() on each, then shuts down the
ThreadPoolExecutor. Exceptions raised by one Database don't prevent the
others from being closed; the first exception is re-raised afterwards.
Idempotent.

Refs #2692

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:10:18 -07:00
Simon Willison
dabf8e4199 Database.close() shuts down write thread and raises DatasetteClosedError
After this commit, Database.close() sends a sentinel to the write queue so
the background write thread exits cleanly, closes cached read/write
connections, and marks the instance closed. Subsequent calls to execute*()
raise DatasetteClosedError. close() remains idempotent and one-way.

Refs #2692

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:09:47 -07:00
Simon Willison
ade0ef8a60 Restore compatibility with existing execute_write_fn() callbacks
Closes #2691
2026-04-16 19:14:32 -07:00
Simon Willison
2638200d26
Link to datasette.io preview tool 2026-04-15 17:19:43 -07:00
Simon Willison
1f99d5dd20 Release 1.0a27 1.0a27
Refs #1936, #2678, #2681, #2682, #2683, #2684, #2688, #2689
2026-04-15 16:11:54 -07:00
Simon Willison
67349e0e02 New :pr:ID shortcut for docs 2026-04-15 16:04:17 -07:00
Simon Willison
1a7030d668 API explorer special case for rowid in /-/upsert
Refs #1936
2026-04-15 15:47:48 -07:00
Simon Willison
5f39036b9b ok: true in /db.json for consistency 2026-04-15 15:44:06 -07:00
Simon Willison
73f338b9f3 Better example in API explorer for /-/upsert, closes #1936 2026-04-15 15:29:59 -07:00
Simon Willison
4922fc2e39 Disallow null primary keys in upsert
Refs https://github.com/simonw/datasette/issues/1936#issuecomment-1341849496
2026-04-15 15:11:33 -07:00
Simon Willison
a973e3ffa1 Normalize headers in CSRF checks, refs #2689 2026-04-14 19:24:38 -07:00
Simon Willison
028cc2446f Don't allow cookies with Authorization: Bearer to bypass CSRF
Refs #2689
2026-04-14 19:23:21 -07:00
Simon Willison
f02484c3de From 409 warnings down to 52 warnings.
By closing unclosed database connections.

Refs #2614
2026-04-14 18:46:47 -07:00
Simon Willison
9c164572d3
Add actor= parameter to datasette.client methods (#2688)
`datasette.client.get(path, actor={"id": "root"}` now makes the internal request with that actor as `request.actor` - same for the other HTTP verb methods on `datasette.client`.

Upgraded relevant tests to use the new `actor=` mechanism.
2026-04-14 18:31:57 -07:00
Simon Willison
0b639a8122
Replace token-based CSRF with Sec-Fetch-Site header protection (#2689)
- New CSRF protection middleware inspired by Go 1.25 and research by Filippo Valsorda - https://words.filippo.io/csrf/ - this replaces the old CSRF token based protection.
- Removes all instances of `<input type="hidden" name="csrftoken" value="{{ csrftoken() }}">` in the templates - they are no longer needed.
- Removes the `def skip_csrf(datasette, scope):` plugin hook defined in `datasette/hookspecs.py` and its documentation and tests.
- Updated CSRF protection documentation to describe the new approach.
- Upgrade guide now describes the CSRF change.
2026-04-14 17:11:36 -07:00
Simon Willison
fc1794719a
Database(is_temp_disk=True) option, used for internal database (#2684)
Closes #2683

* Add is_temp_disk option to Database for temp file-backed databases

Replace the default in-memory internal database with a temporary
file-backed database using WAL mode. This fixes concurrent read/write
locking errors that occur with named in-memory SQLite databases.

The new is_temp_disk parameter on Database creates a temp file via
tempfile.mkstemp, connects to it as a regular file-based database
with WAL mode enabled, and cleans it up on close() and via atexit.

https://claude.ai/code/session_01TteLrUjpDcARjnP1GMRqz2
2026-03-30 21:03:21 -07:00
Simon Willison
94d14e3d37 Warning note about VACUUM and RenameTableEvent
I noticed that VACUUM can update the rootpage for tables in a way that
could confuse our rename table detection logic - but using the
execute_isolated_fn() method to run VACUUM avoids this problem.

Refs #2681
2026-03-30 16:11:06 -07:00
Simon Willison
312f41b0c2
RenameTableEvent, plus write connection track_event() mechanism (#2682)
* Add track_event callback to execute_write_fn and write_wrapper

Allows write functions and write_wrapper generators to queue events
during a write operation that are dispatched after successful commit.
The fn or wrapper can optionally accept a `track_event` parameter
(detected via call_with_supported_arguments). Events are discarded
if the write raises an exception.

Does not yet handle the block=False (non-blocking) case - events
queued during non-blocking writes are currently silently discarded.

Refs https://github.com/simonw/datasette/issues/2681

* Dispatch track_event events for non-blocking (block=False) writes

Spawns a background asyncio task that awaits the write thread's reply
queue and dispatches pending events after a successful non-blocking
write. Events are still discarded if the write raises an exception.

Refs https://github.com/simonw/datasette/issues/2681

* Warn that events won't fire for other processes

Refs https://github.com/simonw/datasette/issues/2681#issuecomment-4157118662
2026-03-30 11:20:46 -07:00
dependabot[bot]
9b5cb1347c
Bump rollup from 3.29.5 to 3.30.0 (#2651)
Bumps [rollup](https://github.com/rollup/rollup) from 3.29.5 to 3.30.0.
- [Release notes](https://github.com/rollup/rollup/releases)
- [Changelog](https://github.com/rollup/rollup/blob/v3.30.0/CHANGELOG.md)
- [Commits](https://github.com/rollup/rollup/compare/v3.29.5...v3.30.0)

---
updated-dependencies:
- dependency-name: rollup
  dependency-version: 3.30.0
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-30 10:54:48 -07:00
dependabot[bot]
1a64d5e55e
Bump picomatch from 2.3.1 to 2.3.2 (#2679)
Bumps [picomatch](https://github.com/micromatch/picomatch) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/2.3.1...2.3.2)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 2.3.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-30 10:54:34 -07:00
Simon Willison
c479e7dec9
Document call_with_supported_arguments as a supported public API (#2678)
* Document call_with_supported_arguments as a supported public API

Mark both call_with_supported_arguments and async_call_with_supported_arguments
with the @documented decorator and add documentation to docs/internals.rst
so plugin authors can use these dependency injection utilities in their own code.

https://claude.ai/code/session_01DKogZpHwzCTrbeG4XjXmNc
2026-03-30 10:44:10 -07:00
Simon Willison
4fcf474088 Release 1.0a26 1.0a26
Refs #1592, #2661, #2664, #2666, #2669, #2670, #2671, #2672
2026-03-18 15:13:37 -07:00
Simon Willison
c673ee9819 Update docs for async def resources_sql(cls, datasette, actor=None) signature 2026-03-18 15:08:52 -07:00
Simon Willison
cb293572c4 UI for setting custom column types, refs #2671 2026-03-18 14:08:44 -07:00
Simon Willison
cb5cc0cc22 Fixed some broken docs/ references, refs #2671 2026-03-18 13:02:31 -07:00
Simon Willison
611b8ad463 blacken-docs 2026-03-18 12:33:09 -07:00
Simon Willison
2b06da29a1 Rename set-column-types action to et-column-type
Refs https://github.com/simonw/datasette/pull/2674#issuecomment-4085015792
2026-03-18 12:33:09 -07:00
Simon Willison
d440c20984 /db/table/-/set-column-type JSON API, refs #2671 2026-03-18 12:33:09 -07:00
Simon Willison
fa1d8f0fa5 set-column-types permission, refs #2671 2026-03-18 12:33:09 -07:00
Simon Willison
feaba9b18b
Optionally limit ColumnType subclasses to specific SQLite types (#2673)
* ColumnTypes now have optional SQLite column types

Refs #2672
2026-03-18 11:37:09 -07:00
Simon Willison
0f81553b3f No hide this column on last remaining column 2026-03-18 10:47:05 -07:00
Simon Willison
68966880c2
Fix mobile column actions not showing items for SQL views (#2670)
* Fix mobile column actions not showing items for SQL views

The previous fix to exclude the Link column from mobile column actions
(d02072b) used .dropdown-menu-icon presence as a proxy, but dropdown
icons are only added to sortable columns (those with <a> tags). This
caused all non-sortable columns to be excluded too.

Instead, explicitly mark the Link column with a data-is-link-column
attribute and filter by that in mobileColumnHeaders, so non-sortable
columns on views and tables still appear in the mobile column actions.

* Prettier formatting for mobile-column-actions.js

https://claude.ai/code/session_01CG545gLcZxet7dS5nMzfCd
2026-03-18 09:46:36 -07:00
Simon Willison
d02072bc9d Do not show mobile column actions for Link column
Refs #2669
2026-03-18 09:08:00 -07:00
Simon Willison
e800312b54 allowed_resources(view-query, actor) fix
Previously we could not filter for canned queries that a
specific actor could view.
2026-03-18 09:05:23 -07:00
Simon Willison
fd016f7986
Column actions panel on mobile (#2669)
On mobile widths the column actions were no longer available.

This adds a new modal to help with that.

https://gisthost.github.io/?ec60eb27e22cf5d96642eec1715586b6
2026-03-18 09:04:28 -07:00
Simon Willison
63d73a806f
Move table configuration docs from metadata.rst to configuration.rst (#2668)
https://claude.ai/code/session_01UqboRB5Wt52BKPhxexUBEn
2026-03-17 08:47:04 -07:00
Simon Willison
bc7a19b39d
Column types system
Closes #2664
2026-03-16 22:29:51 -07:00
Claude
b7578a4884
Remove pointless test_column_type_with_config test
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:24:09 +00:00
Claude
c73a1c907a
Use 'subclass' instead of 'class' in ColumnType docs
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:22:53 +00:00
Claude
da0ea4382c
Update cog-generated plugin list to include default_column_types
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:18:42 +00:00
Claude
dd9b83301c
Refactor ColumnType: register classes, return instances with config
- register_column_types() now returns classes instead of instances
- ColumnType.__init__ takes optional config=, baking it into the instance
- get_column_type() returns a ColumnType instance (or None) instead of a
  (name, config) tuple
- get_column_types() returns {col: ColumnType instance} instead of tuples
- Remove get_column_type_class() - no longer needed
- render_cell/validate/transform_value methods no longer take config arg;
  use self.config instead
- render_cell hook takes column_type (ColumnType or None) instead of
  column_type + column_type_config

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:18:14 +00:00
Claude
8af98c24c2
Move name and description to class attributes on ColumnType
Instead of passing name= and description= as constructor arguments,
define them as class attributes on each subclass. This better reflects
that they are intrinsic to the type, not configurable per-instance.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:00:57 +00:00
Claude
72c8c71518
Move column_type defaults into dict literal
Set column_type and column_type_config to None in the initial
col_dict instead of using an else branch.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:54:36 +00:00
Claude
9fe10cd1aa
Apply black and blacken-docs formatting
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:50:58 +00:00
Claude
77bbfb5f7e
Document column type internal methods in internals.rst
Add documentation for get_column_type, get_column_types,
set_column_type, remove_column_type, and get_column_type_class
methods on the Datasette instance.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:08:58 +00:00
Claude
32e4a31913
Document register_column_types hook and updated render_cell signature
- Add register_column_types(datasette) hook documentation with example
- Update render_cell signature to include column_type and
  column_type_config parameters
- Fixes test_plugin_hooks_are_documented

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:03:52 +00:00
Claude
ad6a020e6d
Add NOT NULL constraints to column_types primary key columns
SQLite allows NULLs in primary key columns by default, so mark
database_name, resource_name, and column_name as NOT NULL explicitly.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 03:58:18 +00:00
Claude
5db4f6953d
Fix linting: remove unused import, apply black formatting
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 03:56:20 +00:00
Claude
de4269629b
Remove duplicate import logging line
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 03:55:51 +00:00