Closes:
- #2709
The key behavior change: after close() starts, no new execute work can be submitted, but already-running execute work is allowed to finish before SQLite connections are closed.
Session-scoped fixtures are cached per worker by pytest itself, so the
manual _ds_client module global is no longer needed.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ds_client already caches a single Datasette for the whole session via a
module-level _ds_client global, so the declared fixture scope should
match. With function scope the auto-close plugin correctly closes it
after the first test that uses it, which then breaks every subsequent
test that reuses the cached (now-closed) instance — as seen in the CI
coverage job, which runs serially rather than under pytest-xdist.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Creates and disposes 50 Datasette instances in a loop and asserts that
the number of open file descriptors and live threads does not grow,
exercising the full close() path end to end.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Installs a pytest11 entry point so that every Datasette() constructed
inside a pytest_runtest_call phase is auto-closed at the end of the test.
Fixture-scoped instances are untouched. Opt out via the
datasette_autoclose = false ini option.
This gives large test suites a safety net against FD exhaustion and leaked
write threads from the now-default temp-disk internal database without
requiring every existing test to be rewritten.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AsgiLifespan now receives an on_shutdown callback that invokes
Datasette.close(), so resources are released cleanly when the ASGI server
delivers a lifespan.shutdown message (SIGTERM / SIGINT for uvicorn).
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Datasette.close() iterates over every attached Database (including the
internal database), calls Database.close() on each, then shuts down the
ThreadPoolExecutor. Exceptions raised by one Database don't prevent the
others from being closed; the first exception is re-raised afterwards.
Idempotent.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After this commit, Database.close() sends a sentinel to the write queue so
the background write thread exits cleanly, closes cached read/write
connections, and marks the instance closed. Subsequent calls to execute*()
raise DatasetteClosedError. close() remains idempotent and one-way.
Refs #2692
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`datasette.client.get(path, actor={"id": "root"}` now makes the internal request with that actor as `request.actor` - same for the other HTTP verb methods on `datasette.client`.
Upgraded relevant tests to use the new `actor=` mechanism.
- New CSRF protection middleware inspired by Go 1.25 and research by Filippo Valsorda - https://words.filippo.io/csrf/ - this replaces the old CSRF token based protection.
- Removes all instances of `<input type="hidden" name="csrftoken" value="{{ csrftoken() }}">` in the templates - they are no longer needed.
- Removes the `def skip_csrf(datasette, scope):` plugin hook defined in `datasette/hookspecs.py` and its documentation and tests.
- Updated CSRF protection documentation to describe the new approach.
- Upgrade guide now describes the CSRF change.
Closes#2683
* Add is_temp_disk option to Database for temp file-backed databases
Replace the default in-memory internal database with a temporary
file-backed database using WAL mode. This fixes concurrent read/write
locking errors that occur with named in-memory SQLite databases.
The new is_temp_disk parameter on Database creates a temp file via
tempfile.mkstemp, connects to it as a regular file-based database
with WAL mode enabled, and cleans it up on close() and via atexit.
https://claude.ai/code/session_01TteLrUjpDcARjnP1GMRqz2
* Add track_event callback to execute_write_fn and write_wrapper
Allows write functions and write_wrapper generators to queue events
during a write operation that are dispatched after successful commit.
The fn or wrapper can optionally accept a `track_event` parameter
(detected via call_with_supported_arguments). Events are discarded
if the write raises an exception.
Does not yet handle the block=False (non-blocking) case - events
queued during non-blocking writes are currently silently discarded.
Refs https://github.com/simonw/datasette/issues/2681
* Dispatch track_event events for non-blocking (block=False) writes
Spawns a background asyncio task that awaits the write thread's reply
queue and dispatches pending events after a successful non-blocking
write. Events are still discarded if the write raises an exception.
Refs https://github.com/simonw/datasette/issues/2681
* Warn that events won't fire for other processes
Refs https://github.com/simonw/datasette/issues/2681#issuecomment-4157118662
* Fix mobile column actions not showing items for SQL views
The previous fix to exclude the Link column from mobile column actions
(d02072b) used .dropdown-menu-icon presence as a proxy, but dropdown
icons are only added to sortable columns (those with <a> tags). This
caused all non-sortable columns to be excluded too.
Instead, explicitly mark the Link column with a data-is-link-column
attribute and filter by that in mobileColumnHeaders, so non-sortable
columns on views and tables still appear in the mobile column actions.
* Prettier formatting for mobile-column-actions.js
https://claude.ai/code/session_01CG545gLcZxet7dS5nMzfCd
- register_column_types() now returns classes instead of instances
- ColumnType.__init__ takes optional config=, baking it into the instance
- get_column_type() returns a ColumnType instance (or None) instead of a
(name, config) tuple
- get_column_types() returns {col: ColumnType instance} instead of tuples
- Remove get_column_type_class() - no longer needed
- render_cell/validate/transform_value methods no longer take config arg;
use self.config instead
- render_cell hook takes column_type (ColumnType or None) instead of
column_type + column_type_config
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
Instead of passing name= and description= as constructor arguments,
define them as class attributes on each subclass. This better reflects
that they are intrinsic to the type, not configurable per-instance.
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
- Add transform_value integration in table JSON endpoint rows
- Add tests for: duplicate type name error, row endpoint rendering,
transform_value in JSON output, column type priority over plugins,
row detail HTML rendering, table HTML rendering, upsert validation,
unknown type warning logging, config overwrite on restart, and
no-config edge case
- Total: 34 column type tests, all passing
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
Implements the column types feature that lets Datasette and plugins annotate
columns with semantic types beyond SQLite storage types (e.g. markdown, email,
url, json, file, point). This enables type-appropriate rendering, validation,
form widgets, and API behavior.
Key changes:
- New `column_types` internal DB table for storing assignments
- `ColumnType` dataclass in datasette/column_types.py with render_cell,
validate, and transform_value methods
- `register_column_types` plugin hook for registering types
- Built-in url, email, and json column types
- Datasette API methods: get/set/remove_column_type(s),
get_column_type_class
- Config loading from datasette.json `column_types` table config key
- `column_types` extra on the table JSON endpoint
- Column type info in display_columns extra
- Column type render_cell gets priority in rendering pipeline
- column_type/column_type_config args added to render_cell hookspec
- Write-path validation on insert and update
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
* Fix startup hook to fire after metadata and schema tables are populated
Previously, the startup() plugin hook fired before internal database
tables were populated from metadata.yaml and before catalog schema
tables were filled. This meant plugins couldn't read or modify metadata
during startup. Now invoke_startup() calls refresh_schemas() before
firing startup hooks, ensuring metadata and catalog tables are available.
* Fix startup hook to fire after metadata and schema tables are populated
Previously, the startup() plugin hook fired before internal database
tables were populated from metadata.yaml and before catalog schema
tables were filled. This meant plugins couldn't read or modify metadata
during startup. Now invoke_startup() calls _refresh_schemas() before
firing startup hooks, ensuring metadata and catalog tables are available.
Updated test_tracer to reflect that internal DB creation SQL now runs
during startup rather than during the first traced request.
* Move check_databases before invoke_startup in CLI serve
Since invoke_startup now calls _refresh_schemas() which queries each
database, the spatialite connection check must run first to provide
the friendly error message instead of a raw OperationalError.
https://claude.ai/code/session_01KL4t5FZYb32rZY7xaqrrZU
