Commit graph

1,134 commits

Author SHA1 Message Date
Claude
170f9de774 Add pks parameter to render_cell() plugin hook
The render_cell() hook now receives a pks parameter containing the list
of primary key column names for the table being rendered. This avoids
plugins needing to make redundant async calls to look up primary keys.

For tables without an explicit primary key, pks is ["rowid"]. For custom
SQL queries and views, pks is an empty list [].

https://claude.ai/code/session_01HFYfevAziq4fSYTNRD9ZCh
2026-02-17 11:13:30 -08:00
Simon Willison
80b7f987ca
write_wrapper plugin hook for intercepting write operations (#2636)
* Implement write_wrapper plugin hook for intercepting database writes

Add a new `write_wrapper` plugin hook that lets plugins wrap write
operations with before/after logic using a generator-based context
manager pattern. The hook receives (datasette, database, request,
transaction) and returns a generator function that takes a conn,
yields once to let the write execute, and can run cleanup after.

The write result is sent back via `generator.send()` and exceptions
are thrown via `generator.throw()`, giving plugins full visibility.

Also adds `request=None` parameter to execute_write, execute_write_fn,
execute_write_script, and execute_write_many, and threads request
through all view-layer call sites (insert, upsert, update, delete,
drop, create table, canned queries).

* Add documentation for wrap_write hook, fix lint issues

Document the wrap_write plugin hook in plugin_hooks.rst with
parameter descriptions and two examples: a simple logging wrapper
and an advanced SQLite authorizer-based table protection pattern.

Also fix black formatting and remove unused variable flagged by ruff.

* Rename wrap_write hook to write_wrapper for consistency with asgi_wrapper
* Move write_wrapper docs to just below prepare_connection
* Refactor write_wrapper tests to use pytest.parametrize

Consolidate duplicate test cases: merge before/after tests for
execute_write_fn and execute_write into one parametrized test, and
merge three parameter-passing tests into one parametrized test.

Claude Code transcript: https://gisthost.github.io/?c4c12079434e69677e4aa8ac664b21b8/index.html
2026-02-09 13:20:33 -08:00
Simon Willison
5873578d49 Release 1.0a24
Refs #2050, #2346, #2608, #2609, #2610, #2611, #2613, #2619, #2624, #2627, #2628, #2629, #2630, #2632
2026-01-29 09:00:22 -08:00
Simon Willison
40a37307de
Add request.form() for multipart form data and file uploads
* Add request.form() for multipart form data and file uploads

New Request.form() method that handles both application/x-www-form-urlencoded
and multipart/form-data content types with streaming parsing.

Features:
- Streaming multipart parser that doesn't buffer entire body in memory
- Files spill to disk above 1MB threshold via SpooledTemporaryFile
- files=False (default) discards file content, files=True stores them
- Security limits: max_request_size, max_file_size, max_fields, max_files
- FormData container with dict-like access and getlist() for multiple values
- UploadedFile class with async read(), seek(), filename, content_type, size
- Support for RFC 5987 filename* encoding for international filenames

Uses multipart-form-data-conformance test suite for validation.

* Update views to use request.form() and document new API

- Migrate PermissionsDebugView, MessagesDebugView, and CreateTokenView
  from post_vars() to form()
- Add documentation for request.form(), FormData, and UploadedFile classes

Centralize multipart defaults and expose stricter limits via Request.form().

Enforce header, part, file, and disk space limits even when files are discarded; detect truncated bodies and client disconnects; and move blocking work off the event loop.

Add FormData close/aclose context managers, update internals docs, and expand multipart tests (including len semantics and stricter conformance expectations).
2026-01-28 18:41:03 -08:00
Simon Willison
b0436faa5e
Fix test isolation bug in test_startup_error_from_plugin_is_click_exception (#2627)
* Fix test isolation bug in test_startup_error_from_plugin_is_click_exception

The test creates a plugin that raises StartupError("boom") and registers it
in the global plugin manager (pm). Without cleanup, this plugin leaks to
subsequent tests, causing test_setting_boolean_validation_false_values to
fail with "Error: boom" instead of "Forbidden".

Add try/finally block to ensure the plugin is unregistered after the test
completes, following the established cleanup pattern used elsewhere in
the test suite.

* Fix blacken-docs formatting in plugin_hooks.rst

Apply blacken-docs formatting to code example that exceeded
the 60 character line limit.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-22 07:03:05 -08:00
Simon Willison
757ce92baf datasette.utils.StartupError() now becomes a click exception, closes #2624 2026-01-06 07:58:48 -08:00
Simon Willison
4cbdfcc07d
dependency-groups and uv (#2611)
* dependency-groups and uv, closes #2610
* New .readthedocs config for --group dev
2025-12-11 17:32:58 -08:00
Simon Willison
2ca00b6c75 Release 1.0a23
Refs #2605, #2599
2025-12-02 19:20:43 -08:00
Simon Willison
93b455239a Release notes for 1.0a22, closes #2596 2025-11-13 10:40:24 -08:00
Simon Willison
4b4add4d31 datasette.pm property, closes #2595 2025-11-13 10:31:03 -08:00
Simon Willison
5125bef573 datasette.in_client() method, closes #2594 2025-11-13 10:00:04 -08:00
Simon Willison
23a640d38b
datasette serve --default-deny option (#2593)
Closes #2592
2025-11-12 16:14:21 -08:00
Simon Willison
291f71ec6b
Remove out-dated plugin_hook_permission_allowed references 2025-11-11 21:59:26 -08:00
Simon Willison
a508fc4a8e Remove permission_allowed hook docs, closes #2588
Refs #2528
2025-11-07 16:50:00 -08:00
Simon Willison
8bc9b1ee03
/-/schema and /db/-/schema and /db/table/-/schema pages (plus .json/.md)
* Add schema endpoints for databases, instances, and tables

Closes: #2586

This commit adds new endpoints to view database schemas in multiple formats:

- /-/schema - View schemas for all databases (HTML, JSON, MD)
- /database/-/schema - View schema for a specific database (HTML, JSON, MD)
- /database/table/-/schema - View schema for a specific table (JSON, MD)

Features:
- Supports HTML, JSON, and Markdown output formats
- Respects view-database and view-table permissions
- Uses group_concat(sql, ';' || CHAR(10)) from sqlite_master to retrieve schemas
- Includes comprehensive tests covering all formats and permission checks

The JSON endpoints return:
- Instance level: {"schemas": [{"database": "name", "schema": "sql"}, ...]}
- Database level: {"database": "name", "schema": "sql"}
- Table level: {"database": "name", "table": "name", "schema": "sql"}

Markdown format provides formatted output with headings and SQL code blocks.

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 12:01:23 -08:00
Simon Willison
1df4028d78 add_memory_database(memory_name, name=None, route=None) 2025-11-05 15:18:17 -08:00
Simon Willison
257e1c1b1b Release 1.0a21
Refs #2429, #2511, #2578, #2583
2025-11-05 13:51:58 -08:00
Simon Willison
d814e81b32
datasette.client.get(..., skip_permission_checks=True)
Closes #2580
2025-11-05 13:38:01 -08:00
Simon Willison
3c2254463b
Release notes for 0.65.2
Adding those to main. Refs #2579
2025-11-05 10:25:37 -08:00
Simon Willison
f12f6cc2ab
Get publish cloudrun working with latest Cloud Run (#2581)
Refs:
- #2511

Filter out bad services, refs:
- https://github.com/simonw/datasette/pull/2581#issuecomment-3492243400
2025-11-05 09:28:41 -08:00
Simon Willison
ce464da34b datasette --get --headers option, closes #2578 2025-11-04 18:12:15 -08:00
Simon Willison
dc3f9fe9e4 Python 3.10, not 3.8 2025-11-03 14:42:59 -08:00
Simon Willison
5d4dfcec6b Fix for link from changelog not working
Annoyingly we now get a warning in the docs build about a duplicate label,
but it seems harmless enough.
2025-11-03 14:38:57 -08:00
Simon Willison
b3b8c5831b Fixed some broken reference links on upgrade guide 2025-11-03 14:34:29 -08:00
Simon Willison
b212895b97 Updated release notes for 1.0a20
Refs #2550
2025-11-03 14:27:41 -08:00
Simon Willison
fa978ec100 More upgrade tips, written by Claude Code
Refs #2549

From the datasette-atom upgrade, https://gistpreview.github.io/?d5047e04bbd9c20c59437916e21754ae
2025-11-02 12:02:45 -08:00
Simon Willison
2459285052 Additional upgrade notes by Codex CLI
Refs https://github.com/simonw/datasette/issues/2549#issuecomment-3477398336

Refs #2564
2025-11-01 20:32:42 -07:00
Simon Willison
506ce5b0ac Remove docs for obsolete register_permissions() hook, refs #2528
Also removed docs for datasette.get_permission() method which no longer exists.
2025-11-01 20:23:37 -07:00
Simon Willison
063bf7a96f Action() is kw_only, abbr= is optional, closes #2571 2025-11-01 20:20:17 -07:00
Simon Willison
b8cee8768e Completed upgrade guide, closes #2564 2025-11-01 18:57:56 -07:00
Simon Willison
a528555e84
Additional actor restriction should not grant access to additional actions (#2569)
Closes #2568
2025-11-01 18:38:29 -07:00
Simon Willison
2b962beaeb Fix permissions_execute_sql warnings in documentation 2025-11-01 11:52:23 -07:00
Simon Willison
5705ce0d95
Move takes_child/takes_parent information from Action to Resource (#2567)
Simplified Action by moving takes_child/takes_parent logic to Resource

- Removed InstanceResource - global actions are now simply those with resource_class=None
- Resource.parent_class - Replaced parent_name: str with parent_class: type[Resource] | None for direct class references
- Simplified Action dataclass - No more redundant fields, everything is derived from the Resource class structure
- Validation - The __init_subclass__ method now checks parent_class.parent_class to enforce the 2-level hierarchy

Closes #2563
2025-11-01 11:35:08 -07:00
Simon Willison
1f8995e776 upgrade-1.0a20.md, refs #2564
And another Markdown conversion, refs #2565
2025-10-31 19:13:41 -07:00
Simon Willison
47e4060469 Enable MyST Markdown docs, port events.rst, refs #2565 2025-10-31 16:38:04 -07:00
Simon Willison
48982a0ff5 Mark 1.0a20 unreleased
Refs #2550
2025-10-31 16:12:54 -07:00
Simon Willison
223dcc7c0e Remove unused link 2025-10-31 16:11:53 -07:00
Simon Willison
3184bfae54 Release notes for 1.0a20, refs #2550 2025-10-31 15:37:30 -07:00
Simon Willison
e5f392ae7a datasette.allowed_resources_sql() returns namedtuple 2025-10-31 15:07:37 -07:00
Simon Willison
400fa08e4c
Add keyset pagination to allowed_resources() (#2562)
* Add keyset pagination to allowed_resources()

This replaces the unbounded list return with PaginatedResources,
which supports efficient keyset pagination for handling thousands
of resources.

Closes #2560

Changes:
- allowed_resources() now returns PaginatedResources instead of list
- Added limit (1-1000, default 100) and next (keyset token) parameters
- Added include_reasons parameter (replaces allowed_resources_with_reasons)
- Removed allowed_resources_with_reasons() method entirely
- PaginatedResources.all() async generator for automatic pagination
- Uses tilde-encoding for tokens (matching table pagination)
- Updated all callers to use .resources accessor
- Updated documentation with new API and examples

The PaginatedResources object has:
- resources: List of Resource objects for current page
- next: Token for next page (None if no more results)
- all(): Async generator that yields all resources across pages

Example usage:
    page = await ds.allowed_resources("view-table", actor, limit=100)
    for table in page.resources:
        print(table.child)

    # Iterate all pages automatically
    async for table in page.all():
        print(table.child)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 14:50:46 -07:00
Simon Willison
b7ef968c6f Fixed some rST labels I broke 2025-10-31 09:15:39 -07:00
Simon Willison
e4be95b16c
Update permissions documentation for new action system (#2551) 2025-10-30 17:59:54 -07:00
Simon Willison
6a71bde37f
Permissions SQL API improvements (#2558)
* Neater design for PermissionSQL class, refs #2556
  - source is now automatically set to the source plugin
  - params is optional
* PermissionSQL.allow() and PermissionSQL.deny() shortcuts

Closes #2556

* Filter out temp database from attached_databases()

Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
2025-10-30 15:48:46 -07:00
Simon Willison
ce4b0794b2
Ported setup.py to pyproject.toml (#2555)
* Ported setup.py to pyproject.toml, refs #2553

* Make fixtures tests less flaky

The in-memory fixtures table was being shared between different
instances of the test client, leading to occasional errors when
running the full test suite.
2025-10-30 10:41:41 -07:00
Simon Willison
4fe1765dc3 Add test for RST heading underline lengths, closes #2544
Added test_rst_heading_underlines_match_title_length() to verify that RST
heading underlines match their title lengths. The test properly handles:
- Overline+underline style headings (skips validation for those)
- Empty lines before underlines (ignores them)
- Minimum 5-character underline length (avoids false positives)

Running this test identified 14 heading underline mismatches which have
been fixed across 5 documentation files:
- docs/authentication.rst (3 headings)
- docs/plugin_hooks.rst (4 headings)
- docs/internals.rst (5 headings)
- docs/deploying.rst (1 heading)
- docs/changelog.rst (1 heading)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
653c94209c Remove broken reference to datasette_ensure_permissions in changelog 2025-10-26 09:49:49 -07:00
Simon Willison
95286fbb60 Refactor check_visibility() to use Resource objects, refs #2537
Updated check_visibility() method signature to accept Resource objects
(DatabaseResource, TableResource, QueryResource) instead of plain strings
and tuples.

Changes:
- Updated check_visibility() signature to only accept Resource objects
- Added validation with helpful error message for incorrect types
- Updated all check_visibility() calls throughout the codebase:
  - datasette/views/database.py: Use DatabaseResource and QueryResource
  - datasette/views/special.py: Use DatabaseResource and TableResource
  - datasette/views/row.py: Use TableResource
  - datasette/views/table.py: Use TableResource
  - datasette/app.py: Use TableResource in expand_foreign_keys
- Updated tests to use Resource objects
- Updated documentation in docs/internals.rst:
  - Removed outdated permissions parameter
  - Updated examples to use Resource objects

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
e7c7e21277 Ran blacken-docs 2025-10-25 15:38:07 -07:00
Simon Willison
d7d7ead0ef Ran cog 2025-10-25 15:38:07 -07:00
Simon Willison
fabcfd68ad Add datasette.ensure_permission() method, refs #2525, refs #2528
Implements a new ensure_permission() method that is a convenience wrapper
around allowed() that raises Forbidden instead of returning False.

Changes:
- Added ensure_permission() method to datasette/app.py
- Updated all views to use ensure_permission() instead of the pattern:
  if not await self.ds.allowed(...): raise Forbidden(...)
- Updated docs/internals.rst to document the new method
- Removed old ensure_permissions() documentation (that method was already removed)

The new method simplifies permission enforcement in views and makes the
code more concise and consistent.
2025-10-25 15:38:07 -07:00