asgi_send_redirect() only collapsed leading forward slashes, so a path
like /\example.com/ produced a Location of /\example.com. Browsers
normalise backslashes to forward slashes, turning that into the
protocol-relative //example.com and redirecting off-site. Collapse any
run of leading slashes and backslashes to a single slash.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
escape_sqlite() wrapped identifiers in [brackets] without escaping any ]
characters inside the string. Since SQLite does not support escaping ]
within bracket quoting, an identifier containing ] could break out and
inject arbitrary SQL. Fall back to double-quote quoting (doubling any
embedded ") when the identifier contains ].
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Fix for execute write returning, closes#2762
* Fix stored write returning rowcount message
* Add configurable execute_write returning limit
* Return rows/truncated from execute query if it used RETURNING
* INSERT ... RETURNING shows rows in /-/execute-write
* Skip RETURNING tests if SQLite version does not support it
Screenshot: https://github.com/simonw/datasette/issues/2762#issuecomment-4588111545
Use the router-stripped route_path when building request-derived export
URLs, so table, row, and query JSON/CSV links do not apply base_url twice.
Keep urls.path() behavior unchanged, and add coverage for both /prefix/
exports and a /data/ base_url with a data database.
Closes#2759
Remove the redundant SQLTableKind alias from the write SQL analysis model. Operation.table_kind and the analyzer cache now use the SQLite metadata classification type directly, making the source of table-kind values clearer.
Stop marking sqlite_master and sqlite_schema reads as internal as soon as the SQLite authorizer reports them. The later DDL-aware pass still treats schema catalog access as internal when it accompanies semantic CREATE, ALTER, or DROP operations.
This makes explicit catalog reads in write SQL fall through to the deny-by-default path as unsupported read schema operations, preventing queries from copying private table definitions into writable tables.
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4559073803
Require view-table permission for reads discovered inside write SQL analysis, including INSERT ... SELECT and CREATE TABLE ... AS SELECT.
Record additional SQLite authorizer callbacks as Operation values so unsupported functions, savepoints, virtual table DDL, and unknown callbacks are denied unless explicitly handled.
Experimental, we may need this for the upcoming canned query
work so that we can tell if a user should be able to save
a writable canned query by confirming they have the right
permissions to update the affected tables.
Refs #2735
- New CSRF protection middleware inspired by Go 1.25 and research by Filippo Valsorda - https://words.filippo.io/csrf/ - this replaces the old CSRF token based protection.
- Removes all instances of `<input type="hidden" name="csrftoken" value="{{ csrftoken() }}">` in the templates - they are no longer needed.
- Removes the `def skip_csrf(datasette, scope):` plugin hook defined in `datasette/hookspecs.py` and its documentation and tests.
- Updated CSRF protection documentation to describe the new approach.
- Upgrade guide now describes the CSRF change.
* Document call_with_supported_arguments as a supported public API
Mark both call_with_supported_arguments and async_call_with_supported_arguments
with the @documented decorator and add documentation to docs/internals.rst
so plugin authors can use these dependency injection utilities in their own code.
https://claude.ai/code/session_01DKogZpHwzCTrbeG4XjXmNc
Implements the column types feature that lets Datasette and plugins annotate
columns with semantic types beyond SQLite storage types (e.g. markdown, email,
url, json, file, point). This enables type-appropriate rendering, validation,
form widgets, and API behavior.
Key changes:
- New `column_types` internal DB table for storing assignments
- `ColumnType` dataclass in datasette/column_types.py with render_cell,
validate, and transform_value methods
- `register_column_types` plugin hook for registering types
- Built-in url, email, and json column types
- Datasette API methods: get/set/remove_column_type(s),
get_column_type_class
- Config loading from datasette.json `column_types` table config key
- `column_types` extra on the table JSON endpoint
- Column type info in display_columns extra
- Column type render_cell gets priority in rendering pipeline
- column_type/column_type_config args added to render_cell hookspec
- Write-path validation on insert and update
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
* Add request.form() for multipart form data and file uploads
New Request.form() method that handles both application/x-www-form-urlencoded
and multipart/form-data content types with streaming parsing.
Features:
- Streaming multipart parser that doesn't buffer entire body in memory
- Files spill to disk above 1MB threshold via SpooledTemporaryFile
- files=False (default) discards file content, files=True stores them
- Security limits: max_request_size, max_file_size, max_fields, max_files
- FormData container with dict-like access and getlist() for multiple values
- UploadedFile class with async read(), seek(), filename, content_type, size
- Support for RFC 5987 filename* encoding for international filenames
Uses multipart-form-data-conformance test suite for validation.
* Update views to use request.form() and document new API
- Migrate PermissionsDebugView, MessagesDebugView, and CreateTokenView
from post_vars() to form()
- Add documentation for request.form(), FormData, and UploadedFile classes
Centralize multipart defaults and expose stricter limits via Request.form().
Enforce header, part, file, and disk space limits even when files are discarded; detect truncated bodies and client disconnects; and move blocking work off the event loop.
Add FormData close/aclose context managers, update internals docs, and expand multipart tests (including len semantics and stricter conformance expectations).
* Fix flaky test_database_page test with deterministic ordering
- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst
* Update test_row_foreign_key_tables for new deterministic FK ordering
The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.
* Update test_table_names for new alphabetical ordering
The table_names() method now returns tables sorted alphabetically.
* Fix for test that fails prior to SQLite 3.37
---------
Co-authored-by: Claude <noreply@anthropic.com>
Implement INTERSECT-based actor restrictions to prevent permission bypass
Actor restrictions are now implemented as SQL filters using INTERSECT rather
than as deny/allow permission rules. This ensures restrictions act as hard
limits that cannot be overridden by other permission plugins or config blocks.
Previously, actor restrictions (_r in actor dict) were implemented by
generating permission rules with deny/allow logic. This approach had a
critical flaw: database-level config allow blocks could bypass table-level
restrictions, granting access to tables not in the actor's allowlist.
The new approach separates concerns:
- Permission rules determine what's allowed based on config and plugins
- Restriction filters limit the result set to only allowlisted resources
- Restrictions use INTERSECT to ensure all restriction criteria are met
- Database-level restrictions (parent, NULL) properly match all child tables
Implementation details:
- Added restriction_sql field to PermissionSQL dataclass
- Made PermissionSQL.sql optional to support restriction-only plugins
- Updated actor_restrictions_sql() to return restriction filters instead of rules
- Modified SQL builders to apply restrictions via INTERSECT and EXISTS clauses
Closes#2572
* Add keyset pagination to allowed_resources()
This replaces the unbounded list return with PaginatedResources,
which supports efficient keyset pagination for handling thousands
of resources.
Closes#2560
Changes:
- allowed_resources() now returns PaginatedResources instead of list
- Added limit (1-1000, default 100) and next (keyset token) parameters
- Added include_reasons parameter (replaces allowed_resources_with_reasons)
- Removed allowed_resources_with_reasons() method entirely
- PaginatedResources.all() async generator for automatic pagination
- Uses tilde-encoding for tokens (matching table pagination)
- Updated all callers to use .resources accessor
- Updated documentation with new API and examples
The PaginatedResources object has:
- resources: List of Resource objects for current page
- next: Token for next page (None if no more results)
- all(): Async generator that yields all resources across pages
Example usage:
page = await ds.allowed_resources("view-table", actor, limit=100)
for table in page.resources:
print(table.child)
# Iterate all pages automatically
async for table in page.all():
print(table.child)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Neater design for PermissionSQL class, refs #2556
- source is now automatically set to the source plugin
- params is optional
* PermissionSQL.allow() and PermissionSQL.deny() shortcuts
Closes#2556
* Filter out temp database from attached_databases()
Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:
- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
(was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
(fixed 71 test errors caused by fixtures not being registered)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Modified /-/allowed to show all reasons that grant access to a resource
- Changed from MAX(reason) to json_group_array() in SQL to collect all reasons
- Reasons now displayed as JSON arrays in both HTML and JSON responses
- Only show Reason column to users with permissions-debug permission
- Removed obsolete "Source Plugin" column from /-/rules interface
- Updated allowed_resources_with_reasons() to parse and return reason lists
- Fixed alert() on /-/allowed by replacing with disabled input state
This change integrates canned queries with Datasette's new SQL-based
permissions system by making the following changes:
1. **Default canned_queries plugin hook**: Added a new hookimpl in
default_permissions.py that returns canned queries from datasette
configuration. This extracts config-reading logic into a plugin hook,
allowing QueryResource to discover all queries.
2. **Async resources_sql()**: Converted Resource.resources_sql() from a
synchronous class method returning a string to an async method that
receives the datasette instance. This allows QueryResource to call
plugin hooks and query the database.
3. **QueryResource implementation**: Implemented QueryResource.resources_sql()
to gather all canned queries by:
- Querying catalog_databases for all databases
- Calling canned_queries hooks for each database with actor=None
- Building a UNION ALL SQL query of all (database, query_name) pairs
- Properly escaping single quotes in resource names
4. **Simplified get_canned_queries()**: Removed config-reading logic since
it's now handled by the default plugin hook.
5. **Added view-query to default allow**: Added "view-query" to the
default_allow_actions set so canned queries are accessible by default.
6. **Removed xfail markers**: Removed test xfail markers from:
- tests/test_canned_queries.py (entire module)
- tests/test_html.py (2 tests)
- tests/test_permissions.py (1 test)
- tests/test_plugins.py (1 test)
All canned query tests now pass with the new permission system.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Rename permission_name to action_name in debug templates for consistency
- Remove confusing WHERE 0 check from check_permission_for_resource()
- Rename tests/test_special.py to tests/test_search_tables.py
- Remove tests/vec.db that shouldn't have been committed
