Commit graph

165 commits

Author SHA1 Message Date
Simon Willison
02a1468f1b Renamed canned queries to queries / stored queries in docs
And a few renames in code and YAML as well.
2026-05-26 15:17:51 -07:00
Simon Willison
b1029acc68 top_canned_query is now top_stored_query, closes #2747 2026-05-26 15:05:41 -07:00
Simon Willison
4bf1c4b065 Rename canned queries to queries/stored queries in docs 2026-05-26 14:54:35 -07:00
Simon Willison
3b26b7aff0 Document canned query hook removal
Refs #2735
2026-05-24 23:00:00 -07:00
Simon Willison
f403ea4e53 No need to alias description as description 2026-05-24 16:47:49 -07:00
Simon Willison
c1525cb467 Improved examples in JumpSQL docs 2026-05-23 21:01:18 -07:00
Simon Willison
c980234c41 JumpSQL(database=) parameter
Refs https://github.com/simonw/datasette/pull/2732#issuecomment-4527304912
2026-05-23 21:00:04 -07:00
Simon Willison
cef6aa85b6 Remove source and source_key columns from JumpSQL
Refs https://github.com/simonw/datasette/pull/2732#issuecomment-4527290391
2026-05-23 20:41:32 -07:00
Simon Willison
21a79b34b8 Improvements to Jump SQL columns
- Removed database_name and resource_name
- url can now optionally return JSON to reuse datasette.urls. methods
- description is now used as a truncated text description
2026-05-23 20:28:02 -07:00
Simon Willison
9c1f8621eb Request is always set for jump_items_sql() hook 2026-05-23 16:59:45 -07:00
Simon Willison
be1b5b2b5c Move debug links into jump menu 2026-05-23 16:57:09 -07:00
Simon Willison
f46c245563 blacken-docs 2026-05-23 08:58:51 -07:00
Simon Willison
8568320a23 Replace jump_start() hook with JavaScript makeJumpSections() hook 2026-05-22 21:13:49 -07:00
Simon Willison
6057c76165 Initial docs for jump_items_sql and jump_start hooks
Refs #2731
2026-05-21 23:28:35 -07:00
Simon Willison
0b639a8122
Replace token-based CSRF with Sec-Fetch-Site header protection (#2689)
- New CSRF protection middleware inspired by Go 1.25 and research by Filippo Valsorda - https://words.filippo.io/csrf/ - this replaces the old CSRF token based protection.
- Removes all instances of `<input type="hidden" name="csrftoken" value="{{ csrftoken() }}">` in the templates - they are no longer needed.
- Removes the `def skip_csrf(datasette, scope):` plugin hook defined in `datasette/hookspecs.py` and its documentation and tests.
- Updated CSRF protection documentation to describe the new approach.
- Upgrade guide now describes the CSRF change.
2026-04-14 17:11:36 -07:00
Simon Willison
312f41b0c2
RenameTableEvent, plus write connection track_event() mechanism (#2682)
* Add track_event callback to execute_write_fn and write_wrapper

Allows write functions and write_wrapper generators to queue events
during a write operation that are dispatched after successful commit.
The fn or wrapper can optionally accept a `track_event` parameter
(detected via call_with_supported_arguments). Events are discarded
if the write raises an exception.

Does not yet handle the block=False (non-blocking) case - events
queued during non-blocking writes are currently silently discarded.

Refs https://github.com/simonw/datasette/issues/2681

* Dispatch track_event events for non-blocking (block=False) writes

Spawns a background asyncio task that awaits the write thread's reply
queue and dispatches pending events after a successful non-blocking
write. Events are still discarded if the write raises an exception.

Refs https://github.com/simonw/datasette/issues/2681

* Warn that events won't fire for other processes

Refs https://github.com/simonw/datasette/issues/2681#issuecomment-4157118662
2026-03-30 11:20:46 -07:00
Simon Willison
c673ee9819 Update docs for async def resources_sql(cls, datasette, actor=None) signature 2026-03-18 15:08:52 -07:00
Simon Willison
cb5cc0cc22 Fixed some broken docs/ references, refs #2671 2026-03-18 13:02:31 -07:00
Simon Willison
feaba9b18b
Optionally limit ColumnType subclasses to specific SQLite types (#2673)
* ColumnTypes now have optional SQLite column types

Refs #2672
2026-03-18 11:37:09 -07:00
Claude
c73a1c907a
Use 'subclass' instead of 'class' in ColumnType docs
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:22:53 +00:00
Claude
dd9b83301c
Refactor ColumnType: register classes, return instances with config
- register_column_types() now returns classes instead of instances
- ColumnType.__init__ takes optional config=, baking it into the instance
- get_column_type() returns a ColumnType instance (or None) instead of a
  (name, config) tuple
- get_column_types() returns {col: ColumnType instance} instead of tuples
- Remove get_column_type_class() - no longer needed
- render_cell/validate/transform_value methods no longer take config arg;
  use self.config instead
- render_cell hook takes column_type (ColumnType or None) instead of
  column_type + column_type_config

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:18:14 +00:00
Claude
8af98c24c2
Move name and description to class attributes on ColumnType
Instead of passing name= and description= as constructor arguments,
define them as class attributes on each subclass. This better reflects
that they are intrinsic to the type, not configurable per-instance.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:00:57 +00:00
Claude
9fe10cd1aa
Apply black and blacken-docs formatting
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:50:58 +00:00
Claude
32e4a31913
Document register_column_types hook and updated render_cell signature
- Add register_column_types(datasette) hook documentation with example
- Update render_cell signature to include column_type and
  column_type_config parameters
- Fixes test_plugin_hooks_are_documented

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:03:52 +00:00
Simon Willison
c96dc5ce26
register_token_handler() plugin hook for custom API token backends (#2650)
Closes #2649

* Add register_token_handler plugin hook for pluggable token backends

Adds a new register_token_handler hook that allows plugins to provide
custom token creation and verification backends. This enables plugins
like datasette-oauth to issue tokens without depending on specific
backend plugins like datasette-auth-tokens.

Key changes:
- New datasette/tokens.py with TokenHandler base class and SignedTokenHandler
  (the default signed-token implementation moved here)
- New register_token_handler hookspec in hookspecs.py
- Datasette.create_token() is now async and delegates to token handlers
- New Datasette.verify_token() method tries all handlers in sequence
- handler= parameter on create_token() to select a specific backend
- TokenHandler exported from datasette package for plugin use
- Fixed actor_from_request loop to await all coroutines (avoids warnings)

* Add documentation and hook test for register_token_handler

Fixes CI failures: the new hook needs a section in docs/plugin_hooks.rst
(checked by test_plugin_hooks_are_documented) and a test_hook_* function
in test_plugins.py (checked by test_plugin_hooks_have_tests).

* Register tokens module as separate default plugin

Instead of re-exporting hookimpls from default_permissions/__init__.py,
register datasette.default_permissions.tokens as its own DEFAULT_PLUGINS
entry. Cleaner and avoids confusing import-for-side-effect patterns.

* Replace restrict_x params with TokenRestrictions dataclass

Consolidates the three separate restrict_all, restrict_database, and
restrict_resource parameters into a single TokenRestrictions dataclass.
Cleaner API surface for both Datasette.create_token() and
TokenHandler.create_token().

Also clarifies docs re: default handler selection via pluggy ordering.

* Add builder methods to TokenRestrictions

Adds allow_all(), allow_database(), and allow_resource() methods that
return self for chaining. Callers no longer need to manipulate nested
dicts directly:

    restrictions = (TokenRestrictions()
        .allow_all("view-instance")
        .allow_database("mydb", "create-table")
        .allow_resource("mydb", "mytable", "insert-row"))

* docs: add 1.0a25 upgrade guide section for create_token() signature change

Ref: https://github.com/simonw/datasette/issues/2649#issuecomment-3962639393

* docs: note that create_token() is now async in upgrade guide

* docs: update internals, plugin_hooks, authentication for new token API

- internals.rst: new async create_token() signature with restrictions
  and handler params, add TokenRestrictions reference docs
- plugin_hooks.rst: show full create_token signature in TokenHandler
  example, note list returns and error cases
- authentication.rst: cross-reference TokenRestrictions from the
  restrictions section

* style: apply black formatting to token handler files

* docs: fix RST heading underline length in internals.rst

* tests: add restrictions round-trip and expiration tests for token handler

Covers allow_database/allow_resource builders, _r payload encoding,
and token_expires in verified actors. Coverage 76% -> 90%.

* tests: add test for signed tokens disabled

* fix: add TokenRestrictions TYPE_CHECKING import to fix ruff F821

* docs: regenerate plugins.rst with cog

* docs: reformat code blocks in plugin_hooks.rst with blacken-docs

* docs: add await .verify_token() to internals.rst

* tests: rewrite register_token_handler test to use real plugin handler

Adds a HardcodedTokenHandler to the test plugins dir that creates
tokens like dstok_hardcoded_token_1. The test now exercises creating
tokens via the default handler (which is the plugin's hardcoded one),
by explicitly naming the hardcoded handler, and by explicitly naming
the signed handler -- then verifies each token round-trips correctly.

* tests: clarify test_token_handler_via_http tests the default signed handler

* fix: use handler="signed" explicitly where signed tokens are expected

The HardcodedTokenHandler in my_plugin.py gets globally registered,
so create_token() without a handler name picks it up as the default.
Fix the create-token view, CLI, and tests to explicitly request the
signed handler where they depend on signed token behavior.

* fix: use handler="signed" in test_create_table_permissions

https://claude.ai/code/session_013cQFiDQjYRrRBH2biFfKuS
2026-02-25 16:32:45 -08:00
Simon Willison
6a2c27b15b blacken-docs 2026-02-20 11:28:39 -08:00
Claude
170f9de774 Add pks parameter to render_cell() plugin hook
The render_cell() hook now receives a pks parameter containing the list
of primary key column names for the table being rendered. This avoids
plugins needing to make redundant async calls to look up primary keys.

For tables without an explicit primary key, pks is ["rowid"]. For custom
SQL queries and views, pks is an empty list [].

https://claude.ai/code/session_01HFYfevAziq4fSYTNRD9ZCh
2026-02-17 11:13:30 -08:00
Simon Willison
80b7f987ca
write_wrapper plugin hook for intercepting write operations (#2636)
* Implement write_wrapper plugin hook for intercepting database writes

Add a new `write_wrapper` plugin hook that lets plugins wrap write
operations with before/after logic using a generator-based context
manager pattern. The hook receives (datasette, database, request,
transaction) and returns a generator function that takes a conn,
yields once to let the write execute, and can run cleanup after.

The write result is sent back via `generator.send()` and exceptions
are thrown via `generator.throw()`, giving plugins full visibility.

Also adds `request=None` parameter to execute_write, execute_write_fn,
execute_write_script, and execute_write_many, and threads request
through all view-layer call sites (insert, upsert, update, delete,
drop, create table, canned queries).

* Add documentation for wrap_write hook, fix lint issues

Document the wrap_write plugin hook in plugin_hooks.rst with
parameter descriptions and two examples: a simple logging wrapper
and an advanced SQLite authorizer-based table protection pattern.

Also fix black formatting and remove unused variable flagged by ruff.

* Rename wrap_write hook to write_wrapper for consistency with asgi_wrapper
* Move write_wrapper docs to just below prepare_connection
* Refactor write_wrapper tests to use pytest.parametrize

Consolidate duplicate test cases: merge before/after tests for
execute_write_fn and execute_write into one parametrized test, and
merge three parameter-passing tests into one parametrized test.

Claude Code transcript: https://gisthost.github.io/?c4c12079434e69677e4aa8ac664b21b8/index.html
2026-02-09 13:20:33 -08:00
Simon Willison
b0436faa5e
Fix test isolation bug in test_startup_error_from_plugin_is_click_exception (#2627)
* Fix test isolation bug in test_startup_error_from_plugin_is_click_exception

The test creates a plugin that raises StartupError("boom") and registers it
in the global plugin manager (pm). Without cleanup, this plugin leaks to
subsequent tests, causing test_setting_boolean_validation_false_values to
fail with "Error: boom" instead of "Forbidden".

Add try/finally block to ensure the plugin is unregistered after the test
completes, following the established cleanup pattern used elsewhere in
the test suite.

* Fix blacken-docs formatting in plugin_hooks.rst

Apply blacken-docs formatting to code example that exceeded
the 60 character line limit.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-22 07:03:05 -08:00
Simon Willison
757ce92baf datasette.utils.StartupError() now becomes a click exception, closes #2624 2026-01-06 07:58:48 -08:00
Simon Willison
a508fc4a8e Remove permission_allowed hook docs, closes #2588
Refs #2528
2025-11-07 16:50:00 -08:00
Simon Willison
506ce5b0ac Remove docs for obsolete register_permissions() hook, refs #2528
Also removed docs for datasette.get_permission() method which no longer exists.
2025-11-01 20:23:37 -07:00
Simon Willison
063bf7a96f Action() is kw_only, abbr= is optional, closes #2571 2025-11-01 20:20:17 -07:00
Simon Willison
2b962beaeb Fix permissions_execute_sql warnings in documentation 2025-11-01 11:52:23 -07:00
Simon Willison
5705ce0d95
Move takes_child/takes_parent information from Action to Resource (#2567)
Simplified Action by moving takes_child/takes_parent logic to Resource

- Removed InstanceResource - global actions are now simply those with resource_class=None
- Resource.parent_class - Replaced parent_name: str with parent_class: type[Resource] | None for direct class references
- Simplified Action dataclass - No more redundant fields, everything is derived from the Resource class structure
- Validation - The __init_subclass__ method now checks parent_class.parent_class to enforce the 2-level hierarchy

Closes #2563
2025-11-01 11:35:08 -07:00
Simon Willison
6a71bde37f
Permissions SQL API improvements (#2558)
* Neater design for PermissionSQL class, refs #2556
  - source is now automatically set to the source plugin
  - params is optional
* PermissionSQL.allow() and PermissionSQL.deny() shortcuts

Closes #2556

* Filter out temp database from attached_databases()

Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
2025-10-30 15:48:46 -07:00
Simon Willison
ce4b0794b2
Ported setup.py to pyproject.toml (#2555)
* Ported setup.py to pyproject.toml, refs #2553

* Make fixtures tests less flaky

The in-memory fixtures table was being shared between different
instances of the test client, leading to occasional errors when
running the full test suite.
2025-10-30 10:41:41 -07:00
Simon Willison
4fe1765dc3 Add test for RST heading underline lengths, closes #2544
Added test_rst_heading_underlines_match_title_length() to verify that RST
heading underlines match their title lengths. The test properly handles:
- Overline+underline style headings (skips validation for those)
- Empty lines before underlines (ignores them)
- Minimum 5-character underline length (avoids false positives)

Running this test identified 14 heading underline mismatches which have
been fixed across 5 documentation files:
- docs/authentication.rst (3 headings)
- docs/plugin_hooks.rst (4 headings)
- docs/internals.rst (5 headings)
- docs/deploying.rst (1 heading)
- docs/changelog.rst (1 heading)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
a2994cc5bb Remove automatic parameter namespacing from permission plugins
Simplifies the permission system by removing automatic parameter namespacing.
Plugins are now responsible for using unique parameter names. The recommended
convention is to prefix parameters with the plugin source name (e.g.,
:myplugin_user_id). System reserves :actor, :actor_id, :action, :filter_parent.

- Remove _namespace_params() function from datasette/utils/permissions.py
- Update build_rules_union() to use plugin params directly
- Document parameter naming convention in plugin_hooks.rst
- Update example plugins to use prefixed parameters
- Add test_multiple_plugins_with_own_parameters() to verify convention works
2025-10-24 11:44:43 -07:00
Simon Willison
b8d26754df Document datasette.allowed(), PermissionSQL class, and SQL parameters
- Added documentation for datasette.allowed() method with keyword-only arguments
- Added comprehensive PermissionSQL class documentation with examples
- Documented the three SQL parameters available: :actor, :actor_id, :action
- Included examples of using json_extract() to access actor fields
- Explained permission resolution rules (specificity, deny over allow, implicit deny)
- Fixed RST formatting warnings (escaped asterisk, fixed underline length)
2025-10-24 10:32:18 -07:00
Simon Willison
2b879e462f Implement resource-based permission system with SQL-driven access control
This introduces a new hierarchical permission system that uses SQL queries
for efficient permission checking across resources. The system replaces the
older permission_allowed() pattern with a more flexible resource-based
approach.

Core changes:

- New Resource ABC and Action dataclass in datasette/permissions.py
  * Resources represent hierarchical entities (instance, database, table)
  * Each resource type implements resources_sql() to list all instances
  * Actions define operations on resources with cascading rules

- New plugin hook: register_actions(datasette)
  * Plugins register actions with their associated resource types
  * Replaces register_permissions() and register_resource_types()
  * See docs/plugin_hooks.rst for full documentation

- Three new Datasette methods for permission checks:
  * allowed_resources(action, actor) - returns list[Resource]
  * allowed_resources_with_reasons(action, actor) - for debugging
  * allowed(action, resource, actor) - checks single resource
  * All use SQL for filtering, never Python iteration

- New /-/tables endpoint (TablesView)
  * Returns JSON list of tables user can view
  * Supports ?q= parameter for regex filtering
  * Format: {"matches": [{"name": "db/table", "url": "/db/table"}]}
  * Respects all permission rules from configuration and plugins

- SQL-based permission evaluation (datasette/utils/actions_sql.py)
  * Cascading rules: child-level → parent-level → global-level
  * DENY beats ALLOW at same specificity
  * Uses CTEs for efficient SQL-only filtering
  * Combines permission_resources_sql() hook results

- Default actions in datasette/default_actions.py
  * InstanceResource, DatabaseResource, TableResource, QueryResource
  * Core actions: view-instance, view-database, view-table, etc.

- Fixed default_permissions.py to handle database-level allow blocks
  * Now creates parent-level rules for view-table action
  * Fixes: datasette ... -s databases.fixtures.allow.id root

Documentation:

- Comprehensive register_actions() hook documentation
- Detailed resources_sql() method explanation
- /-/tables endpoint documentation in docs/introspection.rst
- Deprecated register_permissions() with migration guide

Tests:

- tests/test_actions_sql.py: 7 tests for core permission API
- tests/test_tables_endpoint.py: 13 tests for /-/tables endpoint
- All 118 documentation tests pass
- Tests verify SQL does filtering (not Python)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:32:18 -07:00
Simon Willison
27084caa04
New allowed_resources_sql plugin hook and debug tools (#2505)
* allowed_resources_sql plugin hook and infrastructure
* New methods for checking permissions with the new system
* New /-/allowed and /-/check and /-/rules special endpoints

Still needs to be integrated more deeply into Datasette, especially for listing visible tables.

Refs: #2502

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-08 14:27:51 -07:00
Simon Willison
d021ce97aa
Note that only first actor_from_request value is respected
https://github.com/datasette/datasette-profiles/issues/4#issuecomment-2758588167
2025-03-27 09:09:57 -07:00
Simon Willison
209bdee0e8 Don't run prepare_connection() on internal database, closes #2468 2025-02-18 10:23:23 -08:00
Simon Willison
cd9182a551 Release 1.0a17
Refs #1690, #1943, #2422, #2424, #2441, #2454, #2455, #2458, #2460, #2465
2025-02-06 11:12:34 -08:00
Simon Willison
e85517dab3 blacken-docs, refs #2441 2024-11-15 13:34:45 -08:00
Simon Willison
dce718961c Async support for magic parameters
Closes #2441
2024-11-15 13:17:45 -08:00
Simon Willison
8dc9bfa2ab Markup tweak for track_event docs 2024-08-05 12:58:10 -07:00
Simon Willison
bd7d3bb70f Tweaks and improvements to upgrade guide, refs ##2374
Also refs #2381
2024-08-05 12:11:17 -07:00
Alex Garcia
e1bfab3fca
Move Metadata to --internal database
Refs:
- https://github.com/simonw/datasette/pull/2343
- https://github.com/simonw/datasette/issues/2341
2024-06-11 09:33:23 -07:00