Commit graph

194 commits

Author SHA1 Message Date
Simon Willison
040e42ddca Enforce query ownership and remove canned query hook
Refs #2735
2026-05-24 22:58:50 -07:00
Simon Willison
1000d50220 datasette.fixtures module, closes #2733
https://gist.github.com/simonw/613be79094d491dd08f45e05f4f70691
2026-05-21 23:05:37 -07:00
Simon Willison
f02484c3de From 409 warnings down to 52 warnings.
By closing unclosed database connections.

Refs #2614
2026-04-14 18:46:47 -07:00
Simon Willison
0b639a8122
Replace token-based CSRF with Sec-Fetch-Site header protection (#2689)
- New CSRF protection middleware inspired by Go 1.25 and research by Filippo Valsorda - https://words.filippo.io/csrf/ - this replaces the old CSRF token based protection.
- Removes all instances of `<input type="hidden" name="csrftoken" value="{{ csrftoken() }}">` in the templates - they are no longer needed.
- Removes the `def skip_csrf(datasette, scope):` plugin hook defined in `datasette/hookspecs.py` and its documentation and tests.
- Updated CSRF protection documentation to describe the new approach.
- Upgrade guide now describes the CSRF change.
2026-04-14 17:11:36 -07:00
Simon Willison
c96dc5ce26
register_token_handler() plugin hook for custom API token backends (#2650)
Closes #2649

* Add register_token_handler plugin hook for pluggable token backends

Adds a new register_token_handler hook that allows plugins to provide
custom token creation and verification backends. This enables plugins
like datasette-oauth to issue tokens without depending on specific
backend plugins like datasette-auth-tokens.

Key changes:
- New datasette/tokens.py with TokenHandler base class and SignedTokenHandler
  (the default signed-token implementation moved here)
- New register_token_handler hookspec in hookspecs.py
- Datasette.create_token() is now async and delegates to token handlers
- New Datasette.verify_token() method tries all handlers in sequence
- handler= parameter on create_token() to select a specific backend
- TokenHandler exported from datasette package for plugin use
- Fixed actor_from_request loop to await all coroutines (avoids warnings)

* Add documentation and hook test for register_token_handler

Fixes CI failures: the new hook needs a section in docs/plugin_hooks.rst
(checked by test_plugin_hooks_are_documented) and a test_hook_* function
in test_plugins.py (checked by test_plugin_hooks_have_tests).

* Register tokens module as separate default plugin

Instead of re-exporting hookimpls from default_permissions/__init__.py,
register datasette.default_permissions.tokens as its own DEFAULT_PLUGINS
entry. Cleaner and avoids confusing import-for-side-effect patterns.

* Replace restrict_x params with TokenRestrictions dataclass

Consolidates the three separate restrict_all, restrict_database, and
restrict_resource parameters into a single TokenRestrictions dataclass.
Cleaner API surface for both Datasette.create_token() and
TokenHandler.create_token().

Also clarifies docs re: default handler selection via pluggy ordering.

* Add builder methods to TokenRestrictions

Adds allow_all(), allow_database(), and allow_resource() methods that
return self for chaining. Callers no longer need to manipulate nested
dicts directly:

    restrictions = (TokenRestrictions()
        .allow_all("view-instance")
        .allow_database("mydb", "create-table")
        .allow_resource("mydb", "mytable", "insert-row"))

* docs: add 1.0a25 upgrade guide section for create_token() signature change

Ref: https://github.com/simonw/datasette/issues/2649#issuecomment-3962639393

* docs: note that create_token() is now async in upgrade guide

* docs: update internals, plugin_hooks, authentication for new token API

- internals.rst: new async create_token() signature with restrictions
  and handler params, add TokenRestrictions reference docs
- plugin_hooks.rst: show full create_token signature in TokenHandler
  example, note list returns and error cases
- authentication.rst: cross-reference TokenRestrictions from the
  restrictions section

* style: apply black formatting to token handler files

* docs: fix RST heading underline length in internals.rst

* tests: add restrictions round-trip and expiration tests for token handler

Covers allow_database/allow_resource builders, _r payload encoding,
and token_expires in verified actors. Coverage 76% -> 90%.

* tests: add test for signed tokens disabled

* fix: add TokenRestrictions TYPE_CHECKING import to fix ruff F821

* docs: regenerate plugins.rst with cog

* docs: reformat code blocks in plugin_hooks.rst with blacken-docs

* docs: add await .verify_token() to internals.rst

* tests: rewrite register_token_handler test to use real plugin handler

Adds a HardcodedTokenHandler to the test plugins dir that creates
tokens like dstok_hardcoded_token_1. The test now exercises creating
tokens via the default handler (which is the plugin's hardcoded one),
by explicitly naming the hardcoded handler, and by explicitly naming
the signed handler -- then verifies each token round-trips correctly.

* tests: clarify test_token_handler_via_http tests the default signed handler

* fix: use handler="signed" explicitly where signed tokens are expected

The HardcodedTokenHandler in my_plugin.py gets globally registered,
so create_token() without a handler name picks it up as the default.
Fix the create-token view, CLI, and tests to explicitly request the
signed handler where they depend on signed token behavior.

* fix: use handler="signed" in test_create_table_permissions

https://claude.ai/code/session_013cQFiDQjYRrRBH2biFfKuS
2026-02-25 16:32:45 -08:00
Simon Willison
5c3137d148 Black formatting 2026-02-17 13:30:24 -08:00
Claude
170f9de774 Add pks parameter to render_cell() plugin hook
The render_cell() hook now receives a pks parameter containing the list
of primary key column names for the table being rendered. This avoids
plugins needing to make redundant async calls to look up primary keys.

For tables without an explicit primary key, pks is ["rowid"]. For custom
SQL queries and views, pks is an empty list [].

https://claude.ai/code/session_01HFYfevAziq4fSYTNRD9ZCh
2026-02-17 11:13:30 -08:00
Simon Willison
232a404743 Switch searchable_fts test table to FTS5, closes #2613 2025-12-12 22:18:35 -08:00
Simon Willison
66f2dbb64a Fix assert_permissions_checked to handle PermissionCheck dataclass
Updated the assert_permissions_checked() helper function to work with the
new PermissionCheck dataclass instead of dictionaries. The function now:
- Uses dataclass attributes (pc.action) instead of dict subscripting
- Converts parent/child to old resource format for comparison
- Updates error message formatting to show dataclass fields

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
5feb5fcf5d Remove permission_allowed hook entirely, refs #2528
The permission_allowed hook has been fully replaced by permission_resources_sql.
This commit removes:
- hookspec definition from hookspecs.py
- 4 implementations from default_permissions.py
- implementations from test plugins (my_plugin.py, my_plugin_2.py)
- hook monitoring infrastructure from conftest.py
- references from fixtures.py
- Also fixes test_get_permission to use ds.get_action() instead of ds.get_permission()
- Removes 5th column (source_plugin) from PermissionSQL queries

This completes the migration to the SQL-based permission system.
2025-10-25 15:38:07 -07:00
Simon Willison
fe2084df66 Update test infrastructure to use register_actions hook
- Consolidated register_permissions and register_actions hooks in my_plugin.py
- Added permission_resources_sql hook to provide SQL-based permission rules
- Updated conftest.py to reference datasette.actions instead of datasette.permissions
- Updated fixtures.py to include permission_resources_sql hook and remove register_permissions
- Added backwards compatibility support for old datasette-register-permissions config
- Converted test actions (this_is_allowed, this_is_denied, etc.) to use permission_resources_sql

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
d73b6f169f Add register_actions hook to test plugin and improve test 2025-10-24 10:32:18 -07:00
Simon Willison
4dff846271 simple_primary_key now uses integer id, helps close #2458 2025-02-01 21:44:53 -08:00
Simon Willison
39dfc7d7d7
Removed units functionality and Pint dependency
Closes #2400, unblocks #2320
2024-08-20 19:03:33 -07:00
Simon Willison
b8711988b9 row_actions() plugin hook, closes #2299 2024-03-12 16:16:05 -07:00
Simon Willison
909c85cd2b view_actions plugin hook, closes #2297 2024-03-12 14:25:28 -07:00
Simon Willison
daf5ca02ca homepage_actions() plugin hook, closes #2298 2024-03-12 13:46:06 -07:00
Simon Willison
6ec0081f5d
query_actions plugin hook
* New query_actions plugin hook, closes #2283
2024-02-27 21:55:16 -08:00
Alex Garcia
35deaabcb1
Move non-metadata configuration from metadata.yaml to datasette.yaml
* Allow and permission blocks moved to datasette.yaml
* Documentation updates, initial framework for configuration reference
2023-10-12 09:16:37 -07:00
Simon Willison
b7cf0200e2 Swap order of config and metadata options, refs #2194 2023-09-21 13:22:40 -07:00
Alex Garcia
b2ec8717c3
Plugin configuration now lives in datasette.yaml/json
* Checkpoint, moving top-level plugin config to datasette.json
* Support database-level and table-level plugin configuration in datasette.yaml

Refs #2093
2023-09-13 14:06:25 -07:00
Simon Willison
8bf06a76b5
register_permissions() plugin hook (#1940)
* Docs for permissions: in metadata, refs #1636
* Refactor default_permissions.py to help with implementation of #1636
* register_permissions() plugin hook, closes #1939 - also refs #1938
* Tests for register_permissions() hook, refs #1939
* Documentation for datasette.permissions, refs #1939
* permission_allowed() falls back on Permission.default, refs #1939
* Raise StartupError on duplicate permissions
* Allow dupe permisisons if exact matches
2022-12-12 18:05:54 -08:00
Simon Willison
2355067ef5 Tests now close SQLite database connections and files explicitly, refs #1843
Also added a db.close() method to the Database class.
2022-11-03 13:36:11 -07:00
Simon Willison
c556fad65d Try to address too many files error again, refs #1843 2022-10-25 21:25:47 -07:00
Simon Willison
c36a74ece1 Try shutting down executor in tests to free up thread local SQLite connections, refs #1843 2022-10-25 21:04:39 -07:00
Simon Willison
6887c12ea3 Workaround for 'too many open files' error, refs #1843 2022-10-23 15:17:33 -07:00
Simon Willison
ddc999ad12 Async support for prepare_jinja2_environment, closes #1809 2022-09-16 20:38:24 -07:00
Simon Willison
d0737e4de5 truncate_cells_html now affects URLs too, refs #1805 2022-09-06 16:50:43 -07:00
Simon Willison
c09c53f345 New handle_exception plugin hook, refs #1770
Also refs:
- https://github.com/simonw/datasette-sentry/issues/1
- https://github.com/simonw/datasette-show-errors/issues/2
2022-07-17 16:24:39 -07:00
Simon Willison
4e47a2d894 Fixed bug where tables with a column called n caused 500 errors
Closes #1228
2022-03-18 18:37:54 -07:00
Simon Willison
d4f60c2388
Remove hashed URL mode
Also simplified how view class routing works.

Refs #1661
2022-03-18 17:12:03 -07:00
Simon Willison
1baa030eca
Switch to dash encoding for table/database/row-pk in paths
* Dash encoding functions, tests and docs, refs #1439
* dash encoding is now like percent encoding but with dashes
* Use dash-encoding for row PKs and ?_next=, refs #1439
* Use dash encoding for table names, refs #1439
* Use dash encoding for database names, too, refs #1439

See also https://simonwillison.net/2022/Mar/5/dash-encoding/
2022-03-07 07:38:29 -08:00
Simon Willison
3a0cae4d7f Fix bug introduced by refactor in c35b84a2aa 2021-12-17 18:19:09 -08:00
Simon Willison
c35b84a2aa Remove undocumented sqlite_functions mechanism, closes #1567 2021-12-17 17:54:39 -08:00
Simon Willison
69244a617b Rename city_id to _city_id in fixtures, refs #1525 2021-11-29 22:20:42 -08:00
Simon Willison
c306b696de Correct facet links for columns with a leading underscore, closes #1506 2021-11-13 20:44:54 -08:00
Simon Willison
ca4f83dc7b Rename config= to settings=, refs #1432 2021-08-12 18:10:36 -07:00
Simon Willison
e837095ef3
Column metadata, closes #942 2021-08-12 16:53:23 -07:00
Simon Willison
3bb6409a6c render_cell() can now return an awaitable, refs 2021-08-08 16:05:00 -07:00
Simon Willison
66e143c76e New hide_sql canned query option, refs #1422 2021-08-06 22:17:36 -07:00
Simon Willison
eccfeb0871 register_routes() plugin hook datasette argument, closes #1404 2021-07-26 16:16:46 -07:00
Simon Willison
2e8d924cdc Refactored generated_columns test, no longer in fixtures.db - refs #1391 2021-07-10 12:03:19 -07:00
Simon Willison
02b19c7a9a Removed rogue pdb=True, refs #1377 2021-06-23 15:50:48 -07:00
Simon Willison
b1fd24ac9f skip_csrf(datasette, scope) plugin hook, refs #1377 2021-06-23 15:40:09 -07:00
Simon Willison
ff29dd55fa ?_trace=1 now depends on trace_debug setting, closes #1359 2021-06-05 13:18:37 -07:00
Simon Willison
807de378d0 /-/databases and homepage maintain connection order, closes #1216 2021-06-01 20:10:15 -07:00
Simon Willison
7e983fede6 ?_facet_size=max, ... now links to that, closes #1337
Refs #1332
2021-05-27 09:00:58 -07:00
Simon Willison
6ad544df5e Fixed master -> main in a bunch of places, mainly docs 2021-03-23 09:19:41 -07:00
Konstantin Baikov
8e18c79431
Use context manager instead of plain open (#1211)
Context manager with open closes the files after usage.

When the object is already a pathlib.Path i used read_text
write_text functions

In some cases pathlib.Path.open were used in context manager,
it is basically the same as builtin open.

Thanks, Konstantin Baikov!
2021-03-11 08:15:49 -08:00
Simon Willison
6f41c8a2be
--crossdb option for joining across databases (#1232)
* Test for cross-database join, refs #283
* Warn if --crossdb used with more than 10 DBs, refs #283
* latest.datasette.io demo of --crossdb joins, refs #283
* Show attached databases on /_memory page, refs #283
* Documentation for cross-database queries, refs #283
2021-02-18 14:09:12 -08:00