Commit graph

157 commits

Author SHA1 Message Date
Simon Willison
feaba9b18b
Optionally limit ColumnType subclasses to specific SQLite types (#2673)
* ColumnTypes now have optional SQLite column types

Refs #2672
2026-03-18 11:37:09 -07:00
Simon Willison
63d73a806f
Move table configuration docs from metadata.rst to configuration.rst (#2668)
https://claude.ai/code/session_01UqboRB5Wt52BKPhxexUBEn
2026-03-17 08:47:04 -07:00
Claude
c73a1c907a
Use 'subclass' instead of 'class' in ColumnType docs
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:22:53 +00:00
Claude
dd9b83301c
Refactor ColumnType: register classes, return instances with config
- register_column_types() now returns classes instead of instances
- ColumnType.__init__ takes optional config=, baking it into the instance
- get_column_type() returns a ColumnType instance (or None) instead of a
  (name, config) tuple
- get_column_types() returns {col: ColumnType instance} instead of tuples
- Remove get_column_type_class() - no longer needed
- render_cell/validate/transform_value methods no longer take config arg;
  use self.config instead
- render_cell hook takes column_type (ColumnType or None) instead of
  column_type + column_type_config

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:18:14 +00:00
Claude
9fe10cd1aa
Apply black and blacken-docs formatting
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:50:58 +00:00
Claude
77bbfb5f7e
Document column type internal methods in internals.rst
Add documentation for get_column_type, get_column_types,
set_column_type, remove_column_type, and get_column_type_class
methods on the Datasette instance.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:08:58 +00:00
Simon Willison
c96dc5ce26
register_token_handler() plugin hook for custom API token backends (#2650)
Closes #2649

* Add register_token_handler plugin hook for pluggable token backends

Adds a new register_token_handler hook that allows plugins to provide
custom token creation and verification backends. This enables plugins
like datasette-oauth to issue tokens without depending on specific
backend plugins like datasette-auth-tokens.

Key changes:
- New datasette/tokens.py with TokenHandler base class and SignedTokenHandler
  (the default signed-token implementation moved here)
- New register_token_handler hookspec in hookspecs.py
- Datasette.create_token() is now async and delegates to token handlers
- New Datasette.verify_token() method tries all handlers in sequence
- handler= parameter on create_token() to select a specific backend
- TokenHandler exported from datasette package for plugin use
- Fixed actor_from_request loop to await all coroutines (avoids warnings)

* Add documentation and hook test for register_token_handler

Fixes CI failures: the new hook needs a section in docs/plugin_hooks.rst
(checked by test_plugin_hooks_are_documented) and a test_hook_* function
in test_plugins.py (checked by test_plugin_hooks_have_tests).

* Register tokens module as separate default plugin

Instead of re-exporting hookimpls from default_permissions/__init__.py,
register datasette.default_permissions.tokens as its own DEFAULT_PLUGINS
entry. Cleaner and avoids confusing import-for-side-effect patterns.

* Replace restrict_x params with TokenRestrictions dataclass

Consolidates the three separate restrict_all, restrict_database, and
restrict_resource parameters into a single TokenRestrictions dataclass.
Cleaner API surface for both Datasette.create_token() and
TokenHandler.create_token().

Also clarifies docs re: default handler selection via pluggy ordering.

* Add builder methods to TokenRestrictions

Adds allow_all(), allow_database(), and allow_resource() methods that
return self for chaining. Callers no longer need to manipulate nested
dicts directly:

    restrictions = (TokenRestrictions()
        .allow_all("view-instance")
        .allow_database("mydb", "create-table")
        .allow_resource("mydb", "mytable", "insert-row"))

* docs: add 1.0a25 upgrade guide section for create_token() signature change

Ref: https://github.com/simonw/datasette/issues/2649#issuecomment-3962639393

* docs: note that create_token() is now async in upgrade guide

* docs: update internals, plugin_hooks, authentication for new token API

- internals.rst: new async create_token() signature with restrictions
  and handler params, add TokenRestrictions reference docs
- plugin_hooks.rst: show full create_token signature in TokenHandler
  example, note list returns and error cases
- authentication.rst: cross-reference TokenRestrictions from the
  restrictions section

* style: apply black formatting to token handler files

* docs: fix RST heading underline length in internals.rst

* tests: add restrictions round-trip and expiration tests for token handler

Covers allow_database/allow_resource builders, _r payload encoding,
and token_expires in verified actors. Coverage 76% -> 90%.

* tests: add test for signed tokens disabled

* fix: add TokenRestrictions TYPE_CHECKING import to fix ruff F821

* docs: regenerate plugins.rst with cog

* docs: reformat code blocks in plugin_hooks.rst with blacken-docs

* docs: add await .verify_token() to internals.rst

* tests: rewrite register_token_handler test to use real plugin handler

Adds a HardcodedTokenHandler to the test plugins dir that creates
tokens like dstok_hardcoded_token_1. The test now exercises creating
tokens via the default handler (which is the plugin's hardcoded one),
by explicitly naming the hardcoded handler, and by explicitly naming
the signed handler -- then verifies each token round-trips correctly.

* tests: clarify test_token_handler_via_http tests the default signed handler

* fix: use handler="signed" explicitly where signed tokens are expected

The HardcodedTokenHandler in my_plugin.py gets globally registered,
so create_token() without a handler name picks it up as the default.
Fix the create-token view, CLI, and tests to explicitly request the
signed handler where they depend on signed token behavior.

* fix: use handler="signed" in test_create_table_permissions

https://claude.ai/code/session_013cQFiDQjYRrRBH2biFfKuS
2026-02-25 16:32:45 -08:00
Simon Willison
40a37307de
Add request.form() for multipart form data and file uploads
* Add request.form() for multipart form data and file uploads

New Request.form() method that handles both application/x-www-form-urlencoded
and multipart/form-data content types with streaming parsing.

Features:
- Streaming multipart parser that doesn't buffer entire body in memory
- Files spill to disk above 1MB threshold via SpooledTemporaryFile
- files=False (default) discards file content, files=True stores them
- Security limits: max_request_size, max_file_size, max_fields, max_files
- FormData container with dict-like access and getlist() for multiple values
- UploadedFile class with async read(), seek(), filename, content_type, size
- Support for RFC 5987 filename* encoding for international filenames

Uses multipart-form-data-conformance test suite for validation.

* Update views to use request.form() and document new API

- Migrate PermissionsDebugView, MessagesDebugView, and CreateTokenView
  from post_vars() to form()
- Add documentation for request.form(), FormData, and UploadedFile classes

Centralize multipart defaults and expose stricter limits via Request.form().

Enforce header, part, file, and disk space limits even when files are discarded; detect truncated bodies and client disconnects; and move blocking work off the event loop.

Add FormData close/aclose context managers, update internals docs, and expand multipart tests (including len semantics and stricter conformance expectations).
2026-01-28 18:41:03 -08:00
Simon Willison
93b455239a Release notes for 1.0a22, closes #2596 2025-11-13 10:40:24 -08:00
Simon Willison
4b4add4d31 datasette.pm property, closes #2595 2025-11-13 10:31:03 -08:00
Simon Willison
5125bef573 datasette.in_client() method, closes #2594 2025-11-13 10:00:04 -08:00
Simon Willison
1df4028d78 add_memory_database(memory_name, name=None, route=None) 2025-11-05 15:18:17 -08:00
Simon Willison
d814e81b32
datasette.client.get(..., skip_permission_checks=True)
Closes #2580
2025-11-05 13:38:01 -08:00
Simon Willison
506ce5b0ac Remove docs for obsolete register_permissions() hook, refs #2528
Also removed docs for datasette.get_permission() method which no longer exists.
2025-11-01 20:23:37 -07:00
Simon Willison
e5f392ae7a datasette.allowed_resources_sql() returns namedtuple 2025-10-31 15:07:37 -07:00
Simon Willison
400fa08e4c
Add keyset pagination to allowed_resources() (#2562)
* Add keyset pagination to allowed_resources()

This replaces the unbounded list return with PaginatedResources,
which supports efficient keyset pagination for handling thousands
of resources.

Closes #2560

Changes:
- allowed_resources() now returns PaginatedResources instead of list
- Added limit (1-1000, default 100) and next (keyset token) parameters
- Added include_reasons parameter (replaces allowed_resources_with_reasons)
- Removed allowed_resources_with_reasons() method entirely
- PaginatedResources.all() async generator for automatic pagination
- Uses tilde-encoding for tokens (matching table pagination)
- Updated all callers to use .resources accessor
- Updated documentation with new API and examples

The PaginatedResources object has:
- resources: List of Resource objects for current page
- next: Token for next page (None if no more results)
- all(): Async generator that yields all resources across pages

Example usage:
    page = await ds.allowed_resources("view-table", actor, limit=100)
    for table in page.resources:
        print(table.child)

    # Iterate all pages automatically
    async for table in page.all():
        print(table.child)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 14:50:46 -07:00
Simon Willison
e4be95b16c
Update permissions documentation for new action system (#2551) 2025-10-30 17:59:54 -07:00
Simon Willison
4fe1765dc3 Add test for RST heading underline lengths, closes #2544
Added test_rst_heading_underlines_match_title_length() to verify that RST
heading underlines match their title lengths. The test properly handles:
- Overline+underline style headings (skips validation for those)
- Empty lines before underlines (ignores them)
- Minimum 5-character underline length (avoids false positives)

Running this test identified 14 heading underline mismatches which have
been fixed across 5 documentation files:
- docs/authentication.rst (3 headings)
- docs/plugin_hooks.rst (4 headings)
- docs/internals.rst (5 headings)
- docs/deploying.rst (1 heading)
- docs/changelog.rst (1 heading)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
95286fbb60 Refactor check_visibility() to use Resource objects, refs #2537
Updated check_visibility() method signature to accept Resource objects
(DatabaseResource, TableResource, QueryResource) instead of plain strings
and tuples.

Changes:
- Updated check_visibility() signature to only accept Resource objects
- Added validation with helpful error message for incorrect types
- Updated all check_visibility() calls throughout the codebase:
  - datasette/views/database.py: Use DatabaseResource and QueryResource
  - datasette/views/special.py: Use DatabaseResource and TableResource
  - datasette/views/row.py: Use TableResource
  - datasette/views/table.py: Use TableResource
  - datasette/app.py: Use TableResource in expand_foreign_keys
- Updated tests to use Resource objects
- Updated documentation in docs/internals.rst:
  - Removed outdated permissions parameter
  - Updated examples to use Resource objects

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
e7c7e21277 Ran blacken-docs 2025-10-25 15:38:07 -07:00
Simon Willison
fabcfd68ad Add datasette.ensure_permission() method, refs #2525, refs #2528
Implements a new ensure_permission() method that is a convenience wrapper
around allowed() that raises Forbidden instead of returning False.

Changes:
- Added ensure_permission() method to datasette/app.py
- Updated all views to use ensure_permission() instead of the pattern:
  if not await self.ds.allowed(...): raise Forbidden(...)
- Updated docs/internals.rst to document the new method
- Removed old ensure_permissions() documentation (that method was already removed)

The new method simplifies permission enforcement in views and makes the
code more concise and consistent.
2025-10-25 15:38:07 -07:00
Simon Willison
a21a1b6c14 Ran blacken-docs 2025-10-24 10:32:18 -07:00
Simon Willison
b8d26754df Document datasette.allowed(), PermissionSQL class, and SQL parameters
- Added documentation for datasette.allowed() method with keyword-only arguments
- Added comprehensive PermissionSQL class documentation with examples
- Documented the three SQL parameters available: :actor, :actor_id, :action
- Included examples of using json_extract() to access actor fields
- Explained permission resolution rules (specificity, deny over allow, implicit deny)
- Fixed RST formatting warnings (escaped asterisk, fixed underline length)
2025-10-24 10:32:18 -07:00
Simon Willison
7a602140df catalog_views table, closes #2495
Refs https://github.com/datasette/datasette-queries/issues/1#issuecomment-3074491003
2025-07-15 10:22:56 -07:00
Simon Willison
7945f4fbf2 Improved docs for db.get_all_foreign_keys() 2025-03-12 15:42:11 -07:00
Simon Willison
e59fd01757 Fix for incorrect REFERENCES in internal DB
Refs #2466
2025-02-12 19:40:43 -08:00
Simon Willison
92c4d41ca6 results.dicts() method, closes #2414 2024-09-01 17:20:41 -07:00
Simon Willison
2e82eb108c Group docs on get_/set_ metadata methods, refs #2381 2024-08-05 14:16:34 -07:00
Simon Willison
2b0a61ee19 Rename metadata tables and add schema to docs, refs #2382 2024-08-05 13:53:55 -07:00
Alex Garcia
169ee5d710
Initial upgrade guide for v0.XX to v1 2024-08-05 10:35:38 -07:00
Simon Willison
d118d5c5bb named_parameters(sql) sync function, refs #2354
Also refs #2353 and #2352
2024-06-12 16:51:07 -07:00
Simon Willison
8f9509f00c
datasette, not self.ds, in internals documentation 2024-04-22 16:01:37 -07:00
Simon Willison
63714cb2b7 Fixed some typos spotted by Gemini Pro 1.5, closes #2318 2024-04-10 17:05:15 -07:00
Simon Willison
d32176c5b8
Typo fix triggera -> triggers 2024-04-10 16:50:09 -07:00
Simon Willison
c863443ea1 Documentation for derive_named_parameters()
Closes #2284

Refs https://github.com/simonw/datasette-write/issues/7#issuecomment-1967593883
2024-02-27 13:24:47 -08:00
Simon Willison
5e0e440f2c database.execute_write_fn(transaction=True) parameter, closes #2277 2024-02-17 20:28:15 -08:00
Simon Willison
bcc4f6bf1f
track_event() mechanism for analytics and plugins
* Closes #2240
* Documentation for event plugin hooks, refs #2240
* Include example track_event plugin in docs, refs #2240
* Tests for track_event() and register_events() hooks, refs #2240
* Initial documentation for core events, refs #2240
* Internals documentation for datasette.track_event()
2024-01-31 15:21:40 -08:00
Simon Willison
959e020297 Ran blacken-docs 2024-01-30 20:40:18 -08:00
Simon Willison
4284c74bc1
db.execute_isolated_fn() method (#2220)
Closes #2218
2023-12-19 10:51:03 -08:00
Alex Garcia
35deaabcb1
Move non-metadata configuration from metadata.yaml to datasette.yaml
* Allow and permission blocks moved to datasette.yaml
* Documentation updates, initial framework for configuration reference
2023-10-12 09:16:37 -07:00
Simon Willison
b0e5d8afa3
Stop using parallel SQL queries for tables
Refs:
- #2189
2023-09-20 15:10:55 -07:00
Alex Garcia
b2ec8717c3
Plugin configuration now lives in datasette.yaml/json
* Checkpoint, moving top-level plugin config to datasette.json
* Support database-level and table-level plugin configuration in datasette.yaml

Refs #2093
2023-09-13 14:06:25 -07:00
Simon Willison
b645174271
actors_from_ids plugin hook and datasette.actors_from_ids() method (#2181)
* Prototype of actors_from_ids plugin hook, refs #2180
* datasette-remote-actors example plugin, refs #2180
2023-09-07 21:23:59 -07:00
Simon Willison
9cead33fb9
OperationalError: database table is locked fix
See also:
- https://til.simonwillison.net/datasette/remember-to-commit
2023-08-31 10:46:07 -07:00
Simon Willison
4c3ef03311
Another ReST fix 2023-08-30 16:19:59 -07:00
Simon Willison
2caa53a52a
ReST fix 2023-08-30 16:19:24 -07:00
Simon Willison
bb12229794 Rename core_ to catalog_, closes #2163 2023-08-29 10:01:28 -07:00
Simon Willison
50da908213
Cascade for restricted token view-table/view-database/view-instance operations (#2154)
Closes #2102

* Permission is now a dataclass, not a namedtuple - refs https://github.com/simonw/datasette/pull/2154/#discussion_r1308087800
* datasette.get_permission() method
2023-08-29 09:32:34 -07:00
Alex Garcia
92b8bf38c0
Add new --internal internal.db option, deprecate legacy _internal database
Refs:
- #2157 
---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2023-08-28 20:24:23 -07:00
Simon Willison
e054704fb6 Added missing rST label 2022-12-14 21:38:28 -08:00