Commit graph

2,855 commits

Author SHA1 Message Date
Simon Willison
dc3f9fe9e4 Python 3.10, not 3.8 2025-11-03 14:42:59 -08:00
Simon Willison
5d4dfcec6b Fix for link from changelog not working
Annoyingly we now get a warning in the docs build about a duplicate label,
but it seems harmless enough.
2025-11-03 14:38:57 -08:00
Simon Willison
b3b8c5831b Fixed some broken reference links on upgrade guide 2025-11-03 14:34:29 -08:00
Simon Willison
b212895b97 Updated release notes for 1.0a20
Refs #2550
2025-11-03 14:27:41 -08:00
Simon Willison
18fd373a8f
New PermissionSQL.restriction_sql mechanism for actor restrictions
Implement INTERSECT-based actor restrictions to prevent permission bypass

Actor restrictions are now implemented as SQL filters using INTERSECT rather
than as deny/allow permission rules. This ensures restrictions act as hard
limits that cannot be overridden by other permission plugins or config blocks.

Previously, actor restrictions (_r in actor dict) were implemented by 
generating permission rules with deny/allow logic. This approach had a 
critical flaw: database-level config allow blocks could bypass table-level 
restrictions, granting access to tables not in the actor's allowlist.

The new approach separates concerns:

- Permission rules determine what's allowed based on config and plugins
- Restriction filters limit the result set to only allowlisted resources
- Restrictions use INTERSECT to ensure all restriction criteria are met
- Database-level restrictions (parent, NULL) properly match all child tables

Implementation details:

- Added restriction_sql field to PermissionSQL dataclass
- Made PermissionSQL.sql optional to support restriction-only plugins
- Updated actor_restrictions_sql() to return restriction filters instead of rules
- Modified SQL builders to apply restrictions via INTERSECT and EXISTS clauses

Closes #2572
2025-11-03 14:17:51 -08:00
Simon Willison
c76c3e6e6f facet_suggest_time_limit_ms 200ms in tests, closes #2574 2025-11-03 11:52:12 -08:00
Simon Willison
fa978ec100 More upgrade tips, written by Claude Code
Refs #2549

From the datasette-atom upgrade, https://gistpreview.github.io/?d5047e04bbd9c20c59437916e21754ae
2025-11-02 12:02:45 -08:00
Simon Willison
2459285052 Additional upgrade notes by Codex CLI
Refs https://github.com/simonw/datasette/issues/2549#issuecomment-3477398336

Refs #2564
2025-11-01 20:32:42 -07:00
Simon Willison
506ce5b0ac Remove docs for obsolete register_permissions() hook, refs #2528
Also removed docs for datasette.get_permission() method which no longer exists.
2025-11-01 20:23:37 -07:00
Simon Willison
063bf7a96f Action() is kw_only, abbr= is optional, closes #2571 2025-11-01 20:20:17 -07:00
Simon Willison
7e09e1bf1b Removed obsolete actor ID v.s. actor dict code, refs #2570 2025-11-01 19:30:56 -07:00
Simon Willison
e37aa37edc Further refactor to collapse some utility functions
Refs #2570
2025-11-01 19:28:31 -07:00
Simon Willison
b8cee8768e Completed upgrade guide, closes #2564 2025-11-01 18:57:56 -07:00
Simon Willison
5c16c6687d Split permissions_resources_sql() into 5 for readability
Also remove an obsolete test that caused trouble with the new split plugin hook.

Closes #2570
2025-11-01 18:38:47 -07:00
Simon Willison
a528555e84
Additional actor restriction should not grant access to additional actions (#2569)
Closes #2568
2025-11-01 18:38:29 -07:00
Simon Willison
2b962beaeb Fix permissions_execute_sql warnings in documentation 2025-11-01 11:52:23 -07:00
Simon Willison
5705ce0d95
Move takes_child/takes_parent information from Action to Resource (#2567)
Simplified Action by moving takes_child/takes_parent logic to Resource

- Removed InstanceResource - global actions are now simply those with resource_class=None
- Resource.parent_class - Replaced parent_name: str with parent_class: type[Resource] | None for direct class references
- Simplified Action dataclass - No more redundant fields, everything is derived from the Resource class structure
- Validation - The __init_subclass__ method now checks parent_class.parent_class to enforce the 2-level hierarchy

Closes #2563
2025-11-01 11:35:08 -07:00
Simon Willison
1f8995e776 upgrade-1.0a20.md, refs #2564
And another Markdown conversion, refs #2565
2025-10-31 19:13:41 -07:00
Simon Willison
47e4060469 Enable MyST Markdown docs, port events.rst, refs #2565 2025-10-31 16:38:04 -07:00
Simon Willison
48982a0ff5 Mark 1.0a20 unreleased
Refs #2550
2025-10-31 16:12:54 -07:00
Simon Willison
223dcc7c0e Remove unused link 2025-10-31 16:11:53 -07:00
Simon Willison
3184bfae54 Release notes for 1.0a20, refs #2550 2025-10-31 15:37:30 -07:00
Simon Willison
e5f392ae7a datasette.allowed_resources_sql() returns namedtuple 2025-10-31 15:07:37 -07:00
Simon Willison
400fa08e4c
Add keyset pagination to allowed_resources() (#2562)
* Add keyset pagination to allowed_resources()

This replaces the unbounded list return with PaginatedResources,
which supports efficient keyset pagination for handling thousands
of resources.

Closes #2560

Changes:
- allowed_resources() now returns PaginatedResources instead of list
- Added limit (1-1000, default 100) and next (keyset token) parameters
- Added include_reasons parameter (replaces allowed_resources_with_reasons)
- Removed allowed_resources_with_reasons() method entirely
- PaginatedResources.all() async generator for automatic pagination
- Uses tilde-encoding for tokens (matching table pagination)
- Updated all callers to use .resources accessor
- Updated documentation with new API and examples

The PaginatedResources object has:
- resources: List of Resource objects for current page
- next: Token for next page (None if no more results)
- all(): Async generator that yields all resources across pages

Example usage:
    page = await ds.allowed_resources("view-table", actor, limit=100)
    for table in page.resources:
        print(table.child)

    # Iterate all pages automatically
    async for table in page.all():
        print(table.child)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 14:50:46 -07:00
Simon Willison
b7ef968c6f Fixed some rST labels I broke 2025-10-31 09:15:39 -07:00
Simon Willison
ba654b5576 Forbid same DB passed twice or via config_dir, closes #2561 2025-10-30 21:40:09 -07:00
Simon Willison
e4be95b16c
Update permissions documentation for new action system (#2551) 2025-10-30 17:59:54 -07:00
Simon Willison
87aa798148 Permission tabs include allow debug page
Closes #2559
2025-10-30 17:54:07 -07:00
Simon Willison
6a71bde37f
Permissions SQL API improvements (#2558)
* Neater design for PermissionSQL class, refs #2556
  - source is now automatically set to the source plugin
  - params is optional
* PermissionSQL.allow() and PermissionSQL.deny() shortcuts

Closes #2556

* Filter out temp database from attached_databases()

Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
2025-10-30 15:48:46 -07:00
Simon Willison
5247856bd4 Filter out temp database from attached_databases()
Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
2025-10-30 15:48:10 -07:00
Simon Willison
ce4b0794b2
Ported setup.py to pyproject.toml (#2555)
* Ported setup.py to pyproject.toml, refs #2553

* Make fixtures tests less flaky

The in-memory fixtures table was being shared between different
instances of the test client, leading to occasional errors when
running the full test suite.
2025-10-30 10:41:41 -07:00
Simon Willison
53e6a72a95 Move black to YAML, not pytest 2025-10-30 10:40:46 -07:00
Simon Willison
1289eb0589 Fix SQLite locking issue in execute_write_script
The execute_write_script() method was causing SQLite database locking
errors when multiple executescript() calls ran in quick succession.

Root cause: SQLite's executescript() method has special behavior - it
implicitly commits any pending transaction and operates in autocommit
mode. However, execute_write_script() was passing these calls through
execute_write_fn() with the default transaction=True, which wrapped
the executescript() call in a transaction context (with conn:).

This created a conflict where sequential executescript() calls would
cause the second call to fail with "OperationalError: database table
is locked: sqlite_master" because the sqlite_master table was still
locked from the first operation's implicit commit.

Fix: Pass transaction=False to execute_write_fn() since executescript()
manages its own transactions and should not be wrapped in an additional
transaction context.

This was causing test_hook_extra_body_script to fail because the
internal database initialization (which calls executescript twice in
succession) would fail, preventing the application from rendering
pages correctly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 10:30:09 -07:00
Simon Willison
5da3c9f4bd Better display of recent permissions checks, refs #2543 2025-10-30 10:28:04 -07:00
Simon Willison
b018eb3171 Simplified the code for the permission debug pages
Decided not to use as much JavaScript

Used Codex CLI for this. Refs #2543
2025-10-30 10:28:04 -07:00
Simon Willison
73014abe8b Improved permissions UI WIP 2025-10-30 10:28:04 -07:00
Simon Willison
b3721eaf50 Add /-/actions endpoint to list registered actions
This adds a new endpoint at /-/actions that lists all registered actions
in the permission system. The endpoint supports both JSON and HTML output.

Changes:
- Added _actions() method to Datasette class to return action list
- Added route for /-/actions with JsonDataView
- Created actions.html template for nice HTML display
- Added template parameter to JsonDataView for custom templates
- Moved respond_json_or_html from BaseView to JsonDataView
- Added test for the new endpoint

The endpoint requires view-instance permission and provides details about
each action including name, abbreviation, description, resource class,
and parent/child requirements.

Closes #2547

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:14:58 -07:00
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
2c8e92acf2 Require permissions-debug permission for /-/check endpoint
The /-/check endpoint now requires the permissions-debug permission
to access. This prevents unauthorized users from probing the permission
system. Administrators can grant this permission to specific users or
anonymous users if they want to allow open access.

Added test to verify anonymous and regular users are denied access,
while root user (who has all permissions) can access the endpoint.

Closes #2546
2025-10-26 11:16:07 -07:00
Simon Willison
e7ed948238 Use ruff to upgrade Optional[x] to x | None
Refs #2545
2025-10-26 10:50:29 -07:00
Simon Willison
06b442c894 Applied Black, refs #2544 2025-10-26 10:05:12 -07:00
Simon Willison
6de83bf3a9 Make deploy-latest.yml workflow dispatch-only
It is currently broken, will revert once I fix it.
2025-10-26 09:51:09 -07:00
Simon Willison
4fe1765dc3 Add test for RST heading underline lengths, closes #2544
Added test_rst_heading_underlines_match_title_length() to verify that RST
heading underlines match their title lengths. The test properly handles:
- Overline+underline style headings (skips validation for those)
- Empty lines before underlines (ignores them)
- Minimum 5-character underline length (avoids false positives)

Running this test identified 14 heading underline mismatches which have
been fixed across 5 documentation files:
- docs/authentication.rst (3 headings)
- docs/plugin_hooks.rst (4 headings)
- docs/internals.rst (5 headings)
- docs/deploying.rst (1 heading)
- docs/changelog.rst (1 heading)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
653c94209c Remove broken reference to datasette_ensure_permissions in changelog 2025-10-26 09:49:49 -07:00
Simon Willison
95286fbb60 Refactor check_visibility() to use Resource objects, refs #2537
Updated check_visibility() method signature to accept Resource objects
(DatabaseResource, TableResource, QueryResource) instead of plain strings
and tuples.

Changes:
- Updated check_visibility() signature to only accept Resource objects
- Added validation with helpful error message for incorrect types
- Updated all check_visibility() calls throughout the codebase:
  - datasette/views/database.py: Use DatabaseResource and QueryResource
  - datasette/views/special.py: Use DatabaseResource and TableResource
  - datasette/views/row.py: Use TableResource
  - datasette/views/table.py: Use TableResource
  - datasette/app.py: Use TableResource in expand_foreign_keys
- Updated tests to use Resource objects
- Updated documentation in docs/internals.rst:
  - Removed outdated permissions parameter
  - Updated examples to use Resource objects

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 09:49:49 -07:00
Simon Willison
653475edde Fix permissions_debug.html to use takes_parent/takes_child, refs #2530
The JavaScript was still referencing the old field names takes_database
and takes_resource instead of the new takes_parent and takes_child. This
caused the resource input fields to not show/hide properly when selecting
different permission actions.
2025-10-26 09:49:49 -07:00
dependabot[bot]
c652e92049 Bump the python-packages group across 1 directory with 3 updates
Bumps the python-packages group with 3 updates in the / directory: [furo](https://github.com/pradyunsg/furo), [blacken-docs](https://github.com/asottile/blacken-docs) and [black](https://github.com/psf/black).


Updates `furo` from 2024.8.6 to 2025.7.19
- [Release notes](https://github.com/pradyunsg/furo/releases)
- [Changelog](https://github.com/pradyunsg/furo/blob/main/docs/changelog.md)
- [Commits](https://github.com/pradyunsg/furo/compare/2024.08.06...2025.07.19)

Updates `blacken-docs` from 1.19.1 to 1.20.0
- [Changelog](https://github.com/adamchainz/blacken-docs/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/asottile/blacken-docs/compare/1.19.1...1.20.0)

Updates `black` from 25.1.0 to 25.9.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.1.0...25.9.0)

---
updated-dependencies:
- dependency-name: furo
  dependency-version: 2025.7.19
  dependency-type: direct:development
  update-type: version-update:semver-major
  dependency-group: python-packages
- dependency-name: blacken-docs
  dependency-version: 1.20.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-packages
- dependency-name: black
  dependency-version: 25.9.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-10-25 21:32:52 -07:00
Simon Willison
d769e97ab8 Show multiple permission reasons as JSON arrays, refs #2531
- Modified /-/allowed to show all reasons that grant access to a resource
- Changed from MAX(reason) to json_group_array() in SQL to collect all reasons
- Reasons now displayed as JSON arrays in both HTML and JSON responses
- Only show Reason column to users with permissions-debug permission
- Removed obsolete "Source Plugin" column from /-/rules interface
- Updated allowed_resources_with_reasons() to parse and return reason lists
- Fixed alert() on /-/allowed by replacing with disabled input state
2025-10-25 21:24:05 -07:00
Simon Willison
ee4fcff5c0 On /-/allowed show reason column if vsible to user 2025-10-25 21:08:59 -07:00
Simon Willison
ee62bf9bdc Fix minor irritation with /-/allowed UI 2025-10-25 18:02:26 -07:00