Commit graph

2,744 commits

Author SHA1 Message Date
Simon Willison
7c6bc0b902 Fix #2509: Settings-based deny rules now override root user privileges
The root user's permission_resources_sql hook was returning early with a
blanket "allow all" rule, preventing settings-based deny rules from being
considered. This caused /-/allowed and /-/rules endpoints to incorrectly
show resources that were denied via settings.

Changed permission_resources_sql to append root permissions to the rules
list instead of returning early, allowing config-based deny rules to be
evaluated. The SQL cascading logic correctly applies: deny rules at the
same depth beat allow rules, so database-level denies override root's
global-level allow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 11:13:19 -07:00
Simon Willison
5138e95d69 Migrate homepage to use bulk allowed_resources() and fix NULL handling in SQL JOINs
- Updated IndexView in datasette/views/index.py to fetch all allowed databases and tables
  in bulk upfront using allowed_resources() instead of calling check_visibility() for each
  database, table, and view individually
- Fixed SQL bug in build_allowed_resources_sql() where USING (parent, child) clauses failed
  for database resources because NULL = NULL evaluates to NULL in SQL, not TRUE
- Changed all INNER JOINs to use explicit ON conditions with NULL-safe comparisons:
  ON b.parent = x.parent AND (b.child = x.child OR (b.child IS NULL AND x.child IS NULL))

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:32:18 -07:00
Simon Willison
8674aaa392 Add parent filter and include_is_private to allowed_resources()
Major improvements to the allowed_resources() API:

1. **parent filter**: Filter results to specific database in SQL, not Python
   - Avoids loading thousands of tables into Python memory
   - Filtering happens efficiently in SQLite

2. **include_is_private flag**: Detect private resources in single SQL query
   - Compares actor permissions vs anonymous permissions in SQL
   - LEFT JOIN between actor_allowed and anon_allowed CTEs
   - Returns is_private column: 1 if anonymous blocked, 0 otherwise
   - No individual check_visibility() calls needed

3. **Resource.private property**: Safe access with clear error messages
   - Raises AttributeError if accessed without include_is_private=True
   - Prevents accidental misuse of the property

4. **Database view optimization**: Use new API to eliminate redundant checks
   - Single bulk query replaces N individual permission checks
   - Private flag computed in SQL, not via check_visibility() calls
   - Views filtered from allowed_dict instead of checking db.view_names()

All permission filtering now happens in SQLite where it belongs, with
minimal data transferred to Python.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:32:18 -07:00
Simon Willison
2620938661 Migrate /database view to use bulk allowed_resources()
Replace one-by-one permission checks with bulk allowed_resources() call:
- DatabaseView and QueryView now fetch all allowed tables once
- Filter views and tables using pre-fetched allowed_table_set
- Update TableResource.resources_sql() to include views from catalog_views

This improves performance by reducing permission checks from O(n) to O(1) per
table/view, where n is the number of tables in the database.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:32:18 -07:00
Simon Willison
23715d6c00 Error on startup if invalid setting types 2025-10-24 10:32:18 -07:00
Simon Willison
58ac5ccd6e Simplify types in datasette/permissions.py 2025-10-24 10:32:18 -07:00
Simon Willison
b311f735f9 Fix schema mismatch in empty result query
When no permission rules exist, the query was returning 2 columns (parent, child)
but the function contract specifies 3 columns (parent, child, reason). This could
cause schema mismatches in consuming code.

Added 'NULL AS reason' to match the documented 3-column schema.

Added regression test that verifies the schema has 3 columns even when no
permission rules are returned. The test fails without the fix (showing only
2 columns) and passes with it.

Thanks to @asg017 for catching this
2025-10-24 10:32:18 -07:00
Simon Willison
79879b834a Address PR #2515 review comments
- Add URL to sqlite-permissions-poc in module docstring
- Replace Optional with | None for modern Python syntax
- Add Datasette type annotations
- Add SQL comment explaining cascading permission logic
- Refactor duplicated plugin result processing into helper function
2025-10-24 10:32:18 -07:00
Simon Willison
a21a1b6c14 Ran blacken-docs 2025-10-24 10:32:18 -07:00
Simon Willison
c0b5ce04c3 Ran cog 2025-10-24 10:32:18 -07:00
Simon Willison
9172020535 Removed unneccessary isinstance(candidate, PermissionSQL) 2025-10-24 10:32:18 -07:00
Simon Willison
4d6730e3c4 Remove unused methods from Resource base class 2025-10-24 10:32:18 -07:00
Simon Willison
c7278c73f3 Ran latest prettier 2025-10-24 10:32:18 -07:00
Simon Willison
96d2e16e83 Use allowed_resources_sql() with CTE for table filtering 2025-10-24 10:32:18 -07:00
Simon Willison
eb5a95ee6e Rewrite tables endpoint to use SQL LIKE instead of Python regex 2025-10-24 10:32:18 -07:00
Simon Willison
8e47f99874 Fix /-/tables endpoint: add .json support and correct response format 2025-10-24 10:32:18 -07:00
Simon Willison
7d04211559 Fix test_tables_endpoint_config_database_allow by using unique database names 2025-10-24 10:32:18 -07:00
Simon Willison
d73b6f169f Add register_actions hook to test plugin and improve test 2025-10-24 10:32:18 -07:00
Simon Willison
130dad268d Fix test_navigation_menu_links by enabling root_enabled for root actor 2025-10-24 10:32:18 -07:00
Simon Willison
e333827687 permission_allowed_default_allow_sql 2025-10-24 10:32:18 -07:00
Simon Willison
8b098e4b3e Applied Black 2025-10-24 10:32:18 -07:00
Simon Willison
06af34240f Fix permission endpoint tests by resolving method signature conflicts
- Renamed internal allowed_resources_sql() to _build_permission_rules_sql()
  to avoid conflict with public method
- Made public allowed_resources_sql() keyword-only to prevent argument order bugs
- Fixed PermissionRulesView to use _build_permission_rules_sql() which returns
  full permission rules (with allow/deny) instead of filtered resources
- Fixed _build_permission_rules_sql() to pass actor dict to build_rules_union()
- Added actor_id extraction in AllowedResourcesView
- Added root_enabled=True to test fixture to grant permissions-debug to root user

All 51 tests in test_permission_endpoints.py now pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:32:18 -07:00
Simon Willison
7423c1a999 Fixed some more tests 2025-10-24 10:32:18 -07:00
Simon Willison
98493b7587 Fix permission_allowed_sql_bridge to not apply defaults, closes #2526
The bridge was incorrectly using the new allowed() method which applies
default allow rules. This caused actors without restrictions to get True
instead of USE_DEFAULT, breaking backward compatibility.

Fixed by:
- Removing the code that converted to resource objects and called allowed()
- Bridge now ONLY checks config-based rules via _config_permission_rules()
- Returns None when no config rules exist, allowing Permission.default to apply
- This maintains backward compatibility with the permission_allowed() API

All 177 permission tests now pass, including test_actor_restricted_permissions
and test_permissions_checked which were previously failing.
2025-10-24 10:32:18 -07:00
Simon Willison
8b5bf3e487 Mark test_permissions_checked database download test as xfail, refs #2526
The test expects ensure_permissions() to check all three permissions
(view-database-download, view-database, view-instance) but the current
implementation short-circuits after the first successful check.

Created issue #2526 to track the investigation of the expected behavior.
2025-10-24 10:32:18 -07:00
Simon Willison
2ed2849a14 Eliminate duplicate config checking by removing old permission_allowed hooks
- Removed permission_allowed_default() hook (checked config twice)
- Removed _resolve_config_view_permissions() and _resolve_config_permissions_blocks() helpers
- Added permission_allowed_sql_bridge() to bridge old permission_allowed() API to new SQL system
- Moved default_allow_sql setting check into permission_resources_sql()
- Made root-level allow blocks apply to all view-* actions (view-database, view-table, view-query)
- Added add_row_allow_block() helper for allow blocks that should deny when no match

This resolves the duplicate checking issue where config blocks were evaluated twice:
once in permission_allowed hooks and once in permission_resources_sql hooks.

Note: One test still failing (test_permissions_checked for database download) - needs investigation
2025-10-24 10:32:18 -07:00
Simon Willison
b8d26754df Document datasette.allowed(), PermissionSQL class, and SQL parameters
- Added documentation for datasette.allowed() method with keyword-only arguments
- Added comprehensive PermissionSQL class documentation with examples
- Documented the three SQL parameters available: :actor, :actor_id, :action
- Included examples of using json_extract() to access actor fields
- Explained permission resolution rules (specificity, deny over allow, implicit deny)
- Fixed RST formatting warnings (escaped asterisk, fixed underline length)
2025-10-24 10:32:18 -07:00
Simon Willison
c06e05b7db New --root mechanism with datasette.root_enabled, closes #2521 2025-10-24 10:32:18 -07:00
Simon Willison
65c427e4ee Ensure :actor, :actor_id and :action are all available to permissions SQL, closes #2520
- Updated build_rules_union() to accept actor as dict and provide :actor (JSON) and :actor_id
- Updated resolve_permissions_from_catalog() and resolve_permissions_with_candidates() to accept actor dict
- :actor is now the full actor dict as JSON (use json_extract() to access fields)
- :actor_id is the actor's id field for simple comparisons
- :action continues to be available as before
- Updated all call sites and tests to use new parameter format
- Added test demonstrating all three parameters working together
2025-10-24 10:32:18 -07:00
Simon Willison
b9c6e7a0f6 PluginSQL renamed to PermissionSQL, closes #2524 2025-10-24 10:32:18 -07:00
Simon Willison
159b9f3fec ds.allowed() is now keyword-argument only, closes #2519 2025-10-24 10:32:18 -07:00
Simon Willison
8e9916b286 Update allowed_resources_sql() and refactor allowed_resources() 2025-10-24 10:32:18 -07:00
Simon Willison
b1080e7d30 Moved Resource defaults to datasette/resources.py 2025-10-24 10:32:18 -07:00
Simon Willison
5b0baf7cd5 Ran prettier 2025-10-24 10:32:18 -07:00
Simon Willison
2b879e462f Implement resource-based permission system with SQL-driven access control
This introduces a new hierarchical permission system that uses SQL queries
for efficient permission checking across resources. The system replaces the
older permission_allowed() pattern with a more flexible resource-based
approach.

Core changes:

- New Resource ABC and Action dataclass in datasette/permissions.py
  * Resources represent hierarchical entities (instance, database, table)
  * Each resource type implements resources_sql() to list all instances
  * Actions define operations on resources with cascading rules

- New plugin hook: register_actions(datasette)
  * Plugins register actions with their associated resource types
  * Replaces register_permissions() and register_resource_types()
  * See docs/plugin_hooks.rst for full documentation

- Three new Datasette methods for permission checks:
  * allowed_resources(action, actor) - returns list[Resource]
  * allowed_resources_with_reasons(action, actor) - for debugging
  * allowed(action, resource, actor) - checks single resource
  * All use SQL for filtering, never Python iteration

- New /-/tables endpoint (TablesView)
  * Returns JSON list of tables user can view
  * Supports ?q= parameter for regex filtering
  * Format: {"matches": [{"name": "db/table", "url": "/db/table"}]}
  * Respects all permission rules from configuration and plugins

- SQL-based permission evaluation (datasette/utils/actions_sql.py)
  * Cascading rules: child-level → parent-level → global-level
  * DENY beats ALLOW at same specificity
  * Uses CTEs for efficient SQL-only filtering
  * Combines permission_resources_sql() hook results

- Default actions in datasette/default_actions.py
  * InstanceResource, DatabaseResource, TableResource, QueryResource
  * Core actions: view-instance, view-database, view-table, etc.

- Fixed default_permissions.py to handle database-level allow blocks
  * Now creates parent-level rules for view-table action
  * Fixes: datasette ... -s databases.fixtures.allow.id root

Documentation:

- Comprehensive register_actions() hook documentation
- Detailed resources_sql() method explanation
- /-/tables endpoint documentation in docs/introspection.rst
- Deprecated register_permissions() with migration guide

Tests:

- tests/test_actions_sql.py: 7 tests for core permission API
- tests/test_tables_endpoint.py: 13 tests for /-/tables endpoint
- All 118 documentation tests pass
- Tests verify SQL does filtering (not Python)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:32:18 -07:00
Simon Willison
e951f7e81f
models: read permission for tmate 2025-10-22 16:16:49 -07:00
Simon Willison
2df06e1fda
GITHUB_TOKEN env for tmate.yml 2025-10-22 16:14:27 -07:00
Simon Willison
7ce723edcf
Reformat JavaScript files with Prettier (#2517)
* Reformat JavaScript files with Prettier

Ran `npm run fix` to apply consistent code formatting across JavaScript
files using the project's Prettier configuration (2 spaces, no tabs).

Files reformatted:
- datasette/static/datasette-manager.js
- datasette/static/json-format-highlight-1.0.1.js
- datasette/static/table.js

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Upgrade Prettier from 2.2.1 to 3.6.2

Updated package.json and package-lock.json to use Prettier 3.6.2,
ensuring consistent formatting between local development and CI.

The existing JavaScript files are already formatted with Prettier 3.x
style from the previous commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-20 16:41:09 -07:00
Simon Willison
ec38ad3768
Add DatabaseContext dataclass for consistent template context documentation (#2513)
Refs:
- #1510
- #2333

Claude Code:

Created DatabaseContext as a documented dataclass following the same pattern
as the existing QueryContext. This change replaces the inline dictionary
context creation with an explicit dataclass that:

- Documents all 21 template context variables with help metadata
- Inherits from the Context base class for identification
- Provides better IDE support and type safety
- Makes template variables discoverable without reading code

Also updated QueryContext to inherit from Context for consistency.
2025-10-09 12:54:02 -07:00
Simon Willison
659673614a Refactor debug templates to use shared JavaScript functions
Extracted common JavaScript utilities from debug_allowed.html, debug_check.html, and debug_rules.html into a new _debug_common_functions.html include template. This eliminates code duplication and improves maintainability.

The shared functions include:
- populateFormFromURL(): Populates form fields from URL query parameters
- updateURL(formId, page): Updates browser URL with form values
- escapeHtml(text): HTML escaping utility

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-08 21:53:34 -07:00
Simon Willison
e2a739c496 Fix for asyncio.iscoroutinefunction deprecation warnings
Closes #2512

Refs https://github.com/simonw/asyncinject/issues/18
2025-10-08 20:32:16 -07:00
Simon Willison
27084caa04
New allowed_resources_sql plugin hook and debug tools (#2505)
* allowed_resources_sql plugin hook and infrastructure
* New methods for checking permissions with the new system
* New /-/allowed and /-/check and /-/rules special endpoints

Still needs to be integrated more deeply into Datasette, especially for listing visible tables.

Refs: #2502

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-08 14:27:51 -07:00
Simon Willison
85da8474d4
Python 3.14, drop Python 3.9
Closes #2506
2025-10-08 13:11:32 -07:00
Simon Willison
909448fb7a Run CLI coroutines on explicit event loops
With the help of Codex CLI: https://gist.github.com/simonw/d2de93bfdf85a014a29093720c511093
2025-10-01 12:59:14 -07:00
Simon Willison
5d09ab3ff1 Remove legacy event_loop fixture usage 2025-10-01 12:51:23 -07:00
Simon Willison
571ce651c1 Use venv Python to launch datasette fixtures 2025-10-01 12:49:09 -07:00
Simon Willison
d87bd12dbc Remove obsolete mix_stderr=False 2025-09-30 14:33:24 -07:00
Simon Willison
9dc2a3ffe5 Removed broken refs to Glitch, closes #2503 2025-09-28 21:15:58 -07:00
Simon Willison
7a602140df catalog_views table, closes #2495
Refs https://github.com/datasette/datasette-queries/issues/1#issuecomment-3074491003
2025-07-15 10:22:56 -07:00
Simon Willison
e2497fdb59 Replace Glitch with Codespaces, closes #2488 2025-05-28 19:17:22 -07:00