Commit graph

2,742 commits

Author SHA1 Message Date
Simon Willison
785f2ad0bd Apply database-level allow blocks to view-query action, refs #2510
When a database has an "allow" block in the configuration, it should
apply to all queries in that database, not just tables and the database
itself. This fix ensures that queries respect database-level access
controls.

This fixes the test_padlocks_on_database_page test which expects
plugin-defined queries (from_async_hook, from_hook) to show padlock
indicators when the database has restricted access.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 13:44:24 -07:00
Simon Willison
16b2729847 Fix #2509: Settings-based deny rules now override root user privileges
The root user's permission_resources_sql hook was returning early with a
blanket "allow all" rule, preventing settings-based deny rules from being
considered. This caused /-/allowed and /-/rules endpoints to incorrectly
show resources that were denied via settings.

Changed permission_resources_sql to append root permissions to the rules
list instead of returning early, allowing config-based deny rules to be
evaluated. The SQL cascading logic correctly applies: deny rules at the
same depth beat allow rules, so database-level denies override root's
global-level allow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 11:10:58 -07:00
Simon Willison
d1ea067fde Migrate homepage to use bulk allowed_resources() and fix NULL handling in SQL JOINs
- Updated IndexView in datasette/views/index.py to fetch all allowed databases and tables
  in bulk upfront using allowed_resources() instead of calling check_visibility() for each
  database, table, and view individually
- Fixed SQL bug in build_allowed_resources_sql() where USING (parent, child) clauses failed
  for database resources because NULL = NULL evaluates to NULL in SQL, not TRUE
- Changed all INNER JOINs to use explicit ON conditions with NULL-safe comparisons:
  ON b.parent = x.parent AND (b.child = x.child OR (b.child IS NULL AND x.child IS NULL))

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 10:03:40 -07:00
Simon Willison
3adddad6aa Add parent filter and include_is_private to allowed_resources()
Major improvements to the allowed_resources() API:

1. **parent filter**: Filter results to specific database in SQL, not Python
   - Avoids loading thousands of tables into Python memory
   - Filtering happens efficiently in SQLite

2. **include_is_private flag**: Detect private resources in single SQL query
   - Compares actor permissions vs anonymous permissions in SQL
   - LEFT JOIN between actor_allowed and anon_allowed CTEs
   - Returns is_private column: 1 if anonymous blocked, 0 otherwise
   - No individual check_visibility() calls needed

3. **Resource.private property**: Safe access with clear error messages
   - Raises AttributeError if accessed without include_is_private=True
   - Prevents accidental misuse of the property

4. **Database view optimization**: Use new API to eliminate redundant checks
   - Single bulk query replaces N individual permission checks
   - Private flag computed in SQL, not via check_visibility() calls
   - Views filtered from allowed_dict instead of checking db.view_names()

All permission filtering now happens in SQLite where it belongs, with
minimal data transferred to Python.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 09:30:37 -07:00
Simon Willison
1134b22a27 Migrate /database view to use bulk allowed_resources()
Replace one-by-one permission checks with bulk allowed_resources() call:
- DatabaseView and QueryView now fetch all allowed tables once
- Filter views and tables using pre-fetched allowed_table_set
- Update TableResource.resources_sql() to include views from catalog_views

This improves performance by reducing permission checks from O(n) to O(1) per
table/view, where n is the number of tables in the database.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-24 00:28:16 -07:00
Simon Willison
11c039d35e Error on startup if invalid setting types 2025-10-24 00:14:28 -07:00
Simon Willison
3e6e8ee047 Simplify types in datasette/permissions.py 2025-10-23 23:54:04 -07:00
Simon Willison
e4f5b5c30f Fix schema mismatch in empty result query
When no permission rules exist, the query was returning 2 columns (parent, child)
but the function contract specifies 3 columns (parent, child, reason). This could
cause schema mismatches in consuming code.

Added 'NULL AS reason' to match the documented 3-column schema.

Added regression test that verifies the schema has 3 columns even when no
permission rules are returned. The test fails without the fix (showing only
2 columns) and passes with it.

Thanks to @asg017 for catching this
2025-10-23 21:48:12 -07:00
Simon Willison
bd5e969c8b Address PR #2515 review comments
- Add URL to sqlite-permissions-poc in module docstring
- Replace Optional with | None for modern Python syntax
- Add Datasette type annotations
- Add SQL comment explaining cascading permission logic
- Refactor duplicated plugin result processing into helper function
2025-10-23 16:08:56 -07:00
Simon Willison
e71c083700 Ran blacken-docs 2025-10-23 15:53:49 -07:00
Simon Willison
e5316215aa Ran cog 2025-10-23 15:50:26 -07:00
Simon Willison
092ada7b7d Removed unneccessary isinstance(candidate, PermissionSQL) 2025-10-23 15:48:48 -07:00
Simon Willison
5919de0384 Remove unused methods from Resource base class 2025-10-23 15:48:31 -07:00
Simon Willison
4880102b5d Ran latest prettier 2025-10-23 15:36:16 -07:00
Simon Willison
4b50cc7bc1 Use allowed_resources_sql() with CTE for table filtering 2025-10-23 15:34:36 -07:00
Simon Willison
c4f0365130 Rewrite tables endpoint to use SQL LIKE instead of Python regex 2025-10-23 15:29:51 -07:00
Simon Willison
19a37303c7 Fix /-/tables endpoint: add .json support and correct response format 2025-10-23 15:28:37 -07:00
Simon Willison
8de5b9431c Fix test_tables_endpoint_config_database_allow by using unique database names 2025-10-23 15:26:14 -07:00
Simon Willison
275c06fbe4 Add register_actions hook to test plugin and improve test 2025-10-23 15:24:10 -07:00
Simon Willison
d4dd08933e Fix test_navigation_menu_links by enabling root_enabled for root actor 2025-10-23 15:20:16 -07:00
Simon Willison
28a69d19a2 permission_allowed_default_allow_sql 2025-10-23 15:17:29 -07:00
Simon Willison
8bb07f80b1 Applied Black 2025-10-23 15:08:34 -07:00
Simon Willison
4d93149c2b Fix permission endpoint tests by resolving method signature conflicts
- Renamed internal allowed_resources_sql() to _build_permission_rules_sql()
  to avoid conflict with public method
- Made public allowed_resources_sql() keyword-only to prevent argument order bugs
- Fixed PermissionRulesView to use _build_permission_rules_sql() which returns
  full permission rules (with allow/deny) instead of filtered resources
- Fixed _build_permission_rules_sql() to pass actor dict to build_rules_union()
- Added actor_id extraction in AllowedResourcesView
- Added root_enabled=True to test fixture to grant permissions-debug to root user

All 51 tests in test_permission_endpoints.py now pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-23 14:53:07 -07:00
Simon Willison
475f817c5a Fixed some more tests 2025-10-23 14:43:51 -07:00
Simon Willison
2039e238d9 Fix permission_allowed_sql_bridge to not apply defaults, closes #2526
The bridge was incorrectly using the new allowed() method which applies
default allow rules. This caused actors without restrictions to get True
instead of USE_DEFAULT, breaking backward compatibility.

Fixed by:
- Removing the code that converted to resource objects and called allowed()
- Bridge now ONLY checks config-based rules via _config_permission_rules()
- Returns None when no config rules exist, allowing Permission.default to apply
- This maintains backward compatibility with the permission_allowed() API

All 177 permission tests now pass, including test_actor_restricted_permissions
and test_permissions_checked which were previously failing.
2025-10-23 14:34:48 -07:00
Simon Willison
e42b040055 Mark test_permissions_checked database download test as xfail, refs #2526
The test expects ensure_permissions() to check all three permissions
(view-database-download, view-database, view-instance) but the current
implementation short-circuits after the first successful check.

Created issue #2526 to track the investigation of the expected behavior.
2025-10-23 14:23:46 -07:00
Simon Willison
f4245dce66 Eliminate duplicate config checking by removing old permission_allowed hooks
- Removed permission_allowed_default() hook (checked config twice)
- Removed _resolve_config_view_permissions() and _resolve_config_permissions_blocks() helpers
- Added permission_allowed_sql_bridge() to bridge old permission_allowed() API to new SQL system
- Moved default_allow_sql setting check into permission_resources_sql()
- Made root-level allow blocks apply to all view-* actions (view-database, view-table, view-query)
- Added add_row_allow_block() helper for allow blocks that should deny when no match

This resolves the duplicate checking issue where config blocks were evaluated twice:
once in permission_allowed hooks and once in permission_resources_sql hooks.

Note: One test still failing (test_permissions_checked for database download) - needs investigation
2025-10-23 13:53:01 -07:00
Simon Willison
faef51ad05 Document datasette.allowed(), PermissionSQL class, and SQL parameters
- Added documentation for datasette.allowed() method with keyword-only arguments
- Added comprehensive PermissionSQL class documentation with examples
- Documented the three SQL parameters available: :actor, :actor_id, :action
- Included examples of using json_extract() to access actor fields
- Explained permission resolution rules (specificity, deny over allow, implicit deny)
- Fixed RST formatting warnings (escaped asterisk, fixed underline length)
2025-10-23 12:42:10 -07:00
Simon Willison
5fc58c8775 New --root mechanism with datasette.root_enabled, closes #2521 2025-10-23 12:40:50 -07:00
Simon Willison
1d37d30c2a Ensure :actor, :actor_id and :action are all available to permissions SQL, closes #2520
- Updated build_rules_union() to accept actor as dict and provide :actor (JSON) and :actor_id
- Updated resolve_permissions_from_catalog() and resolve_permissions_with_candidates() to accept actor dict
- :actor is now the full actor dict as JSON (use json_extract() to access fields)
- :actor_id is the actor's id field for simple comparisons
- :action continues to be available as before
- Updated all call sites and tests to use new parameter format
- Added test demonstrating all three parameters working together
2025-10-23 09:48:55 -07:00
Simon Willison
5ed57607e5 PluginSQL renamed to PermissionSQL, closes #2524 2025-10-23 09:34:19 -07:00
Simon Willison
cf887e0277 ds.allowed() is now keyword-argument only, closes #2519 2025-10-23 09:25:33 -07:00
Simon Willison
7dfd14bb07 Update allowed_resources_sql() and refactor allowed_resources() 2025-10-20 16:26:54 -07:00
Simon Willison
3663b9df2d Moved Resource defaults to datasette/resources.py 2025-10-20 16:23:14 -07:00
Simon Willison
9e5c64c3de Ran prettier 2025-10-20 16:03:22 -07:00
Simon Willison
7db754c284 Implement resource-based permission system with SQL-driven access control
This introduces a new hierarchical permission system that uses SQL queries
for efficient permission checking across resources. The system replaces the
older permission_allowed() pattern with a more flexible resource-based
approach.

Core changes:

- New Resource ABC and Action dataclass in datasette/permissions.py
  * Resources represent hierarchical entities (instance, database, table)
  * Each resource type implements resources_sql() to list all instances
  * Actions define operations on resources with cascading rules

- New plugin hook: register_actions(datasette)
  * Plugins register actions with their associated resource types
  * Replaces register_permissions() and register_resource_types()
  * See docs/plugin_hooks.rst for full documentation

- Three new Datasette methods for permission checks:
  * allowed_resources(action, actor) - returns list[Resource]
  * allowed_resources_with_reasons(action, actor) - for debugging
  * allowed(action, resource, actor) - checks single resource
  * All use SQL for filtering, never Python iteration

- New /-/tables endpoint (TablesView)
  * Returns JSON list of tables user can view
  * Supports ?q= parameter for regex filtering
  * Format: {"matches": [{"name": "db/table", "url": "/db/table"}]}
  * Respects all permission rules from configuration and plugins

- SQL-based permission evaluation (datasette/utils/actions_sql.py)
  * Cascading rules: child-level → parent-level → global-level
  * DENY beats ALLOW at same specificity
  * Uses CTEs for efficient SQL-only filtering
  * Combines permission_resources_sql() hook results

- Default actions in datasette/default_actions.py
  * InstanceResource, DatabaseResource, TableResource, QueryResource
  * Core actions: view-instance, view-database, view-table, etc.

- Fixed default_permissions.py to handle database-level allow blocks
  * Now creates parent-level rules for view-table action
  * Fixes: datasette ... -s databases.fixtures.allow.id root

Documentation:

- Comprehensive register_actions() hook documentation
- Detailed resources_sql() method explanation
- /-/tables endpoint documentation in docs/introspection.rst
- Deprecated register_permissions() with migration guide

Tests:

- tests/test_actions_sql.py: 7 tests for core permission API
- tests/test_tables_endpoint.py: 13 tests for /-/tables endpoint
- All 118 documentation tests pass
- Tests verify SQL does filtering (not Python)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-20 16:00:36 -07:00
Simon Willison
ec38ad3768
Add DatabaseContext dataclass for consistent template context documentation (#2513)
Refs:
- #1510
- #2333

Claude Code:

Created DatabaseContext as a documented dataclass following the same pattern
as the existing QueryContext. This change replaces the inline dictionary
context creation with an explicit dataclass that:

- Documents all 21 template context variables with help metadata
- Inherits from the Context base class for identification
- Provides better IDE support and type safety
- Makes template variables discoverable without reading code

Also updated QueryContext to inherit from Context for consistency.
2025-10-09 12:54:02 -07:00
Simon Willison
659673614a Refactor debug templates to use shared JavaScript functions
Extracted common JavaScript utilities from debug_allowed.html, debug_check.html, and debug_rules.html into a new _debug_common_functions.html include template. This eliminates code duplication and improves maintainability.

The shared functions include:
- populateFormFromURL(): Populates form fields from URL query parameters
- updateURL(formId, page): Updates browser URL with form values
- escapeHtml(text): HTML escaping utility

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-08 21:53:34 -07:00
Simon Willison
e2a739c496 Fix for asyncio.iscoroutinefunction deprecation warnings
Closes #2512

Refs https://github.com/simonw/asyncinject/issues/18
2025-10-08 20:32:16 -07:00
Simon Willison
27084caa04
New allowed_resources_sql plugin hook and debug tools (#2505)
* allowed_resources_sql plugin hook and infrastructure
* New methods for checking permissions with the new system
* New /-/allowed and /-/check and /-/rules special endpoints

Still needs to be integrated more deeply into Datasette, especially for listing visible tables.

Refs: #2502

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-08 14:27:51 -07:00
Simon Willison
85da8474d4
Python 3.14, drop Python 3.9
Closes #2506
2025-10-08 13:11:32 -07:00
Simon Willison
909448fb7a Run CLI coroutines on explicit event loops
With the help of Codex CLI: https://gist.github.com/simonw/d2de93bfdf85a014a29093720c511093
2025-10-01 12:59:14 -07:00
Simon Willison
5d09ab3ff1 Remove legacy event_loop fixture usage 2025-10-01 12:51:23 -07:00
Simon Willison
571ce651c1 Use venv Python to launch datasette fixtures 2025-10-01 12:49:09 -07:00
Simon Willison
d87bd12dbc Remove obsolete mix_stderr=False 2025-09-30 14:33:24 -07:00
Simon Willison
9dc2a3ffe5 Removed broken refs to Glitch, closes #2503 2025-09-28 21:15:58 -07:00
Simon Willison
7a602140df catalog_views table, closes #2495
Refs https://github.com/datasette/datasette-queries/issues/1#issuecomment-3074491003
2025-07-15 10:22:56 -07:00
Simon Willison
e2497fdb59 Replace Glitch with Codespaces, closes #2488 2025-05-28 19:17:22 -07:00
Simon Willison
1c77a7e33f Fix global-power-points references
Refs https://github.com/simonw/datasette.io/issues/167
2025-05-28 19:07:46 -07:00
Simon Willison
6f7f4c7d89 Release 1.0a19 1.0a19
Refs #2479
2025-04-21 22:38:53 -07:00