Commit graph

144 commits

Author SHA1 Message Date
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
82cc3d5c86 Migrate view-query permission to SQL-based system, refs #2510
This change integrates canned queries with Datasette's new SQL-based
permissions system by making the following changes:

1. **Default canned_queries plugin hook**: Added a new hookimpl in
   default_permissions.py that returns canned queries from datasette
   configuration. This extracts config-reading logic into a plugin hook,
   allowing QueryResource to discover all queries.

2. **Async resources_sql()**: Converted Resource.resources_sql() from a
   synchronous class method returning a string to an async method that
   receives the datasette instance. This allows QueryResource to call
   plugin hooks and query the database.

3. **QueryResource implementation**: Implemented QueryResource.resources_sql()
   to gather all canned queries by:
   - Querying catalog_databases for all databases
   - Calling canned_queries hooks for each database with actor=None
   - Building a UNION ALL SQL query of all (database, query_name) pairs
   - Properly escaping single quotes in resource names

4. **Simplified get_canned_queries()**: Removed config-reading logic since
   it's now handled by the default plugin hook.

5. **Added view-query to default allow**: Added "view-query" to the
   default_allow_actions set so canned queries are accessible by default.

6. **Removed xfail markers**: Removed test xfail markers from:
   - tests/test_canned_queries.py (entire module)
   - tests/test_html.py (2 tests)
   - tests/test_permissions.py (1 test)
   - tests/test_plugins.py (1 test)

All canned query tests now pass with the new permission system.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
ee1d7983ba Mark canned query tests as xfail, refs #2510, refs #2528
Canned queries are not accessible because view-query permission
has not yet been migrated to the SQL-based permission system.

Marks the following tests with xfail:
- test_config_cache_size (test_api.py)
- test_edit_sql_link_not_shown_if_user_lacks_permission (test_html.py)
- test_database_color - removes canned query path (test_html.py)
- test_hook_register_output_renderer_* (test_plugins.py - 3 tests)
- test_hook_query_actions canned query parameter (test_plugins.py)
- test_custom_query_with_unicode_characters (test_table_api.py)
- test_permissions_checked neighborhood_search (test_permissions.py)
- test_padlocks_on_database_page (test_permissions.py)

All reference issue #2510 for tracking view-query migration.
2025-10-25 15:38:07 -07:00
Simon Willison
60a38cee85 Run black formatter 2025-10-25 15:38:07 -07:00
Simon Willison
d07f8944fa Mark canned query and magic parameter tests as xfail, refs #2510
These tests involve canned queries which use the view-query permission
that has not yet been migrated to the new SQL-based permission system.

Tests marked:
- test_hook_canned_queries (4 tests in test_plugins.py)
- test_hook_register_magic_parameters (test_plugins.py)
- test_hook_top_canned_query (test_plugins.py)
- test_canned_query_* (4 tests in test_html.py)
- test_edit_sql_link_on_canned_queries (test_html.py)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
59ccf797c4 Rename register_permissions tests to register_actions
- Renamed test_hook_register_permissions to test_hook_register_actions
- Renamed test_hook_register_permissions_no_duplicates to test_hook_register_actions_no_duplicates
- Renamed test_hook_register_permissions_allows_identical_duplicates to test_hook_register_actions_allows_identical_duplicates
- Updated all tests to use Action objects instead of Permission objects
- Updated config structures from datasette-register-permissions to datasette-register-actions
- Changed assertions from ds.permissions to ds.actions
- Updated test_hook_permission_allowed to register custom actions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
7aaff5e3d2 Update tests to use new allowed() method instead of permission_allowed() 2025-10-25 15:38:07 -07:00
Simon Willison
d73b6f169f Add register_actions hook to test plugin and improve test 2025-10-24 10:32:18 -07:00
Simon Willison
b9c6e7a0f6 PluginSQL renamed to PermissionSQL, closes #2524 2025-10-24 10:32:18 -07:00
Simon Willison
27084caa04
New allowed_resources_sql plugin hook and debug tools (#2505)
* allowed_resources_sql plugin hook and infrastructure
* New methods for checking permissions with the new system
* New /-/allowed and /-/check and /-/rules special endpoints

Still needs to be integrated more deeply into Datasette, especially for listing visible tables.

Refs: #2502

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-08 14:27:51 -07:00
Simon Willison
209bdee0e8 Don't run prepare_connection() on internal database, closes #2468 2025-02-18 10:23:23 -08:00
Simon Willison
53a3b3c80e
Test improvements and fixed deprecation warnings (#2464)
* `asyncio_default_fixture_loop_scope = function`
* Fix a bunch of BeautifulSoup deprecation warnings
* Fix for PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]>
* xfail for sql_time_limit tests (these can be flaky in CI)

Refs #2461
2025-02-04 14:49:52 -08:00
Simon Willison
4dff846271 simple_primary_key now uses integer id, helps close #2458 2025-02-01 21:44:53 -08:00
Simon Willison
dce718961c Async support for magic parameters
Closes #2441
2024-11-15 13:17:45 -08:00
Simon Willison
39dfc7d7d7
Removed units functionality and Pint dependency
Closes #2400, unblocks #2320
2024-08-20 19:03:33 -07:00
Alex Garcia
a23c2aee00
Introduce new /$DB/-/query endpoint, soft replaces /$DB?sql=... (#2363)
* Introduce new default /$DB/-/query endpoint
* Fix a lot of tests
* Update pyodide test to use query endpoint
* Link to /fixtures/-/query in a few places
* Documentation for QueryView

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-07-15 10:33:51 -07:00
Alex Garcia
e1bfab3fca
Move Metadata to --internal database
Refs:
- https://github.com/simonw/datasette/pull/2343
- https://github.com/simonw/datasette/issues/2341
2024-06-11 09:33:23 -07:00
Simon Willison
b8711988b9 row_actions() plugin hook, closes #2299 2024-03-12 16:16:05 -07:00
Simon Willison
06281a0b8e Test for labels on Table/View action buttons, refs #2297 2024-03-12 14:32:48 -07:00
Simon Willison
909c85cd2b view_actions plugin hook, closes #2297 2024-03-12 14:25:28 -07:00
Simon Willison
daf5ca02ca homepage_actions() plugin hook, closes #2298 2024-03-12 13:46:06 -07:00
Simon Willison
090dff542b
Action menu descriptions
* Refactor tests to extract get_actions_links() helper
* Table, database and query action menu items now support optional descriptions

Closes #2294
2024-03-06 22:54:06 -05:00
Simon Willison
4d24bf6b34 Don't explain an explain even in the demo, refs #2293 2024-03-05 18:14:55 -08:00
Simon Willison
5de6797d4a Better demo plugin for query_actions, refs #2293 2024-03-05 18:06:38 -08:00
Simon Willison
6ec0081f5d
query_actions plugin hook
* New query_actions plugin hook, closes #2283
2024-02-27 21:55:16 -08:00
Simon Willison
52a1dac5d2 Test proving $env works for datasette.yml, closes #2255 2024-02-06 21:00:55 -08:00
Simon Willison
69c6e95323 Fixed a bunch of unused imports spotted with ruff 2024-02-06 17:27:20 -08:00
Simon Willison
be4f02335f Treat plugins in metadata as if they were in config, closes #2248 2024-02-01 15:33:33 -08:00
Simon Willison
d4bc2b2dfc Remove fail_if_plugins_in_metadata, part of #2248 2024-02-01 14:44:16 -08:00
Simon Willison
bcc4f6bf1f
track_event() mechanism for analytics and plugins
* Closes #2240
* Documentation for event plugin hooks, refs #2240
* Include example track_event plugin in docs, refs #2240
* Tests for track_event() and register_events() hooks, refs #2240
* Initial documentation for core events, refs #2240
* Internals documentation for datasette.track_event()
2024-01-31 15:21:40 -08:00
Simon Willison
5c64af6936 Upgrade to latest Black, closes #2239 2024-01-30 19:55:26 -08:00
Simon Willison
c3caf36af7
Template slot family of plugin hooks - top_homepage() and others
New plugin hooks:

top_homepage
top_database
top_table
top_row
top_query
top_canned_query

New datasette.utils.make_slot_function()

Closes #1191
2024-01-30 19:54:03 -08:00
Simon Willison
c7a4706bcc
jinja2_environment_from_request() plugin hook
Closes #2225
2024-01-05 14:33:23 -08:00
Alex Garcia
3d6d1e3050
Raise an exception if a "plugins" block exists in metadata.json 2023-10-12 09:20:50 -07:00
Alex Garcia
35deaabcb1
Move non-metadata configuration from metadata.yaml to datasette.yaml
* Allow and permission blocks moved to datasette.yaml
* Documentation updates, initial framework for configuration reference
2023-10-12 09:16:37 -07:00
Simon Willison
852f501485 Switch from pkg_resources to importlib.metadata in app.py, refs #2057 2023-09-16 09:35:18 -07:00
Alex Garcia
b2ec8717c3
Plugin configuration now lives in datasette.yaml/json
* Checkpoint, moving top-level plugin config to datasette.json
* Support database-level and table-level plugin configuration in datasette.yaml

Refs #2093
2023-09-13 14:06:25 -07:00
Simon Willison
b645174271
actors_from_ids plugin hook and datasette.actors_from_ids() method (#2181)
* Prototype of actors_from_ids plugin hook, refs #2180
* datasette-remote-actors example plugin, refs #2180
2023-09-07 21:23:59 -07:00
Alex Garcia
92b8bf38c0
Add new --internal internal.db option, deprecate legacy _internal database
Refs:
- #2157 
---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2023-08-28 20:24:23 -07:00
Simon Willison
1377a290cd
New JSON design for query views (#2118)
* Refs #2111, closes #2110
* New Context dataclass/subclass mechanism, refs #2127
* Define QueryContext and extract get_tables() method, refs #2127
* Fix OPTIONS bug by porting DaatbaseView to be a View subclass
* Expose async_view_for_class.view_class for test_routes test
* Error/truncated aruments for renderers, closes #2130
2023-08-07 18:47:39 -07:00
Simon Willison
c39d600aef Fix all E741 Ambiguous variable name warnings, refs #2090 2023-06-29 08:05:24 -07:00
Simon Willison
d97e82df3c
?_extra= support and TableView refactor to table_view
* Implemented ?_extra= option for JSON views, refs #262
* New dependency: asyncinject
* Remove now-obsolete TableView class
2023-03-22 15:49:39 -07:00
Simon Willison
0b4a286914 render_cell(..., request) argument, closes #2007 2023-01-27 19:34:14 -08:00
Simon Willison
234230e595 Default JSON shape is now objects - refs #1914, #1709 2022-12-31 10:52:37 -08:00
Simon Willison
89cffcf14c Reset _metadata_local in a couple of tests
Refs https://github.com/simonw/datasette/pull/1960#issuecomment-1356476886
2022-12-17 13:47:55 -08:00
Simon Willison
9c43b4164d Removed @pytest.mark.ds_client mark - refs #1959
I don't need it - can run 'pytest -k ds_client' instead.

See https://github.com/simonw/datasette/pull/1960#issuecomment-1355685828
2022-12-17 13:47:55 -08:00
Simon Willison
30f1a0705b ds_client for test_plugins.py, refs #1959 2022-12-17 13:47:55 -08:00
Simon Willison
45979eb723 Rename permission created by demo plugin
It was showing up as 'new-permission' on https://latest.datasette.io/-/permissions
which I thought was confusing
2022-12-12 21:21:01 -08:00
Simon Willison
8bf06a76b5
register_permissions() plugin hook (#1940)
* Docs for permissions: in metadata, refs #1636
* Refactor default_permissions.py to help with implementation of #1636
* register_permissions() plugin hook, closes #1939 - also refs #1938
* Tests for register_permissions() hook, refs #1939
* Documentation for datasette.permissions, refs #1939
* permission_allowed() falls back on Permission.default, refs #1939
* Raise StartupError on duplicate permissions
* Allow dupe permisisons if exact matches
2022-12-12 18:05:54 -08:00
Simon Willison
c7956eed77 datasette create-token command, refs #1859 2022-10-25 21:26:12 -07:00