Commit graph

228 commits

Author SHA1 Message Date
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
82cc3d5c86 Migrate view-query permission to SQL-based system, refs #2510
This change integrates canned queries with Datasette's new SQL-based
permissions system by making the following changes:

1. **Default canned_queries plugin hook**: Added a new hookimpl in
   default_permissions.py that returns canned queries from datasette
   configuration. This extracts config-reading logic into a plugin hook,
   allowing QueryResource to discover all queries.

2. **Async resources_sql()**: Converted Resource.resources_sql() from a
   synchronous class method returning a string to an async method that
   receives the datasette instance. This allows QueryResource to call
   plugin hooks and query the database.

3. **QueryResource implementation**: Implemented QueryResource.resources_sql()
   to gather all canned queries by:
   - Querying catalog_databases for all databases
   - Calling canned_queries hooks for each database with actor=None
   - Building a UNION ALL SQL query of all (database, query_name) pairs
   - Properly escaping single quotes in resource names

4. **Simplified get_canned_queries()**: Removed config-reading logic since
   it's now handled by the default plugin hook.

5. **Added view-query to default allow**: Added "view-query" to the
   default_allow_actions set so canned queries are accessible by default.

6. **Removed xfail markers**: Removed test xfail markers from:
   - tests/test_canned_queries.py (entire module)
   - tests/test_html.py (2 tests)
   - tests/test_permissions.py (1 test)
   - tests/test_plugins.py (1 test)

All canned query tests now pass with the new permission system.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
ee1d7983ba Mark canned query tests as xfail, refs #2510, refs #2528
Canned queries are not accessible because view-query permission
has not yet been migrated to the SQL-based permission system.

Marks the following tests with xfail:
- test_config_cache_size (test_api.py)
- test_edit_sql_link_not_shown_if_user_lacks_permission (test_html.py)
- test_database_color - removes canned query path (test_html.py)
- test_hook_register_output_renderer_* (test_plugins.py - 3 tests)
- test_hook_query_actions canned query parameter (test_plugins.py)
- test_custom_query_with_unicode_characters (test_table_api.py)
- test_permissions_checked neighborhood_search (test_permissions.py)
- test_padlocks_on_database_page (test_permissions.py)

All reference issue #2510 for tracking view-query migration.
2025-10-25 15:38:07 -07:00
Simon Willison
60a38cee85 Run black formatter 2025-10-25 15:38:07 -07:00
Simon Willison
559a13a8c6 Mark additional canned query tests in test_html.py as xfail, refs #2510
Marked specific parameter combinations that test canned queries:
- test_css_classes_on_body with /fixtures/neighborhood_search
- test_alternate_url_json with /fixtures/neighborhood_search

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
09194c72f8 Replace permission_allowed_2() with allowed() in test_config_permission_rules.py
Updated all test_config_permission_rules.py tests to use the new allowed()
method with Resource objects instead of the old permission_allowed_2()
method.

Also marked test_database_page in test_html.py as xfail since it expects
to see canned queries (view-query permission not yet migrated).

All 7 config_permission_rules tests now pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
d07f8944fa Mark canned query and magic parameter tests as xfail, refs #2510
These tests involve canned queries which use the view-query permission
that has not yet been migrated to the new SQL-based permission system.

Tests marked:
- test_hook_canned_queries (4 tests in test_plugins.py)
- test_hook_register_magic_parameters (test_plugins.py)
- test_hook_top_canned_query (test_plugins.py)
- test_canned_query_* (4 tests in test_html.py)
- test_edit_sql_link_on_canned_queries (test_html.py)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
130dad268d Fix test_navigation_menu_links by enabling root_enabled for root actor 2025-10-24 10:32:18 -07:00
Simon Willison
f2485dce9c
Hide FTS tables that have content=
* Hide FTS tables that have content=, closes #2477
2025-04-16 21:44:09 -07:00
Simon Willison
333f786cb0 Correct syntax for link headers, closes #2470 2025-03-09 20:05:43 -05:00
Simon Willison
53a3b3c80e
Test improvements and fixed deprecation warnings (#2464)
* `asyncio_default_fixture_loop_scope = function`
* Fix a bunch of BeautifulSoup deprecation warnings
* Fix for PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]>
* xfail for sql_time_limit tests (these can be flaky in CI)

Refs #2461
2025-02-04 14:49:52 -08:00
Simon Willison
4dff846271 simple_primary_key now uses integer id, helps close #2458 2025-02-01 21:44:53 -08:00
Simon Willison
37873e02b0 Better breadcrumbs on database and table page, closes #2454 2025-01-09 10:07:03 -08:00
Alex Garcia
6d91d082e0
Hide shadow tables, don't hide virtual tables
Closes #2296
2024-08-15 13:19:22 -07:00
Simon Willison
05dfd34fd0 Use text/html for CSRF error page, refs #2390 2024-08-15 08:48:47 -07:00
Simon Willison
06d4ffb92e Custom error on CSRF failures, closes #2390
Uses https://github.com/simonw/asgi-csrf/issues/28
2024-08-14 21:29:16 -07:00
Simon Willison
93067668fe /-/ alternative URL for homepage, closes #2393 2024-08-14 17:57:13 -07:00
Alex Garcia
a23c2aee00
Introduce new /$DB/-/query endpoint, soft replaces /$DB?sql=... (#2363)
* Introduce new default /$DB/-/query endpoint
* Fix a lot of tests
* Update pyodide test to use query endpoint
* Link to /fixtures/-/query in a few places
* Documentation for QueryView

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-07-15 10:33:51 -07:00
Simon Willison
7437d40e5d <html lang="en">, closes #2348 2024-06-11 10:17:02 -07:00
Alex Garcia
e1bfab3fca
Move Metadata to --internal database
Refs:
- https://github.com/simonw/datasette/pull/2343
- https://github.com/simonw/datasette/issues/2341
2024-06-11 09:33:23 -07:00
Agustin Bacigalup
67e66f36c1
Add ETag header for static responses (#2306)
* add etag to static responses

* fix RuntimeError related to static headers

* Remove unnecessary import

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-03-17 12:18:40 -07:00
Simon Willison
60c6692f68
table_config instead of table_metadata (#2257)
Table configuration that was incorrectly placed in metadata is now treated as if it was in config.

New await datasette.table_config() method.

Closes #2247
2024-02-06 21:57:09 -08:00
Alex Garcia
35deaabcb1
Move non-metadata configuration from metadata.yaml to datasette.yaml
* Allow and permission blocks moved to datasette.yaml
* Documentation updates, initial framework for configuration reference
2023-10-12 09:16:37 -07:00
Simon Willison
4535568f2c Fixed display of database color
Closes #2139, closes #2119
2023-08-10 22:16:19 -07:00
Simon Willison
cd57b0f712 Brought back parameter fields, closes #2132 2023-08-08 06:45:04 -07:00
Simon Willison
1377a290cd
New JSON design for query views (#2118)
* Refs #2111, closes #2110
* New Context dataclass/subclass mechanism, refs #2127
* Define QueryContext and extract get_tables() method, refs #2127
* Fix OPTIONS bug by porting DaatbaseView to be a View subclass
* Expose async_view_for_class.view_class for test_routes test
* Error/truncated aruments for renderers, closes #2130
2023-08-07 18:47:39 -07:00
Simon Willison
9c43b4164d Removed @pytest.mark.ds_client mark - refs #1959
I don't need it - can run 'pytest -k ds_client' instead.

See https://github.com/simonw/datasette/pull/1960#issuecomment-1355685828
2022-12-17 13:47:55 -08:00
Simon Willison
42a66c2f04 A bunch of remaining ds_client conversions, refs #1959 2022-12-17 13:47:55 -08:00
Simon Willison
95900b9d02 Port app_client to ds_client for most of test_html.py, refs #1959 2022-12-17 13:47:55 -08:00
Simon Willison
4c18730e71 Update tests to export 200 for OPTIONS calls, refs #1922 2022-11-30 10:29:48 -08:00
Simon Willison
8d9a957c63 Fix for redirects ignoring base_url, refs #1883 2022-11-10 22:49:54 -08:00
Simon Willison
bbaab3b38e Test form actions use prefix, refs #1883 2022-11-10 22:20:40 -08:00
Simon Willison
2ec5583629 Show interrupted query in resizing textarea, closes #1876 2022-11-01 10:22:26 -07:00
Simon Willison
79aa0de083 Test that breadcrumbs respect permissions, closes #1831 2022-10-13 14:51:59 -07:00
Simon Willison
5f9f567acb Show SQL query when reporting time limit error, closes #1819 2022-09-26 16:06:01 -07:00
Simon Willison
5aa359b869 Apply cell truncation on query page too, refs #1805 2022-09-06 16:58:30 -07:00
Simon Willison
ba35105eee
Test --load-extension in GitHub Actions (#1792)
* Run the --load-extension test, refs #1789
* Ran cog, refs #1789
2022-08-23 17:11:45 -07:00
Simon Willison
a3e6f1b167 Increase height of non-JS textarea to fit query
Closes #1786
2022-08-18 09:06:02 -07:00
Simon Willison
280ff372ab ETag support for .db downloads, closes #1739 2022-05-03 07:59:46 -07:00
Simon Willison
90d1be9952 Tilde encoding now encodes space as plus, closes #1701
Refs #1657
2022-04-06 08:55:01 -07:00
Simon Willison
d4f60c2388
Remove hashed URL mode
Also simplified how view class routing works.

Refs #1661
2022-03-18 17:12:03 -07:00
Simon Willison
a35393b29c
Tilde encoding (#1659)
Closes #1657

Refs #1439
2022-03-15 11:01:57 -07:00
Simon Willison
77e718c3ff Revert "Fix bug with percentage redirects, close #1650"
This reverts commit c85d669de3.

Refs #1658
2022-03-15 08:37:31 -07:00
Simon Willison
c85d669de3 Fix bug with percentage redirects, close #1650 2022-03-07 11:26:08 -08:00
Simon Willison
020effe47b Preserve query string in % to - redirects, refs #1650 2022-03-07 08:18:07 -08:00
Simon Willison
644d25d1de Redirect old % URLs to new - encoded URLs, closes #1650
Refs #1439
2022-03-07 08:01:42 -08:00
Simon Willison
1baa030eca
Switch to dash encoding for table/database/row-pk in paths
* Dash encoding functions, tests and docs, refs #1439
* dash encoding is now like percent encoding but with dashes
* Use dash-encoding for row PKs and ?_next=, refs #1439
* Use dash encoding for table names, refs #1439
* Use dash encoding for database names, too, refs #1439

See also https://simonwillison.net/2022/Mar/5/dash-encoding/
2022-03-07 07:38:29 -08:00
Simon Willison
23a09b0f6a Remove JSON rel=alternate from some pages, closes #1623 2022-02-02 13:48:52 -08:00
Simon Willison
8d5779acf0 Refactored alternate_url_json mechanism, refs #1620, #1533 2022-02-02 13:32:47 -08:00
Simon Willison
b72b2423c7 rel=alternate JSON for queries and database pages, closes #1620 2022-02-02 13:22:45 -08:00