Commit graph

240 commits

Author SHA1 Message Date
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
82cc3d5c86 Migrate view-query permission to SQL-based system, refs #2510
This change integrates canned queries with Datasette's new SQL-based
permissions system by making the following changes:

1. **Default canned_queries plugin hook**: Added a new hookimpl in
   default_permissions.py that returns canned queries from datasette
   configuration. This extracts config-reading logic into a plugin hook,
   allowing QueryResource to discover all queries.

2. **Async resources_sql()**: Converted Resource.resources_sql() from a
   synchronous class method returning a string to an async method that
   receives the datasette instance. This allows QueryResource to call
   plugin hooks and query the database.

3. **QueryResource implementation**: Implemented QueryResource.resources_sql()
   to gather all canned queries by:
   - Querying catalog_databases for all databases
   - Calling canned_queries hooks for each database with actor=None
   - Building a UNION ALL SQL query of all (database, query_name) pairs
   - Properly escaping single quotes in resource names

4. **Simplified get_canned_queries()**: Removed config-reading logic since
   it's now handled by the default plugin hook.

5. **Added view-query to default allow**: Added "view-query" to the
   default_allow_actions set so canned queries are accessible by default.

6. **Removed xfail markers**: Removed test xfail markers from:
   - tests/test_canned_queries.py (entire module)
   - tests/test_html.py (2 tests)
   - tests/test_permissions.py (1 test)
   - tests/test_plugins.py (1 test)

All canned query tests now pass with the new permission system.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-25 15:38:07 -07:00
Simon Willison
ee1d7983ba Mark canned query tests as xfail, refs #2510, refs #2528
Canned queries are not accessible because view-query permission
has not yet been migrated to the SQL-based permission system.

Marks the following tests with xfail:
- test_config_cache_size (test_api.py)
- test_edit_sql_link_not_shown_if_user_lacks_permission (test_html.py)
- test_database_color - removes canned query path (test_html.py)
- test_hook_register_output_renderer_* (test_plugins.py - 3 tests)
- test_hook_query_actions canned query parameter (test_plugins.py)
- test_custom_query_with_unicode_characters (test_table_api.py)
- test_permissions_checked neighborhood_search (test_permissions.py)
- test_padlocks_on_database_page (test_permissions.py)

All reference issue #2510 for tracking view-query migration.
2025-10-25 15:38:07 -07:00
Simon Willison
f2485dce9c
Hide FTS tables that have content=
* Hide FTS tables that have content=, closes #2477
2025-04-16 21:44:09 -07:00
Simon Willison
53a3b3c80e
Test improvements and fixed deprecation warnings (#2464)
* `asyncio_default_fixture_loop_scope = function`
* Fix a bunch of BeautifulSoup deprecation warnings
* Fix for PytestUnraisableExceptionWarning: Exception ignored in: <_io.FileIO [closed]>
* xfail for sql_time_limit tests (these can be flaky in CI)

Refs #2461
2025-02-04 14:49:52 -08:00
Simon Willison
4dff846271 simple_primary_key now uses integer id, helps close #2458 2025-02-01 21:44:53 -08:00
Simon Willison
39dfc7d7d7
Removed units functionality and Pint dependency
Closes #2400, unblocks #2320
2024-08-20 19:03:33 -07:00
Simon Willison
53a8ae1871 Applied Black, refs #2327, #2326 2024-08-15 17:16:47 -07:00
Seb Bacon
9cb5700d60
bugfix: correctly detect json1 in versions.json (#2327)
Fixes #2326
2024-08-15 13:20:26 -07:00
Alex Garcia
6d91d082e0
Hide shadow tables, don't hide virtual tables
Closes #2296
2024-08-15 13:19:22 -07:00
Alex Garcia
a23c2aee00
Introduce new /$DB/-/query endpoint, soft replaces /$DB?sql=... (#2363)
* Introduce new default /$DB/-/query endpoint
* Fix a lot of tests
* Update pyodide test to use query endpoint
* Link to /fixtures/-/query in a few places
* Documentation for QueryView

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-07-15 10:33:51 -07:00
Alex Garcia
e1bfab3fca
Move Metadata to --internal database
Refs:
- https://github.com/simonw/datasette/pull/2343
- https://github.com/simonw/datasette/issues/2341
2024-06-11 09:33:23 -07:00
Simon Willison
7818e8b9d1 Hide tables starting with an _, refs #2104 2024-03-07 00:03:42 -05:00
Simon Willison
398a92cf1e Include database in name of _execute_writes thread, closes #2265 2024-02-08 20:12:31 -08:00
Simon Willison
60c6692f68
table_config instead of table_metadata (#2257)
Table configuration that was incorrectly placed in metadata is now treated as if it was in config.

New await datasette.table_config() method.

Closes #2247
2024-02-06 21:57:09 -08:00
Simon Willison
1e901aa690 /-/config page, closes #2254 2024-02-06 12:33:46 -08:00
Simon Willison
bcc4f6bf1f
track_event() mechanism for analytics and plugins
* Closes #2240
* Documentation for event plugin hooks, refs #2240
* Include example track_event plugin in docs, refs #2240
* Tests for track_event() and register_events() hooks, refs #2240
* Initial documentation for core events, refs #2240
* Internals documentation for datasette.track_event()
2024-01-31 15:21:40 -08:00
Simon Willison
2200abfa17 Fix for flaky test_hidden_sqlite_stat1_table, closes #2179 2023-09-07 15:49:50 -07:00
Simon Willison
e34d09c6ec Don't include columns in query JSON, refs #2136 2023-08-09 12:01:59 -07:00
Simon Willison
1377a290cd
New JSON design for query views (#2118)
* Refs #2111, closes #2110
* New Context dataclass/subclass mechanism, refs #2127
* Define QueryContext and extract get_tables() method, refs #2127
* Fix OPTIONS bug by porting DaatbaseView to be a View subclass
* Expose async_view_for_class.view_class for test_routes test
* Error/truncated aruments for renderers, closes #2130
2023-08-07 18:47:39 -07:00
Simon Willison
8cd60fd1d8 Homepage test now just asserts isinstance(x, int) - closes #2092 2023-06-29 08:24:09 -07:00
Simon Willison
b49fa446d6 --cors Access-Control-Max-Age: 3600, closes #2079 2023-05-25 15:05:58 -07:00
Simon Willison
d97e82df3c
?_extra= support and TableView refactor to table_view
* Implemented ?_extra= option for JSON views, refs #262
* New dependency: asyncinject
* Remove now-obsolete TableView class
2023-03-22 15:49:39 -07:00
Simon Willison
c41278b46f default_allow_sql setting, closes #1409
Refs #1410
2023-01-04 16:51:26 -08:00
Simon Willison
5bbe2bcc50 Rename filtered_table_rows_count to count, refs #782 2022-12-31 12:52:57 -08:00
Simon Willison
234230e595 Default JSON shape is now objects - refs #1914, #1709 2022-12-31 10:52:37 -08:00
Simon Willison
9c43b4164d Removed @pytest.mark.ds_client mark - refs #1959
I don't need it - can run 'pytest -k ds_client' instead.

See https://github.com/simonw/datasette/pull/1960#issuecomment-1355685828
2022-12-17 13:47:55 -08:00
Simon Willison
42a66c2f04 A bunch of remaining ds_client conversions, refs #1959 2022-12-17 13:47:55 -08:00
Simon Willison
b077e63dc6 Ported test_api.py app_client test to ds_client, refs #1959 2022-12-17 13:47:55 -08:00
Simon Willison
31d6a0bc5e Applied Black, refs #1922 2022-11-30 15:17:39 -08:00
Simon Willison
f0fadc28dd Access-Control-Allow-Headers: Authorization, Content-Type - refs #1922 2022-11-30 15:11:18 -08:00
Simon Willison
2cd7ecaa0a Apply Black, refs #1922 2022-11-30 13:54:47 -08:00
Simon Willison
6bfd71f5c6 Access-Control-Allow-Methods: GET, POST, HEAD, OPTIONS - refs #1922 2022-11-30 12:25:12 -08:00
Simon Willison
48725bb4ea CORS headers for write APIs, refs #1922 2022-11-30 09:27:10 -08:00
Simon Willison
93a02281da Show interrupted query in resizing textarea, closes #1876 2022-11-01 10:38:24 -07:00
Simon Willison
c35859ae3d API for bulk inserts, closes #1866 2022-10-29 23:03:45 -07:00
Simon Willison
382a871583 max_signed_tokens_ttl setting, closes #1858
Also redesigned token format to include creation time and optional duration.
2022-10-26 20:14:59 -07:00
Simon Willison
c7956eed77 datasette create-token command, refs #1859 2022-10-25 21:26:12 -07:00
Simon Willison
5f9f567acb Show SQL query when reporting time limit error, closes #1819 2022-09-26 16:06:01 -07:00
Simon Willison
d0737e4de5 truncate_cells_html now affects URLs too, refs #1805 2022-09-06 16:50:43 -07:00
Simon Willison
ba35105eee
Test --load-extension in GitHub Actions (#1792)
* Run the --load-extension test, refs #1789
* Ran cog, refs #1789
2022-08-23 17:11:45 -07:00
Simon Willison
b9c2b1cfc8 Consistent treatment of format in route capturing, refs #1667
Also refs #1660
2022-03-19 13:29:10 -07:00
Simon Willison
4e47a2d894 Fixed bug where tables with a column called n caused 500 errors
Closes #1228
2022-03-18 18:37:54 -07:00
Simon Willison
9979dcd07f Also remove default_cache_ttl_hashed setting, refs #1661 2022-03-18 17:25:14 -07:00
Simon Willison
d4f60c2388
Remove hashed URL mode
Also simplified how view class routing works.

Refs #1661
2022-03-18 17:12:03 -07:00
Simon Willison
a35393b29c
Tilde encoding (#1659)
Closes #1657

Refs #1439
2022-03-15 11:01:57 -07:00
Simon Willison
1baa030eca
Switch to dash encoding for table/database/row-pk in paths
* Dash encoding functions, tests and docs, refs #1439
* dash encoding is now like percent encoding but with dashes
* Use dash-encoding for row PKs and ?_next=, refs #1439
* Use dash encoding for table names, refs #1439
* Use dash encoding for database names, too, refs #1439

See also https://simonwillison.net/2022/Mar/5/dash-encoding/
2022-03-07 07:38:29 -08:00
Simon Willison
ac239d34ab Refactor test_trace into separate test module, refs #1576 2022-02-04 20:45:13 -08:00
Simon Willison
3ef47a0896 Link rel=alternate header for tables and rows
Also added Access-Control-Expose-Headers: Link to --cors mode.

Closes #1533

Refs https://github.com/simonw/datasette-notebook/issues/2

LL#	metadata.json.1
2022-02-01 23:49:09 -08:00
Simon Willison
58652dd925 Hidden tables sqlite1/2/3/4, closes #1587 2022-01-19 20:12:46 -08:00