Commit graph

2,906 commits

Author SHA1 Message Date
Simon Willison
5602f3efa5 Extract query page rendering into new QueryPage module
Refactor QueryView.get() by moving query execution, CSV streaming,
renderer dispatch, and HTML template rendering into a new
datasette/query_page.py module. Share renderer dispatch logic between
query and table views via dispatch_renderer(). Fix query view passing
view_name="query" instead of "table" to output renderers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 08:49:41 -08:00
Simon Willison
5873578d49 Release 1.0a24 1.0a24
Refs #2050, #2346, #2608, #2609, #2610, #2611, #2613, #2619, #2624, #2627, #2628, #2629, #2630, #2632
2026-01-29 09:00:22 -08:00
Daniel Olasubomi Sobowale
b771e930bc
Fix filter-input and search-input zoom on iOS Safari
Closes #2346
2026-01-28 18:41:58 -08:00
Simon Willison
40a37307de
Add request.form() for multipart form data and file uploads
* Add request.form() for multipart form data and file uploads

New Request.form() method that handles both application/x-www-form-urlencoded
and multipart/form-data content types with streaming parsing.

Features:
- Streaming multipart parser that doesn't buffer entire body in memory
- Files spill to disk above 1MB threshold via SpooledTemporaryFile
- files=False (default) discards file content, files=True stores them
- Security limits: max_request_size, max_file_size, max_fields, max_files
- FormData container with dict-like access and getlist() for multiple values
- UploadedFile class with async read(), seek(), filename, content_type, size
- Support for RFC 5987 filename* encoding for international filenames

Uses multipart-form-data-conformance test suite for validation.

* Update views to use request.form() and document new API

- Migrate PermissionsDebugView, MessagesDebugView, and CreateTokenView
  from post_vars() to form()
- Add documentation for request.form(), FormData, and UploadedFile classes

Centralize multipart defaults and expose stricter limits via Request.form().

Enforce header, part, file, and disk space limits even when files are discarded; detect truncated bodies and client disconnects; and move blocking work off the event loop.

Add FormData close/aclose context managers, update internals docs, and expand multipart tests (including len semantics and stricter conformance expectations).
2026-01-28 18:41:03 -08:00
Simon Willison
ffadb5f74c
Workaround for intermittent test failure on SQLite 3.25.3
Closes:
- #2632
2026-01-28 18:34:00 -08:00
Simon Willison
3f8f97e92a Close more connections in test suite
To try and avoid too many open files on macOS
2026-01-28 09:55:25 -08:00
Simon Willison
2f7b120177 Minor speedup for remove_infinites, refs #2629 2026-01-24 22:07:54 -08:00
Simon Willison
7988a179fe Throttle schema refreshes to at most once per second, refs #2629 2026-01-23 21:03:16 -08:00
Simon Willison
7915c46ddd
Fix flaky test_database_page test with deterministic ordering (#2628)
* Fix flaky test_database_page test with deterministic ordering

- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
  500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst

* Update test_row_foreign_key_tables for new deterministic FK ordering

The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.

* Update test_table_names for new alphabetical ordering

The table_names() method now returns tables sorted alphabetically.

* Fix for test that fails prior to SQLite 3.37

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 20:57:25 -08:00
Simon Willison
66d2a033f8 Switch to ruff and fix all lint errors, refs #2630 2026-01-23 20:43:16 -08:00
Simon Willison
b0436faa5e
Fix test isolation bug in test_startup_error_from_plugin_is_click_exception (#2627)
* Fix test isolation bug in test_startup_error_from_plugin_is_click_exception

The test creates a plugin that raises StartupError("boom") and registers it
in the global plugin manager (pm). Without cleanup, this plugin leaks to
subsequent tests, causing test_setting_boolean_validation_false_values to
fail with "Error: boom" instead of "Forbidden".

Add try/finally block to ensure the plugin is unregistered after the test
completes, following the established cleanup pattern used elsewhere in
the test suite.

* Fix blacken-docs formatting in plugin_hooks.rst

Apply blacken-docs formatting to code example that exceeded
the 60 character line limit.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-22 07:03:05 -08:00
Simon Willison
b52655e856 Ignore *.db in gitignore 2026-01-06 07:59:07 -08:00
Simon Willison
757ce92baf datasette.utils.StartupError() now becomes a click exception, closes #2624 2026-01-06 07:58:48 -08:00
Simon Willison
6fede23a2e Only return render_coll columns that differ from default, refs #2619 2025-12-21 20:18:26 -08:00
Simon Willison
eae94dc2c3 Initial render_cell and foreign_key_tables extras for row
Closes #2619, refs #2050
2025-12-21 20:03:10 -08:00
Simon Willison
97496d5a67 ?_extra=render_cells for tables, refs #2619 2025-12-21 19:52:57 -08:00
Simon Willison
232a404743 Switch searchable_fts test table to FTS5, closes #2613 2025-12-12 22:18:35 -08:00
Simon Willison
3b4c7e1abe {"ok": true} on row API, to be consistent with table 2025-12-12 21:43:00 -08:00
Simon Willison
4cbdfcc07d
dependency-groups and uv (#2611)
* dependency-groups and uv, closes #2610
* New .readthedocs config for --group dev
2025-12-11 17:32:58 -08:00
Simon Willison
1d4448fc56
Use subtests in tests/test_docs.py (#2609)
Closes #2608
2025-12-04 21:36:39 -08:00
Simon Willison
2ca00b6c75 Release 1.0a23 1.0a23
Refs #2605, #2599
2025-12-02 19:20:43 -08:00
Simon Willison
03ab359208 tool.uv.package = true 2025-12-02 19:19:48 -08:00
Simon Willison
3eca3ad6d4 Better recipe for 'just docs' 2025-12-02 19:16:39 -08:00
Simon Willison
0a924524be
Split default_permissions.py into a package (#2603)
* Split default_permissions.py into a package, refs #2602

* Remove unused is_resource_allowed() method, improve test coverage

- Remove dead code: is_resource_allowed() method was never called
- Change isinstance check to assertion with error message
- Add test cases for table-level restrictions in restrictions_allow_action()
- Coverage for restrictions.py improved from 79% to 99%

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Additional permission test for gap spotted by coverage
2025-12-02 19:11:31 -08:00
Simon Willison
170b3ff61c Better fix for stale catalog_databases, closes #2606
Refs 2605
2025-12-02 19:00:13 -08:00
Simon Willison
c6c2a238c3 Fix for stale internal database bug, closes #2605 2025-12-02 16:22:42 -08:00
Simon Willison
68f1179bac Fix for text None shown on /-/actions, closes #2599 2025-11-26 17:12:52 -08:00
Simon Willison
2125115cd9 Release 1.0a22 1.0a22
Refs #2592, #2594, #2595, #2596
2025-11-13 10:41:02 -08:00
Simon Willison
93b455239a Release notes for 1.0a22, closes #2596 2025-11-13 10:40:24 -08:00
Simon Willison
4b4add4d31 datasette.pm property, closes #2595 2025-11-13 10:31:03 -08:00
Simon Willison
5125bef573 datasette.in_client() method, closes #2594 2025-11-13 10:00:04 -08:00
Simon Willison
23a640d38b
datasette serve --default-deny option (#2593)
Closes #2592
2025-11-12 16:14:21 -08:00
dependabot[bot]
32a425868c
Bump black from 25.9.0 to 25.11.0 in the python-packages group (#2590)
Bumps the python-packages group with 1 update: [black](https://github.com/psf/black).


Updates `black` from 25.9.0 to 25.11.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.9.0...25.11.0)

---
updated-dependencies:
- dependency-name: black
  dependency-version: 25.11.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-12 06:07:16 -08:00
Simon Willison
291f71ec6b
Remove out-dated plugin_hook_permission_allowed references 2025-11-11 21:59:26 -08:00
Simon Willison
354d7a2873
Bump a few versions, deploy on push to main
Refs:
- #2511
2025-11-09 15:42:11 -08:00
Simon Willison
a508fc4a8e Remove permission_allowed hook docs, closes #2588
Refs #2528
2025-11-07 16:50:00 -08:00
Simon Willison
8bc9b1ee03
/-/schema and /db/-/schema and /db/table/-/schema pages (plus .json/.md)
* Add schema endpoints for databases, instances, and tables

Closes: #2586

This commit adds new endpoints to view database schemas in multiple formats:

- /-/schema - View schemas for all databases (HTML, JSON, MD)
- /database/-/schema - View schema for a specific database (HTML, JSON, MD)
- /database/table/-/schema - View schema for a specific table (JSON, MD)

Features:
- Supports HTML, JSON, and Markdown output formats
- Respects view-database and view-table permissions
- Uses group_concat(sql, ';' || CHAR(10)) from sqlite_master to retrieve schemas
- Includes comprehensive tests covering all formats and permission checks

The JSON endpoints return:
- Instance level: {"schemas": [{"database": "name", "schema": "sql"}, ...]}
- Database level: {"database": "name", "schema": "sql"}
- Table level: {"database": "name", "table": "name", "schema": "sql"}

Markdown format provides formatted output with headings and SQL code blocks.

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 12:01:23 -08:00
Simon Willison
1df4028d78 add_memory_database(memory_name, name=None, route=None) 2025-11-05 15:18:17 -08:00
Simon Willison
257e1c1b1b Release 1.0a21 1.0a21
Refs #2429, #2511, #2578, #2583
2025-11-05 13:51:58 -08:00
Simon Willison
d814e81b32
datasette.client.get(..., skip_permission_checks=True)
Closes #2580
2025-11-05 13:38:01 -08:00
Simon Willison
ec99bb46f8 stable-docs YAML workflow, refs #2582 2025-11-05 10:51:46 -08:00
Simon Willison
3c2254463b
Release notes for 0.65.2
Adding those to main. Refs #2579
2025-11-05 10:25:37 -08:00
Simon Willison
f12f6cc2ab
Get publish cloudrun working with latest Cloud Run (#2581)
Refs:
- #2511

Filter out bad services, refs:
- https://github.com/simonw/datasette/pull/2581#issuecomment-3492243400
2025-11-05 09:28:41 -08:00
Simon Willison
12016342e7 Fix test_metadata_yaml I broke in #2578 2025-11-04 18:40:58 -08:00
Simon Willison
b4385a3ff7 Made test_serve_with_get_headers a bit more forgiving 2025-11-04 18:39:25 -08:00
Simon Willison
ce464da34b datasette --get --headers option, closes #2578 2025-11-04 18:12:15 -08:00
Simon Willison
9f74dc22a8 Run cog with --extra test
Previously it kept on adding stuff to cli-reference.rst
that came from other plugins installed for my global environment
2025-11-04 18:11:24 -08:00
Simon Willison
8b371495dc Move open redirect fix to asgi_send_redirect, refs #2429
See https://github.com/simonw/datasette/pull/2500#issuecomment-3488632278
2025-11-04 17:08:06 -08:00
James Jefferies
f257ca6edb
Fix for open redirect - identified in Issue 2429 (#2500)
* Issue 2429 indicates the possiblity of an open redirect

The 404 processing ends up redirecting a request with multiple path
slashes to that site, i.e.

https://my-site//shedcode.co.uk will redirect to https://shedcode.co.uk

This commit uses a regular expression to remove the multiple leading
slashes before redirecting.
2025-11-04 17:04:12 -08:00
Simon Willison
295e4a2e87 Pin to httpx<1.0
Refs https://github.com/encode/httpx/issues/3635
Closes #2576
2025-11-03 15:05:17 -08:00