Commit graph

2,905 commits

Author SHA1 Message Date
Simon Willison
5873578d49 Release 1.0a24 1.0a24
Refs #2050, #2346, #2608, #2609, #2610, #2611, #2613, #2619, #2624, #2627, #2628, #2629, #2630, #2632
2026-01-29 09:00:22 -08:00
Daniel Olasubomi Sobowale
b771e930bc
Fix filter-input and search-input zoom on iOS Safari
Closes #2346
2026-01-28 18:41:58 -08:00
Simon Willison
40a37307de
Add request.form() for multipart form data and file uploads
* Add request.form() for multipart form data and file uploads

New Request.form() method that handles both application/x-www-form-urlencoded
and multipart/form-data content types with streaming parsing.

Features:
- Streaming multipart parser that doesn't buffer entire body in memory
- Files spill to disk above 1MB threshold via SpooledTemporaryFile
- files=False (default) discards file content, files=True stores them
- Security limits: max_request_size, max_file_size, max_fields, max_files
- FormData container with dict-like access and getlist() for multiple values
- UploadedFile class with async read(), seek(), filename, content_type, size
- Support for RFC 5987 filename* encoding for international filenames

Uses multipart-form-data-conformance test suite for validation.

* Update views to use request.form() and document new API

- Migrate PermissionsDebugView, MessagesDebugView, and CreateTokenView
  from post_vars() to form()
- Add documentation for request.form(), FormData, and UploadedFile classes

Centralize multipart defaults and expose stricter limits via Request.form().

Enforce header, part, file, and disk space limits even when files are discarded; detect truncated bodies and client disconnects; and move blocking work off the event loop.

Add FormData close/aclose context managers, update internals docs, and expand multipart tests (including len semantics and stricter conformance expectations).
2026-01-28 18:41:03 -08:00
Simon Willison
ffadb5f74c
Workaround for intermittent test failure on SQLite 3.25.3
Closes:
- #2632
2026-01-28 18:34:00 -08:00
Simon Willison
3f8f97e92a Close more connections in test suite
To try and avoid too many open files on macOS
2026-01-28 09:55:25 -08:00
Simon Willison
2f7b120177 Minor speedup for remove_infinites, refs #2629 2026-01-24 22:07:54 -08:00
Simon Willison
7988a179fe Throttle schema refreshes to at most once per second, refs #2629 2026-01-23 21:03:16 -08:00
Simon Willison
7915c46ddd
Fix flaky test_database_page test with deterministic ordering (#2628)
* Fix flaky test_database_page test with deterministic ordering

- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
  500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst

* Update test_row_foreign_key_tables for new deterministic FK ordering

The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.

* Update test_table_names for new alphabetical ordering

The table_names() method now returns tables sorted alphabetically.

* Fix for test that fails prior to SQLite 3.37

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 20:57:25 -08:00
Simon Willison
66d2a033f8 Switch to ruff and fix all lint errors, refs #2630 2026-01-23 20:43:16 -08:00
Simon Willison
b0436faa5e
Fix test isolation bug in test_startup_error_from_plugin_is_click_exception (#2627)
* Fix test isolation bug in test_startup_error_from_plugin_is_click_exception

The test creates a plugin that raises StartupError("boom") and registers it
in the global plugin manager (pm). Without cleanup, this plugin leaks to
subsequent tests, causing test_setting_boolean_validation_false_values to
fail with "Error: boom" instead of "Forbidden".

Add try/finally block to ensure the plugin is unregistered after the test
completes, following the established cleanup pattern used elsewhere in
the test suite.

* Fix blacken-docs formatting in plugin_hooks.rst

Apply blacken-docs formatting to code example that exceeded
the 60 character line limit.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-22 07:03:05 -08:00
Simon Willison
b52655e856 Ignore *.db in gitignore 2026-01-06 07:59:07 -08:00
Simon Willison
757ce92baf datasette.utils.StartupError() now becomes a click exception, closes #2624 2026-01-06 07:58:48 -08:00
Simon Willison
6fede23a2e Only return render_coll columns that differ from default, refs #2619 2025-12-21 20:18:26 -08:00
Simon Willison
eae94dc2c3 Initial render_cell and foreign_key_tables extras for row
Closes #2619, refs #2050
2025-12-21 20:03:10 -08:00
Simon Willison
97496d5a67 ?_extra=render_cells for tables, refs #2619 2025-12-21 19:52:57 -08:00
Simon Willison
232a404743 Switch searchable_fts test table to FTS5, closes #2613 2025-12-12 22:18:35 -08:00
Simon Willison
3b4c7e1abe {"ok": true} on row API, to be consistent with table 2025-12-12 21:43:00 -08:00
Simon Willison
4cbdfcc07d
dependency-groups and uv (#2611)
* dependency-groups and uv, closes #2610
* New .readthedocs config for --group dev
2025-12-11 17:32:58 -08:00
Simon Willison
1d4448fc56
Use subtests in tests/test_docs.py (#2609)
Closes #2608
2025-12-04 21:36:39 -08:00
Simon Willison
2ca00b6c75 Release 1.0a23 1.0a23
Refs #2605, #2599
2025-12-02 19:20:43 -08:00
Simon Willison
03ab359208 tool.uv.package = true 2025-12-02 19:19:48 -08:00
Simon Willison
3eca3ad6d4 Better recipe for 'just docs' 2025-12-02 19:16:39 -08:00
Simon Willison
0a924524be
Split default_permissions.py into a package (#2603)
* Split default_permissions.py into a package, refs #2602

* Remove unused is_resource_allowed() method, improve test coverage

- Remove dead code: is_resource_allowed() method was never called
- Change isinstance check to assertion with error message
- Add test cases for table-level restrictions in restrictions_allow_action()
- Coverage for restrictions.py improved from 79% to 99%

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Additional permission test for gap spotted by coverage
2025-12-02 19:11:31 -08:00
Simon Willison
170b3ff61c Better fix for stale catalog_databases, closes #2606
Refs 2605
2025-12-02 19:00:13 -08:00
Simon Willison
c6c2a238c3 Fix for stale internal database bug, closes #2605 2025-12-02 16:22:42 -08:00
Simon Willison
68f1179bac Fix for text None shown on /-/actions, closes #2599 2025-11-26 17:12:52 -08:00
Simon Willison
2125115cd9 Release 1.0a22 1.0a22
Refs #2592, #2594, #2595, #2596
2025-11-13 10:41:02 -08:00
Simon Willison
93b455239a Release notes for 1.0a22, closes #2596 2025-11-13 10:40:24 -08:00
Simon Willison
4b4add4d31 datasette.pm property, closes #2595 2025-11-13 10:31:03 -08:00
Simon Willison
5125bef573 datasette.in_client() method, closes #2594 2025-11-13 10:00:04 -08:00
Simon Willison
23a640d38b
datasette serve --default-deny option (#2593)
Closes #2592
2025-11-12 16:14:21 -08:00
dependabot[bot]
32a425868c
Bump black from 25.9.0 to 25.11.0 in the python-packages group (#2590)
Bumps the python-packages group with 1 update: [black](https://github.com/psf/black).


Updates `black` from 25.9.0 to 25.11.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.9.0...25.11.0)

---
updated-dependencies:
- dependency-name: black
  dependency-version: 25.11.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-12 06:07:16 -08:00
Simon Willison
291f71ec6b
Remove out-dated plugin_hook_permission_allowed references 2025-11-11 21:59:26 -08:00
Simon Willison
354d7a2873
Bump a few versions, deploy on push to main
Refs:
- #2511
2025-11-09 15:42:11 -08:00
Simon Willison
a508fc4a8e Remove permission_allowed hook docs, closes #2588
Refs #2528
2025-11-07 16:50:00 -08:00
Simon Willison
8bc9b1ee03
/-/schema and /db/-/schema and /db/table/-/schema pages (plus .json/.md)
* Add schema endpoints for databases, instances, and tables

Closes: #2586

This commit adds new endpoints to view database schemas in multiple formats:

- /-/schema - View schemas for all databases (HTML, JSON, MD)
- /database/-/schema - View schema for a specific database (HTML, JSON, MD)
- /database/table/-/schema - View schema for a specific table (JSON, MD)

Features:
- Supports HTML, JSON, and Markdown output formats
- Respects view-database and view-table permissions
- Uses group_concat(sql, ';' || CHAR(10)) from sqlite_master to retrieve schemas
- Includes comprehensive tests covering all formats and permission checks

The JSON endpoints return:
- Instance level: {"schemas": [{"database": "name", "schema": "sql"}, ...]}
- Database level: {"database": "name", "schema": "sql"}
- Table level: {"database": "name", "table": "name", "schema": "sql"}

Markdown format provides formatted output with headings and SQL code blocks.

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 12:01:23 -08:00
Simon Willison
1df4028d78 add_memory_database(memory_name, name=None, route=None) 2025-11-05 15:18:17 -08:00
Simon Willison
257e1c1b1b Release 1.0a21 1.0a21
Refs #2429, #2511, #2578, #2583
2025-11-05 13:51:58 -08:00
Simon Willison
d814e81b32
datasette.client.get(..., skip_permission_checks=True)
Closes #2580
2025-11-05 13:38:01 -08:00
Simon Willison
ec99bb46f8 stable-docs YAML workflow, refs #2582 2025-11-05 10:51:46 -08:00
Simon Willison
3c2254463b
Release notes for 0.65.2
Adding those to main. Refs #2579
2025-11-05 10:25:37 -08:00
Simon Willison
f12f6cc2ab
Get publish cloudrun working with latest Cloud Run (#2581)
Refs:
- #2511

Filter out bad services, refs:
- https://github.com/simonw/datasette/pull/2581#issuecomment-3492243400
2025-11-05 09:28:41 -08:00
Simon Willison
12016342e7 Fix test_metadata_yaml I broke in #2578 2025-11-04 18:40:58 -08:00
Simon Willison
b4385a3ff7 Made test_serve_with_get_headers a bit more forgiving 2025-11-04 18:39:25 -08:00
Simon Willison
ce464da34b datasette --get --headers option, closes #2578 2025-11-04 18:12:15 -08:00
Simon Willison
9f74dc22a8 Run cog with --extra test
Previously it kept on adding stuff to cli-reference.rst
that came from other plugins installed for my global environment
2025-11-04 18:11:24 -08:00
Simon Willison
8b371495dc Move open redirect fix to asgi_send_redirect, refs #2429
See https://github.com/simonw/datasette/pull/2500#issuecomment-3488632278
2025-11-04 17:08:06 -08:00
James Jefferies
f257ca6edb
Fix for open redirect - identified in Issue 2429 (#2500)
* Issue 2429 indicates the possiblity of an open redirect

The 404 processing ends up redirecting a request with multiple path
slashes to that site, i.e.

https://my-site//shedcode.co.uk will redirect to https://shedcode.co.uk

This commit uses a regular expression to remove the multiple leading
slashes before redirecting.
2025-11-04 17:04:12 -08:00
Simon Willison
295e4a2e87 Pin to httpx<1.0
Refs https://github.com/encode/httpx/issues/3635
Closes #2576
2025-11-03 15:05:17 -08:00
Simon Willison
95a1fef280 Release 1.0a20 1.0a20
Refs #2488, #2495, #2503, #2505, #2509, #2510, #2513, #2515, #2517, #2519, #2520, #2521,
#2524, #2525, #2526, #2528, #2530, #2531, #2534, #2537, #2543, #2544, #2550, #2551,
#2555, #2558, #2561, #2562, #2564, #2565, #2567, #2569, #2570, #2571, #2574
2025-11-03 14:47:24 -08:00