Commit graph

2,875 commits

Author SHA1 Message Date
Simon Willison
23a640d38b
datasette serve --default-deny option (#2593)
Closes #2592
2025-11-12 16:14:21 -08:00
dependabot[bot]
32a425868c
Bump black from 25.9.0 to 25.11.0 in the python-packages group (#2590)
Bumps the python-packages group with 1 update: [black](https://github.com/psf/black).


Updates `black` from 25.9.0 to 25.11.0
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/25.9.0...25.11.0)

---
updated-dependencies:
- dependency-name: black
  dependency-version: 25.11.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-packages
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-12 06:07:16 -08:00
Simon Willison
291f71ec6b
Remove out-dated plugin_hook_permission_allowed references 2025-11-11 21:59:26 -08:00
Simon Willison
354d7a2873
Bump a few versions, deploy on push to main
Refs:
- #2511
2025-11-09 15:42:11 -08:00
Simon Willison
a508fc4a8e Remove permission_allowed hook docs, closes #2588
Refs #2528
2025-11-07 16:50:00 -08:00
Simon Willison
8bc9b1ee03
/-/schema and /db/-/schema and /db/table/-/schema pages (plus .json/.md)
* Add schema endpoints for databases, instances, and tables

Closes: #2586

This commit adds new endpoints to view database schemas in multiple formats:

- /-/schema - View schemas for all databases (HTML, JSON, MD)
- /database/-/schema - View schema for a specific database (HTML, JSON, MD)
- /database/table/-/schema - View schema for a specific table (JSON, MD)

Features:
- Supports HTML, JSON, and Markdown output formats
- Respects view-database and view-table permissions
- Uses group_concat(sql, ';' || CHAR(10)) from sqlite_master to retrieve schemas
- Includes comprehensive tests covering all formats and permission checks

The JSON endpoints return:
- Instance level: {"schemas": [{"database": "name", "schema": "sql"}, ...]}
- Database level: {"database": "name", "schema": "sql"}
- Table level: {"database": "name", "table": "name", "schema": "sql"}

Markdown format provides formatted output with headings and SQL code blocks.

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-07 12:01:23 -08:00
Simon Willison
1df4028d78 add_memory_database(memory_name, name=None, route=None) 2025-11-05 15:18:17 -08:00
Simon Willison
257e1c1b1b Release 1.0a21 1.0a21
Refs #2429, #2511, #2578, #2583
2025-11-05 13:51:58 -08:00
Simon Willison
d814e81b32
datasette.client.get(..., skip_permission_checks=True)
Closes #2580
2025-11-05 13:38:01 -08:00
Simon Willison
ec99bb46f8 stable-docs YAML workflow, refs #2582 2025-11-05 10:51:46 -08:00
Simon Willison
3c2254463b
Release notes for 0.65.2
Adding those to main. Refs #2579
2025-11-05 10:25:37 -08:00
Simon Willison
f12f6cc2ab
Get publish cloudrun working with latest Cloud Run (#2581)
Refs:
- #2511

Filter out bad services, refs:
- https://github.com/simonw/datasette/pull/2581#issuecomment-3492243400
2025-11-05 09:28:41 -08:00
Simon Willison
12016342e7 Fix test_metadata_yaml I broke in #2578 2025-11-04 18:40:58 -08:00
Simon Willison
b4385a3ff7 Made test_serve_with_get_headers a bit more forgiving 2025-11-04 18:39:25 -08:00
Simon Willison
ce464da34b datasette --get --headers option, closes #2578 2025-11-04 18:12:15 -08:00
Simon Willison
9f74dc22a8 Run cog with --extra test
Previously it kept on adding stuff to cli-reference.rst
that came from other plugins installed for my global environment
2025-11-04 18:11:24 -08:00
Simon Willison
8b371495dc Move open redirect fix to asgi_send_redirect, refs #2429
See https://github.com/simonw/datasette/pull/2500#issuecomment-3488632278
2025-11-04 17:08:06 -08:00
James Jefferies
f257ca6edb
Fix for open redirect - identified in Issue 2429 (#2500)
* Issue 2429 indicates the possiblity of an open redirect

The 404 processing ends up redirecting a request with multiple path
slashes to that site, i.e.

https://my-site//shedcode.co.uk will redirect to https://shedcode.co.uk

This commit uses a regular expression to remove the multiple leading
slashes before redirecting.
2025-11-04 17:04:12 -08:00
Simon Willison
295e4a2e87 Pin to httpx<1.0
Refs https://github.com/encode/httpx/issues/3635
Closes #2576
2025-11-03 15:05:17 -08:00
Simon Willison
95a1fef280 Release 1.0a20 1.0a20
Refs #2488, #2495, #2503, #2505, #2509, #2510, #2513, #2515, #2517, #2519, #2520, #2521,
#2524, #2525, #2526, #2528, #2530, #2531, #2534, #2537, #2543, #2544, #2550, #2551,
#2555, #2558, #2561, #2562, #2564, #2565, #2567, #2569, #2570, #2571, #2574
2025-11-03 14:47:24 -08:00
Simon Willison
dc3f9fe9e4 Python 3.10, not 3.8 2025-11-03 14:42:59 -08:00
Simon Willison
5d4dfcec6b Fix for link from changelog not working
Annoyingly we now get a warning in the docs build about a duplicate label,
but it seems harmless enough.
2025-11-03 14:38:57 -08:00
Simon Willison
b3b8c5831b Fixed some broken reference links on upgrade guide 2025-11-03 14:34:29 -08:00
Simon Willison
b212895b97 Updated release notes for 1.0a20
Refs #2550
2025-11-03 14:27:41 -08:00
Simon Willison
18fd373a8f
New PermissionSQL.restriction_sql mechanism for actor restrictions
Implement INTERSECT-based actor restrictions to prevent permission bypass

Actor restrictions are now implemented as SQL filters using INTERSECT rather
than as deny/allow permission rules. This ensures restrictions act as hard
limits that cannot be overridden by other permission plugins or config blocks.

Previously, actor restrictions (_r in actor dict) were implemented by 
generating permission rules with deny/allow logic. This approach had a 
critical flaw: database-level config allow blocks could bypass table-level 
restrictions, granting access to tables not in the actor's allowlist.

The new approach separates concerns:

- Permission rules determine what's allowed based on config and plugins
- Restriction filters limit the result set to only allowlisted resources
- Restrictions use INTERSECT to ensure all restriction criteria are met
- Database-level restrictions (parent, NULL) properly match all child tables

Implementation details:

- Added restriction_sql field to PermissionSQL dataclass
- Made PermissionSQL.sql optional to support restriction-only plugins
- Updated actor_restrictions_sql() to return restriction filters instead of rules
- Modified SQL builders to apply restrictions via INTERSECT and EXISTS clauses

Closes #2572
2025-11-03 14:17:51 -08:00
Simon Willison
c76c3e6e6f facet_suggest_time_limit_ms 200ms in tests, closes #2574 2025-11-03 11:52:12 -08:00
Simon Willison
fa978ec100 More upgrade tips, written by Claude Code
Refs #2549

From the datasette-atom upgrade, https://gistpreview.github.io/?d5047e04bbd9c20c59437916e21754ae
2025-11-02 12:02:45 -08:00
Simon Willison
2459285052 Additional upgrade notes by Codex CLI
Refs https://github.com/simonw/datasette/issues/2549#issuecomment-3477398336

Refs #2564
2025-11-01 20:32:42 -07:00
Simon Willison
506ce5b0ac Remove docs for obsolete register_permissions() hook, refs #2528
Also removed docs for datasette.get_permission() method which no longer exists.
2025-11-01 20:23:37 -07:00
Simon Willison
063bf7a96f Action() is kw_only, abbr= is optional, closes #2571 2025-11-01 20:20:17 -07:00
Simon Willison
7e09e1bf1b Removed obsolete actor ID v.s. actor dict code, refs #2570 2025-11-01 19:30:56 -07:00
Simon Willison
e37aa37edc Further refactor to collapse some utility functions
Refs #2570
2025-11-01 19:28:31 -07:00
Simon Willison
b8cee8768e Completed upgrade guide, closes #2564 2025-11-01 18:57:56 -07:00
Simon Willison
5c16c6687d Split permissions_resources_sql() into 5 for readability
Also remove an obsolete test that caused trouble with the new split plugin hook.

Closes #2570
2025-11-01 18:38:47 -07:00
Simon Willison
a528555e84
Additional actor restriction should not grant access to additional actions (#2569)
Closes #2568
2025-11-01 18:38:29 -07:00
Simon Willison
2b962beaeb Fix permissions_execute_sql warnings in documentation 2025-11-01 11:52:23 -07:00
Simon Willison
5705ce0d95
Move takes_child/takes_parent information from Action to Resource (#2567)
Simplified Action by moving takes_child/takes_parent logic to Resource

- Removed InstanceResource - global actions are now simply those with resource_class=None
- Resource.parent_class - Replaced parent_name: str with parent_class: type[Resource] | None for direct class references
- Simplified Action dataclass - No more redundant fields, everything is derived from the Resource class structure
- Validation - The __init_subclass__ method now checks parent_class.parent_class to enforce the 2-level hierarchy

Closes #2563
2025-11-01 11:35:08 -07:00
Simon Willison
1f8995e776 upgrade-1.0a20.md, refs #2564
And another Markdown conversion, refs #2565
2025-10-31 19:13:41 -07:00
Simon Willison
47e4060469 Enable MyST Markdown docs, port events.rst, refs #2565 2025-10-31 16:38:04 -07:00
Simon Willison
48982a0ff5 Mark 1.0a20 unreleased
Refs #2550
2025-10-31 16:12:54 -07:00
Simon Willison
223dcc7c0e Remove unused link 2025-10-31 16:11:53 -07:00
Simon Willison
3184bfae54 Release notes for 1.0a20, refs #2550 2025-10-31 15:37:30 -07:00
Simon Willison
e5f392ae7a datasette.allowed_resources_sql() returns namedtuple 2025-10-31 15:07:37 -07:00
Simon Willison
400fa08e4c
Add keyset pagination to allowed_resources() (#2562)
* Add keyset pagination to allowed_resources()

This replaces the unbounded list return with PaginatedResources,
which supports efficient keyset pagination for handling thousands
of resources.

Closes #2560

Changes:
- allowed_resources() now returns PaginatedResources instead of list
- Added limit (1-1000, default 100) and next (keyset token) parameters
- Added include_reasons parameter (replaces allowed_resources_with_reasons)
- Removed allowed_resources_with_reasons() method entirely
- PaginatedResources.all() async generator for automatic pagination
- Uses tilde-encoding for tokens (matching table pagination)
- Updated all callers to use .resources accessor
- Updated documentation with new API and examples

The PaginatedResources object has:
- resources: List of Resource objects for current page
- next: Token for next page (None if no more results)
- all(): Async generator that yields all resources across pages

Example usage:
    page = await ds.allowed_resources("view-table", actor, limit=100)
    for table in page.resources:
        print(table.child)

    # Iterate all pages automatically
    async for table in page.all():
        print(table.child)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 14:50:46 -07:00
Simon Willison
b7ef968c6f Fixed some rST labels I broke 2025-10-31 09:15:39 -07:00
Simon Willison
ba654b5576 Forbid same DB passed twice or via config_dir, closes #2561 2025-10-30 21:40:09 -07:00
Simon Willison
e4be95b16c
Update permissions documentation for new action system (#2551) 2025-10-30 17:59:54 -07:00
Simon Willison
87aa798148 Permission tabs include allow debug page
Closes #2559
2025-10-30 17:54:07 -07:00
Simon Willison
6a71bde37f
Permissions SQL API improvements (#2558)
* Neater design for PermissionSQL class, refs #2556
  - source is now automatically set to the source plugin
  - params is optional
* PermissionSQL.allow() and PermissionSQL.deny() shortcuts

Closes #2556

* Filter out temp database from attached_databases()

Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
2025-10-30 15:48:46 -07:00
Simon Willison
5247856bd4 Filter out temp database from attached_databases()
Refs https://github.com/simonw/datasette/issues/2557#issuecomment-3470510837
2025-10-30 15:48:10 -07:00