Commit graph

2,953 commits

Author SHA1 Message Date
Simon Willison
cb293572c4 UI for setting custom column types, refs #2671 2026-03-18 14:08:44 -07:00
Simon Willison
cb5cc0cc22 Fixed some broken docs/ references, refs #2671 2026-03-18 13:02:31 -07:00
Simon Willison
611b8ad463 blacken-docs 2026-03-18 12:33:09 -07:00
Simon Willison
2b06da29a1 Rename set-column-types action to et-column-type
Refs https://github.com/simonw/datasette/pull/2674#issuecomment-4085015792
2026-03-18 12:33:09 -07:00
Simon Willison
d440c20984 /db/table/-/set-column-type JSON API, refs #2671 2026-03-18 12:33:09 -07:00
Simon Willison
fa1d8f0fa5 set-column-types permission, refs #2671 2026-03-18 12:33:09 -07:00
Simon Willison
feaba9b18b
Optionally limit ColumnType subclasses to specific SQLite types (#2673)
* ColumnTypes now have optional SQLite column types

Refs #2672
2026-03-18 11:37:09 -07:00
Simon Willison
0f81553b3f No hide this column on last remaining column 2026-03-18 10:47:05 -07:00
Simon Willison
68966880c2
Fix mobile column actions not showing items for SQL views (#2670)
* Fix mobile column actions not showing items for SQL views

The previous fix to exclude the Link column from mobile column actions
(d02072b) used .dropdown-menu-icon presence as a proxy, but dropdown
icons are only added to sortable columns (those with <a> tags). This
caused all non-sortable columns to be excluded too.

Instead, explicitly mark the Link column with a data-is-link-column
attribute and filter by that in mobileColumnHeaders, so non-sortable
columns on views and tables still appear in the mobile column actions.

* Prettier formatting for mobile-column-actions.js

https://claude.ai/code/session_01CG545gLcZxet7dS5nMzfCd
2026-03-18 09:46:36 -07:00
Simon Willison
d02072bc9d Do not show mobile column actions for Link column
Refs #2669
2026-03-18 09:08:00 -07:00
Simon Willison
e800312b54 allowed_resources(view-query, actor) fix
Previously we could not filter for canned queries that a
specific actor could view.
2026-03-18 09:05:23 -07:00
Simon Willison
fd016f7986
Column actions panel on mobile (#2669)
On mobile widths the column actions were no longer available.

This adds a new modal to help with that.

https://gisthost.github.io/?ec60eb27e22cf5d96642eec1715586b6
2026-03-18 09:04:28 -07:00
Simon Willison
63d73a806f
Move table configuration docs from metadata.rst to configuration.rst (#2668)
https://claude.ai/code/session_01UqboRB5Wt52BKPhxexUBEn
2026-03-17 08:47:04 -07:00
Simon Willison
bc7a19b39d
Column types system
Closes #2664
2026-03-16 22:29:51 -07:00
Claude
b7578a4884
Remove pointless test_column_type_with_config test
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:24:09 +00:00
Claude
c73a1c907a
Use 'subclass' instead of 'class' in ColumnType docs
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:22:53 +00:00
Claude
da0ea4382c
Update cog-generated plugin list to include default_column_types
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:18:42 +00:00
Claude
dd9b83301c
Refactor ColumnType: register classes, return instances with config
- register_column_types() now returns classes instead of instances
- ColumnType.__init__ takes optional config=, baking it into the instance
- get_column_type() returns a ColumnType instance (or None) instead of a
  (name, config) tuple
- get_column_types() returns {col: ColumnType instance} instead of tuples
- Remove get_column_type_class() - no longer needed
- render_cell/validate/transform_value methods no longer take config arg;
  use self.config instead
- render_cell hook takes column_type (ColumnType or None) instead of
  column_type + column_type_config

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:18:14 +00:00
Claude
8af98c24c2
Move name and description to class attributes on ColumnType
Instead of passing name= and description= as constructor arguments,
define them as class attributes on each subclass. This better reflects
that they are intrinsic to the type, not configurable per-instance.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 05:00:57 +00:00
Claude
72c8c71518
Move column_type defaults into dict literal
Set column_type and column_type_config to None in the initial
col_dict instead of using an else branch.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:54:36 +00:00
Claude
9fe10cd1aa
Apply black and blacken-docs formatting
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:50:58 +00:00
Claude
77bbfb5f7e
Document column type internal methods in internals.rst
Add documentation for get_column_type, get_column_types,
set_column_type, remove_column_type, and get_column_type_class
methods on the Datasette instance.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:08:58 +00:00
Claude
32e4a31913
Document register_column_types hook and updated render_cell signature
- Add register_column_types(datasette) hook documentation with example
- Update render_cell signature to include column_type and
  column_type_config parameters
- Fixes test_plugin_hooks_are_documented

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 04:03:52 +00:00
Claude
ad6a020e6d
Add NOT NULL constraints to column_types primary key columns
SQLite allows NULLs in primary key columns by default, so mark
database_name, resource_name, and column_name as NOT NULL explicitly.

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 03:58:18 +00:00
Claude
5db4f6953d
Fix linting: remove unused import, apply black formatting
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 03:56:20 +00:00
Claude
de4269629b
Remove duplicate import logging line
https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 03:55:51 +00:00
Claude
e8472bc0cd
Add missing tests and transform_value integration
- Add transform_value integration in table JSON endpoint rows
- Add tests for: duplicate type name error, row endpoint rendering,
  transform_value in JSON output, column type priority over plugins,
  row detail HTML rendering, table HTML rendering, upsert validation,
  unknown type warning logging, config overwrite on restart, and
  no-config edge case
- Total: 34 column type tests, all passing

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 02:48:55 +00:00
Claude
73225ccad0
Add column types system for semantic column annotations
Implements the column types feature that lets Datasette and plugins annotate
columns with semantic types beyond SQLite storage types (e.g. markdown, email,
url, json, file, point). This enables type-appropriate rendering, validation,
form widgets, and API behavior.

Key changes:
- New `column_types` internal DB table for storing assignments
- `ColumnType` dataclass in datasette/column_types.py with render_cell,
  validate, and transform_value methods
- `register_column_types` plugin hook for registering types
- Built-in url, email, and json column types
- Datasette API methods: get/set/remove_column_type(s),
  get_column_type_class
- Config loading from datasette.json `column_types` table config key
- `column_types` extra on the table JSON endpoint
- Column type info in display_columns extra
- Column type render_cell gets priority in rendering pipeline
- column_type/column_type_config args added to render_cell hookspec
- Write-path validation on insert and update

https://claude.ai/code/session_01SvPEPqHgURTWESRp28pTC3
2026-03-17 02:40:37 +00:00
Simon Willison
7f93353549
Fix startup hook to fire after metadata and schema tables are populated (#2666)
* Fix startup hook to fire after metadata and schema tables are populated

Previously, the startup() plugin hook fired before internal database
tables were populated from metadata.yaml and before catalog schema
tables were filled. This meant plugins couldn't read or modify metadata
during startup. Now invoke_startup() calls refresh_schemas() before
firing startup hooks, ensuring metadata and catalog tables are available.

* Fix startup hook to fire after metadata and schema tables are populated

Previously, the startup() plugin hook fired before internal database
tables were populated from metadata.yaml and before catalog schema
tables were filled. This meant plugins couldn't read or modify metadata
during startup. Now invoke_startup() calls _refresh_schemas() before
firing startup hooks, ensuring metadata and catalog tables are available.

Updated test_tracer to reflect that internal DB creation SQL now runs
during startup rather than during the first traced request.

* Move check_databases before invoke_startup in CLI serve

Since invoke_startup now calls _refresh_schemas() which queries each
database, the spatialite connection check must run first to provide
the friendly error message instead of a raw OperationalError.

https://claude.ai/code/session_01KL4t5FZYb32rZY7xaqrrZU
2026-03-16 17:56:40 -07:00
Simon Willison
5805a126db Fix for column select on Mobile Safari
Refs https://github.com/simonw/datasette/issues/2661#issuecomment-4027902138
2026-03-09 18:05:01 -07:00
Simon Willison
e2c1e81ec9
UI for selecting and re-ordering columns on the table page (#2662)
New Web Component on table/view page with a dialog for selecting and re-ordering columns.

Closes #2661
Refs #1298
2026-03-09 17:45:24 -07:00
Simon Willison
97201f067c Row pages link to foreign keys from table display, closes #1592
https://gisthost.github.io/?40813f5b3e4d83c0efe1c09135f84290/index.html

Also now shows primary key column first and in bold on that page.
2026-03-06 20:16:50 -08:00
Simon Willison
1263380ea6
Better heading for write_wrapper() 2026-02-25 20:50:46 -08:00
Simon Willison
8f0d60236f Bump version for 1.0a25 1.0a25 2026-02-25 17:01:03 -08:00
Simon Willison
e4ff5e27d3 Fix RST heading underlin 2026-02-25 16:54:51 -08:00
Simon Willison
1246c6576b Release 1.0a25
Refs #2636, #2641, #2646, #2647, #2650
2026-02-25 16:49:14 -08:00
Daniel Bates
2bc1dd2275
Fix --reload interpreting 'serve' command as a file argument (#2646)
When hupper spawns the worker process, it calls the function specified by
worker_path directly. Using "datasette.cli.serve" causes Click to parse
sys.argv without going through the CLI group, so the literal word "serve"
from the original command gets treated as a positional file argument.

Change the worker path to "datasette.cli.cli" so the worker process goes
through the Click group dispatcher, which properly recognizes "serve" as
a subcommand and strips it from the argument list.

Closes #2123

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Simon Willison <swillison@gmail.com>
2026-02-25 16:46:29 -08:00
Simon Willison
24d801b7f7
Respect metadata-defined facet ordering in sorted_facet_results (#2648)
* Preserve metadata-defined facet ordering on table pages

When facets are explicitly defined in table metadata/config, they now
appear in the order specified in the configuration rather than being
sorted by result count. Request-added facets still appear after
metadata-defined facets, sorted by count as before.

* Document metadata-defined facet ordering behavior

* Apply black formatting

https://claude.ai/code/session_01PbSHtjsUpNk3Fx7xjvVqDb
2026-02-25 16:33:27 -08:00
Simon Willison
c96dc5ce26
register_token_handler() plugin hook for custom API token backends (#2650)
Closes #2649

* Add register_token_handler plugin hook for pluggable token backends

Adds a new register_token_handler hook that allows plugins to provide
custom token creation and verification backends. This enables plugins
like datasette-oauth to issue tokens without depending on specific
backend plugins like datasette-auth-tokens.

Key changes:
- New datasette/tokens.py with TokenHandler base class and SignedTokenHandler
  (the default signed-token implementation moved here)
- New register_token_handler hookspec in hookspecs.py
- Datasette.create_token() is now async and delegates to token handlers
- New Datasette.verify_token() method tries all handlers in sequence
- handler= parameter on create_token() to select a specific backend
- TokenHandler exported from datasette package for plugin use
- Fixed actor_from_request loop to await all coroutines (avoids warnings)

* Add documentation and hook test for register_token_handler

Fixes CI failures: the new hook needs a section in docs/plugin_hooks.rst
(checked by test_plugin_hooks_are_documented) and a test_hook_* function
in test_plugins.py (checked by test_plugin_hooks_have_tests).

* Register tokens module as separate default plugin

Instead of re-exporting hookimpls from default_permissions/__init__.py,
register datasette.default_permissions.tokens as its own DEFAULT_PLUGINS
entry. Cleaner and avoids confusing import-for-side-effect patterns.

* Replace restrict_x params with TokenRestrictions dataclass

Consolidates the three separate restrict_all, restrict_database, and
restrict_resource parameters into a single TokenRestrictions dataclass.
Cleaner API surface for both Datasette.create_token() and
TokenHandler.create_token().

Also clarifies docs re: default handler selection via pluggy ordering.

* Add builder methods to TokenRestrictions

Adds allow_all(), allow_database(), and allow_resource() methods that
return self for chaining. Callers no longer need to manipulate nested
dicts directly:

    restrictions = (TokenRestrictions()
        .allow_all("view-instance")
        .allow_database("mydb", "create-table")
        .allow_resource("mydb", "mytable", "insert-row"))

* docs: add 1.0a25 upgrade guide section for create_token() signature change

Ref: https://github.com/simonw/datasette/issues/2649#issuecomment-3962639393

* docs: note that create_token() is now async in upgrade guide

* docs: update internals, plugin_hooks, authentication for new token API

- internals.rst: new async create_token() signature with restrictions
  and handler params, add TokenRestrictions reference docs
- plugin_hooks.rst: show full create_token signature in TokenHandler
  example, note list returns and error cases
- authentication.rst: cross-reference TokenRestrictions from the
  restrictions section

* style: apply black formatting to token handler files

* docs: fix RST heading underline length in internals.rst

* tests: add restrictions round-trip and expiration tests for token handler

Covers allow_database/allow_resource builders, _r payload encoding,
and token_expires in verified actors. Coverage 76% -> 90%.

* tests: add test for signed tokens disabled

* fix: add TokenRestrictions TYPE_CHECKING import to fix ruff F821

* docs: regenerate plugins.rst with cog

* docs: reformat code blocks in plugin_hooks.rst with blacken-docs

* docs: add await .verify_token() to internals.rst

* tests: rewrite register_token_handler test to use real plugin handler

Adds a HardcodedTokenHandler to the test plugins dir that creates
tokens like dstok_hardcoded_token_1. The test now exercises creating
tokens via the default handler (which is the plugin's hardcoded one),
by explicitly naming the hardcoded handler, and by explicitly naming
the signed handler -- then verifies each token round-trips correctly.

* tests: clarify test_token_handler_via_http tests the default signed handler

* fix: use handler="signed" explicitly where signed tokens are expected

The HardcodedTokenHandler in my_plugin.py gets globally registered,
so create_token() without a handler name picks it up as the default.
Fix the create-token view, CLI, and tests to explicitly request the
signed handler where they depend on signed token behavior.

* fix: use handler="signed" in test_create_table_permissions

https://claude.ai/code/session_013cQFiDQjYRrRBH2biFfKuS
2026-02-25 16:32:45 -08:00
Simon Willison
6a2c27b15b blacken-docs 2026-02-20 11:28:39 -08:00
Simon Willison
2f0e64df68
black==26.1.0
I'm getting CI failures for Black, maybe this will help
2026-02-20 11:24:52 -08:00
Simon Willison
7a66456615
black --version 2026-02-20 11:19:19 -08:00
Simon Willison
1c6c6d2e68 Fix test_write_wrapper_set_authorizer: use permissive callback instead of None
conn.set_authorizer(None) does not clear the authorizer - SQLite treats
None as an invalid callback. The denied state persists on the shared
write connection, causing subsequent non-deny test cases to fail.

Fixes test added in 8a315f3d.
2026-02-17 13:30:46 -08:00
Simon Willison
5c3137d148 Black formatting 2026-02-17 13:30:24 -08:00
Claude
51e341b06a Fix test assertions broken by new fixture rows in 170f9de
The render_cell pks parameter commit added rows to compound_primary_key
(2->3 rows) and no_primary_key (201->202 rows) tables but did not
update existing tests that had hardcoded row count expectations.

https://claude.ai/code/session_01XfPSZfK57bzRRiEa7Kz5n1
2026-02-17 13:22:57 -08:00
Claude
170f9de774 Add pks parameter to render_cell() plugin hook
The render_cell() hook now receives a pks parameter containing the list
of primary key column names for the table being rendered. This avoids
plugins needing to make redundant async calls to look up primary keys.

For tables without an explicit primary key, pks is ["rowid"]. For custom
SQL queries and views, pks is an empty list [].

https://claude.ai/code/session_01HFYfevAziq4fSYTNRD9ZCh
2026-02-17 11:13:30 -08:00
Simon Willison
8a315f3d7d Added a test to exercise the write_wrapper example
This example in the docs is now dulicated in a test:

80b7f987ca/docs/plugin_hooks.rst (write-wrapper-datasette-database-request-transaction)

Refs #2637
2026-02-09 13:27:23 -08:00
Simon Willison
80b7f987ca
write_wrapper plugin hook for intercepting write operations (#2636)
* Implement write_wrapper plugin hook for intercepting database writes

Add a new `write_wrapper` plugin hook that lets plugins wrap write
operations with before/after logic using a generator-based context
manager pattern. The hook receives (datasette, database, request,
transaction) and returns a generator function that takes a conn,
yields once to let the write execute, and can run cleanup after.

The write result is sent back via `generator.send()` and exceptions
are thrown via `generator.throw()`, giving plugins full visibility.

Also adds `request=None` parameter to execute_write, execute_write_fn,
execute_write_script, and execute_write_many, and threads request
through all view-layer call sites (insert, upsert, update, delete,
drop, create table, canned queries).

* Add documentation for wrap_write hook, fix lint issues

Document the wrap_write plugin hook in plugin_hooks.rst with
parameter descriptions and two examples: a simple logging wrapper
and an advanced SQLite authorizer-based table protection pattern.

Also fix black formatting and remove unused variable flagged by ruff.

* Rename wrap_write hook to write_wrapper for consistency with asgi_wrapper
* Move write_wrapper docs to just below prepare_connection
* Refactor write_wrapper tests to use pytest.parametrize

Consolidate duplicate test cases: merge before/after tests for
execute_write_fn and execute_write into one parametrized test, and
merge three parameter-passing tests into one parametrized test.

Claude Code transcript: https://gisthost.github.io/?c4c12079434e69677e4aa8ac664b21b8/index.html
2026-02-09 13:20:33 -08:00
Simon Willison
5873578d49 Release 1.0a24 1.0a24
Refs #2050, #2346, #2608, #2609, #2610, #2611, #2613, #2619, #2624, #2627, #2628, #2629, #2630, #2632
2026-01-29 09:00:22 -08:00
Daniel Olasubomi Sobowale
b771e930bc
Fix filter-input and search-input zoom on iOS Safari
Closes #2346
2026-01-28 18:41:58 -08:00