Commit graph

3,260 commits

Author SHA1 Message Date
Simon Willison
f0645c6ddf
Create table, alter table - APIs and modal dialog UI
* Add create table UI

Adds a permission-gated database action that opens a create table modal on database pages, backed by the existing create-table JSON API.

The modal starts with an id integer primary key column plus a blank text column, supports SQLite type selection, and shows custom column type controls only when the actor can set column types.

Selected custom column types are applied after table creation with follow-up set-column-type API calls. Includes styling plus HTML and Playwright coverage for the action payload and create-table flow.

* Add alter table JSON API

- Add POST /<database>/<table>/-/alter with Pydantic validation and dry-run support.
- Support add, rename, alter, drop, primary-key and reorder operations, including allow-listed default expressions.
- Document the endpoint and cover schema changes, validation, permissions, events and dry runs.

Refs #2788

* Add alter table modal

- Register a built-in table action and expose alter-table metadata to table pages.
- Build the client-side modal for editing columns, defaults, ordering, primary keys, and custom column types.
- Add a review/apply confirmation flow with HTML and Playwright coverage.

Refs #2788

* Ran Prettier

* Isolate Unix domain socket test server paths

- Use a per-process socket path for the UDS test fixture.
- Clean up stale socket files before and after the fixture runs.
- Close the HTTP client and wait for the Datasette subprocess to exit.

* Precompute action permissions for table pages

- Extract reusable helpers for database and table action permission preloading.
- Precompute those permissions before building table-page HTML data.
- Document the default table actions plugin.

* Test against pyodide/v314.0.0

Now that we depend on pydantic we need a more recent
pyodide in order to load the emscripten build
of pydantic-core.

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733412763

* Split table create and alter views

- Move create-table and alter-table API views into table_create_alter.py.
- Keep create and alter schema-editing constants and helpers together.
- Rename the create table modal context helper.

* Add foreign keys to create table API

- Add fk_table and optional fk_column support to create-table columns.
- Validate create-table requests with Pydantic while preserving existing errors.
- Document the API and cover inferred primary-key and validation cases.
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733544452

* Add foreign keys to alter table API

- Add add_foreign_key, drop_foreign_key, and set_foreign_keys operations.
- Validate flat fk_table and fk_column arguments with Pydantic.
- Document the API and cover inferred primary-key and validation cases.

* /db/table/-/foreign-key-suggestions API

Improved version of the implementation datasette-edit-schema

* /<database>/-/foreign-key-targets API endpoint

Returns a list of tables with a single primary key, and for each one
the name of that primary key column and its SQLite type affinity.

This will be used by the create table UI to suggest foreign keys.

* Expose foreign key targets to create table UI

- Add foreignKeyTargetsPath to create table page data
- Filter hidden tables from database-level foreign key target results
- Update JSON API docs and tests for filtered targets

* Add foreign key controls to create table dialog

- Add create table advanced controls for foreign keys and first-column primary keys
- Share schema dialog row helpers between create and alter dialogs
- Move custom type into advanced options and add Add column icons

* More robust test_datasette_https_server.sh test

* Make custom type and foreign key mutually exclusive

In the create table dialog a column can now have either a custom display
type or a foreign key target, but not both - a foreign key column's type
is determined by the referenced primary key, so a custom type doesn't
apply. Setting one clears and disables the other, and the foreign key
select stays disabled on the primary key column and when no targets exist.

Also add "Controls how Datasette displays and edits this column" help
text (with aria-describedby) under the custom type selector in both the
create and alter dialogs, and style the alter dialog help text.

* Drop table button in alter dialog

* Object not chain of ifs

Refs https://github.com/simonw/datasette/pull/2789/changes#r3453964430

* Fix broken Playwright tests

* Keyword arguments for readability

* Ran prettier

* Removed the alter table dry run feature

It works by doing conn.backup(memory_conn) which could use
a lot of memory for a large database.

* sqlite-utils>=3.30,<4.0

So we don't get test failures from reformatted SQL.

* not_null, default and default_exr support for create table API columns

* Alter table API can now rename tables, refs #2788

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4771774289

* Fix for Safari select box heights

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4772241681

* Expose foreign key data to alter table UI

Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table.

Add API and HTML data tests for the new alter-table foreign key support.

* Unify create and alter table modal controls

Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend.

Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.

* Add rename table controls to alter table dialog

Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow.

Redirect to the renamed table URL after applying changes and cover the review text in Playwright.

* current_unixtime and current_unixtime_ms default_expr options

Plus tweaked how alter table changing those works a bit.

* Draft changelog for create/alter table UI, refs #2787, #2788
2026-06-22 13:54:36 -07:00
Simon Willison
b3b5c25df8 Draft changelog for create/alter table UI, refs #2787, #2788 2026-06-22 13:51:32 -07:00
Simon Willison
b932d0dc78 current_unixtime and current_unixtime_ms default_expr options
Plus tweaked how alter table changing those works a bit.
2026-06-22 13:42:35 -07:00
Copilot
1717ab02f7 Replace row edit icon SVG in table.py
Closes #2796
2026-06-22 13:23:18 -07:00
Simon Willison
2ebae5ed71 Add rename table controls to alter table dialog
Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow.

Redirect to the renamed table URL after applying changes and cover the review text in Playwright.
2026-06-22 12:51:23 -07:00
Simon Willison
fa43aba309 Unify create and alter table modal controls
Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend.

Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.
2026-06-22 12:50:57 -07:00
Simon Willison
063b04ad83 Expose foreign key data to alter table UI
Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table.

Add API and HTML data tests for the new alter-table foreign key support.
2026-06-22 12:47:02 -07:00
Simon Willison
dada4de172 Fix for Safari select box heights
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4772241681
2026-06-22 12:35:34 -07:00
Simon Willison
4b219be8bd Alter table API can now rename tables, refs #2788
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4771774289
2026-06-22 12:09:07 -07:00
Simon Willison
87354cf94e not_null, default and default_exr support for create table API columns 2026-06-22 11:04:19 -07:00
Simon Willison
a2e75967ce sqlite-utils>=3.30,<4.0
So we don't get test failures from reformatted SQL.
2026-06-22 11:03:49 -07:00
Simon Willison
084df1fba2 Removed the alter table dry run feature
It works by doing conn.backup(memory_conn) which could use
a lot of memory for a large database.
2026-06-22 10:18:01 -07:00
Simon Willison
c4aead65ee Ran prettier 2026-06-22 10:11:56 -07:00
Simon Willison
17876ccf45 Keyword arguments for readability 2026-06-22 10:11:56 -07:00
Simon Willison
b02999bca6 Fix broken Playwright tests 2026-06-22 10:11:56 -07:00
Simon Willison
a87c4ac555 Object not chain of ifs
Refs https://github.com/simonw/datasette/pull/2789/changes#r3453964430
2026-06-22 10:11:56 -07:00
Simon Willison
354780a136 Drop table button in alter dialog 2026-06-22 10:11:56 -07:00
Simon Willison
e834008075 Make custom type and foreign key mutually exclusive
In the create table dialog a column can now have either a custom display
type or a foreign key target, but not both - a foreign key column's type
is determined by the referenced primary key, so a custom type doesn't
apply. Setting one clears and disables the other, and the foreign key
select stays disabled on the primary key column and when no targets exist.

Also add "Controls how Datasette displays and edits this column" help
text (with aria-describedby) under the custom type selector in both the
create and alter dialogs, and style the alter dialog help text.
2026-06-22 10:11:56 -07:00
Simon Willison
c77dad910b More robust test_datasette_https_server.sh test 2026-06-22 10:11:56 -07:00
Simon Willison
1f863def5e Add foreign key controls to create table dialog
- Add create table advanced controls for foreign keys and first-column primary keys
- Share schema dialog row helpers between create and alter dialogs
- Move custom type into advanced options and add Add column icons
2026-06-22 10:11:56 -07:00
Simon Willison
21c156dfb1 Expose foreign key targets to create table UI
- Add foreignKeyTargetsPath to create table page data
- Filter hidden tables from database-level foreign key target results
- Update JSON API docs and tests for filtered targets
2026-06-22 10:11:56 -07:00
Simon Willison
a6ef65f90d /<database>/-/foreign-key-targets API endpoint
Returns a list of tables with a single primary key, and for each one
the name of that primary key column and its SQLite type affinity.

This will be used by the create table UI to suggest foreign keys.
2026-06-22 10:11:56 -07:00
Simon Willison
2900efb32d /db/table/-/foreign-key-suggestions API
Improved version of the implementation datasette-edit-schema
2026-06-22 10:11:56 -07:00
Simon Willison
9d9a2d3ff3 Add foreign keys to alter table API
- Add add_foreign_key, drop_foreign_key, and set_foreign_keys operations.
- Validate flat fk_table and fk_column arguments with Pydantic.
- Document the API and cover inferred primary-key and validation cases.
2026-06-22 10:11:56 -07:00
Simon Willison
9766a9c087 Add foreign keys to create table API
- Add fk_table and optional fk_column support to create-table columns.
- Validate create-table requests with Pydantic while preserving existing errors.
- Document the API and cover inferred primary-key and validation cases.
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733544452
2026-06-22 10:11:56 -07:00
Simon Willison
1972ba8952 Split table create and alter views
- Move create-table and alter-table API views into table_create_alter.py.
- Keep create and alter schema-editing constants and helpers together.
- Rename the create table modal context helper.
2026-06-22 10:11:56 -07:00
Simon Willison
8cec528eeb Test against pyodide/v314.0.0
Now that we depend on pydantic we need a more recent
pyodide in order to load the emscripten build
of pydantic-core.

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733412763
2026-06-22 10:11:56 -07:00
Simon Willison
4115213e17 Precompute action permissions for table pages
- Extract reusable helpers for database and table action permission preloading.
- Precompute those permissions before building table-page HTML data.
- Document the default table actions plugin.
2026-06-22 10:11:56 -07:00
Simon Willison
c9c79fdfc8 Isolate Unix domain socket test server paths
- Use a per-process socket path for the UDS test fixture.
- Clean up stale socket files before and after the fixture runs.
- Close the HTTP client and wait for the Datasette subprocess to exit.
2026-06-22 10:11:56 -07:00
Simon Willison
15a3ac58cc Ran Prettier 2026-06-22 10:11:56 -07:00
Simon Willison
fdd1b61a3e Add alter table modal
- Register a built-in table action and expose alter-table metadata to table pages.
- Build the client-side modal for editing columns, defaults, ordering, primary keys, and custom column types.
- Add a review/apply confirmation flow with HTML and Playwright coverage.

Refs #2788
2026-06-22 10:11:56 -07:00
Simon Willison
b40665dd14 Add alter table JSON API
- Add POST /<database>/<table>/-/alter with Pydantic validation and dry-run support.
- Support add, rename, alter, drop, primary-key and reorder operations, including allow-listed default expressions.
- Document the endpoint and cover schema changes, validation, permissions, events and dry runs.

Refs #2788
2026-06-22 10:11:56 -07:00
Simon Willison
2d3c85dfc0 Add create table UI
Adds a permission-gated database action that opens a create table modal on database pages, backed by the existing create-table JSON API.

The modal starts with an id integer primary key column plus a blank text column, supports SQLite type selection, and shows custom column type controls only when the actor can set column types.

Selected custom column types are applied after table creation with follow-up set-column-type API calls. Includes styling plus HTML and Playwright coverage for the action payload and create-table flow.
2026-06-22 10:11:56 -07:00
Simon Willison
57e7bba38f Stop matrix testing against sqlite-utils 4.0rc1
Refs https://github.com/simonw/sqlite-utils/issues/758#issuecomment-4770349630

I'm going to test this in a branch instead.
2026-06-22 09:06:00 -07:00
Simon Willison
bccb7f17e5 Don't fail test run on sqlite-utils 4.0rc1 failures
Refs https://github.com/simonw/sqlite-utils/issues/758#issuecomment-4763675905
2026-06-21 16:52:14 -07:00
Simon Willison
387f4dd4bc Try sqlite-utils 4.0rc1 in CI
Also output current sqlite-utils version in pytest headers

Refs https://github.com/simonw/sqlite-utils/issues/758
2026-06-21 16:46:20 -07:00
Simon Willison
ad6fe47a95 Better way of setting min height on CodeMirror
Previously there were four lines of whitespace, but that
meant users had to delete that whitespace themselves when
they started editing. Now it is four lines tall without that.
2026-06-19 22:40:37 -07:00
Simon Willison
f673e7416f
Add create table execute-write template (#2794)
Add a Create table starter template to the execute-write page, alongside the existing table-specific insert/update/delete templates.

Keep template button clicks in-page by updating CodeMirror and the URL without collapsing the template controls.

Refresh schema metadata after successful schema-changing execute-write statements so newly created tables appear immediately.
2026-06-19 17:14:33 -07:00
Simon Willison
dfd5b95ec8 Document --headed option, closes #2791 2026-06-17 09:52:02 -07:00
Simon Willison
1ae55007f2 Release 1.0a34 1.0a34
Refs #2767, #2775, #2776, #2779, #2780, #2782
2026-06-16 14:27:26 -07:00
Simon Willison
8c700f5337 Renamed base_url parameter to ds_base_url in tests
Integrating Playwright brought in pytest-base-url which declares a
base_url fixture that was colliding with our own base_url parameters.
2026-06-16 14:25:42 -07:00
Simon Willison
39a397e5b7
Test Datasette JavaScript features using Playwright
Merge pull request #2785 from simonw/playwright-tests
2026-06-16 14:16:20 -07:00
Simon Willison
3edc98a871 Playwright tests exercise the real UI now
Refs https://github.com/simonw/datasette/pull/2785#issuecomment-4723583959
2026-06-16 14:13:44 -07:00
Simon Willison
a8124172dd Remove load_edit_tools(), use a new fixture instead
Refs https://github.com/simonw/datasette/pull/2785/changes#r3423913454
2026-06-16 13:54:00 -07:00
Simon Willison
2f27b08405 Use code-block:: bash for Playwright docs 2026-06-16 13:41:24 -07:00
Simon Willison
77fa49cf17 Just recipes for running Playwright tests
Plus the pytest header now reports selected Playwright browsers
2026-06-16 13:35:17 -07:00
Simon Willison
2a887e5853 Merge branch 'main' into playwright-tests 2026-06-16 13:29:22 -07:00
Simon Willison
b5ba41c9db Fix for Safari esc-closes-confirm bug, closes #2784 2026-06-16 13:28:18 -07:00
Simon Willison
0a0491568a Fix flaky autocomplete timeout test 2026-06-16 11:59:42 -07:00
Simon Willison
dc6f207d37 Update unreleased release notes section
Refs #2780 and many more.
2026-06-14 20:29:59 -07:00