Commit graph

3,310 commits

Author SHA1 Message Date
Simon Willison
3d67168ee5 Docs and unreleased release notes for bulk insert/create table from data 2026-07-03 12:35:59 -07:00
Simon Willison
3aea678eab Show auto primary keys in bulk insert preview
Render omitted auto integer primary key values as muted auto text in the bulk insert preview so generated IDs are easier to understand.

Expose the auto primary key flag in insert UI metadata and cover the preview behavior in Playwright.
2026-07-03 12:27:02 -07:00
Simon Willison
c864bc866d Add bulk insert upsert controls
Expose conflict handling in the bulk insert UI when pasted rows include primary key columns, and route update-or-insert selections through the existing upsert API.

Add live validation for bulk textarea column errors and Playwright coverage for upsert permissions, conflict options, and validation behavior.

Refs shttps://github.com/simonw/datasette/pull/2813#issuecomment-4878320713
2026-07-03 10:44:36 -07:00
Simon Willison
ed6235e59a Ran cog 2026-07-03 10:30:38 -07:00
Simon Willison
141fe194c6 Hide create-from-data UI without insert permission
Expose whether the actor can insert rows in the create-table page data, then use that flag to hide and guard the create-table-from-data path. This avoids offering a flow that fails after preview for create-table-only actors.

Refs https://github.com/simonw/datasette/pull/2813#issuecomment-4878146376
2026-07-03 09:56:09 -07:00
Simon Willison
4a39e44eb6 Preserve numeric inference for blank CSV values
Convert blank delimited cells to null when create-table-from-data inferred an integer or float column. This keeps sqlite-utils from treating mixed numeric and empty-string values as text.

Add a Playwright regression covering CSV table creation with a blank numeric value.
2026-07-03 09:53:57 -07:00
Simon Willison
4bd9d41c43 Preserve SQLite defaults in bulk insert
Omit columns that are absent from pasted bulk insert data so SQLite defaults can fire during insert. Keep preview cells blank for omitted values while preserving explicit null rendering.

Refs [https://github.com/simonw/datasette/pull/2813#issuecomment-4878063728](https://github.com/simonw/datasette/pull/2813#issuecomment-4878063728)
2026-07-03 09:45:34 -07:00
Simon Willison
b759ea5486 Mock navigator.clipboard so Playwright passes in all three browsers 2026-06-29 17:38:51 -07:00
Simon Willison
9ec42b2dad Improvements to create table from data 2026-06-29 16:02:56 -07:00
Simon Willison
e0a138ffbd Tweaks to bulp insert screen
Moved text around, added clickable 'open a file' button for mobile.

Screenshot: https://github.com/simonw/datasette/pull/2813#issuecomment-4837736664
2026-06-29 15:17:19 -07:00
Simon Willison
cb622a3dd6 Switching to create table from data tab no longer marks as dirty
I was getting a confirm() dialog when I clicked outside the dialog on the
create table from data page despite not having filled out any fields.
2026-06-25 22:35:17 -07:00
Simon Willison
7ae601588e Create table from data feature
Similar to insert rows from JSON/CSV/TSV
but it creates a brand new table instead.
2026-06-25 22:32:47 -07:00
Simon Willison
b7a896a803 Insert multiple rows feature in the insert dialog
Works entirely using existing JSON API.

User can paste in CSV, TSV or JSON (or drop a text file on).

They get a preview of the rows to be imported, then they
are imported in max_insert_rows batches.
2026-06-25 22:02:39 -07:00
Simon Willison
d621bdfbfe Ignore ignored/ directory 2026-06-25 21:20:29 -07:00
Chris Amico
fc2922a300
Document custom json encoder (#1996)
Closes #1983
2026-06-23 16:24:03 -07:00
Simon Willison
488c9cf3d3 Small tweaks to 1.0a35 release notes prior to release 1.0a35 2026-06-23 14:33:20 -07:00
Simon Willison
753fa3b316 Release 1.0a35
Refs #1477, #1510, #1975, #2002, #2127, #2787, #2788, #2794, #2796, #2798, #2803
2026-06-23 14:30:08 -07:00
Simon Willison
85be50ac71 Changelog entry for #1975
Plus fixed broken rst link
2026-06-23 14:22:14 -07:00
Ritesh Kewlani
463eea2bd0
Fix _col=<pk> producing duplicate column in output (#2774)
Closes #1975
2026-06-23 14:18:49 -07:00
Simon Willison
a913ba372a Changelog note for #2002
Refs #2792
2026-06-23 14:09:31 -07:00
Sebastian Cao
a5931594de
Document how actors are displayed (#2792)
Closes #2002
2026-06-23 14:07:56 -07:00
Simon Willison
22ccd8a087 escape_sqlite() favors double quotes, closes #2795 2026-06-23 14:04:20 -07:00
Simon Willison
39f1df5997 Improve docs structure for static(), refs #2804, #2800 2026-06-23 13:59:01 -07:00
Simon Willison
5eca46a4bc
Add cache-busted static asset helper (#2804)
* Add cache-busted static asset helper

Add a static() helper for Datasette, plugin, and mounted static assets that appends content-based hashes, caches hashes in production, and serves matching hashed asset URLs with immutable far-future cache headers.

Closes #2800
2026-06-23 13:44:58 -07:00
Simon Willison
a4f74d1d2b More unreleased changes in changelog 2026-06-23 12:50:23 -07:00
Simon Willison
e3ff63b0f9 Warn plugin authors to avoid name conflicts with extra_template_vars
Closes #1988
2026-06-23 12:46:15 -07:00
Simon Willison
e0cdd38786
Documentation for template context variables
Merge pull request #2803

Closes #1510, closes #2127, closes #1477, refs #2333
2026-06-23 12:37:06 -07:00
Simon Willison
0c523dda20 Remove count truncated context test 2026-06-23 12:29:21 -07:00
Simon Willison
8276879997 Construct table context explicitly 2026-06-23 12:24:42 -07:00
Simon Willison
0d1c097396 Document database views and queries as dataclasses 2026-06-23 12:16:42 -07:00
Simon Willison
34d9a3bf33 Use dataclasses for database table context 2026-06-23 12:11:26 -07:00
Simon Willison
59ab0c0ca0 Clarify template context metadata names 2026-06-23 11:30:30 -07:00
Simon Willison
a43e76c31a Construct row template context explicitly 2026-06-23 09:08:40 -07:00
Simon Willison
cda8f7bbef Remove DataView base class 2026-06-23 09:02:11 -07:00
Simon Willison
17ec88503e Document table mutation UI context 2026-06-23 07:36:07 -07:00
Simon Willison
2680e3c4bd Refresh JSON API extra descriptions 2026-06-23 07:32:37 -07:00
Simon Willison
4ac795e20c Expand template context field documentation 2026-06-23 07:30:04 -07:00
Simon Willison
29971d9729 Clarify base template context docs 2026-06-23 07:29:14 -07:00
Simon Willison
4d031c8562 Add count_truncated template context 2026-06-22 19:47:21 -07:00
Simon Willison
49b1adba7b Merge remote-tracking branch 'origin/main' into template-context-docs
# Conflicts:
#	datasette/views/row.py
2026-06-22 18:44:37 -07:00
Simon Willison
86ea1d4722 Visual improvements to table filter UI
- Looks nicer
- Add / remove buttons work properly

Closes #2798
2026-06-22 18:20:06 -07:00
Simon Willison
f831352551 Avoid SQLite RETURNING for compat with older SQLite
Refs #2789
2026-06-22 14:24:34 -07:00
Simon Willison
f0645c6ddf
Create table, alter table - APIs and modal dialog UI
* Add create table UI

Adds a permission-gated database action that opens a create table modal on database pages, backed by the existing create-table JSON API.

The modal starts with an id integer primary key column plus a blank text column, supports SQLite type selection, and shows custom column type controls only when the actor can set column types.

Selected custom column types are applied after table creation with follow-up set-column-type API calls. Includes styling plus HTML and Playwright coverage for the action payload and create-table flow.

* Add alter table JSON API

- Add POST /<database>/<table>/-/alter with Pydantic validation and dry-run support.
- Support add, rename, alter, drop, primary-key and reorder operations, including allow-listed default expressions.
- Document the endpoint and cover schema changes, validation, permissions, events and dry runs.

Refs #2788

* Add alter table modal

- Register a built-in table action and expose alter-table metadata to table pages.
- Build the client-side modal for editing columns, defaults, ordering, primary keys, and custom column types.
- Add a review/apply confirmation flow with HTML and Playwright coverage.

Refs #2788

* Ran Prettier

* Isolate Unix domain socket test server paths

- Use a per-process socket path for the UDS test fixture.
- Clean up stale socket files before and after the fixture runs.
- Close the HTTP client and wait for the Datasette subprocess to exit.

* Precompute action permissions for table pages

- Extract reusable helpers for database and table action permission preloading.
- Precompute those permissions before building table-page HTML data.
- Document the default table actions plugin.

* Test against pyodide/v314.0.0

Now that we depend on pydantic we need a more recent
pyodide in order to load the emscripten build
of pydantic-core.

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733412763

* Split table create and alter views

- Move create-table and alter-table API views into table_create_alter.py.
- Keep create and alter schema-editing constants and helpers together.
- Rename the create table modal context helper.

* Add foreign keys to create table API

- Add fk_table and optional fk_column support to create-table columns.
- Validate create-table requests with Pydantic while preserving existing errors.
- Document the API and cover inferred primary-key and validation cases.
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4733544452

* Add foreign keys to alter table API

- Add add_foreign_key, drop_foreign_key, and set_foreign_keys operations.
- Validate flat fk_table and fk_column arguments with Pydantic.
- Document the API and cover inferred primary-key and validation cases.

* /db/table/-/foreign-key-suggestions API

Improved version of the implementation datasette-edit-schema

* /<database>/-/foreign-key-targets API endpoint

Returns a list of tables with a single primary key, and for each one
the name of that primary key column and its SQLite type affinity.

This will be used by the create table UI to suggest foreign keys.

* Expose foreign key targets to create table UI

- Add foreignKeyTargetsPath to create table page data
- Filter hidden tables from database-level foreign key target results
- Update JSON API docs and tests for filtered targets

* Add foreign key controls to create table dialog

- Add create table advanced controls for foreign keys and first-column primary keys
- Share schema dialog row helpers between create and alter dialogs
- Move custom type into advanced options and add Add column icons

* More robust test_datasette_https_server.sh test

* Make custom type and foreign key mutually exclusive

In the create table dialog a column can now have either a custom display
type or a foreign key target, but not both - a foreign key column's type
is determined by the referenced primary key, so a custom type doesn't
apply. Setting one clears and disables the other, and the foreign key
select stays disabled on the primary key column and when no targets exist.

Also add "Controls how Datasette displays and edits this column" help
text (with aria-describedby) under the custom type selector in both the
create and alter dialogs, and style the alter dialog help text.

* Drop table button in alter dialog

* Object not chain of ifs

Refs https://github.com/simonw/datasette/pull/2789/changes#r3453964430

* Fix broken Playwright tests

* Keyword arguments for readability

* Ran prettier

* Removed the alter table dry run feature

It works by doing conn.backup(memory_conn) which could use
a lot of memory for a large database.

* sqlite-utils>=3.30,<4.0

So we don't get test failures from reformatted SQL.

* not_null, default and default_exr support for create table API columns

* Alter table API can now rename tables, refs #2788

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4771774289

* Fix for Safari select box heights

Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4772241681

* Expose foreign key data to alter table UI

Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table.

Add API and HTML data tests for the new alter-table foreign key support.

* Unify create and alter table modal controls

Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend.

Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.

* Add rename table controls to alter table dialog

Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow.

Redirect to the renamed table URL after applying changes and cover the review text in Playwright.

* current_unixtime and current_unixtime_ms default_expr options

Plus tweaked how alter table changing those works a bit.

* Draft changelog for create/alter table UI, refs #2787, #2788
2026-06-22 13:54:36 -07:00
Simon Willison
b3b5c25df8 Draft changelog for create/alter table UI, refs #2787, #2788 2026-06-22 13:51:32 -07:00
Simon Willison
b932d0dc78 current_unixtime and current_unixtime_ms default_expr options
Plus tweaked how alter table changing those works a bit.
2026-06-22 13:42:35 -07:00
Copilot
1717ab02f7 Replace row edit icon SVG in table.py
Closes #2796
2026-06-22 13:23:18 -07:00
Simon Willison
2ebae5ed71 Add rename table controls to alter table dialog
Add a collapsed rename-table section to the alter table modal and include rename_table operations in the review/apply flow.

Redirect to the renamed table URL after applying changes and cover the review text in Playwright.
2026-06-22 12:51:23 -07:00
Simon Willison
fa43aba309 Unify create and alter table modal controls
Share default value controls between the create and alter table dialogs and expose create-table default expressions to the frontend.

Add create-table not-null/default handling and align the shared foreign key picker behavior across both dialogs.
2026-06-22 12:50:57 -07:00
Simon Willison
063b04ad83 Expose foreign key data to alter table UI
Include current foreign key metadata in the alter table page data and allow the foreign-key-targets endpoint to be read by actors with alter-table permission for a specific table.

Add API and HTML data tests for the new alter-table foreign key support.
2026-06-22 12:47:02 -07:00
Simon Willison
dada4de172 Fix for Safari select box heights
Refs https://github.com/simonw/datasette/pull/2789#issuecomment-4772241681
2026-06-22 12:35:34 -07:00