Commit graph

1,197 commits

Author SHA1 Message Date
Simon Willison
f6d73112c8
Fix for sqlite-utils 4.2 crashing bug (#843)
- Remove from typing_extensions import Self
- Smoke test: uv run --no-default-groups sqlite-utils --help

Closes #842
2026-08-13 16:52:03 -07:00
Simon Willison
1d98613f28 Release 4.2 4.2
Refs #488, #602, #762, #790, #805, #808, #811, #816, #821, #822, #824, #825, #828, #829, #831, #833, #834, #836, #837
2026-08-13 13:09:42 -07:00
ikatyal2110
e4935e0644
transform: coerce empty strings to NULL when converting TEXT columns to numeric types (#805)
* transform: coerce empty strings to NULL when converting TEXT columns to numeric types

When a TEXT column is transformed to INTEGER, FLOAT, or REAL and a row
contains an empty string, the empty string is now converted to NULL during
the INSERT...SELECT copy, matching the expected behavior described in #488.

Fixes #488
2026-08-13 12:56:51 -07:00
Simon Willison
75ba588462 Preserve composite UNIQUE constraints in transforms 2026-08-12 18:46:18 -07:00
Simon Willison
2b52b5ed6f Preserve AUTOINCREMENT through transforms 2026-08-12 18:39:00 -07:00
Simon Willison
fcfccea813 Support ANY column types for strict tables
Closes #790, #820
2026-08-12 16:44:01 -07:00
Simon Willison
57192ef4e3 table.transform(rename=...) now preserves indexes, closes #822 2026-08-12 14:42:24 -07:00
Simon Willison
e4784ec120 Changelog updates
Refs #808, #811, #816, #821, #824, #825, #828, #829, #833, #836, #837
2026-08-12 14:38:30 -07:00
Simon Willison
88b48fa167 Fixed introspection of default values TRUE / FALSE / NULL
Closes #836
2026-08-12 14:19:38 -07:00
nyxst4ck
e6be6267a4
Use quote_identifier() in indexes/xindexes PRAGMA statements (#825)
Closes #824
2026-08-12 14:15:17 -07:00
Simon Willison
c5063f67b1 Use quoted SQL identifiers in convert --dry-run, closes #829 2026-08-12 14:14:27 -07:00
Rami Abdelrazzaq
25c632fbbc
Handle empty input in rows_from_file
Closes #808
2026-08-12 14:05:09 -07:00
Simon Willison
ebb04a97de Fixes for Pyright, closes #833 2026-08-12 14:03:04 -07:00
Simon Willison
38fe466700 Use db.table() and db.view() in tests, closes #838 2026-08-12 14:03:04 -07:00
ethanhawkes-gif
43d5d3331f
Emit LIMIT -1 when offset is used without limit (#821)
* Emit LIMIT -1 when offset is used without limit, closes #816

SQLite requires a LIMIT clause to appear before OFFSET, so passing offset
without limit generated invalid SQL such as:

    select * from "t" offset 2

which raised OperationalError: near "2": syntax error.

A negative limit means "no upper bound" in SQLite, so "limit -1 offset N"
returns all rows from position N onwards.

Fixed in three places that build LIMIT/OFFSET SQL:

- Queryable.rows_where() - also covers pks_and_rows_where()
- Table.search_sql() - also covers search()
- the "sqlite-utils rows" CLI command

* Remove duplicate comments

---------

Co-authored-by: ethanhawkes-gif <259455325+ethanhawkes-gif@users.noreply.github.com>
2026-08-11 22:52:43 -07:00
Bunlong Heng
2d3c6b9a1e
Escape tokenize argument in enable_fts (#828)
The tokenize value passed to Table.enable_fts() was interpolated directly
into the CREATE VIRTUAL TABLE statement inside a single-quoted string
literal. A value containing a single quote could break out of that literal
and inject arbitrary SQL, which executes via executescript(). This is
reachable from the CLI via 'enable-fts --tokenize'.

Route the value through the existing Database.quote() helper so SQLite
itself escapes it. Legitimate tokenizers such as 'porter' are unaffected.
Adds a regression test.
2026-08-11 22:48:06 -07:00
Simon Willison
b37b8cf8c8 Preserve column before/after comments through .transform()
Refs #762

The before comment comes before the column definition - the after
comment is anything after it but before its trailing comma.
2026-08-11 22:45:00 -07:00
Simon Willison
b432e686ca Use sqlite_master not sqlite_schema for older SQLite compatibility 2026-08-11 22:45:00 -07:00
Simon Willison
2303b80aef .transform() preserves check constraints, refs #762 2026-08-11 22:45:00 -07:00
Simon Willison
3db0c57a3b table.checks, table.column_checks, table.table_checks, closes #834
Refs #762
2026-08-11 22:45:00 -07:00
Simon Willison
f726ea4a65
transform() now works for tables referenced by views (#832)
Closes #831
2026-08-11 20:48:11 -07:00
ikatyal2110
6a456830ca
Fix _decode_default_value to unescape doubled single quotes in string defaults (#811)
* Fix _decode_default_value to unescape doubled single quotes in string defaults

SQLite stores string defaults with single quotes doubled (e.g. DEFAULT 'O''Brien'
is stored as the literal "'O''Brien'" in sqlite_master). The previous code
stripped the outer quotes with value[1:-1] but never converted '' back to ',
so default_values returned the raw escaped form instead of the true string value.

* Test for doubled single quotes in string defaults
2026-07-25 21:52:04 -07:00
Simon Willison
a7b734946f Changelog entry for 3.39.1
Refs #815

Copied from e1d55de8f8
2026-07-25 21:50:59 -07:00
Simon Willison
c621499ed1 codespell should check sqlite_utils as well
It did in CI but did not in the Justfile
2026-07-25 14:53:46 -07:00
Simon Willison
69a1c0d960
Fixes for Ruff>=0.16.0 (#814)
* Automated upgrades by Ruff

    uvx --with 'ruff>=0.16.0' ruff check . --fix --unsafe-fixes

* Fix remaining Ruff errors with GPT-5.6 Sol high

https://gist.github.com/simonw/6da7906a9fea6e90da131c21a9055199

* Fix flake E501 long lines
* New Protocol for migrations to make ty happy
2026-07-25 14:53:12 -07:00
Simon Willison
a947dc6739 Changelog now links to CLI and Python API in most recent entry 2026-07-12 17:14:01 -07:00
Simon Willison
458b3ab5b1
Release 4.1.1 4.1.1
Refs #791, #792, #794, #795
2026-07-12 13:52:14 -07:00
Simon Willison
f66ddcb215
Transform now refuses to run inside a transaction if destructive foreign keys exist (#795)
* Transform now refuses to run inside a transaction if destructive foreign keys exist

Closes #794

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014StVTWQJpFhfZJK2CYVBwv
2026-07-12 08:43:51 -07:00
Simon Willison
d714200659
Add test: transform does not cascade-delete referencing records (#792)
> Add a test that covers what happens if you run transform against a table with ON CASCADE DELETE for one of its foreign keys - those records should not be deleted during the transform even though the table is dropped as part of that procedure
2026-07-12 05:00:32 -07:00
Simon Willison
3f0471701b
Add cross-reference notes between CLI and Python API documentation (#791) 2026-07-11 21:45:21 -07:00
Simon Willison
5e8822efd2
Clarify usage of named parameters in CLI documentation 2026-07-11 18:52:09 -07:00
Simon Willison
57c1617391 Release 4.1 4.1
Refs #131, #626, #684, #765, #787
2026-07-11 16:48:41 -07:00
Simon Willison
b74b727035
.transform(strict=) and sqlite-utils transform --strict/--no-strict (#788)
* .transform(strict=) and sqlite-utils transform --strict/--no-strict

Closes #787
2026-07-11 16:43:37 -07:00
Simon Willison
dc61f75a0b sqlite-utils upsert optional --pk in changelog 2026-07-11 16:42:59 -07:00
Simon Willison
6531a57863 Delete obsolete test_memory_attribute_for_existing_connection test
Refs https://github.com/simonw/sqlite-utils/issues/789#issuecomment-4949146177

Closes #789
2026-07-11 16:28:36 -07:00
Simon Willison
0f2d525d06 Clarify transaction documentation
Based on extensive digging into how this stuff all works.
2026-07-11 16:20:37 -07:00
Simon Willison
7a52214624 drop-index command and table.drop_index(index_name)
Closes #626
2026-07-07 21:00:43 -07:00
Sai Asish Y
d302835d57 feat(db): document and test Database.memory and Database.memory_name
Refs #590, closes #734
2026-07-07 19:40:56 -07:00
Simon Willison
d2ac3765ed sqlite-utils insert/upsert --type colunm-name type option, closes #131 2026-07-07 19:17:29 -07:00
Simon Willison
092f0919c3 Changelog tweak refs #684 2026-07-07 18:57:27 -07:00
Simon Willison
569608e40f sqlite-utils insert --code option, closes #684 2026-07-07 18:49:25 -07:00
Simon Willison
23a21c1d6b Ran Black 2026-07-07 18:11:34 -07:00
Simon Willison
ebafb84c93 Allow sqlite-utils upsert to infer --pk from existing table 2026-07-07 18:09:56 -07:00
Simon Willison
aa300942bf Update sqlite-utils convert --help, refs #686 2026-07-07 18:03:25 -07:00
Simon Willison
cf3373e7b7 Refactor --functions option, improve help 2026-07-07 17:59:58 -07:00
Simon Willison
8ee0b7c65c Fix for rogue quote in enable-fts --help 2026-07-07 17:57:35 -07:00
Simon Willison
fa5d66bf53 Update create-table --help to mention real 2026-07-07 17:56:57 -07:00
Simon Willison
619770bf42 sqlite-utils query - to read SQL from stdin, closes #765 2026-07-07 17:52:26 -07:00
Simon Willison
353baf280d Corrected imports in migrations docs 2026-07-07 11:51:53 -07:00
Simon Willison
8bc9213a8e Release 4.0 4.0
Refs #781, #783, #769
2026-07-07 08:40:31 -07:00