Commit graph

1,200 commits

Author SHA1 Message Date
Chris (ChrisJr404)
81e2ad8a44
tables and views commands accept optional table/view names
The tables and views commands now take one or more optional table (or view)
names as positional arguments after the database path, restricting the output
to just those tables. This is handy with --counts against a database that has
a large table you would rather skip.

Names are listed in the order they are passed. If any named table or view does
not exist the command raises an error and exits with a non-zero status, so the
exit code can be used to confirm a set of expected tables is present.

Closes #478
2026-08-18 06:22:20 -04:00
Simon Willison
56dd09702f Run no-default-groups smoke test from Justfile
Refs #842

I had to add --isolated because otherwise this test would pass if a .venv
folder already existed with the dev dependencies installed in it.
2026-08-13 17:01:47 -07:00
Simon Willison
28dc6278cc Release 4.2.1 4.2.1
Refs #842, #843
2026-08-13 16:52:30 -07:00
Simon Willison
f6d73112c8
Fix for sqlite-utils 4.2 crashing bug (#843)
- Remove from typing_extensions import Self
- Smoke test: uv run --no-default-groups sqlite-utils --help

Closes #842
2026-08-13 16:52:03 -07:00
Simon Willison
1d98613f28 Release 4.2 4.2
Refs #488, #602, #762, #790, #805, #808, #811, #816, #821, #822, #824, #825, #828, #829, #831, #833, #834, #836, #837
2026-08-13 13:09:42 -07:00
ikatyal2110
e4935e0644
transform: coerce empty strings to NULL when converting TEXT columns to numeric types (#805)
* transform: coerce empty strings to NULL when converting TEXT columns to numeric types

When a TEXT column is transformed to INTEGER, FLOAT, or REAL and a row
contains an empty string, the empty string is now converted to NULL during
the INSERT...SELECT copy, matching the expected behavior described in #488.

Fixes #488
2026-08-13 12:56:51 -07:00
Simon Willison
75ba588462 Preserve composite UNIQUE constraints in transforms 2026-08-12 18:46:18 -07:00
Simon Willison
2b52b5ed6f Preserve AUTOINCREMENT through transforms 2026-08-12 18:39:00 -07:00
Simon Willison
fcfccea813 Support ANY column types for strict tables
Closes #790, #820
2026-08-12 16:44:01 -07:00
Simon Willison
57192ef4e3 table.transform(rename=...) now preserves indexes, closes #822 2026-08-12 14:42:24 -07:00
Simon Willison
e4784ec120 Changelog updates
Refs #808, #811, #816, #821, #824, #825, #828, #829, #833, #836, #837
2026-08-12 14:38:30 -07:00
Simon Willison
88b48fa167 Fixed introspection of default values TRUE / FALSE / NULL
Closes #836
2026-08-12 14:19:38 -07:00
nyxst4ck
e6be6267a4
Use quote_identifier() in indexes/xindexes PRAGMA statements (#825)
Closes #824
2026-08-12 14:15:17 -07:00
Simon Willison
c5063f67b1 Use quoted SQL identifiers in convert --dry-run, closes #829 2026-08-12 14:14:27 -07:00
Rami Abdelrazzaq
25c632fbbc
Handle empty input in rows_from_file
Closes #808
2026-08-12 14:05:09 -07:00
Simon Willison
ebb04a97de Fixes for Pyright, closes #833 2026-08-12 14:03:04 -07:00
Simon Willison
38fe466700 Use db.table() and db.view() in tests, closes #838 2026-08-12 14:03:04 -07:00
ethanhawkes-gif
43d5d3331f
Emit LIMIT -1 when offset is used without limit (#821)
* Emit LIMIT -1 when offset is used without limit, closes #816

SQLite requires a LIMIT clause to appear before OFFSET, so passing offset
without limit generated invalid SQL such as:

    select * from "t" offset 2

which raised OperationalError: near "2": syntax error.

A negative limit means "no upper bound" in SQLite, so "limit -1 offset N"
returns all rows from position N onwards.

Fixed in three places that build LIMIT/OFFSET SQL:

- Queryable.rows_where() - also covers pks_and_rows_where()
- Table.search_sql() - also covers search()
- the "sqlite-utils rows" CLI command

* Remove duplicate comments

---------

Co-authored-by: ethanhawkes-gif <259455325+ethanhawkes-gif@users.noreply.github.com>
2026-08-11 22:52:43 -07:00
Bunlong Heng
2d3c6b9a1e
Escape tokenize argument in enable_fts (#828)
The tokenize value passed to Table.enable_fts() was interpolated directly
into the CREATE VIRTUAL TABLE statement inside a single-quoted string
literal. A value containing a single quote could break out of that literal
and inject arbitrary SQL, which executes via executescript(). This is
reachable from the CLI via 'enable-fts --tokenize'.

Route the value through the existing Database.quote() helper so SQLite
itself escapes it. Legitimate tokenizers such as 'porter' are unaffected.
Adds a regression test.
2026-08-11 22:48:06 -07:00
Simon Willison
b37b8cf8c8 Preserve column before/after comments through .transform()
Refs #762

The before comment comes before the column definition - the after
comment is anything after it but before its trailing comma.
2026-08-11 22:45:00 -07:00
Simon Willison
b432e686ca Use sqlite_master not sqlite_schema for older SQLite compatibility 2026-08-11 22:45:00 -07:00
Simon Willison
2303b80aef .transform() preserves check constraints, refs #762 2026-08-11 22:45:00 -07:00
Simon Willison
3db0c57a3b table.checks, table.column_checks, table.table_checks, closes #834
Refs #762
2026-08-11 22:45:00 -07:00
Simon Willison
f726ea4a65
transform() now works for tables referenced by views (#832)
Closes #831
2026-08-11 20:48:11 -07:00
ikatyal2110
6a456830ca
Fix _decode_default_value to unescape doubled single quotes in string defaults (#811)
* Fix _decode_default_value to unescape doubled single quotes in string defaults

SQLite stores string defaults with single quotes doubled (e.g. DEFAULT 'O''Brien'
is stored as the literal "'O''Brien'" in sqlite_master). The previous code
stripped the outer quotes with value[1:-1] but never converted '' back to ',
so default_values returned the raw escaped form instead of the true string value.

* Test for doubled single quotes in string defaults
2026-07-25 21:52:04 -07:00
Simon Willison
a7b734946f Changelog entry for 3.39.1
Refs #815

Copied from e1d55de8f8
2026-07-25 21:50:59 -07:00
Simon Willison
c621499ed1 codespell should check sqlite_utils as well
It did in CI but did not in the Justfile
2026-07-25 14:53:46 -07:00
Simon Willison
69a1c0d960
Fixes for Ruff>=0.16.0 (#814)
* Automated upgrades by Ruff

    uvx --with 'ruff>=0.16.0' ruff check . --fix --unsafe-fixes

* Fix remaining Ruff errors with GPT-5.6 Sol high

https://gist.github.com/simonw/6da7906a9fea6e90da131c21a9055199

* Fix flake E501 long lines
* New Protocol for migrations to make ty happy
2026-07-25 14:53:12 -07:00
Simon Willison
a947dc6739 Changelog now links to CLI and Python API in most recent entry 2026-07-12 17:14:01 -07:00
Simon Willison
458b3ab5b1
Release 4.1.1 4.1.1
Refs #791, #792, #794, #795
2026-07-12 13:52:14 -07:00
Simon Willison
f66ddcb215
Transform now refuses to run inside a transaction if destructive foreign keys exist (#795)
* Transform now refuses to run inside a transaction if destructive foreign keys exist

Closes #794

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014StVTWQJpFhfZJK2CYVBwv
2026-07-12 08:43:51 -07:00
Simon Willison
d714200659
Add test: transform does not cascade-delete referencing records (#792)
> Add a test that covers what happens if you run transform against a table with ON CASCADE DELETE for one of its foreign keys - those records should not be deleted during the transform even though the table is dropped as part of that procedure
2026-07-12 05:00:32 -07:00
Simon Willison
3f0471701b
Add cross-reference notes between CLI and Python API documentation (#791) 2026-07-11 21:45:21 -07:00
Simon Willison
5e8822efd2
Clarify usage of named parameters in CLI documentation 2026-07-11 18:52:09 -07:00
Simon Willison
57c1617391 Release 4.1 4.1
Refs #131, #626, #684, #765, #787
2026-07-11 16:48:41 -07:00
Simon Willison
b74b727035
.transform(strict=) and sqlite-utils transform --strict/--no-strict (#788)
* .transform(strict=) and sqlite-utils transform --strict/--no-strict

Closes #787
2026-07-11 16:43:37 -07:00
Simon Willison
dc61f75a0b sqlite-utils upsert optional --pk in changelog 2026-07-11 16:42:59 -07:00
Simon Willison
6531a57863 Delete obsolete test_memory_attribute_for_existing_connection test
Refs https://github.com/simonw/sqlite-utils/issues/789#issuecomment-4949146177

Closes #789
2026-07-11 16:28:36 -07:00
Simon Willison
0f2d525d06 Clarify transaction documentation
Based on extensive digging into how this stuff all works.
2026-07-11 16:20:37 -07:00
Simon Willison
7a52214624 drop-index command and table.drop_index(index_name)
Closes #626
2026-07-07 21:00:43 -07:00
Sai Asish Y
d302835d57 feat(db): document and test Database.memory and Database.memory_name
Refs #590, closes #734
2026-07-07 19:40:56 -07:00
Simon Willison
d2ac3765ed sqlite-utils insert/upsert --type colunm-name type option, closes #131 2026-07-07 19:17:29 -07:00
Simon Willison
092f0919c3 Changelog tweak refs #684 2026-07-07 18:57:27 -07:00
Simon Willison
569608e40f sqlite-utils insert --code option, closes #684 2026-07-07 18:49:25 -07:00
Simon Willison
23a21c1d6b Ran Black 2026-07-07 18:11:34 -07:00
Simon Willison
ebafb84c93 Allow sqlite-utils upsert to infer --pk from existing table 2026-07-07 18:09:56 -07:00
Simon Willison
aa300942bf Update sqlite-utils convert --help, refs #686 2026-07-07 18:03:25 -07:00
Simon Willison
cf3373e7b7 Refactor --functions option, improve help 2026-07-07 17:59:58 -07:00
Simon Willison
8ee0b7c65c Fix for rogue quote in enable-fts --help 2026-07-07 17:57:35 -07:00
Simon Willison
fa5d66bf53 Update create-table --help to mention real 2026-07-07 17:56:57 -07:00