- Changelog entries for all the compound foreign key work
- Upgrading guide notes the transform() behavior changes
- ForeignKey added to the API reference
- Updated .foreign_keys introspection example output
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
For foreign keys declared as "REFERENCES other_table" with no explicit
columns, PRAGMA foreign_key_list returns None for the referenced
column. The foreign_keys property now resolves these to the other
table's primary key columns, so other_column=None unambiguously
indicates a compound foreign key.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ForeignKey gains on_delete and on_update fields (default "NO ACTION"),
populated from PRAGMA foreign_key_list. create_table_sql() renders the
corresponding clauses for both inline and compound table-level foreign
keys, which means table.transform() now preserves actions such as
ON DELETE CASCADE - previously they were silently stripped whenever a
table was transformed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
table.add_foreign_key() and db.add_foreign_keys() now accept lists of
column names to create compound foreign keys:
table.add_foreign_key(
["campus_name", "dept_code"], "departments"
)
Omitting the other columns uses the compound primary key of the other
table. Duplicate detection compares the full column lists.
db.index_foreign_keys() now creates a single composite index across the
columns of a compound foreign key, rather than an index per column.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Compound foreign keys are passed through table.transform() intact
instead of being degraded to per-column single foreign keys:
- rename= applies to each member column of a compound key
- drop= of any member column drops the whole constraint, matching
the existing single-column behavior
- drop_foreign_keys= accepts a bare column name (drops any foreign
key that column participates in) or a tuple of columns (drops the
compound key with exactly those columns)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
create_table() and friends now accept compound foreign keys, specified
as lists of column names in the existing tuple forms:
foreign_keys=[
(["campus_name", "dept_code"], "departments"),
(["campus_name", "dept_code"], "departments", ["campus_name", "dept_code"]),
]
The two-item form guesses the compound primary key of the other table.
Compound keys are rendered as table-level FOREIGN KEY constraints,
while single-column keys keep their inline REFERENCES clauses.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sorted() on a foreign_keys list containing both compound (column=None)
and single-column ForeignKeys raised TypeError because dataclass
ordering compared None against str. column/other_column are now
excluded from comparison - ordering and equality use the always
populated columns/other_columns lists instead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rejecting statements that return no rows is the change most likely
to require code edits when upgrading to 4.0, so it now leads the
Python API changes section - before db.table() - with the full list
of rejected statement types and a before/after example.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
query() and execute() classified statements using
sql.lstrip().upper().startswith(...), which a leading SQL comment
defeated: db.query('/* c */ COMMIT') inside db.atomic() committed
the caller's transaction and masked the ValueError with a 'no such
savepoint' error, comment-prefixed PRAGMAs ran inside the savepoint
guard where journal mode changes are refused, and a comment-prefixed
BEGIN passed to db.execute() was instantly auto-committed.
The new _first_keyword() helper skips leading whitespace and -- or
/* */ comments - the only things SQLite's tokenizer allows before
the first token - and returns that token for exact comparison, so
keyword detection now matches what SQLite itself will see.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
db.query() rejected statements roll back, RETURNING commits immediately.
Fixes two transaction bugs in db.query():
- A non-row-returning statement such as an UPDATE was executed and
auto-committed by execute() before the ValueError was raised, so the
write took effect despite being rejected. query() now runs the
statement inside a savepoint and rolls it back before raising, so a
rejected statement has no effect on the database - in every
connection mode, including autocommit=True.
- INSERT ... RETURNING only committed once the returned generator was
fully exhausted, so calling query() without iterating - or partially
iterating with next() - left the transaction open and the write could
be rolled back on close. The write is now completed and committed at
call time, as the documentation already promised. Plain SELECTs are
still fetched lazily.
Transaction control statements, VACUUM, ATTACH and DETACH never return
rows, so query() now rejects them without executing them. PRAGMA
statements skip the savepoint guard because some of them - such as
pragma journal_mode=wal - refuse to run inside a transaction.
Claude-Session: https://claude.ai/code/session_012U3iRfJoTZ5vd22cBSF2nJ
The documentation style for this project is one line per paragraph,
not text-wrapped RST. Unwraps three recently added paragraphs in
python-api.rst and two older ones in plugins.rst.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
parametrize received a generator from Path.glob(), which pytest 10
will reject. sorted() materializes the list and makes the parameter
order deterministic as a bonus.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Runs the entire test suite against connections created with the
Python 3.12+ sqlite3.connect(autocommit=True) mode, by patching
connect() to default autocommit=True for internally-created
connections. Errors on older Python versions. The full suite
passes in both modes with no test changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Raw write statements previously opened an implicit transaction that
stayed open until something committed it - the write was visible on
the same connection, making it look saved, but was silently rolled
back when the connection closed. execute() now commits any implicit
transaction it opens, so raw writes behave like every other write
in the library: committed as soon as they run, unless an explicit
transaction (db.begin() or db.atomic()) is open, in which case they
join it.
Row-returning writes such as INSERT ... RETURNING via db.query()
commit once their rows have been iterated. atomic(), commit() and
rollback() now issue literal COMMIT/ROLLBACK statements, which have
identical semantics in every sqlite3 connection mode. The CLI bulk
command uses db.atomic() instead of 'with db.conn:'.
This simplifies the transaction contract to: everything commits
immediately unless you explicitly opened a transaction. Docs
updated throughout; changelog and upgrading guide cover the
breaking change for code that relied on rolling back uncommitted
execute() writes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Connections created with sqlite3.connect(autocommit=True) make
commit() a documented no-op, and autocommit=False connections are
permanently inside a transaction - in both modes every write made
by this library appeared to work in-process but was silently
discarded when the connection closed. Database() now raises
TransactionError for these connections instead of losing data.
Tests are skipped on Python versions before 3.12, where the
autocommit parameter does not exist.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
A dedicated sqlite_utils.db.TransactionError exception is clearer
to catch than a generic RuntimeError, and gives future
transaction-state guards a home.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Manual transaction control previously required mixing raw
db.execute("begin") strings with methods on db.conn. These thin
wrappers give the documentation and callers a single consistent
idiom. Docs updated to use them throughout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Expands the closing-a-database docs: exit is equivalent to close(),
open transactions can only come from manual transaction control,
auto-committing could silently persist half-finished work, and the
behavior deliberately differs from sqlite3.Connection's own context
manager.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
The manual-transaction warning could be read as covering
db.atomic() too. Nested atomic() blocks are safe - they become
savepoints inside the migration's transaction, preserving the
all-or-nothing rollback guarantee.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
The method existed only to raise NotImplementedError, since
full-text search is not supported for views, and it showed up in
the generated API reference as a documented View method. Calling
enable_fts() on a View now raises AttributeError like any other
missing method. The sqlite-utils enable-fts command uses db.table()
and shows a clean error when pointed at a view instead of a
traceback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
New top-level 'Transactions and saving your changes' section in the
Python API documentation, stating the fundamental contract up
front: every write method commits its own changes before returning,
no commit() call is ever needed. The db.atomic() documentation
moves under it, alongside new subsections covering the two cases
where the user owns the commit - raw db.execute() writes and
manually managed transactions - and the supported connection modes.
Cross-referenced from Getting started (the first insert example now
says the rows are saved immediately), a warning on db.execute(),
the closing-a-database section and the upgrading guide.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
These were working documents for the 4.0 pre-release review - the
actionable findings have all been fixed on this branch and the
changelog and upgrading guide carry the user-facing record.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
New docs page describing the changes needed to upgrade between
major versions, with a detailed 3.x to 4.0 section covering CLI,
Python API and packaging changes, plus brief sections for the
2.x to 3.0 and 1.x to 2.0 upgrades.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
User-facing argument validation in db.py previously used bare
assert statements, which vanish entirely under python -O and raise
AssertionError - an exception type callers should not have to
catch for input validation. Fifteen validation sites now raise
ValueError with the same messages. Internal invariants (an
unreachable branch and a post-update rowcount check) remain
asserts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Type detection has been the default for CSV/TSV data since 4.0a1,
making this flag a no-op kept only for backwards compatibility.
4.0 is the release where it can be removed - the flag now errors,
prompting scripts to drop it. --no-detect-types is unchanged. The
dead detect_types parameter has been removed from
insert_upsert_implementation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
- insert/upsert into a name that is actually a view now shows a
clean error instead of a traceback
- db.view() on a name that is a table now says so in its error
- Database.__getitem__ docstring documents that views are returned
for view names; hash_id docstring corrected (it is a column name,
not a boolean)
- Registering two migrations with the same name in one set now
raises ValueError at registration time instead of executing both
and failing with IntegrityError at apply time
- build-system requires setuptools>=77, needed for the PEP 639
license expression
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
pending() and applied() no longer create the _sqlite_migrations
table or perform the one-way legacy schema upgrade - that now only
happens in apply(). The CLI --list path no longer creates the
database file when it does not exist. applied() results are now
explicitly ordered by insertion.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
An unknown --stop-before value previously matched nothing and every
migration was silently applied, including the one the user meant to
stop before. The CLI now errors unless each value matches a known
migration.
The CLI also always passed stop_before as a list, but older
duck-typed sqlite-migrate Migrations objects compare it against a
single string name - so the flag was silently ignored for exactly
the migration files the compatibility shim exists to support. Legacy
sets now receive a single string, with an error if multiple values
target one legacy set.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Documents that atomic() opens a deferred BEGIN, that calling it
while the connection is already inside a transaction produces a
savepoint whose changes only persist when the outer transaction
commits, and that Python 3.12+ autocommit=True/False connections
are not supported. Adds a test pinning the savepoint-inside-manual-
BEGIN behavior.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
with Database(...) closes the connection on exit without
committing, so uncommitted changes are rolled back. This was the
existing behavior but was undocumented and untested - it is now
stated in the docs and pinned by a test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
test_query_executes_eagerly failed on the sqlean CI matrix jobs
because sqlean.dbapi2.OperationalError is not a subclass of the
stdlib sqlite3.OperationalError. Import sqlite3 via
sqlite_utils.utils like the other test modules do.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
_AppliedMigration.applied_at was annotated datetime.datetime but
the value is the TEXT timestamp read straight from the
_sqlite_migrations table - always a string, matching the format
written by both this module and the older sqlite-migrate package.
Added a test pinning the runtime type.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Changing the journal mode assigns conn.isolation_level, which
commits any open transaction as a side effect - silently breaking
the rollback guarantee of atomic() blocks and of user-managed
transactions. Both methods now raise RuntimeError if a transaction
is open. Calling them when the database is already in the requested
mode remains a no-op.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
A record missing a primary key value - or with None for one - can
never match an existing row, because a NULL primary key never
satisfies ON CONFLICT. Such records were previously inserted as
brand new rows (silently, for multi-record upsert_all calls) or
triggered a KeyError after the insert had already happened. Both
cases, including upserting an empty record, now raise
PrimaryKeyRequired before any SQL is executed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Both ran their INSERT INTO fts(fts) statements via a bare execute()
with no commit, leaving the connection inside an open implicit
transaction - the FTS operation and all subsequent writes were then
silently rolled back when the connection closed. Both are now
wrapped in db.atomic(), matching delete_where() and the other
write operations.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Previously query() was a generator function, so nothing - including
the SQL itself - ran until the result was first iterated. A write
statement passed to query() silently did nothing, and SQL errors
surfaced far from the call site. The SQL now executes as soon as
query() is called, while rows are still fetched lazily during
iteration.
Statements that return no rows now raise a ValueError directing
callers to execute() instead. As a side effect, statements like
INSERT ... RETURNING now work naturally with query().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
drop-table on a view name silently dropped the view, and drop-view
on a table name silently dropped the table, because both used
db[name].drop() which dispatches on the actual object type. They
now use db.table() / db.view() and exit with an explanatory error
pointing at the correct command. --ignore still exits cleanly but
no longer drops anything.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Migrations.apply() now wraps each migration function and its
_sqlite_migrations tracking row in db.atomic(), so they commit
together: a failing migration rolls back cleanly, is not recorded,
and stays pending, eliminating the double-apply hazard where a
partially-failed migration left committed side effects behind.
Migrations that cannot run inside a transaction (VACUUM, journal
mode changes, manual transaction management) can opt out by
registering with @migrations(transactional=False).
Also fixed the test fixtures which used db.query() for INSERT
statements - query() is a lazy generator, so those inserts never
actually executed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Table.delete_where() ran its DELETE via a bare execute() with no
commit, unlike Table.delete() which wraps in db.atomic(). The
connection was left with an open implicit transaction, so the
deletion (and all subsequent writes, including later atomic()
blocks which switched to savepoint mode) was silently rolled back
when the connection closed.
Wrap the DELETE in db.atomic() and remove the now-unnecessary
atomic() wrapper from the delete_where documentation example.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Assessment of the CREATE TABLE parser as the foundation for 4.1
transform() improvements: fit for 4.1, two tokenizer bugs to fix
first (comments, numeric literal defaults), test wiring, and
design decisions around round-trip fidelity and API visibility.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
Consolidated findings from a final review before the stable 4.0
release: five release blockers, semantic decisions to lock in now,
polish items, and verified-sound areas.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
