Improve db.query() error handling and transaction safety

db.query() rejected statements roll back, RETURNING commits immediately.

Fixes two transaction bugs in db.query():

- A non-row-returning statement such as an UPDATE was executed and
  auto-committed by execute() before the ValueError was raised, so the
  write took effect despite being rejected. query() now runs the
  statement inside a savepoint and rolls it back before raising, so a
  rejected statement has no effect on the database - in every
  connection mode, including autocommit=True.

- INSERT ... RETURNING only committed once the returned generator was
  fully exhausted, so calling query() without iterating - or partially
  iterating with next() - left the transaction open and the write could
  be rolled back on close. The write is now completed and committed at
  call time, as the documentation already promised. Plain SELECTs are
  still fetched lazily.

Transaction control statements, VACUUM, ATTACH and DETACH never return
rows, so query() now rejects them without executing them. PRAGMA
statements skip the savepoint guard because some of them - such as
pragma journal_mode=wal - refuse to run inside a transaction.

Claude-Session: https://claude.ai/code/session_012U3iRfJoTZ5vd22cBSF2nJ
This commit is contained in:
Simon Willison 2026-07-04 17:40:48 -07:00 committed by GitHub
commit 0566a9f128
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 171 additions and 26 deletions

View file

@ -58,7 +58,7 @@ Python API changes
**db.query() executes immediately.** ``db.query(sql)`` previously returned a generator that did not execute the SQL until you started iterating over it. The SQL now runs as soon as the method is called - rows are still fetched lazily. Two consequences:
- Errors in your SQL now raise at the ``db.query()`` call site rather than on first iteration.
- Passing a statement that returns no rows - such as an ``INSERT`` or ``UPDATE`` without a ``RETURNING`` clause - previously did nothing at all, silently. It now raises a ``ValueError``. Use ``db.execute()`` for statements that do not return rows.
- Passing a statement that returns no rows - such as an ``INSERT`` or ``UPDATE`` without a ``RETURNING`` clause - previously did nothing at all, silently. It now raises a ``ValueError``, and the statement is rolled back so it has no effect on the database. Use ``db.execute()`` for statements that do not return rows.
**Upserts use INSERT ... ON CONFLICT.** Upsert operations now use SQLite's ``INSERT ... ON CONFLICT SET`` syntax rather than the previous ``INSERT OR IGNORE`` followed by ``UPDATE``. If your code depends on the old behavior, pass ``use_old_upsert=True`` to the ``Database()`` constructor - see :ref:`python_api_old_upsert`.