Support sqlite3.connect(autocommit=False) connections too

Database() now accepts all three Python transaction handling modes and
behaves identically in each. For autocommit=False connections - where
the driver holds an implicit transaction open at all times - the
library tracks transaction ownership itself:

- A new _explicit_transaction flag distinguishes transactions opened
  with begin()/atomic() from the driver's implicit one, exposed as a
  new db.in_transaction property (conn.in_transaction is always True
  in this mode)
- begin() claims the driver's implicit transaction instead of
  executing BEGIN, which the driver would reject; BEGIN/COMMIT/
  ROLLBACK passed to db.execute() are routed through begin()/commit()/
  rollback()
- Writes outside a user transaction commit the implicit transaction
  immediately, preserving the library's auto-commit contract
- Row-returning statements outside a transaction fetch eagerly and
  commit, so the implicit read transaction does not hold a shared
  lock that blocks writes from other connections
- PRAGMA and VACUUM run in temporary driver autocommit mode
  (ensure_autocommit_on() now flips conn.autocommit), since PRAGMAs
  are silently ignored and VACUUM refused inside the implicit
  transaction

A new pytest --sqlite-autocommit-false option runs the entire suite
in this mode, wired into CI alongside --sqlite-autocommit. Tests that
asserted on conn.in_transaction or wrote through db.conn without
committing now use the mode-aware library API instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PFkrS2nuP8mo1jmT594VCd

docs/changelog.rst

@@ -13,7 +13,7 @@ Unreleased
 - ``sqlite-utils insert`` and ``sqlite-utils upsert`` now accept a ``--code`` option for :ref:`providing a block of Python code <cli_insert_code>` (or a path to a ``.py`` file) that defines a ``rows()`` function or ``rows`` iterable of rows to insert, as an alternative to importing from a file. (:issue:`684`)
 - ``sqlite-utils insert`` and ``sqlite-utils upsert`` now accept ``--type column-name type`` to :ref:`override the type automatically chosen when the table is created <cli_insert_csv_tsv_column_types>`. This is useful for CSV or TSV columns such as ZIP codes that look like integers but should be stored as ``TEXT`` to preserve leading zeros. (:issue:`131`)
 - New ``table.drop_index(name)`` method and ``sqlite-utils drop-index`` command for dropping an index by name. Both accept ``ignore=True``/``--ignore`` to ignore a missing index. (:issue:`626`)
-- ``Database()`` now accepts connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` option. The library manages transactions using explicit ``BEGIN``/``COMMIT``/``ROLLBACK`` and savepoint statements, which behave identically in that mode. Connections created with ``autocommit=False`` are still rejected with a ``TransactionError``, because the driver holds an implicit transaction open at all times in that mode, breaking explicit transaction handling. See :ref:`python_api_transactions_modes`.
+- ``Database()`` now accepts connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` and ``autocommit=False`` options, and adapts its transaction handling so every connection mode behaves identically. Writes still commit automatically unless a transaction opened with ``db.begin()`` or ``db.atomic()`` is open. On ``autocommit=False`` connections - where the driver holds an implicit transaction open at all times - row-returning statements executed outside of a transaction are fetched eagerly so their read transactions do not hold locks that block other connections, and ``PRAGMA``/``VACUUM`` statements run in temporary driver-level autocommit mode. A new ``db.in_transaction`` property reports whether a ``begin()``/``atomic()`` transaction is open, which ``conn.in_transaction`` cannot answer for ``autocommit=False`` connections. See :ref:`python_api_transactions_modes`.
 
 .. _v4_0:
 

docs/python-api.rst

@@ -95,7 +95,7 @@ Instead of a file path you can pass in an existing SQLite connection:
 
     db = Database(sqlite3.connect("my_database.db"))
 
-The connection can use Python's default transaction handling or the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` mode. Connections created with ``autocommit=False`` are rejected with a ``sqlite_utils.db.TransactionError`` - see :ref:`python_api_transactions_modes`.
+The connection can use Python's default transaction handling, or the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` or ``autocommit=False`` modes - see :ref:`python_api_transactions_modes`.
 
 If you want to create an in-memory database, you can do so like this:
 
@@ -409,9 +409,19 @@ Two related safeguards to be aware of:
 Supported connection modes
 --------------------------
 
-``db.atomic()`` and the automatic per-method transactions work with connections in Python's default transaction handling mode and with connections created using the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` option. The library manages transactions itself using explicit ``BEGIN``, ``COMMIT``, ``ROLLBACK`` and savepoint statements, which behave identically in both modes.
+``db.atomic()`` and the automatic per-method transactions work with connections in all three of Python's transaction handling modes: the default mode, and the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` and ``autocommit=False`` modes.
 
-Passing a connection created with ``autocommit=False`` to ``Database()`` raises a ``sqlite_utils.db.TransactionError``. In that mode the ``sqlite3`` driver holds an implicit transaction open at all times, which breaks the explicit transaction handling used by every write method - for example ``BEGIN`` fails because a transaction is always already open.
+In the default mode and with ``autocommit=True`` the library manages transactions itself using explicit ``BEGIN``, ``COMMIT``, ``ROLLBACK`` and savepoint statements, which behave identically in both modes.
+
+With ``autocommit=False`` the ``sqlite3`` driver holds an implicit transaction open at all times, so the library adapts to preserve the same behavior:
+
+- Writes made through the library still commit automatically, unless a transaction opened with ``db.begin()`` or ``db.atomic()`` is open. ``db.begin()`` claims the driver's implicit transaction rather than executing ``BEGIN``, which the driver would reject.
+- ``BEGIN``, ``COMMIT`` and ``ROLLBACK`` statements passed to ``db.execute()`` are routed through ``db.begin()``, ``db.commit()`` and ``db.rollback()``.
+- ``PRAGMA`` and ``VACUUM`` statements executed outside of a transaction run in temporary driver-level autocommit mode, because both are silently ignored - or refused - inside the driver's implicit transaction.
+- Row-returning statements executed with ``db.execute()`` or ``db.query()`` outside of a transaction fetch their results eagerly, then commit. A lazily-read cursor would hold the implicit read transaction - and its shared database lock, which blocks writes from other connections - open indefinitely. This means very large result sets are buffered in memory in this mode.
+- Statements executed directly on ``db.conn`` are not managed: the driver's own rules apply, and nothing is committed until you call ``db.conn.commit()`` yourself.
+
+Use the ``db.in_transaction`` property to check whether a transaction opened with ``db.begin()`` or ``db.atomic()`` is currently open - on ``autocommit=False`` connections ``db.conn.in_transaction`` is always ``True``, so it cannot answer that question.
 
 .. _python_api_table:
 

docs/upgrading.rst

@@ -109,7 +109,7 @@ Two related behavior changes to ``table.transform()``: compound foreign keys now
 - Multi-step operations such as ``table.transform()`` no longer commit an existing transaction you have open - they use savepoints inside it instead.
 - ``db.enable_wal()`` and ``db.disable_wal()`` raise a ``sqlite_utils.db.TransactionError`` if called while a transaction is open, instead of silently committing it.
 - Using ``Database`` as a context manager (``with Database(path) as db:``) closes the connection on exit *without* committing - a transaction you explicitly opened with ``db.begin()`` and did not commit is rolled back.
-- ``Database()`` rejects connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=False)`` option, raising ``sqlite_utils.db.TransactionError``. In that mode the driver holds an implicit transaction open at all times, breaking the library's explicit transaction handling. Connections created with ``autocommit=True`` are supported.
+- ``Database()`` accepts connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` and ``autocommit=False`` options, and adapts its transaction handling so all connection modes behave identically - see :ref:`python_api_transactions_modes`.
 
 Packaging changes
 -----------------