mirror of
https://github.com/simonw/sqlite-utils.git
synced 2026-07-23 01:14:31 +02:00
Reject Python 3.12+ autocommit connections in the Database constructor
Connections created with sqlite3.connect(autocommit=True) make commit() a documented no-op, and autocommit=False connections are permanently inside a transaction - in both modes every write made by this library appeared to work in-process but was silently discarded when the connection closed. Database() now raises TransactionError for these connections instead of losing data. Tests are skipped on Python versions before 3.12, where the autocommit parameter does not exist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
This commit is contained in:
parent
96793668ed
commit
bcd9a26560
5 changed files with 51 additions and 1 deletions
|
|
@ -17,6 +17,7 @@ Breaking changes:
|
|||
- ``db.enable_wal()`` and ``db.disable_wal()`` now raise a ``sqlite_utils.db.TransactionError`` if called while a transaction is open. Previously they would silently commit the open transaction as a side effect of changing the journal mode, breaking the rollback guarantee of ``db.atomic()`` and of user-managed transactions.
|
||||
- The ``View`` class no longer has an ``enable_fts()`` method. It existed only to raise ``NotImplementedError``, since full-text search is not supported for views - calling it now raises ``AttributeError`` instead, and the method no longer appears in the API reference. The ``sqlite-utils enable-fts`` command shows a clean error when pointed at a view.
|
||||
- The no-op ``-d/--detect-types`` flag has been removed from the ``insert`` and ``upsert`` commands. Type detection has been the default for CSV/TSV data since 4.0a1, so the flag did nothing - invocations using it should simply drop it. ``--no-detect-types`` remains available to disable detection.
|
||||
- ``Database()`` now raises a ``sqlite_utils.db.TransactionError`` if passed a connection created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` or ``autocommit=False`` options. ``commit()`` and ``rollback()`` behave differently on those connections, which previously caused every write made by the library to be silently discarded when the connection closed.
|
||||
|
||||
Everything else:
|
||||
|
||||
|
|
|
|||
|
|
@ -95,6 +95,8 @@ Instead of a file path you can pass in an existing SQLite connection:
|
|||
|
||||
db = Database(sqlite3.connect("my_database.db"))
|
||||
|
||||
The connection must use Python's default transaction handling. Connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` or ``autocommit=False`` options are rejected with a ``sqlite_utils.db.TransactionError`` - see :ref:`python_api_transactions_modes`.
|
||||
|
||||
If you want to create an in-memory database, you can do so like this:
|
||||
|
||||
.. code-block:: python
|
||||
|
|
@ -391,10 +393,14 @@ Two related safeguards to be aware of:
|
|||
- ``db.enable_wal()`` and ``db.disable_wal()`` raise a ``sqlite_utils.db.TransactionError`` if called while a transaction is open, because changing the journal mode would commit it as a side effect.
|
||||
- Closing the database - explicitly with ``db.close()``, or by exiting a ``with Database(...) as db:`` block - rolls back any transaction that is still open, see :ref:`python_api_close`.
|
||||
|
||||
.. _python_api_transactions_modes:
|
||||
|
||||
Supported connection modes
|
||||
--------------------------
|
||||
|
||||
``db.atomic()`` and the automatic per-method transactions are designed for connections in Python's default transaction handling mode. Connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` or ``autocommit=False`` options are not supported, because ``commit()`` and ``rollback()`` behave differently on those connections.
|
||||
``db.atomic()`` and the automatic per-method transactions require a connection in Python's default transaction handling mode. Passing a connection created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` or ``autocommit=False`` options to ``Database()`` raises a ``sqlite_utils.db.TransactionError``.
|
||||
|
||||
This is because ``commit()`` and ``rollback()`` behave differently on those connections - under ``autocommit=True`` they are documented no-ops - which would cause every write made by this library to be silently discarded when the connection closed, rather than failing loudly.
|
||||
|
||||
.. _python_api_table:
|
||||
|
||||
|
|
|
|||
|
|
@ -79,6 +79,7 @@ Python API changes
|
|||
- Multi-step operations such as ``table.transform()`` no longer commit an existing transaction you have open - they use savepoints inside it instead.
|
||||
- ``db.enable_wal()`` and ``db.disable_wal()`` raise a ``sqlite_utils.db.TransactionError`` if called while a transaction is open, instead of silently committing it.
|
||||
- Using ``Database`` as a context manager (``with Database(path) as db:``) closes the connection on exit *without* committing - uncommitted changes from raw ``db.execute()`` calls are rolled back.
|
||||
- ``Database()`` rejects connections created with the Python 3.12+ ``sqlite3.connect(..., autocommit=True)`` or ``autocommit=False`` options, raising ``sqlite_utils.db.TransactionError``. On those connections every write the library made was silently discarded when the connection closed.
|
||||
|
||||
Packaging changes
|
||||
-----------------
|
||||
|
|
|
|||
|
|
@ -402,6 +402,18 @@ class Database:
|
|||
if recreate:
|
||||
raise ValueError("recreate cannot be used with connections, only paths")
|
||||
self.conn = cast(sqlite3.Connection, filename_or_conn)
|
||||
# Python 3.12+ autocommit=True/False connections make commit()
|
||||
# and rollback() behave differently, silently breaking the
|
||||
# transaction handling used by every write method
|
||||
autocommit = getattr(self.conn, "autocommit", None)
|
||||
if autocommit is not None and autocommit != getattr(
|
||||
sqlite3, "LEGACY_TRANSACTION_CONTROL", -1
|
||||
):
|
||||
raise TransactionError(
|
||||
"sqlite-utils requires a connection that uses the default "
|
||||
"transaction handling - connections created with "
|
||||
"autocommit=True or autocommit=False are not supported"
|
||||
)
|
||||
self._tracer: Optional[Tracer] = tracer
|
||||
if recursive_triggers:
|
||||
self.execute("PRAGMA recursive_triggers=on;")
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
from sqlite_utils import Database
|
||||
from sqlite_utils.db import TransactionError
|
||||
from sqlite_utils.utils import sqlite3
|
||||
import pytest
|
||||
import sys
|
||||
|
||||
|
||||
def test_recursive_triggers():
|
||||
|
|
@ -54,3 +56,31 @@ def test_database_close(tmpdir, memory):
|
|||
db.close()
|
||||
with pytest.raises(sqlite3.ProgrammingError):
|
||||
db.execute("select 1 + 1")
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
sys.version_info < (3, 12),
|
||||
reason="sqlite3.connect(autocommit=) requires Python 3.12",
|
||||
)
|
||||
@pytest.mark.parametrize("autocommit", [True, False])
|
||||
def test_autocommit_connections_are_rejected(tmpdir, autocommit):
|
||||
# These connection modes break commit()/rollback() in ways that
|
||||
# silently lose data, so the constructor refuses them
|
||||
conn = sqlite3.connect(str(tmpdir / "test.db"), autocommit=autocommit)
|
||||
with pytest.raises(TransactionError):
|
||||
Database(conn)
|
||||
conn.close()
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
sys.version_info < (3, 12),
|
||||
reason="sqlite3.LEGACY_TRANSACTION_CONTROL requires Python 3.12",
|
||||
)
|
||||
def test_legacy_transaction_control_connection_is_accepted(tmpdir):
|
||||
conn = sqlite3.connect(
|
||||
str(tmpdir / "test.db"), autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL
|
||||
)
|
||||
db = Database(conn)
|
||||
db["t"].insert({"id": 1}, pk="id")
|
||||
assert [r["id"] for r in db["t"].rows] == [1]
|
||||
db.close()
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue