sqlite-utils/tests/conftest.py

102 lines
2.8 KiB
Python
Raw Permalink Normal View History

from sqlite_utils import Database
from sqlite_utils.utils import sqlite3
import pytest
CREATE_TABLES = """
create table Gosh (c1 text, c2 text, c3 text);
create table Gosh2 (c1 text, c2 text, c3 text);
"""
def pytest_addoption(parser):
parser.addoption(
"--sqlite-autocommit",
action="store_true",
default=False,
help=(
"Run every test against connections created with the Python 3.12+ "
"sqlite3.connect(autocommit=True) mode"
),
)
Support sqlite3.connect(autocommit=False) connections too Database() now accepts all three Python transaction handling modes and behaves identically in each. For autocommit=False connections - where the driver holds an implicit transaction open at all times - the library tracks transaction ownership itself: - A new _explicit_transaction flag distinguishes transactions opened with begin()/atomic() from the driver's implicit one, exposed as a new db.in_transaction property (conn.in_transaction is always True in this mode) - begin() claims the driver's implicit transaction instead of executing BEGIN, which the driver would reject; BEGIN/COMMIT/ ROLLBACK passed to db.execute() are routed through begin()/commit()/ rollback() - Writes outside a user transaction commit the implicit transaction immediately, preserving the library's auto-commit contract - Row-returning statements outside a transaction fetch eagerly and commit, so the implicit read transaction does not hold a shared lock that blocks writes from other connections - PRAGMA and VACUUM run in temporary driver autocommit mode (ensure_autocommit_on() now flips conn.autocommit), since PRAGMAs are silently ignored and VACUUM refused inside the implicit transaction A new pytest --sqlite-autocommit-false option runs the entire suite in this mode, wired into CI alongside --sqlite-autocommit. Tests that asserted on conn.in_transaction or wrote through db.conn without committing now use the mode-aware library API instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFkrS2nuP8mo1jmT594VCd
2026-07-09 22:06:46 +00:00
parser.addoption(
"--sqlite-autocommit-false",
action="store_true",
default=False,
help=(
"Run every test against connections created with the Python 3.12+ "
"sqlite3.connect(autocommit=False) mode"
),
)
def pytest_configure(config):
import sys
Type fixes now enforced by ty * Fix type warning for pipe.stdout possibly being None Add conditional check before calling .read() on pipe.stdout since Popen can return None for stdout. * Use ctx.meta instead of dynamic attribute for database cleanup Click's Context.meta dictionary is the proper way to store arbitrary data on the context object, avoiding type checker warnings about dynamic attribute assignment. * Add assert for tables.callback before calling Click's callback attribute is typed as Optional[Callable], so add assert to satisfy type checker that it's not None. * Fix type errors in cli.py and db.py - Add type annotation for Database.conn to fix context manager errors - Convert exception objects to str() when raising ClickException - Handle None return from find_spatialite() with proper error message * Fix remaining type errors in cli.py - Add typing import and type annotations for dict kwargs - Use db.table() instead of db[] for extract command - Fix missing str() conversion for exception * Fix type errors in db.py - Add type annotation for Database.conn - Add type: ignore for optional sqlite_dump import - Update execute/query parameter types to Sequence|Dict for sqlite3 compatibility - Use getattr for fn.__name__ access to handle callables without __name__ - Handle None return from find_spatialite() with OSError - Fix pk_values assignment to use local variable * Add type: ignore for optional pysqlite3 and sqlean imports These are alternative sqlite3 implementations that may not be installed. * Fix type errors in tests and plugins - Add type: ignore for monkey-patching Database.__init__ in conftest - Fix CLI test to pass string "2" instead of integer to Click invoke - Add type: ignore for optional sqlean import - Fix add_geometry_column test to use "XY" instead of integer 2 - Add type: ignore for click.Context as context manager - Add type: ignore for enable_fts test that intentionally omits argument - Add type: ignore for sys._called_from_test dynamic attribute - Fix rows_from_file test type error for intentional wrong argument - Handle None from pm.get_hookcallers in plugins.py * Use db.table() instead of db[] for Table-specific operations Changes db[table] to db.table(table) in CLI commands where we know we're working with tables, not views. This resolves most of the Table | View disambiguation type warnings since db.table() returns Table directly rather than Table | View. * Fix remaining type warnings in sqlite_utils package - Add assert for sniff_buffer not being None - Handle cursor.fetchone() potentially returning None - Use db.table() for counts_table and index_foreign_keys - Add type: ignore for cursor union type in raw mode * Ran Black * Run ty in CI * ty check sqlite_utils * Skip running ty on Windows --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 19:34:45 -08:00
sys._called_from_test = True # type: ignore[attr-defined]
Support sqlite3.connect(autocommit=False) connections too Database() now accepts all three Python transaction handling modes and behaves identically in each. For autocommit=False connections - where the driver holds an implicit transaction open at all times - the library tracks transaction ownership itself: - A new _explicit_transaction flag distinguishes transactions opened with begin()/atomic() from the driver's implicit one, exposed as a new db.in_transaction property (conn.in_transaction is always True in this mode) - begin() claims the driver's implicit transaction instead of executing BEGIN, which the driver would reject; BEGIN/COMMIT/ ROLLBACK passed to db.execute() are routed through begin()/commit()/ rollback() - Writes outside a user transaction commit the implicit transaction immediately, preserving the library's auto-commit contract - Row-returning statements outside a transaction fetch eagerly and commit, so the implicit read transaction does not hold a shared lock that blocks writes from other connections - PRAGMA and VACUUM run in temporary driver autocommit mode (ensure_autocommit_on() now flips conn.autocommit), since PRAGMAs are silently ignored and VACUUM refused inside the implicit transaction A new pytest --sqlite-autocommit-false option runs the entire suite in this mode, wired into CI alongside --sqlite-autocommit. Tests that asserted on conn.in_transaction or wrote through db.conn without committing now use the mode-aware library API instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFkrS2nuP8mo1jmT594VCd
2026-07-09 22:06:46 +00:00
autocommit_true = config.getoption("--sqlite-autocommit")
autocommit_false = config.getoption("--sqlite-autocommit-false")
if autocommit_true and autocommit_false:
raise pytest.UsageError(
"--sqlite-autocommit and --sqlite-autocommit-false are mutually exclusive"
)
if autocommit_true or autocommit_false:
if sys.version_info < (3, 12):
raise pytest.UsageError(
Support sqlite3.connect(autocommit=False) connections too Database() now accepts all three Python transaction handling modes and behaves identically in each. For autocommit=False connections - where the driver holds an implicit transaction open at all times - the library tracks transaction ownership itself: - A new _explicit_transaction flag distinguishes transactions opened with begin()/atomic() from the driver's implicit one, exposed as a new db.in_transaction property (conn.in_transaction is always True in this mode) - begin() claims the driver's implicit transaction instead of executing BEGIN, which the driver would reject; BEGIN/COMMIT/ ROLLBACK passed to db.execute() are routed through begin()/commit()/ rollback() - Writes outside a user transaction commit the implicit transaction immediately, preserving the library's auto-commit contract - Row-returning statements outside a transaction fetch eagerly and commit, so the implicit read transaction does not hold a shared lock that blocks writes from other connections - PRAGMA and VACUUM run in temporary driver autocommit mode (ensure_autocommit_on() now flips conn.autocommit), since PRAGMAs are silently ignored and VACUUM refused inside the implicit transaction A new pytest --sqlite-autocommit-false option runs the entire suite in this mode, wired into CI alongside --sqlite-autocommit. Tests that asserted on conn.in_transaction or wrote through db.conn without committing now use the mode-aware library API instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFkrS2nuP8mo1jmT594VCd
2026-07-09 22:06:46 +00:00
"--sqlite-autocommit and --sqlite-autocommit-false require "
"Python 3.12 or higher"
)
real_connect = sqlite3.connect
def autocommit_connect(*args, **kwargs):
Support sqlite3.connect(autocommit=False) connections too Database() now accepts all three Python transaction handling modes and behaves identically in each. For autocommit=False connections - where the driver holds an implicit transaction open at all times - the library tracks transaction ownership itself: - A new _explicit_transaction flag distinguishes transactions opened with begin()/atomic() from the driver's implicit one, exposed as a new db.in_transaction property (conn.in_transaction is always True in this mode) - begin() claims the driver's implicit transaction instead of executing BEGIN, which the driver would reject; BEGIN/COMMIT/ ROLLBACK passed to db.execute() are routed through begin()/commit()/ rollback() - Writes outside a user transaction commit the implicit transaction immediately, preserving the library's auto-commit contract - Row-returning statements outside a transaction fetch eagerly and commit, so the implicit read transaction does not hold a shared lock that blocks writes from other connections - PRAGMA and VACUUM run in temporary driver autocommit mode (ensure_autocommit_on() now flips conn.autocommit), since PRAGMAs are silently ignored and VACUUM refused inside the implicit transaction A new pytest --sqlite-autocommit-false option runs the entire suite in this mode, wired into CI alongside --sqlite-autocommit. Tests that asserted on conn.in_transaction or wrote through db.conn without committing now use the mode-aware library API instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFkrS2nuP8mo1jmT594VCd
2026-07-09 22:06:46 +00:00
kwargs.setdefault("autocommit", autocommit_true)
return real_connect(*args, **kwargs)
sqlite3.connect = autocommit_connect
@pytest.fixture(autouse=True)
def close_all_databases():
"""Automatically close all Database objects created during a test."""
databases = []
original_init = Database.__init__
def tracking_init(self, *args, **kwargs):
original_init(self, *args, **kwargs)
databases.append(self)
Type fixes now enforced by ty * Fix type warning for pipe.stdout possibly being None Add conditional check before calling .read() on pipe.stdout since Popen can return None for stdout. * Use ctx.meta instead of dynamic attribute for database cleanup Click's Context.meta dictionary is the proper way to store arbitrary data on the context object, avoiding type checker warnings about dynamic attribute assignment. * Add assert for tables.callback before calling Click's callback attribute is typed as Optional[Callable], so add assert to satisfy type checker that it's not None. * Fix type errors in cli.py and db.py - Add type annotation for Database.conn to fix context manager errors - Convert exception objects to str() when raising ClickException - Handle None return from find_spatialite() with proper error message * Fix remaining type errors in cli.py - Add typing import and type annotations for dict kwargs - Use db.table() instead of db[] for extract command - Fix missing str() conversion for exception * Fix type errors in db.py - Add type annotation for Database.conn - Add type: ignore for optional sqlite_dump import - Update execute/query parameter types to Sequence|Dict for sqlite3 compatibility - Use getattr for fn.__name__ access to handle callables without __name__ - Handle None return from find_spatialite() with OSError - Fix pk_values assignment to use local variable * Add type: ignore for optional pysqlite3 and sqlean imports These are alternative sqlite3 implementations that may not be installed. * Fix type errors in tests and plugins - Add type: ignore for monkey-patching Database.__init__ in conftest - Fix CLI test to pass string "2" instead of integer to Click invoke - Add type: ignore for optional sqlean import - Fix add_geometry_column test to use "XY" instead of integer 2 - Add type: ignore for click.Context as context manager - Add type: ignore for enable_fts test that intentionally omits argument - Add type: ignore for sys._called_from_test dynamic attribute - Fix rows_from_file test type error for intentional wrong argument - Handle None from pm.get_hookcallers in plugins.py * Use db.table() instead of db[] for Table-specific operations Changes db[table] to db.table(table) in CLI commands where we know we're working with tables, not views. This resolves most of the Table | View disambiguation type warnings since db.table() returns Table directly rather than Table | View. * Fix remaining type warnings in sqlite_utils package - Add assert for sniff_buffer not being None - Handle cursor.fetchone() potentially returning None - Use db.table() for counts_table and index_foreign_keys - Add type: ignore for cursor union type in raw mode * Ran Black * Run ty in CI * ty check sqlite_utils * Skip running ty on Windows --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 19:34:45 -08:00
Database.__init__ = tracking_init # type: ignore[method-assign]
yield
Type fixes now enforced by ty * Fix type warning for pipe.stdout possibly being None Add conditional check before calling .read() on pipe.stdout since Popen can return None for stdout. * Use ctx.meta instead of dynamic attribute for database cleanup Click's Context.meta dictionary is the proper way to store arbitrary data on the context object, avoiding type checker warnings about dynamic attribute assignment. * Add assert for tables.callback before calling Click's callback attribute is typed as Optional[Callable], so add assert to satisfy type checker that it's not None. * Fix type errors in cli.py and db.py - Add type annotation for Database.conn to fix context manager errors - Convert exception objects to str() when raising ClickException - Handle None return from find_spatialite() with proper error message * Fix remaining type errors in cli.py - Add typing import and type annotations for dict kwargs - Use db.table() instead of db[] for extract command - Fix missing str() conversion for exception * Fix type errors in db.py - Add type annotation for Database.conn - Add type: ignore for optional sqlite_dump import - Update execute/query parameter types to Sequence|Dict for sqlite3 compatibility - Use getattr for fn.__name__ access to handle callables without __name__ - Handle None return from find_spatialite() with OSError - Fix pk_values assignment to use local variable * Add type: ignore for optional pysqlite3 and sqlean imports These are alternative sqlite3 implementations that may not be installed. * Fix type errors in tests and plugins - Add type: ignore for monkey-patching Database.__init__ in conftest - Fix CLI test to pass string "2" instead of integer to Click invoke - Add type: ignore for optional sqlean import - Fix add_geometry_column test to use "XY" instead of integer 2 - Add type: ignore for click.Context as context manager - Add type: ignore for enable_fts test that intentionally omits argument - Add type: ignore for sys._called_from_test dynamic attribute - Fix rows_from_file test type error for intentional wrong argument - Handle None from pm.get_hookcallers in plugins.py * Use db.table() instead of db[] for Table-specific operations Changes db[table] to db.table(table) in CLI commands where we know we're working with tables, not views. This resolves most of the Table | View disambiguation type warnings since db.table() returns Table directly rather than Table | View. * Fix remaining type warnings in sqlite_utils package - Add assert for sniff_buffer not being None - Handle cursor.fetchone() potentially returning None - Use db.table() for counts_table and index_foreign_keys - Add type: ignore for cursor union type in raw mode * Ran Black * Run ty in CI * ty check sqlite_utils * Skip running ty on Windows --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 19:34:45 -08:00
Database.__init__ = original_init # type: ignore[method-assign]
for db in databases:
try:
db.close()
except Exception:
pass
@pytest.fixture
def fresh_db():
return Database(memory=True)
@pytest.fixture
def existing_db():
database = Database(memory=True)
database.executescript("""
CREATE TABLE foo (text TEXT);
INSERT INTO foo (text) values ("one");
INSERT INTO foo (text) values ("two");
INSERT INTO foo (text) values ("three");
""")
return database
@pytest.fixture
def db_path(tmpdir):
path = str(tmpdir / "test.db")
db = sqlite3.connect(path)
db.executescript(CREATE_TABLES)
Support sqlite3.connect(autocommit=False) connections too Database() now accepts all three Python transaction handling modes and behaves identically in each. For autocommit=False connections - where the driver holds an implicit transaction open at all times - the library tracks transaction ownership itself: - A new _explicit_transaction flag distinguishes transactions opened with begin()/atomic() from the driver's implicit one, exposed as a new db.in_transaction property (conn.in_transaction is always True in this mode) - begin() claims the driver's implicit transaction instead of executing BEGIN, which the driver would reject; BEGIN/COMMIT/ ROLLBACK passed to db.execute() are routed through begin()/commit()/ rollback() - Writes outside a user transaction commit the implicit transaction immediately, preserving the library's auto-commit contract - Row-returning statements outside a transaction fetch eagerly and commit, so the implicit read transaction does not hold a shared lock that blocks writes from other connections - PRAGMA and VACUUM run in temporary driver autocommit mode (ensure_autocommit_on() now flips conn.autocommit), since PRAGMAs are silently ignored and VACUUM refused inside the implicit transaction A new pytest --sqlite-autocommit-false option runs the entire suite in this mode, wired into CI alongside --sqlite-autocommit. Tests that asserted on conn.in_transaction or wrote through db.conn without committing now use the mode-aware library API instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFkrS2nuP8mo1jmT594VCd
2026-07-09 22:06:46 +00:00
db.commit()
db.close()
return path