Compare commits

..

4 commits

Author SHA1 Message Date
Simon Willison
36e36dcd3f Preserve column before/after comments through .transform()
Refs #762

The before comment comes before the column definition - the after
comment is anything after it but before its trailing comma.
2026-08-11 22:37:15 -07:00
Simon Willison
9db7578cde Use sqlite_master not sqlite_schema for older SQLite compatibility 2026-08-11 22:02:28 -07:00
Simon Willison
1ac90b4dad .transform() preserves check constraints, refs #762 2026-08-11 21:59:57 -07:00
Simon Willison
4cc0418816 table.checks, table.column_checks, table.table_checks, closes #834
Refs #762
2026-08-11 21:58:00 -07:00
59 changed files with 1424 additions and 2680 deletions

View file

@ -20,7 +20,7 @@ jobs:
cache: pip
cache-dependency-path: pyproject.toml
- name: Install SpatiaLite
run: sudo apt-get update && sudo apt-get install -y libsqlite3-mod-spatialite
run: sudo apt-get install libsqlite3-mod-spatialite
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip

View file

@ -10,19 +10,18 @@ jobs:
runs-on: ${{ matrix.os }}
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15-dev"]
numpy: [0, 1]
os: [ubuntu-latest, macos-latest, windows-latest, macos-14]
steps:
- uses: actions/checkout@v7
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v7
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
allow-prereleases: true
cache: pip
cache-dependency-path: pyproject.toml
check-latest: true
- name: Install dependencies
run: |
pip install . --group dev
@ -31,7 +30,7 @@ jobs:
run: pip install numpy
- name: Install SpatiaLite
if: matrix.os == 'ubuntu-latest'
run: sudo apt-get update && sudo apt-get install -y libsqlite3-mod-spatialite
run: sudo apt-get install libsqlite3-mod-spatialite
- name: Build extension for --load-extension test
if: matrix.os == 'ubuntu-latest'
run: |-
@ -44,9 +43,6 @@ jobs:
run: pytest --sqlite-autocommit
- name: run mypy
run: mypy sqlite_utils tests
- name: run pyright regression checks
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.14'
run: pyright sqlite_utils tests
- name: run flake8
run: flake8
- name: run ty
@ -54,11 +50,6 @@ jobs:
run: |
pip install uv
uv run ty check sqlite_utils
- name: Check no accidental dev= dependencies needed
if: matrix.os == 'ubuntu-latest'
run: |
pip install uv
uv run --no-default-groups sqlite-utils --help
- name: Check formatting
run: black . --check
- name: Check if cog needs to be run

View file

@ -2,21 +2,17 @@
@default: test lint
# Run pytest with supplied options
@test *options: test-no-dev-dependencies
@test *options:
uv run pytest {{options}}
@test-no-dev-dependencies:
uv run --isolated --no-default-groups sqlite-utils --help > /dev/null
@run *options:
uv run -- {{options}}
# Run linters: black, flake8, mypy, pyright, ty, cog
# Run linters: black, flake8, mypy, ty, cog
@lint:
just run black . --check
uv run flake8
uv run mypy sqlite_utils tests
uv run pyright sqlite_utils tests
uv run ty check sqlite_utils
uv run cog --check README.md docs/*.rst
uv run --group docs codespell docs/*.rst --ignore-words docs/codespell-ignore-words.txt

View file

@ -4,40 +4,16 @@
Changelog
===========
.. _v4_2_1:
.. _unreleased:
4.2.1 (2026-08-13)
------------------
- Fix for ``No module named 'typing_extensions'`` crashing bug accidentally shipped in version 4.2. (:issue:`842`)
.. _v4_2:
4.2 (2026-08-13)
----------------
Unreleased
----------
- New ``table.checks``, ``table.column_checks`` and ``table.table_checks`` introspection properties expose column-level and table-level ``CHECK`` constraints. (:issue:`834`)
- New ``sqlite_utils.ANY`` marker type for creating and introspecting SQLite ``ANY`` columns. The Python API and CLI can create, add and transform these columns, and ``table.transform()`` and ``table.extract()`` now preserve ``ANY`` columns and their values in ``STRICT`` tables. (:issue:`790`)
- ``table.default_values`` now unescapes doubled single quotes in string defaults, so a default such as ``'O''Brien'`` is returned as ``"O'Brien"``. Thanks, `ikatyal2110 <https://github.com/ikatyal2110>`__. (`#811 <https://github.com/simonw/sqlite-utils/pull/811>`__)
- ``table.default_values`` now decodes unquoted ``TRUE``, ``FALSE`` and ``NULL`` default literals as ``True``, ``False`` and ``None`` respectively. (:issue:`836`)
- ``table.enable_fts(..., tokenize=...)`` and ``sqlite-utils enable-fts --tokenize`` now safely quote the tokenizer argument, preventing a crafted value from injecting additional SQL. Thanks, `Bunlong Heng <https://github.com/bunlongheng>`__. (`#828 <https://github.com/simonw/sqlite-utils/pull/828>`__)
- ``rows_where()``, ``pks_and_rows_where()``, ``search()`` and ``search_sql()`` now support ``offset=`` without requiring ``limit=``. The ``sqlite-utils rows --offset`` option now works without ``--limit`` too. Thanks, `ethanhawkes-gif <https://github.com/ethanhawkes-gif>`__. (:issue:`816`, `#821 <https://github.com/simonw/sqlite-utils/pull/821>`__)
- Empty or whitespace-only input passed to ``rows_from_file()`` is now handled as an empty CSV file instead of raising ``csv.Error``. Thanks, `Rami Abdelrazzaq <https://github.com/RamiNoodle733>`__. (:issue:`808`, `#837 <https://github.com/simonw/sqlite-utils/pull/837>`__)
- ``sqlite-utils convert --dry-run`` now works for table and column names containing closing square brackets. (:issue:`829`)
- ``table.indexes`` and ``table.xindexes`` now work for table, index and column names containing double quotes. This also fixes ``table.transform()`` for tables with those identifiers. Thanks, `nyxst4ck <https://github.com/nyxst4ck>`__. (:issue:`824`, `#825 <https://github.com/simonw/sqlite-utils/pull/825>`__)
- Improved type annotations throughout the package and added Pyright regression checks to CI. (:issue:`833`)
- Changing a ``TEXT`` column to ``INTEGER``, ``FLOAT`` or ``REAL`` using ``table.transform()`` or ``sqlite-utils transform`` now converts exact empty strings to ``NULL``. Previously they remained empty strings in the numeric column. Thanks, `ikatyal2110 <https://github.com/ikatyal2110>`__. (:issue:`488`, `#805 <https://github.com/simonw/sqlite-utils/pull/805>`__)
``table.transform()`` can handle many more edge-cases:
- ``table.transform()`` now preserves column-level and composite ``UNIQUE`` constraints, including constraint names, collations, sort order and ``ON CONFLICT`` behavior. Renaming columns updates those constraints, while dropping any constituent column removes the entire constraint. (:issue:`762`)
- ``table.transform()`` now preserves ``AUTOINCREMENT`` primary keys and their sequence high-water marks. Previously a transform removed ``AUTOINCREMENT`` and could reuse deleted row IDs. (:issue:`602`)
- ``table.transform()`` now preserves ``CHECK`` constraints, including comments within their expressions. Renaming a column rewrites identifier references in checks without changing string literals or function names. Dropping a column drops a check owned by that column, and raises ``TransformError`` if a remaining check depends on it. (:issue:`762`)
- ``table.transform()`` now preserves comments immediately before or after column definitions. These comments move with the column if it is renamed or reordered, and are removed if the column is dropped. (:issue:`762`)
- ``table.transform(rename=...)`` now preserves explicit indexes on renamed columns by dropping and recreating those indexes against the new column names. Previously this raised a ``TransformError``. (:issue:`822`)
- ``table.transform()`` now works for tables that are referenced by views. Previously the ``ALTER TABLE ... RENAME TO`` step raised ``no such table`` if a view referenced the table being transformed. View definitions are left unchanged - see :ref:`python_api_transform_views`. This also fixes a bug where ``transform(keep_table=...)`` silently rewrote dependent views to point at the frozen backup table instead of the live one. (:issue:`831`)
.. _v3_39_1:
3.39.1 (2026-07-25)

View file

@ -494,7 +494,7 @@ See :ref:`cli_transform_table`.
Options:
--type <TEXT CHOICE>... Change column type to INTEGER, TEXT, FLOAT,
REAL, BLOB or ANY
REAL or BLOB
--drop TEXT Drop this column
--rename <TEXT TEXT>... Rename this column to X
-o, --column-order TEXT Reorder columns
@ -963,7 +963,7 @@ See :ref:`cli_create_table`.
height real \
photo blob --pk id
Valid column types are text, integer, real, float, blob and any.
Valid column types are text, integer, real, float and blob.
Options:
--pk TEXT Column to use as primary key
@ -1257,7 +1257,7 @@ See :ref:`cli_add_column`.
::
Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME
[integer|int|float|real|text|str|blob|bytes|any]
[integer|int|float|real|text|str|blob|bytes]
Add a column to the specified table

View file

@ -1390,14 +1390,7 @@ Use ``--type column-name type`` to override the type automatically chosen when t
This is useful for values such as ZIP codes, which may look like integers but should be stored as ``TEXT`` to preserve leading zeros.
The column type should be one of ``TEXT``, ``INTEGER``, ``FLOAT``, ``REAL``, ``BLOB`` or ``ANY``. Column types are matched case-insensitively.
``ANY`` is especially useful with ``--strict``. An ``ANY`` column in a strict table preserves values without coercion, so text such as ``000123`` remains text instead of being converted to an integer:
.. code-block:: bash
sqlite-utils insert events.db events events.csv --csv --strict \
--type payload any
The column type should be one of ``TEXT``, ``INTEGER``, ``FLOAT``, ``REAL`` or ``BLOB``. Column types are matched case-insensitively.
As with detected column types, ``--type`` only affects tables created by the command. If the table already exists, its existing column types are left unchanged.
@ -2148,12 +2141,6 @@ You can create a table in `SQLite STRICT mode <https://www.sqlite.org/stricttabl
sqlite-utils create-table mydb.db mytable id integer name text --strict
Use the ``any`` type for a strict column that should accept integers, floating point values, text, binary data or null without coercion:
.. code-block:: bash
sqlite-utils create-table events.db events id integer payload any --strict
.. code-block:: bash
sqlite-utils tables mydb.db --schema -t
@ -2236,7 +2223,7 @@ The ``transform`` command allows you to apply complex transformations to a table
Every option for this table (with the exception of ``--pk-none``) can be specified multiple times. The options are as follows:
``--type column-name new-type``
Change the type of the specified column. Valid types are ``integer``, ``text``, ``float``, ``real``, ``blob`` and ``any``. Changing a ``TEXT`` column to ``INTEGER``, ``FLOAT`` or ``REAL`` converts exact empty-string values to ``NULL``.
Change the type of the specified column. Valid types are ``integer``, ``text``, ``float``, ``blob``.
``--drop column-name``
Drop the specified column.

View file

@ -828,19 +828,6 @@ You can pass ``strict=True`` to create a table in ``STRICT`` mode:
"name": str,
}, strict=True)
SQLite ``STRICT`` tables can use the ``ANY`` column type for values that should retain their exact SQLite storage class without coercion. Use the ``sqlite_utils.ANY`` marker type:
.. code-block:: python
import sqlite_utils
db.table("events").create({
"id": int,
"payload": sqlite_utils.ANY,
}, pk="id", strict=True)
An ``ANY`` column can store integers, floating point values, text, binary data or ``None``. In a ``STRICT`` table a text value such as ``"000123"`` remains text with its leading zeroes intact. SQLite also accepts ``ANY`` columns in ordinary non-``STRICT`` tables, but those columns apply numeric affinity and would store that same value as the integer ``123``.
.. note::
In the CLI: :ref:`sqlite-utils create-table <cli_create_table>`
@ -1582,7 +1569,7 @@ You can specify the ``col_type`` argument either using a SQLite type as a string
The ``col_type`` is optional - if you omit it the type of ``TEXT`` will be used.
SQLite types you can specify are ``"TEXT"``, ``"INTEGER"``, ``"FLOAT"``, ``"REAL"``, ``"BLOB"`` or ``"ANY"``. You can use the ``sqlite_utils.ANY`` marker instead of the ``"ANY"`` string.
SQLite types you can specify are ``"TEXT"``, ``"INTEGER"``, ``"FLOAT"``, ``"REAL"`` or ``"BLOB"``.
If you pass a Python type, it will be mapped to SQLite types as shown here::
@ -1595,7 +1582,6 @@ If you pass a Python type, it will be mapped to SQLite types as shown here::
datetime.date: "TEXT"
datetime.time: "TEXT"
datetime.timedelta: "TEXT"
sqlite_utils.ANY: "ANY"
# If numpy is installed
np.int8: "INTEGER"
@ -1826,8 +1812,6 @@ To alter the type of a column, use the ``types=`` argument:
# Convert the 'age' column to an integer, and 'weight' to a float
table.transform(types={"age": int, "weight": float})
When a ``TEXT`` column is changed to ``INTEGER``, ``FLOAT`` or ``REAL``, exact empty-string values are stored as ``NULL``. Other values, including whitespace-only strings, are copied normally.
See :ref:`python_api_add_column` for a list of available types.
.. _python_api_transform_strict:
@ -1847,8 +1831,6 @@ Pass ``strict=False`` to convert a strict table back to a regular non-strict tab
table.transform(strict=False)
If the table has ``ANY`` columns, converting it to non-strict mode can coerce text values that look numeric. For example, SQLite converts ``"000123"`` to the integer ``123`` when copying it into an ordinary ``ANY`` column. This is SQLite's documented distinction between `STRICT and ordinary ANY columns <https://www.sqlite.org/stricttables.html#the_any_datatype>`__.
The default is ``strict=None``, which preserves the table's existing strict mode.
Passing ``strict=True`` raises ``sqlite_utils.db.TransformError`` if the available SQLite version does not support strict tables.
@ -2476,11 +2458,6 @@ The ``.columns_dict`` property returns a dictionary version of the columns with
>>> db.table("PlantType").columns_dict
{'id': <class 'int'>, 'value': <class 'str'>}
SQLite ``ANY`` columns are represented by the ``sqlite_utils.ANY`` marker type::
>>> db.table("events").columns_dict
{'id': <class 'int'>, 'payload': <class 'sqlite_utils.utils.ANY'>}
.. _python_api_introspection_default_values:
.default_values

View file

@ -1,6 +1,6 @@
[project]
name = "sqlite-utils"
version = "4.2.1"
version = "4.1.1"
description = "CLI tool and Python library for manipulating SQLite databases"
readme = { file = "README.md", content-type = "text/markdown" }
authors = [
@ -48,7 +48,6 @@ dev = [
# flake8
"flake8",
"flake8-pyproject",
"pyright>=1.1.411",
"ty>=0.0.37",
# For stable cog:
"tabulate>=0.10.0",

View file

@ -1,13 +1,6 @@
from .db import Database
from .hookspecs import hookimpl, hookspec
from .migrations import Migrations
from .utils import ANY, suggest_column_types
from .utils import suggest_column_types
__all__ = [
"ANY",
"Database",
"Migrations",
"hookimpl",
"hookspec",
"suggest_column_types",
]
__all__ = ["Database", "Migrations", "hookimpl", "hookspec", "suggest_column_types"]

View file

@ -76,7 +76,7 @@ def _close_databases(ctx):
pass
VALID_COLUMN_TYPES = ("INTEGER", "TEXT", "FLOAT", "REAL", "BLOB", "ANY")
VALID_COLUMN_TYPES = ("INTEGER", "TEXT", "FLOAT", "REAL", "BLOB")
UNICODE_ERROR = """
{}
@ -489,17 +489,7 @@ def dump(path, load_extension):
@click.argument(
"col_type",
type=click.Choice(
[
"integer",
"int",
"float",
"real",
"text",
"str",
"blob",
"bytes",
"any",
],
["integer", "int", "float", "real", "text", "str", "blob", "bytes"],
case_sensitive=False,
),
required=False,
@ -1103,7 +1093,7 @@ def insert_upsert_implementation(
column_type_overrides = {column: ctype.upper() for column, ctype in (types or [])}
def _insert_docs(docs, tracker=None):
extra_kwargs: dict[str, Any] = {
extra_kwargs = {
"ignore": ignore,
"replace": replace,
"truncate": truncate,
@ -1768,7 +1758,7 @@ def create_table(
height real \\
photo blob --pk id
Valid column types are text, integer, real, float, blob and any.
Valid column types are text, integer, real, float and blob.
"""
db = sqlite_utils.Database(path)
_register_db_for_cleanup(db)
@ -2488,8 +2478,6 @@ def rows(
if limit:
sql += f" limit {limit}"
if offset:
if not limit:
sql += " limit -1"
sql += f" offset {offset}"
ctx.invoke(
query,
@ -2678,10 +2666,12 @@ def schema(
"--type",
type=(
str,
click.Choice(list(VALID_COLUMN_TYPES), case_sensitive=False),
click.Choice(
["INTEGER", "TEXT", "FLOAT", "REAL", "BLOB"], case_sensitive=False
),
),
multiple=True,
help="Change column type to INTEGER, TEXT, FLOAT, REAL, BLOB or ANY",
help="Change column type to INTEGER, TEXT, FLOAT, REAL or BLOB",
)
@click.option("--drop", type=str, multiple=True, help="Drop this column")
@click.option(
@ -3283,20 +3273,25 @@ def convert(
raise click.ClickException(str(e))
if dry_run:
# Pull first 20 values for first column and preview them
def preview(v):
if multi:
if multi:
def preview(v):
return json.dumps(fn(v), default=repr, ensure_ascii=False) if v else v
return fn(v) if v else v
else:
def preview(v):
return fn(v) if v else v
db.conn.create_function("preview_transform", 1, preview)
sql = """
select
{column} as value,
preview_transform({column}) as preview
from {table}{where} limit 10
[{column}] as value,
preview_transform([{column}]) as preview
from [{table}]{where} limit 10
""".format(
column=quote_identifier(columns[0]),
table=quote_identifier(table),
column=columns[0],
table=table,
where=f" where {where}" if where is not None else "",
)
for row in db.conn.execute(sql, where_args).fetchall():
@ -3791,12 +3786,12 @@ def _rows_from_code(code):
code = pathlib.Path(code).read_text()
except FileNotFoundError:
raise click.ClickException(f"File not found: {code}")
namespace: dict[str, Any] = {}
namespace = {}
try:
exec(code, namespace) # noqa: S102
except SyntaxError as ex:
raise click.ClickException(f"Error in --code: {ex}")
rows: Any = namespace.get("rows")
rows = namespace.get("rows")
if callable(rows):
rows = rows()
if isinstance(rows, dict):

View file

@ -1,4 +1,4 @@
"""Helpers for parsing constraints from SQLite CREATE TABLE SQL.
"""Helpers for parsing CHECK constraints from SQLite CREATE TABLE SQL.
SQLite does not expose CHECK constraints through a pragma, so preserving them
across a table rebuild requires reading ``sqlite_schema.sql``. This module is
@ -32,24 +32,6 @@ class ColumnComments:
after: str = ""
@dataclass(frozen=True)
class UniqueColumn:
name: str
collation: str = ""
order: str = ""
@dataclass
class Unique:
columns: tuple[UniqueColumn, ...]
name: str = ""
column: str = ""
conflict: str = ""
sql: str = field(default="", compare=False, repr=False)
start: int = field(default=-1, compare=False, repr=False)
end: int = field(default=-1, compare=False, repr=False)
class ParseError(ValueError):
pass
@ -582,209 +564,6 @@ def parse_checks(create_sql: str) -> list[Check]:
return checks
def parse_autoincrement(create_sql: str) -> str | None:
"""Return the AUTOINCREMENT column from a valid CREATE TABLE statement."""
body_info = _table_body(create_sql)
if body_info is None:
return None
body, _ = body_info
for item, _, _ in _split_spans(body, _lex(body)):
item_tokens = _meaningful(_lex(item))
if not item_tokens:
continue
head = item_tokens[0]
if (
head.kind == "word" and head.text.upper() in _TABLE_CONSTRAINT_KEYWORDS
) or head.is_keyword("CONSTRAINT"):
continue
column = _unquote(head.text)
index = 1
while index < len(item_tokens):
token = item_tokens[index]
if token.text == "(":
index = _matching_paren(item_tokens, index) + 1
continue
if token.is_keyword("AUTOINCREMENT"):
return column
index += 1
return None
_CONFLICT_ACTIONS = frozenset(("ROLLBACK", "ABORT", "FAIL", "IGNORE", "REPLACE"))
def _conflict_after(tokens: list[_Token], index: int) -> tuple[str, int]:
if index >= len(tokens) or not tokens[index].is_keyword("ON"):
return "", index
if index + 2 >= len(tokens) or not tokens[index + 1].is_keyword("CONFLICT"):
raise ParseError("ON after UNIQUE must be followed by CONFLICT and an action")
action = tokens[index + 2].text.upper()
if tokens[index + 2].kind != "word" or action not in _CONFLICT_ACTIONS:
raise ParseError("Invalid UNIQUE ON CONFLICT action")
return action, index + 3
def _unique_columns(
item: str, tokens: list[_Token], open_index: int
) -> tuple[tuple[UniqueColumn, ...], int]:
close = _matching_paren(tokens, open_index)
inner = item[tokens[open_index].end : tokens[close].start]
columns: list[UniqueColumn] = []
for raw_column in _split_ranges(inner, _lex(inner)):
column_tokens = _meaningful(_lex(raw_column))
if not column_tokens or column_tokens[0].kind not in (
"word",
"identifier",
"string",
):
raise ParseError("UNIQUE constraint has an invalid column")
name = _unquote(column_tokens[0].text)
collation = ""
order = ""
index = 1
if index < len(column_tokens) and column_tokens[index].is_keyword("COLLATE"):
if index + 1 >= len(column_tokens):
raise ParseError("COLLATE in UNIQUE constraint is missing its name")
collation = _unquote(column_tokens[index + 1].text)
index += 2
if index < len(column_tokens) and (
column_tokens[index].is_keyword("ASC")
or column_tokens[index].is_keyword("DESC")
):
order = column_tokens[index].text.upper()
index += 1
if index != len(column_tokens):
raise ParseError("UNIQUE constraint has an invalid indexed column")
columns.append(UniqueColumn(name, collation=collation, order=order))
if not columns:
raise ParseError("UNIQUE constraint must include at least one column")
return tuple(columns), close + 1
def _column_uniques(
item: str, tokens: list[_Token], column: str, base_offset: int
) -> list[Unique]:
uniques: list[Unique] = []
collation = ""
collation_index = 1
while collation_index < len(tokens):
token = tokens[collation_index]
if token.text == "(":
collation_index = _matching_paren(tokens, collation_index) + 1
continue
if token.is_keyword("COLLATE"):
if collation_index + 1 >= len(tokens):
raise ParseError("COLLATE is missing its name")
collation = _unquote(tokens[collation_index + 1].text)
collation_index += 2
continue
collation_index += 1
pending_name = ""
pending_start: int | None = None
index = 1
while index < len(tokens):
token = tokens[index]
if token.text == "(":
index = _matching_paren(tokens, index) + 1
continue
if token.is_keyword("CONSTRAINT"):
if index + 1 >= len(tokens):
raise ParseError("CONSTRAINT is missing its name")
pending_name = _unquote(tokens[index + 1].text)
pending_start = index
index += 2
continue
if token.is_keyword("UNIQUE"):
source_start = tokens[
pending_start if pending_start is not None else index
].start
conflict, next_index = _conflict_after(tokens, index + 1)
source_end = tokens[next_index - 1].end
uniques.append(
Unique(
(UniqueColumn(column, collation=collation),),
name=pending_name,
column=column,
conflict=conflict,
sql=item[source_start:source_end],
start=base_offset + source_start,
end=base_offset + source_end,
)
)
pending_name = ""
pending_start = None
index = next_index
continue
if (
token.kind == "word"
and token.text.upper() in _OTHER_COLUMN_CONSTRAINT_KEYWORDS
):
pending_name = ""
pending_start = None
index += 1
return uniques
def parse_uniques(create_sql: str) -> list[Unique]:
"""Return column-level and table-level UNIQUE constraints."""
body_info = _table_body(create_sql)
if body_info is None:
return []
body, body_start = body_info
uniques: list[Unique] = []
for item, item_start, _ in _split_spans(body, _lex(body)):
item_tokens = _meaningful(_lex(item))
if not item_tokens:
continue
item_index = 0
constraint_name = ""
if item_tokens[item_index].is_keyword("CONSTRAINT"):
if len(item_tokens) < 2:
raise ParseError("CONSTRAINT is missing its name")
constraint_name = _unquote(item_tokens[1].text)
item_index = 2
head = item_tokens[item_index] if item_index < len(item_tokens) else None
if head and head.is_keyword("UNIQUE"):
if (
item_index + 1 >= len(item_tokens)
or item_tokens[item_index + 1].text != "("
):
raise ParseError("Table UNIQUE must be followed by a column list")
columns, next_index = _unique_columns(item, item_tokens, item_index + 1)
conflict, next_index = _conflict_after(item_tokens, next_index)
if next_index != len(item_tokens):
raise ParseError("Unexpected SQL after UNIQUE constraint")
source_start = item_tokens[0].start
source_end = item_tokens[next_index - 1].end
uniques.append(
Unique(
columns,
name=constraint_name,
conflict=conflict,
sql=item[source_start:source_end],
start=body_start + item_start + source_start,
end=body_start + item_start + source_end,
)
)
continue
if (
head
and head.kind == "word"
and head.text.upper() in _TABLE_CONSTRAINT_KEYWORDS
):
continue
column = _unquote(item_tokens[0].text)
uniques.extend(
_column_uniques(
item,
item_tokens,
column,
body_start + item_start,
)
)
return uniques
def parse_column_comments(create_sql: str) -> dict[str, ColumnComments]:
"""Return comments immediately before and after each column definition."""
body_info = _table_body(create_sql)

View file

@ -18,30 +18,26 @@ from dataclasses import dataclass, field
from types import TracebackType
from typing import (
Any,
TypeVar,
Union,
cast,
)
from sqlite_fts4 import rank_bm25
from typing_extensions import Self
from sqlite_utils.plugins import ensure_plugins_loaded, pm
from .create_table_parser import (
Check,
ColumnComments,
ParseError,
Unique,
UniqueColumn,
check_references_identifier,
parse_autoincrement,
parse_checks,
parse_column_comments,
parse_uniques,
rewrite_check_expression,
sql_ends_in_line_comment,
)
from .utils import (
ANY,
OperationalError,
chunks,
column_affinity,
@ -105,24 +101,6 @@ def _check_constraint_sql(check: Check) -> str:
return f"{prefix}CHECK ({check.check}{newline})"
def _unique_constraint_sql(unique: Unique) -> str:
prefix = f"CONSTRAINT {quote_identifier(unique.name)} " if unique.name else ""
if unique.column:
constraint = "UNIQUE"
else:
columns = []
for column in unique.columns:
column_sql = quote_identifier(column.name)
if column.collation:
column_sql += f" COLLATE {quote_identifier(column.collation)}"
if column.order:
column_sql += f" {column.order}"
columns.append(column_sql)
constraint = "UNIQUE ({})".format(", ".join(columns))
conflict = f" ON CONFLICT {unique.conflict}" if unique.conflict else ""
return f"{prefix}{constraint}{conflict}"
def _column_definition_with_comments(
definition: str, comments: ColumnComments | None
) -> str:
@ -278,8 +256,8 @@ class ForeignKey:
column: str | None = field(compare=False)
other_table: str
other_column: str | None = field(compare=False)
columns: tuple[str, ...] | list[str] = ()
other_columns: tuple[str, ...] | list[str] = ()
columns: tuple[str, ...] = ()
other_columns: tuple[str, ...] = ()
is_compound: bool = False
on_delete: str = "NO ACTION"
on_update: str = "NO ACTION"
@ -342,8 +320,6 @@ ForeignKeyIndicator = (
ForeignKeysType = Iterable[ForeignKeyIndicator] | list[ForeignKeyIndicator]
PrimaryKey = str | tuple[str, ...] | list[str]
class Default:
pass
@ -351,8 +327,6 @@ class Default:
DEFAULT = Default()
T = TypeVar("T")
Tracer = Callable[[str, Sequence[Any] | dict[str, Any] | None], None]
@ -387,7 +361,6 @@ COLUMN_TYPE_MAPPING: dict[Any, str] = {
decimal.Decimal: "REAL",
None.__class__: "TEXT",
uuid.UUID: "TEXT",
ANY: "ANY",
# SQLite explicit types
"TEXT": "TEXT",
"INTEGER": "INTEGER",
@ -402,8 +375,6 @@ COLUMN_TYPE_MAPPING: dict[Any, str] = {
"real": "REAL",
"blob": "BLOB",
"bytes": "BLOB",
"ANY": "ANY",
"any": "ANY",
}
# If numpy is available, add more types
if np:
@ -635,7 +606,7 @@ class Database:
pm.hook.prepare_connection(conn=self.conn)
self.strict = strict
def __enter__(self):
def __enter__(self) -> Self:
return self
def __exit__(
@ -1442,8 +1413,6 @@ class Database:
strict: bool = False,
_checks: Iterable[Check] | None = None,
_column_comments: Mapping[str, ColumnComments] | None = None,
_autoincrement: str | None = None,
_uniques: Iterable[Unique] | None = None,
) -> str:
"""
Returns the SQL ``CREATE TABLE`` statement for creating the specified table.
@ -1506,60 +1475,6 @@ class Database:
checks_by_column.setdefault(column, []).append(check)
else:
table_checks.append(check)
uniques_by_column: dict[str, list[Unique]] = {}
table_uniques: list[Unique] = []
for unique in _uniques or ():
resolved_unique = Unique(
tuple(
UniqueColumn(
resolve_casing(column.name, columns),
collation=column.collation,
order=column.order,
)
for column in unique.columns
),
name=unique.name,
column=(
resolve_casing(unique.column, columns) if unique.column else ""
),
conflict=unique.conflict,
)
missing = [
column.name
for column in resolved_unique.columns
if column.name not in columns
]
if missing:
raise AlterError(
"No such column for UNIQUE constraint: {}".format(
", ".join(missing)
)
)
if resolved_unique.column:
if (
len(resolved_unique.columns) != 1
or resolved_unique.columns[0].name != resolved_unique.column
):
raise AlterError("Invalid column-level UNIQUE constraint")
if any(
column.collation or column.order
for column in resolved_unique.columns
):
# Render this as a table constraint so the collation or sort
# order that governs uniqueness can be represented explicitly.
table_uniques.append(
Unique(
resolved_unique.columns,
name=resolved_unique.name,
conflict=resolved_unique.conflict,
)
)
else:
uniques_by_column.setdefault(resolved_unique.column, []).append(
resolved_unique
)
else:
table_uniques.append(resolved_unique)
if not columns:
raise ValueError("Tables must have at least one column")
if not all(n in columns for n in not_null):
@ -1601,22 +1516,10 @@ class Database:
column_items.insert(0, (pk, int))
elif pk:
pk = [resolve_casing(p, [c[0] for c in column_items]) for p in pk]
if _autoincrement is not None:
_autoincrement = resolve_casing(
_autoincrement, [c[0] for c in column_items]
)
if _autoincrement != single_pk:
raise ValueError("AUTOINCREMENT requires a single-column primary key")
for column_name, column_type in column_items:
column_extras = []
if column_name == single_pk:
column_extras.append("PRIMARY KEY")
if column_name == _autoincrement:
if COLUMN_TYPE_MAPPING[column_type] != "INTEGER":
raise ValueError(
"AUTOINCREMENT requires an INTEGER PRIMARY KEY column"
)
column_extras.append("AUTOINCREMENT")
if column_name in not_null:
column_extras.append("NOT NULL")
if column_name in defaults and defaults[column_name] is not None:
@ -1628,10 +1531,6 @@ class Database:
column_extras.append(
f"REFERENCES {quote_identifier(fk.other_table)}({quote_identifier(cast(str, fk.other_column))}){_fk_actions_sql(fk)}"
)
column_extras.extend(
_unique_constraint_sql(unique)
for unique in uniques_by_column.get(column_name, ())
)
column_extras.extend(
_check_constraint_sql(check)
for check in checks_by_column.get(column_name, ())
@ -1678,9 +1577,6 @@ class Database:
actions=_fk_actions_sql(fk),
)
)
column_defs.extend(
f" {_unique_constraint_sql(unique)}" for unique in table_uniques
)
column_defs.extend(
f" {_check_constraint_sql(check)}" for check in table_checks
)
@ -1957,8 +1853,8 @@ class Database:
fk_object = self._resolve_foreign_key_casing(
fk_object, table_obj.columns_dict
)
columns = tuple(fk_object.columns)
other_columns = tuple(fk_object.other_columns)
columns = fk_object.columns
other_columns = fk_object.other_columns
for column in columns:
if column not in table_obj.columns_dict:
raise AlterError(f"No such column: {column} in {table}")
@ -2018,10 +1914,9 @@ class Database:
existing_indexes = {tuple(i.columns) for i in table.indexes}
for fk in table.foreign_keys:
# A compound foreign key gets a single composite index
fk_columns = tuple(fk.columns)
if fk_columns not in existing_indexes:
if fk.columns not in existing_indexes:
table.create_index(fk.columns, find_unique_name=True)
existing_indexes.add(fk_columns)
existing_indexes.add(fk.columns)
def vacuum(self) -> None:
"Run a SQLite ``VACUUM`` against the database."
@ -2169,10 +2064,6 @@ class Queryable:
if limit is not None:
sql += f" limit {limit}"
if offset is not None:
# SQLite requires a limit clause before offset - a negative limit
# means "no upper bound", so offset works without an explicit limit
if limit is None:
sql += " limit -1"
sql += f" offset {offset}"
cursor = self.db.execute(sql, where_args or [])
columns = dedupe_keys(c[0] for c in cursor.description)
@ -2474,11 +2365,14 @@ class Table(Queryable):
@property
def indexes(self) -> list[Index]:
"List of indexes defined on this table."
sql = f"PRAGMA index_list({quote_identifier(self.name)})"
sql = f'PRAGMA index_list("{self.name}")'
indexes = []
for row in self.db.execute_returning_dicts(sql):
index_name = row["name"]
column_sql = f"PRAGMA index_info({quote_identifier(index_name)})"
index_name_quoted = (
f'"{index_name}"' if not index_name.startswith('"') else index_name
)
column_sql = f"PRAGMA index_info({index_name_quoted})"
columns = []
for seqno, cid, name in self.db.execute(column_sql).fetchall():
columns.append(name)
@ -2493,11 +2387,14 @@ class Table(Queryable):
@property
def xindexes(self) -> list[XIndex]:
"List of indexes defined on this table using the more detailed ``XIndex`` format."
sql = f"PRAGMA index_list({quote_identifier(self.name)})"
sql = f'PRAGMA index_list("{self.name}")'
indexes = []
for row in self.db.execute_returning_dicts(sql):
index_name = row["name"]
column_sql = f"PRAGMA index_xinfo({quote_identifier(index_name)})"
index_name_quoted = (
f'"{index_name}"' if not index_name.startswith('"') else index_name
)
column_sql = f"PRAGMA index_xinfo({index_name_quoted})"
index_columns = []
for info in self.db.execute(column_sql).fetchall():
index_columns.append(XIndexColumn(*info))
@ -2552,7 +2449,7 @@ class Table(Queryable):
replace: bool = False,
ignore: bool = False,
transform: bool = False,
strict: bool | Default | None = DEFAULT,
strict: bool | Default = DEFAULT,
) -> "Table":
"""
Create a table with the specified columns.
@ -2623,7 +2520,7 @@ class Table(Queryable):
replace=replace,
ignore=ignore,
transform=transform,
strict=cast(bool, strict),
strict=strict, # type: ignore[arg-type]
)
return self
@ -2843,8 +2740,6 @@ class Table(Queryable):
try:
existing_checks = self.checks
existing_column_comments = parse_column_comments(self.schema)
existing_autoincrement = parse_autoincrement(self.schema)
existing_uniques = parse_uniques(self.schema)
except ParseError as ex:
raise TransformError(
f"Could not parse table schema for table {self.name!r}: {ex}"
@ -2871,37 +2766,6 @@ class Table(Queryable):
)
)
create_table_uniques: list[Unique] = []
for unique in existing_uniques:
columns = tuple(
UniqueColumn(
resolve_casing(column.name, existing_columns),
collation=column.collation,
order=column.order,
)
for column in unique.columns
)
if any(column.name in drop for column in columns):
continue
owner = (
resolve_casing(unique.column, existing_columns) if unique.column else ""
)
create_table_uniques.append(
Unique(
tuple(
UniqueColumn(
rename.get(column.name) or column.name,
collation=column.collation,
order=column.order,
)
for column in columns
),
name=unique.name,
column=rename.get(owner) or owner,
conflict=unique.conflict,
)
)
create_table_column_comments: dict[str, ColumnComments] = {}
for column, comments in existing_column_comments.items():
owner = resolve_casing(column, existing_columns)
@ -2992,17 +2856,12 @@ class Table(Queryable):
for name, type_ in current_column_pairs:
type_ = types.get(name) or type_
if name in drop:
del copy_from_to[name]
del [copy_from_to[name]]
continue
new_name = rename.get(name) or name
new_column_pairs.append((new_name, type_))
copy_from_to[name] = new_name
if existing_autoincrement:
existing_autoincrement = resolve_casing(
existing_autoincrement, existing_columns
)
if pk is DEFAULT:
pks_renamed = tuple(
rename.get(pk_name) or pk_name
@ -3013,28 +2872,6 @@ class Table(Queryable):
else:
pk = pks_renamed
create_table_autoincrement = None
if existing_autoincrement and existing_autoincrement not in drop:
renamed_autoincrement = (
rename.get(existing_autoincrement) or existing_autoincrement
)
single_pk = pk[0] if isinstance(pk, (list, tuple)) and len(pk) == 1 else pk
new_column_types = dict(new_column_pairs)
if (
single_pk == renamed_autoincrement
and COLUMN_TYPE_MAPPING.get(new_column_types.get(renamed_autoincrement))
== "INTEGER"
):
create_table_autoincrement = renamed_autoincrement
autoincrement_sequence = None
if create_table_autoincrement:
sequence_row = self.db.execute(
"SELECT seq FROM sqlite_sequence WHERE name = ?", [self.name]
).fetchone()
if sequence_row is not None:
autoincrement_sequence = sequence_row[0]
# not_null may be a set or dict, need to convert to a set
create_table_not_null = {
rename.get(c.name) or c.name
@ -3086,24 +2923,9 @@ class Table(Queryable):
strict=self.strict if strict is None else strict,
_checks=create_table_checks,
_column_comments=create_table_column_comments,
_autoincrement=create_table_autoincrement,
_uniques=create_table_uniques,
).strip()
)
# Columns being changed from TEXT to a numeric type: coerce empty strings to NULL
_numeric_sql_types = {"INTEGER", "REAL", "FLOAT", "NUMERIC"}
text_to_numeric_cols = {
col_name
for col_name, new_type in types.items()
if existing_columns.get(col_name) == str
and COLUMN_TYPE_MAPPING.get(
new_type,
new_type.upper() if isinstance(new_type, str) else "",
)
in _numeric_sql_types
}
# Copy across data, respecting any renamed columns
new_cols = []
old_cols = []
@ -3114,96 +2936,13 @@ class Table(Queryable):
if "rowid" not in new_cols:
new_cols.insert(0, "rowid")
old_cols.insert(0, "rowid")
def _copy_expr(col):
if col in text_to_numeric_cols:
return "NULLIF({}, '')".format(quote_identifier(col))
return quote_identifier(col)
copy_sql = "INSERT INTO {} ({new_cols})\n SELECT {old_cols} FROM {};".format(
quote_identifier(new_table_name),
quote_identifier(self.name),
old_cols=", ".join(_copy_expr(col) for col in old_cols),
old_cols=", ".join(quote_identifier(col) for col in old_cols),
new_cols=", ".join(quote_identifier(col) for col in new_cols),
)
sqls.append(copy_sql)
# Capture indexes before the old table is changed. Simple indexes that
# reference renamed columns are recreated from structured PRAGMA
# metadata instead of editing their stored CREATE INDEX SQL.
index_drop_sqls = []
index_create_sqls = []
xindexes_by_name = {index.name: index for index in self.xindexes}
for index in self.indexes:
if index.origin == "pk":
continue
index_sql = self.db.execute(
"""SELECT sql FROM sqlite_master WHERE type = 'index' AND name = :index_name;""",
{"index_name": index.name},
).fetchall()[0][0]
if index_sql is None:
if index.origin == "u":
# UNIQUE constraints are reproduced in CREATE TABLE above.
continue
raise TransformError(
f"Index '{index.name}' on table '{self.name}' does not have a "
"CREATE INDEX statement. You must manually drop this index prior to running this "
"transformation and manually recreate the new index after running this transformation."
)
dropped_index_column = next(
(column for column in index.columns if column in drop), None
)
renamed_index_column = next(
(column for column in index.columns if column in rename), None
)
if dropped_index_column is not None:
raise TransformError(
f"Index '{index.name}' column '{dropped_index_column}' is not in updated table '{self.name}'. "
f"You must manually drop this index prior to running this transformation "
f"and manually recreate the new index after running this transformation. "
f"The original index sql statement is: `{index_sql}`. No changes have been applied to this table."
)
xindex = xindexes_by_name[index.name]
indexed_columns = sorted(
(column for column in xindex.columns if column.key),
key=lambda column: column.seqno,
)
if (rename or drop) and (
index.partial or any(column.name is None for column in indexed_columns)
):
raise TransformError(
f"Index '{index.name}' is a partial or expression index, so it "
f"cannot be safely recreated while columns are renamed or dropped. "
f"You must manually drop this index prior to running this transformation "
f"and manually recreate the new index after running this transformation. "
f"The original index sql statement is: `{index_sql}`. No changes have been applied to this table."
)
if renamed_index_column is not None:
columns_sql = []
for column in indexed_columns:
assert column.name is not None
column_sql = quote_identifier(
rename.get(column.name) or column.name
)
if column.coll and column.coll.upper() != "BINARY":
column_sql += f" COLLATE {quote_identifier(column.coll)}"
if column.desc:
column_sql += " DESC"
columns_sql.append(column_sql)
index_sql = "CREATE {unique}INDEX {index_name} ON {table_name} ({columns})".format(
unique="UNIQUE " if index.unique else "",
index_name=quote_identifier(index.name),
table_name=quote_identifier(self.name),
columns=", ".join(columns_sql),
)
index_drop_sqls.append(
f"DROP INDEX IF EXISTS {quote_identifier(index.name)};"
)
elif keep_table:
index_drop_sqls.append(
f"DROP INDEX IF EXISTS {quote_identifier(index.name)};"
)
index_create_sqls.append(index_sql)
sqls.extend(index_drop_sqls)
# Drop (or keep) the old table, then rename the new one into place.
# Since SQLite 3.25 ALTER TABLE ... RENAME TO rewrites references to
# the renamed table in every view definition, which fails if a view
@ -3232,25 +2971,30 @@ class Table(Queryable):
"ON" if legacy_alter_table_was_on else "OFF"
)
)
if autoincrement_sequence is not None:
table_name_literal = self.db.quote(self.name)
sqls.extend(
(
"UPDATE sqlite_sequence SET seq = MAX(seq, {sequence}) "
"WHERE name = {table_name};".format(
sequence=autoincrement_sequence,
table_name=table_name_literal,
),
"INSERT INTO sqlite_sequence (name, seq) "
"SELECT {table_name}, {sequence} WHERE NOT EXISTS "
"(SELECT 1 FROM sqlite_sequence WHERE name = {table_name});".format(
sequence=autoincrement_sequence,
table_name=table_name_literal,
),
)
)
# Re-add existing indexes
sqls.extend(index_create_sqls)
for index in self.indexes:
if index.origin != "pk":
index_sql = self.db.execute(
"""SELECT sql FROM sqlite_master WHERE type = 'index' AND name = :index_name;""",
{"index_name": index.name},
).fetchall()[0][0]
if index_sql is None:
raise TransformError(
f"Index '{index.name}' on table '{self.name}' does not have a "
"CREATE INDEX statement. You must manually drop this index prior to running this "
"transformation and manually recreate the new index after running this transformation."
)
if keep_table:
sqls.append(f"DROP INDEX IF EXISTS {quote_identifier(index.name)};")
for col in index.columns:
if col in rename or col in drop:
raise TransformError(
f"Index '{index.name}' column '{col}' is not in updated table '{self.name}'. "
f"You must manually drop this index prior to running this transformation "
f"and manually recreate the new index after running this transformation. "
f"The original index sql statement is: `{index_sql}`. No changes have been applied to this table."
)
sqls.append(index_sql)
return sqls
def extract(
@ -3292,15 +3036,6 @@ class Table(Queryable):
if col in columns
}
if lookup_table.exists():
if (
self.strict
and ANY in lookup_columns_definition.values()
and not lookup_table.strict
):
raise InvalidColumns(
f"Lookup table {table} already exists but is not STRICT, "
"so it cannot preserve ANY column values"
)
if not set(lookup_columns_definition.items()).issubset(
lookup_table.columns_dict.items()
):
@ -3314,7 +3049,6 @@ class Table(Queryable):
**lookup_columns_definition,
},
pk="id",
strict=self.strict,
)
lookup_columns = [(rename.get(col) or col) for col in columns]
lookup_table.create_index(lookup_columns, unique=True, if_not_exists=True)
@ -3605,10 +3339,7 @@ class Table(Queryable):
:param on_update: ``ON UPDATE`` action for the foreign key.
"""
columns = (column,) if isinstance(column, str) else tuple(column)
if not columns:
raise ValueError("column must contain at least one column name")
columns = tuple(resolve_casing(c, self.columns_dict) for c in columns)
assert columns
# Ensure columns exist
for col in columns:
if col not in self.columns_dict:
@ -3619,7 +3350,7 @@ class Table(Queryable):
raise ValueError(
"other_table must be specified for a compound foreign key"
)
other_table = self.guess_foreign_table(next(iter(columns)))
other_table = self.guess_foreign_table(columns[0])
# If other_column is not specified, detect the primary key on other_table
if other_column is None:
if len(columns) > 1:
@ -3783,9 +3514,7 @@ class Table(Queryable):
table_fts=quote_identifier(self.name + "_fts"),
columns=", ".join(quote_identifier(c) for c in columns),
fts_version=fts_version,
tokenize=(
f"\n tokenize={self.db.quote(tokenize)}," if tokenize else ""
),
tokenize=f"\n tokenize='{tokenize}'," if tokenize else "",
)
)
should_recreate = False
@ -4001,8 +3730,6 @@ class Table(Queryable):
if limit is not None:
limit_offset += f" limit {limit}"
if offset is not None:
if limit is None:
limit_offset += " limit -1"
limit_offset += f" offset {offset}"
return sql.format(
dbtable=quote_identifier(self.name),
@ -4066,10 +3793,8 @@ class Table(Queryable):
for row in cursor:
yield dict(zip(columns, row))
def value_or_default(self, key: str, value: T | Default) -> T:
if value is DEFAULT:
return cast(T, self._defaults[key])
return cast(T, value)
def value_or_default(self, key: str, value: Any) -> Any:
return self._defaults[key] if value is DEFAULT else value
def delete(self, pk_values: list | tuple | str | float) -> "Table":
"""
@ -4254,7 +3979,7 @@ class Table(Queryable):
def _convert_multi(
self, column, fn, drop, show_progress, where=None, where_args=None
) -> "Table":
):
# First we execute the function
pk_to_values = {}
new_column_types: dict[str, set[type]] = {}
@ -4300,7 +4025,6 @@ class Table(Queryable):
bar.update(1)
if drop:
self.transform(drop=(column,))
return self
def build_insert_queries_and_params(
self,
@ -4604,8 +4328,8 @@ class Table(Queryable):
def insert(
self,
record: dict[str, Any],
pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys: ForeignKeysType | Default | None = DEFAULT,
pk=DEFAULT,
foreign_keys=DEFAULT,
column_order: list[str] | Default | None = DEFAULT,
not_null: Iterable[str] | Default | None = DEFAULT,
defaults: dict[str, Any] | Default | None = DEFAULT,
@ -4673,24 +4397,24 @@ class Table(Queryable):
def insert_all(
self,
records: Iterable[dict[str, Any]] | Iterable[Sequence[Any]],
pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order: list[str] | Default | None = DEFAULT,
not_null: Iterable[str] | Default | None = DEFAULT,
defaults: dict[str, Any] | Default | None = DEFAULT,
batch_size: int | Default = DEFAULT,
hash_id: str | Default | None = DEFAULT,
hash_id_columns: Iterable[str] | Default | None = DEFAULT,
alter: bool | Default | None = DEFAULT,
ignore: bool | Default | None = DEFAULT,
replace: bool | Default | None = DEFAULT,
truncate: bool = False,
extracts: dict[str, str] | list[str] | Default | None = DEFAULT,
conversions: dict[str, str] | Default | None = DEFAULT,
columns: dict[str, Any] | Default | None = DEFAULT,
upsert: bool = False,
analyze: bool = False,
strict: bool | Default | None = DEFAULT,
pk=DEFAULT,
foreign_keys=DEFAULT,
column_order=DEFAULT,
not_null=DEFAULT,
defaults=DEFAULT,
batch_size=DEFAULT,
hash_id=DEFAULT,
hash_id_columns=DEFAULT,
alter=DEFAULT,
ignore=DEFAULT,
replace=DEFAULT,
truncate=False,
extracts=DEFAULT,
conversions=DEFAULT,
columns=DEFAULT,
upsert=False,
analyze=False,
strict=DEFAULT,
) -> "Table":
"""
Like ``.insert()`` but takes a list of records and ensures that the table
@ -4983,7 +4707,6 @@ class Table(Queryable):
elif isinstance(pk, str):
self.last_pk = row[resolve_casing(pk, row)]
else:
assert pk is not None
self.last_pk = tuple(
row[resolve_casing(p, row)] for p in pk
)
@ -5001,7 +4724,6 @@ class Table(Queryable):
pk_index = column_names.index(resolve_casing(pk, column_names))
self.last_pk = first_record_list[pk_index]
else:
assert pk is not None
self.last_pk = tuple(
first_record_list[
column_names.index(resolve_casing(p, column_names))
@ -5013,7 +4735,6 @@ class Table(Queryable):
if hash_id:
self.last_pk = hash_record(first_record_dict, hash_id_columns)
else:
assert pk is not None
self.last_pk = (
first_record_dict[resolve_casing(pk, first_record_dict)]
if isinstance(pk, str)
@ -5030,19 +4751,19 @@ class Table(Queryable):
def upsert(
self,
record: dict[str, Any],
pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order: list[str] | Default | None = DEFAULT,
not_null: Iterable[str] | Default | None = DEFAULT,
defaults: dict[str, Any] | Default | None = DEFAULT,
hash_id: str | Default | None = DEFAULT,
hash_id_columns: Iterable[str] | Default | None = DEFAULT,
alter: bool | Default | None = DEFAULT,
extracts: dict[str, str] | list[str] | Default | None = DEFAULT,
conversions: dict[str, str] | Default | None = DEFAULT,
columns: dict[str, Any] | Default | None = DEFAULT,
strict: bool | Default | None = DEFAULT,
record,
pk=DEFAULT,
foreign_keys=DEFAULT,
column_order=DEFAULT,
not_null=DEFAULT,
defaults=DEFAULT,
hash_id=DEFAULT,
hash_id_columns=DEFAULT,
alter=DEFAULT,
extracts=DEFAULT,
conversions=DEFAULT,
columns=DEFAULT,
strict=DEFAULT,
) -> "Table":
"""
Like ``.insert()`` but performs an ``UPSERT``, where records are inserted if they do
@ -5069,20 +4790,20 @@ class Table(Queryable):
def upsert_all(
self,
records: Iterable[dict[str, Any]] | Iterable[Sequence[Any]],
pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order: list[str] | Default | None = DEFAULT,
not_null: Iterable[str] | Default | None = DEFAULT,
defaults: dict[str, Any] | Default | None = DEFAULT,
batch_size: int | Default = DEFAULT,
hash_id: str | Default | None = DEFAULT,
hash_id_columns: Iterable[str] | Default | None = DEFAULT,
alter: bool | Default | None = DEFAULT,
extracts: dict[str, str] | list[str] | Default | None = DEFAULT,
conversions: dict[str, str] | Default | None = DEFAULT,
columns: dict[str, Any] | Default | None = DEFAULT,
analyze: bool = False,
strict: bool | Default | None = DEFAULT,
pk=DEFAULT,
foreign_keys=DEFAULT,
column_order=DEFAULT,
not_null=DEFAULT,
defaults=DEFAULT,
batch_size=DEFAULT,
hash_id=DEFAULT,
hash_id_columns=DEFAULT,
alter=DEFAULT,
extracts=DEFAULT,
conversions=DEFAULT,
columns=DEFAULT,
analyze=False,
strict=DEFAULT,
) -> "Table":
"""
Like ``.upsert()`` but can be applied to a list of records.
@ -5532,13 +5253,6 @@ def _decode_default_value(value: str) -> object:
# It's a binary string, stored as hex
to_decode = value[2:-1]
return binascii.unhexlify(to_decode)
upper = value.upper()
if upper == "TRUE":
return True
if upper == "FALSE":
return False
if upper == "NULL":
return None
# If it is a string containing a floating point number:
try:
return float(value)

View file

@ -14,7 +14,6 @@ from typing import (
TYPE_CHECKING,
Any,
BinaryIO,
Generic,
TypeVar,
Union,
cast,
@ -59,10 +58,6 @@ Row = dict[str, RowValue]
T = TypeVar("T")
class ANY:
"""Marker type for an SQLite ``ANY`` column."""
class _CloseableIterator(Iterator[Row]):
"""Iterator wrapper that closes a file when iteration is complete."""
@ -182,8 +177,6 @@ def column_affinity(column_type: str) -> type:
return bytes
if "REAL" in column_type or "FLOA" in column_type or "DOUB" in column_type:
return float
if column_type == "ANY":
return ANY
# Default is 'NUMERIC', which we currently also treat as float
return float
@ -351,11 +344,7 @@ def rows_from_file(
reader = csv.DictReader(decoded_fp, dialect=dialect)
else:
reader = csv.DictReader(decoded_fp)
rows = _extra_key_strategy(
cast(Iterable[dict[str | None, object]], reader),
ignore_extras,
extras_key,
)
rows = _extra_key_strategy(reader, ignore_extras, extras_key)
return _CloseableIterator(iter(rows), decoded_fp), Format.CSV
elif format == Format.TSV:
rows, _ = rows_from_file(
@ -379,8 +368,6 @@ def rows_from_file(
raise TypeError(
"rows_from_file() requires a file-like object that supports peek(), such as io.BytesIO"
)
if not first_bytes:
return (), Format.CSV
if first_bytes.startswith((b"[", b"{")):
# TODO: Detect newline-JSON
return rows_from_file(buffered, format=Format.JSON)
@ -500,12 +487,12 @@ class ValueTracker:
del self.couldbe[key]
class NullProgressBar(Generic[T]):
class NullProgressBar:
def __init__(self, *args: Iterable[T]) -> None:
self.args = args
def __iter__(self) -> Iterator[T]:
yield from self.args[0]
yield from self.args[0] # type: ignore
def update(self, value: int) -> None:
pass

View file

@ -3,13 +3,11 @@ import pytest
@pytest.fixture
def db(fresh_db):
fresh_db.table("one_index").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.table("one_index").create_index(["name"])
fresh_db.table("two_indexes").insert(
{"id": 1, "name": "Cleo", "species": "dog"}, pk="id"
)
fresh_db.table("two_indexes").create_index(["name"])
fresh_db.table("two_indexes").create_index(["species"])
fresh_db["one_index"].insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db["one_index"].create_index(["name"])
fresh_db["two_indexes"].insert({"id": 1, "name": "Cleo", "species": "dog"}, pk="id")
fresh_db["two_indexes"].create_index(["name"])
fresh_db["two_indexes"].create_index(["species"])
return fresh_db
@ -19,7 +17,7 @@ def test_analyze_whole_database(db):
assert set(db.table_names()).issuperset(
{"one_index", "two_indexes", "sqlite_stat1"}
)
assert list(db.table("sqlite_stat1").rows) == [
assert list(db["sqlite_stat1"].rows) == [
{"tbl": "two_indexes", "idx": "idx_two_indexes_species", "stat": "1 1"},
{"tbl": "two_indexes", "idx": "idx_two_indexes_name", "stat": "1 1"},
{"tbl": "one_index", "idx": "idx_one_index_name", "stat": "1 1"},
@ -32,12 +30,12 @@ def test_analyze_one_table(db, method):
if method == "db_method_with_name":
db.analyze("one_index")
elif method == "table_method":
db.table("one_index").analyze()
db["one_index"].analyze()
assert set(db.table_names()).issuperset(
{"one_index", "two_indexes", "sqlite_stat1"}
)
assert list(db.table("sqlite_stat1").rows) == [
assert list(db["sqlite_stat1"].rows) == [
{"tbl": "one_index", "idx": "idx_one_index_name", "stat": "1 1"}
]
@ -48,6 +46,6 @@ def test_analyze_index_by_name(db):
assert set(db.table_names()).issuperset(
{"one_index", "two_indexes", "sqlite_stat1"}
)
assert list(db.table("sqlite_stat1").rows) == [
assert list(db["sqlite_stat1"].rows) == [
{"tbl": "two_indexes", "idx": "idx_two_indexes_species", "stat": "1 1"},
]

View file

@ -9,7 +9,7 @@ from sqlite_utils.db import ColumnDetails, Database
@pytest.fixture
def db_to_analyze(fresh_db):
stuff = fresh_db.table("stuff")
stuff = fresh_db["stuff"]
stuff.insert_all(
[
{"id": 1, "owner": "Terryterryterry", "size": 5},
@ -45,7 +45,7 @@ def big_db_to_analyze_path(tmpdir):
"all_null": None,
}
)
db.table("stuff").insert_all(to_insert)
db["stuff"].insert_all(to_insert)
return path
@ -126,7 +126,7 @@ def big_db_to_analyze_path(tmpdir):
)
def test_analyze_column(db_to_analyze, column, extra_kwargs, expected):
assert (
db_to_analyze.table("stuff").analyze_column(
db_to_analyze["stuff"].analyze_column(
column, common_limit=2, value_truncate=5, **extra_kwargs
)
== expected
@ -186,7 +186,7 @@ def test_analyze_table_save(db_to_analyze_path):
cli.cli, ["analyze-tables", db_to_analyze_path, "--save"]
)
assert result.exit_code == 0
rows = list(Database(db_to_analyze_path).table("_analyze_tables_").rows)
rows = list(Database(db_to_analyze_path)["_analyze_tables_"].rows)
assert rows == [
{
"table": "stuff",
@ -248,7 +248,7 @@ def test_analyze_table_save_no_most_no_least_options(
args.append("--no-least")
result = CliRunner().invoke(cli.cli, args)
assert result.exit_code == 0
rows = list(Database(big_db_to_analyze_path).table("_analyze_tables_").rows)
rows = list(Database(big_db_to_analyze_path)["_analyze_tables_"].rows)
expected = {
"table": "stuff",
"column": "category",
@ -297,13 +297,13 @@ def test_analyze_table_column_all_nulls(big_db_to_analyze_path):
def test_analyze_table_validate_columns(tmpdir, args, expected_error):
path = str(tmpdir / "test_validate_columns.db")
db = Database(path)
db.table("one").insert(
db["one"].insert(
{
"id": 1,
"name": "one",
}
)
db.table("two").insert(
db["two"].insert(
{
"id": 1,
"age": 5,

View file

@ -45,30 +45,30 @@ def test_iter_complete_sql_statements(sql, expected):
def test_atomic_commits(fresh_db):
with fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id")
assert list(fresh_db.table("dogs").rows) == [{"id": 1, "name": "Cleo"}]
assert list(fresh_db["dogs"].rows) == [{"id": 1, "name": "Cleo"}]
def test_atomic_rolls_back(fresh_db):
with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id")
raise RuntimeError("boom")
assert not fresh_db.table("dogs").exists()
assert not fresh_db["dogs"].exists()
def test_nested_atomic_rolls_back_to_savepoint(fresh_db):
fresh_db.table("dogs").create({"id": int, "name": str}, pk="id")
fresh_db["dogs"].create({"id": int, "name": str}, pk="id")
with fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"})
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"})
with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 2, "name": "Pancakes"})
fresh_db["dogs"].insert({"id": 2, "name": "Pancakes"})
raise RuntimeError("boom")
fresh_db.table("dogs").insert({"id": 3, "name": "Marnie"})
fresh_db["dogs"].insert({"id": 3, "name": "Marnie"})
assert list(fresh_db.table("dogs").rows) == [
assert list(fresh_db["dogs"].rows) == [
{"id": 1, "name": "Cleo"},
{"id": 3, "name": "Marnie"},
]
@ -76,12 +76,12 @@ def test_nested_atomic_rolls_back_to_savepoint(fresh_db):
def test_outer_atomic_rolls_back_released_savepoint(fresh_db):
with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id")
with fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 2, "name": "Pancakes"})
fresh_db["dogs"].insert({"id": 2, "name": "Pancakes"})
raise RuntimeError("boom")
assert not fresh_db.table("dogs").exists()
assert not fresh_db["dogs"].exists()
def test_executescript_does_not_commit_open_atomic_block(fresh_db):
@ -97,41 +97,41 @@ def test_executescript_does_not_commit_open_atomic_block(fresh_db):
""")
raise RuntimeError("boom")
assert not fresh_db.table("dogs").exists()
assert not fresh_db["dogs"].exists()
def test_transform_does_not_commit_open_atomic_block(fresh_db):
fresh_db.table("dogs").insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
fresh_db["dogs"].insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db.table("dogs").insert({"id": 2, "name": "Pancakes", "age": "6"})
fresh_db.table("dogs").transform(rename={"age": "dog_age"})
fresh_db["dogs"].insert({"id": 2, "name": "Pancakes", "age": "6"})
fresh_db["dogs"].transform(rename={"age": "dog_age"})
raise RuntimeError("boom")
assert (
fresh_db.table("dogs").schema
fresh_db["dogs"].schema
== 'CREATE TABLE "dogs" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "age" TEXT\n)'
)
assert list(fresh_db.table("dogs").rows) == [
assert list(fresh_db["dogs"].rows) == [
{"id": 1, "name": "Cleo", "age": "5"},
]
def test_transform_parent_table_with_foreign_keys_in_atomic(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db.table("authors").insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db.table("books").insert(
fresh_db["authors"].insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db["books"].insert(
{"id": 1, "title": "Book", "author_id": 1},
pk="id",
foreign_keys={"author_id"},
)
with fresh_db.atomic():
fresh_db.table("authors").transform(rename={"name": "full_name"})
fresh_db["authors"].transform(rename={"name": "full_name"})
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
assert (
fresh_db.table("authors").schema
fresh_db["authors"].schema
== 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "full_name" TEXT\n)'
)
assert fresh_db.execute("PRAGMA foreign_key_check").fetchall() == []
@ -139,19 +139,19 @@ def test_transform_parent_table_with_foreign_keys_in_atomic(fresh_db):
def test_transform_parent_table_with_foreign_keys_rolls_back(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db.table("authors").insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db.table("books").insert(
fresh_db["authors"].insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db["books"].insert(
{"id": 1, "title": "Book", "author_id": 1},
pk="id",
foreign_keys={"author_id"},
)
with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db.table("authors").transform(rename={"name": "full_name"})
fresh_db["authors"].transform(rename={"name": "full_name"})
raise RuntimeError("boom")
assert (
fresh_db.table("authors").schema
fresh_db["authors"].schema
== 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT\n)'
)
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -160,51 +160,49 @@ def test_transform_parent_table_with_foreign_keys_rolls_back(fresh_db):
def test_transform_detects_foreign_key_check_violations(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db.table("authors").insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db.table("books").insert({"id": 1, "author_id": 2}, pk="id")
fresh_db["authors"].insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 2}, pk="id")
with pytest.raises(sqlite3.IntegrityError):
fresh_db.table("books").transform(
add_foreign_keys=(("author_id", "authors", "id"),)
)
fresh_db["books"].transform(add_foreign_keys=(("author_id", "authors", "id"),))
assert fresh_db.table("books").foreign_keys == []
assert fresh_db["books"].foreign_keys == []
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
def test_atomic_inside_manual_transaction_uses_savepoint(fresh_db):
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
fresh_db.execute("begin")
with fresh_db.atomic():
fresh_db.table("t").insert({"id": 2}, pk="id")
fresh_db["t"].insert({"id": 2}, pk="id")
# Nothing is committed until the user's own transaction commits
assert fresh_db.conn.in_transaction
fresh_db.rollback()
assert [r["id"] for r in fresh_db.table("t").rows] == [1]
assert [r["id"] for r in fresh_db["t"].rows] == [1]
# And with a commit instead, the atomic block's writes persist
fresh_db.execute("begin")
with fresh_db.atomic():
fresh_db.table("t").insert({"id": 3}, pk="id")
fresh_db["t"].insert({"id": 3}, pk="id")
fresh_db.commit()
assert [r["id"] for r in fresh_db.table("t").rows] == [1, 3]
assert [r["id"] for r in fresh_db["t"].rows] == [1, 3]
def test_begin_commit_rollback(tmpdir):
path = str(tmpdir / "test.db")
db = Database(path)
db.table("t").insert({"id": 1}, pk="id")
db["t"].insert({"id": 1}, pk="id")
db.begin()
db.table("t").insert({"id": 2}, pk="id")
db["t"].insert({"id": 2}, pk="id")
assert db.conn.in_transaction
db.rollback()
assert not db.conn.in_transaction
assert [r["id"] for r in db.table("t").rows] == [1]
assert [r["id"] for r in db["t"].rows] == [1]
db.begin()
db.table("t").insert({"id": 3}, pk="id")
db["t"].insert({"id": 3}, pk="id")
db.commit()
db.close()
db2 = Database(path)
assert [r["id"] for r in db2.table("t").rows] == [1, 3]
assert [r["id"] for r in db2["t"].rows] == [1, 3]
db2.close()
@ -224,7 +222,7 @@ def test_commit_and_rollback_without_transaction_are_noops(fresh_db):
def test_execute_write_commits_immediately(tmpdir):
path = str(tmpdir / "test.db")
db = Database(path)
db.table("t").insert({"id": 1}, pk="id")
db["t"].insert({"id": 1}, pk="id")
db.execute("insert into t (id) values (2)")
# No implicit transaction is left open
assert not db.conn.in_transaction
@ -236,24 +234,24 @@ def test_execute_write_commits_immediately(tmpdir):
def test_execute_write_respects_explicit_transaction(fresh_db):
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
fresh_db.begin()
fresh_db.execute("insert into t (id) values (2)")
# Still inside the explicit transaction - not committed
assert fresh_db.conn.in_transaction
fresh_db.rollback()
assert [r["id"] for r in fresh_db.table("t").rows] == [1]
assert [r["id"] for r in fresh_db["t"].rows] == [1]
def test_execute_comment_prefixed_begin_leaves_transaction_open(fresh_db):
# A BEGIN hidden behind a leading comment must not be auto-committed
# out from under the caller
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
fresh_db.execute("-- start a transaction\nbegin")
assert fresh_db.conn.in_transaction
fresh_db.execute("insert into t (id) values (2)")
fresh_db.rollback()
assert [r["id"] for r in fresh_db.table("t").rows] == [1]
assert [r["id"] for r in fresh_db["t"].rows] == [1]
def _sqlite_accepts_bom():
@ -271,12 +269,12 @@ def test_execute_prefixed_begin_leaves_transaction_open(fresh_db, begin_sql):
# out from under the caller
if begin_sql.startswith("\ufeff") and not _sqlite_accepts_bom():
pytest.skip("This SQLite version rejects a leading byte order mark")
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
fresh_db.execute(begin_sql)
assert fresh_db.conn.in_transaction
fresh_db.execute("insert into t (id) values (2)")
fresh_db.rollback()
assert [r["id"] for r in fresh_db.table("t").rows] == [1]
assert [r["id"] for r in fresh_db["t"].rows] == [1]
def test_execute_failed_write_rolls_back_implicit_transaction(tmpdir):
@ -284,40 +282,40 @@ def test_execute_failed_write_rolls_back_implicit_transaction(tmpdir):
# that would silently disable auto-commit for every subsequent write
path = str(tmpdir / "test.db")
db = Database(path)
db.table("t").insert({"id": 1}, pk="id")
db["t"].insert({"id": 1}, pk="id")
with pytest.raises(sqlite3.IntegrityError):
db.execute("insert into t (id) values (1)")
assert not db.conn.in_transaction
# Subsequent writes commit as normal and survive closing the connection
db.table("other").insert({"id": 2})
db["other"].insert({"id": 2})
db.close()
db2 = Database(path)
assert db2.table("other").exists()
assert db2["other"].exists()
db2.close()
def test_execute_failed_write_preserves_explicit_transaction(fresh_db):
# A failed write inside an explicit transaction must not roll back
# the caller's earlier work - only the caller decides that
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
fresh_db.begin()
fresh_db.execute("insert into t (id) values (2)")
with pytest.raises(sqlite3.IntegrityError):
fresh_db.execute("insert into t (id) values (1)")
assert fresh_db.conn.in_transaction
fresh_db.commit()
assert [r["id"] for r in fresh_db.table("t").rows] == [1, 2]
assert [r["id"] for r in fresh_db["t"].rows] == [1, 2]
def test_execute_failed_write_inside_atomic_preserves_block(fresh_db):
# A caught failure inside an atomic() block must leave the block's
# transaction open so its other work still commits
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
with fresh_db.atomic():
fresh_db.execute("insert into t (id) values (2)")
with pytest.raises(sqlite3.IntegrityError):
fresh_db.execute("insert into t (id) values (1)")
assert [r["id"] for r in fresh_db.table("t").rows] == [1, 2]
assert [r["id"] for r in fresh_db["t"].rows] == [1, 2]
def test_query_returning_commits_after_iteration(tmpdir):
@ -327,7 +325,7 @@ def test_query_returning_commits_after_iteration(tmpdir):
_pytest.skip("RETURNING requires SQLite 3.35.0 or higher")
path = str(tmpdir / "test.db")
db = Database(path)
db.table("t").insert({"id": 1}, pk="id")
db["t"].insert({"id": 1}, pk="id")
rows = list(db.query("insert into t (id) values (2) returning id"))
assert rows == [{"id": 2}]
assert not db.conn.in_transaction
@ -377,7 +375,7 @@ def test_nested_atomic_preserves_error_from_transaction_destroying_trigger(
def test_atomic_preserves_error_from_insert_or_rollback(fresh_db):
fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db["t"].insert({"id": 1}, pk="id")
with pytest.raises(sqlite3.IntegrityError), fresh_db.atomic():
fresh_db.execute("insert or rollback into t (id) values (1)")
assert not fresh_db.conn.in_transaction

View file

@ -6,10 +6,10 @@ def test_attach(tmpdir):
bar_path = str(tmpdir / "bar.db")
db = Database(foo_path)
with db.conn:
db.table("foo").insert({"id": 1, "text": "foo"})
db["foo"].insert({"id": 1, "text": "foo"})
db2 = Database(bar_path)
with db2.conn:
db2.table("bar").insert({"id": 1, "text": "bar"})
db2["bar"].insert({"id": 1, "text": "bar"})
db.attach("bar", bar_path)
assert db.execute(
"select * from foo union all select * from bar.bar"

File diff suppressed because it is too large Load diff

View file

@ -13,7 +13,7 @@ from sqlite_utils import Database, cli
def test_db_and_path(tmpdir):
db_path = str(pathlib.Path(tmpdir) / "data.db")
db = Database(db_path)
db.table("example").insert_all(
db["example"].insert_all(
[
{"id": 1, "name": "One"},
{"id": 2, "name": "Two"},
@ -44,7 +44,7 @@ def test_cli_bulk(test_db_and_path):
{"id": 2, "name": "Two"},
{"id": 3, "name": "THREE"},
{"id": 4, "name": "FOUR"},
] == list(db.table("example").rows)
] == list(db["example"].rows)
def test_cli_bulk_multiple_functions(test_db_and_path):
@ -70,7 +70,7 @@ def test_cli_bulk_multiple_functions(test_db_and_path):
{"id": 2, "name": "Two"},
{"id": 3, "name": "THREE"},
{"id": 4, "name": "FOUR"},
] == list(db.table("example").rows)
] == list(db["example"].rows)
def test_cli_bulk_batch_size(test_db_and_path):
@ -91,18 +91,17 @@ def test_cli_bulk_batch_size(test_db_and_path):
stdin=subprocess.PIPE,
stdout=sys.stdout,
)
assert proc.stdin is not None
# Writing one record should not commit
proc.stdin.write(b'{"id": 3, "name": "Three"}\n\n')
proc.stdin.flush()
time.sleep(1)
assert db.table("example").count == 2
assert db["example"].count == 2
# Writing another should trigger a commit:
proc.stdin.write(b'{"id": 4, "name": "Four"}\n\n')
proc.stdin.flush()
time.sleep(1)
assert db.table("example").count == 4
assert db["example"].count == 4
proc.stdin.close()
proc.wait()

View file

@ -12,7 +12,7 @@ from sqlite_utils import cli
@pytest.fixture
def test_db_and_path(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("example").insert_all(
db["example"].insert_all(
[
{"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6th October 2019 00:05:06"},
@ -47,12 +47,12 @@ def fresh_db_and_path(tmpdir):
)
def test_convert_code(fresh_db_and_path, code):
db, db_path = fresh_db_and_path
db.table("t").insert({"text": "October"})
db["t"].insert({"text": "October"})
result = CliRunner().invoke(
cli.cli, ["convert", db_path, "t", "text", code], catch_exceptions=False
)
assert result.exit_code == 0, result.output
value = next(iter(db.table("t").rows))["text"]
value = next(iter(db["t"].rows))["text"]
assert value == "Spooktober"
@ -65,7 +65,7 @@ def test_convert_code(fresh_db_and_path, code):
)
def test_convert_code_errors(fresh_db_and_path, bad_code):
db, db_path = fresh_db_and_path
db.table("t").insert({"text": "October"})
db["t"].insert({"text": "October"})
result = CliRunner().invoke(
cli.cli, ["convert", db_path, "t", "text", bad_code], catch_exceptions=False
)
@ -93,12 +93,12 @@ def test_convert_import(test_db_and_path):
{"id": 2, "dt": "6th OXXober 2019 00:05:06"},
{"id": 3, "dt": ""},
{"id": 4, "dt": None},
] == list(db.table("example").rows)
] == list(db["example"].rows)
def test_convert_import_nested(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("example").insert({"xml": '<item name="Cleo" />'})
db["example"].insert({"xml": '<item name="Cleo" />'})
result = CliRunner().invoke(
cli.cli,
[
@ -114,7 +114,7 @@ def test_convert_import_nested(fresh_db_and_path):
assert result.exit_code == 0, result.output
assert [
{"xml": "Cleo"},
] == list(db.table("example").rows)
] == list(db["example"].rows)
def test_convert_dryrun(test_db_and_path):
@ -152,7 +152,7 @@ def test_convert_dryrun(test_db_and_path):
"Would affect 4 rows"
)
# But it should not have actually modified the table data
assert list(db.table("example").rows) == [
assert list(db["example"].rows) == [
{"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6th October 2019 00:05:06"},
{"id": 3, "dt": ""},
@ -181,34 +181,6 @@ def test_convert_dryrun(test_db_and_path):
assert result.output.strip().split("\n")[-1] == "Would affect 1 row"
def test_convert_dryrun_table_and_column_names_containing_closing_bracket(
fresh_db_and_path,
):
db, db_path = fresh_db_and_path
table_name = "table]name"
column_name = "column]name"
db[table_name].insert({column_name: "hello"})
result = CliRunner().invoke(
cli.cli,
[
"convert",
db_path,
table_name,
column_name,
"value.upper()",
"--dry-run",
],
catch_exceptions=False,
)
assert result.exit_code == 0
assert result.output.strip() == (
"hello\n --- becomes:\nHELLO\n\nWould affect 1 row"
)
assert list(db[table_name].rows) == [{column_name: "hello"}]
def test_convert_multi_dryrun(test_db_and_path):
db_path = test_db_and_path[1]
result = CliRunner().invoke(
@ -297,7 +269,7 @@ def test_convert_output_column(test_db_and_path, drop):
if drop:
for row in expected:
del row["dt"]
assert list(db.table("example").rows) == expected
assert list(db["example"].rows) == expected
@pytest.mark.parametrize(
@ -380,7 +352,7 @@ def test_convert_output_error(test_db_and_path, options, expected_error):
@pytest.mark.parametrize("drop", (True, False))
def test_convert_multi(fresh_db_and_path, drop):
db, db_path = fresh_db_and_path
db.table("creatures").insert_all(
db["creatures"].insert_all(
[
{"id": 1, "name": "Simon"},
{"id": 2, "name": "Cleo"},
@ -406,12 +378,12 @@ def test_convert_multi(fresh_db_and_path, drop):
if drop:
for row in expected:
del row["name"]
assert list(db.table("creatures").rows) == expected
assert list(db["creatures"].rows) == expected
def test_convert_multi_complex_column_types(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("rows").insert_all(
db["rows"].insert_all(
[
{"id": 1},
{"id": 2},
@ -440,7 +412,7 @@ def test_convert_multi_complex_column_types(fresh_db_and_path):
],
)
assert result.exit_code == 0, result.output
assert list(db.table("rows").rows) == [
assert list(db["rows"].rows) == [
{"id": 1, "is_str": "", "is_float": 1.2, "is_int": None, "is_bytes": None},
{"id": 2, "is_str": None, "is_float": 1.0, "is_int": 12, "is_bytes": None},
{
@ -452,7 +424,7 @@ def test_convert_multi_complex_column_types(fresh_db_and_path):
},
{"id": 4, "is_str": None, "is_float": None, "is_int": None, "is_bytes": None},
]
assert db.table("rows").schema == (
assert db["rows"].schema == (
'CREATE TABLE "rows" (\n'
' "id" INTEGER PRIMARY KEY\n'
', "is_str" TEXT, "is_float" REAL, "is_int" INTEGER, "is_bytes" BLOB)'
@ -463,7 +435,7 @@ def test_convert_multi_complex_column_types(fresh_db_and_path):
def test_recipe_jsonsplit(tmpdir, delimiter):
db_path = str(pathlib.Path(tmpdir) / "data.db")
db = sqlite_utils.Database(db_path)
db.table("example").insert_all(
db["example"].insert_all(
[
{"id": 1, "tags": (delimiter or ",").join(["foo", "bar"])},
{"id": 2, "tags": (delimiter or ",").join(["bar", "baz"])},
@ -476,7 +448,7 @@ def test_recipe_jsonsplit(tmpdir, delimiter):
args = ["convert", db_path, "example", "tags", code]
result = CliRunner().invoke(cli.cli, args)
assert result.exit_code == 0, result.output
assert list(db.table("example").rows) == [
assert list(db["example"].rows) == [
{"id": 1, "tags": '["foo", "bar"]'},
{"id": 2, "tags": '["bar", "baz"]'},
]
@ -492,7 +464,7 @@ def test_recipe_jsonsplit(tmpdir, delimiter):
)
def test_recipe_jsonsplit_type(fresh_db_and_path, type, expected_array):
db, db_path = fresh_db_and_path
db.table("example").insert_all(
db["example"].insert_all(
[
{"id": 1, "records": "1,2,3"},
],
@ -504,13 +476,13 @@ def test_recipe_jsonsplit_type(fresh_db_and_path, type, expected_array):
args = ["convert", db_path, "example", "records", code]
result = CliRunner().invoke(cli.cli, args)
assert result.exit_code == 0, result.output
assert json.loads(db.table("example").get(1)["records"]) == expected_array
assert json.loads(db["example"].get(1)["records"]) == expected_array
@pytest.mark.parametrize("drop", (True, False))
def test_recipe_jsonsplit_output(fresh_db_and_path, drop):
db, db_path = fresh_db_and_path
db.table("example").insert_all(
db["example"].insert_all(
[
{"id": 1, "records": "1,2,3"},
],
@ -529,7 +501,7 @@ def test_recipe_jsonsplit_output(fresh_db_and_path, drop):
}
if drop:
del expected["records"]
assert db.table("example").get(1) == expected
assert db["example"].get(1) == expected
def test_cannot_use_drop_without_multi_or_output(fresh_db_and_path):
@ -586,7 +558,7 @@ def test_convert_where(test_db_and_path):
],
)
assert result.exit_code == 0, result.output
assert list(db.table("example").rows) == [
assert list(db["example"].rows) == [
{"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6TH OCTOBER 2019 00:05:06"},
{"id": 3, "dt": ""},
@ -596,7 +568,7 @@ def test_convert_where(test_db_and_path):
def test_convert_where_multi(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("names").insert_all(
db["names"].insert_all(
[{"id": 1, "name": "Cleo"}, {"id": 2, "name": "Bants"}], pk="id"
)
result = CliRunner().invoke(
@ -616,7 +588,7 @@ def test_convert_where_multi(fresh_db_and_path):
],
)
assert result.exit_code == 0, result.output
assert list(db.table("names").rows) == [
assert list(db["names"].rows) == [
{"id": 1, "name": "Cleo", "upper": None},
{"id": 2, "name": "Bants", "upper": "BANTS"},
]
@ -624,7 +596,7 @@ def test_convert_where_multi(fresh_db_and_path):
def test_convert_code_standard_input(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("names").insert_all([{"id": 1, "name": "Cleo"}], pk="id")
db["names"].insert_all([{"id": 1, "name": "Cleo"}], pk="id")
result = CliRunner().invoke(
cli.cli,
[
@ -637,27 +609,27 @@ def test_convert_code_standard_input(fresh_db_and_path):
input="value.upper()",
)
assert result.exit_code == 0, result.output
assert list(db.table("names").rows) == [
assert list(db["names"].rows) == [
{"id": 1, "name": "CLEO"},
]
def test_convert_hyphen_workaround(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("names").insert_all([{"id": 1, "name": "Cleo"}], pk="id")
db["names"].insert_all([{"id": 1, "name": "Cleo"}], pk="id")
result = CliRunner().invoke(
cli.cli,
["convert", db_path, "names", "name", '"-"'],
)
assert result.exit_code == 0, result.output
assert list(db.table("names").rows) == [
assert list(db["names"].rows) == [
{"id": 1, "name": "-"},
]
def test_convert_initialization_pattern(fresh_db_and_path):
db, db_path = fresh_db_and_path
db.table("names").insert_all([{"id": 1, "name": "Cleo"}], pk="id")
db["names"].insert_all([{"id": 1, "name": "Cleo"}], pk="id")
result = CliRunner().invoke(
cli.cli,
[
@ -670,7 +642,7 @@ def test_convert_initialization_pattern(fresh_db_and_path):
input="import random\nrandom.seed(1)\ndef convert(value): return random.randint(0, 100)",
)
assert result.exit_code == 0, result.output
assert list(db.table("names").rows) == [
assert list(db["names"].rows) == [
{"id": 1, "name": "17"},
]
@ -685,13 +657,13 @@ def test_convert_handles_falsey_values(fresh_db_and_path):
"x",
"-",
]
db.table("t").insert_all([{"x": 0}, {"x": 1}])
assert db.table("t").get(1)["x"] == 0
assert db.table("t").get(2)["x"] == 1
db["t"].insert_all([{"x": 0}, {"x": 1}])
assert db["t"].get(1)["x"] == 0
assert db["t"].get(2)["x"] == 1
result = CliRunner().invoke(cli.cli, args, input="value + 1")
assert result.exit_code == 0, result.output
assert db.table("t").get(1)["x"] == 1
assert db.table("t").get(2)["x"] == 2
assert db["t"].get(1)["x"] == 1
assert db["t"].get(2)["x"] == 2
@pytest.mark.parametrize(
@ -712,7 +684,7 @@ def test_convert_callable_reference(test_db_and_path, code):
cli.cli, ["convert", db_path, "example", "dt", code], catch_exceptions=False
)
assert result.exit_code == 0, result.output
rows = list(db.table("example").rows)
rows = list(db["example"].rows)
assert rows[0]["dt"] == "2019-10-05"
assert rows[1]["dt"] == "2019-10-06"
assert rows[2]["dt"] == ""
@ -722,7 +694,7 @@ def test_convert_callable_reference(test_db_and_path, code):
def test_convert_callable_reference_with_import(fresh_db_and_path):
"""Test callable reference from an imported module"""
db, db_path = fresh_db_and_path
db.table("example").insert({"id": 1, "data": '{"name": "test"}'})
db["example"].insert({"id": 1, "data": '{"name": "test"}'})
result = CliRunner().invoke(
cli.cli,
[
@ -738,5 +710,5 @@ def test_convert_callable_reference_with_import(fresh_db_and_path):
)
assert result.exit_code == 0, result.output
# json.loads returns a dict, which sqlite stores as JSON string
row = db.table("example").get(1)
row = db["example"].get(1)
assert row["data"] == '{"name": "test"}'

View file

@ -21,7 +21,7 @@ def test_insert_simple(tmpdir):
)
db = Database(db_path)
assert ["dogs"] == db.table_names()
assert [] == db.table("dogs").indexes
assert [] == db["dogs"].indexes
def test_insert_from_stdin(tmpdir):
@ -96,7 +96,7 @@ def test_insert_with_primary_keys(db_path, tmpdir, args, expected_pks):
Database(db_path).query("select * from dogs")
)
db = Database(db_path)
assert db.table("dogs").pks == expected_pks
assert db["dogs"].pks == expected_pks
def test_insert_multiple_with_primary_key(db_path, tmpdir):
@ -110,7 +110,7 @@ def test_insert_multiple_with_primary_key(db_path, tmpdir):
assert result.exit_code == 0
db = Database(db_path)
assert dogs == list(db.query("select * from dogs order by id"))
assert ["id"] == db.table("dogs").pks
assert ["id"] == db["dogs"].pks
def test_insert_multiple_with_compound_primary_key(db_path, tmpdir):
@ -127,7 +127,7 @@ def test_insert_multiple_with_compound_primary_key(db_path, tmpdir):
assert result.exit_code == 0
db = Database(db_path)
assert dogs == list(db.query("select * from dogs order by breed, id"))
assert {"breed", "id"} == set(db.table("dogs").pks)
assert {"breed", "id"} == set(db["dogs"].pks)
assert (
'CREATE TABLE "dogs" (\n'
' "breed" TEXT,\n'
@ -136,7 +136,7 @@ def test_insert_multiple_with_compound_primary_key(db_path, tmpdir):
' "age" INTEGER,\n'
' PRIMARY KEY ("id", "breed")\n'
")"
) == db.table("dogs").schema
) == db["dogs"].schema
def test_insert_not_null_default(db_path, tmpdir):
@ -160,7 +160,7 @@ def test_insert_not_null_default(db_path, tmpdir):
' "name" TEXT NOT NULL,\n'
" \"age\" INTEGER NOT NULL DEFAULT '1',\n"
" \"score\" INTEGER DEFAULT '5'\n)"
) == db.table("dogs").schema
) == db["dogs"].schema
def test_insert_binary_base64(db_path):
@ -191,7 +191,7 @@ def test_insert_newline_delimited(db_path):
def test_insert_ignore(db_path, tmpdir):
db = Database(db_path)
db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id")
json_path = str(tmpdir / "dogs.json")
with open(json_path, "w") as fp:
fp.write(json.dumps([{"id": 1, "name": "Bailey"}]))
@ -232,7 +232,7 @@ def test_insert_csv_tsv(content, options, db_path, tmpdir):
catch_exceptions=False,
)
assert result.exit_code == 0
assert [{"foo": "1", "bar": "2", "baz": "cat,dog"}] == list(db.table("data").rows)
assert [{"foo": "1", "bar": "2", "baz": "cat,dog"}] == list(db["data"].rows)
@pytest.mark.parametrize("empty_null", (True, False))
@ -248,7 +248,7 @@ def test_insert_csv_empty_null(db_path, empty_null):
)
assert result.exit_code == 0
db = Database(db_path)
assert [r for r in db.table("data").rows] == [
assert [r for r in db["data"].rows] == [
{"foo": "1", "bar": None if empty_null else "", "baz": "cat"}
]
@ -302,7 +302,7 @@ def test_insert_replace(db_path, tmpdir):
test_insert_multiple_with_primary_key(db_path, tmpdir)
json_path = str(tmpdir / "insert-replace.json")
db = Database(db_path)
assert db.table("dogs").count == 20
assert db["dogs"].count == 20
insert_replace_dogs = [
{"id": 1, "name": "Insert replaced 1", "age": 4},
{"id": 2, "name": "Insert replaced 2", "age": 4},
@ -314,7 +314,7 @@ def test_insert_replace(db_path, tmpdir):
cli.cli, ["insert", db_path, "dogs", json_path, "--pk", "id", "--replace"]
)
assert result.exit_code == 0, result.output
assert db.table("dogs").count == 21
assert db["dogs"].count == 21
assert (
list(db.query("select * from dogs where id in (1, 2, 21) order by id"))
== insert_replace_dogs
@ -377,7 +377,7 @@ def test_insert_alter(db_path, tmpdir):
assert result.exit_code == 0, result.output
# Soundness check the database itself
db = Database(db_path)
assert {"foo": str, "n": int, "baz": int} == db.table("from_json_nl").columns_dict
assert {"foo": str, "n": int, "baz": int} == db["from_json_nl"].columns_dict
assert [
{"foo": "bar", "n": 1, "baz": None},
{"foo": "baz", "n": 2, "baz": None},
@ -387,8 +387,8 @@ def test_insert_alter(db_path, tmpdir):
def test_insert_analyze(db_path):
db = Database(db_path)
db.table("rows").insert({"foo": "x", "n": 3})
db.table("rows").create_index(["n"])
db["rows"].insert({"foo": "x", "n": 3})
db["rows"].create_index(["n"])
assert "sqlite_stat1" not in db.table_names()
result = CliRunner().invoke(
cli.cli,
@ -577,14 +577,13 @@ def test_insert_streaming_batch_size_1(db_path):
stdin=subprocess.PIPE,
stdout=sys.stdout,
)
assert proc.stdin is not None
proc.stdin.write(b'{"name": "Azi"}\n')
proc.stdin.flush()
def try_until(expected):
tries = 0
while True:
rows = list(Database(db_path).table("rows").rows)
rows = list(Database(db_path)["rows"].rows)
if rows == expected:
return
tries += 1
@ -616,13 +615,13 @@ def test_insert_csv_headers_only(tmpdir):
assert result.exit_code == 0
# Table should not exist since there were no data rows
db = Database(db_path)
assert not db.table("data").exists()
assert not db["data"].exists()
def test_insert_into_view_errors(tmpdir):
db_path = str(tmpdir / "test.db")
db = Database(db_path)
db.table("t").insert({"id": 1})
db["t"].insert({"id": 1})
db.create_view("v", "select * from t")
db.close()
result = CliRunner().invoke(
@ -638,7 +637,7 @@ def test_insert_csv_detect_types_leaves_existing_table_alone(db_path):
# table would rewrite its column types and corrupt data such as
# TEXT zip codes with leading zeros
db = Database(db_path)
db.table("places").insert({"name": "Boston", "zip": "01234"})
db["places"].insert({"name": "Boston", "zip": "01234"})
result = CliRunner().invoke(
cli.cli,
["insert", db_path, "places", "-", "--csv"],
@ -646,8 +645,8 @@ def test_insert_csv_detect_types_leaves_existing_table_alone(db_path):
input="name,zip\nSF,94107",
)
assert result.exit_code == 0, result.output
assert db.table("places").columns_dict["zip"] is str
assert list(db.table("places").rows) == [
assert db["places"].columns_dict["zip"] is str
assert list(db["places"].rows) == [
{"name": "Boston", "zip": "01234"},
{"name": "SF", "zip": "94107"},
]
@ -663,7 +662,7 @@ def test_insert_csv_detect_types_new_table(db_path):
)
assert result.exit_code == 0, result.output
db = Database(db_path)
assert db.table("data").columns_dict == {"name": str, "age": int, "weight": float}
assert db["data"].columns_dict == {"name": str, "age": int, "weight": float}
@pytest.mark.parametrize(
@ -709,13 +708,13 @@ def test_insert_upsert_csv_type_overrides_detected_types(
expected_columns = {"zipcode": str, "score": float}
if command == "upsert":
expected_columns = {"id": int, **expected_columns}
assert db.table("places").columns_dict == expected_columns
assert list(db.table("places").rows) == [expected_row]
assert db["places"].columns_dict == expected_columns
assert list(db["places"].rows) == [expected_row]
def test_upsert_csv_detect_types_leaves_existing_table_alone(db_path):
db = Database(db_path)
db.table("places").insert({"id": 1, "name": "Boston", "zip": "01234"}, pk="id")
db["places"].insert({"id": 1, "name": "Boston", "zip": "01234"}, pk="id")
result = CliRunner().invoke(
cli.cli,
["upsert", db_path, "places", "-", "--csv", "--pk", "id"],
@ -723,15 +722,15 @@ def test_upsert_csv_detect_types_leaves_existing_table_alone(db_path):
input="id,name,zip\n2,SF,94107",
)
assert result.exit_code == 0, result.output
assert db.table("places").columns_dict["zip"] is str
assert db.table("places").get(1)["zip"] == "01234"
assert db["places"].columns_dict["zip"] is str
assert db["places"].get(1)["zip"] == "01234"
def test_insert_invalid_pk_clean_error(db_path):
# An invalid --pk against an existing table should be a clean CLI
# error, not a raw InvalidColumns traceback
db = Database(db_path)
db.table("t").insert({"a": 1})
db["t"].insert({"a": 1})
result = CliRunner().invoke(
cli.cli,
["insert", db_path, "t", "-", "--pk", "badcol"],
@ -766,8 +765,8 @@ def test_insert_code(tmpdir, code):
)
assert result.exit_code == 0, result.output
db = Database(db_path)
assert db.table("creatures").pks == ["id"]
assert list(db.table("creatures").rows) == [
assert db["creatures"].pks == ["id"]
assert list(db["creatures"].rows) == [
{"id": 1, "name": "Cleo"},
{"id": 2, "name": "Suna"},
]
@ -783,7 +782,7 @@ def test_insert_code_from_file(tmpdir):
["insert", db_path, "creatures", "--code", code_path],
)
assert result.exit_code == 0, result.output
assert list(Database(db_path).table("creatures").rows) == [
assert list(Database(db_path)["creatures"].rows) == [
{"id": 1, "name": "Cleo"},
{"id": 2, "name": "Suna"},
]
@ -792,7 +791,7 @@ def test_insert_code_from_file(tmpdir):
def test_upsert_code(tmpdir):
db_path = str(tmpdir / "dogs.db")
db = Database(db_path)
db.table("creatures").insert_all(
db["creatures"].insert_all(
[{"id": 1, "name": "old"}, {"id": 2, "name": "Suna"}], pk="id"
)
result = CliRunner().invoke(
@ -800,7 +799,7 @@ def test_upsert_code(tmpdir):
["upsert", db_path, "creatures", "--code", CODE_ROWS_FUNCTION, "--pk", "id"],
)
assert result.exit_code == 0, result.output
assert list(db.table("creatures").rows) == [
assert list(db["creatures"].rows) == [
{"id": 1, "name": "Cleo"},
{"id": 2, "name": "Suna"},
]
@ -859,9 +858,7 @@ def test_insert_code_single_dict(tmpdir):
],
)
assert result.exit_code == 0, result.output
assert list(Database(db_path).table("creatures").rows) == [
{"id": 1, "name": "Cleo"}
]
assert list(Database(db_path)["creatures"].rows) == [{"id": 1, "name": "Cleo"}]
def test_insert_code_not_iterable(tmpdir):

View file

@ -228,7 +228,7 @@ def test_memory_save(tmpdir, extra_args):
)
assert result.exit_code == 0
db = Database(save_to)
assert list(db.table("stdin").rows) == [
assert list(db["stdin"].rows) == [
{"id": 1, "name": "Cleo"},
{"id": 2, "name": "Bants"},
]

View file

@ -13,11 +13,11 @@ m = Migrations("hello")
@m()
def foo(db):
db.table("foo").insert({"hello": "world"})
db["foo"].insert({"hello": "world"})
@m()
def bar(db):
db.table("bar").insert({"hello": "world"})
db["bar"].insert({"hello": "world"})
"""
@ -42,21 +42,21 @@ creatures = Migrations("creatures")
@creatures()
def create_table(db):
db.table("creatures").insert({"name": "Cleo"})
db["creatures"].insert({"name": "Cleo"})
@creatures()
def add_weight(db):
db.table("creature_weights").insert({"weight": 4.2})
db["creature_weights"].insert({"weight": 4.2})
sales = Migrations("sales")
@sales()
def create_table(db):
db.table("sales").insert({"id": 1})
db["sales"].insert({"id": 1})
@sales()
def add_weight(db):
db.table("sales_weights").insert({"weight": 10})
db["sales_weights"].insert({"weight": 10})
""",
"utf-8",
)
@ -99,10 +99,10 @@ def test_basic(two_migrations, arg):
assert " Pending:\n (none)" in list_output
db = sqlite_utils.Database(db_path)
assert db.table("foo").exists()
assert db.table("bar").exists()
assert db.table("_sqlite_migrations").exists()
rows = list(db.table("_sqlite_migrations").rows)
assert db["foo"].exists()
assert db["bar"].exists()
assert db["_sqlite_migrations"].exists()
rows = list(db["_sqlite_migrations"].rows)
assert len(rows) == 2
assert rows[0]["name"] == "foo"
assert rows[1]["name"] == "bar"
@ -113,13 +113,13 @@ def test_list_same_migration_names_in_different_sets(capsys):
@applied(name="foo")
def applied_foo(db):
db.table("applied").insert({"hello": "world"})
db["applied"].insert({"hello": "world"})
pending = sqlite_utils.Migrations("pending")
@pending(name="foo")
def pending_foo(db):
db.table("pending").insert({"hello": "world"})
db["pending"].insert({"hello": "world"})
db = sqlite_utils.Database(memory=True)
applied.apply(db)
@ -144,7 +144,7 @@ m = Migrations("hello")
@m()
def foo(db):
db.table("dogs").insert({"id": 1, "name": "Cleo"})
db["dogs"].insert({"id": 1, "name": "Cleo"})
""",
"utf-8",
)
@ -184,9 +184,9 @@ Schema after:
new_migration = """
@m()
def bar(db):
db.table("dogs").add_column("age", int)
db.table("dogs").add_column("weight", float)
db.table("dogs").transform()
db["dogs"].add_column("age", int)
db["dogs"].add_column("weight", float)
db["dogs"].transform()
"""
migrations_py.write_text(migrations_py.read_text("utf-8") + new_migration)
@ -224,8 +224,8 @@ def test_stop_before(two_migrations):
)
assert result.exit_code == 0
db = sqlite_utils.Database(db_path)
assert db.table("foo").exists()
assert not db.table("bar").exists()
assert db["foo"].exists()
assert not db["bar"].exists()
def test_stop_before_multiple_sets_unqualified(two_migrations):
@ -239,7 +239,7 @@ m = Migrations("hello2")
@m()
def foo(db):
db.table("foo").insert({"hello": "world"})
db["foo"].insert({"hello": "world"})
""",
"utf-8",
)
@ -257,7 +257,7 @@ def foo(db):
assert result.exit_code == 0, result.output
db = sqlite_utils.Database(db_path)
assert db.table_names() == ["_sqlite_migrations"]
assert list(db.table("_sqlite_migrations").rows) == []
assert list(db["_sqlite_migrations"].rows) == []
def test_stop_before_qualified_only_affects_named_set(two_sets_same_migration_name):
@ -275,10 +275,10 @@ def test_stop_before_qualified_only_affects_named_set(two_sets_same_migration_na
)
assert result.exit_code == 0, result.output
db = sqlite_utils.Database(db_path)
assert db.table("creatures").exists()
assert not db.table("creature_weights").exists()
assert db.table("sales").exists()
assert db.table("sales_weights").exists()
assert db["creatures"].exists()
assert not db["creature_weights"].exists()
assert db["sales"].exists()
assert db["sales_weights"].exists()
def test_stop_before_multiple_qualified(two_sets_same_migration_name):
@ -298,10 +298,10 @@ def test_stop_before_multiple_qualified(two_sets_same_migration_name):
)
assert result.exit_code == 0, result.output
db = sqlite_utils.Database(db_path)
assert db.table("creatures").exists()
assert not db.table("creature_weights").exists()
assert db.table("sales").exists()
assert not db.table("sales_weights").exists()
assert db["creatures"].exists()
assert not db["creature_weights"].exists()
assert db["sales"].exists()
assert not db["sales_weights"].exists()
LEGACY_MIGRATIONS = """
@ -331,7 +331,7 @@ class LegacyMigrations:
return fn
def ensure_migrations_table(self, db):
db.table(self.migrations_table).create(
db[self.migrations_table].create(
{"migration_set": str, "name": str, "applied_at": str},
pk=("migration_set", "name"),
if_not_exists=True,
@ -341,7 +341,7 @@ class LegacyMigrations:
self.ensure_migrations_table(db)
return [
_Applied(row["name"], row["applied_at"])
for row in db.table(self.migrations_table).rows_where(
for row in db[self.migrations_table].rows_where(
"migration_set = ?", [self.name]
)
]
@ -355,7 +355,7 @@ class LegacyMigrations:
if migration.name == stop_before:
return
migration.fn(db)
db.table(self.migrations_table).insert(
db[self.migrations_table].insert(
{
"migration_set": self.name,
"name": migration.name,
@ -369,11 +369,11 @@ legacy = LegacyMigrations("legacy_set")
@legacy
def first(db):
db.table("first").insert({"hello": "world"})
db["first"].insert({"hello": "world"})
@legacy
def second(db):
db.table("second").insert({"hello": "world"})
db["second"].insert({"hello": "world"})
"""
@ -446,11 +446,11 @@ def test_list_does_not_upgrade_legacy_migrations_table(two_migrations):
path, _ = two_migrations
db_path = str(path / "test.db")
db = sqlite_utils.Database(db_path)
db.table("_sqlite_migrations").create(
db["_sqlite_migrations"].create(
{"migration_set": str, "name": str, "applied_at": str},
pk=("migration_set", "name"),
)
db.table("_sqlite_migrations").insert(
db["_sqlite_migrations"].insert(
{"migration_set": "hello", "name": "foo", "applied_at": "x"}
)
db.close()
@ -462,7 +462,7 @@ def test_list_does_not_upgrade_legacy_migrations_table(two_migrations):
assert "foo - x" in result.output
# --list must not perform the one-way legacy schema upgrade
db2 = sqlite_utils.Database(db_path)
assert db2.table("_sqlite_migrations").pks == ["migration_set", "name"]
assert db2["_sqlite_migrations"].pks == ["migration_set", "name"]
db2.close()
@ -485,7 +485,7 @@ def test_stop_before_applied_migration_errors(two_migrations):
assert result.exit_code != 0
assert "already been applied" in result.output
db = sqlite_utils.Database(db_path)
assert not db.table("bar").exists()
assert not db["bar"].exists()
def test_list_with_legacy_class_is_read_only(tmpdir):
@ -496,7 +496,7 @@ def test_list_with_legacy_class_is_read_only(tmpdir):
(path / "migrations.py").write_text(LEGACY_MIGRATIONS, "utf-8")
db_path = str(path / "test.db")
db = sqlite_utils.Database(db_path)
db.table("existing").insert({"id": 1})
db["existing"].insert({"id": 1})
db.close()
result = CliRunner().invoke(
sqlite_utils.cli.cli, ["migrate", db_path, str(path), "--list"]

View file

@ -1,6 +1,5 @@
import pytest
from sqlite_utils import ANY
from sqlite_utils.utils import column_affinity
EXAMPLES = [
@ -27,8 +26,6 @@ EXAMPLES = [
("DOUBLE", float),
("DOUBLE PRECISION", float),
("FLOAT", float),
("ANY", ANY),
("any", ANY),
# Numeric, treated as float:
("NUMERIC", float),
("DECIMAL(10,5)", float),
@ -46,4 +43,4 @@ def test_column_affinity(column_def, expected_type):
@pytest.mark.parametrize("column_def,expected_type", EXAMPLES)
def test_columns_dict(fresh_db, column_def, expected_type):
fresh_db.execute(f"create table foo (col {column_def})")
assert {"col": expected_type} == fresh_db.table("foo").columns_dict
assert {"col": expected_type} == fresh_db["foo"].columns_dict

View file

@ -13,14 +13,14 @@ from sqlite_utils.db import ForeignKey
def test_insert_populates_last_pk_case_insensitively(fresh_db):
books = fresh_db.table("books")
books = fresh_db["books"]
books.create({"Id": int, "Title": str}, pk="Id")
books.insert({"Id": 1, "Title": "One"}, pk="id")
assert books.last_pk == 1
def test_insert_populates_last_pk_compound_pk_case_insensitively(fresh_db):
books = fresh_db.table("books")
books = fresh_db["books"]
books.create({"Author": str, "Position": int, "Title": str})
books.insert(
{"Author": "Sue", "Position": 1, "Title": "One"}, pk=("author", "position")
@ -31,7 +31,7 @@ def test_insert_populates_last_pk_compound_pk_case_insensitively(fresh_db):
@pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert_pk_case_differs_from_schema(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert)
books = db.table("books")
books = db["books"]
books.create({"Id": int, "Title": str}, pk="Id")
books.insert({"Id": 1, "Title": "One"})
books.upsert({"id": 1, "title": "Won"}, pk="id")
@ -43,7 +43,7 @@ def test_upsert_pk_case_differs_from_schema(use_old_upsert):
def test_upsert_record_key_case_differs_from_pk(use_old_upsert):
# all_columns comes from the record keys, pk= from the caller
db = Database(memory=True, use_old_upsert=use_old_upsert)
books = db.table("books")
books = db["books"]
books.create({"Id": int, "Title": str}, pk="Id")
books.upsert({"ID": 1, "Title": "One"}, pk="id")
assert list(books.rows) == [{"Id": 1, "Title": "One"}]
@ -52,7 +52,7 @@ def test_upsert_record_key_case_differs_from_pk(use_old_upsert):
def test_upsert_inferred_pk_case_differs_from_record_keys(fresh_db):
# pk is inferred from the existing schema as "Id", records use "id"
books = fresh_db.table("books")
books = fresh_db["books"]
books.create({"Id": int, "Title": str}, pk="Id")
books.upsert({"id": 1, "title": "One"})
assert list(books.rows) == [{"Id": 1, "Title": "One"}]
@ -60,7 +60,7 @@ def test_upsert_inferred_pk_case_differs_from_record_keys(fresh_db):
def test_upsert_list_mode_pk_case_insensitive(fresh_db):
books = fresh_db.table("books")
books = fresh_db["books"]
books.create({"Id": int, "Title": str}, pk="Id")
books.upsert_all([["id", "title"], [1, "One"]], pk="Id")
assert list(books.rows) == [{"Id": 1, "Title": "One"}]
@ -68,84 +68,84 @@ def test_upsert_list_mode_pk_case_insensitive(fresh_db):
def test_lookup_pk_case_insensitive(fresh_db):
fresh_db.table("species").create({"ID": int, "Name": str}, pk="ID")
fresh_db.table("species").insert({"ID": 5, "Name": "Palm"})
fresh_db.table("species").create_index(["Name"], unique=True)
assert fresh_db.table("species").lookup({"Name": "Palm"}, pk="id") == 5
fresh_db["species"].create({"ID": int, "Name": str}, pk="ID")
fresh_db["species"].insert({"ID": 5, "Name": "Palm"})
fresh_db["species"].create_index(["Name"], unique=True)
assert fresh_db["species"].lookup({"Name": "Palm"}, pk="id") == 5
def test_lookup_does_not_create_redundant_index(fresh_db):
fresh_db.table("species").create({"id": int, "Name": str}, pk="id")
fresh_db.table("species").create_index(["Name"], unique=True)
fresh_db.table("species").lookup({"name": "Palm"})
assert len(fresh_db.table("species").indexes) == 1
fresh_db["species"].create({"id": int, "Name": str}, pk="id")
fresh_db["species"].create_index(["Name"], unique=True)
fresh_db["species"].lookup({"name": "Palm"})
assert len(fresh_db["species"].indexes) == 1
def test_create_table_transform_same_columns_different_case(fresh_db):
fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db.table("t").insert({"Name": "Cleo", "Age": 5})
fresh_db["t"].create({"Name": str, "Age": int})
fresh_db["t"].insert({"Name": "Cleo", "Age": 5})
fresh_db.create_table("t", {"name": str, "age": int}, transform=True)
# Schema casing is preserved - SQLite considers these the same columns
assert fresh_db.table("t").columns_dict == {"Name": str, "Age": int}
assert list(fresh_db.table("t").rows) == [{"Name": "Cleo", "Age": 5}]
assert fresh_db["t"].columns_dict == {"Name": str, "Age": int}
assert list(fresh_db["t"].rows) == [{"Name": "Cleo", "Age": 5}]
def test_create_table_transform_case_insensitive_with_changes(fresh_db):
fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db["t"].create({"Name": str, "Age": int})
fresh_db.create_table("t", {"name": str, "age": str, "size": int}, transform=True)
# age changed type, size added, Name untouched
assert fresh_db.table("t").columns_dict == {"Name": str, "Age": str, "size": int}
assert fresh_db["t"].columns_dict == {"Name": str, "Age": str, "size": int}
def test_transform_types_case_insensitive(fresh_db):
fresh_db.table("t").create({"Name": str, "Age": str})
fresh_db.table("t").transform(types={"age": int})
assert fresh_db.table("t").columns_dict == {"Name": str, "Age": int}
fresh_db["t"].create({"Name": str, "Age": str})
fresh_db["t"].transform(types={"age": int})
assert fresh_db["t"].columns_dict == {"Name": str, "Age": int}
def test_transform_rename_case_insensitive(fresh_db):
fresh_db.table("t").create({"Name": str})
fresh_db.table("t").transform(rename={"name": "title"})
assert fresh_db.table("t").columns_dict == {"title": str}
fresh_db["t"].create({"Name": str})
fresh_db["t"].transform(rename={"name": "title"})
assert fresh_db["t"].columns_dict == {"title": str}
def test_transform_drop_case_insensitive(fresh_db):
fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db.table("t").transform(drop=["name"])
assert fresh_db.table("t").columns_dict == {"Age": int}
fresh_db["t"].create({"Name": str, "Age": int})
fresh_db["t"].transform(drop=["name"])
assert fresh_db["t"].columns_dict == {"Age": int}
def test_transform_not_null_and_defaults_case_insensitive(fresh_db):
fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db.table("t").transform(not_null={"name"}, defaults={"age": 3})
columns = {c.name: c for c in fresh_db.table("t").columns}
fresh_db["t"].create({"Name": str, "Age": int})
fresh_db["t"].transform(not_null={"name"}, defaults={"age": 3})
columns = {c.name: c for c in fresh_db["t"].columns}
assert columns["Name"].notnull
assert fresh_db.table("t").default_values == {"Age": 3}
assert fresh_db["t"].default_values == {"Age": 3}
def test_transform_pk_case_insensitive(fresh_db):
fresh_db.table("t").create({"Id": int, "Name": str})
fresh_db.table("t").transform(pk="id")
assert fresh_db.table("t").pks == ["Id"]
assert fresh_db.table("t").columns_dict == {"Id": int, "Name": str}
fresh_db["t"].create({"Id": int, "Name": str})
fresh_db["t"].transform(pk="id")
assert fresh_db["t"].pks == ["Id"]
assert fresh_db["t"].columns_dict == {"Id": int, "Name": str}
def test_transform_drop_foreign_keys_case_insensitive(fresh_db):
fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db.table("child").create(
fresh_db["parent"].create({"Id": int}, pk="Id")
fresh_db["child"].create(
{"id": int, "Parent_ID": int},
pk="id",
foreign_keys=[("Parent_ID", "parent", "Id")],
)
fresh_db.table("child").transform(drop_foreign_keys=["parent_id"])
assert fresh_db.table("child").foreign_keys == []
fresh_db["child"].transform(drop_foreign_keys=["parent_id"])
assert fresh_db["child"].foreign_keys == []
def test_add_foreign_key_case_insensitive(fresh_db):
fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db.table("child").create({"id": int, "Parent_ID": int}, pk="id")
fresh_db.table("child").add_foreign_key("parent_id", "parent", "id")
fks = fresh_db.table("child").foreign_keys
fresh_db["parent"].create({"Id": int}, pk="Id")
fresh_db["child"].create({"id": int, "Parent_ID": int}, pk="id")
fresh_db["child"].add_foreign_key("parent_id", "parent", "id")
fks = fresh_db["child"].foreign_keys
assert len(fks) == 1
# The foreign key should use the schema casing of the columns
assert fks[0].column == "Parent_ID"
@ -153,83 +153,79 @@ def test_add_foreign_key_case_insensitive(fresh_db):
def test_add_foreign_keys_case_insensitive(fresh_db):
fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db.table("child").create({"id": int, "Parent_ID": int}, pk="id")
fresh_db["parent"].create({"Id": int}, pk="Id")
fresh_db["child"].create({"id": int, "Parent_ID": int}, pk="id")
fresh_db.add_foreign_keys([("child", "parent_id", "parent", "id")])
fks = fresh_db.table("child").foreign_keys
fks = fresh_db["child"].foreign_keys
assert len(fks) == 1
assert fks[0].column == "Parent_ID"
assert fks[0].other_column == "Id"
def test_add_foreign_key_detects_existing_case_insensitively(fresh_db):
fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db.table("child").create(
fresh_db["parent"].create({"Id": int}, pk="Id")
fresh_db["child"].create(
{"id": int, "Parent_ID": int},
pk="id",
foreign_keys=[("Parent_ID", "parent", "Id")],
)
# ignore=True should treat this as already existing, not add a duplicate
fresh_db.table("child").add_foreign_key("parent_id", "parent", "id", ignore=True)
assert len(fresh_db.table("child").foreign_keys) == 1
fresh_db["child"].add_foreign_key("parent_id", "parent", "id", ignore=True)
assert len(fresh_db["child"].foreign_keys) == 1
def test_add_column_fk_col_case_insensitive(fresh_db):
fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db.table("child").create({"id": int}, pk="id")
fresh_db.table("child").add_column("parent_id", int, fk="parent", fk_col="id")
fks = fresh_db.table("child").foreign_keys
fresh_db["parent"].create({"Id": int}, pk="Id")
fresh_db["child"].create({"id": int}, pk="id")
fresh_db["child"].add_column("parent_id", int, fk="parent", fk_col="id")
fks = fresh_db["child"].foreign_keys
assert len(fks) == 1
assert fks[0].other_column == "Id"
def test_extract_case_insensitive(fresh_db):
fresh_db.table("trees").insert({"id": 1, "Species": "Palm"}, pk="id")
fresh_db.table("trees").extract("species")
assert fresh_db.table("trees").columns_dict == {"id": int, "Species_id": int}
assert list(fresh_db.table("Species").rows) == [{"id": 1, "Species": "Palm"}]
fresh_db["trees"].insert({"id": 1, "Species": "Palm"}, pk="id")
fresh_db["trees"].extract("species")
assert fresh_db["trees"].columns_dict == {"id": int, "Species_id": int}
assert list(fresh_db["Species"].rows) == [{"id": 1, "Species": "Palm"}]
def test_convert_multi_case_insensitive(fresh_db):
fresh_db.table("t").insert({"id": 1, "Name": "Cleo"}, pk="id")
fresh_db.table("t").convert("name", lambda v: {"upper": v.upper()}, multi=True)
assert list(fresh_db.table("t").rows) == [
{"id": 1, "Name": "Cleo", "upper": "CLEO"}
]
fresh_db["t"].insert({"id": 1, "Name": "Cleo"}, pk="id")
fresh_db["t"].convert("name", lambda v: {"upper": v.upper()}, multi=True)
assert list(fresh_db["t"].rows) == [{"id": 1, "Name": "Cleo", "upper": "CLEO"}]
def test_convert_output_case_insensitive(fresh_db):
fresh_db.table("t").insert({"id": 1, "Name": "Cleo", "Upper": None}, pk="id")
fresh_db.table("t").convert("name", lambda v: v.upper(), output="upper")
assert list(fresh_db.table("t").rows) == [
{"id": 1, "Name": "Cleo", "Upper": "CLEO"}
]
fresh_db["t"].insert({"id": 1, "Name": "Cleo", "Upper": None}, pk="id")
fresh_db["t"].convert("name", lambda v: v.upper(), output="upper")
assert list(fresh_db["t"].rows) == [{"id": 1, "Name": "Cleo", "Upper": "CLEO"}]
def test_create_table_sql_pk_case_insensitive(fresh_db):
fresh_db.table("t").create({"Id": int, "Name": str}, pk="id")
fresh_db["t"].create({"Id": int, "Name": str}, pk="id")
# Should not have created an extra lowercase "id" column
assert fresh_db.table("t").columns_dict == {"Id": int, "Name": str}
assert fresh_db.table("t").pks == ["Id"]
assert fresh_db["t"].columns_dict == {"Id": int, "Name": str}
assert fresh_db["t"].pks == ["Id"]
def test_create_table_not_null_and_defaults_case_insensitive(fresh_db):
fresh_db.table("t").create(
fresh_db["t"].create(
{"Name": str, "Age": int}, not_null={"name"}, defaults={"age": 1}
)
columns = {c.name: c for c in fresh_db.table("t").columns}
columns = {c.name: c for c in fresh_db["t"].columns}
assert columns["Name"].notnull
assert fresh_db.table("t").default_values == {"Age": 1}
assert fresh_db["t"].default_values == {"Age": 1}
def test_create_table_foreign_keys_case_insensitive(fresh_db):
fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db.table("child").create(
fresh_db["parent"].create({"Id": int}, pk="Id")
fresh_db["child"].create(
{"id": int, "Parent_ID": int},
pk="id",
foreign_keys=[("parent_id", "parent", "id")],
)
fks = fresh_db.table("child").foreign_keys
fks = fresh_db["child"].foreign_keys
assert fks == [
ForeignKey(
table="child", column="Parent_ID", other_table="parent", other_column="Id"

View file

@ -20,8 +20,8 @@ def test_recursive_triggers_off():
def test_memory_name():
db1 = Database(memory_name="shared")
db2 = Database(memory_name="shared")
db1.table("dogs").insert({"name": "Cleo"})
assert list(db2.table("dogs").rows) == [{"name": "Cleo"}]
db1["dogs"].insert({"name": "Cleo"})
assert list(db2["dogs"].rows) == [{"name": "Cleo"}]
def test_sqlite_version():
@ -36,7 +36,7 @@ def test_sqlite_version():
def test_database_context_manager(tmpdir):
path = str(tmpdir / "test.db")
with Database(path) as db:
db.table("t").insert({"id": 1})
db["t"].insert({"id": 1})
# Raw writes commit automatically too
db.execute("insert into t (id) values (2)")
# An explicitly opened transaction left uncommitted on purpose:
@ -47,7 +47,7 @@ def test_database_context_manager(tmpdir):
db.execute("select 1")
# ... and the open explicit transaction was rolled back, not committed
db2 = Database(path)
assert [r["id"] for r in db2.table("t").rows] == [1, 2]
assert [r["id"] for r in db2["t"].rows] == [1, 2]
db2.close()
@ -83,12 +83,11 @@ def test_autocommit_connections_are_rejected(tmpdir, autocommit):
)
def test_legacy_transaction_control_connection_is_accepted(tmpdir):
conn = sqlite3.connect(
str(tmpdir / "test.db"),
autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL, # type: ignore[arg-type]
str(tmpdir / "test.db"), autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL
)
db = Database(conn)
db.table("t").insert({"id": 1}, pk="id")
assert [r["id"] for r in db.table("t").rows] == [1]
db["t"].insert({"id": 1}, pk="id")
assert [r["id"] for r in db["t"].rows] == [1]
db.close()

View file

@ -1,17 +1,17 @@
def test_insert_conversion(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"foo": "bar"}, conversions={"foo": "upper(?)"})
assert [{"foo": "BAR"}] == list(table.rows)
def test_insert_all_conversion(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert_all([{"foo": "bar"}], conversions={"foo": "upper(?)"})
assert [{"foo": "BAR"}] == list(table.rows)
def test_upsert_conversion(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert({"id": 1, "foo": "bar"}, pk="id", conversions={"foo": "upper(?)"})
assert [{"id": 1, "foo": "BAR"}] == list(table.rows)
table.upsert(
@ -21,7 +21,7 @@ def test_upsert_conversion(fresh_db):
def test_upsert_all_conversion(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert_all(
[{"id": 1, "foo": "bar"}], pk="id", conversions={"foo": "upper(?)"}
)
@ -29,7 +29,7 @@ def test_upsert_all_conversion(fresh_db):
def test_update_conversion(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"id": 5, "foo": "bar"}, pk="id")
table.update(5, {"foo": "baz"}, conversions={"foo": "upper(?)"})
assert [{"id": 5, "foo": "BAZ"}] == list(table.rows)

View file

@ -27,7 +27,7 @@ from sqlite_utils.db import BadMultiValues
),
)
def test_convert(fresh_db, columns, fn, expected):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"title": "Mixed Case", "abstract": "Abstract"})
table.convert(columns, fn)
assert list(table.rows) == [expected]
@ -37,7 +37,7 @@ def test_convert(fresh_db, columns, fn, expected):
"where,where_args", (("id > 1", None), ("id > :id", {"id": 1}), ("id > ?", [1]))
)
def test_convert_where(fresh_db, where, where_args):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert_all(
[
{"id": 1, "title": "One"},
@ -53,7 +53,7 @@ def test_convert_where(fresh_db, where, where_args):
def test_convert_handles_falsey_values(fresh_db):
# Falsey values like 0 should be converted (issue #527)
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert_all([{"x": 0}, {"x": 1}])
assert table.get(1)["x"] == 0
assert table.get(2)["x"] == 1
@ -70,14 +70,14 @@ def test_convert_handles_falsey_values(fresh_db):
),
)
def test_convert_output(fresh_db, drop, expected):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"title": "Mixed Case"})
table.convert("title", lambda v: v.upper(), output="other", drop=drop)
assert list(table.rows) == [expected]
def test_convert_output_multiple_column_error(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
with pytest.raises(ValueError) as excinfo:
table.convert(["title", "other"], lambda v: v, output="out")
assert "output= can only be used with a single column" in str(excinfo.value)
@ -91,14 +91,14 @@ def test_convert_output_multiple_column_error(fresh_db):
),
)
def test_convert_output_type(fresh_db, type, expected):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"number": "123"})
table.convert("number", lambda v: v, output="other", output_type=type, drop=True)
assert list(table.rows) == [expected]
def test_convert_multi(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"title": "Mixed Case"})
table.convert(
"title",
@ -123,7 +123,7 @@ def test_convert_multi(fresh_db):
def test_convert_multi_where(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert_all(
[
{"id": 1, "title": "One"},
@ -145,14 +145,14 @@ def test_convert_multi_where(fresh_db):
def test_convert_multi_exception(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"title": "Mixed Case"})
with pytest.raises(BadMultiValues):
table.convert("title", lambda v: v.upper(), multi=True)
def test_convert_repeated(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
col = "num"
table.insert({col: 1})
table.convert(col, lambda x: x * 2)

File diff suppressed because it is too large Load diff

View file

@ -8,12 +8,8 @@ from sqlite_utils.create_table_parser import (
Check,
ColumnComments,
ParseError,
Unique,
UniqueColumn,
parse_autoincrement,
parse_checks,
parse_column_comments,
parse_uniques,
)
@ -121,81 +117,6 @@ def test_virtual_table_has_no_checks():
)
@pytest.mark.parametrize(
"sql,expected",
[
(
"CREATE TABLE t(id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT)",
"id",
),
(
'CREATE TABLE t("quoted id" INTEGER PRIMARY KEY AUTOINCREMENT)',
"quoted id",
),
(
'CREATE TABLE t("autoincrement" INTEGER PRIMARY KEY, value TEXT)',
None,
),
(
"CREATE TABLE t(id INTEGER PRIMARY KEY /* AUTOINCREMENT */, value TEXT)",
None,
),
(
"CREATE TABLE t(id INTEGER PRIMARY KEY, value TEXT CHECK(value != 'AUTOINCREMENT'))",
None,
),
],
)
def test_parse_autoincrement(sql, expected):
sqlite3.connect(":memory:").execute(sql)
assert parse_autoincrement(sql) == expected
def test_parse_column_and_table_uniques():
sql = """
CREATE TABLE memberships (
email TEXT COLLATE RTRIM CONSTRAINT unique_email UNIQUE ON CONFLICT IGNORE,
account_id INTEGER,
CONSTRAINT unique_membership UNIQUE (
account_id DESC,
email COLLATE NOCASE ASC
) ON CONFLICT REPLACE
)
"""
sqlite3.connect(":memory:").execute(sql)
assert parse_uniques(sql) == [
Unique(
(UniqueColumn("email", collation="RTRIM"),),
name="unique_email",
column="email",
conflict="IGNORE",
),
Unique(
(
UniqueColumn("account_id", order="DESC"),
UniqueColumn("email", collation="NOCASE", order="ASC"),
),
name="unique_membership",
conflict="REPLACE",
),
]
uniques = parse_uniques(sql)
assert uniques[0].sql == "CONSTRAINT unique_email UNIQUE ON CONFLICT IGNORE"
assert sql[uniques[1].start : uniques[1].end] == uniques[1].sql
def test_unique_like_text_in_comments_and_checks_is_ignored():
sql = """
CREATE TABLE t (
value TEXT /* UNIQUE ON CONFLICT REPLACE */
CHECK(value != 'UNIQUE(other)'),
other TEXT
)
"""
sqlite3.connect(":memory:").execute(sql)
assert parse_uniques(sql) == []
comment_or_space = st.sampled_from(
[
" ",

View file

@ -32,9 +32,9 @@ EXAMPLES = [
@pytest.mark.parametrize("column_def,initial_value,expected_value", EXAMPLES)
def test_quote_default_value(fresh_db, column_def, initial_value, expected_value):
fresh_db.execute(f"create table foo (col {column_def})")
assert initial_value == fresh_db.table("foo").columns[0].default_value
assert initial_value == fresh_db["foo"].columns[0].default_value
assert expected_value == fresh_db.quote_default_value(
fresh_db.table("foo").columns[0].default_value
fresh_db["foo"].columns[0].default_value
)
@ -48,7 +48,7 @@ def test_insert_empty_record_uses_default_values(fresh_db):
)
""")
table = fresh_db.table("has_defaults")
table = fresh_db["has_defaults"]
table.insert({})
rows = list(table.rows)

View file

@ -2,7 +2,7 @@ import sqlite_utils
def test_delete_rowid_table(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"foo": 1})
rowid = table.insert({"foo": 2}).last_pk
table.delete(rowid)
@ -10,7 +10,7 @@ def test_delete_rowid_table(fresh_db):
def test_delete_pk_table(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"id": 1}, pk="id")
table.insert({"id": 2}, pk="id")
table.delete(1)
@ -18,7 +18,7 @@ def test_delete_pk_table(fresh_db):
def test_delete_where(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
for i in range(1, 11):
table.insert({"id": i}, pk="id")
assert table.count == 10
@ -27,7 +27,7 @@ def test_delete_where(fresh_db):
def test_delete_where_all(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
for i in range(1, 11):
table.insert({"id": i}, pk="id")
assert table.count == 10
@ -38,27 +38,27 @@ def test_delete_where_all(fresh_db):
def test_delete_where_commits(tmpdir):
path = str(tmpdir / "test.db")
db = sqlite_utils.Database(path)
db.table("table").insert_all([{"id": i} for i in range(5)], pk="id")
db.table("table").delete_where("id > ?", [2])
db["table"].insert_all([{"id": i} for i in range(5)], pk="id")
db["table"].delete_where("id > ?", [2])
# The connection must not be left inside an open transaction,
# otherwise subsequent atomic() blocks never commit either
assert not db.conn.in_transaction
db.table("table").insert({"id": 100})
db["table"].insert({"id": 100})
db.close()
db2 = sqlite_utils.Database(path)
assert [r["id"] for r in db2.table("table").rows] == [0, 1, 2, 100]
assert [r["id"] for r in db2["table"].rows] == [0, 1, 2, 100]
db2.close()
def test_delete_where_analyze(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert_all(({"id": i, "i": i} for i in range(10)), pk="id")
table.create_index(["i"], analyze=True)
assert "sqlite_stat1" in fresh_db.table_names()
assert list(fresh_db.table("sqlite_stat1").rows) == [
assert list(fresh_db["sqlite_stat1"].rows) == [
{"tbl": "table", "idx": "idx_table_i", "stat": "10 1"}
]
table.delete_where("id > ?", [5], analyze=True)
assert list(fresh_db.table("sqlite_stat1").rows) == [
assert list(fresh_db["sqlite_stat1"].rows) == [
{"tbl": "table", "idx": "idx_table_i", "stat": "6 1"}
]

View file

@ -22,7 +22,7 @@ def test_duplicate(fresh_db):
"bool_col": True,
"datetime_col": str(dt),
}
table1 = fresh_db.table("table1")
table1 = fresh_db["table1"]
row_id = table1.insert(data).last_rowid
# Duplicate table:
table2 = table1.duplicate("table2")
@ -40,4 +40,4 @@ def test_duplicate(fresh_db):
def test_duplicate_fails_if_table_does_not_exist(fresh_db):
with pytest.raises(NoTable):
fresh_db.table("not_a_table").duplicate("duplicated")
fresh_db["not_a_table"].duplicate("duplicated")

View file

@ -5,7 +5,7 @@ from sqlite_utils import Database, cli
def test_enable_counts_specific_table(fresh_db):
foo = fresh_db.table("foo")
foo = fresh_db["foo"]
assert fresh_db.table_names() == []
for i in range(10):
foo.insert({"name": f"item {i}"})
@ -41,24 +41,24 @@ def test_enable_counts_specific_table(fresh_db):
),
}
assert fresh_db.table_names() == ["foo", "_counts"]
assert list(fresh_db.table("_counts").rows) == [{"count": 10, "table": "foo"}]
assert list(fresh_db["_counts"].rows) == [{"count": 10, "table": "foo"}]
# Add some items to test the triggers
for i in range(5):
foo.insert({"name": f"item {10 + i}"})
assert foo.count == 15
assert list(fresh_db.table("_counts").rows) == [{"count": 15, "table": "foo"}]
assert list(fresh_db["_counts"].rows) == [{"count": 15, "table": "foo"}]
# Delete some items
foo.delete_where("rowid < 7")
assert foo.count == 9
assert list(fresh_db.table("_counts").rows) == [{"count": 9, "table": "foo"}]
assert list(fresh_db["_counts"].rows) == [{"count": 9, "table": "foo"}]
foo.delete_where()
assert foo.count == 0
assert list(fresh_db.table("_counts").rows) == [{"count": 0, "table": "foo"}]
assert list(fresh_db["_counts"].rows) == [{"count": 0, "table": "foo"}]
def test_enable_counts_all_tables(fresh_db):
foo = fresh_db.table("foo")
bar = fresh_db.table("bar")
foo = fresh_db["foo"]
bar = fresh_db["bar"]
foo.insert({"name": "Cleo"})
bar.insert({"name": "Cleo"})
foo.enable_fts(["name"])
@ -73,7 +73,7 @@ def test_enable_counts_all_tables(fresh_db):
"foo_fts_config",
"_counts",
}
assert list(fresh_db.table("_counts").rows) == [
assert list(fresh_db["_counts"].rows) == [
{"count": 1, "table": "foo"},
{"count": 1, "table": "bar"},
{"count": 3, "table": "foo_fts_data"},
@ -87,10 +87,10 @@ def test_enable_counts_all_tables(fresh_db):
def counts_db_path(tmpdir):
path = str(tmpdir / "test.db")
db = Database(path)
db.table("foo").insert({"name": "bar"})
db.table("bar").insert({"name": "bar"})
db.table("bar").insert({"name": "bar"})
db.table("baz").insert({"name": "bar"})
db["foo"].insert({"name": "bar"})
db["bar"].insert({"name": "bar"})
db["bar"].insert({"name": "bar"})
db["baz"].insert({"name": "bar"})
return path
@ -163,25 +163,25 @@ def test_uses_counts_after_enable_counts(counts_db_path):
def test_reset_counts(counts_db_path):
db = Database(counts_db_path)
db.table("foo").enable_counts()
db.table("bar").enable_counts()
db["foo"].enable_counts()
db["bar"].enable_counts()
assert db.cached_counts() == {"foo": 1, "bar": 2}
# Corrupt the value
db.table("_counts").update("foo", {"count": 3})
db["_counts"].update("foo", {"count": 3})
assert db.cached_counts() == {"foo": 3, "bar": 2}
assert db.table("foo").count == 3
assert db["foo"].count == 3
# Reset them
db.reset_counts()
assert db.cached_counts() == {"foo": 1, "bar": 2}
assert db.table("foo").count == 1
assert db["foo"].count == 1
def test_reset_counts_cli(counts_db_path):
db = Database(counts_db_path)
db.table("foo").enable_counts()
db.table("bar").enable_counts()
db["foo"].enable_counts()
db["bar"].enable_counts()
assert db.cached_counts() == {"foo": 1, "bar": 2}
db.table("_counts").update("foo", {"count": 3})
db["_counts"].update("foo", {"count": 3})
result = CliRunner().invoke(cli.cli, ["reset-counts", counts_db_path])
assert result.exit_code == 0
assert db.cached_counts() == {"foo": 1, "bar": 2}

View file

@ -2,7 +2,6 @@ import itertools
import pytest
from sqlite_utils import ANY
from sqlite_utils.db import InvalidColumns
@ -12,7 +11,7 @@ def test_extract_single_column(fresh_db, table, fk_column):
expected_table = table or "species"
expected_fk = fk_column or f"{expected_table}_id"
iter_species = itertools.cycle(["Palm", "Spruce", "Mangrove", "Oak"])
fresh_db.table("tree").insert_all(
fresh_db["tree"].insert_all(
(
{
"id": i,
@ -24,8 +23,8 @@ def test_extract_single_column(fresh_db, table, fk_column):
),
pk="id",
)
fresh_db.table("tree").extract("species", table=table, fk_column=fk_column)
assert fresh_db.table("tree").schema == (
fresh_db["tree"].extract("species", table=table, fk_column=fk_column)
assert fresh_db["tree"].schema == (
'CREATE TABLE "tree" (\n'
' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n'
@ -33,18 +32,18 @@ def test_extract_single_column(fresh_db, table, fk_column):
+ ' "end" INTEGER\n'
+ ")"
)
assert fresh_db.table(expected_table).schema == (
assert fresh_db[expected_table].schema == (
f'CREATE TABLE "{expected_table}" (\n' + ' "id" INTEGER PRIMARY KEY,\n'
' "species" TEXT\n'
")"
)
assert list(fresh_db.table(expected_table).rows) == [
assert list(fresh_db[expected_table].rows) == [
{"id": 1, "species": "Palm"},
{"id": 2, "species": "Spruce"},
{"id": 3, "species": "Mangrove"},
{"id": 4, "species": "Oak"},
]
assert list(itertools.islice(fresh_db.table("tree").rows, 0, 4)) == [
assert list(itertools.islice(fresh_db["tree"].rows, 0, 4)) == [
{"id": 1, "name": "Tree 1", expected_fk: 1, "end": 1},
{"id": 2, "name": "Tree 2", expected_fk: 2, "end": 1},
{"id": 3, "name": "Tree 3", expected_fk: 3, "end": 1},
@ -55,7 +54,7 @@ def test_extract_single_column(fresh_db, table, fk_column):
def test_extract_multiple_columns_with_rename(fresh_db):
iter_common = itertools.cycle(["Palm", "Spruce", "Mangrove", "Oak"])
iter_latin = itertools.cycle(["Arecaceae", "Picea", "Rhizophora", "Quercus"])
fresh_db.table("tree").insert_all(
fresh_db["tree"].insert_all(
(
{
"id": i,
@ -68,30 +67,30 @@ def test_extract_multiple_columns_with_rename(fresh_db):
pk="id",
)
fresh_db.table("tree").extract(
fresh_db["tree"].extract(
["common_name", "latin_name"], rename={"common_name": "name"}
)
assert fresh_db.table("tree").schema == (
assert fresh_db["tree"].schema == (
'CREATE TABLE "tree" (\n'
' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n'
' "common_name_latin_name_id" INTEGER REFERENCES "common_name_latin_name"("id")\n'
")"
)
assert fresh_db.table("common_name_latin_name").schema == (
assert fresh_db["common_name_latin_name"].schema == (
'CREATE TABLE "common_name_latin_name" (\n'
' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n'
' "latin_name" TEXT\n'
")"
)
assert list(fresh_db.table("common_name_latin_name").rows) == [
assert list(fresh_db["common_name_latin_name"].rows) == [
{"name": "Palm", "id": 1, "latin_name": "Arecaceae"},
{"name": "Spruce", "id": 2, "latin_name": "Picea"},
{"name": "Mangrove", "id": 3, "latin_name": "Rhizophora"},
{"name": "Oak", "id": 4, "latin_name": "Quercus"},
]
assert list(itertools.islice(fresh_db.table("tree").rows, 0, 4)) == [
assert list(itertools.islice(fresh_db["tree"].rows, 0, 4)) == [
{"id": 1, "name": "Tree 1", "common_name_latin_name_id": 1},
{"id": 2, "name": "Tree 2", "common_name_latin_name_id": 2},
{"id": 3, "name": "Tree 3", "common_name_latin_name_id": 3},
@ -100,7 +99,7 @@ def test_extract_multiple_columns_with_rename(fresh_db):
def test_extract_invalid_columns(fresh_db):
fresh_db.table("tree").insert(
fresh_db["tree"].insert(
{
"id": 1,
"name": "Tree 1",
@ -110,19 +109,19 @@ def test_extract_invalid_columns(fresh_db):
pk="id",
)
with pytest.raises(InvalidColumns):
fresh_db.table("tree").extract(["bad_column"])
fresh_db["tree"].extract(["bad_column"])
def test_extract_rowid_table(fresh_db):
fresh_db.table("tree").insert(
fresh_db["tree"].insert(
{
"name": "Tree 1",
"common_name": "Palm",
"latin_name": "Arecaceae",
}
)
fresh_db.table("tree").extract(["common_name", "latin_name"])
assert fresh_db.table("tree").schema == (
fresh_db["tree"].extract(["common_name", "latin_name"])
assert fresh_db["tree"].schema == (
'CREATE TABLE "tree" (\n'
' "name" TEXT,\n'
' "common_name_latin_name_id" INTEGER REFERENCES "common_name_latin_name"("id")\n'
@ -140,68 +139,68 @@ def test_extract_rowid_table(fresh_db):
def test_reuse_lookup_table(fresh_db):
fresh_db.table("species").insert({"id": 1, "name": "Wolf"}, pk="id")
fresh_db.table("sightings").insert({"id": 10, "species": "Wolf"}, pk="id")
fresh_db.table("individuals").insert(
fresh_db["species"].insert({"id": 1, "name": "Wolf"}, pk="id")
fresh_db["sightings"].insert({"id": 10, "species": "Wolf"}, pk="id")
fresh_db["individuals"].insert(
{"id": 10, "name": "Terriana", "species": "Fox"}, pk="id"
)
fresh_db.table("sightings").extract("species", rename={"species": "name"})
fresh_db.table("individuals").extract("species", rename={"species": "name"})
assert fresh_db.table("sightings").schema == (
fresh_db["sightings"].extract("species", rename={"species": "name"})
fresh_db["individuals"].extract("species", rename={"species": "name"})
assert fresh_db["sightings"].schema == (
'CREATE TABLE "sightings" (\n'
' "id" INTEGER PRIMARY KEY,\n'
' "species_id" INTEGER REFERENCES "species"("id")\n'
")"
)
assert fresh_db.table("individuals").schema == (
assert fresh_db["individuals"].schema == (
'CREATE TABLE "individuals" (\n'
' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n'
' "species_id" INTEGER REFERENCES "species"("id")\n'
")"
)
assert list(fresh_db.table("species").rows) == [
assert list(fresh_db["species"].rows) == [
{"id": 1, "name": "Wolf"},
{"id": 2, "name": "Fox"},
]
def test_extract_error_on_incompatible_existing_lookup_table(fresh_db):
fresh_db.table("species").insert({"id": 1})
fresh_db.table("tree").insert({"name": "Tree 1", "common_name": "Palm"})
fresh_db["species"].insert({"id": 1})
fresh_db["tree"].insert({"name": "Tree 1", "common_name": "Palm"})
with pytest.raises(InvalidColumns):
fresh_db.table("tree").extract("common_name", table="species")
fresh_db["tree"].extract("common_name", table="species")
# Try again with incompatible existing column type
fresh_db.table("species2").insert({"id": 1, "common_name": 3.5})
fresh_db["species2"].insert({"id": 1, "common_name": 3.5})
with pytest.raises(InvalidColumns):
fresh_db.table("tree").extract("common_name", table="species2")
fresh_db["tree"].extract("common_name", table="species2")
def test_extract_works_with_null_values(fresh_db):
fresh_db.table("listens").insert_all(
fresh_db["listens"].insert_all(
[
{"id": 1, "track_title": "foo", "album_title": "bar"},
{"id": 2, "track_title": "baz", "album_title": None},
],
pk="id",
)
fresh_db.table("listens").extract(
fresh_db["listens"].extract(
columns=["album_title"], table="albums", fk_column="album_id"
)
assert list(fresh_db.table("listens").rows) == [
assert list(fresh_db["listens"].rows) == [
{"id": 1, "track_title": "foo", "album_id": 1},
{"id": 2, "track_title": "baz", "album_id": None},
]
assert list(fresh_db.table("albums").rows) == [
assert list(fresh_db["albums"].rows) == [
{"id": 1, "album_title": "bar"},
]
def test_extract_null_values_single_column(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/186
fresh_db.table("species").insert({"id": 1, "species": "Wolf"}, pk="id")
fresh_db.table("individuals").insert_all(
fresh_db["species"].insert({"id": 1, "species": "Wolf"}, pk="id")
fresh_db["individuals"].insert_all(
[
{"id": 10, "name": "Terriana", "species": "Fox"},
{"id": 11, "name": "Spenidorm", "species": None},
@ -211,13 +210,13 @@ def test_extract_null_values_single_column(fresh_db):
],
pk="id",
)
fresh_db.table("individuals").extract("species")
fresh_db["individuals"].extract("species")
# No null row should have been added to species
assert list(fresh_db.table("species").rows) == [
assert list(fresh_db["species"].rows) == [
{"id": 1, "species": "Wolf"},
{"id": 2, "species": "Fox"},
]
assert list(fresh_db.table("individuals").rows) == [
assert list(fresh_db["individuals"].rows) == [
{"id": 10, "name": "Terriana", "species_id": 2},
{"id": 11, "name": "Spenidorm", "species_id": None},
{"id": 12, "name": "Grantheim", "species_id": 1},
@ -229,7 +228,7 @@ def test_extract_null_values_single_column(fresh_db):
def test_extract_null_values_multiple_columns(fresh_db):
# A row should be extracted if at least one column is not null -
# only rows where ALL extracted columns are null are left alone
fresh_db.table("circulation").insert_all(
fresh_db["circulation"].insert_all(
[
{"id": 1, "title": "title one", "creator": "creator one", "year": 2018},
{"id": 2, "title": "title two", "creator": None, "year": 2019},
@ -238,14 +237,14 @@ def test_extract_null_values_multiple_columns(fresh_db):
],
pk="id",
)
fresh_db.table("circulation").extract(
fresh_db["circulation"].extract(
["title", "creator"], table="books", fk_column="book_id"
)
assert list(fresh_db.table("books").rows) == [
assert list(fresh_db["books"].rows) == [
{"id": 1, "title": "title one", "creator": "creator one"},
{"id": 2, "title": "title two", "creator": None},
]
assert list(fresh_db.table("circulation").rows) == [
assert list(fresh_db["circulation"].rows) == [
{"id": 1, "book_id": 1, "year": 2018},
{"id": 2, "book_id": 2, "year": 2019},
{"id": 3, "book_id": None, "year": 2020},
@ -256,20 +255,20 @@ def test_extract_null_values_multiple_columns(fresh_db):
def test_extract_null_values_existing_lookup_table_with_null_row(fresh_db):
# Even if the lookup table already contains an all-null row, rows where
# every extracted column is null should keep a null foreign key
fresh_db.table("species").insert({"id": 1, "species": None}, pk="id")
fresh_db.table("individuals").insert_all(
fresh_db["species"].insert({"id": 1, "species": None}, pk="id")
fresh_db["individuals"].insert_all(
[
{"id": 10, "name": "Terriana", "species": "Fox"},
{"id": 11, "name": "Spenidorm", "species": None},
],
pk="id",
)
fresh_db.table("individuals").extract("species")
assert list(fresh_db.table("species").rows) == [
fresh_db["individuals"].extract("species")
assert list(fresh_db["species"].rows) == [
{"id": 1, "species": None},
{"id": 2, "species": "Fox"},
]
assert list(fresh_db.table("individuals").rows) == [
assert list(fresh_db["individuals"].rows) == [
{"id": 10, "name": "Terriana", "species_id": 2},
{"id": 11, "name": "Spenidorm", "species_id": None},
]
@ -280,19 +279,17 @@ def test_extract_repeated_into_shared_lookup_with_nulls(fresh_db):
# cannot dedupe NULL-containing rows against the existing lookup
# table - extracting a second table into the same lookup previously
# inserted duplicate rows that nothing pointed to
fresh_db.table("t1").insert_all(
fresh_db["t1"].insert_all(
[
{"id": 1, "species": None, "common": "X"},
{"id": 2, "species": "Oak", "common": "Oak"},
],
pk="id",
)
fresh_db.table("t2").insert_all(
[{"id": 1, "species": None, "common": "X"}], pk="id"
)
fresh_db.table("t1").extract(["species", "common"], table="lk")
fresh_db.table("t2").extract(["species", "common"], table="lk")
assert fresh_db.table("lk").count == 2
fresh_db["t2"].insert_all([{"id": 1, "species": None, "common": "X"}], pk="id")
fresh_db["t1"].extract(["species", "common"], table="lk")
fresh_db["t2"].extract(["species", "common"], table="lk")
assert fresh_db["lk"].count == 2
# Both tables point at the same lookup row
t1_fk = fresh_db.execute("select lk_id from t1 where id = 1").fetchone()[0]
t2_fk = fresh_db.execute("select lk_id from t2 where id = 1").fetchone()[0]
@ -301,43 +298,8 @@ def test_extract_repeated_into_shared_lookup_with_nulls(fresh_db):
def test_extract_repeated_into_shared_lookup_no_nulls(fresh_db):
# Non-NULL rows were already deduped by the unique index - keep it so
fresh_db.table("t1").insert_all([{"id": 1, "species": "Oak"}], pk="id")
fresh_db.table("t2").insert_all([{"id": 1, "species": "Oak"}], pk="id")
fresh_db.table("t1").extract(["species"], table="lk")
fresh_db.table("t2").extract(["species"], table="lk")
assert fresh_db.table("lk").count == 1
def test_extract_preserves_strict_any(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (id integer primary key, data any) strict")
fresh_db.execute("insert into items values (1, ?)", ("000123",))
fresh_db["items"].extract("data", table="data_values")
lookup = fresh_db["data_values"]
assert lookup.strict is True
assert lookup.columns_dict == {"id": int, "data": ANY}
assert fresh_db.execute(
"select typeof(data), data from data_values"
).fetchone() == ("text", "000123")
def test_extract_strict_any_rejects_non_strict_lookup(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (data any) strict")
fresh_db.execute("insert into items values (?)", ("000123",))
fresh_db.execute("create table data_values (id integer primary key, data any)")
with pytest.raises(
InvalidColumns,
match="is not STRICT, so it cannot preserve ANY column values",
):
fresh_db["items"].extract("data", table="data_values")
assert fresh_db.execute("select typeof(data), data from items").fetchone() == (
"text",
"000123",
)
fresh_db["t1"].insert_all([{"id": 1, "species": "Oak"}], pk="id")
fresh_db["t2"].insert_all([{"id": 1, "species": "Oak"}], pk="id")
fresh_db["t1"].extract(["species"], table="lk")
fresh_db["t2"].extract(["species"], table="lk")
assert fresh_db["lk"].count == 1

View file

@ -32,15 +32,15 @@ def test_extracts(fresh_db, kwargs, expected_table, use_table_factory):
assert {expected_table, "Trees"} == set(fresh_db.table_names())
assert (
f'CREATE TABLE "{expected_table}" (\n "id" INTEGER PRIMARY KEY,\n "value" TEXT\n)'
== fresh_db.table(expected_table).schema
== fresh_db[expected_table].schema
)
assert (
f'CREATE TABLE "Trees" (\n "id" INTEGER,\n "species_id" INTEGER REFERENCES "{expected_table}"("id")\n)'
== fresh_db.table("Trees").schema
== fresh_db["Trees"].schema
)
# Should have a foreign key reference
assert len(fresh_db.table("Trees").foreign_keys) == 1
fk = fresh_db.table("Trees").foreign_keys[0]
assert len(fresh_db["Trees"].foreign_keys) == 1
fk = fresh_db["Trees"].foreign_keys[0]
assert fk.table == "Trees"
assert fk.column == "species_id"
@ -54,22 +54,22 @@ def test_extracts(fresh_db, kwargs, expected_table, use_table_factory):
partial=0,
columns=["value"],
)
] == fresh_db.table(expected_table).indexes
] == fresh_db[expected_table].indexes
# Finally, check the rows
assert [{"id": 1, "value": "Oak"}, {"id": 2, "value": "Palm"}] == list(
fresh_db.table(expected_table).rows
fresh_db[expected_table].rows
)
assert [
{"id": 1, "species_id": 1},
{"id": 2, "species_id": 1},
{"id": 3, "species_id": 2},
] == list(fresh_db.table("Trees").rows)
] == list(fresh_db["Trees"].rows)
def test_extracts_null_values(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/186
# Null values should stay null, not be extracted into the lookup table
fresh_db.table("Trees").insert_all(
fresh_db["Trees"].insert_all(
[
{"id": 1, "species_id": "Oak"},
{"id": 2, "species_id": None},
@ -78,11 +78,11 @@ def test_extracts_null_values(fresh_db):
],
extracts={"species_id": "Species"},
)
assert list(fresh_db.table("Species").rows) == [
assert list(fresh_db["Species"].rows) == [
{"id": 1, "value": "Oak"},
{"id": 2, "value": "Palm"},
]
assert list(fresh_db.table("Trees").rows) == [
assert list(fresh_db["Trees"].rows) == [
{"id": 1, "species_id": 1},
{"id": 2, "species_id": None},
{"id": 3, "species_id": 2},
@ -92,7 +92,7 @@ def test_extracts_null_values(fresh_db):
def test_extracts_null_values_list_mode(fresh_db):
# Same as test_extracts_null_values but for list-based records
fresh_db.table("Trees").insert_all(
fresh_db["Trees"].insert_all(
[
["id", "species_id"],
[1, "Oak"],
@ -102,11 +102,11 @@ def test_extracts_null_values_list_mode(fresh_db):
],
extracts={"species_id": "Species"},
)
assert list(fresh_db.table("Species").rows) == [
assert list(fresh_db["Species"].rows) == [
{"id": 1, "value": "Oak"},
{"id": 2, "value": "Palm"},
]
assert list(fresh_db.table("Trees").rows) == [
assert list(fresh_db["Trees"].rows) == [
{"id": 1, "species_id": 1},
{"id": 2, "species_id": None},
{"id": 3, "species_id": 2},

View file

@ -32,7 +32,7 @@ def compound_db():
def test_compound_foreign_key(compound_db):
fks = compound_db.table("courses").foreign_keys
fks = compound_db["courses"].foreign_keys
assert len(fks) == 1
fk = fks[0]
assert fk.is_compound is True
@ -46,10 +46,10 @@ def test_compound_foreign_key(compound_db):
def test_single_foreign_key_gets_columns_fields(fresh_db):
fresh_db.table("authors").insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db.table("books").insert({"title": "Hedgehogs", "author_id": 1})
fresh_db.table("books").add_foreign_key("author_id", "authors", "id")
fk = fresh_db.table("books").foreign_keys[0]
fresh_db["authors"].insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db["books"].insert({"title": "Hedgehogs", "author_id": 1})
fresh_db["books"].add_foreign_key("author_id", "authors", "id")
fk = fresh_db["books"].foreign_keys[0]
assert fk.is_compound is False
assert fk.column == "author_id"
assert fk.other_column == "id"
@ -60,10 +60,10 @@ def test_single_foreign_key_gets_columns_fields(fresh_db):
def test_foreign_key_no_longer_unpacks_as_tuple(fresh_db):
# Clean break in 4.0: ForeignKey is a dataclass, not a namedtuple, so the
# old tuple unpacking and indexing patterns now fail hard.
fresh_db.table("authors").insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db.table("books").insert({"title": "Hedgehogs", "author_id": 1})
fresh_db.table("books").add_foreign_key("author_id", "authors", "id")
fk = fresh_db.table("books").foreign_keys[0]
fresh_db["authors"].insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db["books"].insert({"title": "Hedgehogs", "author_id": 1})
fresh_db["books"].add_foreign_key("author_id", "authors", "id")
fk = fresh_db["books"].foreign_keys[0]
with pytest.raises(TypeError):
_table, _column, _other_table, _other_column = fk
with pytest.raises(TypeError):
@ -71,18 +71,16 @@ def test_foreign_key_no_longer_unpacks_as_tuple(fresh_db):
def test_foreign_keys_are_sortable(fresh_db):
fresh_db.table("authors").insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db.table("categories").insert({"id": 1, "name": "Wildlife"}, pk="id")
fresh_db.table("books").insert(
{"title": "Hedgehogs", "author_id": 1, "category_id": 1}
)
fresh_db["authors"].insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db["categories"].insert({"id": 1, "name": "Wildlife"}, pk="id")
fresh_db["books"].insert({"title": "Hedgehogs", "author_id": 1, "category_id": 1})
fresh_db.add_foreign_keys(
[
("books", "author_id", "authors", "id"),
("books", "category_id", "categories", "id"),
]
)
fks = sorted(fresh_db.table("books").foreign_keys)
fks = sorted(fresh_db["books"].foreign_keys)
assert fks[0].column == "author_id"
assert fks[1].column == "category_id"
@ -107,7 +105,7 @@ def test_mixed_compound_and_single_foreign_keys_are_sortable():
REFERENCES departments(campus_name, dept_code)
);
""")
fks = db.table("courses").foreign_keys
fks = db["courses"].foreign_keys
assert len(fks) == 2
assert {fk.is_compound for fk in fks} == {True, False}
fks_sorted = sorted(fks)
@ -165,8 +163,8 @@ def test_create_table_with_compound_foreign_key(departments_db, foreign_keys):
pk="course_code",
foreign_keys=foreign_keys,
)
assert departments_db.table("courses").schema == EXPECTED_COURSES_SCHEMA
fks = departments_db.table("courses").foreign_keys
assert departments_db["courses"].schema == EXPECTED_COURSES_SCHEMA
fks = departments_db["courses"].foreign_keys
assert len(fks) == 1
fk = fks[0]
assert fk.is_compound is True
@ -183,10 +181,10 @@ def test_create_table_compound_foreign_key_enforced(departments_db):
pk="course_code",
foreign_keys=[(("campus_name", "dept_code"), "departments")],
)
departments_db.table("departments").insert(
departments_db["departments"].insert(
{"campus_name": "Berkeley", "dept_code": "CS", "dept_name": "Computer Science"}
)
departments_db.table("courses").insert(
departments_db["courses"].insert(
{"course_code": "CS101", "campus_name": "Berkeley", "dept_code": "CS"}
)
with pytest.raises(sqlite3.IntegrityError):
@ -209,8 +207,8 @@ def test_create_table_compound_foreign_key_missing_other_column(departments_db):
def test_transform_preserves_compound_foreign_key(compound_db):
compound_db.table("courses").transform(rename={"course_name": "title"})
fks = compound_db.table("courses").foreign_keys
compound_db["courses"].transform(rename={"course_name": "title"})
fks = compound_db["courses"].foreign_keys
assert len(fks) == 1
fk = fks[0]
assert fk.is_compound is True
@ -220,8 +218,8 @@ def test_transform_preserves_compound_foreign_key(compound_db):
def test_transform_rename_member_column_updates_compound_foreign_key(compound_db):
compound_db.table("courses").transform(rename={"campus_name": "campus"})
fks = compound_db.table("courses").foreign_keys
compound_db["courses"].transform(rename={"campus_name": "campus"})
fks = compound_db["courses"].foreign_keys
assert len(fks) == 1
fk = fks[0]
assert fk.is_compound is True
@ -233,9 +231,9 @@ def test_transform_rename_member_column_updates_compound_foreign_key(compound_db
def test_transform_drop_member_column_drops_compound_foreign_key(compound_db):
# Matches single-column behavior: dropping the column silently
# drops the foreign key that used it
compound_db.table("courses").transform(drop={"dept_code"})
assert compound_db.table("courses").foreign_keys == []
assert "FOREIGN KEY" not in compound_db.table("courses").schema
compound_db["courses"].transform(drop={"dept_code"})
assert compound_db["courses"].foreign_keys == []
assert "FOREIGN KEY" not in compound_db["courses"].schema
@pytest.mark.parametrize(
@ -248,11 +246,11 @@ def test_transform_drop_member_column_drops_compound_foreign_key(compound_db):
),
)
def test_transform_drop_compound_foreign_key(compound_db, drop_foreign_keys):
compound_db.table("courses").transform(drop_foreign_keys=drop_foreign_keys)
assert compound_db.table("courses").foreign_keys == []
compound_db["courses"].transform(drop_foreign_keys=drop_foreign_keys)
assert compound_db["courses"].foreign_keys == []
# The columns themselves survive
assert {"campus_name", "dept_code"} <= set(
compound_db.table("courses").columns_dict.keys()
compound_db["courses"].columns_dict.keys()
)
@ -267,12 +265,12 @@ def courses_db(departments_db):
def test_add_compound_foreign_key(courses_db):
t = courses_db.table("courses").add_foreign_key(
t = courses_db["courses"].add_foreign_key(
("campus_name", "dept_code"), "departments", ("campus_name", "dept_code")
)
# Returns self
assert t.name == "courses"
fks = courses_db.table("courses").foreign_keys
fks = courses_db["courses"].foreign_keys
assert len(fks) == 1
fk = fks[0]
assert fk.is_compound is True
@ -283,33 +281,27 @@ def test_add_compound_foreign_key(courses_db):
def test_add_compound_foreign_key_guesses_other_columns(courses_db):
# Lists work here too, though tuples are the documented form
courses_db.table("courses").add_foreign_key(
["campus_name", "dept_code"], "departments"
)
fk = courses_db.table("courses").foreign_keys[0]
courses_db["courses"].add_foreign_key(["campus_name", "dept_code"], "departments")
fk = courses_db["courses"].foreign_keys[0]
assert fk.other_columns == ("campus_name", "dept_code")
def test_add_compound_foreign_key_error_if_already_exists(courses_db):
courses_db.table("courses").add_foreign_key(
("campus_name", "dept_code"), "departments"
)
courses_db["courses"].add_foreign_key(("campus_name", "dept_code"), "departments")
with pytest.raises(AlterError) as ex:
courses_db.table("courses").add_foreign_key(
courses_db["courses"].add_foreign_key(
("campus_name", "dept_code"), "departments"
)
assert "already exists" in ex.value.args[0]
# ignore=True should not raise
courses_db.table("courses").add_foreign_key(
courses_db["courses"].add_foreign_key(
("campus_name", "dept_code"), "departments", ignore=True
)
def test_add_compound_foreign_key_error_if_column_missing(courses_db):
with pytest.raises(AlterError):
courses_db.table("courses").add_foreign_key(
("campus_name", "nope"), "departments"
)
courses_db["courses"].add_foreign_key(("campus_name", "nope"), "departments")
def test_db_add_foreign_keys_compound(courses_db):
@ -323,14 +315,14 @@ def test_db_add_foreign_keys_compound(courses_db):
)
]
)
fk = courses_db.table("courses").foreign_keys[0]
fk = courses_db["courses"].foreign_keys[0]
assert fk.is_compound is True
assert fk.columns == ("campus_name", "dept_code")
def test_index_foreign_keys_compound_creates_composite_index(compound_db):
compound_db.index_foreign_keys()
index_columns = [i.columns for i in compound_db.table("courses").indexes]
index_columns = [i.columns for i in compound_db["courses"].indexes]
assert ["campus_name", "dept_code"] in index_columns
# No separate single-column indexes for the members
assert ["campus_name"] not in index_columns
@ -347,22 +339,22 @@ def test_foreign_key_captures_on_delete_and_on_update():
ON DELETE CASCADE ON UPDATE RESTRICT
);
""")
fk = db.table("books").foreign_keys[0]
fk = db["books"].foreign_keys[0]
assert fk.on_delete == "CASCADE"
assert fk.on_update == "RESTRICT"
def test_foreign_key_on_delete_defaults_to_no_action(fresh_db):
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db.table("books").add_foreign_key("author_id", "authors", "id")
fk = fresh_db.table("books").foreign_keys[0]
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["books"].add_foreign_key("author_id", "authors", "id")
fk = fresh_db["books"].foreign_keys[0]
assert fk.on_delete == "NO ACTION"
assert fk.on_update == "NO ACTION"
def test_create_table_foreign_key_with_on_delete(fresh_db):
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db.create_table(
"books",
{"id": int, "author_id": int},
@ -377,8 +369,8 @@ def test_create_table_foreign_key_with_on_delete(fresh_db):
)
],
)
assert "ON DELETE CASCADE" in fresh_db.table("books").schema
assert fresh_db.table("books").foreign_keys[0].on_delete == "CASCADE"
assert "ON DELETE CASCADE" in fresh_db["books"].schema
assert fresh_db["books"].foreign_keys[0].on_delete == "CASCADE"
def test_transform_preserves_on_delete_cascade():
@ -391,11 +383,11 @@ def test_transform_preserves_on_delete_cascade():
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
);
""")
db.table("books").transform(rename={"title": "book_title"})
fk = db.table("books").foreign_keys[0]
db["books"].transform(rename={"title": "book_title"})
fk = db["books"].foreign_keys[0]
assert fk.on_delete == "CASCADE"
assert fk.on_update == "NO ACTION"
assert "ON DELETE CASCADE" in db.table("books").schema
assert "ON DELETE CASCADE" in db["books"].schema
def test_transform_preserves_compound_foreign_key_on_delete():
@ -414,11 +406,11 @@ def test_transform_preserves_compound_foreign_key_on_delete():
REFERENCES departments(campus_name, dept_code) ON DELETE CASCADE
);
""")
db.table("courses").transform(rename={"course_code": "code"})
fk = db.table("courses").foreign_keys[0]
db["courses"].transform(rename={"course_code": "code"})
fk = db["courses"].foreign_keys[0]
assert fk.is_compound is True
assert fk.on_delete == "CASCADE"
assert "ON DELETE CASCADE" in db.table("courses").schema
assert "ON DELETE CASCADE" in db["courses"].schema
def test_implicit_primary_key_reference_is_resolved():
@ -432,7 +424,7 @@ def test_implicit_primary_key_reference_is_resolved():
author_id INTEGER REFERENCES authors
);
""")
fk = db.table("books").foreign_keys[0]
fk = db["books"].foreign_keys[0]
assert fk.is_compound is False
assert fk.other_column == "author_id"
assert fk.other_columns == ("author_id",)
@ -453,7 +445,7 @@ def test_implicit_compound_primary_key_reference_is_resolved():
FOREIGN KEY (campus_name, dept_code) REFERENCES departments
);
""")
fk = db.table("courses").foreign_keys[0]
fk = db["courses"].foreign_keys[0]
assert fk.is_compound is True
assert fk.other_columns == ("campus_name", "dept_code")
@ -478,14 +470,14 @@ def test_add_foreign_keys_preserves_actions(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/594 review finding:
# ForeignKey objects passed to db.add_foreign_keys() were flattened
# to plain tuples, losing on_delete/on_update
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id")
fresh_db.add_foreign_keys(
[ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")]
)
fk = fresh_db.table("books").foreign_keys[0]
fk = fresh_db["books"].foreign_keys[0]
assert fk.on_delete == "CASCADE"
assert "ON DELETE CASCADE" in fresh_db.table("books").schema
assert "ON DELETE CASCADE" in fresh_db["books"].schema
def test_add_foreign_keys_preserves_actions_compound(courses_db):
@ -503,36 +495,36 @@ def test_add_foreign_keys_preserves_actions_compound(courses_db):
)
]
)
fk = courses_db.table("courses").foreign_keys[0]
fk = courses_db["courses"].foreign_keys[0]
assert fk.is_compound is True
assert fk.on_delete == "CASCADE"
assert "ON DELETE CASCADE" in courses_db.table("courses").schema
assert "ON DELETE CASCADE" in courses_db["courses"].schema
def test_add_foreign_key_on_delete_on_update(fresh_db):
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db.table("books").add_foreign_key(
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["books"].add_foreign_key(
"author_id", "authors", "id", on_delete="CASCADE", on_update="RESTRICT"
)
fk = fresh_db.table("books").foreign_keys[0]
fk = fresh_db["books"].foreign_keys[0]
assert fk.on_delete == "CASCADE"
assert fk.on_update == "RESTRICT"
assert "ON UPDATE RESTRICT ON DELETE CASCADE" in fresh_db.table("books").schema
assert "ON UPDATE RESTRICT ON DELETE CASCADE" in fresh_db["books"].schema
# The cascade should actually fire
fresh_db.execute("PRAGMA foreign_keys = ON")
fresh_db.execute("delete from authors where id = 1")
assert fresh_db.table("books").count == 0
assert fresh_db["books"].count == 0
def test_add_compound_foreign_key_on_delete(courses_db):
courses_db.table("courses").add_foreign_key(
courses_db["courses"].add_foreign_key(
("campus_name", "dept_code"), "departments", on_delete="SET NULL"
)
fk = courses_db.table("courses").foreign_keys[0]
fk = courses_db["courses"].foreign_keys[0]
assert fk.is_compound is True
assert fk.on_delete == "SET NULL"
assert "ON DELETE SET NULL" in courses_db.table("courses").schema
assert "ON DELETE SET NULL" in courses_db["courses"].schema
def test_implicit_compound_foreign_key_resolves_pk_declaration_order(fresh_db):
@ -544,7 +536,7 @@ def test_implicit_compound_foreign_key_resolves_pk_declaration_order(fresh_db):
fresh_db.execute(
"create table child (x text, y text, foreign key (x, y) references other)"
)
fk = fresh_db.table("child").foreign_keys[0]
fk = fresh_db["child"].foreign_keys[0]
assert fk.other_columns == ("a", "b")
@ -557,46 +549,46 @@ def test_transform_implicit_compound_foreign_key_stays_valid(fresh_db):
"create table child (x text, y text, foreign key (x, y) references other)"
)
fresh_db.execute("PRAGMA foreign_keys = ON")
fresh_db.table("other").insert({"a": "A", "b": "B"})
fresh_db.table("child").insert({"x": "A", "y": "B"})
fresh_db.table("child").transform(types={"x": str})
assert fresh_db.table("child").foreign_keys[0].other_columns == ("a", "b")
fresh_db["other"].insert({"a": "A", "b": "B"})
fresh_db["child"].insert({"x": "A", "y": "B"})
fresh_db["child"].transform(types={"x": str})
assert fresh_db["child"].foreign_keys[0].other_columns == ("a", "b")
# The constraint still points the right way around
fresh_db.table("child").insert({"x": "A", "y": "B"})
fresh_db["child"].insert({"x": "A", "y": "B"})
with pytest.raises(sqlite3.IntegrityError):
fresh_db.table("child").insert({"x": "B", "y": "A"})
fresh_db["child"].insert({"x": "B", "y": "A"})
def test_create_compound_foreign_key_guesses_pk_declaration_order(fresh_db):
fresh_db.execute("create table other (b text, a text, primary key (a, b))")
fresh_db.table("other").insert({"a": "A", "b": "B"})
fresh_db.table("child").create(
fresh_db["other"].insert({"a": "A", "b": "B"})
fresh_db["child"].create(
{"id": int, "x": str, "y": str},
pk="id",
foreign_keys=[(("x", "y"), "other")],
)
assert fresh_db.table("child").foreign_keys[0].other_columns == ("a", "b")
assert fresh_db["child"].foreign_keys[0].other_columns == ("a", "b")
fresh_db.execute("PRAGMA foreign_keys = ON")
fresh_db.table("child").insert({"id": 1, "x": "A", "y": "B"})
fresh_db["child"].insert({"id": 1, "x": "A", "y": "B"})
with pytest.raises(sqlite3.IntegrityError):
fresh_db.table("child").insert({"id": 2, "x": "B", "y": "A"})
fresh_db["child"].insert({"id": 2, "x": "B", "y": "A"})
def test_add_compound_foreign_key_guesses_pk_declaration_order(fresh_db):
fresh_db.execute("create table other (b text, a text, primary key (a, b))")
fresh_db.table("child").insert({"id": 1, "x": "A", "y": "B"}, pk="id")
fresh_db.table("child").add_foreign_key(("x", "y"), "other")
assert fresh_db.table("child").foreign_keys[0].other_columns == ("a", "b")
fresh_db["child"].insert({"id": 1, "x": "A", "y": "B"}, pk="id")
fresh_db["child"].add_foreign_key(("x", "y"), "other")
assert fresh_db["child"].foreign_keys[0].other_columns == ("a", "b")
def test_foreign_keys_are_hashable(fresh_db):
# set() over foreign_keys worked with the 3.x namedtuple and must
# keep working with the dataclass
fresh_db.table("p").insert({"id": 1}, pk="id")
fresh_db.table("c").insert(
fresh_db["p"].insert({"id": 1}, pk="id")
fresh_db["c"].insert(
{"id": 1, "pid": 1}, pk="id", foreign_keys=[("pid", "p", "id")]
)
fks = set(fresh_db.table("c").foreign_keys)
fks = set(fresh_db["c"].foreign_keys)
assert len(fks) == 1
assert ForeignKey("c", "pid", "p", "id") in fks
# Usable as dict keys too
@ -608,7 +600,7 @@ def test_foreign_key_is_immutable():
fk = ForeignKey("c", "pid", "p", "id")
with pytest.raises(dataclasses.FrozenInstanceError):
setattr(fk, "table", "other")
fk.table = "other"
def test_foreign_key_equality_and_hash_include_actions():
@ -625,9 +617,9 @@ def test_create_table_mixed_foreign_keys_list(fresh_db):
# 3.x accepted a mix of ForeignKey objects, tuples and bare column
# strings in foreign_keys= (ForeignKey was a namedtuple, so it passed
# the tuple check) - keep accepting the mix
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("publishers").insert({"id": 1}, pk="id")
fresh_db.table("books").create(
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["publishers"].insert({"id": 1}, pk="id")
fresh_db["books"].create(
{"id": int, "author_id": int, "publisher_id": int},
pk="id",
foreign_keys=[
@ -635,14 +627,14 @@ def test_create_table_mixed_foreign_keys_list(fresh_db):
("publisher_id", "publishers", "id"),
],
)
fks = {fk.column: fk.other_table for fk in fresh_db.table("books").foreign_keys}
fks = {fk.column: fk.other_table for fk in fresh_db["books"].foreign_keys}
assert fks == {"author_id": "authors", "publisher_id": "publishers"}
def test_create_table_mixed_foreign_keys_with_string(fresh_db):
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("publishers").insert({"id": 1}, pk="id")
fresh_db.table("books").create(
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["publishers"].insert({"id": 1}, pk="id")
fresh_db["books"].create(
{"id": int, "author_id": int, "publisher_id": int},
pk="id",
foreign_keys=[
@ -650,15 +642,15 @@ def test_create_table_mixed_foreign_keys_with_string(fresh_db):
("publisher_id", "publishers", "id"),
],
)
fks = {fk.column: fk.other_table for fk in fresh_db.table("books").foreign_keys}
fks = {fk.column: fk.other_table for fk in fresh_db["books"].foreign_keys}
assert fks == {"author_id": "authors", "publisher_id": "publishers"}
def test_add_foreign_keys_existing_with_different_actions_errors(fresh_db):
# Requesting an existing foreign key with different ON DELETE/ON UPDATE
# actions was silently skipped, dropping the requested change
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("books").insert(
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["books"].insert(
{"id": 1, "author_id": 1},
pk="id",
foreign_keys=[("author_id", "authors", "id")],
@ -668,21 +660,19 @@ def test_add_foreign_keys_existing_with_different_actions_errors(fresh_db):
[ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")]
)
assert "ON DELETE" in str(ex.value)
assert fresh_db.table("books").foreign_keys[0].on_delete == "NO ACTION"
assert fresh_db["books"].foreign_keys[0].on_delete == "NO ACTION"
def test_add_foreign_keys_identical_existing_is_noop(fresh_db):
# An exact match, including actions, is silently skipped so repeated
# calls stay idempotent
fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db.table("books").add_foreign_key(
"author_id", "authors", "id", on_delete="CASCADE"
)
fresh_db["authors"].insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["books"].add_foreign_key("author_id", "authors", "id", on_delete="CASCADE")
fresh_db.add_foreign_keys(
[ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")]
)
fks = fresh_db.table("books").foreign_keys
fks = fresh_db["books"].foreign_keys
assert len(fks) == 1
assert fks[0].on_delete == "CASCADE"
@ -690,13 +680,13 @@ def test_add_foreign_keys_identical_existing_is_noop(fresh_db):
def test_add_foreign_keys_compound_column_count_mismatch_errors(fresh_db):
# Previously the extra other-column was silently discarded, creating
# a single-column foreign key to just ("id")
fresh_db.table("departments").insert(
fresh_db["departments"].insert(
{"campus": "north", "code": "cs"}, pk=("campus", "code")
)
fresh_db.table("courses").insert({"id": 1, "campus": "north"}, pk="id")
fresh_db["courses"].insert({"id": 1, "campus": "north"}, pk="id")
with pytest.raises(ValueError) as ex:
fresh_db.add_foreign_keys(
[("courses", ("campus",), "departments", ("campus", "code"))]
)
assert "same number of columns" in str(ex.value)
assert fresh_db.table("courses").foreign_keys == []
assert fresh_db["courses"].foreign_keys == []

View file

@ -20,7 +20,7 @@ search_records = [
def test_enable_fts(fresh_db):
table = fresh_db.table("searchable")
table = fresh_db["searchable"]
table.insert_all(search_records)
assert ["searchable"] == fresh_db.table_names()
table.enable_fts(["text", "country"], fts_version="FTS4")
@ -54,7 +54,7 @@ def test_enable_fts(fresh_db):
def test_enable_fts_escape_table_names(fresh_db):
# Table names with restricted chars are handled correctly.
# colons and dots are restricted characters for table names.
table = fresh_db.table("http://example.com")
table = fresh_db["http://example.com"]
table.insert_all(search_records)
assert ["http://example.com"] == fresh_db.table_names()
table.enable_fts(["text", "country"], fts_version="FTS4")
@ -87,7 +87,7 @@ def test_enable_fts_escape_table_names(fresh_db):
def test_search_duplicate_columns_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624
table = fresh_db.table("t")
table = fresh_db["t"]
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS4")
rows = list(table.search("tanuki", columns=["text", "text"]))
@ -100,7 +100,7 @@ def test_search_duplicate_columns_are_deduped(fresh_db):
def test_search_limit_offset(fresh_db):
table = fresh_db.table("t")
table = fresh_db["t"]
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS4")
assert len(list(table.search("are"))) == 2
@ -112,20 +112,9 @@ def test_search_limit_offset(fresh_db):
)
def test_search_offset_without_limit(fresh_db):
table = fresh_db.table("t")
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS4")
assert [row["rowid"] for row in table.search("are", order_by="rowid")] == [1, 2]
assert [
row["rowid"] for row in table.search("are", offset=1, order_by="rowid")
] == [2]
assert table.search_sql(offset=1).strip().endswith("limit -1 offset 1")
@pytest.mark.parametrize("fts_version", ("FTS4", "FTS5"))
def test_search_where(fresh_db, fts_version):
table = fresh_db.table("t")
table = fresh_db["t"]
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version=fts_version)
results = list(
@ -142,7 +131,7 @@ def test_search_where(fresh_db, fts_version):
def test_search_where_args_disallows_query(fresh_db):
table = fresh_db.table("t")
table = fresh_db["t"]
with pytest.raises(ValueError) as ex:
list(
table.search(
@ -156,7 +145,7 @@ def test_search_where_args_disallows_query(fresh_db):
def test_search_include_rank(fresh_db):
table = fresh_db.table("t")
table = fresh_db["t"]
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS5")
results = list(table.search("are", include_rank=True))
@ -182,7 +171,7 @@ def test_search_include_rank(fresh_db):
def test_enable_fts_table_names_containing_spaces(fresh_db):
table = fresh_db.table("test")
table = fresh_db["test"]
table.insert({"column with spaces": "in its name"})
table.enable_fts(["column with spaces"])
assert [
@ -196,7 +185,7 @@ def test_enable_fts_table_names_containing_spaces(fresh_db):
def test_populate_fts(fresh_db):
table = fresh_db.table("populatable")
table = fresh_db["populatable"]
table.insert(search_records[0])
table.enable_fts(["text", "country"], fts_version="FTS4")
assert [] == list(table.search("trash pandas"))
@ -217,7 +206,7 @@ def test_populate_fts(fresh_db):
def test_populate_fts_escape_table_names(fresh_db):
# Restricted characters such as colon and dots should be escaped.
table = fresh_db.table("http://example.com")
table = fresh_db["http://example.com"]
table.insert(search_records[0])
table.enable_fts(["text", "country"], fts_version="FTS4")
assert [] == list(table.search("trash pandas"))
@ -238,7 +227,7 @@ def test_populate_fts_escape_table_names(fresh_db):
@pytest.mark.parametrize("fts_version", ("4", "5"))
def test_fts_tokenize(fresh_db, fts_version):
table_name = f"searchable_{fts_version}"
table = fresh_db.table(table_name)
table = fresh_db[table_name]
table.insert_all(search_records)
# Test without porter stemming
table.enable_fts(
@ -263,22 +252,10 @@ def test_fts_tokenize(fresh_db, fts_version):
}.items() <= rows[0].items()
def test_fts_tokenize_escaped(fresh_db):
# A malicious tokenize value must not be able to break out of the
# string literal in the CREATE VIRTUAL TABLE statement.
table = fresh_db.table("searchable")
table.insert_all(search_records)
malicious = "porter'); CREATE TABLE injected(x); --"
with pytest.raises(Exception):
table.enable_fts(["text"], tokenize=malicious)
# The injected statement must not have executed
assert "injected" not in fresh_db.table_names()
def test_optimize_fts(fresh_db):
for fts_version in ("4", "5"):
table_name = f"searchable_{fts_version}"
table = fresh_db.table(table_name)
table = fresh_db[table_name]
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version=f"FTS{fts_version}")
# You can call optimize successfully against the tables OR their _fts equivalents:
@ -288,11 +265,11 @@ def test_optimize_fts(fresh_db):
"searchable_4_fts",
"searchable_5_fts",
):
fresh_db.table(table_name).optimize()
fresh_db[table_name].optimize()
def test_enable_fts_with_triggers(fresh_db):
table = fresh_db.table("searchable")
table = fresh_db["searchable"]
table.insert(search_records[0])
table.enable_fts(["text", "country"], fts_version="FTS4", create_triggers=True)
rows1 = list(table.search("tanuki"))
@ -321,7 +298,7 @@ def test_enable_fts_with_triggers(fresh_db):
@pytest.mark.parametrize("create_triggers", [True, False])
def test_disable_fts(fresh_db, create_triggers):
table = fresh_db.table("searchable")
table = fresh_db["searchable"]
table.insert(search_records[0])
table.enable_fts(["text", "country"], create_triggers=create_triggers)
assert {
@ -354,7 +331,7 @@ def test_disable_fts(fresh_db, create_triggers):
def test_rebuild_fts(fresh_db):
table = fresh_db.table("searchable")
table = fresh_db["searchable"]
table.insert(search_records[0])
table.enable_fts(["text", "country"])
# Run a search
@ -380,7 +357,7 @@ def test_rebuild_fts(fresh_db):
def test_optimize_and_rebuild_fts_commit(tmpdir, method):
path = str(tmpdir / "test.db")
db = Database(path)
table = db.table("searchable")
table = db["searchable"]
table.insert(search_records[0])
table.enable_fts(["text", "country"])
getattr(table, method)()
@ -390,16 +367,16 @@ def test_optimize_and_rebuild_fts_commit(tmpdir, method):
table.insert(search_records[1])
db.close()
db2 = Database(path)
assert db2.table("searchable").count == 2
assert db2["searchable"].count == 2
db2.close()
@pytest.mark.parametrize("invalid_table", ["does_not_exist", "not_searchable"])
def test_rebuild_fts_invalid(fresh_db, invalid_table):
fresh_db.table("not_searchable").insert({"foo": "bar"})
fresh_db["not_searchable"].insert({"foo": "bar"})
# Raise OperationalError on invalid table
with pytest.raises(sqlite3.OperationalError):
fresh_db.table(invalid_table).rebuild_fts()
fresh_db[invalid_table].rebuild_fts()
@pytest.mark.parametrize("fts_version", ["FTS4", "FTS5"])
@ -408,17 +385,15 @@ def test_rebuild_removes_junk_docsize_rows(tmpdir, fts_version):
path = tmpdir / "test.db"
db = Database(str(path), recursive_triggers=False)
licenses = [{"key": "apache2", "name": "Apache 2"}, {"key": "bsd", "name": "BSD"}]
db.table("licenses").insert_all(licenses, pk="key", replace=True)
db.table("licenses").enable_fts(
["name"], create_triggers=True, fts_version=fts_version
)
assert db.table("licenses_fts_docsize").count == 2
db["licenses"].insert_all(licenses, pk="key", replace=True)
db["licenses"].enable_fts(["name"], create_triggers=True, fts_version=fts_version)
assert db["licenses_fts_docsize"].count == 2
# Bug: insert with replace increases the number of rows in _docsize:
db.table("licenses").insert_all(licenses, pk="key", replace=True)
assert db.table("licenses_fts_docsize").count == 4
db["licenses"].insert_all(licenses, pk="key", replace=True)
assert db["licenses_fts_docsize"].count == 4
# rebuild should fix this:
db.table("licenses_fts").rebuild_fts()
assert db.table("licenses_fts_docsize").count == 2
db["licenses_fts"].rebuild_fts()
assert db["licenses_fts_docsize"].count == 2
@pytest.mark.parametrize(
@ -432,7 +407,7 @@ def test_rebuild_removes_junk_docsize_rows(tmpdir, fts_version):
)
def test_enable_fts_replace(kwargs):
db = Database(memory=True)
db.table("books").insert(
db["books"].insert(
{
"id": 1,
"title": "Habits of Australian Marsupials",
@ -440,31 +415,31 @@ def test_enable_fts_replace(kwargs):
},
pk="id",
)
db.table("books").enable_fts(["title", "author"])
assert not db.table("books").triggers
assert db.table("books_fts").columns_dict.keys() == {"title", "author"}
assert "FTS5" in db.table("books_fts").schema
assert "porter" not in db.table("books_fts").schema
db["books"].enable_fts(["title", "author"])
assert not db["books"].triggers
assert db["books_fts"].columns_dict.keys() == {"title", "author"}
assert "FTS5" in db["books_fts"].schema
assert "porter" not in db["books_fts"].schema
# Now modify the FTS configuration
should_have_changed_columns = "columns" in kwargs
if "columns" not in kwargs:
kwargs["columns"] = ["title", "author"]
db.table("books").enable_fts(**kwargs, replace=True)
db["books"].enable_fts(**kwargs, replace=True)
# Check that the new configuration is correct
if should_have_changed_columns:
assert db.table("books_fts").columns_dict.keys() == {"title"}
assert db["books_fts"].columns_dict.keys() == {"title"}
if "create_triggers" in kwargs:
assert db.table("books").triggers
assert db["books"].triggers
if "fts_version" in kwargs:
assert "FTS4" in db.table("books_fts").schema
assert "FTS4" in db["books_fts"].schema
if "tokenize" in kwargs:
assert "porter" in db.table("books_fts").schema
assert "porter" in db["books_fts"].schema
def test_enable_fts_replace_does_nothing_if_args_the_same():
queries = []
db = Database(memory=True, tracer=lambda sql, params: queries.append((sql, params)))
db.table("books").insert(
db["books"].insert(
{
"id": 1,
"title": "Habits of Australian Marsupials",
@ -472,19 +447,17 @@ def test_enable_fts_replace_does_nothing_if_args_the_same():
},
pk="id",
)
db.table("books").enable_fts(["title", "author"], create_triggers=True)
db["books"].enable_fts(["title", "author"], create_triggers=True)
queries.clear()
# Running that again shouldn't run much SQL:
db.table("books").enable_fts(
["title", "author"], create_triggers=True, replace=True
)
db["books"].enable_fts(["title", "author"], create_triggers=True, replace=True)
# The only SQL that executed should be select statements
assert all(q[0].startswith("select ") for q in queries)
def test_enable_fts_replace_handles_legacy_bracket_quoted_content_table():
db = Database(memory=True)
db.table("books").insert(
db["books"].insert(
{
"id": 1,
"title": "Habits of Australian Marsupials",
@ -499,10 +472,10 @@ def test_enable_fts_replace_handles_legacy_bracket_quoted_content_table():
);
""")
db.table("books").enable_fts(["title", "author"], replace=True)
db["books"].enable_fts(["title", "author"], replace=True)
assert db.table("books_fts").columns_dict.keys() == {"title", "author"}
assert 'content="books"' in db.table("books_fts").schema
assert db["books_fts"].columns_dict.keys() == {"title", "author"}
assert 'content="books"' in db["books_fts"].schema
def test_view_has_no_enable_fts():
@ -510,7 +483,7 @@ def test_view_has_no_enable_fts():
db.create_view("hello", "select 1 + 1")
# Views deliberately do not have an enable_fts() method
with pytest.raises(AttributeError):
db.view("hello").enable_fts() # type: ignore[attr-defined]
db["hello"].enable_fts() # type: ignore[union-attr]
@pytest.mark.parametrize(
@ -716,14 +689,14 @@ def test_view_has_no_enable_fts():
)
def test_search_sql(kwargs, fts, expected):
db = Database(memory=True)
db.table("books").insert(
db["books"].insert(
{
"title": "Habits of Australian Marsupials",
"author": "Marlee Hawkins",
}
)
db.table("books").enable_fts(["title", "author"], fts_version=fts)
sql = db.table("books").search_sql(**kwargs)
db["books"].enable_fts(["title", "author"], fts_version=fts)
sql = db["books"].search_sql(**kwargs)
assert sql == expected
@ -744,7 +717,7 @@ def test_search_sql(kwargs, fts, expected):
),
)
def test_quote_fts_query(fresh_db, input, expected):
table = fresh_db.table("searchable")
table = fresh_db["searchable"]
table.insert_all(search_records)
table.enable_fts(["text", "country"])
quoted = fresh_db.quote_fts(input)
@ -754,7 +727,7 @@ def test_quote_fts_query(fresh_db, input, expected):
def test_search_quote(fresh_db):
table = fresh_db.table("searchable")
table = fresh_db["searchable"]
table.insert_all(search_records)
table.enable_fts(["text", "country"])
query = "cat's"
@ -767,7 +740,7 @@ def test_search_quote(fresh_db):
def test_enable_fts_cli_on_view_errors(tmpdir):
db_path = str(tmpdir / "test.db")
db = Database(db_path)
db.table("t").insert({"text": "hello"})
db["t"].insert({"text": "hello"})
db.create_view("v", "select * from t")
db.close()
from click.testing import CliRunner

View file

@ -4,14 +4,14 @@ from sqlite_utils.db import NotFoundError
def test_get_rowid(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
cleo = {"name": "Cleo", "age": 4}
row_id = dogs.insert(cleo).last_rowid
assert cleo == dogs.get(row_id)
def test_get_primary_key(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
cleo = {"name": "Cleo", "age": 4, "id": 5}
last_pk = dogs.insert(cleo, pk="id").last_pk
assert 5 == last_pk
@ -23,10 +23,10 @@ def test_get_primary_key(fresh_db):
[(100, None), (None, None), ((1, 2), "Need 1 primary key value"), ("2", None)],
)
def test_get_not_found(argument, expected_msg, fresh_db):
fresh_db.table("dogs").insert(
fresh_db["dogs"].insert(
{"id": 1, "name": "Cleo", "age": 4, "is_good": True}, pk="id"
)
with pytest.raises(NotFoundError) as excinfo:
fresh_db.table("dogs").get(argument)
fresh_db["dogs"].get(argument)
if expected_msg is not None:
assert expected_msg == excinfo.value.args[0]

View file

@ -45,7 +45,7 @@ def test_add_geometry_column():
coord_dimension="XY",
)
assert db.table("geometry_columns").get(["locations", "geometry"]) == {
assert db["geometry_columns"].get(["locations", "geometry"]) == {
"f_table_name": "locations",
"f_geometry_column": "geometry",
"geometry_type": 1, # point
@ -133,7 +133,7 @@ def test_cli_add_geometry_column(tmpdir):
db = Database(str(db_path))
db.init_spatialite()
table = db.table("locations").create({"name": str})
table = db["locations"].create({"name": str})
result = CliRunner().invoke(
cli,
@ -149,7 +149,7 @@ def test_cli_add_geometry_column(tmpdir):
assert result.exit_code == 0
assert db.table("geometry_columns").get(["locations", "geometry"]) == {
assert db["geometry_columns"].get(["locations", "geometry"]) == {
"f_table_name": "locations",
"f_geometry_column": "geometry",
"geometry_type": 1, # point
@ -164,7 +164,7 @@ def test_cli_add_geometry_column_options(tmpdir):
db_path = tmpdir / "spatial.db"
db = Database(str(db_path))
db.init_spatialite()
table = db.table("locations").create({"name": str})
table = db["locations"].create({"name": str})
result = CliRunner().invoke(
cli,
@ -183,7 +183,7 @@ def test_cli_add_geometry_column_options(tmpdir):
assert result.exit_code == 0
assert db.table("geometry_columns").get(["locations", "geometry"]) == {
assert db["geometry_columns"].get(["locations", "geometry"]) == {
"f_table_name": "locations",
"f_geometry_column": "geometry",
"geometry_type": 3, # polygon
@ -202,7 +202,7 @@ def test_cli_add_geometry_column_invalid_type(tmpdir):
db = Database(str(db_path))
db.init_spatialite()
table = db.table("locations").create({"name": str})
table = db["locations"].create({"name": str})
result = CliRunner().invoke(
cli,
@ -225,7 +225,7 @@ def test_cli_create_spatial_index(tmpdir):
db = Database(str(db_path))
db.init_spatialite()
table = db.table("locations").create({"name": str})
table = db["locations"].create({"name": str})
table.add_geometry_column("geometry", "POINT")
result = CliRunner().invoke(

View file

@ -11,8 +11,8 @@ def test_roundtrip_integers(integer):
row = {
"integer": integer,
}
db.table("test").insert(row)
assert list(db.table("test").rows) == [row]
db["test"].insert(row)
assert list(db["test"].rows) == [row]
@given(st.text())
@ -21,8 +21,8 @@ def test_roundtrip_text(text):
row = {
"text": text,
}
db.table("test").insert(row)
assert list(db.table("test").rows) == [row]
db["test"].insert(row)
assert list(db["test"].rows) == [row]
@given(st.binary(max_size=1024 * 1024))
@ -31,8 +31,8 @@ def test_roundtrip_binary(binary):
row = {
"binary": binary,
}
db.table("test").insert(row)
assert list(db.table("test").rows) == [row]
db["test"].insert(row)
assert list(db["test"].rows) == [row]
@given(st.floats(allow_nan=False))
@ -41,5 +41,5 @@ def test_roundtrip_floats(floats):
row = {
"floats": floats,
}
db.table("test").insert(row)
assert list(db.table("test").rows) == [row]
db["test"].insert(row)
assert list(db["test"].rows) == [row]

View file

@ -57,7 +57,7 @@ def test_insert_files(silent, pk_args, expected_pks):
)
assert result.exit_code == 0, result.stdout
db = Database(db_path)
rows_by_path = {r["path"]: r for r in db.table("files").rows}
rows_by_path = {r["path"]: r for r in db["files"].rows}
one, two, three = (
rows_by_path["one.txt"],
rows_by_path["two.txt"],
@ -114,7 +114,7 @@ def test_insert_files(silent, pk_args, expected_pks):
for colname, expected_type in expected_types.items():
for row in (one, two, three):
assert isinstance(row[colname], expected_type)
assert set(db.table("files").pks) == set(expected_pks)
assert set(db["files"].pks) == set(expected_pks)
@pytest.mark.parametrize(
@ -144,7 +144,7 @@ def test_insert_files_stdin(use_text, encoding, input, expected):
)
assert result.exit_code == 0, result.stdout
db = Database(db_path)
row = next(iter(db.table("files").rows))
row = next(iter(db["files"].rows))
key = "content"
if use_text:
key = "content_text"

View file

@ -1,6 +1,6 @@
import pytest
from sqlite_utils.db import Check, Database, Index, Table, View, XIndex, XIndexColumn
from sqlite_utils.db import Check, Database, Index, View, XIndex, XIndexColumn
def _check_supports_strict():
@ -21,10 +21,10 @@ def test_view_names(fresh_db):
def test_table_names_fts4(existing_db):
existing_db.table("woo").insert({"title": "Hello"}).enable_fts(
existing_db["woo"].insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4"
)
existing_db.table("woo2").insert({"title": "Hello"}).enable_fts(
existing_db["woo2"].insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS5"
)
assert ["woo_fts"] == existing_db.table_names(fts4=True)
@ -32,17 +32,17 @@ def test_table_names_fts4(existing_db):
def test_detect_fts(existing_db):
existing_db.table("woo").insert({"title": "Hello"}).enable_fts(
existing_db["woo"].insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4"
)
existing_db.table("woo2").insert({"title": "Hello"}).enable_fts(
existing_db["woo2"].insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS5"
)
assert "woo_fts" == existing_db.table("woo").detect_fts()
assert "woo_fts" == existing_db.table("woo_fts").detect_fts()
assert "woo2_fts" == existing_db.table("woo2").detect_fts()
assert "woo2_fts" == existing_db.table("woo2_fts").detect_fts()
assert existing_db.table("foo").detect_fts() is None
assert "woo_fts" == existing_db["woo"].detect_fts()
assert "woo_fts" == existing_db["woo_fts"].detect_fts()
assert "woo2_fts" == existing_db["woo2"].detect_fts()
assert "woo2_fts" == existing_db["woo2_fts"].detect_fts()
assert existing_db["foo"].detect_fts() is None
@pytest.mark.parametrize("reverse_order", (True, False))
@ -52,14 +52,14 @@ def test_detect_fts_similar_tables(fresh_db, reverse_order):
if reverse_order:
table1, table2 = table2, table1
fresh_db.table(table1).insert({"title": "Hello"}).enable_fts(
fresh_db[table1].insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4"
)
fresh_db.table(table2).insert({"title": "Hello"}).enable_fts(
fresh_db[table2].insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4"
)
assert fresh_db.table(table1).detect_fts() == f"{table1}_fts"
assert fresh_db.table(table2).detect_fts() == f"{table2}_fts"
assert fresh_db[table1].detect_fts() == f"{table1}_fts"
assert fresh_db[table2].detect_fts() == f"{table2}_fts"
def test_tables(existing_db):
@ -77,34 +77,26 @@ def test_views(fresh_db):
assert view.columns_dict == {"1": str}
def test_getitem_returns_table_or_view(fresh_db):
fresh_db.table("items").insert({"id": 1}, pk="id")
fresh_db.create_view("item_ids", "select id from items")
assert isinstance(fresh_db["items"], Table)
assert isinstance(fresh_db["item_ids"], View)
def test_count(existing_db):
assert existing_db.table("foo").count == 3
assert existing_db.table("foo").count_where() == 3
assert existing_db.table("foo").execute_count() == 3
assert existing_db["foo"].count == 3
assert existing_db["foo"].count_where() == 3
assert existing_db["foo"].execute_count() == 3
def test_count_where(existing_db):
assert existing_db.table("foo").count_where("text != ?", ["two"]) == 2
assert existing_db.table("foo").count_where("text != :t", {"t": "two"}) == 2
assert existing_db["foo"].count_where("text != ?", ["two"]) == 2
assert existing_db["foo"].count_where("text != :t", {"t": "two"}) == 2
def test_columns(existing_db):
table = existing_db.table("foo")
table = existing_db["foo"]
assert [{"name": "text", "type": "TEXT"}] == [
{"name": col.name, "type": col.type} for col in table.columns
]
def test_table_schema(existing_db):
assert existing_db.table("foo").schema == "CREATE TABLE foo (text TEXT)"
assert existing_db["foo"].schema == "CREATE TABLE foo (text TEXT)"
def test_database_schema(existing_db):
@ -112,9 +104,9 @@ def test_database_schema(existing_db):
def test_table_repr(fresh_db):
table = fresh_db.table("dogs").insert({"name": "Cleo", "age": 4})
table = fresh_db["dogs"].insert({"name": "Cleo", "age": 4})
assert "<Table dogs (name, age)>" == repr(table)
assert "<Table cats (does not exist yet)>" == repr(fresh_db.table("cats"))
assert "<Table cats (does not exist yet)>" == repr(fresh_db["cats"])
def test_indexes(fresh_db):
@ -133,7 +125,7 @@ def test_indexes(fresh_db):
columns=["c2", "c3"],
),
Index(seq=1, name="Gosh_c1", unique=0, origin="c", partial=0, columns=["c1"]),
] == fresh_db.table("Gosh").indexes
] == fresh_db["Gosh"].indexes
def test_xindexes(fresh_db):
@ -142,7 +134,7 @@ def test_xindexes(fresh_db):
create index Gosh_c1 on Gosh(c1);
create index Gosh_c2c3 on Gosh(c2, c3 desc);
""")
assert fresh_db.table("Gosh").xindexes == [
assert fresh_db["Gosh"].xindexes == [
XIndex(
name="Gosh_c2c3",
columns=[
@ -161,31 +153,6 @@ def test_xindexes(fresh_db):
]
def test_indexes_with_double_quotes_in_identifiers(fresh_db):
fresh_db['Go"sh'].insert({"id": 1, 'c"1': 2}, pk="id")
fresh_db['Go"sh'].create_index(['c"1'])
assert [(index.name, index.columns) for index in fresh_db['Go"sh'].indexes] == [
('idx_Go"sh_c"1', ['c"1'])
]
assert fresh_db['Go"sh'].xindexes == [
XIndex(
name='idx_Go"sh_c"1',
columns=[
XIndexColumn(seqno=0, cid=1, name='c"1', desc=0, coll="BINARY", key=1),
XIndexColumn(seqno=1, cid=-1, name=None, desc=0, coll="BINARY", key=0),
],
)
]
def test_transform_table_with_double_quotes_in_identifiers(fresh_db):
fresh_db['Go"sh'].insert({"id": 1, 'c"1': 2, "c2": 3}, pk="id")
fresh_db['Go"sh'].create_index(['c"1'])
fresh_db['Go"sh'].transform(types={"c2": str})
assert fresh_db['Go"sh'].columns_dict["c2"] is str
assert [index.columns for index in fresh_db['Go"sh'].indexes] == [['c"1']]
@pytest.mark.parametrize(
"column,expected_table_guess",
(
@ -199,15 +166,15 @@ def test_transform_table_with_double_quotes_in_identifiers(fresh_db):
def test_guess_foreign_table(fresh_db, column, expected_table_guess):
fresh_db.create_table("authors", {"name": str})
fresh_db.create_table("genre", {"name": str})
assert expected_table_guess == fresh_db.table("books").guess_foreign_table(column)
assert expected_table_guess == fresh_db["books"].guess_foreign_table(column)
@pytest.mark.parametrize(
"pk,expected", ((None, ["rowid"]), ("id", ["id"]), (["id", "id2"], ["id", "id2"]))
)
def test_pks(fresh_db, pk, expected):
fresh_db.table("foo").insert_all([{"id": 1, "id2": 2}], pk=pk)
assert expected == fresh_db.table("foo").pks
fresh_db["foo"].insert_all([{"id": 1, "id2": 2}], pk=pk)
assert expected == fresh_db["foo"].pks
def test_checks(fresh_db):
@ -218,7 +185,7 @@ def test_checks(fresh_db):
CONSTRAINT within_maximum CHECK(score <= maximum)
)
""")
scores = fresh_db.table("scores")
scores = fresh_db["scores"]
expected_column = Check("score > 0", name="positive", column="score")
expected_table = Check("score <= maximum", name="within_maximum")
assert scores.checks == [expected_column, expected_table]
@ -228,26 +195,26 @@ def test_checks(fresh_db):
def test_checks_nonexistent_and_virtual_tables(fresh_db):
assert fresh_db.table("does_not_exist").checks == []
fresh_db.table("searchable").insert({"text": "hello"}).enable_fts(
assert fresh_db["does_not_exist"].checks == []
fresh_db["searchable"].insert({"text": "hello"}).enable_fts(
["text"], fts_version="FTS5"
)
assert fresh_db.table("searchable_fts").checks == []
assert fresh_db["searchable_fts"].checks == []
def test_triggers_and_triggers_dict(fresh_db):
assert [] == fresh_db.triggers
authors = fresh_db.table("authors")
authors = fresh_db["authors"]
authors.insert_all(
[
{"name": "Frank Herbert", "famous_works": "Dune"},
{"name": "Neal Stephenson", "famous_works": "Cryptonomicon"},
]
)
fresh_db.table("other").insert({"foo": "bar"})
fresh_db["other"].insert({"foo": "bar"})
assert authors.triggers == []
assert authors.triggers_dict == {}
assert fresh_db.table("other").triggers == []
assert fresh_db["other"].triggers == []
assert fresh_db.triggers_dict == {}
authors.enable_fts(
["name", "famous_works"], fts_version="FTS4", create_triggers=True
@ -259,7 +226,7 @@ def test_triggers_and_triggers_dict(fresh_db):
}
assert expected_triggers == {(t.name, t.table) for t in fresh_db.triggers}
assert expected_triggers == {
(t.name, t.table) for t in fresh_db.table("authors").triggers
(t.name, t.table) for t in fresh_db["authors"].triggers
}
expected_triggers = {
"authors_ai": (
@ -279,13 +246,13 @@ def test_triggers_and_triggers_dict(fresh_db):
),
}
assert authors.triggers_dict == expected_triggers
assert fresh_db.table("other").triggers == []
assert fresh_db.table("other").triggers_dict == {}
assert fresh_db["other"].triggers == []
assert fresh_db["other"].triggers_dict == {}
assert fresh_db.triggers_dict == expected_triggers
def test_has_counts_triggers(fresh_db):
authors = fresh_db.table("authors")
authors = fresh_db["authors"]
authors.insert({"name": "Frank Herbert"})
assert not authors.has_counts_triggers
authors.enable_counts()
@ -334,14 +301,14 @@ def test_has_counts_triggers(fresh_db):
)
def test_virtual_table_using(fresh_db, sql, expected_name, expected_using):
fresh_db.execute(sql)
assert fresh_db.table(expected_name).virtual_table_using == expected_using
assert fresh_db[expected_name].virtual_table_using == expected_using
def test_use_rowid(fresh_db):
fresh_db.table("rowid_table").insert({"name": "Cleo"})
fresh_db.table("regular_table").insert({"id": 1, "name": "Cleo"}, pk="id")
assert fresh_db.table("rowid_table").use_rowid
assert not fresh_db.table("regular_table").use_rowid
fresh_db["rowid_table"].insert({"name": "Cleo"})
fresh_db["regular_table"].insert({"id": 1, "name": "Cleo"}, pk="id")
assert fresh_db["rowid_table"].use_rowid
assert not fresh_db["regular_table"].use_rowid
@pytest.mark.skipif(
@ -360,7 +327,7 @@ def test_use_rowid(fresh_db):
)
def test_table_strict(fresh_db, create_table, expected_strict):
fresh_db.execute(create_table)
table = fresh_db.table("t")
table = fresh_db["t"]
assert table.strict == expected_strict
@ -376,10 +343,10 @@ def test_table_strict(fresh_db, create_table, expected_strict):
),
)
def test_table_default_values(fresh_db, value):
fresh_db.table("default_values").insert(
fresh_db["default_values"].insert(
{"nodefault": 1, "value": value}, defaults={"value": value}
)
default_values = fresh_db.table("default_values").default_values
default_values = fresh_db["default_values"].default_values
assert default_values == {"value": value}
@ -389,23 +356,8 @@ def test_table_default_values_escaped_quotes(fresh_db):
fresh_db.execute(
"create table t (id integer primary key, name text default 'O''Brien')"
)
assert "default 'O''Brien'" in fresh_db.table("t").schema
assert fresh_db.table("t").default_values == {"name": "O'Brien"}
def test_table_default_values_keyword_literals(fresh_db):
fresh_db.execute(
"create table t ("
"enabled integer default TRUE, "
"disabled integer default false, "
"nullable text default NULL"
")"
)
assert fresh_db.table("t").default_values == {
"enabled": True,
"disabled": False,
"nullable": None,
}
assert "default 'O''Brien'" in fresh_db["t"].schema
assert fresh_db["t"].default_values == {"name": "O'Brien"}
def test_pks_use_primary_key_declaration_order(fresh_db):
@ -413,11 +365,11 @@ def test_pks_use_primary_key_declaration_order(fresh_db):
# pks must follow the declaration order, which is what SQLite uses to
# resolve implicit foreign key references and compound pk lookups
fresh_db.execute("create table t (b text, a text, primary key (a, b))")
assert fresh_db.table("t").pks == ["a", "b"]
assert fresh_db["t"].pks == ["a", "b"]
def test_transform_preserves_compound_pk_declaration_order(fresh_db):
fresh_db.execute("create table t (a text, b text, c text, primary key (b, a))")
fresh_db.table("t").transform(drop={"c"})
assert fresh_db.table("t").pks == ["b", "a"]
assert 'PRIMARY KEY ("b", "a")' in fresh_db.table("t").schema
fresh_db["t"].transform(drop={"c"})
assert fresh_db["t"].pks == ["b", "a"]
assert 'PRIMARY KEY ("b", "a")' in fresh_db["t"].schema

View file

@ -19,9 +19,9 @@ def test_insert_all_list_mode_basic():
yield [2, "Bob", 25]
yield [3, "Charlie", 35]
db.table("people").insert_all(data_generator())
db["people"].insert_all(data_generator())
rows = list(db.table("people").rows)
rows = list(db["people"].rows)
assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
assert rows[1] == {"id": 2, "name": "Bob", "age": 25}
@ -37,10 +37,10 @@ def test_insert_all_list_mode_with_pk():
yield [1, "Alice", 95]
yield [2, "Bob", 87]
db.table("scores").insert_all(data_generator(), pk="id")
db["scores"].insert_all(data_generator(), pk="id")
assert db.table("scores").pks == ["id"]
rows = list(db.table("scores").rows)
assert db["scores"].pks == ["id"]
rows = list(db["scores"].rows)
assert len(rows) == 2
@ -54,7 +54,7 @@ def test_upsert_all_list_mode():
yield [1, "Alice", 100]
yield [2, "Bob", 200]
db.table("data").insert_all(initial_data(), pk="id")
db["data"].insert_all(initial_data(), pk="id")
# Upsert with some updates and new records
def upsert_data():
@ -62,9 +62,9 @@ def test_upsert_all_list_mode():
yield [1, "Alice", 150] # Update existing
yield [3, "Charlie", 300] # Insert new
db.table("data").upsert_all(upsert_data(), pk="id")
db["data"].upsert_all(upsert_data(), pk="id")
rows = list(db.table("data").rows_where(order_by="id"))
rows = list(db["data"].rows_where(order_by="id"))
assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "value": 150}
assert rows[1] == {"id": 2, "name": "Bob", "value": 200}
@ -81,9 +81,9 @@ def test_list_mode_with_various_types():
yield [2, "Bob", 87.3, False]
yield [3, "Charlie", None, True]
db.table("mixed").insert_all(data_generator())
db["mixed"].insert_all(data_generator())
rows = list(db.table("mixed").rows)
rows = list(db["mixed"].rows)
assert len(rows) == 3
assert rows[0]["score"] == 95.5
assert rows[1]["active"] == 0 # SQLite stores boolean as int
@ -99,7 +99,7 @@ def test_list_mode_error_non_string_columns():
yield ["a", "b", "c"]
with pytest.raises(ValueError, match="must be a list of column name strings"):
db.table("bad").insert_all(bad_data()) # type: ignore[arg-type]
db["bad"].insert_all(bad_data())
def test_list_mode_error_mixed_types():
@ -111,7 +111,7 @@ def test_list_mode_error_mixed_types():
yield {"id": 1, "name": "Alice"} # Should be a list, not dict
with pytest.raises(ValueError, match="must also be lists"):
db.table("bad").insert_all(bad_data()) # type: ignore[arg-type]
db["bad"].insert_all(bad_data())
def test_list_mode_empty_after_headers():
@ -122,9 +122,9 @@ def test_list_mode_empty_after_headers():
yield ["id", "name", "age"]
# No data rows
result = db.table("people").insert_all(data_generator())
result = db["people"].insert_all(data_generator())
assert result is not None
assert not db.table("people").exists()
assert not db["people"].exists()
def test_list_mode_batch_processing():
@ -136,7 +136,7 @@ def test_list_mode_batch_processing():
for i in range(1000):
yield [i, f"value_{i}"]
db.table("large").insert_all(large_data(), batch_size=100)
db["large"].insert_all(large_data(), batch_size=100)
count = db.execute("SELECT COUNT(*) as c FROM large").fetchone()[0]
assert count == 1000
@ -152,9 +152,9 @@ def test_list_mode_shorter_rows():
yield [2, "Bob"] # Missing age and city
yield [3, "Charlie", 35] # Missing city
db.table("people").insert_all(data_generator())
db["people"].insert_all(data_generator())
rows = list(db.table("people").rows_where(order_by="id"))
rows = list(db["people"].rows_where(order_by="id"))
assert rows[0] == {"id": 1, "name": "Alice", "age": 30, "city": "NYC"}
assert rows[1] == {"id": 2, "name": "Bob", "age": None, "city": None}
assert rows[2] == {"id": 3, "name": "Charlie", "age": 35, "city": None}
@ -170,9 +170,9 @@ def test_backwards_compatibility_dict_mode():
{"id": 2, "name": "Bob", "age": 25},
]
db.table("people").insert_all(data)
db["people"].insert_all(data)
rows = list(db.table("people").rows)
rows = list(db["people"].rows)
assert len(rows) == 2
assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
@ -189,9 +189,9 @@ def test_insert_all_tuple_mode_basic():
yield (2, "Bob", 25)
yield (3, "Charlie", 35)
db.table("people").insert_all(data_generator())
db["people"].insert_all(data_generator())
rows = list(db.table("people").rows)
rows = list(db["people"].rows)
assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
assert rows[1] == {"id": 2, "name": "Bob", "age": 25}
@ -211,9 +211,9 @@ def test_insert_all_mixed_list_tuple():
yield [3, "Charlie", 35]
yield (4, "Diana", 40)
db.table("people").insert_all(data_generator())
db["people"].insert_all(data_generator())
rows = list(db.table("people").rows)
rows = list(db["people"].rows)
assert len(rows) == 4
assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
assert rows[1] == {"id": 2, "name": "Bob", "age": 25}
@ -231,7 +231,7 @@ def test_upsert_all_tuple_mode():
yield (1, "Alice", 100)
yield (2, "Bob", 200)
db.table("data").insert_all(initial_data(), pk="id")
db["data"].insert_all(initial_data(), pk="id")
# Upsert with tuples
def upsert_data():
@ -239,9 +239,9 @@ def test_upsert_all_tuple_mode():
yield (1, "Alice", 150) # Update existing
yield (3, "Charlie", 300) # Insert new
db.table("data").upsert_all(upsert_data(), pk="id")
db["data"].upsert_all(upsert_data(), pk="id")
rows = list(db.table("data").rows_where(order_by="id"))
rows = list(db["data"].rows_where(order_by="id"))
assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "value": 150}
assert rows[1] == {"id": 2, "name": "Bob", "value": 200}
@ -258,9 +258,9 @@ def test_tuple_mode_shorter_rows():
yield 2, "Bob" # Missing age and city
yield 3, "Charlie", 35 # Missing city
db.table("people").insert_all(data_generator())
db["people"].insert_all(data_generator())
rows = list(db.table("people").rows_where(order_by="id"))
rows = list(db["people"].rows_where(order_by="id"))
assert rows[0] == {"id": 1, "name": "Alice", "age": 30, "city": "NYC"}
assert rows[1] == {"id": 2, "name": "Bob", "age": None, "city": None}
assert rows[2] == {"id": 3, "name": "Charlie", "age": 35, "city": None}
@ -271,18 +271,18 @@ def test_list_mode_single_record_upsert_last_pk():
db = Database(memory=True)
# Create table first
db.table("data").insert({"id": 1, "name": "Alice", "value": 100}, pk="id")
db["data"].insert({"id": 1, "name": "Alice", "value": 100}, pk="id")
# Now upsert a single record using list mode
def upsert_data():
yield ["id", "name", "value"]
yield [1, "Alice", 150] # Update existing
table = db.table("data")
table = db["data"]
table.upsert_all(upsert_data(), pk="id")
# Verify the data was updated
rows = list(db.table("data").rows)
rows = list(db["data"].rows)
assert rows == [{"id": 1, "name": "Alice", "value": 150}]
# Verify last_pk is populated correctly

View file

@ -4,7 +4,7 @@ from sqlite_utils.db import Index
def test_lookup_new_table(fresh_db):
species = fresh_db.table("species")
species = fresh_db["species"]
palm_id = species.lookup({"name": "Palm"})
oak_id = species.lookup({"name": "Oak"})
cherry_id = species.lookup({"name": "Cherry"})
@ -26,7 +26,7 @@ def test_lookup_new_table(fresh_db):
def test_lookup_new_table_compound_key(fresh_db):
species = fresh_db.table("species")
species = fresh_db["species"]
palm_id = species.lookup({"name": "Palm", "type": "Tree"})
oak_id = species.lookup({"name": "Oak", "type": "Tree"})
assert palm_id == species.lookup({"name": "Palm", "type": "Tree"})
@ -70,7 +70,7 @@ def test_lookup_fails_if_constraint_cannot_be_added(fresh_db):
def test_lookup_with_extra_values(fresh_db):
species = fresh_db.table("species")
species = fresh_db["species"]
id = species.lookup({"name": "Palm", "type": "Tree"}, {"first_seen": "2020-01-01"})
assert species.get(id) == {
"id": 1,
@ -90,9 +90,9 @@ def test_lookup_with_extra_values(fresh_db):
def test_lookup_with_extra_insert_parameters(fresh_db):
other_table = fresh_db.table("other_table")
other_table = fresh_db["other_table"]
other_table.insert({"id": 1, "name": "Name"}, pk="id")
species = fresh_db.table("species")
species = fresh_db["species"]
id = species.lookup(
{"name": "Palm", "type": "Tree"},
{
@ -156,15 +156,15 @@ def test_lookup_with_extra_insert_parameters(fresh_db):
@pytest.mark.parametrize("strict", (False, True))
def test_lookup_new_table_strict(fresh_db, strict):
fresh_db.table("species").lookup({"name": "Palm"}, strict=strict)
assert fresh_db.table("species").strict == strict or not fresh_db.supports_strict
fresh_db["species"].lookup({"name": "Palm"}, strict=strict)
assert fresh_db["species"].strict == strict or not fresh_db.supports_strict
def test_lookup_null_value_idempotent(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/186
# Repeated lookups of a null value should return the same row,
# not insert a duplicate row each time
species = fresh_db.table("species")
species = fresh_db["species"]
first_id = species.lookup({"name": None})
second_id = species.lookup({"name": None})
assert first_id == second_id
@ -172,7 +172,7 @@ def test_lookup_null_value_idempotent(fresh_db):
def test_lookup_compound_key_with_null_idempotent(fresh_db):
species = fresh_db.table("species")
species = fresh_db["species"]
palm_id = species.lookup({"name": "Palm", "type": None})
oak_id = species.lookup({"name": "Oak", "type": "Tree"})
assert palm_id == species.lookup({"name": "Palm", "type": None})

View file

@ -4,45 +4,45 @@ from sqlite_utils.db import ForeignKey, NoObviousTable
def test_insert_m2m_single(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m(
"humans", {"id": 1, "name": "Natalie D"}, pk="id"
)
assert {"dogs_humans", "humans", "dogs"} == set(fresh_db.table_names())
humans = fresh_db.table("humans")
dogs_humans = fresh_db.table("dogs_humans")
humans = fresh_db["humans"]
dogs_humans = fresh_db["dogs_humans"]
assert [{"id": 1, "name": "Natalie D"}] == list(humans.rows)
assert [{"humans_id": 1, "dogs_id": 1}] == list(dogs_humans.rows)
def test_insert_m2m_alter(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m(
"humans", {"id": 1, "name": "Natalie D"}, pk="id"
)
dogs.update(1).m2m(
"humans", {"id": 2, "name": "Simon W", "nerd": True}, pk="id", alter=True
)
assert list(fresh_db.table("humans").rows) == [
assert list(fresh_db["humans"].rows) == [
{"id": 1, "name": "Natalie D", "nerd": None},
{"id": 2, "name": "Simon W", "nerd": 1},
]
assert list(fresh_db.table("dogs_humans").rows) == [
assert list(fresh_db["dogs_humans"].rows) == [
{"humans_id": 1, "dogs_id": 1},
{"humans_id": 2, "dogs_id": 1},
]
def test_insert_m2m_list(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m(
"humans",
[{"id": 1, "name": "Natalie D"}, {"id": 2, "name": "Simon W"}],
pk="id",
)
assert {"dogs", "humans", "dogs_humans"} == set(fresh_db.table_names())
humans = fresh_db.table("humans")
dogs_humans = fresh_db.table("dogs_humans")
humans = fresh_db["humans"]
dogs_humans = fresh_db["dogs_humans"]
assert [{"humans_id": 1, "dogs_id": 1}, {"humans_id": 2, "dogs_id": 1}] == list(
dogs_humans.rows
)
@ -68,7 +68,7 @@ def test_insert_m2m_iterable(fresh_db):
def iterable():
yield from iterable_records
platypuses = fresh_db.table("platypuses")
platypuses = fresh_db["platypuses"]
platypuses.insert({"id": 1, "name": "Perry"}, pk="id").m2m(
"humans",
iterable(),
@ -76,8 +76,8 @@ def test_insert_m2m_iterable(fresh_db):
)
assert {"platypuses", "humans", "humans_platypuses"} == set(fresh_db.table_names())
humans = fresh_db.table("humans")
humans_platypuses = fresh_db.table("humans_platypuses")
humans = fresh_db["humans"]
humans_platypuses = fresh_db["humans_platypuses"]
assert [
{"humans_id": 1, "platypuses_id": 1},
{"humans_id": 2, "platypuses_id": 1},
@ -111,14 +111,14 @@ def test_m2m_with_table_objects(fresh_db):
assert expected_tables == set(fresh_db.table_names())
assert dogs.count == 1
assert humans.count == 2
assert fresh_db.table("dogs_humans").count == 2
assert fresh_db["dogs_humans"].count == 2
def test_m2m_lookup(fresh_db):
people = fresh_db.table("people", pk="id")
people.insert({"name": "Wahyu"}).m2m("tags", lookup={"tag": "Coworker"})
people_tags = fresh_db.table("people_tags")
tags = fresh_db.table("tags")
people_tags = fresh_db["people_tags"]
tags = fresh_db["tags"]
assert people_tags.exists()
assert tags.exists()
assert [
@ -150,9 +150,9 @@ def test_m2m_explicit_table_name_argument(fresh_db):
people.insert({"name": "Wahyu"}).m2m(
"tags", lookup={"tag": "Coworker"}, m2m_table="tagged"
)
assert fresh_db.table("tags").exists
assert fresh_db.table("tagged").exists
assert not fresh_db.table("people_tags").exists()
assert fresh_db["tags"].exists
assert fresh_db["tagged"].exists
assert not fresh_db["people_tags"].exists()
def test_m2m_table_candidates(fresh_db):
@ -181,25 +181,25 @@ def test_uses_existing_m2m_table_if_exists(fresh_db):
# Code should look for an existing table with fks to both tables
# and use that if it exists.
people = fresh_db.create_table("people", {"id": int, "name": str}, pk="id")
fresh_db.table("tags").lookup({"tag": "Coworker"})
fresh_db["tags"].lookup({"tag": "Coworker"})
fresh_db.create_table(
"tagged",
{"people_id": int, "tags_id": int},
foreign_keys=["people_id", "tags_id"],
)
people.insert({"name": "Wahyu"}).m2m("tags", lookup={"tag": "Coworker"})
assert fresh_db.table("tags").exists()
assert fresh_db.table("tagged").exists()
assert not fresh_db.table("people_tags").exists()
assert not fresh_db.table("tags_people").exists()
assert [{"people_id": 1, "tags_id": 1}] == list(fresh_db.table("tagged").rows)
assert fresh_db["tags"].exists()
assert fresh_db["tagged"].exists()
assert not fresh_db["people_tags"].exists()
assert not fresh_db["tags_people"].exists()
assert [{"people_id": 1, "tags_id": 1}] == list(fresh_db["tagged"].rows)
def test_requires_explicit_m2m_table_if_multiple_options(fresh_db):
# If the code scans for m2m tables and finds more than one candidate
# it should require that the m2m_table=x argument is used
people = fresh_db.create_table("people", {"id": int, "name": str}, pk="id")
fresh_db.table("tags").lookup({"tag": "Coworker"})
fresh_db["tags"].lookup({"tag": "Coworker"})
fresh_db.create_table(
"tagged",
{"people_id": int, "tags_id": int},

View file

@ -10,11 +10,11 @@ def migrations():
@migrations()
def m001(db):
db.table("dogs").insert({"name": "Cleo"})
db["dogs"].insert({"name": "Cleo"})
@migrations()
def m002(db):
db.table("cats").create({"name": str})
db["cats"].create({"name": str})
db.execute("insert into dogs (name) values ('Pancakes')")
return migrations
@ -28,11 +28,11 @@ def migrations_not_ordered_alphabetically():
@migrations()
def m002(db):
db.table("dogs").insert({"name": "Cleo"})
db["dogs"].insert({"name": "Cleo"})
@migrations()
def m001(db):
db.table("cats").create({"name": str})
db["cats"].create({"name": str})
db.execute("insert into dogs (name) values ('Pancakes')")
return migrations
@ -44,7 +44,7 @@ def migrations2():
@migrations()
def m001(db):
db.table("dogs2").insert({"name": "Cleo"})
db["dogs2"].insert({"name": "Cleo"})
return migrations
@ -96,7 +96,7 @@ def test_applied_at_is_a_string(migrations):
def test_failing_migration_rolls_back(migrations):
@migrations()
def m003(db):
db.table("birds").create({"name": str})
db["birds"].create({"name": str})
db.execute("insert into dogs (name) values ('Dozer')")
raise ValueError("boom")
@ -105,7 +105,7 @@ def test_failing_migration_rolls_back(migrations):
migrations.apply(db)
# m001 and m002 committed before the failure and stay applied
assert set(db.table_names()) == {"_sqlite_migrations", "dogs", "cats"}
assert [r["name"] for r in db.table("dogs").rows] == ["Cleo", "Pancakes"]
assert [r["name"] for r in db["dogs"].rows] == ["Cleo", "Pancakes"]
assert [m.name for m in migrations.applied(db)] == ["m001", "m002"]
# Everything m003 did was rolled back and it is still pending
assert [m.name for m in migrations.pending(db)] == ["m003"]
@ -117,11 +117,11 @@ def test_rerun_after_failure_applies_each_migration_once():
@migrations()
def m001(db):
db.table("dogs").insert({"name": "Cleo"})
db["dogs"].insert({"name": "Cleo"})
@migrations()
def m002(db):
db.table("dogs").insert({"name": "Pancakes"})
db["dogs"].insert({"name": "Pancakes"})
if state["fail"]:
raise ValueError("boom")
@ -131,7 +131,7 @@ def test_rerun_after_failure_applies_each_migration_once():
state["fail"] = False
migrations.apply(db)
# m001 must not have been re-applied, m002 applied exactly once
assert [r["name"] for r in db.table("dogs").rows] == ["Cleo", "Pancakes"]
assert [r["name"] for r in db["dogs"].rows] == ["Cleo", "Pancakes"]
assert [m.name for m in migrations.applied(db)] == ["m001", "m002"]
@ -142,7 +142,7 @@ def test_non_transactional_migration_allows_vacuum(tmpdir):
@migrations()
def m001(db):
db.table("dogs").insert({"name": "Cleo"})
db["dogs"].insert({"name": "Cleo"})
@migrations(transactional=False)
def m002(db):
@ -185,13 +185,11 @@ def test_apply_composes_inside_outer_transaction(migrations):
)
def test_upgrades_sqlite_migrations(migrations, create_table, pk):
db = sqlite_utils.Database(memory=True)
db.table("_sqlite_migrations").create(create_table, pk=pk)
db["_sqlite_migrations"].create(create_table, pk=pk)
assert db.table_names() == ["_sqlite_migrations"]
assert db.table("_sqlite_migrations").pks == (
[pk] if isinstance(pk, str) else list(pk)
)
assert db["_sqlite_migrations"].pks == ([pk] if isinstance(pk, str) else list(pk))
migrations.apply(db)
assert db.table("_sqlite_migrations").pks == ["id"]
assert db["_sqlite_migrations"].pks == ["id"]
def test_pending_and_applied_are_read_only(migrations):
@ -229,7 +227,7 @@ def test_stop_before_applied_migration_errors(migrations):
assert "m001" in str(ex.value)
assert "already been applied" in str(ex.value)
# Nothing else was applied
assert not db.table("cats").exists()
assert not db["cats"].exists()
def test_stop_before_applied_migration_errors_before_any_apply(migrations):
@ -240,9 +238,9 @@ def test_stop_before_applied_migration_errors_before_any_apply(migrations):
@only_second()
def m002(db):
db.table("cats").create({"name": str})
db["cats"].create({"name": str})
only_second.apply(db) # m002 applied, m001 still pending
with pytest.raises(ValueError):
migrations.apply(db, stop_before="m002")
assert not db.table("dogs").exists()
assert not db["dogs"].exists()

View file

@ -112,7 +112,7 @@ def test_mutator_commits_by_default(tmp_path, mutate, expected_rows):
db = seed_database(path)
assert not db.conn.in_transaction
mutate(db.table("items"))
mutate(db["items"])
assert current_rows(db) == expected_rows
assert not db.conn.in_transaction
@ -127,7 +127,7 @@ def test_mutator_commits_with_outer_atomic(tmp_path, mutate, expected_rows):
with db.atomic():
assert db.conn.in_transaction
mutate(db.table("items"))
mutate(db["items"])
assert current_rows(db) == expected_rows
assert db.conn.in_transaction
@ -143,7 +143,7 @@ def test_mutator_rolls_back_outer_atomic(tmp_path, mutate, expected_rows):
db = seed_database(path)
with pytest.raises(RollbackTest), db.atomic():
mutate(db.table("items"))
mutate(db["items"])
assert current_rows(db) == expected_rows
assert db.conn.in_transaction
raise RollbackTest

View file

@ -6,7 +6,7 @@ from sqlite_utils.utils import sqlite3
def test_query(fresh_db):
fresh_db.table("dogs").insert_all([{"name": "Cleo"}, {"name": "Pancakes"}])
fresh_db["dogs"].insert_all([{"name": "Cleo"}, {"name": "Pancakes"}])
results = fresh_db.query("select * from dogs order by name desc")
assert isinstance(results, types.GeneratorType)
assert list(results) == [{"name": "Pancakes"}, {"name": "Cleo"}]
@ -20,13 +20,13 @@ def test_query_executes_eagerly(fresh_db):
def test_query_rejects_statements_that_return_no_rows(fresh_db):
fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db["dogs"].insert({"name": "Cleo"})
with pytest.raises(ValueError) as ex:
fresh_db.query("update dogs set name = 'Cleopaws'")
assert "execute()" in str(ex.value)
# The rejected update was rolled back, and no transaction is left open
assert not fresh_db.conn.in_transaction
assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"]
def test_query_rejected_ddl_is_rolled_back(fresh_db):
@ -37,7 +37,7 @@ def test_query_rejected_ddl_is_rolled_back(fresh_db):
def test_query_rejected_write_inside_transaction_is_rolled_back(fresh_db):
fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db["dogs"].insert({"name": "Cleo"})
fresh_db.begin()
fresh_db.execute("insert into dogs (name) values ('Pancakes')")
with pytest.raises(ValueError):
@ -45,7 +45,7 @@ def test_query_rejected_write_inside_transaction_is_rolled_back(fresh_db):
# The transaction is still open and the earlier insert is intact
assert fresh_db.conn.in_transaction
fresh_db.commit()
assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo", "Pancakes"]
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo", "Pancakes"]
@pytest.mark.parametrize(
@ -77,7 +77,7 @@ def test_query_comment_prefixed_commit_does_not_commit_transaction(fresh_db):
# A COMMIT hidden behind a leading comment must not slip past the
# keyword check - previously it committed the caller's open
# transaction before the ValueError was raised
fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db["dogs"].insert({"name": "Cleo"})
fresh_db.begin()
fresh_db.execute("insert into dogs (name) values ('Pancakes')")
with pytest.raises(ValueError):
@ -85,7 +85,7 @@ def test_query_comment_prefixed_commit_does_not_commit_transaction(fresh_db):
# The explicit transaction is still open and can still be rolled back
assert fresh_db.conn.in_transaction
fresh_db.rollback()
assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"]
@pytest.mark.parametrize("sql", ["; COMMIT", "\ufeffCOMMIT"])
@ -94,7 +94,7 @@ def test_query_prefixed_commit_does_not_commit_transaction(fresh_db, sql):
# real token, so the keyword scanner must skip them too - previously
# '; COMMIT' slipped past the check and committed the caller's open
# transaction before raising OperationalError
fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db["dogs"].insert({"name": "Cleo"})
fresh_db.begin()
fresh_db.execute("insert into dogs (name) values ('Pancakes')")
with pytest.raises(ValueError):
@ -102,7 +102,7 @@ def test_query_prefixed_commit_does_not_commit_transaction(fresh_db, sql):
# The explicit transaction is still open and can still be rolled back
assert fresh_db.conn.in_transaction
fresh_db.rollback()
assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"]
def test_query_error_leaves_no_transaction_open(fresh_db):
@ -190,12 +190,12 @@ def test_first_keyword(sql, expected):
reason="RETURNING requires SQLite 3.35.0 or higher",
)
def test_query_insert_returning(fresh_db):
fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db["dogs"].insert({"name": "Cleo"})
rows = list(
fresh_db.query("insert into dogs (name) values ('Pancakes') returning name")
)
assert rows == [{"name": "Pancakes"}]
assert fresh_db.table("dogs").count == 2
assert fresh_db["dogs"].count == 2
@pytest.mark.skipif(
@ -207,7 +207,7 @@ def test_query_insert_returning_commits_without_iteration(tmpdir):
path = str(tmpdir / "test.db")
db = Database(path)
db.table("dogs").insert({"name": "Cleo"})
db["dogs"].insert({"name": "Cleo"})
# Never iterate over the results
db.query("insert into dogs (name) values ('Pancakes') returning name")
assert not db.conn.in_transaction
@ -227,7 +227,7 @@ def test_query_insert_returning_partial_iteration_still_commits(tmpdir):
path = str(tmpdir / "test.db")
db = Database(path)
db.table("dogs").insert({"name": "Cleo"})
db["dogs"].insert({"name": "Cleo"})
row = next(
db.query(
"insert into dogs (name) values ('Pancakes'), ('Marnie') returning name"
@ -246,7 +246,7 @@ def test_query_insert_returning_partial_iteration_still_commits(tmpdir):
reason="RETURNING requires SQLite 3.35.0 or higher",
)
def test_query_insert_returning_respects_explicit_transaction(fresh_db):
fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db["dogs"].insert({"name": "Cleo"})
fresh_db.begin()
rows = list(
fresh_db.query("insert into dogs (name) values ('Pancakes') returning name")
@ -255,13 +255,13 @@ def test_query_insert_returning_respects_explicit_transaction(fresh_db):
# Still inside the explicit transaction - not committed
assert fresh_db.conn.in_transaction
fresh_db.rollback()
assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"]
def test_query_duplicate_column_names_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624
fresh_db.table("one").insert({"id": 1, "value": "left"})
fresh_db.table("two").insert({"id": 2, "value": "right"})
fresh_db["one"].insert({"id": 1, "value": "left"})
fresh_db["two"].insert({"id": 2, "value": "right"})
rows = list(
fresh_db.query("select one.id, two.id, one.value, two.value from one, two")
)
@ -277,7 +277,7 @@ def test_query_deduped_column_avoids_existing_names(fresh_db):
def test_execute_returning_dicts(fresh_db):
# Like db.query() but returns a list, included for backwards compatibility
# see https://github.com/simonw/sqlite-utils/issues/290
fresh_db.table("test").insert({"id": 1, "bar": 2}, pk="id")
fresh_db["test"].insert({"id": 1, "bar": 2}, pk="id")
assert fresh_db.execute_returning_dicts("select * from test") == [
{"id": 1, "bar": 2}
]

View file

@ -8,7 +8,7 @@ from sqlite_utils.utils import sqlite3
@pytest.fixture
def dates_db(fresh_db):
fresh_db.table("example").insert_all(
fresh_db["example"].insert_all(
[
{"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6th October 2019 00:05:06"},
@ -21,8 +21,8 @@ def dates_db(fresh_db):
def test_parsedate(dates_db):
dates_db.table("example").convert("dt", recipes.parsedate)
assert list(dates_db.table("example").rows) == [
dates_db["example"].convert("dt", recipes.parsedate)
assert list(dates_db["example"].rows) == [
{"id": 1, "dt": "2019-10-05"},
{"id": 2, "dt": "2019-10-06"},
{"id": 3, "dt": ""},
@ -31,8 +31,8 @@ def test_parsedate(dates_db):
def test_parsedatetime(dates_db):
dates_db.table("example").convert("dt", recipes.parsedatetime)
assert list(dates_db.table("example").rows) == [
dates_db["example"].convert("dt", recipes.parsedatetime)
assert list(dates_db["example"].rows) == [
{"id": 1, "dt": "2019-10-05T12:04:00"},
{"id": 2, "dt": "2019-10-06T00:05:06"},
{"id": 3, "dt": ""},
@ -50,16 +50,16 @@ def test_parsedatetime(dates_db):
),
)
def test_dayfirst_yearfirst(fresh_db, recipe, kwargs, expected):
fresh_db.table("example").insert_all(
fresh_db["example"].insert_all(
[
{"id": 1, "dt": "03/04/05"},
],
pk="id",
)
fresh_db.table("example").convert(
fresh_db["example"].convert(
"dt", lambda value: getattr(recipes, recipe)(value, **kwargs)
)
assert list(fresh_db.table("example").rows) == [
assert list(fresh_db["example"].rows) == [
{"id": 1, "dt": expected},
]
@ -68,7 +68,7 @@ def test_dayfirst_yearfirst(fresh_db, recipe, kwargs, expected):
@pytest.mark.parametrize("fn", ("parsedate", "parsedatetime"))
def test_dateparse_errors_raises(fresh_db, fn):
"""Test that invalid dates raise errors when errors=None"""
fresh_db.table("example").insert_all(
fresh_db["example"].insert_all(
[
{"id": 1, "dt": "invalid"},
],
@ -76,32 +76,30 @@ def test_dateparse_errors_raises(fresh_db, fn):
)
# Exception in SQLite callback surfaces as OperationalError
with pytest.raises(sqlite3.OperationalError):
fresh_db.table("example").convert(
"dt", lambda value: getattr(recipes, fn)(value)
)
fresh_db["example"].convert("dt", lambda value: getattr(recipes, fn)(value))
@pytest.mark.parametrize("fn", ("parsedate", "parsedatetime"))
@pytest.mark.parametrize("errors", (recipes.SET_NULL, recipes.IGNORE))
def test_dateparse_errors_handled(fresh_db, fn, errors):
"""Test error handling modes for invalid dates"""
fresh_db.table("example").insert_all(
fresh_db["example"].insert_all(
[
{"id": 1, "dt": "invalid"},
],
pk="id",
)
fresh_db.table("example").convert(
fresh_db["example"].convert(
"dt", lambda value: getattr(recipes, fn)(value, errors=errors)
)
rows = list(fresh_db.table("example").rows)
rows = list(fresh_db["example"].rows)
expected = [{"id": 1, "dt": None if errors is recipes.SET_NULL else "invalid"}]
assert rows == expected
@pytest.mark.parametrize("delimiter", [None, ";", "-"])
def test_jsonsplit(fresh_db, delimiter):
fresh_db.table("example").insert_all(
fresh_db["example"].insert_all(
[
{"id": 1, "tags": (delimiter or ",").join(["foo", "bar"])},
{"id": 2, "tags": (delimiter or ",").join(["bar", "baz"])},
@ -116,8 +114,8 @@ def test_jsonsplit(fresh_db, delimiter):
else:
fn = recipes.jsonsplit
fresh_db.table("example").convert("tags", fn)
assert list(fresh_db.table("example").rows) == [
fresh_db["example"].convert("tags", fn)
assert list(fresh_db["example"].rows) == [
{"id": 1, "tags": '["foo", "bar"]'},
{"id": 2, "tags": '["bar", "baz"]'},
]
@ -132,7 +130,7 @@ def test_jsonsplit(fresh_db, delimiter):
),
)
def test_jsonsplit_type(fresh_db, type, expected):
fresh_db.table("example").insert_all(
fresh_db["example"].insert_all(
[
{"id": 1, "records": "1,2,3"},
],
@ -146,5 +144,5 @@ def test_jsonsplit_type(fresh_db, type, expected):
else:
fn = recipes.jsonsplit
fresh_db.table("example").convert("records", fn)
assert json.loads(fresh_db.table("example").get(1)["records"]) == expected
fresh_db["example"].convert("records", fn)
assert json.loads(fresh_db["example"].get(1)["records"]) == expected

View file

@ -33,8 +33,8 @@ def test_recreate(tmp_path, use_path, create_file_first):
filepath = pathlib.Path(filepath)
if create_file_first:
db = Database(filepath)
db.table("t1").insert({"foo": "bar"})
db["t1"].insert({"foo": "bar"})
assert ["t1"] == db.table_names()
db.close()
Database(filepath, recreate=True).table("t2").insert({"foo": "bar"})
Database(filepath, recreate=True)["t2"].insert({"foo": "bar"})
assert ["t2"] == Database(filepath).table_names()

View file

@ -86,21 +86,21 @@ def test_register_function_deterministic_tries_again_if_exception_raised(fresh_d
def test_register_function_replace(fresh_db):
@fresh_db.register_function()
def one(): # pyright: ignore[reportRedeclaration]
def one():
return "one"
assert "one" == fresh_db.execute("select one()").fetchone()[0]
# This will silently fail to replaec the function
@fresh_db.register_function()
def one(): # pyright: ignore[reportRedeclaration]
def one(): # noqa
return "two"
assert "one" == fresh_db.execute("select one()").fetchone()[0]
# This will replace it
@fresh_db.register_function(replace=True)
def one(): # pyright: ignore[reportRedeclaration]
def one(): # noqa
return "two"
assert "two" == fresh_db.execute("select one()").fetchone()[0]

View file

@ -3,7 +3,7 @@ import pytest
def test_rows(existing_db):
assert [{"text": "one"}, {"text": "two"}, {"text": "three"}] == list(
existing_db.table("foo").rows
existing_db["foo"].rows
)
@ -18,7 +18,7 @@ def test_rows(existing_db):
],
)
def test_rows_where(where, where_args, expected_ids, fresh_db):
table = fresh_db.table("dogs")
table = fresh_db["dogs"]
table.insert_all(
[
{"id": 1, "name": "Cleo", "age": 4, "is_good": True},
@ -41,7 +41,7 @@ def test_rows_where(where, where_args, expected_ids, fresh_db):
],
)
def test_rows_where_order_by(where, order_by, expected_ids, fresh_db):
table = fresh_db.table("dogs")
table = fresh_db["dogs"]
table.insert_all(
[
{"id": 1, "name": "Cleo", "age": 4},
@ -59,13 +59,10 @@ def test_rows_where_order_by(where, order_by, expected_ids, fresh_db):
(None, 3, [1, 2, 3]),
(0, 3, [1, 2, 3]),
(3, 3, [4, 5, 6]),
# offset without limit should return every remaining row
(97, None, [98, 99, 100]),
(0, None, list(range(1, 101))),
],
)
def test_rows_where_offset_limit(fresh_db, offset, limit, expected):
table = fresh_db.table("rows")
table = fresh_db["rows"]
table.insert_all([{"id": id} for id in range(1, 101)], pk="id")
assert table.count == 100
assert expected == [
@ -73,14 +70,8 @@ def test_rows_where_offset_limit(fresh_db, offset, limit, expected):
]
def test_pks_and_rows_where_offset_without_limit(fresh_db):
table = fresh_db.table("rows")
table.insert_all([{"id": id} for id in range(1, 6)], pk="id")
assert [pk for pk, _ in table.pks_and_rows_where(offset=3, order_by="id")] == [4, 5]
def test_pks_and_rows_where_rowid(fresh_db):
table = fresh_db.table("rowid_table")
table = fresh_db["rowid_table"]
table.insert_all({"number": i + 10} for i in range(3))
pks_and_rows = list(table.pks_and_rows_where())
assert pks_and_rows == [
@ -91,7 +82,7 @@ def test_pks_and_rows_where_rowid(fresh_db):
def test_pks_and_rows_where_simple_pk(fresh_db):
table = fresh_db.table("simple_pk_table")
table = fresh_db["simple_pk_table"]
table.insert_all(({"id": i + 10} for i in range(3)), pk="id")
pks_and_rows = list(table.pks_and_rows_where())
assert pks_and_rows == [
@ -102,7 +93,7 @@ def test_pks_and_rows_where_simple_pk(fresh_db):
def test_pks_and_rows_where_compound_pk(fresh_db):
table = fresh_db.table("compound_pk_table")
table = fresh_db["compound_pk_table"]
table.insert_all(
({"type": "number", "number": i, "plusone": i + 1} for i in range(3)),
pk=("type", "number"),
@ -117,8 +108,8 @@ def test_pks_and_rows_where_compound_pk(fresh_db):
def test_rows_where_duplicate_select_columns_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624
fresh_db.table("t").insert({"id": 1, "name": "Cleo"})
rows = list(fresh_db.table("t").rows_where(select="id, id, name"))
fresh_db["t"].insert({"id": 1, "name": "Cleo"})
rows = list(fresh_db["t"].rows_where(select="id, id, name"))
assert rows == [{"id": 1, "id_2": 1, "name": "Cleo"}]
@ -130,10 +121,10 @@ def test_pks_and_rows_where_view(fresh_db):
# an AttributeError from View lacking Table-only properties
from sqlite_utils.utils import sqlite3
fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.create_view("dog_names", "select name from dogs")
try:
result = list(fresh_db.view("dog_names").pks_and_rows_where())
result = list(fresh_db["dog_names"].pks_and_rows_where())
except sqlite3.OperationalError:
pass # SQLite 3.36+: no such column: rowid
else:
@ -144,6 +135,6 @@ def test_pks_and_rows_where_view(fresh_db):
def test_pks_and_rows_where_compound_pk_declaration_order(fresh_db):
# Compound pks are returned in PRIMARY KEY declaration order
fresh_db.execute("create table t (b text, a text, primary key (a, b))")
fresh_db.table("t").insert({"a": "A", "b": "B"})
pks_and_rows = list(fresh_db.table("t").pks_and_rows_where())
fresh_db["t"].insert({"a": "A", "b": "B"})
pks_and_rows = list(fresh_db["t"].pks_and_rows_where())
assert pks_and_rows == [(("A", "B"), {"b": "B", "a": "A"})]

View file

@ -20,13 +20,6 @@ def test_rows_from_file_detect_format(input, expected_format):
assert rows_list == [{"id": "1", "name": "Cleo"}]
@pytest.mark.parametrize("input", (b"", b" \n\t"))
def test_rows_from_file_empty_input(input):
rows, format = rows_from_file(BytesIO(input))
assert format == Format.CSV
assert list(rows) == []
@pytest.mark.parametrize(
"ignore_extras,extras_key,expected",
(

View file

@ -19,7 +19,7 @@ def test_sniff(tmpdir, filepath):
)
assert result.exit_code == 0, result.stdout
db = Database(db_path)
assert list(db.table("creatures").rows) == [
assert list(db["creatures"].rows) == [
{"id": "1", "species": "dog", "name": "Cleo", "age": "5"},
{"id": "2", "species": "dog", "name": "Pancakes", "age": "4"},
{"id": "3", "species": "cat", "name": "Mozie", "age": "8"},

View file

@ -2,7 +2,6 @@ import sqlite3
import pytest
from sqlite_utils import ANY
from sqlite_utils.db import Check, ForeignKey, TransactionError, TransformError
from sqlite_utils.utils import OperationalError
@ -27,7 +26,7 @@ from sqlite_utils.utils import OperationalError
{"types": {"age": int}},
[
'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "age" INTEGER\n);',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", NULLIF("age", \'\') FROM "dogs";',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", "age" FROM "dogs";',
'DROP TABLE "dogs";',
"PRAGMA legacy_alter_table=ON;",
'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";',
@ -63,7 +62,7 @@ from sqlite_utils.utils import OperationalError
{"types": {"age": int}, "rename": {"age": "dog_age"}},
[
'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "dog_age" INTEGER\n);',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "dog_age")\n SELECT "rowid", "id", "name", NULLIF("age", \'\') FROM "dogs";',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "dog_age")\n SELECT "rowid", "id", "name", "age" FROM "dogs";',
'DROP TABLE "dogs";',
"PRAGMA legacy_alter_table=ON;",
'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";',
@ -129,7 +128,7 @@ def test_transform_sql_table_with_primary_key(
def tracer(sql, params):
return captured.append((sql, params))
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
if use_pragma_foreign_keys:
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
@ -168,7 +167,7 @@ def test_transform_sql_table_with_primary_key(
{"types": {"age": int}},
[
'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER,\n "name" TEXT,\n "age" INTEGER\n);',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", NULLIF("age", \'\') FROM "dogs";',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", "age" FROM "dogs";',
'DROP TABLE "dogs";',
"PRAGMA legacy_alter_table=ON;",
'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";',
@ -210,7 +209,7 @@ def test_transform_sql_table_with_no_primary_key(
def tracer(sql, params):
return captured.append((sql, params))
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
if use_pragma_foreign_keys:
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"})
@ -230,7 +229,7 @@ def test_transform_sql_table_with_no_primary_key(
def test_transform_sql_with_no_primary_key_to_primary_key_of_id(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": "5"})
assert (
dogs.schema
@ -245,7 +244,7 @@ def test_transform_sql_with_no_primary_key_to_primary_key_of_id(fresh_db):
def test_transform_rename_pk(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
dogs.transform(rename={"id": "pk"})
assert (
@ -266,7 +265,7 @@ def test_transform_preserves_keyword_literal_defaults(fresh_db):
" note TEXT DEFAULT NULL"
")"
)
table = fresh_db.table("t")
table = fresh_db["t"]
table.insert({"id": 1})
before = fresh_db.execute("SELECT is_active, flag, note FROM t").fetchone()
assert before == (1, 0, None)
@ -289,7 +288,7 @@ def test_transform_preserves_keyword_literal_defaults(fresh_db):
def test_transform_not_null(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
dogs.transform(not_null={"name"})
assert (
@ -299,7 +298,7 @@ def test_transform_not_null(fresh_db):
def test_transform_remove_a_not_null(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, not_null={"age"}, pk="id")
dogs.transform(not_null={"name": True, "age": False})
assert (
@ -310,7 +309,7 @@ def test_transform_remove_a_not_null(fresh_db):
@pytest.mark.parametrize("not_null", [{"age"}, {"age": True}])
def test_transform_add_not_null_with_rename(fresh_db, not_null):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
dogs.transform(not_null=not_null, rename={"age": "dog_age"})
assert (
@ -320,7 +319,7 @@ def test_transform_add_not_null_with_rename(fresh_db, not_null):
def test_transform_defaults(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id")
dogs.transform(defaults={"age": 1})
assert (
@ -330,7 +329,7 @@ def test_transform_defaults(fresh_db):
def test_transform_defaults_and_rename_column(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id")
dogs.transform(rename={"age": "dog_age"}, defaults={"age": 1})
assert (
@ -340,7 +339,7 @@ def test_transform_defaults_and_rename_column(fresh_db):
def test_remove_defaults(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, defaults={"age": 1}, pk="id")
dogs.transform(defaults={"age": None})
assert (
@ -351,8 +350,8 @@ def test_remove_defaults(fresh_db):
@pytest.fixture
def authors_db(fresh_db):
books = fresh_db.table("books")
authors = fresh_db.table("authors")
books = fresh_db["books"]
authors = fresh_db["authors"]
authors.insert({"id": 5, "name": "Jane McGonical"}, pk="id")
books.insert(
{"id": 2, "title": "Reality is Broken", "author_id": 5},
@ -363,13 +362,13 @@ def authors_db(fresh_db):
def test_transform_foreign_keys_persist(authors_db):
assert authors_db.table("books").foreign_keys == [
assert authors_db["books"].foreign_keys == [
ForeignKey(
table="books", column="author_id", other_table="authors", other_column="id"
)
]
authors_db.table("books").transform(rename={"title": "book_title"})
assert authors_db.table("books").foreign_keys == [
authors_db["books"].transform(rename={"title": "book_title"})
assert authors_db["books"].foreign_keys == [
ForeignKey(
table="books", column="author_id", other_table="authors", other_column="id"
)
@ -382,8 +381,8 @@ def test_transform_foreign_keys_survive_renamed_column(
):
if use_pragma_foreign_keys:
authors_db.conn.execute("PRAGMA foreign_keys=ON")
authors_db.table("books").transform(rename={"author_id": "author_id_2"})
assert authors_db.table("books").foreign_keys == [
authors_db["books"].transform(rename={"author_id": "author_id_2"})
assert authors_db["books"].foreign_keys == [
ForeignKey(
table="books",
column="author_id_2",
@ -394,9 +393,9 @@ def test_transform_foreign_keys_survive_renamed_column(
def _add_country_city_continent(db):
db.table("country").insert({"id": 1, "name": "France"}, pk="id")
db.table("continent").insert({"id": 2, "name": "Europe"}, pk="id")
db.table("city").insert({"id": 24, "name": "Paris"}, pk="id")
db["country"].insert({"id": 1, "name": "France"}, pk="id")
db["continent"].insert({"id": 2, "name": "Europe"}, pk="id")
db["city"].insert({"id": 24, "name": "Paris"}, pk="id")
_CAVEAU = {
@ -414,11 +413,11 @@ def test_transform_drop_foreign_keys(fresh_db, use_pragma_foreign_keys):
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
# Create table with three foreign keys so we can drop two of them
_add_country_city_continent(fresh_db)
fresh_db.table("places").insert(
fresh_db["places"].insert(
_CAVEAU,
foreign_keys=("country", "continent", "city"),
)
assert fresh_db.table("places").foreign_keys == [
assert fresh_db["places"].foreign_keys == [
ForeignKey(
table="places", column="city", other_table="city", other_column="id"
),
@ -433,9 +432,9 @@ def test_transform_drop_foreign_keys(fresh_db, use_pragma_foreign_keys):
),
]
# Drop two of those foreign keys
fresh_db.table("places").transform(drop_foreign_keys=("country", "continent"))
fresh_db["places"].transform(drop_foreign_keys=("country", "continent"))
# Should be only one foreign key now
assert fresh_db.table("places").foreign_keys == [
assert fresh_db["places"].foreign_keys == [
ForeignKey(table="places", column="city", other_table="city", other_column="id")
]
if use_pragma_foreign_keys:
@ -444,17 +443,17 @@ def test_transform_drop_foreign_keys(fresh_db, use_pragma_foreign_keys):
def test_transform_verify_foreign_keys(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db.table("authors").insert({"id": 3, "name": "Tina"}, pk="id")
fresh_db.table("books").insert(
fresh_db["authors"].insert({"id": 3, "name": "Tina"}, pk="id")
fresh_db["books"].insert(
{"id": 1, "title": "Book", "author_id": 3}, pk="id", foreign_keys={"author_id"}
)
# Renaming the id column on authors should break everything
with pytest.raises(OperationalError) as e:
fresh_db.table("authors").transform(rename={"id": "id2"})
fresh_db["authors"].transform(rename={"id": "id2"})
assert e.value.args[0] == 'foreign key mismatch - "books" referencing "authors"'
# This should have rolled us back
assert (
fresh_db.table("authors").schema
fresh_db["authors"].schema
== 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT\n)'
)
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -477,22 +476,20 @@ def test_transform_on_delete_cascade_does_not_delete_records(
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
);
""")
fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1})
# Transform the table on the other end of the cascading foreign key
fresh_db.table("authors").transform(rename={"name": "author_name"})
assert list(fresh_db.table("authors").rows) == [
fresh_db["authors"].transform(rename={"name": "author_name"})
assert list(fresh_db["authors"].rows) == [
{"id": 1, "author_name": "Ursula K. Le Guin"}
]
assert list(fresh_db.table("books").rows) == [
assert list(fresh_db["books"].rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1}
]
# Transforming the table with the cascading foreign key should not
# delete its records either
fresh_db.table("books").transform(rename={"title": "book_title"})
assert list(fresh_db.table("books").rows) == [
fresh_db["books"].transform(rename={"title": "book_title"})
assert list(fresh_db["books"].rows) == [
{"id": 1, "book_title": "The Dispossessed", "author_id": 1}
]
if use_pragma_foreign_keys:
@ -514,19 +511,17 @@ def test_transform_in_transaction_refuses_destructive_on_delete(fresh_db, on_del
author_id INTEGER REFERENCES authors(id) ON DELETE {on_delete}
);
""")
fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
previous_schema = fresh_db.table("authors").schema
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1})
previous_schema = fresh_db["authors"].schema
with fresh_db.atomic(), pytest.raises(TransactionError) as excinfo:
fresh_db.table("authors").transform(rename={"name": "author_name"})
fresh_db["authors"].transform(rename={"name": "author_name"})
message = str(excinfo.value)
assert "books" in message
assert f"ON DELETE {on_delete.upper()}" in message
# Nothing should have changed
assert fresh_db.table("authors").schema == previous_schema
assert list(fresh_db.table("books").rows) == [
assert fresh_db["authors"].schema == previous_schema
assert list(fresh_db["books"].rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1}
]
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -543,16 +538,16 @@ def test_transform_in_transaction_refuses_self_referential_cascade(fresh_db):
parent_id INTEGER REFERENCES categories(id) ON DELETE CASCADE
);
""")
fresh_db.table("categories").insert_all(
fresh_db["categories"].insert_all(
[
{"id": 1, "name": "Fiction", "parent_id": None},
{"id": 2, "name": "Science Fiction", "parent_id": 1},
]
)
with fresh_db.atomic(), pytest.raises(TransactionError) as excinfo:
fresh_db.table("categories").transform(rename={"name": "title"})
fresh_db["categories"].transform(rename={"name": "title"})
assert "categories" in str(excinfo.value)
assert fresh_db.table("categories").count == 2
assert fresh_db["categories"].count == 2
def test_transform_in_transaction_allowed_with_no_action_foreign_key(fresh_db):
@ -567,16 +562,14 @@ def test_transform_in_transaction_allowed_with_no_action_foreign_key(fresh_db):
author_id INTEGER REFERENCES authors(id)
);
""")
fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1})
with fresh_db.atomic():
fresh_db.table("authors").transform(rename={"name": "author_name"})
assert list(fresh_db.table("authors").rows) == [
fresh_db["authors"].transform(rename={"name": "author_name"})
assert list(fresh_db["authors"].rows) == [
{"id": 1, "author_name": "Ursula K. Le Guin"}
]
assert list(fresh_db.table("books").rows) == [
assert list(fresh_db["books"].rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1}
]
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -594,13 +587,11 @@ def test_transform_in_transaction_allowed_for_child_table(fresh_db):
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
);
""")
fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1})
with fresh_db.atomic():
fresh_db.table("books").transform(rename={"title": "book_title"})
assert list(fresh_db.table("books").rows) == [
fresh_db["books"].transform(rename={"title": "book_title"})
assert list(fresh_db["books"].rows) == [
{"id": 1, "book_title": "The Dispossessed", "author_id": 1}
]
@ -616,28 +607,24 @@ def test_transform_in_transaction_allowed_with_foreign_keys_off(fresh_db):
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
);
""")
fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1})
with fresh_db.atomic():
fresh_db.table("authors").transform(rename={"name": "author_name"})
assert list(fresh_db.table("books").rows) == [
fresh_db["authors"].transform(rename={"name": "author_name"})
assert list(fresh_db["books"].rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1}
]
def test_transform_add_foreign_keys_from_scratch(fresh_db):
_add_country_city_continent(fresh_db)
fresh_db.table("places").insert(_CAVEAU)
fresh_db["places"].insert(_CAVEAU)
# Should have no foreign keys
assert fresh_db.table("places").foreign_keys == []
assert fresh_db["places"].foreign_keys == []
# Now add them using .transform()
fresh_db.table("places").transform(
add_foreign_keys=("country", "continent", "city")
)
fresh_db["places"].transform(add_foreign_keys=("country", "continent", "city"))
# Should now have all three:
assert fresh_db.table("places").foreign_keys == [
assert fresh_db["places"].foreign_keys == [
ForeignKey(
table="places", column="city", other_table="city", other_column="id"
),
@ -651,7 +638,7 @@ def test_transform_add_foreign_keys_from_scratch(fresh_db):
table="places", column="country", other_table="country", other_column="id"
),
]
assert fresh_db.table("places").schema == (
assert fresh_db["places"].schema == (
'CREATE TABLE "places" (\n'
' "id" INTEGER,\n'
' "name" TEXT,\n'
@ -675,18 +662,18 @@ def test_transform_add_foreign_keys_from_scratch(fresh_db):
)
def test_transform_add_foreign_keys_from_partial(fresh_db, add_foreign_keys):
_add_country_city_continent(fresh_db)
fresh_db.table("places").insert(
fresh_db["places"].insert(
_CAVEAU,
foreign_keys=("city",),
)
# Should have one foreign keys
assert fresh_db.table("places").foreign_keys == [
assert fresh_db["places"].foreign_keys == [
ForeignKey(table="places", column="city", other_table="city", other_column="id")
]
# Now add three more using .transform()
fresh_db.table("places").transform(add_foreign_keys=add_foreign_keys)
fresh_db["places"].transform(add_foreign_keys=add_foreign_keys)
# Should now have all three:
assert fresh_db.table("places").foreign_keys == [
assert fresh_db["places"].foreign_keys == [
ForeignKey(
table="places", column="city", other_table="city", other_column="id"
),
@ -715,14 +702,14 @@ def test_transform_add_foreign_keys_from_partial(fresh_db, add_foreign_keys):
)
def test_transform_replace_foreign_keys(fresh_db, foreign_keys):
_add_country_city_continent(fresh_db)
fresh_db.table("places").insert(
fresh_db["places"].insert(
_CAVEAU,
foreign_keys=("city",),
)
assert len(fresh_db.table("places").foreign_keys) == 1
assert len(fresh_db["places"].foreign_keys) == 1
# Replace with two different ones
fresh_db.table("places").transform(foreign_keys=foreign_keys)
assert fresh_db.table("places").schema == (
fresh_db["places"].transform(foreign_keys=foreign_keys)
assert fresh_db["places"].schema == (
'CREATE TABLE "places" (\n'
' "id" INTEGER,\n'
' "name" TEXT,\n'
@ -742,7 +729,7 @@ def test_transform_preserves_rowids(fresh_db, table_type):
pk = ("id", "name")
elif table_type == "rowid":
pk = None
fresh_db.table("places").insert_all(
fresh_db["places"].insert_all(
[
{"id": "1", "name": "Paris", "country": "France"},
{"id": "2", "name": "London", "country": "UK"},
@ -751,13 +738,13 @@ def test_transform_preserves_rowids(fresh_db, table_type):
pk=pk,
)
# Now delete and insert a row to mix up the `rowid` sequence
fresh_db.table("places").delete_where("id = ?", ["2"])
fresh_db.table("places").insert({"id": "4", "name": "London", "country": "UK"})
fresh_db["places"].delete_where("id = ?", ["2"])
fresh_db["places"].insert({"id": "4", "name": "London", "country": "UK"})
previous_rows = [
tuple(row) for row in fresh_db.execute("select rowid, id, name from places")
]
# Transform it
fresh_db.table("places").transform(column_order=("country", "name"))
fresh_db["places"].transform(column_order=("country", "name"))
# Should be the same
next_rows = [
tuple(row) for row in fresh_db.execute("select rowid, id, name from places")
@ -787,7 +774,7 @@ def test_transform_strict(fresh_db, initial_strict, transform_strict, expected_s
def test_transform_to_strict_with_invalid_data(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.create({"id": int})
dogs.insert({"id": "not-an-integer"})
@ -814,7 +801,7 @@ def test_transform_strict_updates_default(fresh_db):
@pytest.mark.parametrize("method_name", ("transform", "transform_sql"))
def test_transform_to_strict_not_supported(fresh_db, method_name):
table = fresh_db.table("items")
table = fresh_db["items"]
table.create({"id": int})
fresh_db._supports_strict = False
@ -824,55 +811,6 @@ def test_transform_to_strict_not_supported(fresh_db, method_name):
assert table.strict is False
def test_transform_preserves_any_column_in_strict_table(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (id integer primary key, data any) strict")
fresh_db.conn.executemany(
"insert into items values (?, ?)",
[
(1, 42),
(2, "000123"),
(3, 3.14),
(4, b"bytes"),
(5, None),
],
)
table = fresh_db["items"]
table.transform()
assert table.strict is True
assert table.columns_dict == {"id": int, "data": ANY}
assert fresh_db.execute(
"select typeof(data), data from items order by id"
).fetchall() == [
("integer", 42),
("text", "000123"),
("real", 3.14),
("blob", b"bytes"),
("null", None),
]
def test_transform_any_column_from_strict_to_non_strict(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (data any) strict")
fresh_db.execute("insert into items values (?)", ("000123",))
table = fresh_db["items"]
table.transform(strict=False)
assert table.strict is False
assert table.columns_dict == {"data": ANY}
# Ordinary non-STRICT ANY columns apply NUMERIC affinity
assert fresh_db.execute("select typeof(data), data from items").fetchone() == (
"integer",
123,
)
@pytest.mark.parametrize(
"indexes, transform_params",
[
@ -885,7 +823,7 @@ def test_transform_any_column_from_strict_to_non_strict(fresh_db):
def test_transform_indexes(fresh_db, indexes, transform_params):
# https://github.com/simonw/sqlite-utils/issues/633
# New table should have same indexes as old table after transformation
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": 5, "breed": "Labrador"}, pk="id")
for index in indexes:
@ -911,13 +849,13 @@ def test_transform_indexes(fresh_db, indexes, transform_params):
if "keep_table" in transform_params:
assert all(
index.origin == "pk"
for index in fresh_db.table(transform_params["keep_table"]).indexes
for index in fresh_db[transform_params["keep_table"]].indexes
)
def test_transform_retains_indexes_with_foreign_keys(fresh_db):
dogs = fresh_db.table("dogs")
owners = fresh_db.table("owners")
dogs = fresh_db["dogs"]
owners = fresh_db["owners"]
dogs.insert({"id": 1, "name": "Cleo", "owner_id": 1}, pk="id")
owners.insert({"id": 1, "name": "Alice"}, pk="id")
@ -943,15 +881,22 @@ def test_transform_retains_indexes_with_foreign_keys(fresh_db):
), f"Indexes before transform: {indexes_before_transform}\nIndexes after transform: {dogs.indexes}"
def test_transform_with_indexes_errors(fresh_db):
# Should error with a compound (name, age) index if age is dropped
dogs = fresh_db.table("dogs")
@pytest.mark.parametrize(
"transform_params",
[
{"rename": {"age": "dog_age"}},
{"drop": ["age"]},
],
)
def test_transform_with_indexes_errors(fresh_db, transform_params):
# Should error with a compound (name, age) index if age is renamed or dropped
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id")
dogs.create_index(["name", "age"])
with pytest.raises(TransformError) as excinfo:
dogs.transform(drop=["age"])
dogs.transform(**transform_params)
assert (
"Index 'idx_dogs_name_age' column 'age' is not in updated table 'dogs'. "
@ -960,204 +905,35 @@ def test_transform_with_indexes_errors(fresh_db):
)
@pytest.mark.parametrize(
("table_name", "index_name"),
(("name", "idx_name"), ("t", "name")),
)
def test_transform_rename_column_with_index(fresh_db, table_name, index_name):
# https://github.com/simonw/sqlite-utils/issues/822
# Use the same name for the table, column and index to ensure only the
# indexed column changes.
table = fresh_db.table(table_name)
table.insert({"id": 1, "name": "Cleo"}, pk="id")
table.create_index(["name"], index_name=index_name)
sqls = table.transform_sql(rename={"name": "full_name"}, tmp_suffix="suffix")
drop_index_sql = f'DROP INDEX IF EXISTS "{index_name}";'
assert drop_index_sql in sqls
assert sqls.index(drop_index_sql) < sqls.index(f'DROP TABLE "{table_name}";')
table.transform(rename={"name": "full_name"})
assert [column.name for column in table.columns] == ["id", "full_name"]
assert [(index.name, index.columns) for index in table.indexes] == [
(index_name, ["full_name"])
]
def test_transform_recreates_renamed_index_from_metadata(fresh_db):
table = fresh_db.table("t")
table.insert({"alpha": "one", "beta": "two"})
# Deliberately use unquoted SQL and index details that need to survive the
# reconstruction. Renaming both columns also guards against cascading
# string substitutions.
fresh_db.execute(
"CREATE UNIQUE INDEX swap_idx ON t(alpha COLLATE NOCASE DESC, beta)"
)
table.transform(rename={"alpha": "beta", "beta": "alpha"})
assert table.columns_dict == {"beta": str, "alpha": str}
assert [(index.name, index.unique, index.columns) for index in table.indexes] == [
("swap_idx", 1, ["beta", "alpha"])
]
key_columns = [column for column in table.xindexes[0].columns if column.key]
assert [(column.name, column.desc, column.coll) for column in key_columns] == [
("beta", 1, "NOCASE"),
("alpha", 0, "BINARY"),
]
@pytest.mark.parametrize(
"index_sql",
(
"CREATE INDEX idx_t_name ON t(lower(name))",
"CREATE INDEX idx_t_name ON t(name) WHERE name IS NOT NULL",
),
)
def test_transform_rename_complex_index_errors(fresh_db, index_sql):
table = fresh_db.table("t")
table.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute(index_sql)
with pytest.raises(TransformError, match="partial or expression index"):
table.transform(rename={"name": "full_name"})
assert table.columns_dict == {"id": int, "name": str}
assert [index.name for index in table.indexes] == ["idx_t_name"]
def test_transform_with_unique_constraint_implicit_index(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
# Create a table with a UNIQUE constraint on 'name', which creates an implicit index
fresh_db.execute("""
CREATE TABLE dogs (
id INTEGER PRIMARY KEY,
name TEXT UNIQUE ON CONFLICT IGNORE,
name TEXT UNIQUE,
age INTEGER
);
""")
dogs.insert({"id": 1, "name": "Cleo", "age": 5})
dogs.transform(types={"age": str}, rename={"name": "dog_name"})
assert 'dog_name" TEXT UNIQUE ON CONFLICT IGNORE' in dogs.schema
dogs.insert({"id": 2, "dog_name": "Cleo", "age": "6"})
assert list(dogs.rows) == [{"id": 1, "dog_name": "Cleo", "age": "5"}]
def test_transform_preserves_composite_unique_constraint(fresh_db):
fresh_db.execute("""
CREATE TABLE memberships (
account_id INTEGER,
email TEXT,
note TEXT,
CONSTRAINT unique_membership
UNIQUE (account_id DESC, email COLLATE NOCASE)
ON CONFLICT ABORT
)
""")
memberships = fresh_db.table("memberships")
memberships.insert({"account_id": 1, "email": "one@example.com", "note": "x"})
memberships.transform(rename={"account_id": "organization_id"}, types={"note": str})
# Attempt to transform the table without modifying 'name'
with pytest.raises(TransformError) as excinfo:
dogs.transform(types={"age": str})
assert (
'CONSTRAINT "unique_membership" UNIQUE '
'("organization_id" DESC, "email" COLLATE "NOCASE") ON CONFLICT ABORT'
in memberships.schema
"Index 'sqlite_autoindex_dogs_1' on table 'dogs' does not have a CREATE INDEX statement."
in str(excinfo.value)
)
with pytest.raises(sqlite3.IntegrityError):
memberships.insert(
{"organization_id": 1, "email": "ONE@example.com", "note": "y"}
)
def test_transform_preserves_column_unique_collation(fresh_db):
fresh_db.execute("""
CREATE TABLE people (
id INTEGER PRIMARY KEY,
name TEXT COLLATE NOCASE UNIQUE
)
""")
people = fresh_db.table("people")
people.insert({"id": 1, "name": "Cleo"})
people.transform(rename={"name": "full_name"})
assert 'UNIQUE ("full_name" COLLATE "NOCASE")' in people.schema
with pytest.raises(sqlite3.IntegrityError):
people.insert({"id": 2, "full_name": "cleo"})
def test_transform_drops_entire_composite_unique_constraint(fresh_db):
fresh_db.execute("""
CREATE TABLE memberships (
account_id INTEGER,
email TEXT,
UNIQUE (account_id, email)
)
""")
memberships = fresh_db.table("memberships")
memberships.insert({"account_id": 1, "email": "one@example.com"})
memberships.transform(drop={"email"})
assert "UNIQUE" not in memberships.schema
memberships.insert({"account_id": 1})
def test_transform_preserves_autoincrement_and_sequence(fresh_db):
fresh_db.execute(
"CREATE TABLE entries (id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT)"
assert (
"You must manually drop this index prior to running this transformation and manually recreate the new index after running this transformation."
in str(excinfo.value)
)
entries = fresh_db.table("entries")
entries.insert_all(({"value": "one"}, {"value": "two"}))
entries.delete(2)
entries.transform(rename={"value": "label"})
assert "PRIMARY KEY AUTOINCREMENT" in entries.schema
entries.insert({"label": "three"})
assert list(entries.rows) == [
{"id": 1, "label": "one"},
{"id": 3, "label": "three"},
]
@pytest.mark.parametrize(
"new_type,expected_value,expected_type",
[
(int, 42, int),
(float, 42.0, float),
("integer", 42, int),
("float", 42.0, float),
("REAL", 42.0, float),
],
)
def test_transform_empty_string_to_null_for_numeric_types(
fresh_db, new_type, expected_value, expected_type
):
fresh_db["test"].insert_all(
[
{"id": 1, "value": "42"},
{"id": 2, "value": ""},
{"id": 3, "value": None},
{"id": 4, "value": " "},
]
)
fresh_db["test"].transform(types={"value": new_type})
rows = {r["id"]: r["value"] for r in fresh_db["test"].rows}
assert rows[1] == expected_value
assert type(rows[1]) is expected_type
assert rows[2] is None
assert rows[3] is None
assert rows[4] == " "
def test_transform_preserves_view(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/831
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs")
view_sql_before = fresh_db.execute(
@ -1182,8 +958,8 @@ def test_transform_preserves_view(fresh_db):
def test_transform_variants_preserve_view(fresh_db, transform_params):
# Covers retyping, changing primary key and foreign key modifications,
# with a view whose columns are untouched by the transform
fresh_db.table("other").insert({"id": 1}, pk="id")
dogs = fresh_db.table("dogs")
fresh_db["other"].insert({"id": 1}, pk="id")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "other_id": 1}, pk="id")
if "drop_foreign_keys" in transform_params:
dogs.transform(add_foreign_keys=[("other_id", "other", "id")])
@ -1196,13 +972,13 @@ def test_transform_variants_preserve_view(fresh_db, transform_params):
"select sql from sqlite_master where name = 'dogs_view'"
).fetchone()[0]
assert view_sql_before == view_sql_after
assert list(fresh_db.view("dogs_view").rows) == [{"id": 1, "name": "Cleo"}]
assert list(fresh_db["dogs_view"].rows) == [{"id": 1, "name": "Cleo"}]
def test_transform_view_referencing_renamed_column(fresh_db):
# The view survives but querying it raises "no such column" - inherent
# to SQLite views, whose SQL is stored as text
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs")
dogs.transform(rename={"name": "title"})
@ -1211,7 +987,7 @@ def test_transform_view_referencing_renamed_column(fresh_db):
def test_transform_view_on_view(fresh_db):
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view v1 as select id, name from dogs")
fresh_db.execute("create view v2 as select name from v1")
@ -1223,13 +999,13 @@ def test_transform_view_on_view(fresh_db):
"select sql from sqlite_master where type = 'view' order by name"
).fetchall()
assert sqls_before == sqls_after
assert list(fresh_db.view("v2").rows) == [{"name": "Cleo"}]
assert list(fresh_db["v2"].rows) == [{"name": "Cleo"}]
def test_transform_keep_table_does_not_repoint_view(fresh_db):
# Without legacy_alter_table the ALTER TABLE dogs RENAME TO dogs_backup
# step would rewrite the view to select from "dogs_backup"
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs")
dogs.transform(types={"name": str}, keep_table="dogs_backup")
@ -1239,7 +1015,7 @@ def test_transform_keep_table_does_not_repoint_view(fresh_db):
assert "dogs_backup" not in view_sql
# View reads from the live table, not the frozen backup
dogs.insert({"id": 2, "name": "Pancakes"})
assert list(fresh_db.view("dogs_view").rows) == [
assert list(fresh_db["dogs_view"].rows) == [
{"id": 1, "name": "Cleo"},
{"id": 2, "name": "Pancakes"},
]
@ -1248,7 +1024,7 @@ def test_transform_keep_table_does_not_repoint_view(fresh_db):
def test_transform_sql_standalone_statements_work_with_view(fresh_db):
# The documented "run these statements yourself" workflow should be
# standalone-correct, so the pragmas must come from transform_sql()
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs")
sqls = dogs.transform_sql(types={"name": str}, tmp_suffix="suffix")
@ -1257,12 +1033,12 @@ def test_transform_sql_standalone_statements_work_with_view(fresh_db):
assert sqls[-1] == "PRAGMA legacy_alter_table=OFF;"
for sql in sqls:
fresh_db.execute(sql)
assert list(fresh_db.view("dogs_view").rows) == [{"id": 1, "name": "Cleo"}]
assert list(fresh_db["dogs_view"].rows) == [{"id": 1, "name": "Cleo"}]
def test_transform_with_view_in_open_transaction(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON")
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs")
with fresh_db.conn:
@ -1278,7 +1054,7 @@ def test_transform_with_view_in_open_transaction(fresh_db):
def test_transform_restores_legacy_alter_table_setting(fresh_db):
if sqlite3.sqlite_version_info < (3, 25, 0):
pytest.skip("legacy_alter_table pragma requires SQLite 3.25 or higher")
dogs = fresh_db.table("dogs")
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
# Default is OFF, reset to OFF afterwards
dogs.transform(types={"name": str})
@ -1299,7 +1075,7 @@ def test_transform_preserves_check_constraints(fresh_db):
CONSTRAINT nonzero_id CHECK(id != 0)
)
""")
scores = fresh_db.table("scores")
scores = fresh_db["scores"]
scores.insert({"id": 1, "score": 50})
scores.transform()
assert scores.checks == [
@ -1319,7 +1095,7 @@ def test_transform_preserves_check_ending_in_line_comment(fresh_db):
)
)
""")
inventory = fresh_db.table("inventory")
inventory = fresh_db["inventory"]
inventory.transform(types={"quantity": float})
assert inventory.checks == [Check("quantity >= 0 -- Quantity cannot be negative")]
with pytest.raises(sqlite3.IntegrityError, match="CHECK constraint failed"):
@ -1337,7 +1113,7 @@ def test_transform_preserves_comments_owned_by_columns(fresh_db):
age INTEGER -- May be NULL
)
""")
people = fresh_db.table("people")
people = fresh_db["people"]
people.insert({"id": 1, "name": "Cleo", "age": 5})
people.transform(
rename={"name": "display_name"},
@ -1367,8 +1143,8 @@ def test_transform_drops_comments_owned_by_dropped_column(fresh_db):
obsolete TEXT /* Drop this too */
)
""")
fresh_db.table("t").transform(drop={"obsolete"})
schema = fresh_db.table("t").schema
fresh_db["t"].transform(drop={"obsolete"})
schema = fresh_db["t"].schema
assert "Keep this explanation" in schema
assert "Drop this explanation" not in schema
assert "Drop this too" not in schema
@ -1383,7 +1159,7 @@ def test_transform_renames_columns_inside_check_constraints(fresh_db):
CONSTRAINT within_maximum CHECK(quantity <= maximum)
)
""")
inventory = fresh_db.table("inventory")
inventory = fresh_db["inventory"]
inventory.insert({"quantity": 2, "maximum": 3})
inventory.transform(rename={"quantity": "amount"})
assert inventory.checks == [
@ -1406,7 +1182,7 @@ def test_transform_check_rewrite_preserves_functions_and_quotes(fresh_db):
CHECK(length("old name") > 0 AND length != '')
)
""")
items = fresh_db.table("items")
items = fresh_db["items"]
items.insert({"length": "label", "old name": "hello"})
items.transform(rename={"length": "description", "old name": "new name"})
assert items.checks == [Check("length(\"new name\") > 0 AND description != ''")]
@ -1414,9 +1190,9 @@ def test_transform_check_rewrite_preserves_functions_and_quotes(fresh_db):
def test_transform_check_rewrite_quotes_keyword_column(fresh_db):
fresh_db.execute("CREATE TABLE t(old_name TEXT CHECK(old_name != ''))")
fresh_db.table("t").insert({"old_name": "value"})
fresh_db.table("t").transform(rename={"old_name": "select"})
assert fresh_db.table("t").checks == [Check("\"select\" != ''", column="select")]
fresh_db["t"].insert({"old_name": "value"})
fresh_db["t"].transform(rename={"old_name": "select"})
assert fresh_db["t"].checks == [Check("\"select\" != ''", column="select")]
def test_transform_check_rewrite_does_not_rename_collations_or_cast_types(fresh_db):
@ -1433,9 +1209,9 @@ def test_transform_check_rewrite_does_not_rename_collations_or_cast_types(fresh_
)
)
""")
fresh_db.table("t").insert({"nocase": "n", "kind": "k", "other": "o"})
fresh_db.table("t").transform(rename={"nocase": "label", "kind": "category"})
check = fresh_db.table("t").checks[0].check
fresh_db["t"].insert({"nocase": "n", "kind": "k", "other": "o"})
fresh_db["t"].transform(rename={"nocase": "label", "kind": "category"})
check = fresh_db["t"].checks[0].check
assert "COLLATE nocase" in check
assert "AS kind" in check
assert "AND label != ''" in check
@ -1450,9 +1226,9 @@ def test_transform_drops_check_owned_by_dropped_column(fresh_db):
CHECK(id > 0)
)
""")
fresh_db.table("t").insert({"id": 1, "obsolete": 2})
fresh_db.table("t").transform(drop={"obsolete"})
assert fresh_db.table("t").checks == [Check("id > 0")]
fresh_db["t"].insert({"id": 1, "obsolete": 2})
fresh_db["t"].transform(drop={"obsolete"})
assert fresh_db["t"].checks == [Check("id > 0")]
def test_transform_refuses_to_drop_column_used_by_remaining_check(fresh_db):
@ -1463,7 +1239,7 @@ def test_transform_refuses_to_drop_column_used_by_remaining_check(fresh_db):
CHECK(minimum <= maximum)
)
""")
ranges = fresh_db.table("ranges")
ranges = fresh_db["ranges"]
ranges.insert({"minimum": 1, "maximum": 2})
schema_before = ranges.schema
with pytest.raises(

View file

@ -7,14 +7,14 @@ from sqlite_utils.db import NotFoundError
def test_update_rowid_table(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
rowid = table.insert({"foo": "bar"}).last_pk
table.update(rowid, {"foo": "baz"})
assert [{"foo": "baz"}] == list(table.rows)
def test_update_pk_table(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
pk = table.insert({"foo": "bar", "id": 5}, pk="id").last_pk
assert 5 == pk
table.update(pk, {"foo": "baz"})
@ -22,7 +22,7 @@ def test_update_pk_table(fresh_db):
def test_update_compound_pk_table(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
pk = table.insert({"id1": 5, "id2": 3, "v": 1}, pk=("id1", "id2")).last_pk
assert (5, 3) == pk
table.update(pk, {"v": 2})
@ -42,14 +42,14 @@ def test_update_compound_pk_table(fresh_db):
),
)
def test_update_invalid_pk(fresh_db, pk, update_pk):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"id1": 5, "id2": 3, "v": 1}, pk=pk)
with pytest.raises(NotFoundError):
table.update(update_pk, {"v": 2})
def test_update_alter(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
rowid = table.insert({"foo": "bar"}).last_pk
table.update(rowid, {"new_col": 1.2}, alter=True)
assert [{"foo": "bar", "new_col": 1.2}] == list(table.rows)
@ -72,7 +72,7 @@ def test_update_alter(fresh_db):
def test_update_alter_with_special_column_characters(fresh_db):
# With double-quote escaping, columns with special characters are now valid
table = fresh_db.table("table")
table = fresh_db["table"]
rowid = table.insert({"foo": "bar"}).last_pk
table.update(rowid, {"new_col[abc]": 1.2}, alter=True)
assert list(table.rows) == [{"foo": "bar", "new_col[abc]": 1.2}]
@ -106,8 +106,8 @@ def test_update_with_no_values_sets_last_pk(fresh_db):
),
)
def test_update_dictionaries_and_lists_as_json(fresh_db, data_structure):
fresh_db.table("test").insert({"id": 1, "data": ""}, pk="id")
fresh_db.table("test").update(1, {"data": data_structure})
fresh_db["test"].insert({"id": 1, "data": ""}, pk="id")
fresh_db["test"].update(1, {"data": data_structure})
row = fresh_db.execute("select id, data from test").fetchone()
assert row[0] == 1
assert data_structure == json.loads(row[1])

View file

@ -7,15 +7,15 @@ from sqlite_utils.db import PrimaryKeyRequired
@pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert)
table = db.table("table")
table.insert_all([{"id": 1, "name": "Cleo"}], pk="id", replace=True)
table = db["table"]
table.insert({"id": 1, "name": "Cleo"}, pk="id")
table.upsert({"id": 1, "age": 5}, pk="id", alter=True)
assert list(table.rows) == [{"id": 1, "name": "Cleo", "age": 5}]
assert table.last_pk == 1
def test_upsert_all(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert_all([{"id": 1, "name": "Cleo"}, {"id": 2, "name": "Nixie"}], pk="id")
table.upsert_all([{"id": 1, "age": 5}, {"id": 2, "age": 5}], pk="id", alter=True)
assert list(table.rows) == [
@ -26,7 +26,7 @@ def test_upsert_all(fresh_db):
def test_upsert_all_single_column(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert_all([{"name": "Cleo"}], pk="name")
assert list(table.rows) == [{"name": "Cleo"}]
assert table.pks == ["name"]
@ -34,16 +34,16 @@ def test_upsert_all_single_column(fresh_db):
def test_upsert_all_not_null(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/538
fresh_db.table("comments").upsert_all(
fresh_db["comments"].upsert_all(
[{"id": 1, "name": "Cleo"}],
pk="id",
not_null=["name"],
)
assert list(fresh_db.table("comments").rows) == [{"id": 1, "name": "Cleo"}]
assert list(fresh_db["comments"].rows) == [{"id": 1, "name": "Cleo"}]
def test_upsert_error_if_no_pk(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
with pytest.raises(PrimaryKeyRequired):
table.upsert_all([{"id": 1, "name": "Cleo"}])
with pytest.raises(PrimaryKeyRequired):
@ -53,7 +53,7 @@ def test_upsert_error_if_no_pk(fresh_db):
@pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert_empty_record_errors(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert)
table = db.table("table")
table = db["table"]
table.insert({"id": 1, "name": "Cleo"}, pk="id")
with pytest.raises(PrimaryKeyRequired):
table.upsert({}, pk="id")
@ -66,7 +66,7 @@ def test_upsert_empty_record_errors(use_old_upsert):
@pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert_missing_pk_value_errors(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert)
table = db.table("table")
table = db["table"]
table.insert({"id": 1, "name": "Cleo"}, pk="id")
# Records that omit the pk column entirely
with pytest.raises(PrimaryKeyRequired):
@ -78,7 +78,7 @@ def test_upsert_missing_pk_value_errors(use_old_upsert):
def test_upsert_missing_compound_pk_value_errors(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.insert({"a": "x", "b": "y", "v": 1}, pk=("a", "b"))
# Missing one component of the detected compound primary key
with pytest.raises(PrimaryKeyRequired):
@ -105,7 +105,7 @@ def test_upsert_uses_compound_pk_from_existing_table(use_old_upsert):
primary key (Source, Object, Category)
)
""")
table = db.table("summary")
table = db["summary"]
table.upsert(
{
"Source": "Client A",
@ -134,7 +134,7 @@ def test_upsert_uses_compound_pk_from_existing_table(use_old_upsert):
def test_upsert_with_hash_id(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert({"foo": "bar"}, hash_id="pk")
assert [{"pk": "a5e744d0164540d33b1d7ea616c28f2fa97e754a", "foo": "bar"}] == list(
table.rows
@ -144,7 +144,7 @@ def test_upsert_with_hash_id(fresh_db):
@pytest.mark.parametrize("hash_id", (None, "custom_id"))
def test_upsert_with_hash_id_columns(fresh_db, hash_id):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert({"a": 1, "b": 2, "c": 3}, hash_id=hash_id, hash_id_columns=("a", "b"))
assert list(table.rows) == [
{
@ -167,7 +167,7 @@ def test_upsert_with_hash_id_columns(fresh_db, hash_id):
def test_upsert_compound_primary_key(fresh_db):
table = fresh_db.table("table")
table = fresh_db["table"]
table.upsert_all(
[
{"species": "dog", "id": 1, "name": "Cleo", "age": 4},

View file

@ -18,7 +18,7 @@ def test_enable_disable_wal(db_path_tmpdir):
assert "test.db-wal" not in [f.basename for f in tmpdir.listdir()]
db.enable_wal()
assert "wal" == db.journal_mode
db.table("test").insert({"foo": "bar"})
db["test"].insert({"foo": "bar"})
assert "test.db-wal" in [f.basename for f in tmpdir.listdir()]
db.disable_wal()
assert "delete" == db.journal_mode
@ -27,25 +27,25 @@ def test_enable_disable_wal(db_path_tmpdir):
def test_enable_wal_inside_transaction_raises(db_path_tmpdir):
db, _path, _tmpdir = db_path_tmpdir
db.table("test").insert({"id": 1}, pk="id")
db["test"].insert({"id": 1}, pk="id")
with pytest.raises(TransactionError), db.atomic():
db.table("test").insert({"id": 2}, pk="id")
db["test"].insert({"id": 2}, pk="id")
db.enable_wal()
# The atomic() block must have rolled back cleanly and the
# journal mode must be unchanged
assert db.journal_mode == "delete"
assert [r["id"] for r in db.table("test").rows] == [1]
assert [r["id"] for r in db["test"].rows] == [1]
def test_disable_wal_inside_transaction_raises(db_path_tmpdir):
db, _path, _tmpdir = db_path_tmpdir
db.enable_wal()
db.table("test").insert({"id": 1}, pk="id")
db["test"].insert({"id": 1}, pk="id")
with pytest.raises(TransactionError), db.atomic():
db.table("test").insert({"id": 2}, pk="id")
db["test"].insert({"id": 2}, pk="id")
db.disable_wal()
assert db.journal_mode == "wal"
assert [r["id"] for r in db.table("test").rows] == [1]
assert [r["id"] for r in db["test"].rows] == [1]
def test_ensure_autocommit_on(db_path_tmpdir):
@ -65,9 +65,9 @@ def test_enable_wal_noop_inside_transaction_is_allowed(db_path_tmpdir):
db, _path, _tmpdir = db_path_tmpdir
db.enable_wal()
with db.atomic():
db.table("test").insert({"id": 1}, pk="id")
db["test"].insert({"id": 1}, pk="id")
db.enable_wal()
assert [r["id"] for r in db.table("test").rows] == [1]
assert [r["id"] for r in db["test"].rows] == [1]
def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir):
@ -75,7 +75,7 @@ def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir):
# effect, silently breaking the caller's rollback guarantee - so
# entering autocommit mode with a transaction open is an error
db, _path, _tmpdir = db_path_tmpdir
db.table("test").insert({"id": 1}, pk="id")
db["test"].insert({"id": 1}, pk="id")
db.begin()
db.execute("insert into test (id) values (2)")
with pytest.raises(TransactionError), db.ensure_autocommit_on():
@ -83,4 +83,4 @@ def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir):
# The transaction is still open and can still be rolled back
assert db.conn.in_transaction
db.rollback()
assert [r["id"] for r in db.table("test").rows] == [1]
assert [r["id"] for r in db["test"].rows] == [1]