Compare commits

..

25 commits

Author SHA1 Message Date
Simon Willison
85b1be10c8 Same fix for test-coverage.yml
Refs https://github.com/simonw/sqlite-utils/pull/852/changes/ef50b31a21104351445c4856ff3a80b81b52e847
2026-09-01 21:44:52 -07:00
Simon Willison
f7e3174401 Fix for SpatialLite installation failure
Fable 5.1 explains:

> apt on the runner image has stale package lists. It tried to download libminizip1t64_1.3.dfsg-3.1ubuntu2.1 from security.ubuntu.com and got a 404, because Ubuntu has since published a newer build of that package and pulled the old .deb from the mirror.
2026-09-01 21:44:52 -07:00
Simon Willison
b97295271c Test against 3.15 RCs
See https://simonwillison.net/2026/Sep/1/python-315-rc-2/
2026-09-01 21:44:52 -07:00
Simon Willison
56dd09702f Run no-default-groups smoke test from Justfile
Refs #842

I had to add --isolated because otherwise this test would pass if a .venv
folder already existed with the dev dependencies installed in it.
2026-08-13 17:01:47 -07:00
Simon Willison
28dc6278cc Release 4.2.1
Refs #842, #843
2026-08-13 16:52:30 -07:00
Simon Willison
f6d73112c8
Fix for sqlite-utils 4.2 crashing bug (#843)
- Remove from typing_extensions import Self
- Smoke test: uv run --no-default-groups sqlite-utils --help

Closes #842
2026-08-13 16:52:03 -07:00
Simon Willison
1d98613f28 Release 4.2
Refs #488, #602, #762, #790, #805, #808, #811, #816, #821, #822, #824, #825, #828, #829, #831, #833, #834, #836, #837
2026-08-13 13:09:42 -07:00
ikatyal2110
e4935e0644
transform: coerce empty strings to NULL when converting TEXT columns to numeric types (#805)
* transform: coerce empty strings to NULL when converting TEXT columns to numeric types

When a TEXT column is transformed to INTEGER, FLOAT, or REAL and a row
contains an empty string, the empty string is now converted to NULL during
the INSERT...SELECT copy, matching the expected behavior described in #488.

Fixes #488
2026-08-13 12:56:51 -07:00
Simon Willison
75ba588462 Preserve composite UNIQUE constraints in transforms 2026-08-12 18:46:18 -07:00
Simon Willison
2b52b5ed6f Preserve AUTOINCREMENT through transforms 2026-08-12 18:39:00 -07:00
Simon Willison
fcfccea813 Support ANY column types for strict tables
Closes #790, #820
2026-08-12 16:44:01 -07:00
Simon Willison
57192ef4e3 table.transform(rename=...) now preserves indexes, closes #822 2026-08-12 14:42:24 -07:00
Simon Willison
e4784ec120 Changelog updates
Refs #808, #811, #816, #821, #824, #825, #828, #829, #833, #836, #837
2026-08-12 14:38:30 -07:00
Simon Willison
88b48fa167 Fixed introspection of default values TRUE / FALSE / NULL
Closes #836
2026-08-12 14:19:38 -07:00
nyxst4ck
e6be6267a4
Use quote_identifier() in indexes/xindexes PRAGMA statements (#825)
Closes #824
2026-08-12 14:15:17 -07:00
Simon Willison
c5063f67b1 Use quoted SQL identifiers in convert --dry-run, closes #829 2026-08-12 14:14:27 -07:00
Rami Abdelrazzaq
25c632fbbc
Handle empty input in rows_from_file
Closes #808
2026-08-12 14:05:09 -07:00
Simon Willison
ebb04a97de Fixes for Pyright, closes #833 2026-08-12 14:03:04 -07:00
Simon Willison
38fe466700 Use db.table() and db.view() in tests, closes #838 2026-08-12 14:03:04 -07:00
ethanhawkes-gif
43d5d3331f
Emit LIMIT -1 when offset is used without limit (#821)
* Emit LIMIT -1 when offset is used without limit, closes #816

SQLite requires a LIMIT clause to appear before OFFSET, so passing offset
without limit generated invalid SQL such as:

    select * from "t" offset 2

which raised OperationalError: near "2": syntax error.

A negative limit means "no upper bound" in SQLite, so "limit -1 offset N"
returns all rows from position N onwards.

Fixed in three places that build LIMIT/OFFSET SQL:

- Queryable.rows_where() - also covers pks_and_rows_where()
- Table.search_sql() - also covers search()
- the "sqlite-utils rows" CLI command

* Remove duplicate comments

---------

Co-authored-by: ethanhawkes-gif <259455325+ethanhawkes-gif@users.noreply.github.com>
2026-08-11 22:52:43 -07:00
Bunlong Heng
2d3c6b9a1e
Escape tokenize argument in enable_fts (#828)
The tokenize value passed to Table.enable_fts() was interpolated directly
into the CREATE VIRTUAL TABLE statement inside a single-quoted string
literal. A value containing a single quote could break out of that literal
and inject arbitrary SQL, which executes via executescript(). This is
reachable from the CLI via 'enable-fts --tokenize'.

Route the value through the existing Database.quote() helper so SQLite
itself escapes it. Legitimate tokenizers such as 'porter' are unaffected.
Adds a regression test.
2026-08-11 22:48:06 -07:00
Simon Willison
b37b8cf8c8 Preserve column before/after comments through .transform()
Refs #762

The before comment comes before the column definition - the after
comment is anything after it but before its trailing comma.
2026-08-11 22:45:00 -07:00
Simon Willison
b432e686ca Use sqlite_master not sqlite_schema for older SQLite compatibility 2026-08-11 22:45:00 -07:00
Simon Willison
2303b80aef .transform() preserves check constraints, refs #762 2026-08-11 22:45:00 -07:00
Simon Willison
3db0c57a3b table.checks, table.column_checks, table.table_checks, closes #834
Refs #762
2026-08-11 22:45:00 -07:00
59 changed files with 2680 additions and 1424 deletions

View file

@ -20,7 +20,7 @@ jobs:
cache: pip cache: pip
cache-dependency-path: pyproject.toml cache-dependency-path: pyproject.toml
- name: Install SpatiaLite - name: Install SpatiaLite
run: sudo apt-get install libsqlite3-mod-spatialite run: sudo apt-get update && sudo apt-get install -y libsqlite3-mod-spatialite
- name: Install Python dependencies - name: Install Python dependencies
run: | run: |
python -m pip install --upgrade pip python -m pip install --upgrade pip

View file

@ -10,18 +10,19 @@ jobs:
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
matrix: matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15-dev"] python-version: ["3.10", "3.11", "3.12", "3.13", "3.14", "3.15"]
numpy: [0, 1] numpy: [0, 1]
os: [ubuntu-latest, macos-latest, windows-latest, macos-14] os: [ubuntu-latest, macos-latest, windows-latest, macos-14]
steps: steps:
- uses: actions/checkout@v7 - uses: actions/checkout@v7
- name: Set up Python ${{ matrix.python-version }} - name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v6 uses: actions/setup-python@v7
with: with:
python-version: ${{ matrix.python-version }} python-version: ${{ matrix.python-version }}
allow-prereleases: true allow-prereleases: true
cache: pip cache: pip
cache-dependency-path: pyproject.toml cache-dependency-path: pyproject.toml
check-latest: true
- name: Install dependencies - name: Install dependencies
run: | run: |
pip install . --group dev pip install . --group dev
@ -30,7 +31,7 @@ jobs:
run: pip install numpy run: pip install numpy
- name: Install SpatiaLite - name: Install SpatiaLite
if: matrix.os == 'ubuntu-latest' if: matrix.os == 'ubuntu-latest'
run: sudo apt-get install libsqlite3-mod-spatialite run: sudo apt-get update && sudo apt-get install -y libsqlite3-mod-spatialite
- name: Build extension for --load-extension test - name: Build extension for --load-extension test
if: matrix.os == 'ubuntu-latest' if: matrix.os == 'ubuntu-latest'
run: |- run: |-
@ -43,6 +44,9 @@ jobs:
run: pytest --sqlite-autocommit run: pytest --sqlite-autocommit
- name: run mypy - name: run mypy
run: mypy sqlite_utils tests run: mypy sqlite_utils tests
- name: run pyright regression checks
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.14'
run: pyright sqlite_utils tests
- name: run flake8 - name: run flake8
run: flake8 run: flake8
- name: run ty - name: run ty
@ -50,6 +54,11 @@ jobs:
run: | run: |
pip install uv pip install uv
uv run ty check sqlite_utils uv run ty check sqlite_utils
- name: Check no accidental dev= dependencies needed
if: matrix.os == 'ubuntu-latest'
run: |
pip install uv
uv run --no-default-groups sqlite-utils --help
- name: Check formatting - name: Check formatting
run: black . --check run: black . --check
- name: Check if cog needs to be run - name: Check if cog needs to be run

View file

@ -2,17 +2,21 @@
@default: test lint @default: test lint
# Run pytest with supplied options # Run pytest with supplied options
@test *options: @test *options: test-no-dev-dependencies
uv run pytest {{options}} uv run pytest {{options}}
@test-no-dev-dependencies:
uv run --isolated --no-default-groups sqlite-utils --help > /dev/null
@run *options: @run *options:
uv run -- {{options}} uv run -- {{options}}
# Run linters: black, flake8, mypy, ty, cog # Run linters: black, flake8, mypy, pyright, ty, cog
@lint: @lint:
just run black . --check just run black . --check
uv run flake8 uv run flake8
uv run mypy sqlite_utils tests uv run mypy sqlite_utils tests
uv run pyright sqlite_utils tests
uv run ty check sqlite_utils uv run ty check sqlite_utils
uv run cog --check README.md docs/*.rst uv run cog --check README.md docs/*.rst
uv run --group docs codespell docs/*.rst --ignore-words docs/codespell-ignore-words.txt uv run --group docs codespell docs/*.rst --ignore-words docs/codespell-ignore-words.txt

View file

@ -4,16 +4,40 @@
Changelog Changelog
=========== ===========
.. _unreleased: .. _v4_2_1:
Unreleased 4.2.1 (2026-08-13)
---------- ------------------
- Fix for ``No module named 'typing_extensions'`` crashing bug accidentally shipped in version 4.2. (:issue:`842`)
.. _v4_2:
4.2 (2026-08-13)
----------------
- New ``table.checks``, ``table.column_checks`` and ``table.table_checks`` introspection properties expose column-level and table-level ``CHECK`` constraints. (:issue:`834`) - New ``table.checks``, ``table.column_checks`` and ``table.table_checks`` introspection properties expose column-level and table-level ``CHECK`` constraints. (:issue:`834`)
- New ``sqlite_utils.ANY`` marker type for creating and introspecting SQLite ``ANY`` columns. The Python API and CLI can create, add and transform these columns, and ``table.transform()`` and ``table.extract()`` now preserve ``ANY`` columns and their values in ``STRICT`` tables. (:issue:`790`)
- ``table.default_values`` now unescapes doubled single quotes in string defaults, so a default such as ``'O''Brien'`` is returned as ``"O'Brien"``. Thanks, `ikatyal2110 <https://github.com/ikatyal2110>`__. (`#811 <https://github.com/simonw/sqlite-utils/pull/811>`__)
- ``table.default_values`` now decodes unquoted ``TRUE``, ``FALSE`` and ``NULL`` default literals as ``True``, ``False`` and ``None`` respectively. (:issue:`836`)
- ``table.enable_fts(..., tokenize=...)`` and ``sqlite-utils enable-fts --tokenize`` now safely quote the tokenizer argument, preventing a crafted value from injecting additional SQL. Thanks, `Bunlong Heng <https://github.com/bunlongheng>`__. (`#828 <https://github.com/simonw/sqlite-utils/pull/828>`__)
- ``rows_where()``, ``pks_and_rows_where()``, ``search()`` and ``search_sql()`` now support ``offset=`` without requiring ``limit=``. The ``sqlite-utils rows --offset`` option now works without ``--limit`` too. Thanks, `ethanhawkes-gif <https://github.com/ethanhawkes-gif>`__. (:issue:`816`, `#821 <https://github.com/simonw/sqlite-utils/pull/821>`__)
- Empty or whitespace-only input passed to ``rows_from_file()`` is now handled as an empty CSV file instead of raising ``csv.Error``. Thanks, `Rami Abdelrazzaq <https://github.com/RamiNoodle733>`__. (:issue:`808`, `#837 <https://github.com/simonw/sqlite-utils/pull/837>`__)
- ``sqlite-utils convert --dry-run`` now works for table and column names containing closing square brackets. (:issue:`829`)
- ``table.indexes`` and ``table.xindexes`` now work for table, index and column names containing double quotes. This also fixes ``table.transform()`` for tables with those identifiers. Thanks, `nyxst4ck <https://github.com/nyxst4ck>`__. (:issue:`824`, `#825 <https://github.com/simonw/sqlite-utils/pull/825>`__)
- Improved type annotations throughout the package and added Pyright regression checks to CI. (:issue:`833`)
- Changing a ``TEXT`` column to ``INTEGER``, ``FLOAT`` or ``REAL`` using ``table.transform()`` or ``sqlite-utils transform`` now converts exact empty strings to ``NULL``. Previously they remained empty strings in the numeric column. Thanks, `ikatyal2110 <https://github.com/ikatyal2110>`__. (:issue:`488`, `#805 <https://github.com/simonw/sqlite-utils/pull/805>`__)
``table.transform()`` can handle many more edge-cases:
- ``table.transform()`` now preserves column-level and composite ``UNIQUE`` constraints, including constraint names, collations, sort order and ``ON CONFLICT`` behavior. Renaming columns updates those constraints, while dropping any constituent column removes the entire constraint. (:issue:`762`)
- ``table.transform()`` now preserves ``AUTOINCREMENT`` primary keys and their sequence high-water marks. Previously a transform removed ``AUTOINCREMENT`` and could reuse deleted row IDs. (:issue:`602`)
- ``table.transform()`` now preserves ``CHECK`` constraints, including comments within their expressions. Renaming a column rewrites identifier references in checks without changing string literals or function names. Dropping a column drops a check owned by that column, and raises ``TransformError`` if a remaining check depends on it. (:issue:`762`) - ``table.transform()`` now preserves ``CHECK`` constraints, including comments within their expressions. Renaming a column rewrites identifier references in checks without changing string literals or function names. Dropping a column drops a check owned by that column, and raises ``TransformError`` if a remaining check depends on it. (:issue:`762`)
- ``table.transform()`` now preserves comments immediately before or after column definitions. These comments move with the column if it is renamed or reordered, and are removed if the column is dropped. (:issue:`762`) - ``table.transform()`` now preserves comments immediately before or after column definitions. These comments move with the column if it is renamed or reordered, and are removed if the column is dropped. (:issue:`762`)
- ``table.transform(rename=...)`` now preserves explicit indexes on renamed columns by dropping and recreating those indexes against the new column names. Previously this raised a ``TransformError``. (:issue:`822`)
- ``table.transform()`` now works for tables that are referenced by views. Previously the ``ALTER TABLE ... RENAME TO`` step raised ``no such table`` if a view referenced the table being transformed. View definitions are left unchanged - see :ref:`python_api_transform_views`. This also fixes a bug where ``transform(keep_table=...)`` silently rewrote dependent views to point at the frozen backup table instead of the live one. (:issue:`831`) - ``table.transform()`` now works for tables that are referenced by views. Previously the ``ALTER TABLE ... RENAME TO`` step raised ``no such table`` if a view referenced the table being transformed. View definitions are left unchanged - see :ref:`python_api_transform_views`. This also fixes a bug where ``transform(keep_table=...)`` silently rewrote dependent views to point at the frozen backup table instead of the live one. (:issue:`831`)
.. _v3_39_1: .. _v3_39_1:
3.39.1 (2026-07-25) 3.39.1 (2026-07-25)

View file

@ -494,7 +494,7 @@ See :ref:`cli_transform_table`.
Options: Options:
--type <TEXT CHOICE>... Change column type to INTEGER, TEXT, FLOAT, --type <TEXT CHOICE>... Change column type to INTEGER, TEXT, FLOAT,
REAL or BLOB REAL, BLOB or ANY
--drop TEXT Drop this column --drop TEXT Drop this column
--rename <TEXT TEXT>... Rename this column to X --rename <TEXT TEXT>... Rename this column to X
-o, --column-order TEXT Reorder columns -o, --column-order TEXT Reorder columns
@ -963,7 +963,7 @@ See :ref:`cli_create_table`.
height real \ height real \
photo blob --pk id photo blob --pk id
Valid column types are text, integer, real, float and blob. Valid column types are text, integer, real, float, blob and any.
Options: Options:
--pk TEXT Column to use as primary key --pk TEXT Column to use as primary key
@ -1257,7 +1257,7 @@ See :ref:`cli_add_column`.
:: ::
Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME Usage: sqlite-utils add-column [OPTIONS] PATH TABLE COL_NAME
[integer|int|float|real|text|str|blob|bytes] [integer|int|float|real|text|str|blob|bytes|any]
Add a column to the specified table Add a column to the specified table

View file

@ -1390,7 +1390,14 @@ Use ``--type column-name type`` to override the type automatically chosen when t
This is useful for values such as ZIP codes, which may look like integers but should be stored as ``TEXT`` to preserve leading zeros. This is useful for values such as ZIP codes, which may look like integers but should be stored as ``TEXT`` to preserve leading zeros.
The column type should be one of ``TEXT``, ``INTEGER``, ``FLOAT``, ``REAL`` or ``BLOB``. Column types are matched case-insensitively. The column type should be one of ``TEXT``, ``INTEGER``, ``FLOAT``, ``REAL``, ``BLOB`` or ``ANY``. Column types are matched case-insensitively.
``ANY`` is especially useful with ``--strict``. An ``ANY`` column in a strict table preserves values without coercion, so text such as ``000123`` remains text instead of being converted to an integer:
.. code-block:: bash
sqlite-utils insert events.db events events.csv --csv --strict \
--type payload any
As with detected column types, ``--type`` only affects tables created by the command. If the table already exists, its existing column types are left unchanged. As with detected column types, ``--type`` only affects tables created by the command. If the table already exists, its existing column types are left unchanged.
@ -2141,6 +2148,12 @@ You can create a table in `SQLite STRICT mode <https://www.sqlite.org/stricttabl
sqlite-utils create-table mydb.db mytable id integer name text --strict sqlite-utils create-table mydb.db mytable id integer name text --strict
Use the ``any`` type for a strict column that should accept integers, floating point values, text, binary data or null without coercion:
.. code-block:: bash
sqlite-utils create-table events.db events id integer payload any --strict
.. code-block:: bash .. code-block:: bash
sqlite-utils tables mydb.db --schema -t sqlite-utils tables mydb.db --schema -t
@ -2223,7 +2236,7 @@ The ``transform`` command allows you to apply complex transformations to a table
Every option for this table (with the exception of ``--pk-none``) can be specified multiple times. The options are as follows: Every option for this table (with the exception of ``--pk-none``) can be specified multiple times. The options are as follows:
``--type column-name new-type`` ``--type column-name new-type``
Change the type of the specified column. Valid types are ``integer``, ``text``, ``float``, ``blob``. Change the type of the specified column. Valid types are ``integer``, ``text``, ``float``, ``real``, ``blob`` and ``any``. Changing a ``TEXT`` column to ``INTEGER``, ``FLOAT`` or ``REAL`` converts exact empty-string values to ``NULL``.
``--drop column-name`` ``--drop column-name``
Drop the specified column. Drop the specified column.

View file

@ -828,6 +828,19 @@ You can pass ``strict=True`` to create a table in ``STRICT`` mode:
"name": str, "name": str,
}, strict=True) }, strict=True)
SQLite ``STRICT`` tables can use the ``ANY`` column type for values that should retain their exact SQLite storage class without coercion. Use the ``sqlite_utils.ANY`` marker type:
.. code-block:: python
import sqlite_utils
db.table("events").create({
"id": int,
"payload": sqlite_utils.ANY,
}, pk="id", strict=True)
An ``ANY`` column can store integers, floating point values, text, binary data or ``None``. In a ``STRICT`` table a text value such as ``"000123"`` remains text with its leading zeroes intact. SQLite also accepts ``ANY`` columns in ordinary non-``STRICT`` tables, but those columns apply numeric affinity and would store that same value as the integer ``123``.
.. note:: .. note::
In the CLI: :ref:`sqlite-utils create-table <cli_create_table>` In the CLI: :ref:`sqlite-utils create-table <cli_create_table>`
@ -1569,7 +1582,7 @@ You can specify the ``col_type`` argument either using a SQLite type as a string
The ``col_type`` is optional - if you omit it the type of ``TEXT`` will be used. The ``col_type`` is optional - if you omit it the type of ``TEXT`` will be used.
SQLite types you can specify are ``"TEXT"``, ``"INTEGER"``, ``"FLOAT"``, ``"REAL"`` or ``"BLOB"``. SQLite types you can specify are ``"TEXT"``, ``"INTEGER"``, ``"FLOAT"``, ``"REAL"``, ``"BLOB"`` or ``"ANY"``. You can use the ``sqlite_utils.ANY`` marker instead of the ``"ANY"`` string.
If you pass a Python type, it will be mapped to SQLite types as shown here:: If you pass a Python type, it will be mapped to SQLite types as shown here::
@ -1582,6 +1595,7 @@ If you pass a Python type, it will be mapped to SQLite types as shown here::
datetime.date: "TEXT" datetime.date: "TEXT"
datetime.time: "TEXT" datetime.time: "TEXT"
datetime.timedelta: "TEXT" datetime.timedelta: "TEXT"
sqlite_utils.ANY: "ANY"
# If numpy is installed # If numpy is installed
np.int8: "INTEGER" np.int8: "INTEGER"
@ -1812,6 +1826,8 @@ To alter the type of a column, use the ``types=`` argument:
# Convert the 'age' column to an integer, and 'weight' to a float # Convert the 'age' column to an integer, and 'weight' to a float
table.transform(types={"age": int, "weight": float}) table.transform(types={"age": int, "weight": float})
When a ``TEXT`` column is changed to ``INTEGER``, ``FLOAT`` or ``REAL``, exact empty-string values are stored as ``NULL``. Other values, including whitespace-only strings, are copied normally.
See :ref:`python_api_add_column` for a list of available types. See :ref:`python_api_add_column` for a list of available types.
.. _python_api_transform_strict: .. _python_api_transform_strict:
@ -1831,6 +1847,8 @@ Pass ``strict=False`` to convert a strict table back to a regular non-strict tab
table.transform(strict=False) table.transform(strict=False)
If the table has ``ANY`` columns, converting it to non-strict mode can coerce text values that look numeric. For example, SQLite converts ``"000123"`` to the integer ``123`` when copying it into an ordinary ``ANY`` column. This is SQLite's documented distinction between `STRICT and ordinary ANY columns <https://www.sqlite.org/stricttables.html#the_any_datatype>`__.
The default is ``strict=None``, which preserves the table's existing strict mode. The default is ``strict=None``, which preserves the table's existing strict mode.
Passing ``strict=True`` raises ``sqlite_utils.db.TransformError`` if the available SQLite version does not support strict tables. Passing ``strict=True`` raises ``sqlite_utils.db.TransformError`` if the available SQLite version does not support strict tables.
@ -2458,6 +2476,11 @@ The ``.columns_dict`` property returns a dictionary version of the columns with
>>> db.table("PlantType").columns_dict >>> db.table("PlantType").columns_dict
{'id': <class 'int'>, 'value': <class 'str'>} {'id': <class 'int'>, 'value': <class 'str'>}
SQLite ``ANY`` columns are represented by the ``sqlite_utils.ANY`` marker type::
>>> db.table("events").columns_dict
{'id': <class 'int'>, 'payload': <class 'sqlite_utils.utils.ANY'>}
.. _python_api_introspection_default_values: .. _python_api_introspection_default_values:
.default_values .default_values

View file

@ -1,6 +1,6 @@
[project] [project]
name = "sqlite-utils" name = "sqlite-utils"
version = "4.1.1" version = "4.2.1"
description = "CLI tool and Python library for manipulating SQLite databases" description = "CLI tool and Python library for manipulating SQLite databases"
readme = { file = "README.md", content-type = "text/markdown" } readme = { file = "README.md", content-type = "text/markdown" }
authors = [ authors = [
@ -48,6 +48,7 @@ dev = [
# flake8 # flake8
"flake8", "flake8",
"flake8-pyproject", "flake8-pyproject",
"pyright>=1.1.411",
"ty>=0.0.37", "ty>=0.0.37",
# For stable cog: # For stable cog:
"tabulate>=0.10.0", "tabulate>=0.10.0",

View file

@ -1,6 +1,13 @@
from .db import Database from .db import Database
from .hookspecs import hookimpl, hookspec from .hookspecs import hookimpl, hookspec
from .migrations import Migrations from .migrations import Migrations
from .utils import suggest_column_types from .utils import ANY, suggest_column_types
__all__ = ["Database", "Migrations", "hookimpl", "hookspec", "suggest_column_types"] __all__ = [
"ANY",
"Database",
"Migrations",
"hookimpl",
"hookspec",
"suggest_column_types",
]

View file

@ -76,7 +76,7 @@ def _close_databases(ctx):
pass pass
VALID_COLUMN_TYPES = ("INTEGER", "TEXT", "FLOAT", "REAL", "BLOB") VALID_COLUMN_TYPES = ("INTEGER", "TEXT", "FLOAT", "REAL", "BLOB", "ANY")
UNICODE_ERROR = """ UNICODE_ERROR = """
{} {}
@ -489,7 +489,17 @@ def dump(path, load_extension):
@click.argument( @click.argument(
"col_type", "col_type",
type=click.Choice( type=click.Choice(
["integer", "int", "float", "real", "text", "str", "blob", "bytes"], [
"integer",
"int",
"float",
"real",
"text",
"str",
"blob",
"bytes",
"any",
],
case_sensitive=False, case_sensitive=False,
), ),
required=False, required=False,
@ -1093,7 +1103,7 @@ def insert_upsert_implementation(
column_type_overrides = {column: ctype.upper() for column, ctype in (types or [])} column_type_overrides = {column: ctype.upper() for column, ctype in (types or [])}
def _insert_docs(docs, tracker=None): def _insert_docs(docs, tracker=None):
extra_kwargs = { extra_kwargs: dict[str, Any] = {
"ignore": ignore, "ignore": ignore,
"replace": replace, "replace": replace,
"truncate": truncate, "truncate": truncate,
@ -1758,7 +1768,7 @@ def create_table(
height real \\ height real \\
photo blob --pk id photo blob --pk id
Valid column types are text, integer, real, float and blob. Valid column types are text, integer, real, float, blob and any.
""" """
db = sqlite_utils.Database(path) db = sqlite_utils.Database(path)
_register_db_for_cleanup(db) _register_db_for_cleanup(db)
@ -2478,6 +2488,8 @@ def rows(
if limit: if limit:
sql += f" limit {limit}" sql += f" limit {limit}"
if offset: if offset:
if not limit:
sql += " limit -1"
sql += f" offset {offset}" sql += f" offset {offset}"
ctx.invoke( ctx.invoke(
query, query,
@ -2666,12 +2678,10 @@ def schema(
"--type", "--type",
type=( type=(
str, str,
click.Choice( click.Choice(list(VALID_COLUMN_TYPES), case_sensitive=False),
["INTEGER", "TEXT", "FLOAT", "REAL", "BLOB"], case_sensitive=False
),
), ),
multiple=True, multiple=True,
help="Change column type to INTEGER, TEXT, FLOAT, REAL or BLOB", help="Change column type to INTEGER, TEXT, FLOAT, REAL, BLOB or ANY",
) )
@click.option("--drop", type=str, multiple=True, help="Drop this column") @click.option("--drop", type=str, multiple=True, help="Drop this column")
@click.option( @click.option(
@ -3273,25 +3283,20 @@ def convert(
raise click.ClickException(str(e)) raise click.ClickException(str(e))
if dry_run: if dry_run:
# Pull first 20 values for first column and preview them # Pull first 20 values for first column and preview them
def preview(v):
if multi: if multi:
def preview(v):
return json.dumps(fn(v), default=repr, ensure_ascii=False) if v else v return json.dumps(fn(v), default=repr, ensure_ascii=False) if v else v
else:
def preview(v):
return fn(v) if v else v return fn(v) if v else v
db.conn.create_function("preview_transform", 1, preview) db.conn.create_function("preview_transform", 1, preview)
sql = """ sql = """
select select
[{column}] as value, {column} as value,
preview_transform([{column}]) as preview preview_transform({column}) as preview
from [{table}]{where} limit 10 from {table}{where} limit 10
""".format( """.format(
column=columns[0], column=quote_identifier(columns[0]),
table=table, table=quote_identifier(table),
where=f" where {where}" if where is not None else "", where=f" where {where}" if where is not None else "",
) )
for row in db.conn.execute(sql, where_args).fetchall(): for row in db.conn.execute(sql, where_args).fetchall():
@ -3786,12 +3791,12 @@ def _rows_from_code(code):
code = pathlib.Path(code).read_text() code = pathlib.Path(code).read_text()
except FileNotFoundError: except FileNotFoundError:
raise click.ClickException(f"File not found: {code}") raise click.ClickException(f"File not found: {code}")
namespace = {} namespace: dict[str, Any] = {}
try: try:
exec(code, namespace) # noqa: S102 exec(code, namespace) # noqa: S102
except SyntaxError as ex: except SyntaxError as ex:
raise click.ClickException(f"Error in --code: {ex}") raise click.ClickException(f"Error in --code: {ex}")
rows = namespace.get("rows") rows: Any = namespace.get("rows")
if callable(rows): if callable(rows):
rows = rows() rows = rows()
if isinstance(rows, dict): if isinstance(rows, dict):

View file

@ -1,4 +1,4 @@
"""Helpers for parsing CHECK constraints from SQLite CREATE TABLE SQL. """Helpers for parsing constraints from SQLite CREATE TABLE SQL.
SQLite does not expose CHECK constraints through a pragma, so preserving them SQLite does not expose CHECK constraints through a pragma, so preserving them
across a table rebuild requires reading ``sqlite_schema.sql``. This module is across a table rebuild requires reading ``sqlite_schema.sql``. This module is
@ -32,6 +32,24 @@ class ColumnComments:
after: str = "" after: str = ""
@dataclass(frozen=True)
class UniqueColumn:
name: str
collation: str = ""
order: str = ""
@dataclass
class Unique:
columns: tuple[UniqueColumn, ...]
name: str = ""
column: str = ""
conflict: str = ""
sql: str = field(default="", compare=False, repr=False)
start: int = field(default=-1, compare=False, repr=False)
end: int = field(default=-1, compare=False, repr=False)
class ParseError(ValueError): class ParseError(ValueError):
pass pass
@ -564,6 +582,209 @@ def parse_checks(create_sql: str) -> list[Check]:
return checks return checks
def parse_autoincrement(create_sql: str) -> str | None:
"""Return the AUTOINCREMENT column from a valid CREATE TABLE statement."""
body_info = _table_body(create_sql)
if body_info is None:
return None
body, _ = body_info
for item, _, _ in _split_spans(body, _lex(body)):
item_tokens = _meaningful(_lex(item))
if not item_tokens:
continue
head = item_tokens[0]
if (
head.kind == "word" and head.text.upper() in _TABLE_CONSTRAINT_KEYWORDS
) or head.is_keyword("CONSTRAINT"):
continue
column = _unquote(head.text)
index = 1
while index < len(item_tokens):
token = item_tokens[index]
if token.text == "(":
index = _matching_paren(item_tokens, index) + 1
continue
if token.is_keyword("AUTOINCREMENT"):
return column
index += 1
return None
_CONFLICT_ACTIONS = frozenset(("ROLLBACK", "ABORT", "FAIL", "IGNORE", "REPLACE"))
def _conflict_after(tokens: list[_Token], index: int) -> tuple[str, int]:
if index >= len(tokens) or not tokens[index].is_keyword("ON"):
return "", index
if index + 2 >= len(tokens) or not tokens[index + 1].is_keyword("CONFLICT"):
raise ParseError("ON after UNIQUE must be followed by CONFLICT and an action")
action = tokens[index + 2].text.upper()
if tokens[index + 2].kind != "word" or action not in _CONFLICT_ACTIONS:
raise ParseError("Invalid UNIQUE ON CONFLICT action")
return action, index + 3
def _unique_columns(
item: str, tokens: list[_Token], open_index: int
) -> tuple[tuple[UniqueColumn, ...], int]:
close = _matching_paren(tokens, open_index)
inner = item[tokens[open_index].end : tokens[close].start]
columns: list[UniqueColumn] = []
for raw_column in _split_ranges(inner, _lex(inner)):
column_tokens = _meaningful(_lex(raw_column))
if not column_tokens or column_tokens[0].kind not in (
"word",
"identifier",
"string",
):
raise ParseError("UNIQUE constraint has an invalid column")
name = _unquote(column_tokens[0].text)
collation = ""
order = ""
index = 1
if index < len(column_tokens) and column_tokens[index].is_keyword("COLLATE"):
if index + 1 >= len(column_tokens):
raise ParseError("COLLATE in UNIQUE constraint is missing its name")
collation = _unquote(column_tokens[index + 1].text)
index += 2
if index < len(column_tokens) and (
column_tokens[index].is_keyword("ASC")
or column_tokens[index].is_keyword("DESC")
):
order = column_tokens[index].text.upper()
index += 1
if index != len(column_tokens):
raise ParseError("UNIQUE constraint has an invalid indexed column")
columns.append(UniqueColumn(name, collation=collation, order=order))
if not columns:
raise ParseError("UNIQUE constraint must include at least one column")
return tuple(columns), close + 1
def _column_uniques(
item: str, tokens: list[_Token], column: str, base_offset: int
) -> list[Unique]:
uniques: list[Unique] = []
collation = ""
collation_index = 1
while collation_index < len(tokens):
token = tokens[collation_index]
if token.text == "(":
collation_index = _matching_paren(tokens, collation_index) + 1
continue
if token.is_keyword("COLLATE"):
if collation_index + 1 >= len(tokens):
raise ParseError("COLLATE is missing its name")
collation = _unquote(tokens[collation_index + 1].text)
collation_index += 2
continue
collation_index += 1
pending_name = ""
pending_start: int | None = None
index = 1
while index < len(tokens):
token = tokens[index]
if token.text == "(":
index = _matching_paren(tokens, index) + 1
continue
if token.is_keyword("CONSTRAINT"):
if index + 1 >= len(tokens):
raise ParseError("CONSTRAINT is missing its name")
pending_name = _unquote(tokens[index + 1].text)
pending_start = index
index += 2
continue
if token.is_keyword("UNIQUE"):
source_start = tokens[
pending_start if pending_start is not None else index
].start
conflict, next_index = _conflict_after(tokens, index + 1)
source_end = tokens[next_index - 1].end
uniques.append(
Unique(
(UniqueColumn(column, collation=collation),),
name=pending_name,
column=column,
conflict=conflict,
sql=item[source_start:source_end],
start=base_offset + source_start,
end=base_offset + source_end,
)
)
pending_name = ""
pending_start = None
index = next_index
continue
if (
token.kind == "word"
and token.text.upper() in _OTHER_COLUMN_CONSTRAINT_KEYWORDS
):
pending_name = ""
pending_start = None
index += 1
return uniques
def parse_uniques(create_sql: str) -> list[Unique]:
"""Return column-level and table-level UNIQUE constraints."""
body_info = _table_body(create_sql)
if body_info is None:
return []
body, body_start = body_info
uniques: list[Unique] = []
for item, item_start, _ in _split_spans(body, _lex(body)):
item_tokens = _meaningful(_lex(item))
if not item_tokens:
continue
item_index = 0
constraint_name = ""
if item_tokens[item_index].is_keyword("CONSTRAINT"):
if len(item_tokens) < 2:
raise ParseError("CONSTRAINT is missing its name")
constraint_name = _unquote(item_tokens[1].text)
item_index = 2
head = item_tokens[item_index] if item_index < len(item_tokens) else None
if head and head.is_keyword("UNIQUE"):
if (
item_index + 1 >= len(item_tokens)
or item_tokens[item_index + 1].text != "("
):
raise ParseError("Table UNIQUE must be followed by a column list")
columns, next_index = _unique_columns(item, item_tokens, item_index + 1)
conflict, next_index = _conflict_after(item_tokens, next_index)
if next_index != len(item_tokens):
raise ParseError("Unexpected SQL after UNIQUE constraint")
source_start = item_tokens[0].start
source_end = item_tokens[next_index - 1].end
uniques.append(
Unique(
columns,
name=constraint_name,
conflict=conflict,
sql=item[source_start:source_end],
start=body_start + item_start + source_start,
end=body_start + item_start + source_end,
)
)
continue
if (
head
and head.kind == "word"
and head.text.upper() in _TABLE_CONSTRAINT_KEYWORDS
):
continue
column = _unquote(item_tokens[0].text)
uniques.extend(
_column_uniques(
item,
item_tokens,
column,
body_start + item_start,
)
)
return uniques
def parse_column_comments(create_sql: str) -> dict[str, ColumnComments]: def parse_column_comments(create_sql: str) -> dict[str, ColumnComments]:
"""Return comments immediately before and after each column definition.""" """Return comments immediately before and after each column definition."""
body_info = _table_body(create_sql) body_info = _table_body(create_sql)

View file

@ -18,26 +18,30 @@ from dataclasses import dataclass, field
from types import TracebackType from types import TracebackType
from typing import ( from typing import (
Any, Any,
TypeVar,
Union, Union,
cast, cast,
) )
from sqlite_fts4 import rank_bm25 from sqlite_fts4 import rank_bm25
from typing_extensions import Self
from sqlite_utils.plugins import ensure_plugins_loaded, pm from sqlite_utils.plugins import ensure_plugins_loaded, pm
from .create_table_parser import ( from .create_table_parser import (
Check, Check,
ColumnComments, ColumnComments,
ParseError, ParseError,
Unique,
UniqueColumn,
check_references_identifier, check_references_identifier,
parse_autoincrement,
parse_checks, parse_checks,
parse_column_comments, parse_column_comments,
parse_uniques,
rewrite_check_expression, rewrite_check_expression,
sql_ends_in_line_comment, sql_ends_in_line_comment,
) )
from .utils import ( from .utils import (
ANY,
OperationalError, OperationalError,
chunks, chunks,
column_affinity, column_affinity,
@ -101,6 +105,24 @@ def _check_constraint_sql(check: Check) -> str:
return f"{prefix}CHECK ({check.check}{newline})" return f"{prefix}CHECK ({check.check}{newline})"
def _unique_constraint_sql(unique: Unique) -> str:
prefix = f"CONSTRAINT {quote_identifier(unique.name)} " if unique.name else ""
if unique.column:
constraint = "UNIQUE"
else:
columns = []
for column in unique.columns:
column_sql = quote_identifier(column.name)
if column.collation:
column_sql += f" COLLATE {quote_identifier(column.collation)}"
if column.order:
column_sql += f" {column.order}"
columns.append(column_sql)
constraint = "UNIQUE ({})".format(", ".join(columns))
conflict = f" ON CONFLICT {unique.conflict}" if unique.conflict else ""
return f"{prefix}{constraint}{conflict}"
def _column_definition_with_comments( def _column_definition_with_comments(
definition: str, comments: ColumnComments | None definition: str, comments: ColumnComments | None
) -> str: ) -> str:
@ -256,8 +278,8 @@ class ForeignKey:
column: str | None = field(compare=False) column: str | None = field(compare=False)
other_table: str other_table: str
other_column: str | None = field(compare=False) other_column: str | None = field(compare=False)
columns: tuple[str, ...] = () columns: tuple[str, ...] | list[str] = ()
other_columns: tuple[str, ...] = () other_columns: tuple[str, ...] | list[str] = ()
is_compound: bool = False is_compound: bool = False
on_delete: str = "NO ACTION" on_delete: str = "NO ACTION"
on_update: str = "NO ACTION" on_update: str = "NO ACTION"
@ -320,6 +342,8 @@ ForeignKeyIndicator = (
ForeignKeysType = Iterable[ForeignKeyIndicator] | list[ForeignKeyIndicator] ForeignKeysType = Iterable[ForeignKeyIndicator] | list[ForeignKeyIndicator]
PrimaryKey = str | tuple[str, ...] | list[str]
class Default: class Default:
pass pass
@ -327,6 +351,8 @@ class Default:
DEFAULT = Default() DEFAULT = Default()
T = TypeVar("T")
Tracer = Callable[[str, Sequence[Any] | dict[str, Any] | None], None] Tracer = Callable[[str, Sequence[Any] | dict[str, Any] | None], None]
@ -361,6 +387,7 @@ COLUMN_TYPE_MAPPING: dict[Any, str] = {
decimal.Decimal: "REAL", decimal.Decimal: "REAL",
None.__class__: "TEXT", None.__class__: "TEXT",
uuid.UUID: "TEXT", uuid.UUID: "TEXT",
ANY: "ANY",
# SQLite explicit types # SQLite explicit types
"TEXT": "TEXT", "TEXT": "TEXT",
"INTEGER": "INTEGER", "INTEGER": "INTEGER",
@ -375,6 +402,8 @@ COLUMN_TYPE_MAPPING: dict[Any, str] = {
"real": "REAL", "real": "REAL",
"blob": "BLOB", "blob": "BLOB",
"bytes": "BLOB", "bytes": "BLOB",
"ANY": "ANY",
"any": "ANY",
} }
# If numpy is available, add more types # If numpy is available, add more types
if np: if np:
@ -606,7 +635,7 @@ class Database:
pm.hook.prepare_connection(conn=self.conn) pm.hook.prepare_connection(conn=self.conn)
self.strict = strict self.strict = strict
def __enter__(self) -> Self: def __enter__(self):
return self return self
def __exit__( def __exit__(
@ -1413,6 +1442,8 @@ class Database:
strict: bool = False, strict: bool = False,
_checks: Iterable[Check] | None = None, _checks: Iterable[Check] | None = None,
_column_comments: Mapping[str, ColumnComments] | None = None, _column_comments: Mapping[str, ColumnComments] | None = None,
_autoincrement: str | None = None,
_uniques: Iterable[Unique] | None = None,
) -> str: ) -> str:
""" """
Returns the SQL ``CREATE TABLE`` statement for creating the specified table. Returns the SQL ``CREATE TABLE`` statement for creating the specified table.
@ -1475,6 +1506,60 @@ class Database:
checks_by_column.setdefault(column, []).append(check) checks_by_column.setdefault(column, []).append(check)
else: else:
table_checks.append(check) table_checks.append(check)
uniques_by_column: dict[str, list[Unique]] = {}
table_uniques: list[Unique] = []
for unique in _uniques or ():
resolved_unique = Unique(
tuple(
UniqueColumn(
resolve_casing(column.name, columns),
collation=column.collation,
order=column.order,
)
for column in unique.columns
),
name=unique.name,
column=(
resolve_casing(unique.column, columns) if unique.column else ""
),
conflict=unique.conflict,
)
missing = [
column.name
for column in resolved_unique.columns
if column.name not in columns
]
if missing:
raise AlterError(
"No such column for UNIQUE constraint: {}".format(
", ".join(missing)
)
)
if resolved_unique.column:
if (
len(resolved_unique.columns) != 1
or resolved_unique.columns[0].name != resolved_unique.column
):
raise AlterError("Invalid column-level UNIQUE constraint")
if any(
column.collation or column.order
for column in resolved_unique.columns
):
# Render this as a table constraint so the collation or sort
# order that governs uniqueness can be represented explicitly.
table_uniques.append(
Unique(
resolved_unique.columns,
name=resolved_unique.name,
conflict=resolved_unique.conflict,
)
)
else:
uniques_by_column.setdefault(resolved_unique.column, []).append(
resolved_unique
)
else:
table_uniques.append(resolved_unique)
if not columns: if not columns:
raise ValueError("Tables must have at least one column") raise ValueError("Tables must have at least one column")
if not all(n in columns for n in not_null): if not all(n in columns for n in not_null):
@ -1516,10 +1601,22 @@ class Database:
column_items.insert(0, (pk, int)) column_items.insert(0, (pk, int))
elif pk: elif pk:
pk = [resolve_casing(p, [c[0] for c in column_items]) for p in pk] pk = [resolve_casing(p, [c[0] for c in column_items]) for p in pk]
if _autoincrement is not None:
_autoincrement = resolve_casing(
_autoincrement, [c[0] for c in column_items]
)
if _autoincrement != single_pk:
raise ValueError("AUTOINCREMENT requires a single-column primary key")
for column_name, column_type in column_items: for column_name, column_type in column_items:
column_extras = [] column_extras = []
if column_name == single_pk: if column_name == single_pk:
column_extras.append("PRIMARY KEY") column_extras.append("PRIMARY KEY")
if column_name == _autoincrement:
if COLUMN_TYPE_MAPPING[column_type] != "INTEGER":
raise ValueError(
"AUTOINCREMENT requires an INTEGER PRIMARY KEY column"
)
column_extras.append("AUTOINCREMENT")
if column_name in not_null: if column_name in not_null:
column_extras.append("NOT NULL") column_extras.append("NOT NULL")
if column_name in defaults and defaults[column_name] is not None: if column_name in defaults and defaults[column_name] is not None:
@ -1531,6 +1628,10 @@ class Database:
column_extras.append( column_extras.append(
f"REFERENCES {quote_identifier(fk.other_table)}({quote_identifier(cast(str, fk.other_column))}){_fk_actions_sql(fk)}" f"REFERENCES {quote_identifier(fk.other_table)}({quote_identifier(cast(str, fk.other_column))}){_fk_actions_sql(fk)}"
) )
column_extras.extend(
_unique_constraint_sql(unique)
for unique in uniques_by_column.get(column_name, ())
)
column_extras.extend( column_extras.extend(
_check_constraint_sql(check) _check_constraint_sql(check)
for check in checks_by_column.get(column_name, ()) for check in checks_by_column.get(column_name, ())
@ -1577,6 +1678,9 @@ class Database:
actions=_fk_actions_sql(fk), actions=_fk_actions_sql(fk),
) )
) )
column_defs.extend(
f" {_unique_constraint_sql(unique)}" for unique in table_uniques
)
column_defs.extend( column_defs.extend(
f" {_check_constraint_sql(check)}" for check in table_checks f" {_check_constraint_sql(check)}" for check in table_checks
) )
@ -1853,8 +1957,8 @@ class Database:
fk_object = self._resolve_foreign_key_casing( fk_object = self._resolve_foreign_key_casing(
fk_object, table_obj.columns_dict fk_object, table_obj.columns_dict
) )
columns = fk_object.columns columns = tuple(fk_object.columns)
other_columns = fk_object.other_columns other_columns = tuple(fk_object.other_columns)
for column in columns: for column in columns:
if column not in table_obj.columns_dict: if column not in table_obj.columns_dict:
raise AlterError(f"No such column: {column} in {table}") raise AlterError(f"No such column: {column} in {table}")
@ -1914,9 +2018,10 @@ class Database:
existing_indexes = {tuple(i.columns) for i in table.indexes} existing_indexes = {tuple(i.columns) for i in table.indexes}
for fk in table.foreign_keys: for fk in table.foreign_keys:
# A compound foreign key gets a single composite index # A compound foreign key gets a single composite index
if fk.columns not in existing_indexes: fk_columns = tuple(fk.columns)
if fk_columns not in existing_indexes:
table.create_index(fk.columns, find_unique_name=True) table.create_index(fk.columns, find_unique_name=True)
existing_indexes.add(fk.columns) existing_indexes.add(fk_columns)
def vacuum(self) -> None: def vacuum(self) -> None:
"Run a SQLite ``VACUUM`` against the database." "Run a SQLite ``VACUUM`` against the database."
@ -2064,6 +2169,10 @@ class Queryable:
if limit is not None: if limit is not None:
sql += f" limit {limit}" sql += f" limit {limit}"
if offset is not None: if offset is not None:
# SQLite requires a limit clause before offset - a negative limit
# means "no upper bound", so offset works without an explicit limit
if limit is None:
sql += " limit -1"
sql += f" offset {offset}" sql += f" offset {offset}"
cursor = self.db.execute(sql, where_args or []) cursor = self.db.execute(sql, where_args or [])
columns = dedupe_keys(c[0] for c in cursor.description) columns = dedupe_keys(c[0] for c in cursor.description)
@ -2365,14 +2474,11 @@ class Table(Queryable):
@property @property
def indexes(self) -> list[Index]: def indexes(self) -> list[Index]:
"List of indexes defined on this table." "List of indexes defined on this table."
sql = f'PRAGMA index_list("{self.name}")' sql = f"PRAGMA index_list({quote_identifier(self.name)})"
indexes = [] indexes = []
for row in self.db.execute_returning_dicts(sql): for row in self.db.execute_returning_dicts(sql):
index_name = row["name"] index_name = row["name"]
index_name_quoted = ( column_sql = f"PRAGMA index_info({quote_identifier(index_name)})"
f'"{index_name}"' if not index_name.startswith('"') else index_name
)
column_sql = f"PRAGMA index_info({index_name_quoted})"
columns = [] columns = []
for seqno, cid, name in self.db.execute(column_sql).fetchall(): for seqno, cid, name in self.db.execute(column_sql).fetchall():
columns.append(name) columns.append(name)
@ -2387,14 +2493,11 @@ class Table(Queryable):
@property @property
def xindexes(self) -> list[XIndex]: def xindexes(self) -> list[XIndex]:
"List of indexes defined on this table using the more detailed ``XIndex`` format." "List of indexes defined on this table using the more detailed ``XIndex`` format."
sql = f'PRAGMA index_list("{self.name}")' sql = f"PRAGMA index_list({quote_identifier(self.name)})"
indexes = [] indexes = []
for row in self.db.execute_returning_dicts(sql): for row in self.db.execute_returning_dicts(sql):
index_name = row["name"] index_name = row["name"]
index_name_quoted = ( column_sql = f"PRAGMA index_xinfo({quote_identifier(index_name)})"
f'"{index_name}"' if not index_name.startswith('"') else index_name
)
column_sql = f"PRAGMA index_xinfo({index_name_quoted})"
index_columns = [] index_columns = []
for info in self.db.execute(column_sql).fetchall(): for info in self.db.execute(column_sql).fetchall():
index_columns.append(XIndexColumn(*info)) index_columns.append(XIndexColumn(*info))
@ -2449,7 +2552,7 @@ class Table(Queryable):
replace: bool = False, replace: bool = False,
ignore: bool = False, ignore: bool = False,
transform: bool = False, transform: bool = False,
strict: bool | Default = DEFAULT, strict: bool | Default | None = DEFAULT,
) -> "Table": ) -> "Table":
""" """
Create a table with the specified columns. Create a table with the specified columns.
@ -2520,7 +2623,7 @@ class Table(Queryable):
replace=replace, replace=replace,
ignore=ignore, ignore=ignore,
transform=transform, transform=transform,
strict=strict, # type: ignore[arg-type] strict=cast(bool, strict),
) )
return self return self
@ -2740,6 +2843,8 @@ class Table(Queryable):
try: try:
existing_checks = self.checks existing_checks = self.checks
existing_column_comments = parse_column_comments(self.schema) existing_column_comments = parse_column_comments(self.schema)
existing_autoincrement = parse_autoincrement(self.schema)
existing_uniques = parse_uniques(self.schema)
except ParseError as ex: except ParseError as ex:
raise TransformError( raise TransformError(
f"Could not parse table schema for table {self.name!r}: {ex}" f"Could not parse table schema for table {self.name!r}: {ex}"
@ -2766,6 +2871,37 @@ class Table(Queryable):
) )
) )
create_table_uniques: list[Unique] = []
for unique in existing_uniques:
columns = tuple(
UniqueColumn(
resolve_casing(column.name, existing_columns),
collation=column.collation,
order=column.order,
)
for column in unique.columns
)
if any(column.name in drop for column in columns):
continue
owner = (
resolve_casing(unique.column, existing_columns) if unique.column else ""
)
create_table_uniques.append(
Unique(
tuple(
UniqueColumn(
rename.get(column.name) or column.name,
collation=column.collation,
order=column.order,
)
for column in columns
),
name=unique.name,
column=rename.get(owner) or owner,
conflict=unique.conflict,
)
)
create_table_column_comments: dict[str, ColumnComments] = {} create_table_column_comments: dict[str, ColumnComments] = {}
for column, comments in existing_column_comments.items(): for column, comments in existing_column_comments.items():
owner = resolve_casing(column, existing_columns) owner = resolve_casing(column, existing_columns)
@ -2856,12 +2992,17 @@ class Table(Queryable):
for name, type_ in current_column_pairs: for name, type_ in current_column_pairs:
type_ = types.get(name) or type_ type_ = types.get(name) or type_
if name in drop: if name in drop:
del [copy_from_to[name]] del copy_from_to[name]
continue continue
new_name = rename.get(name) or name new_name = rename.get(name) or name
new_column_pairs.append((new_name, type_)) new_column_pairs.append((new_name, type_))
copy_from_to[name] = new_name copy_from_to[name] = new_name
if existing_autoincrement:
existing_autoincrement = resolve_casing(
existing_autoincrement, existing_columns
)
if pk is DEFAULT: if pk is DEFAULT:
pks_renamed = tuple( pks_renamed = tuple(
rename.get(pk_name) or pk_name rename.get(pk_name) or pk_name
@ -2872,6 +3013,28 @@ class Table(Queryable):
else: else:
pk = pks_renamed pk = pks_renamed
create_table_autoincrement = None
if existing_autoincrement and existing_autoincrement not in drop:
renamed_autoincrement = (
rename.get(existing_autoincrement) or existing_autoincrement
)
single_pk = pk[0] if isinstance(pk, (list, tuple)) and len(pk) == 1 else pk
new_column_types = dict(new_column_pairs)
if (
single_pk == renamed_autoincrement
and COLUMN_TYPE_MAPPING.get(new_column_types.get(renamed_autoincrement))
== "INTEGER"
):
create_table_autoincrement = renamed_autoincrement
autoincrement_sequence = None
if create_table_autoincrement:
sequence_row = self.db.execute(
"SELECT seq FROM sqlite_sequence WHERE name = ?", [self.name]
).fetchone()
if sequence_row is not None:
autoincrement_sequence = sequence_row[0]
# not_null may be a set or dict, need to convert to a set # not_null may be a set or dict, need to convert to a set
create_table_not_null = { create_table_not_null = {
rename.get(c.name) or c.name rename.get(c.name) or c.name
@ -2923,9 +3086,24 @@ class Table(Queryable):
strict=self.strict if strict is None else strict, strict=self.strict if strict is None else strict,
_checks=create_table_checks, _checks=create_table_checks,
_column_comments=create_table_column_comments, _column_comments=create_table_column_comments,
_autoincrement=create_table_autoincrement,
_uniques=create_table_uniques,
).strip() ).strip()
) )
# Columns being changed from TEXT to a numeric type: coerce empty strings to NULL
_numeric_sql_types = {"INTEGER", "REAL", "FLOAT", "NUMERIC"}
text_to_numeric_cols = {
col_name
for col_name, new_type in types.items()
if existing_columns.get(col_name) == str
and COLUMN_TYPE_MAPPING.get(
new_type,
new_type.upper() if isinstance(new_type, str) else "",
)
in _numeric_sql_types
}
# Copy across data, respecting any renamed columns # Copy across data, respecting any renamed columns
new_cols = [] new_cols = []
old_cols = [] old_cols = []
@ -2936,13 +3114,96 @@ class Table(Queryable):
if "rowid" not in new_cols: if "rowid" not in new_cols:
new_cols.insert(0, "rowid") new_cols.insert(0, "rowid")
old_cols.insert(0, "rowid") old_cols.insert(0, "rowid")
def _copy_expr(col):
if col in text_to_numeric_cols:
return "NULLIF({}, '')".format(quote_identifier(col))
return quote_identifier(col)
copy_sql = "INSERT INTO {} ({new_cols})\n SELECT {old_cols} FROM {};".format( copy_sql = "INSERT INTO {} ({new_cols})\n SELECT {old_cols} FROM {};".format(
quote_identifier(new_table_name), quote_identifier(new_table_name),
quote_identifier(self.name), quote_identifier(self.name),
old_cols=", ".join(quote_identifier(col) for col in old_cols), old_cols=", ".join(_copy_expr(col) for col in old_cols),
new_cols=", ".join(quote_identifier(col) for col in new_cols), new_cols=", ".join(quote_identifier(col) for col in new_cols),
) )
sqls.append(copy_sql) sqls.append(copy_sql)
# Capture indexes before the old table is changed. Simple indexes that
# reference renamed columns are recreated from structured PRAGMA
# metadata instead of editing their stored CREATE INDEX SQL.
index_drop_sqls = []
index_create_sqls = []
xindexes_by_name = {index.name: index for index in self.xindexes}
for index in self.indexes:
if index.origin == "pk":
continue
index_sql = self.db.execute(
"""SELECT sql FROM sqlite_master WHERE type = 'index' AND name = :index_name;""",
{"index_name": index.name},
).fetchall()[0][0]
if index_sql is None:
if index.origin == "u":
# UNIQUE constraints are reproduced in CREATE TABLE above.
continue
raise TransformError(
f"Index '{index.name}' on table '{self.name}' does not have a "
"CREATE INDEX statement. You must manually drop this index prior to running this "
"transformation and manually recreate the new index after running this transformation."
)
dropped_index_column = next(
(column for column in index.columns if column in drop), None
)
renamed_index_column = next(
(column for column in index.columns if column in rename), None
)
if dropped_index_column is not None:
raise TransformError(
f"Index '{index.name}' column '{dropped_index_column}' is not in updated table '{self.name}'. "
f"You must manually drop this index prior to running this transformation "
f"and manually recreate the new index after running this transformation. "
f"The original index sql statement is: `{index_sql}`. No changes have been applied to this table."
)
xindex = xindexes_by_name[index.name]
indexed_columns = sorted(
(column for column in xindex.columns if column.key),
key=lambda column: column.seqno,
)
if (rename or drop) and (
index.partial or any(column.name is None for column in indexed_columns)
):
raise TransformError(
f"Index '{index.name}' is a partial or expression index, so it "
f"cannot be safely recreated while columns are renamed or dropped. "
f"You must manually drop this index prior to running this transformation "
f"and manually recreate the new index after running this transformation. "
f"The original index sql statement is: `{index_sql}`. No changes have been applied to this table."
)
if renamed_index_column is not None:
columns_sql = []
for column in indexed_columns:
assert column.name is not None
column_sql = quote_identifier(
rename.get(column.name) or column.name
)
if column.coll and column.coll.upper() != "BINARY":
column_sql += f" COLLATE {quote_identifier(column.coll)}"
if column.desc:
column_sql += " DESC"
columns_sql.append(column_sql)
index_sql = "CREATE {unique}INDEX {index_name} ON {table_name} ({columns})".format(
unique="UNIQUE " if index.unique else "",
index_name=quote_identifier(index.name),
table_name=quote_identifier(self.name),
columns=", ".join(columns_sql),
)
index_drop_sqls.append(
f"DROP INDEX IF EXISTS {quote_identifier(index.name)};"
)
elif keep_table:
index_drop_sqls.append(
f"DROP INDEX IF EXISTS {quote_identifier(index.name)};"
)
index_create_sqls.append(index_sql)
sqls.extend(index_drop_sqls)
# Drop (or keep) the old table, then rename the new one into place. # Drop (or keep) the old table, then rename the new one into place.
# Since SQLite 3.25 ALTER TABLE ... RENAME TO rewrites references to # Since SQLite 3.25 ALTER TABLE ... RENAME TO rewrites references to
# the renamed table in every view definition, which fails if a view # the renamed table in every view definition, which fails if a view
@ -2971,30 +3232,25 @@ class Table(Queryable):
"ON" if legacy_alter_table_was_on else "OFF" "ON" if legacy_alter_table_was_on else "OFF"
) )
) )
if autoincrement_sequence is not None:
table_name_literal = self.db.quote(self.name)
sqls.extend(
(
"UPDATE sqlite_sequence SET seq = MAX(seq, {sequence}) "
"WHERE name = {table_name};".format(
sequence=autoincrement_sequence,
table_name=table_name_literal,
),
"INSERT INTO sqlite_sequence (name, seq) "
"SELECT {table_name}, {sequence} WHERE NOT EXISTS "
"(SELECT 1 FROM sqlite_sequence WHERE name = {table_name});".format(
sequence=autoincrement_sequence,
table_name=table_name_literal,
),
)
)
# Re-add existing indexes # Re-add existing indexes
for index in self.indexes: sqls.extend(index_create_sqls)
if index.origin != "pk":
index_sql = self.db.execute(
"""SELECT sql FROM sqlite_master WHERE type = 'index' AND name = :index_name;""",
{"index_name": index.name},
).fetchall()[0][0]
if index_sql is None:
raise TransformError(
f"Index '{index.name}' on table '{self.name}' does not have a "
"CREATE INDEX statement. You must manually drop this index prior to running this "
"transformation and manually recreate the new index after running this transformation."
)
if keep_table:
sqls.append(f"DROP INDEX IF EXISTS {quote_identifier(index.name)};")
for col in index.columns:
if col in rename or col in drop:
raise TransformError(
f"Index '{index.name}' column '{col}' is not in updated table '{self.name}'. "
f"You must manually drop this index prior to running this transformation "
f"and manually recreate the new index after running this transformation. "
f"The original index sql statement is: `{index_sql}`. No changes have been applied to this table."
)
sqls.append(index_sql)
return sqls return sqls
def extract( def extract(
@ -3036,6 +3292,15 @@ class Table(Queryable):
if col in columns if col in columns
} }
if lookup_table.exists(): if lookup_table.exists():
if (
self.strict
and ANY in lookup_columns_definition.values()
and not lookup_table.strict
):
raise InvalidColumns(
f"Lookup table {table} already exists but is not STRICT, "
"so it cannot preserve ANY column values"
)
if not set(lookup_columns_definition.items()).issubset( if not set(lookup_columns_definition.items()).issubset(
lookup_table.columns_dict.items() lookup_table.columns_dict.items()
): ):
@ -3049,6 +3314,7 @@ class Table(Queryable):
**lookup_columns_definition, **lookup_columns_definition,
}, },
pk="id", pk="id",
strict=self.strict,
) )
lookup_columns = [(rename.get(col) or col) for col in columns] lookup_columns = [(rename.get(col) or col) for col in columns]
lookup_table.create_index(lookup_columns, unique=True, if_not_exists=True) lookup_table.create_index(lookup_columns, unique=True, if_not_exists=True)
@ -3339,7 +3605,10 @@ class Table(Queryable):
:param on_update: ``ON UPDATE`` action for the foreign key. :param on_update: ``ON UPDATE`` action for the foreign key.
""" """
columns = (column,) if isinstance(column, str) else tuple(column) columns = (column,) if isinstance(column, str) else tuple(column)
if not columns:
raise ValueError("column must contain at least one column name")
columns = tuple(resolve_casing(c, self.columns_dict) for c in columns) columns = tuple(resolve_casing(c, self.columns_dict) for c in columns)
assert columns
# Ensure columns exist # Ensure columns exist
for col in columns: for col in columns:
if col not in self.columns_dict: if col not in self.columns_dict:
@ -3350,7 +3619,7 @@ class Table(Queryable):
raise ValueError( raise ValueError(
"other_table must be specified for a compound foreign key" "other_table must be specified for a compound foreign key"
) )
other_table = self.guess_foreign_table(columns[0]) other_table = self.guess_foreign_table(next(iter(columns)))
# If other_column is not specified, detect the primary key on other_table # If other_column is not specified, detect the primary key on other_table
if other_column is None: if other_column is None:
if len(columns) > 1: if len(columns) > 1:
@ -3514,7 +3783,9 @@ class Table(Queryable):
table_fts=quote_identifier(self.name + "_fts"), table_fts=quote_identifier(self.name + "_fts"),
columns=", ".join(quote_identifier(c) for c in columns), columns=", ".join(quote_identifier(c) for c in columns),
fts_version=fts_version, fts_version=fts_version,
tokenize=f"\n tokenize='{tokenize}'," if tokenize else "", tokenize=(
f"\n tokenize={self.db.quote(tokenize)}," if tokenize else ""
),
) )
) )
should_recreate = False should_recreate = False
@ -3730,6 +4001,8 @@ class Table(Queryable):
if limit is not None: if limit is not None:
limit_offset += f" limit {limit}" limit_offset += f" limit {limit}"
if offset is not None: if offset is not None:
if limit is None:
limit_offset += " limit -1"
limit_offset += f" offset {offset}" limit_offset += f" offset {offset}"
return sql.format( return sql.format(
dbtable=quote_identifier(self.name), dbtable=quote_identifier(self.name),
@ -3793,8 +4066,10 @@ class Table(Queryable):
for row in cursor: for row in cursor:
yield dict(zip(columns, row)) yield dict(zip(columns, row))
def value_or_default(self, key: str, value: Any) -> Any: def value_or_default(self, key: str, value: T | Default) -> T:
return self._defaults[key] if value is DEFAULT else value if value is DEFAULT:
return cast(T, self._defaults[key])
return cast(T, value)
def delete(self, pk_values: list | tuple | str | float) -> "Table": def delete(self, pk_values: list | tuple | str | float) -> "Table":
""" """
@ -3979,7 +4254,7 @@ class Table(Queryable):
def _convert_multi( def _convert_multi(
self, column, fn, drop, show_progress, where=None, where_args=None self, column, fn, drop, show_progress, where=None, where_args=None
): ) -> "Table":
# First we execute the function # First we execute the function
pk_to_values = {} pk_to_values = {}
new_column_types: dict[str, set[type]] = {} new_column_types: dict[str, set[type]] = {}
@ -4025,6 +4300,7 @@ class Table(Queryable):
bar.update(1) bar.update(1)
if drop: if drop:
self.transform(drop=(column,)) self.transform(drop=(column,))
return self
def build_insert_queries_and_params( def build_insert_queries_and_params(
self, self,
@ -4328,8 +4604,8 @@ class Table(Queryable):
def insert( def insert(
self, self,
record: dict[str, Any], record: dict[str, Any],
pk=DEFAULT, pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys=DEFAULT, foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order: list[str] | Default | None = DEFAULT, column_order: list[str] | Default | None = DEFAULT,
not_null: Iterable[str] | Default | None = DEFAULT, not_null: Iterable[str] | Default | None = DEFAULT,
defaults: dict[str, Any] | Default | None = DEFAULT, defaults: dict[str, Any] | Default | None = DEFAULT,
@ -4397,24 +4673,24 @@ class Table(Queryable):
def insert_all( def insert_all(
self, self,
records: Iterable[dict[str, Any]] | Iterable[Sequence[Any]], records: Iterable[dict[str, Any]] | Iterable[Sequence[Any]],
pk=DEFAULT, pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys=DEFAULT, foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order=DEFAULT, column_order: list[str] | Default | None = DEFAULT,
not_null=DEFAULT, not_null: Iterable[str] | Default | None = DEFAULT,
defaults=DEFAULT, defaults: dict[str, Any] | Default | None = DEFAULT,
batch_size=DEFAULT, batch_size: int | Default = DEFAULT,
hash_id=DEFAULT, hash_id: str | Default | None = DEFAULT,
hash_id_columns=DEFAULT, hash_id_columns: Iterable[str] | Default | None = DEFAULT,
alter=DEFAULT, alter: bool | Default | None = DEFAULT,
ignore=DEFAULT, ignore: bool | Default | None = DEFAULT,
replace=DEFAULT, replace: bool | Default | None = DEFAULT,
truncate=False, truncate: bool = False,
extracts=DEFAULT, extracts: dict[str, str] | list[str] | Default | None = DEFAULT,
conversions=DEFAULT, conversions: dict[str, str] | Default | None = DEFAULT,
columns=DEFAULT, columns: dict[str, Any] | Default | None = DEFAULT,
upsert=False, upsert: bool = False,
analyze=False, analyze: bool = False,
strict=DEFAULT, strict: bool | Default | None = DEFAULT,
) -> "Table": ) -> "Table":
""" """
Like ``.insert()`` but takes a list of records and ensures that the table Like ``.insert()`` but takes a list of records and ensures that the table
@ -4707,6 +4983,7 @@ class Table(Queryable):
elif isinstance(pk, str): elif isinstance(pk, str):
self.last_pk = row[resolve_casing(pk, row)] self.last_pk = row[resolve_casing(pk, row)]
else: else:
assert pk is not None
self.last_pk = tuple( self.last_pk = tuple(
row[resolve_casing(p, row)] for p in pk row[resolve_casing(p, row)] for p in pk
) )
@ -4724,6 +5001,7 @@ class Table(Queryable):
pk_index = column_names.index(resolve_casing(pk, column_names)) pk_index = column_names.index(resolve_casing(pk, column_names))
self.last_pk = first_record_list[pk_index] self.last_pk = first_record_list[pk_index]
else: else:
assert pk is not None
self.last_pk = tuple( self.last_pk = tuple(
first_record_list[ first_record_list[
column_names.index(resolve_casing(p, column_names)) column_names.index(resolve_casing(p, column_names))
@ -4735,6 +5013,7 @@ class Table(Queryable):
if hash_id: if hash_id:
self.last_pk = hash_record(first_record_dict, hash_id_columns) self.last_pk = hash_record(first_record_dict, hash_id_columns)
else: else:
assert pk is not None
self.last_pk = ( self.last_pk = (
first_record_dict[resolve_casing(pk, first_record_dict)] first_record_dict[resolve_casing(pk, first_record_dict)]
if isinstance(pk, str) if isinstance(pk, str)
@ -4751,19 +5030,19 @@ class Table(Queryable):
def upsert( def upsert(
self, self,
record, record: dict[str, Any],
pk=DEFAULT, pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys=DEFAULT, foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order=DEFAULT, column_order: list[str] | Default | None = DEFAULT,
not_null=DEFAULT, not_null: Iterable[str] | Default | None = DEFAULT,
defaults=DEFAULT, defaults: dict[str, Any] | Default | None = DEFAULT,
hash_id=DEFAULT, hash_id: str | Default | None = DEFAULT,
hash_id_columns=DEFAULT, hash_id_columns: Iterable[str] | Default | None = DEFAULT,
alter=DEFAULT, alter: bool | Default | None = DEFAULT,
extracts=DEFAULT, extracts: dict[str, str] | list[str] | Default | None = DEFAULT,
conversions=DEFAULT, conversions: dict[str, str] | Default | None = DEFAULT,
columns=DEFAULT, columns: dict[str, Any] | Default | None = DEFAULT,
strict=DEFAULT, strict: bool | Default | None = DEFAULT,
) -> "Table": ) -> "Table":
""" """
Like ``.insert()`` but performs an ``UPSERT``, where records are inserted if they do Like ``.insert()`` but performs an ``UPSERT``, where records are inserted if they do
@ -4790,20 +5069,20 @@ class Table(Queryable):
def upsert_all( def upsert_all(
self, self,
records: Iterable[dict[str, Any]] | Iterable[Sequence[Any]], records: Iterable[dict[str, Any]] | Iterable[Sequence[Any]],
pk=DEFAULT, pk: PrimaryKey | Default | None = DEFAULT,
foreign_keys=DEFAULT, foreign_keys: ForeignKeysType | Default | None = DEFAULT,
column_order=DEFAULT, column_order: list[str] | Default | None = DEFAULT,
not_null=DEFAULT, not_null: Iterable[str] | Default | None = DEFAULT,
defaults=DEFAULT, defaults: dict[str, Any] | Default | None = DEFAULT,
batch_size=DEFAULT, batch_size: int | Default = DEFAULT,
hash_id=DEFAULT, hash_id: str | Default | None = DEFAULT,
hash_id_columns=DEFAULT, hash_id_columns: Iterable[str] | Default | None = DEFAULT,
alter=DEFAULT, alter: bool | Default | None = DEFAULT,
extracts=DEFAULT, extracts: dict[str, str] | list[str] | Default | None = DEFAULT,
conversions=DEFAULT, conversions: dict[str, str] | Default | None = DEFAULT,
columns=DEFAULT, columns: dict[str, Any] | Default | None = DEFAULT,
analyze=False, analyze: bool = False,
strict=DEFAULT, strict: bool | Default | None = DEFAULT,
) -> "Table": ) -> "Table":
""" """
Like ``.upsert()`` but can be applied to a list of records. Like ``.upsert()`` but can be applied to a list of records.
@ -5253,6 +5532,13 @@ def _decode_default_value(value: str) -> object:
# It's a binary string, stored as hex # It's a binary string, stored as hex
to_decode = value[2:-1] to_decode = value[2:-1]
return binascii.unhexlify(to_decode) return binascii.unhexlify(to_decode)
upper = value.upper()
if upper == "TRUE":
return True
if upper == "FALSE":
return False
if upper == "NULL":
return None
# If it is a string containing a floating point number: # If it is a string containing a floating point number:
try: try:
return float(value) return float(value)

View file

@ -14,6 +14,7 @@ from typing import (
TYPE_CHECKING, TYPE_CHECKING,
Any, Any,
BinaryIO, BinaryIO,
Generic,
TypeVar, TypeVar,
Union, Union,
cast, cast,
@ -58,6 +59,10 @@ Row = dict[str, RowValue]
T = TypeVar("T") T = TypeVar("T")
class ANY:
"""Marker type for an SQLite ``ANY`` column."""
class _CloseableIterator(Iterator[Row]): class _CloseableIterator(Iterator[Row]):
"""Iterator wrapper that closes a file when iteration is complete.""" """Iterator wrapper that closes a file when iteration is complete."""
@ -177,6 +182,8 @@ def column_affinity(column_type: str) -> type:
return bytes return bytes
if "REAL" in column_type or "FLOA" in column_type or "DOUB" in column_type: if "REAL" in column_type or "FLOA" in column_type or "DOUB" in column_type:
return float return float
if column_type == "ANY":
return ANY
# Default is 'NUMERIC', which we currently also treat as float # Default is 'NUMERIC', which we currently also treat as float
return float return float
@ -344,7 +351,11 @@ def rows_from_file(
reader = csv.DictReader(decoded_fp, dialect=dialect) reader = csv.DictReader(decoded_fp, dialect=dialect)
else: else:
reader = csv.DictReader(decoded_fp) reader = csv.DictReader(decoded_fp)
rows = _extra_key_strategy(reader, ignore_extras, extras_key) rows = _extra_key_strategy(
cast(Iterable[dict[str | None, object]], reader),
ignore_extras,
extras_key,
)
return _CloseableIterator(iter(rows), decoded_fp), Format.CSV return _CloseableIterator(iter(rows), decoded_fp), Format.CSV
elif format == Format.TSV: elif format == Format.TSV:
rows, _ = rows_from_file( rows, _ = rows_from_file(
@ -368,6 +379,8 @@ def rows_from_file(
raise TypeError( raise TypeError(
"rows_from_file() requires a file-like object that supports peek(), such as io.BytesIO" "rows_from_file() requires a file-like object that supports peek(), such as io.BytesIO"
) )
if not first_bytes:
return (), Format.CSV
if first_bytes.startswith((b"[", b"{")): if first_bytes.startswith((b"[", b"{")):
# TODO: Detect newline-JSON # TODO: Detect newline-JSON
return rows_from_file(buffered, format=Format.JSON) return rows_from_file(buffered, format=Format.JSON)
@ -487,12 +500,12 @@ class ValueTracker:
del self.couldbe[key] del self.couldbe[key]
class NullProgressBar: class NullProgressBar(Generic[T]):
def __init__(self, *args: Iterable[T]) -> None: def __init__(self, *args: Iterable[T]) -> None:
self.args = args self.args = args
def __iter__(self) -> Iterator[T]: def __iter__(self) -> Iterator[T]:
yield from self.args[0] # type: ignore yield from self.args[0]
def update(self, value: int) -> None: def update(self, value: int) -> None:
pass pass

View file

@ -3,11 +3,13 @@ import pytest
@pytest.fixture @pytest.fixture
def db(fresh_db): def db(fresh_db):
fresh_db["one_index"].insert({"id": 1, "name": "Cleo"}, pk="id") fresh_db.table("one_index").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db["one_index"].create_index(["name"]) fresh_db.table("one_index").create_index(["name"])
fresh_db["two_indexes"].insert({"id": 1, "name": "Cleo", "species": "dog"}, pk="id") fresh_db.table("two_indexes").insert(
fresh_db["two_indexes"].create_index(["name"]) {"id": 1, "name": "Cleo", "species": "dog"}, pk="id"
fresh_db["two_indexes"].create_index(["species"]) )
fresh_db.table("two_indexes").create_index(["name"])
fresh_db.table("two_indexes").create_index(["species"])
return fresh_db return fresh_db
@ -17,7 +19,7 @@ def test_analyze_whole_database(db):
assert set(db.table_names()).issuperset( assert set(db.table_names()).issuperset(
{"one_index", "two_indexes", "sqlite_stat1"} {"one_index", "two_indexes", "sqlite_stat1"}
) )
assert list(db["sqlite_stat1"].rows) == [ assert list(db.table("sqlite_stat1").rows) == [
{"tbl": "two_indexes", "idx": "idx_two_indexes_species", "stat": "1 1"}, {"tbl": "two_indexes", "idx": "idx_two_indexes_species", "stat": "1 1"},
{"tbl": "two_indexes", "idx": "idx_two_indexes_name", "stat": "1 1"}, {"tbl": "two_indexes", "idx": "idx_two_indexes_name", "stat": "1 1"},
{"tbl": "one_index", "idx": "idx_one_index_name", "stat": "1 1"}, {"tbl": "one_index", "idx": "idx_one_index_name", "stat": "1 1"},
@ -30,12 +32,12 @@ def test_analyze_one_table(db, method):
if method == "db_method_with_name": if method == "db_method_with_name":
db.analyze("one_index") db.analyze("one_index")
elif method == "table_method": elif method == "table_method":
db["one_index"].analyze() db.table("one_index").analyze()
assert set(db.table_names()).issuperset( assert set(db.table_names()).issuperset(
{"one_index", "two_indexes", "sqlite_stat1"} {"one_index", "two_indexes", "sqlite_stat1"}
) )
assert list(db["sqlite_stat1"].rows) == [ assert list(db.table("sqlite_stat1").rows) == [
{"tbl": "one_index", "idx": "idx_one_index_name", "stat": "1 1"} {"tbl": "one_index", "idx": "idx_one_index_name", "stat": "1 1"}
] ]
@ -46,6 +48,6 @@ def test_analyze_index_by_name(db):
assert set(db.table_names()).issuperset( assert set(db.table_names()).issuperset(
{"one_index", "two_indexes", "sqlite_stat1"} {"one_index", "two_indexes", "sqlite_stat1"}
) )
assert list(db["sqlite_stat1"].rows) == [ assert list(db.table("sqlite_stat1").rows) == [
{"tbl": "two_indexes", "idx": "idx_two_indexes_species", "stat": "1 1"}, {"tbl": "two_indexes", "idx": "idx_two_indexes_species", "stat": "1 1"},
] ]

View file

@ -9,7 +9,7 @@ from sqlite_utils.db import ColumnDetails, Database
@pytest.fixture @pytest.fixture
def db_to_analyze(fresh_db): def db_to_analyze(fresh_db):
stuff = fresh_db["stuff"] stuff = fresh_db.table("stuff")
stuff.insert_all( stuff.insert_all(
[ [
{"id": 1, "owner": "Terryterryterry", "size": 5}, {"id": 1, "owner": "Terryterryterry", "size": 5},
@ -45,7 +45,7 @@ def big_db_to_analyze_path(tmpdir):
"all_null": None, "all_null": None,
} }
) )
db["stuff"].insert_all(to_insert) db.table("stuff").insert_all(to_insert)
return path return path
@ -126,7 +126,7 @@ def big_db_to_analyze_path(tmpdir):
) )
def test_analyze_column(db_to_analyze, column, extra_kwargs, expected): def test_analyze_column(db_to_analyze, column, extra_kwargs, expected):
assert ( assert (
db_to_analyze["stuff"].analyze_column( db_to_analyze.table("stuff").analyze_column(
column, common_limit=2, value_truncate=5, **extra_kwargs column, common_limit=2, value_truncate=5, **extra_kwargs
) )
== expected == expected
@ -186,7 +186,7 @@ def test_analyze_table_save(db_to_analyze_path):
cli.cli, ["analyze-tables", db_to_analyze_path, "--save"] cli.cli, ["analyze-tables", db_to_analyze_path, "--save"]
) )
assert result.exit_code == 0 assert result.exit_code == 0
rows = list(Database(db_to_analyze_path)["_analyze_tables_"].rows) rows = list(Database(db_to_analyze_path).table("_analyze_tables_").rows)
assert rows == [ assert rows == [
{ {
"table": "stuff", "table": "stuff",
@ -248,7 +248,7 @@ def test_analyze_table_save_no_most_no_least_options(
args.append("--no-least") args.append("--no-least")
result = CliRunner().invoke(cli.cli, args) result = CliRunner().invoke(cli.cli, args)
assert result.exit_code == 0 assert result.exit_code == 0
rows = list(Database(big_db_to_analyze_path)["_analyze_tables_"].rows) rows = list(Database(big_db_to_analyze_path).table("_analyze_tables_").rows)
expected = { expected = {
"table": "stuff", "table": "stuff",
"column": "category", "column": "category",
@ -297,13 +297,13 @@ def test_analyze_table_column_all_nulls(big_db_to_analyze_path):
def test_analyze_table_validate_columns(tmpdir, args, expected_error): def test_analyze_table_validate_columns(tmpdir, args, expected_error):
path = str(tmpdir / "test_validate_columns.db") path = str(tmpdir / "test_validate_columns.db")
db = Database(path) db = Database(path)
db["one"].insert( db.table("one").insert(
{ {
"id": 1, "id": 1,
"name": "one", "name": "one",
} }
) )
db["two"].insert( db.table("two").insert(
{ {
"id": 1, "id": 1,
"age": 5, "age": 5,

View file

@ -45,30 +45,30 @@ def test_iter_complete_sql_statements(sql, expected):
def test_atomic_commits(fresh_db): def test_atomic_commits(fresh_db):
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id") fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
assert list(fresh_db["dogs"].rows) == [{"id": 1, "name": "Cleo"}] assert list(fresh_db.table("dogs").rows) == [{"id": 1, "name": "Cleo"}]
def test_atomic_rolls_back(fresh_db): def test_atomic_rolls_back(fresh_db):
with pytest.raises(RuntimeError), fresh_db.atomic(): with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id") fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
raise RuntimeError("boom") raise RuntimeError("boom")
assert not fresh_db["dogs"].exists() assert not fresh_db.table("dogs").exists()
def test_nested_atomic_rolls_back_to_savepoint(fresh_db): def test_nested_atomic_rolls_back_to_savepoint(fresh_db):
fresh_db["dogs"].create({"id": int, "name": str}, pk="id") fresh_db.table("dogs").create({"id": int, "name": str}, pk="id")
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}) fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"})
with pytest.raises(RuntimeError), fresh_db.atomic(): with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db["dogs"].insert({"id": 2, "name": "Pancakes"}) fresh_db.table("dogs").insert({"id": 2, "name": "Pancakes"})
raise RuntimeError("boom") raise RuntimeError("boom")
fresh_db["dogs"].insert({"id": 3, "name": "Marnie"}) fresh_db.table("dogs").insert({"id": 3, "name": "Marnie"})
assert list(fresh_db["dogs"].rows) == [ assert list(fresh_db.table("dogs").rows) == [
{"id": 1, "name": "Cleo"}, {"id": 1, "name": "Cleo"},
{"id": 3, "name": "Marnie"}, {"id": 3, "name": "Marnie"},
] ]
@ -76,12 +76,12 @@ def test_nested_atomic_rolls_back_to_savepoint(fresh_db):
def test_outer_atomic_rolls_back_released_savepoint(fresh_db): def test_outer_atomic_rolls_back_released_savepoint(fresh_db):
with pytest.raises(RuntimeError), fresh_db.atomic(): with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id") fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["dogs"].insert({"id": 2, "name": "Pancakes"}) fresh_db.table("dogs").insert({"id": 2, "name": "Pancakes"})
raise RuntimeError("boom") raise RuntimeError("boom")
assert not fresh_db["dogs"].exists() assert not fresh_db.table("dogs").exists()
def test_executescript_does_not_commit_open_atomic_block(fresh_db): def test_executescript_does_not_commit_open_atomic_block(fresh_db):
@ -97,41 +97,41 @@ def test_executescript_does_not_commit_open_atomic_block(fresh_db):
""") """)
raise RuntimeError("boom") raise RuntimeError("boom")
assert not fresh_db["dogs"].exists() assert not fresh_db.table("dogs").exists()
def test_transform_does_not_commit_open_atomic_block(fresh_db): def test_transform_does_not_commit_open_atomic_block(fresh_db):
fresh_db["dogs"].insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id") fresh_db.table("dogs").insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
with pytest.raises(RuntimeError), fresh_db.atomic(): with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db["dogs"].insert({"id": 2, "name": "Pancakes", "age": "6"}) fresh_db.table("dogs").insert({"id": 2, "name": "Pancakes", "age": "6"})
fresh_db["dogs"].transform(rename={"age": "dog_age"}) fresh_db.table("dogs").transform(rename={"age": "dog_age"})
raise RuntimeError("boom") raise RuntimeError("boom")
assert ( assert (
fresh_db["dogs"].schema fresh_db.table("dogs").schema
== 'CREATE TABLE "dogs" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "age" TEXT\n)' == 'CREATE TABLE "dogs" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "age" TEXT\n)'
) )
assert list(fresh_db["dogs"].rows) == [ assert list(fresh_db.table("dogs").rows) == [
{"id": 1, "name": "Cleo", "age": "5"}, {"id": 1, "name": "Cleo", "age": "5"},
] ]
def test_transform_parent_table_with_foreign_keys_in_atomic(fresh_db): def test_transform_parent_table_with_foreign_keys_in_atomic(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db["authors"].insert({"id": 1, "name": "Tina"}, pk="id") fresh_db.table("authors").insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db["books"].insert( fresh_db.table("books").insert(
{"id": 1, "title": "Book", "author_id": 1}, {"id": 1, "title": "Book", "author_id": 1},
pk="id", pk="id",
foreign_keys={"author_id"}, foreign_keys={"author_id"},
) )
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["authors"].transform(rename={"name": "full_name"}) fresh_db.table("authors").transform(rename={"name": "full_name"})
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0] assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
assert ( assert (
fresh_db["authors"].schema fresh_db.table("authors").schema
== 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "full_name" TEXT\n)' == 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "full_name" TEXT\n)'
) )
assert fresh_db.execute("PRAGMA foreign_key_check").fetchall() == [] assert fresh_db.execute("PRAGMA foreign_key_check").fetchall() == []
@ -139,19 +139,19 @@ def test_transform_parent_table_with_foreign_keys_in_atomic(fresh_db):
def test_transform_parent_table_with_foreign_keys_rolls_back(fresh_db): def test_transform_parent_table_with_foreign_keys_rolls_back(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db["authors"].insert({"id": 1, "name": "Tina"}, pk="id") fresh_db.table("authors").insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db["books"].insert( fresh_db.table("books").insert(
{"id": 1, "title": "Book", "author_id": 1}, {"id": 1, "title": "Book", "author_id": 1},
pk="id", pk="id",
foreign_keys={"author_id"}, foreign_keys={"author_id"},
) )
with pytest.raises(RuntimeError), fresh_db.atomic(): with pytest.raises(RuntimeError), fresh_db.atomic():
fresh_db["authors"].transform(rename={"name": "full_name"}) fresh_db.table("authors").transform(rename={"name": "full_name"})
raise RuntimeError("boom") raise RuntimeError("boom")
assert ( assert (
fresh_db["authors"].schema fresh_db.table("authors").schema
== 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT\n)' == 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT\n)'
) )
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0] assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -160,49 +160,51 @@ def test_transform_parent_table_with_foreign_keys_rolls_back(fresh_db):
def test_transform_detects_foreign_key_check_violations(fresh_db): def test_transform_detects_foreign_key_check_violations(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db["authors"].insert({"id": 1, "name": "Tina"}, pk="id") fresh_db.table("authors").insert({"id": 1, "name": "Tina"}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 2}, pk="id") fresh_db.table("books").insert({"id": 1, "author_id": 2}, pk="id")
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
fresh_db["books"].transform(add_foreign_keys=(("author_id", "authors", "id"),)) fresh_db.table("books").transform(
add_foreign_keys=(("author_id", "authors", "id"),)
)
assert fresh_db["books"].foreign_keys == [] assert fresh_db.table("books").foreign_keys == []
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0] assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
def test_atomic_inside_manual_transaction_uses_savepoint(fresh_db): def test_atomic_inside_manual_transaction_uses_savepoint(fresh_db):
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db.execute("begin") fresh_db.execute("begin")
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["t"].insert({"id": 2}, pk="id") fresh_db.table("t").insert({"id": 2}, pk="id")
# Nothing is committed until the user's own transaction commits # Nothing is committed until the user's own transaction commits
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.rollback() fresh_db.rollback()
assert [r["id"] for r in fresh_db["t"].rows] == [1] assert [r["id"] for r in fresh_db.table("t").rows] == [1]
# And with a commit instead, the atomic block's writes persist # And with a commit instead, the atomic block's writes persist
fresh_db.execute("begin") fresh_db.execute("begin")
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["t"].insert({"id": 3}, pk="id") fresh_db.table("t").insert({"id": 3}, pk="id")
fresh_db.commit() fresh_db.commit()
assert [r["id"] for r in fresh_db["t"].rows] == [1, 3] assert [r["id"] for r in fresh_db.table("t").rows] == [1, 3]
def test_begin_commit_rollback(tmpdir): def test_begin_commit_rollback(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["t"].insert({"id": 1}, pk="id") db.table("t").insert({"id": 1}, pk="id")
db.begin() db.begin()
db["t"].insert({"id": 2}, pk="id") db.table("t").insert({"id": 2}, pk="id")
assert db.conn.in_transaction assert db.conn.in_transaction
db.rollback() db.rollback()
assert not db.conn.in_transaction assert not db.conn.in_transaction
assert [r["id"] for r in db["t"].rows] == [1] assert [r["id"] for r in db.table("t").rows] == [1]
db.begin() db.begin()
db["t"].insert({"id": 3}, pk="id") db.table("t").insert({"id": 3}, pk="id")
db.commit() db.commit()
db.close() db.close()
db2 = Database(path) db2 = Database(path)
assert [r["id"] for r in db2["t"].rows] == [1, 3] assert [r["id"] for r in db2.table("t").rows] == [1, 3]
db2.close() db2.close()
@ -222,7 +224,7 @@ def test_commit_and_rollback_without_transaction_are_noops(fresh_db):
def test_execute_write_commits_immediately(tmpdir): def test_execute_write_commits_immediately(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["t"].insert({"id": 1}, pk="id") db.table("t").insert({"id": 1}, pk="id")
db.execute("insert into t (id) values (2)") db.execute("insert into t (id) values (2)")
# No implicit transaction is left open # No implicit transaction is left open
assert not db.conn.in_transaction assert not db.conn.in_transaction
@ -234,24 +236,24 @@ def test_execute_write_commits_immediately(tmpdir):
def test_execute_write_respects_explicit_transaction(fresh_db): def test_execute_write_respects_explicit_transaction(fresh_db):
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db.begin() fresh_db.begin()
fresh_db.execute("insert into t (id) values (2)") fresh_db.execute("insert into t (id) values (2)")
# Still inside the explicit transaction - not committed # Still inside the explicit transaction - not committed
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.rollback() fresh_db.rollback()
assert [r["id"] for r in fresh_db["t"].rows] == [1] assert [r["id"] for r in fresh_db.table("t").rows] == [1]
def test_execute_comment_prefixed_begin_leaves_transaction_open(fresh_db): def test_execute_comment_prefixed_begin_leaves_transaction_open(fresh_db):
# A BEGIN hidden behind a leading comment must not be auto-committed # A BEGIN hidden behind a leading comment must not be auto-committed
# out from under the caller # out from under the caller
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db.execute("-- start a transaction\nbegin") fresh_db.execute("-- start a transaction\nbegin")
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.execute("insert into t (id) values (2)") fresh_db.execute("insert into t (id) values (2)")
fresh_db.rollback() fresh_db.rollback()
assert [r["id"] for r in fresh_db["t"].rows] == [1] assert [r["id"] for r in fresh_db.table("t").rows] == [1]
def _sqlite_accepts_bom(): def _sqlite_accepts_bom():
@ -269,12 +271,12 @@ def test_execute_prefixed_begin_leaves_transaction_open(fresh_db, begin_sql):
# out from under the caller # out from under the caller
if begin_sql.startswith("\ufeff") and not _sqlite_accepts_bom(): if begin_sql.startswith("\ufeff") and not _sqlite_accepts_bom():
pytest.skip("This SQLite version rejects a leading byte order mark") pytest.skip("This SQLite version rejects a leading byte order mark")
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db.execute(begin_sql) fresh_db.execute(begin_sql)
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.execute("insert into t (id) values (2)") fresh_db.execute("insert into t (id) values (2)")
fresh_db.rollback() fresh_db.rollback()
assert [r["id"] for r in fresh_db["t"].rows] == [1] assert [r["id"] for r in fresh_db.table("t").rows] == [1]
def test_execute_failed_write_rolls_back_implicit_transaction(tmpdir): def test_execute_failed_write_rolls_back_implicit_transaction(tmpdir):
@ -282,40 +284,40 @@ def test_execute_failed_write_rolls_back_implicit_transaction(tmpdir):
# that would silently disable auto-commit for every subsequent write # that would silently disable auto-commit for every subsequent write
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["t"].insert({"id": 1}, pk="id") db.table("t").insert({"id": 1}, pk="id")
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
db.execute("insert into t (id) values (1)") db.execute("insert into t (id) values (1)")
assert not db.conn.in_transaction assert not db.conn.in_transaction
# Subsequent writes commit as normal and survive closing the connection # Subsequent writes commit as normal and survive closing the connection
db["other"].insert({"id": 2}) db.table("other").insert({"id": 2})
db.close() db.close()
db2 = Database(path) db2 = Database(path)
assert db2["other"].exists() assert db2.table("other").exists()
db2.close() db2.close()
def test_execute_failed_write_preserves_explicit_transaction(fresh_db): def test_execute_failed_write_preserves_explicit_transaction(fresh_db):
# A failed write inside an explicit transaction must not roll back # A failed write inside an explicit transaction must not roll back
# the caller's earlier work - only the caller decides that # the caller's earlier work - only the caller decides that
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
fresh_db.begin() fresh_db.begin()
fresh_db.execute("insert into t (id) values (2)") fresh_db.execute("insert into t (id) values (2)")
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
fresh_db.execute("insert into t (id) values (1)") fresh_db.execute("insert into t (id) values (1)")
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.commit() fresh_db.commit()
assert [r["id"] for r in fresh_db["t"].rows] == [1, 2] assert [r["id"] for r in fresh_db.table("t").rows] == [1, 2]
def test_execute_failed_write_inside_atomic_preserves_block(fresh_db): def test_execute_failed_write_inside_atomic_preserves_block(fresh_db):
# A caught failure inside an atomic() block must leave the block's # A caught failure inside an atomic() block must leave the block's
# transaction open so its other work still commits # transaction open so its other work still commits
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db.execute("insert into t (id) values (2)") fresh_db.execute("insert into t (id) values (2)")
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
fresh_db.execute("insert into t (id) values (1)") fresh_db.execute("insert into t (id) values (1)")
assert [r["id"] for r in fresh_db["t"].rows] == [1, 2] assert [r["id"] for r in fresh_db.table("t").rows] == [1, 2]
def test_query_returning_commits_after_iteration(tmpdir): def test_query_returning_commits_after_iteration(tmpdir):
@ -325,7 +327,7 @@ def test_query_returning_commits_after_iteration(tmpdir):
_pytest.skip("RETURNING requires SQLite 3.35.0 or higher") _pytest.skip("RETURNING requires SQLite 3.35.0 or higher")
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["t"].insert({"id": 1}, pk="id") db.table("t").insert({"id": 1}, pk="id")
rows = list(db.query("insert into t (id) values (2) returning id")) rows = list(db.query("insert into t (id) values (2) returning id"))
assert rows == [{"id": 2}] assert rows == [{"id": 2}]
assert not db.conn.in_transaction assert not db.conn.in_transaction
@ -375,7 +377,7 @@ def test_nested_atomic_preserves_error_from_transaction_destroying_trigger(
def test_atomic_preserves_error_from_insert_or_rollback(fresh_db): def test_atomic_preserves_error_from_insert_or_rollback(fresh_db):
fresh_db["t"].insert({"id": 1}, pk="id") fresh_db.table("t").insert({"id": 1}, pk="id")
with pytest.raises(sqlite3.IntegrityError), fresh_db.atomic(): with pytest.raises(sqlite3.IntegrityError), fresh_db.atomic():
fresh_db.execute("insert or rollback into t (id) values (1)") fresh_db.execute("insert or rollback into t (id) values (1)")
assert not fresh_db.conn.in_transaction assert not fresh_db.conn.in_transaction

View file

@ -6,10 +6,10 @@ def test_attach(tmpdir):
bar_path = str(tmpdir / "bar.db") bar_path = str(tmpdir / "bar.db")
db = Database(foo_path) db = Database(foo_path)
with db.conn: with db.conn:
db["foo"].insert({"id": 1, "text": "foo"}) db.table("foo").insert({"id": 1, "text": "foo"})
db2 = Database(bar_path) db2 = Database(bar_path)
with db2.conn: with db2.conn:
db2["bar"].insert({"id": 1, "text": "bar"}) db2.table("bar").insert({"id": 1, "text": "bar"})
db.attach("bar", bar_path) db.attach("bar", bar_path)
assert db.execute( assert db.execute(
"select * from foo union all select * from bar.bar" "select * from foo union all select * from bar.bar"

File diff suppressed because it is too large Load diff

View file

@ -13,7 +13,7 @@ from sqlite_utils import Database, cli
def test_db_and_path(tmpdir): def test_db_and_path(tmpdir):
db_path = str(pathlib.Path(tmpdir) / "data.db") db_path = str(pathlib.Path(tmpdir) / "data.db")
db = Database(db_path) db = Database(db_path)
db["example"].insert_all( db.table("example").insert_all(
[ [
{"id": 1, "name": "One"}, {"id": 1, "name": "One"},
{"id": 2, "name": "Two"}, {"id": 2, "name": "Two"},
@ -44,7 +44,7 @@ def test_cli_bulk(test_db_and_path):
{"id": 2, "name": "Two"}, {"id": 2, "name": "Two"},
{"id": 3, "name": "THREE"}, {"id": 3, "name": "THREE"},
{"id": 4, "name": "FOUR"}, {"id": 4, "name": "FOUR"},
] == list(db["example"].rows) ] == list(db.table("example").rows)
def test_cli_bulk_multiple_functions(test_db_and_path): def test_cli_bulk_multiple_functions(test_db_and_path):
@ -70,7 +70,7 @@ def test_cli_bulk_multiple_functions(test_db_and_path):
{"id": 2, "name": "Two"}, {"id": 2, "name": "Two"},
{"id": 3, "name": "THREE"}, {"id": 3, "name": "THREE"},
{"id": 4, "name": "FOUR"}, {"id": 4, "name": "FOUR"},
] == list(db["example"].rows) ] == list(db.table("example").rows)
def test_cli_bulk_batch_size(test_db_and_path): def test_cli_bulk_batch_size(test_db_and_path):
@ -91,17 +91,18 @@ def test_cli_bulk_batch_size(test_db_and_path):
stdin=subprocess.PIPE, stdin=subprocess.PIPE,
stdout=sys.stdout, stdout=sys.stdout,
) )
assert proc.stdin is not None
# Writing one record should not commit # Writing one record should not commit
proc.stdin.write(b'{"id": 3, "name": "Three"}\n\n') proc.stdin.write(b'{"id": 3, "name": "Three"}\n\n')
proc.stdin.flush() proc.stdin.flush()
time.sleep(1) time.sleep(1)
assert db["example"].count == 2 assert db.table("example").count == 2
# Writing another should trigger a commit: # Writing another should trigger a commit:
proc.stdin.write(b'{"id": 4, "name": "Four"}\n\n') proc.stdin.write(b'{"id": 4, "name": "Four"}\n\n')
proc.stdin.flush() proc.stdin.flush()
time.sleep(1) time.sleep(1)
assert db["example"].count == 4 assert db.table("example").count == 4
proc.stdin.close() proc.stdin.close()
proc.wait() proc.wait()

View file

@ -12,7 +12,7 @@ from sqlite_utils import cli
@pytest.fixture @pytest.fixture
def test_db_and_path(fresh_db_and_path): def test_db_and_path(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["example"].insert_all( db.table("example").insert_all(
[ [
{"id": 1, "dt": "5th October 2019 12:04"}, {"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6th October 2019 00:05:06"}, {"id": 2, "dt": "6th October 2019 00:05:06"},
@ -47,12 +47,12 @@ def fresh_db_and_path(tmpdir):
) )
def test_convert_code(fresh_db_and_path, code): def test_convert_code(fresh_db_and_path, code):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["t"].insert({"text": "October"}) db.table("t").insert({"text": "October"})
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, ["convert", db_path, "t", "text", code], catch_exceptions=False cli.cli, ["convert", db_path, "t", "text", code], catch_exceptions=False
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
value = next(iter(db["t"].rows))["text"] value = next(iter(db.table("t").rows))["text"]
assert value == "Spooktober" assert value == "Spooktober"
@ -65,7 +65,7 @@ def test_convert_code(fresh_db_and_path, code):
) )
def test_convert_code_errors(fresh_db_and_path, bad_code): def test_convert_code_errors(fresh_db_and_path, bad_code):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["t"].insert({"text": "October"}) db.table("t").insert({"text": "October"})
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, ["convert", db_path, "t", "text", bad_code], catch_exceptions=False cli.cli, ["convert", db_path, "t", "text", bad_code], catch_exceptions=False
) )
@ -93,12 +93,12 @@ def test_convert_import(test_db_and_path):
{"id": 2, "dt": "6th OXXober 2019 00:05:06"}, {"id": 2, "dt": "6th OXXober 2019 00:05:06"},
{"id": 3, "dt": ""}, {"id": 3, "dt": ""},
{"id": 4, "dt": None}, {"id": 4, "dt": None},
] == list(db["example"].rows) ] == list(db.table("example").rows)
def test_convert_import_nested(fresh_db_and_path): def test_convert_import_nested(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["example"].insert({"xml": '<item name="Cleo" />'}) db.table("example").insert({"xml": '<item name="Cleo" />'})
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
[ [
@ -114,7 +114,7 @@ def test_convert_import_nested(fresh_db_and_path):
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert [ assert [
{"xml": "Cleo"}, {"xml": "Cleo"},
] == list(db["example"].rows) ] == list(db.table("example").rows)
def test_convert_dryrun(test_db_and_path): def test_convert_dryrun(test_db_and_path):
@ -152,7 +152,7 @@ def test_convert_dryrun(test_db_and_path):
"Would affect 4 rows" "Would affect 4 rows"
) )
# But it should not have actually modified the table data # But it should not have actually modified the table data
assert list(db["example"].rows) == [ assert list(db.table("example").rows) == [
{"id": 1, "dt": "5th October 2019 12:04"}, {"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6th October 2019 00:05:06"}, {"id": 2, "dt": "6th October 2019 00:05:06"},
{"id": 3, "dt": ""}, {"id": 3, "dt": ""},
@ -181,6 +181,34 @@ def test_convert_dryrun(test_db_and_path):
assert result.output.strip().split("\n")[-1] == "Would affect 1 row" assert result.output.strip().split("\n")[-1] == "Would affect 1 row"
def test_convert_dryrun_table_and_column_names_containing_closing_bracket(
fresh_db_and_path,
):
db, db_path = fresh_db_and_path
table_name = "table]name"
column_name = "column]name"
db[table_name].insert({column_name: "hello"})
result = CliRunner().invoke(
cli.cli,
[
"convert",
db_path,
table_name,
column_name,
"value.upper()",
"--dry-run",
],
catch_exceptions=False,
)
assert result.exit_code == 0
assert result.output.strip() == (
"hello\n --- becomes:\nHELLO\n\nWould affect 1 row"
)
assert list(db[table_name].rows) == [{column_name: "hello"}]
def test_convert_multi_dryrun(test_db_and_path): def test_convert_multi_dryrun(test_db_and_path):
db_path = test_db_and_path[1] db_path = test_db_and_path[1]
result = CliRunner().invoke( result = CliRunner().invoke(
@ -269,7 +297,7 @@ def test_convert_output_column(test_db_and_path, drop):
if drop: if drop:
for row in expected: for row in expected:
del row["dt"] del row["dt"]
assert list(db["example"].rows) == expected assert list(db.table("example").rows) == expected
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -352,7 +380,7 @@ def test_convert_output_error(test_db_and_path, options, expected_error):
@pytest.mark.parametrize("drop", (True, False)) @pytest.mark.parametrize("drop", (True, False))
def test_convert_multi(fresh_db_and_path, drop): def test_convert_multi(fresh_db_and_path, drop):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["creatures"].insert_all( db.table("creatures").insert_all(
[ [
{"id": 1, "name": "Simon"}, {"id": 1, "name": "Simon"},
{"id": 2, "name": "Cleo"}, {"id": 2, "name": "Cleo"},
@ -378,12 +406,12 @@ def test_convert_multi(fresh_db_and_path, drop):
if drop: if drop:
for row in expected: for row in expected:
del row["name"] del row["name"]
assert list(db["creatures"].rows) == expected assert list(db.table("creatures").rows) == expected
def test_convert_multi_complex_column_types(fresh_db_and_path): def test_convert_multi_complex_column_types(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["rows"].insert_all( db.table("rows").insert_all(
[ [
{"id": 1}, {"id": 1},
{"id": 2}, {"id": 2},
@ -412,7 +440,7 @@ def test_convert_multi_complex_column_types(fresh_db_and_path):
], ],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["rows"].rows) == [ assert list(db.table("rows").rows) == [
{"id": 1, "is_str": "", "is_float": 1.2, "is_int": None, "is_bytes": None}, {"id": 1, "is_str": "", "is_float": 1.2, "is_int": None, "is_bytes": None},
{"id": 2, "is_str": None, "is_float": 1.0, "is_int": 12, "is_bytes": None}, {"id": 2, "is_str": None, "is_float": 1.0, "is_int": 12, "is_bytes": None},
{ {
@ -424,7 +452,7 @@ def test_convert_multi_complex_column_types(fresh_db_and_path):
}, },
{"id": 4, "is_str": None, "is_float": None, "is_int": None, "is_bytes": None}, {"id": 4, "is_str": None, "is_float": None, "is_int": None, "is_bytes": None},
] ]
assert db["rows"].schema == ( assert db.table("rows").schema == (
'CREATE TABLE "rows" (\n' 'CREATE TABLE "rows" (\n'
' "id" INTEGER PRIMARY KEY\n' ' "id" INTEGER PRIMARY KEY\n'
', "is_str" TEXT, "is_float" REAL, "is_int" INTEGER, "is_bytes" BLOB)' ', "is_str" TEXT, "is_float" REAL, "is_int" INTEGER, "is_bytes" BLOB)'
@ -435,7 +463,7 @@ def test_convert_multi_complex_column_types(fresh_db_and_path):
def test_recipe_jsonsplit(tmpdir, delimiter): def test_recipe_jsonsplit(tmpdir, delimiter):
db_path = str(pathlib.Path(tmpdir) / "data.db") db_path = str(pathlib.Path(tmpdir) / "data.db")
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
db["example"].insert_all( db.table("example").insert_all(
[ [
{"id": 1, "tags": (delimiter or ",").join(["foo", "bar"])}, {"id": 1, "tags": (delimiter or ",").join(["foo", "bar"])},
{"id": 2, "tags": (delimiter or ",").join(["bar", "baz"])}, {"id": 2, "tags": (delimiter or ",").join(["bar", "baz"])},
@ -448,7 +476,7 @@ def test_recipe_jsonsplit(tmpdir, delimiter):
args = ["convert", db_path, "example", "tags", code] args = ["convert", db_path, "example", "tags", code]
result = CliRunner().invoke(cli.cli, args) result = CliRunner().invoke(cli.cli, args)
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["example"].rows) == [ assert list(db.table("example").rows) == [
{"id": 1, "tags": '["foo", "bar"]'}, {"id": 1, "tags": '["foo", "bar"]'},
{"id": 2, "tags": '["bar", "baz"]'}, {"id": 2, "tags": '["bar", "baz"]'},
] ]
@ -464,7 +492,7 @@ def test_recipe_jsonsplit(tmpdir, delimiter):
) )
def test_recipe_jsonsplit_type(fresh_db_and_path, type, expected_array): def test_recipe_jsonsplit_type(fresh_db_and_path, type, expected_array):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["example"].insert_all( db.table("example").insert_all(
[ [
{"id": 1, "records": "1,2,3"}, {"id": 1, "records": "1,2,3"},
], ],
@ -476,13 +504,13 @@ def test_recipe_jsonsplit_type(fresh_db_and_path, type, expected_array):
args = ["convert", db_path, "example", "records", code] args = ["convert", db_path, "example", "records", code]
result = CliRunner().invoke(cli.cli, args) result = CliRunner().invoke(cli.cli, args)
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert json.loads(db["example"].get(1)["records"]) == expected_array assert json.loads(db.table("example").get(1)["records"]) == expected_array
@pytest.mark.parametrize("drop", (True, False)) @pytest.mark.parametrize("drop", (True, False))
def test_recipe_jsonsplit_output(fresh_db_and_path, drop): def test_recipe_jsonsplit_output(fresh_db_and_path, drop):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["example"].insert_all( db.table("example").insert_all(
[ [
{"id": 1, "records": "1,2,3"}, {"id": 1, "records": "1,2,3"},
], ],
@ -501,7 +529,7 @@ def test_recipe_jsonsplit_output(fresh_db_and_path, drop):
} }
if drop: if drop:
del expected["records"] del expected["records"]
assert db["example"].get(1) == expected assert db.table("example").get(1) == expected
def test_cannot_use_drop_without_multi_or_output(fresh_db_and_path): def test_cannot_use_drop_without_multi_or_output(fresh_db_and_path):
@ -558,7 +586,7 @@ def test_convert_where(test_db_and_path):
], ],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["example"].rows) == [ assert list(db.table("example").rows) == [
{"id": 1, "dt": "5th October 2019 12:04"}, {"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6TH OCTOBER 2019 00:05:06"}, {"id": 2, "dt": "6TH OCTOBER 2019 00:05:06"},
{"id": 3, "dt": ""}, {"id": 3, "dt": ""},
@ -568,7 +596,7 @@ def test_convert_where(test_db_and_path):
def test_convert_where_multi(fresh_db_and_path): def test_convert_where_multi(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["names"].insert_all( db.table("names").insert_all(
[{"id": 1, "name": "Cleo"}, {"id": 2, "name": "Bants"}], pk="id" [{"id": 1, "name": "Cleo"}, {"id": 2, "name": "Bants"}], pk="id"
) )
result = CliRunner().invoke( result = CliRunner().invoke(
@ -588,7 +616,7 @@ def test_convert_where_multi(fresh_db_and_path):
], ],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["names"].rows) == [ assert list(db.table("names").rows) == [
{"id": 1, "name": "Cleo", "upper": None}, {"id": 1, "name": "Cleo", "upper": None},
{"id": 2, "name": "Bants", "upper": "BANTS"}, {"id": 2, "name": "Bants", "upper": "BANTS"},
] ]
@ -596,7 +624,7 @@ def test_convert_where_multi(fresh_db_and_path):
def test_convert_code_standard_input(fresh_db_and_path): def test_convert_code_standard_input(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["names"].insert_all([{"id": 1, "name": "Cleo"}], pk="id") db.table("names").insert_all([{"id": 1, "name": "Cleo"}], pk="id")
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
[ [
@ -609,27 +637,27 @@ def test_convert_code_standard_input(fresh_db_and_path):
input="value.upper()", input="value.upper()",
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["names"].rows) == [ assert list(db.table("names").rows) == [
{"id": 1, "name": "CLEO"}, {"id": 1, "name": "CLEO"},
] ]
def test_convert_hyphen_workaround(fresh_db_and_path): def test_convert_hyphen_workaround(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["names"].insert_all([{"id": 1, "name": "Cleo"}], pk="id") db.table("names").insert_all([{"id": 1, "name": "Cleo"}], pk="id")
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
["convert", db_path, "names", "name", '"-"'], ["convert", db_path, "names", "name", '"-"'],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["names"].rows) == [ assert list(db.table("names").rows) == [
{"id": 1, "name": "-"}, {"id": 1, "name": "-"},
] ]
def test_convert_initialization_pattern(fresh_db_and_path): def test_convert_initialization_pattern(fresh_db_and_path):
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["names"].insert_all([{"id": 1, "name": "Cleo"}], pk="id") db.table("names").insert_all([{"id": 1, "name": "Cleo"}], pk="id")
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
[ [
@ -642,7 +670,7 @@ def test_convert_initialization_pattern(fresh_db_and_path):
input="import random\nrandom.seed(1)\ndef convert(value): return random.randint(0, 100)", input="import random\nrandom.seed(1)\ndef convert(value): return random.randint(0, 100)",
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["names"].rows) == [ assert list(db.table("names").rows) == [
{"id": 1, "name": "17"}, {"id": 1, "name": "17"},
] ]
@ -657,13 +685,13 @@ def test_convert_handles_falsey_values(fresh_db_and_path):
"x", "x",
"-", "-",
] ]
db["t"].insert_all([{"x": 0}, {"x": 1}]) db.table("t").insert_all([{"x": 0}, {"x": 1}])
assert db["t"].get(1)["x"] == 0 assert db.table("t").get(1)["x"] == 0
assert db["t"].get(2)["x"] == 1 assert db.table("t").get(2)["x"] == 1
result = CliRunner().invoke(cli.cli, args, input="value + 1") result = CliRunner().invoke(cli.cli, args, input="value + 1")
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert db["t"].get(1)["x"] == 1 assert db.table("t").get(1)["x"] == 1
assert db["t"].get(2)["x"] == 2 assert db.table("t").get(2)["x"] == 2
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -684,7 +712,7 @@ def test_convert_callable_reference(test_db_and_path, code):
cli.cli, ["convert", db_path, "example", "dt", code], catch_exceptions=False cli.cli, ["convert", db_path, "example", "dt", code], catch_exceptions=False
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
rows = list(db["example"].rows) rows = list(db.table("example").rows)
assert rows[0]["dt"] == "2019-10-05" assert rows[0]["dt"] == "2019-10-05"
assert rows[1]["dt"] == "2019-10-06" assert rows[1]["dt"] == "2019-10-06"
assert rows[2]["dt"] == "" assert rows[2]["dt"] == ""
@ -694,7 +722,7 @@ def test_convert_callable_reference(test_db_and_path, code):
def test_convert_callable_reference_with_import(fresh_db_and_path): def test_convert_callable_reference_with_import(fresh_db_and_path):
"""Test callable reference from an imported module""" """Test callable reference from an imported module"""
db, db_path = fresh_db_and_path db, db_path = fresh_db_and_path
db["example"].insert({"id": 1, "data": '{"name": "test"}'}) db.table("example").insert({"id": 1, "data": '{"name": "test"}'})
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
[ [
@ -710,5 +738,5 @@ def test_convert_callable_reference_with_import(fresh_db_and_path):
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
# json.loads returns a dict, which sqlite stores as JSON string # json.loads returns a dict, which sqlite stores as JSON string
row = db["example"].get(1) row = db.table("example").get(1)
assert row["data"] == '{"name": "test"}' assert row["data"] == '{"name": "test"}'

View file

@ -21,7 +21,7 @@ def test_insert_simple(tmpdir):
) )
db = Database(db_path) db = Database(db_path)
assert ["dogs"] == db.table_names() assert ["dogs"] == db.table_names()
assert [] == db["dogs"].indexes assert [] == db.table("dogs").indexes
def test_insert_from_stdin(tmpdir): def test_insert_from_stdin(tmpdir):
@ -96,7 +96,7 @@ def test_insert_with_primary_keys(db_path, tmpdir, args, expected_pks):
Database(db_path).query("select * from dogs") Database(db_path).query("select * from dogs")
) )
db = Database(db_path) db = Database(db_path)
assert db["dogs"].pks == expected_pks assert db.table("dogs").pks == expected_pks
def test_insert_multiple_with_primary_key(db_path, tmpdir): def test_insert_multiple_with_primary_key(db_path, tmpdir):
@ -110,7 +110,7 @@ def test_insert_multiple_with_primary_key(db_path, tmpdir):
assert result.exit_code == 0 assert result.exit_code == 0
db = Database(db_path) db = Database(db_path)
assert dogs == list(db.query("select * from dogs order by id")) assert dogs == list(db.query("select * from dogs order by id"))
assert ["id"] == db["dogs"].pks assert ["id"] == db.table("dogs").pks
def test_insert_multiple_with_compound_primary_key(db_path, tmpdir): def test_insert_multiple_with_compound_primary_key(db_path, tmpdir):
@ -127,7 +127,7 @@ def test_insert_multiple_with_compound_primary_key(db_path, tmpdir):
assert result.exit_code == 0 assert result.exit_code == 0
db = Database(db_path) db = Database(db_path)
assert dogs == list(db.query("select * from dogs order by breed, id")) assert dogs == list(db.query("select * from dogs order by breed, id"))
assert {"breed", "id"} == set(db["dogs"].pks) assert {"breed", "id"} == set(db.table("dogs").pks)
assert ( assert (
'CREATE TABLE "dogs" (\n' 'CREATE TABLE "dogs" (\n'
' "breed" TEXT,\n' ' "breed" TEXT,\n'
@ -136,7 +136,7 @@ def test_insert_multiple_with_compound_primary_key(db_path, tmpdir):
' "age" INTEGER,\n' ' "age" INTEGER,\n'
' PRIMARY KEY ("id", "breed")\n' ' PRIMARY KEY ("id", "breed")\n'
")" ")"
) == db["dogs"].schema ) == db.table("dogs").schema
def test_insert_not_null_default(db_path, tmpdir): def test_insert_not_null_default(db_path, tmpdir):
@ -160,7 +160,7 @@ def test_insert_not_null_default(db_path, tmpdir):
' "name" TEXT NOT NULL,\n' ' "name" TEXT NOT NULL,\n'
" \"age\" INTEGER NOT NULL DEFAULT '1',\n" " \"age\" INTEGER NOT NULL DEFAULT '1',\n"
" \"score\" INTEGER DEFAULT '5'\n)" " \"score\" INTEGER DEFAULT '5'\n)"
) == db["dogs"].schema ) == db.table("dogs").schema
def test_insert_binary_base64(db_path): def test_insert_binary_base64(db_path):
@ -191,7 +191,7 @@ def test_insert_newline_delimited(db_path):
def test_insert_ignore(db_path, tmpdir): def test_insert_ignore(db_path, tmpdir):
db = Database(db_path) db = Database(db_path)
db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id") db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
json_path = str(tmpdir / "dogs.json") json_path = str(tmpdir / "dogs.json")
with open(json_path, "w") as fp: with open(json_path, "w") as fp:
fp.write(json.dumps([{"id": 1, "name": "Bailey"}])) fp.write(json.dumps([{"id": 1, "name": "Bailey"}]))
@ -232,7 +232,7 @@ def test_insert_csv_tsv(content, options, db_path, tmpdir):
catch_exceptions=False, catch_exceptions=False,
) )
assert result.exit_code == 0 assert result.exit_code == 0
assert [{"foo": "1", "bar": "2", "baz": "cat,dog"}] == list(db["data"].rows) assert [{"foo": "1", "bar": "2", "baz": "cat,dog"}] == list(db.table("data").rows)
@pytest.mark.parametrize("empty_null", (True, False)) @pytest.mark.parametrize("empty_null", (True, False))
@ -248,7 +248,7 @@ def test_insert_csv_empty_null(db_path, empty_null):
) )
assert result.exit_code == 0 assert result.exit_code == 0
db = Database(db_path) db = Database(db_path)
assert [r for r in db["data"].rows] == [ assert [r for r in db.table("data").rows] == [
{"foo": "1", "bar": None if empty_null else "", "baz": "cat"} {"foo": "1", "bar": None if empty_null else "", "baz": "cat"}
] ]
@ -302,7 +302,7 @@ def test_insert_replace(db_path, tmpdir):
test_insert_multiple_with_primary_key(db_path, tmpdir) test_insert_multiple_with_primary_key(db_path, tmpdir)
json_path = str(tmpdir / "insert-replace.json") json_path = str(tmpdir / "insert-replace.json")
db = Database(db_path) db = Database(db_path)
assert db["dogs"].count == 20 assert db.table("dogs").count == 20
insert_replace_dogs = [ insert_replace_dogs = [
{"id": 1, "name": "Insert replaced 1", "age": 4}, {"id": 1, "name": "Insert replaced 1", "age": 4},
{"id": 2, "name": "Insert replaced 2", "age": 4}, {"id": 2, "name": "Insert replaced 2", "age": 4},
@ -314,7 +314,7 @@ def test_insert_replace(db_path, tmpdir):
cli.cli, ["insert", db_path, "dogs", json_path, "--pk", "id", "--replace"] cli.cli, ["insert", db_path, "dogs", json_path, "--pk", "id", "--replace"]
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert db["dogs"].count == 21 assert db.table("dogs").count == 21
assert ( assert (
list(db.query("select * from dogs where id in (1, 2, 21) order by id")) list(db.query("select * from dogs where id in (1, 2, 21) order by id"))
== insert_replace_dogs == insert_replace_dogs
@ -377,7 +377,7 @@ def test_insert_alter(db_path, tmpdir):
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
# Soundness check the database itself # Soundness check the database itself
db = Database(db_path) db = Database(db_path)
assert {"foo": str, "n": int, "baz": int} == db["from_json_nl"].columns_dict assert {"foo": str, "n": int, "baz": int} == db.table("from_json_nl").columns_dict
assert [ assert [
{"foo": "bar", "n": 1, "baz": None}, {"foo": "bar", "n": 1, "baz": None},
{"foo": "baz", "n": 2, "baz": None}, {"foo": "baz", "n": 2, "baz": None},
@ -387,8 +387,8 @@ def test_insert_alter(db_path, tmpdir):
def test_insert_analyze(db_path): def test_insert_analyze(db_path):
db = Database(db_path) db = Database(db_path)
db["rows"].insert({"foo": "x", "n": 3}) db.table("rows").insert({"foo": "x", "n": 3})
db["rows"].create_index(["n"]) db.table("rows").create_index(["n"])
assert "sqlite_stat1" not in db.table_names() assert "sqlite_stat1" not in db.table_names()
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
@ -577,13 +577,14 @@ def test_insert_streaming_batch_size_1(db_path):
stdin=subprocess.PIPE, stdin=subprocess.PIPE,
stdout=sys.stdout, stdout=sys.stdout,
) )
assert proc.stdin is not None
proc.stdin.write(b'{"name": "Azi"}\n') proc.stdin.write(b'{"name": "Azi"}\n')
proc.stdin.flush() proc.stdin.flush()
def try_until(expected): def try_until(expected):
tries = 0 tries = 0
while True: while True:
rows = list(Database(db_path)["rows"].rows) rows = list(Database(db_path).table("rows").rows)
if rows == expected: if rows == expected:
return return
tries += 1 tries += 1
@ -615,13 +616,13 @@ def test_insert_csv_headers_only(tmpdir):
assert result.exit_code == 0 assert result.exit_code == 0
# Table should not exist since there were no data rows # Table should not exist since there were no data rows
db = Database(db_path) db = Database(db_path)
assert not db["data"].exists() assert not db.table("data").exists()
def test_insert_into_view_errors(tmpdir): def test_insert_into_view_errors(tmpdir):
db_path = str(tmpdir / "test.db") db_path = str(tmpdir / "test.db")
db = Database(db_path) db = Database(db_path)
db["t"].insert({"id": 1}) db.table("t").insert({"id": 1})
db.create_view("v", "select * from t") db.create_view("v", "select * from t")
db.close() db.close()
result = CliRunner().invoke( result = CliRunner().invoke(
@ -637,7 +638,7 @@ def test_insert_csv_detect_types_leaves_existing_table_alone(db_path):
# table would rewrite its column types and corrupt data such as # table would rewrite its column types and corrupt data such as
# TEXT zip codes with leading zeros # TEXT zip codes with leading zeros
db = Database(db_path) db = Database(db_path)
db["places"].insert({"name": "Boston", "zip": "01234"}) db.table("places").insert({"name": "Boston", "zip": "01234"})
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
["insert", db_path, "places", "-", "--csv"], ["insert", db_path, "places", "-", "--csv"],
@ -645,8 +646,8 @@ def test_insert_csv_detect_types_leaves_existing_table_alone(db_path):
input="name,zip\nSF,94107", input="name,zip\nSF,94107",
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert db["places"].columns_dict["zip"] is str assert db.table("places").columns_dict["zip"] is str
assert list(db["places"].rows) == [ assert list(db.table("places").rows) == [
{"name": "Boston", "zip": "01234"}, {"name": "Boston", "zip": "01234"},
{"name": "SF", "zip": "94107"}, {"name": "SF", "zip": "94107"},
] ]
@ -662,7 +663,7 @@ def test_insert_csv_detect_types_new_table(db_path):
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
db = Database(db_path) db = Database(db_path)
assert db["data"].columns_dict == {"name": str, "age": int, "weight": float} assert db.table("data").columns_dict == {"name": str, "age": int, "weight": float}
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -708,13 +709,13 @@ def test_insert_upsert_csv_type_overrides_detected_types(
expected_columns = {"zipcode": str, "score": float} expected_columns = {"zipcode": str, "score": float}
if command == "upsert": if command == "upsert":
expected_columns = {"id": int, **expected_columns} expected_columns = {"id": int, **expected_columns}
assert db["places"].columns_dict == expected_columns assert db.table("places").columns_dict == expected_columns
assert list(db["places"].rows) == [expected_row] assert list(db.table("places").rows) == [expected_row]
def test_upsert_csv_detect_types_leaves_existing_table_alone(db_path): def test_upsert_csv_detect_types_leaves_existing_table_alone(db_path):
db = Database(db_path) db = Database(db_path)
db["places"].insert({"id": 1, "name": "Boston", "zip": "01234"}, pk="id") db.table("places").insert({"id": 1, "name": "Boston", "zip": "01234"}, pk="id")
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
["upsert", db_path, "places", "-", "--csv", "--pk", "id"], ["upsert", db_path, "places", "-", "--csv", "--pk", "id"],
@ -722,15 +723,15 @@ def test_upsert_csv_detect_types_leaves_existing_table_alone(db_path):
input="id,name,zip\n2,SF,94107", input="id,name,zip\n2,SF,94107",
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert db["places"].columns_dict["zip"] is str assert db.table("places").columns_dict["zip"] is str
assert db["places"].get(1)["zip"] == "01234" assert db.table("places").get(1)["zip"] == "01234"
def test_insert_invalid_pk_clean_error(db_path): def test_insert_invalid_pk_clean_error(db_path):
# An invalid --pk against an existing table should be a clean CLI # An invalid --pk against an existing table should be a clean CLI
# error, not a raw InvalidColumns traceback # error, not a raw InvalidColumns traceback
db = Database(db_path) db = Database(db_path)
db["t"].insert({"a": 1}) db.table("t").insert({"a": 1})
result = CliRunner().invoke( result = CliRunner().invoke(
cli.cli, cli.cli,
["insert", db_path, "t", "-", "--pk", "badcol"], ["insert", db_path, "t", "-", "--pk", "badcol"],
@ -765,8 +766,8 @@ def test_insert_code(tmpdir, code):
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
db = Database(db_path) db = Database(db_path)
assert db["creatures"].pks == ["id"] assert db.table("creatures").pks == ["id"]
assert list(db["creatures"].rows) == [ assert list(db.table("creatures").rows) == [
{"id": 1, "name": "Cleo"}, {"id": 1, "name": "Cleo"},
{"id": 2, "name": "Suna"}, {"id": 2, "name": "Suna"},
] ]
@ -782,7 +783,7 @@ def test_insert_code_from_file(tmpdir):
["insert", db_path, "creatures", "--code", code_path], ["insert", db_path, "creatures", "--code", code_path],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(Database(db_path)["creatures"].rows) == [ assert list(Database(db_path).table("creatures").rows) == [
{"id": 1, "name": "Cleo"}, {"id": 1, "name": "Cleo"},
{"id": 2, "name": "Suna"}, {"id": 2, "name": "Suna"},
] ]
@ -791,7 +792,7 @@ def test_insert_code_from_file(tmpdir):
def test_upsert_code(tmpdir): def test_upsert_code(tmpdir):
db_path = str(tmpdir / "dogs.db") db_path = str(tmpdir / "dogs.db")
db = Database(db_path) db = Database(db_path)
db["creatures"].insert_all( db.table("creatures").insert_all(
[{"id": 1, "name": "old"}, {"id": 2, "name": "Suna"}], pk="id" [{"id": 1, "name": "old"}, {"id": 2, "name": "Suna"}], pk="id"
) )
result = CliRunner().invoke( result = CliRunner().invoke(
@ -799,7 +800,7 @@ def test_upsert_code(tmpdir):
["upsert", db_path, "creatures", "--code", CODE_ROWS_FUNCTION, "--pk", "id"], ["upsert", db_path, "creatures", "--code", CODE_ROWS_FUNCTION, "--pk", "id"],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(db["creatures"].rows) == [ assert list(db.table("creatures").rows) == [
{"id": 1, "name": "Cleo"}, {"id": 1, "name": "Cleo"},
{"id": 2, "name": "Suna"}, {"id": 2, "name": "Suna"},
] ]
@ -858,7 +859,9 @@ def test_insert_code_single_dict(tmpdir):
], ],
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
assert list(Database(db_path)["creatures"].rows) == [{"id": 1, "name": "Cleo"}] assert list(Database(db_path).table("creatures").rows) == [
{"id": 1, "name": "Cleo"}
]
def test_insert_code_not_iterable(tmpdir): def test_insert_code_not_iterable(tmpdir):

View file

@ -228,7 +228,7 @@ def test_memory_save(tmpdir, extra_args):
) )
assert result.exit_code == 0 assert result.exit_code == 0
db = Database(save_to) db = Database(save_to)
assert list(db["stdin"].rows) == [ assert list(db.table("stdin").rows) == [
{"id": 1, "name": "Cleo"}, {"id": 1, "name": "Cleo"},
{"id": 2, "name": "Bants"}, {"id": 2, "name": "Bants"},
] ]

View file

@ -13,11 +13,11 @@ m = Migrations("hello")
@m() @m()
def foo(db): def foo(db):
db["foo"].insert({"hello": "world"}) db.table("foo").insert({"hello": "world"})
@m() @m()
def bar(db): def bar(db):
db["bar"].insert({"hello": "world"}) db.table("bar").insert({"hello": "world"})
""" """
@ -42,21 +42,21 @@ creatures = Migrations("creatures")
@creatures() @creatures()
def create_table(db): def create_table(db):
db["creatures"].insert({"name": "Cleo"}) db.table("creatures").insert({"name": "Cleo"})
@creatures() @creatures()
def add_weight(db): def add_weight(db):
db["creature_weights"].insert({"weight": 4.2}) db.table("creature_weights").insert({"weight": 4.2})
sales = Migrations("sales") sales = Migrations("sales")
@sales() @sales()
def create_table(db): def create_table(db):
db["sales"].insert({"id": 1}) db.table("sales").insert({"id": 1})
@sales() @sales()
def add_weight(db): def add_weight(db):
db["sales_weights"].insert({"weight": 10}) db.table("sales_weights").insert({"weight": 10})
""", """,
"utf-8", "utf-8",
) )
@ -99,10 +99,10 @@ def test_basic(two_migrations, arg):
assert " Pending:\n (none)" in list_output assert " Pending:\n (none)" in list_output
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
assert db["foo"].exists() assert db.table("foo").exists()
assert db["bar"].exists() assert db.table("bar").exists()
assert db["_sqlite_migrations"].exists() assert db.table("_sqlite_migrations").exists()
rows = list(db["_sqlite_migrations"].rows) rows = list(db.table("_sqlite_migrations").rows)
assert len(rows) == 2 assert len(rows) == 2
assert rows[0]["name"] == "foo" assert rows[0]["name"] == "foo"
assert rows[1]["name"] == "bar" assert rows[1]["name"] == "bar"
@ -113,13 +113,13 @@ def test_list_same_migration_names_in_different_sets(capsys):
@applied(name="foo") @applied(name="foo")
def applied_foo(db): def applied_foo(db):
db["applied"].insert({"hello": "world"}) db.table("applied").insert({"hello": "world"})
pending = sqlite_utils.Migrations("pending") pending = sqlite_utils.Migrations("pending")
@pending(name="foo") @pending(name="foo")
def pending_foo(db): def pending_foo(db):
db["pending"].insert({"hello": "world"}) db.table("pending").insert({"hello": "world"})
db = sqlite_utils.Database(memory=True) db = sqlite_utils.Database(memory=True)
applied.apply(db) applied.apply(db)
@ -144,7 +144,7 @@ m = Migrations("hello")
@m() @m()
def foo(db): def foo(db):
db["dogs"].insert({"id": 1, "name": "Cleo"}) db.table("dogs").insert({"id": 1, "name": "Cleo"})
""", """,
"utf-8", "utf-8",
) )
@ -184,9 +184,9 @@ Schema after:
new_migration = """ new_migration = """
@m() @m()
def bar(db): def bar(db):
db["dogs"].add_column("age", int) db.table("dogs").add_column("age", int)
db["dogs"].add_column("weight", float) db.table("dogs").add_column("weight", float)
db["dogs"].transform() db.table("dogs").transform()
""" """
migrations_py.write_text(migrations_py.read_text("utf-8") + new_migration) migrations_py.write_text(migrations_py.read_text("utf-8") + new_migration)
@ -224,8 +224,8 @@ def test_stop_before(two_migrations):
) )
assert result.exit_code == 0 assert result.exit_code == 0
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
assert db["foo"].exists() assert db.table("foo").exists()
assert not db["bar"].exists() assert not db.table("bar").exists()
def test_stop_before_multiple_sets_unqualified(two_migrations): def test_stop_before_multiple_sets_unqualified(two_migrations):
@ -239,7 +239,7 @@ m = Migrations("hello2")
@m() @m()
def foo(db): def foo(db):
db["foo"].insert({"hello": "world"}) db.table("foo").insert({"hello": "world"})
""", """,
"utf-8", "utf-8",
) )
@ -257,7 +257,7 @@ def foo(db):
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
assert db.table_names() == ["_sqlite_migrations"] assert db.table_names() == ["_sqlite_migrations"]
assert list(db["_sqlite_migrations"].rows) == [] assert list(db.table("_sqlite_migrations").rows) == []
def test_stop_before_qualified_only_affects_named_set(two_sets_same_migration_name): def test_stop_before_qualified_only_affects_named_set(two_sets_same_migration_name):
@ -275,10 +275,10 @@ def test_stop_before_qualified_only_affects_named_set(two_sets_same_migration_na
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
assert db["creatures"].exists() assert db.table("creatures").exists()
assert not db["creature_weights"].exists() assert not db.table("creature_weights").exists()
assert db["sales"].exists() assert db.table("sales").exists()
assert db["sales_weights"].exists() assert db.table("sales_weights").exists()
def test_stop_before_multiple_qualified(two_sets_same_migration_name): def test_stop_before_multiple_qualified(two_sets_same_migration_name):
@ -298,10 +298,10 @@ def test_stop_before_multiple_qualified(two_sets_same_migration_name):
) )
assert result.exit_code == 0, result.output assert result.exit_code == 0, result.output
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
assert db["creatures"].exists() assert db.table("creatures").exists()
assert not db["creature_weights"].exists() assert not db.table("creature_weights").exists()
assert db["sales"].exists() assert db.table("sales").exists()
assert not db["sales_weights"].exists() assert not db.table("sales_weights").exists()
LEGACY_MIGRATIONS = """ LEGACY_MIGRATIONS = """
@ -331,7 +331,7 @@ class LegacyMigrations:
return fn return fn
def ensure_migrations_table(self, db): def ensure_migrations_table(self, db):
db[self.migrations_table].create( db.table(self.migrations_table).create(
{"migration_set": str, "name": str, "applied_at": str}, {"migration_set": str, "name": str, "applied_at": str},
pk=("migration_set", "name"), pk=("migration_set", "name"),
if_not_exists=True, if_not_exists=True,
@ -341,7 +341,7 @@ class LegacyMigrations:
self.ensure_migrations_table(db) self.ensure_migrations_table(db)
return [ return [
_Applied(row["name"], row["applied_at"]) _Applied(row["name"], row["applied_at"])
for row in db[self.migrations_table].rows_where( for row in db.table(self.migrations_table).rows_where(
"migration_set = ?", [self.name] "migration_set = ?", [self.name]
) )
] ]
@ -355,7 +355,7 @@ class LegacyMigrations:
if migration.name == stop_before: if migration.name == stop_before:
return return
migration.fn(db) migration.fn(db)
db[self.migrations_table].insert( db.table(self.migrations_table).insert(
{ {
"migration_set": self.name, "migration_set": self.name,
"name": migration.name, "name": migration.name,
@ -369,11 +369,11 @@ legacy = LegacyMigrations("legacy_set")
@legacy @legacy
def first(db): def first(db):
db["first"].insert({"hello": "world"}) db.table("first").insert({"hello": "world"})
@legacy @legacy
def second(db): def second(db):
db["second"].insert({"hello": "world"}) db.table("second").insert({"hello": "world"})
""" """
@ -446,11 +446,11 @@ def test_list_does_not_upgrade_legacy_migrations_table(two_migrations):
path, _ = two_migrations path, _ = two_migrations
db_path = str(path / "test.db") db_path = str(path / "test.db")
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
db["_sqlite_migrations"].create( db.table("_sqlite_migrations").create(
{"migration_set": str, "name": str, "applied_at": str}, {"migration_set": str, "name": str, "applied_at": str},
pk=("migration_set", "name"), pk=("migration_set", "name"),
) )
db["_sqlite_migrations"].insert( db.table("_sqlite_migrations").insert(
{"migration_set": "hello", "name": "foo", "applied_at": "x"} {"migration_set": "hello", "name": "foo", "applied_at": "x"}
) )
db.close() db.close()
@ -462,7 +462,7 @@ def test_list_does_not_upgrade_legacy_migrations_table(two_migrations):
assert "foo - x" in result.output assert "foo - x" in result.output
# --list must not perform the one-way legacy schema upgrade # --list must not perform the one-way legacy schema upgrade
db2 = sqlite_utils.Database(db_path) db2 = sqlite_utils.Database(db_path)
assert db2["_sqlite_migrations"].pks == ["migration_set", "name"] assert db2.table("_sqlite_migrations").pks == ["migration_set", "name"]
db2.close() db2.close()
@ -485,7 +485,7 @@ def test_stop_before_applied_migration_errors(two_migrations):
assert result.exit_code != 0 assert result.exit_code != 0
assert "already been applied" in result.output assert "already been applied" in result.output
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
assert not db["bar"].exists() assert not db.table("bar").exists()
def test_list_with_legacy_class_is_read_only(tmpdir): def test_list_with_legacy_class_is_read_only(tmpdir):
@ -496,7 +496,7 @@ def test_list_with_legacy_class_is_read_only(tmpdir):
(path / "migrations.py").write_text(LEGACY_MIGRATIONS, "utf-8") (path / "migrations.py").write_text(LEGACY_MIGRATIONS, "utf-8")
db_path = str(path / "test.db") db_path = str(path / "test.db")
db = sqlite_utils.Database(db_path) db = sqlite_utils.Database(db_path)
db["existing"].insert({"id": 1}) db.table("existing").insert({"id": 1})
db.close() db.close()
result = CliRunner().invoke( result = CliRunner().invoke(
sqlite_utils.cli.cli, ["migrate", db_path, str(path), "--list"] sqlite_utils.cli.cli, ["migrate", db_path, str(path), "--list"]

View file

@ -1,5 +1,6 @@
import pytest import pytest
from sqlite_utils import ANY
from sqlite_utils.utils import column_affinity from sqlite_utils.utils import column_affinity
EXAMPLES = [ EXAMPLES = [
@ -26,6 +27,8 @@ EXAMPLES = [
("DOUBLE", float), ("DOUBLE", float),
("DOUBLE PRECISION", float), ("DOUBLE PRECISION", float),
("FLOAT", float), ("FLOAT", float),
("ANY", ANY),
("any", ANY),
# Numeric, treated as float: # Numeric, treated as float:
("NUMERIC", float), ("NUMERIC", float),
("DECIMAL(10,5)", float), ("DECIMAL(10,5)", float),
@ -43,4 +46,4 @@ def test_column_affinity(column_def, expected_type):
@pytest.mark.parametrize("column_def,expected_type", EXAMPLES) @pytest.mark.parametrize("column_def,expected_type", EXAMPLES)
def test_columns_dict(fresh_db, column_def, expected_type): def test_columns_dict(fresh_db, column_def, expected_type):
fresh_db.execute(f"create table foo (col {column_def})") fresh_db.execute(f"create table foo (col {column_def})")
assert {"col": expected_type} == fresh_db["foo"].columns_dict assert {"col": expected_type} == fresh_db.table("foo").columns_dict

View file

@ -13,14 +13,14 @@ from sqlite_utils.db import ForeignKey
def test_insert_populates_last_pk_case_insensitively(fresh_db): def test_insert_populates_last_pk_case_insensitively(fresh_db):
books = fresh_db["books"] books = fresh_db.table("books")
books.create({"Id": int, "Title": str}, pk="Id") books.create({"Id": int, "Title": str}, pk="Id")
books.insert({"Id": 1, "Title": "One"}, pk="id") books.insert({"Id": 1, "Title": "One"}, pk="id")
assert books.last_pk == 1 assert books.last_pk == 1
def test_insert_populates_last_pk_compound_pk_case_insensitively(fresh_db): def test_insert_populates_last_pk_compound_pk_case_insensitively(fresh_db):
books = fresh_db["books"] books = fresh_db.table("books")
books.create({"Author": str, "Position": int, "Title": str}) books.create({"Author": str, "Position": int, "Title": str})
books.insert( books.insert(
{"Author": "Sue", "Position": 1, "Title": "One"}, pk=("author", "position") {"Author": "Sue", "Position": 1, "Title": "One"}, pk=("author", "position")
@ -31,7 +31,7 @@ def test_insert_populates_last_pk_compound_pk_case_insensitively(fresh_db):
@pytest.mark.parametrize("use_old_upsert", (False, True)) @pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert_pk_case_differs_from_schema(use_old_upsert): def test_upsert_pk_case_differs_from_schema(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert) db = Database(memory=True, use_old_upsert=use_old_upsert)
books = db["books"] books = db.table("books")
books.create({"Id": int, "Title": str}, pk="Id") books.create({"Id": int, "Title": str}, pk="Id")
books.insert({"Id": 1, "Title": "One"}) books.insert({"Id": 1, "Title": "One"})
books.upsert({"id": 1, "title": "Won"}, pk="id") books.upsert({"id": 1, "title": "Won"}, pk="id")
@ -43,7 +43,7 @@ def test_upsert_pk_case_differs_from_schema(use_old_upsert):
def test_upsert_record_key_case_differs_from_pk(use_old_upsert): def test_upsert_record_key_case_differs_from_pk(use_old_upsert):
# all_columns comes from the record keys, pk= from the caller # all_columns comes from the record keys, pk= from the caller
db = Database(memory=True, use_old_upsert=use_old_upsert) db = Database(memory=True, use_old_upsert=use_old_upsert)
books = db["books"] books = db.table("books")
books.create({"Id": int, "Title": str}, pk="Id") books.create({"Id": int, "Title": str}, pk="Id")
books.upsert({"ID": 1, "Title": "One"}, pk="id") books.upsert({"ID": 1, "Title": "One"}, pk="id")
assert list(books.rows) == [{"Id": 1, "Title": "One"}] assert list(books.rows) == [{"Id": 1, "Title": "One"}]
@ -52,7 +52,7 @@ def test_upsert_record_key_case_differs_from_pk(use_old_upsert):
def test_upsert_inferred_pk_case_differs_from_record_keys(fresh_db): def test_upsert_inferred_pk_case_differs_from_record_keys(fresh_db):
# pk is inferred from the existing schema as "Id", records use "id" # pk is inferred from the existing schema as "Id", records use "id"
books = fresh_db["books"] books = fresh_db.table("books")
books.create({"Id": int, "Title": str}, pk="Id") books.create({"Id": int, "Title": str}, pk="Id")
books.upsert({"id": 1, "title": "One"}) books.upsert({"id": 1, "title": "One"})
assert list(books.rows) == [{"Id": 1, "Title": "One"}] assert list(books.rows) == [{"Id": 1, "Title": "One"}]
@ -60,7 +60,7 @@ def test_upsert_inferred_pk_case_differs_from_record_keys(fresh_db):
def test_upsert_list_mode_pk_case_insensitive(fresh_db): def test_upsert_list_mode_pk_case_insensitive(fresh_db):
books = fresh_db["books"] books = fresh_db.table("books")
books.create({"Id": int, "Title": str}, pk="Id") books.create({"Id": int, "Title": str}, pk="Id")
books.upsert_all([["id", "title"], [1, "One"]], pk="Id") books.upsert_all([["id", "title"], [1, "One"]], pk="Id")
assert list(books.rows) == [{"Id": 1, "Title": "One"}] assert list(books.rows) == [{"Id": 1, "Title": "One"}]
@ -68,84 +68,84 @@ def test_upsert_list_mode_pk_case_insensitive(fresh_db):
def test_lookup_pk_case_insensitive(fresh_db): def test_lookup_pk_case_insensitive(fresh_db):
fresh_db["species"].create({"ID": int, "Name": str}, pk="ID") fresh_db.table("species").create({"ID": int, "Name": str}, pk="ID")
fresh_db["species"].insert({"ID": 5, "Name": "Palm"}) fresh_db.table("species").insert({"ID": 5, "Name": "Palm"})
fresh_db["species"].create_index(["Name"], unique=True) fresh_db.table("species").create_index(["Name"], unique=True)
assert fresh_db["species"].lookup({"Name": "Palm"}, pk="id") == 5 assert fresh_db.table("species").lookup({"Name": "Palm"}, pk="id") == 5
def test_lookup_does_not_create_redundant_index(fresh_db): def test_lookup_does_not_create_redundant_index(fresh_db):
fresh_db["species"].create({"id": int, "Name": str}, pk="id") fresh_db.table("species").create({"id": int, "Name": str}, pk="id")
fresh_db["species"].create_index(["Name"], unique=True) fresh_db.table("species").create_index(["Name"], unique=True)
fresh_db["species"].lookup({"name": "Palm"}) fresh_db.table("species").lookup({"name": "Palm"})
assert len(fresh_db["species"].indexes) == 1 assert len(fresh_db.table("species").indexes) == 1
def test_create_table_transform_same_columns_different_case(fresh_db): def test_create_table_transform_same_columns_different_case(fresh_db):
fresh_db["t"].create({"Name": str, "Age": int}) fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db["t"].insert({"Name": "Cleo", "Age": 5}) fresh_db.table("t").insert({"Name": "Cleo", "Age": 5})
fresh_db.create_table("t", {"name": str, "age": int}, transform=True) fresh_db.create_table("t", {"name": str, "age": int}, transform=True)
# Schema casing is preserved - SQLite considers these the same columns # Schema casing is preserved - SQLite considers these the same columns
assert fresh_db["t"].columns_dict == {"Name": str, "Age": int} assert fresh_db.table("t").columns_dict == {"Name": str, "Age": int}
assert list(fresh_db["t"].rows) == [{"Name": "Cleo", "Age": 5}] assert list(fresh_db.table("t").rows) == [{"Name": "Cleo", "Age": 5}]
def test_create_table_transform_case_insensitive_with_changes(fresh_db): def test_create_table_transform_case_insensitive_with_changes(fresh_db):
fresh_db["t"].create({"Name": str, "Age": int}) fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db.create_table("t", {"name": str, "age": str, "size": int}, transform=True) fresh_db.create_table("t", {"name": str, "age": str, "size": int}, transform=True)
# age changed type, size added, Name untouched # age changed type, size added, Name untouched
assert fresh_db["t"].columns_dict == {"Name": str, "Age": str, "size": int} assert fresh_db.table("t").columns_dict == {"Name": str, "Age": str, "size": int}
def test_transform_types_case_insensitive(fresh_db): def test_transform_types_case_insensitive(fresh_db):
fresh_db["t"].create({"Name": str, "Age": str}) fresh_db.table("t").create({"Name": str, "Age": str})
fresh_db["t"].transform(types={"age": int}) fresh_db.table("t").transform(types={"age": int})
assert fresh_db["t"].columns_dict == {"Name": str, "Age": int} assert fresh_db.table("t").columns_dict == {"Name": str, "Age": int}
def test_transform_rename_case_insensitive(fresh_db): def test_transform_rename_case_insensitive(fresh_db):
fresh_db["t"].create({"Name": str}) fresh_db.table("t").create({"Name": str})
fresh_db["t"].transform(rename={"name": "title"}) fresh_db.table("t").transform(rename={"name": "title"})
assert fresh_db["t"].columns_dict == {"title": str} assert fresh_db.table("t").columns_dict == {"title": str}
def test_transform_drop_case_insensitive(fresh_db): def test_transform_drop_case_insensitive(fresh_db):
fresh_db["t"].create({"Name": str, "Age": int}) fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db["t"].transform(drop=["name"]) fresh_db.table("t").transform(drop=["name"])
assert fresh_db["t"].columns_dict == {"Age": int} assert fresh_db.table("t").columns_dict == {"Age": int}
def test_transform_not_null_and_defaults_case_insensitive(fresh_db): def test_transform_not_null_and_defaults_case_insensitive(fresh_db):
fresh_db["t"].create({"Name": str, "Age": int}) fresh_db.table("t").create({"Name": str, "Age": int})
fresh_db["t"].transform(not_null={"name"}, defaults={"age": 3}) fresh_db.table("t").transform(not_null={"name"}, defaults={"age": 3})
columns = {c.name: c for c in fresh_db["t"].columns} columns = {c.name: c for c in fresh_db.table("t").columns}
assert columns["Name"].notnull assert columns["Name"].notnull
assert fresh_db["t"].default_values == {"Age": 3} assert fresh_db.table("t").default_values == {"Age": 3}
def test_transform_pk_case_insensitive(fresh_db): def test_transform_pk_case_insensitive(fresh_db):
fresh_db["t"].create({"Id": int, "Name": str}) fresh_db.table("t").create({"Id": int, "Name": str})
fresh_db["t"].transform(pk="id") fresh_db.table("t").transform(pk="id")
assert fresh_db["t"].pks == ["Id"] assert fresh_db.table("t").pks == ["Id"]
assert fresh_db["t"].columns_dict == {"Id": int, "Name": str} assert fresh_db.table("t").columns_dict == {"Id": int, "Name": str}
def test_transform_drop_foreign_keys_case_insensitive(fresh_db): def test_transform_drop_foreign_keys_case_insensitive(fresh_db):
fresh_db["parent"].create({"Id": int}, pk="Id") fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db["child"].create( fresh_db.table("child").create(
{"id": int, "Parent_ID": int}, {"id": int, "Parent_ID": int},
pk="id", pk="id",
foreign_keys=[("Parent_ID", "parent", "Id")], foreign_keys=[("Parent_ID", "parent", "Id")],
) )
fresh_db["child"].transform(drop_foreign_keys=["parent_id"]) fresh_db.table("child").transform(drop_foreign_keys=["parent_id"])
assert fresh_db["child"].foreign_keys == [] assert fresh_db.table("child").foreign_keys == []
def test_add_foreign_key_case_insensitive(fresh_db): def test_add_foreign_key_case_insensitive(fresh_db):
fresh_db["parent"].create({"Id": int}, pk="Id") fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db["child"].create({"id": int, "Parent_ID": int}, pk="id") fresh_db.table("child").create({"id": int, "Parent_ID": int}, pk="id")
fresh_db["child"].add_foreign_key("parent_id", "parent", "id") fresh_db.table("child").add_foreign_key("parent_id", "parent", "id")
fks = fresh_db["child"].foreign_keys fks = fresh_db.table("child").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
# The foreign key should use the schema casing of the columns # The foreign key should use the schema casing of the columns
assert fks[0].column == "Parent_ID" assert fks[0].column == "Parent_ID"
@ -153,79 +153,83 @@ def test_add_foreign_key_case_insensitive(fresh_db):
def test_add_foreign_keys_case_insensitive(fresh_db): def test_add_foreign_keys_case_insensitive(fresh_db):
fresh_db["parent"].create({"Id": int}, pk="Id") fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db["child"].create({"id": int, "Parent_ID": int}, pk="id") fresh_db.table("child").create({"id": int, "Parent_ID": int}, pk="id")
fresh_db.add_foreign_keys([("child", "parent_id", "parent", "id")]) fresh_db.add_foreign_keys([("child", "parent_id", "parent", "id")])
fks = fresh_db["child"].foreign_keys fks = fresh_db.table("child").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
assert fks[0].column == "Parent_ID" assert fks[0].column == "Parent_ID"
assert fks[0].other_column == "Id" assert fks[0].other_column == "Id"
def test_add_foreign_key_detects_existing_case_insensitively(fresh_db): def test_add_foreign_key_detects_existing_case_insensitively(fresh_db):
fresh_db["parent"].create({"Id": int}, pk="Id") fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db["child"].create( fresh_db.table("child").create(
{"id": int, "Parent_ID": int}, {"id": int, "Parent_ID": int},
pk="id", pk="id",
foreign_keys=[("Parent_ID", "parent", "Id")], foreign_keys=[("Parent_ID", "parent", "Id")],
) )
# ignore=True should treat this as already existing, not add a duplicate # ignore=True should treat this as already existing, not add a duplicate
fresh_db["child"].add_foreign_key("parent_id", "parent", "id", ignore=True) fresh_db.table("child").add_foreign_key("parent_id", "parent", "id", ignore=True)
assert len(fresh_db["child"].foreign_keys) == 1 assert len(fresh_db.table("child").foreign_keys) == 1
def test_add_column_fk_col_case_insensitive(fresh_db): def test_add_column_fk_col_case_insensitive(fresh_db):
fresh_db["parent"].create({"Id": int}, pk="Id") fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db["child"].create({"id": int}, pk="id") fresh_db.table("child").create({"id": int}, pk="id")
fresh_db["child"].add_column("parent_id", int, fk="parent", fk_col="id") fresh_db.table("child").add_column("parent_id", int, fk="parent", fk_col="id")
fks = fresh_db["child"].foreign_keys fks = fresh_db.table("child").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
assert fks[0].other_column == "Id" assert fks[0].other_column == "Id"
def test_extract_case_insensitive(fresh_db): def test_extract_case_insensitive(fresh_db):
fresh_db["trees"].insert({"id": 1, "Species": "Palm"}, pk="id") fresh_db.table("trees").insert({"id": 1, "Species": "Palm"}, pk="id")
fresh_db["trees"].extract("species") fresh_db.table("trees").extract("species")
assert fresh_db["trees"].columns_dict == {"id": int, "Species_id": int} assert fresh_db.table("trees").columns_dict == {"id": int, "Species_id": int}
assert list(fresh_db["Species"].rows) == [{"id": 1, "Species": "Palm"}] assert list(fresh_db.table("Species").rows) == [{"id": 1, "Species": "Palm"}]
def test_convert_multi_case_insensitive(fresh_db): def test_convert_multi_case_insensitive(fresh_db):
fresh_db["t"].insert({"id": 1, "Name": "Cleo"}, pk="id") fresh_db.table("t").insert({"id": 1, "Name": "Cleo"}, pk="id")
fresh_db["t"].convert("name", lambda v: {"upper": v.upper()}, multi=True) fresh_db.table("t").convert("name", lambda v: {"upper": v.upper()}, multi=True)
assert list(fresh_db["t"].rows) == [{"id": 1, "Name": "Cleo", "upper": "CLEO"}] assert list(fresh_db.table("t").rows) == [
{"id": 1, "Name": "Cleo", "upper": "CLEO"}
]
def test_convert_output_case_insensitive(fresh_db): def test_convert_output_case_insensitive(fresh_db):
fresh_db["t"].insert({"id": 1, "Name": "Cleo", "Upper": None}, pk="id") fresh_db.table("t").insert({"id": 1, "Name": "Cleo", "Upper": None}, pk="id")
fresh_db["t"].convert("name", lambda v: v.upper(), output="upper") fresh_db.table("t").convert("name", lambda v: v.upper(), output="upper")
assert list(fresh_db["t"].rows) == [{"id": 1, "Name": "Cleo", "Upper": "CLEO"}] assert list(fresh_db.table("t").rows) == [
{"id": 1, "Name": "Cleo", "Upper": "CLEO"}
]
def test_create_table_sql_pk_case_insensitive(fresh_db): def test_create_table_sql_pk_case_insensitive(fresh_db):
fresh_db["t"].create({"Id": int, "Name": str}, pk="id") fresh_db.table("t").create({"Id": int, "Name": str}, pk="id")
# Should not have created an extra lowercase "id" column # Should not have created an extra lowercase "id" column
assert fresh_db["t"].columns_dict == {"Id": int, "Name": str} assert fresh_db.table("t").columns_dict == {"Id": int, "Name": str}
assert fresh_db["t"].pks == ["Id"] assert fresh_db.table("t").pks == ["Id"]
def test_create_table_not_null_and_defaults_case_insensitive(fresh_db): def test_create_table_not_null_and_defaults_case_insensitive(fresh_db):
fresh_db["t"].create( fresh_db.table("t").create(
{"Name": str, "Age": int}, not_null={"name"}, defaults={"age": 1} {"Name": str, "Age": int}, not_null={"name"}, defaults={"age": 1}
) )
columns = {c.name: c for c in fresh_db["t"].columns} columns = {c.name: c for c in fresh_db.table("t").columns}
assert columns["Name"].notnull assert columns["Name"].notnull
assert fresh_db["t"].default_values == {"Age": 1} assert fresh_db.table("t").default_values == {"Age": 1}
def test_create_table_foreign_keys_case_insensitive(fresh_db): def test_create_table_foreign_keys_case_insensitive(fresh_db):
fresh_db["parent"].create({"Id": int}, pk="Id") fresh_db.table("parent").create({"Id": int}, pk="Id")
fresh_db["child"].create( fresh_db.table("child").create(
{"id": int, "Parent_ID": int}, {"id": int, "Parent_ID": int},
pk="id", pk="id",
foreign_keys=[("parent_id", "parent", "id")], foreign_keys=[("parent_id", "parent", "id")],
) )
fks = fresh_db["child"].foreign_keys fks = fresh_db.table("child").foreign_keys
assert fks == [ assert fks == [
ForeignKey( ForeignKey(
table="child", column="Parent_ID", other_table="parent", other_column="Id" table="child", column="Parent_ID", other_table="parent", other_column="Id"

View file

@ -20,8 +20,8 @@ def test_recursive_triggers_off():
def test_memory_name(): def test_memory_name():
db1 = Database(memory_name="shared") db1 = Database(memory_name="shared")
db2 = Database(memory_name="shared") db2 = Database(memory_name="shared")
db1["dogs"].insert({"name": "Cleo"}) db1.table("dogs").insert({"name": "Cleo"})
assert list(db2["dogs"].rows) == [{"name": "Cleo"}] assert list(db2.table("dogs").rows) == [{"name": "Cleo"}]
def test_sqlite_version(): def test_sqlite_version():
@ -36,7 +36,7 @@ def test_sqlite_version():
def test_database_context_manager(tmpdir): def test_database_context_manager(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
with Database(path) as db: with Database(path) as db:
db["t"].insert({"id": 1}) db.table("t").insert({"id": 1})
# Raw writes commit automatically too # Raw writes commit automatically too
db.execute("insert into t (id) values (2)") db.execute("insert into t (id) values (2)")
# An explicitly opened transaction left uncommitted on purpose: # An explicitly opened transaction left uncommitted on purpose:
@ -47,7 +47,7 @@ def test_database_context_manager(tmpdir):
db.execute("select 1") db.execute("select 1")
# ... and the open explicit transaction was rolled back, not committed # ... and the open explicit transaction was rolled back, not committed
db2 = Database(path) db2 = Database(path)
assert [r["id"] for r in db2["t"].rows] == [1, 2] assert [r["id"] for r in db2.table("t").rows] == [1, 2]
db2.close() db2.close()
@ -83,11 +83,12 @@ def test_autocommit_connections_are_rejected(tmpdir, autocommit):
) )
def test_legacy_transaction_control_connection_is_accepted(tmpdir): def test_legacy_transaction_control_connection_is_accepted(tmpdir):
conn = sqlite3.connect( conn = sqlite3.connect(
str(tmpdir / "test.db"), autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL str(tmpdir / "test.db"),
autocommit=sqlite3.LEGACY_TRANSACTION_CONTROL, # type: ignore[arg-type]
) )
db = Database(conn) db = Database(conn)
db["t"].insert({"id": 1}, pk="id") db.table("t").insert({"id": 1}, pk="id")
assert [r["id"] for r in db["t"].rows] == [1] assert [r["id"] for r in db.table("t").rows] == [1]
db.close() db.close()

View file

@ -1,17 +1,17 @@
def test_insert_conversion(fresh_db): def test_insert_conversion(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"foo": "bar"}, conversions={"foo": "upper(?)"}) table.insert({"foo": "bar"}, conversions={"foo": "upper(?)"})
assert [{"foo": "BAR"}] == list(table.rows) assert [{"foo": "BAR"}] == list(table.rows)
def test_insert_all_conversion(fresh_db): def test_insert_all_conversion(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert_all([{"foo": "bar"}], conversions={"foo": "upper(?)"}) table.insert_all([{"foo": "bar"}], conversions={"foo": "upper(?)"})
assert [{"foo": "BAR"}] == list(table.rows) assert [{"foo": "BAR"}] == list(table.rows)
def test_upsert_conversion(fresh_db): def test_upsert_conversion(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert({"id": 1, "foo": "bar"}, pk="id", conversions={"foo": "upper(?)"}) table.upsert({"id": 1, "foo": "bar"}, pk="id", conversions={"foo": "upper(?)"})
assert [{"id": 1, "foo": "BAR"}] == list(table.rows) assert [{"id": 1, "foo": "BAR"}] == list(table.rows)
table.upsert( table.upsert(
@ -21,7 +21,7 @@ def test_upsert_conversion(fresh_db):
def test_upsert_all_conversion(fresh_db): def test_upsert_all_conversion(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert_all( table.upsert_all(
[{"id": 1, "foo": "bar"}], pk="id", conversions={"foo": "upper(?)"} [{"id": 1, "foo": "bar"}], pk="id", conversions={"foo": "upper(?)"}
) )
@ -29,7 +29,7 @@ def test_upsert_all_conversion(fresh_db):
def test_update_conversion(fresh_db): def test_update_conversion(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"id": 5, "foo": "bar"}, pk="id") table.insert({"id": 5, "foo": "bar"}, pk="id")
table.update(5, {"foo": "baz"}, conversions={"foo": "upper(?)"}) table.update(5, {"foo": "baz"}, conversions={"foo": "upper(?)"})
assert [{"id": 5, "foo": "BAZ"}] == list(table.rows) assert [{"id": 5, "foo": "BAZ"}] == list(table.rows)

View file

@ -27,7 +27,7 @@ from sqlite_utils.db import BadMultiValues
), ),
) )
def test_convert(fresh_db, columns, fn, expected): def test_convert(fresh_db, columns, fn, expected):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"title": "Mixed Case", "abstract": "Abstract"}) table.insert({"title": "Mixed Case", "abstract": "Abstract"})
table.convert(columns, fn) table.convert(columns, fn)
assert list(table.rows) == [expected] assert list(table.rows) == [expected]
@ -37,7 +37,7 @@ def test_convert(fresh_db, columns, fn, expected):
"where,where_args", (("id > 1", None), ("id > :id", {"id": 1}), ("id > ?", [1])) "where,where_args", (("id > 1", None), ("id > :id", {"id": 1}), ("id > ?", [1]))
) )
def test_convert_where(fresh_db, where, where_args): def test_convert_where(fresh_db, where, where_args):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert_all( table.insert_all(
[ [
{"id": 1, "title": "One"}, {"id": 1, "title": "One"},
@ -53,7 +53,7 @@ def test_convert_where(fresh_db, where, where_args):
def test_convert_handles_falsey_values(fresh_db): def test_convert_handles_falsey_values(fresh_db):
# Falsey values like 0 should be converted (issue #527) # Falsey values like 0 should be converted (issue #527)
table = fresh_db["table"] table = fresh_db.table("table")
table.insert_all([{"x": 0}, {"x": 1}]) table.insert_all([{"x": 0}, {"x": 1}])
assert table.get(1)["x"] == 0 assert table.get(1)["x"] == 0
assert table.get(2)["x"] == 1 assert table.get(2)["x"] == 1
@ -70,14 +70,14 @@ def test_convert_handles_falsey_values(fresh_db):
), ),
) )
def test_convert_output(fresh_db, drop, expected): def test_convert_output(fresh_db, drop, expected):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"title": "Mixed Case"}) table.insert({"title": "Mixed Case"})
table.convert("title", lambda v: v.upper(), output="other", drop=drop) table.convert("title", lambda v: v.upper(), output="other", drop=drop)
assert list(table.rows) == [expected] assert list(table.rows) == [expected]
def test_convert_output_multiple_column_error(fresh_db): def test_convert_output_multiple_column_error(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
with pytest.raises(ValueError) as excinfo: with pytest.raises(ValueError) as excinfo:
table.convert(["title", "other"], lambda v: v, output="out") table.convert(["title", "other"], lambda v: v, output="out")
assert "output= can only be used with a single column" in str(excinfo.value) assert "output= can only be used with a single column" in str(excinfo.value)
@ -91,14 +91,14 @@ def test_convert_output_multiple_column_error(fresh_db):
), ),
) )
def test_convert_output_type(fresh_db, type, expected): def test_convert_output_type(fresh_db, type, expected):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"number": "123"}) table.insert({"number": "123"})
table.convert("number", lambda v: v, output="other", output_type=type, drop=True) table.convert("number", lambda v: v, output="other", output_type=type, drop=True)
assert list(table.rows) == [expected] assert list(table.rows) == [expected]
def test_convert_multi(fresh_db): def test_convert_multi(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"title": "Mixed Case"}) table.insert({"title": "Mixed Case"})
table.convert( table.convert(
"title", "title",
@ -123,7 +123,7 @@ def test_convert_multi(fresh_db):
def test_convert_multi_where(fresh_db): def test_convert_multi_where(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert_all( table.insert_all(
[ [
{"id": 1, "title": "One"}, {"id": 1, "title": "One"},
@ -145,14 +145,14 @@ def test_convert_multi_where(fresh_db):
def test_convert_multi_exception(fresh_db): def test_convert_multi_exception(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"title": "Mixed Case"}) table.insert({"title": "Mixed Case"})
with pytest.raises(BadMultiValues): with pytest.raises(BadMultiValues):
table.convert("title", lambda v: v.upper(), multi=True) table.convert("title", lambda v: v.upper(), multi=True)
def test_convert_repeated(fresh_db): def test_convert_repeated(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
col = "num" col = "num"
table.insert({col: 1}) table.insert({col: 1})
table.convert(col, lambda x: x * 2) table.convert(col, lambda x: x * 2)

File diff suppressed because it is too large Load diff

View file

@ -8,8 +8,12 @@ from sqlite_utils.create_table_parser import (
Check, Check,
ColumnComments, ColumnComments,
ParseError, ParseError,
Unique,
UniqueColumn,
parse_autoincrement,
parse_checks, parse_checks,
parse_column_comments, parse_column_comments,
parse_uniques,
) )
@ -117,6 +121,81 @@ def test_virtual_table_has_no_checks():
) )
@pytest.mark.parametrize(
"sql,expected",
[
(
"CREATE TABLE t(id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT)",
"id",
),
(
'CREATE TABLE t("quoted id" INTEGER PRIMARY KEY AUTOINCREMENT)',
"quoted id",
),
(
'CREATE TABLE t("autoincrement" INTEGER PRIMARY KEY, value TEXT)',
None,
),
(
"CREATE TABLE t(id INTEGER PRIMARY KEY /* AUTOINCREMENT */, value TEXT)",
None,
),
(
"CREATE TABLE t(id INTEGER PRIMARY KEY, value TEXT CHECK(value != 'AUTOINCREMENT'))",
None,
),
],
)
def test_parse_autoincrement(sql, expected):
sqlite3.connect(":memory:").execute(sql)
assert parse_autoincrement(sql) == expected
def test_parse_column_and_table_uniques():
sql = """
CREATE TABLE memberships (
email TEXT COLLATE RTRIM CONSTRAINT unique_email UNIQUE ON CONFLICT IGNORE,
account_id INTEGER,
CONSTRAINT unique_membership UNIQUE (
account_id DESC,
email COLLATE NOCASE ASC
) ON CONFLICT REPLACE
)
"""
sqlite3.connect(":memory:").execute(sql)
assert parse_uniques(sql) == [
Unique(
(UniqueColumn("email", collation="RTRIM"),),
name="unique_email",
column="email",
conflict="IGNORE",
),
Unique(
(
UniqueColumn("account_id", order="DESC"),
UniqueColumn("email", collation="NOCASE", order="ASC"),
),
name="unique_membership",
conflict="REPLACE",
),
]
uniques = parse_uniques(sql)
assert uniques[0].sql == "CONSTRAINT unique_email UNIQUE ON CONFLICT IGNORE"
assert sql[uniques[1].start : uniques[1].end] == uniques[1].sql
def test_unique_like_text_in_comments_and_checks_is_ignored():
sql = """
CREATE TABLE t (
value TEXT /* UNIQUE ON CONFLICT REPLACE */
CHECK(value != 'UNIQUE(other)'),
other TEXT
)
"""
sqlite3.connect(":memory:").execute(sql)
assert parse_uniques(sql) == []
comment_or_space = st.sampled_from( comment_or_space = st.sampled_from(
[ [
" ", " ",

View file

@ -32,9 +32,9 @@ EXAMPLES = [
@pytest.mark.parametrize("column_def,initial_value,expected_value", EXAMPLES) @pytest.mark.parametrize("column_def,initial_value,expected_value", EXAMPLES)
def test_quote_default_value(fresh_db, column_def, initial_value, expected_value): def test_quote_default_value(fresh_db, column_def, initial_value, expected_value):
fresh_db.execute(f"create table foo (col {column_def})") fresh_db.execute(f"create table foo (col {column_def})")
assert initial_value == fresh_db["foo"].columns[0].default_value assert initial_value == fresh_db.table("foo").columns[0].default_value
assert expected_value == fresh_db.quote_default_value( assert expected_value == fresh_db.quote_default_value(
fresh_db["foo"].columns[0].default_value fresh_db.table("foo").columns[0].default_value
) )
@ -48,7 +48,7 @@ def test_insert_empty_record_uses_default_values(fresh_db):
) )
""") """)
table = fresh_db["has_defaults"] table = fresh_db.table("has_defaults")
table.insert({}) table.insert({})
rows = list(table.rows) rows = list(table.rows)

View file

@ -2,7 +2,7 @@ import sqlite_utils
def test_delete_rowid_table(fresh_db): def test_delete_rowid_table(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"foo": 1}) table.insert({"foo": 1})
rowid = table.insert({"foo": 2}).last_pk rowid = table.insert({"foo": 2}).last_pk
table.delete(rowid) table.delete(rowid)
@ -10,7 +10,7 @@ def test_delete_rowid_table(fresh_db):
def test_delete_pk_table(fresh_db): def test_delete_pk_table(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"id": 1}, pk="id") table.insert({"id": 1}, pk="id")
table.insert({"id": 2}, pk="id") table.insert({"id": 2}, pk="id")
table.delete(1) table.delete(1)
@ -18,7 +18,7 @@ def test_delete_pk_table(fresh_db):
def test_delete_where(fresh_db): def test_delete_where(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
for i in range(1, 11): for i in range(1, 11):
table.insert({"id": i}, pk="id") table.insert({"id": i}, pk="id")
assert table.count == 10 assert table.count == 10
@ -27,7 +27,7 @@ def test_delete_where(fresh_db):
def test_delete_where_all(fresh_db): def test_delete_where_all(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
for i in range(1, 11): for i in range(1, 11):
table.insert({"id": i}, pk="id") table.insert({"id": i}, pk="id")
assert table.count == 10 assert table.count == 10
@ -38,27 +38,27 @@ def test_delete_where_all(fresh_db):
def test_delete_where_commits(tmpdir): def test_delete_where_commits(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = sqlite_utils.Database(path) db = sqlite_utils.Database(path)
db["table"].insert_all([{"id": i} for i in range(5)], pk="id") db.table("table").insert_all([{"id": i} for i in range(5)], pk="id")
db["table"].delete_where("id > ?", [2]) db.table("table").delete_where("id > ?", [2])
# The connection must not be left inside an open transaction, # The connection must not be left inside an open transaction,
# otherwise subsequent atomic() blocks never commit either # otherwise subsequent atomic() blocks never commit either
assert not db.conn.in_transaction assert not db.conn.in_transaction
db["table"].insert({"id": 100}) db.table("table").insert({"id": 100})
db.close() db.close()
db2 = sqlite_utils.Database(path) db2 = sqlite_utils.Database(path)
assert [r["id"] for r in db2["table"].rows] == [0, 1, 2, 100] assert [r["id"] for r in db2.table("table").rows] == [0, 1, 2, 100]
db2.close() db2.close()
def test_delete_where_analyze(fresh_db): def test_delete_where_analyze(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert_all(({"id": i, "i": i} for i in range(10)), pk="id") table.insert_all(({"id": i, "i": i} for i in range(10)), pk="id")
table.create_index(["i"], analyze=True) table.create_index(["i"], analyze=True)
assert "sqlite_stat1" in fresh_db.table_names() assert "sqlite_stat1" in fresh_db.table_names()
assert list(fresh_db["sqlite_stat1"].rows) == [ assert list(fresh_db.table("sqlite_stat1").rows) == [
{"tbl": "table", "idx": "idx_table_i", "stat": "10 1"} {"tbl": "table", "idx": "idx_table_i", "stat": "10 1"}
] ]
table.delete_where("id > ?", [5], analyze=True) table.delete_where("id > ?", [5], analyze=True)
assert list(fresh_db["sqlite_stat1"].rows) == [ assert list(fresh_db.table("sqlite_stat1").rows) == [
{"tbl": "table", "idx": "idx_table_i", "stat": "6 1"} {"tbl": "table", "idx": "idx_table_i", "stat": "6 1"}
] ]

View file

@ -22,7 +22,7 @@ def test_duplicate(fresh_db):
"bool_col": True, "bool_col": True,
"datetime_col": str(dt), "datetime_col": str(dt),
} }
table1 = fresh_db["table1"] table1 = fresh_db.table("table1")
row_id = table1.insert(data).last_rowid row_id = table1.insert(data).last_rowid
# Duplicate table: # Duplicate table:
table2 = table1.duplicate("table2") table2 = table1.duplicate("table2")
@ -40,4 +40,4 @@ def test_duplicate(fresh_db):
def test_duplicate_fails_if_table_does_not_exist(fresh_db): def test_duplicate_fails_if_table_does_not_exist(fresh_db):
with pytest.raises(NoTable): with pytest.raises(NoTable):
fresh_db["not_a_table"].duplicate("duplicated") fresh_db.table("not_a_table").duplicate("duplicated")

View file

@ -5,7 +5,7 @@ from sqlite_utils import Database, cli
def test_enable_counts_specific_table(fresh_db): def test_enable_counts_specific_table(fresh_db):
foo = fresh_db["foo"] foo = fresh_db.table("foo")
assert fresh_db.table_names() == [] assert fresh_db.table_names() == []
for i in range(10): for i in range(10):
foo.insert({"name": f"item {i}"}) foo.insert({"name": f"item {i}"})
@ -41,24 +41,24 @@ def test_enable_counts_specific_table(fresh_db):
), ),
} }
assert fresh_db.table_names() == ["foo", "_counts"] assert fresh_db.table_names() == ["foo", "_counts"]
assert list(fresh_db["_counts"].rows) == [{"count": 10, "table": "foo"}] assert list(fresh_db.table("_counts").rows) == [{"count": 10, "table": "foo"}]
# Add some items to test the triggers # Add some items to test the triggers
for i in range(5): for i in range(5):
foo.insert({"name": f"item {10 + i}"}) foo.insert({"name": f"item {10 + i}"})
assert foo.count == 15 assert foo.count == 15
assert list(fresh_db["_counts"].rows) == [{"count": 15, "table": "foo"}] assert list(fresh_db.table("_counts").rows) == [{"count": 15, "table": "foo"}]
# Delete some items # Delete some items
foo.delete_where("rowid < 7") foo.delete_where("rowid < 7")
assert foo.count == 9 assert foo.count == 9
assert list(fresh_db["_counts"].rows) == [{"count": 9, "table": "foo"}] assert list(fresh_db.table("_counts").rows) == [{"count": 9, "table": "foo"}]
foo.delete_where() foo.delete_where()
assert foo.count == 0 assert foo.count == 0
assert list(fresh_db["_counts"].rows) == [{"count": 0, "table": "foo"}] assert list(fresh_db.table("_counts").rows) == [{"count": 0, "table": "foo"}]
def test_enable_counts_all_tables(fresh_db): def test_enable_counts_all_tables(fresh_db):
foo = fresh_db["foo"] foo = fresh_db.table("foo")
bar = fresh_db["bar"] bar = fresh_db.table("bar")
foo.insert({"name": "Cleo"}) foo.insert({"name": "Cleo"})
bar.insert({"name": "Cleo"}) bar.insert({"name": "Cleo"})
foo.enable_fts(["name"]) foo.enable_fts(["name"])
@ -73,7 +73,7 @@ def test_enable_counts_all_tables(fresh_db):
"foo_fts_config", "foo_fts_config",
"_counts", "_counts",
} }
assert list(fresh_db["_counts"].rows) == [ assert list(fresh_db.table("_counts").rows) == [
{"count": 1, "table": "foo"}, {"count": 1, "table": "foo"},
{"count": 1, "table": "bar"}, {"count": 1, "table": "bar"},
{"count": 3, "table": "foo_fts_data"}, {"count": 3, "table": "foo_fts_data"},
@ -87,10 +87,10 @@ def test_enable_counts_all_tables(fresh_db):
def counts_db_path(tmpdir): def counts_db_path(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["foo"].insert({"name": "bar"}) db.table("foo").insert({"name": "bar"})
db["bar"].insert({"name": "bar"}) db.table("bar").insert({"name": "bar"})
db["bar"].insert({"name": "bar"}) db.table("bar").insert({"name": "bar"})
db["baz"].insert({"name": "bar"}) db.table("baz").insert({"name": "bar"})
return path return path
@ -163,25 +163,25 @@ def test_uses_counts_after_enable_counts(counts_db_path):
def test_reset_counts(counts_db_path): def test_reset_counts(counts_db_path):
db = Database(counts_db_path) db = Database(counts_db_path)
db["foo"].enable_counts() db.table("foo").enable_counts()
db["bar"].enable_counts() db.table("bar").enable_counts()
assert db.cached_counts() == {"foo": 1, "bar": 2} assert db.cached_counts() == {"foo": 1, "bar": 2}
# Corrupt the value # Corrupt the value
db["_counts"].update("foo", {"count": 3}) db.table("_counts").update("foo", {"count": 3})
assert db.cached_counts() == {"foo": 3, "bar": 2} assert db.cached_counts() == {"foo": 3, "bar": 2}
assert db["foo"].count == 3 assert db.table("foo").count == 3
# Reset them # Reset them
db.reset_counts() db.reset_counts()
assert db.cached_counts() == {"foo": 1, "bar": 2} assert db.cached_counts() == {"foo": 1, "bar": 2}
assert db["foo"].count == 1 assert db.table("foo").count == 1
def test_reset_counts_cli(counts_db_path): def test_reset_counts_cli(counts_db_path):
db = Database(counts_db_path) db = Database(counts_db_path)
db["foo"].enable_counts() db.table("foo").enable_counts()
db["bar"].enable_counts() db.table("bar").enable_counts()
assert db.cached_counts() == {"foo": 1, "bar": 2} assert db.cached_counts() == {"foo": 1, "bar": 2}
db["_counts"].update("foo", {"count": 3}) db.table("_counts").update("foo", {"count": 3})
result = CliRunner().invoke(cli.cli, ["reset-counts", counts_db_path]) result = CliRunner().invoke(cli.cli, ["reset-counts", counts_db_path])
assert result.exit_code == 0 assert result.exit_code == 0
assert db.cached_counts() == {"foo": 1, "bar": 2} assert db.cached_counts() == {"foo": 1, "bar": 2}

View file

@ -2,6 +2,7 @@ import itertools
import pytest import pytest
from sqlite_utils import ANY
from sqlite_utils.db import InvalidColumns from sqlite_utils.db import InvalidColumns
@ -11,7 +12,7 @@ def test_extract_single_column(fresh_db, table, fk_column):
expected_table = table or "species" expected_table = table or "species"
expected_fk = fk_column or f"{expected_table}_id" expected_fk = fk_column or f"{expected_table}_id"
iter_species = itertools.cycle(["Palm", "Spruce", "Mangrove", "Oak"]) iter_species = itertools.cycle(["Palm", "Spruce", "Mangrove", "Oak"])
fresh_db["tree"].insert_all( fresh_db.table("tree").insert_all(
( (
{ {
"id": i, "id": i,
@ -23,8 +24,8 @@ def test_extract_single_column(fresh_db, table, fk_column):
), ),
pk="id", pk="id",
) )
fresh_db["tree"].extract("species", table=table, fk_column=fk_column) fresh_db.table("tree").extract("species", table=table, fk_column=fk_column)
assert fresh_db["tree"].schema == ( assert fresh_db.table("tree").schema == (
'CREATE TABLE "tree" (\n' 'CREATE TABLE "tree" (\n'
' "id" INTEGER PRIMARY KEY,\n' ' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
@ -32,18 +33,18 @@ def test_extract_single_column(fresh_db, table, fk_column):
+ ' "end" INTEGER\n' + ' "end" INTEGER\n'
+ ")" + ")"
) )
assert fresh_db[expected_table].schema == ( assert fresh_db.table(expected_table).schema == (
f'CREATE TABLE "{expected_table}" (\n' + ' "id" INTEGER PRIMARY KEY,\n' f'CREATE TABLE "{expected_table}" (\n' + ' "id" INTEGER PRIMARY KEY,\n'
' "species" TEXT\n' ' "species" TEXT\n'
")" ")"
) )
assert list(fresh_db[expected_table].rows) == [ assert list(fresh_db.table(expected_table).rows) == [
{"id": 1, "species": "Palm"}, {"id": 1, "species": "Palm"},
{"id": 2, "species": "Spruce"}, {"id": 2, "species": "Spruce"},
{"id": 3, "species": "Mangrove"}, {"id": 3, "species": "Mangrove"},
{"id": 4, "species": "Oak"}, {"id": 4, "species": "Oak"},
] ]
assert list(itertools.islice(fresh_db["tree"].rows, 0, 4)) == [ assert list(itertools.islice(fresh_db.table("tree").rows, 0, 4)) == [
{"id": 1, "name": "Tree 1", expected_fk: 1, "end": 1}, {"id": 1, "name": "Tree 1", expected_fk: 1, "end": 1},
{"id": 2, "name": "Tree 2", expected_fk: 2, "end": 1}, {"id": 2, "name": "Tree 2", expected_fk: 2, "end": 1},
{"id": 3, "name": "Tree 3", expected_fk: 3, "end": 1}, {"id": 3, "name": "Tree 3", expected_fk: 3, "end": 1},
@ -54,7 +55,7 @@ def test_extract_single_column(fresh_db, table, fk_column):
def test_extract_multiple_columns_with_rename(fresh_db): def test_extract_multiple_columns_with_rename(fresh_db):
iter_common = itertools.cycle(["Palm", "Spruce", "Mangrove", "Oak"]) iter_common = itertools.cycle(["Palm", "Spruce", "Mangrove", "Oak"])
iter_latin = itertools.cycle(["Arecaceae", "Picea", "Rhizophora", "Quercus"]) iter_latin = itertools.cycle(["Arecaceae", "Picea", "Rhizophora", "Quercus"])
fresh_db["tree"].insert_all( fresh_db.table("tree").insert_all(
( (
{ {
"id": i, "id": i,
@ -67,30 +68,30 @@ def test_extract_multiple_columns_with_rename(fresh_db):
pk="id", pk="id",
) )
fresh_db["tree"].extract( fresh_db.table("tree").extract(
["common_name", "latin_name"], rename={"common_name": "name"} ["common_name", "latin_name"], rename={"common_name": "name"}
) )
assert fresh_db["tree"].schema == ( assert fresh_db.table("tree").schema == (
'CREATE TABLE "tree" (\n' 'CREATE TABLE "tree" (\n'
' "id" INTEGER PRIMARY KEY,\n' ' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
' "common_name_latin_name_id" INTEGER REFERENCES "common_name_latin_name"("id")\n' ' "common_name_latin_name_id" INTEGER REFERENCES "common_name_latin_name"("id")\n'
")" ")"
) )
assert fresh_db["common_name_latin_name"].schema == ( assert fresh_db.table("common_name_latin_name").schema == (
'CREATE TABLE "common_name_latin_name" (\n' 'CREATE TABLE "common_name_latin_name" (\n'
' "id" INTEGER PRIMARY KEY,\n' ' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
' "latin_name" TEXT\n' ' "latin_name" TEXT\n'
")" ")"
) )
assert list(fresh_db["common_name_latin_name"].rows) == [ assert list(fresh_db.table("common_name_latin_name").rows) == [
{"name": "Palm", "id": 1, "latin_name": "Arecaceae"}, {"name": "Palm", "id": 1, "latin_name": "Arecaceae"},
{"name": "Spruce", "id": 2, "latin_name": "Picea"}, {"name": "Spruce", "id": 2, "latin_name": "Picea"},
{"name": "Mangrove", "id": 3, "latin_name": "Rhizophora"}, {"name": "Mangrove", "id": 3, "latin_name": "Rhizophora"},
{"name": "Oak", "id": 4, "latin_name": "Quercus"}, {"name": "Oak", "id": 4, "latin_name": "Quercus"},
] ]
assert list(itertools.islice(fresh_db["tree"].rows, 0, 4)) == [ assert list(itertools.islice(fresh_db.table("tree").rows, 0, 4)) == [
{"id": 1, "name": "Tree 1", "common_name_latin_name_id": 1}, {"id": 1, "name": "Tree 1", "common_name_latin_name_id": 1},
{"id": 2, "name": "Tree 2", "common_name_latin_name_id": 2}, {"id": 2, "name": "Tree 2", "common_name_latin_name_id": 2},
{"id": 3, "name": "Tree 3", "common_name_latin_name_id": 3}, {"id": 3, "name": "Tree 3", "common_name_latin_name_id": 3},
@ -99,7 +100,7 @@ def test_extract_multiple_columns_with_rename(fresh_db):
def test_extract_invalid_columns(fresh_db): def test_extract_invalid_columns(fresh_db):
fresh_db["tree"].insert( fresh_db.table("tree").insert(
{ {
"id": 1, "id": 1,
"name": "Tree 1", "name": "Tree 1",
@ -109,19 +110,19 @@ def test_extract_invalid_columns(fresh_db):
pk="id", pk="id",
) )
with pytest.raises(InvalidColumns): with pytest.raises(InvalidColumns):
fresh_db["tree"].extract(["bad_column"]) fresh_db.table("tree").extract(["bad_column"])
def test_extract_rowid_table(fresh_db): def test_extract_rowid_table(fresh_db):
fresh_db["tree"].insert( fresh_db.table("tree").insert(
{ {
"name": "Tree 1", "name": "Tree 1",
"common_name": "Palm", "common_name": "Palm",
"latin_name": "Arecaceae", "latin_name": "Arecaceae",
} }
) )
fresh_db["tree"].extract(["common_name", "latin_name"]) fresh_db.table("tree").extract(["common_name", "latin_name"])
assert fresh_db["tree"].schema == ( assert fresh_db.table("tree").schema == (
'CREATE TABLE "tree" (\n' 'CREATE TABLE "tree" (\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
' "common_name_latin_name_id" INTEGER REFERENCES "common_name_latin_name"("id")\n' ' "common_name_latin_name_id" INTEGER REFERENCES "common_name_latin_name"("id")\n'
@ -139,68 +140,68 @@ def test_extract_rowid_table(fresh_db):
def test_reuse_lookup_table(fresh_db): def test_reuse_lookup_table(fresh_db):
fresh_db["species"].insert({"id": 1, "name": "Wolf"}, pk="id") fresh_db.table("species").insert({"id": 1, "name": "Wolf"}, pk="id")
fresh_db["sightings"].insert({"id": 10, "species": "Wolf"}, pk="id") fresh_db.table("sightings").insert({"id": 10, "species": "Wolf"}, pk="id")
fresh_db["individuals"].insert( fresh_db.table("individuals").insert(
{"id": 10, "name": "Terriana", "species": "Fox"}, pk="id" {"id": 10, "name": "Terriana", "species": "Fox"}, pk="id"
) )
fresh_db["sightings"].extract("species", rename={"species": "name"}) fresh_db.table("sightings").extract("species", rename={"species": "name"})
fresh_db["individuals"].extract("species", rename={"species": "name"}) fresh_db.table("individuals").extract("species", rename={"species": "name"})
assert fresh_db["sightings"].schema == ( assert fresh_db.table("sightings").schema == (
'CREATE TABLE "sightings" (\n' 'CREATE TABLE "sightings" (\n'
' "id" INTEGER PRIMARY KEY,\n' ' "id" INTEGER PRIMARY KEY,\n'
' "species_id" INTEGER REFERENCES "species"("id")\n' ' "species_id" INTEGER REFERENCES "species"("id")\n'
")" ")"
) )
assert fresh_db["individuals"].schema == ( assert fresh_db.table("individuals").schema == (
'CREATE TABLE "individuals" (\n' 'CREATE TABLE "individuals" (\n'
' "id" INTEGER PRIMARY KEY,\n' ' "id" INTEGER PRIMARY KEY,\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
' "species_id" INTEGER REFERENCES "species"("id")\n' ' "species_id" INTEGER REFERENCES "species"("id")\n'
")" ")"
) )
assert list(fresh_db["species"].rows) == [ assert list(fresh_db.table("species").rows) == [
{"id": 1, "name": "Wolf"}, {"id": 1, "name": "Wolf"},
{"id": 2, "name": "Fox"}, {"id": 2, "name": "Fox"},
] ]
def test_extract_error_on_incompatible_existing_lookup_table(fresh_db): def test_extract_error_on_incompatible_existing_lookup_table(fresh_db):
fresh_db["species"].insert({"id": 1}) fresh_db.table("species").insert({"id": 1})
fresh_db["tree"].insert({"name": "Tree 1", "common_name": "Palm"}) fresh_db.table("tree").insert({"name": "Tree 1", "common_name": "Palm"})
with pytest.raises(InvalidColumns): with pytest.raises(InvalidColumns):
fresh_db["tree"].extract("common_name", table="species") fresh_db.table("tree").extract("common_name", table="species")
# Try again with incompatible existing column type # Try again with incompatible existing column type
fresh_db["species2"].insert({"id": 1, "common_name": 3.5}) fresh_db.table("species2").insert({"id": 1, "common_name": 3.5})
with pytest.raises(InvalidColumns): with pytest.raises(InvalidColumns):
fresh_db["tree"].extract("common_name", table="species2") fresh_db.table("tree").extract("common_name", table="species2")
def test_extract_works_with_null_values(fresh_db): def test_extract_works_with_null_values(fresh_db):
fresh_db["listens"].insert_all( fresh_db.table("listens").insert_all(
[ [
{"id": 1, "track_title": "foo", "album_title": "bar"}, {"id": 1, "track_title": "foo", "album_title": "bar"},
{"id": 2, "track_title": "baz", "album_title": None}, {"id": 2, "track_title": "baz", "album_title": None},
], ],
pk="id", pk="id",
) )
fresh_db["listens"].extract( fresh_db.table("listens").extract(
columns=["album_title"], table="albums", fk_column="album_id" columns=["album_title"], table="albums", fk_column="album_id"
) )
assert list(fresh_db["listens"].rows) == [ assert list(fresh_db.table("listens").rows) == [
{"id": 1, "track_title": "foo", "album_id": 1}, {"id": 1, "track_title": "foo", "album_id": 1},
{"id": 2, "track_title": "baz", "album_id": None}, {"id": 2, "track_title": "baz", "album_id": None},
] ]
assert list(fresh_db["albums"].rows) == [ assert list(fresh_db.table("albums").rows) == [
{"id": 1, "album_title": "bar"}, {"id": 1, "album_title": "bar"},
] ]
def test_extract_null_values_single_column(fresh_db): def test_extract_null_values_single_column(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/186 # https://github.com/simonw/sqlite-utils/issues/186
fresh_db["species"].insert({"id": 1, "species": "Wolf"}, pk="id") fresh_db.table("species").insert({"id": 1, "species": "Wolf"}, pk="id")
fresh_db["individuals"].insert_all( fresh_db.table("individuals").insert_all(
[ [
{"id": 10, "name": "Terriana", "species": "Fox"}, {"id": 10, "name": "Terriana", "species": "Fox"},
{"id": 11, "name": "Spenidorm", "species": None}, {"id": 11, "name": "Spenidorm", "species": None},
@ -210,13 +211,13 @@ def test_extract_null_values_single_column(fresh_db):
], ],
pk="id", pk="id",
) )
fresh_db["individuals"].extract("species") fresh_db.table("individuals").extract("species")
# No null row should have been added to species # No null row should have been added to species
assert list(fresh_db["species"].rows) == [ assert list(fresh_db.table("species").rows) == [
{"id": 1, "species": "Wolf"}, {"id": 1, "species": "Wolf"},
{"id": 2, "species": "Fox"}, {"id": 2, "species": "Fox"},
] ]
assert list(fresh_db["individuals"].rows) == [ assert list(fresh_db.table("individuals").rows) == [
{"id": 10, "name": "Terriana", "species_id": 2}, {"id": 10, "name": "Terriana", "species_id": 2},
{"id": 11, "name": "Spenidorm", "species_id": None}, {"id": 11, "name": "Spenidorm", "species_id": None},
{"id": 12, "name": "Grantheim", "species_id": 1}, {"id": 12, "name": "Grantheim", "species_id": 1},
@ -228,7 +229,7 @@ def test_extract_null_values_single_column(fresh_db):
def test_extract_null_values_multiple_columns(fresh_db): def test_extract_null_values_multiple_columns(fresh_db):
# A row should be extracted if at least one column is not null - # A row should be extracted if at least one column is not null -
# only rows where ALL extracted columns are null are left alone # only rows where ALL extracted columns are null are left alone
fresh_db["circulation"].insert_all( fresh_db.table("circulation").insert_all(
[ [
{"id": 1, "title": "title one", "creator": "creator one", "year": 2018}, {"id": 1, "title": "title one", "creator": "creator one", "year": 2018},
{"id": 2, "title": "title two", "creator": None, "year": 2019}, {"id": 2, "title": "title two", "creator": None, "year": 2019},
@ -237,14 +238,14 @@ def test_extract_null_values_multiple_columns(fresh_db):
], ],
pk="id", pk="id",
) )
fresh_db["circulation"].extract( fresh_db.table("circulation").extract(
["title", "creator"], table="books", fk_column="book_id" ["title", "creator"], table="books", fk_column="book_id"
) )
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "title": "title one", "creator": "creator one"}, {"id": 1, "title": "title one", "creator": "creator one"},
{"id": 2, "title": "title two", "creator": None}, {"id": 2, "title": "title two", "creator": None},
] ]
assert list(fresh_db["circulation"].rows) == [ assert list(fresh_db.table("circulation").rows) == [
{"id": 1, "book_id": 1, "year": 2018}, {"id": 1, "book_id": 1, "year": 2018},
{"id": 2, "book_id": 2, "year": 2019}, {"id": 2, "book_id": 2, "year": 2019},
{"id": 3, "book_id": None, "year": 2020}, {"id": 3, "book_id": None, "year": 2020},
@ -255,20 +256,20 @@ def test_extract_null_values_multiple_columns(fresh_db):
def test_extract_null_values_existing_lookup_table_with_null_row(fresh_db): def test_extract_null_values_existing_lookup_table_with_null_row(fresh_db):
# Even if the lookup table already contains an all-null row, rows where # Even if the lookup table already contains an all-null row, rows where
# every extracted column is null should keep a null foreign key # every extracted column is null should keep a null foreign key
fresh_db["species"].insert({"id": 1, "species": None}, pk="id") fresh_db.table("species").insert({"id": 1, "species": None}, pk="id")
fresh_db["individuals"].insert_all( fresh_db.table("individuals").insert_all(
[ [
{"id": 10, "name": "Terriana", "species": "Fox"}, {"id": 10, "name": "Terriana", "species": "Fox"},
{"id": 11, "name": "Spenidorm", "species": None}, {"id": 11, "name": "Spenidorm", "species": None},
], ],
pk="id", pk="id",
) )
fresh_db["individuals"].extract("species") fresh_db.table("individuals").extract("species")
assert list(fresh_db["species"].rows) == [ assert list(fresh_db.table("species").rows) == [
{"id": 1, "species": None}, {"id": 1, "species": None},
{"id": 2, "species": "Fox"}, {"id": 2, "species": "Fox"},
] ]
assert list(fresh_db["individuals"].rows) == [ assert list(fresh_db.table("individuals").rows) == [
{"id": 10, "name": "Terriana", "species_id": 2}, {"id": 10, "name": "Terriana", "species_id": 2},
{"id": 11, "name": "Spenidorm", "species_id": None}, {"id": 11, "name": "Spenidorm", "species_id": None},
] ]
@ -279,17 +280,19 @@ def test_extract_repeated_into_shared_lookup_with_nulls(fresh_db):
# cannot dedupe NULL-containing rows against the existing lookup # cannot dedupe NULL-containing rows against the existing lookup
# table - extracting a second table into the same lookup previously # table - extracting a second table into the same lookup previously
# inserted duplicate rows that nothing pointed to # inserted duplicate rows that nothing pointed to
fresh_db["t1"].insert_all( fresh_db.table("t1").insert_all(
[ [
{"id": 1, "species": None, "common": "X"}, {"id": 1, "species": None, "common": "X"},
{"id": 2, "species": "Oak", "common": "Oak"}, {"id": 2, "species": "Oak", "common": "Oak"},
], ],
pk="id", pk="id",
) )
fresh_db["t2"].insert_all([{"id": 1, "species": None, "common": "X"}], pk="id") fresh_db.table("t2").insert_all(
fresh_db["t1"].extract(["species", "common"], table="lk") [{"id": 1, "species": None, "common": "X"}], pk="id"
fresh_db["t2"].extract(["species", "common"], table="lk") )
assert fresh_db["lk"].count == 2 fresh_db.table("t1").extract(["species", "common"], table="lk")
fresh_db.table("t2").extract(["species", "common"], table="lk")
assert fresh_db.table("lk").count == 2
# Both tables point at the same lookup row # Both tables point at the same lookup row
t1_fk = fresh_db.execute("select lk_id from t1 where id = 1").fetchone()[0] t1_fk = fresh_db.execute("select lk_id from t1 where id = 1").fetchone()[0]
t2_fk = fresh_db.execute("select lk_id from t2 where id = 1").fetchone()[0] t2_fk = fresh_db.execute("select lk_id from t2 where id = 1").fetchone()[0]
@ -298,8 +301,43 @@ def test_extract_repeated_into_shared_lookup_with_nulls(fresh_db):
def test_extract_repeated_into_shared_lookup_no_nulls(fresh_db): def test_extract_repeated_into_shared_lookup_no_nulls(fresh_db):
# Non-NULL rows were already deduped by the unique index - keep it so # Non-NULL rows were already deduped by the unique index - keep it so
fresh_db["t1"].insert_all([{"id": 1, "species": "Oak"}], pk="id") fresh_db.table("t1").insert_all([{"id": 1, "species": "Oak"}], pk="id")
fresh_db["t2"].insert_all([{"id": 1, "species": "Oak"}], pk="id") fresh_db.table("t2").insert_all([{"id": 1, "species": "Oak"}], pk="id")
fresh_db["t1"].extract(["species"], table="lk") fresh_db.table("t1").extract(["species"], table="lk")
fresh_db["t2"].extract(["species"], table="lk") fresh_db.table("t2").extract(["species"], table="lk")
assert fresh_db["lk"].count == 1 assert fresh_db.table("lk").count == 1
def test_extract_preserves_strict_any(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (id integer primary key, data any) strict")
fresh_db.execute("insert into items values (1, ?)", ("000123",))
fresh_db["items"].extract("data", table="data_values")
lookup = fresh_db["data_values"]
assert lookup.strict is True
assert lookup.columns_dict == {"id": int, "data": ANY}
assert fresh_db.execute(
"select typeof(data), data from data_values"
).fetchone() == ("text", "000123")
def test_extract_strict_any_rejects_non_strict_lookup(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (data any) strict")
fresh_db.execute("insert into items values (?)", ("000123",))
fresh_db.execute("create table data_values (id integer primary key, data any)")
with pytest.raises(
InvalidColumns,
match="is not STRICT, so it cannot preserve ANY column values",
):
fresh_db["items"].extract("data", table="data_values")
assert fresh_db.execute("select typeof(data), data from items").fetchone() == (
"text",
"000123",
)

View file

@ -32,15 +32,15 @@ def test_extracts(fresh_db, kwargs, expected_table, use_table_factory):
assert {expected_table, "Trees"} == set(fresh_db.table_names()) assert {expected_table, "Trees"} == set(fresh_db.table_names())
assert ( assert (
f'CREATE TABLE "{expected_table}" (\n "id" INTEGER PRIMARY KEY,\n "value" TEXT\n)' f'CREATE TABLE "{expected_table}" (\n "id" INTEGER PRIMARY KEY,\n "value" TEXT\n)'
== fresh_db[expected_table].schema == fresh_db.table(expected_table).schema
) )
assert ( assert (
f'CREATE TABLE "Trees" (\n "id" INTEGER,\n "species_id" INTEGER REFERENCES "{expected_table}"("id")\n)' f'CREATE TABLE "Trees" (\n "id" INTEGER,\n "species_id" INTEGER REFERENCES "{expected_table}"("id")\n)'
== fresh_db["Trees"].schema == fresh_db.table("Trees").schema
) )
# Should have a foreign key reference # Should have a foreign key reference
assert len(fresh_db["Trees"].foreign_keys) == 1 assert len(fresh_db.table("Trees").foreign_keys) == 1
fk = fresh_db["Trees"].foreign_keys[0] fk = fresh_db.table("Trees").foreign_keys[0]
assert fk.table == "Trees" assert fk.table == "Trees"
assert fk.column == "species_id" assert fk.column == "species_id"
@ -54,22 +54,22 @@ def test_extracts(fresh_db, kwargs, expected_table, use_table_factory):
partial=0, partial=0,
columns=["value"], columns=["value"],
) )
] == fresh_db[expected_table].indexes ] == fresh_db.table(expected_table).indexes
# Finally, check the rows # Finally, check the rows
assert [{"id": 1, "value": "Oak"}, {"id": 2, "value": "Palm"}] == list( assert [{"id": 1, "value": "Oak"}, {"id": 2, "value": "Palm"}] == list(
fresh_db[expected_table].rows fresh_db.table(expected_table).rows
) )
assert [ assert [
{"id": 1, "species_id": 1}, {"id": 1, "species_id": 1},
{"id": 2, "species_id": 1}, {"id": 2, "species_id": 1},
{"id": 3, "species_id": 2}, {"id": 3, "species_id": 2},
] == list(fresh_db["Trees"].rows) ] == list(fresh_db.table("Trees").rows)
def test_extracts_null_values(fresh_db): def test_extracts_null_values(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/186 # https://github.com/simonw/sqlite-utils/issues/186
# Null values should stay null, not be extracted into the lookup table # Null values should stay null, not be extracted into the lookup table
fresh_db["Trees"].insert_all( fresh_db.table("Trees").insert_all(
[ [
{"id": 1, "species_id": "Oak"}, {"id": 1, "species_id": "Oak"},
{"id": 2, "species_id": None}, {"id": 2, "species_id": None},
@ -78,11 +78,11 @@ def test_extracts_null_values(fresh_db):
], ],
extracts={"species_id": "Species"}, extracts={"species_id": "Species"},
) )
assert list(fresh_db["Species"].rows) == [ assert list(fresh_db.table("Species").rows) == [
{"id": 1, "value": "Oak"}, {"id": 1, "value": "Oak"},
{"id": 2, "value": "Palm"}, {"id": 2, "value": "Palm"},
] ]
assert list(fresh_db["Trees"].rows) == [ assert list(fresh_db.table("Trees").rows) == [
{"id": 1, "species_id": 1}, {"id": 1, "species_id": 1},
{"id": 2, "species_id": None}, {"id": 2, "species_id": None},
{"id": 3, "species_id": 2}, {"id": 3, "species_id": 2},
@ -92,7 +92,7 @@ def test_extracts_null_values(fresh_db):
def test_extracts_null_values_list_mode(fresh_db): def test_extracts_null_values_list_mode(fresh_db):
# Same as test_extracts_null_values but for list-based records # Same as test_extracts_null_values but for list-based records
fresh_db["Trees"].insert_all( fresh_db.table("Trees").insert_all(
[ [
["id", "species_id"], ["id", "species_id"],
[1, "Oak"], [1, "Oak"],
@ -102,11 +102,11 @@ def test_extracts_null_values_list_mode(fresh_db):
], ],
extracts={"species_id": "Species"}, extracts={"species_id": "Species"},
) )
assert list(fresh_db["Species"].rows) == [ assert list(fresh_db.table("Species").rows) == [
{"id": 1, "value": "Oak"}, {"id": 1, "value": "Oak"},
{"id": 2, "value": "Palm"}, {"id": 2, "value": "Palm"},
] ]
assert list(fresh_db["Trees"].rows) == [ assert list(fresh_db.table("Trees").rows) == [
{"id": 1, "species_id": 1}, {"id": 1, "species_id": 1},
{"id": 2, "species_id": None}, {"id": 2, "species_id": None},
{"id": 3, "species_id": 2}, {"id": 3, "species_id": 2},

View file

@ -32,7 +32,7 @@ def compound_db():
def test_compound_foreign_key(compound_db): def test_compound_foreign_key(compound_db):
fks = compound_db["courses"].foreign_keys fks = compound_db.table("courses").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
fk = fks[0] fk = fks[0]
assert fk.is_compound is True assert fk.is_compound is True
@ -46,10 +46,10 @@ def test_compound_foreign_key(compound_db):
def test_single_foreign_key_gets_columns_fields(fresh_db): def test_single_foreign_key_gets_columns_fields(fresh_db):
fresh_db["authors"].insert({"id": 1, "name": "Sally"}, pk="id") fresh_db.table("authors").insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db["books"].insert({"title": "Hedgehogs", "author_id": 1}) fresh_db.table("books").insert({"title": "Hedgehogs", "author_id": 1})
fresh_db["books"].add_foreign_key("author_id", "authors", "id") fresh_db.table("books").add_foreign_key("author_id", "authors", "id")
fk = fresh_db["books"].foreign_keys[0] fk = fresh_db.table("books").foreign_keys[0]
assert fk.is_compound is False assert fk.is_compound is False
assert fk.column == "author_id" assert fk.column == "author_id"
assert fk.other_column == "id" assert fk.other_column == "id"
@ -60,10 +60,10 @@ def test_single_foreign_key_gets_columns_fields(fresh_db):
def test_foreign_key_no_longer_unpacks_as_tuple(fresh_db): def test_foreign_key_no_longer_unpacks_as_tuple(fresh_db):
# Clean break in 4.0: ForeignKey is a dataclass, not a namedtuple, so the # Clean break in 4.0: ForeignKey is a dataclass, not a namedtuple, so the
# old tuple unpacking and indexing patterns now fail hard. # old tuple unpacking and indexing patterns now fail hard.
fresh_db["authors"].insert({"id": 1, "name": "Sally"}, pk="id") fresh_db.table("authors").insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db["books"].insert({"title": "Hedgehogs", "author_id": 1}) fresh_db.table("books").insert({"title": "Hedgehogs", "author_id": 1})
fresh_db["books"].add_foreign_key("author_id", "authors", "id") fresh_db.table("books").add_foreign_key("author_id", "authors", "id")
fk = fresh_db["books"].foreign_keys[0] fk = fresh_db.table("books").foreign_keys[0]
with pytest.raises(TypeError): with pytest.raises(TypeError):
_table, _column, _other_table, _other_column = fk _table, _column, _other_table, _other_column = fk
with pytest.raises(TypeError): with pytest.raises(TypeError):
@ -71,16 +71,18 @@ def test_foreign_key_no_longer_unpacks_as_tuple(fresh_db):
def test_foreign_keys_are_sortable(fresh_db): def test_foreign_keys_are_sortable(fresh_db):
fresh_db["authors"].insert({"id": 1, "name": "Sally"}, pk="id") fresh_db.table("authors").insert({"id": 1, "name": "Sally"}, pk="id")
fresh_db["categories"].insert({"id": 1, "name": "Wildlife"}, pk="id") fresh_db.table("categories").insert({"id": 1, "name": "Wildlife"}, pk="id")
fresh_db["books"].insert({"title": "Hedgehogs", "author_id": 1, "category_id": 1}) fresh_db.table("books").insert(
{"title": "Hedgehogs", "author_id": 1, "category_id": 1}
)
fresh_db.add_foreign_keys( fresh_db.add_foreign_keys(
[ [
("books", "author_id", "authors", "id"), ("books", "author_id", "authors", "id"),
("books", "category_id", "categories", "id"), ("books", "category_id", "categories", "id"),
] ]
) )
fks = sorted(fresh_db["books"].foreign_keys) fks = sorted(fresh_db.table("books").foreign_keys)
assert fks[0].column == "author_id" assert fks[0].column == "author_id"
assert fks[1].column == "category_id" assert fks[1].column == "category_id"
@ -105,7 +107,7 @@ def test_mixed_compound_and_single_foreign_keys_are_sortable():
REFERENCES departments(campus_name, dept_code) REFERENCES departments(campus_name, dept_code)
); );
""") """)
fks = db["courses"].foreign_keys fks = db.table("courses").foreign_keys
assert len(fks) == 2 assert len(fks) == 2
assert {fk.is_compound for fk in fks} == {True, False} assert {fk.is_compound for fk in fks} == {True, False}
fks_sorted = sorted(fks) fks_sorted = sorted(fks)
@ -163,8 +165,8 @@ def test_create_table_with_compound_foreign_key(departments_db, foreign_keys):
pk="course_code", pk="course_code",
foreign_keys=foreign_keys, foreign_keys=foreign_keys,
) )
assert departments_db["courses"].schema == EXPECTED_COURSES_SCHEMA assert departments_db.table("courses").schema == EXPECTED_COURSES_SCHEMA
fks = departments_db["courses"].foreign_keys fks = departments_db.table("courses").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
fk = fks[0] fk = fks[0]
assert fk.is_compound is True assert fk.is_compound is True
@ -181,10 +183,10 @@ def test_create_table_compound_foreign_key_enforced(departments_db):
pk="course_code", pk="course_code",
foreign_keys=[(("campus_name", "dept_code"), "departments")], foreign_keys=[(("campus_name", "dept_code"), "departments")],
) )
departments_db["departments"].insert( departments_db.table("departments").insert(
{"campus_name": "Berkeley", "dept_code": "CS", "dept_name": "Computer Science"} {"campus_name": "Berkeley", "dept_code": "CS", "dept_name": "Computer Science"}
) )
departments_db["courses"].insert( departments_db.table("courses").insert(
{"course_code": "CS101", "campus_name": "Berkeley", "dept_code": "CS"} {"course_code": "CS101", "campus_name": "Berkeley", "dept_code": "CS"}
) )
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
@ -207,8 +209,8 @@ def test_create_table_compound_foreign_key_missing_other_column(departments_db):
def test_transform_preserves_compound_foreign_key(compound_db): def test_transform_preserves_compound_foreign_key(compound_db):
compound_db["courses"].transform(rename={"course_name": "title"}) compound_db.table("courses").transform(rename={"course_name": "title"})
fks = compound_db["courses"].foreign_keys fks = compound_db.table("courses").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
fk = fks[0] fk = fks[0]
assert fk.is_compound is True assert fk.is_compound is True
@ -218,8 +220,8 @@ def test_transform_preserves_compound_foreign_key(compound_db):
def test_transform_rename_member_column_updates_compound_foreign_key(compound_db): def test_transform_rename_member_column_updates_compound_foreign_key(compound_db):
compound_db["courses"].transform(rename={"campus_name": "campus"}) compound_db.table("courses").transform(rename={"campus_name": "campus"})
fks = compound_db["courses"].foreign_keys fks = compound_db.table("courses").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
fk = fks[0] fk = fks[0]
assert fk.is_compound is True assert fk.is_compound is True
@ -231,9 +233,9 @@ def test_transform_rename_member_column_updates_compound_foreign_key(compound_db
def test_transform_drop_member_column_drops_compound_foreign_key(compound_db): def test_transform_drop_member_column_drops_compound_foreign_key(compound_db):
# Matches single-column behavior: dropping the column silently # Matches single-column behavior: dropping the column silently
# drops the foreign key that used it # drops the foreign key that used it
compound_db["courses"].transform(drop={"dept_code"}) compound_db.table("courses").transform(drop={"dept_code"})
assert compound_db["courses"].foreign_keys == [] assert compound_db.table("courses").foreign_keys == []
assert "FOREIGN KEY" not in compound_db["courses"].schema assert "FOREIGN KEY" not in compound_db.table("courses").schema
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -246,11 +248,11 @@ def test_transform_drop_member_column_drops_compound_foreign_key(compound_db):
), ),
) )
def test_transform_drop_compound_foreign_key(compound_db, drop_foreign_keys): def test_transform_drop_compound_foreign_key(compound_db, drop_foreign_keys):
compound_db["courses"].transform(drop_foreign_keys=drop_foreign_keys) compound_db.table("courses").transform(drop_foreign_keys=drop_foreign_keys)
assert compound_db["courses"].foreign_keys == [] assert compound_db.table("courses").foreign_keys == []
# The columns themselves survive # The columns themselves survive
assert {"campus_name", "dept_code"} <= set( assert {"campus_name", "dept_code"} <= set(
compound_db["courses"].columns_dict.keys() compound_db.table("courses").columns_dict.keys()
) )
@ -265,12 +267,12 @@ def courses_db(departments_db):
def test_add_compound_foreign_key(courses_db): def test_add_compound_foreign_key(courses_db):
t = courses_db["courses"].add_foreign_key( t = courses_db.table("courses").add_foreign_key(
("campus_name", "dept_code"), "departments", ("campus_name", "dept_code") ("campus_name", "dept_code"), "departments", ("campus_name", "dept_code")
) )
# Returns self # Returns self
assert t.name == "courses" assert t.name == "courses"
fks = courses_db["courses"].foreign_keys fks = courses_db.table("courses").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
fk = fks[0] fk = fks[0]
assert fk.is_compound is True assert fk.is_compound is True
@ -281,27 +283,33 @@ def test_add_compound_foreign_key(courses_db):
def test_add_compound_foreign_key_guesses_other_columns(courses_db): def test_add_compound_foreign_key_guesses_other_columns(courses_db):
# Lists work here too, though tuples are the documented form # Lists work here too, though tuples are the documented form
courses_db["courses"].add_foreign_key(["campus_name", "dept_code"], "departments") courses_db.table("courses").add_foreign_key(
fk = courses_db["courses"].foreign_keys[0] ["campus_name", "dept_code"], "departments"
)
fk = courses_db.table("courses").foreign_keys[0]
assert fk.other_columns == ("campus_name", "dept_code") assert fk.other_columns == ("campus_name", "dept_code")
def test_add_compound_foreign_key_error_if_already_exists(courses_db): def test_add_compound_foreign_key_error_if_already_exists(courses_db):
courses_db["courses"].add_foreign_key(("campus_name", "dept_code"), "departments") courses_db.table("courses").add_foreign_key(
("campus_name", "dept_code"), "departments"
)
with pytest.raises(AlterError) as ex: with pytest.raises(AlterError) as ex:
courses_db["courses"].add_foreign_key( courses_db.table("courses").add_foreign_key(
("campus_name", "dept_code"), "departments" ("campus_name", "dept_code"), "departments"
) )
assert "already exists" in ex.value.args[0] assert "already exists" in ex.value.args[0]
# ignore=True should not raise # ignore=True should not raise
courses_db["courses"].add_foreign_key( courses_db.table("courses").add_foreign_key(
("campus_name", "dept_code"), "departments", ignore=True ("campus_name", "dept_code"), "departments", ignore=True
) )
def test_add_compound_foreign_key_error_if_column_missing(courses_db): def test_add_compound_foreign_key_error_if_column_missing(courses_db):
with pytest.raises(AlterError): with pytest.raises(AlterError):
courses_db["courses"].add_foreign_key(("campus_name", "nope"), "departments") courses_db.table("courses").add_foreign_key(
("campus_name", "nope"), "departments"
)
def test_db_add_foreign_keys_compound(courses_db): def test_db_add_foreign_keys_compound(courses_db):
@ -315,14 +323,14 @@ def test_db_add_foreign_keys_compound(courses_db):
) )
] ]
) )
fk = courses_db["courses"].foreign_keys[0] fk = courses_db.table("courses").foreign_keys[0]
assert fk.is_compound is True assert fk.is_compound is True
assert fk.columns == ("campus_name", "dept_code") assert fk.columns == ("campus_name", "dept_code")
def test_index_foreign_keys_compound_creates_composite_index(compound_db): def test_index_foreign_keys_compound_creates_composite_index(compound_db):
compound_db.index_foreign_keys() compound_db.index_foreign_keys()
index_columns = [i.columns for i in compound_db["courses"].indexes] index_columns = [i.columns for i in compound_db.table("courses").indexes]
assert ["campus_name", "dept_code"] in index_columns assert ["campus_name", "dept_code"] in index_columns
# No separate single-column indexes for the members # No separate single-column indexes for the members
assert ["campus_name"] not in index_columns assert ["campus_name"] not in index_columns
@ -339,22 +347,22 @@ def test_foreign_key_captures_on_delete_and_on_update():
ON DELETE CASCADE ON UPDATE RESTRICT ON DELETE CASCADE ON UPDATE RESTRICT
); );
""") """)
fk = db["books"].foreign_keys[0] fk = db.table("books").foreign_keys[0]
assert fk.on_delete == "CASCADE" assert fk.on_delete == "CASCADE"
assert fk.on_update == "RESTRICT" assert fk.on_update == "RESTRICT"
def test_foreign_key_on_delete_defaults_to_no_action(fresh_db): def test_foreign_key_on_delete_defaults_to_no_action(fresh_db):
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id") fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["books"].add_foreign_key("author_id", "authors", "id") fresh_db.table("books").add_foreign_key("author_id", "authors", "id")
fk = fresh_db["books"].foreign_keys[0] fk = fresh_db.table("books").foreign_keys[0]
assert fk.on_delete == "NO ACTION" assert fk.on_delete == "NO ACTION"
assert fk.on_update == "NO ACTION" assert fk.on_update == "NO ACTION"
def test_create_table_foreign_key_with_on_delete(fresh_db): def test_create_table_foreign_key_with_on_delete(fresh_db):
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db.create_table( fresh_db.create_table(
"books", "books",
{"id": int, "author_id": int}, {"id": int, "author_id": int},
@ -369,8 +377,8 @@ def test_create_table_foreign_key_with_on_delete(fresh_db):
) )
], ],
) )
assert "ON DELETE CASCADE" in fresh_db["books"].schema assert "ON DELETE CASCADE" in fresh_db.table("books").schema
assert fresh_db["books"].foreign_keys[0].on_delete == "CASCADE" assert fresh_db.table("books").foreign_keys[0].on_delete == "CASCADE"
def test_transform_preserves_on_delete_cascade(): def test_transform_preserves_on_delete_cascade():
@ -383,11 +391,11 @@ def test_transform_preserves_on_delete_cascade():
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
); );
""") """)
db["books"].transform(rename={"title": "book_title"}) db.table("books").transform(rename={"title": "book_title"})
fk = db["books"].foreign_keys[0] fk = db.table("books").foreign_keys[0]
assert fk.on_delete == "CASCADE" assert fk.on_delete == "CASCADE"
assert fk.on_update == "NO ACTION" assert fk.on_update == "NO ACTION"
assert "ON DELETE CASCADE" in db["books"].schema assert "ON DELETE CASCADE" in db.table("books").schema
def test_transform_preserves_compound_foreign_key_on_delete(): def test_transform_preserves_compound_foreign_key_on_delete():
@ -406,11 +414,11 @@ def test_transform_preserves_compound_foreign_key_on_delete():
REFERENCES departments(campus_name, dept_code) ON DELETE CASCADE REFERENCES departments(campus_name, dept_code) ON DELETE CASCADE
); );
""") """)
db["courses"].transform(rename={"course_code": "code"}) db.table("courses").transform(rename={"course_code": "code"})
fk = db["courses"].foreign_keys[0] fk = db.table("courses").foreign_keys[0]
assert fk.is_compound is True assert fk.is_compound is True
assert fk.on_delete == "CASCADE" assert fk.on_delete == "CASCADE"
assert "ON DELETE CASCADE" in db["courses"].schema assert "ON DELETE CASCADE" in db.table("courses").schema
def test_implicit_primary_key_reference_is_resolved(): def test_implicit_primary_key_reference_is_resolved():
@ -424,7 +432,7 @@ def test_implicit_primary_key_reference_is_resolved():
author_id INTEGER REFERENCES authors author_id INTEGER REFERENCES authors
); );
""") """)
fk = db["books"].foreign_keys[0] fk = db.table("books").foreign_keys[0]
assert fk.is_compound is False assert fk.is_compound is False
assert fk.other_column == "author_id" assert fk.other_column == "author_id"
assert fk.other_columns == ("author_id",) assert fk.other_columns == ("author_id",)
@ -445,7 +453,7 @@ def test_implicit_compound_primary_key_reference_is_resolved():
FOREIGN KEY (campus_name, dept_code) REFERENCES departments FOREIGN KEY (campus_name, dept_code) REFERENCES departments
); );
""") """)
fk = db["courses"].foreign_keys[0] fk = db.table("courses").foreign_keys[0]
assert fk.is_compound is True assert fk.is_compound is True
assert fk.other_columns == ("campus_name", "dept_code") assert fk.other_columns == ("campus_name", "dept_code")
@ -470,14 +478,14 @@ def test_add_foreign_keys_preserves_actions(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/594 review finding: # https://github.com/simonw/sqlite-utils/issues/594 review finding:
# ForeignKey objects passed to db.add_foreign_keys() were flattened # ForeignKey objects passed to db.add_foreign_keys() were flattened
# to plain tuples, losing on_delete/on_update # to plain tuples, losing on_delete/on_update
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id") fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db.add_foreign_keys( fresh_db.add_foreign_keys(
[ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")] [ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")]
) )
fk = fresh_db["books"].foreign_keys[0] fk = fresh_db.table("books").foreign_keys[0]
assert fk.on_delete == "CASCADE" assert fk.on_delete == "CASCADE"
assert "ON DELETE CASCADE" in fresh_db["books"].schema assert "ON DELETE CASCADE" in fresh_db.table("books").schema
def test_add_foreign_keys_preserves_actions_compound(courses_db): def test_add_foreign_keys_preserves_actions_compound(courses_db):
@ -495,36 +503,36 @@ def test_add_foreign_keys_preserves_actions_compound(courses_db):
) )
] ]
) )
fk = courses_db["courses"].foreign_keys[0] fk = courses_db.table("courses").foreign_keys[0]
assert fk.is_compound is True assert fk.is_compound is True
assert fk.on_delete == "CASCADE" assert fk.on_delete == "CASCADE"
assert "ON DELETE CASCADE" in courses_db["courses"].schema assert "ON DELETE CASCADE" in courses_db.table("courses").schema
def test_add_foreign_key_on_delete_on_update(fresh_db): def test_add_foreign_key_on_delete_on_update(fresh_db):
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id") fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["books"].add_foreign_key( fresh_db.table("books").add_foreign_key(
"author_id", "authors", "id", on_delete="CASCADE", on_update="RESTRICT" "author_id", "authors", "id", on_delete="CASCADE", on_update="RESTRICT"
) )
fk = fresh_db["books"].foreign_keys[0] fk = fresh_db.table("books").foreign_keys[0]
assert fk.on_delete == "CASCADE" assert fk.on_delete == "CASCADE"
assert fk.on_update == "RESTRICT" assert fk.on_update == "RESTRICT"
assert "ON UPDATE RESTRICT ON DELETE CASCADE" in fresh_db["books"].schema assert "ON UPDATE RESTRICT ON DELETE CASCADE" in fresh_db.table("books").schema
# The cascade should actually fire # The cascade should actually fire
fresh_db.execute("PRAGMA foreign_keys = ON") fresh_db.execute("PRAGMA foreign_keys = ON")
fresh_db.execute("delete from authors where id = 1") fresh_db.execute("delete from authors where id = 1")
assert fresh_db["books"].count == 0 assert fresh_db.table("books").count == 0
def test_add_compound_foreign_key_on_delete(courses_db): def test_add_compound_foreign_key_on_delete(courses_db):
courses_db["courses"].add_foreign_key( courses_db.table("courses").add_foreign_key(
("campus_name", "dept_code"), "departments", on_delete="SET NULL" ("campus_name", "dept_code"), "departments", on_delete="SET NULL"
) )
fk = courses_db["courses"].foreign_keys[0] fk = courses_db.table("courses").foreign_keys[0]
assert fk.is_compound is True assert fk.is_compound is True
assert fk.on_delete == "SET NULL" assert fk.on_delete == "SET NULL"
assert "ON DELETE SET NULL" in courses_db["courses"].schema assert "ON DELETE SET NULL" in courses_db.table("courses").schema
def test_implicit_compound_foreign_key_resolves_pk_declaration_order(fresh_db): def test_implicit_compound_foreign_key_resolves_pk_declaration_order(fresh_db):
@ -536,7 +544,7 @@ def test_implicit_compound_foreign_key_resolves_pk_declaration_order(fresh_db):
fresh_db.execute( fresh_db.execute(
"create table child (x text, y text, foreign key (x, y) references other)" "create table child (x text, y text, foreign key (x, y) references other)"
) )
fk = fresh_db["child"].foreign_keys[0] fk = fresh_db.table("child").foreign_keys[0]
assert fk.other_columns == ("a", "b") assert fk.other_columns == ("a", "b")
@ -549,46 +557,46 @@ def test_transform_implicit_compound_foreign_key_stays_valid(fresh_db):
"create table child (x text, y text, foreign key (x, y) references other)" "create table child (x text, y text, foreign key (x, y) references other)"
) )
fresh_db.execute("PRAGMA foreign_keys = ON") fresh_db.execute("PRAGMA foreign_keys = ON")
fresh_db["other"].insert({"a": "A", "b": "B"}) fresh_db.table("other").insert({"a": "A", "b": "B"})
fresh_db["child"].insert({"x": "A", "y": "B"}) fresh_db.table("child").insert({"x": "A", "y": "B"})
fresh_db["child"].transform(types={"x": str}) fresh_db.table("child").transform(types={"x": str})
assert fresh_db["child"].foreign_keys[0].other_columns == ("a", "b") assert fresh_db.table("child").foreign_keys[0].other_columns == ("a", "b")
# The constraint still points the right way around # The constraint still points the right way around
fresh_db["child"].insert({"x": "A", "y": "B"}) fresh_db.table("child").insert({"x": "A", "y": "B"})
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
fresh_db["child"].insert({"x": "B", "y": "A"}) fresh_db.table("child").insert({"x": "B", "y": "A"})
def test_create_compound_foreign_key_guesses_pk_declaration_order(fresh_db): def test_create_compound_foreign_key_guesses_pk_declaration_order(fresh_db):
fresh_db.execute("create table other (b text, a text, primary key (a, b))") fresh_db.execute("create table other (b text, a text, primary key (a, b))")
fresh_db["other"].insert({"a": "A", "b": "B"}) fresh_db.table("other").insert({"a": "A", "b": "B"})
fresh_db["child"].create( fresh_db.table("child").create(
{"id": int, "x": str, "y": str}, {"id": int, "x": str, "y": str},
pk="id", pk="id",
foreign_keys=[(("x", "y"), "other")], foreign_keys=[(("x", "y"), "other")],
) )
assert fresh_db["child"].foreign_keys[0].other_columns == ("a", "b") assert fresh_db.table("child").foreign_keys[0].other_columns == ("a", "b")
fresh_db.execute("PRAGMA foreign_keys = ON") fresh_db.execute("PRAGMA foreign_keys = ON")
fresh_db["child"].insert({"id": 1, "x": "A", "y": "B"}) fresh_db.table("child").insert({"id": 1, "x": "A", "y": "B"})
with pytest.raises(sqlite3.IntegrityError): with pytest.raises(sqlite3.IntegrityError):
fresh_db["child"].insert({"id": 2, "x": "B", "y": "A"}) fresh_db.table("child").insert({"id": 2, "x": "B", "y": "A"})
def test_add_compound_foreign_key_guesses_pk_declaration_order(fresh_db): def test_add_compound_foreign_key_guesses_pk_declaration_order(fresh_db):
fresh_db.execute("create table other (b text, a text, primary key (a, b))") fresh_db.execute("create table other (b text, a text, primary key (a, b))")
fresh_db["child"].insert({"id": 1, "x": "A", "y": "B"}, pk="id") fresh_db.table("child").insert({"id": 1, "x": "A", "y": "B"}, pk="id")
fresh_db["child"].add_foreign_key(("x", "y"), "other") fresh_db.table("child").add_foreign_key(("x", "y"), "other")
assert fresh_db["child"].foreign_keys[0].other_columns == ("a", "b") assert fresh_db.table("child").foreign_keys[0].other_columns == ("a", "b")
def test_foreign_keys_are_hashable(fresh_db): def test_foreign_keys_are_hashable(fresh_db):
# set() over foreign_keys worked with the 3.x namedtuple and must # set() over foreign_keys worked with the 3.x namedtuple and must
# keep working with the dataclass # keep working with the dataclass
fresh_db["p"].insert({"id": 1}, pk="id") fresh_db.table("p").insert({"id": 1}, pk="id")
fresh_db["c"].insert( fresh_db.table("c").insert(
{"id": 1, "pid": 1}, pk="id", foreign_keys=[("pid", "p", "id")] {"id": 1, "pid": 1}, pk="id", foreign_keys=[("pid", "p", "id")]
) )
fks = set(fresh_db["c"].foreign_keys) fks = set(fresh_db.table("c").foreign_keys)
assert len(fks) == 1 assert len(fks) == 1
assert ForeignKey("c", "pid", "p", "id") in fks assert ForeignKey("c", "pid", "p", "id") in fks
# Usable as dict keys too # Usable as dict keys too
@ -600,7 +608,7 @@ def test_foreign_key_is_immutable():
fk = ForeignKey("c", "pid", "p", "id") fk = ForeignKey("c", "pid", "p", "id")
with pytest.raises(dataclasses.FrozenInstanceError): with pytest.raises(dataclasses.FrozenInstanceError):
fk.table = "other" setattr(fk, "table", "other")
def test_foreign_key_equality_and_hash_include_actions(): def test_foreign_key_equality_and_hash_include_actions():
@ -617,9 +625,9 @@ def test_create_table_mixed_foreign_keys_list(fresh_db):
# 3.x accepted a mix of ForeignKey objects, tuples and bare column # 3.x accepted a mix of ForeignKey objects, tuples and bare column
# strings in foreign_keys= (ForeignKey was a namedtuple, so it passed # strings in foreign_keys= (ForeignKey was a namedtuple, so it passed
# the tuple check) - keep accepting the mix # the tuple check) - keep accepting the mix
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["publishers"].insert({"id": 1}, pk="id") fresh_db.table("publishers").insert({"id": 1}, pk="id")
fresh_db["books"].create( fresh_db.table("books").create(
{"id": int, "author_id": int, "publisher_id": int}, {"id": int, "author_id": int, "publisher_id": int},
pk="id", pk="id",
foreign_keys=[ foreign_keys=[
@ -627,14 +635,14 @@ def test_create_table_mixed_foreign_keys_list(fresh_db):
("publisher_id", "publishers", "id"), ("publisher_id", "publishers", "id"),
], ],
) )
fks = {fk.column: fk.other_table for fk in fresh_db["books"].foreign_keys} fks = {fk.column: fk.other_table for fk in fresh_db.table("books").foreign_keys}
assert fks == {"author_id": "authors", "publisher_id": "publishers"} assert fks == {"author_id": "authors", "publisher_id": "publishers"}
def test_create_table_mixed_foreign_keys_with_string(fresh_db): def test_create_table_mixed_foreign_keys_with_string(fresh_db):
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["publishers"].insert({"id": 1}, pk="id") fresh_db.table("publishers").insert({"id": 1}, pk="id")
fresh_db["books"].create( fresh_db.table("books").create(
{"id": int, "author_id": int, "publisher_id": int}, {"id": int, "author_id": int, "publisher_id": int},
pk="id", pk="id",
foreign_keys=[ foreign_keys=[
@ -642,15 +650,15 @@ def test_create_table_mixed_foreign_keys_with_string(fresh_db):
("publisher_id", "publishers", "id"), ("publisher_id", "publishers", "id"),
], ],
) )
fks = {fk.column: fk.other_table for fk in fresh_db["books"].foreign_keys} fks = {fk.column: fk.other_table for fk in fresh_db.table("books").foreign_keys}
assert fks == {"author_id": "authors", "publisher_id": "publishers"} assert fks == {"author_id": "authors", "publisher_id": "publishers"}
def test_add_foreign_keys_existing_with_different_actions_errors(fresh_db): def test_add_foreign_keys_existing_with_different_actions_errors(fresh_db):
# Requesting an existing foreign key with different ON DELETE/ON UPDATE # Requesting an existing foreign key with different ON DELETE/ON UPDATE
# actions was silently skipped, dropping the requested change # actions was silently skipped, dropping the requested change
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["books"].insert( fresh_db.table("books").insert(
{"id": 1, "author_id": 1}, {"id": 1, "author_id": 1},
pk="id", pk="id",
foreign_keys=[("author_id", "authors", "id")], foreign_keys=[("author_id", "authors", "id")],
@ -660,19 +668,21 @@ def test_add_foreign_keys_existing_with_different_actions_errors(fresh_db):
[ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")] [ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")]
) )
assert "ON DELETE" in str(ex.value) assert "ON DELETE" in str(ex.value)
assert fresh_db["books"].foreign_keys[0].on_delete == "NO ACTION" assert fresh_db.table("books").foreign_keys[0].on_delete == "NO ACTION"
def test_add_foreign_keys_identical_existing_is_noop(fresh_db): def test_add_foreign_keys_identical_existing_is_noop(fresh_db):
# An exact match, including actions, is silently skipped so repeated # An exact match, including actions, is silently skipped so repeated
# calls stay idempotent # calls stay idempotent
fresh_db["authors"].insert({"id": 1}, pk="id") fresh_db.table("authors").insert({"id": 1}, pk="id")
fresh_db["books"].insert({"id": 1, "author_id": 1}, pk="id") fresh_db.table("books").insert({"id": 1, "author_id": 1}, pk="id")
fresh_db["books"].add_foreign_key("author_id", "authors", "id", on_delete="CASCADE") fresh_db.table("books").add_foreign_key(
"author_id", "authors", "id", on_delete="CASCADE"
)
fresh_db.add_foreign_keys( fresh_db.add_foreign_keys(
[ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")] [ForeignKey("books", "author_id", "authors", "id", on_delete="CASCADE")]
) )
fks = fresh_db["books"].foreign_keys fks = fresh_db.table("books").foreign_keys
assert len(fks) == 1 assert len(fks) == 1
assert fks[0].on_delete == "CASCADE" assert fks[0].on_delete == "CASCADE"
@ -680,13 +690,13 @@ def test_add_foreign_keys_identical_existing_is_noop(fresh_db):
def test_add_foreign_keys_compound_column_count_mismatch_errors(fresh_db): def test_add_foreign_keys_compound_column_count_mismatch_errors(fresh_db):
# Previously the extra other-column was silently discarded, creating # Previously the extra other-column was silently discarded, creating
# a single-column foreign key to just ("id") # a single-column foreign key to just ("id")
fresh_db["departments"].insert( fresh_db.table("departments").insert(
{"campus": "north", "code": "cs"}, pk=("campus", "code") {"campus": "north", "code": "cs"}, pk=("campus", "code")
) )
fresh_db["courses"].insert({"id": 1, "campus": "north"}, pk="id") fresh_db.table("courses").insert({"id": 1, "campus": "north"}, pk="id")
with pytest.raises(ValueError) as ex: with pytest.raises(ValueError) as ex:
fresh_db.add_foreign_keys( fresh_db.add_foreign_keys(
[("courses", ("campus",), "departments", ("campus", "code"))] [("courses", ("campus",), "departments", ("campus", "code"))]
) )
assert "same number of columns" in str(ex.value) assert "same number of columns" in str(ex.value)
assert fresh_db["courses"].foreign_keys == [] assert fresh_db.table("courses").foreign_keys == []

View file

@ -20,7 +20,7 @@ search_records = [
def test_enable_fts(fresh_db): def test_enable_fts(fresh_db):
table = fresh_db["searchable"] table = fresh_db.table("searchable")
table.insert_all(search_records) table.insert_all(search_records)
assert ["searchable"] == fresh_db.table_names() assert ["searchable"] == fresh_db.table_names()
table.enable_fts(["text", "country"], fts_version="FTS4") table.enable_fts(["text", "country"], fts_version="FTS4")
@ -54,7 +54,7 @@ def test_enable_fts(fresh_db):
def test_enable_fts_escape_table_names(fresh_db): def test_enable_fts_escape_table_names(fresh_db):
# Table names with restricted chars are handled correctly. # Table names with restricted chars are handled correctly.
# colons and dots are restricted characters for table names. # colons and dots are restricted characters for table names.
table = fresh_db["http://example.com"] table = fresh_db.table("http://example.com")
table.insert_all(search_records) table.insert_all(search_records)
assert ["http://example.com"] == fresh_db.table_names() assert ["http://example.com"] == fresh_db.table_names()
table.enable_fts(["text", "country"], fts_version="FTS4") table.enable_fts(["text", "country"], fts_version="FTS4")
@ -87,7 +87,7 @@ def test_enable_fts_escape_table_names(fresh_db):
def test_search_duplicate_columns_are_deduped(fresh_db): def test_search_duplicate_columns_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624 # https://github.com/simonw/sqlite-utils/issues/624
table = fresh_db["t"] table = fresh_db.table("t")
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS4") table.enable_fts(["text", "country"], fts_version="FTS4")
rows = list(table.search("tanuki", columns=["text", "text"])) rows = list(table.search("tanuki", columns=["text", "text"]))
@ -100,7 +100,7 @@ def test_search_duplicate_columns_are_deduped(fresh_db):
def test_search_limit_offset(fresh_db): def test_search_limit_offset(fresh_db):
table = fresh_db["t"] table = fresh_db.table("t")
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS4") table.enable_fts(["text", "country"], fts_version="FTS4")
assert len(list(table.search("are"))) == 2 assert len(list(table.search("are"))) == 2
@ -112,9 +112,20 @@ def test_search_limit_offset(fresh_db):
) )
def test_search_offset_without_limit(fresh_db):
table = fresh_db.table("t")
table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS4")
assert [row["rowid"] for row in table.search("are", order_by="rowid")] == [1, 2]
assert [
row["rowid"] for row in table.search("are", offset=1, order_by="rowid")
] == [2]
assert table.search_sql(offset=1).strip().endswith("limit -1 offset 1")
@pytest.mark.parametrize("fts_version", ("FTS4", "FTS5")) @pytest.mark.parametrize("fts_version", ("FTS4", "FTS5"))
def test_search_where(fresh_db, fts_version): def test_search_where(fresh_db, fts_version):
table = fresh_db["t"] table = fresh_db.table("t")
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version=fts_version) table.enable_fts(["text", "country"], fts_version=fts_version)
results = list( results = list(
@ -131,7 +142,7 @@ def test_search_where(fresh_db, fts_version):
def test_search_where_args_disallows_query(fresh_db): def test_search_where_args_disallows_query(fresh_db):
table = fresh_db["t"] table = fresh_db.table("t")
with pytest.raises(ValueError) as ex: with pytest.raises(ValueError) as ex:
list( list(
table.search( table.search(
@ -145,7 +156,7 @@ def test_search_where_args_disallows_query(fresh_db):
def test_search_include_rank(fresh_db): def test_search_include_rank(fresh_db):
table = fresh_db["t"] table = fresh_db.table("t")
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version="FTS5") table.enable_fts(["text", "country"], fts_version="FTS5")
results = list(table.search("are", include_rank=True)) results = list(table.search("are", include_rank=True))
@ -171,7 +182,7 @@ def test_search_include_rank(fresh_db):
def test_enable_fts_table_names_containing_spaces(fresh_db): def test_enable_fts_table_names_containing_spaces(fresh_db):
table = fresh_db["test"] table = fresh_db.table("test")
table.insert({"column with spaces": "in its name"}) table.insert({"column with spaces": "in its name"})
table.enable_fts(["column with spaces"]) table.enable_fts(["column with spaces"])
assert [ assert [
@ -185,7 +196,7 @@ def test_enable_fts_table_names_containing_spaces(fresh_db):
def test_populate_fts(fresh_db): def test_populate_fts(fresh_db):
table = fresh_db["populatable"] table = fresh_db.table("populatable")
table.insert(search_records[0]) table.insert(search_records[0])
table.enable_fts(["text", "country"], fts_version="FTS4") table.enable_fts(["text", "country"], fts_version="FTS4")
assert [] == list(table.search("trash pandas")) assert [] == list(table.search("trash pandas"))
@ -206,7 +217,7 @@ def test_populate_fts(fresh_db):
def test_populate_fts_escape_table_names(fresh_db): def test_populate_fts_escape_table_names(fresh_db):
# Restricted characters such as colon and dots should be escaped. # Restricted characters such as colon and dots should be escaped.
table = fresh_db["http://example.com"] table = fresh_db.table("http://example.com")
table.insert(search_records[0]) table.insert(search_records[0])
table.enable_fts(["text", "country"], fts_version="FTS4") table.enable_fts(["text", "country"], fts_version="FTS4")
assert [] == list(table.search("trash pandas")) assert [] == list(table.search("trash pandas"))
@ -227,7 +238,7 @@ def test_populate_fts_escape_table_names(fresh_db):
@pytest.mark.parametrize("fts_version", ("4", "5")) @pytest.mark.parametrize("fts_version", ("4", "5"))
def test_fts_tokenize(fresh_db, fts_version): def test_fts_tokenize(fresh_db, fts_version):
table_name = f"searchable_{fts_version}" table_name = f"searchable_{fts_version}"
table = fresh_db[table_name] table = fresh_db.table(table_name)
table.insert_all(search_records) table.insert_all(search_records)
# Test without porter stemming # Test without porter stemming
table.enable_fts( table.enable_fts(
@ -252,10 +263,22 @@ def test_fts_tokenize(fresh_db, fts_version):
}.items() <= rows[0].items() }.items() <= rows[0].items()
def test_fts_tokenize_escaped(fresh_db):
# A malicious tokenize value must not be able to break out of the
# string literal in the CREATE VIRTUAL TABLE statement.
table = fresh_db.table("searchable")
table.insert_all(search_records)
malicious = "porter'); CREATE TABLE injected(x); --"
with pytest.raises(Exception):
table.enable_fts(["text"], tokenize=malicious)
# The injected statement must not have executed
assert "injected" not in fresh_db.table_names()
def test_optimize_fts(fresh_db): def test_optimize_fts(fresh_db):
for fts_version in ("4", "5"): for fts_version in ("4", "5"):
table_name = f"searchable_{fts_version}" table_name = f"searchable_{fts_version}"
table = fresh_db[table_name] table = fresh_db.table(table_name)
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"], fts_version=f"FTS{fts_version}") table.enable_fts(["text", "country"], fts_version=f"FTS{fts_version}")
# You can call optimize successfully against the tables OR their _fts equivalents: # You can call optimize successfully against the tables OR their _fts equivalents:
@ -265,11 +288,11 @@ def test_optimize_fts(fresh_db):
"searchable_4_fts", "searchable_4_fts",
"searchable_5_fts", "searchable_5_fts",
): ):
fresh_db[table_name].optimize() fresh_db.table(table_name).optimize()
def test_enable_fts_with_triggers(fresh_db): def test_enable_fts_with_triggers(fresh_db):
table = fresh_db["searchable"] table = fresh_db.table("searchable")
table.insert(search_records[0]) table.insert(search_records[0])
table.enable_fts(["text", "country"], fts_version="FTS4", create_triggers=True) table.enable_fts(["text", "country"], fts_version="FTS4", create_triggers=True)
rows1 = list(table.search("tanuki")) rows1 = list(table.search("tanuki"))
@ -298,7 +321,7 @@ def test_enable_fts_with_triggers(fresh_db):
@pytest.mark.parametrize("create_triggers", [True, False]) @pytest.mark.parametrize("create_triggers", [True, False])
def test_disable_fts(fresh_db, create_triggers): def test_disable_fts(fresh_db, create_triggers):
table = fresh_db["searchable"] table = fresh_db.table("searchable")
table.insert(search_records[0]) table.insert(search_records[0])
table.enable_fts(["text", "country"], create_triggers=create_triggers) table.enable_fts(["text", "country"], create_triggers=create_triggers)
assert { assert {
@ -331,7 +354,7 @@ def test_disable_fts(fresh_db, create_triggers):
def test_rebuild_fts(fresh_db): def test_rebuild_fts(fresh_db):
table = fresh_db["searchable"] table = fresh_db.table("searchable")
table.insert(search_records[0]) table.insert(search_records[0])
table.enable_fts(["text", "country"]) table.enable_fts(["text", "country"])
# Run a search # Run a search
@ -357,7 +380,7 @@ def test_rebuild_fts(fresh_db):
def test_optimize_and_rebuild_fts_commit(tmpdir, method): def test_optimize_and_rebuild_fts_commit(tmpdir, method):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
table = db["searchable"] table = db.table("searchable")
table.insert(search_records[0]) table.insert(search_records[0])
table.enable_fts(["text", "country"]) table.enable_fts(["text", "country"])
getattr(table, method)() getattr(table, method)()
@ -367,16 +390,16 @@ def test_optimize_and_rebuild_fts_commit(tmpdir, method):
table.insert(search_records[1]) table.insert(search_records[1])
db.close() db.close()
db2 = Database(path) db2 = Database(path)
assert db2["searchable"].count == 2 assert db2.table("searchable").count == 2
db2.close() db2.close()
@pytest.mark.parametrize("invalid_table", ["does_not_exist", "not_searchable"]) @pytest.mark.parametrize("invalid_table", ["does_not_exist", "not_searchable"])
def test_rebuild_fts_invalid(fresh_db, invalid_table): def test_rebuild_fts_invalid(fresh_db, invalid_table):
fresh_db["not_searchable"].insert({"foo": "bar"}) fresh_db.table("not_searchable").insert({"foo": "bar"})
# Raise OperationalError on invalid table # Raise OperationalError on invalid table
with pytest.raises(sqlite3.OperationalError): with pytest.raises(sqlite3.OperationalError):
fresh_db[invalid_table].rebuild_fts() fresh_db.table(invalid_table).rebuild_fts()
@pytest.mark.parametrize("fts_version", ["FTS4", "FTS5"]) @pytest.mark.parametrize("fts_version", ["FTS4", "FTS5"])
@ -385,15 +408,17 @@ def test_rebuild_removes_junk_docsize_rows(tmpdir, fts_version):
path = tmpdir / "test.db" path = tmpdir / "test.db"
db = Database(str(path), recursive_triggers=False) db = Database(str(path), recursive_triggers=False)
licenses = [{"key": "apache2", "name": "Apache 2"}, {"key": "bsd", "name": "BSD"}] licenses = [{"key": "apache2", "name": "Apache 2"}, {"key": "bsd", "name": "BSD"}]
db["licenses"].insert_all(licenses, pk="key", replace=True) db.table("licenses").insert_all(licenses, pk="key", replace=True)
db["licenses"].enable_fts(["name"], create_triggers=True, fts_version=fts_version) db.table("licenses").enable_fts(
assert db["licenses_fts_docsize"].count == 2 ["name"], create_triggers=True, fts_version=fts_version
)
assert db.table("licenses_fts_docsize").count == 2
# Bug: insert with replace increases the number of rows in _docsize: # Bug: insert with replace increases the number of rows in _docsize:
db["licenses"].insert_all(licenses, pk="key", replace=True) db.table("licenses").insert_all(licenses, pk="key", replace=True)
assert db["licenses_fts_docsize"].count == 4 assert db.table("licenses_fts_docsize").count == 4
# rebuild should fix this: # rebuild should fix this:
db["licenses_fts"].rebuild_fts() db.table("licenses_fts").rebuild_fts()
assert db["licenses_fts_docsize"].count == 2 assert db.table("licenses_fts_docsize").count == 2
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -407,7 +432,7 @@ def test_rebuild_removes_junk_docsize_rows(tmpdir, fts_version):
) )
def test_enable_fts_replace(kwargs): def test_enable_fts_replace(kwargs):
db = Database(memory=True) db = Database(memory=True)
db["books"].insert( db.table("books").insert(
{ {
"id": 1, "id": 1,
"title": "Habits of Australian Marsupials", "title": "Habits of Australian Marsupials",
@ -415,31 +440,31 @@ def test_enable_fts_replace(kwargs):
}, },
pk="id", pk="id",
) )
db["books"].enable_fts(["title", "author"]) db.table("books").enable_fts(["title", "author"])
assert not db["books"].triggers assert not db.table("books").triggers
assert db["books_fts"].columns_dict.keys() == {"title", "author"} assert db.table("books_fts").columns_dict.keys() == {"title", "author"}
assert "FTS5" in db["books_fts"].schema assert "FTS5" in db.table("books_fts").schema
assert "porter" not in db["books_fts"].schema assert "porter" not in db.table("books_fts").schema
# Now modify the FTS configuration # Now modify the FTS configuration
should_have_changed_columns = "columns" in kwargs should_have_changed_columns = "columns" in kwargs
if "columns" not in kwargs: if "columns" not in kwargs:
kwargs["columns"] = ["title", "author"] kwargs["columns"] = ["title", "author"]
db["books"].enable_fts(**kwargs, replace=True) db.table("books").enable_fts(**kwargs, replace=True)
# Check that the new configuration is correct # Check that the new configuration is correct
if should_have_changed_columns: if should_have_changed_columns:
assert db["books_fts"].columns_dict.keys() == {"title"} assert db.table("books_fts").columns_dict.keys() == {"title"}
if "create_triggers" in kwargs: if "create_triggers" in kwargs:
assert db["books"].triggers assert db.table("books").triggers
if "fts_version" in kwargs: if "fts_version" in kwargs:
assert "FTS4" in db["books_fts"].schema assert "FTS4" in db.table("books_fts").schema
if "tokenize" in kwargs: if "tokenize" in kwargs:
assert "porter" in db["books_fts"].schema assert "porter" in db.table("books_fts").schema
def test_enable_fts_replace_does_nothing_if_args_the_same(): def test_enable_fts_replace_does_nothing_if_args_the_same():
queries = [] queries = []
db = Database(memory=True, tracer=lambda sql, params: queries.append((sql, params))) db = Database(memory=True, tracer=lambda sql, params: queries.append((sql, params)))
db["books"].insert( db.table("books").insert(
{ {
"id": 1, "id": 1,
"title": "Habits of Australian Marsupials", "title": "Habits of Australian Marsupials",
@ -447,17 +472,19 @@ def test_enable_fts_replace_does_nothing_if_args_the_same():
}, },
pk="id", pk="id",
) )
db["books"].enable_fts(["title", "author"], create_triggers=True) db.table("books").enable_fts(["title", "author"], create_triggers=True)
queries.clear() queries.clear()
# Running that again shouldn't run much SQL: # Running that again shouldn't run much SQL:
db["books"].enable_fts(["title", "author"], create_triggers=True, replace=True) db.table("books").enable_fts(
["title", "author"], create_triggers=True, replace=True
)
# The only SQL that executed should be select statements # The only SQL that executed should be select statements
assert all(q[0].startswith("select ") for q in queries) assert all(q[0].startswith("select ") for q in queries)
def test_enable_fts_replace_handles_legacy_bracket_quoted_content_table(): def test_enable_fts_replace_handles_legacy_bracket_quoted_content_table():
db = Database(memory=True) db = Database(memory=True)
db["books"].insert( db.table("books").insert(
{ {
"id": 1, "id": 1,
"title": "Habits of Australian Marsupials", "title": "Habits of Australian Marsupials",
@ -472,10 +499,10 @@ def test_enable_fts_replace_handles_legacy_bracket_quoted_content_table():
); );
""") """)
db["books"].enable_fts(["title", "author"], replace=True) db.table("books").enable_fts(["title", "author"], replace=True)
assert db["books_fts"].columns_dict.keys() == {"title", "author"} assert db.table("books_fts").columns_dict.keys() == {"title", "author"}
assert 'content="books"' in db["books_fts"].schema assert 'content="books"' in db.table("books_fts").schema
def test_view_has_no_enable_fts(): def test_view_has_no_enable_fts():
@ -483,7 +510,7 @@ def test_view_has_no_enable_fts():
db.create_view("hello", "select 1 + 1") db.create_view("hello", "select 1 + 1")
# Views deliberately do not have an enable_fts() method # Views deliberately do not have an enable_fts() method
with pytest.raises(AttributeError): with pytest.raises(AttributeError):
db["hello"].enable_fts() # type: ignore[union-attr] db.view("hello").enable_fts() # type: ignore[attr-defined]
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -689,14 +716,14 @@ def test_view_has_no_enable_fts():
) )
def test_search_sql(kwargs, fts, expected): def test_search_sql(kwargs, fts, expected):
db = Database(memory=True) db = Database(memory=True)
db["books"].insert( db.table("books").insert(
{ {
"title": "Habits of Australian Marsupials", "title": "Habits of Australian Marsupials",
"author": "Marlee Hawkins", "author": "Marlee Hawkins",
} }
) )
db["books"].enable_fts(["title", "author"], fts_version=fts) db.table("books").enable_fts(["title", "author"], fts_version=fts)
sql = db["books"].search_sql(**kwargs) sql = db.table("books").search_sql(**kwargs)
assert sql == expected assert sql == expected
@ -717,7 +744,7 @@ def test_search_sql(kwargs, fts, expected):
), ),
) )
def test_quote_fts_query(fresh_db, input, expected): def test_quote_fts_query(fresh_db, input, expected):
table = fresh_db["searchable"] table = fresh_db.table("searchable")
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"]) table.enable_fts(["text", "country"])
quoted = fresh_db.quote_fts(input) quoted = fresh_db.quote_fts(input)
@ -727,7 +754,7 @@ def test_quote_fts_query(fresh_db, input, expected):
def test_search_quote(fresh_db): def test_search_quote(fresh_db):
table = fresh_db["searchable"] table = fresh_db.table("searchable")
table.insert_all(search_records) table.insert_all(search_records)
table.enable_fts(["text", "country"]) table.enable_fts(["text", "country"])
query = "cat's" query = "cat's"
@ -740,7 +767,7 @@ def test_search_quote(fresh_db):
def test_enable_fts_cli_on_view_errors(tmpdir): def test_enable_fts_cli_on_view_errors(tmpdir):
db_path = str(tmpdir / "test.db") db_path = str(tmpdir / "test.db")
db = Database(db_path) db = Database(db_path)
db["t"].insert({"text": "hello"}) db.table("t").insert({"text": "hello"})
db.create_view("v", "select * from t") db.create_view("v", "select * from t")
db.close() db.close()
from click.testing import CliRunner from click.testing import CliRunner

View file

@ -4,14 +4,14 @@ from sqlite_utils.db import NotFoundError
def test_get_rowid(fresh_db): def test_get_rowid(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
cleo = {"name": "Cleo", "age": 4} cleo = {"name": "Cleo", "age": 4}
row_id = dogs.insert(cleo).last_rowid row_id = dogs.insert(cleo).last_rowid
assert cleo == dogs.get(row_id) assert cleo == dogs.get(row_id)
def test_get_primary_key(fresh_db): def test_get_primary_key(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
cleo = {"name": "Cleo", "age": 4, "id": 5} cleo = {"name": "Cleo", "age": 4, "id": 5}
last_pk = dogs.insert(cleo, pk="id").last_pk last_pk = dogs.insert(cleo, pk="id").last_pk
assert 5 == last_pk assert 5 == last_pk
@ -23,10 +23,10 @@ def test_get_primary_key(fresh_db):
[(100, None), (None, None), ((1, 2), "Need 1 primary key value"), ("2", None)], [(100, None), (None, None), ((1, 2), "Need 1 primary key value"), ("2", None)],
) )
def test_get_not_found(argument, expected_msg, fresh_db): def test_get_not_found(argument, expected_msg, fresh_db):
fresh_db["dogs"].insert( fresh_db.table("dogs").insert(
{"id": 1, "name": "Cleo", "age": 4, "is_good": True}, pk="id" {"id": 1, "name": "Cleo", "age": 4, "is_good": True}, pk="id"
) )
with pytest.raises(NotFoundError) as excinfo: with pytest.raises(NotFoundError) as excinfo:
fresh_db["dogs"].get(argument) fresh_db.table("dogs").get(argument)
if expected_msg is not None: if expected_msg is not None:
assert expected_msg == excinfo.value.args[0] assert expected_msg == excinfo.value.args[0]

View file

@ -45,7 +45,7 @@ def test_add_geometry_column():
coord_dimension="XY", coord_dimension="XY",
) )
assert db["geometry_columns"].get(["locations", "geometry"]) == { assert db.table("geometry_columns").get(["locations", "geometry"]) == {
"f_table_name": "locations", "f_table_name": "locations",
"f_geometry_column": "geometry", "f_geometry_column": "geometry",
"geometry_type": 1, # point "geometry_type": 1, # point
@ -133,7 +133,7 @@ def test_cli_add_geometry_column(tmpdir):
db = Database(str(db_path)) db = Database(str(db_path))
db.init_spatialite() db.init_spatialite()
table = db["locations"].create({"name": str}) table = db.table("locations").create({"name": str})
result = CliRunner().invoke( result = CliRunner().invoke(
cli, cli,
@ -149,7 +149,7 @@ def test_cli_add_geometry_column(tmpdir):
assert result.exit_code == 0 assert result.exit_code == 0
assert db["geometry_columns"].get(["locations", "geometry"]) == { assert db.table("geometry_columns").get(["locations", "geometry"]) == {
"f_table_name": "locations", "f_table_name": "locations",
"f_geometry_column": "geometry", "f_geometry_column": "geometry",
"geometry_type": 1, # point "geometry_type": 1, # point
@ -164,7 +164,7 @@ def test_cli_add_geometry_column_options(tmpdir):
db_path = tmpdir / "spatial.db" db_path = tmpdir / "spatial.db"
db = Database(str(db_path)) db = Database(str(db_path))
db.init_spatialite() db.init_spatialite()
table = db["locations"].create({"name": str}) table = db.table("locations").create({"name": str})
result = CliRunner().invoke( result = CliRunner().invoke(
cli, cli,
@ -183,7 +183,7 @@ def test_cli_add_geometry_column_options(tmpdir):
assert result.exit_code == 0 assert result.exit_code == 0
assert db["geometry_columns"].get(["locations", "geometry"]) == { assert db.table("geometry_columns").get(["locations", "geometry"]) == {
"f_table_name": "locations", "f_table_name": "locations",
"f_geometry_column": "geometry", "f_geometry_column": "geometry",
"geometry_type": 3, # polygon "geometry_type": 3, # polygon
@ -202,7 +202,7 @@ def test_cli_add_geometry_column_invalid_type(tmpdir):
db = Database(str(db_path)) db = Database(str(db_path))
db.init_spatialite() db.init_spatialite()
table = db["locations"].create({"name": str}) table = db.table("locations").create({"name": str})
result = CliRunner().invoke( result = CliRunner().invoke(
cli, cli,
@ -225,7 +225,7 @@ def test_cli_create_spatial_index(tmpdir):
db = Database(str(db_path)) db = Database(str(db_path))
db.init_spatialite() db.init_spatialite()
table = db["locations"].create({"name": str}) table = db.table("locations").create({"name": str})
table.add_geometry_column("geometry", "POINT") table.add_geometry_column("geometry", "POINT")
result = CliRunner().invoke( result = CliRunner().invoke(

View file

@ -11,8 +11,8 @@ def test_roundtrip_integers(integer):
row = { row = {
"integer": integer, "integer": integer,
} }
db["test"].insert(row) db.table("test").insert(row)
assert list(db["test"].rows) == [row] assert list(db.table("test").rows) == [row]
@given(st.text()) @given(st.text())
@ -21,8 +21,8 @@ def test_roundtrip_text(text):
row = { row = {
"text": text, "text": text,
} }
db["test"].insert(row) db.table("test").insert(row)
assert list(db["test"].rows) == [row] assert list(db.table("test").rows) == [row]
@given(st.binary(max_size=1024 * 1024)) @given(st.binary(max_size=1024 * 1024))
@ -31,8 +31,8 @@ def test_roundtrip_binary(binary):
row = { row = {
"binary": binary, "binary": binary,
} }
db["test"].insert(row) db.table("test").insert(row)
assert list(db["test"].rows) == [row] assert list(db.table("test").rows) == [row]
@given(st.floats(allow_nan=False)) @given(st.floats(allow_nan=False))
@ -41,5 +41,5 @@ def test_roundtrip_floats(floats):
row = { row = {
"floats": floats, "floats": floats,
} }
db["test"].insert(row) db.table("test").insert(row)
assert list(db["test"].rows) == [row] assert list(db.table("test").rows) == [row]

View file

@ -57,7 +57,7 @@ def test_insert_files(silent, pk_args, expected_pks):
) )
assert result.exit_code == 0, result.stdout assert result.exit_code == 0, result.stdout
db = Database(db_path) db = Database(db_path)
rows_by_path = {r["path"]: r for r in db["files"].rows} rows_by_path = {r["path"]: r for r in db.table("files").rows}
one, two, three = ( one, two, three = (
rows_by_path["one.txt"], rows_by_path["one.txt"],
rows_by_path["two.txt"], rows_by_path["two.txt"],
@ -114,7 +114,7 @@ def test_insert_files(silent, pk_args, expected_pks):
for colname, expected_type in expected_types.items(): for colname, expected_type in expected_types.items():
for row in (one, two, three): for row in (one, two, three):
assert isinstance(row[colname], expected_type) assert isinstance(row[colname], expected_type)
assert set(db["files"].pks) == set(expected_pks) assert set(db.table("files").pks) == set(expected_pks)
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -144,7 +144,7 @@ def test_insert_files_stdin(use_text, encoding, input, expected):
) )
assert result.exit_code == 0, result.stdout assert result.exit_code == 0, result.stdout
db = Database(db_path) db = Database(db_path)
row = next(iter(db["files"].rows)) row = next(iter(db.table("files").rows))
key = "content" key = "content"
if use_text: if use_text:
key = "content_text" key = "content_text"

View file

@ -1,6 +1,6 @@
import pytest import pytest
from sqlite_utils.db import Check, Database, Index, View, XIndex, XIndexColumn from sqlite_utils.db import Check, Database, Index, Table, View, XIndex, XIndexColumn
def _check_supports_strict(): def _check_supports_strict():
@ -21,10 +21,10 @@ def test_view_names(fresh_db):
def test_table_names_fts4(existing_db): def test_table_names_fts4(existing_db):
existing_db["woo"].insert({"title": "Hello"}).enable_fts( existing_db.table("woo").insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4" ["title"], fts_version="FTS4"
) )
existing_db["woo2"].insert({"title": "Hello"}).enable_fts( existing_db.table("woo2").insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS5" ["title"], fts_version="FTS5"
) )
assert ["woo_fts"] == existing_db.table_names(fts4=True) assert ["woo_fts"] == existing_db.table_names(fts4=True)
@ -32,17 +32,17 @@ def test_table_names_fts4(existing_db):
def test_detect_fts(existing_db): def test_detect_fts(existing_db):
existing_db["woo"].insert({"title": "Hello"}).enable_fts( existing_db.table("woo").insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4" ["title"], fts_version="FTS4"
) )
existing_db["woo2"].insert({"title": "Hello"}).enable_fts( existing_db.table("woo2").insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS5" ["title"], fts_version="FTS5"
) )
assert "woo_fts" == existing_db["woo"].detect_fts() assert "woo_fts" == existing_db.table("woo").detect_fts()
assert "woo_fts" == existing_db["woo_fts"].detect_fts() assert "woo_fts" == existing_db.table("woo_fts").detect_fts()
assert "woo2_fts" == existing_db["woo2"].detect_fts() assert "woo2_fts" == existing_db.table("woo2").detect_fts()
assert "woo2_fts" == existing_db["woo2_fts"].detect_fts() assert "woo2_fts" == existing_db.table("woo2_fts").detect_fts()
assert existing_db["foo"].detect_fts() is None assert existing_db.table("foo").detect_fts() is None
@pytest.mark.parametrize("reverse_order", (True, False)) @pytest.mark.parametrize("reverse_order", (True, False))
@ -52,14 +52,14 @@ def test_detect_fts_similar_tables(fresh_db, reverse_order):
if reverse_order: if reverse_order:
table1, table2 = table2, table1 table1, table2 = table2, table1
fresh_db[table1].insert({"title": "Hello"}).enable_fts( fresh_db.table(table1).insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4" ["title"], fts_version="FTS4"
) )
fresh_db[table2].insert({"title": "Hello"}).enable_fts( fresh_db.table(table2).insert({"title": "Hello"}).enable_fts(
["title"], fts_version="FTS4" ["title"], fts_version="FTS4"
) )
assert fresh_db[table1].detect_fts() == f"{table1}_fts" assert fresh_db.table(table1).detect_fts() == f"{table1}_fts"
assert fresh_db[table2].detect_fts() == f"{table2}_fts" assert fresh_db.table(table2).detect_fts() == f"{table2}_fts"
def test_tables(existing_db): def test_tables(existing_db):
@ -77,26 +77,34 @@ def test_views(fresh_db):
assert view.columns_dict == {"1": str} assert view.columns_dict == {"1": str}
def test_getitem_returns_table_or_view(fresh_db):
fresh_db.table("items").insert({"id": 1}, pk="id")
fresh_db.create_view("item_ids", "select id from items")
assert isinstance(fresh_db["items"], Table)
assert isinstance(fresh_db["item_ids"], View)
def test_count(existing_db): def test_count(existing_db):
assert existing_db["foo"].count == 3 assert existing_db.table("foo").count == 3
assert existing_db["foo"].count_where() == 3 assert existing_db.table("foo").count_where() == 3
assert existing_db["foo"].execute_count() == 3 assert existing_db.table("foo").execute_count() == 3
def test_count_where(existing_db): def test_count_where(existing_db):
assert existing_db["foo"].count_where("text != ?", ["two"]) == 2 assert existing_db.table("foo").count_where("text != ?", ["two"]) == 2
assert existing_db["foo"].count_where("text != :t", {"t": "two"}) == 2 assert existing_db.table("foo").count_where("text != :t", {"t": "two"}) == 2
def test_columns(existing_db): def test_columns(existing_db):
table = existing_db["foo"] table = existing_db.table("foo")
assert [{"name": "text", "type": "TEXT"}] == [ assert [{"name": "text", "type": "TEXT"}] == [
{"name": col.name, "type": col.type} for col in table.columns {"name": col.name, "type": col.type} for col in table.columns
] ]
def test_table_schema(existing_db): def test_table_schema(existing_db):
assert existing_db["foo"].schema == "CREATE TABLE foo (text TEXT)" assert existing_db.table("foo").schema == "CREATE TABLE foo (text TEXT)"
def test_database_schema(existing_db): def test_database_schema(existing_db):
@ -104,9 +112,9 @@ def test_database_schema(existing_db):
def test_table_repr(fresh_db): def test_table_repr(fresh_db):
table = fresh_db["dogs"].insert({"name": "Cleo", "age": 4}) table = fresh_db.table("dogs").insert({"name": "Cleo", "age": 4})
assert "<Table dogs (name, age)>" == repr(table) assert "<Table dogs (name, age)>" == repr(table)
assert "<Table cats (does not exist yet)>" == repr(fresh_db["cats"]) assert "<Table cats (does not exist yet)>" == repr(fresh_db.table("cats"))
def test_indexes(fresh_db): def test_indexes(fresh_db):
@ -125,7 +133,7 @@ def test_indexes(fresh_db):
columns=["c2", "c3"], columns=["c2", "c3"],
), ),
Index(seq=1, name="Gosh_c1", unique=0, origin="c", partial=0, columns=["c1"]), Index(seq=1, name="Gosh_c1", unique=0, origin="c", partial=0, columns=["c1"]),
] == fresh_db["Gosh"].indexes ] == fresh_db.table("Gosh").indexes
def test_xindexes(fresh_db): def test_xindexes(fresh_db):
@ -134,7 +142,7 @@ def test_xindexes(fresh_db):
create index Gosh_c1 on Gosh(c1); create index Gosh_c1 on Gosh(c1);
create index Gosh_c2c3 on Gosh(c2, c3 desc); create index Gosh_c2c3 on Gosh(c2, c3 desc);
""") """)
assert fresh_db["Gosh"].xindexes == [ assert fresh_db.table("Gosh").xindexes == [
XIndex( XIndex(
name="Gosh_c2c3", name="Gosh_c2c3",
columns=[ columns=[
@ -153,6 +161,31 @@ def test_xindexes(fresh_db):
] ]
def test_indexes_with_double_quotes_in_identifiers(fresh_db):
fresh_db['Go"sh'].insert({"id": 1, 'c"1': 2}, pk="id")
fresh_db['Go"sh'].create_index(['c"1'])
assert [(index.name, index.columns) for index in fresh_db['Go"sh'].indexes] == [
('idx_Go"sh_c"1', ['c"1'])
]
assert fresh_db['Go"sh'].xindexes == [
XIndex(
name='idx_Go"sh_c"1',
columns=[
XIndexColumn(seqno=0, cid=1, name='c"1', desc=0, coll="BINARY", key=1),
XIndexColumn(seqno=1, cid=-1, name=None, desc=0, coll="BINARY", key=0),
],
)
]
def test_transform_table_with_double_quotes_in_identifiers(fresh_db):
fresh_db['Go"sh'].insert({"id": 1, 'c"1': 2, "c2": 3}, pk="id")
fresh_db['Go"sh'].create_index(['c"1'])
fresh_db['Go"sh'].transform(types={"c2": str})
assert fresh_db['Go"sh'].columns_dict["c2"] is str
assert [index.columns for index in fresh_db['Go"sh'].indexes] == [['c"1']]
@pytest.mark.parametrize( @pytest.mark.parametrize(
"column,expected_table_guess", "column,expected_table_guess",
( (
@ -166,15 +199,15 @@ def test_xindexes(fresh_db):
def test_guess_foreign_table(fresh_db, column, expected_table_guess): def test_guess_foreign_table(fresh_db, column, expected_table_guess):
fresh_db.create_table("authors", {"name": str}) fresh_db.create_table("authors", {"name": str})
fresh_db.create_table("genre", {"name": str}) fresh_db.create_table("genre", {"name": str})
assert expected_table_guess == fresh_db["books"].guess_foreign_table(column) assert expected_table_guess == fresh_db.table("books").guess_foreign_table(column)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"pk,expected", ((None, ["rowid"]), ("id", ["id"]), (["id", "id2"], ["id", "id2"])) "pk,expected", ((None, ["rowid"]), ("id", ["id"]), (["id", "id2"], ["id", "id2"]))
) )
def test_pks(fresh_db, pk, expected): def test_pks(fresh_db, pk, expected):
fresh_db["foo"].insert_all([{"id": 1, "id2": 2}], pk=pk) fresh_db.table("foo").insert_all([{"id": 1, "id2": 2}], pk=pk)
assert expected == fresh_db["foo"].pks assert expected == fresh_db.table("foo").pks
def test_checks(fresh_db): def test_checks(fresh_db):
@ -185,7 +218,7 @@ def test_checks(fresh_db):
CONSTRAINT within_maximum CHECK(score <= maximum) CONSTRAINT within_maximum CHECK(score <= maximum)
) )
""") """)
scores = fresh_db["scores"] scores = fresh_db.table("scores")
expected_column = Check("score > 0", name="positive", column="score") expected_column = Check("score > 0", name="positive", column="score")
expected_table = Check("score <= maximum", name="within_maximum") expected_table = Check("score <= maximum", name="within_maximum")
assert scores.checks == [expected_column, expected_table] assert scores.checks == [expected_column, expected_table]
@ -195,26 +228,26 @@ def test_checks(fresh_db):
def test_checks_nonexistent_and_virtual_tables(fresh_db): def test_checks_nonexistent_and_virtual_tables(fresh_db):
assert fresh_db["does_not_exist"].checks == [] assert fresh_db.table("does_not_exist").checks == []
fresh_db["searchable"].insert({"text": "hello"}).enable_fts( fresh_db.table("searchable").insert({"text": "hello"}).enable_fts(
["text"], fts_version="FTS5" ["text"], fts_version="FTS5"
) )
assert fresh_db["searchable_fts"].checks == [] assert fresh_db.table("searchable_fts").checks == []
def test_triggers_and_triggers_dict(fresh_db): def test_triggers_and_triggers_dict(fresh_db):
assert [] == fresh_db.triggers assert [] == fresh_db.triggers
authors = fresh_db["authors"] authors = fresh_db.table("authors")
authors.insert_all( authors.insert_all(
[ [
{"name": "Frank Herbert", "famous_works": "Dune"}, {"name": "Frank Herbert", "famous_works": "Dune"},
{"name": "Neal Stephenson", "famous_works": "Cryptonomicon"}, {"name": "Neal Stephenson", "famous_works": "Cryptonomicon"},
] ]
) )
fresh_db["other"].insert({"foo": "bar"}) fresh_db.table("other").insert({"foo": "bar"})
assert authors.triggers == [] assert authors.triggers == []
assert authors.triggers_dict == {} assert authors.triggers_dict == {}
assert fresh_db["other"].triggers == [] assert fresh_db.table("other").triggers == []
assert fresh_db.triggers_dict == {} assert fresh_db.triggers_dict == {}
authors.enable_fts( authors.enable_fts(
["name", "famous_works"], fts_version="FTS4", create_triggers=True ["name", "famous_works"], fts_version="FTS4", create_triggers=True
@ -226,7 +259,7 @@ def test_triggers_and_triggers_dict(fresh_db):
} }
assert expected_triggers == {(t.name, t.table) for t in fresh_db.triggers} assert expected_triggers == {(t.name, t.table) for t in fresh_db.triggers}
assert expected_triggers == { assert expected_triggers == {
(t.name, t.table) for t in fresh_db["authors"].triggers (t.name, t.table) for t in fresh_db.table("authors").triggers
} }
expected_triggers = { expected_triggers = {
"authors_ai": ( "authors_ai": (
@ -246,13 +279,13 @@ def test_triggers_and_triggers_dict(fresh_db):
), ),
} }
assert authors.triggers_dict == expected_triggers assert authors.triggers_dict == expected_triggers
assert fresh_db["other"].triggers == [] assert fresh_db.table("other").triggers == []
assert fresh_db["other"].triggers_dict == {} assert fresh_db.table("other").triggers_dict == {}
assert fresh_db.triggers_dict == expected_triggers assert fresh_db.triggers_dict == expected_triggers
def test_has_counts_triggers(fresh_db): def test_has_counts_triggers(fresh_db):
authors = fresh_db["authors"] authors = fresh_db.table("authors")
authors.insert({"name": "Frank Herbert"}) authors.insert({"name": "Frank Herbert"})
assert not authors.has_counts_triggers assert not authors.has_counts_triggers
authors.enable_counts() authors.enable_counts()
@ -301,14 +334,14 @@ def test_has_counts_triggers(fresh_db):
) )
def test_virtual_table_using(fresh_db, sql, expected_name, expected_using): def test_virtual_table_using(fresh_db, sql, expected_name, expected_using):
fresh_db.execute(sql) fresh_db.execute(sql)
assert fresh_db[expected_name].virtual_table_using == expected_using assert fresh_db.table(expected_name).virtual_table_using == expected_using
def test_use_rowid(fresh_db): def test_use_rowid(fresh_db):
fresh_db["rowid_table"].insert({"name": "Cleo"}) fresh_db.table("rowid_table").insert({"name": "Cleo"})
fresh_db["regular_table"].insert({"id": 1, "name": "Cleo"}, pk="id") fresh_db.table("regular_table").insert({"id": 1, "name": "Cleo"}, pk="id")
assert fresh_db["rowid_table"].use_rowid assert fresh_db.table("rowid_table").use_rowid
assert not fresh_db["regular_table"].use_rowid assert not fresh_db.table("regular_table").use_rowid
@pytest.mark.skipif( @pytest.mark.skipif(
@ -327,7 +360,7 @@ def test_use_rowid(fresh_db):
) )
def test_table_strict(fresh_db, create_table, expected_strict): def test_table_strict(fresh_db, create_table, expected_strict):
fresh_db.execute(create_table) fresh_db.execute(create_table)
table = fresh_db["t"] table = fresh_db.table("t")
assert table.strict == expected_strict assert table.strict == expected_strict
@ -343,10 +376,10 @@ def test_table_strict(fresh_db, create_table, expected_strict):
), ),
) )
def test_table_default_values(fresh_db, value): def test_table_default_values(fresh_db, value):
fresh_db["default_values"].insert( fresh_db.table("default_values").insert(
{"nodefault": 1, "value": value}, defaults={"value": value} {"nodefault": 1, "value": value}, defaults={"value": value}
) )
default_values = fresh_db["default_values"].default_values default_values = fresh_db.table("default_values").default_values
assert default_values == {"value": value} assert default_values == {"value": value}
@ -356,8 +389,23 @@ def test_table_default_values_escaped_quotes(fresh_db):
fresh_db.execute( fresh_db.execute(
"create table t (id integer primary key, name text default 'O''Brien')" "create table t (id integer primary key, name text default 'O''Brien')"
) )
assert "default 'O''Brien'" in fresh_db["t"].schema assert "default 'O''Brien'" in fresh_db.table("t").schema
assert fresh_db["t"].default_values == {"name": "O'Brien"} assert fresh_db.table("t").default_values == {"name": "O'Brien"}
def test_table_default_values_keyword_literals(fresh_db):
fresh_db.execute(
"create table t ("
"enabled integer default TRUE, "
"disabled integer default false, "
"nullable text default NULL"
")"
)
assert fresh_db.table("t").default_values == {
"enabled": True,
"disabled": False,
"nullable": None,
}
def test_pks_use_primary_key_declaration_order(fresh_db): def test_pks_use_primary_key_declaration_order(fresh_db):
@ -365,11 +413,11 @@ def test_pks_use_primary_key_declaration_order(fresh_db):
# pks must follow the declaration order, which is what SQLite uses to # pks must follow the declaration order, which is what SQLite uses to
# resolve implicit foreign key references and compound pk lookups # resolve implicit foreign key references and compound pk lookups
fresh_db.execute("create table t (b text, a text, primary key (a, b))") fresh_db.execute("create table t (b text, a text, primary key (a, b))")
assert fresh_db["t"].pks == ["a", "b"] assert fresh_db.table("t").pks == ["a", "b"]
def test_transform_preserves_compound_pk_declaration_order(fresh_db): def test_transform_preserves_compound_pk_declaration_order(fresh_db):
fresh_db.execute("create table t (a text, b text, c text, primary key (b, a))") fresh_db.execute("create table t (a text, b text, c text, primary key (b, a))")
fresh_db["t"].transform(drop={"c"}) fresh_db.table("t").transform(drop={"c"})
assert fresh_db["t"].pks == ["b", "a"] assert fresh_db.table("t").pks == ["b", "a"]
assert 'PRIMARY KEY ("b", "a")' in fresh_db["t"].schema assert 'PRIMARY KEY ("b", "a")' in fresh_db.table("t").schema

View file

@ -19,9 +19,9 @@ def test_insert_all_list_mode_basic():
yield [2, "Bob", 25] yield [2, "Bob", 25]
yield [3, "Charlie", 35] yield [3, "Charlie", 35]
db["people"].insert_all(data_generator()) db.table("people").insert_all(data_generator())
rows = list(db["people"].rows) rows = list(db.table("people").rows)
assert len(rows) == 3 assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "age": 30} assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
assert rows[1] == {"id": 2, "name": "Bob", "age": 25} assert rows[1] == {"id": 2, "name": "Bob", "age": 25}
@ -37,10 +37,10 @@ def test_insert_all_list_mode_with_pk():
yield [1, "Alice", 95] yield [1, "Alice", 95]
yield [2, "Bob", 87] yield [2, "Bob", 87]
db["scores"].insert_all(data_generator(), pk="id") db.table("scores").insert_all(data_generator(), pk="id")
assert db["scores"].pks == ["id"] assert db.table("scores").pks == ["id"]
rows = list(db["scores"].rows) rows = list(db.table("scores").rows)
assert len(rows) == 2 assert len(rows) == 2
@ -54,7 +54,7 @@ def test_upsert_all_list_mode():
yield [1, "Alice", 100] yield [1, "Alice", 100]
yield [2, "Bob", 200] yield [2, "Bob", 200]
db["data"].insert_all(initial_data(), pk="id") db.table("data").insert_all(initial_data(), pk="id")
# Upsert with some updates and new records # Upsert with some updates and new records
def upsert_data(): def upsert_data():
@ -62,9 +62,9 @@ def test_upsert_all_list_mode():
yield [1, "Alice", 150] # Update existing yield [1, "Alice", 150] # Update existing
yield [3, "Charlie", 300] # Insert new yield [3, "Charlie", 300] # Insert new
db["data"].upsert_all(upsert_data(), pk="id") db.table("data").upsert_all(upsert_data(), pk="id")
rows = list(db["data"].rows_where(order_by="id")) rows = list(db.table("data").rows_where(order_by="id"))
assert len(rows) == 3 assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "value": 150} assert rows[0] == {"id": 1, "name": "Alice", "value": 150}
assert rows[1] == {"id": 2, "name": "Bob", "value": 200} assert rows[1] == {"id": 2, "name": "Bob", "value": 200}
@ -81,9 +81,9 @@ def test_list_mode_with_various_types():
yield [2, "Bob", 87.3, False] yield [2, "Bob", 87.3, False]
yield [3, "Charlie", None, True] yield [3, "Charlie", None, True]
db["mixed"].insert_all(data_generator()) db.table("mixed").insert_all(data_generator())
rows = list(db["mixed"].rows) rows = list(db.table("mixed").rows)
assert len(rows) == 3 assert len(rows) == 3
assert rows[0]["score"] == 95.5 assert rows[0]["score"] == 95.5
assert rows[1]["active"] == 0 # SQLite stores boolean as int assert rows[1]["active"] == 0 # SQLite stores boolean as int
@ -99,7 +99,7 @@ def test_list_mode_error_non_string_columns():
yield ["a", "b", "c"] yield ["a", "b", "c"]
with pytest.raises(ValueError, match="must be a list of column name strings"): with pytest.raises(ValueError, match="must be a list of column name strings"):
db["bad"].insert_all(bad_data()) db.table("bad").insert_all(bad_data()) # type: ignore[arg-type]
def test_list_mode_error_mixed_types(): def test_list_mode_error_mixed_types():
@ -111,7 +111,7 @@ def test_list_mode_error_mixed_types():
yield {"id": 1, "name": "Alice"} # Should be a list, not dict yield {"id": 1, "name": "Alice"} # Should be a list, not dict
with pytest.raises(ValueError, match="must also be lists"): with pytest.raises(ValueError, match="must also be lists"):
db["bad"].insert_all(bad_data()) db.table("bad").insert_all(bad_data()) # type: ignore[arg-type]
def test_list_mode_empty_after_headers(): def test_list_mode_empty_after_headers():
@ -122,9 +122,9 @@ def test_list_mode_empty_after_headers():
yield ["id", "name", "age"] yield ["id", "name", "age"]
# No data rows # No data rows
result = db["people"].insert_all(data_generator()) result = db.table("people").insert_all(data_generator())
assert result is not None assert result is not None
assert not db["people"].exists() assert not db.table("people").exists()
def test_list_mode_batch_processing(): def test_list_mode_batch_processing():
@ -136,7 +136,7 @@ def test_list_mode_batch_processing():
for i in range(1000): for i in range(1000):
yield [i, f"value_{i}"] yield [i, f"value_{i}"]
db["large"].insert_all(large_data(), batch_size=100) db.table("large").insert_all(large_data(), batch_size=100)
count = db.execute("SELECT COUNT(*) as c FROM large").fetchone()[0] count = db.execute("SELECT COUNT(*) as c FROM large").fetchone()[0]
assert count == 1000 assert count == 1000
@ -152,9 +152,9 @@ def test_list_mode_shorter_rows():
yield [2, "Bob"] # Missing age and city yield [2, "Bob"] # Missing age and city
yield [3, "Charlie", 35] # Missing city yield [3, "Charlie", 35] # Missing city
db["people"].insert_all(data_generator()) db.table("people").insert_all(data_generator())
rows = list(db["people"].rows_where(order_by="id")) rows = list(db.table("people").rows_where(order_by="id"))
assert rows[0] == {"id": 1, "name": "Alice", "age": 30, "city": "NYC"} assert rows[0] == {"id": 1, "name": "Alice", "age": 30, "city": "NYC"}
assert rows[1] == {"id": 2, "name": "Bob", "age": None, "city": None} assert rows[1] == {"id": 2, "name": "Bob", "age": None, "city": None}
assert rows[2] == {"id": 3, "name": "Charlie", "age": 35, "city": None} assert rows[2] == {"id": 3, "name": "Charlie", "age": 35, "city": None}
@ -170,9 +170,9 @@ def test_backwards_compatibility_dict_mode():
{"id": 2, "name": "Bob", "age": 25}, {"id": 2, "name": "Bob", "age": 25},
] ]
db["people"].insert_all(data) db.table("people").insert_all(data)
rows = list(db["people"].rows) rows = list(db.table("people").rows)
assert len(rows) == 2 assert len(rows) == 2
assert rows[0] == {"id": 1, "name": "Alice", "age": 30} assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
@ -189,9 +189,9 @@ def test_insert_all_tuple_mode_basic():
yield (2, "Bob", 25) yield (2, "Bob", 25)
yield (3, "Charlie", 35) yield (3, "Charlie", 35)
db["people"].insert_all(data_generator()) db.table("people").insert_all(data_generator())
rows = list(db["people"].rows) rows = list(db.table("people").rows)
assert len(rows) == 3 assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "age": 30} assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
assert rows[1] == {"id": 2, "name": "Bob", "age": 25} assert rows[1] == {"id": 2, "name": "Bob", "age": 25}
@ -211,9 +211,9 @@ def test_insert_all_mixed_list_tuple():
yield [3, "Charlie", 35] yield [3, "Charlie", 35]
yield (4, "Diana", 40) yield (4, "Diana", 40)
db["people"].insert_all(data_generator()) db.table("people").insert_all(data_generator())
rows = list(db["people"].rows) rows = list(db.table("people").rows)
assert len(rows) == 4 assert len(rows) == 4
assert rows[0] == {"id": 1, "name": "Alice", "age": 30} assert rows[0] == {"id": 1, "name": "Alice", "age": 30}
assert rows[1] == {"id": 2, "name": "Bob", "age": 25} assert rows[1] == {"id": 2, "name": "Bob", "age": 25}
@ -231,7 +231,7 @@ def test_upsert_all_tuple_mode():
yield (1, "Alice", 100) yield (1, "Alice", 100)
yield (2, "Bob", 200) yield (2, "Bob", 200)
db["data"].insert_all(initial_data(), pk="id") db.table("data").insert_all(initial_data(), pk="id")
# Upsert with tuples # Upsert with tuples
def upsert_data(): def upsert_data():
@ -239,9 +239,9 @@ def test_upsert_all_tuple_mode():
yield (1, "Alice", 150) # Update existing yield (1, "Alice", 150) # Update existing
yield (3, "Charlie", 300) # Insert new yield (3, "Charlie", 300) # Insert new
db["data"].upsert_all(upsert_data(), pk="id") db.table("data").upsert_all(upsert_data(), pk="id")
rows = list(db["data"].rows_where(order_by="id")) rows = list(db.table("data").rows_where(order_by="id"))
assert len(rows) == 3 assert len(rows) == 3
assert rows[0] == {"id": 1, "name": "Alice", "value": 150} assert rows[0] == {"id": 1, "name": "Alice", "value": 150}
assert rows[1] == {"id": 2, "name": "Bob", "value": 200} assert rows[1] == {"id": 2, "name": "Bob", "value": 200}
@ -258,9 +258,9 @@ def test_tuple_mode_shorter_rows():
yield 2, "Bob" # Missing age and city yield 2, "Bob" # Missing age and city
yield 3, "Charlie", 35 # Missing city yield 3, "Charlie", 35 # Missing city
db["people"].insert_all(data_generator()) db.table("people").insert_all(data_generator())
rows = list(db["people"].rows_where(order_by="id")) rows = list(db.table("people").rows_where(order_by="id"))
assert rows[0] == {"id": 1, "name": "Alice", "age": 30, "city": "NYC"} assert rows[0] == {"id": 1, "name": "Alice", "age": 30, "city": "NYC"}
assert rows[1] == {"id": 2, "name": "Bob", "age": None, "city": None} assert rows[1] == {"id": 2, "name": "Bob", "age": None, "city": None}
assert rows[2] == {"id": 3, "name": "Charlie", "age": 35, "city": None} assert rows[2] == {"id": 3, "name": "Charlie", "age": 35, "city": None}
@ -271,18 +271,18 @@ def test_list_mode_single_record_upsert_last_pk():
db = Database(memory=True) db = Database(memory=True)
# Create table first # Create table first
db["data"].insert({"id": 1, "name": "Alice", "value": 100}, pk="id") db.table("data").insert({"id": 1, "name": "Alice", "value": 100}, pk="id")
# Now upsert a single record using list mode # Now upsert a single record using list mode
def upsert_data(): def upsert_data():
yield ["id", "name", "value"] yield ["id", "name", "value"]
yield [1, "Alice", 150] # Update existing yield [1, "Alice", 150] # Update existing
table = db["data"] table = db.table("data")
table.upsert_all(upsert_data(), pk="id") table.upsert_all(upsert_data(), pk="id")
# Verify the data was updated # Verify the data was updated
rows = list(db["data"].rows) rows = list(db.table("data").rows)
assert rows == [{"id": 1, "name": "Alice", "value": 150}] assert rows == [{"id": 1, "name": "Alice", "value": 150}]
# Verify last_pk is populated correctly # Verify last_pk is populated correctly

View file

@ -4,7 +4,7 @@ from sqlite_utils.db import Index
def test_lookup_new_table(fresh_db): def test_lookup_new_table(fresh_db):
species = fresh_db["species"] species = fresh_db.table("species")
palm_id = species.lookup({"name": "Palm"}) palm_id = species.lookup({"name": "Palm"})
oak_id = species.lookup({"name": "Oak"}) oak_id = species.lookup({"name": "Oak"})
cherry_id = species.lookup({"name": "Cherry"}) cherry_id = species.lookup({"name": "Cherry"})
@ -26,7 +26,7 @@ def test_lookup_new_table(fresh_db):
def test_lookup_new_table_compound_key(fresh_db): def test_lookup_new_table_compound_key(fresh_db):
species = fresh_db["species"] species = fresh_db.table("species")
palm_id = species.lookup({"name": "Palm", "type": "Tree"}) palm_id = species.lookup({"name": "Palm", "type": "Tree"})
oak_id = species.lookup({"name": "Oak", "type": "Tree"}) oak_id = species.lookup({"name": "Oak", "type": "Tree"})
assert palm_id == species.lookup({"name": "Palm", "type": "Tree"}) assert palm_id == species.lookup({"name": "Palm", "type": "Tree"})
@ -70,7 +70,7 @@ def test_lookup_fails_if_constraint_cannot_be_added(fresh_db):
def test_lookup_with_extra_values(fresh_db): def test_lookup_with_extra_values(fresh_db):
species = fresh_db["species"] species = fresh_db.table("species")
id = species.lookup({"name": "Palm", "type": "Tree"}, {"first_seen": "2020-01-01"}) id = species.lookup({"name": "Palm", "type": "Tree"}, {"first_seen": "2020-01-01"})
assert species.get(id) == { assert species.get(id) == {
"id": 1, "id": 1,
@ -90,9 +90,9 @@ def test_lookup_with_extra_values(fresh_db):
def test_lookup_with_extra_insert_parameters(fresh_db): def test_lookup_with_extra_insert_parameters(fresh_db):
other_table = fresh_db["other_table"] other_table = fresh_db.table("other_table")
other_table.insert({"id": 1, "name": "Name"}, pk="id") other_table.insert({"id": 1, "name": "Name"}, pk="id")
species = fresh_db["species"] species = fresh_db.table("species")
id = species.lookup( id = species.lookup(
{"name": "Palm", "type": "Tree"}, {"name": "Palm", "type": "Tree"},
{ {
@ -156,15 +156,15 @@ def test_lookup_with_extra_insert_parameters(fresh_db):
@pytest.mark.parametrize("strict", (False, True)) @pytest.mark.parametrize("strict", (False, True))
def test_lookup_new_table_strict(fresh_db, strict): def test_lookup_new_table_strict(fresh_db, strict):
fresh_db["species"].lookup({"name": "Palm"}, strict=strict) fresh_db.table("species").lookup({"name": "Palm"}, strict=strict)
assert fresh_db["species"].strict == strict or not fresh_db.supports_strict assert fresh_db.table("species").strict == strict or not fresh_db.supports_strict
def test_lookup_null_value_idempotent(fresh_db): def test_lookup_null_value_idempotent(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/186 # https://github.com/simonw/sqlite-utils/issues/186
# Repeated lookups of a null value should return the same row, # Repeated lookups of a null value should return the same row,
# not insert a duplicate row each time # not insert a duplicate row each time
species = fresh_db["species"] species = fresh_db.table("species")
first_id = species.lookup({"name": None}) first_id = species.lookup({"name": None})
second_id = species.lookup({"name": None}) second_id = species.lookup({"name": None})
assert first_id == second_id assert first_id == second_id
@ -172,7 +172,7 @@ def test_lookup_null_value_idempotent(fresh_db):
def test_lookup_compound_key_with_null_idempotent(fresh_db): def test_lookup_compound_key_with_null_idempotent(fresh_db):
species = fresh_db["species"] species = fresh_db.table("species")
palm_id = species.lookup({"name": "Palm", "type": None}) palm_id = species.lookup({"name": "Palm", "type": None})
oak_id = species.lookup({"name": "Oak", "type": "Tree"}) oak_id = species.lookup({"name": "Oak", "type": "Tree"})
assert palm_id == species.lookup({"name": "Palm", "type": None}) assert palm_id == species.lookup({"name": "Palm", "type": None})

View file

@ -4,45 +4,45 @@ from sqlite_utils.db import ForeignKey, NoObviousTable
def test_insert_m2m_single(fresh_db): def test_insert_m2m_single(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m( dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m(
"humans", {"id": 1, "name": "Natalie D"}, pk="id" "humans", {"id": 1, "name": "Natalie D"}, pk="id"
) )
assert {"dogs_humans", "humans", "dogs"} == set(fresh_db.table_names()) assert {"dogs_humans", "humans", "dogs"} == set(fresh_db.table_names())
humans = fresh_db["humans"] humans = fresh_db.table("humans")
dogs_humans = fresh_db["dogs_humans"] dogs_humans = fresh_db.table("dogs_humans")
assert [{"id": 1, "name": "Natalie D"}] == list(humans.rows) assert [{"id": 1, "name": "Natalie D"}] == list(humans.rows)
assert [{"humans_id": 1, "dogs_id": 1}] == list(dogs_humans.rows) assert [{"humans_id": 1, "dogs_id": 1}] == list(dogs_humans.rows)
def test_insert_m2m_alter(fresh_db): def test_insert_m2m_alter(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m( dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m(
"humans", {"id": 1, "name": "Natalie D"}, pk="id" "humans", {"id": 1, "name": "Natalie D"}, pk="id"
) )
dogs.update(1).m2m( dogs.update(1).m2m(
"humans", {"id": 2, "name": "Simon W", "nerd": True}, pk="id", alter=True "humans", {"id": 2, "name": "Simon W", "nerd": True}, pk="id", alter=True
) )
assert list(fresh_db["humans"].rows) == [ assert list(fresh_db.table("humans").rows) == [
{"id": 1, "name": "Natalie D", "nerd": None}, {"id": 1, "name": "Natalie D", "nerd": None},
{"id": 2, "name": "Simon W", "nerd": 1}, {"id": 2, "name": "Simon W", "nerd": 1},
] ]
assert list(fresh_db["dogs_humans"].rows) == [ assert list(fresh_db.table("dogs_humans").rows) == [
{"humans_id": 1, "dogs_id": 1}, {"humans_id": 1, "dogs_id": 1},
{"humans_id": 2, "dogs_id": 1}, {"humans_id": 2, "dogs_id": 1},
] ]
def test_insert_m2m_list(fresh_db): def test_insert_m2m_list(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m( dogs.insert({"id": 1, "name": "Cleo"}, pk="id").m2m(
"humans", "humans",
[{"id": 1, "name": "Natalie D"}, {"id": 2, "name": "Simon W"}], [{"id": 1, "name": "Natalie D"}, {"id": 2, "name": "Simon W"}],
pk="id", pk="id",
) )
assert {"dogs", "humans", "dogs_humans"} == set(fresh_db.table_names()) assert {"dogs", "humans", "dogs_humans"} == set(fresh_db.table_names())
humans = fresh_db["humans"] humans = fresh_db.table("humans")
dogs_humans = fresh_db["dogs_humans"] dogs_humans = fresh_db.table("dogs_humans")
assert [{"humans_id": 1, "dogs_id": 1}, {"humans_id": 2, "dogs_id": 1}] == list( assert [{"humans_id": 1, "dogs_id": 1}, {"humans_id": 2, "dogs_id": 1}] == list(
dogs_humans.rows dogs_humans.rows
) )
@ -68,7 +68,7 @@ def test_insert_m2m_iterable(fresh_db):
def iterable(): def iterable():
yield from iterable_records yield from iterable_records
platypuses = fresh_db["platypuses"] platypuses = fresh_db.table("platypuses")
platypuses.insert({"id": 1, "name": "Perry"}, pk="id").m2m( platypuses.insert({"id": 1, "name": "Perry"}, pk="id").m2m(
"humans", "humans",
iterable(), iterable(),
@ -76,8 +76,8 @@ def test_insert_m2m_iterable(fresh_db):
) )
assert {"platypuses", "humans", "humans_platypuses"} == set(fresh_db.table_names()) assert {"platypuses", "humans", "humans_platypuses"} == set(fresh_db.table_names())
humans = fresh_db["humans"] humans = fresh_db.table("humans")
humans_platypuses = fresh_db["humans_platypuses"] humans_platypuses = fresh_db.table("humans_platypuses")
assert [ assert [
{"humans_id": 1, "platypuses_id": 1}, {"humans_id": 1, "platypuses_id": 1},
{"humans_id": 2, "platypuses_id": 1}, {"humans_id": 2, "platypuses_id": 1},
@ -111,14 +111,14 @@ def test_m2m_with_table_objects(fresh_db):
assert expected_tables == set(fresh_db.table_names()) assert expected_tables == set(fresh_db.table_names())
assert dogs.count == 1 assert dogs.count == 1
assert humans.count == 2 assert humans.count == 2
assert fresh_db["dogs_humans"].count == 2 assert fresh_db.table("dogs_humans").count == 2
def test_m2m_lookup(fresh_db): def test_m2m_lookup(fresh_db):
people = fresh_db.table("people", pk="id") people = fresh_db.table("people", pk="id")
people.insert({"name": "Wahyu"}).m2m("tags", lookup={"tag": "Coworker"}) people.insert({"name": "Wahyu"}).m2m("tags", lookup={"tag": "Coworker"})
people_tags = fresh_db["people_tags"] people_tags = fresh_db.table("people_tags")
tags = fresh_db["tags"] tags = fresh_db.table("tags")
assert people_tags.exists() assert people_tags.exists()
assert tags.exists() assert tags.exists()
assert [ assert [
@ -150,9 +150,9 @@ def test_m2m_explicit_table_name_argument(fresh_db):
people.insert({"name": "Wahyu"}).m2m( people.insert({"name": "Wahyu"}).m2m(
"tags", lookup={"tag": "Coworker"}, m2m_table="tagged" "tags", lookup={"tag": "Coworker"}, m2m_table="tagged"
) )
assert fresh_db["tags"].exists assert fresh_db.table("tags").exists
assert fresh_db["tagged"].exists assert fresh_db.table("tagged").exists
assert not fresh_db["people_tags"].exists() assert not fresh_db.table("people_tags").exists()
def test_m2m_table_candidates(fresh_db): def test_m2m_table_candidates(fresh_db):
@ -181,25 +181,25 @@ def test_uses_existing_m2m_table_if_exists(fresh_db):
# Code should look for an existing table with fks to both tables # Code should look for an existing table with fks to both tables
# and use that if it exists. # and use that if it exists.
people = fresh_db.create_table("people", {"id": int, "name": str}, pk="id") people = fresh_db.create_table("people", {"id": int, "name": str}, pk="id")
fresh_db["tags"].lookup({"tag": "Coworker"}) fresh_db.table("tags").lookup({"tag": "Coworker"})
fresh_db.create_table( fresh_db.create_table(
"tagged", "tagged",
{"people_id": int, "tags_id": int}, {"people_id": int, "tags_id": int},
foreign_keys=["people_id", "tags_id"], foreign_keys=["people_id", "tags_id"],
) )
people.insert({"name": "Wahyu"}).m2m("tags", lookup={"tag": "Coworker"}) people.insert({"name": "Wahyu"}).m2m("tags", lookup={"tag": "Coworker"})
assert fresh_db["tags"].exists() assert fresh_db.table("tags").exists()
assert fresh_db["tagged"].exists() assert fresh_db.table("tagged").exists()
assert not fresh_db["people_tags"].exists() assert not fresh_db.table("people_tags").exists()
assert not fresh_db["tags_people"].exists() assert not fresh_db.table("tags_people").exists()
assert [{"people_id": 1, "tags_id": 1}] == list(fresh_db["tagged"].rows) assert [{"people_id": 1, "tags_id": 1}] == list(fresh_db.table("tagged").rows)
def test_requires_explicit_m2m_table_if_multiple_options(fresh_db): def test_requires_explicit_m2m_table_if_multiple_options(fresh_db):
# If the code scans for m2m tables and finds more than one candidate # If the code scans for m2m tables and finds more than one candidate
# it should require that the m2m_table=x argument is used # it should require that the m2m_table=x argument is used
people = fresh_db.create_table("people", {"id": int, "name": str}, pk="id") people = fresh_db.create_table("people", {"id": int, "name": str}, pk="id")
fresh_db["tags"].lookup({"tag": "Coworker"}) fresh_db.table("tags").lookup({"tag": "Coworker"})
fresh_db.create_table( fresh_db.create_table(
"tagged", "tagged",
{"people_id": int, "tags_id": int}, {"people_id": int, "tags_id": int},

View file

@ -10,11 +10,11 @@ def migrations():
@migrations() @migrations()
def m001(db): def m001(db):
db["dogs"].insert({"name": "Cleo"}) db.table("dogs").insert({"name": "Cleo"})
@migrations() @migrations()
def m002(db): def m002(db):
db["cats"].create({"name": str}) db.table("cats").create({"name": str})
db.execute("insert into dogs (name) values ('Pancakes')") db.execute("insert into dogs (name) values ('Pancakes')")
return migrations return migrations
@ -28,11 +28,11 @@ def migrations_not_ordered_alphabetically():
@migrations() @migrations()
def m002(db): def m002(db):
db["dogs"].insert({"name": "Cleo"}) db.table("dogs").insert({"name": "Cleo"})
@migrations() @migrations()
def m001(db): def m001(db):
db["cats"].create({"name": str}) db.table("cats").create({"name": str})
db.execute("insert into dogs (name) values ('Pancakes')") db.execute("insert into dogs (name) values ('Pancakes')")
return migrations return migrations
@ -44,7 +44,7 @@ def migrations2():
@migrations() @migrations()
def m001(db): def m001(db):
db["dogs2"].insert({"name": "Cleo"}) db.table("dogs2").insert({"name": "Cleo"})
return migrations return migrations
@ -96,7 +96,7 @@ def test_applied_at_is_a_string(migrations):
def test_failing_migration_rolls_back(migrations): def test_failing_migration_rolls_back(migrations):
@migrations() @migrations()
def m003(db): def m003(db):
db["birds"].create({"name": str}) db.table("birds").create({"name": str})
db.execute("insert into dogs (name) values ('Dozer')") db.execute("insert into dogs (name) values ('Dozer')")
raise ValueError("boom") raise ValueError("boom")
@ -105,7 +105,7 @@ def test_failing_migration_rolls_back(migrations):
migrations.apply(db) migrations.apply(db)
# m001 and m002 committed before the failure and stay applied # m001 and m002 committed before the failure and stay applied
assert set(db.table_names()) == {"_sqlite_migrations", "dogs", "cats"} assert set(db.table_names()) == {"_sqlite_migrations", "dogs", "cats"}
assert [r["name"] for r in db["dogs"].rows] == ["Cleo", "Pancakes"] assert [r["name"] for r in db.table("dogs").rows] == ["Cleo", "Pancakes"]
assert [m.name for m in migrations.applied(db)] == ["m001", "m002"] assert [m.name for m in migrations.applied(db)] == ["m001", "m002"]
# Everything m003 did was rolled back and it is still pending # Everything m003 did was rolled back and it is still pending
assert [m.name for m in migrations.pending(db)] == ["m003"] assert [m.name for m in migrations.pending(db)] == ["m003"]
@ -117,11 +117,11 @@ def test_rerun_after_failure_applies_each_migration_once():
@migrations() @migrations()
def m001(db): def m001(db):
db["dogs"].insert({"name": "Cleo"}) db.table("dogs").insert({"name": "Cleo"})
@migrations() @migrations()
def m002(db): def m002(db):
db["dogs"].insert({"name": "Pancakes"}) db.table("dogs").insert({"name": "Pancakes"})
if state["fail"]: if state["fail"]:
raise ValueError("boom") raise ValueError("boom")
@ -131,7 +131,7 @@ def test_rerun_after_failure_applies_each_migration_once():
state["fail"] = False state["fail"] = False
migrations.apply(db) migrations.apply(db)
# m001 must not have been re-applied, m002 applied exactly once # m001 must not have been re-applied, m002 applied exactly once
assert [r["name"] for r in db["dogs"].rows] == ["Cleo", "Pancakes"] assert [r["name"] for r in db.table("dogs").rows] == ["Cleo", "Pancakes"]
assert [m.name for m in migrations.applied(db)] == ["m001", "m002"] assert [m.name for m in migrations.applied(db)] == ["m001", "m002"]
@ -142,7 +142,7 @@ def test_non_transactional_migration_allows_vacuum(tmpdir):
@migrations() @migrations()
def m001(db): def m001(db):
db["dogs"].insert({"name": "Cleo"}) db.table("dogs").insert({"name": "Cleo"})
@migrations(transactional=False) @migrations(transactional=False)
def m002(db): def m002(db):
@ -185,11 +185,13 @@ def test_apply_composes_inside_outer_transaction(migrations):
) )
def test_upgrades_sqlite_migrations(migrations, create_table, pk): def test_upgrades_sqlite_migrations(migrations, create_table, pk):
db = sqlite_utils.Database(memory=True) db = sqlite_utils.Database(memory=True)
db["_sqlite_migrations"].create(create_table, pk=pk) db.table("_sqlite_migrations").create(create_table, pk=pk)
assert db.table_names() == ["_sqlite_migrations"] assert db.table_names() == ["_sqlite_migrations"]
assert db["_sqlite_migrations"].pks == ([pk] if isinstance(pk, str) else list(pk)) assert db.table("_sqlite_migrations").pks == (
[pk] if isinstance(pk, str) else list(pk)
)
migrations.apply(db) migrations.apply(db)
assert db["_sqlite_migrations"].pks == ["id"] assert db.table("_sqlite_migrations").pks == ["id"]
def test_pending_and_applied_are_read_only(migrations): def test_pending_and_applied_are_read_only(migrations):
@ -227,7 +229,7 @@ def test_stop_before_applied_migration_errors(migrations):
assert "m001" in str(ex.value) assert "m001" in str(ex.value)
assert "already been applied" in str(ex.value) assert "already been applied" in str(ex.value)
# Nothing else was applied # Nothing else was applied
assert not db["cats"].exists() assert not db.table("cats").exists()
def test_stop_before_applied_migration_errors_before_any_apply(migrations): def test_stop_before_applied_migration_errors_before_any_apply(migrations):
@ -238,9 +240,9 @@ def test_stop_before_applied_migration_errors_before_any_apply(migrations):
@only_second() @only_second()
def m002(db): def m002(db):
db["cats"].create({"name": str}) db.table("cats").create({"name": str})
only_second.apply(db) # m002 applied, m001 still pending only_second.apply(db) # m002 applied, m001 still pending
with pytest.raises(ValueError): with pytest.raises(ValueError):
migrations.apply(db, stop_before="m002") migrations.apply(db, stop_before="m002")
assert not db["dogs"].exists() assert not db.table("dogs").exists()

View file

@ -112,7 +112,7 @@ def test_mutator_commits_by_default(tmp_path, mutate, expected_rows):
db = seed_database(path) db = seed_database(path)
assert not db.conn.in_transaction assert not db.conn.in_transaction
mutate(db["items"]) mutate(db.table("items"))
assert current_rows(db) == expected_rows assert current_rows(db) == expected_rows
assert not db.conn.in_transaction assert not db.conn.in_transaction
@ -127,7 +127,7 @@ def test_mutator_commits_with_outer_atomic(tmp_path, mutate, expected_rows):
with db.atomic(): with db.atomic():
assert db.conn.in_transaction assert db.conn.in_transaction
mutate(db["items"]) mutate(db.table("items"))
assert current_rows(db) == expected_rows assert current_rows(db) == expected_rows
assert db.conn.in_transaction assert db.conn.in_transaction
@ -143,7 +143,7 @@ def test_mutator_rolls_back_outer_atomic(tmp_path, mutate, expected_rows):
db = seed_database(path) db = seed_database(path)
with pytest.raises(RollbackTest), db.atomic(): with pytest.raises(RollbackTest), db.atomic():
mutate(db["items"]) mutate(db.table("items"))
assert current_rows(db) == expected_rows assert current_rows(db) == expected_rows
assert db.conn.in_transaction assert db.conn.in_transaction
raise RollbackTest raise RollbackTest

View file

@ -6,7 +6,7 @@ from sqlite_utils.utils import sqlite3
def test_query(fresh_db): def test_query(fresh_db):
fresh_db["dogs"].insert_all([{"name": "Cleo"}, {"name": "Pancakes"}]) fresh_db.table("dogs").insert_all([{"name": "Cleo"}, {"name": "Pancakes"}])
results = fresh_db.query("select * from dogs order by name desc") results = fresh_db.query("select * from dogs order by name desc")
assert isinstance(results, types.GeneratorType) assert isinstance(results, types.GeneratorType)
assert list(results) == [{"name": "Pancakes"}, {"name": "Cleo"}] assert list(results) == [{"name": "Pancakes"}, {"name": "Cleo"}]
@ -20,13 +20,13 @@ def test_query_executes_eagerly(fresh_db):
def test_query_rejects_statements_that_return_no_rows(fresh_db): def test_query_rejects_statements_that_return_no_rows(fresh_db):
fresh_db["dogs"].insert({"name": "Cleo"}) fresh_db.table("dogs").insert({"name": "Cleo"})
with pytest.raises(ValueError) as ex: with pytest.raises(ValueError) as ex:
fresh_db.query("update dogs set name = 'Cleopaws'") fresh_db.query("update dogs set name = 'Cleopaws'")
assert "execute()" in str(ex.value) assert "execute()" in str(ex.value)
# The rejected update was rolled back, and no transaction is left open # The rejected update was rolled back, and no transaction is left open
assert not fresh_db.conn.in_transaction assert not fresh_db.conn.in_transaction
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"] assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
def test_query_rejected_ddl_is_rolled_back(fresh_db): def test_query_rejected_ddl_is_rolled_back(fresh_db):
@ -37,7 +37,7 @@ def test_query_rejected_ddl_is_rolled_back(fresh_db):
def test_query_rejected_write_inside_transaction_is_rolled_back(fresh_db): def test_query_rejected_write_inside_transaction_is_rolled_back(fresh_db):
fresh_db["dogs"].insert({"name": "Cleo"}) fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db.begin() fresh_db.begin()
fresh_db.execute("insert into dogs (name) values ('Pancakes')") fresh_db.execute("insert into dogs (name) values ('Pancakes')")
with pytest.raises(ValueError): with pytest.raises(ValueError):
@ -45,7 +45,7 @@ def test_query_rejected_write_inside_transaction_is_rolled_back(fresh_db):
# The transaction is still open and the earlier insert is intact # The transaction is still open and the earlier insert is intact
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.commit() fresh_db.commit()
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo", "Pancakes"] assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo", "Pancakes"]
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -77,7 +77,7 @@ def test_query_comment_prefixed_commit_does_not_commit_transaction(fresh_db):
# A COMMIT hidden behind a leading comment must not slip past the # A COMMIT hidden behind a leading comment must not slip past the
# keyword check - previously it committed the caller's open # keyword check - previously it committed the caller's open
# transaction before the ValueError was raised # transaction before the ValueError was raised
fresh_db["dogs"].insert({"name": "Cleo"}) fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db.begin() fresh_db.begin()
fresh_db.execute("insert into dogs (name) values ('Pancakes')") fresh_db.execute("insert into dogs (name) values ('Pancakes')")
with pytest.raises(ValueError): with pytest.raises(ValueError):
@ -85,7 +85,7 @@ def test_query_comment_prefixed_commit_does_not_commit_transaction(fresh_db):
# The explicit transaction is still open and can still be rolled back # The explicit transaction is still open and can still be rolled back
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.rollback() fresh_db.rollback()
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"] assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
@pytest.mark.parametrize("sql", ["; COMMIT", "\ufeffCOMMIT"]) @pytest.mark.parametrize("sql", ["; COMMIT", "\ufeffCOMMIT"])
@ -94,7 +94,7 @@ def test_query_prefixed_commit_does_not_commit_transaction(fresh_db, sql):
# real token, so the keyword scanner must skip them too - previously # real token, so the keyword scanner must skip them too - previously
# '; COMMIT' slipped past the check and committed the caller's open # '; COMMIT' slipped past the check and committed the caller's open
# transaction before raising OperationalError # transaction before raising OperationalError
fresh_db["dogs"].insert({"name": "Cleo"}) fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db.begin() fresh_db.begin()
fresh_db.execute("insert into dogs (name) values ('Pancakes')") fresh_db.execute("insert into dogs (name) values ('Pancakes')")
with pytest.raises(ValueError): with pytest.raises(ValueError):
@ -102,7 +102,7 @@ def test_query_prefixed_commit_does_not_commit_transaction(fresh_db, sql):
# The explicit transaction is still open and can still be rolled back # The explicit transaction is still open and can still be rolled back
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.rollback() fresh_db.rollback()
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"] assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
def test_query_error_leaves_no_transaction_open(fresh_db): def test_query_error_leaves_no_transaction_open(fresh_db):
@ -190,12 +190,12 @@ def test_first_keyword(sql, expected):
reason="RETURNING requires SQLite 3.35.0 or higher", reason="RETURNING requires SQLite 3.35.0 or higher",
) )
def test_query_insert_returning(fresh_db): def test_query_insert_returning(fresh_db):
fresh_db["dogs"].insert({"name": "Cleo"}) fresh_db.table("dogs").insert({"name": "Cleo"})
rows = list( rows = list(
fresh_db.query("insert into dogs (name) values ('Pancakes') returning name") fresh_db.query("insert into dogs (name) values ('Pancakes') returning name")
) )
assert rows == [{"name": "Pancakes"}] assert rows == [{"name": "Pancakes"}]
assert fresh_db["dogs"].count == 2 assert fresh_db.table("dogs").count == 2
@pytest.mark.skipif( @pytest.mark.skipif(
@ -207,7 +207,7 @@ def test_query_insert_returning_commits_without_iteration(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["dogs"].insert({"name": "Cleo"}) db.table("dogs").insert({"name": "Cleo"})
# Never iterate over the results # Never iterate over the results
db.query("insert into dogs (name) values ('Pancakes') returning name") db.query("insert into dogs (name) values ('Pancakes') returning name")
assert not db.conn.in_transaction assert not db.conn.in_transaction
@ -227,7 +227,7 @@ def test_query_insert_returning_partial_iteration_still_commits(tmpdir):
path = str(tmpdir / "test.db") path = str(tmpdir / "test.db")
db = Database(path) db = Database(path)
db["dogs"].insert({"name": "Cleo"}) db.table("dogs").insert({"name": "Cleo"})
row = next( row = next(
db.query( db.query(
"insert into dogs (name) values ('Pancakes'), ('Marnie') returning name" "insert into dogs (name) values ('Pancakes'), ('Marnie') returning name"
@ -246,7 +246,7 @@ def test_query_insert_returning_partial_iteration_still_commits(tmpdir):
reason="RETURNING requires SQLite 3.35.0 or higher", reason="RETURNING requires SQLite 3.35.0 or higher",
) )
def test_query_insert_returning_respects_explicit_transaction(fresh_db): def test_query_insert_returning_respects_explicit_transaction(fresh_db):
fresh_db["dogs"].insert({"name": "Cleo"}) fresh_db.table("dogs").insert({"name": "Cleo"})
fresh_db.begin() fresh_db.begin()
rows = list( rows = list(
fresh_db.query("insert into dogs (name) values ('Pancakes') returning name") fresh_db.query("insert into dogs (name) values ('Pancakes') returning name")
@ -255,13 +255,13 @@ def test_query_insert_returning_respects_explicit_transaction(fresh_db):
# Still inside the explicit transaction - not committed # Still inside the explicit transaction - not committed
assert fresh_db.conn.in_transaction assert fresh_db.conn.in_transaction
fresh_db.rollback() fresh_db.rollback()
assert [row["name"] for row in fresh_db["dogs"].rows] == ["Cleo"] assert [row["name"] for row in fresh_db.table("dogs").rows] == ["Cleo"]
def test_query_duplicate_column_names_are_deduped(fresh_db): def test_query_duplicate_column_names_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624 # https://github.com/simonw/sqlite-utils/issues/624
fresh_db["one"].insert({"id": 1, "value": "left"}) fresh_db.table("one").insert({"id": 1, "value": "left"})
fresh_db["two"].insert({"id": 2, "value": "right"}) fresh_db.table("two").insert({"id": 2, "value": "right"})
rows = list( rows = list(
fresh_db.query("select one.id, two.id, one.value, two.value from one, two") fresh_db.query("select one.id, two.id, one.value, two.value from one, two")
) )
@ -277,7 +277,7 @@ def test_query_deduped_column_avoids_existing_names(fresh_db):
def test_execute_returning_dicts(fresh_db): def test_execute_returning_dicts(fresh_db):
# Like db.query() but returns a list, included for backwards compatibility # Like db.query() but returns a list, included for backwards compatibility
# see https://github.com/simonw/sqlite-utils/issues/290 # see https://github.com/simonw/sqlite-utils/issues/290
fresh_db["test"].insert({"id": 1, "bar": 2}, pk="id") fresh_db.table("test").insert({"id": 1, "bar": 2}, pk="id")
assert fresh_db.execute_returning_dicts("select * from test") == [ assert fresh_db.execute_returning_dicts("select * from test") == [
{"id": 1, "bar": 2} {"id": 1, "bar": 2}
] ]

View file

@ -8,7 +8,7 @@ from sqlite_utils.utils import sqlite3
@pytest.fixture @pytest.fixture
def dates_db(fresh_db): def dates_db(fresh_db):
fresh_db["example"].insert_all( fresh_db.table("example").insert_all(
[ [
{"id": 1, "dt": "5th October 2019 12:04"}, {"id": 1, "dt": "5th October 2019 12:04"},
{"id": 2, "dt": "6th October 2019 00:05:06"}, {"id": 2, "dt": "6th October 2019 00:05:06"},
@ -21,8 +21,8 @@ def dates_db(fresh_db):
def test_parsedate(dates_db): def test_parsedate(dates_db):
dates_db["example"].convert("dt", recipes.parsedate) dates_db.table("example").convert("dt", recipes.parsedate)
assert list(dates_db["example"].rows) == [ assert list(dates_db.table("example").rows) == [
{"id": 1, "dt": "2019-10-05"}, {"id": 1, "dt": "2019-10-05"},
{"id": 2, "dt": "2019-10-06"}, {"id": 2, "dt": "2019-10-06"},
{"id": 3, "dt": ""}, {"id": 3, "dt": ""},
@ -31,8 +31,8 @@ def test_parsedate(dates_db):
def test_parsedatetime(dates_db): def test_parsedatetime(dates_db):
dates_db["example"].convert("dt", recipes.parsedatetime) dates_db.table("example").convert("dt", recipes.parsedatetime)
assert list(dates_db["example"].rows) == [ assert list(dates_db.table("example").rows) == [
{"id": 1, "dt": "2019-10-05T12:04:00"}, {"id": 1, "dt": "2019-10-05T12:04:00"},
{"id": 2, "dt": "2019-10-06T00:05:06"}, {"id": 2, "dt": "2019-10-06T00:05:06"},
{"id": 3, "dt": ""}, {"id": 3, "dt": ""},
@ -50,16 +50,16 @@ def test_parsedatetime(dates_db):
), ),
) )
def test_dayfirst_yearfirst(fresh_db, recipe, kwargs, expected): def test_dayfirst_yearfirst(fresh_db, recipe, kwargs, expected):
fresh_db["example"].insert_all( fresh_db.table("example").insert_all(
[ [
{"id": 1, "dt": "03/04/05"}, {"id": 1, "dt": "03/04/05"},
], ],
pk="id", pk="id",
) )
fresh_db["example"].convert( fresh_db.table("example").convert(
"dt", lambda value: getattr(recipes, recipe)(value, **kwargs) "dt", lambda value: getattr(recipes, recipe)(value, **kwargs)
) )
assert list(fresh_db["example"].rows) == [ assert list(fresh_db.table("example").rows) == [
{"id": 1, "dt": expected}, {"id": 1, "dt": expected},
] ]
@ -68,7 +68,7 @@ def test_dayfirst_yearfirst(fresh_db, recipe, kwargs, expected):
@pytest.mark.parametrize("fn", ("parsedate", "parsedatetime")) @pytest.mark.parametrize("fn", ("parsedate", "parsedatetime"))
def test_dateparse_errors_raises(fresh_db, fn): def test_dateparse_errors_raises(fresh_db, fn):
"""Test that invalid dates raise errors when errors=None""" """Test that invalid dates raise errors when errors=None"""
fresh_db["example"].insert_all( fresh_db.table("example").insert_all(
[ [
{"id": 1, "dt": "invalid"}, {"id": 1, "dt": "invalid"},
], ],
@ -76,30 +76,32 @@ def test_dateparse_errors_raises(fresh_db, fn):
) )
# Exception in SQLite callback surfaces as OperationalError # Exception in SQLite callback surfaces as OperationalError
with pytest.raises(sqlite3.OperationalError): with pytest.raises(sqlite3.OperationalError):
fresh_db["example"].convert("dt", lambda value: getattr(recipes, fn)(value)) fresh_db.table("example").convert(
"dt", lambda value: getattr(recipes, fn)(value)
)
@pytest.mark.parametrize("fn", ("parsedate", "parsedatetime")) @pytest.mark.parametrize("fn", ("parsedate", "parsedatetime"))
@pytest.mark.parametrize("errors", (recipes.SET_NULL, recipes.IGNORE)) @pytest.mark.parametrize("errors", (recipes.SET_NULL, recipes.IGNORE))
def test_dateparse_errors_handled(fresh_db, fn, errors): def test_dateparse_errors_handled(fresh_db, fn, errors):
"""Test error handling modes for invalid dates""" """Test error handling modes for invalid dates"""
fresh_db["example"].insert_all( fresh_db.table("example").insert_all(
[ [
{"id": 1, "dt": "invalid"}, {"id": 1, "dt": "invalid"},
], ],
pk="id", pk="id",
) )
fresh_db["example"].convert( fresh_db.table("example").convert(
"dt", lambda value: getattr(recipes, fn)(value, errors=errors) "dt", lambda value: getattr(recipes, fn)(value, errors=errors)
) )
rows = list(fresh_db["example"].rows) rows = list(fresh_db.table("example").rows)
expected = [{"id": 1, "dt": None if errors is recipes.SET_NULL else "invalid"}] expected = [{"id": 1, "dt": None if errors is recipes.SET_NULL else "invalid"}]
assert rows == expected assert rows == expected
@pytest.mark.parametrize("delimiter", [None, ";", "-"]) @pytest.mark.parametrize("delimiter", [None, ";", "-"])
def test_jsonsplit(fresh_db, delimiter): def test_jsonsplit(fresh_db, delimiter):
fresh_db["example"].insert_all( fresh_db.table("example").insert_all(
[ [
{"id": 1, "tags": (delimiter or ",").join(["foo", "bar"])}, {"id": 1, "tags": (delimiter or ",").join(["foo", "bar"])},
{"id": 2, "tags": (delimiter or ",").join(["bar", "baz"])}, {"id": 2, "tags": (delimiter or ",").join(["bar", "baz"])},
@ -114,8 +116,8 @@ def test_jsonsplit(fresh_db, delimiter):
else: else:
fn = recipes.jsonsplit fn = recipes.jsonsplit
fresh_db["example"].convert("tags", fn) fresh_db.table("example").convert("tags", fn)
assert list(fresh_db["example"].rows) == [ assert list(fresh_db.table("example").rows) == [
{"id": 1, "tags": '["foo", "bar"]'}, {"id": 1, "tags": '["foo", "bar"]'},
{"id": 2, "tags": '["bar", "baz"]'}, {"id": 2, "tags": '["bar", "baz"]'},
] ]
@ -130,7 +132,7 @@ def test_jsonsplit(fresh_db, delimiter):
), ),
) )
def test_jsonsplit_type(fresh_db, type, expected): def test_jsonsplit_type(fresh_db, type, expected):
fresh_db["example"].insert_all( fresh_db.table("example").insert_all(
[ [
{"id": 1, "records": "1,2,3"}, {"id": 1, "records": "1,2,3"},
], ],
@ -144,5 +146,5 @@ def test_jsonsplit_type(fresh_db, type, expected):
else: else:
fn = recipes.jsonsplit fn = recipes.jsonsplit
fresh_db["example"].convert("records", fn) fresh_db.table("example").convert("records", fn)
assert json.loads(fresh_db["example"].get(1)["records"]) == expected assert json.loads(fresh_db.table("example").get(1)["records"]) == expected

View file

@ -33,8 +33,8 @@ def test_recreate(tmp_path, use_path, create_file_first):
filepath = pathlib.Path(filepath) filepath = pathlib.Path(filepath)
if create_file_first: if create_file_first:
db = Database(filepath) db = Database(filepath)
db["t1"].insert({"foo": "bar"}) db.table("t1").insert({"foo": "bar"})
assert ["t1"] == db.table_names() assert ["t1"] == db.table_names()
db.close() db.close()
Database(filepath, recreate=True)["t2"].insert({"foo": "bar"}) Database(filepath, recreate=True).table("t2").insert({"foo": "bar"})
assert ["t2"] == Database(filepath).table_names() assert ["t2"] == Database(filepath).table_names()

View file

@ -86,21 +86,21 @@ def test_register_function_deterministic_tries_again_if_exception_raised(fresh_d
def test_register_function_replace(fresh_db): def test_register_function_replace(fresh_db):
@fresh_db.register_function() @fresh_db.register_function()
def one(): def one(): # pyright: ignore[reportRedeclaration]
return "one" return "one"
assert "one" == fresh_db.execute("select one()").fetchone()[0] assert "one" == fresh_db.execute("select one()").fetchone()[0]
# This will silently fail to replaec the function # This will silently fail to replaec the function
@fresh_db.register_function() @fresh_db.register_function()
def one(): # noqa def one(): # pyright: ignore[reportRedeclaration]
return "two" return "two"
assert "one" == fresh_db.execute("select one()").fetchone()[0] assert "one" == fresh_db.execute("select one()").fetchone()[0]
# This will replace it # This will replace it
@fresh_db.register_function(replace=True) @fresh_db.register_function(replace=True)
def one(): # noqa def one(): # pyright: ignore[reportRedeclaration]
return "two" return "two"
assert "two" == fresh_db.execute("select one()").fetchone()[0] assert "two" == fresh_db.execute("select one()").fetchone()[0]

View file

@ -3,7 +3,7 @@ import pytest
def test_rows(existing_db): def test_rows(existing_db):
assert [{"text": "one"}, {"text": "two"}, {"text": "three"}] == list( assert [{"text": "one"}, {"text": "two"}, {"text": "three"}] == list(
existing_db["foo"].rows existing_db.table("foo").rows
) )
@ -18,7 +18,7 @@ def test_rows(existing_db):
], ],
) )
def test_rows_where(where, where_args, expected_ids, fresh_db): def test_rows_where(where, where_args, expected_ids, fresh_db):
table = fresh_db["dogs"] table = fresh_db.table("dogs")
table.insert_all( table.insert_all(
[ [
{"id": 1, "name": "Cleo", "age": 4, "is_good": True}, {"id": 1, "name": "Cleo", "age": 4, "is_good": True},
@ -41,7 +41,7 @@ def test_rows_where(where, where_args, expected_ids, fresh_db):
], ],
) )
def test_rows_where_order_by(where, order_by, expected_ids, fresh_db): def test_rows_where_order_by(where, order_by, expected_ids, fresh_db):
table = fresh_db["dogs"] table = fresh_db.table("dogs")
table.insert_all( table.insert_all(
[ [
{"id": 1, "name": "Cleo", "age": 4}, {"id": 1, "name": "Cleo", "age": 4},
@ -59,10 +59,13 @@ def test_rows_where_order_by(where, order_by, expected_ids, fresh_db):
(None, 3, [1, 2, 3]), (None, 3, [1, 2, 3]),
(0, 3, [1, 2, 3]), (0, 3, [1, 2, 3]),
(3, 3, [4, 5, 6]), (3, 3, [4, 5, 6]),
# offset without limit should return every remaining row
(97, None, [98, 99, 100]),
(0, None, list(range(1, 101))),
], ],
) )
def test_rows_where_offset_limit(fresh_db, offset, limit, expected): def test_rows_where_offset_limit(fresh_db, offset, limit, expected):
table = fresh_db["rows"] table = fresh_db.table("rows")
table.insert_all([{"id": id} for id in range(1, 101)], pk="id") table.insert_all([{"id": id} for id in range(1, 101)], pk="id")
assert table.count == 100 assert table.count == 100
assert expected == [ assert expected == [
@ -70,8 +73,14 @@ def test_rows_where_offset_limit(fresh_db, offset, limit, expected):
] ]
def test_pks_and_rows_where_offset_without_limit(fresh_db):
table = fresh_db.table("rows")
table.insert_all([{"id": id} for id in range(1, 6)], pk="id")
assert [pk for pk, _ in table.pks_and_rows_where(offset=3, order_by="id")] == [4, 5]
def test_pks_and_rows_where_rowid(fresh_db): def test_pks_and_rows_where_rowid(fresh_db):
table = fresh_db["rowid_table"] table = fresh_db.table("rowid_table")
table.insert_all({"number": i + 10} for i in range(3)) table.insert_all({"number": i + 10} for i in range(3))
pks_and_rows = list(table.pks_and_rows_where()) pks_and_rows = list(table.pks_and_rows_where())
assert pks_and_rows == [ assert pks_and_rows == [
@ -82,7 +91,7 @@ def test_pks_and_rows_where_rowid(fresh_db):
def test_pks_and_rows_where_simple_pk(fresh_db): def test_pks_and_rows_where_simple_pk(fresh_db):
table = fresh_db["simple_pk_table"] table = fresh_db.table("simple_pk_table")
table.insert_all(({"id": i + 10} for i in range(3)), pk="id") table.insert_all(({"id": i + 10} for i in range(3)), pk="id")
pks_and_rows = list(table.pks_and_rows_where()) pks_and_rows = list(table.pks_and_rows_where())
assert pks_and_rows == [ assert pks_and_rows == [
@ -93,7 +102,7 @@ def test_pks_and_rows_where_simple_pk(fresh_db):
def test_pks_and_rows_where_compound_pk(fresh_db): def test_pks_and_rows_where_compound_pk(fresh_db):
table = fresh_db["compound_pk_table"] table = fresh_db.table("compound_pk_table")
table.insert_all( table.insert_all(
({"type": "number", "number": i, "plusone": i + 1} for i in range(3)), ({"type": "number", "number": i, "plusone": i + 1} for i in range(3)),
pk=("type", "number"), pk=("type", "number"),
@ -108,8 +117,8 @@ def test_pks_and_rows_where_compound_pk(fresh_db):
def test_rows_where_duplicate_select_columns_are_deduped(fresh_db): def test_rows_where_duplicate_select_columns_are_deduped(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/624 # https://github.com/simonw/sqlite-utils/issues/624
fresh_db["t"].insert({"id": 1, "name": "Cleo"}) fresh_db.table("t").insert({"id": 1, "name": "Cleo"})
rows = list(fresh_db["t"].rows_where(select="id, id, name")) rows = list(fresh_db.table("t").rows_where(select="id, id, name"))
assert rows == [{"id": 1, "id_2": 1, "name": "Cleo"}] assert rows == [{"id": 1, "id_2": 1, "name": "Cleo"}]
@ -121,10 +130,10 @@ def test_pks_and_rows_where_view(fresh_db):
# an AttributeError from View lacking Table-only properties # an AttributeError from View lacking Table-only properties
from sqlite_utils.utils import sqlite3 from sqlite_utils.utils import sqlite3
fresh_db["dogs"].insert({"id": 1, "name": "Cleo"}, pk="id") fresh_db.table("dogs").insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.create_view("dog_names", "select name from dogs") fresh_db.create_view("dog_names", "select name from dogs")
try: try:
result = list(fresh_db["dog_names"].pks_and_rows_where()) result = list(fresh_db.view("dog_names").pks_and_rows_where())
except sqlite3.OperationalError: except sqlite3.OperationalError:
pass # SQLite 3.36+: no such column: rowid pass # SQLite 3.36+: no such column: rowid
else: else:
@ -135,6 +144,6 @@ def test_pks_and_rows_where_view(fresh_db):
def test_pks_and_rows_where_compound_pk_declaration_order(fresh_db): def test_pks_and_rows_where_compound_pk_declaration_order(fresh_db):
# Compound pks are returned in PRIMARY KEY declaration order # Compound pks are returned in PRIMARY KEY declaration order
fresh_db.execute("create table t (b text, a text, primary key (a, b))") fresh_db.execute("create table t (b text, a text, primary key (a, b))")
fresh_db["t"].insert({"a": "A", "b": "B"}) fresh_db.table("t").insert({"a": "A", "b": "B"})
pks_and_rows = list(fresh_db["t"].pks_and_rows_where()) pks_and_rows = list(fresh_db.table("t").pks_and_rows_where())
assert pks_and_rows == [(("A", "B"), {"b": "B", "a": "A"})] assert pks_and_rows == [(("A", "B"), {"b": "B", "a": "A"})]

View file

@ -20,6 +20,13 @@ def test_rows_from_file_detect_format(input, expected_format):
assert rows_list == [{"id": "1", "name": "Cleo"}] assert rows_list == [{"id": "1", "name": "Cleo"}]
@pytest.mark.parametrize("input", (b"", b" \n\t"))
def test_rows_from_file_empty_input(input):
rows, format = rows_from_file(BytesIO(input))
assert format == Format.CSV
assert list(rows) == []
@pytest.mark.parametrize( @pytest.mark.parametrize(
"ignore_extras,extras_key,expected", "ignore_extras,extras_key,expected",
( (

View file

@ -19,7 +19,7 @@ def test_sniff(tmpdir, filepath):
) )
assert result.exit_code == 0, result.stdout assert result.exit_code == 0, result.stdout
db = Database(db_path) db = Database(db_path)
assert list(db["creatures"].rows) == [ assert list(db.table("creatures").rows) == [
{"id": "1", "species": "dog", "name": "Cleo", "age": "5"}, {"id": "1", "species": "dog", "name": "Cleo", "age": "5"},
{"id": "2", "species": "dog", "name": "Pancakes", "age": "4"}, {"id": "2", "species": "dog", "name": "Pancakes", "age": "4"},
{"id": "3", "species": "cat", "name": "Mozie", "age": "8"}, {"id": "3", "species": "cat", "name": "Mozie", "age": "8"},

View file

@ -2,6 +2,7 @@ import sqlite3
import pytest import pytest
from sqlite_utils import ANY
from sqlite_utils.db import Check, ForeignKey, TransactionError, TransformError from sqlite_utils.db import Check, ForeignKey, TransactionError, TransformError
from sqlite_utils.utils import OperationalError from sqlite_utils.utils import OperationalError
@ -26,7 +27,7 @@ from sqlite_utils.utils import OperationalError
{"types": {"age": int}}, {"types": {"age": int}},
[ [
'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "age" INTEGER\n);', 'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "age" INTEGER\n);',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", "age" FROM "dogs";', 'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", NULLIF("age", \'\') FROM "dogs";',
'DROP TABLE "dogs";', 'DROP TABLE "dogs";',
"PRAGMA legacy_alter_table=ON;", "PRAGMA legacy_alter_table=ON;",
'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";', 'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";',
@ -62,7 +63,7 @@ from sqlite_utils.utils import OperationalError
{"types": {"age": int}, "rename": {"age": "dog_age"}}, {"types": {"age": int}, "rename": {"age": "dog_age"}},
[ [
'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "dog_age" INTEGER\n);', 'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT,\n "dog_age" INTEGER\n);',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "dog_age")\n SELECT "rowid", "id", "name", "age" FROM "dogs";', 'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "dog_age")\n SELECT "rowid", "id", "name", NULLIF("age", \'\') FROM "dogs";',
'DROP TABLE "dogs";', 'DROP TABLE "dogs";',
"PRAGMA legacy_alter_table=ON;", "PRAGMA legacy_alter_table=ON;",
'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";', 'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";',
@ -128,7 +129,7 @@ def test_transform_sql_table_with_primary_key(
def tracer(sql, params): def tracer(sql, params):
return captured.append((sql, params)) return captured.append((sql, params))
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
if use_pragma_foreign_keys: if use_pragma_foreign_keys:
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
@ -167,7 +168,7 @@ def test_transform_sql_table_with_primary_key(
{"types": {"age": int}}, {"types": {"age": int}},
[ [
'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER,\n "name" TEXT,\n "age" INTEGER\n);', 'CREATE TABLE "dogs_new_suffix" (\n "id" INTEGER,\n "name" TEXT,\n "age" INTEGER\n);',
'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", "age" FROM "dogs";', 'INSERT INTO "dogs_new_suffix" ("rowid", "id", "name", "age")\n SELECT "rowid", "id", "name", NULLIF("age", \'\') FROM "dogs";',
'DROP TABLE "dogs";', 'DROP TABLE "dogs";',
"PRAGMA legacy_alter_table=ON;", "PRAGMA legacy_alter_table=ON;",
'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";', 'ALTER TABLE "dogs_new_suffix" RENAME TO "dogs";',
@ -209,7 +210,7 @@ def test_transform_sql_table_with_no_primary_key(
def tracer(sql, params): def tracer(sql, params):
return captured.append((sql, params)) return captured.append((sql, params))
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
if use_pragma_foreign_keys: if use_pragma_foreign_keys:
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}) dogs.insert({"id": 1, "name": "Cleo", "age": "5"})
@ -229,7 +230,7 @@ def test_transform_sql_table_with_no_primary_key(
def test_transform_sql_with_no_primary_key_to_primary_key_of_id(fresh_db): def test_transform_sql_with_no_primary_key_to_primary_key_of_id(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}) dogs.insert({"id": 1, "name": "Cleo", "age": "5"})
assert ( assert (
dogs.schema dogs.schema
@ -244,7 +245,7 @@ def test_transform_sql_with_no_primary_key_to_primary_key_of_id(fresh_db):
def test_transform_rename_pk(fresh_db): def test_transform_rename_pk(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
dogs.transform(rename={"id": "pk"}) dogs.transform(rename={"id": "pk"})
assert ( assert (
@ -265,7 +266,7 @@ def test_transform_preserves_keyword_literal_defaults(fresh_db):
" note TEXT DEFAULT NULL" " note TEXT DEFAULT NULL"
")" ")"
) )
table = fresh_db["t"] table = fresh_db.table("t")
table.insert({"id": 1}) table.insert({"id": 1})
before = fresh_db.execute("SELECT is_active, flag, note FROM t").fetchone() before = fresh_db.execute("SELECT is_active, flag, note FROM t").fetchone()
assert before == (1, 0, None) assert before == (1, 0, None)
@ -288,7 +289,7 @@ def test_transform_preserves_keyword_literal_defaults(fresh_db):
def test_transform_not_null(fresh_db): def test_transform_not_null(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
dogs.transform(not_null={"name"}) dogs.transform(not_null={"name"})
assert ( assert (
@ -298,7 +299,7 @@ def test_transform_not_null(fresh_db):
def test_transform_remove_a_not_null(fresh_db): def test_transform_remove_a_not_null(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, not_null={"age"}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, not_null={"age"}, pk="id")
dogs.transform(not_null={"name": True, "age": False}) dogs.transform(not_null={"name": True, "age": False})
assert ( assert (
@ -309,7 +310,7 @@ def test_transform_remove_a_not_null(fresh_db):
@pytest.mark.parametrize("not_null", [{"age"}, {"age": True}]) @pytest.mark.parametrize("not_null", [{"age"}, {"age": True}])
def test_transform_add_not_null_with_rename(fresh_db, not_null): def test_transform_add_not_null_with_rename(fresh_db, not_null):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": "5"}, pk="id")
dogs.transform(not_null=not_null, rename={"age": "dog_age"}) dogs.transform(not_null=not_null, rename={"age": "dog_age"})
assert ( assert (
@ -319,7 +320,7 @@ def test_transform_add_not_null_with_rename(fresh_db, not_null):
def test_transform_defaults(fresh_db): def test_transform_defaults(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id")
dogs.transform(defaults={"age": 1}) dogs.transform(defaults={"age": 1})
assert ( assert (
@ -329,7 +330,7 @@ def test_transform_defaults(fresh_db):
def test_transform_defaults_and_rename_column(fresh_db): def test_transform_defaults_and_rename_column(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id")
dogs.transform(rename={"age": "dog_age"}, defaults={"age": 1}) dogs.transform(rename={"age": "dog_age"}, defaults={"age": 1})
assert ( assert (
@ -339,7 +340,7 @@ def test_transform_defaults_and_rename_column(fresh_db):
def test_remove_defaults(fresh_db): def test_remove_defaults(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, defaults={"age": 1}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": 5}, defaults={"age": 1}, pk="id")
dogs.transform(defaults={"age": None}) dogs.transform(defaults={"age": None})
assert ( assert (
@ -350,8 +351,8 @@ def test_remove_defaults(fresh_db):
@pytest.fixture @pytest.fixture
def authors_db(fresh_db): def authors_db(fresh_db):
books = fresh_db["books"] books = fresh_db.table("books")
authors = fresh_db["authors"] authors = fresh_db.table("authors")
authors.insert({"id": 5, "name": "Jane McGonical"}, pk="id") authors.insert({"id": 5, "name": "Jane McGonical"}, pk="id")
books.insert( books.insert(
{"id": 2, "title": "Reality is Broken", "author_id": 5}, {"id": 2, "title": "Reality is Broken", "author_id": 5},
@ -362,13 +363,13 @@ def authors_db(fresh_db):
def test_transform_foreign_keys_persist(authors_db): def test_transform_foreign_keys_persist(authors_db):
assert authors_db["books"].foreign_keys == [ assert authors_db.table("books").foreign_keys == [
ForeignKey( ForeignKey(
table="books", column="author_id", other_table="authors", other_column="id" table="books", column="author_id", other_table="authors", other_column="id"
) )
] ]
authors_db["books"].transform(rename={"title": "book_title"}) authors_db.table("books").transform(rename={"title": "book_title"})
assert authors_db["books"].foreign_keys == [ assert authors_db.table("books").foreign_keys == [
ForeignKey( ForeignKey(
table="books", column="author_id", other_table="authors", other_column="id" table="books", column="author_id", other_table="authors", other_column="id"
) )
@ -381,8 +382,8 @@ def test_transform_foreign_keys_survive_renamed_column(
): ):
if use_pragma_foreign_keys: if use_pragma_foreign_keys:
authors_db.conn.execute("PRAGMA foreign_keys=ON") authors_db.conn.execute("PRAGMA foreign_keys=ON")
authors_db["books"].transform(rename={"author_id": "author_id_2"}) authors_db.table("books").transform(rename={"author_id": "author_id_2"})
assert authors_db["books"].foreign_keys == [ assert authors_db.table("books").foreign_keys == [
ForeignKey( ForeignKey(
table="books", table="books",
column="author_id_2", column="author_id_2",
@ -393,9 +394,9 @@ def test_transform_foreign_keys_survive_renamed_column(
def _add_country_city_continent(db): def _add_country_city_continent(db):
db["country"].insert({"id": 1, "name": "France"}, pk="id") db.table("country").insert({"id": 1, "name": "France"}, pk="id")
db["continent"].insert({"id": 2, "name": "Europe"}, pk="id") db.table("continent").insert({"id": 2, "name": "Europe"}, pk="id")
db["city"].insert({"id": 24, "name": "Paris"}, pk="id") db.table("city").insert({"id": 24, "name": "Paris"}, pk="id")
_CAVEAU = { _CAVEAU = {
@ -413,11 +414,11 @@ def test_transform_drop_foreign_keys(fresh_db, use_pragma_foreign_keys):
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
# Create table with three foreign keys so we can drop two of them # Create table with three foreign keys so we can drop two of them
_add_country_city_continent(fresh_db) _add_country_city_continent(fresh_db)
fresh_db["places"].insert( fresh_db.table("places").insert(
_CAVEAU, _CAVEAU,
foreign_keys=("country", "continent", "city"), foreign_keys=("country", "continent", "city"),
) )
assert fresh_db["places"].foreign_keys == [ assert fresh_db.table("places").foreign_keys == [
ForeignKey( ForeignKey(
table="places", column="city", other_table="city", other_column="id" table="places", column="city", other_table="city", other_column="id"
), ),
@ -432,9 +433,9 @@ def test_transform_drop_foreign_keys(fresh_db, use_pragma_foreign_keys):
), ),
] ]
# Drop two of those foreign keys # Drop two of those foreign keys
fresh_db["places"].transform(drop_foreign_keys=("country", "continent")) fresh_db.table("places").transform(drop_foreign_keys=("country", "continent"))
# Should be only one foreign key now # Should be only one foreign key now
assert fresh_db["places"].foreign_keys == [ assert fresh_db.table("places").foreign_keys == [
ForeignKey(table="places", column="city", other_table="city", other_column="id") ForeignKey(table="places", column="city", other_table="city", other_column="id")
] ]
if use_pragma_foreign_keys: if use_pragma_foreign_keys:
@ -443,17 +444,17 @@ def test_transform_drop_foreign_keys(fresh_db, use_pragma_foreign_keys):
def test_transform_verify_foreign_keys(fresh_db): def test_transform_verify_foreign_keys(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
fresh_db["authors"].insert({"id": 3, "name": "Tina"}, pk="id") fresh_db.table("authors").insert({"id": 3, "name": "Tina"}, pk="id")
fresh_db["books"].insert( fresh_db.table("books").insert(
{"id": 1, "title": "Book", "author_id": 3}, pk="id", foreign_keys={"author_id"} {"id": 1, "title": "Book", "author_id": 3}, pk="id", foreign_keys={"author_id"}
) )
# Renaming the id column on authors should break everything # Renaming the id column on authors should break everything
with pytest.raises(OperationalError) as e: with pytest.raises(OperationalError) as e:
fresh_db["authors"].transform(rename={"id": "id2"}) fresh_db.table("authors").transform(rename={"id": "id2"})
assert e.value.args[0] == 'foreign key mismatch - "books" referencing "authors"' assert e.value.args[0] == 'foreign key mismatch - "books" referencing "authors"'
# This should have rolled us back # This should have rolled us back
assert ( assert (
fresh_db["authors"].schema fresh_db.table("authors").schema
== 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT\n)' == 'CREATE TABLE "authors" (\n "id" INTEGER PRIMARY KEY,\n "name" TEXT\n)'
) )
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0] assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -476,20 +477,22 @@ def test_transform_on_delete_cascade_does_not_delete_records(
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
); );
""") """)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"}) fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1}) fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
# Transform the table on the other end of the cascading foreign key # Transform the table on the other end of the cascading foreign key
fresh_db["authors"].transform(rename={"name": "author_name"}) fresh_db.table("authors").transform(rename={"name": "author_name"})
assert list(fresh_db["authors"].rows) == [ assert list(fresh_db.table("authors").rows) == [
{"id": 1, "author_name": "Ursula K. Le Guin"} {"id": 1, "author_name": "Ursula K. Le Guin"}
] ]
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1} {"id": 1, "title": "The Dispossessed", "author_id": 1}
] ]
# Transforming the table with the cascading foreign key should not # Transforming the table with the cascading foreign key should not
# delete its records either # delete its records either
fresh_db["books"].transform(rename={"title": "book_title"}) fresh_db.table("books").transform(rename={"title": "book_title"})
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "book_title": "The Dispossessed", "author_id": 1} {"id": 1, "book_title": "The Dispossessed", "author_id": 1}
] ]
if use_pragma_foreign_keys: if use_pragma_foreign_keys:
@ -511,17 +514,19 @@ def test_transform_in_transaction_refuses_destructive_on_delete(fresh_db, on_del
author_id INTEGER REFERENCES authors(id) ON DELETE {on_delete} author_id INTEGER REFERENCES authors(id) ON DELETE {on_delete}
); );
""") """)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"}) fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1}) fresh_db.table("books").insert(
previous_schema = fresh_db["authors"].schema {"id": 1, "title": "The Dispossessed", "author_id": 1}
)
previous_schema = fresh_db.table("authors").schema
with fresh_db.atomic(), pytest.raises(TransactionError) as excinfo: with fresh_db.atomic(), pytest.raises(TransactionError) as excinfo:
fresh_db["authors"].transform(rename={"name": "author_name"}) fresh_db.table("authors").transform(rename={"name": "author_name"})
message = str(excinfo.value) message = str(excinfo.value)
assert "books" in message assert "books" in message
assert f"ON DELETE {on_delete.upper()}" in message assert f"ON DELETE {on_delete.upper()}" in message
# Nothing should have changed # Nothing should have changed
assert fresh_db["authors"].schema == previous_schema assert fresh_db.table("authors").schema == previous_schema
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1} {"id": 1, "title": "The Dispossessed", "author_id": 1}
] ]
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0] assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -538,16 +543,16 @@ def test_transform_in_transaction_refuses_self_referential_cascade(fresh_db):
parent_id INTEGER REFERENCES categories(id) ON DELETE CASCADE parent_id INTEGER REFERENCES categories(id) ON DELETE CASCADE
); );
""") """)
fresh_db["categories"].insert_all( fresh_db.table("categories").insert_all(
[ [
{"id": 1, "name": "Fiction", "parent_id": None}, {"id": 1, "name": "Fiction", "parent_id": None},
{"id": 2, "name": "Science Fiction", "parent_id": 1}, {"id": 2, "name": "Science Fiction", "parent_id": 1},
] ]
) )
with fresh_db.atomic(), pytest.raises(TransactionError) as excinfo: with fresh_db.atomic(), pytest.raises(TransactionError) as excinfo:
fresh_db["categories"].transform(rename={"name": "title"}) fresh_db.table("categories").transform(rename={"name": "title"})
assert "categories" in str(excinfo.value) assert "categories" in str(excinfo.value)
assert fresh_db["categories"].count == 2 assert fresh_db.table("categories").count == 2
def test_transform_in_transaction_allowed_with_no_action_foreign_key(fresh_db): def test_transform_in_transaction_allowed_with_no_action_foreign_key(fresh_db):
@ -562,14 +567,16 @@ def test_transform_in_transaction_allowed_with_no_action_foreign_key(fresh_db):
author_id INTEGER REFERENCES authors(id) author_id INTEGER REFERENCES authors(id)
); );
""") """)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"}) fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1}) fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["authors"].transform(rename={"name": "author_name"}) fresh_db.table("authors").transform(rename={"name": "author_name"})
assert list(fresh_db["authors"].rows) == [ assert list(fresh_db.table("authors").rows) == [
{"id": 1, "author_name": "Ursula K. Le Guin"} {"id": 1, "author_name": "Ursula K. Le Guin"}
] ]
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1} {"id": 1, "title": "The Dispossessed", "author_id": 1}
] ]
assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0] assert fresh_db.conn.execute("PRAGMA foreign_keys").fetchone()[0]
@ -587,11 +594,13 @@ def test_transform_in_transaction_allowed_for_child_table(fresh_db):
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
); );
""") """)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"}) fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1}) fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["books"].transform(rename={"title": "book_title"}) fresh_db.table("books").transform(rename={"title": "book_title"})
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "book_title": "The Dispossessed", "author_id": 1} {"id": 1, "book_title": "The Dispossessed", "author_id": 1}
] ]
@ -607,24 +616,28 @@ def test_transform_in_transaction_allowed_with_foreign_keys_off(fresh_db):
author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE author_id INTEGER REFERENCES authors(id) ON DELETE CASCADE
); );
""") """)
fresh_db["authors"].insert({"id": 1, "name": "Ursula K. Le Guin"}) fresh_db.table("authors").insert({"id": 1, "name": "Ursula K. Le Guin"})
fresh_db["books"].insert({"id": 1, "title": "The Dispossessed", "author_id": 1}) fresh_db.table("books").insert(
{"id": 1, "title": "The Dispossessed", "author_id": 1}
)
with fresh_db.atomic(): with fresh_db.atomic():
fresh_db["authors"].transform(rename={"name": "author_name"}) fresh_db.table("authors").transform(rename={"name": "author_name"})
assert list(fresh_db["books"].rows) == [ assert list(fresh_db.table("books").rows) == [
{"id": 1, "title": "The Dispossessed", "author_id": 1} {"id": 1, "title": "The Dispossessed", "author_id": 1}
] ]
def test_transform_add_foreign_keys_from_scratch(fresh_db): def test_transform_add_foreign_keys_from_scratch(fresh_db):
_add_country_city_continent(fresh_db) _add_country_city_continent(fresh_db)
fresh_db["places"].insert(_CAVEAU) fresh_db.table("places").insert(_CAVEAU)
# Should have no foreign keys # Should have no foreign keys
assert fresh_db["places"].foreign_keys == [] assert fresh_db.table("places").foreign_keys == []
# Now add them using .transform() # Now add them using .transform()
fresh_db["places"].transform(add_foreign_keys=("country", "continent", "city")) fresh_db.table("places").transform(
add_foreign_keys=("country", "continent", "city")
)
# Should now have all three: # Should now have all three:
assert fresh_db["places"].foreign_keys == [ assert fresh_db.table("places").foreign_keys == [
ForeignKey( ForeignKey(
table="places", column="city", other_table="city", other_column="id" table="places", column="city", other_table="city", other_column="id"
), ),
@ -638,7 +651,7 @@ def test_transform_add_foreign_keys_from_scratch(fresh_db):
table="places", column="country", other_table="country", other_column="id" table="places", column="country", other_table="country", other_column="id"
), ),
] ]
assert fresh_db["places"].schema == ( assert fresh_db.table("places").schema == (
'CREATE TABLE "places" (\n' 'CREATE TABLE "places" (\n'
' "id" INTEGER,\n' ' "id" INTEGER,\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
@ -662,18 +675,18 @@ def test_transform_add_foreign_keys_from_scratch(fresh_db):
) )
def test_transform_add_foreign_keys_from_partial(fresh_db, add_foreign_keys): def test_transform_add_foreign_keys_from_partial(fresh_db, add_foreign_keys):
_add_country_city_continent(fresh_db) _add_country_city_continent(fresh_db)
fresh_db["places"].insert( fresh_db.table("places").insert(
_CAVEAU, _CAVEAU,
foreign_keys=("city",), foreign_keys=("city",),
) )
# Should have one foreign keys # Should have one foreign keys
assert fresh_db["places"].foreign_keys == [ assert fresh_db.table("places").foreign_keys == [
ForeignKey(table="places", column="city", other_table="city", other_column="id") ForeignKey(table="places", column="city", other_table="city", other_column="id")
] ]
# Now add three more using .transform() # Now add three more using .transform()
fresh_db["places"].transform(add_foreign_keys=add_foreign_keys) fresh_db.table("places").transform(add_foreign_keys=add_foreign_keys)
# Should now have all three: # Should now have all three:
assert fresh_db["places"].foreign_keys == [ assert fresh_db.table("places").foreign_keys == [
ForeignKey( ForeignKey(
table="places", column="city", other_table="city", other_column="id" table="places", column="city", other_table="city", other_column="id"
), ),
@ -702,14 +715,14 @@ def test_transform_add_foreign_keys_from_partial(fresh_db, add_foreign_keys):
) )
def test_transform_replace_foreign_keys(fresh_db, foreign_keys): def test_transform_replace_foreign_keys(fresh_db, foreign_keys):
_add_country_city_continent(fresh_db) _add_country_city_continent(fresh_db)
fresh_db["places"].insert( fresh_db.table("places").insert(
_CAVEAU, _CAVEAU,
foreign_keys=("city",), foreign_keys=("city",),
) )
assert len(fresh_db["places"].foreign_keys) == 1 assert len(fresh_db.table("places").foreign_keys) == 1
# Replace with two different ones # Replace with two different ones
fresh_db["places"].transform(foreign_keys=foreign_keys) fresh_db.table("places").transform(foreign_keys=foreign_keys)
assert fresh_db["places"].schema == ( assert fresh_db.table("places").schema == (
'CREATE TABLE "places" (\n' 'CREATE TABLE "places" (\n'
' "id" INTEGER,\n' ' "id" INTEGER,\n'
' "name" TEXT,\n' ' "name" TEXT,\n'
@ -729,7 +742,7 @@ def test_transform_preserves_rowids(fresh_db, table_type):
pk = ("id", "name") pk = ("id", "name")
elif table_type == "rowid": elif table_type == "rowid":
pk = None pk = None
fresh_db["places"].insert_all( fresh_db.table("places").insert_all(
[ [
{"id": "1", "name": "Paris", "country": "France"}, {"id": "1", "name": "Paris", "country": "France"},
{"id": "2", "name": "London", "country": "UK"}, {"id": "2", "name": "London", "country": "UK"},
@ -738,13 +751,13 @@ def test_transform_preserves_rowids(fresh_db, table_type):
pk=pk, pk=pk,
) )
# Now delete and insert a row to mix up the `rowid` sequence # Now delete and insert a row to mix up the `rowid` sequence
fresh_db["places"].delete_where("id = ?", ["2"]) fresh_db.table("places").delete_where("id = ?", ["2"])
fresh_db["places"].insert({"id": "4", "name": "London", "country": "UK"}) fresh_db.table("places").insert({"id": "4", "name": "London", "country": "UK"})
previous_rows = [ previous_rows = [
tuple(row) for row in fresh_db.execute("select rowid, id, name from places") tuple(row) for row in fresh_db.execute("select rowid, id, name from places")
] ]
# Transform it # Transform it
fresh_db["places"].transform(column_order=("country", "name")) fresh_db.table("places").transform(column_order=("country", "name"))
# Should be the same # Should be the same
next_rows = [ next_rows = [
tuple(row) for row in fresh_db.execute("select rowid, id, name from places") tuple(row) for row in fresh_db.execute("select rowid, id, name from places")
@ -774,7 +787,7 @@ def test_transform_strict(fresh_db, initial_strict, transform_strict, expected_s
def test_transform_to_strict_with_invalid_data(fresh_db): def test_transform_to_strict_with_invalid_data(fresh_db):
if not fresh_db.supports_strict: if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables") pytest.skip("SQLite version does not support strict tables")
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.create({"id": int}) dogs.create({"id": int})
dogs.insert({"id": "not-an-integer"}) dogs.insert({"id": "not-an-integer"})
@ -801,7 +814,7 @@ def test_transform_strict_updates_default(fresh_db):
@pytest.mark.parametrize("method_name", ("transform", "transform_sql")) @pytest.mark.parametrize("method_name", ("transform", "transform_sql"))
def test_transform_to_strict_not_supported(fresh_db, method_name): def test_transform_to_strict_not_supported(fresh_db, method_name):
table = fresh_db["items"] table = fresh_db.table("items")
table.create({"id": int}) table.create({"id": int})
fresh_db._supports_strict = False fresh_db._supports_strict = False
@ -811,6 +824,55 @@ def test_transform_to_strict_not_supported(fresh_db, method_name):
assert table.strict is False assert table.strict is False
def test_transform_preserves_any_column_in_strict_table(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (id integer primary key, data any) strict")
fresh_db.conn.executemany(
"insert into items values (?, ?)",
[
(1, 42),
(2, "000123"),
(3, 3.14),
(4, b"bytes"),
(5, None),
],
)
table = fresh_db["items"]
table.transform()
assert table.strict is True
assert table.columns_dict == {"id": int, "data": ANY}
assert fresh_db.execute(
"select typeof(data), data from items order by id"
).fetchall() == [
("integer", 42),
("text", "000123"),
("real", 3.14),
("blob", b"bytes"),
("null", None),
]
def test_transform_any_column_from_strict_to_non_strict(fresh_db):
if not fresh_db.supports_strict:
pytest.skip("SQLite version does not support strict tables")
fresh_db.execute("create table items (data any) strict")
fresh_db.execute("insert into items values (?)", ("000123",))
table = fresh_db["items"]
table.transform(strict=False)
assert table.strict is False
assert table.columns_dict == {"data": ANY}
# Ordinary non-STRICT ANY columns apply NUMERIC affinity
assert fresh_db.execute("select typeof(data), data from items").fetchone() == (
"integer",
123,
)
@pytest.mark.parametrize( @pytest.mark.parametrize(
"indexes, transform_params", "indexes, transform_params",
[ [
@ -823,7 +885,7 @@ def test_transform_to_strict_not_supported(fresh_db, method_name):
def test_transform_indexes(fresh_db, indexes, transform_params): def test_transform_indexes(fresh_db, indexes, transform_params):
# https://github.com/simonw/sqlite-utils/issues/633 # https://github.com/simonw/sqlite-utils/issues/633
# New table should have same indexes as old table after transformation # New table should have same indexes as old table after transformation
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "age": 5, "breed": "Labrador"}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": 5, "breed": "Labrador"}, pk="id")
for index in indexes: for index in indexes:
@ -849,13 +911,13 @@ def test_transform_indexes(fresh_db, indexes, transform_params):
if "keep_table" in transform_params: if "keep_table" in transform_params:
assert all( assert all(
index.origin == "pk" index.origin == "pk"
for index in fresh_db[transform_params["keep_table"]].indexes for index in fresh_db.table(transform_params["keep_table"]).indexes
) )
def test_transform_retains_indexes_with_foreign_keys(fresh_db): def test_transform_retains_indexes_with_foreign_keys(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
owners = fresh_db["owners"] owners = fresh_db.table("owners")
dogs.insert({"id": 1, "name": "Cleo", "owner_id": 1}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "owner_id": 1}, pk="id")
owners.insert({"id": 1, "name": "Alice"}, pk="id") owners.insert({"id": 1, "name": "Alice"}, pk="id")
@ -881,22 +943,15 @@ def test_transform_retains_indexes_with_foreign_keys(fresh_db):
), f"Indexes before transform: {indexes_before_transform}\nIndexes after transform: {dogs.indexes}" ), f"Indexes before transform: {indexes_before_transform}\nIndexes after transform: {dogs.indexes}"
@pytest.mark.parametrize( def test_transform_with_indexes_errors(fresh_db):
"transform_params", # Should error with a compound (name, age) index if age is dropped
[ dogs = fresh_db.table("dogs")
{"rename": {"age": "dog_age"}},
{"drop": ["age"]},
],
)
def test_transform_with_indexes_errors(fresh_db, transform_params):
# Should error with a compound (name, age) index if age is renamed or dropped
dogs = fresh_db["dogs"]
dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "age": 5}, pk="id")
dogs.create_index(["name", "age"]) dogs.create_index(["name", "age"])
with pytest.raises(TransformError) as excinfo: with pytest.raises(TransformError) as excinfo:
dogs.transform(**transform_params) dogs.transform(drop=["age"])
assert ( assert (
"Index 'idx_dogs_name_age' column 'age' is not in updated table 'dogs'. " "Index 'idx_dogs_name_age' column 'age' is not in updated table 'dogs'. "
@ -905,35 +960,204 @@ def test_transform_with_indexes_errors(fresh_db, transform_params):
) )
@pytest.mark.parametrize(
("table_name", "index_name"),
(("name", "idx_name"), ("t", "name")),
)
def test_transform_rename_column_with_index(fresh_db, table_name, index_name):
# https://github.com/simonw/sqlite-utils/issues/822
# Use the same name for the table, column and index to ensure only the
# indexed column changes.
table = fresh_db.table(table_name)
table.insert({"id": 1, "name": "Cleo"}, pk="id")
table.create_index(["name"], index_name=index_name)
sqls = table.transform_sql(rename={"name": "full_name"}, tmp_suffix="suffix")
drop_index_sql = f'DROP INDEX IF EXISTS "{index_name}";'
assert drop_index_sql in sqls
assert sqls.index(drop_index_sql) < sqls.index(f'DROP TABLE "{table_name}";')
table.transform(rename={"name": "full_name"})
assert [column.name for column in table.columns] == ["id", "full_name"]
assert [(index.name, index.columns) for index in table.indexes] == [
(index_name, ["full_name"])
]
def test_transform_recreates_renamed_index_from_metadata(fresh_db):
table = fresh_db.table("t")
table.insert({"alpha": "one", "beta": "two"})
# Deliberately use unquoted SQL and index details that need to survive the
# reconstruction. Renaming both columns also guards against cascading
# string substitutions.
fresh_db.execute(
"CREATE UNIQUE INDEX swap_idx ON t(alpha COLLATE NOCASE DESC, beta)"
)
table.transform(rename={"alpha": "beta", "beta": "alpha"})
assert table.columns_dict == {"beta": str, "alpha": str}
assert [(index.name, index.unique, index.columns) for index in table.indexes] == [
("swap_idx", 1, ["beta", "alpha"])
]
key_columns = [column for column in table.xindexes[0].columns if column.key]
assert [(column.name, column.desc, column.coll) for column in key_columns] == [
("beta", 1, "NOCASE"),
("alpha", 0, "BINARY"),
]
@pytest.mark.parametrize(
"index_sql",
(
"CREATE INDEX idx_t_name ON t(lower(name))",
"CREATE INDEX idx_t_name ON t(name) WHERE name IS NOT NULL",
),
)
def test_transform_rename_complex_index_errors(fresh_db, index_sql):
table = fresh_db.table("t")
table.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute(index_sql)
with pytest.raises(TransformError, match="partial or expression index"):
table.transform(rename={"name": "full_name"})
assert table.columns_dict == {"id": int, "name": str}
assert [index.name for index in table.indexes] == ["idx_t_name"]
def test_transform_with_unique_constraint_implicit_index(fresh_db): def test_transform_with_unique_constraint_implicit_index(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
# Create a table with a UNIQUE constraint on 'name', which creates an implicit index # Create a table with a UNIQUE constraint on 'name', which creates an implicit index
fresh_db.execute(""" fresh_db.execute("""
CREATE TABLE dogs ( CREATE TABLE dogs (
id INTEGER PRIMARY KEY, id INTEGER PRIMARY KEY,
name TEXT UNIQUE, name TEXT UNIQUE ON CONFLICT IGNORE,
age INTEGER age INTEGER
); );
""") """)
dogs.insert({"id": 1, "name": "Cleo", "age": 5}) dogs.insert({"id": 1, "name": "Cleo", "age": 5})
# Attempt to transform the table without modifying 'name' dogs.transform(types={"age": str}, rename={"name": "dog_name"})
with pytest.raises(TransformError) as excinfo:
dogs.transform(types={"age": str}) assert 'dog_name" TEXT UNIQUE ON CONFLICT IGNORE' in dogs.schema
dogs.insert({"id": 2, "dog_name": "Cleo", "age": "6"})
assert list(dogs.rows) == [{"id": 1, "dog_name": "Cleo", "age": "5"}]
def test_transform_preserves_composite_unique_constraint(fresh_db):
fresh_db.execute("""
CREATE TABLE memberships (
account_id INTEGER,
email TEXT,
note TEXT,
CONSTRAINT unique_membership
UNIQUE (account_id DESC, email COLLATE NOCASE)
ON CONFLICT ABORT
)
""")
memberships = fresh_db.table("memberships")
memberships.insert({"account_id": 1, "email": "one@example.com", "note": "x"})
memberships.transform(rename={"account_id": "organization_id"}, types={"note": str})
assert ( assert (
"Index 'sqlite_autoindex_dogs_1' on table 'dogs' does not have a CREATE INDEX statement." 'CONSTRAINT "unique_membership" UNIQUE '
in str(excinfo.value) '("organization_id" DESC, "email" COLLATE "NOCASE") ON CONFLICT ABORT'
in memberships.schema
) )
assert ( with pytest.raises(sqlite3.IntegrityError):
"You must manually drop this index prior to running this transformation and manually recreate the new index after running this transformation." memberships.insert(
in str(excinfo.value) {"organization_id": 1, "email": "ONE@example.com", "note": "y"}
) )
def test_transform_preserves_column_unique_collation(fresh_db):
fresh_db.execute("""
CREATE TABLE people (
id INTEGER PRIMARY KEY,
name TEXT COLLATE NOCASE UNIQUE
)
""")
people = fresh_db.table("people")
people.insert({"id": 1, "name": "Cleo"})
people.transform(rename={"name": "full_name"})
assert 'UNIQUE ("full_name" COLLATE "NOCASE")' in people.schema
with pytest.raises(sqlite3.IntegrityError):
people.insert({"id": 2, "full_name": "cleo"})
def test_transform_drops_entire_composite_unique_constraint(fresh_db):
fresh_db.execute("""
CREATE TABLE memberships (
account_id INTEGER,
email TEXT,
UNIQUE (account_id, email)
)
""")
memberships = fresh_db.table("memberships")
memberships.insert({"account_id": 1, "email": "one@example.com"})
memberships.transform(drop={"email"})
assert "UNIQUE" not in memberships.schema
memberships.insert({"account_id": 1})
def test_transform_preserves_autoincrement_and_sequence(fresh_db):
fresh_db.execute(
"CREATE TABLE entries (id INTEGER PRIMARY KEY AUTOINCREMENT, value TEXT)"
)
entries = fresh_db.table("entries")
entries.insert_all(({"value": "one"}, {"value": "two"}))
entries.delete(2)
entries.transform(rename={"value": "label"})
assert "PRIMARY KEY AUTOINCREMENT" in entries.schema
entries.insert({"label": "three"})
assert list(entries.rows) == [
{"id": 1, "label": "one"},
{"id": 3, "label": "three"},
]
@pytest.mark.parametrize(
"new_type,expected_value,expected_type",
[
(int, 42, int),
(float, 42.0, float),
("integer", 42, int),
("float", 42.0, float),
("REAL", 42.0, float),
],
)
def test_transform_empty_string_to_null_for_numeric_types(
fresh_db, new_type, expected_value, expected_type
):
fresh_db["test"].insert_all(
[
{"id": 1, "value": "42"},
{"id": 2, "value": ""},
{"id": 3, "value": None},
{"id": 4, "value": " "},
]
)
fresh_db["test"].transform(types={"value": new_type})
rows = {r["id"]: r["value"] for r in fresh_db["test"].rows}
assert rows[1] == expected_value
assert type(rows[1]) is expected_type
assert rows[2] is None
assert rows[3] is None
assert rows[4] == " "
def test_transform_preserves_view(fresh_db): def test_transform_preserves_view(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/831 # https://github.com/simonw/sqlite-utils/issues/831
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs") fresh_db.execute("create view dogs_view as select id, name from dogs")
view_sql_before = fresh_db.execute( view_sql_before = fresh_db.execute(
@ -958,8 +1182,8 @@ def test_transform_preserves_view(fresh_db):
def test_transform_variants_preserve_view(fresh_db, transform_params): def test_transform_variants_preserve_view(fresh_db, transform_params):
# Covers retyping, changing primary key and foreign key modifications, # Covers retyping, changing primary key and foreign key modifications,
# with a view whose columns are untouched by the transform # with a view whose columns are untouched by the transform
fresh_db["other"].insert({"id": 1}, pk="id") fresh_db.table("other").insert({"id": 1}, pk="id")
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo", "other_id": 1}, pk="id") dogs.insert({"id": 1, "name": "Cleo", "other_id": 1}, pk="id")
if "drop_foreign_keys" in transform_params: if "drop_foreign_keys" in transform_params:
dogs.transform(add_foreign_keys=[("other_id", "other", "id")]) dogs.transform(add_foreign_keys=[("other_id", "other", "id")])
@ -972,13 +1196,13 @@ def test_transform_variants_preserve_view(fresh_db, transform_params):
"select sql from sqlite_master where name = 'dogs_view'" "select sql from sqlite_master where name = 'dogs_view'"
).fetchone()[0] ).fetchone()[0]
assert view_sql_before == view_sql_after assert view_sql_before == view_sql_after
assert list(fresh_db["dogs_view"].rows) == [{"id": 1, "name": "Cleo"}] assert list(fresh_db.view("dogs_view").rows) == [{"id": 1, "name": "Cleo"}]
def test_transform_view_referencing_renamed_column(fresh_db): def test_transform_view_referencing_renamed_column(fresh_db):
# The view survives but querying it raises "no such column" - inherent # The view survives but querying it raises "no such column" - inherent
# to SQLite views, whose SQL is stored as text # to SQLite views, whose SQL is stored as text
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs") fresh_db.execute("create view dogs_view as select id, name from dogs")
dogs.transform(rename={"name": "title"}) dogs.transform(rename={"name": "title"})
@ -987,7 +1211,7 @@ def test_transform_view_referencing_renamed_column(fresh_db):
def test_transform_view_on_view(fresh_db): def test_transform_view_on_view(fresh_db):
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view v1 as select id, name from dogs") fresh_db.execute("create view v1 as select id, name from dogs")
fresh_db.execute("create view v2 as select name from v1") fresh_db.execute("create view v2 as select name from v1")
@ -999,13 +1223,13 @@ def test_transform_view_on_view(fresh_db):
"select sql from sqlite_master where type = 'view' order by name" "select sql from sqlite_master where type = 'view' order by name"
).fetchall() ).fetchall()
assert sqls_before == sqls_after assert sqls_before == sqls_after
assert list(fresh_db["v2"].rows) == [{"name": "Cleo"}] assert list(fresh_db.view("v2").rows) == [{"name": "Cleo"}]
def test_transform_keep_table_does_not_repoint_view(fresh_db): def test_transform_keep_table_does_not_repoint_view(fresh_db):
# Without legacy_alter_table the ALTER TABLE dogs RENAME TO dogs_backup # Without legacy_alter_table the ALTER TABLE dogs RENAME TO dogs_backup
# step would rewrite the view to select from "dogs_backup" # step would rewrite the view to select from "dogs_backup"
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs") fresh_db.execute("create view dogs_view as select id, name from dogs")
dogs.transform(types={"name": str}, keep_table="dogs_backup") dogs.transform(types={"name": str}, keep_table="dogs_backup")
@ -1015,7 +1239,7 @@ def test_transform_keep_table_does_not_repoint_view(fresh_db):
assert "dogs_backup" not in view_sql assert "dogs_backup" not in view_sql
# View reads from the live table, not the frozen backup # View reads from the live table, not the frozen backup
dogs.insert({"id": 2, "name": "Pancakes"}) dogs.insert({"id": 2, "name": "Pancakes"})
assert list(fresh_db["dogs_view"].rows) == [ assert list(fresh_db.view("dogs_view").rows) == [
{"id": 1, "name": "Cleo"}, {"id": 1, "name": "Cleo"},
{"id": 2, "name": "Pancakes"}, {"id": 2, "name": "Pancakes"},
] ]
@ -1024,7 +1248,7 @@ def test_transform_keep_table_does_not_repoint_view(fresh_db):
def test_transform_sql_standalone_statements_work_with_view(fresh_db): def test_transform_sql_standalone_statements_work_with_view(fresh_db):
# The documented "run these statements yourself" workflow should be # The documented "run these statements yourself" workflow should be
# standalone-correct, so the pragmas must come from transform_sql() # standalone-correct, so the pragmas must come from transform_sql()
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs") fresh_db.execute("create view dogs_view as select id, name from dogs")
sqls = dogs.transform_sql(types={"name": str}, tmp_suffix="suffix") sqls = dogs.transform_sql(types={"name": str}, tmp_suffix="suffix")
@ -1033,12 +1257,12 @@ def test_transform_sql_standalone_statements_work_with_view(fresh_db):
assert sqls[-1] == "PRAGMA legacy_alter_table=OFF;" assert sqls[-1] == "PRAGMA legacy_alter_table=OFF;"
for sql in sqls: for sql in sqls:
fresh_db.execute(sql) fresh_db.execute(sql)
assert list(fresh_db["dogs_view"].rows) == [{"id": 1, "name": "Cleo"}] assert list(fresh_db.view("dogs_view").rows) == [{"id": 1, "name": "Cleo"}]
def test_transform_with_view_in_open_transaction(fresh_db): def test_transform_with_view_in_open_transaction(fresh_db):
fresh_db.conn.execute("PRAGMA foreign_keys=ON") fresh_db.conn.execute("PRAGMA foreign_keys=ON")
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
fresh_db.execute("create view dogs_view as select id, name from dogs") fresh_db.execute("create view dogs_view as select id, name from dogs")
with fresh_db.conn: with fresh_db.conn:
@ -1054,7 +1278,7 @@ def test_transform_with_view_in_open_transaction(fresh_db):
def test_transform_restores_legacy_alter_table_setting(fresh_db): def test_transform_restores_legacy_alter_table_setting(fresh_db):
if sqlite3.sqlite_version_info < (3, 25, 0): if sqlite3.sqlite_version_info < (3, 25, 0):
pytest.skip("legacy_alter_table pragma requires SQLite 3.25 or higher") pytest.skip("legacy_alter_table pragma requires SQLite 3.25 or higher")
dogs = fresh_db["dogs"] dogs = fresh_db.table("dogs")
dogs.insert({"id": 1, "name": "Cleo"}, pk="id") dogs.insert({"id": 1, "name": "Cleo"}, pk="id")
# Default is OFF, reset to OFF afterwards # Default is OFF, reset to OFF afterwards
dogs.transform(types={"name": str}) dogs.transform(types={"name": str})
@ -1075,7 +1299,7 @@ def test_transform_preserves_check_constraints(fresh_db):
CONSTRAINT nonzero_id CHECK(id != 0) CONSTRAINT nonzero_id CHECK(id != 0)
) )
""") """)
scores = fresh_db["scores"] scores = fresh_db.table("scores")
scores.insert({"id": 1, "score": 50}) scores.insert({"id": 1, "score": 50})
scores.transform() scores.transform()
assert scores.checks == [ assert scores.checks == [
@ -1095,7 +1319,7 @@ def test_transform_preserves_check_ending_in_line_comment(fresh_db):
) )
) )
""") """)
inventory = fresh_db["inventory"] inventory = fresh_db.table("inventory")
inventory.transform(types={"quantity": float}) inventory.transform(types={"quantity": float})
assert inventory.checks == [Check("quantity >= 0 -- Quantity cannot be negative")] assert inventory.checks == [Check("quantity >= 0 -- Quantity cannot be negative")]
with pytest.raises(sqlite3.IntegrityError, match="CHECK constraint failed"): with pytest.raises(sqlite3.IntegrityError, match="CHECK constraint failed"):
@ -1113,7 +1337,7 @@ def test_transform_preserves_comments_owned_by_columns(fresh_db):
age INTEGER -- May be NULL age INTEGER -- May be NULL
) )
""") """)
people = fresh_db["people"] people = fresh_db.table("people")
people.insert({"id": 1, "name": "Cleo", "age": 5}) people.insert({"id": 1, "name": "Cleo", "age": 5})
people.transform( people.transform(
rename={"name": "display_name"}, rename={"name": "display_name"},
@ -1143,8 +1367,8 @@ def test_transform_drops_comments_owned_by_dropped_column(fresh_db):
obsolete TEXT /* Drop this too */ obsolete TEXT /* Drop this too */
) )
""") """)
fresh_db["t"].transform(drop={"obsolete"}) fresh_db.table("t").transform(drop={"obsolete"})
schema = fresh_db["t"].schema schema = fresh_db.table("t").schema
assert "Keep this explanation" in schema assert "Keep this explanation" in schema
assert "Drop this explanation" not in schema assert "Drop this explanation" not in schema
assert "Drop this too" not in schema assert "Drop this too" not in schema
@ -1159,7 +1383,7 @@ def test_transform_renames_columns_inside_check_constraints(fresh_db):
CONSTRAINT within_maximum CHECK(quantity <= maximum) CONSTRAINT within_maximum CHECK(quantity <= maximum)
) )
""") """)
inventory = fresh_db["inventory"] inventory = fresh_db.table("inventory")
inventory.insert({"quantity": 2, "maximum": 3}) inventory.insert({"quantity": 2, "maximum": 3})
inventory.transform(rename={"quantity": "amount"}) inventory.transform(rename={"quantity": "amount"})
assert inventory.checks == [ assert inventory.checks == [
@ -1182,7 +1406,7 @@ def test_transform_check_rewrite_preserves_functions_and_quotes(fresh_db):
CHECK(length("old name") > 0 AND length != '') CHECK(length("old name") > 0 AND length != '')
) )
""") """)
items = fresh_db["items"] items = fresh_db.table("items")
items.insert({"length": "label", "old name": "hello"}) items.insert({"length": "label", "old name": "hello"})
items.transform(rename={"length": "description", "old name": "new name"}) items.transform(rename={"length": "description", "old name": "new name"})
assert items.checks == [Check("length(\"new name\") > 0 AND description != ''")] assert items.checks == [Check("length(\"new name\") > 0 AND description != ''")]
@ -1190,9 +1414,9 @@ def test_transform_check_rewrite_preserves_functions_and_quotes(fresh_db):
def test_transform_check_rewrite_quotes_keyword_column(fresh_db): def test_transform_check_rewrite_quotes_keyword_column(fresh_db):
fresh_db.execute("CREATE TABLE t(old_name TEXT CHECK(old_name != ''))") fresh_db.execute("CREATE TABLE t(old_name TEXT CHECK(old_name != ''))")
fresh_db["t"].insert({"old_name": "value"}) fresh_db.table("t").insert({"old_name": "value"})
fresh_db["t"].transform(rename={"old_name": "select"}) fresh_db.table("t").transform(rename={"old_name": "select"})
assert fresh_db["t"].checks == [Check("\"select\" != ''", column="select")] assert fresh_db.table("t").checks == [Check("\"select\" != ''", column="select")]
def test_transform_check_rewrite_does_not_rename_collations_or_cast_types(fresh_db): def test_transform_check_rewrite_does_not_rename_collations_or_cast_types(fresh_db):
@ -1209,9 +1433,9 @@ def test_transform_check_rewrite_does_not_rename_collations_or_cast_types(fresh_
) )
) )
""") """)
fresh_db["t"].insert({"nocase": "n", "kind": "k", "other": "o"}) fresh_db.table("t").insert({"nocase": "n", "kind": "k", "other": "o"})
fresh_db["t"].transform(rename={"nocase": "label", "kind": "category"}) fresh_db.table("t").transform(rename={"nocase": "label", "kind": "category"})
check = fresh_db["t"].checks[0].check check = fresh_db.table("t").checks[0].check
assert "COLLATE nocase" in check assert "COLLATE nocase" in check
assert "AS kind" in check assert "AS kind" in check
assert "AND label != ''" in check assert "AND label != ''" in check
@ -1226,9 +1450,9 @@ def test_transform_drops_check_owned_by_dropped_column(fresh_db):
CHECK(id > 0) CHECK(id > 0)
) )
""") """)
fresh_db["t"].insert({"id": 1, "obsolete": 2}) fresh_db.table("t").insert({"id": 1, "obsolete": 2})
fresh_db["t"].transform(drop={"obsolete"}) fresh_db.table("t").transform(drop={"obsolete"})
assert fresh_db["t"].checks == [Check("id > 0")] assert fresh_db.table("t").checks == [Check("id > 0")]
def test_transform_refuses_to_drop_column_used_by_remaining_check(fresh_db): def test_transform_refuses_to_drop_column_used_by_remaining_check(fresh_db):
@ -1239,7 +1463,7 @@ def test_transform_refuses_to_drop_column_used_by_remaining_check(fresh_db):
CHECK(minimum <= maximum) CHECK(minimum <= maximum)
) )
""") """)
ranges = fresh_db["ranges"] ranges = fresh_db.table("ranges")
ranges.insert({"minimum": 1, "maximum": 2}) ranges.insert({"minimum": 1, "maximum": 2})
schema_before = ranges.schema schema_before = ranges.schema
with pytest.raises( with pytest.raises(

View file

@ -7,14 +7,14 @@ from sqlite_utils.db import NotFoundError
def test_update_rowid_table(fresh_db): def test_update_rowid_table(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
rowid = table.insert({"foo": "bar"}).last_pk rowid = table.insert({"foo": "bar"}).last_pk
table.update(rowid, {"foo": "baz"}) table.update(rowid, {"foo": "baz"})
assert [{"foo": "baz"}] == list(table.rows) assert [{"foo": "baz"}] == list(table.rows)
def test_update_pk_table(fresh_db): def test_update_pk_table(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
pk = table.insert({"foo": "bar", "id": 5}, pk="id").last_pk pk = table.insert({"foo": "bar", "id": 5}, pk="id").last_pk
assert 5 == pk assert 5 == pk
table.update(pk, {"foo": "baz"}) table.update(pk, {"foo": "baz"})
@ -22,7 +22,7 @@ def test_update_pk_table(fresh_db):
def test_update_compound_pk_table(fresh_db): def test_update_compound_pk_table(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
pk = table.insert({"id1": 5, "id2": 3, "v": 1}, pk=("id1", "id2")).last_pk pk = table.insert({"id1": 5, "id2": 3, "v": 1}, pk=("id1", "id2")).last_pk
assert (5, 3) == pk assert (5, 3) == pk
table.update(pk, {"v": 2}) table.update(pk, {"v": 2})
@ -42,14 +42,14 @@ def test_update_compound_pk_table(fresh_db):
), ),
) )
def test_update_invalid_pk(fresh_db, pk, update_pk): def test_update_invalid_pk(fresh_db, pk, update_pk):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"id1": 5, "id2": 3, "v": 1}, pk=pk) table.insert({"id1": 5, "id2": 3, "v": 1}, pk=pk)
with pytest.raises(NotFoundError): with pytest.raises(NotFoundError):
table.update(update_pk, {"v": 2}) table.update(update_pk, {"v": 2})
def test_update_alter(fresh_db): def test_update_alter(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
rowid = table.insert({"foo": "bar"}).last_pk rowid = table.insert({"foo": "bar"}).last_pk
table.update(rowid, {"new_col": 1.2}, alter=True) table.update(rowid, {"new_col": 1.2}, alter=True)
assert [{"foo": "bar", "new_col": 1.2}] == list(table.rows) assert [{"foo": "bar", "new_col": 1.2}] == list(table.rows)
@ -72,7 +72,7 @@ def test_update_alter(fresh_db):
def test_update_alter_with_special_column_characters(fresh_db): def test_update_alter_with_special_column_characters(fresh_db):
# With double-quote escaping, columns with special characters are now valid # With double-quote escaping, columns with special characters are now valid
table = fresh_db["table"] table = fresh_db.table("table")
rowid = table.insert({"foo": "bar"}).last_pk rowid = table.insert({"foo": "bar"}).last_pk
table.update(rowid, {"new_col[abc]": 1.2}, alter=True) table.update(rowid, {"new_col[abc]": 1.2}, alter=True)
assert list(table.rows) == [{"foo": "bar", "new_col[abc]": 1.2}] assert list(table.rows) == [{"foo": "bar", "new_col[abc]": 1.2}]
@ -106,8 +106,8 @@ def test_update_with_no_values_sets_last_pk(fresh_db):
), ),
) )
def test_update_dictionaries_and_lists_as_json(fresh_db, data_structure): def test_update_dictionaries_and_lists_as_json(fresh_db, data_structure):
fresh_db["test"].insert({"id": 1, "data": ""}, pk="id") fresh_db.table("test").insert({"id": 1, "data": ""}, pk="id")
fresh_db["test"].update(1, {"data": data_structure}) fresh_db.table("test").update(1, {"data": data_structure})
row = fresh_db.execute("select id, data from test").fetchone() row = fresh_db.execute("select id, data from test").fetchone()
assert row[0] == 1 assert row[0] == 1
assert data_structure == json.loads(row[1]) assert data_structure == json.loads(row[1])

View file

@ -7,15 +7,15 @@ from sqlite_utils.db import PrimaryKeyRequired
@pytest.mark.parametrize("use_old_upsert", (False, True)) @pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert(use_old_upsert): def test_upsert(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert) db = Database(memory=True, use_old_upsert=use_old_upsert)
table = db["table"] table = db.table("table")
table.insert({"id": 1, "name": "Cleo"}, pk="id") table.insert_all([{"id": 1, "name": "Cleo"}], pk="id", replace=True)
table.upsert({"id": 1, "age": 5}, pk="id", alter=True) table.upsert({"id": 1, "age": 5}, pk="id", alter=True)
assert list(table.rows) == [{"id": 1, "name": "Cleo", "age": 5}] assert list(table.rows) == [{"id": 1, "name": "Cleo", "age": 5}]
assert table.last_pk == 1 assert table.last_pk == 1
def test_upsert_all(fresh_db): def test_upsert_all(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert_all([{"id": 1, "name": "Cleo"}, {"id": 2, "name": "Nixie"}], pk="id") table.upsert_all([{"id": 1, "name": "Cleo"}, {"id": 2, "name": "Nixie"}], pk="id")
table.upsert_all([{"id": 1, "age": 5}, {"id": 2, "age": 5}], pk="id", alter=True) table.upsert_all([{"id": 1, "age": 5}, {"id": 2, "age": 5}], pk="id", alter=True)
assert list(table.rows) == [ assert list(table.rows) == [
@ -26,7 +26,7 @@ def test_upsert_all(fresh_db):
def test_upsert_all_single_column(fresh_db): def test_upsert_all_single_column(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert_all([{"name": "Cleo"}], pk="name") table.upsert_all([{"name": "Cleo"}], pk="name")
assert list(table.rows) == [{"name": "Cleo"}] assert list(table.rows) == [{"name": "Cleo"}]
assert table.pks == ["name"] assert table.pks == ["name"]
@ -34,16 +34,16 @@ def test_upsert_all_single_column(fresh_db):
def test_upsert_all_not_null(fresh_db): def test_upsert_all_not_null(fresh_db):
# https://github.com/simonw/sqlite-utils/issues/538 # https://github.com/simonw/sqlite-utils/issues/538
fresh_db["comments"].upsert_all( fresh_db.table("comments").upsert_all(
[{"id": 1, "name": "Cleo"}], [{"id": 1, "name": "Cleo"}],
pk="id", pk="id",
not_null=["name"], not_null=["name"],
) )
assert list(fresh_db["comments"].rows) == [{"id": 1, "name": "Cleo"}] assert list(fresh_db.table("comments").rows) == [{"id": 1, "name": "Cleo"}]
def test_upsert_error_if_no_pk(fresh_db): def test_upsert_error_if_no_pk(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
with pytest.raises(PrimaryKeyRequired): with pytest.raises(PrimaryKeyRequired):
table.upsert_all([{"id": 1, "name": "Cleo"}]) table.upsert_all([{"id": 1, "name": "Cleo"}])
with pytest.raises(PrimaryKeyRequired): with pytest.raises(PrimaryKeyRequired):
@ -53,7 +53,7 @@ def test_upsert_error_if_no_pk(fresh_db):
@pytest.mark.parametrize("use_old_upsert", (False, True)) @pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert_empty_record_errors(use_old_upsert): def test_upsert_empty_record_errors(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert) db = Database(memory=True, use_old_upsert=use_old_upsert)
table = db["table"] table = db.table("table")
table.insert({"id": 1, "name": "Cleo"}, pk="id") table.insert({"id": 1, "name": "Cleo"}, pk="id")
with pytest.raises(PrimaryKeyRequired): with pytest.raises(PrimaryKeyRequired):
table.upsert({}, pk="id") table.upsert({}, pk="id")
@ -66,7 +66,7 @@ def test_upsert_empty_record_errors(use_old_upsert):
@pytest.mark.parametrize("use_old_upsert", (False, True)) @pytest.mark.parametrize("use_old_upsert", (False, True))
def test_upsert_missing_pk_value_errors(use_old_upsert): def test_upsert_missing_pk_value_errors(use_old_upsert):
db = Database(memory=True, use_old_upsert=use_old_upsert) db = Database(memory=True, use_old_upsert=use_old_upsert)
table = db["table"] table = db.table("table")
table.insert({"id": 1, "name": "Cleo"}, pk="id") table.insert({"id": 1, "name": "Cleo"}, pk="id")
# Records that omit the pk column entirely # Records that omit the pk column entirely
with pytest.raises(PrimaryKeyRequired): with pytest.raises(PrimaryKeyRequired):
@ -78,7 +78,7 @@ def test_upsert_missing_pk_value_errors(use_old_upsert):
def test_upsert_missing_compound_pk_value_errors(fresh_db): def test_upsert_missing_compound_pk_value_errors(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.insert({"a": "x", "b": "y", "v": 1}, pk=("a", "b")) table.insert({"a": "x", "b": "y", "v": 1}, pk=("a", "b"))
# Missing one component of the detected compound primary key # Missing one component of the detected compound primary key
with pytest.raises(PrimaryKeyRequired): with pytest.raises(PrimaryKeyRequired):
@ -105,7 +105,7 @@ def test_upsert_uses_compound_pk_from_existing_table(use_old_upsert):
primary key (Source, Object, Category) primary key (Source, Object, Category)
) )
""") """)
table = db["summary"] table = db.table("summary")
table.upsert( table.upsert(
{ {
"Source": "Client A", "Source": "Client A",
@ -134,7 +134,7 @@ def test_upsert_uses_compound_pk_from_existing_table(use_old_upsert):
def test_upsert_with_hash_id(fresh_db): def test_upsert_with_hash_id(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert({"foo": "bar"}, hash_id="pk") table.upsert({"foo": "bar"}, hash_id="pk")
assert [{"pk": "a5e744d0164540d33b1d7ea616c28f2fa97e754a", "foo": "bar"}] == list( assert [{"pk": "a5e744d0164540d33b1d7ea616c28f2fa97e754a", "foo": "bar"}] == list(
table.rows table.rows
@ -144,7 +144,7 @@ def test_upsert_with_hash_id(fresh_db):
@pytest.mark.parametrize("hash_id", (None, "custom_id")) @pytest.mark.parametrize("hash_id", (None, "custom_id"))
def test_upsert_with_hash_id_columns(fresh_db, hash_id): def test_upsert_with_hash_id_columns(fresh_db, hash_id):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert({"a": 1, "b": 2, "c": 3}, hash_id=hash_id, hash_id_columns=("a", "b")) table.upsert({"a": 1, "b": 2, "c": 3}, hash_id=hash_id, hash_id_columns=("a", "b"))
assert list(table.rows) == [ assert list(table.rows) == [
{ {
@ -167,7 +167,7 @@ def test_upsert_with_hash_id_columns(fresh_db, hash_id):
def test_upsert_compound_primary_key(fresh_db): def test_upsert_compound_primary_key(fresh_db):
table = fresh_db["table"] table = fresh_db.table("table")
table.upsert_all( table.upsert_all(
[ [
{"species": "dog", "id": 1, "name": "Cleo", "age": 4}, {"species": "dog", "id": 1, "name": "Cleo", "age": 4},

View file

@ -18,7 +18,7 @@ def test_enable_disable_wal(db_path_tmpdir):
assert "test.db-wal" not in [f.basename for f in tmpdir.listdir()] assert "test.db-wal" not in [f.basename for f in tmpdir.listdir()]
db.enable_wal() db.enable_wal()
assert "wal" == db.journal_mode assert "wal" == db.journal_mode
db["test"].insert({"foo": "bar"}) db.table("test").insert({"foo": "bar"})
assert "test.db-wal" in [f.basename for f in tmpdir.listdir()] assert "test.db-wal" in [f.basename for f in tmpdir.listdir()]
db.disable_wal() db.disable_wal()
assert "delete" == db.journal_mode assert "delete" == db.journal_mode
@ -27,25 +27,25 @@ def test_enable_disable_wal(db_path_tmpdir):
def test_enable_wal_inside_transaction_raises(db_path_tmpdir): def test_enable_wal_inside_transaction_raises(db_path_tmpdir):
db, _path, _tmpdir = db_path_tmpdir db, _path, _tmpdir = db_path_tmpdir
db["test"].insert({"id": 1}, pk="id") db.table("test").insert({"id": 1}, pk="id")
with pytest.raises(TransactionError), db.atomic(): with pytest.raises(TransactionError), db.atomic():
db["test"].insert({"id": 2}, pk="id") db.table("test").insert({"id": 2}, pk="id")
db.enable_wal() db.enable_wal()
# The atomic() block must have rolled back cleanly and the # The atomic() block must have rolled back cleanly and the
# journal mode must be unchanged # journal mode must be unchanged
assert db.journal_mode == "delete" assert db.journal_mode == "delete"
assert [r["id"] for r in db["test"].rows] == [1] assert [r["id"] for r in db.table("test").rows] == [1]
def test_disable_wal_inside_transaction_raises(db_path_tmpdir): def test_disable_wal_inside_transaction_raises(db_path_tmpdir):
db, _path, _tmpdir = db_path_tmpdir db, _path, _tmpdir = db_path_tmpdir
db.enable_wal() db.enable_wal()
db["test"].insert({"id": 1}, pk="id") db.table("test").insert({"id": 1}, pk="id")
with pytest.raises(TransactionError), db.atomic(): with pytest.raises(TransactionError), db.atomic():
db["test"].insert({"id": 2}, pk="id") db.table("test").insert({"id": 2}, pk="id")
db.disable_wal() db.disable_wal()
assert db.journal_mode == "wal" assert db.journal_mode == "wal"
assert [r["id"] for r in db["test"].rows] == [1] assert [r["id"] for r in db.table("test").rows] == [1]
def test_ensure_autocommit_on(db_path_tmpdir): def test_ensure_autocommit_on(db_path_tmpdir):
@ -65,9 +65,9 @@ def test_enable_wal_noop_inside_transaction_is_allowed(db_path_tmpdir):
db, _path, _tmpdir = db_path_tmpdir db, _path, _tmpdir = db_path_tmpdir
db.enable_wal() db.enable_wal()
with db.atomic(): with db.atomic():
db["test"].insert({"id": 1}, pk="id") db.table("test").insert({"id": 1}, pk="id")
db.enable_wal() db.enable_wal()
assert [r["id"] for r in db["test"].rows] == [1] assert [r["id"] for r in db.table("test").rows] == [1]
def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir): def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir):
@ -75,7 +75,7 @@ def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir):
# effect, silently breaking the caller's rollback guarantee - so # effect, silently breaking the caller's rollback guarantee - so
# entering autocommit mode with a transaction open is an error # entering autocommit mode with a transaction open is an error
db, _path, _tmpdir = db_path_tmpdir db, _path, _tmpdir = db_path_tmpdir
db["test"].insert({"id": 1}, pk="id") db.table("test").insert({"id": 1}, pk="id")
db.begin() db.begin()
db.execute("insert into test (id) values (2)") db.execute("insert into test (id) values (2)")
with pytest.raises(TransactionError), db.ensure_autocommit_on(): with pytest.raises(TransactionError), db.ensure_autocommit_on():
@ -83,4 +83,4 @@ def test_ensure_autocommit_on_inside_transaction_raises(db_path_tmpdir):
# The transaction is still open and can still be rolled back # The transaction is still open and can still be rolled back
assert db.conn.in_transaction assert db.conn.in_transaction
db.rollback() db.rollback()
assert [r["id"] for r in db["test"].rows] == [1] assert [r["id"] for r in db.table("test").rows] == [1]