remote/sqlite-utils
1
0
Fork 0
mirror of https://github.com/simonw/sqlite-utils.git synced 2026-10-26 10:17:04 +01:00
Code Issues Projects Releases Packages Wiki Activity

feat: support public AUTOINCREMENT creation and CLI options

Browse source 
Refs #664
This commit is contained in:
WU Leizhi 2026-10-03 06:10:42 +08:00
commit 85dd980ccf
9 changed files with 639 additions and 78 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/changelog.rst
View file

@ -4,6 +4,11 @@
Changelog
===========
Unreleased
----------
- The Python table creation, insert, upsert and lookup APIs now accept ``autoincrement=`` for single integer primary keys. The ``create-table``, ``insert``, ``upsert`` and ``transform`` commands accept ``--autoincrement`` and ``--no-autoincrement``. Existing table mode changes can be applied or previewed with ``transform()`` and ``transform_sql()``. (:issue:`664`)
.. _v4_2_1:
4.2.1 (2026-08-13)

171
docs/cli-reference.rst
View file

@ -287,40 +287,47 @@ See :ref:`cli_inserting_data`, :ref:`cli_insert_csv_tsv`, :ref:`cli_insert_unstr
' --pk id
Options:
--pk TEXT Columns to use as the primary key, e.g. id
--code TEXT Python code defining a rows() function or iterable
of rows to insert
--flatten Flatten nested JSON objects, so {"a": {"b": 1}}
becomes {"a_b": 1}
--nl Expect newline-delimited JSON
-c, --csv Expect CSV input
--tsv Expect TSV input
--empty-null Treat empty strings as NULL
--lines Treat each line as a single value called 'line'
--text Treat input as a single value called 'text'
--convert TEXT Python code to convert each item
--import TEXT Python modules to import
--delimiter TEXT Delimiter to use for CSV files
--quotechar TEXT Quote character to use for CSV/TSV
--sniff Detect delimiter and quote character
--no-headers CSV file has no header row
--encoding TEXT Character encoding for input, defaults to utf-8
--batch-size INTEGER Commit every X records
--stop-after INTEGER Stop after X records
--alter Alter existing table to add any missing columns
--not-null TEXT Columns that should be created as NOT NULL
--default <TEXT TEXT>... Default value that should be set for a column
--type <TEXT CHOICE>... Column types to use when creating the table
--no-detect-types Treat all CSV/TSV columns as TEXT
--analyze Run ANALYZE at the end of this operation
--load-extension TEXT Path to SQLite extension, with optional :entrypoint
--silent Do not show progress bar
--strict Apply STRICT mode to created table
--ignore Ignore records if pk already exists
--replace Replace records if pk already exists
--truncate Truncate table before inserting records, if table
already exists
-h, --help Show this message and exit.
--pk TEXT Columns to use as the primary key, e.g. id
--code TEXT Python code defining a rows() function or
iterable of rows to insert
--flatten Flatten nested JSON objects, so {"a": {"b":
1}} becomes {"a_b": 1}
--nl Expect newline-delimited JSON
-c, --csv Expect CSV input
--tsv Expect TSV input
--empty-null Treat empty strings as NULL
--lines Treat each line as a single value called
'line'
--text Treat input as a single value called 'text'
--convert TEXT Python code to convert each item
--import TEXT Python modules to import
--delimiter TEXT Delimiter to use for CSV files
--quotechar TEXT Quote character to use for CSV/TSV
--sniff Detect delimiter and quote character
--no-headers CSV file has no header row
--encoding TEXT Character encoding for input, defaults to
utf-8
--batch-size INTEGER Commit every X records
--stop-after INTEGER Stop after X records
--alter Alter existing table to add any missing
columns
--not-null TEXT Columns that should be created as NOT NULL
--default <TEXT TEXT>... Default value that should be set for a column
--type <TEXT CHOICE>... Column types to use when creating the table
--no-detect-types Treat all CSV/TSV columns as TEXT
--analyze Run ANALYZE at the end of this operation
--load-extension TEXT Path to SQLite extension, with optional
:entrypoint
--silent Do not show progress bar
--strict Apply STRICT mode to created table
--autoincrement / --no-autoincrement
Use AUTOINCREMENT for the INTEGER PRIMARY KEY
when creating the table
--ignore Ignore records if pk already exists
--replace Replace records if pk already exists
--truncate Truncate table before inserting records, if
table already exists
-h, --help Show this message and exit.
.. _cli_ref_upsert:
@ -351,36 +358,43 @@ See :ref:`cli_upsert`.
]' | sqlite-utils upsert data.db chickens - --pk id
Options:
--pk TEXT Columns to use as the primary key, e.g. id
--code TEXT Python code defining a rows() function or iterable
of rows to insert
--flatten Flatten nested JSON objects, so {"a": {"b": 1}}
becomes {"a_b": 1}
--nl Expect newline-delimited JSON
-c, --csv Expect CSV input
--tsv Expect TSV input
--empty-null Treat empty strings as NULL
--lines Treat each line as a single value called 'line'
--text Treat input as a single value called 'text'
--convert TEXT Python code to convert each item
--import TEXT Python modules to import
--delimiter TEXT Delimiter to use for CSV files
--quotechar TEXT Quote character to use for CSV/TSV
--sniff Detect delimiter and quote character
--no-headers CSV file has no header row
--encoding TEXT Character encoding for input, defaults to utf-8
--batch-size INTEGER Commit every X records
--stop-after INTEGER Stop after X records
--alter Alter existing table to add any missing columns
--not-null TEXT Columns that should be created as NOT NULL
--default <TEXT TEXT>... Default value that should be set for a column
--type <TEXT CHOICE>... Column types to use when creating the table
--no-detect-types Treat all CSV/TSV columns as TEXT
--analyze Run ANALYZE at the end of this operation
--load-extension TEXT Path to SQLite extension, with optional :entrypoint
--silent Do not show progress bar
--strict Apply STRICT mode to created table
-h, --help Show this message and exit.
--pk TEXT Columns to use as the primary key, e.g. id
--code TEXT Python code defining a rows() function or
iterable of rows to insert
--flatten Flatten nested JSON objects, so {"a": {"b":
1}} becomes {"a_b": 1}
--nl Expect newline-delimited JSON
-c, --csv Expect CSV input
--tsv Expect TSV input
--empty-null Treat empty strings as NULL
--lines Treat each line as a single value called
'line'
--text Treat input as a single value called 'text'
--convert TEXT Python code to convert each item
--import TEXT Python modules to import
--delimiter TEXT Delimiter to use for CSV files
--quotechar TEXT Quote character to use for CSV/TSV
--sniff Detect delimiter and quote character
--no-headers CSV file has no header row
--encoding TEXT Character encoding for input, defaults to
utf-8
--batch-size INTEGER Commit every X records
--stop-after INTEGER Stop after X records
--alter Alter existing table to add any missing
columns
--not-null TEXT Columns that should be created as NOT NULL
--default <TEXT TEXT>... Default value that should be set for a column
--type <TEXT CHOICE>... Column types to use when creating the table
--no-detect-types Treat all CSV/TSV columns as TEXT
--analyze Run ANALYZE at the end of this operation
--load-extension TEXT Path to SQLite extension, with optional
:entrypoint
--silent Do not show progress bar
--strict Apply STRICT mode to created table
--autoincrement / --no-autoincrement
Use AUTOINCREMENT for the INTEGER PRIMARY KEY
when creating the table
-h, --help Show this message and exit.
.. _cli_ref_bulk:
@ -510,6 +524,9 @@ See :ref:`cli_transform_table`.
--drop-foreign-key TEXT Drop foreign key constraint for this column
--strict / --no-strict Enable or disable STRICT mode (default:
preserve current mode)
--autoincrement / --no-autoincrement
Enable or disable AUTOINCREMENT (default:
preserve current mode)
--sql Output SQL without executing it
--load-extension TEXT Path to SQLite extension, with optional
:entrypoint
@ -966,17 +983,21 @@ See :ref:`cli_create_table`.
Valid column types are text, integer, real, float, blob and any.
Options:
--pk TEXT Column to use as primary key
--not-null TEXT Columns that should be created as NOT NULL
--default <TEXT TEXT>... Default value that should be set for a column
--fk <TEXT TEXT TEXT>... Column, other table, other column to set as a
foreign key
--ignore If table already exists, do nothing
--replace If table already exists, replace it
--transform If table already exists, try to transform the schema
--load-extension TEXT Path to SQLite extension, with optional :entrypoint
--strict Apply STRICT mode to created table
-h, --help Show this message and exit.
--pk TEXT Column to use as primary key
--not-null TEXT Columns that should be created as NOT NULL
--default <TEXT TEXT>... Default value that should be set for a column
--fk <TEXT TEXT TEXT>... Column, other table, other column to set as a
foreign key
--ignore If table already exists, do nothing
--replace If table already exists, replace it
--transform If table already exists, try to transform the
schema
--load-extension TEXT Path to SQLite extension, with optional
:entrypoint
--strict Apply STRICT mode to created table
--autoincrement / --no-autoincrement
Use AUTOINCREMENT for the INTEGER PRIMARY KEY
-h, --help Show this message and exit.
.. _cli_ref_create_index:

19
docs/cli.rst
View file

@ -2079,6 +2079,22 @@ You can pass as many column-name column-type pairs as you like. Valid types are
Pass ``--pk`` more than once for a compound primary key that covers multiple columns.
For a single integer primary key, add ``--autoincrement`` to prevent reuse of deleted IDs:
.. code-block:: bash
sqlite-utils create-table events.db events id integer name text --pk id --autoincrement
The option also works when ``insert`` or ``upsert`` creates a new table:
.. code-block:: bash
echo '{"name": "first event"}' | sqlite-utils insert events.db events - --pk id --autoincrement
An omitted primary key column is created as an integer column. ``upsert`` still requires primary key values in the input. Compound and non-integer primary keys cannot use this option. Ordinary integer primary keys already generate missing IDs; ``AUTOINCREMENT`` prevents their reuse and does not guarantee consecutive values. Use ``--no-autoincrement`` to explicitly create a table without it.
Insert options do not change the mode of an existing table. A conflicting explicit mode is rejected; use ``transform --autoincrement`` or ``transform --no-autoincrement`` to change it. ``create-table --transform`` also applies an explicit mode change.
You can specify columns that should be NOT NULL using ``--not-null colname``. You can specify default values for columns using ``--default colname defaultvalue``.
.. code-block:: bash
@ -2271,6 +2287,9 @@ Every option for this table (with the exception of ``--pk-none``) can be specifi
``--add-foreign-key column other_table other_column``
Add a foreign key constraint to ``column`` pointing to ``other_table.other_column``.
``--autoincrement`` / ``--no-autoincrement``
Enable or disable ``AUTOINCREMENT`` for a single integer primary key. If neither option is supplied the current mode is preserved. Enabling it tracks the current IDs, but cannot recover IDs deleted before it was enabled. ``--sql`` previews the same change without applying it.
``--strict``
Convert the table to a `SQLite STRICT table <https://www.sqlite.org/stricttables.html>`__. The command fails if the available SQLite version does not support strict tables. If existing rows contain values that are incompatible with their declared column types the transformation fails and the original table is left unchanged.

27
docs/python-api.rst
View file

@ -844,6 +844,33 @@ An ``ANY`` column can store integers, floating point values, text, binary data o
.. note::
In the CLI: :ref:`sqlite-utils create-table <cli_create_table>`
.. _python_api_autoincrement:
Preventing reuse of deleted IDs
-------------------------------
An ``INTEGER PRIMARY KEY`` already generates an ID when an inserted record omits that column. Pass ``autoincrement=True`` to prevent SQLite from reusing IDs of deleted rows:
.. code-block:: python
table = db["events"].create({"id": int, "name": str}, pk="id", autoincrement=True)
table.insert({"name": "first event"})
This option also works with ``db.create_table()``, ``db.create_table_sql()``, ``insert()``, ``insert_all()``, ``upsert()``, ``upsert_all()`` and ``lookup()``. To create the table on the first insert:
.. code-block:: python
db["events"].insert({"name": "first event"}, pk="id", autoincrement=True)
You can configure it as a table default using ``db.table("events", pk="id", autoincrement=True)``. The primary key must be a single ``INTEGER`` column; compound, text and hash primary keys cannot use this option. A primary key omitted from the column definitions is created as an integer column. ``upsert()`` still requires primary key values in every record.
The default ``None`` keeps the usual ID allocation when creating a new table. An explicit ``False`` creates a table without ``AUTOINCREMENT``. An explicit option that conflicts with an existing table is rejected by inserts and lookups; use ``table.transform(autoincrement=True)`` or ``table.transform(autoincrement=False)`` to change the mode. ``create(..., transform=True)`` also applies an explicit mode change. Ordinary transformations preserve the existing mode and its sequence.
Enabling the mode on an existing table starts tracking its current IDs. It cannot recover the history of IDs deleted before it was enabled. IDs need not be consecutive. SQLite documents the additional cost of this mode in `its AUTOINCREMENT documentation <https://www.sqlite.org/autoinc.html>`__.
.. note::
In the CLI: :ref:`sqlite-utils create-table <cli_create_table>` and :ref:`sqlite-utils transform <cli_transform_table>`.
.. _python_api_compound_primary_keys:
Compound primary keys