Add --extract to CLI insert and upsert

docs/cli-reference.rst

@@ -315,6 +315,7 @@ See :ref:`cli_inserting_data`, :ref:`cli_insert_csv_tsv`, :ref:`cli_insert_unstr
       --analyze                 Run ANALYZE at the end of this operation
       --load-extension TEXT     Path to SQLite extension, with optional :entrypoint
       --silent                  Do not show progress bar
+      --extract TEXT            Columns to extract to same-name lookup tables
       --strict                  Apply STRICT mode to created table
       --ignore                  Ignore records if pk already exists
       --replace                 Replace records if pk already exists
@@ -379,6 +380,7 @@ See :ref:`cli_upsert`.
       --analyze                 Run ANALYZE at the end of this operation
       --load-extension TEXT     Path to SQLite extension, with optional :entrypoint
       --silent                  Do not show progress bar
+      --extract TEXT            Columns to extract to same-name lookup tables
       --strict                  Apply STRICT mode to created table
       -h, --help                Show this message and exit.
 

docs/cli.rst

@@ -1312,6 +1312,33 @@ With the ``--flatten`` option columns will be created using ``topkey_nextkey`` c
        "labels_service" TEXT
     );
 
+.. _cli_insert_extract:
+
+Extracting values during an import
+==================================
+
+Use ``--extract COLUMN`` with ``insert`` or ``upsert`` to move repeated values into a lookup table as records are imported:
+
+.. code-block:: bash
+
+    echo '[{"id": 1, "species": "oak"}, {"id": 2, "species": "oak"}]' \
+      | sqlite-utils insert trees.db trees - --pk id --extract species
+
+This creates a ``species`` table with ``id`` and ``value`` columns and a unique index on ``value``. Both input records reference the same lookup row. In a newly created ``trees`` table, the ``species`` column is an integer foreign key referencing ``species.id``. Extracted ``null`` values stay ``null`` and do not create lookup rows.
+
+Repeat the option to extract multiple columns into separate lookup tables:
+
+.. code-block:: bash
+
+    sqlite-utils insert trees.db trees trees.csv --csv \
+      --extract species --extract country
+
+The option also works with TSV, newline-delimited JSON, converted rows and ``--code``. CSV/TSV type detection still applies to other columns; extracted columns in a newly created table keep their integer foreign key type. Subsequent imports and upserts reuse existing lookup rows.
+
+This uses the Python API's :ref:`extracts= mechanism <python_api_extracts>` and keeps its behavior for existing tables: it does not change an existing column's type or add a foreign key constraint to that column. Start with a new table to create the foreign key schema, or use :ref:`sqlite-utils extract <cli_extract>` to normalize values already stored in a table.
+
+Without ``--extract``, imported values are stored directly as before.
+
 .. _cli_insert_csv_tsv:
 
 Inserting CSV or TSV data