insert --convert: validate the result instead of falling back on falsy

The JSON/CSV branch used 'fn(doc) or doc', so any falsy result was
discarded and the unconverted row inserted. Exit code 0, no warning:

    $ echo '[{"id": 1, "name": "Cleo"}]' | sqlite-utils insert d.db t - --convert '{}'
    $ sqlite-utils 'select * from d.db'      # -> the original row, not {}

The realistic shape of this is a filter:

    --convert 'row if row["ok"] else {}'    # meant to drop the bad rows
    # every row inserted, including the ones meant to be dropped

'--lines' runs a different expression twelve lines up and has no such
fallback, so the same --convert code is rejected there and silently
accepted here:

    --convert 0   JSON input  -> exit 0, rows inserted
    --convert 0   --lines     -> exit 1, Rows must all be dictionaries, got: 0

test_insert_convert_error_messages already asserts that a non-dict
return is rejected (--convert 1); 0, False, [] and "" were the outliers,
purely because they are falsy.

None is the one falsy result that has to keep meaning 'modify doc in
place' - a --convert that is a bare statement returns it, which
test_insert_convert_row_modifying_in_place covers - so the fallback is
now keyed on 'is None' rather than truthiness.

Tests: 7 new cases in tests/test_cli_insert.py, all failing on 6bc1d33.
Suite 1505 passed, 16 skipped. black, flake8, mypy clean. pyright
reports one error and ty two diagnostics, both identical on unmodified
main (tests/test_constructor.py:74 and two unused-type-ignore comments
in db.py/utils.py) and none in the files this touches.
This commit is contained in:
feiiiiii5 2026-09-29 00:24:06 +08:00
commit 210c119907
No known key found for this signature in database
2 changed files with 80 additions and 1 deletions

View file

@ -1312,7 +1312,19 @@ def insert_upsert_implementation(
"--convert must return dict or iterator"
)
else:
docs = (fn(doc) or doc for doc in docs)
def convert_doc(doc, fn=fn):
# A --convert expression that is a statement returns
# None, and means "modify doc in place", so None is
# the one falsy result that falls back to the
# original row. Any other falsy result is a value
# the caller returned deliberately and has to be
# validated like --lines already does, rather than
# silently inserting the row unconverted.
result = fn(doc)
return doc if result is None else result
docs = (convert_doc(doc) for doc in docs)
_insert_docs(docs, tracker=tracker)