mirror of
https://github.com/simonw/sqlite-utils.git
synced 2026-10-23 17:57:03 +02:00
insert --convert: validate the result instead of falling back on falsy
The JSON/CSV branch used 'fn(doc) or doc', so any falsy result was
discarded and the unconverted row inserted. Exit code 0, no warning:
$ echo '[{"id": 1, "name": "Cleo"}]' | sqlite-utils insert d.db t - --convert '{}'
$ sqlite-utils 'select * from d.db' # -> the original row, not {}
The realistic shape of this is a filter:
--convert 'row if row["ok"] else {}' # meant to drop the bad rows
# every row inserted, including the ones meant to be dropped
'--lines' runs a different expression twelve lines up and has no such
fallback, so the same --convert code is rejected there and silently
accepted here:
--convert 0 JSON input -> exit 0, rows inserted
--convert 0 --lines -> exit 1, Rows must all be dictionaries, got: 0
test_insert_convert_error_messages already asserts that a non-dict
return is rejected (--convert 1); 0, False, [] and "" were the outliers,
purely because they are falsy.
None is the one falsy result that has to keep meaning 'modify doc in
place' - a --convert that is a bare statement returns it, which
test_insert_convert_row_modifying_in_place covers - so the fallback is
now keyed on 'is None' rather than truthiness.
Tests: 7 new cases in tests/test_cli_insert.py, all failing on 6bc1d33.
Suite 1505 passed, 16 skipped. black, flake8, mypy clean. pyright
reports one error and ty two diagnostics, both identical on unmodified
main (tests/test_constructor.py:74 and two unused-type-ignore comments
in db.py/utils.py) and none in the files this touches.
This commit is contained in:
parent
6bc1d33d58
commit
210c119907
2 changed files with 80 additions and 1 deletions
|
|
@ -1312,7 +1312,19 @@ def insert_upsert_implementation(
|
|||
"--convert must return dict or iterator"
|
||||
)
|
||||
else:
|
||||
docs = (fn(doc) or doc for doc in docs)
|
||||
|
||||
def convert_doc(doc, fn=fn):
|
||||
# A --convert expression that is a statement returns
|
||||
# None, and means "modify doc in place", so None is
|
||||
# the one falsy result that falls back to the
|
||||
# original row. Any other falsy result is a value
|
||||
# the caller returned deliberately and has to be
|
||||
# validated like --lines already does, rather than
|
||||
# silently inserting the row unconverted.
|
||||
result = fn(doc)
|
||||
return doc if result is None else result
|
||||
|
||||
docs = (convert_doc(doc) for doc in docs)
|
||||
|
||||
_insert_docs(docs, tracker=tracker)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue