Commit graph

25 commits

Author SHA1 Message Date
Simon Willison
548a886ca1 Clean CLI errors for InvalidColumns from insert --pk and extract
sqlite-utils insert db t - --pk badcol and sqlite-utils extract db t
nosuchcol dumped raw InvalidColumns tracebacks - the insert error
handling caught NoTable and OperationalError but not the InvalidColumns
introduced for #732, and the extract command had no handling at all
(including for NoTable when pointed at a view). Both now exit with
click-style Error: messages.

Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4900034150

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 21:57:14 -07:00
Simon Willison
884574685f insert/upsert --csv no longer rewrites column types of existing tables
Type detection is the 4.0 default for CSV/TSV data, and the detected-type
transform ran even when the target table already existed - inserting a
CSV into a table with a TEXT zip column converted the column to INTEGER,
corrupting values with leading zeros ('01234' became 1234) with no
warning. Detected types now only apply to tables the command created.

Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4900034150

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 21:44:56 -07:00
Claude
397cdcc491
Polish: clearer errors, corrected docstrings, setuptools pin
- insert/upsert into a name that is actually a view now shows a
  clean error instead of a traceback
- db.view() on a name that is a table now says so in its error
- Database.__getitem__ docstring documents that views are returned
  for view names; hash_id docstring corrected (it is a column name,
  not a boolean)
- Registering two migrations with the same name in one set now
  raises ValueError at registration time instead of executing both
  and failing with IntegrityError at apply time
- build-system requires setuptools>=77, needed for the PEP 639
  license expression

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
2026-07-04 19:13:15 +00:00
Rami Abdelrazzaq
2b0cc04c8d
Fix issue #702: Handle CSV with only header row in --detect-types (#707)
When inserting a CSV file that contains only a header row (no data rows),
the --detect-types option would crash with an AssertionError because it
tried to transform a table that didn't exist.

This fix adds a check to ensure the table exists before attempting to
apply type transformations. The fix is applied in two places:
1. insert_upsert_implementation() for the insert command
2. memory() command for CSV/TSV files

Added test case: test_insert_csv_headers_only

Co-authored-by: Test User <test@example.com>
2026-06-21 15:57:06 -07:00
Simon Willison
35377a874b
Detect CSV/TSV column types by default (#683)
The `--detect-types` option is now automatically turned on for all commands that deal with CSV or CSV.

A new `--no-detect-types` option can be used to have all columns treated as text.

Closes #679
2025-11-23 22:23:15 -08:00
Simon Willison
fb93452ea8
Use double quotes not braces for tables and columns (#678)
Closes #677
2025-11-23 20:43:26 -08:00
Simon Willison
da92a30679 Test for insert CLI with multiple --pk, refs #621 2024-03-16 17:26:02 -07:00
Simon Willison
d2bcdc00c6 Swapped the order of a bunch of pytest comparisons
When I wrote this I thought constant == value was a better assertion. I no longer think that.
2023-08-17 18:05:13 -07:00
Simon Willison
f7af23837d --empty-null option for CSV and TSV imports, closes #563 2023-07-02 22:42:26 -07:00
Simon Willison
8c739558f7 --stop-after option, closes #561 2023-06-27 11:50:04 -07:00
Simon Willison
e4ed372517 Show more detailed error on invalid JSON, closes #532 2023-05-08 13:31:56 -07:00
Simon Willison
a256d7de98 Fix a bunch of warnings in the tests, refs #541 2023-05-08 12:57:43 -07:00
Simon Willison
9dcb099905 Better error messages for --convert, closes #363 2022-02-03 14:51:25 -08:00
Simon Willison
1b84c175b4 --analyze option for create-index, insert, update commands, closes #379, closes #365 2022-01-10 17:36:41 -08:00
Simon Willison
b6dad08a83 Keep trying up to ten times, refs #364 2022-01-09 21:04:51 -08:00
Simon Willison
046e5246c9 Longer delay to hopefully get test to pass, refs #364 2022-01-09 20:51:07 -08:00
Simon Willison
e6ae643497 Did not need WAL after all, refs #364 2022-01-09 20:41:00 -08:00
Simon Willison
cfb3f12358 Only buffer input if --sniff, closes #364 2022-01-09 20:39:58 -08:00
Simon Willison
d2a79d200f --nl now ignores blank lines, closes #376 2022-01-09 20:12:39 -08:00
Simon Willison
22c8d10dd3 --convert function can now modify row in place, closes #371 2022-01-09 12:06:02 -08:00
Simon Willison
413f8ed754 --convert --text for iterators, docs for --convert 2022-01-05 22:24:53 -08:00
Simon Willison
2e4847e493 Implemented --convert for different things, renamed --all to --text 2022-01-05 22:24:53 -08:00
Simon Willison
e66299c6ed Implemented and documented sqlite-utils insert --all 2022-01-05 22:24:53 -08:00
Simon Willison
f1569c9f7f Implemented sqlite-utils insert --lines 2022-01-05 22:24:53 -08:00
Simon Willison
d1ed2f423d Refactored sqlite-utils insert tests into test_cli_insert.py 2022-01-05 22:24:53 -08:00