Commit graph

68 commits

Author SHA1 Message Date
Simon Willison
ebafb84c93 Allow sqlite-utils upsert to infer --pk from existing table 2026-07-07 18:09:56 -07:00
Simon Willison
aa300942bf Update sqlite-utils convert --help, refs #686 2026-07-07 18:03:25 -07:00
Simon Willison
cf3373e7b7 Refactor --functions option, improve help 2026-07-07 17:59:58 -07:00
Simon Willison
8ee0b7c65c Fix for rogue quote in enable-fts --help 2026-07-07 17:57:35 -07:00
Simon Willison
fa5d66bf53 Update create-table --help to mention real 2026-07-07 17:56:57 -07:00
Simon Willison
619770bf42 sqlite-utils query - to read SQL from stdin, closes #765 2026-07-07 17:52:26 -07:00
Simon Willison
815b6a7d3d
JSON output no longer escapes non-ASCII characters, new --ascii option (#777)
Closes #625

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JaHan1NhaTRAxJ9LQtSLf9
2026-07-06 11:10:07 -07:00
Johnson K C
d516e58543
Honor --no-headers for --fmt and --table output (#566) (#751)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 09:54:17 -07:00
Claude
f1cdceaca9
Remove the no-op -d/--detect-types flag from insert and upsert
Type detection has been the default for CSV/TSV data since 4.0a1,
making this flag a no-op kept only for backwards compatibility.
4.0 is the release where it can be removed - the flag now errors,
prompting scripts to drop it. --no-detect-types is unchanged. The
dead detect_types parameter has been removed from
insert_upsert_implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
2026-07-04 19:15:31 +00:00
Simon Willison
e71b4ca3ee Ran cog 2026-06-24 13:11:29 -07:00
Simon Willison
3cc27d69bc
New migrations system, ported from sqlite-migrate (#754)
Closes #752
2026-06-21 09:40:21 -07:00
Simon Willison
8f0c06e188
Test against Python 3.15-dev, bump ty and Black (#738)
* Add Python 3.15-dev to test matrix
* Run ty check only on 3.14
* Bump Black version
* Update tabulate and use that in 
* Bump to latest ty
2026-05-17 16:52:48 -07:00
Simon Willison
8d74ffc932
More type annotations (#697)
* Add comprehensive type annotations

- mypy.ini: expanded configuration with module-specific settings
- hookspecs.py: type annotations for hook functions
- plugins.py: typed get_plugins() return value
- recipes.py: full type annotations for parsedate, parsedatetime, jsonsplit
- utils.py: extensive type annotations including Row type alias,
  TypeTracker, ValueTracker, and all utility functions
- db.py: type annotations for Database methods (__exit__,
  ensure_autocommit_off, tracer, register_function, etc.) and
  Queryable class methods
- tests/test_docs.py: updated to match new signature display format

* Fix type errors caught by ty check

- Add type: ignore comments for external library type stub limitations
  (csv.reader, click.progressbar, IOBase.name, Callable.__name__)
- Change Iterable to Sequence for SQL where_args parameters
- Use db.table() instead of db[name] for proper Table return type
- Fix rebuild_fts return type from None to Table
- Update test_tracer to expect fewer queries (optimization side effect)

* Fix mypy type errors

- Add type: ignore comments for runtime-valid patterns mypy can't verify
- Fix new_column_types annotation to Dict[str, Set[type]]
- Add type: ignore for Default sentinel values passed to create_table

* mypy skip tests directory

* Fix CI: exclude typing imports from recipe docs, skip mypy on tests

- Add Callable and Optional to exclusion list in _generate_convert_help()
- Regenerate docs/cli-reference.rst with cog
- Add [mypy-tests.*] ignore_errors = True to skip test type errors

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 22:11:47 -08:00
Simon Willison
35377a874b
Detect CSV/TSV column types by default (#683)
The `--detect-types` option is now automatically turned on for all commands that deal with CSV or CSV.

A new `--no-detect-types` option can be used to have all columns treated as text.

Closes #679
2025-11-23 22:23:15 -08:00
Simon Willison
0bbc68089c
--functions can take filenames, can be used multiple times (#681)
Closes #659

The --functions option now accepts:
- File paths ending in .py (e.g., --functions my_funcs.py)
- Multiple invocations (e.g., --functions foo.py --functions 'def bar(): ...')
- Inline Python code (existing behavior)

Implementation follows the same pattern as llm's --functions flag
(simonw/llm@a880c123).

Changes:
- Added multiple=True to --functions Click option in query, bulk, and memory commands
- Modified _register_functions() to detect and read .py files
- Updated _maybe_register_functions() to iterate over multiple function sources
- Removed unused bytes/bytearray handling
- Added comprehensive tests for file paths and multiple invocations
- Updated documentation with examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Shorter help for --functions

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-23 21:46:51 -08:00
Simon Willison
c872d27bb3
Use REAL not FLOAT as SQLite column type (#680)
* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
2025-11-23 21:37:59 -08:00
Simon Willison
96fab69256 Removed convert skip_false and --skip-false, closes #542 2025-11-23 15:40:28 -08:00
Simon Willison
370318c695 Lowercase for geometry types
Cog test failed in CI, but these are case insensitive anyway.
2025-10-01 13:52:10 -07:00
Simon Willison
72f6c820f6 Small documentation updates to fix warnings 2025-05-07 17:54:29 -07:00
Simon Willison
0e4e270d44
Test against multiple SQLite versions (#654)
* Test against pre-upsert SQLite 3.23.1

Borrowed from 8f86d2af6

* Try this on Python 3.9

* select ... from pragma_function_list()

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860898278

* Fix spelling error

* Compatible with latest black

* Skip plugin test that needs pragma_function_list

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860924225

* Ran cog
2025-05-07 17:49:50 -07:00
Simon Willison
5bd7aec4d2
Test against Python 3.13 pre-release (#619)
* Test against Python 3.13 pre-release

* Skip tests for numpy on Python 3.13

Refs https://github.com/simonw/sqlite-utils/pull/619#issuecomment-1998798451

* Try to avoid Python 3.13 cog differences

* Hide \b characters in cli-reference

* Fixed .rST warning
2024-03-14 21:14:10 -07:00
Taj Khattra
1500c19bd0
Add more STRICT table support (#604)
* Add more STRICT table support per https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776.
* Make `table.transform()` preserve STRICT mode.
* Fix mypy failures in PR #604
* Link to SQLITE strict page in a few places
2023-12-07 21:05:27 -08:00
Simon Willison
88bd372205 str, int, bytes aliases for column types, closes #606 2023-12-06 10:49:21 -08:00
Simon Willison
56093de078 sqlite-utils transform --add-foreign-key option, closes #585 2023-08-17 18:51:04 -07:00
Simon Willison
70717dc0e1 Remove unneccessary warning, refs #577, #585 2023-08-17 18:28:09 -07:00
Simon Willison
619cea8681 sqlite-utils convert --pdb option, closes #581 2023-07-26 14:06:05 -07:00
Simon Willison
5e9a02153d Fix bug with --editable
See also: https://github.com/simonw/llm/issues/136
2023-07-25 22:20:49 -07:00
Simon Willison
18f190e283 sqlite-utils rename-table command, refs #565 2023-07-22 12:48:04 -07:00
Simon Willison
b379a2a0c3 Added pluggy and first hook, register_commands - refs #569, #567 2023-07-22 12:04:31 -07:00
Simon Willison
7e48502b5a Update CLI reference docs for -e, refs #570 2023-07-22 11:37:53 -07:00
Simon Willison
f7af23837d --empty-null option for CSV and TSV imports, closes #563 2023-07-02 22:42:26 -07:00
Simon Willison
8c739558f7 --stop-after option, closes #561 2023-06-27 11:50:04 -07:00
Simon Willison
718b0cba9b
Experimental TUI powered by Trogon
* sqlite-utils tui command if Trogon is installed, closes #545
* Documentation for trogon TUI
* Screenshot of TUI
* Ignore trogon mypy error
* only run flake8 on Python 3.8 or higher, closes #550
2023-05-21 11:41:56 -07:00
Simon Willison
d2a7b15b2b
Analyze tables options: --common-limit, --no-most, --no-least
Closes #544
2023-05-21 09:19:30 -07:00
Simon Willison
9662d4ce26 Updated cog, refs #527 2023-05-08 14:05:30 -07:00
Simon Willison
373b7886d2 --raw-lines option, closes #539 2023-05-07 11:26:14 -07:00
Simon Willison
ebe504ab21 Clarify column types in create-table help 2022-11-29 09:03:35 -08:00
Simon Willison
eb67fc69a2 Run cog -r against latest tabulate, refs #501 2022-10-18 11:08:34 -07:00
Simon Willison
85247038f7 install and uninstall commands, closes #483 2022-09-26 11:57:11 -07:00
Simon Willison
104f37fa4d
db[table].create(..., transform=True) and create-table --transform
Closes #467
2022-08-27 16:17:55 -07:00
Simon Willison
19dd077944
Support entrypoints for --load-extension (#473)
* Entrypoint support, closes #470
2022-08-26 22:55:47 -07:00
Simon Willison
23ef1d6c20 bulk --functions, closes #471 2022-08-26 22:10:43 -07:00
Simon Willison
59e2cfbdc1 sqlite-utils memory --functions, refs #471 2022-08-26 22:03:53 -07:00
Simon Willison
31f062d4a7 sqlite-utils query --functions option, refs #471 2022-08-26 21:53:55 -07:00
Simon Willison
7a9a6363ff sqlite-utils rows --order option, closes #469 2022-08-26 21:10:20 -07:00
Simon Willison
f4fb78fa95
Cross-link CLI to Python docs (#460)
* Start cross-linking CLI to Python docs, refs #426
* More links to Python from CLI page, refs #426
2022-08-18 14:58:07 -07:00
Simon Willison
b9a89a0f2c duplicate --ignore option, refs #450 2022-07-15 15:35:58 -07:00
Simon Willison
5fa823f03f add-column --ignore option, refs #450 2022-07-15 15:31:37 -07:00
Simon Willison
2c77f4467e Add --ignore to create-index as alias of --if-not-exists, refs #450 2022-07-15 15:25:49 -07:00
Simon Willison
40b6947255 enable-fts --replace option, refs #450
Also fixed up some sqlite3.OperationalError imports.
2022-07-15 15:20:26 -07:00