Commit graph

75 commits

Author SHA1 Message Date
Chris (ChrisJr404)
81e2ad8a44
tables and views commands accept optional table/view names
The tables and views commands now take one or more optional table (or view)
names as positional arguments after the database path, restricting the output
to just those tables. This is handy with --counts against a database that has
a large table you would rather skip.

Names are listed in the order they are passed. If any named table or view does
not exist the command raises an error and exits with a non-zero status, so the
exit code can be used to confirm a set of expected tables is present.

Closes #478
2026-08-18 06:22:20 -04:00
Simon Willison
fcfccea813 Support ANY column types for strict tables
Closes #790, #820
2026-08-12 16:44:01 -07:00
Simon Willison
69a1c0d960
Fixes for Ruff>=0.16.0 (#814)
* Automated upgrades by Ruff

    uvx --with 'ruff>=0.16.0' ruff check . --fix --unsafe-fixes

* Fix remaining Ruff errors with GPT-5.6 Sol high

https://gist.github.com/simonw/6da7906a9fea6e90da131c21a9055199

* Fix flake E501 long lines
* New Protocol for migrations to make ty happy
2026-07-25 14:53:12 -07:00
Simon Willison
b74b727035
.transform(strict=) and sqlite-utils transform --strict/--no-strict (#788)
* .transform(strict=) and sqlite-utils transform --strict/--no-strict

Closes #787
2026-07-11 16:43:37 -07:00
Simon Willison
7a52214624 drop-index command and table.drop_index(index_name)
Closes #626
2026-07-07 21:00:43 -07:00
Simon Willison
d2ac3765ed sqlite-utils insert/upsert --type colunm-name type option, closes #131 2026-07-07 19:17:29 -07:00
Simon Willison
569608e40f sqlite-utils insert --code option, closes #684 2026-07-07 18:49:25 -07:00
Simon Willison
ebafb84c93 Allow sqlite-utils upsert to infer --pk from existing table 2026-07-07 18:09:56 -07:00
Simon Willison
aa300942bf Update sqlite-utils convert --help, refs #686 2026-07-07 18:03:25 -07:00
Simon Willison
cf3373e7b7 Refactor --functions option, improve help 2026-07-07 17:59:58 -07:00
Simon Willison
8ee0b7c65c Fix for rogue quote in enable-fts --help 2026-07-07 17:57:35 -07:00
Simon Willison
fa5d66bf53 Update create-table --help to mention real 2026-07-07 17:56:57 -07:00
Simon Willison
619770bf42 sqlite-utils query - to read SQL from stdin, closes #765 2026-07-07 17:52:26 -07:00
Simon Willison
815b6a7d3d
JSON output no longer escapes non-ASCII characters, new --ascii option (#777)
Closes #625

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JaHan1NhaTRAxJ9LQtSLf9
2026-07-06 11:10:07 -07:00
Johnson K C
d516e58543
Honor --no-headers for --fmt and --table output (#566) (#751)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 09:54:17 -07:00
Claude
f1cdceaca9
Remove the no-op -d/--detect-types flag from insert and upsert
Type detection has been the default for CSV/TSV data since 4.0a1,
making this flag a no-op kept only for backwards compatibility.
4.0 is the release where it can be removed - the flag now errors,
prompting scripts to drop it. --no-detect-types is unchanged. The
dead detect_types parameter has been removed from
insert_upsert_implementation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
2026-07-04 19:15:31 +00:00
Simon Willison
e71b4ca3ee Ran cog 2026-06-24 13:11:29 -07:00
Simon Willison
3cc27d69bc
New migrations system, ported from sqlite-migrate (#754)
Closes #752
2026-06-21 09:40:21 -07:00
Simon Willison
8f0c06e188
Test against Python 3.15-dev, bump ty and Black (#738)
* Add Python 3.15-dev to test matrix
* Run ty check only on 3.14
* Bump Black version
* Update tabulate and use that in 
* Bump to latest ty
2026-05-17 16:52:48 -07:00
Simon Willison
8d74ffc932
More type annotations (#697)
* Add comprehensive type annotations

- mypy.ini: expanded configuration with module-specific settings
- hookspecs.py: type annotations for hook functions
- plugins.py: typed get_plugins() return value
- recipes.py: full type annotations for parsedate, parsedatetime, jsonsplit
- utils.py: extensive type annotations including Row type alias,
  TypeTracker, ValueTracker, and all utility functions
- db.py: type annotations for Database methods (__exit__,
  ensure_autocommit_off, tracer, register_function, etc.) and
  Queryable class methods
- tests/test_docs.py: updated to match new signature display format

* Fix type errors caught by ty check

- Add type: ignore comments for external library type stub limitations
  (csv.reader, click.progressbar, IOBase.name, Callable.__name__)
- Change Iterable to Sequence for SQL where_args parameters
- Use db.table() instead of db[name] for proper Table return type
- Fix rebuild_fts return type from None to Table
- Update test_tracer to expect fewer queries (optimization side effect)

* Fix mypy type errors

- Add type: ignore comments for runtime-valid patterns mypy can't verify
- Fix new_column_types annotation to Dict[str, Set[type]]
- Add type: ignore for Default sentinel values passed to create_table

* mypy skip tests directory

* Fix CI: exclude typing imports from recipe docs, skip mypy on tests

- Add Callable and Optional to exclusion list in _generate_convert_help()
- Regenerate docs/cli-reference.rst with cog
- Add [mypy-tests.*] ignore_errors = True to skip test type errors

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 22:11:47 -08:00
Simon Willison
35377a874b
Detect CSV/TSV column types by default (#683)
The `--detect-types` option is now automatically turned on for all commands that deal with CSV or CSV.

A new `--no-detect-types` option can be used to have all columns treated as text.

Closes #679
2025-11-23 22:23:15 -08:00
Simon Willison
0bbc68089c
--functions can take filenames, can be used multiple times (#681)
Closes #659

The --functions option now accepts:
- File paths ending in .py (e.g., --functions my_funcs.py)
- Multiple invocations (e.g., --functions foo.py --functions 'def bar(): ...')
- Inline Python code (existing behavior)

Implementation follows the same pattern as llm's --functions flag
(simonw/llm@a880c123).

Changes:
- Added multiple=True to --functions Click option in query, bulk, and memory commands
- Modified _register_functions() to detect and read .py files
- Updated _maybe_register_functions() to iterate over multiple function sources
- Removed unused bytes/bytearray handling
- Added comprehensive tests for file paths and multiple invocations
- Updated documentation with examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Shorter help for --functions

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-23 21:46:51 -08:00
Simon Willison
c872d27bb3
Use REAL not FLOAT as SQLite column type (#680)
* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
2025-11-23 21:37:59 -08:00
Simon Willison
96fab69256 Removed convert skip_false and --skip-false, closes #542 2025-11-23 15:40:28 -08:00
Simon Willison
370318c695 Lowercase for geometry types
Cog test failed in CI, but these are case insensitive anyway.
2025-10-01 13:52:10 -07:00
Simon Willison
72f6c820f6 Small documentation updates to fix warnings 2025-05-07 17:54:29 -07:00
Simon Willison
0e4e270d44
Test against multiple SQLite versions (#654)
* Test against pre-upsert SQLite 3.23.1

Borrowed from 8f86d2af6

* Try this on Python 3.9

* select ... from pragma_function_list()

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860898278

* Fix spelling error

* Compatible with latest black

* Skip plugin test that needs pragma_function_list

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860924225

* Ran cog
2025-05-07 17:49:50 -07:00
Simon Willison
5bd7aec4d2
Test against Python 3.13 pre-release (#619)
* Test against Python 3.13 pre-release

* Skip tests for numpy on Python 3.13

Refs https://github.com/simonw/sqlite-utils/pull/619#issuecomment-1998798451

* Try to avoid Python 3.13 cog differences

* Hide \b characters in cli-reference

* Fixed .rST warning
2024-03-14 21:14:10 -07:00
Taj Khattra
1500c19bd0
Add more STRICT table support (#604)
* Add more STRICT table support per https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776.
* Make `table.transform()` preserve STRICT mode.
* Fix mypy failures in PR #604
* Link to SQLITE strict page in a few places
2023-12-07 21:05:27 -08:00
Simon Willison
88bd372205 str, int, bytes aliases for column types, closes #606 2023-12-06 10:49:21 -08:00
Simon Willison
56093de078 sqlite-utils transform --add-foreign-key option, closes #585 2023-08-17 18:51:04 -07:00
Simon Willison
70717dc0e1 Remove unneccessary warning, refs #577, #585 2023-08-17 18:28:09 -07:00
Simon Willison
619cea8681 sqlite-utils convert --pdb option, closes #581 2023-07-26 14:06:05 -07:00
Simon Willison
5e9a02153d Fix bug with --editable
See also: https://github.com/simonw/llm/issues/136
2023-07-25 22:20:49 -07:00
Simon Willison
18f190e283 sqlite-utils rename-table command, refs #565 2023-07-22 12:48:04 -07:00
Simon Willison
b379a2a0c3 Added pluggy and first hook, register_commands - refs #569, #567 2023-07-22 12:04:31 -07:00
Simon Willison
7e48502b5a Update CLI reference docs for -e, refs #570 2023-07-22 11:37:53 -07:00
Simon Willison
f7af23837d --empty-null option for CSV and TSV imports, closes #563 2023-07-02 22:42:26 -07:00
Simon Willison
8c739558f7 --stop-after option, closes #561 2023-06-27 11:50:04 -07:00
Simon Willison
718b0cba9b
Experimental TUI powered by Trogon
* sqlite-utils tui command if Trogon is installed, closes #545
* Documentation for trogon TUI
* Screenshot of TUI
* Ignore trogon mypy error
* only run flake8 on Python 3.8 or higher, closes #550
2023-05-21 11:41:56 -07:00
Simon Willison
d2a7b15b2b
Analyze tables options: --common-limit, --no-most, --no-least
Closes #544
2023-05-21 09:19:30 -07:00
Simon Willison
9662d4ce26 Updated cog, refs #527 2023-05-08 14:05:30 -07:00
Simon Willison
373b7886d2 --raw-lines option, closes #539 2023-05-07 11:26:14 -07:00
Simon Willison
ebe504ab21 Clarify column types in create-table help 2022-11-29 09:03:35 -08:00
Simon Willison
eb67fc69a2 Run cog -r against latest tabulate, refs #501 2022-10-18 11:08:34 -07:00
Simon Willison
85247038f7 install and uninstall commands, closes #483 2022-09-26 11:57:11 -07:00
Simon Willison
104f37fa4d
db[table].create(..., transform=True) and create-table --transform
Closes #467
2022-08-27 16:17:55 -07:00
Simon Willison
19dd077944
Support entrypoints for --load-extension (#473)
* Entrypoint support, closes #470
2022-08-26 22:55:47 -07:00
Simon Willison
23ef1d6c20 bulk --functions, closes #471 2022-08-26 22:10:43 -07:00
Simon Willison
59e2cfbdc1 sqlite-utils memory --functions, refs #471 2022-08-26 22:03:53 -07:00