Commit graph

1,031 commits

Author SHA1 Message Date
Simon Willison
5a7f522980
sqlite-utils convert can now use callable references, closes #686
This now works:

    echo '{"date": "13th January 2025"}' | sqlite-utils insert dates.db dates -
    sqlite-utils convert dates.db dates date r.parsedate
    sqlite-utils rows dates.db dates

Previously the command would have been:

    sqlite-utils convert dates.db dates date 'r.parsedate(value)'
2025-11-29 14:43:02 -08:00
Simon Willison
f9e5fbb3bd Backport release notes for 3.39, closes #688 2025-11-24 10:46:34 -08:00
Copilot
e66e82f2c8
Remove Sentinel workaround for Click 8.3.1+ (#685)
Closes #666

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2025-11-24 09:11:18 -08:00
Simon Willison
a0d8bc3932 Add pip as a dependency to ensure plugin installation, closes #687 2025-11-24 07:58:54 -08:00
Simon Willison
ae58faf274 Add 3.14 to the publish.yml workflow 2025-11-23 22:34:59 -08:00
Simon Willison
8c40732e4f Changelog markup fixes, refs #682 4.0a1 2025-11-23 22:32:34 -08:00
Simon Willison
9ea950e3ba Removed duplicate changelog entry, refs #682 2025-11-23 22:30:15 -08:00
Simon Willison
69ee540454 Release 4.0a1
Refs #682, #542, #645, #655, #657, #659, #672, #675, #677, #679
2025-11-23 22:27:18 -08:00
Simon Willison
6dad400e14 Release notes for 4.0a1
Refs #682, #542, #645, #655, #657, #659, #672, #675, #677, #679
2025-11-23 22:26:32 -08:00
Simon Willison
35377a874b
Detect CSV/TSV column types by default (#683)
The `--detect-types` option is now automatically turned on for all commands that deal with CSV or CSV.

A new `--no-detect-types` option can be used to have all columns treated as text.

Closes #679
2025-11-23 22:23:15 -08:00
Simon Willison
0bbc68089c
--functions can take filenames, can be used multiple times (#681)
Closes #659

The --functions option now accepts:
- File paths ending in .py (e.g., --functions my_funcs.py)
- Multiple invocations (e.g., --functions foo.py --functions 'def bar(): ...')
- Inline Python code (existing behavior)

Implementation follows the same pattern as llm's --functions flag
(simonw/llm@a880c123).

Changes:
- Added multiple=True to --functions Click option in query, bulk, and memory commands
- Modified _register_functions() to detect and read .py files
- Updated _maybe_register_functions() to iterate over multiple function sources
- Removed unused bytes/bytearray handling
- Added comprehensive tests for file paths and multiple invocations
- Updated documentation with examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Shorter help for --functions

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-23 21:46:51 -08:00
Simon Willison
c872d27bb3
Use REAL not FLOAT as SQLite column type (#680)
* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
2025-11-23 21:37:59 -08:00
Simon Willison
fb93452ea8
Use double quotes not braces for tables and columns (#678)
Closes #677
2025-11-23 20:43:26 -08:00
Simon Willison
bf1ac778a3 Use uv in the Justfile, refs #674 2025-11-23 19:50:06 -08:00
Simon Willison
d328f7e765 Fix flake8 error 2025-11-23 19:11:10 -08:00
Simon Willison
52ea7d21f4 Fix for mypy errors 2025-11-23 15:53:19 -08:00
Simon Willison
96fab69256 Removed convert skip_false and --skip-false, closes #542 2025-11-23 15:40:28 -08:00
Simon Willison
fafa966300 Table.create() now stores configuration in _defaults
When configuration parameters like pk, foreign_keys, not_null, etc.
are passed to Table.create(), they are now stored in self._defaults
so that subsequent operations on the same Table instance (like insert,
upsert) will use those settings automatically.

Previously, calling db["table"].insert({...}, pk="id") would create
the table correctly but subsequent calls to insert() or upsert() on
the same table object would not know about the pk setting, causing
issues like upsert() failing with "requires a pk" error.

Closes #655

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 15:18:55 -08:00
Simon Willison
d5f113de48 .gitignore uv.lock and dist 2025-11-23 14:12:18 -08:00
Simon Willison
4f12c7a452 Port from setup.py to pyproject.toml
Refs #676, closes #675
2025-11-23 14:07:11 -08:00
Benson Muite
1587feca6f Use PEP639 license metadata formatting 2025-11-23 12:18:18 -08:00
Simon Willison
7ffd5052e9 table.insert_all() and table.upsert_all() now take generators of lists/tuples
Closes #672, PR #673
2025-11-23 12:17:23 -08:00
Simon Willison
dfb2dbe967 Update GitHub Actions workflow cache configuration 2025-11-23 12:12:55 -08:00
Simon Willison
076c1a0aa2 Ran black 2025-11-23 11:59:19 -08:00
fry69
42d23f5954 test: suppress PytestUnraisableExceptionWarning in dateparse_errors 2025-11-23 11:58:06 -08:00
fry69
10957305be test: fix Python 3.14 datetime deprecation 2025-11-23 11:58:06 -08:00
Simon Willison
c479ca0f44 SQLite tests against Python 3.10 2025-10-01 13:52:10 -07:00
Simon Willison
370318c695 Lowercase for geometry types
Cog test failed in CI, but these are case insensitive anyway.
2025-10-01 13:52:10 -07:00
Simon Willison
f91e4c9e52 Run cog with --diff to show errors 2025-10-01 13:52:10 -07:00
Simon Willison
1361ed5711 A bunch of fixes for Click sentinal stuff 2025-10-01 13:52:10 -07:00
Simon Willison
bccd05c9b4 Test on 3.14 and 3.14t, drop 3.9 2025-10-01 13:52:10 -07:00
Alex Chan
cf1b407207
Add a type hint for db.close() (#663)
Closes #662
2025-10-01 13:37:05 -07:00
Simon Willison
094b010fd8
db.table() only returns tables, added db.view()
* db.table() only returns tables, added db.view(), closes #657
* Massive documentation update for db.table()

Refs #656
2025-05-08 23:19:36 -07:00
Simon Willison
d892d2ae49 Fix underline length, refs #656 4.0a0 2025-05-08 20:53:24 -07:00
Simon Willison
9e458dea7d Release 0.4a0
Refs #646, #648, #652, #654, #656
2025-05-08 20:51:48 -07:00
Simon Willison
8e7d018fa2
Use ON CONFLICT for upsert, refs #652
* New upsert implementation, refs #652
* supports_strict now caches on self._supports_strict

PR: https://github.com/simonw/sqlite-utils/pull/653
2025-05-08 20:37:49 -07:00
Simon Willison
72f6c820f6 Small documentation updates to fix warnings 2025-05-07 17:54:29 -07:00
Simon Willison
0e4e270d44
Test against multiple SQLite versions (#654)
* Test against pre-upsert SQLite 3.23.1

Borrowed from 8f86d2af6

* Try this on Python 3.9

* select ... from pragma_function_list()

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860898278

* Fix spelling error

* Compatible with latest black

* Skip plugin test that needs pragma_function_list

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860924225

* Ran cog
2025-05-07 17:49:50 -07:00
Simon Willison
88a83f0b7e
Link to JSON Lines instead
ndjson.org has an expired domain.
2025-05-04 22:50:10 -07:00
Simon Willison
0aefbb634d Removed trogon feature in favor of sqlite-util-tui
Refs #648, #545
2025-01-10 17:15:02 -08:00
Simon Willison
04107d3fea Remove note from docs about 3.8 and deterministic, closes #646 2024-11-23 14:49:15 -08:00
Simon Willison
c544f75446 Remove 3.8 from trove tags, refs #646 2024-11-23 14:48:55 -08:00
Simon Willison
1290c50f71 Drop Python 3.9, add Python 3.13 to all CI
Refs #646
2024-11-23 14:43:04 -08:00
Simon Willison
9d7da0606e Release 3.38 3.38
Refs #628, #633, #643, #644
2024-11-23 14:38:40 -08:00
Simon Willison
4381390cf1 Skip this test on DBs that don't support strict, refs #644 2024-11-23 14:32:58 -08:00
Simon Willison
4dc2e2e9c8 Use REAL for floating point columns if table is strict, closes #644
Refs #645
2024-11-23 14:27:21 -08:00
liunux4odoo
7423296ec7
include_rank parameter for Table.search
* Add `include_rank` parameter to `Table.search`
* Test for .search(include_rank)
* Docs for table.search(include_rank)

https://github.com/simonw/sqlite-utils/pull/628

Refs #480

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-11-23 12:34:27 -08:00
Mat Miller
42230709f7
Recreate indexes when calling transform when possible (#634)
* Recreate indexes when calling transform when possible and raise an error when they cannot be retained automatically
* Docs for sqlite_utils.db.TransformError

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-11-23 12:17:15 -08:00
Gavin Wahl
cbddfb28f9
typo in command name (#640) 2024-11-09 08:31:21 -08:00
Simon Willison
0acb1f29c2 Release 3.38a0 3.38a0
Refs #643
2024-11-08 12:13:57 -08:00