Commit graph

252 commits

Author SHA1 Message Date
Simon Willison
2ba282db8a Use db.table() instead of db[] for Table-specific operations
Changes db[table] to db.table(table) in CLI commands where we know
we're working with tables, not views. This resolves most of the
Table | View disambiguation type warnings since db.table() returns
Table directly rather than Table | View.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 16:45:46 -08:00
Simon Willison
5c15db90f4 Fix remaining type errors in cli.py
- Add typing import and type annotations for dict kwargs
- Use db.table() instead of db[] for extract command
- Fix missing str() conversion for exception

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 16:26:57 -08:00
Simon Willison
29842be858 Fix type errors in cli.py and db.py
- Add type annotation for Database.conn to fix context manager errors
- Convert exception objects to str() when raising ClickException
- Handle None return from find_spatialite() with proper error message

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 16:25:03 -08:00
Simon Willison
116cc99791 Add assert for tables.callback before calling
Click's callback attribute is typed as Optional[Callable], so add
assert to satisfy type checker that it's not None.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 16:23:12 -08:00
Simon Willison
0ee090f541 Use ctx.meta instead of dynamic attribute for database cleanup
Click's Context.meta dictionary is the proper way to store arbitrary
data on the context object, avoiding type checker warnings about
dynamic attribute assignment.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 16:22:35 -08:00
Simon Willison
fd5b09f64b
Database as a context manager, fixed many pytest warnings
* Database can now work as a context manager
* Claude Code helped fix a ton of .close() warnings

https://gistpreview.github.io/?730f0c5dc38528a1dd0615f330bd5481

* New autouse fixture to help with test warnings

Refs https://github.com/simonw/sqlite-utils/issues/692#issuecomment-3644371889

* Fix all remaining resource warnings

https://gistpreview.github.io/?0bb8e869b82f6ff0db647de755182502

Closes #692
2025-12-11 16:56:12 -08:00
Copilot
e66e82f2c8
Remove Sentinel workaround for Click 8.3.1+ (#685)
Closes #666

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2025-11-24 09:11:18 -08:00
Simon Willison
35377a874b
Detect CSV/TSV column types by default (#683)
The `--detect-types` option is now automatically turned on for all commands that deal with CSV or CSV.

A new `--no-detect-types` option can be used to have all columns treated as text.

Closes #679
2025-11-23 22:23:15 -08:00
Simon Willison
0bbc68089c
--functions can take filenames, can be used multiple times (#681)
Closes #659

The --functions option now accepts:
- File paths ending in .py (e.g., --functions my_funcs.py)
- Multiple invocations (e.g., --functions foo.py --functions 'def bar(): ...')
- Inline Python code (existing behavior)

Implementation follows the same pattern as llm's --functions flag
(simonw/llm@a880c123).

Changes:
- Added multiple=True to --functions Click option in query, bulk, and memory commands
- Modified _register_functions() to detect and read .py files
- Updated _maybe_register_functions() to iterate over multiple function sources
- Removed unused bytes/bytearray handling
- Added comprehensive tests for file paths and multiple invocations
- Updated documentation with examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Shorter help for --functions

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-23 21:46:51 -08:00
Simon Willison
c872d27bb3
Use REAL not FLOAT as SQLite column type (#680)
* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
2025-11-23 21:37:59 -08:00
Simon Willison
fb93452ea8
Use double quotes not braces for tables and columns (#678)
Closes #677
2025-11-23 20:43:26 -08:00
Simon Willison
96fab69256 Removed convert skip_false and --skip-false, closes #542 2025-11-23 15:40:28 -08:00
Simon Willison
076c1a0aa2 Ran black 2025-11-23 11:59:19 -08:00
fry69
10957305be test: fix Python 3.14 datetime deprecation 2025-11-23 11:58:06 -08:00
Simon Willison
370318c695 Lowercase for geometry types
Cog test failed in CI, but these are case insensitive anyway.
2025-10-01 13:52:10 -07:00
Simon Willison
1361ed5711 A bunch of fixes for Click sentinal stuff 2025-10-01 13:52:10 -07:00
Simon Willison
094b010fd8
db.table() only returns tables, added db.view()
* db.table() only returns tables, added db.view(), closes #657
* Massive documentation update for db.table()

Refs #656
2025-05-08 23:19:36 -07:00
Simon Willison
8e7d018fa2
Use ON CONFLICT for upsert, refs #652
* New upsert implementation, refs #652
* supports_strict now caches on self._supports_strict

PR: https://github.com/simonw/sqlite-utils/pull/653
2025-05-08 20:37:49 -07:00
Simon Willison
0aefbb634d Removed trogon feature in favor of sqlite-util-tui
Refs #648, #545
2025-01-10 17:15:02 -08:00
Simon Willison
2258b431d4 Neater return_db pattern for reusing memory command, closes #643 2024-11-08 12:11:17 -08:00
Simon Willison
8906f57740 Hack to support reuse of memory command, closes #643 2024-11-08 11:17:17 -08:00
Simon Willison
1d050dcdc7 insert-files multiple --pk support, closes #621 2024-03-16 17:33:31 -07:00
Simon Willison
23be5be1dc create-table now supports multiple --pk, refs #620 2024-03-16 17:05:39 -07:00
Simon Willison
5bd7aec4d2
Test against Python 3.13 pre-release (#619)
* Test against Python 3.13 pre-release

* Skip tests for numpy on Python 3.13

Refs https://github.com/simonw/sqlite-utils/pull/619#issuecomment-1998798451

* Try to avoid Python 3.13 cog differences

* Hide \b characters in cli-reference

* Fixed .rST warning
2024-03-14 21:14:10 -07:00
Taj Khattra
1500c19bd0
Add more STRICT table support (#604)
* Add more STRICT table support per https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776.
* Make `table.transform()` preserve STRICT mode.
* Fix mypy failures in PR #604
* Link to SQLITE strict page in a few places
2023-12-07 21:05:27 -08:00
Simon Willison
88bd372205 str, int, bytes aliases for column types, closes #606 2023-12-06 10:49:21 -08:00
Luke Plant
37273d7f63
Fixed issue #433 - CLI eats cursor (#598)
The issue is that underlying iterator is not fully consumed within the body of
the `with file_progress()` block. Instead, that block creates generator
expressions like `docs = (dict(zip(headers, row)) for row in reader)`

These iterables are consumed later, outside the `with file_progress()` block,
which consumes the underlying iterator, and in turn updates the progress bar.

This means that the `ProgressBar.__exit__` method gets called before the last
time the `ProgressBar.update` method gets called. The result is that the code to
make the cursor invisible (inside the `update()` method) is called after the
cleanup code to make it visible (in the `__exit__` method).

The fix is to move consumption of the `docs` iterators within the progress bar block.

(An additional fix, to make ProgressBar more robust against this kind of misuse, would
to make it refusing to update after its `__exit__` method had been called, just
like files cannot be `read()` after they are closed. That requires a in the
click library).
2023-11-03 17:40:29 -07:00
Simon Willison
56093de078 sqlite-utils transform --add-foreign-key option, closes #585 2023-08-17 18:51:04 -07:00
Simon Willison
70717dc0e1 Remove unneccessary warning, refs #577, #585 2023-08-17 18:28:09 -07:00
Simon Willison
619cea8681 sqlite-utils convert --pdb option, closes #581 2023-07-26 14:06:05 -07:00
Simon Willison
5e9a02153d Fix bug with --editable
See also: https://github.com/simonw/llm/issues/136
2023-07-25 22:20:49 -07:00
Simon Willison
c728c25555 Undo change which broke the tests 2023-07-22 16:25:14 -07:00
Simon Willison
778dad789e More robust _called_from_test pattern 2023-07-22 16:21:27 -07:00
Simon Willison
18f190e283 sqlite-utils rename-table command, refs #565 2023-07-22 12:48:04 -07:00
Simon Willison
2d55f185ff Applied Black, refs #568 2023-07-22 12:22:15 -07:00
Simon Willison
58b577279f table.create(..., replace=True / ignore = True) closes #568 2023-07-22 12:15:40 -07:00
Simon Willison
b379a2a0c3 Added pluggy and first hook, register_commands - refs #569, #567 2023-07-22 12:04:31 -07:00
Simon Willison
ef31210bf0 sqlite-utils install -e, closes #570 2023-07-22 11:32:42 -07:00
Simon Willison
f7af23837d --empty-null option for CSV and TSV imports, closes #563 2023-07-02 22:42:26 -07:00
Simon Willison
63dc7ab1a5 Fixed a complaint from ruff check 2023-06-29 13:41:08 -07:00
Simon Willison
8c739558f7 --stop-after option, closes #561 2023-06-27 11:50:04 -07:00
Simon Willison
f5c63088e1
Use sqlean if available in environment (#560)
Closes #559
Closes #235

Refs https://github.com/simonw/llm/issues/60

- Uses `sqlean` in place of `sqlite3` if `sqlean.py` is installed
- Uses `sqlite-dump` if available and `conn.iterdump()` does not exist
- New `with db.ensure_autocommit_off()` method for ensuring autocommit is off, used by `enable_wal()` and `disable_wal()`.
2023-06-25 16:25:51 -07:00
Simon Willison
718b0cba9b
Experimental TUI powered by Trogon
* sqlite-utils tui command if Trogon is installed, closes #545
* Documentation for trogon TUI
* Screenshot of TUI
* Ignore trogon mypy error
* only run flake8 on Python 3.8 or higher, closes #550
2023-05-21 11:41:56 -07:00
Simon Willison
e8c5b042e4 Validate column names in analyze-columns, closes #548 2023-05-21 10:35:48 -07:00
Simon Willison
6027f3ea69 No need to show common values if everything is null
Closes #547
2023-05-21 10:19:16 -07:00
Simon Willison
d2a7b15b2b
Analyze tables options: --common-limit, --no-most, --no-least
Closes #544
2023-05-21 09:19:30 -07:00
Simon Willison
e0ec4c3451 --no-skip-false option, plus docs - closes #527 2023-05-08 14:03:20 -07:00
Simon Willison
e4ed372517 Show more detailed error on invalid JSON, closes #532 2023-05-08 13:31:56 -07:00
Simon Willison
a256d7de98 Fix a bunch of warnings in the tests, refs #541 2023-05-08 12:57:43 -07:00
Simon Willison
4fc2f12c88 Fix ResourceWarning in sqlite-utils insert, refs #534 2023-05-08 12:39:06 -07:00