Commit graph

21 commits

Author SHA1 Message Date
Simon Willison
733a67490f Test for CSV headers with memory command, refs #702, #707 2026-06-21 16:05:10 -07:00
Simon Willison
871038505b
Type fixes now enforced by ty
* Fix type warning for pipe.stdout possibly being None

Add conditional check before calling .read() on pipe.stdout since
Popen can return None for stdout.

* Use ctx.meta instead of dynamic attribute for database cleanup

Click's Context.meta dictionary is the proper way to store arbitrary
data on the context object, avoiding type checker warnings about
dynamic attribute assignment.

* Add assert for tables.callback before calling

Click's callback attribute is typed as Optional[Callable], so add
assert to satisfy type checker that it's not None.

* Fix type errors in cli.py and db.py

- Add type annotation for Database.conn to fix context manager errors
- Convert exception objects to str() when raising ClickException
- Handle None return from find_spatialite() with proper error message

* Fix remaining type errors in cli.py

- Add typing import and type annotations for dict kwargs
- Use db.table() instead of db[] for extract command
- Fix missing str() conversion for exception

* Fix type errors in db.py

- Add type annotation for Database.conn
- Add type: ignore for optional sqlite_dump import
- Update execute/query parameter types to Sequence|Dict for sqlite3 compatibility
- Use getattr for fn.__name__ access to handle callables without __name__
- Handle None return from find_spatialite() with OSError
- Fix pk_values assignment to use local variable

* Add type: ignore for optional pysqlite3 and sqlean imports

These are alternative sqlite3 implementations that may not be installed.

* Fix type errors in tests and plugins

- Add type: ignore for monkey-patching Database.__init__ in conftest
- Fix CLI test to pass string "2" instead of integer to Click invoke
- Add type: ignore for optional sqlean import
- Fix add_geometry_column test to use "XY" instead of integer 2
- Add type: ignore for click.Context as context manager
- Add type: ignore for enable_fts test that intentionally omits argument
- Add type: ignore for sys._called_from_test dynamic attribute
- Fix rows_from_file test type error for intentional wrong argument
- Handle None from pm.get_hookcallers in plugins.py

* Use db.table() instead of db[] for Table-specific operations

Changes db[table] to db.table(table) in CLI commands where we know
we're working with tables, not views. This resolves most of the
Table | View disambiguation type warnings since db.table() returns
Table directly rather than Table | View.

* Fix remaining type warnings in sqlite_utils package

- Add assert for sniff_buffer not being None
- Handle cursor.fetchone() potentially returning None
- Use db.table() for counts_table and index_foreign_keys
- Add type: ignore for cursor union type in raw mode

* Ran Black

* Run ty in CI

* ty check sqlite_utils

* Skip running ty on Windows

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-16 19:34:45 -08:00
Simon Willison
fd5b09f64b
Database as a context manager, fixed many pytest warnings
* Database can now work as a context manager
* Claude Code helped fix a ton of .close() warnings

https://gistpreview.github.io/?730f0c5dc38528a1dd0615f330bd5481

* New autouse fixture to help with test warnings

Refs https://github.com/simonw/sqlite-utils/issues/692#issuecomment-3644371889

* Fix all remaining resource warnings

https://gistpreview.github.io/?0bb8e869b82f6ff0db647de755182502

Closes #692
2025-12-11 16:56:12 -08:00
Simon Willison
0bbc68089c
--functions can take filenames, can be used multiple times (#681)
Closes #659

The --functions option now accepts:
- File paths ending in .py (e.g., --functions my_funcs.py)
- Multiple invocations (e.g., --functions foo.py --functions 'def bar(): ...')
- Inline Python code (existing behavior)

Implementation follows the same pattern as llm's --functions flag
(simonw/llm@a880c123).

Changes:
- Added multiple=True to --functions Click option in query, bulk, and memory commands
- Modified _register_functions() to detect and read .py files
- Updated _maybe_register_functions() to iterate over multiple function sources
- Removed unused bytes/bytearray handling
- Added comprehensive tests for file paths and multiple invocations
- Updated documentation with examples

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Shorter help for --functions

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-23 21:46:51 -08:00
Simon Willison
fb93452ea8
Use double quotes not braces for tables and columns (#678)
Closes #677
2025-11-23 20:43:26 -08:00
Simon Willison
2258b431d4 Neater return_db pattern for reusing memory command, closes #643 2024-11-08 12:11:17 -08:00
Simon Willison
8906f57740 Hack to support reuse of memory command, closes #643 2024-11-08 11:17:17 -08:00
Simon Willison
f5c63088e1
Use sqlean if available in environment (#560)
Closes #559
Closes #235

Refs https://github.com/simonw/llm/issues/60

- Uses `sqlean` in place of `sqlite3` if `sqlean.py` is installed
- Uses `sqlite-dump` if available and `conn.iterdump()` does not exist
- New `with db.ensure_autocommit_off()` method for ensuring autocommit is off, used by `enable_wal()` and `disable_wal()`.
2023-06-25 16:25:51 -07:00
Simon Willison
a256d7de98 Fix a bunch of warnings in the tests, refs #541 2023-05-08 12:57:43 -07:00
Simon Willison
59e2cfbdc1 sqlite-utils memory --functions, refs #471 2022-08-26 22:03:53 -07:00
Simon Willison
12b8c9de25 sqlite-utils memory --flatten, closes #332 2021-11-14 15:05:00 -08:00
Simon Willison
c1b26eed03 sqlite-utils memory handles files with same filename, closes #325 2021-09-22 13:45:37 -07:00
Simon Willison
9258f4bd84 sqlite-utils memory --analyze, closes #320 2021-08-22 08:44:25 -07:00
Simon Willison
90e211e3e2 Now complies with flake8, refs #291 2021-06-22 18:22:08 -07:00
Simon Willison
933be66eba sqlite-utils memory --schema, closes #288
Also updated some rowid examples, closes #287
2021-06-20 11:25:21 -07:00
Simon Willison
0e797033f9 .transform() on rowid (non-pk) tables bug fix, closes #284 2021-06-19 08:28:26 -07:00
Simon Willison
dc94f4bb8c Don't detect types on JSON input to memory, closes #283 2021-06-19 07:52:44 -07:00
Simon Willison
ec5174ed40 Detect types for sqlite-utils memory CSV, opt out with --no-detect-types - closes #282 2021-06-18 21:37:56 -07:00
Simon Willison
00e4bd5ff1 TSV and JSON support for sqlite-utils memory
Closes #281, closes #279, refs #272
2021-06-18 20:11:54 -07:00
Simon Willison
7684bbf097 --encoding option for sqlite-utils memory, closes #280
Refs #272
2021-06-18 08:29:41 -07:00
Simon Willison
42ec59d8ee
sqlite-utils memory command for directly querying CSV/JSON data
* Turn SQL errors into click errors
* Initial CSV-only prototype of sqlite-utils memory, refs #272
* Implement --save plus tests for --save and --dump, refs #272
* Re-arranged CLI query documentation, refs #272
* Re-organized CLI query docs, refs #272
* Docs for --save and --dump plus made SQL optional for those, refs #273
* Replaced one last :memory: example
* Documented --attach option for memory command, refs #272
* Improved arrangement of CLI query documentation
2021-06-18 08:00:52 -07:00