Commit graph

496 commits

Author SHA1 Message Date
Simon Willison
c872d27bb3
Use REAL not FLOAT as SQLite column type (#680)
* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
2025-11-23 21:37:59 -08:00
Simon Willison
fb93452ea8
Use double quotes not braces for tables and columns (#678)
Closes #677
2025-11-23 20:43:26 -08:00
Simon Willison
52ea7d21f4 Fix for mypy errors 2025-11-23 15:53:19 -08:00
Simon Willison
96fab69256 Removed convert skip_false and --skip-false, closes #542 2025-11-23 15:40:28 -08:00
Simon Willison
fafa966300 Table.create() now stores configuration in _defaults
When configuration parameters like pk, foreign_keys, not_null, etc.
are passed to Table.create(), they are now stored in self._defaults
so that subsequent operations on the same Table instance (like insert,
upsert) will use those settings automatically.

Previously, calling db["table"].insert({...}, pk="id") would create
the table correctly but subsequent calls to insert() or upsert() on
the same table object would not know about the pk setting, causing
issues like upsert() failing with "requires a pk" error.

Closes #655

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 15:18:55 -08:00
Simon Willison
7ffd5052e9 table.insert_all() and table.upsert_all() now take generators of lists/tuples
Closes #672, PR #673
2025-11-23 12:17:23 -08:00
Simon Willison
076c1a0aa2 Ran black 2025-11-23 11:59:19 -08:00
fry69
10957305be test: fix Python 3.14 datetime deprecation 2025-11-23 11:58:06 -08:00
Simon Willison
370318c695 Lowercase for geometry types
Cog test failed in CI, but these are case insensitive anyway.
2025-10-01 13:52:10 -07:00
Simon Willison
1361ed5711 A bunch of fixes for Click sentinal stuff 2025-10-01 13:52:10 -07:00
Alex Chan
cf1b407207
Add a type hint for db.close() (#663)
Closes #662
2025-10-01 13:37:05 -07:00
Simon Willison
094b010fd8
db.table() only returns tables, added db.view()
* db.table() only returns tables, added db.view(), closes #657
* Massive documentation update for db.table()

Refs #656
2025-05-08 23:19:36 -07:00
Simon Willison
8e7d018fa2
Use ON CONFLICT for upsert, refs #652
* New upsert implementation, refs #652
* supports_strict now caches on self._supports_strict

PR: https://github.com/simonw/sqlite-utils/pull/653
2025-05-08 20:37:49 -07:00
Simon Willison
0e4e270d44
Test against multiple SQLite versions (#654)
* Test against pre-upsert SQLite 3.23.1

Borrowed from 8f86d2af6

* Try this on Python 3.9

* select ... from pragma_function_list()

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860898278

* Fix spelling error

* Compatible with latest black

* Skip plugin test that needs pragma_function_list

Refs https://github.com/simonw/sqlite-utils/pull/654#issuecomment-2860924225

* Ran cog
2025-05-07 17:49:50 -07:00
Simon Willison
0aefbb634d Removed trogon feature in favor of sqlite-util-tui
Refs #648, #545
2025-01-10 17:15:02 -08:00
Simon Willison
4dc2e2e9c8 Use REAL for floating point columns if table is strict, closes #644
Refs #645
2024-11-23 14:27:21 -08:00
liunux4odoo
7423296ec7
include_rank parameter for Table.search
* Add `include_rank` parameter to `Table.search`
* Test for .search(include_rank)
* Docs for table.search(include_rank)

https://github.com/simonw/sqlite-utils/pull/628

Refs #480

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-11-23 12:34:27 -08:00
Mat Miller
42230709f7
Recreate indexes when calling transform when possible (#634)
* Recreate indexes when calling transform when possible and raise an error when they cannot be retained automatically
* Docs for sqlite_utils.db.TransformError

Co-authored-by: Simon Willison <swillison@gmail.com>
2024-11-23 12:17:15 -08:00
Simon Willison
2258b431d4 Neater return_db pattern for reusing memory command, closes #643 2024-11-08 12:11:17 -08:00
Simon Willison
8906f57740 Hack to support reuse of memory command, closes #643 2024-11-08 11:17:17 -08:00
Simon Willison
dc79454234
Fix for np.int8 error, refs #632
Also refs:
- https://github.com/simonw/llm/issues/531
2024-07-18 11:32:48 -07:00
Simon Willison
1d050dcdc7 insert-files multiple --pk support, closes #621 2024-03-16 17:33:31 -07:00
Simon Willison
23be5be1dc create-table now supports multiple --pk, refs #620 2024-03-16 17:05:39 -07:00
Simon Willison
5bd7aec4d2
Test against Python 3.13 pre-release (#619)
* Test against Python 3.13 pre-release

* Skip tests for numpy on Python 3.13

Refs https://github.com/simonw/sqlite-utils/pull/619#issuecomment-1998798451

* Try to avoid Python 3.13 cog differences

* Hide \b characters in cli-reference

* Fixed .rST warning
2024-03-14 21:14:10 -07:00
Simon Willison
70cc0c91ab
Drop support for Python 3.7
* Drop support for Python 3.7, closes #616
* Bumped versions on a whole lot of GitHub Actions
2024-01-30 18:37:38 -08:00
Simon Willison
ff57a97482 Upgrade to latest Black 2024-01-30 18:18:52 -08:00
Simon Willison
b7def00b8c More consistent indentation 2024-01-14 09:27:58 -08:00
Taj Khattra
1500c19bd0
Add more STRICT table support (#604)
* Add more STRICT table support per https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776.
* Make `table.transform()` preserve STRICT mode.
* Fix mypy failures in PR #604
* Link to SQLITE strict page in a few places
2023-12-07 21:05:27 -08:00
Simon Willison
88bd372205 str, int, bytes aliases for column types, closes #606 2023-12-06 10:49:21 -08:00
Harald Nezbeda
347fdc865e
Map datetime.timedelta to text columns, closes #522 2023-11-03 17:49:50 -07:00
Luke Plant
37273d7f63
Fixed issue #433 - CLI eats cursor (#598)
The issue is that underlying iterator is not fully consumed within the body of
the `with file_progress()` block. Instead, that block creates generator
expressions like `docs = (dict(zip(headers, row)) for row in reader)`

These iterables are consumed later, outside the `with file_progress()` block,
which consumes the underlying iterator, and in turn updates the progress bar.

This means that the `ProgressBar.__exit__` method gets called before the last
time the `ProgressBar.update` method gets called. The result is that the code to
make the cursor invisible (inside the `update()` method) is called after the
cleanup code to make it visible (in the `__exit__` method).

The fix is to move consumption of the `docs` iterators within the progress bar block.

(An additional fix, to make ProgressBar more robust against this kind of misuse, would
to make it refusing to update after its `__exit__` method had been called, just
like files cannot be `read()` after they are closed. That requires a in the
click library).
2023-11-03 17:40:29 -07:00
Simon Willison
b92ea4793c Fixed spelling of wrappping 2023-11-03 17:31:49 -07:00
Mike Coats
4b3c83cd9f Add spatialite arm64 linux path
Issue: #599
2023-11-03 17:31:49 -07:00
Simon Willison
1c6ea54338
.transform() now preserves rowid values
* .transform() now preserves rowid values, refs #592
* Test transform rowids against different table types, closes #592
2023-09-08 17:45:30 -07:00
Simon Willison
56093de078 sqlite-utils transform --add-foreign-key option, closes #585 2023-08-17 18:51:04 -07:00
Simon Willison
70717dc0e1 Remove unneccessary warning, refs #577, #585 2023-08-17 18:28:09 -07:00
Simon Willison
509857ee87
.add_foreign_keys() uses .transform() instead of PRAGMA writable_schema
Closes #577

This should solve all sorts of problems seen by users of platforms that throw errors on writable_schema.

Also added `add_foreign_keys=` and `foreign_keys=` parameters to `table.transform()`.
2023-08-17 17:48:08 -07:00
Simon Willison
619cea8681 sqlite-utils convert --pdb option, closes #581 2023-07-26 14:06:05 -07:00
Simon Willison
5e9a02153d Fix bug with --editable
See also: https://github.com/simonw/llm/issues/136
2023-07-25 22:20:49 -07:00
Simon Willison
c728c25555 Undo change which broke the tests 2023-07-22 16:25:14 -07:00
Simon Willison
778dad789e More robust _called_from_test pattern 2023-07-22 16:21:27 -07:00
Simon Willison
374a816c72 Database(..., execute_plugins=False) mechanism, refs #575 2023-07-22 16:06:11 -07:00
Alex Garcia
3f80a02698
prepare_connection plugin hook
Closes:
- #574

Refs #567

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2023-07-22 15:59:08 -07:00
Simon Willison
61aaa69815 .transform(..., keep_table=name) parameter, closes #571
Also type hints for the transform_sql() method
2023-07-22 15:32:09 -07:00
Simon Willison
18f190e283 sqlite-utils rename-table command, refs #565 2023-07-22 12:48:04 -07:00
Simon Willison
82e8cd3667 db.rename_table() method, refs #565 2023-07-22 12:38:31 -07:00
Simon Willison
86a352f8b7 Fixed mypy error, refs #568 2023-07-22 12:27:41 -07:00
Simon Willison
2d55f185ff Applied Black, refs #568 2023-07-22 12:22:15 -07:00
Simon Willison
58b577279f table.create(..., replace=True / ignore = True) closes #568 2023-07-22 12:15:40 -07:00
Simon Willison
b379a2a0c3 Added pluggy and first hook, register_commands - refs #569, #567 2023-07-22 12:04:31 -07:00