Commit graph

120 commits

Author SHA1 Message Date
Simon Willison
3de8507c6b insert(pk=, alter=True) can add the pk column from records again
The InvalidColumns check added for #732 fired before alter=True had a
chance to add the missing pk column - a regression from 3.x, where
insert({"id": 5, "a": 2}, pk="id", alter=True) against a table without
an id column worked. With alter=True the check is now deferred until the
first batch of record keys is known: a pk column supplied by the records
passes, one found in neither the table nor the records still raises
InvalidColumns before anything is inserted. An empty record iterator
with alter=True returns without error, matching the 3.x no-op.

Refs https://github.com/simonw/sqlite-utils/issues/769#issuecomment-4900034150

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 21:47:59 -07:00
Simon Willison
221774f25a Fix for table.insert(..., pk=..., ignore=True), closes #554 2026-07-06 20:22:08 -07:00
Simon Willison
6225eba5c8 Raise InvalidColumns on insert(pk="invalid")
Closes #732
2026-07-06 20:16:28 -07:00
Claude
a2e6ea2eca
Raise ValueError instead of AssertionError for invalid arguments
User-facing argument validation in db.py previously used bare
assert statements, which vanish entirely under python -O and raise
AssertionError - an exception type callers should not have to
catch for input validation. Fifteen validation sites now raise
ValueError with the same messages. Internal invariants (an
unreachable branch and a post-update rowcount check) remain
asserts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
2026-07-04 19:19:24 +00:00
Claude
397cdcc491
Polish: clearer errors, corrected docstrings, setuptools pin
- insert/upsert into a name that is actually a view now shows a
  clean error instead of a traceback
- db.view() on a name that is a table now says so in its error
- Database.__getitem__ docstring documents that views are returned
  for view names; hash_id docstring corrected (it is a column name,
  not a boolean)
- Registering two migrations with the same name in one set now
  raises ValueError at registration time instead of executing both
  and failing with IntegrityError at apply time
- build-system requires setuptools>=77, needed for the PEP 639
  license expression

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UnLnhsH25Nnv7LHhekUfPd
2026-07-04 19:13:15 +00:00
Simon Willison
8f0c06e188
Test against Python 3.15-dev, bump ty and Black (#738)
* Add Python 3.15-dev to test matrix
* Run ty check only on 3.14
* Bump Black version
* Update tabulate and use that in 
* Bump to latest ty
2026-05-17 16:52:48 -07:00
Simon Willison
c872d27bb3
Use REAL not FLOAT as SQLite column type (#680)
* Use REAL not FLOAT as SQLite column type, refs #645
* Fix for REAL columns by CSV --detect-types

Refs https://github.com/simonw/sqlite-utils/issues/645#issuecomment-3568947189

* Removed note about strict and REAL
2025-11-23 21:37:59 -08:00
Simon Willison
fb93452ea8
Use double quotes not braces for tables and columns (#678)
Closes #677
2025-11-23 20:43:26 -08:00
Simon Willison
d328f7e765 Fix flake8 error 2025-11-23 19:11:10 -08:00
Simon Willison
fafa966300 Table.create() now stores configuration in _defaults
When configuration parameters like pk, foreign_keys, not_null, etc.
are passed to Table.create(), they are now stored in self._defaults
so that subsequent operations on the same Table instance (like insert,
upsert) will use those settings automatically.

Previously, calling db["table"].insert({...}, pk="id") would create
the table correctly but subsequent calls to insert() or upsert() on
the same table object would not know about the pk setting, causing
issues like upsert() failing with "requires a pk" error.

Closes #655

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-23 15:18:55 -08:00
Simon Willison
094b010fd8
db.table() only returns tables, added db.view()
* db.table() only returns tables, added db.view(), closes #657
* Massive documentation update for db.table()

Refs #656
2025-05-08 23:19:36 -07:00
Simon Willison
8e7d018fa2
Use ON CONFLICT for upsert, refs #652
* New upsert implementation, refs #652
* supports_strict now caches on self._supports_strict

PR: https://github.com/simonw/sqlite-utils/pull/653
2025-05-08 20:37:49 -07:00
Simon Willison
4381390cf1 Skip this test on DBs that don't support strict, refs #644 2024-11-23 14:32:58 -08:00
Simon Willison
4dc2e2e9c8 Use REAL for floating point columns if table is strict, closes #644
Refs #645
2024-11-23 14:27:21 -08:00
Taj Khattra
1500c19bd0
Add more STRICT table support (#604)
* Add more STRICT table support per https://github.com/simonw/sqlite-utils/issues/344#issuecomment-982014776.
* Make `table.transform()` preserve STRICT mode.
* Fix mypy failures in PR #604
* Link to SQLITE strict page in a few places
2023-12-07 21:05:27 -08:00
Simon Willison
b2e0cd066d Test and docs for timedelta support, refs #522 2023-11-03 17:56:34 -07:00
Simon Willison
d2bcdc00c6 Swapped the order of a bunch of pytest comparisons
When I wrote this I thought constant == value was a better assertion. I no longer think that.
2023-08-17 18:05:13 -07:00
Simon Willison
b4735f794a Remove test.utils.collapse_whitespace, closes #583 2023-08-17 17:59:09 -07:00
Simon Willison
509857ee87
.add_foreign_keys() uses .transform() instead of PRAGMA writable_schema
Closes #577

This should solve all sorts of problems seen by users of platforms that throw errors on writable_schema.

Also added `add_foreign_keys=` and `foreign_keys=` parameters to `table.transform()`.
2023-08-17 17:48:08 -07:00
Simon Willison
82e8cd3667 db.rename_table() method, refs #565 2023-07-22 12:38:31 -07:00
Simon Willison
58b577279f table.create(..., replace=True / ignore = True) closes #568 2023-07-22 12:15:40 -07:00
Simon Willison
923768db2e Assert on exact error message, refs #537 2023-05-08 14:11:48 -07:00
Scott Perry
39ef137e67
Support self-referencing FKs in create (#537) 2023-05-08 14:10:00 -07:00
Simon Willison
104f37fa4d
db[table].create(..., transform=True) and create-table --transform
Closes #467
2022-08-27 16:17:55 -07:00
Simon Willison
0e60f3c80c Better error message if table has no columns, closes #424 2022-04-13 15:39:48 -07:00
Simon Willison
931b1e1513 .insert(hash_id_columns=) parameter, closes #343 2022-03-01 16:00:51 -08:00
Simon Willison
aa24903113 Create table if_not_exists=True argument, closes #397 2022-02-05 17:28:53 -08:00
Simon Willison
79b5b58354 Basic test for db[t].create(...), refs #397 2022-02-05 17:19:39 -08:00
Simon Willison
ab392157f7 analyze=True for insert_all/upsert_all, refs #378 2022-01-10 17:00:34 -08:00
Simon Willison
0142c2a3c2 Improved test_create_index_analyze test, refs #378 2022-01-10 12:02:08 -08:00
Simon Willison
0d10402f7b table.create_index(..., analyze=True), refs #378 2022-01-10 12:00:24 -08:00
Simon Willison
ffb54427d3 insert now replaces square braces in column name with underscore, closes #341 2021-11-14 18:56:35 -08:00
Simon Willison
9cda5b070f Handle dict/tuple/list mapping to TEXT, closes #338 2021-11-14 16:36:00 -08:00
Simon Willison
13195d8747 Test demonstrating fix for #335 2021-11-14 14:55:42 -08:00
Simon Willison
e8d958109e create_index(..., find_unique_name=True) option, refs #335 2021-11-14 14:49:42 -08:00
Simon Willison
c62363ebdc Run mypy against tests/ too, refs #37 2021-08-18 14:49:12 -07:00
Simon Willison
90e211e3e2 Now complies with flake8, refs #291 2021-06-22 18:22:08 -07:00
Simon Willison
9faeef230b New db.query() method, refs #290 2021-06-21 21:03:59 -07:00
Simon Willison
b9629099ab Fix bug with upsert_all() and single column tables, closes #271 2021-06-12 19:57:21 -07:00
Simon Willison
51d01da30d
Ability to add descending order indexes (#262)
* DescIndex(column) for descending index columns, refs #260
* Ability to add desc indexes using CLI, closes #260
2021-05-28 22:01:38 -07:00
Simon Willison
a95954c481 Tests for unicode characters in nested JSON, refs #257 2021-05-18 19:56:53 -07:00
Simon Willison
c236894caa table.drop(ignore=True) option, refs #237 2021-02-25 09:05:08 -08:00
Simon Willison
a76e3b33ac Fixed bug with add_foreign_key against columns with spaces, closes #238 2021-02-25 08:53:53 -08:00
Simon Willison
8f042ae1fd Fix for bug with extra columns in later chunks, closes #234
Thanks @nieuwenhoven for the fix, proposed in #225
2021-02-14 13:03:31 -08:00
Simon Willison
b6840646ba .add_missing_columns() is now case insensitive, closes #221 2021-01-12 15:17:27 -08:00
Simon Willison
3d041d34d5 Renamed db.escape() to db.quote() and documented it
Closes #217
2021-01-02 20:15:04 -08:00
Simon Willison
7c0ef116ed pk=['id'] now equivalent to pk='id', closes #181 2020-10-14 14:59:38 -07:00
Simon Willison
e23eedb4ce ignore=True argument for add_foreign_key, closes #112
Also --ignore for add-foreign-key command

Plus table.add_foreign_key(...) now returns self, allowing more chaining
2020-09-20 15:17:25 -07:00
Simon Wiles
367082e787
Typos in tests (#156)
Thanks @simonwiles
2020-09-09 11:21:22 -07:00
Simon Wiles
e6d202b742
Handle case where subsequent records (after first batch) include extra columns
Refs #145.

* Extract build_insert_queries_and_params
* Extract insert_chunk so it can be called recursively

Thanks, @simonwiles
2020-09-08 16:20:36 -07:00