Commit graph

58 commits

Author SHA1 Message Date
Simon Willison
58dcedb510 Skip RETURNING tests if SQLite version does not support it
https://github.com/simonw/datasette/pull/2763#issuecomment-4588138314
2026-05-31 15:34:01 -07:00
Simon Willison
b4761bab53 Fixed an edge case with exactly 11 results returned
We select 11 when we return 10 just to detect if the results were truncated.

We were returning rowcount: 11 in this case when it should have been -1.
2026-05-31 14:14:49 -07:00
Simon Willison
1380b90a3f Close execute_write returning cursor on fetch errors
Refs #2762
2026-05-31 12:30:11 -07:00
Simon Willison
1e81be99e4 Add configurable execute_write returning limit
Refs #2762
2026-05-31 12:30:11 -07:00
Simon Willison
bdfd9d5482 Fix for execute write returning issue #2762 2026-05-31 11:36:34 -07:00
Simon Willison
aaf00e9ec2 Refactor hidden_table_names() to use new implemenatation
Refs https://github.com/simonw/datasette/pull/2749#issuecomment-4565727978
2026-05-28 08:42:06 -07:00
Simon Willison
737ff03efb Expanded analysis of SQL operations, refs #2748 2026-05-26 22:11:35 -07:00
Simon Willison
a855a1acec Database.analyze_sql(sql) method
Experimental, we may need this for the upcoming canned query
work so that we can tell if a user should be able to save
a writable canned query by confirming they have the right
permissions to update the affected tables.

Refs #2735
2026-05-24 22:29:49 -07:00
Simon Willison
3110faa0ba
Replace Janus queue with asyncio.Future
Closes #1752

AI generated patch explanation: https://gisthost.github.io/?e2b8d9c7666e988b5c003ff5e5ef3098
2026-05-16 11:45:43 -07:00
Simon Willison
dabf8e4199 Database.close() shuts down write thread and raises DatasetteClosedError
After this commit, Database.close() sends a sentinel to the write queue so
the background write thread exits cleanly, closes cached read/write
connections, and marks the instance closed. Subsequent calls to execute*()
raise DatasetteClosedError. close() remains idempotent and one-way.

Refs #2692

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:09:47 -07:00
Simon Willison
ade0ef8a60 Restore compatibility with existing execute_write_fn() callbacks
Closes #2691
2026-04-16 19:14:32 -07:00
Simon Willison
f02484c3de From 409 warnings down to 52 warnings.
By closing unclosed database connections.

Refs #2614
2026-04-14 18:46:47 -07:00
Simon Willison
fc1794719a
Database(is_temp_disk=True) option, used for internal database (#2684)
Closes #2683

* Add is_temp_disk option to Database for temp file-backed databases

Replace the default in-memory internal database with a temporary
file-backed database using WAL mode. This fixes concurrent read/write
locking errors that occur with named in-memory SQLite databases.

The new is_temp_disk parameter on Database creates a temp file via
tempfile.mkstemp, connects to it as a regular file-based database
with WAL mode enabled, and cleans it up on close() and via atexit.

https://claude.ai/code/session_01TteLrUjpDcARjnP1GMRqz2
2026-03-30 21:03:21 -07:00
Simon Willison
5c3137d148 Black formatting 2026-02-17 13:30:24 -08:00
Simon Willison
7915c46ddd
Fix flaky test_database_page test with deterministic ordering (#2628)
* Fix flaky test_database_page test with deterministic ordering

- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
  500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst

* Update test_row_foreign_key_tables for new deterministic FK ordering

The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.

* Update test_table_names for new alphabetical ordering

The table_names() method now returns tables sorted alphabetically.

* Fix for test that fails prior to SQLite 3.37

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 20:57:25 -08:00
Simon Willison
232a404743 Switch searchable_fts test table to FTS5, closes #2613 2025-12-12 22:18:35 -08:00
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
f2485dce9c
Hide FTS tables that have content=
* Hide FTS tables that have content=, closes #2477
2025-04-16 21:44:09 -07:00
Simon Willison
f95ac19e71 Fix to support replacing a database, closes #2465 2025-02-06 10:32:47 -08:00
Simon Willison
92c4d41ca6 results.dicts() method, closes #2414 2024-09-01 17:20:41 -07:00
Simon Willison
39dfc7d7d7
Removed units functionality and Pint dependency
Closes #2400, unblocks #2320
2024-08-20 19:03:33 -07:00
Alex Garcia
6d91d082e0
Hide shadow tables, don't hide virtual tables
Closes #2296
2024-08-15 13:19:22 -07:00
Simon Willison
10f9ba1a00 Take advantage of execute_write_fn(transaction=True)
A bunch of places no longer need to do manual transaction handling
thanks to this change. Refs #2277
2024-02-17 20:51:19 -08:00
Simon Willison
5e0e440f2c database.execute_write_fn(transaction=True) parameter, closes #2277 2024-02-17 20:28:15 -08:00
Simon Willison
5c64af6936 Upgrade to latest Black, closes #2239 2024-01-30 19:55:26 -08:00
Simon Willison
4284c74bc1
db.execute_isolated_fn() method (#2220)
Closes #2218
2023-12-19 10:51:03 -08:00
Simon Willison
c588a89f26 db.view_exists() method, needed by #1896 2022-11-18 14:16:38 -08:00
Simon Willison
b29ccb59c7 Add test for db.view_names() 2022-11-18 14:13:48 -08:00
Simon Willison
bbf33a7635 Test for bool(results), closes #1832 2022-10-04 21:32:29 -07:00
Simon Willison
fb7e70d5e7 Database(is_mutable=) now defaults to True, closes #1808
Refs https://github.com/simonw/datasette-upload-dbs/issues/6
2022-09-09 09:19:20 -07:00
Simon Willison
4e47a2d894 Fixed bug where tables with a column called n caused 500 errors
Closes #1228
2022-03-18 18:37:54 -07:00
Simon Willison
5010d1359b Fix for test failure caused by SQLite 3.37.0+, closes #1647 2022-03-05 11:46:59 -08:00
Simon Willison
8c401ee0f0 Fixed remaining code and docs for new block=True default, closes #1579 2021-12-23 11:18:20 -08:00
Simon Willison
00a2895cd2 execute_write defaut is now block=True, closes #1579 2021-12-23 11:03:49 -08:00
Simon Willison
5cadc24489 db.execute_write_script() and db.execute_write_many(), closes #1570
Refs #1555
2021-12-18 10:57:22 -08:00
Simon Willison
2e4ba71b53 Optimize create table calls using executescript=True
Refs #1555, #1569
2021-12-18 10:34:15 -08:00
Simon Willison
9e094b7c9d db.execute_write(executescript=True) option, closes #1569 2021-12-18 10:34:15 -08:00
Simon Willison
83bacfa945 Call _prepare_connection() on write connections, closes #1564 2021-12-17 17:58:39 -08:00
Simon Willison
69244a617b Rename city_id to _city_id in fixtures, refs #1525 2021-11-29 22:20:42 -08:00
Simon Willison
c306b696de Correct facet links for columns with a leading underscore, closes #1506 2021-11-13 20:44:54 -08:00
Simon Willison
2e8d924cdc Refactored generated_columns test, no longer in fixtures.db - refs #1391 2021-07-10 12:03:19 -07:00
Simon Willison
7c87532acc New .add_memory_database() method, closes #1247 2021-02-28 20:02:18 -08:00
Simon Willison
6f41c8a2be
--crossdb option for joining across databases (#1232)
* Test for cross-database join, refs #283
* Warn if --crossdb used with more than 10 DBs, refs #283
* latest.datasette.io demo of --crossdb joins, refs #283
* Show attached databases on /_memory page, refs #283
* Documentation for cross-database queries, refs #283
2021-02-18 14:09:12 -08:00
Simon Willison
8919f99c2f Improved .add_database() method design
Closes #1155 - _internal now has a sensible name

Closes #509 - Support opening multiple databases with the same stem
2020-12-22 12:04:18 -08:00
Simon Willison
5e9895c67f Database(memory_name=) for shared in-memory databases, closes #1151 2020-12-17 17:01:18 -08:00
Simon Willison
17cbbb1f7f
generated_columns table in fixtures.py, closes #1119 2020-11-30 16:28:02 -08:00
Simon Willison
461670a0b8
Support for generated columns
* Support for generated columns, closes #1116
* Show SQLite version in pytest report header
* Use table_info() if SQLite < 3.26.0
* Cache sqlite_version() rather than re-calculate every time
* Adjust test_database_page for SQLite 3.26.0 or higher
2020-11-30 13:29:57 -08:00
Simon Willison
dea3c508b3 Revert "Support for generated columns, closes #1116" - it failed CI
This reverts commit 37f87b5e52.
2020-11-30 12:09:32 -08:00
Simon Willison
37f87b5e52 Support for generated columns, closes #1116 2020-11-30 12:01:15 -08:00
Simon Willison
deb0be4ae5 Fix bug where compound foreign keys produced broken links, closes #1098 2020-11-29 11:30:17 -08:00