Commit graph

51 commits

Author SHA1 Message Date
Simon Willison
a855a1acec Database.analyze_sql(sql) method
Experimental, we may need this for the upcoming canned query
work so that we can tell if a user should be able to save
a writable canned query by confirming they have the right
permissions to update the affected tables.

Refs #2735
2026-05-24 22:29:49 -07:00
Simon Willison
3110faa0ba
Replace Janus queue with asyncio.Future
Closes #1752

AI generated patch explanation: https://gisthost.github.io/?e2b8d9c7666e988b5c003ff5e5ef3098
2026-05-16 11:45:43 -07:00
Simon Willison
dabf8e4199 Database.close() shuts down write thread and raises DatasetteClosedError
After this commit, Database.close() sends a sentinel to the write queue so
the background write thread exits cleanly, closes cached read/write
connections, and marks the instance closed. Subsequent calls to execute*()
raise DatasetteClosedError. close() remains idempotent and one-way.

Refs #2692

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 20:09:47 -07:00
Simon Willison
ade0ef8a60 Restore compatibility with existing execute_write_fn() callbacks
Closes #2691
2026-04-16 19:14:32 -07:00
Simon Willison
f02484c3de From 409 warnings down to 52 warnings.
By closing unclosed database connections.

Refs #2614
2026-04-14 18:46:47 -07:00
Simon Willison
fc1794719a
Database(is_temp_disk=True) option, used for internal database (#2684)
Closes #2683

* Add is_temp_disk option to Database for temp file-backed databases

Replace the default in-memory internal database with a temporary
file-backed database using WAL mode. This fixes concurrent read/write
locking errors that occur with named in-memory SQLite databases.

The new is_temp_disk parameter on Database creates a temp file via
tempfile.mkstemp, connects to it as a regular file-based database
with WAL mode enabled, and cleans it up on close() and via atexit.

https://claude.ai/code/session_01TteLrUjpDcARjnP1GMRqz2
2026-03-30 21:03:21 -07:00
Simon Willison
5c3137d148 Black formatting 2026-02-17 13:30:24 -08:00
Simon Willison
7915c46ddd
Fix flaky test_database_page test with deterministic ordering (#2628)
* Fix flaky test_database_page test with deterministic ordering

- Add ORDER BY to table_names() query in database.py
- Sort foreign keys deterministically in get_all_foreign_keys()
- Refactor test_database_page to use property-based assertions instead of
  500+ lines of hardcoded expected data
- Run blacken-docs on plugin_hooks.rst

* Update test_row_foreign_key_tables for new deterministic FK ordering

The foreign keys are now sorted by (other_table, column, other_column),
so complex_foreign_keys comes before foreign_key_references alphabetically.

* Update test_table_names for new alphabetical ordering

The table_names() method now returns tables sorted alphabetically.

* Fix for test that fails prior to SQLite 3.37

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-23 20:57:25 -08:00
Simon Willison
232a404743 Switch searchable_fts test table to FTS5, closes #2613 2025-12-12 22:18:35 -08:00
Simon Willison
5c537e0a3e Fix type annotation bugs and remove unused imports
This fixes issues introduced by the ruff commit e57f391a which converted
Optional[x] to x | None:

- Fixed datasette/app.py line 1024: Dict[id | str, Dict] -> Dict[int | str, Dict]
  (was using id built-in function instead of int type)
- Fixed datasette/app.py line 1074: Optional["Resource"] -> "Resource" | None
- Added 'from __future__ import annotations' for Python 3.10 compatibility
- Added TYPE_CHECKING blocks to avoid circular imports
- Removed dead code (unused variable assignments) from cli.py and views
- Removed unused imports flagged by ruff across multiple files
- Fixed test fixtures: moved app_client fixture imports to conftest.py
  (fixed 71 test errors caused by fixtures not being registered)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-26 16:03:13 -07:00
Simon Willison
f2485dce9c
Hide FTS tables that have content=
* Hide FTS tables that have content=, closes #2477
2025-04-16 21:44:09 -07:00
Simon Willison
f95ac19e71 Fix to support replacing a database, closes #2465 2025-02-06 10:32:47 -08:00
Simon Willison
92c4d41ca6 results.dicts() method, closes #2414 2024-09-01 17:20:41 -07:00
Simon Willison
39dfc7d7d7
Removed units functionality and Pint dependency
Closes #2400, unblocks #2320
2024-08-20 19:03:33 -07:00
Alex Garcia
6d91d082e0
Hide shadow tables, don't hide virtual tables
Closes #2296
2024-08-15 13:19:22 -07:00
Simon Willison
10f9ba1a00 Take advantage of execute_write_fn(transaction=True)
A bunch of places no longer need to do manual transaction handling
thanks to this change. Refs #2277
2024-02-17 20:51:19 -08:00
Simon Willison
5e0e440f2c database.execute_write_fn(transaction=True) parameter, closes #2277 2024-02-17 20:28:15 -08:00
Simon Willison
5c64af6936 Upgrade to latest Black, closes #2239 2024-01-30 19:55:26 -08:00
Simon Willison
4284c74bc1
db.execute_isolated_fn() method (#2220)
Closes #2218
2023-12-19 10:51:03 -08:00
Simon Willison
c588a89f26 db.view_exists() method, needed by #1896 2022-11-18 14:16:38 -08:00
Simon Willison
b29ccb59c7 Add test for db.view_names() 2022-11-18 14:13:48 -08:00
Simon Willison
bbf33a7635 Test for bool(results), closes #1832 2022-10-04 21:32:29 -07:00
Simon Willison
fb7e70d5e7 Database(is_mutable=) now defaults to True, closes #1808
Refs https://github.com/simonw/datasette-upload-dbs/issues/6
2022-09-09 09:19:20 -07:00
Simon Willison
4e47a2d894 Fixed bug where tables with a column called n caused 500 errors
Closes #1228
2022-03-18 18:37:54 -07:00
Simon Willison
5010d1359b Fix for test failure caused by SQLite 3.37.0+, closes #1647 2022-03-05 11:46:59 -08:00
Simon Willison
8c401ee0f0 Fixed remaining code and docs for new block=True default, closes #1579 2021-12-23 11:18:20 -08:00
Simon Willison
00a2895cd2 execute_write defaut is now block=True, closes #1579 2021-12-23 11:03:49 -08:00
Simon Willison
5cadc24489 db.execute_write_script() and db.execute_write_many(), closes #1570
Refs #1555
2021-12-18 10:57:22 -08:00
Simon Willison
2e4ba71b53 Optimize create table calls using executescript=True
Refs #1555, #1569
2021-12-18 10:34:15 -08:00
Simon Willison
9e094b7c9d db.execute_write(executescript=True) option, closes #1569 2021-12-18 10:34:15 -08:00
Simon Willison
83bacfa945 Call _prepare_connection() on write connections, closes #1564 2021-12-17 17:58:39 -08:00
Simon Willison
69244a617b Rename city_id to _city_id in fixtures, refs #1525 2021-11-29 22:20:42 -08:00
Simon Willison
c306b696de Correct facet links for columns with a leading underscore, closes #1506 2021-11-13 20:44:54 -08:00
Simon Willison
2e8d924cdc Refactored generated_columns test, no longer in fixtures.db - refs #1391 2021-07-10 12:03:19 -07:00
Simon Willison
7c87532acc New .add_memory_database() method, closes #1247 2021-02-28 20:02:18 -08:00
Simon Willison
6f41c8a2be
--crossdb option for joining across databases (#1232)
* Test for cross-database join, refs #283
* Warn if --crossdb used with more than 10 DBs, refs #283
* latest.datasette.io demo of --crossdb joins, refs #283
* Show attached databases on /_memory page, refs #283
* Documentation for cross-database queries, refs #283
2021-02-18 14:09:12 -08:00
Simon Willison
8919f99c2f Improved .add_database() method design
Closes #1155 - _internal now has a sensible name

Closes #509 - Support opening multiple databases with the same stem
2020-12-22 12:04:18 -08:00
Simon Willison
5e9895c67f Database(memory_name=) for shared in-memory databases, closes #1151 2020-12-17 17:01:18 -08:00
Simon Willison
17cbbb1f7f
generated_columns table in fixtures.py, closes #1119 2020-11-30 16:28:02 -08:00
Simon Willison
461670a0b8
Support for generated columns
* Support for generated columns, closes #1116
* Show SQLite version in pytest report header
* Use table_info() if SQLite < 3.26.0
* Cache sqlite_version() rather than re-calculate every time
* Adjust test_database_page for SQLite 3.26.0 or higher
2020-11-30 13:29:57 -08:00
Simon Willison
dea3c508b3 Revert "Support for generated columns, closes #1116" - it failed CI
This reverts commit 37f87b5e52.
2020-11-30 12:09:32 -08:00
Simon Willison
37f87b5e52 Support for generated columns, closes #1116 2020-11-30 12:01:15 -08:00
Simon Willison
deb0be4ae5 Fix bug where compound foreign keys produced broken links, closes #1098 2020-11-29 11:30:17 -08:00
Simon Willison
e8e0a6f284
Use FTS4 in fixtures
Closes #1081
2020-11-11 16:02:58 -08:00
Simon Willison
e02f6c1300 Tests for db.table_columns() and db.table_column_details() 2020-10-06 14:02:30 -07:00
Simon Willison
a648bb82ba Upgrade to Black 20.8b1, closes #958 2020-09-02 15:24:55 -07:00
Simon Willison
af12f45c2b Documentation and tests for db.is_mutable 2020-08-15 16:27:32 -07:00
Simon Willison
b86f94883b
Don't hang in db.execute_write_fn() if connection fails
Closes #935

Refs https://github.com/simonw/latest-datasette-with-all-plugins/issues/3
2020-08-15 15:35:31 -07:00
Simon Willison
033a1bb22c Removed rogue print() from test 2020-06-05 12:06:43 -07:00
Simon Willison
4d798ca0e3 Added test for db.mtime_ns 2020-05-30 11:17:20 -07:00