datasette/datasette
Simon Willison 75298db4ae Optimize database page table listing to avoid scanning all tables
Restructured the table listing logic in get_tables() to check permissions
via SQL first using the permission_resources_sql plugin hook, then only
fetch table counts for tables that will actually be displayed.

Previous implementation called table_counts() for all tables before checking
permissions, which defeated the purpose of the optimization - it still
required scanning every table just to list them.

Changes:
- Modified Database.table_counts() to accept optional 'tables' parameter
  that allows selective counting while preserving caching for immutable DBs
- Rewrote get_tables() in database view to query catalog_tables for table
  names first (cheap operation), use resolve_permissions_from_catalog to
  check permissions in bulk, then only call table_counts() with the subset
  of allowed tables
- Fixed bug in default_permissions.py where query_config could be a string
  instead of dict, causing AttributeError
- Correctly handles table-level 'allow: True' blocks that should bypass
  database-level restrictions when determining privacy status

All 177 permission tests passing.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-17 17:04:05 -07:00
..
publish Use service-specific image ID for Cloud Run deploys, refs #2036 2023-03-08 12:25:55 -08:00
static CSS fix for table headings on mobile, closes #2479 2025-04-21 22:33:34 -07:00
templates Refactor debug templates to use shared JavaScript functions 2025-10-08 21:53:34 -07:00
utils Fix for asyncio.iscoroutinefunction deprecation warnings 2025-10-08 20:32:16 -07:00
views Optimize database page table listing to avoid scanning all tables 2025-10-17 17:04:05 -07:00
__init__.py track_event() mechanism for analytics and plugins 2024-01-31 15:21:40 -08:00
__main__.py Add support for running datasette as a module (#556) 2019-07-11 09:07:44 -07:00
actor_auth_cookie.py Remove python-baseconv dependency, refs #1733, closes #1734 2022-05-02 12:39:06 -07:00
app.py New allowed_resources_sql plugin hook and debug tools (#2505) 2025-10-08 14:27:51 -07:00
blob_renderer.py Rename route match groups for consistency, refs #1667, #1660 2022-03-19 09:52:08 -07:00
cli.py Run CLI coroutines on explicit event loops 2025-10-01 12:59:14 -07:00
database.py Optimize database page table listing to avoid scanning all tables 2025-10-17 17:04:05 -07:00
default_magic_parameters.py Fix datetime.utcnow deprecation warning 2024-03-15 15:32:12 -07:00
default_menu_links.py Move Metadata to --internal database 2024-06-11 09:33:23 -07:00
default_permissions.py Optimize database page table listing to avoid scanning all tables 2025-10-17 17:04:05 -07:00
events.py alter table support for /db/-/create API, refs #2101 2024-02-08 13:36:17 -08:00
facets.py Fix huge performance bug in DateFacet, refs #2407 2024-08-21 14:38:11 -07:00
filters.py Removed units functionality and Pint dependency 2024-08-20 19:03:33 -07:00
forbidden.py Fixed a bunch of unused imports spotted with ruff 2024-02-06 17:27:20 -08:00
handle_exception.py debugger: load 'ipdb' if present 2024-08-20 20:02:35 -07:00
hookspecs.py New allowed_resources_sql plugin hook and debug tools (#2505) 2025-10-08 14:27:51 -07:00
inspect.py Modernize code to Python 3.6+ (#1158) 2020-12-23 09:04:32 -08:00
permissions.py Fixed a bunch of unused imports spotted with ruff 2024-02-06 17:27:20 -08:00
plugins.py DATASETTE_TRACE_PLUGINS setting, closes #2274 2024-02-16 13:00:24 -08:00
renderer.py Move Metadata to --internal database 2024-06-11 09:33:23 -07:00
sql_functions.py _search= queries now correctly escaped, fixes #651 2019-12-29 18:48:30 +00:00
tracer.py Tracer now catches errors, closes #2405 2024-08-21 12:19:18 -07:00
url_builder.py count all rows button on table page, refs #2408 2024-08-21 19:09:25 -07:00
version.py Release 1.0a19 2025-04-21 22:38:53 -07:00