mirror of
https://github.com/simonw/datasette.git
synced 2026-10-20 08:17:00 +02:00
d7a2ce91ccSchema watcher:
- Scan coalescing is loop-agnostic: in-flight scans are concurrent.futures
futures any event loop can wait for, taken over only when their loop has
closed or stopped; each refresh() resolves only its own scans. A write on
one loop no longer hangs forever when another loop's scan is in flight.
- At most one polling task across event loops; replaced pollers exit.
- A scan or catalog write that fails transiently (internal database locked,
shared-cache lock, cancellation) re-marks the database and retries it from
later requests and sweeps, with back-off; requests are never failed by it.
Write threads mark schema changes directly (under a lock) instead of
relying on a notification reaching a loop that may have closed.
- add_database() file databases and named in-memory databases use the
default_schema_watch mode (external unless configured). Named in-memory
databases can be external: polled with PRAGMA schema_version on their
write connection. They are not scanned at registration time, which raced
plugins filling them (datasette-copy-to-memory).
- close() no longer persists a fingerprint for an owned database whose file
header shows a schema the catalog has not seen.
- The derived-table permission rule (external-content FTS) trusts the
catalog only when a stat and header read prove it is current; otherwise
it asks the database, and schedules a rescan if the catalog is behind.
Connections and write threads:
- A BaseException from a write or isolated callback is delivered to the
caller as a RuntimeError instead of killing the thread silently.
- conn.close() on a write lease raises ConnectionLeaseError, and the write
thread reopens a connection something else closed.
- Write connections pin PRAGMA recursive_triggers=on (sqlite-utils FTS
triggers rely on it; idle reopen lost the old drift).
- execute_fn() accepts generators that no longer hold the connection or a
cursor.
- --crossdb _memory connections count against max_open_connections, are
reaped, and are discarded when databases are added or removed.
Lifecycle:
- The temp internal database file uses weakref.finalize, not atexit, so
closed or abandoned instances can be garbage collected.
- The read-pool reaper no longer keeps the last pool alive in its frame.
- The scratch directory is loaded (and locked) at the end of __init__, so a
failed construction leaves nothing locked.
- config_dir/scratch is used only if empty or created by Datasette; a second
instance on the same config dir starts without it; sidecar files are never
deleted the first time a directory is opened.
Regression tests: tests/test_connection_review_fixes.py (24 of 27 fail on
|
||
|---|---|---|
| .. | ||
| plugins | ||
| test_templates | ||
| __init__.py | ||
| build_small_spatialite_db.py | ||
| conftest.py | ||
| connection_budget.py | ||
| ext.c | ||
| fixtures.py | ||
| spatialite.db | ||
| test-datasette-load-plugins.sh | ||
| test_actions_sql.py | ||
| test_actor_restriction_bug.py | ||
| test_allowed_many.py | ||
| test_allowed_resources.py | ||
| test_api.py | ||
| test_api_write.py | ||
| test_auth.py | ||
| test_autocomplete.py | ||
| test_background_tasks.py | ||
| test_base_view.py | ||
| test_cli.py | ||
| test_cli_serve_get.py | ||
| test_cli_serve_server.py | ||
| test_column_types.py | ||
| test_config_dir.py | ||
| test_config_permission_rules.py | ||
| test_connection_budget.py | ||
| test_connection_leases.py | ||
| test_connection_review_fixes.py | ||
| test_crossdb.py | ||
| test_csrf_middleware.py | ||
| test_csv.py | ||
| test_custom_pages.py | ||
| test_datasette_https_server.sh | ||
| test_debug_autocomplete.py | ||
| test_default_deny.py | ||
| test_docs.py | ||
| test_docs_plugins.py | ||
| test_error_shape.py | ||
| test_extras.py | ||
| test_facets.py | ||
| test_fd_leak.py | ||
| test_filters.py | ||
| test_fixtures.py | ||
| test_fts_permissions.py | ||
| test_html.py | ||
| test_http_span.py | ||
| test_internal_db.py | ||
| test_internals_database.py | ||
| test_internals_datasette.py | ||
| test_internals_datasette_client.py | ||
| test_internals_request.py | ||
| test_internals_response.py | ||
| test_internals_urls.py | ||
| test_jump.py | ||
| test_label_column_for_table.py | ||
| test_lifespan.py | ||
| test_load_extensions.py | ||
| test_messages.py | ||
| test_multipart.py | ||
| test_numeric_filter_values.py | ||
| test_package.py | ||
| test_permission_endpoints.py | ||
| test_permissions.py | ||
| test_playwright.py | ||
| test_plugins.py | ||
| test_pr76_fts_policy.py | ||
| test_pr76_statistics_policy.py | ||
| test_publish_cloudrun.py | ||
| test_publish_heroku.py | ||
| test_pytest_autoclose_plugin.py | ||
| test_queries.py | ||
| test_read_pool.py | ||
| test_restriction_sql.py | ||
| test_routes.py | ||
| test_schema_endpoints.py | ||
| test_schema_watcher.py | ||
| test_scratch.py | ||
| test_search_tables.py | ||
| test_shutdown.py | ||
| test_spatialite.py | ||
| test_stored_queries.py | ||
| test_success_envelope.py | ||
| test_table_api.py | ||
| test_table_count.py | ||
| test_table_html.py | ||
| test_table_resource_identity.py | ||
| test_tasks_endpoint.py | ||
| test_telemetry.py | ||
| test_telemetry_metrics.py | ||
| test_telemetry_registry.py | ||
| test_telemetry_testing_kit.py | ||
| test_template_context.py | ||
| test_token_handler.py | ||
| test_tracer.py | ||
| test_utils.py | ||
| test_utils_check_callable.py | ||
| test_utils_permissions.py | ||
| test_utils_sql_analysis.py | ||
| test_write_sql_operation_decisions.py | ||
| test_write_thread_idle.py | ||
| test_write_wrapper.py | ||
| utils.py | ||