remote/datasette
1
0
Fork 0
mirror of https://github.com/simonw/datasette.git synced 2026-10-19 15:56:58 +02:00
Code Issues Projects Releases Packages Wiki Activity

Measure table-page extras on free-threaded Python 3.15

Browse source 
Runs the table pages on 3.15.0b4t with parallel and sequential extras,
1-8 SQL threads, and 1 or 8 requests in flight. On the GIL build parallel
extras are slower because the SQL worker threads contend for the GIL on
every fetched row. On the free-threaded build that penalty disappears and
throughput under load is 2.2-3.1x higher. Adds a patch that resolves
extras in parallel only when the GIL is disabled, the stress and gather
microbenchmark tools, and the raw results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8mEgRS8swZfYqwqrVMeo5
This commit is contained in:
Claude 2026-10-13 19:40:24 +00:00
commit 07068144f9
No known key found for this signature in database
27 changed files with 3636 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

146
perf/request-response/README.md
View file

@ -312,6 +312,142 @@ What this shows:
monkeypatched in `tools/serve_variant.py`. It is not in `ALL-combined.diff`
and contributes about 3%.
## Free-threaded Python 3.15 and the table-page extras
Table pages resolve about 30 "extras" (count, facets, suggested facets,
display columns and so on) through asyncinject with `parallel=True`, so each
extra is its own asyncio task and several SQL queries are in flight at once.
On the GIL build, making them sequential (`Registry(parallel=False)`) was
16–21% faster. This section asks why, and whether free-threaded Python
changes the answer.
**Setup.** `uv python install 3.15t` gives `3.15.0b4` free-threaded. Every
dependency installed from wheels (pydantic-core, PyYAML, MarkupSafe and
sqlite3 all declare free-threading support), and `sys._is_gil_enabled()`
stays `False` after Datasette starts and serves pages. **The full test
suite passes on the free-threaded build: 2829 passed**, the same as the GIL
build.
The benchmark added a 500,000-row table (`BIGDB=1`), so the count and facet
queries do real SQLite work, plus `CONCURRENCY=8` to keep 8 requests in
flight. Each cell is ms per page, from 5 interleaved rounds, with all
patches applied (`ALL-combined.diff`). Each "parallel / sequential" pair was
measured interleaved in the same run. Different rows come from different
runs, so compare across rows with about ±10% allowance.
One request at a time (latency):
| Build, SQL threads | Extras | Table page | 100-row page | 500k-row page | 500k, 2 facets |
|---|---|---:|---:|---:|---:|
| GIL, 3 (default) | parallel | 36.2 | 27.7 | 37.6 | 429 |
| GIL, 3 (default) | sequential | **30.0** | **21.9** | **30.5** | 429 |
| GIL, 1 | parallel | **28.4** | **21.7** | 31.2 | 433 |
| GIL, 1 | sequential | 29.5 | 25.8 | 34.7 | 427 |
| Free-threaded, 3 | parallel | 33.8 | 26.7 | **36.3** | 419 |
| Free-threaded, 3 | sequential | 33.4 | 26.9 | 40.4 | 437 |
| Free-threaded, 8 | parallel | 35.0 | 28.8 | 36.4 | 419 |
| Free-threaded, 8 | sequential | 29.8 | 30.1 | 38.9 | 440 |
8 requests in flight (ms per page = wall time ÷ pages served):
| Build, SQL threads | Extras | Table page | 100-row page | 500k-row page | 500k, 2 facets |
|---|---|---:|---:|---:|---:|
| GIL, 3 (default) | parallel | 42.0 | 35.1 | 48.1 | 167 |
| GIL, 3 (default) | sequential | 42.0 | 30.9 | 44.5 | 163 |
| GIL, 1 | parallel | 22.3 | 19.4 | 26.4 | 424 |
| GIL, 1 | sequential | 22.5 | 19.5 | 28.0 | 427 |
| Free-threaded, 3 | parallel | 13.4 | 15.7 | 18.6 | 153 |
| Free-threaded, 3 | sequential | 12.4 | 14.9 | 19.6 | 146 |
| Free-threaded, 8 | parallel | 15.2 | 15.9 | **17.2** | **72** |
| Free-threaded, 8 | sequential | 14.9 | 15.1 | 19.7 | 72 |
Without the patches the picture is the same: on the GIL build sequential
extras were 14–16% faster on the table and 500k-row pages, and on the
free-threaded build they were level or 8–13% slower. Under load the
unpatched table page went from 55.9 ms per page (GIL) to 20.3 ms
(free-threaded).
### Why parallel extras were slower on the GIL build
It is GIL contention between the SQL worker threads, not the extras
machinery.
- Python's `sqlite3` releases the GIL around each `sqlite3_step()` and takes
it back for every row. Datasette also sets a Python `text_factory`, which
runs for every TEXT value. A query that fetches rows therefore wants the GIL
constantly.
- When parallel extras put several queries on the 3-thread pool at once,
those threads and the event loop fight over the GIL. Each contended
hand-off is a futex wait plus a context switch, and the same work costs
far more CPU.
- **The 3.15 sampler shows it directly.** In `--mode cpu -a` profiles of the
table page on the GIL build, sampled CPU per page was 57 ms with parallel
extras against 33 ms sequential. The SQL worker function
(`sql_operation_in_thread`, with the C sqlite calls charged to it) cost
14.5 ms per page against 6.9 ms. On the free-threaded build both were about
32 ms.
- **Shrinking the switch interval tenfold barely helps:** sequential still
won by 13% at 0.5 ms, against 17% at the default 5 ms. So the cost is the
per-row hand-offs, not the 5 ms interval.
- `tools/micro_gather.py` isolates it. It runs the same `db.execute()` 10 at
a time with `asyncio.gather` instead of one at a time, and reports CPU per
query relative to sequential:
| `num_sql_threads` | GIL build | Free-threaded |
|---|---:|---:|
| 1 | ×0.57–1.07 | ×0.67–0.78 |
| 3 (default) | **×2.3–5.0** | ×0.62–0.77 |
| 8 | **×3.6–31.5** | ×0.60–0.83 |
With one SQL thread there is no contention, and the GIL penalty
disappears. With 8 threads it collapses: ten concurrent 100-row queries
take 9.2 ms each instead of 0.5 ms. On the free-threaded build, running
queries together is always cheaper per query. Per-query wall time drops
2.3–4.3× with 3 threads.
### What this means for the extras question
- **The sequential-extras win is an artefact of the GIL plus a multi-thread
SQL pool.** On the free-threaded build, parallel and sequential extras are
level within about ±5% on most pages, with no consistent winner: the
500k-row page favours parallel by 7–11%, and with 8 threads the small table
page favours sequential by 15%. With 1 SQL thread on the GIL build,
parallel extras *beat* sequential (sequential is 19% slower on the
100-row page). So
`critic-3-extras-sequential` should not land as written.
- **`patches/extras-parallel-only-without-gil.diff`** is the targeted
version. It resolves extras in parallel only when `sys._is_gil_enabled()`
is false, checked when each registry is first built, so the setting
follows the GIL if a later extension import turns it back on. That gives
the best-measured arm with the default 3 threads on each build. The full
suite passes with it on the GIL build, where its behaviour changes.
- **The bigger lever on GIL builds is the thread pool itself.** One SQL
thread nearly doubled table-page throughput under load (42.0 → 22.3 ms per
page). It removes contention between concurrent *requests*, not just
between one request's extras. But it serialises long SQLite-bound queries:
the 500k-row facet page went from 167 to 424 ms per page. Neither default
suits both workloads. Reducing per-row GIL traffic would help either way:
dropping the Python `text_factory` for a plain `str` with a fallback on
decode errors, the completeness critic's item 8.
- **Free-threading is the largest throughput change measured in this whole
study.** With 8 in flight it served table pages 2.2–3.1× faster than the
GIL build with the default pool. Heavy SQLite queries scaled with threads:
the 500k-row facet page took 72 ms per page with 8 threads, against 167 ms
on the GIL build. The single-request cost is pure-Python overhead: light
requests were 40–70% slower (redirect 94 → 160 µs), and single table pages
were 11–22% slower than the best GIL configuration. It would pay off for busy deployments and cost a little for
quiet ones.
**Correctness under real parallelism.** `tools/ft_stress.py` sent 800
requests with 16 in flight to 8 table pages on the free-threaded build.
The pages covered the fixtures tables, the 500k-row table and a named
in-memory database (the case behind #2189), all with parallel extras. It
compared every response with the first. That was 3,200 responses across both
trees and 3 or 8 SQL threads, with **zero failures**. The GIL-build control
lost one suggested facet in 800 requests: a suggestion query exceeded the
50 ms `facet_suggest_time_limit_ms` under contention. Suggested facets can
vary under load on GIL builds for the same reason.
## Bugs found along the way
Fixed by the patches above:
@ -360,11 +496,11 @@ not adversarially reviewed or carrying a trade-off to decide first:
8–13% of server CPU on light requests.
- **Send `Content-Length` on non-streaming responses.** About 3% end to end.
It also gives clients sizes and progress.
- `critic-3-extras-sequential`: table HTML resolves about 30 "extras" as
concurrent asyncio tasks (asyncinject `parallel=True`). Running them
sequentially measured −18 to −20% on table HTML. This contradicts
`run_sequential` in `table.py` (#2189), but the count and facet queries
may genuinely benefit from overlapping on large tables.
- `critic-3-extras-sequential`: always resolve table-page extras
sequentially. It measured −16 to −21% on GIL builds, but see
[the free-threading section](#free-threaded-python-315-and-the-table-page-extras).
The win comes from GIL contention in the SQL pool and disappears on
free-threaded Python, so prefer `patches/extras-parallel-only-without-gil.diff`.
- `critic-5-request-scoped-introspection-memo`: foreign keys are
introspected 5×, and primary keys and `PRAGMA schema_version` 2×, per
table HTML request. A request-scoped memo plus a one-trip prefetch

36
perf/request-response/patches/extras-parallel-only-without-gil.diff Normal file
View file

@ -0,0 +1,36 @@
diff --git a/datasette/extras.py b/datasette/extras.py
index fb8c2e0..0e2fbe5 100644
--- a/datasette/extras.py
+++ b/datasette/extras.py
@@ -1,4 +1,5 @@
import re
+import sys
from dataclasses import dataclass
from enum import Enum
from typing import ClassVar
@@ -8,6 +9,16 @@ from asyncinject import Registry
from datasette.utils.asgi import BadRequest
+def _resolve_extras_in_parallel():
+ # With the GIL, resolving extras concurrently puts several queries on the
+ # SQL thread pool at once, and the workers then contend for the GIL on
+ # every fetched row - measured at 2-5x the CPU of running them one at a
+ # time. On a free-threaded build concurrent queries run in parallel.
+ # Checked when a registry is built, since importing an extension module
+ # without free-threading support can turn the GIL back on.
+ return not getattr(sys, "_is_gil_enabled", lambda: True)()
+
+
def extra_names_from_request(request):
extra_bits = request.args.getlist("_extra")
extras = set()
@@ -96,7 +107,7 @@ class ExtraRegistry:
def _registry_for_scope(self, scope):
registry = self._scope_registries.get(scope)
if registry is None:
- registry = Registry()
+ registry = Registry(parallel=_resolve_extras_in_parallel())
for cls in self.classes_for_scope(scope):
registry.register(cls().resolve, name=cls.key())
self._scope_registries[scope] = registry

452
perf/request-response/results/free-threading/freethreaded_c1.json Normal file
View file

@ -0,0 +1,452 @@
{
"table_html": {
"base": {
"median_us": 43871.0,
"runs": [
43871.0,
44357.7,
43599.5,
46125.5,
40705.8
],
"status_bytes": [
200,
26552
],
"cpu_us": 44955.6,
"cpu_runs": [
44955.6,
44273.6,
45696.8,
47558.6,
41993.8
]
},
"base_seq": {
"median_us": 43309.3,
"runs": [
44124.8,
43309.3,
45309.1,
37084.8,
34109.4
],
"status_bytes": [
200,
26552
],
"cpu_us": 42346.3,
"cpu_runs": [
42346.3,
42882.1,
44283.5,
36386.3,
34457.4
]
},
"combined": {
"median_us": 33837.3,
"runs": [
36948.0,
29868.1,
37668.5,
33837.3,
31009.3
],
"status_bytes": [
200,
26552
],
"cpu_us": 35728.0,
"cpu_runs": [
38348.6,
31307.0,
37755.8,
35728.0,
32834.8
]
},
"combined_seq": {
"median_us": 33373.5,
"runs": [
36819.7,
34901.3,
25410.0,
31811.8,
33373.5
],
"status_bytes": [
200,
26552
],
"cpu_us": 31976.5,
"cpu_runs": [
36969.7,
34372.7,
25878.1,
30504.9,
31976.5
]
}
},
"table_html_100": {
"base": {
"median_us": 40415.2,
"runs": [
45496.2,
40415.2,
39221.5,
43562.6,
37436.5
],
"status_bytes": [
200,
66669
],
"cpu_us": 42918.7,
"cpu_runs": [
48015.0,
42918.7,
41740.9,
45991.0,
39111.3
]
},
"base_seq": {
"median_us": 43468.4,
"runs": [
40910.3,
45526.4,
46428.0,
40650.6,
43468.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 42619.1,
"cpu_runs": [
40387.7,
43968.0,
45368.3,
39635.3,
42619.1
]
},
"combined": {
"median_us": 26724.4,
"runs": [
29658.8,
27078.3,
26223.9,
26724.4,
24932.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 28701.9,
"cpu_runs": [
30590.0,
29155.1,
28155.5,
28701.9,
27614.9
]
},
"combined_seq": {
"median_us": 26867.7,
"runs": [
30034.6,
34023.5,
22900.1,
26867.7,
21389.8
],
"status_bytes": [
200,
66670
],
"cpu_us": 25400.8,
"cpu_runs": [
29599.6,
32881.4,
22562.8,
25400.8,
21336.0
]
}
},
"big_html": {
"base": {
"median_us": 49135.6,
"runs": [
53062.1,
49135.6,
42865.2,
55607.8,
47956.8
],
"status_bytes": [
200,
86459
],
"cpu_us": 51199.6,
"cpu_runs": [
54519.9,
51199.6,
44761.0,
56466.6,
50410.9
]
},
"base_seq": {
"median_us": 55371.8,
"runs": [
51639.3,
65047.1,
56713.3,
55371.8,
51062.8
],
"status_bytes": [
200,
86459
],
"cpu_us": 53922.4,
"cpu_runs": [
50314.5,
59773.9,
55420.1,
53922.4,
49771.2
]
},
"combined": {
"median_us": 36299.8,
"runs": [
37940.7,
36299.8,
33726.9,
37396.5,
32461.2
],
"status_bytes": [
200,
86459
],
"cpu_us": 36634.8,
"cpu_runs": [
39646.9,
36634.8,
35441.4,
39186.1,
33450.0
]
},
"combined_seq": {
"median_us": 40371.8,
"runs": [
44281.8,
40371.8,
34935.8,
36150.2,
41274.7
],
"status_bytes": [
200,
86459
],
"cpu_us": 38581.7,
"cpu_runs": [
43356.8,
39186.1,
33863.6,
35311.4,
38581.7
]
}
},
"big_facets_html": {
"base": {
"median_us": 436276.1,
"runs": [
439074.1,
435315.7,
436276.1,
444049.6,
431548.3
],
"status_bytes": [
200,
90956
],
"cpu_us": 449273.4,
"cpu_runs": [
453009.3,
449273.4,
449240.9,
456523.5,
444008.0
]
},
"base_seq": {
"median_us": 451279.6,
"runs": [
455576.6,
456216.7,
451060.6,
451033.3,
451279.6
],
"status_bytes": [
200,
90956
],
"cpu_us": 445378.8,
"cpu_runs": [
446964.4,
449458.5,
445378.8,
443124.3,
444085.5
]
},
"combined": {
"median_us": 418904.4,
"runs": [
427159.2,
418904.4,
420496.0,
415503.1,
415558.2
],
"status_bytes": [
200,
87157
],
"cpu_us": 427476.8,
"cpu_runs": [
436970.9,
427476.8,
427571.8,
427279.8,
426346.0
]
},
"combined_seq": {
"median_us": 437213.3,
"runs": [
439106.4,
439541.8,
435953.7,
433532.4,
437213.3
],
"status_bytes": [
200,
87157
],
"cpu_us": 431081.8,
"cpu_runs": [
434441.7,
434451.3,
428971.5,
428830.8,
431081.8
]
}
},
"big_json_extras": {
"base": {
"median_us": 206230.7,
"runs": [
207105.5,
204980.3,
209447.8,
206230.7,
203319.6
],
"status_bytes": [
200,
14468
],
"cpu_us": 214626.2,
"cpu_runs": [
215387.1,
213953.6,
220001.8,
214626.2,
211076.6
]
},
"base_seq": {
"median_us": 220080.8,
"runs": [
219032.3,
223417.9,
219964.5,
220080.8,
228293.9
],
"status_bytes": [
200,
17428
],
"cpu_us": 216250.8,
"cpu_runs": [
216097.5,
217030.6,
216250.8,
215577.1,
223218.7
]
},
"combined": {
"median_us": 207025.7,
"runs": [
207025.7,
210685.9,
209470.6,
200896.8,
199705.0
],
"status_bytes": [
200,
17428
],
"cpu_us": 215221.7,
"cpu_runs": [
215221.7,
217835.4,
217475.4,
208773.7,
205521.3
]
},
"combined_seq": {
"median_us": 215655.5,
"runs": [
216749.0,
211619.4,
213081.2,
215655.5,
217202.8
],
"status_bytes": [
200,
17428
],
"cpu_us": 212518.2,
"cpu_runs": [
213552.7,
208046.7,
210516.6,
212518.2,
212905.8
]
}
}
}

17
perf/request-response/results/free-threading/freethreaded_c1.txt Normal file
View file

@ -0,0 +1,17 @@
endpoint base base_seq combined combined_seq
table_html 43871.0us 43309.3us -1% 33837.3us -23% 33373.5us -24%
table_html_100 40415.2us 43468.4us +8% 26724.4us -34% 26867.7us -34%
big_html 49135.6us 55371.8us +13% 36299.8us -26% 40371.8us -18%
big_facets_html 436276.1us 451279.6us +3% 418904.4us -4% 437213.3us +0%
big_json_extras 206230.7us 220080.8us +7% 207025.7us +0% 215655.5us +5%
CPU time per request (process_time, all threads):
endpoint base base_seq combined combined_seq
table_html 44955.6us 42346.3us -6% 35728.0us -21% 31976.5us -29%
table_html_100 42918.7us 42619.1us -1% 28701.9us -33% 25400.8us -41%
big_html 51199.6us 53922.4us +5% 36634.8us -28% 38581.7us -25%
big_facets_html 449273.4us 445378.8us -1% 427476.8us -5% 431081.8us -4%
big_json_extras 214626.2us 216250.8us +1% 215221.7us +0% 212518.2us -1%
WARNING: status/bytes differ for table_html_100: {'base': (200, 66669), 'base_seq': (200, 66670), 'combined': (200, 66670), 'combined_seq': (200, 66670)}
WARNING: status/bytes differ for big_facets_html: {'base': (200, 90956), 'base_seq': (200, 90956), 'combined': (200, 87157), 'combined_seq': (200, 87157)}
WARNING: status/bytes differ for big_json_extras: {'base': (200, 14468), 'base_seq': (200, 17428), 'combined': (200, 17428), 'combined_seq': (200, 17428)}

452
perf/request-response/results/free-threading/freethreaded_c8.json Normal file
View file

@ -0,0 +1,452 @@
{
"table_html": {
"base": {
"median_us": 20267.3,
"runs": [
17824.2,
18769.6,
22057.6,
24516.7,
20267.3
],
"status_bytes": [
200,
26552
],
"cpu_us": 32450.7,
"cpu_runs": [
31775.3,
29979.2,
35008.3,
37576.5,
32450.7
]
},
"base_seq": {
"median_us": 19539.0,
"runs": [
19639.6,
19108.7,
18292.1,
20280.5,
19539.0
],
"status_bytes": [
200,
26552
],
"cpu_us": 32815.1,
"cpu_runs": [
32119.9,
32815.1,
30286.8,
33823.2,
33970.2
]
},
"combined": {
"median_us": 13410.0,
"runs": [
13410.0,
13056.2,
21626.6,
13048.4,
14432.9
],
"status_bytes": [
200,
26552
],
"cpu_us": 23373.6,
"cpu_runs": [
23210.4,
21937.1,
32830.6,
23373.6,
23411.0
]
},
"combined_seq": {
"median_us": 12370.8,
"runs": [
14722.4,
11785.0,
12370.8,
11908.6,
15040.8
],
"status_bytes": [
200,
26552
],
"cpu_us": 22529.8,
"cpu_runs": [
23879.5,
22529.8,
20463.5,
20546.3,
25056.6
]
}
},
"table_html_100": {
"base": {
"median_us": 25633.7,
"runs": [
24087.7,
30387.1,
26492.7,
25633.7,
24190.7
],
"status_bytes": [
200,
66670
],
"cpu_us": 37758.4,
"cpu_runs": [
36036.3,
44986.8,
37758.4,
37795.1,
36558.7
]
},
"base_seq": {
"median_us": 26993.9,
"runs": [
24664.7,
32562.4,
32042.7,
26993.9,
23133.3
],
"status_bytes": [
200,
66670
],
"cpu_us": 38652.7,
"cpu_runs": [
36417.6,
44246.5,
44041.6,
38652.7,
35637.2
]
},
"combined": {
"median_us": 15667.6,
"runs": [
17574.8,
16396.4,
15667.6,
13536.7,
12991.9
],
"status_bytes": [
200,
66670
],
"cpu_us": 25699.8,
"cpu_runs": [
27439.0,
26509.2,
25699.8,
22217.2,
21951.1
]
},
"combined_seq": {
"median_us": 14860.7,
"runs": [
19064.2,
14860.7,
14254.0,
14243.9,
15110.1
],
"status_bytes": [
200,
66669
],
"cpu_us": 23044.6,
"cpu_runs": [
27186.8,
23044.6,
22320.8,
22918.9,
23502.3
]
}
},
"big_html": {
"base": {
"median_us": 32140.0,
"runs": [
32140.0,
34334.2,
28244.8,
32493.7,
27560.1
],
"status_bytes": [
200,
86459
],
"cpu_us": 47521.9,
"cpu_runs": [
48246.2,
50265.1,
43771.2,
47521.9,
42252.8
]
},
"base_seq": {
"median_us": 30629.3,
"runs": [
29346.8,
32889.4,
30629.3,
31312.6,
30617.6
],
"status_bytes": [
200,
86459
],
"cpu_us": 45540.3,
"cpu_runs": [
44127.0,
49658.2,
46226.1,
44117.0,
45540.3
]
},
"combined": {
"median_us": 18564.9,
"runs": [
19051.5,
18564.9,
16456.9,
18316.0,
20369.0
],
"status_bytes": [
200,
86459
],
"cpu_us": 30912.8,
"cpu_runs": [
30752.6,
30912.8,
27671.4,
31376.5,
31901.7
]
},
"combined_seq": {
"median_us": 19607.1,
"runs": [
26335.5,
18041.0,
19607.1,
19277.8,
19931.5
],
"status_bytes": [
200,
86459
],
"cpu_us": 30503.3,
"cpu_runs": [
41075.8,
29880.9,
30232.3,
30503.3,
30942.2
]
}
},
"big_facets_html": {
"base": {
"median_us": 153071.4,
"runs": [
146930.2,
156584.7,
150042.2,
153071.4,
155761.0
],
"status_bytes": [
200,
90956
],
"cpu_us": 430757.7,
"cpu_runs": [
426345.6,
442366.1,
430757.7,
430266.8,
439000.6
]
},
"base_seq": {
"median_us": 156348.8,
"runs": [
157725.3,
156348.8,
150967.2,
151371.0,
156878.5
],
"status_bytes": [
200,
90956
],
"cpu_us": 436700.9,
"cpu_runs": [
440248.7,
436700.9,
429677.8,
433221.0,
439827.6
]
},
"combined": {
"median_us": 152529.7,
"runs": [
151653.5,
160218.3,
152529.7,
145625.5,
158308.0
],
"status_bytes": [
200,
90956
],
"cpu_us": 416349.0,
"cpu_runs": [
414815.4,
432270.0,
415925.8,
416349.0,
426514.4
]
},
"combined_seq": {
"median_us": 146384.6,
"runs": [
150181.9,
146384.6,
145943.2,
143131.7,
146798.8
],
"status_bytes": [
200,
90956
],
"cpu_us": 418970.1,
"cpu_runs": [
417535.6,
422411.5,
418970.1,
413217.3,
419294.7
]
}
},
"big_json_extras": {
"base": {
"median_us": 69360.6,
"runs": [
67398.6,
69360.6,
67826.1,
70117.1,
70229.0
],
"status_bytes": [
200,
14468
],
"cpu_us": 205427.7,
"cpu_runs": [
201451.8,
208433.4,
204163.1,
207793.2,
205427.7
]
},
"base_seq": {
"median_us": 74573.0,
"runs": [
76157.2,
75542.2,
74573.0,
71768.8,
70216.5
],
"status_bytes": [
200,
14468
],
"cpu_us": 211874.2,
"cpu_runs": [
211874.2,
212395.8,
215329.4,
206496.5,
200192.2
]
},
"combined": {
"median_us": 71468.6,
"runs": [
68738.3,
71647.6,
71824.1,
70178.0,
71468.6
],
"status_bytes": [
200,
14468
],
"cpu_us": 200218.6,
"cpu_runs": [
200218.6,
207698.2,
198219.4,
200121.5,
204259.5
]
},
"combined_seq": {
"median_us": 71755.9,
"runs": [
71755.9,
73599.3,
72803.9,
69703.6,
69141.6
],
"status_bytes": [
200,
14468
],
"cpu_us": 203118.1,
"cpu_runs": [
203118.1,
204780.2,
206865.2,
197732.3,
198824.2
]
}
}
}

15
perf/request-response/results/free-threading/freethreaded_c8.txt Normal file
View file

@ -0,0 +1,15 @@
endpoint base base_seq combined combined_seq
table_html 20267.3us 19539.0us -4% 13410.0us -34% 12370.8us -39%
table_html_100 25633.7us 26993.9us +5% 15667.6us -39% 14860.7us -42%
big_html 32140.0us 30629.3us -5% 18564.9us -42% 19607.1us -39%
big_facets_html 153071.4us 156348.8us +2% 152529.7us -0% 146384.6us -4%
big_json_extras 69360.6us 74573.0us +8% 71468.6us +3% 71755.9us +3%
CPU time per request (process_time, all threads):
endpoint base base_seq combined combined_seq
table_html 32450.7us 32815.1us +1% 23373.6us -28% 22529.8us -31%
table_html_100 37758.4us 38652.7us +2% 25699.8us -32% 23044.6us -39%
big_html 47521.9us 45540.3us -4% 30912.8us -35% 30503.3us -36%
big_facets_html 430757.7us 436700.9us +1% 416349.0us -3% 418970.1us -3%
big_json_extras 205427.7us 211874.2us +3% 200218.6us -3% 203118.1us -1%
WARNING: status/bytes differ for table_html_100: {'base': (200, 66670), 'base_seq': (200, 66670), 'combined': (200, 66670), 'combined_seq': (200, 66669)}

232
perf/request-response/results/free-threading/freethreaded_t8_c1.json Normal file
View file

@ -0,0 +1,232 @@
{
"table_html": {
"combined": {
"median_us": 34975.8,
"runs": [
39357.6,
31788.5,
33903.6,
34975.8,
35074.1
],
"status_bytes": [
200,
26551
],
"cpu_us": 35247.5,
"cpu_runs": [
41014.2,
33136.0,
35247.5,
35037.7,
36566.5
]
},
"combined_seq": {
"median_us": 29762.6,
"runs": [
28785.2,
29762.6,
33021.0,
37197.3,
27900.6
],
"status_bytes": [
200,
26552
],
"cpu_us": 28326.2,
"cpu_runs": [
27729.0,
28326.2,
32473.6,
35277.5,
27930.3
]
}
},
"table_html_100": {
"combined": {
"median_us": 28804.9,
"runs": [
28804.9,
29074.0,
23106.0,
28942.2,
24640.2
],
"status_bytes": [
200,
66670
],
"cpu_us": 29970.0,
"cpu_runs": [
29970.0,
31507.7,
25005.7,
30548.5,
26688.1
]
},
"combined_seq": {
"median_us": 30127.7,
"runs": [
30127.7,
30685.7,
24992.2,
31707.4,
23042.7
],
"status_bytes": [
200,
66670
],
"cpu_us": 29178.3,
"cpu_runs": [
29178.3,
29644.8,
24398.9,
30427.8,
22852.2
]
}
},
"big_html": {
"combined": {
"median_us": 36421.8,
"runs": [
36421.8,
33125.6,
42760.8,
35308.8,
36826.2
],
"status_bytes": [
200,
86458
],
"cpu_us": 36884.2,
"cpu_runs": [
36596.8,
34704.8,
43465.7,
36884.2,
38710.0
]
},
"combined_seq": {
"median_us": 38917.1,
"runs": [
42625.8,
42171.0,
38625.6,
37974.8,
38917.1
],
"status_bytes": [
200,
86459
],
"cpu_us": 38048.2,
"cpu_runs": [
41169.5,
41111.7,
35976.1,
36922.5,
38048.2
]
}
},
"big_facets_html": {
"combined": {
"median_us": 418525.9,
"runs": [
418583.0,
422842.6,
417112.4,
417463.1,
418525.9
],
"status_bytes": [
200,
90956
],
"cpu_us": 427737.0,
"cpu_runs": [
428553.8,
429643.4,
425837.3,
422867.2,
427737.0
]
},
"combined_seq": {
"median_us": 440381.0,
"runs": [
440472.6,
449913.2,
440381.0,
439167.4,
438127.8
],
"status_bytes": [
200,
87157
],
"cpu_us": 433207.1,
"cpu_runs": [
433207.1,
439543.5,
433958.2,
431820.4,
430869.2
]
}
},
"big_json_extras": {
"combined": {
"median_us": 205540.5,
"runs": [
205030.0,
206769.0,
205703.8,
205540.5,
205150.3
],
"status_bytes": [
200,
17428
],
"cpu_us": 212786.6,
"cpu_runs": [
211042.5,
213575.7,
212786.6,
213859.4,
211107.5
]
},
"combined_seq": {
"median_us": 214527.8,
"runs": [
214527.8,
212318.8,
212776.0,
217214.6,
216249.0
],
"status_bytes": [
200,
17428
],
"cpu_us": 211274.8,
"cpu_runs": [
211274.8,
207165.6,
209016.9,
211738.7,
213587.1
]
}
}
}

17
perf/request-response/results/free-threading/freethreaded_t8_c1.txt Normal file
View file

@ -0,0 +1,17 @@
endpoint combined combined_seq
table_html 34975.8us 29762.6us -15%
table_html_100 28804.9us 30127.7us +5%
big_html 36421.8us 38917.1us +7%
big_facets_html 418525.9us 440381.0us +5%
big_json_extras 205540.5us 214527.8us +4%
CPU time per request (process_time, all threads):
endpoint combined combined_seq
table_html 35247.5us 28326.2us -20%
table_html_100 29970.0us 29178.3us -3%
big_html 36884.2us 38048.2us +3%
big_facets_html 427737.0us 433207.1us +1%
big_json_extras 212786.6us 211274.8us -1%
WARNING: status/bytes differ for table_html: {'combined': (200, 26551), 'combined_seq': (200, 26552)}
WARNING: status/bytes differ for big_html: {'combined': (200, 86458), 'combined_seq': (200, 86459)}
WARNING: status/bytes differ for big_facets_html: {'combined': (200, 90956), 'combined_seq': (200, 87157)}

232
perf/request-response/results/free-threading/freethreaded_t8_c8.json Normal file
View file

@ -0,0 +1,232 @@
{
"table_html": {
"combined": {
"median_us": 15238.2,
"runs": [
13778.0,
13864.2,
20202.6,
15238.2,
16372.8
],
"status_bytes": [
200,
26552
],
"cpu_us": 28569.2,
"cpu_runs": [
28362.9,
27726.4,
37427.9,
28569.2,
29171.5
]
},
"combined_seq": {
"median_us": 14914.1,
"runs": [
12911.7,
15927.1,
13358.5,
18344.3,
14914.1
],
"status_bytes": [
200,
26552
],
"cpu_us": 27843.0,
"cpu_runs": [
25295.1,
27843.0,
24582.7,
33385.0,
29404.2
]
}
},
"table_html_100": {
"combined": {
"median_us": 15888.1,
"runs": [
14640.5,
19117.7,
14657.0,
15888.1,
16459.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 26727.6,
"cpu_runs": [
26419.2,
31279.6,
26224.7,
26727.6,
27023.2
]
},
"combined_seq": {
"median_us": 15116.9,
"runs": [
15116.9,
13832.2,
15131.7,
15971.3,
14433.6
],
"status_bytes": [
200,
66670
],
"cpu_us": 26677.5,
"cpu_runs": [
26965.2,
25293.7,
25120.6,
29304.1,
26677.5
]
}
},
"big_html": {
"combined": {
"median_us": 17186.4,
"runs": [
16670.5,
18173.9,
14915.3,
17186.4,
18619.1
],
"status_bytes": [
200,
86459
],
"cpu_us": 32678.5,
"cpu_runs": [
32678.5,
32853.0,
27290.5,
30545.4,
34393.9
]
},
"combined_seq": {
"median_us": 19747.5,
"runs": [
17311.7,
17440.7,
19747.5,
20205.6,
20277.8
],
"status_bytes": [
200,
86459
],
"cpu_us": 34292.1,
"cpu_runs": [
31521.9,
30721.1,
34411.6,
34292.1,
35986.7
]
}
},
"big_facets_html": {
"combined": {
"median_us": 72082.8,
"runs": [
72082.8,
72459.7,
69348.6,
71597.9,
74655.6
],
"status_bytes": [
200,
90956
],
"cpu_us": 246842.7,
"cpu_runs": [
243096.2,
250012.8,
235532.1,
246842.7,
261236.0
]
},
"combined_seq": {
"median_us": 72313.7,
"runs": [
69210.1,
72873.8,
72313.7,
72650.6,
70327.9
],
"status_bytes": [
200,
90956
],
"cpu_us": 256127.4,
"cpu_runs": [
242729.6,
256127.4,
258836.4,
256515.0,
245821.2
]
}
},
"big_json_extras": {
"combined": {
"median_us": 33967.2,
"runs": [
35755.0,
32316.6,
33967.2,
32894.0,
35117.7
],
"status_bytes": [
200,
14468
],
"cpu_us": 122862.7,
"cpu_runs": [
124527.9,
121430.7,
122862.7,
121877.2,
123818.5
]
},
"combined_seq": {
"median_us": 33935.9,
"runs": [
34191.1,
32539.0,
33935.9,
34334.9,
31916.8
],
"status_bytes": [
200,
14468
],
"cpu_us": 123638.9,
"cpu_runs": [
124406.7,
123638.9,
122587.6,
127244.6,
119710.1
]
}
}
}

14
perf/request-response/results/free-threading/freethreaded_t8_c8.txt Normal file
View file

@ -0,0 +1,14 @@
endpoint combined combined_seq
table_html 15238.2us 14914.1us -2%
table_html_100 15888.1us 15116.9us -5%
big_html 17186.4us 19747.5us +15%
big_facets_html 72082.8us 72313.7us +0%
big_json_extras 33967.2us 33935.9us -0%
CPU time per request (process_time, all threads):
endpoint combined combined_seq
table_html 28569.2us 27843.0us -3%
table_html_100 26727.6us 26677.5us -0%
big_html 32678.5us 34292.1us +5%
big_facets_html 246842.7us 256127.4us +4%
big_json_extras 122862.7us 123638.9us +1%

452
perf/request-response/results/free-threading/gil_c1.json Normal file
View file

@ -0,0 +1,452 @@
{
"table_html": {
"base": {
"median_us": 48723.8,
"runs": [
50790.4,
45845.4,
57572.0,
48723.8,
46642.7
],
"status_bytes": [
200,
26552
],
"cpu_us": 47686.8,
"cpu_runs": [
50982.0,
47373.4,
54956.2,
45895.8,
47686.8
]
},
"base_seq": {
"median_us": 40882.9,
"runs": [
38473.5,
40882.9,
41477.6,
47273.5,
40550.4
],
"status_bytes": [
200,
26551
],
"cpu_us": 39163.6,
"cpu_runs": [
36067.8,
38967.5,
39538.3,
45420.3,
39163.6
]
},
"combined": {
"median_us": 36199.3,
"runs": [
34223.4,
36199.3,
38435.8,
37995.0,
30870.5
],
"status_bytes": [
200,
26552
],
"cpu_us": 37599.6,
"cpu_runs": [
35854.8,
37599.6,
40073.6,
38761.6,
33173.2
]
},
"combined_seq": {
"median_us": 30000.0,
"runs": [
30000.0,
24645.3,
30349.3,
32959.8,
22150.9
],
"status_bytes": [
200,
26552
],
"cpu_us": 28937.2,
"cpu_runs": [
28937.2,
24374.0,
29042.4,
31073.7,
22013.7
]
}
},
"table_html_100": {
"base": {
"median_us": 41860.2,
"runs": [
40650.6,
40965.9,
43199.3,
41860.2,
42097.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 42602.9,
"cpu_runs": [
41096.0,
41154.0,
43032.8,
42602.9,
43018.2
]
},
"base_seq": {
"median_us": 40041.4,
"runs": [
41104.0,
42824.9,
36529.9,
40041.4,
37799.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 38197.2,
"cpu_runs": [
39844.9,
41058.4,
35147.9,
38197.2,
35238.7
]
},
"combined": {
"median_us": 27735.3,
"runs": [
30495.3,
27735.3,
26114.5,
26234.1,
31007.3
],
"status_bytes": [
200,
66670
],
"cpu_us": 28789.8,
"cpu_runs": [
29654.9,
28789.8,
26882.1,
27482.8,
32330.0
]
},
"combined_seq": {
"median_us": 21869.3,
"runs": [
25051.0,
27724.0,
21869.3,
21657.3,
21639.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 21476.9,
"cpu_runs": [
23940.0,
26325.5,
21476.9,
20862.0,
21284.6
]
}
},
"big_html": {
"base": {
"median_us": 56635.2,
"runs": [
56635.2,
53610.9,
62632.8,
55964.2,
63027.7
],
"status_bytes": [
200,
86459
],
"cpu_us": 55834.0,
"cpu_runs": [
55275.5,
53372.9,
57477.1,
55834.0,
57775.5
]
},
"base_seq": {
"median_us": 48601.0,
"runs": [
49653.2,
45418.1,
47730.3,
51183.0,
48601.0
],
"status_bytes": [
200,
86459
],
"cpu_us": 47013.7,
"cpu_runs": [
48519.4,
44258.1,
46554.8,
49700.3,
47013.7
]
},
"combined": {
"median_us": 37583.9,
"runs": [
37974.0,
37583.9,
38760.4,
37499.1,
35710.8
],
"status_bytes": [
200,
86459
],
"cpu_us": 37896.8,
"cpu_runs": [
37896.8,
38321.0,
38090.5,
37350.1,
36243.8
]
},
"combined_seq": {
"median_us": 30481.6,
"runs": [
33714.1,
29087.2,
34077.9,
29247.1,
30481.6
],
"status_bytes": [
200,
86459
],
"cpu_us": 27683.0,
"cpu_runs": [
28629.5,
27683.0,
33241.6,
27607.2,
26145.6
]
}
},
"big_facets_html": {
"base": {
"median_us": 444848.9,
"runs": [
454020.9,
448949.4,
443883.6,
438942.3,
444848.9
],
"status_bytes": [
200,
87157
],
"cpu_us": 448225.0,
"cpu_runs": [
452224.0,
451364.1,
446864.1,
443375.6,
448225.0
]
},
"base_seq": {
"median_us": 454491.1,
"runs": [
450282.7,
458516.1,
454491.1,
454496.1,
449258.3
],
"status_bytes": [
200,
87157
],
"cpu_us": 446723.6,
"cpu_runs": [
441079.8,
450381.8,
446723.6,
446852.7,
441442.7
]
},
"combined": {
"median_us": 428737.2,
"runs": [
429004.6,
428974.6,
426955.4,
428737.2,
425167.4
],
"status_bytes": [
200,
90956
],
"cpu_us": 429839.7,
"cpu_runs": [
430390.1,
430767.9,
427927.4,
429839.7,
426904.9
]
},
"combined_seq": {
"median_us": 428745.4,
"runs": [
432549.1,
426932.0,
432569.2,
428745.4,
428613.8
],
"status_bytes": [
200,
90956
],
"cpu_us": 422246.6,
"cpu_runs": [
423334.7,
421065.0,
426357.1,
422246.6,
422074.4
]
}
},
"big_json_extras": {
"base": {
"median_us": 211351.0,
"runs": [
208831.0,
209196.9,
211351.0,
215350.9,
211583.3
],
"status_bytes": [
200,
17428
],
"cpu_us": 212990.4,
"cpu_runs": [
211620.7,
211803.2,
212990.4,
216353.4,
213772.7
]
},
"base_seq": {
"median_us": 218914.8,
"runs": [
216172.4,
218914.8,
225930.5,
221172.0,
205416.3
],
"status_bytes": [
200,
14468
],
"cpu_us": 213888.7,
"cpu_runs": [
213228.9,
213888.7,
217207.5,
217090.4,
202933.4
]
},
"combined": {
"median_us": 206525.9,
"runs": [
211433.9,
204760.5,
206525.9,
205381.9,
206981.9
],
"status_bytes": [
200,
17428
],
"cpu_us": 209886.9,
"cpu_runs": [
214489.4,
208481.0,
209886.9,
210399.3,
209443.8
]
},
"combined_seq": {
"median_us": 211165.8,
"runs": [
211165.8,
207163.3,
215228.9,
218132.1,
208905.4
],
"status_bytes": [
200,
17428
],
"cpu_us": 208036.4,
"cpu_runs": [
208036.4,
204234.6,
211840.1,
214489.4,
204674.2
]
}
}
}

17
perf/request-response/results/free-threading/gil_c1.txt Normal file
View file

@ -0,0 +1,17 @@
endpoint base base_seq combined combined_seq
table_html 48723.8us 40882.9us -16% 36199.3us -26% 30000.0us -38%
table_html_100 41860.2us 40041.4us -4% 27735.3us -34% 21869.3us -48%
big_html 56635.2us 48601.0us -14% 37583.9us -34% 30481.6us -46%
big_facets_html 444848.9us 454491.1us +2% 428737.2us -4% 428745.4us -4%
big_json_extras 211351.0us 218914.8us +4% 206525.9us -2% 211165.8us -0%
CPU time per request (process_time, all threads):
endpoint base base_seq combined combined_seq
table_html 47686.8us 39163.6us -18% 37599.6us -21% 28937.2us -39%
table_html_100 42602.9us 38197.2us -10% 28789.8us -32% 21476.9us -50%
big_html 55834.0us 47013.7us -16% 37896.8us -32% 27683.0us -50%
big_facets_html 448225.0us 446723.6us -0% 429839.7us -4% 422246.6us -6%
big_json_extras 212990.4us 213888.7us +0% 209886.9us -1% 208036.4us -2%
WARNING: status/bytes differ for table_html: {'base': (200, 26552), 'base_seq': (200, 26551), 'combined': (200, 26552), 'combined_seq': (200, 26552)}
WARNING: status/bytes differ for big_facets_html: {'base': (200, 87157), 'base_seq': (200, 87157), 'combined': (200, 90956), 'combined_seq': (200, 90956)}
WARNING: status/bytes differ for big_json_extras: {'base': (200, 17428), 'base_seq': (200, 14468), 'combined': (200, 17428), 'combined_seq': (200, 17428)}

272
perf/request-response/results/free-threading/gil_c1_switch05.json Normal file
View file

@ -0,0 +1,272 @@
{
"table_html": {
"base": {
"median_us": 47049.7,
"runs": [
47049.7,
46425.7,
47301.1,
47061.2,
42555.7
],
"status_bytes": [
200,
26552
],
"cpu_us": 49918.0,
"cpu_runs": [
50191.3,
49918.0,
50648.8,
49487.5,
46456.6
]
},
"base_seq": {
"median_us": 42553.1,
"runs": [
52077.0,
38370.7,
37836.0,
42553.1,
43023.1
],
"status_bytes": [
200,
26551
],
"cpu_us": 40484.2,
"cpu_runs": [
45649.7,
36955.8,
35414.4,
40484.2,
41011.0
]
},
"combined": {
"median_us": 36105.3,
"runs": [
36105.3,
38305.5,
34827.3,
32790.1,
39106.1
],
"status_bytes": [
200,
26552
],
"cpu_us": 40255.3,
"cpu_runs": [
40255.3,
41123.9,
38633.7,
37208.4,
43397.5
]
},
"combined_seq": {
"median_us": 31361.5,
"runs": [
31361.5,
41789.2,
30781.4,
31499.2,
28819.8
],
"status_bytes": [
200,
26551
],
"cpu_us": 29692.9,
"cpu_runs": [
30394.1,
35195.6,
28912.0,
29692.9,
27250.5
]
}
},
"table_html_100": {
"base": {
"median_us": 41961.4,
"runs": [
44938.5,
44527.6,
39676.1,
39305.3,
41961.4
],
"status_bytes": [
200,
66670
],
"cpu_us": 43227.9,
"cpu_runs": [
46353.5,
46367.7,
41432.9,
40762.8,
43227.9
]
},
"base_seq": {
"median_us": 39018.8,
"runs": [
38156.7,
43142.6,
39018.8,
33284.5,
41400.1
],
"status_bytes": [
200,
66670
],
"cpu_us": 36491.6,
"cpu_runs": [
36046.7,
40984.4,
36491.6,
32544.0,
37927.9
]
},
"combined": {
"median_us": 27836.5,
"runs": [
25701.4,
32671.0,
30171.7,
24459.6,
27836.5
],
"status_bytes": [
200,
66670
],
"cpu_us": 30423.6,
"cpu_runs": [
27407.0,
35428.4,
32121.9,
26631.6,
30423.6
]
},
"combined_seq": {
"median_us": 22521.1,
"runs": [
28498.0,
19413.3,
22521.1,
19967.2,
24544.5
],
"status_bytes": [
200,
66669
],
"cpu_us": 22035.4,
"cpu_runs": [
27397.1,
18577.2,
22035.4,
19482.8,
23383.9
]
}
},
"big_html": {
"base": {
"median_us": 52826.7,
"runs": [
51708.3,
61066.4,
52826.7,
48757.2,
56478.2
],
"status_bytes": [
200,
86459
],
"cpu_us": 54046.6,
"cpu_runs": [
51897.2,
57491.1,
54046.6,
49724.6,
56925.6
]
},
"base_seq": {
"median_us": 46792.5,
"runs": [
46792.5,
44794.2,
41622.8,
57385.8,
49247.0
],
"status_bytes": [
200,
86459
],
"cpu_us": 45132.3,
"cpu_runs": [
45132.3,
43553.2,
40293.8,
53773.9,
47973.0
]
},
"combined": {
"median_us": 36168.3,
"runs": [
40736.5,
34933.2,
36168.3,
33658.7,
36533.5
],
"status_bytes": [
200,
86459
],
"cpu_us": 36874.7,
"cpu_runs": [
38282.4,
35281.7,
36874.7,
34799.4,
37397.3
]
},
"combined_seq": {
"median_us": 29164.8,
"runs": [
28197.1,
31520.2,
29658.3,
29017.4,
29164.8
],
"status_bytes": [
200,
86459
],
"cpu_us": 27814.4,
"cpu_runs": [
27159.4,
30883.7,
28193.2,
27696.0,
27814.4
]
}
}
}

12
perf/request-response/results/free-threading/gil_c1_switch05.txt Normal file
View file

@ -0,0 +1,12 @@
endpoint base base_seq combined combined_seq
table_html 47049.7us 42553.1us -10% 36105.3us -23% 31361.5us -33%
table_html_100 41961.4us 39018.8us -7% 27836.5us -34% 22521.1us -46%
big_html 52826.7us 46792.5us -11% 36168.3us -32% 29164.8us -45%
CPU time per request (process_time, all threads):
endpoint base base_seq combined combined_seq
table_html 49918.0us 40484.2us -19% 40255.3us -19% 29692.9us -41%
table_html_100 43227.9us 36491.6us -16% 30423.6us -30% 22035.4us -49%
big_html 54046.6us 45132.3us -16% 36874.7us -32% 27814.4us -49%
WARNING: status/bytes differ for table_html: {'base': (200, 26552), 'base_seq': (200, 26551), 'combined': (200, 26552), 'combined_seq': (200, 26551)}
WARNING: status/bytes differ for table_html_100: {'base': (200, 66670), 'base_seq': (200, 66670), 'combined': (200, 66670), 'combined_seq': (200, 66669)}

452
perf/request-response/results/free-threading/gil_c8.json Normal file
View file

@ -0,0 +1,452 @@
{
"table_html": {
"base": {
"median_us": 55940.3,
"runs": [
52547.5,
55940.3,
55573.9,
60532.1,
65107.0
],
"status_bytes": [
200,
26552
],
"cpu_us": 69646.4,
"cpu_runs": [
67762.4,
69646.4,
64674.1,
75686.1,
80620.1
]
},
"base_seq": {
"median_us": 57221.5,
"runs": [
57221.5,
57475.3,
61431.8,
56320.7,
53716.0
],
"status_bytes": [
200,
26552
],
"cpu_us": 71763.8,
"cpu_runs": [
72855.6,
69082.8,
72765.0,
71763.8,
68527.6
]
},
"combined": {
"median_us": 42007.4,
"runs": [
41242.0,
41809.9,
44001.0,
42007.4,
50248.7
],
"status_bytes": [
200,
26551
],
"cpu_us": 54192.4,
"cpu_runs": [
53640.6,
51816.9,
56274.9,
54192.4,
63779.9
]
},
"combined_seq": {
"median_us": 41961.5,
"runs": [
41583.0,
43633.9,
39824.8,
43556.4,
41961.5
],
"status_bytes": [
200,
26552
],
"cpu_us": 53968.0,
"cpu_runs": [
54178.9,
55610.7,
51810.9,
53968.0,
52991.5
]
}
},
"table_html_100": {
"base": {
"median_us": 48828.2,
"runs": [
49399.9,
48056.2,
45611.6,
48828.2,
52181.3
],
"status_bytes": [
200,
66670
],
"cpu_us": 59403.3,
"cpu_runs": [
62351.7,
59403.3,
57088.3,
59009.5,
64085.7
]
},
"base_seq": {
"median_us": 48199.2,
"runs": [
52568.2,
48199.2,
44522.9,
51853.3,
43791.9
],
"status_bytes": [
200,
66670
],
"cpu_us": 59076.8,
"cpu_runs": [
64654.7,
60839.0,
56672.3,
59076.8,
55093.4
]
},
"combined": {
"median_us": 35097.9,
"runs": [
36016.5,
27888.9,
30085.1,
40975.8,
35097.9
],
"status_bytes": [
200,
66670
],
"cpu_us": 43599.8,
"cpu_runs": [
45543.8,
36395.7,
38298.2,
43599.8,
44906.1
]
},
"combined_seq": {
"median_us": 30931.8,
"runs": [
28806.5,
30335.9,
30931.8,
32509.1,
30968.1
],
"status_bytes": [
200,
66670
],
"cpu_us": 39944.2,
"cpu_runs": [
37869.8,
39427.5,
40023.6,
40962.7,
39944.2
]
}
},
"big_html": {
"base": {
"median_us": 61454.6,
"runs": [
57551.0,
66737.1,
64139.7,
61454.6,
56370.2
],
"status_bytes": [
200,
86459
],
"cpu_us": 77951.9,
"cpu_runs": [
74675.4,
80273.9,
81853.3,
77951.9,
71963.2
]
},
"base_seq": {
"median_us": 62140.5,
"runs": [
62140.5,
70177.8,
64586.3,
61493.5,
60612.7
],
"status_bytes": [
200,
86459
],
"cpu_us": 79762.5,
"cpu_runs": [
79762.5,
87725.0,
81451.1,
78456.7,
75643.6
]
},
"combined": {
"median_us": 48061.2,
"runs": [
49688.3,
48061.2,
44427.5,
41613.7,
49574.9
],
"status_bytes": [
200,
86459
],
"cpu_us": 62633.8,
"cpu_runs": [
64483.2,
62633.8,
58751.6,
55916.2,
64431.8
]
},
"combined_seq": {
"median_us": 44529.8,
"runs": [
44529.8,
48277.2,
46014.6,
43729.6,
39213.0
],
"status_bytes": [
200,
86459
],
"cpu_us": 58350.6,
"cpu_runs": [
58350.6,
58386.7,
60092.1,
57561.5,
51368.1
]
}
},
"big_facets_html": {
"base": {
"median_us": 178489.9,
"runs": [
191860.4,
178580.0,
168271.5,
178489.9,
175507.5
],
"status_bytes": [
200,
87157
],
"cpu_us": 419499.3,
"cpu_runs": [
450984.6,
423417.0,
412754.3,
418658.5,
419499.3
]
},
"base_seq": {
"median_us": 178701.6,
"runs": [
182522.5,
178701.6,
185496.1,
175254.3,
174494.7
],
"status_bytes": [
200,
90956
],
"cpu_us": 420419.2,
"cpu_runs": [
426325.6,
420419.2,
424222.6,
404820.7,
416592.0
]
},
"combined": {
"median_us": 167378.1,
"runs": [
166793.5,
167378.1,
168061.0,
168654.2,
166394.6
],
"status_bytes": [
200,
87156
],
"cpu_us": 412931.2,
"cpu_runs": [
413714.0,
412931.2,
412630.5,
421912.4,
408528.3
]
},
"combined_seq": {
"median_us": 162834.7,
"runs": [
172295.6,
162773.6,
162409.0,
164381.2,
162834.7
],
"status_bytes": [
200,
90956
],
"cpu_us": 411673.2,
"cpu_runs": [
424667.0,
411673.2,
409154.0,
413715.6,
407901.1
]
}
},
"big_json_extras": {
"base": {
"median_us": 84135.0,
"runs": [
84881.3,
82346.0,
84217.2,
84135.0,
78823.1
],
"status_bytes": [
200,
14468
],
"cpu_us": 206358.4,
"cpu_runs": [
207132.9,
203867.6,
206615.1,
206358.4,
202222.6
]
},
"base_seq": {
"median_us": 87880.3,
"runs": [
88366.5,
86494.1,
84976.2,
90209.9,
87880.3
],
"status_bytes": [
200,
14468
],
"cpu_us": 211772.7,
"cpu_runs": [
214620.6,
213271.5,
210253.4,
211772.7,
209899.8
]
},
"combined": {
"median_us": 78701.4,
"runs": [
78388.3,
79047.8,
76115.5,
79460.0,
78701.4
],
"status_bytes": [
200,
14468
],
"cpu_us": 201825.3,
"cpu_runs": [
199094.3,
201825.3,
199660.1,
208587.1,
203242.4
]
},
"combined_seq": {
"median_us": 86615.8,
"runs": [
85198.1,
87417.5,
84012.6,
86615.8,
87444.6
],
"status_bytes": [
200,
14468
],
"cpu_us": 210347.3,
"cpu_runs": [
210347.3,
210348.8,
208112.0,
213249.5,
208049.9
]
}
}
}

16
perf/request-response/results/free-threading/gil_c8.txt Normal file
View file

@ -0,0 +1,16 @@
endpoint base base_seq combined combined_seq
table_html 55940.3us 57221.5us +2% 42007.4us -25% 41961.5us -25%
table_html_100 48828.2us 48199.2us -1% 35097.9us -28% 30931.8us -37%
big_html 61454.6us 62140.5us +1% 48061.2us -22% 44529.8us -28%
big_facets_html 178489.9us 178701.6us +0% 167378.1us -6% 162834.7us -9%
big_json_extras 84135.0us 87880.3us +4% 78701.4us -6% 86615.8us +3%
CPU time per request (process_time, all threads):
endpoint base base_seq combined combined_seq
table_html 69646.4us 71763.8us +3% 54192.4us -22% 53968.0us -23%
table_html_100 59403.3us 59076.8us -1% 43599.8us -27% 39944.2us -33%
big_html 77951.9us 79762.5us +2% 62633.8us -20% 58350.6us -25%
big_facets_html 419499.3us 420419.2us +0% 412931.2us -2% 411673.2us -2%
big_json_extras 206358.4us 211772.7us +3% 201825.3us -2% 210347.3us +2%
WARNING: status/bytes differ for table_html: {'base': (200, 26552), 'base_seq': (200, 26552), 'combined': (200, 26551), 'combined_seq': (200, 26552)}
WARNING: status/bytes differ for big_facets_html: {'base': (200, 87157), 'base_seq': (200, 90956), 'combined': (200, 87156), 'combined_seq': (200, 90956)}

232
perf/request-response/results/free-threading/gil_t1_c1.json Normal file
View file

@ -0,0 +1,232 @@
{
"table_html": {
"combined": {
"median_us": 28380.6,
"runs": [
29604.8,
28380.6,
28713.9,
26036.5,
25111.3
],
"status_bytes": [
200,
26552
],
"cpu_us": 28555.7,
"cpu_runs": [
29967.4,
28555.7,
29364.1,
26764.3,
25672.0
]
},
"combined_seq": {
"median_us": 29486.1,
"runs": [
32714.7,
27689.3,
31458.1,
29486.1,
25572.6
],
"status_bytes": [
200,
26551
],
"cpu_us": 28797.3,
"cpu_runs": [
31597.5,
26890.8,
31207.3,
28797.3,
25294.0
]
}
},
"table_html_100": {
"combined": {
"median_us": 21700.6,
"runs": [
21027.6,
26116.8,
21700.6,
21648.6,
27094.1
],
"status_bytes": [
200,
66670
],
"cpu_us": 21572.8,
"cpu_runs": [
20877.9,
26328.7,
20981.5,
21572.8,
23600.8
]
},
"combined_seq": {
"median_us": 25818.8,
"runs": [
27834.2,
25818.8,
26244.8,
20695.6,
25379.2
],
"status_bytes": [
200,
66670
],
"cpu_us": 25250.2,
"cpu_runs": [
27031.5,
25338.8,
25250.2,
20363.0,
24771.5
]
}
},
"big_html": {
"combined": {
"median_us": 31205.3,
"runs": [
31205.3,
30827.9,
35843.1,
32780.5,
28889.9
],
"status_bytes": [
200,
86458
],
"cpu_us": 31281.3,
"cpu_runs": [
31281.3,
30655.7,
36190.6,
33161.4,
28606.0
]
},
"combined_seq": {
"median_us": 34717.9,
"runs": [
35530.9,
41741.5,
34717.9,
26028.8,
27897.6
],
"status_bytes": [
200,
86459
],
"cpu_us": 33678.1,
"cpu_runs": [
34802.8,
40048.0,
33678.1,
25724.0,
26807.8
]
}
},
"big_facets_html": {
"combined": {
"median_us": 433353.3,
"runs": [
433353.3,
439706.3,
430854.3,
429270.6,
435022.0
],
"status_bytes": [
200,
87157
],
"cpu_us": 429131.9,
"cpu_runs": [
429538.4,
433562.8,
426662.9,
424620.5,
429131.9
]
},
"combined_seq": {
"median_us": 426759.4,
"runs": [
426759.4,
441141.0,
429590.0,
423465.7,
423516.0
],
"status_bytes": [
200,
87157
],
"cpu_us": 421007.3,
"cpu_runs": [
421007.3,
429668.7,
423765.4,
419886.6,
415736.5
]
}
},
"big_json_extras": {
"combined": {
"median_us": 215561.3,
"runs": [
219965.5,
216920.5,
211912.3,
215561.3,
205896.5
],
"status_bytes": [
200,
14468
],
"cpu_us": 212128.5,
"cpu_runs": [
217216.8,
213794.9,
209027.3,
212128.5,
204000.8
]
},
"combined_seq": {
"median_us": 213088.5,
"runs": [
213088.5,
215255.6,
212942.0,
214429.0,
210378.1
],
"status_bytes": [
200,
14468
],
"cpu_us": 210569.5,
"cpu_runs": [
210569.5,
211826.2,
206743.2,
211524.9,
205986.2
]
}
}
}

16
perf/request-response/results/free-threading/gil_t1_c1.txt Normal file
View file

@ -0,0 +1,16 @@
endpoint combined combined_seq
table_html 28380.6us 29486.1us +4%
table_html_100 21700.6us 25818.8us +19%
big_html 31205.3us 34717.9us +11%
big_facets_html 433353.3us 426759.4us -2%
big_json_extras 215561.3us 213088.5us -1%
CPU time per request (process_time, all threads):
endpoint combined combined_seq
table_html 28555.7us 28797.3us +1%
table_html_100 21572.8us 25250.2us +17%
big_html 31281.3us 33678.1us +8%
big_facets_html 429131.9us 421007.3us -2%
big_json_extras 212128.5us 210569.5us -1%
WARNING: status/bytes differ for table_html: {'combined': (200, 26552), 'combined_seq': (200, 26551)}
WARNING: status/bytes differ for big_html: {'combined': (200, 86458), 'combined_seq': (200, 86459)}

232
perf/request-response/results/free-threading/gil_t1_c8.json Normal file
View file

@ -0,0 +1,232 @@
{
"table_html": {
"combined": {
"median_us": 22328.7,
"runs": [
22464.6,
22328.7,
21807.7,
21018.9,
23935.7
],
"status_bytes": [
200,
26552
],
"cpu_us": 24085.8,
"cpu_runs": [
24099.2,
24085.8,
23783.8,
22929.2,
25875.9
]
},
"combined_seq": {
"median_us": 22450.9,
"runs": [
24163.2,
24314.3,
20886.4,
21208.2,
22450.9
],
"status_bytes": [
200,
26552
],
"cpu_us": 24089.1,
"cpu_runs": [
26343.1,
25961.5,
22692.4,
22539.7,
24089.1
]
}
},
"table_html_100": {
"combined": {
"median_us": 19396.8,
"runs": [
24703.1,
18095.2,
21489.9,
17740.2,
19396.8
],
"status_bytes": [
200,
66670
],
"cpu_us": 20022.4,
"cpu_runs": [
21707.1,
18865.2,
22726.6,
18657.1,
20022.4
]
},
"combined_seq": {
"median_us": 19498.6,
"runs": [
19498.6,
25449.6,
20071.0,
19204.0,
18264.5
],
"status_bytes": [
200,
66670
],
"cpu_us": 20464.8,
"cpu_runs": [
20464.8,
26445.6,
20791.9,
19619.2,
19200.9
]
}
},
"big_html": {
"combined": {
"median_us": 26366.1,
"runs": [
24873.8,
33707.3,
30098.4,
26366.1,
25196.7
],
"status_bytes": [
200,
86459
],
"cpu_us": 26936.3,
"cpu_runs": [
25972.7,
34981.6,
31370.5,
26936.3,
26299.4
]
},
"combined_seq": {
"median_us": 28049.5,
"runs": [
23635.3,
27076.9,
28049.5,
28542.3,
29020.7
],
"status_bytes": [
200,
86459
],
"cpu_us": 28582.7,
"cpu_runs": [
24791.2,
28466.5,
28582.7,
29773.6,
29967.5
]
}
},
"big_facets_html": {
"combined": {
"median_us": 423658.3,
"runs": [
421112.8,
427362.2,
427048.6,
423658.3,
420737.1
],
"status_bytes": [
200,
90956
],
"cpu_us": 416547.4,
"cpu_runs": [
416099.3,
421854.8,
423415.3,
416547.4,
415221.4
]
},
"combined_seq": {
"median_us": 427133.9,
"runs": [
426996.1,
427133.9,
423135.8,
427804.0,
428807.4
],
"status_bytes": [
200,
87156
],
"cpu_us": 422053.7,
"cpu_runs": [
422053.7,
425340.5,
419642.6,
419878.2,
423186.9
]
}
},
"big_json_extras": {
"combined": {
"median_us": 208527.6,
"runs": [
206048.7,
202818.6,
208527.6,
209078.5,
211001.9
],
"status_bytes": [
200,
14468
],
"cpu_us": 206348.6,
"cpu_runs": [
204936.5,
201273.1,
206348.6,
207110.0,
209876.1
]
},
"combined_seq": {
"median_us": 206522.4,
"runs": [
203244.5,
203901.4,
214114.8,
206522.4,
206654.5
],
"status_bytes": [
200,
14468
],
"cpu_us": 203089.0,
"cpu_runs": [
201692.4,
203089.0,
210444.4,
203013.2,
203868.3
]
}
}
}

15
perf/request-response/results/free-threading/gil_t1_c8.txt Normal file
View file

@ -0,0 +1,15 @@
endpoint combined combined_seq
table_html 22328.7us 22450.9us +1%
table_html_100 19396.8us 19498.6us +1%
big_html 26366.1us 28049.5us +6%
big_facets_html 423658.3us 427133.9us +1%
big_json_extras 208527.6us 206522.4us -1%
CPU time per request (process_time, all threads):
endpoint combined combined_seq
table_html 24085.8us 24089.1us +0%
table_html_100 20022.4us 20464.8us +2%
big_html 26936.3us 28582.7us +6%
big_facets_html 416547.4us 422053.7us +1%
big_json_extras 206348.6us 203089.0us -2%
WARNING: status/bytes differ for big_facets_html: {'combined': (200, 90956), 'combined_seq': (200, 87156)}

29
perf/request-response/results/free-threading/micro_gather.txt Normal file
View file

@ -0,0 +1,29 @@
tools/micro_gather.py: Database.execute() 300 times, either one at a time
("sequential") or 10 at a time with asyncio.gather ("gather10").
Per-query wall time and CPU time (process_time, all threads), medians of 5.
Python 3.15.0b4 (GIL) and 3.15.0b4t (free-threaded), combined patch tree.
GIL on, num_sql_threads=3
1 row sequential wall 167us cpu 165us | gather10 wall 299us cpu 376us (cpu x2.28)
100 text rows sequential wall 500us cpu 470us | gather10 wall 1723us cpu 2311us (cpu x4.91)
introspection sequential wall 309us cpu 300us | gather10 wall 1112us cpu 1493us (cpu x4.98)
GIL off, num_sql_threads=3
1 row sequential wall 133us cpu 148us | gather10 wall 58us cpu 110us (cpu x0.75)
100 text rows sequential wall 330us cpu 323us | gather10 wall 90us cpu 249us (cpu x0.77)
introspection sequential wall 320us cpu 303us | gather10 wall 75us cpu 188us (cpu x0.62)
GIL on, num_sql_threads=1
1 row sequential wall 177us cpu 177us | gather10 wall 94us cpu 101us (cpu x0.57)
100 text rows sequential wall 355us cpu 347us | gather10 wall 362us cpu 370us (cpu x1.07)
introspection sequential wall 321us cpu 312us | gather10 wall 321us cpu 331us (cpu x1.06)
GIL off, num_sql_threads=1
1 row sequential wall 174us cpu 178us | gather10 wall 94us cpu 134us (cpu x0.76)
100 text rows sequential wall 366us cpu 386us | gather10 wall 211us cpu 260us (cpu x0.67)
introspection sequential wall 243us cpu 251us | gather10 wall 145us cpu 197us (cpu x0.78)
GIL on, num_sql_threads=8
1 row sequential wall 155us cpu 152us | gather10 wall 395us cpu 551us (cpu x3.61)
100 text rows sequential wall 496us cpu 473us | gather10 wall 9218us cpu 14903us (cpu x31.50)
introspection sequential wall 288us cpu 273us | gather10 wall 4853us cpu 7475us (cpu x27.35)
GIL off, num_sql_threads=8
1 row sequential wall 142us cpu 152us | gather10 wall 63us cpu 120us (cpu x0.79)
100 text rows sequential wall 508us cpu 476us | gather10 wall 150us cpu 394us (cpu x0.83)
introspection sequential wall 409us cpu 384us | gather10 wall 98us cpu 229us (cpu x0.60)

13
perf/request-response/results/free-threading/stress.txt Normal file
View file

@ -0,0 +1,13 @@
tools/ft_stress.py --concurrency 16 --requests 800: 100 requests each to 8
table pages (fixtures, a 500k-row table, and a named in-memory database with
parallel extras), every response compared with the first one (status only
for the 500k-row pages, whose facet suggestions depend on time limits).
free-threaded, cf113a5, 3 SQL threads 800/800 ok, GIL stayed off
free-threaded, all patches, 3 SQL threads 800/800 ok, GIL stayed off
free-threaded, all patches, 8 SQL threads 800/800 ok, GIL stayed off
free-threaded, cf113a5, 8 SQL threads 800/800 ok, GIL stayed off
GIL build, all patches, 3 SQL threads (run 1) 799/800 - table_html_100 lost
its "pk2" suggested facet: that suggestion query went over the 50 ms
facet_suggest_time_limit_ms under GIL contention
GIL build, all patches, 3 SQL threads (run 2) 800/800 ok

5
perf/request-response/tools/.gitignore vendored Normal file
View file

@ -0,0 +1,5 @@
*.db
__pycache__/
server_*.log
server_*.pid
final_results_*.jsonl

89
perf/request-response/tools/bench_asgi.py
View file

@ -4,8 +4,15 @@ profiles show only Datasette's request/response machinery.
Run it with the source tree you want to measure first on PYTHONPATH, e.g.
PYTHONPATH=/path/to/datasette python bench_asgi.py time --n 1000
fixtures.db is generated next to this script on first run. Set IMMUTABLE=1
to open it as an immutable database, NUM_SQL_THREADS to change the pool.
fixtures.db (and, with BIGDB=1, a 500,000-row bigdb.db) are generated next
to this script on first run.
Environment variables:
IMMUTABLE=1 open the databases as immutable
NUM_SQL_THREADS=N size of Datasette's SQL thread pool (default 3)
BIGDB=1 also attach bigdb.db, for the big_* endpoints
CONCURRENCY=N time mode: keep N requests in flight (throughput)
SWITCH_INTERVAL=S sys.setswitchinterval(S) before running
Usage:
python bench_asgi.py time [--endpoints a,b] [--n 2000] [--rounds 5]
@ -32,6 +39,36 @@ if not os.path.exists(FIXTURES):
write_fixture_database(FIXTURES)
def write_bigdb(path, rows=500_000):
import random
import sqlite3
rnd = random.Random(1)
conn = sqlite3.connect(path)
conn.execute(
"create table big (id integer primary key, category text, city text,"
" state text, score integer, name text, created text)"
)
conn.executemany(
"insert into big values (?, ?, ?, ?, ?, ?, ?)",
(
(
i,
f"category-{rnd.randrange(12)}",
f"city-{rnd.randrange(300)}",
f"S{rnd.randrange(50):02d}",
rnd.randint(0, 1000),
f"name {rnd.random():.8f}",
f"2026-{rnd.randint(1, 12):02d}-{rnd.randint(1, 28):02d}",
)
for i in range(1, rows + 1)
),
)
conn.commit()
conn.close()
ENDPOINTS = {
# name: (method, path, query_string, extra headers, needs actor cookie)
"versions_json": ("GET", "/-/versions.json", "", [], False),
@ -71,6 +108,15 @@ ENDPOINTS = {
[],
False,
),
"big_html": ("GET", "/bigdb/big", "", [], False),
"big_facets_html": ("GET", "/bigdb/big", "_facet=category&_facet=city", [], False),
"big_json_extras": (
"GET",
"/bigdb/big.json",
"_facet=category&_extra=count&_extra=facet_results&_extra=suggested_facets",
[],
False,
),
}
BROWSER_HEADERS = [
@ -85,10 +131,20 @@ BROWSER_HEADERS = [
]
if os.environ.get("SWITCH_INTERVAL"):
sys.setswitchinterval(float(os.environ["SWITCH_INTERVAL"]))
def make_ds():
files = [FIXTURES]
if os.environ.get("BIGDB"):
bigdb = os.path.join(HERE, "bigdb.db")
if not os.path.exists(bigdb):
write_bigdb(bigdb)
files.append(bigdb)
return Datasette(
[] if os.environ.get("IMMUTABLE") else [FIXTURES],
immutables=[FIXTURES] if os.environ.get("IMMUTABLE") else [],
[] if os.environ.get("IMMUTABLE") else files,
immutables=files if os.environ.get("IMMUTABLE") else [],
settings={"num_sql_threads": int(os.environ.get("NUM_SQL_THREADS", "3"))},
)
@ -172,16 +228,33 @@ async def run_time(endpoints, n, rounds):
for _ in range(min(200, n)):
await d.request(name)
per_round = []
for _ in range(rounds):
t0 = time.perf_counter()
for _ in range(n):
concurrency = int(os.environ.get("CONCURRENCY", "1"))
async def worker(count, name=name):
for _ in range(count):
await d.request(name)
per_round.append((time.perf_counter() - t0) / n * 1e6)
cpu_round = []
for _ in range(rounds):
c0 = time.process_time()
t0 = time.perf_counter()
if concurrency == 1:
for _ in range(n):
await d.request(name)
else:
# Throughput: wall time per request with this many in flight
await asyncio.gather(
*(worker(n // concurrency) for _ in range(concurrency))
)
done = n if concurrency == 1 else (n // concurrency) * concurrency
per_round.append((time.perf_counter() - t0) / done * 1e6)
cpu_round.append((time.process_time() - c0) / done * 1e6)
results[name] = {
"status": status,
"bytes": len(body),
"median_us": round(statistics.median(per_round), 1),
"min_us": round(min(per_round), 1),
"cpu_us": round(statistics.median(cpu_round), 1),
}
print(
f"{name:22s} {status} {len(body):7d}B median {results[name]['median_us']:8.1f}us min {results[name]['min_us']:8.1f}us",

114
perf/request-response/tools/ft_stress.py Normal file
View file

@ -0,0 +1,114 @@
"""
Fire many concurrent table-page requests (with parallel extras) at one
Datasette instance and check every response: status 200 and a body that
matches the first response once timing fields are removed.
PYTHONPATH=tree python ft_stress.py --concurrency 16 --requests 400
"""
import argparse
import asyncio
import collections
import os
import re
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
os.environ.setdefault("BIGDB", "1")
import bench_asgi
TIMING = re.compile(rb"(\d+(\.\d+)?\s*ms|\"query_ms\":\s*[\d.e-]+|took [\d.]+ms)")
ENDPOINTS = [
"table_html",
"table_html_100",
"big_html",
"big_json_extras",
"row_json",
"table_json",
"mem_html",
"mem_json_extras",
]
bench_asgi.ENDPOINTS["mem_html"] = ("GET", "/memtest/t", "", [], False)
bench_asgi.ENDPOINTS["mem_json_extras"] = (
"GET",
"/memtest/t.json",
"_facet=cat&_extra=count&_extra=facet_results&_extra=suggested_facets",
[],
False,
)
def normalize(body):
return TIMING.sub(b"T", body)
async def main(concurrency, total):
ds = bench_asgi.make_ds()
# Named in-memory database: the case behind issue #2189
mem = ds.add_memory_database("memtest")
await mem.execute_write(
"create table t (id integer primary key, name text, cat text)"
)
await mem.execute_write_many(
"insert into t (name, cat) values (?, ?)",
[(f"name {i}", f"cat {i % 7}") for i in range(2000)],
)
d = bench_asgi.Driver(ds)
await d.setup()
expected = {}
for name in ENDPOINTS:
status, _, body = await d.request(name)
expected[name] = (status, normalize(body))
outcomes = collections.Counter()
mismatches = []
queue = asyncio.Queue()
for i in range(total):
queue.put_nowait(ENDPOINTS[i % len(ENDPOINTS)])
async def worker():
while not queue.empty():
name = queue.get_nowait()
try:
status, _, body = await d.request(name)
except Exception as ex: # noqa: BLE001 - report every failure
outcomes[f"{name}: exception {type(ex).__name__}: {ex}"] += 1
continue
# Facet suggestions on the 500k-row table depend on query time
# limits, so only the status is compared for those pages
if name.startswith("big_") and status == expected[name][0] or (status, normalize(body)) == expected[name]:
outcomes[f"{name}: ok"] += 1
else:
outcomes[f"{name}: MISMATCH status={status}"] += 1
if len(mismatches) < 3:
mismatches.append((name, status, body[:300]))
import difflib
a = expected[name][1].decode("utf-8", "replace").splitlines()
b = normalize(body).decode("utf-8", "replace").splitlines()
print(
"DIFF",
name,
"\n".join(
list(difflib.unified_diff(a, b, lineterm="", n=1))[:40]
),
)
await asyncio.gather(*(worker() for _ in range(concurrency)))
import sys as _sys
print("gil enabled:", getattr(_sys, "_is_gil_enabled", lambda: True)())
for k, v in sorted(outcomes.items()):
print(f" {v:5d} {k}")
for m in mismatches:
print("MISMATCH SAMPLE", m)
bad = sum(v for k, v in outcomes.items() if not k.endswith(": ok"))
print("FAILURES:", bad)
if __name__ == "__main__":
p = argparse.ArgumentParser()
p.add_argument("--concurrency", type=int, default=16)
p.add_argument("--requests", type=int, default=400)
a = p.parse_args()
asyncio.run(main(a.concurrency, a.requests))

18
perf/request-response/tools/measure_ab.py
View file

@ -60,6 +60,7 @@ def main():
trees = [t.split("=", 1) for t in args.tree]
endpoints = args.endpoints.split(",")
samples = {name: {ep: [] for ep in endpoints} for name, _ in trees}
cpu = {name: {ep: [] for ep in endpoints} for name, _ in trees}
statuses = {name: {} for name, _ in trees}
for r in range(args.rounds):
order = trees[r % len(trees) :] + trees[: r % len(trees)]
@ -67,6 +68,8 @@ def main():
data = run_once(path, endpoints, args.n, {})
for ep in endpoints:
samples[name][ep].append(data[ep]["median_us"])
if "cpu_us" in data[ep]:
cpu[name][ep].append(data[ep]["cpu_us"])
statuses[name][ep] = (data[ep]["status"], data[ep]["bytes"])
print(f"round {r + 1}/{args.rounds} done", file=sys.stderr, flush=True)
result = {}
@ -83,12 +86,27 @@ def main():
"median_us": med,
"runs": samples[name][ep],
"status_bytes": statuses[name][ep],
"cpu_us": statistics.median(cpu[name][ep]) if cpu[name][ep] else None,
"cpu_runs": cpu[name][ep],
}
if name == base_name:
row += f"{med:14.1f}us"
else:
row += f"{med:9.1f}us {100 * (med - base) / base:+5.0f}%"
print(row)
if any(cpu[n][e] for n, _ in trees for e in endpoints):
print("\nCPU time per request (process_time, all threads):")
print(header)
for ep in endpoints:
row = f"{ep:22s}"
base = statistics.median(cpu[base_name][ep])
for name, _ in trees:
med = statistics.median(cpu[name][ep])
if name == base_name:
row += f"{med:14.1f}us"
else:
row += f"{med:9.1f}us {100 * (med - base) / base:+5.0f}%"
print(row)
for ep in endpoints:
sb = {name: statuses[name][ep] for name, _ in trees}
if len(set(sb.values())) > 1:

53
perf/request-response/tools/micro_gather.py Normal file
View file

@ -0,0 +1,53 @@
"""db.execute() run one at a time vs. 10 at a time with asyncio.gather."""
import asyncio
import os
import statistics
import sys
import time
from datasette.app import Datasette
HERE = os.path.dirname(os.path.abspath(__file__)) # run bench_asgi.py once first
QUERIES = {
"1 row": "select 1",
"100 text rows": "select pk1, pk2, pk3, content from compound_three_primary_keys limit 100",
"introspection": "select name, type, sql from sqlite_master",
}
async def main():
ds = Datasette(
[os.path.join(HERE, "fixtures.db")],
settings={"num_sql_threads": int(os.environ.get("SQL_THREADS", "3"))},
)
await ds.invoke_startup()
db = ds.get_database("fixtures")
gil = getattr(sys, "_is_gil_enabled", lambda: True)()
print(
f"GIL {'on' if gil else 'off'}, num_sql_threads={os.environ.get('SQL_THREADS', '3')}"
)
for label, sql in QUERIES.items():
for _ in range(50):
await db.execute(sql)
res = {}
for mode in ("sequential", "gather10"):
walls, cpus = [], []
for _ in range(5):
c0, t0 = time.process_time(), time.perf_counter()
if mode == "sequential":
for _ in range(300):
await db.execute(sql)
else:
for _ in range(30):
await asyncio.gather(*(db.execute(sql) for _ in range(10)))
walls.append((time.perf_counter() - t0) / 300 * 1e6)
cpus.append((time.process_time() - c0) / 300 * 1e6)
res[mode] = (statistics.median(walls), statistics.median(cpus))
s, g = res["sequential"], res["gather10"]
print(
f" {label:14s} sequential wall {s[0]:6.0f}us cpu {s[1]:6.0f}us | gather10 wall {g[0]:6.0f}us cpu {g[1]:6.0f}us (cpu x{g[1] / s[1]:.2f})"
)
asyncio.run(main())