diff --git a/perf/request-response/README.md b/perf/request-response/README.md index faed686b..857974c7 100644 --- a/perf/request-response/README.md +++ b/perf/request-response/README.md @@ -312,6 +312,142 @@ What this shows: monkeypatched in `tools/serve_variant.py`. It is not in `ALL-combined.diff` and contributes about 3%. +## Free-threaded Python 3.15 and the table-page extras + +Table pages resolve about 30 "extras" (count, facets, suggested facets, +display columns and so on) through asyncinject with `parallel=True`, so each +extra is its own asyncio task and several SQL queries are in flight at once. +On the GIL build, making them sequential (`Registry(parallel=False)`) was +16–21% faster. This section asks why, and whether free-threaded Python +changes the answer. + +**Setup.** `uv python install 3.15t` gives `3.15.0b4` free-threaded. Every +dependency installed from wheels (pydantic-core, PyYAML, MarkupSafe and +sqlite3 all declare free-threading support), and `sys._is_gil_enabled()` +stays `False` after Datasette starts and serves pages. **The full test +suite passes on the free-threaded build: 2829 passed**, the same as the GIL +build. + +The benchmark added a 500,000-row table (`BIGDB=1`), so the count and facet +queries do real SQLite work, plus `CONCURRENCY=8` to keep 8 requests in +flight. Each cell is ms per page, from 5 interleaved rounds, with all +patches applied (`ALL-combined.diff`). Each "parallel / sequential" pair was +measured interleaved in the same run. Different rows come from different +runs, so compare across rows with about ±10% allowance. + +One request at a time (latency): + +| Build, SQL threads | Extras | Table page | 100-row page | 500k-row page | 500k, 2 facets | +|---|---|---:|---:|---:|---:| +| GIL, 3 (default) | parallel | 36.2 | 27.7 | 37.6 | 429 | +| GIL, 3 (default) | sequential | **30.0** | **21.9** | **30.5** | 429 | +| GIL, 1 | parallel | **28.4** | **21.7** | 31.2 | 433 | +| GIL, 1 | sequential | 29.5 | 25.8 | 34.7 | 427 | +| Free-threaded, 3 | parallel | 33.8 | 26.7 | **36.3** | 419 | +| Free-threaded, 3 | sequential | 33.4 | 26.9 | 40.4 | 437 | +| Free-threaded, 8 | parallel | 35.0 | 28.8 | 36.4 | 419 | +| Free-threaded, 8 | sequential | 29.8 | 30.1 | 38.9 | 440 | + +8 requests in flight (ms per page = wall time ÷ pages served): + +| Build, SQL threads | Extras | Table page | 100-row page | 500k-row page | 500k, 2 facets | +|---|---|---:|---:|---:|---:| +| GIL, 3 (default) | parallel | 42.0 | 35.1 | 48.1 | 167 | +| GIL, 3 (default) | sequential | 42.0 | 30.9 | 44.5 | 163 | +| GIL, 1 | parallel | 22.3 | 19.4 | 26.4 | 424 | +| GIL, 1 | sequential | 22.5 | 19.5 | 28.0 | 427 | +| Free-threaded, 3 | parallel | 13.4 | 15.7 | 18.6 | 153 | +| Free-threaded, 3 | sequential | 12.4 | 14.9 | 19.6 | 146 | +| Free-threaded, 8 | parallel | 15.2 | 15.9 | **17.2** | **72** | +| Free-threaded, 8 | sequential | 14.9 | 15.1 | 19.7 | 72 | + +Without the patches the picture is the same: on the GIL build sequential +extras were 14–16% faster on the table and 500k-row pages, and on the +free-threaded build they were level or 8–13% slower. Under load the +unpatched table page went from 55.9 ms per page (GIL) to 20.3 ms +(free-threaded). + +### Why parallel extras were slower on the GIL build + +It is GIL contention between the SQL worker threads, not the extras +machinery. + +- Python's `sqlite3` releases the GIL around each `sqlite3_step()` and takes + it back for every row. Datasette also sets a Python `text_factory`, which + runs for every TEXT value. A query that fetches rows therefore wants the GIL + constantly. +- When parallel extras put several queries on the 3-thread pool at once, + those threads and the event loop fight over the GIL. Each contended + hand-off is a futex wait plus a context switch, and the same work costs + far more CPU. +- **The 3.15 sampler shows it directly.** In `--mode cpu -a` profiles of the + table page on the GIL build, sampled CPU per page was 57 ms with parallel + extras against 33 ms sequential. The SQL worker function + (`sql_operation_in_thread`, with the C sqlite calls charged to it) cost + 14.5 ms per page against 6.9 ms. On the free-threaded build both were about + 32 ms. +- **Shrinking the switch interval tenfold barely helps:** sequential still + won by 13% at 0.5 ms, against 17% at the default 5 ms. So the cost is the + per-row hand-offs, not the 5 ms interval. +- `tools/micro_gather.py` isolates it. It runs the same `db.execute()` 10 at + a time with `asyncio.gather` instead of one at a time, and reports CPU per + query relative to sequential: + +| `num_sql_threads` | GIL build | Free-threaded | +|---|---:|---:| +| 1 | ×0.57–1.07 | ×0.67–0.78 | +| 3 (default) | **×2.3–5.0** | ×0.62–0.77 | +| 8 | **×3.6–31.5** | ×0.60–0.83 | + + With one SQL thread there is no contention, and the GIL penalty + disappears. With 8 threads it collapses: ten concurrent 100-row queries + take 9.2 ms each instead of 0.5 ms. On the free-threaded build, running + queries together is always cheaper per query. Per-query wall time drops + 2.3–4.3× with 3 threads. + +### What this means for the extras question + +- **The sequential-extras win is an artefact of the GIL plus a multi-thread + SQL pool.** On the free-threaded build, parallel and sequential extras are + level within about ±5% on most pages, with no consistent winner: the + 500k-row page favours parallel by 7–11%, and with 8 threads the small table + page favours sequential by 15%. With 1 SQL thread on the GIL build, + parallel extras *beat* sequential (sequential is 19% slower on the + 100-row page). So + `critic-3-extras-sequential` should not land as written. +- **`patches/extras-parallel-only-without-gil.diff`** is the targeted + version. It resolves extras in parallel only when `sys._is_gil_enabled()` + is false, checked when each registry is first built, so the setting + follows the GIL if a later extension import turns it back on. That gives + the best-measured arm with the default 3 threads on each build. The full + suite passes with it on the GIL build, where its behaviour changes. +- **The bigger lever on GIL builds is the thread pool itself.** One SQL + thread nearly doubled table-page throughput under load (42.0 → 22.3 ms per + page). It removes contention between concurrent *requests*, not just + between one request's extras. But it serialises long SQLite-bound queries: + the 500k-row facet page went from 167 to 424 ms per page. Neither default + suits both workloads. Reducing per-row GIL traffic would help either way: + dropping the Python `text_factory` for a plain `str` with a fallback on + decode errors, the completeness critic's item 8. +- **Free-threading is the largest throughput change measured in this whole + study.** With 8 in flight it served table pages 2.2–3.1× faster than the + GIL build with the default pool. Heavy SQLite queries scaled with threads: + the 500k-row facet page took 72 ms per page with 8 threads, against 167 ms + on the GIL build. The single-request cost is pure-Python overhead: light + requests were 40–70% slower (redirect 94 → 160 µs), and single table pages + were 11–22% slower than the best GIL configuration. It would pay off for busy deployments and cost a little for + quiet ones. + +**Correctness under real parallelism.** `tools/ft_stress.py` sent 800 +requests with 16 in flight to 8 table pages on the free-threaded build. +The pages covered the fixtures tables, the 500k-row table and a named +in-memory database (the case behind #2189), all with parallel extras. It +compared every response with the first. That was 3,200 responses across both +trees and 3 or 8 SQL threads, with **zero failures**. The GIL-build control +lost one suggested facet in 800 requests: a suggestion query exceeded the +50 ms `facet_suggest_time_limit_ms` under contention. Suggested facets can +vary under load on GIL builds for the same reason. + ## Bugs found along the way Fixed by the patches above: @@ -360,11 +496,11 @@ not adversarially reviewed or carrying a trade-off to decide first: 8–13% of server CPU on light requests. - **Send `Content-Length` on non-streaming responses.** About 3% end to end. It also gives clients sizes and progress. -- `critic-3-extras-sequential`: table HTML resolves about 30 "extras" as - concurrent asyncio tasks (asyncinject `parallel=True`). Running them - sequentially measured −18 to −20% on table HTML. This contradicts - `run_sequential` in `table.py` (#2189), but the count and facet queries - may genuinely benefit from overlapping on large tables. +- `critic-3-extras-sequential`: always resolve table-page extras + sequentially. It measured −16 to −21% on GIL builds, but see + [the free-threading section](#free-threaded-python-315-and-the-table-page-extras). + The win comes from GIL contention in the SQL pool and disappears on + free-threaded Python, so prefer `patches/extras-parallel-only-without-gil.diff`. - `critic-5-request-scoped-introspection-memo`: foreign keys are introspected 5×, and primary keys and `PRAGMA schema_version` 2×, per table HTML request. A request-scoped memo plus a one-trip prefetch diff --git a/perf/request-response/patches/extras-parallel-only-without-gil.diff b/perf/request-response/patches/extras-parallel-only-without-gil.diff new file mode 100644 index 00000000..dd719ceb --- /dev/null +++ b/perf/request-response/patches/extras-parallel-only-without-gil.diff @@ -0,0 +1,36 @@ +diff --git a/datasette/extras.py b/datasette/extras.py +index fb8c2e0..0e2fbe5 100644 +--- a/datasette/extras.py ++++ b/datasette/extras.py +@@ -1,4 +1,5 @@ + import re ++import sys + from dataclasses import dataclass + from enum import Enum + from typing import ClassVar +@@ -8,6 +9,16 @@ from asyncinject import Registry + from datasette.utils.asgi import BadRequest + + ++def _resolve_extras_in_parallel(): ++ # With the GIL, resolving extras concurrently puts several queries on the ++ # SQL thread pool at once, and the workers then contend for the GIL on ++ # every fetched row - measured at 2-5x the CPU of running them one at a ++ # time. On a free-threaded build concurrent queries run in parallel. ++ # Checked when a registry is built, since importing an extension module ++ # without free-threading support can turn the GIL back on. ++ return not getattr(sys, "_is_gil_enabled", lambda: True)() ++ ++ + def extra_names_from_request(request): + extra_bits = request.args.getlist("_extra") + extras = set() +@@ -96,7 +107,7 @@ class ExtraRegistry: + def _registry_for_scope(self, scope): + registry = self._scope_registries.get(scope) + if registry is None: +- registry = Registry() ++ registry = Registry(parallel=_resolve_extras_in_parallel()) + for cls in self.classes_for_scope(scope): + registry.register(cls().resolve, name=cls.key()) + self._scope_registries[scope] = registry diff --git a/perf/request-response/results/free-threading/freethreaded_c1.json b/perf/request-response/results/free-threading/freethreaded_c1.json new file mode 100644 index 00000000..20377e16 --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_c1.json @@ -0,0 +1,452 @@ +{ + "table_html": { + "base": { + "median_us": 43871.0, + "runs": [ + 43871.0, + 44357.7, + 43599.5, + 46125.5, + 40705.8 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 44955.6, + "cpu_runs": [ + 44955.6, + 44273.6, + 45696.8, + 47558.6, + 41993.8 + ] + }, + "base_seq": { + "median_us": 43309.3, + "runs": [ + 44124.8, + 43309.3, + 45309.1, + 37084.8, + 34109.4 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 42346.3, + "cpu_runs": [ + 42346.3, + 42882.1, + 44283.5, + 36386.3, + 34457.4 + ] + }, + "combined": { + "median_us": 33837.3, + "runs": [ + 36948.0, + 29868.1, + 37668.5, + 33837.3, + 31009.3 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 35728.0, + "cpu_runs": [ + 38348.6, + 31307.0, + 37755.8, + 35728.0, + 32834.8 + ] + }, + "combined_seq": { + "median_us": 33373.5, + "runs": [ + 36819.7, + 34901.3, + 25410.0, + 31811.8, + 33373.5 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 31976.5, + "cpu_runs": [ + 36969.7, + 34372.7, + 25878.1, + 30504.9, + 31976.5 + ] + } + }, + "table_html_100": { + "base": { + "median_us": 40415.2, + "runs": [ + 45496.2, + 40415.2, + 39221.5, + 43562.6, + 37436.5 + ], + "status_bytes": [ + 200, + 66669 + ], + "cpu_us": 42918.7, + "cpu_runs": [ + 48015.0, + 42918.7, + 41740.9, + 45991.0, + 39111.3 + ] + }, + "base_seq": { + "median_us": 43468.4, + "runs": [ + 40910.3, + 45526.4, + 46428.0, + 40650.6, + 43468.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 42619.1, + "cpu_runs": [ + 40387.7, + 43968.0, + 45368.3, + 39635.3, + 42619.1 + ] + }, + "combined": { + "median_us": 26724.4, + "runs": [ + 29658.8, + 27078.3, + 26223.9, + 26724.4, + 24932.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 28701.9, + "cpu_runs": [ + 30590.0, + 29155.1, + 28155.5, + 28701.9, + 27614.9 + ] + }, + "combined_seq": { + "median_us": 26867.7, + "runs": [ + 30034.6, + 34023.5, + 22900.1, + 26867.7, + 21389.8 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 25400.8, + "cpu_runs": [ + 29599.6, + 32881.4, + 22562.8, + 25400.8, + 21336.0 + ] + } + }, + "big_html": { + "base": { + "median_us": 49135.6, + "runs": [ + 53062.1, + 49135.6, + 42865.2, + 55607.8, + 47956.8 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 51199.6, + "cpu_runs": [ + 54519.9, + 51199.6, + 44761.0, + 56466.6, + 50410.9 + ] + }, + "base_seq": { + "median_us": 55371.8, + "runs": [ + 51639.3, + 65047.1, + 56713.3, + 55371.8, + 51062.8 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 53922.4, + "cpu_runs": [ + 50314.5, + 59773.9, + 55420.1, + 53922.4, + 49771.2 + ] + }, + "combined": { + "median_us": 36299.8, + "runs": [ + 37940.7, + 36299.8, + 33726.9, + 37396.5, + 32461.2 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 36634.8, + "cpu_runs": [ + 39646.9, + 36634.8, + 35441.4, + 39186.1, + 33450.0 + ] + }, + "combined_seq": { + "median_us": 40371.8, + "runs": [ + 44281.8, + 40371.8, + 34935.8, + 36150.2, + 41274.7 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 38581.7, + "cpu_runs": [ + 43356.8, + 39186.1, + 33863.6, + 35311.4, + 38581.7 + ] + } + }, + "big_facets_html": { + "base": { + "median_us": 436276.1, + "runs": [ + 439074.1, + 435315.7, + 436276.1, + 444049.6, + 431548.3 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 449273.4, + "cpu_runs": [ + 453009.3, + 449273.4, + 449240.9, + 456523.5, + 444008.0 + ] + }, + "base_seq": { + "median_us": 451279.6, + "runs": [ + 455576.6, + 456216.7, + 451060.6, + 451033.3, + 451279.6 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 445378.8, + "cpu_runs": [ + 446964.4, + 449458.5, + 445378.8, + 443124.3, + 444085.5 + ] + }, + "combined": { + "median_us": 418904.4, + "runs": [ + 427159.2, + 418904.4, + 420496.0, + 415503.1, + 415558.2 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 427476.8, + "cpu_runs": [ + 436970.9, + 427476.8, + 427571.8, + 427279.8, + 426346.0 + ] + }, + "combined_seq": { + "median_us": 437213.3, + "runs": [ + 439106.4, + 439541.8, + 435953.7, + 433532.4, + 437213.3 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 431081.8, + "cpu_runs": [ + 434441.7, + 434451.3, + 428971.5, + 428830.8, + 431081.8 + ] + } + }, + "big_json_extras": { + "base": { + "median_us": 206230.7, + "runs": [ + 207105.5, + 204980.3, + 209447.8, + 206230.7, + 203319.6 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 214626.2, + "cpu_runs": [ + 215387.1, + 213953.6, + 220001.8, + 214626.2, + 211076.6 + ] + }, + "base_seq": { + "median_us": 220080.8, + "runs": [ + 219032.3, + 223417.9, + 219964.5, + 220080.8, + 228293.9 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 216250.8, + "cpu_runs": [ + 216097.5, + 217030.6, + 216250.8, + 215577.1, + 223218.7 + ] + }, + "combined": { + "median_us": 207025.7, + "runs": [ + 207025.7, + 210685.9, + 209470.6, + 200896.8, + 199705.0 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 215221.7, + "cpu_runs": [ + 215221.7, + 217835.4, + 217475.4, + 208773.7, + 205521.3 + ] + }, + "combined_seq": { + "median_us": 215655.5, + "runs": [ + 216749.0, + 211619.4, + 213081.2, + 215655.5, + 217202.8 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 212518.2, + "cpu_runs": [ + 213552.7, + 208046.7, + 210516.6, + 212518.2, + 212905.8 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/freethreaded_c1.txt b/perf/request-response/results/free-threading/freethreaded_c1.txt new file mode 100644 index 00000000..56b0f2fe --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_c1.txt @@ -0,0 +1,17 @@ +endpoint base base_seq combined combined_seq +table_html 43871.0us 43309.3us -1% 33837.3us -23% 33373.5us -24% +table_html_100 40415.2us 43468.4us +8% 26724.4us -34% 26867.7us -34% +big_html 49135.6us 55371.8us +13% 36299.8us -26% 40371.8us -18% +big_facets_html 436276.1us 451279.6us +3% 418904.4us -4% 437213.3us +0% +big_json_extras 206230.7us 220080.8us +7% 207025.7us +0% 215655.5us +5% + +CPU time per request (process_time, all threads): +endpoint base base_seq combined combined_seq +table_html 44955.6us 42346.3us -6% 35728.0us -21% 31976.5us -29% +table_html_100 42918.7us 42619.1us -1% 28701.9us -33% 25400.8us -41% +big_html 51199.6us 53922.4us +5% 36634.8us -28% 38581.7us -25% +big_facets_html 449273.4us 445378.8us -1% 427476.8us -5% 431081.8us -4% +big_json_extras 214626.2us 216250.8us +1% 215221.7us +0% 212518.2us -1% +WARNING: status/bytes differ for table_html_100: {'base': (200, 66669), 'base_seq': (200, 66670), 'combined': (200, 66670), 'combined_seq': (200, 66670)} +WARNING: status/bytes differ for big_facets_html: {'base': (200, 90956), 'base_seq': (200, 90956), 'combined': (200, 87157), 'combined_seq': (200, 87157)} +WARNING: status/bytes differ for big_json_extras: {'base': (200, 14468), 'base_seq': (200, 17428), 'combined': (200, 17428), 'combined_seq': (200, 17428)} diff --git a/perf/request-response/results/free-threading/freethreaded_c8.json b/perf/request-response/results/free-threading/freethreaded_c8.json new file mode 100644 index 00000000..3245d3a6 --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_c8.json @@ -0,0 +1,452 @@ +{ + "table_html": { + "base": { + "median_us": 20267.3, + "runs": [ + 17824.2, + 18769.6, + 22057.6, + 24516.7, + 20267.3 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 32450.7, + "cpu_runs": [ + 31775.3, + 29979.2, + 35008.3, + 37576.5, + 32450.7 + ] + }, + "base_seq": { + "median_us": 19539.0, + "runs": [ + 19639.6, + 19108.7, + 18292.1, + 20280.5, + 19539.0 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 32815.1, + "cpu_runs": [ + 32119.9, + 32815.1, + 30286.8, + 33823.2, + 33970.2 + ] + }, + "combined": { + "median_us": 13410.0, + "runs": [ + 13410.0, + 13056.2, + 21626.6, + 13048.4, + 14432.9 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 23373.6, + "cpu_runs": [ + 23210.4, + 21937.1, + 32830.6, + 23373.6, + 23411.0 + ] + }, + "combined_seq": { + "median_us": 12370.8, + "runs": [ + 14722.4, + 11785.0, + 12370.8, + 11908.6, + 15040.8 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 22529.8, + "cpu_runs": [ + 23879.5, + 22529.8, + 20463.5, + 20546.3, + 25056.6 + ] + } + }, + "table_html_100": { + "base": { + "median_us": 25633.7, + "runs": [ + 24087.7, + 30387.1, + 26492.7, + 25633.7, + 24190.7 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 37758.4, + "cpu_runs": [ + 36036.3, + 44986.8, + 37758.4, + 37795.1, + 36558.7 + ] + }, + "base_seq": { + "median_us": 26993.9, + "runs": [ + 24664.7, + 32562.4, + 32042.7, + 26993.9, + 23133.3 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 38652.7, + "cpu_runs": [ + 36417.6, + 44246.5, + 44041.6, + 38652.7, + 35637.2 + ] + }, + "combined": { + "median_us": 15667.6, + "runs": [ + 17574.8, + 16396.4, + 15667.6, + 13536.7, + 12991.9 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 25699.8, + "cpu_runs": [ + 27439.0, + 26509.2, + 25699.8, + 22217.2, + 21951.1 + ] + }, + "combined_seq": { + "median_us": 14860.7, + "runs": [ + 19064.2, + 14860.7, + 14254.0, + 14243.9, + 15110.1 + ], + "status_bytes": [ + 200, + 66669 + ], + "cpu_us": 23044.6, + "cpu_runs": [ + 27186.8, + 23044.6, + 22320.8, + 22918.9, + 23502.3 + ] + } + }, + "big_html": { + "base": { + "median_us": 32140.0, + "runs": [ + 32140.0, + 34334.2, + 28244.8, + 32493.7, + 27560.1 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 47521.9, + "cpu_runs": [ + 48246.2, + 50265.1, + 43771.2, + 47521.9, + 42252.8 + ] + }, + "base_seq": { + "median_us": 30629.3, + "runs": [ + 29346.8, + 32889.4, + 30629.3, + 31312.6, + 30617.6 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 45540.3, + "cpu_runs": [ + 44127.0, + 49658.2, + 46226.1, + 44117.0, + 45540.3 + ] + }, + "combined": { + "median_us": 18564.9, + "runs": [ + 19051.5, + 18564.9, + 16456.9, + 18316.0, + 20369.0 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 30912.8, + "cpu_runs": [ + 30752.6, + 30912.8, + 27671.4, + 31376.5, + 31901.7 + ] + }, + "combined_seq": { + "median_us": 19607.1, + "runs": [ + 26335.5, + 18041.0, + 19607.1, + 19277.8, + 19931.5 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 30503.3, + "cpu_runs": [ + 41075.8, + 29880.9, + 30232.3, + 30503.3, + 30942.2 + ] + } + }, + "big_facets_html": { + "base": { + "median_us": 153071.4, + "runs": [ + 146930.2, + 156584.7, + 150042.2, + 153071.4, + 155761.0 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 430757.7, + "cpu_runs": [ + 426345.6, + 442366.1, + 430757.7, + 430266.8, + 439000.6 + ] + }, + "base_seq": { + "median_us": 156348.8, + "runs": [ + 157725.3, + 156348.8, + 150967.2, + 151371.0, + 156878.5 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 436700.9, + "cpu_runs": [ + 440248.7, + 436700.9, + 429677.8, + 433221.0, + 439827.6 + ] + }, + "combined": { + "median_us": 152529.7, + "runs": [ + 151653.5, + 160218.3, + 152529.7, + 145625.5, + 158308.0 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 416349.0, + "cpu_runs": [ + 414815.4, + 432270.0, + 415925.8, + 416349.0, + 426514.4 + ] + }, + "combined_seq": { + "median_us": 146384.6, + "runs": [ + 150181.9, + 146384.6, + 145943.2, + 143131.7, + 146798.8 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 418970.1, + "cpu_runs": [ + 417535.6, + 422411.5, + 418970.1, + 413217.3, + 419294.7 + ] + } + }, + "big_json_extras": { + "base": { + "median_us": 69360.6, + "runs": [ + 67398.6, + 69360.6, + 67826.1, + 70117.1, + 70229.0 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 205427.7, + "cpu_runs": [ + 201451.8, + 208433.4, + 204163.1, + 207793.2, + 205427.7 + ] + }, + "base_seq": { + "median_us": 74573.0, + "runs": [ + 76157.2, + 75542.2, + 74573.0, + 71768.8, + 70216.5 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 211874.2, + "cpu_runs": [ + 211874.2, + 212395.8, + 215329.4, + 206496.5, + 200192.2 + ] + }, + "combined": { + "median_us": 71468.6, + "runs": [ + 68738.3, + 71647.6, + 71824.1, + 70178.0, + 71468.6 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 200218.6, + "cpu_runs": [ + 200218.6, + 207698.2, + 198219.4, + 200121.5, + 204259.5 + ] + }, + "combined_seq": { + "median_us": 71755.9, + "runs": [ + 71755.9, + 73599.3, + 72803.9, + 69703.6, + 69141.6 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 203118.1, + "cpu_runs": [ + 203118.1, + 204780.2, + 206865.2, + 197732.3, + 198824.2 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/freethreaded_c8.txt b/perf/request-response/results/free-threading/freethreaded_c8.txt new file mode 100644 index 00000000..99c9b11d --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_c8.txt @@ -0,0 +1,15 @@ +endpoint base base_seq combined combined_seq +table_html 20267.3us 19539.0us -4% 13410.0us -34% 12370.8us -39% +table_html_100 25633.7us 26993.9us +5% 15667.6us -39% 14860.7us -42% +big_html 32140.0us 30629.3us -5% 18564.9us -42% 19607.1us -39% +big_facets_html 153071.4us 156348.8us +2% 152529.7us -0% 146384.6us -4% +big_json_extras 69360.6us 74573.0us +8% 71468.6us +3% 71755.9us +3% + +CPU time per request (process_time, all threads): +endpoint base base_seq combined combined_seq +table_html 32450.7us 32815.1us +1% 23373.6us -28% 22529.8us -31% +table_html_100 37758.4us 38652.7us +2% 25699.8us -32% 23044.6us -39% +big_html 47521.9us 45540.3us -4% 30912.8us -35% 30503.3us -36% +big_facets_html 430757.7us 436700.9us +1% 416349.0us -3% 418970.1us -3% +big_json_extras 205427.7us 211874.2us +3% 200218.6us -3% 203118.1us -1% +WARNING: status/bytes differ for table_html_100: {'base': (200, 66670), 'base_seq': (200, 66670), 'combined': (200, 66670), 'combined_seq': (200, 66669)} diff --git a/perf/request-response/results/free-threading/freethreaded_t8_c1.json b/perf/request-response/results/free-threading/freethreaded_t8_c1.json new file mode 100644 index 00000000..f2553a1d --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_t8_c1.json @@ -0,0 +1,232 @@ +{ + "table_html": { + "combined": { + "median_us": 34975.8, + "runs": [ + 39357.6, + 31788.5, + 33903.6, + 34975.8, + 35074.1 + ], + "status_bytes": [ + 200, + 26551 + ], + "cpu_us": 35247.5, + "cpu_runs": [ + 41014.2, + 33136.0, + 35247.5, + 35037.7, + 36566.5 + ] + }, + "combined_seq": { + "median_us": 29762.6, + "runs": [ + 28785.2, + 29762.6, + 33021.0, + 37197.3, + 27900.6 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 28326.2, + "cpu_runs": [ + 27729.0, + 28326.2, + 32473.6, + 35277.5, + 27930.3 + ] + } + }, + "table_html_100": { + "combined": { + "median_us": 28804.9, + "runs": [ + 28804.9, + 29074.0, + 23106.0, + 28942.2, + 24640.2 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 29970.0, + "cpu_runs": [ + 29970.0, + 31507.7, + 25005.7, + 30548.5, + 26688.1 + ] + }, + "combined_seq": { + "median_us": 30127.7, + "runs": [ + 30127.7, + 30685.7, + 24992.2, + 31707.4, + 23042.7 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 29178.3, + "cpu_runs": [ + 29178.3, + 29644.8, + 24398.9, + 30427.8, + 22852.2 + ] + } + }, + "big_html": { + "combined": { + "median_us": 36421.8, + "runs": [ + 36421.8, + 33125.6, + 42760.8, + 35308.8, + 36826.2 + ], + "status_bytes": [ + 200, + 86458 + ], + "cpu_us": 36884.2, + "cpu_runs": [ + 36596.8, + 34704.8, + 43465.7, + 36884.2, + 38710.0 + ] + }, + "combined_seq": { + "median_us": 38917.1, + "runs": [ + 42625.8, + 42171.0, + 38625.6, + 37974.8, + 38917.1 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 38048.2, + "cpu_runs": [ + 41169.5, + 41111.7, + 35976.1, + 36922.5, + 38048.2 + ] + } + }, + "big_facets_html": { + "combined": { + "median_us": 418525.9, + "runs": [ + 418583.0, + 422842.6, + 417112.4, + 417463.1, + 418525.9 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 427737.0, + "cpu_runs": [ + 428553.8, + 429643.4, + 425837.3, + 422867.2, + 427737.0 + ] + }, + "combined_seq": { + "median_us": 440381.0, + "runs": [ + 440472.6, + 449913.2, + 440381.0, + 439167.4, + 438127.8 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 433207.1, + "cpu_runs": [ + 433207.1, + 439543.5, + 433958.2, + 431820.4, + 430869.2 + ] + } + }, + "big_json_extras": { + "combined": { + "median_us": 205540.5, + "runs": [ + 205030.0, + 206769.0, + 205703.8, + 205540.5, + 205150.3 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 212786.6, + "cpu_runs": [ + 211042.5, + 213575.7, + 212786.6, + 213859.4, + 211107.5 + ] + }, + "combined_seq": { + "median_us": 214527.8, + "runs": [ + 214527.8, + 212318.8, + 212776.0, + 217214.6, + 216249.0 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 211274.8, + "cpu_runs": [ + 211274.8, + 207165.6, + 209016.9, + 211738.7, + 213587.1 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/freethreaded_t8_c1.txt b/perf/request-response/results/free-threading/freethreaded_t8_c1.txt new file mode 100644 index 00000000..7cd7cb15 --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_t8_c1.txt @@ -0,0 +1,17 @@ +endpoint combined combined_seq +table_html 34975.8us 29762.6us -15% +table_html_100 28804.9us 30127.7us +5% +big_html 36421.8us 38917.1us +7% +big_facets_html 418525.9us 440381.0us +5% +big_json_extras 205540.5us 214527.8us +4% + +CPU time per request (process_time, all threads): +endpoint combined combined_seq +table_html 35247.5us 28326.2us -20% +table_html_100 29970.0us 29178.3us -3% +big_html 36884.2us 38048.2us +3% +big_facets_html 427737.0us 433207.1us +1% +big_json_extras 212786.6us 211274.8us -1% +WARNING: status/bytes differ for table_html: {'combined': (200, 26551), 'combined_seq': (200, 26552)} +WARNING: status/bytes differ for big_html: {'combined': (200, 86458), 'combined_seq': (200, 86459)} +WARNING: status/bytes differ for big_facets_html: {'combined': (200, 90956), 'combined_seq': (200, 87157)} diff --git a/perf/request-response/results/free-threading/freethreaded_t8_c8.json b/perf/request-response/results/free-threading/freethreaded_t8_c8.json new file mode 100644 index 00000000..07e7c0fd --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_t8_c8.json @@ -0,0 +1,232 @@ +{ + "table_html": { + "combined": { + "median_us": 15238.2, + "runs": [ + 13778.0, + 13864.2, + 20202.6, + 15238.2, + 16372.8 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 28569.2, + "cpu_runs": [ + 28362.9, + 27726.4, + 37427.9, + 28569.2, + 29171.5 + ] + }, + "combined_seq": { + "median_us": 14914.1, + "runs": [ + 12911.7, + 15927.1, + 13358.5, + 18344.3, + 14914.1 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 27843.0, + "cpu_runs": [ + 25295.1, + 27843.0, + 24582.7, + 33385.0, + 29404.2 + ] + } + }, + "table_html_100": { + "combined": { + "median_us": 15888.1, + "runs": [ + 14640.5, + 19117.7, + 14657.0, + 15888.1, + 16459.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 26727.6, + "cpu_runs": [ + 26419.2, + 31279.6, + 26224.7, + 26727.6, + 27023.2 + ] + }, + "combined_seq": { + "median_us": 15116.9, + "runs": [ + 15116.9, + 13832.2, + 15131.7, + 15971.3, + 14433.6 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 26677.5, + "cpu_runs": [ + 26965.2, + 25293.7, + 25120.6, + 29304.1, + 26677.5 + ] + } + }, + "big_html": { + "combined": { + "median_us": 17186.4, + "runs": [ + 16670.5, + 18173.9, + 14915.3, + 17186.4, + 18619.1 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 32678.5, + "cpu_runs": [ + 32678.5, + 32853.0, + 27290.5, + 30545.4, + 34393.9 + ] + }, + "combined_seq": { + "median_us": 19747.5, + "runs": [ + 17311.7, + 17440.7, + 19747.5, + 20205.6, + 20277.8 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 34292.1, + "cpu_runs": [ + 31521.9, + 30721.1, + 34411.6, + 34292.1, + 35986.7 + ] + } + }, + "big_facets_html": { + "combined": { + "median_us": 72082.8, + "runs": [ + 72082.8, + 72459.7, + 69348.6, + 71597.9, + 74655.6 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 246842.7, + "cpu_runs": [ + 243096.2, + 250012.8, + 235532.1, + 246842.7, + 261236.0 + ] + }, + "combined_seq": { + "median_us": 72313.7, + "runs": [ + 69210.1, + 72873.8, + 72313.7, + 72650.6, + 70327.9 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 256127.4, + "cpu_runs": [ + 242729.6, + 256127.4, + 258836.4, + 256515.0, + 245821.2 + ] + } + }, + "big_json_extras": { + "combined": { + "median_us": 33967.2, + "runs": [ + 35755.0, + 32316.6, + 33967.2, + 32894.0, + 35117.7 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 122862.7, + "cpu_runs": [ + 124527.9, + 121430.7, + 122862.7, + 121877.2, + 123818.5 + ] + }, + "combined_seq": { + "median_us": 33935.9, + "runs": [ + 34191.1, + 32539.0, + 33935.9, + 34334.9, + 31916.8 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 123638.9, + "cpu_runs": [ + 124406.7, + 123638.9, + 122587.6, + 127244.6, + 119710.1 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/freethreaded_t8_c8.txt b/perf/request-response/results/free-threading/freethreaded_t8_c8.txt new file mode 100644 index 00000000..c99545cc --- /dev/null +++ b/perf/request-response/results/free-threading/freethreaded_t8_c8.txt @@ -0,0 +1,14 @@ +endpoint combined combined_seq +table_html 15238.2us 14914.1us -2% +table_html_100 15888.1us 15116.9us -5% +big_html 17186.4us 19747.5us +15% +big_facets_html 72082.8us 72313.7us +0% +big_json_extras 33967.2us 33935.9us -0% + +CPU time per request (process_time, all threads): +endpoint combined combined_seq +table_html 28569.2us 27843.0us -3% +table_html_100 26727.6us 26677.5us -0% +big_html 32678.5us 34292.1us +5% +big_facets_html 246842.7us 256127.4us +4% +big_json_extras 122862.7us 123638.9us +1% diff --git a/perf/request-response/results/free-threading/gil_c1.json b/perf/request-response/results/free-threading/gil_c1.json new file mode 100644 index 00000000..33f16c68 --- /dev/null +++ b/perf/request-response/results/free-threading/gil_c1.json @@ -0,0 +1,452 @@ +{ + "table_html": { + "base": { + "median_us": 48723.8, + "runs": [ + 50790.4, + 45845.4, + 57572.0, + 48723.8, + 46642.7 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 47686.8, + "cpu_runs": [ + 50982.0, + 47373.4, + 54956.2, + 45895.8, + 47686.8 + ] + }, + "base_seq": { + "median_us": 40882.9, + "runs": [ + 38473.5, + 40882.9, + 41477.6, + 47273.5, + 40550.4 + ], + "status_bytes": [ + 200, + 26551 + ], + "cpu_us": 39163.6, + "cpu_runs": [ + 36067.8, + 38967.5, + 39538.3, + 45420.3, + 39163.6 + ] + }, + "combined": { + "median_us": 36199.3, + "runs": [ + 34223.4, + 36199.3, + 38435.8, + 37995.0, + 30870.5 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 37599.6, + "cpu_runs": [ + 35854.8, + 37599.6, + 40073.6, + 38761.6, + 33173.2 + ] + }, + "combined_seq": { + "median_us": 30000.0, + "runs": [ + 30000.0, + 24645.3, + 30349.3, + 32959.8, + 22150.9 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 28937.2, + "cpu_runs": [ + 28937.2, + 24374.0, + 29042.4, + 31073.7, + 22013.7 + ] + } + }, + "table_html_100": { + "base": { + "median_us": 41860.2, + "runs": [ + 40650.6, + 40965.9, + 43199.3, + 41860.2, + 42097.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 42602.9, + "cpu_runs": [ + 41096.0, + 41154.0, + 43032.8, + 42602.9, + 43018.2 + ] + }, + "base_seq": { + "median_us": 40041.4, + "runs": [ + 41104.0, + 42824.9, + 36529.9, + 40041.4, + 37799.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 38197.2, + "cpu_runs": [ + 39844.9, + 41058.4, + 35147.9, + 38197.2, + 35238.7 + ] + }, + "combined": { + "median_us": 27735.3, + "runs": [ + 30495.3, + 27735.3, + 26114.5, + 26234.1, + 31007.3 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 28789.8, + "cpu_runs": [ + 29654.9, + 28789.8, + 26882.1, + 27482.8, + 32330.0 + ] + }, + "combined_seq": { + "median_us": 21869.3, + "runs": [ + 25051.0, + 27724.0, + 21869.3, + 21657.3, + 21639.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 21476.9, + "cpu_runs": [ + 23940.0, + 26325.5, + 21476.9, + 20862.0, + 21284.6 + ] + } + }, + "big_html": { + "base": { + "median_us": 56635.2, + "runs": [ + 56635.2, + 53610.9, + 62632.8, + 55964.2, + 63027.7 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 55834.0, + "cpu_runs": [ + 55275.5, + 53372.9, + 57477.1, + 55834.0, + 57775.5 + ] + }, + "base_seq": { + "median_us": 48601.0, + "runs": [ + 49653.2, + 45418.1, + 47730.3, + 51183.0, + 48601.0 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 47013.7, + "cpu_runs": [ + 48519.4, + 44258.1, + 46554.8, + 49700.3, + 47013.7 + ] + }, + "combined": { + "median_us": 37583.9, + "runs": [ + 37974.0, + 37583.9, + 38760.4, + 37499.1, + 35710.8 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 37896.8, + "cpu_runs": [ + 37896.8, + 38321.0, + 38090.5, + 37350.1, + 36243.8 + ] + }, + "combined_seq": { + "median_us": 30481.6, + "runs": [ + 33714.1, + 29087.2, + 34077.9, + 29247.1, + 30481.6 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 27683.0, + "cpu_runs": [ + 28629.5, + 27683.0, + 33241.6, + 27607.2, + 26145.6 + ] + } + }, + "big_facets_html": { + "base": { + "median_us": 444848.9, + "runs": [ + 454020.9, + 448949.4, + 443883.6, + 438942.3, + 444848.9 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 448225.0, + "cpu_runs": [ + 452224.0, + 451364.1, + 446864.1, + 443375.6, + 448225.0 + ] + }, + "base_seq": { + "median_us": 454491.1, + "runs": [ + 450282.7, + 458516.1, + 454491.1, + 454496.1, + 449258.3 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 446723.6, + "cpu_runs": [ + 441079.8, + 450381.8, + 446723.6, + 446852.7, + 441442.7 + ] + }, + "combined": { + "median_us": 428737.2, + "runs": [ + 429004.6, + 428974.6, + 426955.4, + 428737.2, + 425167.4 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 429839.7, + "cpu_runs": [ + 430390.1, + 430767.9, + 427927.4, + 429839.7, + 426904.9 + ] + }, + "combined_seq": { + "median_us": 428745.4, + "runs": [ + 432549.1, + 426932.0, + 432569.2, + 428745.4, + 428613.8 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 422246.6, + "cpu_runs": [ + 423334.7, + 421065.0, + 426357.1, + 422246.6, + 422074.4 + ] + } + }, + "big_json_extras": { + "base": { + "median_us": 211351.0, + "runs": [ + 208831.0, + 209196.9, + 211351.0, + 215350.9, + 211583.3 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 212990.4, + "cpu_runs": [ + 211620.7, + 211803.2, + 212990.4, + 216353.4, + 213772.7 + ] + }, + "base_seq": { + "median_us": 218914.8, + "runs": [ + 216172.4, + 218914.8, + 225930.5, + 221172.0, + 205416.3 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 213888.7, + "cpu_runs": [ + 213228.9, + 213888.7, + 217207.5, + 217090.4, + 202933.4 + ] + }, + "combined": { + "median_us": 206525.9, + "runs": [ + 211433.9, + 204760.5, + 206525.9, + 205381.9, + 206981.9 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 209886.9, + "cpu_runs": [ + 214489.4, + 208481.0, + 209886.9, + 210399.3, + 209443.8 + ] + }, + "combined_seq": { + "median_us": 211165.8, + "runs": [ + 211165.8, + 207163.3, + 215228.9, + 218132.1, + 208905.4 + ], + "status_bytes": [ + 200, + 17428 + ], + "cpu_us": 208036.4, + "cpu_runs": [ + 208036.4, + 204234.6, + 211840.1, + 214489.4, + 204674.2 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/gil_c1.txt b/perf/request-response/results/free-threading/gil_c1.txt new file mode 100644 index 00000000..98c0ffff --- /dev/null +++ b/perf/request-response/results/free-threading/gil_c1.txt @@ -0,0 +1,17 @@ +endpoint base base_seq combined combined_seq +table_html 48723.8us 40882.9us -16% 36199.3us -26% 30000.0us -38% +table_html_100 41860.2us 40041.4us -4% 27735.3us -34% 21869.3us -48% +big_html 56635.2us 48601.0us -14% 37583.9us -34% 30481.6us -46% +big_facets_html 444848.9us 454491.1us +2% 428737.2us -4% 428745.4us -4% +big_json_extras 211351.0us 218914.8us +4% 206525.9us -2% 211165.8us -0% + +CPU time per request (process_time, all threads): +endpoint base base_seq combined combined_seq +table_html 47686.8us 39163.6us -18% 37599.6us -21% 28937.2us -39% +table_html_100 42602.9us 38197.2us -10% 28789.8us -32% 21476.9us -50% +big_html 55834.0us 47013.7us -16% 37896.8us -32% 27683.0us -50% +big_facets_html 448225.0us 446723.6us -0% 429839.7us -4% 422246.6us -6% +big_json_extras 212990.4us 213888.7us +0% 209886.9us -1% 208036.4us -2% +WARNING: status/bytes differ for table_html: {'base': (200, 26552), 'base_seq': (200, 26551), 'combined': (200, 26552), 'combined_seq': (200, 26552)} +WARNING: status/bytes differ for big_facets_html: {'base': (200, 87157), 'base_seq': (200, 87157), 'combined': (200, 90956), 'combined_seq': (200, 90956)} +WARNING: status/bytes differ for big_json_extras: {'base': (200, 17428), 'base_seq': (200, 14468), 'combined': (200, 17428), 'combined_seq': (200, 17428)} diff --git a/perf/request-response/results/free-threading/gil_c1_switch05.json b/perf/request-response/results/free-threading/gil_c1_switch05.json new file mode 100644 index 00000000..ce503fda --- /dev/null +++ b/perf/request-response/results/free-threading/gil_c1_switch05.json @@ -0,0 +1,272 @@ +{ + "table_html": { + "base": { + "median_us": 47049.7, + "runs": [ + 47049.7, + 46425.7, + 47301.1, + 47061.2, + 42555.7 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 49918.0, + "cpu_runs": [ + 50191.3, + 49918.0, + 50648.8, + 49487.5, + 46456.6 + ] + }, + "base_seq": { + "median_us": 42553.1, + "runs": [ + 52077.0, + 38370.7, + 37836.0, + 42553.1, + 43023.1 + ], + "status_bytes": [ + 200, + 26551 + ], + "cpu_us": 40484.2, + "cpu_runs": [ + 45649.7, + 36955.8, + 35414.4, + 40484.2, + 41011.0 + ] + }, + "combined": { + "median_us": 36105.3, + "runs": [ + 36105.3, + 38305.5, + 34827.3, + 32790.1, + 39106.1 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 40255.3, + "cpu_runs": [ + 40255.3, + 41123.9, + 38633.7, + 37208.4, + 43397.5 + ] + }, + "combined_seq": { + "median_us": 31361.5, + "runs": [ + 31361.5, + 41789.2, + 30781.4, + 31499.2, + 28819.8 + ], + "status_bytes": [ + 200, + 26551 + ], + "cpu_us": 29692.9, + "cpu_runs": [ + 30394.1, + 35195.6, + 28912.0, + 29692.9, + 27250.5 + ] + } + }, + "table_html_100": { + "base": { + "median_us": 41961.4, + "runs": [ + 44938.5, + 44527.6, + 39676.1, + 39305.3, + 41961.4 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 43227.9, + "cpu_runs": [ + 46353.5, + 46367.7, + 41432.9, + 40762.8, + 43227.9 + ] + }, + "base_seq": { + "median_us": 39018.8, + "runs": [ + 38156.7, + 43142.6, + 39018.8, + 33284.5, + 41400.1 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 36491.6, + "cpu_runs": [ + 36046.7, + 40984.4, + 36491.6, + 32544.0, + 37927.9 + ] + }, + "combined": { + "median_us": 27836.5, + "runs": [ + 25701.4, + 32671.0, + 30171.7, + 24459.6, + 27836.5 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 30423.6, + "cpu_runs": [ + 27407.0, + 35428.4, + 32121.9, + 26631.6, + 30423.6 + ] + }, + "combined_seq": { + "median_us": 22521.1, + "runs": [ + 28498.0, + 19413.3, + 22521.1, + 19967.2, + 24544.5 + ], + "status_bytes": [ + 200, + 66669 + ], + "cpu_us": 22035.4, + "cpu_runs": [ + 27397.1, + 18577.2, + 22035.4, + 19482.8, + 23383.9 + ] + } + }, + "big_html": { + "base": { + "median_us": 52826.7, + "runs": [ + 51708.3, + 61066.4, + 52826.7, + 48757.2, + 56478.2 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 54046.6, + "cpu_runs": [ + 51897.2, + 57491.1, + 54046.6, + 49724.6, + 56925.6 + ] + }, + "base_seq": { + "median_us": 46792.5, + "runs": [ + 46792.5, + 44794.2, + 41622.8, + 57385.8, + 49247.0 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 45132.3, + "cpu_runs": [ + 45132.3, + 43553.2, + 40293.8, + 53773.9, + 47973.0 + ] + }, + "combined": { + "median_us": 36168.3, + "runs": [ + 40736.5, + 34933.2, + 36168.3, + 33658.7, + 36533.5 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 36874.7, + "cpu_runs": [ + 38282.4, + 35281.7, + 36874.7, + 34799.4, + 37397.3 + ] + }, + "combined_seq": { + "median_us": 29164.8, + "runs": [ + 28197.1, + 31520.2, + 29658.3, + 29017.4, + 29164.8 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 27814.4, + "cpu_runs": [ + 27159.4, + 30883.7, + 28193.2, + 27696.0, + 27814.4 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/gil_c1_switch05.txt b/perf/request-response/results/free-threading/gil_c1_switch05.txt new file mode 100644 index 00000000..51183eb7 --- /dev/null +++ b/perf/request-response/results/free-threading/gil_c1_switch05.txt @@ -0,0 +1,12 @@ +endpoint base base_seq combined combined_seq +table_html 47049.7us 42553.1us -10% 36105.3us -23% 31361.5us -33% +table_html_100 41961.4us 39018.8us -7% 27836.5us -34% 22521.1us -46% +big_html 52826.7us 46792.5us -11% 36168.3us -32% 29164.8us -45% + +CPU time per request (process_time, all threads): +endpoint base base_seq combined combined_seq +table_html 49918.0us 40484.2us -19% 40255.3us -19% 29692.9us -41% +table_html_100 43227.9us 36491.6us -16% 30423.6us -30% 22035.4us -49% +big_html 54046.6us 45132.3us -16% 36874.7us -32% 27814.4us -49% +WARNING: status/bytes differ for table_html: {'base': (200, 26552), 'base_seq': (200, 26551), 'combined': (200, 26552), 'combined_seq': (200, 26551)} +WARNING: status/bytes differ for table_html_100: {'base': (200, 66670), 'base_seq': (200, 66670), 'combined': (200, 66670), 'combined_seq': (200, 66669)} diff --git a/perf/request-response/results/free-threading/gil_c8.json b/perf/request-response/results/free-threading/gil_c8.json new file mode 100644 index 00000000..5457a618 --- /dev/null +++ b/perf/request-response/results/free-threading/gil_c8.json @@ -0,0 +1,452 @@ +{ + "table_html": { + "base": { + "median_us": 55940.3, + "runs": [ + 52547.5, + 55940.3, + 55573.9, + 60532.1, + 65107.0 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 69646.4, + "cpu_runs": [ + 67762.4, + 69646.4, + 64674.1, + 75686.1, + 80620.1 + ] + }, + "base_seq": { + "median_us": 57221.5, + "runs": [ + 57221.5, + 57475.3, + 61431.8, + 56320.7, + 53716.0 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 71763.8, + "cpu_runs": [ + 72855.6, + 69082.8, + 72765.0, + 71763.8, + 68527.6 + ] + }, + "combined": { + "median_us": 42007.4, + "runs": [ + 41242.0, + 41809.9, + 44001.0, + 42007.4, + 50248.7 + ], + "status_bytes": [ + 200, + 26551 + ], + "cpu_us": 54192.4, + "cpu_runs": [ + 53640.6, + 51816.9, + 56274.9, + 54192.4, + 63779.9 + ] + }, + "combined_seq": { + "median_us": 41961.5, + "runs": [ + 41583.0, + 43633.9, + 39824.8, + 43556.4, + 41961.5 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 53968.0, + "cpu_runs": [ + 54178.9, + 55610.7, + 51810.9, + 53968.0, + 52991.5 + ] + } + }, + "table_html_100": { + "base": { + "median_us": 48828.2, + "runs": [ + 49399.9, + 48056.2, + 45611.6, + 48828.2, + 52181.3 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 59403.3, + "cpu_runs": [ + 62351.7, + 59403.3, + 57088.3, + 59009.5, + 64085.7 + ] + }, + "base_seq": { + "median_us": 48199.2, + "runs": [ + 52568.2, + 48199.2, + 44522.9, + 51853.3, + 43791.9 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 59076.8, + "cpu_runs": [ + 64654.7, + 60839.0, + 56672.3, + 59076.8, + 55093.4 + ] + }, + "combined": { + "median_us": 35097.9, + "runs": [ + 36016.5, + 27888.9, + 30085.1, + 40975.8, + 35097.9 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 43599.8, + "cpu_runs": [ + 45543.8, + 36395.7, + 38298.2, + 43599.8, + 44906.1 + ] + }, + "combined_seq": { + "median_us": 30931.8, + "runs": [ + 28806.5, + 30335.9, + 30931.8, + 32509.1, + 30968.1 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 39944.2, + "cpu_runs": [ + 37869.8, + 39427.5, + 40023.6, + 40962.7, + 39944.2 + ] + } + }, + "big_html": { + "base": { + "median_us": 61454.6, + "runs": [ + 57551.0, + 66737.1, + 64139.7, + 61454.6, + 56370.2 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 77951.9, + "cpu_runs": [ + 74675.4, + 80273.9, + 81853.3, + 77951.9, + 71963.2 + ] + }, + "base_seq": { + "median_us": 62140.5, + "runs": [ + 62140.5, + 70177.8, + 64586.3, + 61493.5, + 60612.7 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 79762.5, + "cpu_runs": [ + 79762.5, + 87725.0, + 81451.1, + 78456.7, + 75643.6 + ] + }, + "combined": { + "median_us": 48061.2, + "runs": [ + 49688.3, + 48061.2, + 44427.5, + 41613.7, + 49574.9 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 62633.8, + "cpu_runs": [ + 64483.2, + 62633.8, + 58751.6, + 55916.2, + 64431.8 + ] + }, + "combined_seq": { + "median_us": 44529.8, + "runs": [ + 44529.8, + 48277.2, + 46014.6, + 43729.6, + 39213.0 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 58350.6, + "cpu_runs": [ + 58350.6, + 58386.7, + 60092.1, + 57561.5, + 51368.1 + ] + } + }, + "big_facets_html": { + "base": { + "median_us": 178489.9, + "runs": [ + 191860.4, + 178580.0, + 168271.5, + 178489.9, + 175507.5 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 419499.3, + "cpu_runs": [ + 450984.6, + 423417.0, + 412754.3, + 418658.5, + 419499.3 + ] + }, + "base_seq": { + "median_us": 178701.6, + "runs": [ + 182522.5, + 178701.6, + 185496.1, + 175254.3, + 174494.7 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 420419.2, + "cpu_runs": [ + 426325.6, + 420419.2, + 424222.6, + 404820.7, + 416592.0 + ] + }, + "combined": { + "median_us": 167378.1, + "runs": [ + 166793.5, + 167378.1, + 168061.0, + 168654.2, + 166394.6 + ], + "status_bytes": [ + 200, + 87156 + ], + "cpu_us": 412931.2, + "cpu_runs": [ + 413714.0, + 412931.2, + 412630.5, + 421912.4, + 408528.3 + ] + }, + "combined_seq": { + "median_us": 162834.7, + "runs": [ + 172295.6, + 162773.6, + 162409.0, + 164381.2, + 162834.7 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 411673.2, + "cpu_runs": [ + 424667.0, + 411673.2, + 409154.0, + 413715.6, + 407901.1 + ] + } + }, + "big_json_extras": { + "base": { + "median_us": 84135.0, + "runs": [ + 84881.3, + 82346.0, + 84217.2, + 84135.0, + 78823.1 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 206358.4, + "cpu_runs": [ + 207132.9, + 203867.6, + 206615.1, + 206358.4, + 202222.6 + ] + }, + "base_seq": { + "median_us": 87880.3, + "runs": [ + 88366.5, + 86494.1, + 84976.2, + 90209.9, + 87880.3 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 211772.7, + "cpu_runs": [ + 214620.6, + 213271.5, + 210253.4, + 211772.7, + 209899.8 + ] + }, + "combined": { + "median_us": 78701.4, + "runs": [ + 78388.3, + 79047.8, + 76115.5, + 79460.0, + 78701.4 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 201825.3, + "cpu_runs": [ + 199094.3, + 201825.3, + 199660.1, + 208587.1, + 203242.4 + ] + }, + "combined_seq": { + "median_us": 86615.8, + "runs": [ + 85198.1, + 87417.5, + 84012.6, + 86615.8, + 87444.6 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 210347.3, + "cpu_runs": [ + 210347.3, + 210348.8, + 208112.0, + 213249.5, + 208049.9 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/gil_c8.txt b/perf/request-response/results/free-threading/gil_c8.txt new file mode 100644 index 00000000..b8fdc4e6 --- /dev/null +++ b/perf/request-response/results/free-threading/gil_c8.txt @@ -0,0 +1,16 @@ +endpoint base base_seq combined combined_seq +table_html 55940.3us 57221.5us +2% 42007.4us -25% 41961.5us -25% +table_html_100 48828.2us 48199.2us -1% 35097.9us -28% 30931.8us -37% +big_html 61454.6us 62140.5us +1% 48061.2us -22% 44529.8us -28% +big_facets_html 178489.9us 178701.6us +0% 167378.1us -6% 162834.7us -9% +big_json_extras 84135.0us 87880.3us +4% 78701.4us -6% 86615.8us +3% + +CPU time per request (process_time, all threads): +endpoint base base_seq combined combined_seq +table_html 69646.4us 71763.8us +3% 54192.4us -22% 53968.0us -23% +table_html_100 59403.3us 59076.8us -1% 43599.8us -27% 39944.2us -33% +big_html 77951.9us 79762.5us +2% 62633.8us -20% 58350.6us -25% +big_facets_html 419499.3us 420419.2us +0% 412931.2us -2% 411673.2us -2% +big_json_extras 206358.4us 211772.7us +3% 201825.3us -2% 210347.3us +2% +WARNING: status/bytes differ for table_html: {'base': (200, 26552), 'base_seq': (200, 26552), 'combined': (200, 26551), 'combined_seq': (200, 26552)} +WARNING: status/bytes differ for big_facets_html: {'base': (200, 87157), 'base_seq': (200, 90956), 'combined': (200, 87156), 'combined_seq': (200, 90956)} diff --git a/perf/request-response/results/free-threading/gil_t1_c1.json b/perf/request-response/results/free-threading/gil_t1_c1.json new file mode 100644 index 00000000..14f34893 --- /dev/null +++ b/perf/request-response/results/free-threading/gil_t1_c1.json @@ -0,0 +1,232 @@ +{ + "table_html": { + "combined": { + "median_us": 28380.6, + "runs": [ + 29604.8, + 28380.6, + 28713.9, + 26036.5, + 25111.3 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 28555.7, + "cpu_runs": [ + 29967.4, + 28555.7, + 29364.1, + 26764.3, + 25672.0 + ] + }, + "combined_seq": { + "median_us": 29486.1, + "runs": [ + 32714.7, + 27689.3, + 31458.1, + 29486.1, + 25572.6 + ], + "status_bytes": [ + 200, + 26551 + ], + "cpu_us": 28797.3, + "cpu_runs": [ + 31597.5, + 26890.8, + 31207.3, + 28797.3, + 25294.0 + ] + } + }, + "table_html_100": { + "combined": { + "median_us": 21700.6, + "runs": [ + 21027.6, + 26116.8, + 21700.6, + 21648.6, + 27094.1 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 21572.8, + "cpu_runs": [ + 20877.9, + 26328.7, + 20981.5, + 21572.8, + 23600.8 + ] + }, + "combined_seq": { + "median_us": 25818.8, + "runs": [ + 27834.2, + 25818.8, + 26244.8, + 20695.6, + 25379.2 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 25250.2, + "cpu_runs": [ + 27031.5, + 25338.8, + 25250.2, + 20363.0, + 24771.5 + ] + } + }, + "big_html": { + "combined": { + "median_us": 31205.3, + "runs": [ + 31205.3, + 30827.9, + 35843.1, + 32780.5, + 28889.9 + ], + "status_bytes": [ + 200, + 86458 + ], + "cpu_us": 31281.3, + "cpu_runs": [ + 31281.3, + 30655.7, + 36190.6, + 33161.4, + 28606.0 + ] + }, + "combined_seq": { + "median_us": 34717.9, + "runs": [ + 35530.9, + 41741.5, + 34717.9, + 26028.8, + 27897.6 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 33678.1, + "cpu_runs": [ + 34802.8, + 40048.0, + 33678.1, + 25724.0, + 26807.8 + ] + } + }, + "big_facets_html": { + "combined": { + "median_us": 433353.3, + "runs": [ + 433353.3, + 439706.3, + 430854.3, + 429270.6, + 435022.0 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 429131.9, + "cpu_runs": [ + 429538.4, + 433562.8, + 426662.9, + 424620.5, + 429131.9 + ] + }, + "combined_seq": { + "median_us": 426759.4, + "runs": [ + 426759.4, + 441141.0, + 429590.0, + 423465.7, + 423516.0 + ], + "status_bytes": [ + 200, + 87157 + ], + "cpu_us": 421007.3, + "cpu_runs": [ + 421007.3, + 429668.7, + 423765.4, + 419886.6, + 415736.5 + ] + } + }, + "big_json_extras": { + "combined": { + "median_us": 215561.3, + "runs": [ + 219965.5, + 216920.5, + 211912.3, + 215561.3, + 205896.5 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 212128.5, + "cpu_runs": [ + 217216.8, + 213794.9, + 209027.3, + 212128.5, + 204000.8 + ] + }, + "combined_seq": { + "median_us": 213088.5, + "runs": [ + 213088.5, + 215255.6, + 212942.0, + 214429.0, + 210378.1 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 210569.5, + "cpu_runs": [ + 210569.5, + 211826.2, + 206743.2, + 211524.9, + 205986.2 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/gil_t1_c1.txt b/perf/request-response/results/free-threading/gil_t1_c1.txt new file mode 100644 index 00000000..85a0c22a --- /dev/null +++ b/perf/request-response/results/free-threading/gil_t1_c1.txt @@ -0,0 +1,16 @@ +endpoint combined combined_seq +table_html 28380.6us 29486.1us +4% +table_html_100 21700.6us 25818.8us +19% +big_html 31205.3us 34717.9us +11% +big_facets_html 433353.3us 426759.4us -2% +big_json_extras 215561.3us 213088.5us -1% + +CPU time per request (process_time, all threads): +endpoint combined combined_seq +table_html 28555.7us 28797.3us +1% +table_html_100 21572.8us 25250.2us +17% +big_html 31281.3us 33678.1us +8% +big_facets_html 429131.9us 421007.3us -2% +big_json_extras 212128.5us 210569.5us -1% +WARNING: status/bytes differ for table_html: {'combined': (200, 26552), 'combined_seq': (200, 26551)} +WARNING: status/bytes differ for big_html: {'combined': (200, 86458), 'combined_seq': (200, 86459)} diff --git a/perf/request-response/results/free-threading/gil_t1_c8.json b/perf/request-response/results/free-threading/gil_t1_c8.json new file mode 100644 index 00000000..a44db669 --- /dev/null +++ b/perf/request-response/results/free-threading/gil_t1_c8.json @@ -0,0 +1,232 @@ +{ + "table_html": { + "combined": { + "median_us": 22328.7, + "runs": [ + 22464.6, + 22328.7, + 21807.7, + 21018.9, + 23935.7 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 24085.8, + "cpu_runs": [ + 24099.2, + 24085.8, + 23783.8, + 22929.2, + 25875.9 + ] + }, + "combined_seq": { + "median_us": 22450.9, + "runs": [ + 24163.2, + 24314.3, + 20886.4, + 21208.2, + 22450.9 + ], + "status_bytes": [ + 200, + 26552 + ], + "cpu_us": 24089.1, + "cpu_runs": [ + 26343.1, + 25961.5, + 22692.4, + 22539.7, + 24089.1 + ] + } + }, + "table_html_100": { + "combined": { + "median_us": 19396.8, + "runs": [ + 24703.1, + 18095.2, + 21489.9, + 17740.2, + 19396.8 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 20022.4, + "cpu_runs": [ + 21707.1, + 18865.2, + 22726.6, + 18657.1, + 20022.4 + ] + }, + "combined_seq": { + "median_us": 19498.6, + "runs": [ + 19498.6, + 25449.6, + 20071.0, + 19204.0, + 18264.5 + ], + "status_bytes": [ + 200, + 66670 + ], + "cpu_us": 20464.8, + "cpu_runs": [ + 20464.8, + 26445.6, + 20791.9, + 19619.2, + 19200.9 + ] + } + }, + "big_html": { + "combined": { + "median_us": 26366.1, + "runs": [ + 24873.8, + 33707.3, + 30098.4, + 26366.1, + 25196.7 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 26936.3, + "cpu_runs": [ + 25972.7, + 34981.6, + 31370.5, + 26936.3, + 26299.4 + ] + }, + "combined_seq": { + "median_us": 28049.5, + "runs": [ + 23635.3, + 27076.9, + 28049.5, + 28542.3, + 29020.7 + ], + "status_bytes": [ + 200, + 86459 + ], + "cpu_us": 28582.7, + "cpu_runs": [ + 24791.2, + 28466.5, + 28582.7, + 29773.6, + 29967.5 + ] + } + }, + "big_facets_html": { + "combined": { + "median_us": 423658.3, + "runs": [ + 421112.8, + 427362.2, + 427048.6, + 423658.3, + 420737.1 + ], + "status_bytes": [ + 200, + 90956 + ], + "cpu_us": 416547.4, + "cpu_runs": [ + 416099.3, + 421854.8, + 423415.3, + 416547.4, + 415221.4 + ] + }, + "combined_seq": { + "median_us": 427133.9, + "runs": [ + 426996.1, + 427133.9, + 423135.8, + 427804.0, + 428807.4 + ], + "status_bytes": [ + 200, + 87156 + ], + "cpu_us": 422053.7, + "cpu_runs": [ + 422053.7, + 425340.5, + 419642.6, + 419878.2, + 423186.9 + ] + } + }, + "big_json_extras": { + "combined": { + "median_us": 208527.6, + "runs": [ + 206048.7, + 202818.6, + 208527.6, + 209078.5, + 211001.9 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 206348.6, + "cpu_runs": [ + 204936.5, + 201273.1, + 206348.6, + 207110.0, + 209876.1 + ] + }, + "combined_seq": { + "median_us": 206522.4, + "runs": [ + 203244.5, + 203901.4, + 214114.8, + 206522.4, + 206654.5 + ], + "status_bytes": [ + 200, + 14468 + ], + "cpu_us": 203089.0, + "cpu_runs": [ + 201692.4, + 203089.0, + 210444.4, + 203013.2, + 203868.3 + ] + } + } +} \ No newline at end of file diff --git a/perf/request-response/results/free-threading/gil_t1_c8.txt b/perf/request-response/results/free-threading/gil_t1_c8.txt new file mode 100644 index 00000000..fcdee74d --- /dev/null +++ b/perf/request-response/results/free-threading/gil_t1_c8.txt @@ -0,0 +1,15 @@ +endpoint combined combined_seq +table_html 22328.7us 22450.9us +1% +table_html_100 19396.8us 19498.6us +1% +big_html 26366.1us 28049.5us +6% +big_facets_html 423658.3us 427133.9us +1% +big_json_extras 208527.6us 206522.4us -1% + +CPU time per request (process_time, all threads): +endpoint combined combined_seq +table_html 24085.8us 24089.1us +0% +table_html_100 20022.4us 20464.8us +2% +big_html 26936.3us 28582.7us +6% +big_facets_html 416547.4us 422053.7us +1% +big_json_extras 206348.6us 203089.0us -2% +WARNING: status/bytes differ for big_facets_html: {'combined': (200, 90956), 'combined_seq': (200, 87156)} diff --git a/perf/request-response/results/free-threading/micro_gather.txt b/perf/request-response/results/free-threading/micro_gather.txt new file mode 100644 index 00000000..e5156808 --- /dev/null +++ b/perf/request-response/results/free-threading/micro_gather.txt @@ -0,0 +1,29 @@ +tools/micro_gather.py: Database.execute() 300 times, either one at a time +("sequential") or 10 at a time with asyncio.gather ("gather10"). +Per-query wall time and CPU time (process_time, all threads), medians of 5. +Python 3.15.0b4 (GIL) and 3.15.0b4t (free-threaded), combined patch tree. + +GIL on, num_sql_threads=3 + 1 row sequential wall 167us cpu 165us | gather10 wall 299us cpu 376us (cpu x2.28) + 100 text rows sequential wall 500us cpu 470us | gather10 wall 1723us cpu 2311us (cpu x4.91) + introspection sequential wall 309us cpu 300us | gather10 wall 1112us cpu 1493us (cpu x4.98) +GIL off, num_sql_threads=3 + 1 row sequential wall 133us cpu 148us | gather10 wall 58us cpu 110us (cpu x0.75) + 100 text rows sequential wall 330us cpu 323us | gather10 wall 90us cpu 249us (cpu x0.77) + introspection sequential wall 320us cpu 303us | gather10 wall 75us cpu 188us (cpu x0.62) +GIL on, num_sql_threads=1 + 1 row sequential wall 177us cpu 177us | gather10 wall 94us cpu 101us (cpu x0.57) + 100 text rows sequential wall 355us cpu 347us | gather10 wall 362us cpu 370us (cpu x1.07) + introspection sequential wall 321us cpu 312us | gather10 wall 321us cpu 331us (cpu x1.06) +GIL off, num_sql_threads=1 + 1 row sequential wall 174us cpu 178us | gather10 wall 94us cpu 134us (cpu x0.76) + 100 text rows sequential wall 366us cpu 386us | gather10 wall 211us cpu 260us (cpu x0.67) + introspection sequential wall 243us cpu 251us | gather10 wall 145us cpu 197us (cpu x0.78) +GIL on, num_sql_threads=8 + 1 row sequential wall 155us cpu 152us | gather10 wall 395us cpu 551us (cpu x3.61) + 100 text rows sequential wall 496us cpu 473us | gather10 wall 9218us cpu 14903us (cpu x31.50) + introspection sequential wall 288us cpu 273us | gather10 wall 4853us cpu 7475us (cpu x27.35) +GIL off, num_sql_threads=8 + 1 row sequential wall 142us cpu 152us | gather10 wall 63us cpu 120us (cpu x0.79) + 100 text rows sequential wall 508us cpu 476us | gather10 wall 150us cpu 394us (cpu x0.83) + introspection sequential wall 409us cpu 384us | gather10 wall 98us cpu 229us (cpu x0.60) diff --git a/perf/request-response/results/free-threading/stress.txt b/perf/request-response/results/free-threading/stress.txt new file mode 100644 index 00000000..70ee9285 --- /dev/null +++ b/perf/request-response/results/free-threading/stress.txt @@ -0,0 +1,13 @@ +tools/ft_stress.py --concurrency 16 --requests 800: 100 requests each to 8 +table pages (fixtures, a 500k-row table, and a named in-memory database with +parallel extras), every response compared with the first one (status only +for the 500k-row pages, whose facet suggestions depend on time limits). + +free-threaded, cf113a5, 3 SQL threads 800/800 ok, GIL stayed off +free-threaded, all patches, 3 SQL threads 800/800 ok, GIL stayed off +free-threaded, all patches, 8 SQL threads 800/800 ok, GIL stayed off +free-threaded, cf113a5, 8 SQL threads 800/800 ok, GIL stayed off +GIL build, all patches, 3 SQL threads (run 1) 799/800 - table_html_100 lost + its "pk2" suggested facet: that suggestion query went over the 50 ms + facet_suggest_time_limit_ms under GIL contention +GIL build, all patches, 3 SQL threads (run 2) 800/800 ok diff --git a/perf/request-response/tools/.gitignore b/perf/request-response/tools/.gitignore new file mode 100644 index 00000000..9456a9f3 --- /dev/null +++ b/perf/request-response/tools/.gitignore @@ -0,0 +1,5 @@ +*.db +__pycache__/ +server_*.log +server_*.pid +final_results_*.jsonl diff --git a/perf/request-response/tools/bench_asgi.py b/perf/request-response/tools/bench_asgi.py index bd06414c..58347e39 100644 --- a/perf/request-response/tools/bench_asgi.py +++ b/perf/request-response/tools/bench_asgi.py @@ -4,8 +4,15 @@ profiles show only Datasette's request/response machinery. Run it with the source tree you want to measure first on PYTHONPATH, e.g. PYTHONPATH=/path/to/datasette python bench_asgi.py time --n 1000 -fixtures.db is generated next to this script on first run. Set IMMUTABLE=1 -to open it as an immutable database, NUM_SQL_THREADS to change the pool. +fixtures.db (and, with BIGDB=1, a 500,000-row bigdb.db) are generated next +to this script on first run. + +Environment variables: + IMMUTABLE=1 open the databases as immutable + NUM_SQL_THREADS=N size of Datasette's SQL thread pool (default 3) + BIGDB=1 also attach bigdb.db, for the big_* endpoints + CONCURRENCY=N time mode: keep N requests in flight (throughput) + SWITCH_INTERVAL=S sys.setswitchinterval(S) before running Usage: python bench_asgi.py time [--endpoints a,b] [--n 2000] [--rounds 5] @@ -32,6 +39,36 @@ if not os.path.exists(FIXTURES): write_fixture_database(FIXTURES) + +def write_bigdb(path, rows=500_000): + import random + import sqlite3 + + rnd = random.Random(1) + conn = sqlite3.connect(path) + conn.execute( + "create table big (id integer primary key, category text, city text," + " state text, score integer, name text, created text)" + ) + conn.executemany( + "insert into big values (?, ?, ?, ?, ?, ?, ?)", + ( + ( + i, + f"category-{rnd.randrange(12)}", + f"city-{rnd.randrange(300)}", + f"S{rnd.randrange(50):02d}", + rnd.randint(0, 1000), + f"name {rnd.random():.8f}", + f"2026-{rnd.randint(1, 12):02d}-{rnd.randint(1, 28):02d}", + ) + for i in range(1, rows + 1) + ), + ) + conn.commit() + conn.close() + + ENDPOINTS = { # name: (method, path, query_string, extra headers, needs actor cookie) "versions_json": ("GET", "/-/versions.json", "", [], False), @@ -71,6 +108,15 @@ ENDPOINTS = { [], False, ), + "big_html": ("GET", "/bigdb/big", "", [], False), + "big_facets_html": ("GET", "/bigdb/big", "_facet=category&_facet=city", [], False), + "big_json_extras": ( + "GET", + "/bigdb/big.json", + "_facet=category&_extra=count&_extra=facet_results&_extra=suggested_facets", + [], + False, + ), } BROWSER_HEADERS = [ @@ -85,10 +131,20 @@ BROWSER_HEADERS = [ ] +if os.environ.get("SWITCH_INTERVAL"): + sys.setswitchinterval(float(os.environ["SWITCH_INTERVAL"])) + + def make_ds(): + files = [FIXTURES] + if os.environ.get("BIGDB"): + bigdb = os.path.join(HERE, "bigdb.db") + if not os.path.exists(bigdb): + write_bigdb(bigdb) + files.append(bigdb) return Datasette( - [] if os.environ.get("IMMUTABLE") else [FIXTURES], - immutables=[FIXTURES] if os.environ.get("IMMUTABLE") else [], + [] if os.environ.get("IMMUTABLE") else files, + immutables=files if os.environ.get("IMMUTABLE") else [], settings={"num_sql_threads": int(os.environ.get("NUM_SQL_THREADS", "3"))}, ) @@ -172,16 +228,33 @@ async def run_time(endpoints, n, rounds): for _ in range(min(200, n)): await d.request(name) per_round = [] - for _ in range(rounds): - t0 = time.perf_counter() - for _ in range(n): + concurrency = int(os.environ.get("CONCURRENCY", "1")) + + async def worker(count, name=name): + for _ in range(count): await d.request(name) - per_round.append((time.perf_counter() - t0) / n * 1e6) + + cpu_round = [] + for _ in range(rounds): + c0 = time.process_time() + t0 = time.perf_counter() + if concurrency == 1: + for _ in range(n): + await d.request(name) + else: + # Throughput: wall time per request with this many in flight + await asyncio.gather( + *(worker(n // concurrency) for _ in range(concurrency)) + ) + done = n if concurrency == 1 else (n // concurrency) * concurrency + per_round.append((time.perf_counter() - t0) / done * 1e6) + cpu_round.append((time.process_time() - c0) / done * 1e6) results[name] = { "status": status, "bytes": len(body), "median_us": round(statistics.median(per_round), 1), "min_us": round(min(per_round), 1), + "cpu_us": round(statistics.median(cpu_round), 1), } print( f"{name:22s} {status} {len(body):7d}B median {results[name]['median_us']:8.1f}us min {results[name]['min_us']:8.1f}us", diff --git a/perf/request-response/tools/ft_stress.py b/perf/request-response/tools/ft_stress.py new file mode 100644 index 00000000..56f5a387 --- /dev/null +++ b/perf/request-response/tools/ft_stress.py @@ -0,0 +1,114 @@ +""" +Fire many concurrent table-page requests (with parallel extras) at one +Datasette instance and check every response: status 200 and a body that +matches the first response once timing fields are removed. + + PYTHONPATH=tree python ft_stress.py --concurrency 16 --requests 400 +""" + +import argparse +import asyncio +import collections +import os +import re +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +os.environ.setdefault("BIGDB", "1") + +import bench_asgi + +TIMING = re.compile(rb"(\d+(\.\d+)?\s*ms|\"query_ms\":\s*[\d.e-]+|took [\d.]+ms)") +ENDPOINTS = [ + "table_html", + "table_html_100", + "big_html", + "big_json_extras", + "row_json", + "table_json", + "mem_html", + "mem_json_extras", +] +bench_asgi.ENDPOINTS["mem_html"] = ("GET", "/memtest/t", "", [], False) +bench_asgi.ENDPOINTS["mem_json_extras"] = ( + "GET", + "/memtest/t.json", + "_facet=cat&_extra=count&_extra=facet_results&_extra=suggested_facets", + [], + False, +) + + +def normalize(body): + return TIMING.sub(b"T", body) + + +async def main(concurrency, total): + ds = bench_asgi.make_ds() + # Named in-memory database: the case behind issue #2189 + mem = ds.add_memory_database("memtest") + await mem.execute_write( + "create table t (id integer primary key, name text, cat text)" + ) + await mem.execute_write_many( + "insert into t (name, cat) values (?, ?)", + [(f"name {i}", f"cat {i % 7}") for i in range(2000)], + ) + d = bench_asgi.Driver(ds) + await d.setup() + expected = {} + for name in ENDPOINTS: + status, _, body = await d.request(name) + expected[name] = (status, normalize(body)) + outcomes = collections.Counter() + mismatches = [] + queue = asyncio.Queue() + for i in range(total): + queue.put_nowait(ENDPOINTS[i % len(ENDPOINTS)]) + + async def worker(): + while not queue.empty(): + name = queue.get_nowait() + try: + status, _, body = await d.request(name) + except Exception as ex: # noqa: BLE001 - report every failure + outcomes[f"{name}: exception {type(ex).__name__}: {ex}"] += 1 + continue + # Facet suggestions on the 500k-row table depend on query time + # limits, so only the status is compared for those pages + if name.startswith("big_") and status == expected[name][0] or (status, normalize(body)) == expected[name]: + outcomes[f"{name}: ok"] += 1 + else: + outcomes[f"{name}: MISMATCH status={status}"] += 1 + if len(mismatches) < 3: + mismatches.append((name, status, body[:300])) + import difflib + + a = expected[name][1].decode("utf-8", "replace").splitlines() + b = normalize(body).decode("utf-8", "replace").splitlines() + print( + "DIFF", + name, + "\n".join( + list(difflib.unified_diff(a, b, lineterm="", n=1))[:40] + ), + ) + + await asyncio.gather(*(worker() for _ in range(concurrency))) + import sys as _sys + + print("gil enabled:", getattr(_sys, "_is_gil_enabled", lambda: True)()) + for k, v in sorted(outcomes.items()): + print(f" {v:5d} {k}") + for m in mismatches: + print("MISMATCH SAMPLE", m) + bad = sum(v for k, v in outcomes.items() if not k.endswith(": ok")) + print("FAILURES:", bad) + + +if __name__ == "__main__": + p = argparse.ArgumentParser() + p.add_argument("--concurrency", type=int, default=16) + p.add_argument("--requests", type=int, default=400) + a = p.parse_args() + asyncio.run(main(a.concurrency, a.requests)) diff --git a/perf/request-response/tools/measure_ab.py b/perf/request-response/tools/measure_ab.py index 0c0c6401..fce36a82 100644 --- a/perf/request-response/tools/measure_ab.py +++ b/perf/request-response/tools/measure_ab.py @@ -60,6 +60,7 @@ def main(): trees = [t.split("=", 1) for t in args.tree] endpoints = args.endpoints.split(",") samples = {name: {ep: [] for ep in endpoints} for name, _ in trees} + cpu = {name: {ep: [] for ep in endpoints} for name, _ in trees} statuses = {name: {} for name, _ in trees} for r in range(args.rounds): order = trees[r % len(trees) :] + trees[: r % len(trees)] @@ -67,6 +68,8 @@ def main(): data = run_once(path, endpoints, args.n, {}) for ep in endpoints: samples[name][ep].append(data[ep]["median_us"]) + if "cpu_us" in data[ep]: + cpu[name][ep].append(data[ep]["cpu_us"]) statuses[name][ep] = (data[ep]["status"], data[ep]["bytes"]) print(f"round {r + 1}/{args.rounds} done", file=sys.stderr, flush=True) result = {} @@ -83,12 +86,27 @@ def main(): "median_us": med, "runs": samples[name][ep], "status_bytes": statuses[name][ep], + "cpu_us": statistics.median(cpu[name][ep]) if cpu[name][ep] else None, + "cpu_runs": cpu[name][ep], } if name == base_name: row += f"{med:14.1f}us" else: row += f"{med:9.1f}us {100 * (med - base) / base:+5.0f}%" print(row) + if any(cpu[n][e] for n, _ in trees for e in endpoints): + print("\nCPU time per request (process_time, all threads):") + print(header) + for ep in endpoints: + row = f"{ep:22s}" + base = statistics.median(cpu[base_name][ep]) + for name, _ in trees: + med = statistics.median(cpu[name][ep]) + if name == base_name: + row += f"{med:14.1f}us" + else: + row += f"{med:9.1f}us {100 * (med - base) / base:+5.0f}%" + print(row) for ep in endpoints: sb = {name: statuses[name][ep] for name, _ in trees} if len(set(sb.values())) > 1: diff --git a/perf/request-response/tools/micro_gather.py b/perf/request-response/tools/micro_gather.py new file mode 100644 index 00000000..67ef9fce --- /dev/null +++ b/perf/request-response/tools/micro_gather.py @@ -0,0 +1,53 @@ +"""db.execute() run one at a time vs. 10 at a time with asyncio.gather.""" + +import asyncio +import os +import statistics +import sys +import time + +from datasette.app import Datasette + +HERE = os.path.dirname(os.path.abspath(__file__)) # run bench_asgi.py once first +QUERIES = { + "1 row": "select 1", + "100 text rows": "select pk1, pk2, pk3, content from compound_three_primary_keys limit 100", + "introspection": "select name, type, sql from sqlite_master", +} + + +async def main(): + ds = Datasette( + [os.path.join(HERE, "fixtures.db")], + settings={"num_sql_threads": int(os.environ.get("SQL_THREADS", "3"))}, + ) + await ds.invoke_startup() + db = ds.get_database("fixtures") + gil = getattr(sys, "_is_gil_enabled", lambda: True)() + print( + f"GIL {'on' if gil else 'off'}, num_sql_threads={os.environ.get('SQL_THREADS', '3')}" + ) + for label, sql in QUERIES.items(): + for _ in range(50): + await db.execute(sql) + res = {} + for mode in ("sequential", "gather10"): + walls, cpus = [], [] + for _ in range(5): + c0, t0 = time.process_time(), time.perf_counter() + if mode == "sequential": + for _ in range(300): + await db.execute(sql) + else: + for _ in range(30): + await asyncio.gather(*(db.execute(sql) for _ in range(10))) + walls.append((time.perf_counter() - t0) / 300 * 1e6) + cpus.append((time.process_time() - c0) / 300 * 1e6) + res[mode] = (statistics.median(walls), statistics.median(cpus)) + s, g = res["sequential"], res["gather10"] + print( + f" {label:14s} sequential wall {s[0]:6.0f}us cpu {s[1]:6.0f}us | gather10 wall {g[0]:6.0f}us cpu {g[1]:6.0f}us (cpu x{g[1] / s[1]:.2f})" + ) + + +asyncio.run(main())