Lowest median time
199 s
Vitko Runners (warm starts + test splitting)
GitHub-hosted (ubuntu-latest): 603 s
Slowest median
603 s
GitHub-hosted runners (ubuntu-latest, 2 vCPU)
3.0× the lowest
Lowest cost, managed
$0.011
Ubicloud
mean per job
Failed runs
14
Vitko Runners (warm starts + test splitting) 3, GitHub-hosted larger runner (4 vCPU) 2, Blacksmith 3, Self-hosted runners on Google Cloud 1, RunsOn 1, Ubicloud 1, GitHub-hosted runners (ubuntu-latest, 2 vCPU) 3
of 270 measured runs
Median time
Seconds from start to finish · lower is betterTable
Ordered by median time. Self-hosted options are priced all-in at 100,000 jobs a month. Green marks the lowest median time and the lowest cost among the managed runners.| Configuration | Median | p95 | Queue p50 | Queue p95 | Cost | OK |
|---|---|---|---|---|---|---|
| Vitko Runners (warm starts + test splitting) | 199 s | 212 s | 2 s | 13.3 s | $0.013 | 27 of 30 |
| GitHub-hosted larger runner (4 vCPU) | 288 s | 315 s | 3 s | 4 s | $0.060 | 28 of 30 |
| Vitko Runners | 350 s | 389 s | 2 s | 40.7 s | $0.012 | 30 of 30 |
| Blacksmith | 360 s | 511 s | 8 s | 10 s | $0.026 | 27 of 30 |
| Self-hosted runners on Google Cloud | 401 s | 457 s | 2 s | 2 s | $0.023 | 29 of 30 |
| Self-hosted runners on AWS (one VM per job) | 403 s | 503 s | 48.5 s | 90.3 s | $0.016 | 30 of 30 |
| RunsOn | 437 s | 456 s | 19 s | 26.6 s | $0.0053 | 29 of 30 |
| Ubicloud | 502 s | 594 s | 18 s | 28.6 s | $0.011 | 29 of 30 |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | 603 s | 753 s | 3 s | 4.7 s | $0.061 | 27 of 30 |
The job
- Project: vitejs/vite at
v8.3.1(39ddf7ccf7e7469ff6a3ba37bca38c32ea804d6e), licence MIT. Fetched at run time, not vendored. - Upstream job:
.github/workflows/ci.yml, job "Build&Test: node-24, ubuntu-latest". - Benchmark workflow:
.github/workflows/wl-vite.yml.
Portability edits (identical for every provider)
- Node is set up before pnpm, so pnpm's installer never runs on a runner image's own Node.
pnpm playwright install --with-deps chromium: installs Chromium's system libraries, which GitHub's image already has.- One matrix cell (Node 24, Linux); the
changedgate job is dropped. environment-react-ssr > pre-bundling > deps reload: the assertion on the two "dependencies optimized" log lines is made order-insensitive (sorted). Upstream logs the server line after a fixedsetTimeout(…, 2 * debounceMs)(200 ms) and the client line after a browser reload and re-bundle, so the order is a race that faster machines lose. The assertion is unchanged otherwise: both lines must appear.legacy > watch > rebuilds styles only entry on change: after upstream's own waits, the test also waits (up to 10 s, polling every 100 ms) until the rebuilt CSS files referenced by the manifest exist and contain the new colour. Upstream counts watcher END events and races a second one against 100 ms (a workaround for rolldown issue 10613); the number and timing of those events vary, so on any machine the files are sometimes read before the rebuild has written them. The assertions are unchanged.
Split-test arm (Track B)
On the arm that uses vitko-inc/split-tests, the unit step and the three end-to-end steps (serve, bundled dev, build) are shared out by test file. Spec files whose pages load resources from the internet (playground/assets/, playground/css/, playground/css-lightningcss/) run in an unsplit step right after each split step, because the parts have no outside network. Together the two steps run the same spec files and the same tests as the plain steps.
suite/vite holds the workflow and notes.