Short answer
In CI Bench v1, median job time on GitHub-hosted (ubuntu-latest) was cli 368 s, codex-lint 884 s, container 239 s, fd 92 s, flask 23 s, guava 1167 s, monorepo 384 s, pydantic 179 s, ts 784 s, vite 603 s, vue 121 s. The configuration with the lowest median differed by workload: Vitko Runners (warm starts + test splitting), Vitko Runners, RunsOn, Self-hosted runners on Google Cloud. Maintained by Vitko, Inc., which sells one of the products measured.
Median job time
| Configuration | cli | codex-lint | container | fd | flask | guava | monorepo | pydantic | ts | vite | vue |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Vitko Runners (warm starts + test splitting)Vitko | 140 s | 405 s | 178 s | 45 s | 13 s | 651 s | 172 s | 69 s | 243 s | 199 s | 32 s |
| Vitko RunnersVitko | 272 s | 404 s | 179 s | 49 s | 16 s | 655 s | 180 s | 105 s | 372 s | 350 s | 63 s |
| GitHub-hosted larger runner (4 vCPU) | 192 s | 476 s | 185 s | 66 s | 21 s | 946 s | 220 s | 154 s | 435 s | 288 s | 67 s |
| Blacksmith | 286 s | 448 s | 386 s† | 65 s | 22 s | 735 s | 177 s | 103 s | 401 s | 360 s | 76 s |
| Self-hosted runners on Google Cloud | 310 s | 581 s | 178 s | 64 s | 11 s | 820 s | 278 s | 119 s | 533 s | 401 s | 76 s |
| RunsOn | 327 s | 623 s | 174 s | 89 s | 38 s | 897 s | 288 s | 147 s | 570 s | 437 s | 101 s |
| Self-hosted runners on AWS (one VM per job) | 357 s | 512 s | 247 s | 114 s | 70 s | 717 s | 281 s | 154 s | 433 s | 403 s | 117 s |
| Ubicloud | 359 s | 682 s | 213 s | 87 s | 36 s | 881 s | 276 s | 159 s | 560 s | 502 s | 114 s |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | 368 s | 884 s | 239 s | 92 s | 23 s | 1167 s | 384 s | 179 s | 784 s | 603 s | 121 s |
† Layer cache did not hit in this run; a re-run is planned. See method.
Median of 30 runs per cell. p95, queue time and cost: each workload page. Jobs are one project's CI job each; your workflow will differ.
More: all questions answered · 12 options compared · ranked by speed · ranked by cost per job · calculator.
Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 29, 2026.
† Layer cache did not hit in this run; a re-run is planned. See method.