Short answer
CI Bench v1 measured eleven open-source projects' own CI jobs plus a burst, covering Rust (fd, codex-lint), Go (GitHub CLI), Java (guava), TypeScript and Node (TypeScript, vite, vue, a vercel/ai monorepo build), Python (flask, pydantic) and a Docker image build. The configuration with the lowest median time differs by workload: Rust: Vitko Runners (warm starts + test splitting) / Vitko Runners; Go: Vitko Runners (warm starts + test splitting); Java: Vitko Runners (warm starts + test splitting); TypeScript and Node: Vitko Runners (warm starts + test splitting); Python: Self-hosted runners on Google Cloud / Vitko Runners (warm starts + test splitting); Docker image build: RunsOn. That is one project's job per language, so treat it as a guide to measure your own. Maintained by Vitko, Inc., which sells one of the products measured.
Workload by workload
| Language | Workload | Lowest median | Lowest median time | GitHub-hosted (ubuntu-latest) | Vitko Runners | Blacksmith |
|---|---|---|---|---|---|---|
| Rust | fd | Vitko Runners (warm starts + test splitting)Vitko | 45 s | 92 s | 49 s | 65 s |
| Rust | codex-lint | Vitko RunnersVitko | 404 s | 884 s | 404 s | 448 s |
| Go | cli | Vitko Runners (warm starts + test splitting)Vitko | 140 s | 368 s | 272 s | 286 s |
| Java | guava | Vitko Runners (warm starts + test splitting)Vitko | 651 s | 1167 s | 655 s | 735 s |
| TypeScript and Node | ts | Vitko Runners (warm starts + test splitting)Vitko | 243 s | 784 s | 372 s | 401 s |
| TypeScript and Node | vite | Vitko Runners (warm starts + test splitting)Vitko | 199 s | 603 s | 350 s | 360 s |
| TypeScript and Node | vue | Vitko Runners (warm starts + test splitting)Vitko | 32 s | 121 s | 63 s | 76 s |
| TypeScript and Node | monorepo | Vitko Runners (warm starts + test splitting)Vitko | 172 s | 384 s | 180 s | 177 s |
| Python | flask | Self-hosted runners on Google Cloud | 11 s | 23 s | 16 s | 22 s |
| Python | pydantic | Vitko Runners (warm starts + test splitting)Vitko | 69 s | 179 s | 105 s | 103 s |
| Docker image build | container | RunsOn | 174 s | 239 s | 179 s | 386 s† |
† Layer cache did not hit in this run; a re-run is planned. See method. Every workload page lists all nine configurations with p95, queue time and cost: all workloads. List price per minute is not cost per job; faster runners finish sooner.
More: all questions answered · 12 options compared · ranked by speed · ranked by cost per job · free minutes.
Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 28, 2026.