Short answer
In the CI Bench v1 burst workload (20 jobs dispatched at once), the lowest median time was Blacksmith at 97 s; GitHub-hosted (ubuntu-latest) took 259 s. Queue time varied widely: median 3.0 s on GitHub-hosted against 18.0 s on Vitko Runners, whose p95 queue was 294.6 s against GitHub-hosted's 31.3 s. One burst of 20 on default capacity, so treat it as one data point. Maintained by Vitko, Inc., which sells one of the products measured.
Burst workload, ordered by median time
| Configuration | Median time | p95 time | Queue p50 | Queue p95 | Cost for all 20 jobs | OK |
|---|---|---|---|---|---|---|
| Blacksmith | 97 s | 190 s | 9.0 s | 16.0 s | $0.166 | 30 of 30 |
| Vitko Runners (warm starts + test splitting)Vitko | 131 s | 191 s | 17.3 s | 115.1 s | $0.052 | 30 of 30 |
| Self-hosted runners on Google Cloud | 153 s | 165 s | 2.0 s | 2.5 s | $0.135 | 30 of 30 |
| GitHub-hosted larger runner (4 vCPU) | 154 s | 179 s | 3.0 s | 38.6 s | $0.636 | 30 of 30 |
| Vitko RunnersVitko | 160 s | 372 s | 18.0 s | 294.6 s | $0.056 | 30 of 30 |
| Self-hosted runners on AWS (one VM per job) | 161 s | 182 s | 65.3 s | 92.6 s | $0.111 | 30 of 30 |
| RunsOn | 177 s | 340 s | 22.5 s | 193.6 s | $0.051 | 30 of 30 |
| Ubicloud | 203 s | 271 s | 18.0 s | 127.5 s | $0.062 | 30 of 30 |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | 259 s | 283 s | 3.0 s | 31.3 s | $0.500 | 30 of 30 |
Time and cost cover the jobs of the burst; queue time is per job. Vitko Runners and the other managed services ran at default capacity with nothing sized for the benchmark. Workload details: burst workload page. List price per minute is not cost per job; faster runners finish sooner.
More: all questions answered · 12 options compared · ranked by speed · ranked by cost per job · burst workload.
Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 28, 2026.