Short answer
Taking the p95 job time divided by the median for each workload and then the median of those ratios across eleven workloads in CI Bench v1, the most consistent configuration was RunsOn (1.05) and the least Blacksmith (1.42); GitHub-hosted (ubuntu-latest) was 1.07. A ratio near 1 means the slowest typical runs take about as long as the median. 30 runs per cell, so tails are rough. Maintained by Vitko, Inc., which sells one of the products measured.
p95 divided by median, across eleven workloads
| Configuration | Type | Median p95 to median ratio | Largest ratio on any workload |
|---|---|---|---|
| RunsOn | Self-hosted | 1.05 | 1.09 |
| Vitko Runners (warm starts + test splitting)Vitko | Managed | 1.07 | 1.73 |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | Managed | 1.07 | 1.31 |
| Self-hosted runners on Google Cloud | Self-hosted | 1.09 | 2.92 |
| Vitko RunnersVitko | Managed | 1.11 | 1.33 |
| GitHub-hosted larger runner (4 vCPU) | Managed | 1.12 | 1.32 |
| Self-hosted runners on AWS (one VM per job) | Self-hosted | 1.22 | 1.91 |
| Ubicloud | Managed | 1.29 | 1.51 |
| Blacksmith | Managed | 1.42 | 1.72 |
Per-workload p95 and p50 are on each workload page. Small samples: 30 runs per cell, so a p95 is set by the second-slowest run or so.
More: all questions answered · 12 options compared · ranked by speed · ranked by cost per job · calculator.
Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 29, 2026.