Compared with Blacksmith
| Configuration | Type | Speed vs Blacksmith | Cost per job vs Blacksmith (10,000 jobs a month) | Detail |
|---|---|---|---|---|
| Vitko Runners (warm starts + test splitting)Vitko | Managed | 1.5x faster | 67% lower | head to head |
| Vitko RunnersVitko | Managed | 1.1x faster | 67% lower | head to head |
| GitHub-hosted larger runner (4 vCPU) | Managed | about the same | 188% higher | head to head |
| Self-hosted runners on Google Cloud | Self-hosted | about the same | 15% lower | head to head |
| RunsOn | Self-hosted | 1.3x slower | 48% lower | head to head |
| Self-hosted runners on AWS (one VM per job) | Self-hosted | 1.3x slower | 18% lower | head to head |
| Ubicloud | Managed | 1.4x slower | 64% lower | head to head |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | Managed | 1.6x slower | 108% higher | head to head |
Ratios come from the time and cost indexes in results.json (geometric mean over 12 workloads, each against GitHub-hosted (ubuntu-latest)), divided by Blacksmith's. List prices dated 2026-10-05; List price per minute is not cost per job; faster runners finish sooner. Method: how the runs were made.
Providers not measured
- Depot: Not measured in v2: Depot runners were assigned but did not start jobs for our account during the test window (checks on 2026-10-01 and 2026-10-24).
- WarpBuild: Not benchmarked
- Namespace: Not yet measured; joins in v2.
- BuildJet: Shut down (per its own docs, read 2026-10-24).
- Sengi: Not measured.
- CRACI: Not measured.
Their list prices, with sources, are on the list-prices page (read 2026-10-24).
Maintained by Vitko, Inc., which sells one of the products measured. Page generated October 27, 2026.