Self-hosted options spread fixed monthly costs over the monthly volume, so their cost per job changes with it; see the volume table.
Speed
× faster than GitHub-hosted (ubuntu-latest) · higher is betterCost per job
USD per $1.00 spent on GitHub-hosted (ubuntu-latest), at 100,000 jobs a month · lower is betterWorkload by workload
Ubicloud had the lowest median time of the two on 5 of 12 workloads, Self-hosted runners on AWS on 7.| Workload | Ubicloud | Self-hosted runners on AWS (one VM per job) | Lowest median |
|---|---|---|---|
| Burst: 20 jobs at once | 203 s | 161 s | Self-hosted runners on AWS (one VM per job) |
| cli | 359 s | 357 s | Self-hosted runners on AWS (one VM per job) |
| codex-lint | 682 s | 512 s | Self-hosted runners on AWS (one VM per job) |
| container | 213 s | 247 s | Ubicloud |
| fd | 87 s | 114 s | Ubicloud |
| flask | 36 s | 70 s | Ubicloud |
| guava | 881 s | 717 s | Self-hosted runners on AWS (one VM per job) |
| monorepo | 276 s | 281 s | Ubicloud |
| pydantic | 159 s | 154 s | Self-hosted runners on AWS (one VM per job) |
| ts | 560 s | 433 s | Self-hosted runners on AWS (one VM per job) |
| vite | 502 s | 403 s | Self-hosted runners on AWS (one VM per job) |
| vue | 114 s | 117 s | Ubicloud |
Cost by volume
Mean cost per job in USD, from the benchmark’s sensitivity results| Volume | Ubicloud | Self-hosted runners on AWS (one VM per job) |
|---|---|---|
| 100,000 jobs a month | $0.173 | $0.383 |
| 10,000 jobs a month | $0.173 | $0.395 |
| 1,000 jobs a month | $0.173 | $0.508 |
Failed runs
Counted, never replaced- Ubicloud
- 1 of 360 measured runs (0.3%)
- Self-hosted runners on AWS (one VM per job)
- 0 of 360 measured runs (0.0%)
Each configuration page lists where its failures happened. All losses.
All results for this pair
One line per configuration pairing, generated from the benchmark data- no clear difference in time between Ubicloud and Self-hosted runners on AWS (one VM per job) (the 95% interval, 0.94 to 1.05, includes 1.00); Ubicloud cost 55% less per job (at 100,000 jobs a month).