Self-hosted options spread fixed monthly costs over the monthly volume, so their cost per job changes with it; see the volume table.
Speed
× faster than GitHub-hosted (ubuntu-latest) · higher is betterCost per job
USD per $1.00 spent on GitHub-hosted (ubuntu-latest), at 100,000 jobs a month · lower is betterWorkload by workload
Self-hosted runners on AWS had the lowest median time of the two on 9 of 12 workloads, GitHub-hosted (ubuntu-latest) on 3.| Workload | Self-hosted runners on AWS (one VM per job) | GitHub-hosted runners (ubuntu-latest, 2 vCPU) | Lowest median |
|---|---|---|---|
| Burst: 20 jobs at once | 161 s | 259 s | Self-hosted runners on AWS (one VM per job) |
| cli | 357 s | 368 s | Self-hosted runners on AWS (one VM per job) |
| codex-lint | 512 s | 884 s | Self-hosted runners on AWS (one VM per job) |
| container | 247 s | 239 s | GitHub-hosted runners (ubuntu-latest, 2 vCPU) |
| fd | 114 s | 92 s | GitHub-hosted runners (ubuntu-latest, 2 vCPU) |
| flask | 70 s | 23 s | GitHub-hosted runners (ubuntu-latest, 2 vCPU) |
| guava | 717 s | 1167 s | Self-hosted runners on AWS (one VM per job) |
| monorepo | 281 s | 384 s | Self-hosted runners on AWS (one VM per job) |
| pydantic | 154 s | 179 s | Self-hosted runners on AWS (one VM per job) |
| ts | 433 s | 784 s | Self-hosted runners on AWS (one VM per job) |
| vite | 403 s | 603 s | Self-hosted runners on AWS (one VM per job) |
| vue | 117 s | 121 s | Self-hosted runners on AWS (one VM per job) |
Cost by volume
Mean cost per job in USD, from the benchmark’s sensitivity results| Volume | Self-hosted runners on AWS (one VM per job) | GitHub-hosted runners (ubuntu-latest, 2 vCPU) |
|---|---|---|
| 100,000 jobs a month | $0.383 | $1.000 |
| 10,000 jobs a month | $0.395 | $1.000 |
| 1,000 jobs a month | $0.508 | $1.000 |
Failed runs
Counted, never replaced- Self-hosted runners on AWS (one VM per job)
- 0 of 360 measured runs (0.0%)
- GitHub-hosted runners (ubuntu-latest, 2 vCPU)
- 3 of 360 measured runs (0.8%)
Each configuration page lists where its failures happened. All losses.
All results for this pair
One line per configuration pairing, generated from the benchmark data- Self-hosted runners on AWS (one VM per job) ran jobs 1.2× faster than GitHub-hosted runners (ubuntu-latest, 2 vCPU) (95% interval 1.1× to 1.2×); Self-hosted runners on AWS (one VM per job) cost 62% less per job (at 100,000 jobs a month).