Self-hosted options spread fixed monthly costs over the monthly volume, so their cost per job changes with it; see the volume table.
Speed
× faster than GitHub-hosted (ubuntu-latest) · higher is betterCost per job
USD per $1.00 spent on GitHub-hosted (ubuntu-latest), at 100,000 jobs a month · lower is betterWorkload by workload
Self-hosted runners on Google Cloud had the lowest median time of the two on 6 of 12 workloads, GitHub-hosted larger runner (4 vCPU) on 6.| Workload | Self-hosted runners on Google Cloud | GitHub-hosted larger runner (4 vCPU) | Lowest median |
|---|---|---|---|
| Burst: 20 jobs at once | 153 s | 154 s | Self-hosted runners on Google Cloud |
| cli | 310 s | 192 s | GitHub-hosted larger runner (4 vCPU) |
| codex-lint | 581 s | 476 s | GitHub-hosted larger runner (4 vCPU) |
| container | 178 s | 185 s | Self-hosted runners on Google Cloud |
| fd | 64 s | 66 s | Self-hosted runners on Google Cloud |
| flask | 11 s | 21 s | Self-hosted runners on Google Cloud |
| guava | 820 s | 946 s | Self-hosted runners on Google Cloud |
| monorepo | 278 s | 220 s | GitHub-hosted larger runner (4 vCPU) |
| pydantic | 119 s | 154 s | Self-hosted runners on Google Cloud |
| ts | 533 s | 435 s | GitHub-hosted larger runner (4 vCPU) |
| vite | 401 s | 288 s | GitHub-hosted larger runner (4 vCPU) |
| vue | 76 s | 67 s | GitHub-hosted larger runner (4 vCPU) |
Cost by volume
Mean cost per job in USD, from the benchmark’s sensitivity results| Volume | Self-hosted runners on Google Cloud | GitHub-hosted larger runner (4 vCPU) |
|---|---|---|
| 100,000 jobs a month | $0.409 | $1.387 |
| 10,000 jobs a month | $0.409 | $1.387 |
| 1,000 jobs a month | $0.409 | $1.387 |
Failed runs
Counted, never replaced- Self-hosted runners on Google Cloud
- 1 of 360 measured runs (0.3%)
- GitHub-hosted larger runner (4 vCPU)
- 3 of 360 measured runs (0.8%)
Each configuration page lists where its failures happened. All losses.
All results for this pair
One line per configuration pairing, generated from the benchmark data- GitHub-hosted larger runner (4 vCPU) ran jobs 1.0× faster than Self-hosted runners on Google Cloud (95% interval 1.0× to 1.1×); Self-hosted runners on Google Cloud cost 70% less per job (at 100,000 jobs a month).