CI Bench

Managed · CI Bench v1

GitHub-hosted larger runner (4 vCPU)

1.6x faster than GitHub's 2 vCPU runner and 39% higher cost per job, averaged over 12 workloads.

Speed
1.6×
faster than GitHub-hosted (ubuntu-latest)
3rd of 9 by speed
Cost per job
+39%
vs GitHub-hosted (ubuntu-latest)
9th of 9; the same at 1,000 to 100,000 jobs a month
Lowest median time
0 / 12
workloads
none of the workloads
Failed runs
3 of 360
measured runs
codex-lint 1 of 30; vite 2 of 30. Counted, not replaced.

Speed on each workload

× faster than GitHub-hosted (ubuntu-latest) on the same workload
Burst: 20 jobs at once
1.7×
cli
1.9×
codex-lint
1.9×
container
1.3×
fd
1.4×
flask
1.1×
guava
1.2×
monorepo
1.7×
pydantic
1.2×
ts
1.8×
vite
2.1×
vue
1.8×

Configuration

Managed · 4 vCPU / 16 GB
Measured configuration
GitHub-hosted, 4 vCPU
Size
4 vCPU / 16 GB
Label or setup
ubuntu-24.04 in a public repository (GitHub's free 4 vCPU / 16 GB standard runner), priced at the 4-core list price
Price as measured
$0.012 a minute, each job rounded up to a whole minute; list price on 2026-10-05 (source)
Definition
providers/gh4.json

Per workload

Green marks the lowest median time on the workload. The burst row covers all 20 jobs.
WorkloadMedian timevs GitHub-hosted (ubuntu-latest)p95 timeMean costvs GitHub-hosted (ubuntu-latest)Successful runsRank
Burst: 20 jobs at once154 s1.7x faster179 s$0.63627% higher30 of 304th
cli192 s1.9x faster198 s$0.04515% higher30 of 302nd
codex-lint476 s1.9x faster511 s$0.09913% higher29 of 304th
container185 s1.3x faster215 s$0.04259% higher30 of 305th
fd66 s1.4x faster81 s$0.01960% higher30 of 305th
flask21 s1.1x faster24 s$0.012100% higher30 of 304th
guava946 s1.2x faster983 s$0.18166% higher30 of 308th
monorepo220 s1.7x faster235 s$0.04617% higher30 of 304th
pydantic154 s1.2x faster173 s$0.03369% higher30 of 306th
ts435 s1.8x faster548 s$0.09928% higher30 of 305th
vite288 s2.1x faster315 s$0.0601% lower28 of 302nd
vue67 s1.8x faster88 s$0.02043% higher30 of 303rd
Burst queue, p95
39 s, against 31 s for GitHub-hosted (ubuntu-latest)
Indexes and 95% intervals
Time index
0.642 (95% interval 0.629 to 0.669)
Cost index
1.387 (1.350 to 1.424) at 100,000 jobs a month

Compare with…

Head-to-head pages: speed, cost, failures and each workload