CI Bench

Managed · CI Bench v1

Blacksmith

1.6x faster than GitHub's 2 vCPU runner and 52% lower cost per job, averaged over 12 workloads.

Speed
1.6×
faster than GitHub-hosted (ubuntu-latest)
4th of 9 by speed
Cost per job
−52%
vs GitHub-hosted (ubuntu-latest)
7th of 9; the same at 1,000 to 100,000 jobs a month
Lowest median time
1 / 12
workloads
Burst: 20 jobs at once
Failed runs
3 of 360
measured runs
vite 3 of 30. Counted, not replaced.

Speed on each workload

× faster than GitHub-hosted (ubuntu-latest) on the same workload
Burst: 20 jobs at once
2.7×
cli
1.3×
codex-lint
2.0×
container
0.6×
fd
1.4×
flask
1.0×
guava
1.6×
monorepo
2.2×
pydantic
1.7×
ts
2.0×
vite
1.7×
vue
1.6×

Configuration

Managed · 2 vCPU / 8 GB
Measured configuration
Blacksmith, 2 vCPU
Size
2 vCPU / 8 GB
Label or setup
blacksmith-2vcpu-ubuntu-2404
Price as measured
$0.004 a minute, each job rounded up to a whole minute; list price on 2026-10-05 (source)
Accelerators used
Docker layer cache (useblacksmith/setup-docker-builder + build-push-action) on the container workload; actions/cache is served by Blacksmith's cache automatically.
Definition
providers/blacksmith2.json

Per workload

Green marks the lowest median time on the workload. The burst row covers all 20 jobs.
WorkloadMedian timevs GitHub-hosted (ubuntu-latest)p95 timeMean costvs GitHub-hosted (ubuntu-latest)Successful runsRank
Burst: 20 jobs at once97 s2.7x faster190 s$0.16667% lower30 of 301st
cli286 s1.3x faster323 s$0.02048% lower30 of 304th
codex-lint448 s2.0x faster748 s$0.03461% lower30 of 303rd
container386 s1.6x slower663 s$0.03115% higher30 of 309th
fd65 s1.4x faster81 s$0.005554% lower30 of 304th
flask22 s1.0x faster36 s$0.004033% lower30 of 305th
guava735 s1.6x faster875 s$0.05153% lower30 of 304th
monorepo177 s2.2x faster291 s$0.01465% lower30 of 302nd
pydantic103 s1.7x faster145 s$0.008756% lower30 of 302nd
ts401 s2.0x faster570 s$0.03061% lower30 of 303rd
vite360 s1.7x faster511 s$0.02657% lower27 of 304th
vue76 s1.6x faster104 s$0.007745% lower30 of 305th
Slower than GitHub-hosted (ubuntu-latest)
container (1.6x slower)
Burst queue, p95
16 s, against 31 s for GitHub-hosted (ubuntu-latest)
Indexes and 95% intervals
Time index
0.645 (95% interval 0.606 to 0.759)
Cost index
0.482 (0.435 to 0.537) at 100,000 jobs a month

Compare with…

Head-to-head pages: speed, cost, failures and each workload