Vitko Runners vs GitHub-hosted larger runner (4 vCPU): speed and cost on 12 real CI workloads

Speed

× faster than GitHub-hosted (ubuntu-latest) · higher is better

Vitko Runners (warm starts + test splitting)

2.3×

Vitko Runners

1.7×

GitHub-hosted larger runner (4 vCPU)

1.6×

GitHub-hosted (ubuntu-latest)

1.0×

USD per $1.00 spent on GitHub-hosted (ubuntu-latest), at 100,000 jobs a month · lower is better

Vitko Runners (warm starts + test splitting)

$0.16

Vitko Runners

$0.16

GitHub-hosted larger runner (4 vCPU)

$1.39

GitHub-hosted (ubuntu-latest)

$1.00

Vitko Runners had the lowest median time of the two on 12 of 12 workloads, GitHub-hosted larger runner (4 vCPU) on 0.

Workload	Vitko Runners (warm starts + test splitting)	Vitko Runners	GitHub-hosted larger runner (4 vCPU)	Lowest median
Burst: 20 jobs at once	131 s	160 s	154 s	Vitko Runners (warm starts + test splitting)
cli	140 s	272 s	192 s	Vitko Runners (warm starts + test splitting)
codex-lint	405 s	404 s	476 s	Vitko Runners
container	178 s	179 s	185 s	Vitko Runners (warm starts + test splitting)
fd	45 s	49 s	66 s	Vitko Runners (warm starts + test splitting)
flask	13 s	16 s	21 s	Vitko Runners (warm starts + test splitting)
guava	651 s	655 s	946 s	Vitko Runners (warm starts + test splitting)
monorepo	172 s	180 s	220 s	Vitko Runners (warm starts + test splitting)
pydantic	69 s	105 s	154 s	Vitko Runners (warm starts + test splitting)
ts	243 s	372 s	435 s	Vitko Runners (warm starts + test splitting)
vite	199 s	350 s	288 s	Vitko Runners (warm starts + test splitting)
vue	32 s	63 s	67 s	Vitko Runners (warm starts + test splitting)

Mean cost per job in USD, from the benchmark’s sensitivity results

Volume	Vitko Runners (warm starts + test splitting)	Vitko Runners	GitHub-hosted larger runner (4 vCPU)
100,000 jobs a month	$0.158	$0.157	$1.387
10,000 jobs a month	$0.158	$0.157	$1.387
1,000 jobs a month	$0.158	$0.157	$1.387

Counted, never replaced

Each configuration page lists where its failures happened. All losses.

One line per configuration pairing, generated from the benchmark data

Vitko Runners (warm starts + test splitting) ran jobs 1.5× faster than GitHub-hosted larger runner (4 vCPU) (95% interval 1.4× to 1.5×); Vitko Runners (warm starts + test splitting) cost 89% less per job (at 100,000 jobs a month).
Vitko Runners ran jobs 1.1× faster than GitHub-hosted larger runner (4 vCPU) (95% interval 1.1× to 1.2×); Vitko Runners cost 89% less per job (at 100,000 jobs a month).