CI Bench

Comparison · Result set v1 · Measured Oct 7–11, 2026 · Updated October 28, 2026

When does self-hosting runners stop being worth it?

Cost per job of self-hosted and managed configurations at three monthly volumes, and where the lines cross.

Short answer

It depends on job volume, because self-hosted setups carry fixed monthly costs that managed per-minute prices do not. In CI Bench v1, RunsOn (self-hosted in your AWS account) cost 43% higher per job than GitHub-hosted (ubuntu-latest) at 1,000 jobs a month, 75% lower at 10,000 and 88% lower at 100,000. Against the managed runners measured it overtook the lowest managed option (Vitko Runners, 84% lower) only at the 100,000 level. Those figures leave out the time you spend operating the setup. Maintained by Vitko, Inc., which sells one of the products measured.

Cost per job against GitHub-hosted (ubuntu-latest), by monthly volume

Cost per job by monthly volume
ConfigurationType1,000 jobs a month10,000 jobs a month100,000 jobs a month
RunsOnSelf-hosted43% higher75% lower88% lower
Vitko RunnersVitkoManaged84% lower84% lower84% lower
Vitko Runners (warm starts + test splitting)VitkoManaged84% lower84% lower84% lower
UbicloudManaged83% lower83% lower83% lower
Self-hosted runners on AWS (one VM per job)Self-hosted49% lower60% lower62% lower
Self-hosted runners on Google CloudSelf-hosted59% lower59% lower59% lower
BlacksmithManaged52% lower52% lower52% lower
GitHub-hosted larger runner (4 vCPU)Managed39% higher39% higher39% higher

Self-hosted options are priced all-in: what the jobs use in your cloud account plus fixed monthly costs (Self-hosted runners on AWS (one VM per job) $3.07 a month; RunsOn $40.83 a month) spread over the volume. Managed prices do not change with volume. List price per minute is not cost per job; faster runners finish sooner.

What the table leaves out

More: all questions answered · 12 options compared · ranked by cost per job · self-hosted vs managed · price per minute.

Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 28, 2026.