CI Bench

Comparison · Result set v1 · Measured Oct 7–11, 2026

Self-hosted vs managed GitHub Actions runners, measured

3 self-hosted configurations and 4 managed services on the same 12 workloads. Self-hosted cost is all-in (cloud bill plus fixed monthly costs), so it depends on volume.

Side by side

Self-hosted and managed configurations
RankConfigurationTypeSpeed vs GitHub-hostedCost per job at 1,000 a monthAt 10,000At 100,000Failed runs
1Vitko Runners (warm starts + test splitting)VitkoManaged2.3x faster84% lower84% lower84% lower3 of 360
2Vitko RunnersVitkoManaged1.7x faster84% lower84% lower84% lower0 of 360
3BlacksmithManaged1.6x faster52% lower52% lower52% lower3 of 360
4Self-hosted runners on Google CloudSelf-hosted1.5x faster59% lower59% lower59% lower1 of 360
5RunsOnSelf-hosted1.2x faster43% higher75% lower88% lower1 of 360
6Self-hosted runners on AWS (one VM per job)Self-hosted1.2x faster49% lower60% lower62% lower0 of 360
7UbicloudManaged1.1x faster83% lower83% lower83% lower1 of 360

We operated the self-hosted options, which sets how much operations time they took; that time is not in the cost figures. Self-hosted options spread fixed monthly costs over the volume, so their cost per job falls as the volume rises; managed per-minute prices do not change with volume. Limits of the method.

Head to head: Vitko Runners vs Blacksmith · Vitko Runners vs Ubicloud · Vitko Runners vs RunsOn · Vitko Runners vs Self-hosted runners on AWS · Vitko Runners vs Self-hosted runners on Google Cloud · Vitko Runners vs GitHub-hosted larger runner (4 vCPU) · Vitko Runners vs GitHub-hosted (ubuntu-latest) · Blacksmith vs GitHub-hosted (ubuntu-latest).

More: speed ranking · cost ranking · alternatives to GitHub-hosted · monorepo workload · self-hosted vs managed · list prices · free minutes · alternatives to Blacksmith, RunsOn, Ubicloud.

Maintained by Vitko, Inc., which sells one of the products measured. Measured October 7 to 11, 2026 (result set v1); page generated October 27, 2026. Every number comes from results.json; the workflows and every raw job record are in the repository, and the method says how the runs were made.