CI Bench

Sourced list · read 2026-10-27 · Updated October 28, 2026

Fresh-VM-per-job GitHub Actions runners: the options

What each provider's own page says about ephemeral machines, with the measured speed and cost per job of the ones CI Bench v1 covers.

Short answer

Options whose own pages say each job gets a fresh, discarded machine: GitHub-hosted runners, Depot, RunsOn (in your AWS account), the terraform-aws-github-runner module (in your account), Vitko Runners (private beta) and, if you configure it that way, Actions Runner Controller. A statement on a page is a claim, not an audit; ask each provider for its security documentation. Speed and cost per job for the measured ones: Vitko Runners (warm starts + test splitting) 2.3x faster GitHub-hosted at 84% lower cost per job (10,000 jobs a month); Vitko Runners 1.7x faster GitHub-hosted at 84% lower cost per job (10,000 jobs a month); Blacksmith 1.6x faster GitHub-hosted at 52% lower cost per job (10,000 jobs a month); Ubicloud 1.1x faster GitHub-hosted at 83% lower cost per job (10,000 jobs a month); RunsOn 1.2x faster GitHub-hosted at 75% lower cost per job (10,000 jobs a month). Maintained by Vitko, Inc., which sells one of the products measured.

What each page says

Ephemeral runner options
OptionWhat its page saysSource
GitHub-hosted runnersRun code in "ephemeral and clean isolated virtual machines".GitHub Docs, secure use, read 2026-10-27
DepotBuilds run on ephemeral EC2 instances that are never reused; single-tenant infrastructure.Depot docs, read 2026-10-27
RunsOnEvery job gets a fresh EC2 instance in your AWS account, destroyed when the job ends.RunsOn, read 2026-10-27
terraform-aws-github-runnerRunners are created on demand and terminated after use (ephemeral), on AWS spot instances in your account.Module README, read 2026-10-27
Actions Runner ControllerRunner scale sets can be ephemeral and container-based; the guarantee depends on how you configure it.GitHub Docs, ARC, read 2026-10-27
Vitko RunnersEvery job runs in its own fresh virtual machine that is deleted when the job ends; during the private beta each design partner’s jobs run on a server dedicated to its organization.Vitko Runners security page, read 2026-10-27

Related: questions to ask a runner service. Vitko sells one of the products named; the statements are restated from public pages and may have changed.

More: all questions answered · 12 options compared · ranked by cost per job · self-hosted vs managed · price per minute.

Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 28, 2026.