Short answer
It depends on where you want the runners and what you need. In CI Bench v1, RunsOn was 1.2x faster GitHub-hosted (ubuntu-latest), 75% lower cost per job at 10,000 jobs a month; Blacksmith was 1.6x faster GitHub-hosted (ubuntu-latest), 52% lower cost per job at 10,000 jobs a month. Depot and Namespace were not measured (Depot's runners did not start jobs for our account during the test window; Namespace joins in the next result set), so for them only published prices and features are compared. RunsOn runs in your own AWS account; the other three are managed services. Maintained by Vitko, Inc., which sells one of the products measured.
Side by side
| Topic | RunsOn | Blacksmith | Depot | Namespace |
|---|---|---|---|---|
| Where it runs | Your AWS account, EC2 (spot by default); you operate the stack | Managed service | Managed service (ephemeral EC2 instances never reused, per its docs) | Managed service |
| Measured speed vs GitHub-hosted | 1.2x faster | 1.6x faster | Not measured | Not measured |
| Measured cost per job vs GitHub-hosted, 10,000 jobs a month | 75% lower | 52% lower | Not measured | Not measured |
| List price, 2 vCPU Linux | AWS instance cost; license from €300 a year | $0.004 a minute | $0.006 a minute, per second | $0.004 a minute (2 vCPU 8 GB) prepaid; $0.006 overage |
| List price, 4 vCPU Linux | AWS instance cost | Not stated on the pricing page read | $0.012 a minute | $0.008 prepaid; $0.012 overage |
| Included usage | License free for non-commercial use | 3,000 free minutes a month | Developer $20 a month with 2,000 minutes; Startup $200 with 20,000 | Team $100 a month with 100,000 unit minutes; Business $250 with 250,000 |
| Platforms listed | Linux and Windows; macOS stays on GitHub | Ubuntu x64 and Arm, Windows, macOS | Linux x64 and Arm, macOS; organization-owned repositories only | Linux AMD64 and ARM64, macOS, Windows |
Sources, read 2026-10-27: RunsOn pricing, Blacksmith pricing, Depot runner types and pricing, Namespace pricing. Measured rows: indexes in results.json, each against GitHub-hosted (ubuntu-latest); List price per minute is not cost per job; faster runners finish sooner. Blacksmith's container cell: its Docker layer cache did not hit in that run (method).
Who each suits
- RunsOn: teams that want runners inside their own AWS account and are happy to operate the stack.
- Blacksmith: teams that want a managed service with Arm, Windows and macOS runners listed and a measured result to read (Blacksmith vs RunsOn, measured).
- Depot: GitHub-organization repositories that also want its container builds and cache; check the plan allowances.
- Namespace: teams that want one plan across CI, Docker builds and other environments; unit-minute pricing needs a calculation per shape.
Where Vitko Runners fits: it was 1.7x faster GitHub-hosted (ubuntu-latest), 84% lower cost per job at 10,000 jobs a month (2.3x faster with warm starts and test splitting), on Ubuntu 24.04 x64 only, private repositories, in private beta. See Vitko Runners vs Blacksmith and vs RunsOn.
More: 12 options compared · ranked by speed · ranked by cost per job · price per minute, 2 and 4 vCPU · self-hosted vs managed · alternatives to GitHub-hosted.
Provider pages may have changed since October 27, 2026; they are the authority. Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 27, 2026.