Short answer
In CI Bench v1 the two self-hosted setups were not the same design: the AWS one started one on-demand VM per job (m8a.large, 2 vCPU 8 GiB), the Google Cloud one was an always-on pool (n4d-standard-2, 2 vCPU 8 GiB) priced at 50% utilization. Cost per job against GitHub-hosted (ubuntu-latest) at 1,000, 10,000 and 100,000 jobs a month: AWS 49% lower, 60% lower, 62% lower; Google Cloud 59% lower, 59% lower, 59% lower. The pool started jobs faster (median queue 1.0 s against 51.0 s). Both are all-in costs without operations time. Maintained by Vitko, Inc., which sells one of the products measured.
Side by side
| Topic | AWS, one VM per job | Google Cloud, always-on pool |
|---|---|---|
| Design | One on-demand VM per job (m8a.large) | Always-on pool (n4d-standard-2, 50% utilization) |
| Speed vs GitHub-hosted | 1.2x faster | 1.5x faster |
| Cost per job, 1,000 jobs a month | 49% lower | 59% lower |
| Cost per job, 10,000 | 60% lower | 59% lower |
| Cost per job, 100,000 | 62% lower | 59% lower |
| Median queue time (median over workloads) | 51.0 s | 1.0 s |
| Fixed monthly cost in the figures | $3.07 (launcher host) | None (no controller host) |
We operated both; operations time is not in the cost figures. The designs differ, so the gap is not only the cloud. Head to head: self-hosted AWS vs Google Cloud. RunsOn is another self-hosted option on AWS: alternatives to RunsOn. List price per minute is not cost per job; faster runners finish sooner.
More: all questions answered · 12 options compared · ranked by speed · ranked by cost per job · calculator.
Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 29, 2026.