Short answer
On CI Bench v1's container workload (a full Docker image build of mastodon/mastodon v4.7.2, linux/amd64), the lowest median time was RunsOn at 174 s, then Vitko Runners (warm starts + test splitting) 178 s and Self-hosted runners on Google Cloud 178 s; GitHub-hosted (ubuntu-latest) took 239 s. Blacksmith's 386 s is for a run in which its Docker layer cache did not hit; a re-run is planned. It is one project's build, so measure your own. Maintained by Vitko, Inc., which sells one of the products measured.
Container workload, ordered by median time
| Configuration | Type | Median time | p95 time | vs GitHub-hosted (ubuntu-latest) | Mean cost per job | Note |
|---|---|---|---|---|---|---|
| RunsOn | Self-hosted | 174 s | 188 s | 1.4x faster | $0.0060 | |
| Vitko Runners (warm starts + test splitting)Vitko | Managed | 178 s | 208 s | 1.3x faster | $0.0061 | |
| Self-hosted runners on Google Cloud | Self-hosted | 178 s | 201 s | 1.3x faster | $0.053 | |
| Vitko RunnersVitko | Managed | 179 s | 199 s | 1.3x faster | $0.0060 | |
| GitHub-hosted larger runner (4 vCPU) | Managed | 185 s | 215 s | 1.3x faster | $0.042 | |
| Ubicloud | Managed | 213 s | 262 s | 1.1x faster | $0.0047 | |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | Managed | 239 s | 282 s | baseline | $0.027 | |
| Self-hosted runners on AWS (one VM per job) | Self-hosted | 247 s | 300 s | 1.0x slower | $0.044 | |
| Blacksmith | Managed | 386 s† | 663 s | 1.6x slower | $0.031 |
The job is the project's own build-container-image workflow with the portability edits listed on the workload page: each run changes one file so dependency layers can come from cache and later layers rebuild, using upstream's type=gha layer cache; providers' own layer caches were used where documented. 30 runs per configuration. List price per minute is not cost per job; faster runners finish sooner.
Docker layer caching, as each provider publishes it
| Provider | What its page says | Source |
|---|---|---|
| Blacksmith | Docker layer caching is an add-on at +$0.50 per GB a month; its docs list a required cache-key input. | Blacksmith pricing, read 2026-10-27 |
| Depot | Sells container builds as a separate product (500 minutes included on Developer, 5,000 on Startup, then $0.04 a minute); its GitHub Actions runners use its cache system. | Depot pricing, read 2026-10-27 |
| WarpBuild | Lists a Docker layer cache and Docker builders from $0.06 a minute (16 vCPU, 32 GB); cache storage $0.20 per GB-month. | WarpBuild pricing, read 2026-10-27 |
| Namespace | Plans include Docker builds (Team 1,000, Business 2,500; Developer $0.05 per build) shared with unit minutes. | Namespace pricing, read 2026-10-27 |
| RunsOn | An ephemeral ECR registry in your AWS account for BuildKit cache; type=gha and type=s3 caches also work. | RunsOn, read 2026-10-27 |
| Vitko Runners | No Docker layer caching as a feature; warm starts can save Docker images, which is not layer caching. Docker works normally in jobs. | Vitko Runners limits, read 2026-10-27 |
How to choose for Docker-heavy CI
- Measure your own Dockerfile: the order of layers decides how much a cache can save. The workflow in the repository is a starting point.
- Compare the cache's price with the minutes it saves: some providers charge for cache storage or builders separately (table above).
- If your builds already use a registry or
type=ghacache, ask whether the provider's cache replaces or adds to it.
More: 12 options compared · ranked by speed · ranked by cost per job · price per minute, 2 and 4 vCPU · self-hosted vs managed · alternatives to GitHub-hosted.
Provider pages may have changed since October 27, 2026; they are the authority. Method: how the runs were made; data: results.json; every raw record: the repository. Maintained by Vitko, Inc., which sells one of the products measured. Page dated October 27, 2026.
† Layer cache did not hit in this run; a re-run is planned. See method.