Lowest median time
174 s
RunsOn
GitHub-hosted (ubuntu-latest): 239 s
Slowest median
386 s
Blacksmith
2.2× the lowest
Lowest cost, managed
$0.0047
Ubicloud
mean per job
Failed runs
0
no failed runs
of 270 measured runs
Median time
Seconds from start to finish · lower is betterTable
Ordered by median time. Self-hosted options are priced all-in at 100,000 jobs a month. Green marks the lowest median time and the lowest cost among the managed runners.| Configuration | Median | p95 | Queue p50 | Queue p95 | Cost | OK |
|---|---|---|---|---|---|---|
| RunsOn | 174 s | 188 s | 18 s | 21.5 s | $0.0060 | 30 of 30 |
| Vitko Runners (warm starts + test splitting) | 178 s | 208 s | 1 s | 2 s | $0.0061 | 30 of 30 |
| Self-hosted runners on Google Cloud | 178 s | 201 s | 1 s | 2 s | $0.053 | 30 of 30 |
| Vitko Runners | 179 s | 199 s | 1 s | 2 s | $0.0060 | 30 of 30 |
| GitHub-hosted larger runner (4 vCPU) | 185 s | 215 s | 3 s | 4.5 s | $0.042 | 30 of 30 |
| Ubicloud | 213 s | 262 s | 18 s | 21.5 s | $0.0047 | 30 of 30 |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | 239 s | 282 s | 3 s | 3 s | $0.027 | 30 of 30 |
| Self-hosted runners on AWS (one VM per job) | 247 s | 300 s | 52.5 s | 84.7 s | $0.044 | 30 of 30 |
| Blacksmith | 386 s | 663 s | 8 s | 9 s | $0.031 | 30 of 30 |
The job
- Project: mastodon/mastodon at
v4.7.2(3987b9fd624010eeeaeb7cad96606b05aa813f6a), licence AGPL-3.0. Fetched at run time, not vendored. - Upstream job:
.github/workflows/build-container-image.yml, job build-image (linux/amd64). - Benchmark workflow:
.github/workflows/wl-container.yml.
Portability edits (identical for every provider)
- No registry login or push.
- Built with upstream's PR-build layer cache (
type=gha). Each run first appends a comment toconfig/application.rb, as a pull request would change the app: dependency layers can come from the cache, and everything afterCOPY . /opt/mastodon/rebuilds. Providers' own Docker layer caches (where documented) are used the same way.
suite/container holds the workflow and notes.