Lowest median time
651 s
Vitko Runners (warm starts + test splitting)
GitHub-hosted (ubuntu-latest): 1167 s
Slowest median
1167 s
GitHub-hosted runners (ubuntu-latest, 2 vCPU)
1.8× the lowest
Lowest cost, managed
$0.019
Ubicloud
mean per job
Failed runs
0
no failed runs
of 270 measured runs
Median time
Seconds from start to finish · lower is betterTable
Ordered by median time. Self-hosted options are priced all-in at 100,000 jobs a month. Green marks the lowest median time and the lowest cost among the managed runners.| Configuration | Median | p95 | Queue p50 | Queue p95 | Cost | OK |
|---|---|---|---|---|---|---|
| Vitko Runners (warm starts + test splitting) | 651 s | 661 s | 1 s | 2 s | $0.022 | 30 of 30 |
| Vitko Runners | 655 s | 661 s | 1 s | 2 s | $0.022 | 30 of 30 |
| Self-hosted runners on AWS (one VM per job) | 717 s | 783 s | 51 s | 108.1 s | $0.032 | 30 of 30 |
| Blacksmith | 735 s | 875 s | 8 s | 10 s | $0.051 | 30 of 30 |
| Self-hosted runners on Google Cloud | 820 s | 854 s | 1 s | 2 s | $0.045 | 30 of 30 |
| Ubicloud | 881 s | 1086 s | 19 s | 23 s | $0.019 | 30 of 30 |
| RunsOn | 897 s | 919 s | 18 s | 22.5 s | $0.0096 | 30 of 30 |
| GitHub-hosted larger runner (4 vCPU) | 946 s | 983 s | 3 s | 13.6 s | $0.181 | 30 of 30 |
| GitHub-hosted runners (ubuntu-latest, 2 vCPU) | 1167 s | 1249 s | 3 s | 5.6 s | $0.109 | 30 of 30 |
The job
- Project: google/guava at
v33.7.2(666338cc49d5686c07e3e4d17f0aa1cec25a2be6), licence Apache-2.0. Fetched at run time, not vendored. - Upstream job:
.github/workflows/ci.yml, job "pom.xml on JDK 25 on ubuntu-latest" (Maven install, then verify). - Benchmark workflow:
.github/workflows/wl-guava.yml.
Portability edits (identical for every provider)
- One matrix cell (pom.xml, JDK 25, Linux). No other changes.
Why it replaced okhttp
okhttp (Gradle) was the first Java/Kotlin workload. In the pilot it failed the flakiness criterion (one HTTP/2 test failed in 3 of 6 GitHub-hosted runs) and its IPv6-loopback tests fail on plain cloud VMs, so it was replaced by the alternate before any result set.
suite/guava holds the workflow and notes.