Leaderboard
How models and coding agents perform on real Android engineering work, as measured by Google's Android Bench. We reproduce published numbers only — we do not run or invent scores. Data as of 16 Oct 2026; refreshed weekly.
Android Bench 2.0 · long-horizon tasks (model + agent)
30 private multi-day tasks (app creation, migrations, platform features, Flutter/React Native → Compose conversions), 5 runs per task, verified with instrumentation tests plus a multimodal visual judge. Pass rate = runs scoring a perfect 1.0; completion rate = continuous 0–1 score.
| # | Model | Agent | Pass rate | 95% CI | Completion | Avg latency (h) | Total cost | |
|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 · claude-code | claude-code | 32.7% | 18.7–47.3 | 84.7% | 15.2 | $215.4 | 84.7% completion · $215.4 cost |
| 2 | GPT 6 Astra · codex | codex | 28% | 13.3–42 | 82.2% | 7.9 | $375.7 | 82.2% completion · $375.7 cost |
| 3 | Claude Fable 5.1 · claude-code | claude-code | 22.7% | 10.7–36 | 82.4% | 22.2 | $492.6 | 82.4% completion · $492.6 cost |
| 4 | GPT 6 Sol · codex | codex | 18% | 7.3–29.3 | 70.2% | 6 | $93.4 | 70.2% completion · $93.4 cost |
| 5 | Qwen3.8 Max · qwen-coder | qwen-coder | 14% | 4.7–24 | 74.3% | 47.2 | $260.2 | 74.3% completion · $260.2 cost |
| 6 | Kimi K3 · kimi-code | kimi-code | 12% | 3.3–22 | 72.8% | 66.2 | $418.3 | 72.8% completion · $418.3 cost |
| 7 | GPT 6 Luna · codex | codex | 10.7% | 4–17.3 | 53.2% | 6.2 | $5.3 | 53.2% completion · $5.3 cost |
| 8 | Gemini 3.8 Flash · antigravity-sdk | antigravity-sdk | 8% | 3.3–13.3 | 47.4% | 12.1 | $34.5 | 47.4% completion · $34.5 cost |
| 9 | Gemini 3.7 Flash · antigravity-sdk | antigravity-sdk | 7.3% | 1.3–14.7 | 50.1% | 9.9 | $26 | 50.1% completion · $26 cost |
| 10 | Claude Sonnet 5 · claude-code | claude-code | 6.7% | 1.3–13.3 | 59.8% | 14.7 | $283.7 | 59.8% completion · $283.7 cost |
Android Bench 1.0 · 100 localized tasks (model only)
Pull-request-sized fixes; frontier models now saturate this set (~90%).
| # | Model | Weights | Score | 95% CI | Avg latency (h) | Total cost | |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5 | closed | 91.8% | 87.4–95.7 | 13.2 | $175.4 | closed weights · $175.4 cost |
| 2 | Claude Fable 5 | closed | 90.9% | 86.2–95.4 | 8.7 | $144.3 | closed weights · $144.3 cost |
| 3 | GPT 5.6 Sol | closed | 90.8% | 85.6–95.6 | 14.2 | $150 | closed weights · $150 cost |
| 4 | Kimi K3 | open | 90.2% | 85–94.6 | 29.6 | $112.9 | open weights · $112.9 cost |
| 5 | GPT 5.6 Luna | closed | 87.6% | 81.8–92.3 | 10.6 | $7.2 | closed weights · $7.2 cost |
| 6 | Qwen3.8 Max | closed | 87% | 82.2–91.6 | 41.2 | $196.7 | closed weights · $196.7 cost |
| 7 | GPT 5.6 Terra | closed | 86.8% | 81.7–91.2 | 7.6 | $48 | closed weights · $48 cost |
| 8 | GPT 5.5 | closed | 80.2% | 73–86.6 | 11.4 | $138.3 | closed weights · $138.3 cost |
| 9 | Claude Sonnet 5 | closed | 76.2% | 68.8–82.5 | 12.3 | $99.9 | closed weights · $99.9 cost |
| 10 | Gemini 3.6 Flash | closed | 75.6% | 67.5–82.1 | 10.2 | $135.5 | closed weights · $135.5 cost |
| 11 | Gemini 3.1 Pro Preview | closed | 74.3% | 66.8–81.5 | 10.5 | $86.6 | closed weights · $86.6 cost |
| 12 | GPT 5.4 | closed | 74.1% | 66.6–81.2 | 8.4 | $83.4 | closed weights · $83.4 cost |
| 13 | Claude Opus 4.8 | closed | 72.4% | 65.1–79.2 | 6.7 | $88 | closed weights · $88 cost |
| 14 | GLM 5.2 | open | 72.2% | 65.3–79.1 | 38.9 | $117 | open weights · $117 cost |
| 15 | Gemini 3.5 Flash | closed | 71.1% | 63.8–78.5 | 28.3 | $165.6 | closed weights · $165.6 cost |
| 16 | Kimi K2.7 Code | open | 70.4% | 63.4–77.1 | 31.8 | $48.1 | open weights · $48.1 cost |
| 17 | Claude Opus 4.7 | closed | 68.7% | 60.9–76.4 | 7 | $96.5 | closed weights · $96.5 cost |
| 18 | Kimi K2.6 | open | 67.6% | 59.8–74.8 | 57.2 | $49.4 | open weights · $49.4 cost |
| 19 | Claude Sonnet 4.6 | closed | 67% | 58.5–75.1 | 16.9 | $127.6 | closed weights · $127.6 cost |
| 20 | Minimax M3 | closed | 63.6% | 56.1–70.7 | 26 | $41.7 | closed weights · $41.7 cost |
| 21 | GLM 5.1 | open | 63.2% | 55.3–70.2 | 17.6 | $53.5 | open weights · $53.5 cost |
| 22 | Gemini 3 Flash Preview | closed | 62.5% | 54.7–69.7 | 13.1 | $30.1 | closed weights · $30.1 cost |
| 23 | Mimo V2.5 Pro | open | 60.8% | 53.2–68.6 | 13.6 | $9.2 | open weights · $9.2 cost |
| 24 | Deepseek V4 Pro | open | 59.5% | 51.4–67.9 | 9 | $3.7 | open weights · $3.7 cost |
| 25 | Qwen3.7 Plus | closed | 57.7% | 50.2–65.6 | 18.5 | $18.6 | closed weights · $18.6 cost |
| 26 | Deepseek V4 Flash | open | 54.7% | 46.9–62.8 | 8.9 | $1.5 | open weights · $1.5 cost |
| 27 | Qwen3.7 Max | closed | 54.2% | 46.9–62.1 | 14.2 | $58.3 | closed weights · $58.3 cost |
| 28 | Gemini 3.5 Flash-Lite | closed | 50.6% | 42.3–58.4 | 5.4 | $34.1 | closed weights · $34.1 cost |
| 29 | Qwen3.6 27b | open | 45.1% | 37.6–52.8 | 25.8 | $97.3 | open weights · $97.3 cost |
| 30 | MiniMax M2.7 | open | 41.6% | 34.7–49.4 | 18.2 | $14.9 | open weights · $14.9 cost |
| 31 | Gemma 4 31b IT | open | 37.1% | 29.9–44.5 | 36.3 | $10.4 | open weights · $10.4 cost |
| 32 | Qwen3.6 35b A3b | open | 37% | 29.1–44.3 | 16.3 | $17.8 | open weights · $17.8 cost |
Sources & caveats
- Android Bench leaderboard — developer.android.com/bench
- Android Bench 2.0 methodology
- "Android Bench 2.0: Pushing the frontier with challenging long-horizon tasks" — Android Developers Blog, 17 Sep 2026
- Per Google: cost and latency are summed over all attempts, so models that fail early can look cheaper; latency includes network transit; pricing changes between releases.