We partnered with Baseten to build the largest-scale harness leaderboard using 83B tokens across 27,280 agent runs. We wanted to answer two questions: when you switch your model, should you consider switching the harness too? What is the best harness for each model?

Previously, our v1.0 FrontierHarness leaderboard ranked harnesses on a single model, Kimi K3.

V1.1 expands the benchmark to 22 harnesses × 5 models × 62 tasks × 4 attempts, 76× the size of v1.0.

Through our experiments, we found that you should consider switching harnesses when switching models to maximize your experience.