Sage

Report3 min readUpdated 20 November 2026

Benchmarks

On 36 Python, Rust and Go tasks with hidden tests, Sage solved as many as OpenCode on DeepSeek V4.1 Flash for about 40% less cost and time, and more than OpenCode on GLM-5.3 for less than half the cost.

Sage solved
44/72
vs OpenCode’s 44/72
Sage cost
$2.73
vs OpenCode’s $4.51
Sage time per run
3.8 min
vs OpenCode’s 6.2 min
Sage cost per solve
6.2¢
vs OpenCode’s 10.3¢

Python, Rust and Go

36 Exercism tasks, 12 per language. Tests stay hidden until the agent finishes. Each harness ran every task twice per model: 72 runs. Sage 0.3.0 against OpenCode 1.19.5, on 17 November 2026.

Solved, out of 72

DeepSeek V4.1 Flash
Sage44
OpenCode44
GLM-5.3
Sage31
OpenCode19

Total cost, 72 runs

DeepSeek V4.1 Flash
Sage$2.73
OpenCode$4.51
GLM-5.3
Sage$8.86
OpenCode$21.65

Time per run

DeepSeek V4.1 Flash
Sage3.8 min
OpenCode6.2 min
GLM-5.3
Sage4.6 min
OpenCode11.5 min
ModelHarnessSolvedPythonRustGoCostTime per run
DeepSeek V4.1 FlashSage44/7213/2417/2414/24$2.733.8 min
OpenCode44/7213/2417/2414/24$4.516.2 min
GLM-5.3Sage31/728/2415/248/24$8.864.6 min
OpenCode19/723/249/247/24$21.6511.5 min
  • Same effort. Both harnesses asked the model for low reasoning effort. Sage raises it only after three surprises in a row.
  • Why OpenCode was slow on GLM-5.3. Each request wrote five to six times as much and took 30 s against 17 s. 18 runs hit the 20-minute or 60-request limit, against one of Sage’s.
  • Isolated. Each harness ran under its own account, with no network except the model gateway and no way to read the hidden tests. OpenCode tried to download the official tests in most runs; every attempt was blocked.
  • Reruns. Runs that ended on a provider error or a fault of our own machine were run again from scratch.
  • Checked for leaks. On DeepSeek, three runs reproduced the hidden tests or solution almost in full, most likely from memory; without them Sage is 44/71 and OpenCode 42/70. On GLM-5.3, no solved run did.

Earlier: 24 project tasks

Bug fixes, small modules, skill tasks (security audits, dead-code cleanup, tests, performance, debugging, UI) and multi-file tasks with strict specs. Three runs per task on DeepSeek V4.1 Flash: 72 runs, on 12 November 2026, with an earlier Sage.

Solved, out of 72

Sage67
OpenCode70

Total cost, 72 runs

Sage$1.12
OpenCode$2.23

Time per run

Sage59 s
OpenCode116 s

Sage used 5.7 requests per run against OpenCode’s 10.8. Three changes measured that week, each on its own, are still how Sage works:

Start with less reasoning
Same accuracy in half the time.
Allow whole-file rewrites
Refused writes fell from more than ten per 48 runs to a few.
Stick to the task’s own tests
The test watcher no longer follows throwaway scripts, removing 14 to 40 false switches per 48 runs.

Limits and fairness

Gaps of two or three solved runs are within normal model variance. A miss that repeats on one task is the real signal. The results are from two models, DeepSeek V4.1 Flash and GLM-5.3, both through one gateway; other models may rank the harnesses differently.

Times and costs vary through the day, so compare only runs made side by side.

On your own code, results can differ from other harnesses in either direction: a benchmark is a sample, and your work is the test that counts.

The method, raw logs and per-run results for both tests are in the Sage repository, in docs/BENCHMARKS.md.

Try it on your own work

The fairest comparison is a task you already have. Download Sage, add a key for the model you use, and give it something real in a repository you can reset. Keep the effort on Auto: it starts low and rises by itself when the task needs it, which is where Sage saves the most.