Correct and runnable?
Fraction of all tasks where the candidate compiles, executes, and passes the correctness oracle.
Official results · arXiv:2605.15222v1
Systems code
should be fast.
PerfCodeBench measures whether LLMs can optimize real, executable systems code without breaking semantics—across correctness, baseline improvement, and expert-level efficiency.
1,854executable tasks
20evaluated models
6languages
CPU + GPUhardware regimes
01 — Interactive leaderboard
Reliability and speed, ranked separately.
Search models, filter by provider, and sort on any metric. Switch views for the CPU–GPU gap and language-level results reported in the paper.
02 — Evaluation lenses
One benchmark. Five distinct questions.
Efficiency never overrides correctness: a candidate must compile, run, and pass its task-specific oracle before receiving performance credit.
Faster than baseline?
Fraction of all tasks where correct generated code improves on the baseline runtime.
At expert level?
Fraction of comparable correct tasks where the candidate matches or beats the expert reference.
How much gap closed?
Correctness-gated progress across the baseline-to-reference performance gap, clamped to [0, 1].
Strong efficiency?
Fraction of comparable tasks closing at least 80% of the available expert improvement.
03 — What the ranking reveals
The benchmark is far from saturated.
The leader depends on what you value: executable reliability, baseline improvement, or closing the expert optimization gap.
Correctness ≠ efficiency.
GPT-5.4 leads CRR and FBR, while GPT-5 leads RBR, CGRE, and CGRE≥0.8. Generating runnable speedups and reaching expert efficiency are related—but distinct—skills.
CUDA is a separate frontier.
Across six strong models, GPU correctness and efficiency remain dramatically below CPU results. DeepSeek-V4-Pro posts the strongest GPU slice, yet the gap remains large.
Rare successes can still be strong.
Models such as Kimi K2.6 and Qwen3.6-27B have low CRR but high conditional efficiency, showing why a single aggregate score is insufficient.