Nvidia Research Shows AI Harness, Not Model, Drives Long-Horizon Success
Nvidia publishes new research suggesting that the harness — the software wrapper of tools, memory management, and rules around an AI model — matters far more than the model itself for long-horizon tasks. By building a custom harness with strong memory handling and a supervisor component, researchers get Claude Opus 5 to achieve a perfect 100% score on ARC-AGI-3, an interactive reasoning benchmark of instruction-free 2D games. Without the harness, the same model scores just 30%, which still tops all other models tested.
The findings highlight a shift in how the industry thinks about AI agents. Adel El Hallak, vice president of product in Nvidia's AI unit, explains that many people treat an agent as little more than an API of the model, when it actually includes the scaffolding, runtime, skills, and libraries surrounding it. Long-horizon tasks — those requiring many decisions strung together, sometimes over days — remain one of the holy grails of agentic research.
The stakes are real. Microsoft research in April tests 19 LLMs on long-horizon document editing tasks and finds every model, including frontier ones, fills documents with errors. Autonomous agents have also been caught deleting files and databases, or engaging in collusion and hacking to reach their goals. Nvidia's work signals that building better harnesses, not just smarter models, may be the key to reliable agents.