Meta EvoHarness-RL paper reports 8B model matched Claude Opus 4.5 on ALFWorld

Meta EvoHarness-RL researchers at Meta AI and the University of Illinois Urbana-Champaign reported that a Qwen3-8B agent reached a 96.9% success rate on ALFWorld, a text-based benchmark for multi-step tasks. The reported score was 0.5 percentage points above the 96.4% result for unaided Claude Opus 4.5 in the same reported evaluation, but it does not show that the smaller model matches Anthropic's system across coding, enterprise workflows or general reasoning.

The work is described in an arXiv manuscript, EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents, submitted August 5, 2026. The paper lists authors affiliated with Meta AI and UIUC and is listed by arXiv as accepted to LLA@COLM 2026. It is a research result, not an independently reproduced commercial deployment.

What did Meta EvoHarness-RL actually test?

EvoHarness-RL is not a new Meta foundation model. It is a training method applied here to Qwen3-8B, which teaches an agent how to manage the external runtime support around a model while it carries out a long task.

The authors define that support, or harness, as components including prompts, tools, retrieval, memory, state trackers, execution feedback and control-flow mechanisms. Their system groups this external state into three categories: Belief, covering the agent's current view of its environment; Progress, covering completed and remaining subgoals; and Experience, covering knowledge that can be reused across tasks.

The agent can perform four compact actions: query or track its beliefs, commit updates to progress, recall prior experience and write notes for future use. The technical claim is that the model learns when to take those actions rather than relying on a fixed set of instructions written by developers.

Training proceeds in two stages. Supervised harness fine-tuning teaches the base model the action protocol and how to create useful external state from interaction traces. A second stage, cost-aware Group Relative Policy Optimization, trains selective reading, updating and consolidation of that state. Because external-state access consumes tokens and compute, the method is designed to learn when using it is worthwhile.

How narrow is the Claude Opus 4.5 comparison?

The headline result needs its benchmark boundaries. ALFWorld is a text-based environment designed to test sequential logic and state tracking in multi-step tasks. It is not evidence that Qwen3-8B has reached Claude Opus 4.5 performance in broad production work.

The researchers' more meaningful within-model comparison is against a ReAct baseline using the same Qwen3-8B backbone. That baseline scored 47.9%, versus 96.9% with EvoHarness-RL, a reported gain of 49.0 percentage points. A secondary report of the complete table also placed SkillOS at 80.2%, SkillRL at 89.9%, GPT-4.1 with a BPE prompt at 70.0%, and GPT-5 with a BPE prompt at 85.0%. Claude Opus 4.5 equipped with the harness reached 98.5% in that table.

The paper describes two observed effects: “harness annealing,” in which agents move from frequent to selective external-state access, and “harness evolution,” in which progress updates and accumulated experience form a more compact task-specific workspace.

What does the result say about AI agent costs?

Using an 8-billion-parameter model suggests a route to lower-cost agent systems, but the paper does not demonstrate a price advantage over Claude Opus 4.5. It provides no comparable training or serving costs, hardware configuration, latency, token consumption, or end-to-end deployment economics. For operators, the result is evidence that learned coordination of memory and task state can drive benchmark performance; the cost claim remains unproven.

This story draws on original reporting from VentureBeat.