Writer says AI harness cut token use 38% on enterprise AI tasks

Writer researchers say an optimized AI orchestration layer reduced token use per task by 38% and lowered blended task cost by 41% across a controlled enterprise benchmark. The finding matters for AI teams because the work targets the harness around the model, a layer engineering teams can change without fine-tuning or swapping the foundation model.

The paper compares a conventional production agent loop with the Writer Agent Harness on 22 locked enterprise tasks. Those tasks covered grounding and retrieval, multi-step workflows, tool use and content generation. The researchers held the tasks and models constant to isolate the effect of the orchestration layer.

Writer reported that tokens per task fell from 14.2k to 8.8k. Blended cost per task dropped from 21 cents to 12 cents. The company also said cost per successful task declined by as much as 61%, although results varied by model and task. Task success moved from 78% to 81%, a gain the researchers described as directional rather than statistically significant at the benchmark size.

Median end-to-end latency also fell, according to the paper, from 48 seconds to 27 seconds. Writer attributes that reduction to prompt caching and fewer dead-end reasoning loops. The study did not report external replication, customer deployment results or a broader production workload sample.

What Writer changed

The harness is the software layer that formats prompts, routes work, manages tools, handles retrieval and decides what context is passed into the model. Writer’s argument is that many teams spend months comparing models while leaving this layer as thin glue code, even though it determines much of the cost per completed task.

The optimized harness uses techniques including system prompt caching, interaction history compaction, tool management, retrieval controls and error handling. In one pattern described by Writer, search can be delegated to a focused sub-agent that receives only the tool and query it needs, then returns a capped summary to the main agent. That keeps raw search output from accumulating in the primary context window.

Waseem AlShikh, Writer’s CTO and co-founder, told VentureBeat that teams often watch model price per token while ignoring tokens consumed per task. His view is that agent loops can compound spend because each retry can send a growing context back through the model.

The model still matters

The experiments used six models: Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1 and Writer’s Palmyra X6. Writer’s own model is part of the test set, which is relevant context for readers assessing the results.

The paper also found limits to orchestration. Smaller models struggled with sub-agent delegation. Gemini Flash 3.5 and Qwen 3.6 scored 0.45 and 0.42, respectively, below what the researchers considered a usable reliability threshold. Only Palmyra X6 and Claude Sonnet 4.6 crossed that threshold, with scores of 0.86 and 0.85.

That result undercuts a common assumption in agent design: adding more structure does not automatically make a cheaper model production-ready. Extra scaffolding consumes tokens and requires the model to follow more instructions. For weaker models, coordination overhead can erase the intended savings.

Practical takeaways for AI teams

Writer’s recommended playbook starts with prompt structure. Static content such as rules, tool schemas and standard procedures should sit in a stable prefix so model APIs can cache it. Dynamic user requests and recent state should sit lower in the prompt. The point is to avoid paying repeatedly for instructions that do not change.

The second recommendation is to move history and intermediate work out of the active context window, then retrieve only what the current step requires. Writer also argues for hard controls in code: per-task token budgets, caps on steps and tool calls, recursion limits and spend caps after failed validation.

The tradeoff is engineering overhead. Writer’s findings are most relevant to teams running high-volume agentic systems, not early prototypes where a strong model and light harness may be cheaper to build. The broader signal is that enterprise AI cost control is shifting from model selection alone to system design around the model.

This story draws on original reporting from VentureBeat.