Most developers assume their AI agents fail because the model gets things wrong. But new analysis suggests something more frustrating is happening behind the scenes: your agent might be failing perfectly correctly—while still wasting most of its compute budget.
In a controlled benchmark of 200 tasks, a typical ReAct-style agent burned 90.8% of its retries on errors that could never succeed. Not edge cases. Not bad prompts. Just a structural flaw quietly draining resources.
And chances are, your monitoring dashboard isn’t showing it.
It looks like everything is working—until you look closer
On the surface, things seem fine. Success rates look healthy. Latency is within limits. Retries aren’t exceeding thresholds.
But those metrics hide a critical blind spot: how many retries were doomed from the start?
The benchmark revealed that out of 513 retries, 466 were wasted—triggered by errors that no retry could fix. The biggest culprit? The agent trying to call tools that don’t exist.
The tiny design decision causing massive waste
At the heart of the issue is a common pattern in ReAct agents: letting the language model decide which tool to call by generating a string at runtime.
When the model hallucinates a tool name—like “web_browser” or “sql_query”—the system attempts to fetch it. It fails. Then retries. And retries again.
But here’s the catch: that tool will never exist. So every retry is guaranteed waste.
This isn’t a probability problem. It’s a logic problem. A missing key in a dictionary doesn’t magically appear on the second attempt.
Why this is more dangerous than it sounds
Wasted retries don’t just inflate costs—they crowd out legitimate recovery attempts.
Imagine your agent hits a real issue later, like a rate limit or network timeout. By then, the retry budget may already be exhausted on hallucinated tool calls. The task fails—not because it couldn’t recover, but because it never got the chance.
In the benchmark, 19 out of 21 failures traced back to this exact scenario.
A better approach quietly fixes everything
When the architecture was adjusted, the difference wasn’t incremental—it was structural.
The improved workflow eliminated wasted retries entirely. Not reduced—zero.
Success rate climbed to 100%, and retry usage dropped dramatically, all while keeping latency nearly identical at the high end.
So what changed?
Three fixes that make your agent behave like a system—not a guesser
1. Classify errors before retrying
Not all failures are equal. Some can recover (like timeouts), others can’t (like invalid inputs or missing tools). Once errors are categorized, the system can skip retries that are guaranteed to fail.
2. Replace global retries with per-tool limits
A single global retry counter treats all tools as one system. If one tool fails repeatedly, it drains the entire budget. Per-tool circuit breakers isolate failures so one bad dependency doesn’t take everything down.
3. Move tool selection out of the model
This is the big one. Instead of letting the model generate tool names, map predefined step types to tools in code. The model decides what to do, but not what to call.
The result: hallucinated tool names become structurally impossible.
There’s a trade-off—but it’s usually worth it
Deterministic routing does reduce flexibility. If your system relies on dynamically discovering or composing new tools, you’ll need a more hybrid approach.
But for most production use cases—where tasks follow predictable patterns—the benefits are hard to ignore: lower cost, higher reliability, and far more predictable performance.
The surprising insight: even “healthy” systems are leaking
One of the most revealing findings came from low-error scenarios.
Even at a 5% hallucination rate, where the agent achieved a perfect 100% success rate, over half of all retries were still wasted.
In other words, your system can look flawless on the outside while quietly burning through resources underneath.
Why this matters as AI agents scale
As more teams deploy autonomous agents for workflows—customer support, research, automation—efficiency stops being a nice-to-have. It becomes a cost and reliability constraint.
This issue ties into a broader trend in AI engineering: moving from prompt-driven experimentation to system-level design. The biggest gains aren’t coming from better prompts—they’re coming from better architecture.
We’re seeing a shift from “let the model decide everything” to “let the model reason, but keep control where it matters.”
A quick reality check for your own system
If you’re running a ReAct-style agent today, it’s worth asking:
- Does your system retry when a tool isn’t found?
- Do all tools share the same retry budget?
- Can you track which retries were actually useful?
If the answers are yes, yes, and no—you’re likely burning more budget than you think.
Where this leaves you
This isn’t about abandoning ReAct. It’s about tightening the parts that were never designed for production scale.
A few structural changes can turn an unpredictable, wasteful loop into a controlled, reliable system—without sacrificing performance.
So here’s the real question: if your agent is already succeeding, would you notice if half its effort was going to waste?
