Your Recursion Limit Is Not a Budget: Why AI Agent Costs Explode in Production

An agent hit a rate limit, retried, hit it again, and re-planned. It repeated that cycle roughly 4,800 times an hour for sixty-three hours. The bill was forty-two dollars after the first hour, two hundred by the fourth, a thousand by hour twelve, and $4,200 by the time somebody opened a laptop and killed the process. Nothing was broken. Every individual call succeeded or failed exactly as designed.

That postmortem is worth reading in full — Sattyam Jain published it as The Agent That Burned $4,200 in 63 Hours — but the interesting part is not the number. It is that the agent had a step limit configured, and the step limit did not help. This article is about why that happens, and what to put in place instead.

The three multipliers behind a runaway bill

Agent spend does not go wrong linearly. A slow leak in a web service costs twice as much when traffic doubles; an agent loop costs an order of magnitude more because three separate mechanisms multiply each other. Understanding them separately is what makes the fix obvious.

1. Retry without backoff or state

The failure mode in the postmortem above is the canonical one: plan, call a tool, receive a 429, re-plan, call the same tool, receive a 429. The agent is behaving correctly at every step. It has no memory that this exact call already failed, and no instruction that repetition is itself a signal. Left alone, it will do this for as long as the process runs.

This is distinct from a normal retry storm because the retry is not the expensive part. A raw HTTP retry against a rate-limited endpoint costs nothing. It is the re-planning around the retry — a full model call, with the entire conversation as input — that carries the cost.

2. Context re-ingestion during replanning

This is the multiplier almost nobody accounts for, and it is the reason step limits fail as budgets. When an agent replans after a failure, the failure goes into the context so the model can reason about it. Each subsequent turn therefore carries every previous turn, including every previous failure. Input tokens grow on every single step.

The arithmetic is unforgiving. Take a modest agent whose first turn sends 2,000 input tokens and receives 400 output tokens, where each turn appends its predecessor to the context:

step  1    context  2,000 in  ->    400 out
step  2    context  2,400 in  ->    400 out
step  3    context  2,800 in  ->    400 out
...
step 25    context 11,600 in  ->    400 out

input tokens billed across steps 1..25   = 170,000
input tokens if context stayed flat      =  50,000
                                           --------
                            3.4x  on input alone, same step count

Two runs, both terminating cleanly at the configured 25-step limit, can differ by more than threefold in what they bill. The step limit was satisfied in both cases. It is simply not measuring the thing that costs money. And because output tokens are priced higher than input tokens on every major provider, a run that also grows its responses diverges faster still.

3. Tool-call and sub-agent fan-out

A single user request that spawns three sub-agents, each of which makes four tool calls, each of which triggers one model call to interpret its result, is twelve model calls plus the orchestration turns above them. Nothing here is a bug. It is the architecture working. But the step count a framework sees at the top level is not the step count actually executed, and a limit set on the parent graph does not constrain what the children do.

What framework limits actually cap

It is worth being precise about this rather than vague. LangGraph's documented guard is recursion_limit, configured on invoke as graph.invoke({...}, {"recursion_limit": 1000}). It counts steps and raises when the graph exceeds them. Reading the official error documentation for it, the notable thing is what is absent: there is no reference to cost, spend, or tokens anywhere in it. That is not an oversight in the docs. It reflects what the mechanism is for.

Every row except the last is a proxy. Proxies are fine when the relationship to the underlying quantity is stable, and the argument of this article is that for agent runs it is not stable — context growth breaks it by design.

Where the guard belongs: the client boundary

The instinct is to add budget tracking to the orchestration layer, next to the step limit. That is the wrong layer, for a reason that is structural rather than stylistic: the orchestrator does not know what anything costs. It sees nodes and edges. It does not see token counts, and it certainly does not see the asymmetric input/output pricing that determines the actual bill.

The model client does. Every major provider returns token usage on the response object — Anthropic's usage.input_tokens and usage.output_tokens, and the equivalent usage object from OpenAI. That makes the client the one place in the stack where actual spend is observable at the moment it is incurred, which makes it the only honest place to enforce a ceiling.

Implementation: a spend ceiling in about forty lines

The accounting itself is unglamorous, which is the point — it needs to be simple enough that nobody is tempted to skip it. Prices are constructor arguments rather than constants, because they change and a hardcoded price is a silent inaccuracy the moment it does.

from dataclasses import dataclass


class BudgetExceeded(RuntimeError):
    """Raised before a call that would run past the ceiling."""


@dataclass
class Budget:
    ceiling_usd: float
    input_per_mtok: float   # price per 1M input tokens
    output_per_mtok: float  # price per 1M output tokens

    spent_usd: float = 0.0
    calls: int = 0

    def charge(self, input_tokens: int, output_tokens: int) -> float:
        """Record one call. Returns what it cost."""
        cost = (
            (input_tokens / 1_000_000) * self.input_per_mtok
            + (output_tokens / 1_000_000) * self.output_per_mtok
        )
        self.spent_usd += cost
        self.calls += 1
        return cost

    def check(self) -> None:
        if self.spent_usd >= self.ceiling_usd:
            raise BudgetExceeded(
                f"spent ${self.spent_usd:.2f} of ${self.ceiling_usd:.2f} "
                f"ceiling across {self.calls} calls"
            )

The wrapper is where the ordering matters. Check before the call, charge after it. Checking after would let a single expensive call run unbounded past the ceiling; checking before means the run stops within one call of the limit, and that one call is bounded by max_tokens.

class BudgetedMessages:
    """Drop-in wrapper around client.messages with a hard spend ceiling."""

    def __init__(self, messages, budget: Budget):
        self._messages = messages
        self._budget = budget

    def create(self, **kwargs):
        # Before: refuse to start a call we cannot afford.
        self._budget.check()

        response = self._messages.create(**kwargs)

        # After: record what it actually cost, not what we estimated.
        usage = response.usage
        self._budget.charge(usage.input_tokens, usage.output_tokens)
        return response


# Usage — prices passed in, not hardcoded.
budget = Budget(
    ceiling_usd=5.00,
    input_per_mtok=3.00,
    output_per_mtok=15.00,
)
client.messages = BudgetedMessages(client.messages, budget)

Against the sixty-three-hour run described earlier, a five-dollar ceiling would have halted the process inside the first ten minutes. The postmortem's own conclusion was the same: a $50 threshold would have caught it within the hour.

Catching the loop before the ceiling does

A spend ceiling is a backstop, and backstops should rarely fire. The cheaper guard catches the specific pathology: a no-progress loop has a precise, machine-detectable signature, which is the same tool called with the same arguments returning the same result on consecutive steps. That needs no model call to detect and no heuristic tuning.

import hashlib
import json
from collections import deque


class NoProgressDetector:
    """Halts when the last N tool calls are byte-identical."""

    def __init__(self, window: int = 3):
        self._recent: deque[str] = deque(maxlen=window)

    def observe(self, tool_name: str, arguments: dict, result) -> bool:
        fingerprint = hashlib.sha256(
            json.dumps(
                [tool_name, arguments, result],
                sort_keys=True,
                default=str,
            ).encode()
        ).hexdigest()

        self._recent.append(fingerprint)

        # Only meaningful once the window is full.
        return (
            len(self._recent) == self._recent.maxlen
            and len(set(self._recent)) == 1
        )

Content-addressing the call rather than comparing objects is deliberate: it survives dict ordering, handles unhashable arguments, and gives a stable value to log. A window of three is a reasonable default — two consecutive identical calls can be a legitimate retry, three almost never is.

Common mistakes

1. Treating the step limit as the budget. It is a liveness guard. Keep it — it stops genuinely infinite graphs — but do not let it be the only ceiling.
2. Estimating cost from a token counter instead of reading it off the response. Counters approximate; the usage object is what you are billed on, including cached and reasoning tokens the counter does not model.
3. Charging before the call using an estimate. Output length is not known until the response arrives, and output is the more expensive half.
4. Setting the ceiling per call rather than per run. The failure mode is thousands of individually cheap calls, every one of which passes a per-call check.
5. Enforcing only in the parent graph. A sub-agent with its own client bypasses a parent-level guard entirely. The budget object has to be shared down the tree.
6. Relying on the provider dashboard to notice. It aggregates at the organisation level and reports on a delay measured in hours, which is longer than a runaway loop needs to do real damage.

Security and performance considerations

A spend ceiling is a denial-of-wallet control, not only a cost control. An agent that accepts untrusted input and calls tools can be steered into expensive behaviour deliberately — a prompt that induces long outputs or repeated tool calls turns your inference budget into an attacker's lever. A per-run ceiling bounds the blast radius of that attack in the same motion as it bounds an accident.

On performance, the accounting itself is negligible — arithmetic and a hash against a network call measured in hundreds of milliseconds. Two implementation details matter more. In async or concurrent runs the budget is shared mutable state and needs a lock, or you get lost updates and a ceiling that leaks. And the fingerprint should hash the result, not store it, so a long tool output does not sit in memory for the life of the run.

Troubleshooting a bill that is already too high

1. Check the ratio of input to output tokens first. Healthy agent runs are input-heavy but bounded; a ratio that climbs across a single run is context re-ingestion, which points at replanning rather than at any individual tool.
2. Count model calls per user request. If it is far above what the graph implies, you have fan-out through sub-agents that the parent step limit never saw.
3. Look for identical consecutive tool calls in your traces. This is the cheapest signal available and it identifies the loop precisely, without inference.
4. Compare cost per run rather than cost per call. Per-call cost usually looks entirely reasonable in exactly the runs that went wrong, which is why per-call alerting misses them.
5. Confirm which model actually served the request. A silent fallback to a larger model is a common and easily missed multiplier when a primary is rate-limited.

Cost control is a design property rather than an afterthought, and it is worth deciding before an agent reaches production rather than after. If you are scoping a build, the questions to ask a team about failure handling and budget enforcement are covered in our notes on what AI agents cost and how to scope one, and the wider budget picture — build versus run, and where estimates usually go wrong — is in the AI app development cost guide.

If you are building something agentic and want the cost controls designed in from the start, that is what our AI agent development practice does. If you have a system already running and the bill is the problem, an AI consulting engagement starts with instrumenting what you have before recommending any rebuild.