LangGraph Replays Your Nodes — Which Means Your Side Effects Happen Twice

Durable execution is introduced as a safety feature: the process dies, the run resumes, nobody loses work. That is accurate. What follows from it gets much less attention, and it is the part that reaches production as an incident.

The LangGraph documentation states the mechanism plainly — nodes after the checkpoint re-execute, including any LLM calls, API requests, or interrupts. Resuming is replaying. So if one of those nodes issued a refund, sent an email, or inserted a row, it does it again.

The three durability modes, and what each actually promises

The mode controls when state is written, and the choice is a real trade rather than a tuning preference. Checkpoints are taken per super-step — one tick in which all scheduled nodes run, possibly in parallel — and keyed by thread_id.

The instinct is to pick exit for performance on long-running graphs, and for a purely computational pipeline that is defensible — there is nothing to lose but time, as the durable execution overview frames it. The moment a node performs an external effect, exit becomes the worst option available: a crash replays every effect from the beginning of the run, because no intermediate state was ever recorded.

async is the pragmatic middle and it carries a small honest caveat: a crash during the write window loses that checkpoint, so the step is replayed. sync is the only mode where a completed step is guaranteed to be recorded before the next begins, and it is the one to reach for when a replayed step is expensive or irreversible.

Classify nodes before choosing a mode

The useful discipline is to sort every node into one of three categories. It takes an afternoon on an existing graph and it determines everything else.

1. Pure — computes from state and returns state. No external calls, no writes, no clock, no randomness. Replays are free and produce identical results.
2. Idempotent — touches the outside world, but repeating it changes nothing further. Reading a document, upserting by primary key, setting a value rather than incrementing it. Safe to replay by construction.
3. Effectful — repeating it causes a second real thing to happen. Charging a card, sending a message, appending to a ledger, creating a resource with a generated id. These are the nodes that need a guard.

Push effects to the end of a super-step

Before reaching for idempotency machinery, the cheaper structural fix is to isolate effects into their own nodes rather than burying them inside nodes that also do computation. A node that calls a model, transforms the result, and then sends an email replays all three on resume. Split into two nodes, the checkpoint lands between them, and the send is not replayed when the failure occurred after it.

This costs nothing but a graph edge and it removes a large share of the exposure. What it does not remove is the case where the failure happens during the effectful node itself, which is where a key is required.

Idempotency keys that survive replay

The standard remedy is an idempotency key: a value the downstream service uses to recognise a repeat and return the original result rather than performing the action again. Most payment and messaging APIs support one.

The subtlety, and the reason this fails in practice, is where the key comes from. Generated inside the node, it is regenerated on replay — a fresh value, a fresh charge, and an idempotency mechanism that provided no idempotency. The key has to be derived from something that is identical across replays of the same logical step.

import hashlib


def step_key(config: dict, node: str, *parts: str) -> str:
    """Deterministic key for one logical execution of one node.

    Derived from thread identity and the node name, so a replay of the
    same step produces the same key. Anything generated inside the node —
    uuid4(), time.time() — is recreated on replay and defeats the purpose.
    """
    thread_id = config["configurable"]["thread_id"]
    material = "|".join([thread_id, node, *parts])
    return hashlib.sha256(material.encode()).hexdigest()[:32]


def charge_customer(state: dict, config: dict) -> dict:
    # Same thread + same node + same invoice => same key on every replay.
    key = step_key(config, "charge_customer", state["invoice_id"])

    receipt = payments.charge(
        amount=state["amount_cents"],
        customer=state["customer_id"],
        idempotency_key=key,          # replay returns the original charge
    )
    return {"receipt_id": receipt.id}

Including a business identifier such as the invoice id matters as much as the thread id. A retry loop that legitimately charges the same customer twice for two different invoices must produce two keys, and a key built only from thread and node name would collapse them into one — silently dropping the second charge, which is the opposite failure and harder to notice.

Non-determinism quietly breaks resumption

A node that reads the clock, generates a random value, or reads mutable external state produces different output on replay. Nothing errors. The graph simply continues from a state that does not match the one the earlier steps were computed against, and the resulting bug is very hard to trace because it only appears after a resume.

The fix is to treat non-deterministic values as inputs rather than as things a node fetches. Capture the timestamp or the generated identifier once, write it into state, and have downstream nodes read it from there. State is checkpointed; a call to the clock is not.

# Fragile: every replay produces a different value, and downstream nodes
# were computed against the first one.
def prepare(state):
    return {"batch_id": str(uuid.uuid4()), "started_at": time.time()}


# Stable: generated once, then read from checkpointed state on replay.
def prepare(state, config):
    if state.get("batch_id"):
        return {}                      # already established, do not regenerate
    return {
        "batch_id": step_key(config, "prepare"),
        "started_at": time.time(),
    }

The guard clause is the important half. It makes the node self-idempotent: the first execution establishes the values, every replay observes they already exist and returns without changing them.

Common mistakes

1. Assuming a checkpointer gives exactly-once execution. It gives durability. Effects in replayed nodes happen again, and the documentation says so directly.
2. Choosing exit mode for a graph containing effectful nodes. It is the fastest mode and the one that replays the most work after a crash.
3. Generating idempotency keys inside the node with uuid4. They are regenerated on replay, so the deduplication never engages.
4. Building keys from thread and node alone. Two legitimately distinct operations on one thread collapse into one, and the second is silently dropped.
5. Mixing computation and effects in a single node. Splitting them lets the checkpoint land between, which removes most replay exposure for the cost of one edge.
6. Calling time or uuid inside nodes and reading them downstream. The values change on replay and the graph proceeds from an inconsistent state.
7. Testing resume only from a clean interrupt. Interrupts are the tidy case; test resumption after a hard process kill mid-node, which is the case that produces duplicates.

Security and performance considerations

Checkpoints are a data store containing whatever your state carries, which for agent graphs frequently means retrieved documents, user messages, and tool results. That is a copy of potentially sensitive material with a lifetime nobody chose deliberately, sitting outside whatever access controls the source data had. Decide what state is allowed to hold and how long threads are retained, rather than discovering the answer during a data request.

The thread_id is also an access-control boundary and is easy to get wrong. It is the primary key for resumption, so anything able to supply an arbitrary thread_id can resume somebody else's run and read its state. Derive it from authenticated identity rather than accepting it from a client.

On performance, sync mode adds a write to the critical path of every super-step, and for a graph with many small nodes that overhead is real. The instinct is to reach for exit to recover it; the better trade is usually to reduce the number of super-steps by merging trivial nodes, which lowers checkpoint frequency without giving up durability. Replay cost compounds with model spend, which is worth reading alongside why agent bills explode in production — a graph that retries aggressively from early checkpoints can spend a great deal without any single call looking unusual.

Troubleshooting

1. Duplicate emails or charges after a deploy — a rolling restart interrupted runs mid-node and the effectful node replayed. Check whether it carries a derived idempotency key.
2. Resume produces different results from the original run — non-determinism inside a node. Look for clock reads, random values, or reads of mutable external state.
3. Nothing resumes at all — usually a missing or unstable thread_id. Without it the checkpointer cannot save or restore, and a per-request generated id is effectively absent.
4. Costs spike after enabling retries — replayed nodes re-issue their LLM calls. Count model calls per run rather than per node.
5. Works on interrupt-and-resume, fails on crash-and-resume — almost certainly exit or async mode. The interrupt path writes state; the crash path did not.
6. Second run of an identical request does nothing — an idempotency key too coarse to distinguish two legitimate operations. Add the business identifier to the key material.

The framing that makes this tractable is that durable execution changes the contract your node code is written against. A node is no longer a function that runs once; it is a function that may run any number of times and must converge on the same outcome. That is a familiar discipline from distributed systems, arriving in a place where a lot of code was written as if it were a script.

If you are building agent workflows that touch payments, messaging, or anything else where a duplicate is expensive, that boundary deserves designing up front. Our agent engineering work covers it, and where the surrounding transactional system is the harder half, our custom software team handles that side.