--- title: "Idempotency: When the Same Message Arrives Twice" description: "At-least-once delivery means a handler will see the same message twice. The idempotency key, where it comes from, and the dedupe that survives a crash" url: https://sade.dev/en/notes/idempotency-duplicate-delivery/ lang: en author: "Muhammet Şafak" published: 2026-06-17 updated: 2026-09-06 section: Note tags: ["queue","reliability","architecture","async"] summary: "At-least-once is a guarantee to design around, not a bug to remove. The key has to be minted by the sender and never derived from the payload alone, and the gap between doing the work and recording the key decides the shape: record it in the same transaction when you own the effect, keep a seen-set when you do not. Concurrent duplicates need a unique constraint, not a check." --- # Idempotency: When the Same Message Arrives Twice > At-least-once is a guarantee to design around, not a bug to remove. The key has to be minted by the sender and never derived from the payload alone, and the gap between doing the work and recording the key decides the shape: record it in the same transaction when you own the effect, keep a seen-set when you do not. Concurrent duplicates need a unique constraint, not a check. A customer got charged twice for one order. Nothing ran twice on purpose: the payment worker charged the card, then crashed before it could ack the message — so the broker, doing exactly what it promised, delivered the message again. The duplicate wasn't the bug. The handler that wasn't built for a duplicate was. ## At-least-once is a promise, not a glitch Almost every queue — and every retried HTTP call — is **at-least-once**. A handler MAY see the same message more than once: after a crash, a release, a redelivery, or a client that timed out and retried. Exactly-once across a network is impractical without coordination the broker can't give you cheaply, so few brokers offer it — and those that do, like Google Cloud Pub/Sub, fence it in with narrow conditions: a single region, pull subscriptions only. That reframes the problem. The redelivery isn't a failure to eliminate; it's a guarantee to design around. (It's the same trade the [queue](/en/notes/sync-vs-async) makes when it retries a failed job instead of losing it.) ## Idempotent means "twice equals once" An operation is idempotent if applying it many times has the same effect as applying it once — `f(f(x)) = f(x)`. - "Set order status to `paid`" is idempotent. - "Increment balance by 10" is **not** — twice is +20. - "Send the welcome email" is **not** — twice is two emails. The whole job is turning the second kind of operation into the first. ## The idempotency key comes from the sender You need a stable identity for *this operation*, and it must be decided by the **sender**, not the receiver: - For a queue message, it's the **per-message id the producer mints** — one id per message, distinct from the trace/correlation id (which spans many messages). - For an HTTP write, it's a **client-generated `Idempotency-Key` header** — the Stripe model. Two rules keep the key honest: - **Don't derive it from the payload alone.** Two legitimately identical requests — the same customer buying the same item twice — would collide and the second would be silently dropped. - **Don't let the receiver mint it.** The receiver can't tell a retry from a brand-new call; only the sender knows "this is the same operation I tried before." ## Dedupe: remember what you've already done The handler does one check before the work: *has this key been processed?* If yes, skip and ack. If no, do the work, then record the key. That's the entire pattern — the weight is in *where* the record lives. ## The crash between the work and the record The hard part is the gap between doing the work and recording the key. Two shapes, and the right one depends on the side effect: - **Seen-set — record after success.** Store the key in Redis or a table once the handler returns. Cheap and broadly applicable. The window: crash *after* the side effect but *before* recording, and a redelivery reprocesses. That's fine when the side effect is itself idempotent — an `UPSERT`, a "set to paid". - **Transactional — record with the work.** Write the idempotency record in the **same database transaction** as the business change. No window: the key and the effect commit or roll back together. The cost: the effect must be a DB write you control, and the key store must be that same database — not a separate Redis. For an effect you own, this is what kills the duplicate for real. So the side effect chooses for you: a row you own → transactional; a third-party call you can't enlist in your transaction (email, a charge) → seen-set, and make the downstream idempotent too by forwarding your key as *its* `Idempotency-Key`. ## Concurrent duplicates need a constraint, not a check Two deliveries of the same key racing before either records it slip through a "check then write" — that sequence isn't atomic. A **unique constraint on the key column** is: the second insert becomes a conflict you catch and treat as "already handled." This is the same shape as the [gaps a race opens in sequential numbering](/en/systems/race-conditions-and-gaps-in-sequential-numbering) — the database, not the application, is where uniqueness is actually enforced. ## When do you not need any of this? If the handler is already idempotent — a pure `UPSERT` keyed by a stable id, a "set field to value" — you may need nothing at all. Reach for a key plus dedupe when the operation has a **non-idempotent side effect** (money, email, an external `POST`) or **accumulates** (increment, append). Spend the complexity only where a second run does real damage. --- At-least-once isn't the part you fix; it's the part you accept. Build the handler so the second delivery is a no-op, and redelivery stops being an incident and goes back to being what it is — the broker keeping its promise. --- *See also:* the [BabelQueue idempotency spec](https://babelqueue.com/docs/spec/1.x/idempotency) pins down where the key comes from and how dedupe behaves, and the [`idempotency-payments` example](https://github.com/BabelQueue/babelqueue-examples/tree/main/idempotency-payments) is exactly this double-charge case made runnable. ## Frequently asked **Who should generate the idempotency key — the sender or the receiver?** It must be decided by the sender, not the receiver. For a queue message, it's the per-message id the producer mints; for an HTTP write, it's a client-generated Idempotency-Key header — the Stripe model. The receiver can't tell a retry from a brand-new call; only the sender knows "this is the same operation I tried before." **How does a handler know whether it has already processed a message?** The handler does one check before the work: has this key been processed? If yes, skip and ack. If no, do the work, then record the key. That's the entire pattern — the weight is in where the record lives. ## Sources - [RFC 9110, 9.2.2: Idempotent Methods](https://www.rfc-editor.org/rfc/rfc9110.html#name-idempotent-methods) — IETF - [Stripe API: idempotent requests](https://docs.stripe.com/api/idempotent_requests) — Stripe - [Exactly-once delivery: pull subscriptions, single region](https://docs.cloud.google.com/pubsub/docs/exactly-once-delivery) — Google Cloud - [Consumer acknowledgements and publisher confirms](https://www.rabbitmq.com/docs/confirms) — RabbitMQ