Idempotency

Explanation

In distributed systems, network calls can fail two ways: before the server processes the request (safe to retry) or after processing but before the response arrives (dangerous to retry if it has side effects). Idempotency solves the second case: if the operation is idempotent, you can always retry without risking duplication. HTTP method semantics: GET, PUT, DELETE, and HEAD are defined as idempotent. PUT /users/42 applied ten times with the same data should produce the same state as once — it sets, doesn't increment. DELETE /users/42 applied twice: first deletes, second is a no-op or returns 404 — the end state is the same. POST is not inherently idempotent: POST /orders creates a new order each time. Idempotency keys: for non-idempotent operations (POST), clients send a unique idempotency key (UUID) with each request. The server stores the key; if it sees the same key again, it returns the cached response instead of re-executing. Stripe uses this for payments: if a charge request times out, retry with the same key — the charge is only processed once. Design patterns: use UPSERT (insert-or-update) instead of plain INSERT, use SET instead of INCREMENT where possible, verify the operation hasn't already been performed, and make event consumers idempotent (processing the same event twice is safe).

Code Example

// Idempotency key pattern for safe payment retries

// Client: generate key once, resend on failure
const idempotencyKey = crypto.randomUUID();

async function createCharge(amount, retries = 0) {
  try {
    const res = await fetch('/api/charges', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'Idempotency-Key': idempotencyKey, // same key on every retry
      },
      body: JSON.stringify({ amount }),
    });
    return res.json();
  } catch (err) {
    if (retries < 3) return createCharge(amount, retries + 1);
    throw err;
  }
}

// Server: check key before processing
app.post('/api/charges', async (req, res) => {
  const key = req.headers['idempotency-key'];

  const existing = await redis.get(`idem:${key}`);
  if (existing) return res.json(JSON.parse(existing)); // already processed

  const charge = await stripe.charges.create({ amount: req.body.amount });

  await redis.setEx(`idem:${key}`, 86400, JSON.stringify(charge)); // 24h TTL
  res.status(201).json(charge);
});

Why It Matters for Engineers

Idempotency is fundamental to building reliable distributed systems. Without it, retries cause duplicate charges, duplicate emails, and duplicate records — all serious bugs. The idempotency key pattern is industry standard for payment APIs (Stripe, Twilio) and is the correct approach to safe retries. Understanding idempotency also clarifies HTTP method semantics: using POST for updates (instead of PUT/PATCH) means clients can't safely retry on timeout. Correct HTTP semantics make retries safe and APIs more predictable.