If you've ever clicked "Submit Order" twice and ended up with two charges on your credit card, you've experienced what happens when an API isn't idempotent. Idempotency is one of those concepts that sounds academic but has very real consequences for your users and your data integrity.
This guide breaks down what idempotency actually means, which HTTP methods give it to you for free, and how to implement idempotency keys when you need to add it yourself.
An operation is idempotent if performing it multiple times produces the same result as performing it once. The word comes from mathematics — an idempotent function f satisfies f(f(x)) = f(x).
In API terms: if a client sends the same request twice (due to a network timeout, a retry, or a bug), the server should end up in the same state as if the request had been sent once.
This matters because networks are unreliable. A client sends a request, the server processes it, but the response gets lost in transit. The client doesn't know if the request succeeded, so it retries. Without idempotency, you get duplicate orders, double charges, or corrupted data.
The HTTP spec defines idempotency for several methods:
GET — Always idempotent. Reading a resource doesn't change it, so you can call it as many times as you want.
PUT — Idempotent by design. Replacing a resource with the same data leaves the system in the same state.
DELETE — Idempotent. Deleting something that's already deleted should return 404 or 204, not an error that breaks your retry logic.
HEAD — Idempotent, same as GET but without the response body.
OPTIONS — Idempotent.
POST — Not idempotent by default. Creating a resource twice creates two resources. This is where idempotency keys come in.
PATCH — Tricky. It depends on the operation. PATCH /pets/1 {"status": "available"} is idempotent. PATCH /pets/1 {"views": {"$increment": 1}} is not.
Here's a quick reference using the PetStore API:
# Idempotent - safe to retry
GET /pets/1
PUT /pets/1
DELETE /pets/1
# Not idempotent - retrying creates duplicates
POST /pets
POST /orders
Consider this scenario with the PetStore API:
// Client creates an order
const response = await fetch('https://petstore.example.com/orders', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
petId: 42,
quantity: 1,
shipDate: '2024-03-15'
})
});
// Network timeout - client never gets the response
// Client retries...
// Now there are TWO orders for the same pet
The server processed the first request successfully, but the client never got the 200 response. When it retries, the server has no way to know this is a duplicate — it creates another order.
The standard solution is an idempotency key: a unique identifier the client generates and sends with the request. The server stores the result of the first request and returns the cached result for any subsequent requests with the same key.
Generate a UUID before making the request and include it in a header:
const { v4: uuidv4 } = require('uuid');
async function createOrderIdempotent(orderData) {
const idempotencyKey = uuidv4(); // Generate once, before the request
const makeRequest = async () => {
return fetch('https://petstore.example.com/orders', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Idempotency-Key': idempotencyKey // Same key on every retry
},
body: JSON.stringify(orderData)
});
};
// Retry up to 3 times with the same idempotency key
for (let attempt = 0; attempt < 3; attempt++) {
try {
const response = await makeRequest();
if (response.ok) return response.json();
if (response.status < 500) throw new Error(`Client error: ${response.status}`);
} catch (err) {
if (attempt === 2) throw err;
await new Promise(r => setTimeout(r, 1000 * Math.pow(2, attempt))); // Exponential backoff
}
}
}
The key insight: generate the UUID once before the retry loop, not inside it. If you generate a new key on each attempt, you lose the idempotency guarantee.
The server needs to: 1. Check if it's seen this idempotency key before 2. If yes, return the stored response 3. If no, process the request and store the result
Here's a Node.js/Express implementation:
const express = require('express');
const { v4: uuidv4 } = require('uuid');
// In production, use Redis or a database for this
const idempotencyStore = new Map();
function idempotencyMiddleware(req, res, next) {
const key = req.headers['idempotency-key'];
// If no key provided, proceed normally (not all endpoints require it)
if (!key) return next();
// Validate key format
const uuidRegex = /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
if (!uuidRegex.test(key)) {
return res.status(400).json({ error: 'Invalid idempotency key format' });
}
const stored = idempotencyStore.get(key);
if (stored) {
if (stored.status === 'processing') {
// Request is still being processed
return res.status(409).json({ error: 'Request is still being processed' });
}
// Return the cached response
console.log(`Returning cached response for idempotency key: ${key}`);
return res.status(stored.statusCode).json(stored.body);
}
// Mark as processing to handle concurrent requests with the same key
idempotencyStore.set(key, { status: 'processing' });
// Intercept the response to cache it
const originalJson = res.json.bind(res);
res.json = (body) => {
idempotencyStore.set(key, {
status: 'complete',
statusCode: res.statusCode,
body,
createdAt: Date.now()
});
return originalJson(body);
};
next();
}
const app = express();
app.use(express.json());
app.post('/orders', idempotencyMiddleware, async (req, res) => {
const { petId, quantity, shipDate } = req.body;
// Create the order
const order = await createOrder({ petId, quantity, shipDate });
res.status(201).json(order);
});
The in-memory Map won't work across multiple server instances. Use Redis with an expiry:
const redis = require('redis');
const client = redis.createClient();
async function checkIdempotencyKey(key) {
const stored = await client.get(`idempotency:${key}`);
return stored ? JSON.parse(stored) : null;
}
async function storeIdempotencyResult(key, statusCode, body) {
const data = JSON.stringify({ statusCode, body });
// Store for 24 hours
await client.setEx(`idempotency:${key}`, 86400, data);
}
async function markAsProcessing(key) {
// SET NX ensures only one request processes at a time
const result = await client.set(
`idempotency:${key}`,
JSON.stringify({ status: 'processing' }),
{ NX: true, EX: 30 } // 30 second lock
);
return result === 'OK'; // Returns false if key already exists
}
Sometimes you want idempotency at the database level, independent of any application logic. This is useful for operations that might be triggered from multiple sources.
The simplest approach: add a unique constraint on the idempotency key column.
CREATE TABLE orders (
id SERIAL PRIMARY KEY,
idempotency_key UUID UNIQUE NOT NULL,
pet_id INTEGER NOT NULL,
quantity INTEGER NOT NULL,
ship_date DATE,
status VARCHAR(20) DEFAULT 'pending',
created_at TIMESTAMP DEFAULT NOW()
);
-- This will fail with a unique constraint violation on duplicate
INSERT INTO orders (idempotency_key, pet_id, quantity, ship_date)
VALUES ('550e8400-e29b-41d4-a716-446655440000', 42, 1, '2024-03-15')
ON CONFLICT (idempotency_key) DO NOTHING
RETURNING *;
The ON CONFLICT DO NOTHING clause means duplicate inserts silently succeed without creating a new row. You can then fetch the existing row to return to the client.
For more complex cases, use an upsert:
INSERT INTO orders (idempotency_key, pet_id, quantity, ship_date, status)
VALUES ($1, $2, $3, $4, 'pending')
ON CONFLICT (idempotency_key)
DO UPDATE SET updated_at = NOW() -- Touch the row to reset any TTL
RETURNING *;
For operations that span multiple tables:
async function createOrderIdempotent(idempotencyKey, orderData) {
return db.transaction(async (trx) => {
// Try to acquire an advisory lock based on the key hash
const lockId = hashToInt(idempotencyKey);
await trx.raw('SELECT pg_advisory_xact_lock(?)', [lockId]);
// Check if we've already processed this key
const existing = await trx('idempotency_records')
.where({ key: idempotencyKey })
.first();
if (existing) {
return JSON.parse(existing.response_body);
}
// Process the request
const order = await createOrderInDB(trx, orderData);
// Store the result
await trx('idempotency_records').insert({
key: idempotencyKey,
response_body: JSON.stringify(order),
created_at: new Date()
});
return order;
});
}
If a client sends the same idempotency key with different request bodies, you should return an error:
if (stored && stored.requestHash !== hashRequest(req.body)) {
return res.status(422).json({
error: 'Idempotency key reused with different request body'
});
}
Idempotency keys shouldn't live forever. 24 hours is a common window — long enough to handle retries, short enough to not accumulate indefinitely.
If a request fails with a 4xx error (bad input, not found, etc.), should you cache that too? Generally yes — if the client retries with the same bad input, they should get the same error back, not accidentally succeed if the data changes.
For 5xx errors, it's more nuanced. A server error might be transient, so you might want to allow retries without caching the failure.
Here's a complete example showing idempotent order creation with the PetStore API:
class PetStoreClient {
constructor(baseUrl) {
this.baseUrl = baseUrl;
}
async createOrder(petId, quantity) {
const idempotencyKey = uuidv4();
const response = await this.retryWithIdempotency(
'/orders',
{
method: 'POST',
body: { petId, quantity, shipDate: new Date().toISOString() }
},
idempotencyKey
);
return response;
}
async retryWithIdempotency(path, options, idempotencyKey, maxRetries = 3) {
for (let i = 0; i < maxRetries; i++) {
try {
const res = await fetch(`${this.baseUrl}${path}`, {
method: options.method,
headers: {
'Content-Type': 'application/json',
'Idempotency-Key': idempotencyKey
},
body: JSON.stringify(options.body)
});
// Don't retry client errors
if (res.status >= 400 && res.status < 500) {
throw new Error(`Client error: ${res.status}`);
}
if (res.ok) return res.json();
} catch (err) {
if (i === maxRetries - 1) throw err;
await sleep(Math.pow(2, i) * 1000);
}
}
}
}
Idempotency is about making your API safe to retry. The key points:
Getting this right means your users can safely retry failed requests without worrying about duplicate orders, double charges, or corrupted state. It's one of those things that's invisible when it works and very visible when it doesn't.
