Transactional SMS API Idempotency: Preventing Duplicate Messages

Why duplicate messages happen

Duplicate SMS can originate from browser retries, API timeouts, queue redelivery, duplicated webhooks, application loops or worker crashes. The important point is that duplicate prevention must exist at more than one boundary.

A production messaging system should assume that requests and events can be delivered more than once. Idempotency converts repeated processing from a customer-facing incident into a harmless replay.

What idempotency means

An idempotent operation produces the same intended business outcome when the same request is processed repeatedly. In an SMS platform, the objective is usually to ensure that one business notification does not create multiple provider submissions.

Idempotency does not necessarily mean every internal operation happens only once. It means repeated attempts do not create unintended customer-visible effects.

Idempotency keys

The calling application can provide a unique key representing the business event. A payment confirmation might use the payment-event ID, while an appointment reminder might use the appointment-event ID plus notification type.

The key should be generated from a durable business identity rather than a random value generated separately for every retry.

Database uniqueness

A database unique constraint is one of the strongest duplicate-prevention mechanisms because it remains effective across multiple workers and application instances.

The uniqueness scope must be carefully defined. A key may be unique per tenant, event type or provider submission depending on the workflow.

Idempotent API response

If the same request arrives again with the same idempotency key, the API should return the existing message result rather than creating another message.

The response should allow the caller to understand that the request was already accepted without requiring a second customer notification.

Timeouts and uncertain outcomes

A client timeout does not prove that the messaging API rejected the request. The server may have accepted it just before the network failed.

A client should retry using the same idempotency key. The server can then return the original message record instead of creating a duplicate.

Queue-level idempotency

A queue can deliver the same job twice. Workers should therefore perform a duplicate check before provider submission.

Do not assume that a queue's acknowledgement semantics provide exactly-once SMS submission.

Provider-level uncertainty

The most difficult case is a provider timeout after submission. The provider may have accepted the message even though the worker did not receive the response.

Use provider-supported idempotency where available, or reconcile using provider references before deciding whether another submission is safe.

Idempotency retention

Keys cannot necessarily be retained forever. Define a retention period based on how long duplicate events could realistically arrive and how long the business event remains replayable.

For financial or compliance-sensitive workflows, longer retention may be appropriate.

Concurrent requests

Two application instances may submit the same event at almost exactly the same time. The duplicate check must therefore be atomic.

A read-then-insert pattern without a unique constraint can still create two records under concurrency.

Testing

Test repeated requests, concurrent requests, client timeout, queue redelivery, duplicate webhooks and provider timeout. Confirm that every scenario results in the intended number of customer messages.

Implementation checklist

Define key generation, uniqueness scope, database constraints, retry behaviour, queue checks, provider reconciliation, retention and observability before production.

Idempotency across services

In a microservice environment, the same business event may travel through an order service, event broker, notification service and provider adapter. Each boundary can potentially retry. The safest design is to carry a stable event identity through the entire path.

For example, ORDER_CONFIRMED:8472 can remain the business event ID while the messaging service derives a notification key such as tenant + event ID + notification type. A worker can then use a separate provider-submission key where the provider supports it.

This layered identity model makes it possible to prevent duplicates without incorrectly treating two genuinely different notifications as the same event.

Idempotency versus deduplication

Deduplication removes repeated events. Idempotency makes repeated processing safe. They are related but not identical.

A system may deduplicate duplicate webhooks while still allowing a legitimate new event to create a second notification. Developers should therefore define what constitutes a duplicate at each stage instead of applying one global “duplicate” rule.

Race conditions

Consider two workers receiving the same notification at the same time. Both read that no message exists and both attempt to create one. If duplicate prevention depends only on the initial read, both may succeed.

A database uniqueness constraint or atomic insert is required to close this race. Application-level checks should provide a friendly response, but the database should enforce the invariant.

Idempotency key conflicts

A dangerous situation occurs when the same idempotency key is reused with different message content. The API should define whether this is rejected as a conflict or treated according to a deterministic policy.

Rejecting conflicting reuse is usually safer because silently changing the meaning of a previously accepted key can make troubleshooting extremely difficult.

API client guidance

Client developers should generate one idempotency key for one business operation and reuse it for retries. They should not generate a new key after every network timeout.

The API documentation should provide this rule prominently because incorrect client retry behaviour can defeat an otherwise sound server design.

Operational reconciliation

A reconciliation process can search for business events that should have produced a notification but do not have a corresponding message record. It can also identify messages whose provider outcome remains uncertain.

Reconciliation should create controlled recovery work rather than blindly resend everything it finds.

Security considerations

Idempotency keys may contain business identifiers, so avoid exposing sensitive information directly in them. Prefer opaque identifiers or hashed references where appropriate.

Access logs should record the key or a safe fingerprint rather than sensitive message content.

Production scenario

A customer clicks Pay twice because the first page appears frozen. The payment service generates one authoritative payment event. If the notification API receives two requests with the same event-derived key, the first creates the message and the second returns the existing message ID. Only one payment-success SMS is submitted.

The customer experience remains correct even though multiple application requests occurred.

Final implementation checklist

Verify stable event identities, atomic uniqueness, safe client retries, conflicting-key handling, queue redelivery, provider uncertainty, reconciliation, retention and monitoring. Test concurrent requests—not just sequential duplicate requests.

Idempotency storage strategy

For high-volume systems, the idempotency table can become large. Partition or expire records according to the maximum replay window while preserving longer-lived business records separately.

Do not delete keys while a source system can still legitimately retry them. The retention policy should be documented as part of the API contract.

Idempotency observability

Measure duplicate-request rate, conflict rate and idempotency-hit rate. A sudden increase can indicate a client timeout problem, an unstable network or a buggy integration.

These metrics are valuable because they reveal integration quality before customers complain about duplicate messages.

Reference implementation

The API receives an event ID, calculates a scoped idempotency key and attempts an atomic insert. If the key already exists, it returns the existing message ID. If the insert succeeds, the message is queued. The worker uses the same stable identity before provider submission and records the provider reference.

This design remains safe when the API is scaled across multiple servers.

Final quality review

Review the page for concurrent-request handling, timeout uncertainty, key conflicts, retention, provider ambiguity and developer-ready implementation guidance.

Client retry contract

Document a recommended retry sequence for clients: reuse the same idempotency key after network uncertainty, respect server retry guidance, and stop retrying when the API returns a permanent validation or authorization error.

This prevents individual integrations from inventing incompatible retry behaviour.

Idempotency with batch APIs

If the gateway supports batch submission, decide whether idempotency applies to the entire batch or each individual message. Per-message keys are generally easier to reconcile because one invalid item does not make the entire batch ambiguous.

Provider submission record

Store a provider submission attempt separately when detailed auditing is required. This allows the platform to distinguish one logical message from multiple technical attempts without treating every attempt as a new customer notification.

Recovery after database failover

After a database failover, workers may restart with uncertain local state. Durable message records and unique constraints allow the worker to determine whether a notification already exists instead of relying on in-memory state.

Developer takeaway

Idempotency should be designed from the first API request through provider submission. The strongest implementation combines stable business identities, atomic database constraints, safe retries and explicit handling of uncertain provider outcomes.