Transactional SMS API Webhook Architecture: Secure Callbacks, Retries and Event Processing

Why webhooks need their own architecture

Delivery callbacks are inbound events from an external system. They can arrive late, multiple times, out of order or not at all.

Treating webhook processing as a simple HTTP endpoint that directly updates a message table creates reliability and security risks.

Webhook authentication

Validate provider signatures, tokens or other supported authentication mechanisms before accepting an event. Store the raw evidence required for troubleshooting without retaining unnecessary sensitive content.

Fast acknowledgement

The callback endpoint should normally acknowledge valid events quickly and perform heavier processing asynchronously. A provider may retry if the endpoint takes too long.

Event queue

Place validated webhook events into a durable queue before downstream processing when the provider volume or reliability requirements justify it. This isolates provider traffic from database or business-service latency.

Event identity

Use a provider event ID when available. If the provider can send the same event repeatedly, the event ID can be used for deduplication.

Out-of-order events

A delivered event can arrive before an earlier submitted event. The status processor should apply transition rules rather than assuming callback order equals business order.

Status normalization

Providers use different status names. Normalize them into a stable internal model while retaining the original provider status for diagnostics.

Webhook retries

Provider retries should not create duplicate status transitions or duplicate billing events. Processing should be idempotent.

Unknown messages

A callback may reference a message that has been deleted, migrated or was never known to the platform. Define whether the event is discarded, quarantined or sent for investigation.

Monitoring callback health

Monitor callback rate, signature failures, processing latency, queue backlog and unknown-event rate. A sudden drop in callbacks can indicate a provider-side issue or an endpoint configuration problem.

Testing

Test duplicate callbacks, forged signatures, delayed events, out-of-order statuses, unknown message IDs and callback bursts.

Implementation checklist

Secure the endpoint, validate and queue events, deduplicate by event identity, apply status-transition rules, normalize provider states and monitor callback health.

Webhook event envelope

A useful event envelope can contain event_id, event_type, occurred_at, provider, provider_message_id, internal_message_id and payload. The envelope provides consistent processing while the payload contains provider-specific detail.

Event IDs should be stable across provider retries.

Signature validation

Validate the provider signature over the exact payload representation required by the provider. Do not parse and reserialize JSON before verifying a signature if the provider signs the raw body.

Use constant-time comparison where appropriate and reject stale or malformed authentication material.

Queue before business processing

Once the webhook is authenticated, place the event into a durable processing path and acknowledge it according to the provider's requirements. This prevents a slow database or downstream service from causing repeated provider retries.

Status transition guard

A status processor should reject or quarantine impossible transitions. For example, an older SUBMITTED event should not overwrite a previously recorded DELIVERED state.

Keep the raw provider status so support can understand why an event was ignored.

Callback duplication

If event ID 123 arrives three times, the first event should create the delivery record and subsequent copies should be recognized as duplicates. The customer-facing message state should not change three times.

Callback ordering

Do not assume callback order. Use event timestamps, provider sequence numbers where available and internal transition rules. If the provider cannot guarantee ordering, the internal state machine should be designed accordingly.

Burst protection

A provider can send a large callback burst after recovering from an outage. The webhook endpoint should be protected by rate limits and a queue so callback processing does not overwhelm the database.

Operational example

Provider A sends DELIVERED for message M100, then later sends SUBMITTED because its systems were recovering. The status processor retains DELIVERED as the latest valid state and records the late SUBMITTED event for audit. The customer-facing state is not regressed.

Final developer checklist

Verify signature handling, raw-body validation, event identity, durable queueing, duplicate protection, out-of-order handling, status normalization, unknown-event processing and callback monitoring.

Webhook trust boundary

The webhook endpoint sits at the boundary between the provider and the internal messaging system. Treat every inbound request as untrusted until signature or token validation succeeds.

After authentication, the event still needs schema validation and resource correlation. A valid provider signature does not prove that the event references a currently valid message.

Raw payload retention

Where required for troubleshooting or signature verification, retain the raw provider payload for a defined period. Do not retain it indefinitely if it contains unnecessary personal data.

Event processing transaction

A webhook processor should make event insertion and duplicate detection atomic. If the event is already known, it should acknowledge the duplicate without repeating downstream work.

Webhook backpressure

If callback traffic exceeds processing capacity, the inbound queue should absorb the burst while workers scale within database and downstream limits. The endpoint should not perform expensive business operations synchronously.

Replay handling

Operators may need to replay a failed webhook. Replay should use the same event ID and idempotency controls so it cannot create duplicate billing or status transitions.

Provider-specific event types

Store the original provider event type and normalize it internally. When the provider introduces a new event type, unknown events should be quarantined or logged rather than automatically interpreted as an existing state.

Webhook endpoint rotation

If callback credentials or endpoints must be rotated, support a controlled overlap period where possible. Verify the new endpoint before disabling the old configuration.

Webhook security monitoring

Monitor signature failures, unusual source patterns, repeated event IDs, payload-size anomalies and unknown provider message references. These signals can identify both integration errors and attempted abuse.

Operational scenario

A provider retries a DELIVERED callback five times because the first response was delayed. The endpoint authenticates each request, stores the first event and recognizes the next four as duplicates. The message remains DELIVERED and no additional billing or customer event is generated.

Final production checklist

Verify signature handling, raw payload policy, schema validation, event uniqueness, queueing, status transitions, replay safety, endpoint rotation and callback monitoring.

Webhook event ordering strategy

If a provider offers sequence numbers, store them and use them to improve ordering decisions. If it does not, rely on event timestamps and internal state-transition rules while recognizing that timestamps may also be imperfect.

Never allow a late lower-confidence event to blindly overwrite a stronger terminal state.

Webhook and billing

If delivery events affect billing or customer usage, event processing must be idempotent. A duplicated delivery callback must not create two billable usage events or two downstream notifications.

Separate the raw delivery event from the derived billing or reporting event so corrections remain traceable.

Webhook replay queue

A replay queue should retain the original event ID and a replay attempt identifier. Operators should be able to see whether an event was replayed and why.

Replay should pass through normal validation and authorization controls rather than using an undocumented administrative shortcut.

Final decision guide

Authenticate first, acknowledge quickly, process asynchronously, deduplicate by stable event identity, enforce valid state transitions and retain enough provider evidence for reconciliation.

Webhook timeout policy

Set a clear endpoint response-time target and keep synchronous processing minimal. If a callback cannot be accepted safely, return the provider-appropriate response and allow controlled retry rather than partially processing the event and then failing.

Webhook operational dashboard

A useful dashboard should show callbacks received, authenticated, rejected, queued, processed, duplicated, unknown and failed. Queue age and processing latency should be visible alongside provider event volume.

Webhook rule

Never assume one callback equals one state change. A webhook represents an external event that must pass authentication, deduplication and lifecycle validation before changing internal state.

Final note

A reliable webhook pipeline treats external callbacks as untrusted, repeatable events and converts them into controlled internal state changes.

Implementation scenario

A provider sends a signed delivery event three times. The first event is queued and processed; the next two are recognized by event ID and acknowledged without repeating the state transition. If a later failed event conflicts with a terminal delivered state, the state machine prevents regression and retains the raw event for investigation.