Glossary

Outbox Pattern

The Transactional Outbox Pattern replaces an unreliable dual write with one atomic database step and repeatable delivery. It does not, however, promise exactly-once delivery.

Short definition

The business change and intent to send a message are created together.

The more precise name for the transactional variant of the Outbox Pattern is Transactional Outbox Pattern. The application stores both the business-data change and the message intended for later delivery in the same store and one database transaction. If the transaction fails, neither the order nor the message exists. If it commits, the message remains durably ready for delivery even if the application process crashes.

A separate publisher or relay reads unprocessed entries and publishes them to a message broker. The database acknowledgement and broker acknowledgement are still not one atomic operation: the relay may crash after publishing but before recording the “sent” state. Delivery is therefore commonly at least once, and the recipient must safely handle a duplicate.

Problem it solves

A dual write can leave two different truths.

A direct database write and separate broker send leave a moment between them in which the application, network, or broker can fail.

the order is stored, but OrderCreated is never sent because the process crashes
the message is sent before commit, but the database transaction subsequently rolls back
the HTTP request times out even though one of the operations has already completed
the publisher cannot know whether the broker accepted the message and must safely retry
multiple publisher instances compete for the same outbox entries and must coordinate their processing

Practical example and diagram

A new order and the OrderCreated event

A PHP application receives an order and opens a short transaction. It inserts the order into the orders table and an entry with a unique message_id, the OrderCreated type, order identifier, payload, creation time, and optionally an aggregate sequence into the outbox table. Only then does it commit. The request does not need to wait for RabbitMQ, and the network call stays outside the database transaction.

RabbitMQ can be the target broker, but it is not part of the pattern’s definition. A worker periodically selects unsent entries, publishes them, and records processing after the broker acknowledges them. If it crashes between the last two steps, the same message_id may appear again; the consumer therefore checks before the side effect whether it has already processed the message.

Text diagram and its alternative

HTTP request
      ↓
DB TRANSACTION
 ├─ Order(order-42)
 └─ Outbox(message-123)
      ↓ COMMIT
Publisher / Relay
      ↓
Message broker
      ↓ may deliver again
Idempotent consumer

How it works

From a local transaction to message delivery

The sequence also serves as a textual alternative to the diagram. An implementation may use polling or change data capture; the fundamental reliability boundary remains the same.

One local transaction The application inserts or changes business data while creating a durable outbox record. Both changes commit or both roll back.
Selecting waiting entries The publisher reads a batch of unprocessed messages. With multiple workers it needs sensible locking, leasing, or another mechanism that limits wasteful parallel work.
Publishing A message queue or topic accepts the message. The relay distinguishes a transient failure from a permanent one and uses limited retries with backoff.
Recording the result After the broker acknowledgement, the publisher marks the entry as published, moves it, or maintains a separate cursor. Old records are cleaned up according to a retention policy.
Idempotent processing Idempotence protects the business effect from a repeated message_id. Deduplication must be atomic with the consumer’s local change, otherwise a new dual-write gap remains.

Main parts and principles

An outbox is an operational mechanism, not merely a helper table.

Reliability comes from combining an atomic write, the relay, observable retries, and a safe recipient.

Outbox record

It carries a stable message identifier, contract type and version, creation time, payload, and source identity. In a relational database this is often a table; in a document store the message can be part of a document or transactional batch.

Polling publisher

A worker reads waiting entries at intervals. It is easy to understand, but needs a suitable index, batching, concurrency control, and a metric for the age of the oldest message. Polling too aggressively loads the database, while polling too slowly raises latency.

CDC or log tailing

Change Data Capture can follow a database log and forward changes with lower latency. It is only a relay variant and adds a dependency on the particular database and operational infrastructure, so it is not automatically suitable everywhere.

Message ordering

Global ordering is expensive and often unnecessary. Ordering events for one order can matter; an aggregate_id and sequence are then stored and the broker is partitioned accordingly. The outbox alone does not guarantee order at every consumer.

Domain and integration messages

An internal domain event need not directly become a public integration contract. The outbox payload should publish the stable minimum required by consumers rather than thoughtlessly copying the application’s internal model.

Benefits and limitations

Fewer lost messages at the cost of more stored state.

Benefits

removes the gap between committing business data and durably recording the intent to publish
does not require a distributed 2PC transaction between the database and broker
allows publishing to recover after a process crash or temporary broker outage
exposes backlog, age, and delivery failures for monitoring and controlled intervention

Limitations and common mistakes

the Outbox Pattern does not ensure exactly-once delivery; the relay can publish the same message more than once
a consumer without idempotence can send a second email, deduct inventory again, or create another payment
a “published” state stored before broker acknowledgement can lose the message, while storing it after allows duplicates
missing retention, an index, and an alert for backlog age cause table growth and hidden delay
an outbox does not coordinate several business steps by itself and does not solve every distributed transaction

When it fits

When a committed local change must reliably trigger follow-up work.

The pattern suits order events, search-index updates, inventory synchronisation, and starting an asynchronous workflow. It applies in a monolith as well as in microservices; CQRS, Event Sourcing, and a particular broker are not prerequisites. What matters is a database change and a message that must semantically be created together.

It adds less value when the follow-up operation can be safely derived from authoritative data and an occasional scheduler run is enough. An outbox is also not a replacement for the Saga Pattern: it helps reliably carry individual commands and events, while the overall process state, compensations, and decision about the next step need a separate workflow design.

Operations and checks

Monitor delivery as a production process in its own right.

A successful commit means a stored intent, not completed processing in every downstream system.

measure waiting-entry count, age, delivery time, and retry count
use a stable message_id and atomic deduplication at the consumer
define a retry limit, backoff, dead letter, or traceable permanent-failure state
test crashes after the database commit, after broker publish, and while recording processing
version the message contract and protect order only where it has business meaning
maintain outbox retention and indexes so the publisher does not slow ordinary transactions

Common questions

Transactional Outbox Pattern guarantees

Does the Outbox Pattern guarantee exactly-once delivery?

No. The relay may crash after a successful publish but before saving the result, then send the message again after restart. At-least-once delivery with idempotent processing is the usual goal.

Must an outbox always be a separate SQL table?

No. A table is common for a relational database, while a document database can use a document or change feed. The essential property is that the business change and message are created within one supported atomic transaction boundary.

Does the Outbox Pattern require RabbitMQ?

No. The destination may be RabbitMQ, Kafka, a cloud queue, or another delivery system. The pattern connects a local database change with later publishing; it does not choose the broker.

Does an outbox solve the whole distributed transaction?

No. It reliably records and forwards a message tied to a local commit. Subsequent business steps, their state, conflicts, and compensations need a separate design, such as a saga.

Can CDC replace polling?

Yes, when the database and operational environment provide a suitable change feed or log. CDC can lower latency, but it does not remove the need for idempotence, monitoring, and message-schema management.

How I handle integration reliability

I separate data changes from network delivery with a clear and observable boundary.

When designing integration flows I account for crashes between steps, duplicates, retries, and tracing a message’s production state.

View my approach to application architecture Discuss your project without obligation