Jan Mikoláš
Practical guide
Cron or message queue: which should you use for automation?
Cron decides when to start. A queue decides how to distribute individual jobs safely between workers.
In short
A time trigger and job transport are not rivals
Cron or a scheduler suits a time event: close a report daily, find expired reservations every five minutes, or issue documents monthly.
A message queue suits work created by an event or distribution of a large batch. It retains backlog, supports multiple workers, and gives every message its own acknowledgement and retry.
Prepare
Five questions before choosing
Choose by trigger, volume, and recovery needs rather than whichever technology already runs in the project.
- Does work begin at a specific time or immediately after an event?
- Is it one short operation or thousands of independent items?
- What should happen after downtime, overlapping runs, or several hours of backlog?
- Can the operation be repeated idempotently and how do you recognize a completed period?
- Do you need worker scaling, per-item retries, and progress tracking?
Steps 1 to 3
Choose the simplest reliable combination
One short daily task does not need a queue cluster. Thousands of items do not belong in one hour-long process without a checkpoint.
1. Use cron for a small time-based unit
- Choose cron when the main requirement is a calendar or interval and the whole job is short, measurable, and restartable.
- Prevent overlap with a lock or a unique period record. With multiple replicas, choose one scheduler or a shared lock.
- Capture exit code, duration, last success, and failure. Cron without monitoring merely starts an invisible problem regularly.
- Decide whether a missed run catches up after downtime. A system cron does not necessarily replay past schedules.
0 2 * * * php bin/console app:daily-report2. Use a queue for individual jobs
- Choose a queue when an order, webhook, or other event creates work that should not block a web request.
- One message should be reasonably small and independently repeatable. A worker acknowledges only after completion.
- Backlog absorbs a spike and more workers add capacity. Monitor the age of the oldest message, not only count.
- Configure bounded retry and a DLQ. A queue without failure policy turns a transient error into an infinite loop or lost message.
event → queue → worker → ack3. Combine both for large scheduled work
- At the scheduled time, create a run and publish small messages for concrete items. Workers perform the actual work.
- Identify the run with a job-plus-period key. A unique constraint prevents two schedulers from dispatching the same batch twice.
- Publish items in pages through an outbox or another reliable mechanism. A scheduler crash must not leave an ambiguous half-batch.
- Derive run state from stored items and metrics, not one process memory. Support safe resumption and targeted retries.
scheduler → create run 2026-08-11 → enqueue item IDs → workersStep 4
Test time, overlap, and backlog
Automation must stay correct when delayed, started twice, or restarted midway.
-
Run two schedulers concurrently
Create one run for the same period and schedule every item once by business identity.
-
Stop workers and build backlog
After recovery, drain the queue at a controlled rate without duplicates or overwhelming dependencies.
-
Kill the scheduler during fan-out
A new process continues from a stored checkpoint or outbox. No item remains permanently unknown.
When it goes wrong
Common mistakes
Cron runs on every replica
Choose one scheduler or use a shared lock and unique period key. A local container lock does not coordinate replicas.
A long cron job stops halfway
Split the list into small idempotent messages and store progress. The scheduler should control fan-out, not hold the batch in one process.
The queue does not know when to create a job
A queue is transport, not a calendar. Create the time event with a scheduler and publish a message afterward.
A daily run is missing after downtime
Define catch-up and store the last successful period. Deliberately create missing intervals after startup.
Done
Automation has the right trigger and work model.
Cron or a scheduler owns time, while a queue distributes and retains jobs. Combine them for a large scheduled batch through stored runs, idempotency keys, and monitoring.