Cloudflare Queues
Status: Production Ready ✅ Last Updated: 2026-01-09 Dependencies: cloudflare-worker-base (for Worker setup) Latest Versions: wrangler@4.58.0, @cloudflare/workers-types@4.20260109.0
Recent Updates (2025):
April 2025: Pull consumers increased limits (5,000 msg/s per queue, up from 1,200 requests/5min) March 2025: Pause & Purge APIs (wrangler queues pause-delivery, queues purge) 2025: Customizable retention (60s to 14 days, previously fixed at 4 days) 2025: Increased queue limits (10,000 queues per account, up from 10) Quick Start (5 Minutes)
1. Create queue
npx wrangler queues create my-queue
2. Add producer binding to wrangler.jsonc
{ "queues": { "producers": [{ "binding": "MY_QUEUE", "queue": "my-queue" }] } }
3. Send message from Worker
await env.MY_QUEUE.send({ userId: '123', action: 'process-order' });
Or publish via HTTP (May 2025+) from any service
curl -X POST "https://api.cloudflare.com/client/v4/accounts/{account_id}/queues/my-queue/messages" \ -H "Authorization: Bearer YOUR_API_TOKEN" \ -d '{"messages": [{"body": {"userId": "123"}}]}'
4. Add consumer binding to wrangler.jsonc
{ "queues": { "consumers": [{ "queue": "my-queue", "max_batch_size": 10 }] } }
5. Process messages
export default {
async queue(batch: MessageBatch, env: Env): Promise
6. Deploy and test
npx wrangler deploy npx wrangler tail my-consumer
Producer API // Send single message await env.MY_QUEUE.send({ userId: '123', action: 'send-email' });
// Send with delay (max 12 hours) await env.MY_QUEUE.send({ action: 'reminder' }, { delaySeconds: 600 });
// Send batch (max 100 messages or 256 KB) await env.MY_QUEUE.sendBatch([ { body: { userId: '1' } }, { body: { userId: '2' } }, ]);
Critical Limits:
Message size: 128 KB max (including ~100 bytes metadata) Messages >128 KB will fail - store in R2 and send reference instead Batch size: 100 messages or 256 KB total Delay: 0-43200 seconds (12 hours max) HTTP Publishing (May 2025+)
New in May 2025: Publish messages to queues via HTTP from any service or programming language.
Source: Cloudflare Changelog
Authentication: Requires Cloudflare API token with Queues Edit permissions.
Single message
curl -X POST "https://api.cloudflare.com/client/v4/accounts/{account_id}/queues/my-queue/messages" \ -H "Authorization: Bearer YOUR_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "messages": [ {"body": {"userId": "123", "action": "process-order"}} ] }'
Batch messages
curl -X POST "https://api.cloudflare.com/client/v4/accounts/{account_id}/queues/my-queue/messages" \ -H "Authorization: Bearer YOUR_API_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "messages": [ {"body": {"userId": "1"}}, {"body": {"userId": "2"}}, {"body": {"userId": "3"}} ] }'
Use Cases:
Publishing from external microservices (Node.js, Python, Go, etc.) Cron jobs running outside Cloudflare Webhook receivers Legacy systems integration Services without Cloudflare Workers SDK Event Subscriptions (August 2025+)
New in August 2025: Subscribe to events from Cloudflare services and consume via Queues.
Source: Cloudflare Changelog
Supported Event Sources:
R2 (bucket.created, object.uploaded, object.deleted, etc.) Workers KV Workers AI Vectorize Workflows Super Slurper Workers Builds
Create Subscription:
npx wrangler queues subscription create my-queue \ --source r2 \ --events bucket.created,object.uploaded
Event Structure:
interface CloudflareEvent { type: string; // 'r2.bucket.created', 'kv.namespace.created' source: string; // 'r2', 'kv', 'ai', etc. payload: any; // Event-specific data metadata: { accountId: string; timestamp: string; }; }
Consumer Example:
export default {
async queue(batch: MessageBatch, env: Env): Promise
switch (event.type) {
case 'r2.bucket.created':
console.log('New R2 bucket:', event.payload.bucketName);
await notifyAdmin(event.payload);
break;
case 'r2.object.uploaded':
console.log('File uploaded:', event.payload.key);
await processNewFile(event.payload.key);
break;
case 'kv.namespace.created':
console.log('New KV namespace:', event.payload.namespaceId);
break;
case 'ai.inference.completed':
console.log('AI inference done:', event.payload.modelId);
break;
}
message.ack();
}
} };
Use Cases:
Build custom workflows triggered by R2 uploads
Monitor infrastructure changes (new KV namespaces, buckets)
Track AI inference jobs
Audit account activity
Event-driven architectures without custom webhooks
Consumer API
export default {
async queue(batch: MessageBatch, env: Env, ctx: ExecutionContext): Promise
await processMessage(message.body);
message.ack(); // Explicit ack (critical for non-idempotent ops)
}
} };
// Retry with exponential backoff message.retry({ delaySeconds: Math.min(60 * Math.pow(2, message.attempts - 1), 3600) });
// Batch methods batch.ackAll(); // Ack all messages batch.retryAll(); // Retry all messages
Critical:
message.ack() - Mark success, prevents retry even if handler fails later Use explicit ack for non-idempotent operations (DB writes, API calls, payments) Implicit ack - If handler returns successfully without calling ack(), all messages auto-acknowledged Ordering not guaranteed - Don't assume FIFO message order Critical Consumer Patterns Explicit Acknowledgement (Non-Idempotent Operations)
ALWAYS use explicit ack() for: Database writes, API calls, financial transactions
export default {
async queue(batch: MessageBatch, env: Env): PromiseFailed ${message.id}:, error);
// Don't ack - will retry
}
}
}
};
Why? Prevents duplicate writes if one message in batch fails. Failed messages retry independently.
Exponential Backoff for Rate-Limited APIs
export default {
async queue(batch: MessageBatch, env: Env): Promise
Dead Letter Queue (DLQ) - CRITICAL for Production
⚠️ Without DLQ, failed messages are DELETED PERMANENTLY after max_retries
npx wrangler queues create my-dlq
wrangler.jsonc:
{ "queues": { "consumers": [{ "queue": "my-queue", "max_retries": 3, "dead_letter_queue": "my-dlq" // Messages go here after 3 failed retries }] } }
DLQ Consumer:
export default {
async queue(batch: MessageBatch, env: Env): Promise
Known Issues Prevention
This skill prevents 13 documented issues:
Issue #1: Multiple Dev Commands - Queues Don't Flow Between Processes
Error: Queue messages sent in one wrangler dev process don't appear in another wrangler dev consumer process Source: GitHub Issue #9795
Why It Happens: The virtual queue used by wrangler is in-process memory. Separate dev processes cannot share the queue state.
Prevention:
❌ Don't run producer and consumer as separate processes
Terminal 1: wrangler dev (producer)
Terminal 2: wrangler dev (consumer) # Won't receive messages!
✅ Option 1: Run both in single dev command
wrangler dev -c producer/wrangler.jsonc -c consumer/wrangler.jsonc
✅ Option 2: Use Vite plugin with auxiliaryWorkers
vite.config.ts:
export default defineConfig({ plugins: [ cloudflare({ auxiliaryWorkers: ['./consumer/wrangler.jsonc'] }) ] })
Issue #2: Queue Producer Binding Causes 500 Errors with Remote Dev
Error: All routes return 500 Internal Server Error when using wrangler dev --remote with queue bindings Source: GitHub Issue #9642
Why It Happens: Queues are not yet supported in wrangler dev --remote mode. Even routes that don't use the queue binding fail.
Prevention:
// When using remote dev, temporarily comment out queue bindings { "queues": { // "producers": [{ "queue": "my-queue", "binding": "MY_QUEUE" }] } }
// Or use local dev instead // wrangler dev (without --remote)
Issue #3: D1 Remote Breaks When Queue Remote is Set
Error: D1 remote binding stops working when remote: true is set on queue producer binding Source: GitHub Issue #11106
Why It Happens: Binding conflict issue affecting mixed local/remote development.
Prevention:
// ❌ Don't mix D1 remote with queue remote { "d1_databases": [{ "binding": "DB", "database_id": "...", "remote": true }], "queues": { "producers": [{ "binding": "QUEUE", "queue": "my-queue", "remote": true // ❌ Breaks D1 remote }] } }
// ✅ Avoid remote on queues when using D1 remote { "d1_databases": [{ "binding": "DB", "remote": true }], "queues": { "producers": [{ "binding": "QUEUE", "queue": "my-queue" }] } }
Status: No workaround yet. Track issue for updates.
Issue #4: Mixed Local/Remote Bindings - Queue Consumer Missing
Error: Queue consumer binding does not appear when mixing local queues with remote AI/Vectorize bindings Source: GitHub Issue #9887
Why It Happens: Wrangler doesn't support mixed local/remote bindings in the same worker.
Prevention:
// ❌ Don't mix local queues with remote AI { "queues": { "consumers": [{ "queue": "my-queue" }] }, "ai": { "binding": "AI", "experimental_remote": true // ❌ Breaks queue consumer } }
// ✅ Option 1: All local (no remote bindings) wrangler dev
// ✅ Option 2: Separate workers for queues vs AI // Worker 1: Queue processing (local) // Worker 2: AI operations (remote)
Issue #5: http_pull Type Prevents Worker Consumer Execution
Error: Queue consumer with type: "http_pull" doesn't execute in production Source: GitHub Issue #6619
Why It Happens: http_pull is for external HTTP-based consumers, not Worker-based consumers.
Prevention:
// ❌ Don't use type: "http_pull" for Worker consumers { "queues": { "consumers": [{ "queue": "my-queue", "type": "http_pull", // ❌ Wrong for Workers "max_batch_size": 10 }] } }
// ✅ Omit type field for push-based Worker consumers { "queues": { "consumers": [{ "queue": "my-queue", "max_batch_size": 10 // No "type" field - defaults to Worker consumer }] } }
Breaking Changes & Deprecations delivery_delay in Producer Config (Upcoming Removal)
Warning: The delivery_delay parameter in producer bindings will be removed in a future wrangler version.
Source: GitHub Issue #10286
// ❌ Will be removed - don't use { "queues": { "producers": [{ "binding": "MY_QUEUE", "queue": "my-queue", "delivery_delay": 300 // ❌ Don't use this }] } }
Migration: Use per-message delay instead:
// ✅ Correct approach - per-message delay await env.MY_QUEUE.send({ data }, { delaySeconds: 300 });
Why: Workers should not affect queue-level settings. With multiple producers, the setting from the last-deployed producer wins, causing unpredictable behavior.
Community Tips
Note: These tips come from community discussions and GitHub issues. Verify against your wrangler version.
Tip: max_batch_timeout May Break Local Development
Source: GitHub Issue #6619 Confidence: MEDIUM Applies to: Local development with wrangler dev
If your queue consumer doesn't execute locally, try removing max_batch_timeout:
{ "queues": { "consumers": [{ "queue": "my-queue", "max_batch_size": 10 // Remove max_batch_timeout for local dev }] } }
This appears to be version-specific and may not affect all setups.
Tip: Queue Name Not Available on Producer Bindings
Source: GitHub Issue #10131 Confidence: HIGH Applies to: Multi-environment deployments (staging, PR previews, tenant-specific queues)
Queue names are only available via batch.queue in consumer handlers, not on producer bindings. This creates issues with environment-specific queue names like email-queue-staging or email-queue-pr-123.
Current Limitation:
// ❌ Can't access queue name from producer binding const queueName = env.MY_QUEUE.name; // Doesn't exist!
// ❌ Must hardcode or normalize in consumer switch (batch.queue) { case 'email-queue': // What about email-queue-staging? case 'email-queue-staging': // Must handle all variants case 'email-queue-pr-123': // Dynamic env names break this }
Workaround:
// In consumer: Normalize queue name function normalizeQueueName(queueName: string): string { return queueName.replace(/-staging|-pr-\d+|-dev/g, ''); }
switch (normalizeQueueName(batch.queue)) { case 'email-queue': // Handle all email-queue-* variants }
Status: Feature request tracked internally: MQ-923
Consumer Configuration { "queues": { "consumers": [{ "queue": "my-queue", "max_batch_size": 100, // 1-100 (default: 10) "max_batch_timeout": 30, // 0-60s (default: 5s) "max_retries": 5, // 0-100 (default: 3) "retry_delay": 300, // Seconds (default: 0) "max_concurrency": 10, // 1-250 (default: auto-scale) "dead_letter_queue": "my-dlq" // REQUIRED for production }] } }
Critical Settings:
Batching - Consumer called when EITHER condition met (max_batch_size OR max_batch_timeout) max_retries - After exhausted: with DLQ → sent to DLQ, without DLQ → DELETED PERMANENTLY max_concurrency - Only set if upstream has rate limits or connection limits. Otherwise leave unset for auto-scaling (up to 250 concurrent invocations) DLQ - Create separately: npx wrangler queues create my-dlq Wrangler Commands
Create queue
npx wrangler queues create my-queue npx wrangler queues create my-queue --message-retention-period-secs 1209600 # 14 days
Manage queues
npx wrangler queues list npx wrangler queues info my-queue npx wrangler queues delete my-queue # ⚠️ Deletes ALL messages!
Pause/Purge (March 2025 - NEW)
npx wrangler queues pause-delivery my-queue # Pause processing, keep receiving npx wrangler queues resume-delivery my-queue npx wrangler queues purge my-queue # ⚠️ Permanently deletes all messages!
Consumer management
npx wrangler queues consumer add my-queue my-consumer-worker \ --batch-size 50 --batch-timeout 10 --message-retries 5 npx wrangler queues consumer remove my-queue my-consumer-worker
Limits & Quotas Feature Limit Queues per account 10,000 Message size 128 KB (includes ~100 bytes metadata) Message retries 100 max Batch size 1-100 messages Batch timeout 0-60 seconds Messages per sendBatch 100 (or 256 KB total) Queue throughput 5,000 messages/second per queue Message retention 4 days (default), 14 days (max) Queue backlog size 25 GB per queue Concurrent consumers 250 (push-based, auto-scale) Consumer duration 15 minutes (wall clock) Consumer CPU time 30 seconds (default), 5 minutes (max) Visibility timeout 12 hours (pull consumers) Message delay 12 hours (max) API rate limit 1200 requests / 5 minutes Pricing
Requires Workers Paid plan ($5/month)
Operations Pricing:
First 1,000,000 operations/month: FREE After that: $0.40 per million operations
What counts as an operation:
Each 64 KB chunk written, read, or deleted Messages >64 KB count as multiple operations: 65 KB message = 2 operations 127 KB message = 2 operations 128 KB message = 2 operations
Typical message lifecycle:
1 write + 1 read + 1 delete = 3 operations
Retries:
Each retry = additional read operation Message retried 3 times = 1 write + 4 reads + 1 delete = 6 operations
Dead Letter Queue:
Writing to DLQ = additional write operation
Cost examples:
1M messages/month (no retries): ((1M × 3) - 1M) / 1M × $0.40 = $0.80 10M messages/month: ((10M × 3) - 1M) / 1M × $0.40 = $11.60 100M messages/month: ((100M × 3) - 1M) / 1M × $0.40 = $119.60 Error Handling Common Errors 1. Message Too Large // ❌ Bad: Message >128 KB await env.MY_QUEUE.send({ data: largeArray, // >128 KB });
// ✅ Good: Check size before sending const message = { data: largeArray }; const size = new TextEncoder().encode(JSON.stringify(message)).length;
if (size > 128000) {
// Store in R2, send reference
const key = messages/${crypto.randomUUID()}.json;
await env.MY_BUCKET.put(key, JSON.stringify(message));
await env.MY_QUEUE.send({ type: 'large-message', r2Key: key });
} else {
await env.MY_QUEUE.send(message);
}
- Throughput Exceeded // ❌ Bad: Exceeding 5000 msg/s per queue for (let i = 0; i < 10000; i++) { await env.MY_QUEUE.send({ id: i }); // Too fast! }
// ✅ Good: Use sendBatch const messages = Array.from({ length: 10000 }, (_, i) => ({ body: { id: i }, }));
// Send in batches of 100 for (let i = 0; i < messages.length; i += 100) { await env.MY_QUEUE.sendBatch(messages.slice(i, i + 100)); }
// ✅ Even better: Rate limit with delay for (let i = 0; i < messages.length; i += 100) { await env.MY_QUEUE.sendBatch(messages.slice(i, i + 100)); if (i + 100 < messages.length) { await new Promise(resolve => setTimeout(resolve, 100)); // 100ms delay } }
- Consumer Timeout
// ❌ Bad: Long processing without CPU limit increase
export default {
async queue(batch: MessageBatch): Promise
{ for (const message of batch.messages) { await processForMinutes(message.body); // CPU timeout! } }, };
// ✅ Good: Increase CPU limit in wrangler.jsonc
wrangler.jsonc:
{ "limits": { "cpu_ms": 300000 // 5 minutes (max allowed) } }
- Backlog Growing // Issue: Consumer too slow, backlog growing
// ✅ Solution 1: Increase batch size { "queues": { "consumers": [{ "queue": "my-queue", "max_batch_size": 100 // Process more per invocation }] } }
// ✅ Solution 2: Let concurrency auto-scale (don't set max_concurrency)
// ✅ Solution 3: Optimize consumer code
export default {
async queue(batch: MessageBatch, env: Env): Promise
Critical Rules
Always:
✅ Configure DLQ for production (dead_letter_queue in consumer config) ✅ Use explicit message.ack() for non-idempotent ops (DB writes, API calls) ✅ Validate message size <128 KB before sending ✅ Use sendBatch() for multiple messages (more efficient) ✅ Implement exponential backoff: 60 * Math.pow(2, message.attempts - 1) ✅ Let concurrency auto-scale (don't set max_concurrency unless upstream has rate limits)
Never:
❌ Never assume FIFO ordering - not guaranteed ❌ Never rely on implicit ack for non-idempotent ops - use explicit ack() ❌ Never send messages >128 KB - will fail (store in R2 instead) ❌ Never skip DLQ in production - failed messages DELETED PERMANENTLY without DLQ ❌ Never exceed 5,000 msg/s per queue (push consumers) or rate limits apply ❌ Never process messages synchronously - use Promise.all() for parallelism Troubleshooting Issue: Messages not being delivered to consumer
Possible causes:
Consumer not deployed Wrong queue name in wrangler.jsonc Delivery paused Consumer throwing errors
Solution:
Check queue info
npx wrangler queues info my-queue
Check if delivery paused
npx wrangler queues resume-delivery my-queue
Check consumer logs
npx wrangler tail my-consumer
Issue: Entire batch retried when one message fails
Cause: Using implicit acknowledgement with non-idempotent operations
Solution: Use explicit ack()
// ✅ Explicit ack
for (const message of batch.messages) {
try {
await dbWrite(message.body);
message.ack(); // Only ack on success
} catch (error) {
console.error(Failed: ${message.id});
// Don't ack - will retry
}
}
Issue: Messages deleted without processing
Cause: No Dead Letter Queue configured
Solution:
Create DLQ
npx wrangler queues create my-dlq
Add to consumer config
{ "queues": { "consumers": [{ "queue": "my-queue", "dead_letter_queue": "my-dlq" }] } }
Issue: Consumer not auto-scaling
Possible causes:
max_concurrency set to 1 Consumer returning errors (not processing) Batch processing too fast (no backlog)
Solution:
{ "queues": { "consumers": [{ "queue": "my-queue", // Don't set max_concurrency - let it auto-scale "max_batch_size": 50 // Increase batch size instead }] } }
Related Documentation Cloudflare Queues Docs How Queues Works JavaScript APIs Batching & Retries Consumer Concurrency Dead Letter Queues Wrangler Commands Limits Pricing
Last Updated: 2026-01-21 Version: 2.0.0 Changes: Added HTTP Publishing (May 2025), Event Subscriptions (Aug 2025), Known Issues Prevention (13 issues), Breaking Changes section, Community Tips. Error count: 0 → 13. Major feature additions and comprehensive issue documentation. Maintainer: Jeremy Dawes | jeremy@jezweb.net