How to decouple services with Amazon SQS
Decouple producers and consumers with Amazon SQS queues, dead-letter queues, and Lambda triggers. Covers FIFO ordering and visibility timeout tuning.
Amazon SQS is a fully managed message queue. It lets a producer hand off work without waiting for a consumer, smoothing spikes and isolating failures. Standard queues offer at-least-once delivery and best-effort ordering; FIFO queues guarantee order and exactly-once processing within a deduplication window.
Prerequisites
- An AWS account, Node.js, and
@aws-sdk/client-sqs.
Steps
1. Create a queue
aws sqs create-queue --queue-name work-queue
For strict ordering, add --queue-name work-queue.fifo --attributes FifoQueue=true.
2. Send messages
import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
const sqs = new SQSClient({});
await sqs.send(new SendMessageCommand({ QueueUrl: url, MessageBody: JSON.stringify({ jobId: 1 }) }));
3. Receive and delete
Consumers must delete a message after processing, or it reappears after the visibility timeout:
import { ReceiveMessageCommand, DeleteMessageCommand } from "@aws-sdk/client-sqs";
const { Messages } = await sqs.send(new ReceiveMessageCommand({ QueueUrl: url, WaitTimeSeconds: 20 }));
// process, then:
await sqs.send(new DeleteMessageCommand({ QueueUrl: url, ReceiptHandle: Messages[0].ReceiptHandle }));
WaitTimeSeconds enables long polling, reducing empty receives.
4. Add a dead-letter queue
Create a second queue and set a redrive policy so messages that fail repeatedly move there for inspection instead of looping forever. Set maxReceiveCount (for example 5) on the source queue.
5. Trigger Lambda
Map the queue to a Lambda function as an event source. Lambda polls the queue, invokes your handler in batches, and deletes messages automatically on success.
6. Tune visibility timeout
Set the visibility timeout to at least your processing time (plus margin). Too short causes duplicate processing; too long delays retries on crashes.
Verification
Send a batch of messages and confirm the consumer processes each once. Force a handler to throw and confirm the message lands in the dead-letter queue after maxReceiveCount attempts. Check CloudWatch metrics for ApproximateNumberOfMessagesVisible.
Next Steps
Add message attributes for routing, batch sends for throughput, and consider SNS-to-SQS fan-out when multiple consumers need the same message.
Prerequisites
- AWS account
- Node.js installed
- AWS SDK v3 installed
Steps
- 1Create a queue
- 2Send messages
- 3Receive and delete
- 4Add a dead-letter queue
- 5Trigger Lambda
- 6Tune visibility timeout