Skip to main content

How to decouple services with Amazon SQS

Decouple producers and consumers with Amazon SQS queues, dead-letter queues, and Lambda triggers. Covers FIFO ordering and visibility timeout tuning.

Difficulty
Intermediate
Duration
40 minutes
Steps
6

Amazon SQS is a fully managed message queue. It lets a producer hand off work without waiting for a consumer, smoothing spikes and isolating failures. Standard queues offer at-least-once delivery and best-effort ordering; FIFO queues guarantee order and exactly-once processing within a deduplication window.

Prerequisites

  • An AWS account, Node.js, and @aws-sdk/client-sqs.

Steps

1. Create a queue

aws sqs create-queue --queue-name work-queue

For strict ordering, add --queue-name work-queue.fifo --attributes FifoQueue=true.

2. Send messages

import { SQSClient, SendMessageCommand } from "@aws-sdk/client-sqs";
const sqs = new SQSClient({});
await sqs.send(new SendMessageCommand({ QueueUrl: url, MessageBody: JSON.stringify({ jobId: 1 }) }));

3. Receive and delete

Consumers must delete a message after processing, or it reappears after the visibility timeout:

import { ReceiveMessageCommand, DeleteMessageCommand } from "@aws-sdk/client-sqs";
const { Messages } = await sqs.send(new ReceiveMessageCommand({ QueueUrl: url, WaitTimeSeconds: 20 }));
// process, then:
await sqs.send(new DeleteMessageCommand({ QueueUrl: url, ReceiptHandle: Messages[0].ReceiptHandle }));

WaitTimeSeconds enables long polling, reducing empty receives.

4. Add a dead-letter queue

Create a second queue and set a redrive policy so messages that fail repeatedly move there for inspection instead of looping forever. Set maxReceiveCount (for example 5) on the source queue.

5. Trigger Lambda

Map the queue to a Lambda function as an event source. Lambda polls the queue, invokes your handler in batches, and deletes messages automatically on success.

6. Tune visibility timeout

Set the visibility timeout to at least your processing time (plus margin). Too short causes duplicate processing; too long delays retries on crashes.

Verification

Send a batch of messages and confirm the consumer processes each once. Force a handler to throw and confirm the message lands in the dead-letter queue after maxReceiveCount attempts. Check CloudWatch metrics for ApproximateNumberOfMessagesVisible.

Next Steps

Add message attributes for routing, batch sends for throughput, and consider SNS-to-SQS fan-out when multiple consumers need the same message.

Prerequisites

  • AWS account
  • Node.js installed
  • AWS SDK v3 installed

Steps

  • 1
    Create a queue
  • 2
    Send messages
  • 3
    Receive and delete
  • 4
    Add a dead-letter queue
  • 5
    Trigger Lambda
  • 6
    Tune visibility timeout