Dead Letter Queue Sqs Process Again

This is courtesy of Alexandre Pinhel, Specialist SA Manager, in collaboration with Guillaume Marchand and Luke Hargreaves, Solutions Architects.

Amazon Simple Queue Service (Amazon SQS) is a fully managed bulletin queuing service. It enables you lot to decouple and scale microservices, distributed systems, and serverless applications. A commonly used characteristic of Amazon SQS is expressionless-alphabetic character queues. The DLQ (expressionless-letter queue) is used to shop letters that can't be processed (consumed) successfully.

This post describes how to add automatic resilience to an existing SQS queue. Information technology monitors the expressionless-alphabetic character queue and moves a message dorsum to the main queue to run into if it can be processed once again. It besides uses a specific algorithm to brand sure this is not repeated forever. Each time information technology attempts to reprocess the message, the replay time increases until the bulletin is finally considered dead.

I employ Amazon SQS expressionless-alphabetic character queues, AWS Lambda, and a specific algorithm to subtract the charge per unit of retries for failed messages. I so parcel and publish this serverless solution in the AWS Serverless Application Repository.

Dead-letter queues and message replay

The primary job of a dead-letter queue (DLQ) is to handle bulletin failure. It allows y'all to fix aside and isolate non-processed messages to determine why processing failed. Often these failed messages are caused by awarding errors. For example, a consumer application fails to parse a bulletin correctly and throws an unhandled exception. This exception then triggers an error response that sends the bulletin to the DLQ. The AWS documentation contains a tutorial detailing the configuration of an Amazon SQS expressionless-letter queue.

To procedure the failed messages, I build a retry mechanism by implementing an exponential backoff algorithm. The idea behind exponential backoff is to use progressively longer waits between retries for consecutive error responses. About exponential backoff algorithms apply jitter (randomized delay) to forbid successive collisions. This spreads the bulletin retries more evenly across fourth dimension, allowing them to be processed more efficiently.

Solution overview

The flow of the bulletin sent by the producer to SQS is every bit follows:

1. The producer application sends a message to an SQS queue
2. The consumer application fails to procedure the message in the same SQS queue
3. The message is moved from the main SQS queue to the default dead-letter queue as per the component settings.
4. A Lambda part is configured with the SQS main dead-letter of the alphabet queue as an event source. It receives and sends back the message to the original queue adding a message timer.
5. The message timer is defined past the exponential backoff and jitter algorithm.
6. You tin limit the number of retries. If the bulletin exceeds this limit, the message is moved to a second DLQ where an operator processes it manually.

How the replay office works

Each time the SQS expressionless-letter queue receives a message, it triggers Lambda to run the replay function. The replay code uses an SQS message attribute `sqs-dlq-replay-nb` as a persistent counter for the current number of retries attempted. The number of retries is compared to the maximum number (defined in the awarding configuration file). If it exceeds the maximum, the bulletin is moved to the human operated queue. If not, the role uses the AWS Lambda event data to build a new message for the Amazon SQS main queue. Finally information technology updates the retry counter, adds a new bulletin timer to the bulletin, and it sends the message back (replays) to the principal queue.

          def handler(event, context):     """Lambda role handler."""     for record in outcome['Records']:         nbReplay = 0         # number of replay         if 'sqs-dlq-replay-nb' in record['messageAttributes']:             nbReplay = int(tape['messageAttributes']['sqs-dlq-replay-nb']["stringValue"])          nbReplay += 1         if nbReplay > config.MAX_ATTEMPS:             raise MaxAttempsError(replay=nbReplay, max=config.MAX_ATTEMPS)          # SQS attributes         attributes = record['messageAttributes']         attributes.update({'sqs-dlq-replay-nb': {'StringValue': str(nbReplay), 'DataType': 'Number'}})          _sqs_attributes_cleaner(attributes)          # Backoff         b = backoff.ExpoBackoffFullJitter(base=config.BACKOFF_RATE, cap=config.MESSAGE_RETENTION_PERIOD)         delaySeconds = b.backoff(n=int(nbReplay))          # SQS         SQS.send_message(             QueueUrl=config.SQS_MAIN_URL,             MessageBody=record['body'],             DelaySeconds=int(delaySeconds),             MessageAttributes=record['messageAttributes']         )                  

How to employ the awarding

Y'all can use this serverless application via:

• The Lambda panel: choose the "Browse serverless app repository" option to create a role. Select "amazon-sqs-dlq-replay-backoff" awarding in the public applications repository. And so, configure the awarding with the default SQS parameters and the replay characteristic parameters.
• The Serverless Framework, every bit described by Yan Cui in this blog post.
• An AWS CloudFormation template past using the AWS::ServerlessRepo::Application resources, equally described in the documentation.

Here is an example of a CloudFormation template using the AWS Serverless Application Repository application:

          AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31  Resource:   ReplaySqsQueue:     Blazon: AWS::Serverless::Application     Properties:       Location:          ApplicationId: arn:aws:serverlessrepo:eu-west-1:1234123412:applications~sqs-dlq-replay         SemanticVersion: one.0.0       Parameters:         BackoffRate: "2"         MaxAttempts: "iii"                  

Conclusion

I describe how an exponential backoff algorithm (with jitter) enhances the message processing capabilities of an Amazon SQS queue. You can now find the amazon-sqs-dlq-replay-backoff application in the AWS Serverless Application Repository. Download the code from this GitHub repository.

To go started with dead-letter queues in Amazon SQS, read:

• Using Amazon SQS Dead-Letter of the alphabet Queues
• Monitoring Amazon SQS Using CloudWatch

To implement replay mechanisms, see:

• Increase your noesis on the backoff algorithm reading this weblog post by Marc Brooker.
• Leverage SQS Message Timers feature to manage the bulletin visibility in the queue.

For more serverless learning resources, visit https://serverlessland.com.

lawsonmemered.blogspot.com

Source: https://aws.amazon.com/blogs/compute/using-amazon-sqs-dead-letter-queues-to-replay-messages/