cloud messaging ai_generated true

Message reprocessed — SQS visibility timeout too short for processing time

ID: cloud/aws-sqs-visibility-timeout-reprocessing

Also available as: JSON · Markdown
90%Fix Rate
92%Confidence
4Evidence
2023-01-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
any active — — —

Root Cause

SQS visibility timeout shorter than processing time causes message to become visible again.

generic

Workarounds

  1. 93% success Set visibility timeout to 6x max processing time and extend with ChangeMessageVisibility 
    sqs.change_message_visibility(QueueUrl=q, ReceiptHandle=rh, VisibilityTimeout=300)

    Sources: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html

  2. 90% success Make processing idempotent so reprocessing has no side effects 
    Use a deduplication key and check before processing

    Sources: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html

Dead Ends

Common approaches that don't work:

  1. Set visibility timeout equal to average processing time 80% fail

    Average is not max; longer processing will cause reprocessing

  2. Delete message before processing completes 85% fail

    If processing fails after delete, the message is lost forever