AWS.SQS.SdkException cloud resource_error ai_generated true

AWS.SQS.SdkException: Message size exceeds the maximum allowed (256 KB)

ID: cloud/aws-sqs-message-size-exceeded

Also available as: JSON · Markdown · 中文
90%Fix Rate
88%Confidence
1Evidence
2023-05-20First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
AWS SDK for Python 1.32.0 active — — —
AWS SDK for Java 2.21.0 active — — —
SQS API 2012-11-05 active — — —

Root Cause

Amazon SQS limits individual message payloads to 256 KB (including metadata); sending larger messages triggers this exception.

generic

中文

Amazon SQS 将单个消息负载限制为 256 KB（包括元数据）；发送更大的消息会触发此异常。

Official Documentation

https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/quotas-messages.html

Workarounds

  1. 95% success Store the large payload in Amazon S3 and send the S3 object key in the SQS message. Example: upload to S3, then send JSON {'s3_bucket': 'my-bucket', 's3_key': 'path/to/payload.json'}. 
    Store the large payload in Amazon S3 and send the S3 object key in the SQS message. Example: upload to S3, then send JSON {'s3_bucket': 'my-bucket', 's3_key': 'path/to/payload.json'}.
  2. 90% success Use the SQS Extended Client Library (Java or Python) which automatically handles S3-backed large messages. 
    Use the SQS Extended Client Library (Java or Python) which automatically handles S3-backed large messages.
  3. 75% success Reduce the message payload by removing unnecessary metadata or using a more compact format like Protocol Buffers instead of JSON. 
    Reduce the message payload by removing unnecessary metadata or using a more compact format like Protocol Buffers instead of JSON.

中文步骤

  1. 将大负载存储在 Amazon S3 中，并在 SQS 消息中发送 S3 对象键。例如：上传到 S3，然后发送 JSON {'s3_bucket': 'my-bucket', 's3_key': 'path/to/payload.json'}。
  2. 使用 SQS Extended Client Library（Java 或 Python），自动处理基于 S3 的大消息。
  3. 通过移除不必要的元数据或使用更紧凑的格式（如 Protocol Buffers 而非 JSON）来减少消息负载。

Dead Ends

Common approaches that don't work:

  1. 60% fail

    Compression is not transparent; the receiver must implement decompression, and the raw size limit still applies.

  2. 80% fail

    SQS does not guarantee order across multiple messages unless using FIFO queues with a message group ID.