ProvisionedThroughputExceededException terraform resource_error ai_generated true

Error: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed: ProvisionedThroughputExceededException

ID: terraform/state-lock-dynamodb-throughput-exceeded

Also available as: JSON · Markdown · 中文
85%Fix Rate
87%Confidence
1Evidence
2023-11-05First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Terraform v1.5 active — — —
Terraform v1.6 active — — —
Terraform v1.7 active — — —
AWS SDK v1.44 active — — —

Root Cause

The DynamoDB table used for state locking has insufficient read/write capacity units to handle the concurrent lock requests, causing throttling.

generic

中文

用于状态锁定的 DynamoDB 表的读写容量单位不足，无法处理并发锁定请求，导致节流。

Official Documentation

https://developer.hashicorp.com/terraform/language/settings/backends/s3#dynamodb-state-locking

Workarounds

  1. 95% success Increase DynamoDB table provisioned capacity: `aws dynamodb update-table --table-name terraform-locks --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10` or switch to on-demand billing mode. 
    Increase DynamoDB table provisioned capacity: `aws dynamodb update-table --table-name terraform-locks --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10` or switch to on-demand billing mode.
  2. 70% success Implement a retry with jitter in your CI/CD script: `for i in 1 2 3 4 5; do terraform apply && break || sleep $((RANDOM % 10 + 5)); done` 
    Implement a retry with jitter in your CI/CD script: `for i in 1 2 3 4 5; do terraform apply && break || sleep $((RANDOM % 10 + 5)); done`
  3. 60% success Use `terraform apply -lock=false` to bypass locking entirely (only for non-critical environments, as it risks state corruption). 
    Use `terraform apply -lock=false` to bypass locking entirely (only for non-critical environments, as it risks state corruption).

中文步骤

  1. Increase DynamoDB table provisioned capacity: `aws dynamodb update-table --table-name terraform-locks --provisioned-throughput ReadCapacityUnits=10,WriteCapacityUnits=10` or switch to on-demand billing mode.
  2. Implement a retry with jitter in your CI/CD script: `for i in 1 2 3 4 5; do terraform apply && break || sleep $((RANDOM % 10 + 5)); done`
  3. Use `terraform apply -lock=false` to bypass locking entirely (only for non-critical environments, as it risks state corruption).

Dead Ends

Common approaches that don't work:

  1. Increasing DynamoDB table read capacity units without increasing write capacity 80% fail

    State locking uses write operations (PutItem, DeleteItem) for lock acquisition and release; increasing only read capacity doesn't help with throttled writes.

  2. Switching to a different DynamoDB table with same capacity settings 90% fail

    The issue is capacity, not table identity; any table with low capacity will throttle under high concurrency.

  3. Adding a delay in CI/CD pipeline but not using exponential backoff 75% fail

    Fixed delays are ineffective because concurrent lock attempts can still cluster; Terraform's default retry uses exponential backoff, but if capacity is too low, all retries fail.