policy runtime_error ai_generated true

Error: Error acquiring the state lock. Lock Info: Lock ID: <id>. Error: ConditionalCheckFailedException: The conditional request failed

ID: policy/terraform-state-lock-timeout-dynamodb

Also available as: JSON · Markdown · 中文
80%Fix Rate
85%Confidence
1Evidence
2023-08-15First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
Terraform v1.5.0 active — — —
Terraform v1.6.0 active — — —
DynamoDB standard table active — — —

Root Cause

Terraform state locking via DynamoDB fails when another process holds the lock longer than the default retry timeout, often due to long-running applies or stale locks.

generic

中文

Terraform 通过 DynamoDB 的状态锁定在另一个进程持有锁超过默认重试超时时失败，通常是由于长时间运行的 apply 或过时的锁。

Official Documentation

https://developer.hashicorp.com/terraform/language/settings/backends/s3#dynamodb-state-locking

Workarounds

  1. 85% success Identify the lock holder via AWS CLI: aws dynamodb get-item --table-name terraform-locks --key '{"LockID": {"S": "<lock-id>"}}'. If the holder is stale, run terraform force-unlock -force <lock-id>. 
    Identify the lock holder via AWS CLI: aws dynamodb get-item --table-name terraform-locks --key '{"LockID": {"S": "<lock-id>"}}'. If the holder is stale, run terraform force-unlock -force <lock-id>.
  2. 75% success Increase the lock retry timeout in terraform block: terraform { backend "s3" { lock_timeout = "5m" } } to allow longer waits. 
    Increase the lock retry timeout in terraform block: terraform { backend "s3" { lock_timeout = "5m" } } to allow longer waits.
  3. 70% success Implement a pre-apply check script that verifies no other CI pipeline is running using the same state file. 
    Implement a pre-apply check script that verifies no other CI pipeline is running using the same state file.

中文步骤

  1. 通过 AWS CLI 识别锁持有者：aws dynamodb get-item --table-name terraform-locks --key '{"LockID": {"S": "<lock-id>"}}'。如果持有者已过期，运行 terraform force-unlock -force <lock-id>。
  2. 在 terraform 块中增加锁重试超时：terraform { backend "s3" { lock_timeout = "5m" } } 以允许更长的等待。
  3. 实施预 apply 检查脚本，验证没有其他 CI 管道使用相同状态文件运行。

Dead Ends

Common approaches that don't work:

  1. 60% fail

    Running terraform force-unlock without verifying the lock holder can corrupt the state if the original process is still running.

  2. 80% fail

    Deleting the DynamoDB lock table entirely loses all lock state and requires reinitialization.

  3. 50% fail

    Manually editing the DynamoDB item with incorrect permissions may cause AccessDenied errors.