403 policy resource_error ai_generated partial

Error 403: Quota exceeded for quota metric 'Read requests' and limit 'Read requests per minute per user' of service 'cloudresourcemanager.googleapis.com' for consumer 'project_number:123456789'.

ID: policy/gcp-resource-manager-quota-exceeded

Also available as: JSON · Markdown · 中文

82%Fix Rate

88%Confidence

1Evidence

2023-08-20First Seen

Version Compatibility

Version	Status	Introduced	Deprecated	Notes
GCP SDK 400.x	active	—	—	—
GCP SDK 410.x	active	—	—	—
GCP SDK 420.x	active	—	—	—

Root Cause

The Cloud Resource Manager API has a per-user per-minute read request quota (default 60/min) that is being exceeded by automated scripts or CI/CD pipelines making too many API calls in a short period.

generic

中文

Cloud Resource Manager API 有每分钟每用户的读取请求配额（默认 60/分钟），自动化脚本或 CI/CD 流水线在短时间内发出过多 API 调用导致超出该配额。

Official Documentation

https://cloud.google.com/resource-manager/docs/limits

Workarounds

85% success Implement exponential backoff with jitter in your code to slow down request rate when hitting quota limits. Example (Python): import time import random def call_with_backoff(client, request, max_retries=5): for i in range(max_retries): try: return client.execute(request) except Exception as e: if 'Quota exceeded' in str(e): sleep_time = (2 ** i) + random.uniform(0, 1) time.sleep(sleep_time) else: raise raise Exception("Max retries exceeded")
```
Implement exponential backoff with jitter in your code to slow down request rate when hitting quota limits.
Example (Python):
  import time
  import random
  def call_with_backoff(client, request, max_retries=5):
      for i in range(max_retries):
          try:
              return client.execute(request)
          except Exception as e:
              if 'Quota exceeded' in str(e):
                  sleep_time = (2 ** i) + random.uniform(0, 1)
                  time.sleep(sleep_time)
              else:
                  raise
      raise Exception("Max retries exceeded")
```
70% success Request a quota increase from GCP console for the Cloud Resource Manager API read requests per minute per user. This requires filing a support ticket. Example: Go to GCP Console > IAM & Admin > Quotas > Select 'Cloud Resource Manager API' > 'Read requests per minute per user' > Edit > Increase limit.
```
Request a quota increase from GCP console for the Cloud Resource Manager API read requests per minute per user. This requires filing a support ticket.
Example:
  Go to GCP Console > IAM & Admin > Quotas > Select 'Cloud Resource Manager API' > 'Read requests per minute per user' > Edit > Increase limit.
```
75% success Batch multiple read requests into a single API call using the 'getIamPolicy' or 'testIamPermissions' methods with multiple resources, reducing the number of requests. Example: # Use list method with page size to reduce calls request = cloudresourcemanager_v3.Projects().list(pageSize=100) while request: response = service.projects().list(body=request).execute() # process response request = service.projects().list_next(request, response)
```
Batch multiple read requests into a single API call using the 'getIamPolicy' or 'testIamPermissions' methods with multiple resources, reducing the number of requests.
Example:
  # Use list method with page size to reduce calls
  request = cloudresourcemanager_v3.Projects().list(pageSize=100)
  while request:
      response = service.projects().list(body=request).execute()
      # process response
      request = service.projects().list_next(request, response)
```

中文步骤

在代码中实现带抖动的指数退避，以在达到配额限制时减慢请求速率。
示例（Python）：
  import time
  import random
  def call_with_backoff(client, request, max_retries=5):
      for i in range(max_retries):
          try:
              return client.execute(request)
          except Exception as e:
              if 'Quota exceeded' in str(e):
                  sleep_time = (2 ** i) + random.uniform(0, 1)
                  time.sleep(sleep_time)
              else:
                  raise
      raise Exception("Max retries exceeded")

从 GCP 控制台请求增加 Cloud Resource Manager API 每分钟每用户读取请求的配额。这需要提交支持工单。
示例：
  转到 GCP 控制台 > IAM 和管理 > 配额 > 选择 'Cloud Resource Manager API' > '每分钟每用户读取请求' > 编辑 > 增加限制。

将多个读取请求批处理到单个 API 调用中，使用 'getIamPolicy' 或 'testIamPermissions' 方法处理多个资源，从而减少请求数量。
示例：
  # 使用 list 方法并设置较大的 page size 以减少调用次数
  request = cloudresourcemanager_v3.Projects().list(pageSize=100)
  while request:
      response = service.projects().list(body=request).execute()
      # 处理响应
      request = service.projects().list_next(request, response)

Dead Ends

Common approaches that don't work:

Retrying immediately after failure 45% fail
Simply retrying the request after a short delay may work temporarily but the quota will be exceeded again if the same rate of requests continues.
Creating a new service account 60% fail
Creating a new service account and re-running the script will still hit the same per-user quota limit if the script makes requests at the same rate.
Disabling and re-enabling the API 80% fail
Disabling and re-enabling the Cloud Resource Manager API does not reset the quota counter; the quota is based on per-user usage over time.