kubernetes runtime_error ai_generated true

failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)

ID: kubernetes/horizontal-pod-autoscaler-missing-metrics

Also available as: JSON · Markdown · 中文
92%Fix Rate
90%Confidence
1Evidence
2023-11-01First Seen

Version Compatibility

VersionStatusIntroducedDeprecatedNotes
kubernetes 1.27 active — — —
metrics-server 0.6 active — — —
metrics-server 0.7 active — — —

Root Cause

The metrics-server (or custom metrics adapter) is not installed or not functioning, so the Horizontal Pod Autoscaler cannot retrieve CPU/memory metrics.

generic

中文

metrics-server（或自定义指标适配器）未安装或未正常运行，导致 Horizontal Pod Autoscaler 无法检索 CPU/内存指标。

Official Documentation

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-metrics-apis

Workarounds

  1. 95% success Deploy metrics-server using `kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml`. Verify it's running: `kubectl get pods -n kube-system | grep metrics-server`. Check logs for errors: `kubectl logs -n kube-system deployment/metrics-server`. 
    Deploy metrics-server using `kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml`. Verify it's running: `kubectl get pods -n kube-system | grep metrics-server`. Check logs for errors: `kubectl logs -n kube-system deployment/metrics-server`.
  2. 85% success Run `kubectl get apiservices.apiregistration.k8s.io | grep metrics` to see if `v1beta1.metrics.k8s.io` or `v1beta1.custom.metrics.k8s.io` is available. If not, ensure the metrics-server deployment is healthy and its service endpoints are reachable. 
    Run `kubectl get apiservices.apiregistration.k8s.io | grep metrics` to see if `v1beta1.metrics.k8s.io` or `v1beta1.custom.metrics.k8s.io` is available. If not, ensure the metrics-server deployment is healthy and its service endpoints are reachable.

中文步骤

  1. 使用 `kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml` 部署 metrics-server。验证其运行：`kubectl get pods -n kube-system | grep metrics-server`。检查日志错误：`kubectl logs -n kube-system deployment/metrics-server`。
  2. 运行 `kubectl get apiservices.apiregistration.k8s.io | grep metrics` 查看 `v1beta1.metrics.k8s.io` 或 `v1beta1.custom.metrics.k8s.io` 是否可用。如果不可用，确保 metrics-server 部署健康且其服务端点可达。

Dead Ends

Common approaches that don't work:

  1. 95% fail

    Changing the target doesn't fix the missing metrics API; the HPA still can't get any data.

  2. 90% fail

    The HPA controller is not the source of metrics; it relies on the metrics API which is unavailable.

  3. 50% fail

    Conflicting adapters can cause API registration issues; the correct approach is to ensure only one metrics provider is active.