kube-state-metrics is the Kubernetes add-on that exposes the state of every Kubernetes object as a Prometheus metric. It is the second of the two metrics services that every Kubernetes cluster runs, and it is the one most teams do not realize they need until they need it.
Table of contents
What it does
kube-state-metrics watches the Kubernetes API server and generates metrics about the state of every object: pods, deployments, stateful sets, jobs, cron jobs, nodes, services, ingresses, persistent volumes.
The metrics it exposes:
- Pod state. Ready, pending, failed, restart count, container state.
- Deployment state. Replicas available, replicas unavailable, replicas updated, generation.
- Job state. Active, succeeded, failed, completion time.
- Node state. Ready, memory pressure, disk pressure, PID pressure.
- Persistent volume state. Available, bound, released, failed.
What it does not do
kube-state-metrics does not report:
- CPU and memory usage. That is what metrics-server does. The two services are complementary.
- Application-level metrics. That is what the application’s own exporter does.
- Logs. That is what the log pipeline does.
- Traces. That is what the OpenTelemetry collector does.
When you need it
The team needs kube-state-metrics when:
- The team is running any non-trivial workload on Kubernetes.
- The team wants to alert on pod state, deployment state, job state, or node state.
- The team wants to display the state of the cluster in a dashboard.
- The team is running horizontal pod autoscaling (HPA) - the HPA controller needs the metrics to make scaling decisions.
How to install it
The standard install: the kube-state-metrics Helm chart. The chart deploys:
- The kube-state-metrics deployment, with one replica by default.
- A service that exposes the metrics on port 8080.
- A service monitor that configures Prometheus to scrape the service.
The install takes 5 minutes. The chart is the official one; the team should not fork it.
Common queries
The queries most teams end up with:
- Pods in pending state.
kube_pod_status_phase{phase="Pending"} == 1 - Deployments with no available replicas.
kube_deployment_status_replicas_available != kube_deployment_spec_replicas - Jobs that have failed.
kube_job_status_failed > 0 - Nodes not ready.
kube_node_status_condition{condition="Ready",status="true"} == 0 - Pods that have restarted in the last hour.
increase(kube_pod_container_status_restarts_total[1h]) > 0
The gotchas
The gotchas that bite teams:
- Cardinality explosion. kube-state-metrics exposes one metric per object. A cluster with 10,000 pods generates 10,000 metrics. The team should be aware of the cardinality cost and use recording rules to pre-aggregate.
- Stale metrics. kube-state-metrics reads the state from the API server, not from the actual object. If the API server is slow, the metrics are stale. The team should set a reasonable alert threshold for the staleness.
- Pod-name labels. Pod-name labels are high-cardinality and should be aggregated away before being sent to a dashboard. Most teams use the deployment and namespace as the labels.
FAQ
What is kube-state-metrics?
A Kubernetes add-on that exposes the state of every Kubernetes object as a Prometheus metric. It is the second of the two metrics services that every Kubernetes cluster runs.
What is the difference between metrics-server and kube-state-metrics?
metrics-server reports CPU and memory usage for pods and nodes. kube-state-metrics reports the state of every Kubernetes object - pod state, deployment state, job state, node state, persistent volume state.
Do I need kube-state-metrics?
If you are running any non-trivial workload on Kubernetes, yes. It is the source of truth for pod state, deployment state, and job state in your cluster.
How do I install kube-state-metrics?
The standard install is the kube-state-metrics Helm chart. The chart deploys the service, the metrics endpoint, and the Prometheus service monitor.
If you are sizing the observability tier for a new project, the RunxBuild hosting calculator is the place to model the line items. The metrics, the logs, the traces - each one is a separate number, and the team’s mental model for the platform is the sum of those numbers. The RunxBuild dashboard is where the team sees the actual metrics in one place.