Troubleshooting guide · Pod stuck in Pending

Pod stuck in Pending: why Kubernetes will not schedule it

A pod stays Pending when the scheduler cannot find a node that satisfies everything it asks for — its CPU and memory requests, its node selector and affinity, its tolerations, its volumes. The FailedScheduling event lists what ruled out each node, and that list is the diagnosis.

Updated

First, check it is really unscheduled

Pending is a phase, and it also covers a pod that has been placed on a node and is still pulling its image. A pod with no node assigned is the scheduling problem this guide is about:

NODE reads <none> for a pod the scheduler has not placed:

kubectl get pod <pod> -n <namespace> -o wide

Read the scheduler's message

The FailedScheduling event, at the bottom:

kubectl describe pod <pod> -n <namespace>

It reads like: 0/4 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }, 3 Insufficient cpu. Every node is accounted for, grouped by the reason it was ruled out — here, one node is tainted and the other three do not have enough CPU left to reserve.

What each reason means

  • Insufficient cpu or Insufficient memory: no node has that much left to reserve. The scheduler counts REQUESTS, not usage — a node can be nearly idle and still full. Lower the request to what the container really needs, free capacity, or add nodes.
  • A request larger than any node: a pod asking for more CPU or memory than one node has in total will never schedule, however many nodes there are.
  • had untolerated taint: the node is reserved for something else. Add a matching toleration only if the pod belongs there.
  • didn't match Pod's node affinity/selector: the pod's nodeSelector or affinity names labels no available node carries. Check the labels with kubectl get nodes --show-labels.
  • unbound immediate PersistentVolumeClaims: the pod needs a volume that does not exist yet. Check the claim's status, its StorageClass and whether a provisioner is running.
  • Too many pods: the node has reached its maximum pod count.

See what is already reserved

What each node can offer, and what its pods have already requested, under Allocated resources:

kubectl describe nodes

The pod's own requests:

kubectl get pod <pod> -n <namespace> -o jsonpath='{range .spec.containers[*]}{.name}{"\t"}{.resources.requests}{"\n"}{end}'

If the cluster runs an autoscaler — Cluster Autoscaler or Karpenter — a Pending pod is what asks it for a node. When none arrives, the autoscaler's own events or logs say why, often a request no available instance type can fit.

When there is no pod at all

If a Deployment shows fewer ready replicas and there is no Pending pod, the pods were never created. A ResourceQuota in the namespace refuses them at creation; the ReplicaSet's events say so:

Look for exceeded quota:

kubectl describe replicaset -n <namespace> -l app=<label>

What PodSteer does about it.

  • A pod that cannot schedule is a finding on the overview, and the scheduler's FailedScheduling message is quoted beside it.
  • The capacity panel measures CPU and memory against what pods have requested, which is what decides whether the next pod fits — not against usage.
  • Rightsizing findings point at workloads that reserve far more than they use — the requests that fill a cluster while its usage graphs look calm.
PodSteer's cluster overview with its findings open under a banner reading Needs attention, 4 critical and 5 warning findings. Listed: an image that cannot be pulled for two pods of storefront/reviews, two unschedulable pods of storefront/search-indexer, CrashLoopBackOff for the image-resizer deployment, an unschedulable Trivy scanner job, and the start of a warning about failing startup probes. Each finding says what the state means. PodSteer's cluster overview with its findings open under a banner reading Needs attention, 4 critical and 5 warning findings. Listed: an image that cannot be pulled for two pods of storefront/reviews, two unschedulable pods of storefront/search-indexer, CrashLoopBackOff for the image-resizer deployment, an unschedulable Trivy scanner job, and the start of a warning about failing startup probes. Each finding says what the state means.
Findings, named by the workload. Each finding names the workload it belongs to and says what the state means, before a single pod has been opened.

CrashLoopBackOff

The container starts, exits and is restarted with a growing delay. Find the reason in the previous attempt's logs and its exit code.

OOMKilled

The container used more memory than its limit and the kernel killed it. Confirm it, measure the real peak, then size the limit or fix the leak.

ImagePullBackOff

The node cannot fetch the container image. The event message says whether the image is missing, the registry refused, or it could not be reached.