Deployments and Daemonset typically have more than one replica or desired replica count, and although kubectl default formatting will return columns summarizing how many are desired and how many are currently ready, an automated script needs to parse these value in order to determine if full health.
Similiarly, pod status as well as the readiness of each container needs to be parsed to correctly determine full health.
In this article, I will provide simple kubectl commands piped to awk/grep to evaluate health. If you would rather use kubectl json parsed with jq, see my k8s_check_full_health.sh script on github instead.
Daemonset
Here is example output from kubectl requesting all Daemonsets. Then piped to awk+grep to identify any where the desired count does not match the ready count.
# list of daemonset
$ kubectl get daemonsets -A
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE AGE
metallb-system speaker 3 3 3 3 3 133d
ingress-nginx ingress-nginx-controller 3 3 3 3 3 131d
ingress-nginx nginx-ingress-secondary-controller 3 3 3 3 3 131d
default tiny-tools-ds 3 3 3 3 3 136d
# SHOW daemonset not fully healthy:
# pull out NAME, DESIRED and READY columns
# then show rows where the counts do NOT match
# fields: desiredNumberScheduled != numberReady
kubectl get daemonsets -A --no-headers | awk '{printf "%s,%s,%s\n",$2,$3,$5}' | grep -Pv ".*,(\d+),\1"
Deployment
Here is example output from kubectl requesting all Deployments. Then piped to awk+grep to identify any where the replica count does not match the ready count.
# list of deployments
$ kubectl get deployment -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system local-path-provisioner 1/1 1 1 136d
default golang-hello-world-web 1/1 1 1 131d
kube-system coredns 1/1 1 1 136d
metallb-system controller 1/1 1 1 133d
default golang-hello-world-web2 2/2 2 2 131d
kube-system metrics-server 1/1 1 1 136d
# SHOW deployments not fully healthy:
# pull out NAME and READY columns
# then show rows where the counts do NOT match
# fields: replicas != readyReplicas
kubectl get deployments -A --no-headers | awk '{printf "%s,%s\n",$2,$3}' | grep -Pv ".*,(\d+)/\1"
Pods
Here is example output from kubectl requesting all pods. Then piped to awk+grep to identify any where the status does not equal ‘Running’ and the ready count does not match the container count.
# list of pods
$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx ingress-nginx-admission-create-rt7kt 0/1 Completed 0 131d
ingress-nginx ingress-nginx-admission-patch-mfrkj 0/1 Completed 0 131d
default golang-hello-world-web2-74b4bbcb46-zvdw2 1/1 Running 4 (7d21h ago) 131d
kube-system local-path-provisioner-5d56847996-hvxms 1/1 Running 2 (7d21h ago) 136d
default tiny-tools-rs-r4pwf 1/1 Running 4 (7d21h ago) 125d
# SHOW pods not fully healthy:
# exclude 'Completed' pods
# then pull out NAME and READY columns
# then show rows where not 'Running' and the ready counts do NOT match
kubectl get pods -A --no-headers | grep -v Completed | awk '{printf "%s,%s,%s\n",$2,$3,$4}' | grep -Pv ".*,(\d+)/\1,Running"
REFERENCES
fabianlee github, k8s_check_full_health.sh script that uses kubectl json + jq to determine health