Week 5 - Kubelet Internals¶

5.1 Conceptual Core¶

The kubelet is the per-node agent. Its job: watch Pods bound to this node, drive the CRI to make the actual containers match. Plus: report node status, manage volumes, run health checks, evict on resource pressure.
The kubelet is also a PLEG (Pod Lifecycle Event Generator) that polls the runtime to detect actual container state changes-necessary because container exits aren't always pushed events.
Kubelet is the component most often blamed for "weird" Kubernetes behavior; understanding it is non-optional.

Read pkg/kubelet/kubelet.go. Major loops:
`syncLoop - the main reconciliation loop.
`PLEG - pod-lifecycle event generation.
`volumeManager - volume mount/unmount.
`statusManager - Pod status updates back to apiserver.
`evictionManager - resource-pressure eviction.
Static pods-Pods defined as YAML files on disk (/etc/kubernetes/manifests/). Kubelet runs them directly without an apiserver. How control-plane pods bootstrap themselves.
Pod lifecycle phases: Pending → Running → Succeeded/Failed. With container states Waiting / Running / Terminated.
Pod resource enforcement: kubelet sets cgroups based on requests/limits. With cpu-manager-policy=static, the kubelet pins exclusive CPUs to Guaranteed-class Pods. Same idea for memory-manager-policy and topology-manager-policy.
Eviction: when a node runs low on memory, disk, PID space, the kubelet evicts Pods in priority order. Soft vs hard thresholds.

SSH to a node. journalctl -u kubelet -f and trigger a Pod creation. Watch the log.
crictl ps, crictl pods, `crictl inspect - operate at the CRI layer directly.
Place a static pod manifest; observe kubelet picking it up.
Trigger a memory eviction by setting low evictionHard and oversubscribing. Read the eviction event and the kubelet's decision.

Set kubelet args: - -read-only-port=0, - -anonymous-auth=false, - -authorization-mode=Webhook, - -protect-kernel-defaults=true, - -make-iptables-util-chains=true, - -tls-min-version=VersionTLS12.

Wire kubelet metrics: kubelet_pod_start_duration_seconds, kubelet_running_pods, kubelet_volume_stats_used_bytes. Alert on slow Pod starts.