Analyzing Resource Usage
1. Installing Metrics Server
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
| Common Flag | Use |
|---|
--kubelet-insecure-tls | Skip kubelet cert verify (dev only) |
--kubelet-preferred-address-types | InternalIP fallback |
2. Viewing Node Metrics
kubectl top nodes
kubectl top nodes --sort-by=memory
3. Viewing Pod Metrics
kubectl top pods -A
kubectl top pods -n prod --sort-by=cpu
4. Viewing Container Metrics
kubectl top pods --containers
kubectl top pods --containers -l app=web
5. Sorting by Resource Usage
| Flag | Effect |
|---|
--sort-by=cpu | Descending CPU |
--sort-by=memory | Descending memory |
6. Using Prometheus
helm install kps prometheus-community/kube-prometheus-stack \
-n monitoring --create-namespace
| Component | Role |
|---|
| Prometheus | TSDB + scraper |
| Alertmanager | Alert routing |
| Operator | Manages CRs (ServiceMonitor, PodMonitor, PrometheusRule) |
7. Creating Grafana Dashboards
apiVersion: v1
kind: ConfigMap
metadata:
name: cluster-overview
labels: { grafana_dashboard: "1" } # auto-discovered by sidecar
data:
dashboard.json: |
{ "title": "Cluster Overview", "panels": [...] }
8. Setting Up Alerts
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata: { name: node-alerts, namespace: monitoring }
spec:
groups:
- name: node
rules:
- alert: HighCpu
expr: avg by (node)(rate(node_cpu_seconds_total{mode!="idle"}[5m])) > 0.9
for: 10m
labels: { severity: warning }
annotations: { summary: "Node {{ $labels.node }} CPU > 90%" }
9. Using kube-state-metrics
| Metric | Detail |
|---|
kube_pod_status_phase | Pod phase counts |
kube_deployment_status_replicas | Replicas state |
kube_node_status_condition | Node conditions |
kube_pod_container_resource_requests | Requested resources |
10. Monitoring Control Plane
| Endpoint | Detail |
|---|
/metrics on kube-apiserver | API latency, request counts |
/metrics on kube-controller-manager | Reconcile work queue |
/metrics on kube-scheduler | Scheduling latency |
/metrics on etcd | WAL fsync, leader changes |