Performing Node maintenance Tasks

Using metrics server to monitor node and pod permformance

https://github.com/kubernets-sigs/metrics-server.git

# setting up metrics server
kubectl apply -f https://github.com/kubenetes-sigs/metrics-server/releases/latest/download/components.yaml
kubectl -n kube-system get pods
kubectl -n kube-system edit deployments.apps metrics-server
# In `spec.template.spec.containers.args`, use the follwing
- --kubelet-insecure-tls

kubectl -n kube-system logs metrics-server<TAB> # should show
# "Generating self-signed cert" and "Serving securely on [::]443"

kubectl top pods --all--namespaces

## Backing up the ETCD

```sh
# install `etcdctl` tool as root
sudo apt install etcd-client
# to use `etcdctl`, service API endpoint, cacert, cert, key are needed
sudo etcdctl --help; sudo ETCDCTL_API=3 etcdctl --help
ps aux | grep etcd
# test api access
sudo ETCDCTL_API=3 etcdctl --endpoints=localhost:2379 --cacert /ect/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key get / --prefix --keys-only

# stores an etcd node backend snapshot to a given file
sudo ETCDCTL_API=3 etcdctl --endpoints=localhost:2379 --cacert /ect/kubernetes/pki/etcd/ca.crt --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key snapshot save /tmp/etcdbackup.db

# verify the Etcd backup
sudo ETCDCTL_API=3 etcdctl --write-out=table snapshot status /tmp/etcdbackup.db

# restores an etcd member snapshot to an etcd directory
sudo etcd snapshot restore /tmp/etcdbackup.db --data-dir /var/lib/etcd

Performing Cluster node upgrades

View kubernetes.io/docs and search for 'cluster upgrade'

  1. Kubernetes clusters can be upgraded from one to another minor versions
  2. Skipping minor versions (1.30 to 1.32) is not supported
  3. You'll manually need to update the Kubernetes repository in /etc/apt.source.list.d/kubernetes.list to the latest version
  4. First, you'll have to upgrade kubeadm
  5. Next, you'll need to upgrade the control plane node
  6. After that, the worker nodes are upgraded

Control plane node upgrade overview

  1. upgrade kubeadm
  2. use kubeadm upgrade plan to check available versions
  3. use kubeadm upgrade apply v1.xx.y to run the upgrade
  4. use kubectl drain controlnode --ignore-daemonsets to remote all current workloads from the node
  5. upgrade and restart kubelet and kubectl
  6. use kubectl uncordon controlnode to bring back the control node

Performing cluster worker node upgrades


Page Source