K8S storage

Managing Storage

Using Pod Volumes

• Pod Volumes are a part of the Pod specification and have the storage reference hard coded in the Pod manifest
• This is not bad, but it doesn't allow for flexible storage allocation
• Pod Volumes can be used for many storage types
• Also, the ConfigMap can mounted as a Pod Volume

Managing Persistent Volumes

• PersistentVolumes (PV) are API resources that represent specific storage, can be accessed by name through any namespaces
• PVs can be created manually, or automatically using StorageClass and storage provisioners
• PersistentVolumes need a few properties:
  • storageClassName: used as a selector label to allow PVCs to bind, or connect to a specific StorageClass
  • capacity: the PV capacity in GiB
  • accessMode: defines read/write or read-only access
  • the specific type of storage
• Pods do not connect to PVs directly, but indirectly using PersistentVolumeClaim (PVC)

# pv.yaml
kind: PersistentVolume
apiVersion: v1
metadata:
  name: pv-volume
spec:
  storageClassName: demo
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mydata"

Configuring PersistentVolumeClaim

• PVCs are independent API resources that allow Pods to connect to any type of storage that is provided at a specific site
• The purpose of the PVC is to make the application itself agnostic of the specific storage type that is provided
• This makes it flexible to pick up any storage available at a specific site.

# pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pv-claim
spec:
  storageClassName: demo
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

A Pod uses the PVC by referencing it in its volumes section:

kind: Pod
apiVersion: v1
metadata:
  name: pv-pod
spec:
  volumes:
    - name: pv-storage
      persistentVolumeClaim:
        claimName: pv-claim
  containers:
    - name: pv-container
      image: nginx
      ports:
        - containerPort: 80
          name: "http-server"
      volumeMounts:
        - mountPath: "/usr/share/nginx/html"
          name: pv-storage

persistentVolumeReclaimPolicy

• The persistentVolumeReclaimPolicy is set on a PV to determine what happens if it is no longer bound to a PVC
• Retain is the default value. It ensures the volume will be left in its current phase which is released, such that it can be manually reclaimed by an administrator
• Delete means that the PV will be deleted once it is released
• Recycle means that the PV will recycle back into the pool of unused PVs

Understanding ConfigMap

• A ConfigMap is an API resource used to store site-specific data
• A Secret is a base64 encoded ConfigMap
• ConfigMaps are used to store either environment variables, startup parameters or configuration files
• When a Configuration File is used in a ConfigMap or Secret, it is mounted as a volume to provide access to its contents
• When a ConfigMap contains environment variables, kubectl set env can be used to update a Deployment to use it

Creating ConfigMaps for files

• The --from-file parameter is used to create a ConfigMap that contains a text file
• If the file specified is a directory, all files in that directory are added to the ConfigMap:

kubectl create cm mydir --from-file=/my/directory/

Mounting ConfigMaps

• When mounted, the ConfigMap content is created in the mountpoint
• While using pod.spec.containers.volumeMounts.subPath, it is possible to only mount a part of the ConfigMap

# spec.template.spec
volumes:
- name: cmvol
  configMap:
    name: webindex
# spec.template.spec.containers
volumeMounts:
- mountPath: /usr/share/nginx/html
  name: cmvol

# Demo: Creating a ConfigMap
echo "hello world" > index.html
kubectl create cm webindex --from-file=index.html
kubectl describe cm webindex
kubectl create deploy webserver --image=nginx
kubectl edit deploy webserver

Auto-provisioning storage

Understanding StorageClass

• StorageClass is an API resource that allows storage to be automatically provisioned
• StorageClass can also be used as a property that connects PVC and PV without using an actual StorageClass resource
• Multiple StorageClass resources can co-exist in the same cluster to provide access to different types of storage
• For automatic working, one StorageClass must be set as default:

kubectl patch storageclass mysc -p '{"metadata":{"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'

Using StorageClass

• To enable automatic provisioning, StorageClass needs a backing storage provisioner
• In the PV and PVC definition, a storageClass property can be set to connect to a specific StorageClass which is useful if multiple StorageClass resources are available
• If the storageClass property is not set, the PVC will get storage from the default StorageClass
• Also, if no default StorageClass is set, the PVC will get stuck in a status of Pending

Storage Provisioners

• Storage Provisioners are external applications that are provided by the CNCF ecosystem
• Many vendors have created solutions, many are available through artifacthub.io
• These solutions can be installed using the helm utility

Using an NFS Storage Provisioner

• The Storage Provisioner works with a StorageClass to automatically provide storage.
• It runs as a Pod in the Kubernetes cluster, provided with access control configured through Roles, RoleBindings, and ServiceAccounts.

• Once operational, PersistentVolumes can be created automatically on demand.

• To create a storage provisioner, access permissions to the API are required.

• Roles and RoleBindings are created to provide these permissions.

• A ServiceAccount is created to connect the Pod to the appropriate RoleBinding.

# Demo: Configuring a Storage Provisioner
# On control: install the NFS server
sudo apt install nfs-server -y
# On other nodes: install the NFS client
sudo apt install nfs-client
# On control: create and export the shared directory
sudo mkdir /nfsexport
sudo sh -c 'echo "/nfsexport *(rw,no_root_squash)" > /etc/exports'
sudo systemctl restart nfs-server
# On other nodes: verify the export is visible
showmount -e control

Then install the NFS subdir external provisioner with helm:

# Fetch the helm binary from github.com/helm/helm/releases and install it to /usr/local/bin
# Add the helm repo
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
# Install the package (replace xx.xx.xx.yy with the NFS server IP)
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner --set nfs.server=xx.xx.xx.yy --set nfs.path=/nfsexport
# Verify the nfs-subdir-provisioner Pod is running
kubectl get pods

Demo: Creating the PVC

# Verify that currently no PVs are available
kubectl get pv
# Create a PVC
kubectl apply -f nfs-provisioner-pvc-test.yaml
# Verify the PVC is created and bound to an automatically created PV
kubectl get pvc,pv
# Then create any Pod that mounts the PVC storage, and verify data ends up in the NFS share

# nfs-provisioner-pvc-test.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-pvc-test
spec:
  storageClassName: nfs-client   # SAME NAME AS THE STORAGECLASS
  accessModes:
    - ReadWriteMany              # must be the same as PersistentVolume
  resources:
    requests:
      storage: 50Mi