Unit 3
Managing Applicaton Access
Understanding Kubernetes Networking
In Kubernetes, networking happens at different levels:
- Between containers: implemented as IPC
- Between Pods: implemented by network plugins
- Between Pods and Services: implemented by Service resources
- Between external users and Services: implemented by Services, with the help of Ingress or Gateway API
Managing Incoming Traffic
- For a long time, Ingress has been the solution to manage incoming traffic.
- Recently, Ingress has gone into a "feature freeze" and will be replaced by Gateway API.
- Currently, Ingress is still in the exam objectives, this is expected to be replaced with Gateway API in the future.
- For that reason, in this lesson you'll learn about both.
Warning: Do not configure Ingress and Gateway API on the same machine!
Understanding Network Plugins
- Network plugins are required to implement network traffic between Pods
- Network plugins are provided by the Kubernetes Ecosystem
- Vanilla Kubernetes does not come with a default network plugin, and you'll have to install it while installing a cluster
- Different plugins provide different features
- Currently, the Calico plugin is commonly used because of its support for features like NetworkPolicy
- Network plugins add resources to the Kubernetes APIs using Custom Resource Definitions
kubectl get crd # List Custom Resource Definitions added by the network plugin
kubectl get ippools -A -o yaml # Show Calico IP pools (Pod IP ranges) in YAML
kubectl get pods -o wide # List Pods with their assigned IP and node
ps aux | grep service-cluster-ip-range # Find the Service ClusterIP range from the API server process
Understanding Services
- Service resources are used to provide access to Pods
- If multiple Pods are used as Service endpoint, the Service will load balance traffic to the Pods
- Different types of Service can be configured:
ClusterIP: the Service is internally exposed and is reachable only from within the clusterNodePort: the Service is exposed at each node's IP address as a port. The Service can be reached from outside the cluster at nodeip:nodeportLoadBalancer: the cloud provider offers a load balancer that routes traffic to either NodePort- or ClusterIP-based ServicesExternalName: the Service is mapped to an external name that is implemented as a DNS CNAME record
Configuring Services
- Use
kubectl exposeto expose applications through their Pods, ReplicaSet, or Deployment (recommended) - Use
kubectl create serviceas an alternative
kubectl create deploy webshop --image=nginx --replicas=3
kubectl get pods --selector app=webshop -o wide
kubectl expose deploy webshop --type=NodePort --port=80
kubectl describe svc webshop
kubectl get svc
curl nodeip:nodeport
Understanding Ingress
- Ingress is an API object that manages external access to services in a cluster
- Ingress works with external DNS to provide URL-based access to Kubernetes applications
- Ingress consists of two parts
- A physical load balancer available on the external network
- An API resource that contains rules to contact the Service resources to find out about available back-end Pods
Ingress load balancers are provided by the Kubernetes ecosystem, different load balancers are available
Ingress exposes HTTP and HTTPS routes from outside the cluster to Services within the cluster
Ingress uses the
selectorlabelin Services to connect to the Pod endpointsTraffic routing is controlled by rules defined on the Ingress resource
Ingress can be configured to do the following, according to functionality provided by the load balancer
- Give Services externally-reachable URLs
- Load balance traffic
- Terminate SSL/TLS
- Offer name based virtual hosting
helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace
kubectl get pods -n ingress-nginx
kubectl create deploy nginxsvc --image=nginx --port=80
kubectl expose deploy nginxsvc
# Demo: Installing the Nginx Ingress Controller
kubectl create ingress nginxsvc --class=nginx --rule=nginxsvc.info/*=nginxsvc:80
kubectl port-forward -n ingress-nginx svc/ingress-nginx-controller 8080:80
echo "127.0.0.1 nginxsvc.info" >> /etc/hosts
curl nginxsvc.info:8080
Configuring Ingress, managing rules
- Ingress rules catch incoming traffic that matches a specific path and optional hostname and connects that to a Service and port
- Use
kubectl create ingressto create rules - Different paths can be defined on the same host
- Different virtual hosts can be defined in the same Ingress
# Different paths on the same host
kubectl create ingress mygress --rule="/mygress=mygress:80" --rule="/yourgress=yourgress:80"
# Different virtual hosts in the same Ingress
kubectl create ingress nginxsvc --class=nginx --rule=nginxsvc.info/*=nginxsvc:80 --rule=otherserver.org/*=otherserver:80
Understanding IngressClass
- In one cluster, different Ingress controllers can be hosted, each with its own configuration
- Controllers can be included in an
IngressClass - While defining Ingress rules, the
--classoption should be used to implement the role on a specific Ingress controller- If this option is not used, a default
IngressClassmust be defined - Set
ingressclass.kubernetes.io/is-default-class: trueas an annotation on the IngressClass to make it the default
- If this option is not used, a default
- After creating the Ingress controller as described before, an IngressClass API resource has been created
- Use
kubectl get ingressclass -o yamlto investigate its content
# Demo: Configuring Ingress Rules
kubectl get deployment
kubectl get svc webshop
kubectl create ingress webshop-ingress --rule="/=webshop:80" --rule="/hello=newdep:8080"
sudo vim /etc/hosts
# 127.0.0.1 webshop.info
kubectl get ingress
kubectl describe ingress webshop-ingress
# Create the back-end deployment used by the /hello rule
kubectl create deployment newdep --image=gcr.io/google-samples/hello-app:2.0
kubectl expose deployment newdep --port=8080
kubectl describe ingress webshop-ingress
Using Port Forwarding
kubectl port-forwardcan be used to connect to applications for analyzing and troubleshooting- It forwards traffic coming in to a local port on the kubectl client machine to a port that is available in a Pod
- Using port forwarding allows you to test application access without the need to configure Services and Ingress
- Use
kubectl port-forward mypod 1234:80to forward local port 1234 to Pod port 80 - To run in the background, use Ctrl-z or start with a
&at the end of thekubectl port-forwardcommand
Gateway API
- In current Kubernetes, Ingress is in feature freeze and no longer developed.
- The replacement is Gateway API.
- Gateway API adds more advanced features to manage incoming traffic:
- Advanced traffic management
- More options that are integrated in the API resources
- Gateway API may find its way into future versions of the CKA exam.
- Gateway API uses specific API resources which are provided as CRDs:
GatewayClass: represents the physical Gateway Controller, it usessepc.controllerNameto connnect a specific Gateway Controller.Gateway: defines an instance of traffic handling infrastructure, multiple Gateways can connect to one Gateway Controller. it usesgatewayClassNameproperty to connect to the GatewayController, it also defineslistenersto specifiy which protocols should be serviced.HTTPRoute: defines how traffic is routed to one or more Services. Incoming requests are identified by thespec.hostnames,parentRefsproperty connects the HTTPRoute to a Gateway.backendRefsproperty connects the HTTPRoute to a Service.
- To work with Gateway API, a Gateway API Controller needs to be installed.
- Without this controller, there's nothing actually handling the incoming traffic!
- Different Gateway API controllers are provided by the ecosystem.
- In this class, we'll use the Nginx Gateway Fabric, which is easily installed with
helm. - Before installing the controller, you must (currently) install the custom resources!
Do not run Gateway API on a node that already is running an Ingress controller!
Using Gateway API to provide access to Applications.
- First, you'll need to make sure the required custom resources are available.
- Next, install a community Gateway API controller.
- Verify the community Gateway Controller is ready to accept incoming requests.
- Create the Kubernetes application you want to provide access to.
- Configure GatewayClass, Gateway, and HTTPRoute.
- Test by accessing the Service that exposes the community controller.
# Install CRD's (see latest version: https://docs.nginx.com/nginx-gateway-fabric/installation/installing-ngf/helm/)
kubectl kustomize https://github.com/nginxinc/nginx-gateway-fabric/config/crd/gateway-api/standard?ref=v1.5.1 | kubectl apply -f -
# Install nginx-gateway-fabric controller
helm install ngf oci://ghcr.io/nginxinc/charts/nginx-gateway-fabric --create-namespace -n nginx-gateway
# Verify
kubectl get pods,svc -n nginx-gateway
# At this point you have a GatewayController; use kubectl get gc and notice the name (nginx)
kubectl get gc
- Make the gwc Service accessible as a NodePort:
kubectl edit -n nginx-gateway svc ngf-nginx-gateway-fabricand set type to NodePort. - Also, change the NodePorts to 32080 and 32443.
- Changing the NodePorts allows you to configure an external load balancer to catch incoming traffic on port 443 and forward that to NodePort 32443.
Demo: Using Gateway API
- Create Kubernetes resources
- Open
http-routing.yamlfrom course Git repository and verify:gatewayClassName: nginxbackendRefs.name: nginxgw
- Apply the routing config, then test with port-forwarding
# Create Kubernetes resources
kubectl create deploy nginxgw --image=nginx --replicas=3
kubectl expose deploy nginxgw --port=80
# Apply the HTTPRoute config
kubectl apply -f http-routing.yaml
# First test using port-forwarding
sudo sh -c "echo 127.0.0.1 whatever.com >> /etc/hosts"
kubectl -n nginx-gateway port-forward pods/ngf-nginx-gateway-[Tab] 8080:80 8443:443
curl whatever.com:8080
Configuring Gateway API for TLS Access
- Incoming TLS traffic can be terminated at either the Gateway, or on the endpoint application.
- If TLS traffic is terminated at the Gateway, traffic between the Gateway and the endpoint application is unsecured.
- If TLS traffic is terminated by the endpoint application, the Gateway doesn't deal with certificates but just accepts incoming traffic on port 443, which will be passed through to endpoint application port 443.
Terminating TLS at the Gateway
To set up TLS termination at the Gateway, the following steps need to be completed:
- Create a self-signed certificate
- Create a Kubernetes Secret for the self-signed certificate
- Update the Gateway resource to use the Kubernetes Secret
- Add an HTTPRoute resource for the HTTPS traffic
- Configure port forwarding to the Gateway Fabric NodePort Service
# This demo assumes you have completed the demo in Lesson 9.9
# Create a self-signed certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=whatever.com"
# Create a Kubernetes TLS Secret
kubectl create secret tls gateway-tls --cert=tls.crt --key=tls.key
# Update the Gateway resource (see tls-gateway.yaml in Git repository)
# Add an https route (see https-routing.yaml in Git repository)
# Configure port forwarding to the Gateway Fabric NodePort Service
sudo apt install socat
sudo socat TCP4-LISTEN:80,fork TCP4:127.0.0.1:32080 &
sudo socat TCP4-LISTEN:443,fork TCP4:127.0.0.1:32443 &
curl -k https://whatever.com
Networking
Understanding the CNI
- The Container Network Interface (CNI) is the common interface used for networking when starting the kubelet process on a worker node.
- It is a common container network interface, not limited to Kubernetes alone.
- The CNI doesn't take care of networking, that is done by the network plugin.
CNI ensures the pluggable nature of networking and makes it easy to select between different network plugins provided by the ecosystem.
The CNI plugin configuration is in
/etc/cni/net.dSome plugins have the complete network setup in this directory
Other plugins have generic settings, and are using additional configuration
Generic CNI documentation is at https://github.com/containernetworking/cni
While starting, the kube-controller-manager process may define the Pod CIDR, using the
--cluster-cidrparameterNetwork plugins often have their own resources for further defining network properties, often written to a CIDR
In Calico, check the IPPool resource:
kubectl describe ippool
Understanding Service Auto Registration
- Kubernetes runs coredns Pods in the kube-system Namespace as internal DNS servers
- These Pods are exposed by the kubedns Service, which by default offers access on IP address
10.96.0.10 - Kubernetes Service resources register with this kubedns Service
- Pods are automatically configured with the IP address of the kubedns Service as their DNS resolver
- As a result, all Pods can access all Services by name
Accessing Service in other Namespaces
- If a Service is running in the same Namespace, it can be reached from a Pod by using its short hostname
- If a Service is running in another Namespace, an FQDN consisting of
servicename.namespace.svc.clusternamemust be used - The clustername is defined in the coredns ConfigMap Corefile definition and set to
cluster.localif it hasn't been changed, usekubectl get cm -n kube-system coredns -o yamlto verify
# access pod by name in same namespace
kubectl run webserver --image=nginx
kubectl expose pod webserver --port=80
kubectl run testpod --image=busybox -- sleep 3600
kubectl get svc
kubectl exec -it testpod -- wget webserver
# access pod in other namespaces
kubectl create ns remote
kubectl run interginx --image=nginx
kubectl run remotebox --image=busybox -n remote -- sleep 3600
kubectl expose pod interginx --port=80
kubectl exec -it remotebox -n remote -- cat /etc/resolv.conf
kubectl exec -it remotebox -n remote -- nslookup interginx
kubectl exec -it remotebox -n remote -- nslookup interginx.default.svc.cluster.local
Using NetworkPolicies to Manage Traffic Between Pods
- By default, there are no restrictions to network traffic in K8s
- Pods can always communicate, even if they're in other Namespaces
- To limit this, NetworkPolicies can be used
- NetworkPolicies need to be supported by the network plugin though
- The weave plugin does NOT support NetworkPolicies!
- If in a policy there is no match, traffic will be denied
ingressis used to allow incoming trafficegressis used to allow outgoing traffic
If no NetworkPolicy is used, all traffic is allowed
In NetworkPolicy, three different identifiers can be used
- Pods (
podSelector): note that a Pod cannot block access to itself - Namespaces (
namespaceSelector): to grant access to specific Namespaces - IP blocks (
ipBlock): notice that traffic to and from the node where a Pod is running is always allowed
- Pods (
When defining a Pod- or Namespace-based NetworkPolicy, a selector label is used to specify what traffic is allowed to and from the Pods that match the selector
NetworkPolicies do not conflict, they are additive
# exploring NetworkPolicy
kubectl apply -f nwpolicy-complete-example.yaml
kubectl expose pod nginx --port=80
kubectl exec -it busybox -- wget --spider --timeout=1 nginx
kubectl label pod busybox access=true
kubectl exec -it busybox -- wget --spider --timeout=1 nginx
Using NetworkPolicies to Manage Traffic Between Namespaces
- A NetworkPolicy is a Namespaced resource, and always applies its rules to resources within the Namespace where it is created.
- If no matching labels are specified for Pods, it applies to all Pods in that Namespace and Pods from other Namespaces are not affected:
...
spec:
podSelector:
matchLabels:
ingress:
- from:
- podSelector: {}
...
- To allow ingress traffic from Pods in any Namespace, the NetworkPolicy should include an unspecified namespaceSelector:
...
spec:
podSelector:
matchLabels:
ingress:
- from:
- namespaceSelector: {}
podSelector:
matchLabels:
access="true"
...
# demo allow network traffic between namespaces
kubectl create ns nwp-namespace
kubectl create -f nwp-lab10-1.yaml
kubectl expose pod nwp-nginx --port=80
# gives a bad address error
kubectl exec -it nwp-busybox -n nwp-namespace -- wget --spider --timeout=1 nwp-nginx
# explains that it's looking in the wrong ns
kubectl exec -it nwp-busybox -n nwp-namespace -- nslookup nwp-nginx
# is allowed (using FQDN)
kubectl exec -it nwp-busybox -n nwp-namespace -- wget --spider --timeout=1 nwp-nginx.default.svc.cluster.local
# apply a more restrictive policy
kubectl create -f nwp-lab10-2.yaml
# now the FQDN access is not allowed
kubectl exec -it nwp-busybox -n nwp-namespace -- wget --spider --timeout=1 nwp-nginx.default.svc.cluster.local
# create a busybox in the default namespace
kubectl create deployment busybox --image=busybox -- sleep 3600
# from default namespace access is allowed
kubectl exec -it busybox[Tab] -- wget --spider --timeout=1 nwp-nginx
CoreDNS
- The coredns Deployment runs in the kube-system Namespace
- Until a network plugin is available, the coredns Pods will show a status of pending
- The coredns configuration is maintained in the coredns ConfigMap
- Any changes to the configuration (such as adding an external DNS forwarder) need to be added to this ConfigMap
data:
Corefile: |
.:53 {
forward . 8.8.8.8 8.8.4.4
log
errors
}
Page Source