Dive Into KCNA

Dive Into Cloud Native, Containers, Kubernetes and the KCNA Exam - Lab Tutorial ▾

Welcome to the Dive Into Cloud Native - Containers, Kubernetes and the KCNA Exam lab tutorial.

In this introduction section we will familiarise you with the lab environment 🚀

Lets first get you logged into the environment. Click on the terminal and enter the username of root with the password of root

If you desire, you can also use the user of guest with a password of guest (for a non root user, this is configured with sudo and the password for sudo is also guest).

Copying and Pasting to and from the Terminals will vary, depending on your system -

Windows = Copy: Ctrl + Shift + c | Paste: Ctrl + Shift + v
Linux = Copy: Ctrl + Shift + c | Paste: Ctrl + Shift + v
Mac OS X = Copy: CMD + c | Paste: CMD + v

Terminals can be customised by clicking the Settings button, you can select the Font, Font Size, Theme and Tab Name.

Click Settings to close when complete.

At any time, this Tutorial panel can be toggled by clicking the Tutorial button.

Code or command blocks are presented as follows. There are 3 different options you can use when you encounter these -

Clipboard: Copies to the Clipboard
Terminal: Send to the Terminal
Rocket: Send to the Terminal with Return

As you hover over these, you will get a friendly reminder Tooltip, if you wish you can disable this in the Settings option

Try this with the command below which will update apt (Ubuntu's Advanced Packaging Tool) and populate packages available for installation -

apt update

Next we'll install htop using apt, htop is an interactive system monitor, process viewer and process manager designed for Unix systems -

apt install -y htop

htop will be installed to a system path, we can confirm this by executing -

which htop

As /usr/bin is in our system PATH by default, we can run htop just by issuing the command htop -

htop

With htop running, now is a good time to try tweaking the terminal. Click Settings and try changing the Terminal Font, Terminal Font Size and the Terminal Theme.

You can also click the Tutorial button to toggle visibility of the tutorial.

Press CTRL-C or q to exit htop. When you're ready, click the button below to move to the next section 🚀

Installing a k3s Kubernetes Cluster ▾

Optional Step: Life can be busy sometimes so at the start of each lesson is an option like this to synchronize the environment for the topic.

This feature is designed to reset the environment, effectively clearing any steps completed in any other lessons. Additionally, it provides convenience for revisiting and redoing the same lesson if desired.

This uses Ansible to automate previous steps. If you find this useful, see my course 'Dive Into Ansible' -

Important - Windows ❖ - If you're running the lab environment directly on Windows, it is important that WSL is configured as per the lab guidance in the course. See the video 'Kubernetes Lab Setup - Windows Considerations' in the 'Kubernetes Fundamentals' section for guidance.

Without this, k3s will fail to start. You can check by running the following command. If all is as expected the output will be cgroup2fs. If it is anything else then please review that section before proceeding -

stat -fc %T /sys/fs/cgroup

k3s is a convenient lightweight distribution that can be used for setting up a multinode Kubernetes cluster. Let's start by initialising our control-plane node.

We will use the k3s installer helper script, passing in the variables for our desired version and a token value.

This token value will be used later when our individual nodes join the cluster -

curl -sfLk https://get.k3s.io | INSTALL_K3S_VERSION=v1.31.0+k3s1 K3S_TOKEN=KCNA INSTALL_K3S_EXEC="--disable traefik --kubelet-arg=eviction-hard=imagefs.available<1%,nodefs.available<1%" sh -

As part of the k3s setup process, it will create a symbolic link from the k3s binary to /usr/local/bin/kubectl, you can see this by executing the following command -

ls -altrh /usr/local/bin/kubectl

We can also see that k3s has configured, a default kubeconfig configuration file for our instance at /etc/rancher/k3s/k3s.yaml -

cat /etc/rancher/k3s/k3s.yaml

For consistency with other kubernetes distributions, we will move this to ~/.kube/config - k3s will work as expected with this file in both locations -

mkdir -p ~/.kube; mv /etc/rancher/k3s/k3s.yaml ~/.kube/config

We can now query our nodes with kubectl and see our control-plane instance -

kubectl get nodes

Let's configure a k3s node and join this new node to our cluster, for this we'll remotely execute this command via ssh from worker-1, we'll pass a reference to the control-plane and the token that we used during the setup of the control-plane -

ssh worker-1 'curl -sfLk https://get.k3s.io | INSTALL_K3S_VERSION=v1.31.0+k3s1 K3S_URL=https://control-plane:6443 K3S_TOKEN=KCNA INSTALL_K3S_EXEC="--kubelet-arg=eviction-hard=imagefs.available<1%,nodefs.available<1%" sh -'

With this complete, we should now also be able to see our node -

kubectl get nodes

Let's configure an additional node, using the same approach -

ssh worker-2 'curl -sfLk https://get.k3s.io | INSTALL_K3S_VERSION=v1.31.0+k3s1 K3S_URL=https://control-plane:6443 K3S_TOKEN=KCNA INSTALL_K3S_EXEC="--kubelet-arg=eviction-hard=imagefs.available<1%,nodefs.available<1%" sh -'

And if we again query kubectl get nodes, we should see our control-plane and two nodes -

kubectl get nodes

In the next video we'll explore the use of our newly built Kubernetes environment with Kubernetes Pods 🚀

Kubernetes Pods ▾

This video relates to the course section on 'Kubernetes Pods'. I recommend that you either view the video first or watch the video whilst working through this lab. Let's make use of the nginx webserver to run a simple pod with a single nginx container -

kubectl run nginx --image=nginx

And let's query our running pods using kubectl get pods -

kubectl get pods

As this is now running, we see the logs of the pod using the kubectl logs command -

kubectl logs pod/nginx

If we query our running pods with the -o option we will get more information including it's IP address -

kubectl get pods -o wide

Let's capture the IP address for this pod, and confirm we've captured it by echo'ing to the terminal -

NGINX_IP=$(kubectl get pods -o wide | awk '/nginx/ { print $6 }'); echo $NGINX_IP

And we can check that this IP is pingable from our control-plane -

ping -c 3 $NGINX_IP

It will also be accessible from our other nodes, we'll use SSH to test this from worker-1 -

ssh worker-1 ping -c 3 $NGINX_IP

And also check that it is pingable from worker-2 -

ssh worker-2 ping -c 3 $NGINX_IP

Show the nginx pod IP address -

echo $NGINX_IP

🧐 Using the Reverse Proxy in this lab, you can access the nginx page served from the pod. Click the + Tab option, choose Reverse Proxy. use the default value of Control Plane for the Instance. Enter the IP address as the Target and the Port as 80 and leave the Path as the default, then press Go. This page will be accessible from any of the lab instances 🚀

We can also try the kubectl port-forward functionality, go back to our terminal tab, then let's forward pod/nginx to localhost:8080 on the control-plane and 80 in the pod, when you have finished using the port-forward, you'll need to use ctrl-c to close -

kubectl port-forward pod/nginx 8080:80

🧐 With the port-forward running, you can access the page from the Reverse Proxy Tab using Control Plane ➜ localhost ➜ 8080 -- This will only work for the Control Plane as the port-forward is specific for this instance. If you were running outside of the lab, you would access this on your machine in a browser by navigating to http://localhost:8080 - when complete, navigate back to the terminal and press ctrl-c to close the port forward and press return a few times to ensure it's closed 🚀

We can also query the nginx pod by running curl, in another pod and pointing it at the pod IP address -

kubectl run -it --rm curl --image=curlimages/curl:8.4.0 --restart=Never -- http://$NGINX_IP

You'll see that this pod, automatically cleaned itself after it was run owing to the --rm option -

kubectl get pods -o wide

Let's explore an alternative approach, we'll run a pod with ubuntu that just runs with sleep infinity, we'll also pass in the variable for our NGINX_IP pod so that it's accessible as an environment variable within the pod -

kubectl run --image=ubuntu ubuntu --env="NGINX_IP=$NGINX_IP" -- sleep infinity

And check that the pod is running -

kubectl get pods -o wide

We'll now execute a shell process, in the running ubuntu pod -

kubectl exec -it ubuntu -- bash

If we take a look whilst in this shell, we'll see that sleep infinity, is process id 1, we will have our bash shell and then the ps command that we ran, is a subprocess of the bash shell -

ps -ef

As this is a ubuntu container, we can use the apt package manager to install curl. We will do an apt update, followed by an apt install -y curl -

apt update && apt install -y curl

And now we can query the nginx pod, using the NGINX_IP variable that we passed in at the start -

curl $NGINX_IP

Exit from the ubuntu container -

exit

And let's remove these pods -

kubectl delete pod/nginx pod/ubuntu --now

We can use the kubectl options of -o yaml, along with --dry-run=client, to show the yaml declaration that the cli is using, let's tee this to a file called nginx.yaml at the same time -

kubectl run nginx --image=nginx --dry-run=client -o yaml | tee nginx.yaml

Notice the restartPolicy option, if we wish to know more about a particular section of Kubernetes yaml, we can use the kubectl explain command. For example, to query this we would use pod (as per the entry Kind), then the path, spec.restartPolicy -

kubectl explain pod.spec.restartPolicy

Let's create this in kubernetes using the imperative approach of kubectl create with -f to specify the filename -

kubectl create -f nginx.yaml

Whilst this succeeded this time, subsequent attempts will fail as kubectl cannot create an entity that already exists -

kubectl create -f nginx.yaml

An alternative approach is to use the declarative approach of apply, meaning we are applying what this entity should look like. You will get a warning initially if you do this to an entity that was created -

kubectl apply -f nginx.yaml

However, subsequent attempts will succeed -

kubectl apply -f nginx.yaml

Let's capture a ubuntu pod example, as yaml also, we won't need the NGINX_IP variable -

kubectl run --image=ubuntu --dry-run=client -o yaml ubuntu sleep infinity | tee ubuntu.yaml

Then we also apply this -

kubectl apply -f ubuntu.yaml

And check our pods, we will now have both nginx and ubuntu running successfully, via yaml files -

kubectl get pods -o wide

Let's remove both of these pods -

kubectl delete pod/nginx pod/ubuntu --now

Currently we have two separate files, but a yaml file can contain multiple declarations, let's combine the two files with a --- yaml separator, we will use tee to save this to combined.yaml -

{ cat nginx.yaml; echo "---"; cat ubuntu.yaml; } | tee combined.yaml

If we apply this combined.yaml file, it will create both entities at the same time -

kubectl apply -f combined.yaml

And we can see both pods in our Kubernetes cluster -

kubectl get pods -o wide

We can also use this file to delete both entities, at the same time with the kubectl delete command -

kubectl delete -f combined.yaml --now

Modify the combined.yaml file so that we are running a single pod, with two containers, moving the ubuntu containers section, into the nginx section. The ubuntu container will be configured as a sidecar that echo's a log and exit's if a file (/tmp/crash) exists. We will also update the metadata to match. The following command will create a working example -

cat <<EOF > combined.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: mypod
  name: mypod
spec:
  containers:
  - image: nginx
    name: webserver
    resources: {}
  - image: ubuntu
    name: sidecar
    args:
    - /bin/sh
    - -c
    - while true; do echo "\$(date +'%T') - Hello from the sidecar"; sleep 5; if [ -f /tmp/crash ]; then exit 1; fi; done
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

Apply the new combined.yaml file -

kubectl apply -f combined.yaml

You'll now see that we have a single pod showing 2/2 to indicate that it is running 2 of 2 containers in the pod -

kubectl get pods -o wide

Using kubectl we can describe the pod for further information, in particular the IP address and each of the containers (named webserver and sidecar) -

kubectl describe pod/mypod

Let's capture the mypod IP address -

MYPOD_IP=$(kubectl get pods -o wide | awk '/mypod/ { print $6 }'); echo $MYPOD_IP

And check that this is accessible using a throwaway curl pod (n.b. in the video lesson I used curlimages/curl:latest, this image is found to have frequent changes and instability, therefore for the lab, I've fixed the tag to a known working version) -

kubectl run -it --rm --image=curlimages/curl:8.4.0 --restart=Never curl -- http://$MYPOD_IP

If we check the logs for the sidecar container, we will see this is working as expected, note the use of -c to specify the particular container, ctrl-c to exit -

kubectl logs pod/mypod -c sidecar

Let's crash the sidecar container, by creating the /tmp/crash file that it is looking for as an exit criteria -

kubectl exec -it mypod -c sidecar -- touch /tmp/crash

If we check our pods again, this will transition to a restart of 1, where it restarted the crashed container -

kubectl get pods -o wide

And if we check the sidecar logs, this will have restarted -

kubectl logs pod/mypod -c sidecar

If however, we wish to see the previous logs, we can do so with the -p option -

kubectl logs pod/mypod -p -c sidecar

Let's remove the pod -

kubectl delete pod/mypod --now

And remove the yaml files -

rm nginx.yaml ubuntu.yaml combined.yaml

🎯 Challenge! 🎯 Using kubectl, run a pod with -it and --rm, the name of the pod should be matrix and it should use the image spurin/cmatrix, if it runs successfully you should see a matrix style display in the terminal that can be closed with ctrl-c, solution enclosed -

kubectl run -it --rm matrix --image=spurin/cmatrix

In the next chapter there is a further study section on Kubernetes Pods (Init Containers). When you're ready click Next to move on to the next lab 🚀

Kubernetes Pods (Init Containers) ▾

This is the lab that complements the Kubernetes Pods (Init Containers) further study section. If you have not seen this, please first see this section in the course as it contains important KCNA exam information

In this lab we'll explore how to use init containers in our Pod specification!

The following example is a pod manifest that is preconfigured with a simple init container. It will quite simply echo a countdown from 120 to 0, after which the init container will complete and the main container will run. The main container will echo main container every 30 seconds with a counter iteration, until it is stopped. Create a yaml manifest called countdown-pod.yaml -

cat <<EOF > countdown-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: countdown-pod
spec:
  initContainers:
  - name: init-countdown
    image: busybox
    command: ['sh', '-c', 'for i in \$(seq 120 -1 0); do echo init-countdown: \$i; sleep 1; done']

  containers:
  - name: main-container
    image: busybox
    command: ['sh', '-c', 'while true; do count=\$((count + 1)); echo main-container: sleeping for 30 seconds - iteration \$count; sleep 30; done']
EOF

Apply this resource -

kubectl apply -f countdown-pod.yaml

If you check the pod at this stage you'll notice it has an init container reference of 0/1 (0 out of 1 init stages complete) -

kubectl get pods -o wide

Lets watch the logs from each container. We’ll watch the init container logs first and when the logs follow completes, we’ll start following the logs for the main container. We will use an until as the commands may require multiple attempts to start whilst the containers initialise -

until kubectl logs pod/countdown-pod -c init-countdown --follow --pod-running-timeout=5m; do sleep 1; done; until kubectl logs pod/countdown-pod -c main-container --follow --pod-running-timeout=5m; do sleep 1; done

Once you’ve reached the main container, press ctrl-c when you’re ready. Check the pod status for comparison, you’ll notice that the init section is now removed and the status will be Running -

kubectl get pods -o wide

We’ll remove our pod and clean-up -

kubectl delete -f countdown-pod.yaml --now

rm countdown-pod.yaml

In the next video, we'll take a look at Kubernetes Namespaces! Checkout the video and when you're ready click Next to move on to the next lab 🚀

Kubernetes Namespaces ▾

Let's review the resources currently running and the namespaces being used. As our system is predominatently running kubernetes components, these will be assigned to the kube-system namespace -

kubectl get all -A

Kubernetes is provided with a number of default namespaces, we can see these with the following -

kubectl get namespaces

And this can be shorted as follows -

kubectl get ns

Many commands in Kubernetes can be shortened and a convenient way of seeing available shortnames is by querying api-resources, you'll see for example that pods can be shortened to po, you'll also notice that this command conveniently references when a component is namespaced, for example, a pod is namespaced whilst a node is not -

kubectl api-resources | more

If we create a pod without a namespace it will run in the default namespace -

kubectl run nginx --image=nginx

If we query all available resources, we will see this pod assigned to the default namespace -

kubectl get all -A

Let's remove this pod -

kubectl delete pod/nginx --now

We'll create our own namespace called mynamespace -

kubectl create namespace mynamespace

And if we wanted to use this namespace, we could add the -n parameter, for example creating an nginx pod -

kubectl -n mynamespace run nginx --image=nginx

If we query the available pods, we wont see this as it's checking the default namespace -

kubectl get pods

However, if we add -n mynamespace, we will see this running -

kubectl -n mynamespace get pods

Running kubectl config view will show the current kubectl configuration, you will see that our default context is using the default namespace, either with a defined key set to default or no key present -

kubectl config view

If we desire, we can update our kubernetes server configuration to use a specific namespace -

kubectl config set-context --current --namespace=mynamespace

If we re-check our configuration this will have now changed -

kubectl config view

If we query the available pods, we will now see the nginx pod in the mynamespace namespace -

kubectl get pods

Let's revert our configuration back to the default namespace -

kubectl config set-context --current --namespace=default

And we will remove the mynamespace namespace which in turn, will also remove the nginx pod in the mynamespace namespace -

kubectl delete namespace/mynamespace --now

🎯 Challenge! 🎯 Using kubectl, create a namespace called myawesomenamespace -

kubectl create namespace myawesomenamespace

🎯 Challenge! 🎯 Run an nginx pod in the myawesomenamespace -

kubectl -n myawesomenamespace run nginx --image=nginx

🎯 Challenge! 🎯 Update your config to use myawesomenamespace as the default namespace -

kubectl config set-context --current --namespace=myawesomenamespace

🎯 Challenge! 🎯 Verify that the nginx pod is visible in the myawesomenamespace namespace using short naming for pods -

kubectl get po

🎯 Challenge! 🎯 Delete the namespace myawesomenamespace -

kubectl delete namespace/myawesomenamespace --now

🎯 Challenge! 🎯 Revert your configuration back to the default namespace -

kubectl config set-context --current --namespace=default

With our understanding of Kubernetes namespaces, we'll explore takes pods to the next level by looking at the next video on Kubernetes Deployments and ReplicaSets, when you're ready click Next to move on the next lab 🚀

Kubernetes Deployments and ReplicaSets ▾

Let’s create a simple deployment using nginx, as we do so we’ll tee the yaml to a file and we’ll apply it, at the same time, we’re going to revisit this yaml later -

kubectl create deployment nginx --image=nginx --dry-run=client -o yaml | tee nginx-deployment.yaml | kubectl apply -f -

If we query deployments, we will see our nginx deployment -

kubectl get deployment

We will now have a replicaSet, with a unique identifier, note the name of the identifier, make a reference to this for comparison later on -

kubectl get replicaset

If we query our pods, these will be named with the name of the deployment, replicaSet and a unique identifier -

kubectl get pods -o wide

Deployments have a history, lets check the initial rollout history -

kubectl rollout history deployment/nginx

At the moment there is only 1 section in our history, we can annote this to make use of this record -

kubectl annotate deployment/nginx kubernetes.io/change-cause="initial deployment"

And if we check again, we can see our annotation -

kubectl rollout history deployment/nginx

Currently our deployment has a single replica and it's using the nginx image, verify this using get deployment -

kubectl get deployment -o wide

Let's scale the number of replicas, we will run a watch command straight after to see this as it progresses, press ctrl-c to exit -

kubectl scale deployment/nginx --replicas=10; watch kubectl get pods -o wide

Checking our deployment again will show 10/10 replicas -

kubectl get deployment -o wide

Modify the nginx-deployment.yaml file so that it is using 12 replicas, for example -

cat <<EOF > nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 12
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}
EOF

Apply the update and watch the changes to pods in real time -

kubectl apply -f nginx-deployment.yaml; watch kubectl get pods -o wide

Check against deployments, it should now state 12/12 -

kubectl get deployments -o wide

Even though we scaled the deployment, this does not change from a history viewpoint -

kubectl rollout history deployment/nginx

If you review the yaml for the deployment, you will see additional context that has been added. Review the strategy of rollingUpdate and it's parameters -

kubectl get deployment/nginx -o yaml

Modify the nginx-deployment.yaml file so that it is using the image nginx:stable, for example -

cat <<EOF > nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 12
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:stable
        name: nginx
        resources: {}
status: {}
EOF

Let's apply these changes and watch the rollout as it happens

kubectl apply -f nginx-deployment.yaml && kubectl rollout status deployment/nginx

We will now see 12 replicas, in the deployment status -

kubectl get deployments -o wide

Changing the image would have resulted in the creation of a new replicaSet -

kubectl get replicaset

And we can see that our pods, are now using this new replicaSet -

kubectl get pods -o wide

Let’s track this change -

kubectl annotate deployment/nginx kubernetes.io/change-cause="change image to nginx:stable"

And let's check the rollout history -

kubectl rollout history deployment/nginx

We can also change a deployment image, using the CLI. Set the image to alpine and watch the rollout status -

kubectl set image deployment/nginx nginx=nginx:alpine && kubectl rollout status deployment/nginx

We will annotate this change as well -

kubectl annotate deployment/nginx kubernetes.io/change-cause="change image to nginx:alpine"

And check our rollout history again -

kubectl rollout history deployment/nginx

Set the image to nginx:perl but this time, we will watch the pod output in real time, keep an eye on the pod replicaSet name (the left most column) -

kubectl set image deployment/nginx nginx=nginx:perl && watch kubectl get pods -o wide

Let's annotate this change as well -

kubectl annotate deployment/nginx kubernetes.io/change-cause="change image to nginx:perl"

If you describe the deployment, you will now see references to the new and old replicaSets -

kubectl describe deployment/nginx

And you can see the replicaSets via kubectl -

kubectl get replicaset -o wide

If you review the rollout history, you'll see a correlation between the age and the revisions -

kubectl rollout history deployment/nginx

Let's see what happens, if a deployment fails, we will use a tag that doesn't exist ... 🍌 ... Owing to the RollingUpdate Strategy, we can lose a percentage of our containers before it interveens as a safeguard, press ctrl-c to exit -

kubectl set image deployment/nginx nginx=nginx:bananas && watch kubectl get pods -o wide

We'll annotate that this failed -

kubectl annotate deployment/nginx kubernetes.io/change-cause="change image to nginx:bananas - failed"

Review the replicaSets at this point as the deployment is using a combination of the failing replicaSet and the previous replicaSet -

kubectl describe deployment/nginx

We will rollback this change, to get us back to a working version -

kubectl rollout undo deployment/nginx && kubectl rollout status deployment/nginx

Note that revision 4, has now become revision 6-

kubectl rollout history deployment/nginx

If we review our replicaSets, you'll see that our pods, are now running under the previous replicaSet -

kubectl get replicaset -o wide

We can also implicity rollback to a specific revision. If we rollback to the initial deployment which is nginx alone, we’ll see that the first replicaSet, will become the replicaSet in use -

kubectl rollout undo deployment/nginx --to-revision=1 && kubectl rollout status deployment/nginx

We can now see that we're re-using our original replicaSet -

kubectl get replicaset -o wide

And that revision 1, has now become revision 7. Because of the earlier change where revision 4 became revision 5, our history is now 2,3,5,6,7 -

kubectl rollout history deployment/nginx

kubectl describe will show that it is using our original replicaSet -

kubectl describe deployment/nginx

Let's clean up the deployment and it's associated replicaSets -

kubectl delete deployment/nginx --now

🎯 Challenge! 🎯 Create a deployment called web-app using the image httpd -

kubectl create deployment web-app --image=httpd

🎯 Challenge! 🎯 Verify the deployment rollout status -

kubectl rollout status deployment/web-app

🎯 Challenge! 🎯 Scale deployment/web-app to 3 replicas -

kubectl scale deployment/web-app --replicas=3

🎯 Challenge! 🎯 Verify the number of pods for deployment/web-app -

kubectl get pods

🎯 Challenge! 🎯 Annotate the deployment/web-app with "This is my web-app deployment" -

kubectl annotate deployment/web-app kubernetes.io/change-cause="This is my web-app deployment"

🎯 Challenge! 🎯 Show the deployment/web-app history using "rollout history" -

kubectl rollout history deployment/web-app

🎯 Challenge! 🎯 Set the deployment/web-app image to an image that doesn't exist (httpd:nonexistent) -

kubectl set image deployment/web-app httpd=httpd:nonexistent

🎯 Challenge! 🎯 Check the deployment/web-app with rollout status, it should be stuck -

kubectl rollout status deployment/web-app

🎯 Challenge! 🎯 Annotate the deployment/web-app with "Switched to httpd:nonexistent - failed" -

kubectl annotate deployment/web-app kubernetes.io/change-cause="Switched to httpd:nonexistent - failed"

🎯 Challenge! 🎯 Undo the rollout using "rollout undo" and immediately watch the "rollout status" -

kubectl rollout undo deployment/web-app && kubectl rollout status deployment/web-app

🎯 Challenge! 🎯 Show the deployment/web-app history using "rollout history" -

kubectl rollout history deployment/web-app

🎯 Challenge! 🎯 Verify that the pods are running -

kubectl get pods

🎯 Challenge! 🎯 Finally, delete deployment/web-app -

kubectl delete deployment/web-app --now

YAML is an important component that you'll use in your everyday Kubernetes usage, in the next video we'll explore how to quickly capture the YAML declaration from existing commands 🚀

Kubernetes Services ▾

ClusterIP Service

Let's start by creating an nginx deployment using the spurin/nginx-debug image, we'll use 3 replicas and for now, we'll just view the yaml rather than executing, note the use of the label 'app: nginx' and a ports section -

kubectl create deployment nginx --image=spurin/nginx-debug --port=80 --replicas=3 -o yaml --dry-run=client

Let's execute and create this deployment -

kubectl create deployment nginx --image=spurin/nginx-debug --port=80 --replicas=3

We're going to expose this as a ClusterIP service, firstly, let's review the yaml declaration, notice how this has re-used the port information -

kubectl expose deployment/nginx --dry-run=client -o yaml

And let's execute this -

kubectl expose deployment/nginx

If we now check, we should see our nginx service listed as a ClusterIP -

kubectl get service

It will have created endpoints in the background and you can see these by querying endpoints with kubectl -

kubectl get endpoints

Each of these endpoints will relate to our nginx deployment pods -

kubectl get pods -o wide

If we run a describe on the service, we will also see the endpoints listed -

kubectl describe service/nginx

Let's capture our ClusterIP -

CLUSTER_IP=$(kubectl get services | grep ClusterIP | grep nginx | awk {'print $3'}); echo $CLUSTER_IP

And lets try accessing this, with curl -

curl $CLUSTER_IP

We can also access this service from another pod, let's spin up a curlimages/curl:8.4.0 pod -

kubectl run --rm -it curl --image=curlimages/curl:8.4.0 --restart=Never -- sh

And from this pod we'll use curl against the service DNS name of nginx.default.svc.cluster.local -

curl nginx.default.svc.cluster.local

Pods automatically inherit a DNS search path relating to the service, if we check /etc/resolv.conf, we can see this search path -

cat /etc/resolv.conf

This means that technically, we could also just curl nginx and it would work as expected -

curl nginx

Let's exit from this container -

exit

And let's delete this service -

kubectl delete service/nginx

NodePort Service

This time, we are going to re-create our service but as a NodePort -

kubectl expose deployment/nginx --type=NodePort

If we check our services, we will see this listed as a NodePort with two ports, the first being for the application and the second being the NodePort Port -

kubectl get service

Let's view the node information -

kubectl get nodes -o wide

We'll capture the required information, firstly the IP of our control-plane node -

CONTROL_PLANE_IP=$(kubectl get nodes -o wide | grep control-plane | awk {'print $6'}); echo $CONTROL_PLANE_IP

And also the designated NodePort Port -

NODEPORT_PORT=$(kubectl get services | grep NodePort | grep nginx | awk -F'[:/]' '{print $2}'); echo $NODEPORT_PORT

And using both of these, we can query the NodePort service from one of the Nodes -

curl ${CONTROL_PLANE_IP}:${NODEPORT_PORT}

Let's remove this service -

kubectl delete service/nginx

LoadBalancer Service

We'll now create a LoadBalancer service using the built in k3s LoadBalancer by exposing a service of type LoadBalancer -

kubectl expose deployment/nginx --type=LoadBalancer --port 8080 --target-port 80

If we check services, we can see this listed with IP addresses (repeat command until this transitions from pending) -

kubectl get service

❗ If you're using Google Cloud Shell for the lab environment, you may find that the service briefly shows IP addresses but then, reverts back to pending. Further investigation with 'kubectl get all -A' may also show the LoadBalancer pods with an error. The issue relates to an incompatibility on the Google Cloud Shell setup with nf_tables and more recent changes in k3s to support nf_tables. If you encounter this, reverting the internal image used by the LoadBalancer to an earlier version that uses iptables legacy instead will resolve this, use the following script only if you encounter this issue -

/resources/patch_klipper_iptables_legacy.sh

Let's capture the first IP address shared by the LoadBalancer -

LOADBALANCER_IP=$(kubectl get service | grep LoadBalancer | grep nginx | awk '{split($0,a," "); split(a[4],b,","); print b[1]}'); echo $LOADBALANCER_IP

And we will also capture the LoadBalancer Port -

LOADBALANCER_PORT=$(kubectl get service | grep LoadBalancer | grep nginx | awk -F'[:/]' '{print $2}'); echo $LOADBALANCER_PORT

And let's watch the curl connections to the LoadBalancer, press ctrl-c to exit -

watch --differences "curl ${LOADBALANCER_IP}:${LOADBALANCER_PORT} 2>/dev/null"

Let's scale down our deployment to 1 replica and watch connections, press ctrl-c to exit -

kubectl scale deployment/nginx --replicas=1; watch --differences "curl ${LOADBALANCER_IP}:${LOADBALANCER_PORT} 2>/dev/null"

Let's scale up our deployment to 3 replicas and watch connections, press ctrl-c to exit -

kubectl scale deployment/nginx --replicas=3; watch --differences "curl ${LOADBALANCER_IP}:${LOADBALANCER_PORT} 2>/dev/null"

And now, we'll remove our deployment and our service

kubectl delete deployment/nginx service/nginx

ExternalName Service

We're now going to look at the ExternalName service, firstly create a deployment called nginx-red with an image of spurin/nginx-red -

kubectl create deployment nginx-red --image=spurin/nginx-red --port=80

And create the equivalent with nginx-blue -

kubectl create deployment nginx-blue --image=spurin/nginx-blue --port=80

If we check our Deployments, we will see both of these listed -

kubectl get deployment

Let's expose deployment nginx-red -

kubectl expose deployment/nginx-red

And also expose deployment nginx-blue -

kubectl expose deployment/nginx-blue

And if we issue a get service, we can see both of these -

kubectl get service

Lets create an ExternalName service of my-service that points to nginx-red -

kubectl create service externalname my-service --external-name nginx-red.default.svc.cluster.local

Checking our services shows this additional ExternalName service -

kubectl get service

Let's take a look at this from the viewpoint of a pod, we'll execute a curl pod -

kubectl run --rm -it curl --image=curlimages/curl:8.4.0 --restart=Never -- sh

And from within the pod, let's query nginx-red -

curl nginx-red

And then, nginx-blue -

curl nginx-blue

And also our my-service which is currently pointing at nginx-red -

curl my-service

If we take a look with nslookup, we can see that my-service is a CNAME for nginx-red.default.svc.cluster.local -

nslookup my-service

Using the UI interface, bring up a new tab on the Control-Plane, log in, and within this tab, edit the my-service manifest (this will default to using vim), change red to blue and save the file and exit -

kubectl edit service/my-service

Go back to the original tab and once again issue an nslookup, it should now reference nginx-blue -

nslookup my-service

And if we curl my-service, we can see the blue html code -

curl my-service

Let's exit from this curl pod -

exit

And we will clean up our deployments and services -

kubectl delete deployment/nginx-blue deployment/nginx-red service/nginx-blue service/nginx-red service/my-service

Headless Service

We will now take a look at Headless services which are a variation of ClusterIP, let's create a deployment using spurin/nginx with 3 replicas and a specified port of 80 -

kubectl create deployment nginx --image=spurin/nginx-debug --replicas=3 --port=80

To create a headless service, we need to modify a ClusterIP Yaml Declaration, capture the yaml from a basic ClusterIP Deployment and save to headless.yaml -

kubectl expose deployment/nginx --dry-run=client -o yaml --type=ClusterIP | tee headless.yaml

We then need to add an entry of 'clusterIP: None' under spec to this file, for convenience, this command will update the file as expected -

grep -q 'clusterIP: None' headless.yaml || sed -i '/spec:/a\ \ clusterIP: None' headless.yaml; cat headless.yaml

And we will apply this file -

kubectl apply -f headless.yaml

And can now see this as a service -

kubectl get service

Let's use our convenient curl pod to check this -

kubectl run --rm -it curl --image=curlimages/curl:8.4.0 --restart=Never -- sh

And with this setup, if we watch the output of nslookup nginx, we will see this dynamically changing -

watch nslookup nginx

And if we curl this (repeat the command multiple times) we will see this accessing different pods -

curl nginx

Let's exit -

exit

And finally, we will cleanup headless.yaml, our deployment and our service -

rm headless.yaml; kubectl delete deployment/nginx service/nginx

In the next video, we'll take a look at Kubernetes Jobs 🚀

Kubernetes Jobs ▾

Jobs

Create a job to calculate pi using the perl image -

kubectl create job calculatepi --image=perl:5.34.0 -- "perl" "-Mbignum=bpi" "-wle" "print bpi(2000)"

And watch the job completion, can take up to 60 seconds, press ctrl-c to exit -

watch kubectl get jobs

If we run a describe against this job, we can see that this completed -

kubectl describe job/calculatepi

Show the pods -

kubectl get pods -o wide

Let's capture the pod name -

PI_POD=$(kubectl get pods | grep calculatepi | awk {'print $1'}); echo $PI_POD

And lets view the logs from the pod, showing pi to 2000 places -

kubectl logs $PI_POD

Let's delete this job -

kubectl delete job/calculatepi

We'll capture the yaml from our pi job to a file called calculatepi -

kubectl create job calculatepi --image=perl:5.34.0 --dry-run=client -o yaml -- "perl" "-Mbignum=bpi" "-wle" "print bpi(2000)" | tee calculatepi.yaml

We're going to update this to include completions and parallelism, review these options using kubectl explain -

kubectl explain job.spec | more

Update the calculatepi.yaml to include completions of 20 and parallelism of 5 -

cat <<EOF > calculatepi.yaml
apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  name: calculatepi
spec:
  completions: 20
  parallelism: 5
  template:
    metadata:
      creationTimestamp: null
    spec:
      containers:
      - command:
        - perl
        - -Mbignum=bpi
        - -wle
        - print bpi(2000)
        image: perl:5.34.0
        name: calculatepi
        resources: {}
      restartPolicy: Never
status: {}
EOF

Apply this file, we'll add a small sleep and will watch the output of kubectl get pods, this will run until we have 20 pods with up to 5 pods at any time -

kubectl apply -f calculatepi.yaml && sleep 1 && watch kubectl get pods -o wide

We could from any of these pods, get the logs and see pi calculated -

PI_POD=$(kubectl get pods | grep calculatepi | tail -1 | awk {'print $1'}); kubectl logs $PI_POD

Remove the job, this will also delete all of the associated pods -

kubectl delete job/calculatepi

And if we check, all of these pods have been removed -

kubectl get pods -o wide

Let's cleanup, the yaml file -

rm calculatepi.yaml

CronJobs

We'll re-use the previous example but we will repurpose it as a CronJob with a schedule of * * * * * -

kubectl create cronjob calculatepi --image=perl:5.34.0 --schedule="* * * * *" -- "perl" "-Mbignum=bpi" "-wle" "print bpi(2000)"

If we watch this, as the minute ticks over it should start, you may need to toggle the tutorial button to close this pane to see this, press ctrl-c to exit -

watch kubectl get jobs

Regardless of how many jobs are completed, we will only have 3 running in total, it may briefly step up to 4 but then it will go back to 3.

This is because of the successfulJobsHistoryLimit parameter. You can see this by running an edit on the job, whilst here, add in completions:20 and parallelism: 5 under spec.jobTemplate.spec, save the file and exit when done -

kubectl edit cronjob/calculatepi

Watch the jobs again, press ctrl-c to exit -

watch kubectl get jobs

Let's cleanup the CronJob, this will delete associated jobs and pods -

kubectl delete cronjob/calculatepi

We can confirm all jobs are removed -

kubectl get jobs

And all pods -

kubectl get pods

In the next video, we'll take a look at Kubernetes ConfigMaps 🚀

Kubernetes ConfigMaps ▾

Create a ConfigMap with two entries using the --from-literal parameter -

kubectl create configmap colour-configmap --from-literal=COLOUR=red --from-literal=KEY=value

Describe the ConfigMap to show the available values -

kubectl describe configmap/colour-configmap

And lets remove this ConfigMap -

kubectl delete configmap/colour-configmap

We'll use an alternative approach for creating the same ConfigMap, we'll save our values to a file -

cat <<EOF > configmap-colour.properties
COLOUR=green
KEY=value
EOF

Check our file -

cat configmap-colour.properties

And create the ConfigMap using the --from-env-file parameter -

kubectl create configmap colour-configmap --from-env-file=configmap-colour.properties

If we run our describe again, it will be as per the previous attempt -

kubectl describe configmap/colour-configmap

Let's capture the yaml for a pod that dumps environment variables, and then sleeps for infinity -

kubectl run --image=ubuntu --dry-run=client --restart=Never -o yaml ubuntu --command bash -- -c 'env; sleep infinity' | tee env-dump-pod.yaml

We'll apply this pod -

kubectl apply -f env-dump-pod.yaml

And can see it in our get pods output -

kubectl get pods

And if we view the logs for this, we'll see the default environment variables including those added by Kubernetes -

kubectl logs ubuntu

Update our pod yaml so that it includes an envFrom, configMapRef -

cat <<EOF > env-dump-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  containers:
  - command:
    - bash
    - -c
    - env; sleep infinity
    image: ubuntu
    name: ubuntu
    resources: {}
    envFrom:
    - configMapRef:
        name: colour-configmap
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}
EOF

If we attempt to apply this at present it will fail as we can only update certain fields -

kubectl apply -f env-dump-pod.yaml

We will need to delete and recreate the instance -

kubectl delete -f env-dump-pod.yaml --now; kubectl apply -f env-dump-pod.yaml

If we check the logs again, we will see both of our values as environment variables -

kubectl logs ubuntu

ConfigMaps provide a centralised location for configuration data on our cluster, edit the current configmap and change the colour to red -

kubectl edit configmap/colour-configmap

Cycle the pod again -

kubectl delete -f env-dump-pod.yaml --now && kubectl apply -f env-dump-pod.yaml

And check the logs, the output will now show red -

kubectl logs ubuntu

Let's make our ConfigMap immutable. Update the configmap and add 'immutable: true' without any indentation in the file on a new line, also change the colour to purple -

kubectl edit configmap/colour-configmap

Recycle the pod again -

kubectl delete -f env-dump-pod.yaml --now && kubectl apply -f env-dump-pod.yaml

And check the logs to confirm that it shows purple -

kubectl logs ubuntu

If you try to change the configmap again, it will fail as it is now in an immutable state, change the colour and try to save it -

kubectl edit configmap/colour-configmap

We'll clean up the resources -

kubectl delete pod/ubuntu configmap/colour-configmap --now

And remove our files -

rm configmap-colour.properties env-dump-pod.yaml

In the next video, we'll take a look at Kubernetes Secrets 🚀

Kubernetes Secrets ▾

Create a secret using the --from-literal approach which is identical in usage to that of ConfigMaps, send the output to yaml as a dry-run and note the encoded values for COLOUR and KEY -

kubectl create secret generic colour-secret --from-literal=COLOUR=red --from-literal=KEY=value --dry-run=client -o yaml

We can verify that this is a base64 encoded value by passing the same value and encoding it ourselves with base64 manually -

echo -n value | base64

And we can also decode this by reversing the base64 encoding -

echo dmFsdWU= | base64 -d

Let's run our previous example and create the secret -

kubectl create secret generic colour-secret --from-literal=COLOUR=red --from-literal=KEY=value

List available secrets -

kubectl get secrets

And show the yaml output for the secret, we will see the encoded secrets within -

kubectl get secret/colour-secret -o yaml

We'll re-use our ubuntu pod that sleeps for infinity and dumps environment variables, this time we will use envFrom with secretRef, the following is a complete yaml example -

cat <<EOF > env-dump-pod.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  containers:
  - command:
    - bash
    - -c
    - env; sleep infinity
    image: ubuntu
    name: ubuntu
    resources: {}
    envFrom:
    - secretRef:
        name: colour-secret
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}
EOF

We'll apply this and create our ubuntu pod -

kubectl apply -f env-dump-pod.yaml

And if we view the logs for this, we can see our secrets listed accordingly in a similar way to the use of ConfigMaps -

kubectl logs ubuntu

Let's delete these resources -

kubectl delete pod/ubuntu secret/colour-secret --now

And remove our file -

rm env-dump-pod.yaml

In the next video, we'll take a look at Kubernetes Labels 🚀

Kubernetes Labels ▾

Show the yaml output for an nginx pod, note the default label of 'run:nginx' -

kubectl run nginx --image nginx --port 80 -o yaml --dry-run=client

And let's run this -

kubectl run nginx --image nginx --port 80

Review the yaml for exposing this pod, note that the selector references 'run:nginx' which is how it matches our pod -

kubectl expose pod/nginx --dry-run=client -o yaml

We'll also run this -

kubectl expose pod/nginx

With these labels set, we could query these using a selector parameter -

kubectl get all --selector run=nginx

Let's try out labels ourselves, we will create three pods, each with a specific colour as a label. The following yaml will be used as a template to create these (longer pastes like this may not appear to work as expected but, if you review the file they will be okay) -

cat <<EOF > coloured_pods.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
    colour: red
  name: ubuntu-red
spec:
  containers:
  - command:
    - sleep
    - infinity
    image: ubuntu
    name: ubuntu
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
    colour: green
  name: ubuntu-green
spec:
  containers:
  - command:
    - sleep
    - infinity
    image: ubuntu
    name: ubuntu
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
    colour: pink
  name: ubuntu-pink
spec:
  containers:
  - command:
    - sleep
    - infinity
    image: ubuntu
    name: ubuntu
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

We'll apply this and create our pods -

kubectl apply -f coloured_pods.yaml

If we check we will have our coloured pods in the output -

kubectl get pods -o wide

And now if we desire, we can specifically select resources based on that selector -

kubectl get all --selector colour=green

We can also use the shorthand method of -l which is a synonym for --selector -

kubectl get all -l colour=green

Let's cleanup after ourselves -

kubectl delete pod/nginx service/nginx pod/ubuntu-red pod/ubuntu-green pod/ubuntu-pink --now

And remove our yaml file -

rm coloured_pods.yaml

In the next video, we'll take a look at the Kubernetes API 🚀

Kubernetes API ▾

Query the kubernetes nodes -

kubectl get nodes

We can see the API requests that are being made by adding verbosity flags -

kubectl get nodes --v=6

The API resources and version can be referenced with api-resources -

kubectl api-resources | more

Re-review the previous get nodes command and note the https call that is being made -

kubectl get nodes --v=6

If we try the equivalent call ourselves it won't work as it is missing authentication information supplied by our .kube/config file -

curl --insecure https://127.0.0.1:6443/api/v1/nodes?limit=500

Let's run a kubectl proxy to interact with the API in the background, we'll store the pid information to a file to assist with cleaning this up later, press return after executing -

kubectl proxy & echo $! > /var/run/kubectl-proxy.pid

We adjust our previous request and this should now work via the proxy -

curl -s http://127.0.0.1:8001/api/v1/nodes?limit=500 | more

With our proxy running, we can use this to fetch an OpenAPI specification in v2 -

curl localhost:8001/openapi/v2

And in v3, if you desired you could redirect these to a file for use elsewhere -

curl localhost:8001/openapi/v3

Let's create a pod via the API, the process for generating this is explained in the video -

curl --location 'http://localhost:8001/api/v1/namespaces/default/pods?pretty=true' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
    "kind": "Pod",
    "apiVersion": "v1",
    "metadata": {
        "name": "nginx",
        "creationTimestamp": null,
        "labels": {
            "run": "nginx"
        }
    },
    "spec": {
        "containers": [
            {
                "name": "nginx",
                "image": "nginx",
                "resources": {}
            }
        ],
        "restartPolicy": "Always",
        "dnsPolicy": "ClusterFirst"
    },
    "status": {}
}'

If we check, we can now see our pod that was created via the API -

kubectl get pods

We will also delete a pod, via the api, again the process for generating this is outlined in the video -

curl --location --request DELETE 'http://localhost:8001/api/v1/namespaces/default/pods/nginx' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--data '{
    "gracePeriodSeconds": 1,
    "propagationPolicy": "Background"
}'

And if we check our pods again, this will no longer exist - kubectl get pods

We'll terminate the proxy

kill $(cat /var/run/kubectl-proxy.pid)

And remove our pid file -

rm /var/run/kubectl-proxy.pid

In the next video, we'll take a look at Kubernetes RBAC 🚀

Kubernetes RBAC ▾

Default KubeConfig

Let's start by taking a look at our current kubeconfig -

kubectl config view

To see the information that is currently hidden, we can add the parameter of --raw -

kubectl config view --raw

Let's take a closer look at the certificate-authority-data string, firstly we'll capture this -

CERTIFICATE_AUTHORITY_DATA=$(kubectl config view --raw | grep certificate-authority-data | awk -F': ' {'print $2'}) && echo $CERTIFICATE_AUTHORITY_DATA

We'll decode this using base64 -

echo $CERTIFICATE_AUTHORITY_DATA | base64 -d

And we can see the certificate public key using openssl -

echo $CERTIFICATE_AUTHORITY_DATA | base64 -d | openssl x509 -text --noout

Review our configuration again -

kubectl config view --raw

This time we'll take a look at users, we'll capture the user private key -

CLIENT_KEY_DATA=$(kubectl config view --raw | grep client-key-data | awk -F': ' {'print $2'}) && echo $CLIENT_KEY_DATA

And again, we can decode this with base64 to see the users private key -

echo $CLIENT_KEY_DATA | base64 -d

We also have client-certificate-data which is where we enter the main realms of rbac, capture this to a variable -

CLIENT_CERTIFICATE_DATA=$(kubectl config view --raw | grep client-certificate-data | awk -F': ' {'print $2'}) && echo $CLIENT_CERTIFICATE_DATA

We'll decode this, at this point you may see one or more certificates for the client-certificate-data -

echo $CLIENT_CERTIFICATE_DATA | base64 -d

And we will decode this using openssl, note how the client-certificate-data contains a subject line with O and CN, O for a Group and CN for a username -

echo $CLIENT_CERTIFICATE_DATA | base64 -d | openssl x509 -text --noout

Cluster Role Bindings

Let's take a look at this from the view of Cluster Role Bindings in RBAC, review the Users, Groups and ServiceAccounts, note that group system:masters has a binding with cluster-admin and a role of ClusterRole/cluster-admin (you can click the Tutorial button to toggle the sidebar if you need more viewing space) -

kubectl get clusterrolebindings -o wide

If you wish to see every resource and verb that is available we can do so with kube api-resources -

kubectl api-resources --sort-by name -o wide | more

With this knowledge, let's recap the view of clusterrolebindings and how our group, relates to the admin user -

kubectl describe ClusterRole/cluster-admin

Let's build our own alternative super user group using the theme of superheroes, we'll start by creating a clusterrole and we'll assign full permission -

kubectl create clusterrole cluster-superhero --verb='*' --resource='*'

We'll bind this using a clusterrolebinding and we'll instruct it to bind to the group of cluster-superheroes -

kubectl create clusterrolebinding cluster-superhero --clusterrole=cluster-superhero --group=cluster-superheroes

If we check again with clusterrolebindings and filter our output, we'll see out new variation -

kubectl get clusterrolebindings -o wide | egrep 'Name|^cluster-'

We can use kubectl auth can-i to verify permissions, as an example, as our super user account check that we can do access any resource with any verb -

kubectl auth can-i '*' '*'

We can expand this further, for example, we could check this against our group with an assigned user of batman -

kubectl auth can-i '*' '*' --as-group="cluster-superheroes" --as="batman"

As the permissions are assigned at a group level, any user in this group will have the equivalent access, for example -

kubectl auth can-i '*' '*' --as-group="cluster-superheroes" --as="superman"

kubectl auth can-i '*' '*' --as-group="cluster-superheroes" --as="wonder-woman"

Configuring an RBAC User/Group Manually

We'll see how we can configure a user/group for use with RBAC from scratch, first we'll start by generating an rsa private key for our user batman -

openssl genrsa -out batman.key 4096

We'll now create a certificate signing request, we'll reference our private key, an output file and we'll specify the subject information with CN and O for our user and group -

openssl req -new -key batman.key -out batman.csr -subj "/CN=batman/O=cluster-superheroes" -sha256

If we check we'll have two files -

And if you wish you can review the files -

cat batman.key

cat batman.csr

We'll now structure a CSR signing request, first we'll create variables with our data -

CSR_DATA=$(base64 batman.csr | tr -d '\n')

CSR_USER=batman

And if we check these, both will now be set -

echo $CSR_DATA

echo $CSR_USER

Let's template a Kubernetes Certificate Signing Request as yaml, we'll use placeholders for our variables and will redirect to a file -

cat <<EOF > batman-csr-request.yaml
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: ${CSR_USER}
spec:
  request: ${CSR_DATA}
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
EOF

And if we check our file, we will see these entries populated -

cat batman-csr-request.yaml

We'll apply this request -

kubectl apply -f batman-csr-request.yaml

And if we check in Kubernetes we will see this certificate signing request as pending -

kubectl get certificatesigningrequest

We can use the shorthand of csr which is more commonly used in these circumstances -

kubectl get csr

As our current super user we'll approve this request -

kubectl certificate approve batman

And if we check again, we will have an approved csr -

kubectl get csr

If we query the signed certificate using json as output, we will be able to see our certificate in base64 -

kubectl get csr batman -o json

To directly capture this entry, we'll use jsonpath and will tell it to walk the path of status, then certificate -

kubectl get csr batman -o jsonpath='{.status.certificate}'

At the moment this is base64 so let's decode this -

kubectl get csr batman -o jsonpath='{.status.certificate}' | base64 -d

And we'll redirect this to a file -

kubectl get csr batman -o jsonpath='{.status.certificate}' | base64 -d > batman.crt

And now we have the following files -

ls -l

We can use openssl to decode the approved/signed certificate, showing that the subject line contains CN and O -

openssl x509 -in batman.crt -text -noout

Adding Information to a Kubeconfig file

Let's use our existing kubeconfig file as a template -

cp /root/.kube/config batman-clustersuperheroes.config

If we take a look, this currently has a lot of information that we don't want our batman user to have -

cat batman-clustersuperheroes.config

Let's clean this file, we'll use the KUBECONFIG variable and we'll unset users.default -

KUBECONFIG=batman-clustersuperheroes.config kubectl config unset users.default

We'll delete the current context -

KUBECONFIG=batman-clustersuperheroes.config kubectl config delete-context default

And we'll unset the current context -

KUBECONFIG=batman-clustersuperheroes.config kubectl config unset current-context

Our base file is now simplified -

cat batman-clustersuperheroes.config

Let's embed the new information, first set-credentials as batman, we'll pass the certificate data and the private key, we'll also embed this information -

KUBECONFIG=batman-clustersuperheroes.config kubectl config set-credentials batman --client-certificate=batman.crt --client-key=batman.key --embed-certs=true

Set the default context to use the default cluster with a user of batman -

KUBECONFIG=batman-clustersuperheroes.config kubectl config set-context default --cluster=default --user=batman

And use the context of default -

KUBECONFIG=batman-clustersuperheroes.config kubectl config use-context default

Now our configuration files looks like the following -

cat batman-clustersuperheroes.config

And we can test that this is working, we'll be able to get nodes, create a pod and delete a pod, essentially our user is a super user -

KUBECONFIG=batman-clustersuperheroes.config kubectl get nodes

KUBECONFIG=./batman-clustersuperheroes.config kubectl run nginx --image=nginx

KUBECONFIG=./batman-clustersuperheroes.config kubectl delete pod/nginx --now

Automating an RBAC Kubeconfig file

We're going to setup a convenient tool for automating kubeconfig configurations, first install prerequisites -

apt update && apt install -y git jq openssl

We'll clone the project -

git clone https://github.com/spurin/kubeconfig-creator.git

Change into the project directory -

cd kubeconfig-creator

And we'll create another user similar to batman, this time we'll create superman -

./kubeconfig_creator.sh -u superman -g cluster-superheroes

And because superman is also part of the cluster-superheroes group, we can execute commands as if we are a super user -

KUBECONFIG=./superman-clustersuperheroes.config kubectl get nodes

We'll create another user, this time we'll create wonder-woman -

./kubeconfig_creator.sh -u wonder-woman -g cluster-superheroes

And again, she will have full access -

KUBECONFIG=./wonderwoman-clustersuperheroes.config kubectl get nodes

Creating a watch only RBAC group

We'll create a ClusterRole that can only read resources with the verbs list,get,watch -

kubectl create clusterrole cluster-watcher --verb=list,get,watch --resource='*'

We'll create our binding and our cluster-viewers group at the same time -

kubectl create clusterrolebinding cluster-watcher --clusterrole=cluster-watcher --group=cluster-watchers

If we check the auth against a user called uatu in this group, we no longer have full access like before -

kubectl auth can-i '*' '*' --as-group="cluster-watchers" --as="uatu"

However if we check, we can observe -

kubectl auth can-i 'list' '*' --as-group="cluster-watchers" --as="uatu"

Let's create a KUBECONFIG file to test this for the uatu user -

./kubeconfig_creator.sh -u uatu -g cluster-watchers

If we run a get nodes, this will work -

KUBECONFIG=./uatu-clusterwatchers.config kubectl get nodes

We can also query pods -

KUBECONFIG=./uatu-clusterwatchers.config kubectl get pods

However, if we try to create something as this user it will fail -

KUBECONFIG=./uatu-clusterwatchers.config kubectl run nginx --image=nginx

Creating an RBAC managed user

All of our examples so far have been against wildcard ClusterRoles and Verbs, and against groups. Let's try this out for a standalone user, we'll create a ClusterRole as a pod manager with the verbs list,get,create,delete and the resource as pods -

kubectl create clusterrole cluster-pod-manager --verb=list,get,create,delete --resource='pods'

We'll create a ClusterRoleBinding which we'll use to bind to our ClusterRole and a user called deadpool -

kubectl create clusterrolebinding cluster-pod-manager --clusterrole=cluster-pod-manager --user=deadpool

If we check with auth can -i list * we wont be able to do so as deadpool -

kubectl auth can-i 'list' '*' --as="deadpool"

However, if we check against pods, this will be permitted -

kubectl auth can-i 'list' 'pods' --as="deadpool"

If we check the ClusterRoleBindings output, we'll see deadpool as a user in the output -

kubectl get clusterrolebindings -o wide

Let's create a KUBECONFIG file again, this time we wont use the group, just the user and observe the output, there will be no O entry as the subject line -

./kubeconfig_creator.sh -u deadpool

If we try to access pods as this user, it will work -

KUBECONFIG=./deadpool.config kubectl get pods

But if we try and access secrets, it will fail -

KUBECONFIG=./deadpool.config kubectl get secrets

Using Roles and RoleBindings

Let's switch to Roles and RoleBindings which are a namespaced resource, first we'll create a namespace called gryffindor -

kubectl create namespace gryffindor

We'll create a role, in this new namespace with full access -

kubectl -n gryffindor create role gryffindor-admin --verb='*' --resource='*'

And we'll create a RoleBinding and specify our group -

kubectl -n gryffindor create rolebinding gryffindor-admin --role=gryffindor-admin --group=gryffindor-admins

If we check our auth against the group, we'll be rejected if we don't specify a namespace, however, if we do it will work as expected -

kubectl auth can-i '*' '*' --as-group="gryffindor-admins" --as="harry"

kubectl -n gryffindor auth can-i '*' '*' --as-group="gryffindor-admins" --as="harry"

Let's create a kubeconfig file to check this as well, we'll use our convenient tool to do so and we'll set the namespace as we execute -

./kubeconfig_creator.sh -u harry -g gryffindor-admins -n gryffindor

If we check our configuration file, it has a namespace set -

cat harry-gryffindoradmins.config

If we try and access anything outside of a namespace it will fail -

KUBECONFIG=./harry-gryffindoradmins.config kubectl get nodes

If we try and access pods in the default namespace, this will also fail -

KUBECONFIG=./harry-gryffindoradmins.config kubectl -n default get pods

However, if we access resources in the gryffindor namespace, it will succeed -

KUBECONFIG=./harry-gryffindoradmins.config kubectl get pods

Cleanup

Execute the following commands, to cleanup -

cd ..

rm -rf batman* kubeconfig-creator

kubectl delete namespace/gryffindor

kubectl delete clusterrole/cluster-superhero clusterrole/cluster-watcher clusterrole/cluster-pod-manager

kubectl delete clusterrolebinding/cluster-superhero clusterrolebinding/cluster-watcher clusterrolebinding/cluster-pod-manager

Next we have further study on RBAC with Service Accounts, check the text lecture and then try out the next lab 🚀

Kubernetes RBAC (Service Accounts) ▾

This is the lab that complements the Kubernetes RBAC further study section. If you have not seen this, please first see this section in the course as it contains important KCNA exam information. This lab provides an automated step through of this optional further study.

We'll start by creating a simple pod with curlimages/curl -

kubectl run curl --image=curlimages/curl:8.4.0 -- sleep infinity

Inspect the pod, you will see Service Account at the top and it will be assigned the value of default

kubectl describe pod/curl | more

Kubectl exec into the curlimages/curl container with a sh shell

kubectl exec -it curl -- sh

We can capture the default ServiceAccount token that has been assigned to the pod by accessing the token file that is available through /var/run/secrets/kubernetes.io, all pods receive API information in this way, via the pods filesystem -

TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)

echo $TOKEN

Using this token, we can query the API using curl, for example we could query the /api/v1/pods endpoint. We will use the convenient DNS name of https://kubernetes.default.svc which is setup by default for convenience to allow pods to communicate with the API. If you attempt this at this stage, access will be restricted as there are no access permissions with the default account -

curl --header "Authorization: Bearer $TOKEN" --insecure https://kubernetes.default.svc/api/v1/pods

Exit the pod, and clean-up

exit

kubectl delete pod/curl --now

Let’s create a pod-serviceaccount -

kubectl create serviceaccount pod-serviceaccount

Create a ClusterRole or Role as per our video with the desired resources and verbs. We’ll re-use the example we used for deadpool -

kubectl create clusterrole cluster-pod-manager --verb=list,get,create,delete --resource='pods'

Create a ClusterRoleBinding/RoleBinding, again we’ll re-use the deadpool example but instead of specifying a user, we’ll specify a serviceaccount, the format for this is namespace:serviceaccount

kubectl create clusterrolebinding cluster-pod-manager --clusterrole=cluster-pod-manager --serviceaccount=default:pod-serviceaccount

This time we’ll re-create the Pod but we’ll specify our serviceaccount. Sadly it isn’t possible to specify a serviceaccount as part of the CLI but, we can inject this into the yaml on the fly as follows -

kubectl run curl --image=curlimages/curl:8.4.0 --overrides='{ "spec": { "serviceAccount": "pod-serviceaccount" } }' -- sleep infinity

If we describe the pod, this time we will see pod-serviceaccount as the service account -

kubectl describe pod/curl | more

Kubectl exec into the curlimages/curl container with an sh shell -

kubectl exec -it curl -- sh

Again, capture the default ServiceAccount token that has been assigned to the pod by accessing the token file that is available through /var/run/secrets/kubernetes.io -

TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)

echo $TOKEN

Using this token, attempt to query the /api/v1/pods endpoint as the pod-serviceaccount, this time it will be successful -

curl --header "Authorization: Bearer $TOKEN" --insecure https://kubernetes.default.svc/api/v1/pods

Exit the pod, and clean-up all of the resources -

exit

kubectl delete pod/curl serviceaccount/pod-serviceaccount clusterrole/cluster-pod-manager clusterrolebinding/cluster-pod-manager --now

In the next video, we'll take a look at Scheduling and nodeSelector 🚀

Kubernetes Scheduling and NodeSelector ▾

Let's begin by templating an nginx pod that we'll later target with a custom scheduler -

kubectl run nginx --image=nginx -o yaml --dry-run=client | tee nginx_scheduler.yaml

We are interested in a pod.spec option known as schedulerName, use kubectl explain to see more information on this -

kubectl explain pod.spec | more

Update the nginx_scheduler.yaml file to include a schedulerName for a scheduler called my-scheduler -

cat <<EOF > nginx_scheduler.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  schedulerName: my-scheduler
  containers:
  - image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

And let's apply this file -

kubectl apply -f nginx_scheduler.yaml

Our pod will be stuck in pending as there is no scheduler called my-scheduler running -

kubectl get pods -o wide

We're going to make use of a convenient code example that mimicks the functionality of a scheduler as a bash script, firstly install git and jq if required -

apt update && apt install -y git jq

And lets clone this project -

git clone https://github.com/spurin/simple-kubernetes-scheduler-example.git

And lets change to the directory and take a look at this file -

cd simple-kubernetes-scheduler-example; more my-scheduler.sh

And if you're interested in seeing how the json traversal in the script works, take a look at the json as follows -

kubectl get nodes -o json

If we run this scheduler, it should schedule our pending pod, press ctrl-c when done -

./my-scheduler.sh

And if we check our pods, this will now be scheduled -

kubectl get pods -o wide

For now, let's clean this up -

kubectl delete pod/nginx --now

nodeName

An alternative way of scheduling is by directly specifying the nodeName in which to schedule the pod, firstly go back to the previous directory -

cd ..

Review the options for nodeName under pod.spec -

kubectl explain pod.spec | more

Update the yaml so that it directly specifies a nodeName of worker-2 -

cat <<EOF > nginx_scheduler.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  nodeName: worker-2
  containers:
  - image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

And we will apply this file -

kubectl apply -f nginx_scheduler.yaml

Checking our pods again will show this scheduled to worker-2 -

kubectl get pods -o wide

For now, remove this pod -

kubectl delete pod/nginx --now

nodeSelector (Further Study)

An alternative approach is the use of nodeSelector which makes use of known labels, to select a node. If you run a describe on the worker-1 node you'll see the labels section at the top -

kubectl describe node/worker-1 | more

We will make use of the label kubernetes.io/hostname=worker-1 to target a node, update the yaml as follows -

cat <<EOF > nginx_scheduler.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  nodeSelector:
    kubernetes.io/hostname: worker-1
  containers:
  - image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

Apply this update -

kubectl apply -f nginx_scheduler.yaml

And if we check, this will have been scheduled to worker-1 -

kubectl get pods -o wide

Cleanup

Finally, we'll clean this up!

kubectl delete pod/nginx --now; rm -rf simple-kubernetes-scheduler-example; rm -rf nginx_scheduler.yaml

In the next video, we'll take a look at Kubernetes Storage 🚀

Kubernetes Storage ▾

We'll begin by capturing ubuntu pod yaml with sleep infinity as a command -

kubectl run --image=ubuntu ubuntu -o yaml --dry-run=client --command sleep infinity | tee ubuntu_emptydir.yaml

Update this specification so that it includes a volumeMount and an emptyDir volume -

cat <<EOF > ubuntu_emptydir.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  containers:
  - command:
    - sleep
    - infinity
    image: ubuntu
    name: ubuntu
    resources: {}
    volumeMounts:
    - mountPath: /cache
      name: cache-volume
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  volumes:
  - name: cache-volume
    emptyDir: {
      medium: Memory,
    }
status: {}
EOF

And let's apply this file -

kubectl apply -f ubuntu_emptydir.yaml

And lets check our pod is running -

kubectl get pods -o wide

We'll hop into this pod with an interactive shell -

kubectl exec -it ubuntu -- bash

And if we go into cache and do a df -h, we can see that the filesytem being used is tmpfs -

cd cache; df -h .

With this being a memory based filesystem, it will be highly performant. Let's test this using dd -

dd if=/dev/zero of=output oflag=sync bs=1024k count=1000

For now, let's exit the pod -

exit

When we get rid of this container it will also remove the emptyDir volume -

kubectl delete pod/ubuntu --now

Persistent Storage

Let's take a look at the storageclasses available in our cluster, we will have a single storageclass and it will be marked as default -

kubectl get storageclass

Familiarise yourself with the storageclass specication, in particular the reclaimPolicy section -

kubectl explain storageclass | more

We'll create a persistent volume using the k3s storageClass of local-path -

cat <<EOF > manual_pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: manual-pv001
spec:
  storageClassName: local-path
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/var/lib/rancher/k3s/storage/manual-pv001"
    type: DirectoryOrCreate
EOF

And we will apply this file -

kubectl apply -f manual_pv.yaml

Check the available persistent volumes and note that the reclaim policy is set to Retain -

kubectl get pv

Let's now create a manual persistent volume claim against this persistent volume -

cat <<EOF > manual_pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: manual-claim
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 10Gi
  storageClassName: local-path
  volumeName: manual-pv001
EOF

Apply this file -

kubectl apply -f manual_pvc.yaml

And check the persistent volume claim -

kubectl get persistentvolumeclaim

This can also be done the short way -

kubectl get pvc

Dynamic PVC

Let's create a dynamic version of our Persistent Volume Claim, this will be similar to our manual approach but without a volumeName entry -

cat <<EOF > dynamic_pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: dynamic-claim
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 10Gi
  storageClassName: local-path
EOF

Let's apply this -

kubectl apply -f dynamic_pvc.yaml

And if we get our pvc's you'll see that this is Pending -

kubectl get pvc

If we run a describe on the pvc, we will see that it is awaiting a consumer before it is bound -

kubectl describe pvc/dynamic-claim

And if we check our pv's, we only have the manual pv -

kubectl get pv

Let's mock up a ubuntu pod with sleep infinity as a starting base -

kubectl run --image=ubuntu ubuntu -o yaml --dry-run=client --command sleep infinity | tee ubuntu_with_volumes.yaml

And we'll modify this to include volumeMounts and volumes for both our manual and dynamic pvc's -

cat <<EOF > ubuntu_with_volumes.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  containers:
  - command:
    - sleep
    - infinity
    image: ubuntu
    name: ubuntu
    resources: {}
    volumeMounts:
    - mountPath: /manual
      name: manual-volume
    - mountPath: /dynamic
      name: dynamic-volume
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  volumes:
    - name: manual-volume
      persistentVolumeClaim:
        claimName: manual-claim
    - name: dynamic-volume
      persistentVolumeClaim:
        claimName: dynamic-claim
status: {}
EOF

As the k3s storageclass provisioner is very basic, we should use a nodeSelector to guide this pod to a specific node, check the labels for node/worker-1 -

kubectl describe node/worker-1 | more

And update the yaml to include a nodeSelector field -

cat <<EOF > ubuntu_with_volumes.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  nodeSelector:
    kubernetes.io/hostname: worker-1
  containers:
  - command:
    - sleep
    - infinity
    image: ubuntu
    name: ubuntu
    resources: {}
    volumeMounts:
    - mountPath: /manual
      name: manual-volume
    - mountPath: /dynamic
      name: dynamic-volume
  dnsPolicy: ClusterFirst
  restartPolicy: Always
  volumes:
    - name: manual-volume
      persistentVolumeClaim:
        claimName: manual-claim
    - name: dynamic-volume
      persistentVolumeClaim:
        claimName: dynamic-claim
status: {}
EOF

And we can apply this file -

kubectl apply -f ubuntu_with_volumes.yaml

Check that the pod is running -

kubectl get pod

Check pv's, A dynamic pv will have been created, notice the reclaim policies for both -

kubectl get pv

Check pvc's, they should now both be bound -

kubectl get pvc

And let's write data to our volumes, access the pod with an interactive shell -

kubectl exec -it ubuntu -- bash

And if we check both of these, they are mounted volumes -

cd /manual; df -h .; cd /dynamic; df -h .

Let's write a text file to each of these and we'll exit -

echo hello > /manual/test.txt; echo hello > /dynamic/test.txt; exit

As these are persistent volumes, if we delete the pod and recreate it -

kubectl delete pod/ubuntu --now && kubectl apply -f ubuntu_with_volumes.yaml

And again access this pod -

kubectl exec -it ubuntu -- bash

We can see that our files will still exist as our volumes are persistent -

cat /manual/test.txt; cat /dynamic/test.txt

Let's exit -

exit

Before we go, let's see what happens when we delete the pod and pvc's -

kubectl delete pod/ubuntu pvc/dynamic-claim pvc/manual-claim --now

If we check our pv's, the manual one still exists as the Reclaim policy is set to Retain -

kubectl get pv

Finally we'll clean this up and remove our files -

kubectl delete pv/manual-pv001 --now; rm -rf dynamic_pvc.yaml manual_pv* ubuntu_emptydir.yaml ubuntu_with_volumes.yaml

In the next video, we'll take a look at Kubernetes StatefulSets 🚀

Kubernetes StatefulSets ▾

There isn't a quick way of creating a StatefulSet from the CLI but a Deployment gets us very close, we'll mock up a Deployment and tee to a file -

kubectl create deployment nginx --image=nginx --replicas=3 --dry-run=client -o yaml | tee statefulset.yaml

And we'll modify this, to change it to a StatefulSet, we're also going to include a reference to serviceName -

cat <<EOF > statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  serviceName: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
EOF

And let's apply this file -

kubectl apply -f statefulset.yaml

And lets check our statefulset -

kubectl get statefulset

If we check our pods, these will have been named accordingly -

kubectl get pods -o wide

*Bonus Lab Content* - If a pod in a statefulset was removed, causing a restart, whilst the ip may change, the name will stay consistent. Delete pod/nginx-2 -

kubectl delete pod/nginx-2 --now

And now query our pods -

kubectl get pods -o wide

Let's make use of the statefulsets service functionality, we'll create a headless service called nginx -

kubectl create service clusterip --clusterip=None nginx

Check the service is visible -

kubectl get service

And if we take a look at, we can see our endpoint slices -

kubectl get endpoints

The yaml output is great for viewing endpoints -

kubectl get endpoints/nginx -o yaml

Let's check this from the viewpoint of a pod, we'll run a curlimages/curl container -

kubectl run --rm -i --tty curl --image=curlimages/curl:8.4.0 --restart=Never -- sh

And let's curl our statefulset service name -

curl nginx-1.nginx.default.svc.cluster.local

And we'll exit this container -

exit

Take a look at the statefulset yaml, you'll notice that the updateStrategy/rollingUpdate has a partition value of 0 -

kubectl get statefulset/nginx -o yaml | more

In the accompanying video we edit our existing statefulset to change the partition value to 2 but, the following command will do the equivalent -

kubectl patch statefulset/nginx -p '{"spec":{"updateStrategy":{"rollingUpdate":{"partition":2}}}}'

We will set the image to nginx:alpine and watch the rollout -

kubectl set image statefulset/nginx nginx=nginx:alpine && kubectl rollout status statefulset/nginx

Because of the partition value, the rollout only took place on nginx-2, we can verify this with the following commands (check the image) -

kubectl describe pod/nginx-1 | more

kubectl describe pod/nginx-2 | more

StatefulSet Persistent Storage

Update our statefulset to include a dynamic volume -

cat <<EOF > statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  serviceName: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        volumeMounts:
        - name: nginx
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: nginx
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "local-path"
      resources:
        requests:
          storage: 1Gi
EOF

And we will recycle our statefulset -

kubectl delete -f statefulset.yaml && kubectl apply -f statefulset.yaml

Watch our pods start, press ctrl-c to exit -

watch --differences kubectl get pods -o wide

And we can verify that we have an individual pvc and pv for each pod -

kubectl get pvc

kubectl get pv

We'll execute a shell into nginx-0 -

kubectl exec -it nginx-0 -- bash

Create some data -

cd /data; touch this_is_nginx-0; ls -l

And exit -

exit

Let's delete the pod and watch it be recreated -

kubectl delete pod/nginx-0 --now; watch --differences kubectl get pods -o wide

And if we execute a shell again -

kubectl exec -it nginx-0 -- bash

We can verify that our data still exists -

ls /data

Let's exit -

exit

And we can take this further, we can delete the entire statefulset -

kubectl delete -f statefulset.yaml

Our pvc and pv will still exist -

kubectl get pvc; kubectl get pv

And if we apply the statefulset again -

kubectl apply -f statefulset.yaml

Check our pods are running -

kubectl get pods -o wide

And exec back into the pod -

kubectl exec -it nginx-0 -- bash

And check our data, we can confirm that it is available as expected -

ls /data

Let's exit -

exit

And we'll cleanup -

kubectl delete statefulset/nginx --now; for i in 0 1 2; do kubectl delete pvc/nginx-nginx-$i --now; done; kubectl delete service/nginx; rm statefulset.yaml

In the next video, we'll take a look at Kubernetes NetworkPolicies 🚀

Kubernetes NetworkPolicies ▾

Welcome to the interactive lab on Network Policies in Kubernetes. We'll explore how Network Policies can help isolate pods and control their communication within a Kubernetes cluster. This is particularly useful in multi-tenancy environments or when you need to restrict access to certain services within your cluster -

First, let's create an nginx pod -

kubectl run nginx --image=nginx

Next, expose the nginx pod as a service with a ClusterIP. This step makes the nginx pod accessible within the Kubernetes cluster. Use the following command to expose the nginx pod -

kubectl expose pod/nginx --port=80

Now, let's test the accessibility of the nginx service. We'll run a temporary curl pod and use it to send a request to the nginx service. This demonstrates that, by default, any pod in the cluster can access the nginx service. Execute the following command to start an interactive curl session -

kubectl run --rm -i --tty curl --image=curlimages/curl:8.4.0 --restart=Never -- sh

Once in the curl pod, use curl to request the nginx page. Notice that there are no restrictions, and the nginx service responds -

curl nginx.default.svc.cluster.local

After testing, type 'exit' to leave the curl pod -

exit

Now, we will implement a Network Policy to restrict access to the nginx pod. Network Policies in Kubernetes allow you to control the flow of traffic. We will create a policy that allows access to the nginx pod only from pods with specific labels. Start by creating a network policy configuration file named 'networkpolicy.yaml' -

cat <<EOF > networkpolicy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-nginx-access
  namespace: default
spec:
  podSelector:
    matchLabels:
      run: nginx
  policyTypes:
    - Ingress
  ingress:
    - from:
      - podSelector:
          matchLabels:
            run: curl
EOF

Apply the network policy to your cluster. This will enforce the rules defined in the policy, restricting the access to the nginx pod as specified. Run the following command to apply the network policy -

kubectl apply -f networkpolicy.yaml

Test the network policy. Start another curl pod which we'll be using to accessing the nginx service again. Since this pod has the label 'run:curl', it should be allowed by the network policy. Run the following command to start an interactive session in a new curl pod -

kubectl run --rm -i --tty curl --image=curlimages/curl --restart=Never -- sh

Bring up a new control-plane tab and then do a describe on the pod, you'll see the run=curl label, when done close the tab -

kubectl describe pod/curl | more

In the curl pod, use curl to test access. The request should succeed, demonstrating the network policy's effect -

curl nginx.default.svc.cluster.local

Then exit the pod -

exit

To further validate the network policy, let's test with a pod that doesn't match the network policy criteria. Create a new curl pod with a different name, which results in a different label. This pod should not be able to access the nginx service. Run the following command to start an interactive session in the new pod -

kubectl run --rm -i --tty curl2 --image=curlimages/curl --restart=Never -- sh

Inside the curl2 pod, again, attempt to access the nginx service using curl. The request should be blocked by the network policy, illustrating how Network Policies can be used to control access in a Kubernetes cluster -

curl nginx.default.svc.cluster.local

And then exit the pod -

exit

Finally, let's clean up the environment. Execute the following commands to delete the nginx pod, the nginx service, the network policy, and the network policy configuration file -

kubectl delete pod nginx; kubectl delete service nginx; kubectl delete networkpolicy allow-nginx-access; rm -rf networkpolicy.yaml

In the next video, we'll take a look at Kubernetes Pod Disruption Budgets 🚀

Kubernetes Pod Disruption Budgets ▾

Let's create a simple deployment with multiple replicas to see how they distribute across different nodes -

kubectl create deployment nginx --image=nginx --replicas=5

With the deployment created, check how the replicas are spread across the control-plane and worker nodes -

kubectl get pods -o wide

Let's simulate a voluntary disruption. Use the kubectl cordon command to make a node unschedulable, and then delete the pods on that node. This will demonstrate how Kubernetes handles pod rescheduling when a node becomes unavailable. We use a different approach here than in the video to account for randomised pod names -

kubectl cordon control-plane && kubectl delete pods -l app=nginx --field-selector=spec.nodeName=control-plane --now

Recheck the pods to observe how they are rescheduled -

kubectl get pods -o wide

Now, repeat the cordon and delete process for the pods on worker-1 -

kubectl cordon worker-1 && kubectl delete pods -l app=nginx --field-selector=spec.nodeName=worker-1 --now

Recheck the pods again to observe how they are rescheduled, they should all be running on worker-2 -

kubectl get pods -o wide

Familiarise yourself with the kubectl drain command -

kubectl drain --help | more

Introduce a more disruptive scenario by draining worker-2 using the kubectl drain command. This simulates a scenario like node maintenance, where the node is taken offline, and observe the impact on pod availability -

kubectl drain worker-2 --delete-emptydir-data=true --ignore-daemonsets=true

Check the deployment and pod status to observe how the disruption has affected the application's availability. This will highlight the limitations of relying solely on replicas for high availability -

kubectl get deployment; echo; kubectl get pods -o wide

Uncordon all the nodes to make them available for scheduling again. This step is important to bring the cluster back to a normal state before introducing Pod Disruption Budgets -

kubectl uncordon control-plane worker-1 worker-2

Recheck the pods again -

kubectl get pods -o wide

Now, explore the creation of a Pod Disruption Budget. First, describe the existing deployment to understand its configuration and use that information to set up the PDB -

kubectl describe deployment/nginx | more

Create a Pod Disruption Budget for the nginx deployment, ensuring that a minimum of 2 pods are always available -

kubectl create pdb nginx --selector=app=nginx --min-available=2

Test the PDB by cordoning all nodes and attempting to drain them. This will demonstrate how the PDB prevents excessive disruptions that could compromise application availability, this loop will continue, once run, move on to the next step -

kubectl cordon control-plane worker-1 worker-2; kubectl drain control-plane worker-1 worker-2 --delete-emptydir-data=true --ignore-daemonsets=true

In a new control-plane root user tab, uncordon worker-1 and worker-2 and then switch back to the previous tab, this will allow Kubernetes to reschedule pods and adhere to the PDB requirements. This step shows how Kubernetes manages to maintain the minimum number of available pods specified in the PDB (this can take some time for the environment to normalise) -

kubectl uncordon worker-1 worker-2

Finally, uncordon the control-plane and clean up the environment by deleting the deployment and the Pod Disruption Budget -

kubectl uncordon control-plane; kubectl delete deployment/nginx pdb/nginx --now

In the next video, we'll take a look at Kubernetes Security 🚀

Kubernetes Security ▾

Let's start with Security Contexts which define privilege and access control settings for a Pod or Container. We will use a custom image that includes a rogue binary for demonstration purposes. This binary allows escalation from a normal user to root in a Kubernetes pod -

kubectl run ubuntu --image=spurin/rootshell:latest -o yaml --dry-run=client -- sleep infinity | tee ubuntu_secure.yaml

Let's apply this to create the pod in our Kubernetes cluster -

kubectl apply -f ubuntu_secure.yaml

With the pod running, let's execute into it and observe the current user privileges. You'll notice that we have root access -

kubectl exec -it ubuntu -- bash

Let's exit out of the pod -

exit

Modify our YAML file to include a Security Context that specifies running as a non-root user with a specific UID and GID -

cat <<EOF > ubuntu_secure.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000
  containers:
  - args:
    - sleep
    - infinity
    image: spurin/rootshell:latest
    name: ubuntu
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

Let's replace the existing pod with our updated configuration. This replacement might take a little longer -

kubectl replace --force -f ubuntu_secure.yaml

Let's exec into the pod again. This time, you'll observe that we're logged in as a non-root user -

kubectl exec -it ubuntu -- bash

If we run id, this will confirm we are a non root user -

However, if we run the /rootshell binary this will allow privilege escalation -

/rootshell

We can even run commands that require root access like apt update -

apt update

Repeat this command until we exit the container -

exit

To further secure our pod, we will add an additional Security Context to the container specification, disallowing privilege escalation -

cat <<EOF > ubuntu_secure.yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: ubuntu
  name: ubuntu
spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000
  containers:
  - args:
    - sleep
    - infinity
    image: spurin/rootshell:latest
    name: ubuntu
    resources: {}
    securityContext:
      allowPrivilegeEscalation: false
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}
EOF

Again, let's replace the pod with our new security settings -

kubectl replace --force -f ubuntu_secure.yaml

Now, exec into the pod one more time -

kubectl exec -it ubuntu -- bash

And try to escalate privileges, you can try this multiple times, the command will succeed but without root priviledge -

/rootshell

And issue exit until we fully exit the container -

exit

Finally, let's clean up by deleting the pod and removing the YAML file -

kubectl delete -f ubuntu_secure.yaml --now ; rm -rf ubuntu_secure.yaml

In the next video, we'll take a look at Helm and Helm Charts 🚀

Helm and Helm Charts ▾

Before we begin, ensure you have git installed, as it's required for Helm's plugin installation. We'll start by installing git, and at the same time we also going to install tree for a clearer view of directory structures used in Helm -

apt update && apt install -y git tree

Helm installation can be done in various ways. For convenience in this lab, we will use the get_helm.sh script. Normally, you should inspect scripts before executing them, but for this tutorial, we'll safely pipe the script to bash directly. Let's install Helm now -

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Verify that Helm is installed and available in your path by checking its version. This command will show the installed Helm version details -

helm version

Let's create a Helm chart for the Flappy Bird style application. This process generates a basic structure for our application. Run the following command to create a new Helm chart named "flappy-app" -

helm create flappy-app

Navigate into the newly created flappy-app directory to explore the Helm chart's structure. This will help us understand the components of a Helm chart -

cd flappy-app

Let's examine the directory structure of our Helm chart using the tree command. This gives us an overview of the files and directories in our chart -

tree

Now, customize the Chart.yaml file. This file contains key information about our Helm chart, such as the chart name and description. Edit the file to set the description to Flappy Dock Game Helm chart (without quotes) and leave the version as it's default -

vim Chart.yaml

Modify the values.yaml file to set the image repository to spurin/flappy-dock and use the latest tag (again without quotes for both). Also, disable the serviceAccount by setting its related boolean values to false -

vim values.yaml

Package the Helm chart for distribution. This step demonstrates Helm's versatility in chart management and version control -

helm package .

Deploy the application using the packaged Helm chart. This showcases Helm's ability to manage applications from packaged charts, which is useful for version control and distribution -

helm install flappy-app ./flappy-app-0.1.0.tgz

Helm would have provided us with some convenient commands, for ease we can run these as follows, this will also set up a port-forward to our application (you may need to retry if the app isn't running) -

export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=flappy-app,app.kubernetes.io/instance=flappy-app" -o jsonpath="{.items[0].metadata.name}"); export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}"); echo "Visit http://127.0.0.1:8080 to use your application"; kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT

Bring up a new tab to the Reverse Proxy and select the following - Control Plane ➜ 127.0.0.1 ➜ 8080 and then press Go, have some fun with the game and when complete, move back to the previous tab and press ctrl-c

Explore the deployed Kubernetes resources to understand how Helm interacts with Kubernetes. Check the deployment, pods, and services created by our Helm chart -

kubectl get deployment; echo; kubectl get pods; echo; kubectl get svc

Check whats showing from a helm viewpoint -

helm list

And then, let's uninstall the Helm chart to clean up the deployed resources -

helm uninstall flappy-app

Finally, let's clean up the environment -

cd ..; rm -rf flappy-app

Congratulations on completing the Helm lab! You have learned how to install Helm, create and customize a Helm chart, package and deploy an application, and clean up resources. These skills are valuable for managing Kubernetes workloads efficiently. The next two videos on Service Meshes and Observability are theory only, after that the next lab will be Prometheus and Grafana 🚀

Prometheus and Grafana ▾

⚠️🐋 Docker Desktop Notice 🐋⚠️

If you're running the lab under the Docker Desktop Extension, you will need to click 'Switch window.location to http' in the Settings option above for this lab exercise. This option only exists in Docker Desktop and it switches the environment from a file to http protocol to match the Grafana security requirements.

For this lab exercise, we recommend that you're making use of the Tutorial Sidepane in Fixed mode, if you haven't already, click Settings and select 'Fixed' for Tutorial Sidepane. Otherwise you may not see some of the web page options with the tutorial floating over the top.

In this interactive lab, we will dive into the powerful combination of Prometheus and Grafana for monitoring and visualizing metrics in Kubernetes. Prometheus, a CNCF Graduated project, is a monitoring and alerting toolkit widely used in the cloud-native ecosystem. It provides a multi-dimensional data model, a flexible query language (PromQL), and efficient data collection methods. Grafana complements Prometheus by offering advanced visualization, integration with various data sources, and robust alerting capabilities. Understanding these tools is essential for roles like DevOps and Site Reliability Engineering, focusing on service level agreements (SLAs), objectives (SLOs), and indicators (SLIs). Let's start by setting up our environment using Helm to install Prometheus and Grafana, we'll install git -

apt update && apt install -y git

And we'll install helm -

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Now, let's add the Prometheus community's Helm chart repository, which contains preconfigured charts for deploying Prometheus and Grafana in Kubernetes. This step simplifies the installation and management of these monitoring tools -

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

To ensure we install a stable version of Prometheus and Grafana, let's first search for the available versions of the kube-prometheus-stack in the repository. We aim to install a specific version (e.g., 55.5.0) to achieve consistent results throughout this tutorial -

helm search repo prometheus-community/kube-prometheus-stack -l

With the desired version identified, it's time to install the kube-prometheus-stack using Helm. This step will deploy Prometheus, Grafana, and related components into our Kubernetes cluster. We'll name our installation 'my-observability' and specify the version for consistency -

helm install my-observability prometheus-community/kube-prometheus-stack --version 55.5.0

After initiating the installation, take a brief pause to allow Kubernetes to set up the components and start collecting metrics. This waiting period is crucial for generating meaningful data for our later analysis. Consider taking a 10-minute break at this point

With the installation complete and metrics being collected, let's explore what has been installed. Use kubectl to list all the resources in the cluster. Pay attention to resources prefixed with 'my-observability', which include Prometheus, Grafana, and other related components -

kubectl get all -A

To access Prometheus and start exploring its capabilities, we need to locate its service endpoint. Run the following command to list the services and find the ClusterIP of 'my-observability-kube-prom-prometheus'. We will use this IP and port to access Prometheus through our reverse proxy -

kubectl get svc

Now, let's interact with Prometheus's web interface. Use the reverse proxy feature in the lab environment to access Prometheus by entering the previously found ClusterIP and port 9090. Once in the Prometheus UI, explore the Metrics Explorer and try executing queries using PromQL, like 'kube_pod_ips', to see the power of Prometheus in action, switch to the Graph view, click the stacked graph button (next to Show Exemplars) and enter the custom PromQL query of 'sum by (namespace) (kube_pod_ips)

Next, let's generate some activity in our cluster to see how Prometheus tracks changes over time. Run a simple loop in the terminal to create multiple nginx pods, which will be monitored by Prometheus. This process will take around 5 minutes -

for i in {1..10}; do kubectl run nginx-${i} --image=nginx; sleep 30; done

Returning to Prometheus, adjust the time interval to the last 5 minutes and observe how the metrics update, reflecting the newly created pods. This monitoring is a key feature of Prometheus, offering valuable insights into cluster activities

Now, let's turn our attention to Grafana for advanced visualization. First, find the service endpoint for Grafana using kubectl to get the services. Look for the ClusterIP of 'my-observability-grafana' and it's Port (80) and use this for access through the reverse proxy -

kubectl get svc

Access Grafana (see the note at the top if you're using the Docker Desktop Extension lab) via the reverse proxy using the noted ClusterIP and port 80. Log in to Grafana with the default credentials (username: admin, password: prom-operator) and explore the pre-configured dashboards linked to Prometheus. Import community dashboards like '15759' for nodes and '15757' for Kubernetes cluster views to enhance your observability experience

Before concluding this lab, it's important to clean up the environment. Use Helm to uninstall the 'my-observability' release and remove any remaining resources, including the nginx pods we created. This step ensures our lab environment is ready for future use -

helm list

helm uninstall my-observability

kubectl get all -A

kubectl -n kube-system delete service/my-observability-kube-prom-kubelet --now

for i in {1..10}; do kubectl delete pod/nginx-${i} --now; done

Congratulations on completing the Prometheus and Grafana lab! In the next video we'll be looking at GitOps with Argo! 🚀

Argo CD ▾

For this lab exercise as well, we also recommend that you're making use of the Tutorial Sidepane in Fixed mode, if you haven't already, click Settings and select 'Fixed' for Tutorial Sidepane. Otherwise you may not see some of the web page options with the tutorial floating over the top.

Welcome to the interactive lab on Cloud Native Application Delivery and GitOps. We will be exploring modern application deployment, focusing on cloud-native methodologies and the GitOps approach using Argo. Let's start by creating a namespace for ArgoCD -

kubectl create namespace argocd

Next, we'll install ArgoCD in our newly created namespace. We will use a YAML file from the ArgoCD repository to install it. This YAML file contains all the necessary Kubernetes resources for setting up ArgoCD -

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

In our lab environment, we'll simplify the setup by switching ArgoCD's operation from HTTPS to HTTP. This step is for ease of use in the lab and is not recommended for production environments. We'll patch the ArgoCD config map and then restart the ArgoCD server deployment to apply these changes -

kubectl -n argocd patch configmap argocd-cmd-params-cm --type merge -p '{"data":{"server.insecure":"true"}}'

kubectl -n argocd rollout restart deployment/argocd-server

Now, we'll verify that the ArgoCD components are running correctly. The 'watch' command allows us to actively monitor the state of our resources. We'll use it to keep an eye on everything in the argocd namespace until they are all up and running -

watch --differences kubectl get all -n argocd

Our next step involves accessing the ArgoCD server. To log in, we need the ArgoCD admin password, which is stored in a Kubernetes secret. We'll retrieve this password by accessing the secret under .data.password and will decode with base64, copy this to your clipboard -

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

With the password at hand, let's find the ClusterIP of the ArgoCD server. This IP will be used to access the ArgoCD UI through a reverse proxy. We'll run a command to list the services in the argocd namespace and note the ClusterIP for the argocd-server service -

kubectl -n argocd get svc

It's time to access the ArgoCD UI. Use the retrieved ClusterIP and the default port of 80. Then use a username of admin and the clipboard password to log into the ArgoCD web interface. Once logged in, we'll create a new application named my-argo-app. Set the Project Name as default. Set the Sync Policy to Automatic and enable both Prune Resources and Self Heal options. Under Sync Options select Auto-Create Namespace. Use a Git repository for the application source and enter https://github.com/spurin/argo-f-yourself, specify the path as . Then under Destination choose the Cluster URL (select the default option). Then set the namespace to my-argo-ns - Lastly select the Directory Recurse option before clicking create. This process demonstrates how GitOps can be used for application deployment in a cloud native environment -

If you wish, at this point you could interact with the wordpress svc and access and setup wordpress through it's ui, click the svc, select the clusterIP and navigate to this with the reverse proxy. When you're finished, use the argocd-server ClusterIP to navigate back to Argo.

Let's simulate a failure scenario to understand the self-healing capabilities of GitOps with ArgoCD. We'll delete the 'my-argo-ns' namespace, which contains our deployed application. Watch how ArgoCD detects this discrepancy between the desired state in Git and the actual state in the cluster and reconciles it by redeploying the application -

kubectl delete namespace/my-argo-ns --now

Finally, lets clean up our environment. We'll delete the namespaces argocd and my-argo-ns. This step ensures that all resources we created, including applications and ArgoCD components, are removed -

kubectl delete namespace/argocd namespace/my-argo-ns --now

Congratulations! You've completed all of the labs. Now it's time to focus your study with the exam quiz simulators and if you haven't already, book your KCNA examination! Best of luck and thank you - James Spurin! 🚀