A heart rate dynamic admission controller

This post will explore extending the Kubernetes API Admission Controllers by using Webhooks and my own heart rate.

The Kubernetes API has a core concept of an “Admission Controller”, these controllers are compiled into the kube-apiserver binary and intercepts API requests before they are persisted to the Kubernetes storage back-end (etcd).

As these admission controllers are compiled into a binary, it restricts changes to a cluster administrator, which can enable the pre-compiled options by using the —enable-admission-plugins or by modifying the kube-apiserver binary to patch in the functionality that is needed, a scary thought.

There are other ways though, as of Kubernetes 1.8, the concept of Dynamic Admission Controllers was introduced with the alpha API addition of Admission Controller Webhooks, these graduated to beta in 1.9

Validating admission webhooks

ValidatingAdmissionWebhook and the MutatingAdmissionWebhook have moved into beta and are enabled by default in Kubernetes 1.11.

The beauty of these webhooks is that they can be dynamically configured after the start of the api-server, this enables anyone with the correct Role-based access control (RBAC) to extend the concept of Admission Control, and I am going to show you how.

I have been using a heart rate monitor for the last few months whilst cycling to work and I have been enveloped with capturing the data and using it in as many ways as possible.

Which gave me the idea to use the data to gate when I can, and cannot create pods / deployments in my own personal Kubernetes cluster.

After reading about a specific type of Admission Webhook — “ValidatingAdmissionWebhook”, I decided to give the it a whirl and see if I could come up with an implementation.

I am going to be running this example in minikube but any cluster should be suitable if you follow along. There are a number of pre-requisites that need to be fulfilled before continuing, and that is the following:

• Ensuring that you have a Kubernetes cluster version greater than 1.11 (for this blogpost)
• Making sure that the kube-apiserver flag—enable-admission-plugins contains ValidatingAdmissionWebhook.
• And that the admissionregistration.k8s.io/v1beta1 is enabled by: kubectl api-versions | grep admissionregistration.k8s.io/v1beta1

Once the above are all in place we can begin.

To get familiar with the yaml configuration and to ensure that all Pod creation requests are blocked until we implement our Webhook server, we will apply the following ValidatingWebhookConfiguration:

The documentation for this is a bit sparse and difficult to read, however, if you can wrap your head around the Kubernetes API documentation you can take a peek here: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#validatingwebhookconfiguration-v1beta1-admissionregistration-k8s-io, learn by example with a fully functioning kubesec admission controller or have a look at the finished source for this article.

Once we have applied this to our cluster using the magic of kubectl apply -f our.yaml we can create a pod and watch our pod fail to create due to the unavailability of our webhook service, now we are able to build our service.

I feel most comfortable programming using Node.js and decided to implement the Bluetooth heart rate API’s using Node.js, sadly it only works on Node 8 and requires sudo due to it interfacing with the Bluetooth network stack. By using a Node.js module named noble and reading the specification and with a cheeky MIT licensed copy pasta of the bit bashing we had a working server which responds with the following on success:

And on error responds with:

With this now working for requests our last step will be to run the server, and to ensure that the Kubernetes API can connect to it, as I am running the Webhook application on my local machine and Kubernetes in minikube, which is running in virtualbox they share a network adaptor and a network address, which allows minikube to connect to my local machine on 192.168.99.1 , checked by running ip addr and looking for the vboxnet adaptors IP address.

I then proceed to spin up the server with a heart rate limit set to 200 by running: node server.js 200

As we can see, in the video, when looking at the events of the pod creation we see an error message of Error create: admission webhook “deny-heartrate.yaml” — Heart rate: 90 is too low (Lower than 200) .

Great, we see that a print out of allowed: false was also outputted from the server, proving that we denied a creation of a pod based on my heart rate!

The last step is to see if it admits the creation of pods when the bpm limit is set much lower.

Excellent!

You should now have enough information to start implementing a more useful Dynamic Kubernetes Admission controllers, such as https://github.com/stefanprodan/kubesec-webhook and https://github.com/lukebond/grafeas-image-signing-webhook and include them in your production based workloads.