# Kubernetes By Component - Part 4

## K8s Controller Manager

This is part 4 in a series of posts related to Kubernetes. Pervious posts were about kubelet, Kubernetes API, etcd, and kubectl, and kube-scheduler. This time we will explore kube-controller-manager and its role in managing replication of pods (a.k.a. scaling).

We will pick up where part 3 left off. If you do not already have a VM setup in the state at the end of that post, you will want to do that now.

We will see how the kube-controller-manager helps simplify scaling up or down a set of pods in the cluster.

The Kubernetes kube-controller-manager watches the cluster state for differences between the current state and the desired state. It doesn’t do this for everything — that’s pretty much the definition of every controller in Kubernetes. The ones that ship with Kubernetes today are the replication controller, endpoints controller, namespace controller, and serviceaccounts controller. For this post we are only interested in the replication controller included with kube-controller-manager since that governs scaling of pods.

## Introducing Deployments

Before we get into scaling our pods, we need to learn about a new Kubernetes resource — the deployment.

A deployment wraps the pod definition with information about many things including the number of replicas — i.e. instances — of a pod. Below is an example deployment for our nginx pod. Note that what we have in the spec: block is exactly what we had in the pod definition in previous posts.

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: selector: matchLabels: app: nginx replicas: 2 template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent ports: - containerPort: 80 volumeMounts: - mountPath: /var/log/nginx name: nginx-logs - name: log-truncator image: busybox imagePullPolicy: IfNotPresent command: - /bin/sh args: [-c, 'while true; do cat /dev/null > /logdir/access.log; sleep 10; done'] volumeMounts: - mountPath: /logdir name: nginx-logs volumes: - name: nginx-logs emptyDir: {} 

Let’s pull this definition into our VM and deploy it.

 1 2 3 4 5 6 7 8 $wget https://raw.githubusercontent.com/joshuasheppard/k8s-by-component/master/part4/nginx-deployment.yaml$ ./kubectl create -f nginx-deployment.yaml deployment "nginx-deployment" created $./kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 2 0 0 0 12s$ ./kubectl get pods No resources found. 

At this point nothing will happen. Much like when we were deploying pods without explicit node assignments and without the kube-scheduler deployed. Except this time, we will need the kube-controller-manager to make use of deployment resources.

What we see above is that we want (i.e. desire) two instances of the spec: block. This is determined by the replicas: 2 line in the deployment specification. We also see “CURRENT”, “UP-TO-DATE”, and “AVAILABLE” at 0. These will stay like this until the kube-controller-manager is running to act on the deployments.

## Deploying the Contoller-Manager

In yet another terminal - we are up to five now - let’s get the kube-controller-manager in place.

 1 2 3 4 $vagrant ssh$ wget https://storage.googleapis.com/kubernetes-release/release/v1.9.7/bin/linux/amd64/kube-controller-manager $chmod +x kube-controller-manager$ sudo ./kube-controller-manager --kubeconfig=$PWD/kubeconfig  Back in our original terminal window, run the following to see what we have going on besides pods in our cluster.  1 2 3 4 5 $ ./kubectl get componentstatuses NAME STATUS MESSAGE ERROR controller-manager Healthy ok scheduler Healthy ok etcd-0 Healthy {"health":"true"} 

As a recap, etcd stores the state of the cluster and is accessed only through the kube-apiserver. kube-scheduler ensures that pods are assigned to kubelet “nodes” that have sufficient resources to run them.

Now we have added kube-controller-manager with the expectation that it helps us quickly scale up or down the instances of a pod. Let’s take another look at the current state of the cluster.

 1 2 3 4 5 6 7 $./kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 2 2 2 2 5m$ ./kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-76d8597c69-6x4bf 2/2 Running 0 56s nginx-deployment-76d8597c69-kqgd8 2/2 Running 0 56s 

We see above that the deployment has been processed now and that we have two instances of the pod described in the deployment running.

## Scaling in Action

Now let’s test out scaling down.

 1 2 3 4 5 6 7 8 9 $./kubectl scale --replicas=1 -f nginx-deployment.yaml deployment "nginx-deployment" scaled$ ./kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 1 1 1 1 6m $./kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-76d8597c69-6x4bf 2/2 Running 0 3m nginx-deployment-76d8597c69-kqgd8 2/2 Terminating 0 3m  We told the cluster that we only needed one instance in the nginx deployment and it promptly elected one of the pods to terminate so that we would only have one running per our request. When we scale up to three fast enough we can see something interesting.   1 2 3 4 5 6 7 8 9 10 11 $ ./kubectl scale --replicas=3 -f nginx-deployment.yaml deployment "nginx-deployment" scaled $./kubectl get deployments NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE nginx-deployment 3 3 3 3 6m$ ./kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-76d8597c69-6x4bf 2/2 Running 0 3m nginx-deployment-76d8597c69-gkffh 2/2 Running 0 6s nginx-deployment-76d8597c69-kqgd8 2/2 Terminating 0 3m nginx-deployment-76d8597c69-zzhcq 2/2 Running 0 6s 

Note that the pod that was terminating continues to terminate and two new pods are spun up to meet the current needs of the deployment. Once a pod begins terminating it will not be “resuscitated”.

## Cleaning Up

When we are done, we can spin down the deployment and it’s associated pods using the delete command through kubectl as we have done with pods in the past posts.

 1 2 3 4 5 6 7 8 9 $./kubectl delete -f nginx-deployment.yaml deployment "nginx-deployment" deleted$ ./kubectl get deployments No resources found. $./kubectl get pods NAME READY STATUS RESTARTS AGE nginx-deployment-76d8597c69-6x4bf 2/2 Terminating 0 12m nginx-deployment-76d8597c69-gkffh 2/2 Terminating 0 9m nginx-deployment-76d8597c69-zzhcq 2/2 Terminating 0 9m  After a minute or so all the pods related to the deployment will have finished terminating. A last cleanup step is to exit from the Vagrant box and destroy it.  1 2 $ exit \$ vagrant destroy 

## What’s Next

In a follow-up article, we will explore how newer versions of Kubernetes — v1.10+ — change how we deploy pods and start up our controllers like kube-scheduler and kube-controller-manager.