Single Kubernetes Cluster Installation

Adding Linux Master Node, Linux Worker Node and Windows Worker Node

Hi everyone,

In this article, we will install Single Kubernetes on Ubuntu 20.04. We will also learn how to add a master node and a Windows worker node to Ubuntu.

Firstly, Let’s look at what we use Docker and Kubernetes for.

Docker: Docker is an open source containerization platform. It enables developers to package applications into containers-standardized executable components combining application source code with the operating system (OS) libraries and dependencies required to run that code in any environment. Containers simplify delivery of distributed applications, and have become increasingly popular as organizations shift to cloud-native development and hybrid multi cloud environments.

Kubernetes: Kubernetes, often abbreviated as “K8S”, orchestrates containerized applications to run on a cluster of hosts. The K8s system automates the deployment and management of cloud native applications using on-premises infrastructure or public cloud platforms. It distributes application workloads across a Kubernetes cluster and automates dynamic container networking needs. Kubernetes also allocates storage and persistent volumes to running containers, provides automatic scaling, and works continuously to maintain the desired state of applications, providing resiliency.

Prerequisites,

• A compatible Linux host. The Kubernetes project provides generic instructions for Linux distributions based on Debian and Red Hat, and those distributions without a package manager.
• 2 GB or more of RAM per machine (any less will leave little room for your apps).
• 2 CPUs or more.
• Full network connectivity between all machines in the cluster (public or private network is fine).
• Unique hostname, MAC address, and product_uuid for every node.
• Certain ports are open on your machines.
• Swap disabled. You MUST disable swap in order for the kubelet to work properly.
• Windows operating system must be Windows Server 2019.

First, let’s install docker for all the nodes we will include in the node.

Docker Installation for Ubuntu 20.04: There are two different methods to install docker.

1st way: Run the command “.sh” docker script.

2nd way: With the official install instructions on the docker official website.

Docker installation with way 1: Run the following bash commands on Ubuntu 20.04 host,

curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

Note: If we install docker with the 1st way, we will have installed the latest version of docker. To install a specific version, you should choose the 2nd way.

Docker installation with way 2: Before you install Docker Engine for the first time on a new host machine, you need to set up the Docker repository. Afterward, you can install and update docker from the repository.

• Update the apt package index and install packages to allow apt to use a repository over HTTPS.

sudo apt-get update
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
Add Docker’s official GPG key.
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg — dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
Use the following command to set up the stable repository.
echo \
“deb [arch=$(dpkg — print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable” | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt-get install docker-ce docker-ce-cli containerd.io

If you want to install a specific version of docker engine, Run the following commands below.

sudo apt update
sudo apt-get install docker-ce=<VERSION_STRING> docker-ce-cli=<VERSION_STRING> containerd.io

Note: Command to list Docker versions,

sudo apt-cache madison docker-ce

Command to run after installing docker for docker version control.

docker version

For enabling Non-root Users to Run Docker Commands. Run the following commands below.

sudo groupadd docker
sudo gpasswd -a $USER docker
newgrp docker

Docker Installation for Windows Server 2019

• Install the Hyper-V feature: On Windows platforms, you can run containers in two modes: process isolation and Hyper-V isolation. In process isolation mode, containers share the OS kernel with the host and hence are lightweight and similar to how containers work on Linux systems. Conversely, in Hyper-V isolation mode, each container runs inside a special minimal virtual machine. Thus, it provides secure kernel-level isolation and enhanced compatibility. You need to enable Hyper-V in the host OS to run containers in Hyper-V isolation mode.

You can install Hyper-V on Windows Server using the PowerShell command below:

Install-WindowsFeature -Name Hyper-V -IncludeManagementTools - Restart

• Install the containers feature: For containerization to work, you need to install the Windows container feature on the Windows container host. Use the command below to install the containers feature and reboot the computer.

Install-WindowsFeature -Name Containers
Install-WindowsFeature containers -Restart

• Install Docker: The process for installing Docker EE on Windows Server is quite simple with the introduction of the OneGet provider PowerShell Module. As a first step, install the Docker-Microsoft PackageManagement Provider module from the PowerShell Gallery.

Install-Module -Name DockerMsftProvider -Repository PSGallery -Force

• We can use the commands below to view the installed package provider and the docker package made available through it.

Get-PackageProvider -ListAvailableget-packagesource -ProviderName DockerMsftProvider

• We will use the PackageManagement PowerShell module command Install-Package to install the latest version of Docker.

Install-Package -Name docker -ProviderName DockerMsftProvider

• To install a specific version docker engine.

Install-Package -Name docker -ProviderName DockerMsftProvider -Force -RequiredVersion XX.XX.X

Command to run after installing docker for Docker version control.

docker version

For enabling Non-root Users to Run Docker Commands. Run the following commands below.

Install-Module -Name dockeraccesshelper
Import-Module dockeraccesshelper
Add-AccountToDockerAccess “username”

Note: The same version of docker must be installed on all nodes of Linux and Windows operating system.

Kubernetes Installation: There are many different ways to install kubernetes cluster on your system. These,

• Kubeadm
• Rancher Rke 2
• Kubespay

We will install a single kubernetes cluster with kubeadm. Before, we have to install kubelet, kubectl and kubeadm on all linux operating systems.

kubelet: The kubelet is the primary “node agent” that runs on each node. It can register the node with the apiserver using one of: the hostname; a flag to override the hostname; or specific logic for a cloud provider. The kublet in kubernetes manages the POD lifecycle. It receives the podspec which is the declarative definition of the desired POD state and performs the operations to “activate” the POD.

kubectl: The kubectl command line tool lets you control Kubernetes clusters. For configuration, kubectl looks for a file named config in the $HOME/.kube directory. In short, kubectl is kubernetes cli (Command Line Interface) .

kubeadm: Kubeadm is a tool built to provide kubeadm init and kubeadm join as best-practice “fast paths” for creating Kubernetes clusters.

For this, we have to follow the steps below.

• Letting iptables see bridged traffic: As a requirement for your Linux Node’s iptables to correctly see bridged traffic, you should ensure net.bridge.bridge-nf-call-iptables is set to 1 in your sysctl config.

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl — system

• Debian-based operating systems install kubeadm, kubelet and kubectl with following commands.

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg
echo “deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main” | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update

If Run the following command, You will have installed the latest version of kubelet, kubectl and kubeadm.

sudo apt-get install -y kubelet kubeadm kubectl

If you want to install a specific version of kubelet,kubectl and kubeadm. Firstly, kubelet, kubectl and kubeadm command to run to list the versions we can install.

sudo apt list -a kubelet
sudo apt list -a kubectl
sudo apt list -a kubeadm

Then the command we will run to install it with a specific version.

sudo apt install -y kubeadm=X.XX.X-XX kubelet=X.XX.X-XX kubectl=X.XX.X-XX

Command to will run to verify installation of kubelet, kubectl and kubeadm.

sudo apt-mark hold kubelet kubeadm kubectl

Kubernetes Cluster Installation With Kubeadm: Before we start the installation, we must turn off swap in all nodes. There are different way swap disable.

1st way: To temporarily turn off the swap, From terminal the command to run.

sudo swapoff -a

Command to be run from the terminal for control.

cat /proc/swaps

Note: If nothing appears on the terminal screen when the above command is run, it means that swap is turned off.

2nd way: To permanently turn off the swap. This is the most logical way to turn off swap. For this, To “/etc/fstab” open with vim editor and swapfile row delete. Then save the file and exit.

Add To Control Plane Endpoint: We need to add the ip address and domain name of the first master machine that we will install with kubeadm to all nodes “/etc/hosts”. For instance,

192.168.1.28         kubernetes.internal.denizturkmen

Command to run to install kubernetes cluster after adding control plane endpoints for all nodes under “/etc/hosts”.

sudo kubeadm init --control-plane-endpoint=”kubernetes.internal.denizturkmen:6443"
--apiserver-advertise-address=192.168.1.28 
--node-name k8s-master-1 
--pod-network-cidr=10.244.0.0/16

To explain the command a little.

• control-plane-endpoint: String domain name we add to all nodes.
• apiserver-advertise-address: Master machine ip address.
• node-name: The name of your Kubernetes cluster.
• pod-network-cidr: The ip range that the pods included in our Kubernetes cluster will receive.

After the command ends successfully, the master and worker gives us a token to add as a node. We should save these tokens to add other nodes to our cluster.

Generally, the token given to us by the kubeadm cluster command is valid for kubernetes 1.18- but for 1.18+ it may be necessary to create a new token.

If I get an error about the token when I try to add another node. We need to create a new master token with kubeadm. For this,

kubeadm token --help
kubeadm token create --help
sudo kubeadm init phase upload-certs --upload-certs
kubeadm token create --certificate-key xxxxxxxxxx --print-join-command
kubeadm token list

If I get an error about the token when I try to add another node. We need to create a new worker token with kubeadm. For this,

kubeadm token create --print-join-command

To use Kubernetes Cluster kubectl,

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

To Install Container Network Interface(CNI) for Ubuntu:

Note: We are installing flannel CNI as we will add it as a worker node to our windows kubernetes cluster.

Firstly, flannel yml be download and flannel yaml open vim editor.

• wget https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml

net-conf.json: |
{
“Network”: “10.244.0.0/16”,
“Backend”: {
“Type”: “vxlan”,
“VNI”: 4096,
“Port”: 4789
}
}

Note: The network address here is the pod cidr network ip address we gave in the kubeadm command. Additionally, we add VNI and PORT.

Then, “Flannel.yml” be applied.

kubectl apply -f kube-fhannel.yml

The additions made here are for adding windows worker node to kubernetes cluster.

After a few minutes after the command is executed, the state of the node in our kubernetes cluster will go to the ready state.

Command to run to check

kubectl get nodes

Node status to look at more details.

kubectl get nodes -owide

Add Master Node for Ubuntu

Note: Be sure to add swap and endpoint hosts. Then, Run the master token command you received from the master machine.

sudo kubeadm join kubernetes.internal.denizturkmen:6443 --token bxe6rz.dvmxi5fsm8z2sq09 --discovery-token-ca-cert-hash sha256:859ae8797b90f81c1f8de467a0da85e058fb194076 --control-plane -- certificate-key 2fc83490b206ff44c070716b69824a2992d38a --apiserver-advertise-address=XX.XX.X.XX --node-name k8s-master-2

To run kubectl commands, you must also transfer the config files from the main master machine to this machine. For this,

mkdir ~/.kube
scp root@XX.XXX.XX.XX:/etc/kubernetes/admin.conf ~/.kube/config

To check that the master node has been added, the following command is run from the master machine.

kubectl get nodes

Add Worker Node for Ubuntu

Note: Be sure to add swap and endpoint hosts. Then, Run the master token command you received from the master machine.

sudo kubeadm join kubernetes.internal.denizturkmen:6443 --token 0epqtx.ajn7bcp8zo --discovery-token-ca-cert-hash sha256:859ae87940b4b7343f1f4127a0da85e058fb194076 --apiserver-advertise-address=XX.XX.X.XX --node-name k8s-worker-1

• Since Worker is a node, it does not run kubectl commands, there is no need to throw config files with scp from it.

To check that the Worker node has been added, the following command is run from the master machine.

kubectl get nodes

Add Worker Node for Windows Server

We go to the machine we set up as a master and set the kube proxy. For this,

curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed ‘s/VERSION/v1.20.5/g’ | kubectl apply -f -
kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml

Note: If the kube-proxy crossloop is crashing, You can fix it by downgrading the version we downloaded with curl here.

The command we will run to control kube-proxy from the master machine.

kubectl get pod -A -n kube-system

If the pod does not restart, there is no problem.

We are installing kubelet and kubeadm on Windows Server machine. For this,

curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
.\PrepareNode.ps1 -KubernetesVersion v1.21.5

After the installation is complete, we run the worker token command we received from the host machine.

sudo kubeadm join kubernetes.internal.denizturkmen:6443 --token 0epqtx.ajn7bcp8zo --discovery-token-ca-cert-hash sha256:859ae87940b4b7343f1f4127a0da85e058fb194076 --apiserver-advertise-address=XX.XX.X.XX --node-name k8s-win-wroker-1

Finally, to check that the worker node has been added, the following command is run from the master machine.

kubectl get nodes owide

We have come to the end of this article. I will continue to write my articles on my research and the scenarios I have encountered in the sector. I hope it was a useful writing. Thanks for reading my post.

Deniz Türkmen

Deniz Türkmen — Turkey | Professional Profile | LinkedIn

denizturkmen — Overview