Friday, 4 November 2022

Install Kubernetes Master Node on Rocky Linux 9

Install Kubernetes Master Node on Rocky Linux 9

In this configuration guide, you will learn how to install Kubernetes Master Node on Rocky Linux 9 or other RPM based Linux distros.


Table of Contents:


What is Kubernetes?:

Kubernetes (commonly stylized as K8s) is an open-source container orchestration system for automating software deployment, scaling, and management. Google originally designed Kubernetes, but the Cloud Native Computing Foundation now maintains the project.

Kubernetes works with Docker, Containerd, and CRI-O. Originally, it interfaced exclusively with the Docker runtime through a "Dockershim"; however, from November 2020 up to April 2022, Kubernetes has deprecated the shim in favor of directly interfacing with the container through Containerd, or replacing Docker with a runtime that is compliant with the Container Runtime Interface (CRI). With the release of v1.24 in May 2022, "Dockershim" has been removed entirely.

Kubernetes defines a set of building blocks ("primitives") that collectively provide mechanisms that deploy, maintain, and scale applications based on CPU, memory or custom metrics. Kubernetes is loosely coupled and extensible to meet different workloads. The internal components as well as extensions and containers that run on Kubernetes rely on the Kubernetes API. The platform exerts its control over compute and storage resources by defining resources as Objects, which can then be managed as such.

Kubernetes follows the primary/replica architecture. The components of Kubernetes can be divided into those that manage an individual node and those that are part of the control plane.

The Kubernetes master node handles the Kubernetes control plane of the cluster, managing its workload and directing communication across the system. The Kubernetes control plane consists of various components, each its own process, that can run both on a single master node or on multiple masters supporting high-availability clusters.

Read Also:

Environment Specification:

We are using a minimal Rocky Linux 9 virtual machine with following specifications.

  • CPU - 3.4 Ghz (2 cores)
  • Memory - 2 GB
  • Storage - 20 GB
  • Operating System - Rocky Linux release 9.0 (Blue Onyx)
  • Hostname – kubemaster-01.centlinux.com
  • IP Address -

We are using a Lenovo - 2022 - IdeaPad 3i - Essential Laptop Computer - Intel Core i5 12th Gen - 15.6" FHD Display - 8GB Memory - 512GB Storage. You can also use the following link to purchase the same at very reasonable price from Amazon.com.

Lenovo Idealpad 3i


Set Hostname and Name Resolution:

By using a ssh client, connect with your Linux server as root user.

Set a proper FQDN (Fully Qualified Domain Name) for your Kubernetes server. Also use the Local DNS resolver (/etc/hosts) for name resolution of your hostname.

# hostnamectl set-hostname kubemaster-01.centlinux.com
# echo kubemaster-01.centlinux.com kubemaster-01 >> /etc/hosts


Updating Rocky Linux Server:

Refresh your cache of enabled yum repositories.

# dnf makecache --refresh
Rocky Linux 9 - BaseOS                           85 kB/s | 1.7 MB     00:20
Rocky Linux 9 - AppStream                       193 kB/s | 6.0 MB     00:31
Rocky Linux 9 - Extras                          870  B/s | 2.9 kB     00:03
Metadata cache created.

Execute following dnf command to update your Rocky Linux server.

# dnf update -y

Linux Kernel packages may be updated by the above command. Therefore, reboot your Linux server before moving forward.

# reboot

After reboot, check the Linux Kernel and operating system versions that are being used in this configuration guide.

# cat /etc/rocky-release
Rocky Linux release 9.0 (Blue Onyx)

# uname -r


Switch SELinux to Permissive Mode:

Kubernetes doesn't provide a SELinux policy, therefore you can either switch SELinux target to Permissive mode or manually set the File Context of various Kubernetes files and directories.

For the scope of this article, we recommend that you should set permissive target for SELinux. However, if you are configuring a Kubernetes cluster for production then you should identify the files and set the File Context for them, or you can even create your own SELinux policy.

Execute following commands at Linux bash to set SELinux permissive mode.

# setenforce 0
# sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config


Loading K8s required Kernel Modules:

Kubernetes requires "overlay" and "br_netfilter" Kernel modules. Therefore, you can use following group of commands to permanently enable them.

# modprobe overlay
# modprobe br_netfilter

# cat > /etc/modules-load.d/k8s.conf << EOF
> overlay
> br_netfilter

Set following Kernel parameter as required by Kubernetes software.

# cat > /etc/sysctl.d/k8s.conf << EOF
> net.ipv4.ip_forward = 1
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1

Reload Kernel parameter configuration files with above changes.

# sysctl --system
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
* Applying /usr/lib/sysctl.d/50-coredump.conf ...
* Applying /usr/lib/sysctl.d/50-default.conf ...
* Applying /usr/lib/sysctl.d/50-libkcapi-optmem_max.conf ...
* Applying /usr/lib/sysctl.d/50-pid-max.conf ...
* Applying /usr/lib/sysctl.d/50-redhat.conf ...
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/k8s.conf ...
* Applying /etc/sysctl.conf ...
kernel.yama.ptrace_scope = 0
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
kernel.core_pipe_limit = 16
fs.suid_dumpable = 2
kernel.sysrq = 16
kernel.core_uses_pid = 1
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.ens33.rp_filter = 2
net.ipv4.conf.lo.rp_filter = 2
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.ens33.accept_source_route = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.ens33.promote_secondaries = 1
net.ipv4.conf.lo.promote_secondaries = 1
net.ipv4.ping_group_range = 0 2147483647
net.core.default_qdisc = fq_codel
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_regular = 1
fs.protected_fifos = 1
net.core.optmem_max = 81920
kernel.pid_max = 4194304
kernel.kptr_restrict = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.ens33.rp_filter = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1


Disable Swap in Linux:

To disable Swap for your current session you can use swapoff command. However, for permanently disable the Swap storage you have to comment (#) the respective directive in /etc/fstab file.

Execute following commands to do the same.

# swapoff -a
# sed -e '/swap/s/^/#/g' -i /etc/fstab

Verify the usage of Swap storage on your Linux server.

# free -m
               total        used        free      shared  buff/cache   available
Mem:            1748         269        1143           8         335        1302
Swap:              0           0           0


Installing Containerd Runtime:

Containerd is not available in standard yum repositories, therefore, you may need to install Docker Official Yum Repository to install Container runtime.

# dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Adding repo from: https://download.docker.com/linux/centos/docker-ce.repo

Build yum cache for Docker yum repository.

# dnf makecache
Docker CE Stable - x86_64                       6.1 kB/s |  12 kB     00:01
Rocky Linux 9 - BaseOS                          440  B/s | 3.6 kB     00:08
Rocky Linux 9 - AppStream                       1.4 kB/s | 3.6 kB     00:02
Rocky Linux 9 - Extras                          1.5 kB/s | 2.9 kB     00:01
Metadata cache created.

Now, you can easily install Containerd runtime by using dnf command.

# dnf install -y containerd.io

After installation, backup the original containerd configuration file and generate a new file as follows.

# mv /etc/containerd/config.toml /etc/containerd/config.toml.orig
# containerd config default > /etc/containerd/config.toml

Edit Containerd configuration file by using vim text editor.

# vi /etc/containerd/config.toml

Locate and set SystemdCgroup parameter in this file, to enable the systemd cgroup driver for Containerd runtime.

    SystemdCgroup = true

Enable and start Containerd service.

# systemctl enable --now containerd.service
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /usr/lib/systemd/system/containerd.service.

Check the status of Containerd service for any errors.

# systemctl status containerd.service
● containerd.service - containerd container runtime
     Loaded: loaded (/usr/lib/systemd/system/containerd.service; enabled; vendo>
     Active: active (running) since Thu 2022-11-03 12:31:07 CDT; 25s ago
       Docs: https://containerd.io
    Process: 8454 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SU>
   Main PID: 8455 (containerd)
      Tasks: 8
     Memory: 16.1M
        CPU: 172ms
     CGroup: /system.slice/containerd.service
             └─8455 /usr/bin/containerd

Nov 03 12:31:07 rockylinux9-01.centlinux.com containerd[8455]: time="2022-11-03>
Nov 03 12:31:07 rockylinux9-01.centlinux.com systemd[1]: Started containerd con>


Configure Linux Firewall:

Kubernetes uses following service ports at Master node.

Port Protocol Purpose
6443 TCP Kubernetes API server
2379-2380 TCP etcd server client API
10250 TCP Kubelet API
10251 TCP kube-scheduler
10252 TCP kube-controller-manager

Therefore, you need to allow these service ports in Linux firewall.

# firewall-cmd --permanent --add-port={6443,2379,2380,10250,10251,10252}/tcp

# firewall-cmd --reload


Installing Kubernetes Software:

To install Orchestration software, you have to add the Kubernetes Official Yum Repository.

The following command will add the Kubernetes repository in your Linux server.

# cat > /etc/yum.repos.d/k8s.repo << EOF
> [kubernetes]
> name=Kubernetes
> baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
> enabled=1
> gpgcheck=1
> gpgkey=https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
> exclude=kubelet kubeadm kubectl

Build the yum cache for Kubernetes yum repository.

# dnf makecache
Docker CE Stable - x86_64                       1.8 kB/s | 3.5 kB     00:01
Kubernetes                                       36 kB/s | 158 kB     00:04
Rocky Linux 9 - BaseOS                          1.3 kB/s | 3.6 kB     00:02
Rocky Linux 9 - AppStream                       1.8 kB/s | 3.6 kB     00:02
Rocky Linux 9 - AppStream                       593 kB/s | 6.0 MB     00:10
Rocky Linux 9 - Extras                          1.5 kB/s | 2.9 kB     00:01
Metadata cache created.

Now, you can install Kubernetes packages by using dnf command.

# dnf install -y {kubelet,kubeadm,kubectl} --disableexcludes=kubernetes

Enable and start kubelet.service. It is the main Kubernetes service that waits for any events when you initialize the cluster or join a node to this cluster.

# systemctl enable --now kubelet.service
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /usr/lib/systemd/system/kubelet.service.

Verify the status of kubelet.service.

# systemctl status kubelet
● kubelet.service - kubelet: The Kubernetes Node Agent
     Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor p>
    Drop-In: /usr/lib/systemd/system/kubelet.service.d
     Active: active (running) since Thu 2022-11-03 13:30:03 CDT; 128ms ago
       Docs: https://kubernetes.io/docs/
   Main PID: 10967
      Tasks: 1 (limit: 10948)
     Memory: 15.3M
        CPU: 100ms
     CGroup: /system.slice/kubelet.service

Nov 03 13:30:03 kubemaster-01.centlinux.com systemd[1]: Started kubelet: The Ku>


Enable Bash Completion for Kubernetes Commands:

If you plan to manage your Kubernetes cluster from Linux CLI then, the bash completion will be very helpful for you.

To enable automatic completion of kubectl commands, you have to execute the script provided by kubectl command itself. You must ensure that bash-completion package is already installed on your Rocky Linux server.

# source <(kubectl completion bash)
# kubectl completion bash > /etc/bash_completion.d/kubectl


Installing Flannel CNI Plugin:

Kubernetes supports various CNI (Container Network Interface) plugins, such as AWS VPC, Azure CNI, Cilium, Calico, Flannel, and many more.

In this configuration guide, we are using Flannel CNI plugin. Ensure that this plugin must be installed on each Kubernetes node.

Create a directory and download flanneld file therein.

# mkdir /opt/bin
# curl -fsSLo /opt/bin/flanneld https://github.com/flannel-io/flannel/releases/download/v0.20.1/flannel-v0.20.1-linux-amd64.tar.gz

Grant execution permissions to flanneld file to make it an executable.

# chmod +x /opt/bin/flanneld


Initializing Kubernetes Control Plane:

Execute the following command to download container images, that are required to create Kubernetes Cluster.

# kubeadm config images pull
[config/images] Pulled registry.k8s.io/kube-apiserver:v1.25.3
[config/images] Pulled registry.k8s.io/kube-controller-manager:v1.25.3
[config/images] Pulled registry.k8s.io/kube-scheduler:v1.25.3
[config/images] Pulled registry.k8s.io/kube-proxy:v1.25.3
[config/images] Pulled registry.k8s.io/pause:3.8
[config/images] Pulled registry.k8s.io/etcd:3.5.4-0
[config/images] Pulled registry.k8s.io/coredns/coredns:v1.9.3

After successful downloading of images, execute the following command to initialize the Kubernetes cluster on kubemaster-01.centlinux.com server. This nod will be selected as the Kubernetes control plane because it is the first node in the cluster.

# kubeadm init
[init] Using Kubernetes version: v1.25.3
[preflight] Running pre-flight checks
        [WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly
        [WARNING SystemVerification]: missing optional cgroups: blkio
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [kubemaster-01.centlinux.com kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs []
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [kubemaster-01.centlinux.com localhost] and IPs [ ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [kubemaster-01.centlinux.com localhost] and IPs [ ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
[apiclient] All control plane components are healthy after 79.030431 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node kubemaster-01.centlinux.com as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node kubemaster-01.centlinux.com as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]
[bootstrap-token] Using token: pfejvw.wybridg116yi8taa
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join --token pfejvw.wybridg116yi8taa \
        --discovery-token-ca-cert-hash sha256:9db6c88584ab5c3f7987631514960fed83caa834d9faf5435dfb6c5bcf3fe74b

Note down the command to add Kubernetes Worker node for later use.

Execute following command to set KUBECONFIG variable for all sessions.

# echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile.d/k8s.sh

Execute following commands as a user, that is being used to manage your Kubernetes cluster.

# mkdir -p $HOME/.kube
# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# chown $(id -u):$(id -g) $HOME/.kube/config

Execute kubectl commands to check the status of your Kubernetes cluster.

# kubectl get nodes
NAME                          STATUS     ROLES           AGE     VERSION
kubemaster-01.centlinux.com   NotReady   control-plane   2m26s   v1.25.3

# kubectl cluster-info
Kubernetes control plane is running at
CoreDNS is running at

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

After Kubernetes Control Plane is started, run the following command to install the Flannel Pod network plugin. This command will automatically run the "flanneld" binary file and start some flannel pods.

# kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created

Check the list of running pods on your Kubernetes cluster.

# kubectl get pods --all-namespaces
NAMESPACE      NAME                                                  READY   STATUS     RESTARTS   AGE
kube-flannel   kube-flannel-ds-79t24                                 0/1     Init:1/2   0          61s
kube-system    coredns-565d847f94-mq59j                              0/1     Pending    0          5m27s
kube-system    coredns-565d847f94-xngbz                              0/1     Pending    0          5m27s
kube-system    etcd-kubemaster-01.centlinux.com                      1/1     Running    0          5m29s
kube-system    kube-apiserver-kubemaster-01.centlinux.com            1/1     Running    0          5m29s
kube-system    kube-controller-manager-kubemaster-01.centlinux.com   1/1     Running    0          5m29s
kube-system    kube-proxy-fh9v6                                      1/1     Running    0          5m27s
kube-system    kube-scheduler-kubemaster-01.centlinux.com            1/1     Running    0          5m29s

Your Kubernetes master node has been installed successfully.



In this configuration guide, you have learned how to install Kubernetes Master Node on Rocky Linux 9. To start managing your Cluster, you should read The Kubernetes Book: 2022 Edition by Nigel Poulton.

If you find this article useful? Consider supporting us by Buy Me A Coffee

No comments:

Post a Comment

© 2023 CentLinux. All Rights Reserved.