Cara Menyiapkan High Availability Kubernetes Klaster Menggunakan Kubeadm

https://blog.ajiarya.id/posts/kubernetes/cara-menyiapkan-high-availability-kubernetes-klaster-kubeadm/

Cara menyiapkan klaster kubernetes yang High Available

September 10, 2021 · 4 min · Aji Arya | Suggest Changes

Table of Contents

Topologi

Versi

Sistem Operasi

Ubuntu 20.04 (Focal)

Aplikasi

HAProxy 2.0.13-2ubuntu0.3
KeepAlived 1:2.0.19-2
Containerd 1.4.9-1
Kubernetes 1.22.1-00 (kubeadm, kubelet, kubectl)

Panduan

Semua perintah dibawah dijalankan menggunakan user root

Persiapan

Sunting file /etc/hosts

cat<<EOF >> /etc/hosts

### Lab Kubernetes ###
192.168.30.10 kubernetes-vip
192.168.30.11 kubernetes-lb1
192.168.30.12 kubernetes-lb2
192.168.30.13 kubernetes-master01
192.168.30.14 kubernetes-master02
192.168.30.15 kubernetes-master03
192.168.30.16 kubernetes-worker01
EOF

update dan upgrade package-package

apt update -y; apt upgrade -y

Pasang HAproxy & Keepalived pada Load Balancer Node

Pasang haproxy dan keepalived

apt install -y haproxy keepalived

Konfigurasi HAProxy

Sunting file konfigurasi

Tambahkan baris berikut kedalam file konfigurasi. Isikan backend dengan Alamat IP semua master

vim /etc/haproxy/haproxy.cfg

frontend kube-apiserver
    bind *:6443
    mode tcp
    option tcplog
    default_backend kube-apiserver

backend kube-apiserver
    mode tcp
    option tcp-check
    balance roundrobin
    default-server inter 10s downinter 5s rise 2 fall 2 slowstart 60s maxconn 250 maxqueue 256 weight 100
    server kube-apiserver-1 192.168.30.13:6443 check
    server kube-apiserver-2 192.168.30.14:6443 check
    server kube-apiserver-3 192.168.30.15:6443 check

Validasi file konfigurasi

haproxy -c -V -f /etc/haproxy/haproxy.cfg

Restart HAProxy

systemctl restart haproxy

Enable HAProxy

systemctl enable haproxy

Konfigurasi Keepalived

Sunting file keepalived

vim /etc/keepalived/keepalived.conf

lb01

vrrp_script chk_haproxy {
  script "killall -0 haproxy"
  interval 2
  weight 2
}

vrrp_instance haproxy-vip {
  state MASTER
  priority 100
  interface ens3                      # Network card
  virtual_router_id 50
  advert_int 1
  authentication {
    auth_type PASS
    auth_pass nasgorenak
  }

  virtual_ipaddress {
    192.168.30.10/24                  # The VIP address
  }

  track_script {
    chk_haproxy
  }
}

lb02

vrrp_script chk_haproxy {
  script "killall -0 haproxy"
  interval 2
  weight 2
}

vrrp_instance haproxy-vip {
  state BACKUP
  priority 99
  interface ens3                      # Network card
  virtual_router_id 50
  advert_int 1
  authentication {
    auth_type PASS
    auth_pass nasgorenak
  }

  virtual_ipaddress {
    192.168.30.10/24                  # The VIP address
  }

  track_script {
    chk_haproxy
  }
}

Restart Keepalived

systemctl restart keepalived

Enable Keepalived

systemctl enable keepalived

Pasang container runtime pada K8S master & node

Kita akan menggunakan containerd

Muat kernel module dan ubah beberapa parameter kernel

cat <<EOF | tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sysctl --system

Pasang repository resmi docker

# Pasang GPG key repository docker
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# Tambahkan repository docker
echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null

apt update

Pasang containerd.io

apt install containerd.io

Konfigurasi containerd

sudo mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml

Restart containerd

systemctl restart containerd

Pasang package-package kubernetes pada master & worker

Pasang repository

# Pasang GPG key repository kubernetes
sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

# Tambahkan repository kubernetes
echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | tee /etc/apt/sources.list.d/kubernetes.list

apt update

Pasang package-package kubernetes

apt install -y kubelet=1.22.1-00 kubeadm=1.22.1-00 kubectl=1.22.1-00
apt-mark hold kubelet kubeadm kubectl

Tips!
Periksa versi package yang tersedia menggunakan apt-cache madison <name_package>

Bootstrap Master (kubernetes-master01)

Bootstrap klaster kubernetes

kubeadm init --pod-network-cidr=172.16.0.0/16 --control-plane-endpoint "kubernetes-vip:6443" --upload-certs

Output kubeadm init
Simpan dan gunakan untuk mendaftarkan node lain ke dalam klaster

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:
# Gunakan ini untuk mendaftarkan master
  kubeadm join kubernetes-vip:6443 --token tzkfd1.tsgtqfy5q15weour \
        --discovery-token-ca-cert-hash sha256:e4c1619f9050ac1041e198675a7bf1d275a1d8a486ba213260d0cd4144a46b89 \
        --control-plane --certificate-key 496abde789dc5340843bf83272bb3f307ae13bd6f4ad2fc3e9dfe08cc77aa991

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:
# Gunakan ini untuk mendaftarkan worker
kubeadm join kubernetes-vip:6443 --token tzkfd1.tsgtqfy5q15weour \
        --discovery-token-ca-cert-hash sha256:e4c1619f9050ac1041e198675a7bf1d275a1d8a486ba213260d0cd4144a46b89

Pindahkan kubeconfig
kubeconfig berisikan informasi mengenai endpoint kubernetes dan token untuk autentikasi

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Pasang CNI

Unduh manifest calico

curl https://docs.projectcalico.org/manifests/calico.yaml -O

Sunting calico.yaml

# Sunting calico.yaml
vim calico.yaml

# sesuaikan CALICO_IP4POOL_CIDR dengan --pod-network-cidr pada saat bootstrap
            - name: CALICO_IPV4POOL_CIDR
              value: "172.16.0.0/16"

Terapkan pada klaster

kubectl apply -f calico.yaml

Mendaftarkan Master ke Klaster

Daftarkan master

Perintah dibawah didapatkan dari output kubeadm init pada master01

kubeadm join kubernetes-vip:6443 --token tzkfd1.tsgtqfy5q15weour \
      --discovery-token-ca-cert-hash sha256:e4c1619f9050ac1041e198675a7bf1d275a1d8a486ba213260d0cd4144a46b89 \
      --control-plane --certificate-key 496abde789dc5340843bf83272bb3f307ae13bd6f4ad2fc3e9dfe08cc77aa991

Mendaftarkan Worker ke Klaster

Perintah dibawah didapatkan dari output kubeadm init pada master01

Daftarkan worker

kubeadm join kubernetes-vip:6443 --token tzkfd1.tsgtqfy5q15weour \
        --discovery-token-ca-cert-hash sha256:e4c1619f9050ac1041e198675a7bf1d275a1d8a486ba213260d0cd4144a46b89

Hasil

Uji coba High Availability

Matikan salah satu lb

systemctl poweroff

Periksa status klaster dengan menjalankan perintah kubectl

# Sebagai contoh
kubectl get nodes

Kumpulan Tutorial Kito

Senin, 07 Oktober 2024

K8S-setting HA