Minggu, 10 November 2024

HAPROXY-kubernetes

 

SETTING UP KUBERNETES CLUSTER WITH MULTIPLE CONTROL PLANE NODES BEHIND HAPROXY

https://medium.com/@murat.bilal/setting-up-kubernetes-cluster-with-multiple-control-plane-nodes-behind-haproxy-9384c6c040d8
Murat Bilal

8 min read

In this article I demonstrate how to build a kubernetes cluster including 3 Control-plane and 2 worker nodes behind Haproxy.

According to kubernetes documentation You can set up an HA cluster:

  • With stacked control plane nodes, where etcd nodes are colocated with control plane nodes
  • With external etcd nodes, where etcd runs on separate nodes from the control plane.

I am using stacked control plane nodes option, where etcd nodes are colocated with control plane nodes.

A stacked HA cluster is where the distributed data storage cluster provided by etcd is stacked on top of the cluster formed by the nodes managed by kubeadm that run control plane components.

Each control plane node runs an instance of the kube-apiserver, kube-scheduler, and kube-controller-manager. The kube-apiserver is exposed to worker nodes using a load balancer.

Each control plane node creates a local etcd member and this etcd member communicates only with the kube-apiserver of this node. The same applies to the local kube-controller-manager and kube-scheduler instances.

This topology couples the control planes and etcd members on the same nodes. It is simpler to set up than a cluster with external etcd nodes, and simpler to manage for replication.

However, a stacked cluster runs the risk of failed coupling. If one node goes down, both an etcd member and a control plane instance are lost, and redundancy is compromised. You can mitigate this risk by adding more control plane nodes.

You should therefore run a minimum of three stacked control plane nodes for an HA cluster.

Here are the steps for deploying stacked cluster:

  1. Deploy 2G Ram, 2 VCPU, 20G disk Ubuntu 22.04 VMs.Minimal install option is enough. For Kubernetes cluster I am using v1.27

My vm IP addresses:

controller1  10.252.54.45
controller2 10.252.54.133
controller3 10.252.54.88
worker1 10.252.55.221
worker2 10.252.54.106
haproxy 10.252.54.81

2. On all nodes install containerd as a container runtime environment.

sudo apt install containerd -y
sudo mkdir /etc/containerd
sudo bash -c 'containerd config default > /etc/containerd/config.toml'

sudo systemctl start containerd
sudo systemctl enable containerd

alcalab@controller1:~$ sudo ctr version
Client:
Version: 1.7.2
Revision:
Go version: go1.20.3

Server:
Version: 1.7.2
Revision:
UUID: 5cd52fc2-b80c-4a30-9bab-af89035df0f2

3. Install Kubernetes Tools on all nodes(Kubeadm,Kubectl, kubelet)

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl gpg
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.27/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.27/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

4. Disable swap on all nodes

sudo swapoff -a

# Disable it also on /etc/fstab

5. Login haproxy vm. Install and configure haproxy :

sudo apt-get update
sudo apt-get install haproxy -y

# Edit the HAProxy configuration file (usually located at /etc/haproxy/haproxy.cfg)
# Add below config
frontend k8s-control-plane
bind 10.252.54.81:6443
mode tcp
option tcplog
default_backend k8s-control-plane

backend k8s-control-plane
mode tcp
balance roundrobin
option tcp-check
server control-plane-1 10.252.54.45:6443 check
server control-plane-2 10.252.54.133:6443 check
server control-plane-3 10.252.54.88:6443 check

sudo service haproxy restart
sudo service haproxy status

Oct 28 19:04:13 haproxy haproxy[1384]: [WARNING] (1384) : Server k8s-control-plane/control-plane-1 is DOWN, reason: Layer4 connection problem, info: "Connection refused at initial connection step of tcp-c>
Oct 28 19:04:13 haproxy haproxy[1384]: [WARNING] (1384) : Server k8s-control-plane/control-plane-2 is DOWN, reason: Layer4 connection problem, info: "
Connection refused at initial connection step of tcp-c>
Oct 28 19:04:14 haproxy haproxy[1384]: [WARNING] (1384) : Server k8s-control-plane/control-plane-3 is DOWN, reason: Layer4 connection problem, info: "Connection refused at initial connection step of tcp-c>

# Currently it is down, this is normal

6. Please edit /etc/hosts for all vms. For example for controller1:

alcalab@controller1:~$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 controller1
10.252.54.45 controller1
10.252.54.133 controller2
10.252.54.88 controller3
10.252.54.106 worker2
10.252.55.221 worker1
10.252.54.81 haproxy

7. For all nodes, Forwarding IPv4 and letting iptables see bridged traffic

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF


sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF


# Apply sysctl params without reboot
sudo sysctl --system

# Verify that the br_netfilter, overlay modules are loaded by running the following commands:
lsmod | grep br_netfilter
lsmod | grep overlay

# Verify that the net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward system variables are set to 1 in your sysctl config by running the following command:
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

8. Login controller1 . Initialize First Control Plane Node(controller1):

sudo kubeadm init --control-plane-endpoint=10.252.54.81:6443 --upload-certs

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of the control-plane node running the following command on each as root:

kubeadm join 10.252.54.81:6443 --token ou152l.svboxp5m0gxvsu27 \
--discovery-token-ca-cert-hash sha256:d64a4fee845fa7cb499b7d113bb773b8c04a4e29999b3ac8c6c518f1f595d449 \
--control-plane --certificate-key 8921bddc74fade19ce3e752c2f3f8c596cb7185949f512bcdcf7b724c3d68481

9. Monitor First Control Plane node with “kubectl get nodes” and “kubectl get pods -A”. Everything should be ready and running.

10. Apply calico network plugin on First Control Plane Node:

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

11. Login controller2 and run below command.Join Control Plane Node 2 to the cluster.

# Take the previous command output for joining 2 control plane node.In our case controller2

sudo kubeadm join 10.252.54.81:6443 --token ou152l.svboxp5m0gxvsu27 \
--discovery-token-ca-cert-hash sha256:d64a4fee845fa7cb499b7d113bb773b8c04a4e29999b3ac8c6c518f1f595d449 \
--control-plane --certificate-key 8921bddc74fade19ce3e752c2f3f8c596cb7185949f512bcdcf7b724c3d68481

# Monitor with kubectl get nodes and kubectl get pods -A until all are ready and running.
# 10.252.54.81 is our haproxy IP

12. Login controller3 and run below command. Join Control Plane Node 3 to the cluster.

# Take the previous command output for joining control plane node to the cluster .In our case controller3

sudo kubeadm join 10.252.54.81:6443 --token ou152l.svboxp5m0gxvsu27 \
--discovery-token-ca-cert-hash sha256:d64a4fee845fa7cb499b7d113bb773b8c04a4e29999b3ac8c6c518f1f595d449 \
--control-plane --certificate-key 8921bddc74fade19ce3e752c2f3f8c596cb7185949f512bcdcf7b724c3d68481

# Monitor with kubectl get nodes and kubectl get pods -A until all are ready and running.
# 10.252.54.81 is our haproxy IP

13. Login worker node1 and run below commands.

containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

sudo kubeadm join 10.252.54.81:6443 --token w8egbz.ugc5hdxftaz7iop6 --discovery-token-ca-cert-hash sha256:53ee4ee84eb80e9046b030732fa67c0936c936821e736085f6d1a55150ba965b

# Monitor with kubectl get nodes and kubectl get pods -A until all are ready and running.
# 10.252.54.81 is our haproxy IP

14. Login worker node 2 and run below commands.

containerd config default | sudo tee /etc/containerd/config.toml >/dev/null 2>&1
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

sudo kubeadm join 10.252.54.81:6443 --token w8egbz.ugc5hdxftaz7iop6 --discovery-token-ca-cert-hash sha256:53ee4ee84eb80e9046b030732fa67c0936c936821e736085f6d1a55150ba965b

# Monitor with kubectl get nodes and kubectl get pods -A until all are ready and running.
# 10.252.54.81 is our haproxy IP

15. Check nodes and system pods from controller1:

alcalab@controller1:~$ kubectl get pods -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
calico-kube-controllers-85578c44bf-vkc75 1/1 Running 0 132m 172.16.214.66 controller1 <none> <none>
calico-node-cqqln 1/1 Running 25 (11m ago) 91m 10.252.54.106 worker2 <none> <none>
calico-node-htg22 1/1 Running 0 143m 10.252.54.45 controller1 <none> <none>
calico-node-lg79p 1/1 Running 7 (26m ago) 36m 10.252.55.221 worker1 <none> <none>
calico-node-nhdkz 1/1 Running 0 97m 10.252.54.88 controller3 <none> <none>
calico-node-rlppr 1/1 Running 0 104m 10.252.54.133 controller2 <none> <none>
coredns-5d78c9869d-sswnz 1/1 Running 0 132m 172.16.214.67 controller1 <none> <none>
coredns-5d78c9869d-w8pw7 1/1 Running 0 144m 172.16.214.65 controller1 <none> <none>
etcd-controller1 1/1 Running 84 144m 10.252.54.45 controller1 <none> <none>
etcd-controller2 1/1 Running 20 104m 10.252.54.133 controller2 <none> <none>
etcd-controller3 1/1 Running 0 97m 10.252.54.88 controller3 <none> <none>
kube-apiserver-controller1 1/1 Running 94 144m 10.252.54.45 controller1 <none> <none>
kube-apiserver-controller2 1/1 Running 0 104m 10.252.54.133 controller2 <none> <none>
kube-apiserver-controller3 1/1 Running 17 97m 10.252.54.88 controller3 <none> <none>
kube-controller-manager-controller1 1/1 Running 77 (103m ago) 144m 10.252.54.45 controller1 <none> <none>
kube-controller-manager-controller2 1/1 Running 0 104m 10.252.54.133 controller2 <none> <none>
kube-controller-manager-controller3 1/1 Running 12 97m 10.252.54.88 controller3 <none> <none>
kube-proxy-57bxm 1/1 Running 0 104m 10.252.54.133 controller2 <none> <none>
kube-proxy-896v4 1/1 Running 23 (25m ago) 93m 10.252.55.221 worker1 <none> <none>
kube-proxy-pwhsn 1/1 Running 23 (11m ago) 91m 10.252.54.106 worker2 <none> <none>
kube-proxy-vcsz2 1/1 Running 0 144m 10.252.54.45 controller1 <none> <none>
kube-proxy-vzjm5 1/1 Running 0 97m 10.252.54.88 controller3 <none> <none>
kube-scheduler-controller1 1/1 Running 76 (103m ago) 144m 10.252.54.45 controller1 <none> <none>
kube-scheduler-controller2 1/1 Running 0 104m 10.252.54.133 controller2 <none> <none>
kube-scheduler-controller3 1/1 Running 14 97m 10.252.54.88 controller3 <none> <none>

alcalab@controller1:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
controller1 Ready control-plane 3h10m v1.27.7
controller2 Ready control-plane 150m v1.27.7
controller3 Ready control-plane 143m v1.27.7
worker1 Ready <none> 139m v1.27.7
worker2 Ready <none> 137m v1.27.7

16. Check haproxy status:

alcalab@haproxy:~$ echo "show stat" | sudo nc -U /run/haproxy/admin.sock | cut -d "," -f 1,2,8-10,18 | column -s, -t
# pxname svname stot bin bout status
k8s-control-plane FRONTEND 33564 40481017 444896960 OPEN
k8s-control-plane control-plane-1 11620 23858568 264448949 UP
k8s-control-plane control-plane-2 2711 10281107 135957292 UP
k8s-control-plane control-plane-3 638 2213922 44490719 UP
k8s-control-plane BACKEND 33564 40481017 444896960 UP

17. Test Your Kubernetes Cluster by installing nginx.Login to controller1 and run below commands:

kubectl create deployment nginx-app --image=nginx --replicas=2

alcalab@controller1:~$ kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-app-5c64488cdf-5629d 1/1 Running 0 52m
nginx-app-5c64488cdf-qvz9t 1/1 Running 0 52m

18. Expose the deployment as NodePort:

kubectl expose deployment nginx-app --type=NodePort --port=80
service/nginx-app exposed

alcalab@controller1:~$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h19m
nginx-app NodePort 10.103.18.197 <none> 80:31032/TCP

19. Try to access from port 31032 with curl

alcalab@controller1:~$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-app-5c64488cdf-5629d 1/1 Running 0 55m 172.16.189.82 worker2 <none> <none>
nginx-app-5c64488cdf-qvz9t 1/1 Running 0 55m 172.16.235.135 worker1 <none> <none>

alcalab@controller1:~$ curl http://10.252.55.221:31032
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

alcalab@controller1:~$ curl http://worker2:31032
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Congratulations! You have successfully set up a Kubernetes cluster on Ubuntu 22.04. Explore further by deploying more complex applications and services on your Kubernetes cluster.

Senin, 04 November 2024

VIRCTL-remote kubernetes

 

Using virtctl to access virtual machine in Kubernetes

Install the virtctl client tool

Basic VirtualMachineInstance operations can be performed with the stock kubectl utility. However, the virtctl binary utility is required to use advanced features such as:

  • Serial and graphical console access

It also provides convenience commands for:

  • Starting and stopping VirtualMachineInstances
  • Live migrating VirtualMachineInstances
  • Uploading virtual machine disk images

There are two ways to get it:

  • the most recent version of the tool can be retrieved from the official release page
  • it can be installed as a kubectl plugin using krew

Example:

$ export VERSION=v0.48.1
$ wget https://github.com/kubevirt/kubevirt/releases/download/${VERSION}/virtctl-${VERSION}-linux-amd64
$ ln -s virtctl-v0.48.1-linux-amd64 virtctl
$ chmod +x virtctl-v0.48.1-linux-amd64
$ ./virtctl version
Client Version: version.Info{GitVersion:"v0.48.1", ...}

Access the virtual machine console

$ ./virtctl -h
virtctl controls virtual machine related operations on your kubernetes cluster.

Available Commands:
  addvolume         add a volume to a running VM
  console           Connect to a console of a virtual machine instance.
  expose            Expose a virtual machine instance, virtual machine, or virtual machine instance replica set as a new service.
  fslist            Return full list of filesystems available on the guest machine.
  guestfs           Start a shell into the libguestfs pod
  guestosinfo       Return guest agent info about operating system.
  help              Help about any command
  image-upload      Upload a VM image to a DataVolume/PersistentVolumeClaim.
  migrate           Migrate a virtual machine.
  pause             Pause a virtual machine
  permitted-devices List the permitted devices for vmis.
  port-forward      Forward local ports to a virtualmachine or virtualmachineinstance.
  removevolume      remove a volume from a running VM
  restart           Restart a virtual machine.
  soft-reboot       Soft reboot a virtual machine instance
  ssh               Open a SSH connection to a virtual machine instance.
  start             Start a virtual machine.
  stop              Stop a virtual machine.
  unpause           Unpause a virtual machine
  usbredir          Redirect a usb device to a virtual machine instance.
  userlist          Return full list of logged in users on the guest machine.
  version           Print the client and server version information.
  vnc               Open a vnc connection to a virtual machine instance.

Use "virtctl <command> --help" for more information about a given command.
Use "virtctl options" for a list of global command-line options (applies to all commands).

$ ./virtctl console vm1
[root@vm1 output]# hostname
vm1

Reference

COZYSTACK-install OK

https://cozystack.io/docs/get-started/
https://blog.aenix.io/installing-a-kubernetes-cluster-managed-by-cozystack-a-detailed-guide-by-gohost-and-%C3%A6nix-2b2d2e0ddbdb

dbsa@dbsa-X406UA
:~$ ssh root@172.16.79.5
Welcome to Ubuntu 22.04 LTS (GNU/Linux 4.15.0-213-generic x86_64)

* Documentation:  https://help.ubuntu.com
* Management:     https://landscape.canonical.com
* Support:        https://ubuntu.com/advantage

 

Setting up nmap (7.91+dfsg1+really7.80+dfsg1-2ubuntu0.1) ...

Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.8) ...
Выберите версию talos клиента для скачивания:
1) v1.8.2
2) v1.9.0-alpha.0
3) v1.8.1
4) v1.7.7
5) v1.8.0
6) v1.8.0-beta.1
7) v1.8.0-beta.0
8) v1.8.0-alpha.2
9) v1.7.6
10) v1.6.8
#? 9
Вы выбрали версию v1.7.6
--2024-11-04 15:40:06--  https://github.com/siderolabs/talos/releases/download/v1.7.6/talosctl-linux-amd64
Resolving github.com (github.com)... 20.205.243.166
Connecting to github.com (github.com)|20.205.243.166|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/109451092/9cda616a-dab1-41cd-846e-e540a97e3d44?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduct
ion%2F20241104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241104T084007Z&X-Amz-Expires=300&X-Amz-Signature=1d01f0e214c786a15e233aa05911e2b19368add5177de294f0dd86036e74dba7&X-Amz-SignedHeaders=host&resp
onse-content-disposition=attachment%3B%20filename%3Dtalosctl-linux-amd64&response-content-type=application%2Foctet-stream [following]
--2024-11-04 15:40:07--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/109451092/9cda616a-dab1-41cd-846e-e540a97e3d44?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=rele
aseassetproduction%2F20241104%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241104T084007Z&X-Amz-Expires=300&X-Amz-Signature=1d01f0e214c786a15e233aa05911e2b19368add5177de294f0dd86036e74dba7&X-Amz-SignedHe
aders=host&response-content-disposition=attachment%3B%20filename%3Dtalosctl-linux-amd64&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.109.133, 185.199.108.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 79237272 (76M) [application/octet-stream]
Saving to: ‘talosctl’

talosctl                                           100%[===============================================================================================================>]  75,57M  13,7MB/s    in 6,0s     

2024-11-04 15:40:14 (12,6 MB/s) - ‘talosctl’ saved [79237272/79237272]

Получаем список доступных версий kubectl...
Выберите версию kubectl для скачивания:
1) v1.30.4
2) v1.29.8
#? 1
Вы выбрали версию v1.30.4
Загружается kubectl версия v1.30.4 ...
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100 49.0M  100 49.0M    0     0  10.0M      0  0:00:04  0:00:04 --:--:-- 11.7M
kubectl версия v1.30.4 успешно установлена!
Получаем список доступных версий Helm...
Доступные версии Helm:
1) v3.16.2
2) v3.16.1
3) v3.16.0
4) v3.16.0
5) v3.15.4
6) v3.15.3
7) v3.15.2
8) v3.15.1
9) v3.15.0
10) v3.15.0
#? 1
Скачиваем и устанавливаем Helm версии v3.16.2...
Downloading https://get.helm.sh/helm-v3.16.2-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
Helm версии v3.16.2 успешно установлен!
version.BuildInfo{Version:"v3.16.2", GitCommit:"13654a52f7c70a143b1dd51416d633e1071faffb", GitTreeState:"clean", GoVersion:"go1.22.7"}
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4912  100  4912    0     0   5225      0 --:--:-- --:--:-- --:--:--  768k
Получаем список доступных версий talm
Доступные версии:
1) v0.6.4     3) v0.6.2    5) v0.6.0    7) v0.5.11   9) v0.5.8   11) v0.5.6   13) v0.5.4   15) v0.5.2   17) v0.5.0   19) v0.3.1   21) v0.2.2   23) v0.2.0   25) v0.1.0
2) v0.6.3     4) v0.6.1    6) v0.5.12   8) v0.5.10  10) v0.5.7   12) v0.5.5   14) v0.5.3   16) v0.5.1   18) v0.4.0   20) v0.3.0   22) v0.2.1   24) v0.1.1
#? 1
Скачиваем talm версии v0.6.4...
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 97.9M  100 97.9M    0     0  10.3M      0  0:00:09  0:00:09 --:--:-- 13.3M
Проверка установки...
talm version 0.6.4
talm успешно установлен!
Укажите название директории для конфигурационных файлов,
директория будет располагаться в каталоге /opt/. По умолчанию: /opt/cozystack

Введите название директории:                                                                                                                                                                                
                                                                                                                                                                                                           
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 21727  100 21727    0     0  23555      0 --:--:-- --:--:-- --:--:-- 23555
yq v4.44.2 from Mike Farah (mikefarah) installed
Укажите IP-сеть для  etcd и kubelet
По умолчанию:  192.168.100.0/24  
IP-сеть (network/mask): 172.16.79.0/24
grep: : No such file or directory
net.ipv4.ip_forward = 1
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
 libwpe-1.0-1 libwpebackend-fdo-1.0-1
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
 netfilter-persistent
The following NEW packages will be installed:
 iptables-persistent netfilter-persistent
0 upgraded, 2 newly installed, 0 to remove and 3 not upgraded.
Need to get 13,9 kB of archives.
After this operation, 93,2 kB of additional disk space will be used.
Get:1 http://id.archive.ubuntu.com/ubuntu jammy/universe amd64 netfilter-persistent all 1.0.16 [7.440 B]
Get:2 http://id.archive.ubuntu.com/ubuntu jammy/universe amd64 iptables-persistent all 1.0.16 [6.488 B]
Fetched 13,9 kB in 1s (21,9 kB/s)          
Preconfiguring packages ...
Selecting previously unselected package netfilter-persistent.
(Reading database ... 167872 files and directories currently installed.)
Preparing to unpack .../netfilter-persistent_1.0.16_all.deb ...
Unpacking netfilter-persistent (1.0.16) ...
Selecting previously unselected package iptables-persistent.
Preparing to unpack .../iptables-persistent_1.0.16_all.deb ...
Unpacking iptables-persistent (1.0.16) ...
Setting up netfilter-persistent (1.0.16) ...
Created symlink /etc/systemd/system/multi-user.target.wants/netfilter-persistent.service → /lib/systemd/system/netfilter-persistent.service.
Setting up iptables-persistent (1.0.16) ...
update-alternatives: using /lib/systemd/system/netfilter-persistent.service to provide /lib/systemd/system/iptables.service (iptables.service) in auto mode
Processing triggers for man-db (2.10.2-1) ...


>>> Welcome to Webi! - modern tools, instant installs.  <<<
   We expect your experience to be absolutely perfect!

   Success? Star it!   https://github.com/webinstall/webi-installers
   Problem? Report it: https://github.com/webinstall/webi-installers/issues
                       (your system is GNU/Linux/x86_64 with libc & curl+wget)

Bootstrapping Webi
   Downloading https://webinstall.dev/packages/webi/webi.sh
       to ~/.local/bin/webi
   Running ~/.local/bin/webi k9s@stable

Installing k9s ...
   Found  ~/.local/bin
   Initializing ~/.config/envman/
   Edit ~/.profile to source ~/.config/envman/load.sh
   Edit ~/.bashrc to source ~/.config/envman/load.sh
   Downloading k9s from
     https://github.com/derailed/k9s/releases/download/v0.32.5/k9s_Linux_amd64.tar.gz
   Saved as ~/Downloads/webi/k9s/0.32.5/k9s_Linux_amd64.tar.gz
   Extracting ~/Downloads/webi/k9s/0.32.5/k9s_Linux_amd64.tar.gz
   Installing to ~/.local/opt/k9s-v0.32.5/bin/k9s

   Installed 'k9s v0.32.5' as ~/.local/bin/k9s

   Edit ~/.config/envman/PATH.env to add:
       ~/.local/bin

>>> ACTION REQUIRED <<<
       Copy, paste & run the following command:
       source ~/.config/envman/PATH.env
       (newly opened terminal windows will update automatically)

talm       in folder /usr/local/bin/talm
cp: cannot stat 'kubeconfig': No such file or directory
cp: cannot stat 'talosconfig': No such file or directory
========== Installed binary ===========
helm       in folder /usr/local/bin/helm
yq         in folder /snap/bin/yq
kubectl    in folder /usr/local/bin/kubectl
docker     in folder /usr/bin/docker
talosctl   in folder /usr/local/bin/talosctl
dialog     in folder /usr/bin/dialog
nmap       in folder /usr/bin/nmap
talm       in folder /usr/local/bin/talm
k9s       in folder /root/.local/bin/k9s
node_shell       in folder /usr/local/bin/kubectl-node_shell
========== services runing ===========
DNS Bind9       active
NTP             active                                                                                                                                                                                      
========== ADD Iptables Rule ===========                                                                                                                                                                    
-A FORWARD -s 172.16.79.0/24 -m comment --comment cozystack -j ACCEPT
-A FORWARD -d 172.16.79.0/24 -m state --state RELATED,ESTABLISHED -m comment --comment cozystack -j ACCEPT
-A POSTROUTING -s 172.16.79.0/24 -m comment --comment cozystack -j MASQUERADE
!!!  Please change the catalog to work with talos-bootstrap !!!
cd  /opt/cozystack  
root@ubuntu-virtual-machine:~/cozystack# cd /opt/cozystack/
root@ubuntu-virtual-machine:/opt/cozystack# ls
patch-controlplane.yaml  patch.yaml  talos-bootstrap
root@ubuntu-virtual-machine:/opt/cozystack# . talos-bootstrap install
awk: line 1: regular expression compile failed (missing operand)
*"$
generating PKI and tokens
Created /opt/cozystack/controlplane.yaml
Created /opt/cozystack/worker.yaml
Created /opt/cozystack/talosconfig
Dry run summary:
Node is running in maintenance mode and does not have a config yet.
root@ubuntu-virtual-machine:/opt/cozystack# . talos-bootstrap install
awk: line 1: regular expression compile failed (missing operand)
*"$
generating PKI and tokens
Created /opt/cozystack/controlplane.yaml
Created /opt/cozystack/worker.yaml
Created /opt/cozystack/talosconfig
Dry run summary:
Node is running in maintenance mode and does not have a config yet.
root@ubuntu-virtual-machine:/opt/cozystack# . talos-bootstrap install
awk: line 1: regular expression compile failed (missing operand)
*"$
generating PKI and tokens
Created /opt/cozystack/controlplane.yaml
Created /opt/cozystack/worker.yaml
Created /opt/cozystack/talosconfig
Dry run summary:
Node is running in maintenance mode and does not have a config yet.
root@ubuntu-virtual-machine:/opt/cozystack#