k8s学习九:非内网搭建k8s

文章背景

很多人知道,我在自己家里搭建了一台服务器,可以直接访问外网,另外我老家还有2台服务器,也有外网,所以想着集合下阿里云,老家的服务器资源,搭建k8s环境

家里服务器带宽高,但是ip会经常变动,所以需要考虑到ip变动的问题

我打算深圳的服务器作为master节点,进行搭建k8s网络集群

外网搭建配置

由于centos停止维护了,所以我的服务器都是ubuntu 22系统,搭建的k8s版本为1.23.10

配置虚拟网卡

因为我们的服务器都是外网访问的,网卡没有绑定外网ip,所以需要创建虚拟网卡去绑定外网ip

#首先需要获取到自己的网卡名
root@tioncico-pc:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp2s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 00:23:81:22:db:95 brd ff:ff:ff:ff:ff:ff
    inet 119.123.101.117/8 brd 119.255.255.255 scope global enp2s0:1
       valid_lft forever preferred_lft forever
3: wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 34:de:1a:98:33:89 brd ff:ff:ff:ff:ff:ff
    inet 192.168.68.152/24 metric 600 brd 192.168.68.255 scope global dynamic wlp3s0
       valid_lft 10361092sec preferred_lft 10361092sec
    inet6 fd05:a5df:b270:0:36de:1aff:fe98:3389/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::36de:1aff:fe98:3389/64 scope link 
       valid_lft forever preferred_lft forever
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:70:50:46:03 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
5: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default 
    link/ether 5a:8c:ee:0b:13:45 brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.0/32 scope global flannel.1
       valid_lft forever preferred_lft forever
    inet6 fe80::588c:eeff:fe0b:1345/64 scope link 
       valid_lft forever preferred_lft forever
6: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
    link/ether 1e:14:f7:cf:b6:e6 brd ff:ff:ff:ff:ff:ff
    inet 10.244.0.1/24 brd 10.244.0.255 scope global cni0
       valid_lft forever preferred_lft forever
    inet6 fe80::1c14:f7ff:fecf:b6e6/64 scope link 
       valid_lft forever preferred_lft forever
7: veth4273b252@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
    link/ether 2a:34:bc:1c:7e:a9 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::866:8dff:fe95:e61f/64 scope link 
       valid_lft forever preferred_lft forever
8: veth34975173@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default 
    link/ether 4e:4c:cb:ee:b6:55 brd ff:ff:ff:ff:ff:ff link-netnsid 1
    inet6 fe80::fcad:ccff:fe0a:6613/64 scope link 
       valid_lft forever preferred_lft forever
root@tioncico-pc:~# 

### 根据网卡名,去创建一张虚拟网卡: 我这边用的是enp2s0
root@tioncico-pc:~#  ifconfig enp2s0:1 119.123.101.118(外网ip)
root@tioncico-pc:~#

初始化k8s master 集群

该步骤和文章: https://www.php20.cn/article/387 一致,ip改为外网ip

 kubeadm init \
 --apiserver-advertise-address 119.123.101.117 \
 --apiserver-bind-port 6443 \
 --pod-network-cidr 10.244.0.0/16 \
 --image-repository registry.aliyuncs.com/google_containers -v10

等待集群初始化完毕

阿里云节点加入集群

该步骤和文章: 一致

kubeadm join 119.123.101.117:6443 \
--token co521f.29i90vgeen5feer6         \
--discovery-token-ca-cert-hash sha256:881055871d680663e95215e7c34dc16958ea475a0f1789c1eca5a460e4268366  \
-v10

加入成功后,在master节点可以获取到节点状态:

root@tioncico-pc:~# kubectl get nodes
NAME                      STATUS   ROLES                  AGE   VERSION
izwz92md6qnm0l3rvvrn8nz   Ready    <none>                 36m   v1.23.10
tioncico-pc               Ready    control-plane,master   44m   v1.23.10
root@tioncico-pc:~#

如果有错误,依次解决错误就可以了

可能存在的错误以及解决方案

节点显示not ready

通过 kubectl describe nodes 命令查看节点详情

 kubectl describe nodes  节点名

如果详情没发现节点报错,那说明节点连接正常

在not ready子节点上执行查看报错

systemctl status kubelet

子节点 open /run/flannel/subnet.env: no such file or directory

查看master 节点的配置文件

root@tioncico-pc:~# cat /run/flannel/subnet.env
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
root@tioncico-pc:~#

将master的复制到子节点,并重启kubelet

创建nginx服务测试:

 kubectl create deployment nginx  --image=nginx:1.14-alpine

如果报错,可通过命令排查错误:

root@tioncico-pc:~# kubectl describe pod nginx

没报错的情况:

root@tioncico-pc:~# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
nginx-7cbb8cd5d8-d7jwc   1/1     Running   0          20m
root@tioncico-pc:~#

下一章节将说明如果master节点更改之后如何重置集群