在1.19版本之前,kubeadm部署方式启用ipvs模式时,初始化配置文件需要添加以下内容:
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
featureGates:
SupportIPVSProxyMode: true
mode: ipvs
本次在1.20.2版本中,使用kubeadm进行集群初始化时,虽然可以正常部署,但是查看pod情况的时候可以看到kube-proxy无法运行成功,报错部分内容如下:
]# kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
....#省略其他输出内容
kube-system kube-proxy-7vrbv 0/1 CrashLoopBackOff 9 43m
kube-system kube-proxy-ghs7h 0/1 CrashLoopBackOff 9 43m
kube-system kube-proxy-l9twb 0/1 CrashLoopBackOff 1 7s
kube-system kube-proxy-mzfrf 0/1 CrashLoopBackOff 9 42m
kube-system kube-proxy-nxpls 0/1 CrashLoopBackOff 9 3h4m
kube-system kube-proxy-pmmtq 0/1 CrashLoopBackOff 8 42m
#查看日志信息
]# kubectl logs kube-proxy-l9twb -n kube-system
F0114 12:58:34.042769 1 server.go:488] failed complete: unrecognized feature gate: SupportIPVSProxyMode
goroutine 1 [running]:
k8s.io/kubernetes/vendor/k8s.io/klog/v2.stacks(0xc00000e001, 0xc0004b6000, 0x6e, 0xc0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1026 +0xb9
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).output(0x29b65c0, 0xc000000003, 0x0, 0x0, 0xc0003d8230, 0x28edbc9, 0x9, 0x1e8, 0x0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:975 +0x19b
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).printf(0x29b65c0, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x1b3a573, 0x13, 0xc000431310, 0x1, ...)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:750 +0x191
k8s.io/kubernetes/vendor/k8s.io/klog/v2.Fatalf(...)
通过报错可以看到kube-proxy无法识别SupportIPVSProxyMode这个字段,于是访问官方查看最新版本ipvs开启的正确配置,通过https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md可以看到官方说明:
Cluster Created by Kubeadm
If you are using kubeadm with a configuration file, you have to add mode: ipvs below the kubeProxy field as part of the kubeadm configuration.
...
kubeProxy:
config:
mode: ipvs
...
由于集群已经初始化成功了,所以现在改kubeadm初始化配置文件没有意义,因为我们需要直接修改kube-proxy的启动配置
通过查看kube-pxory的资源清单可以知道, kube-proxy的配置文件是通过configmap方式挂载到容器中的,因此我们只需要对应修改configmap中的配置内容,就可以将无效字段删除
]# kubectl -n kube-system get pod kube-proxy-24tkb -o yaml
apiVersion: v1
kind: Pod
metadata:
..... #其他内容省略
containers:
- command:
- /usr/local/bin/kube-proxy
- --config=/var/lib/kube-proxy/config.conf
- --hostname-override=$(NODE_NAME)
..... #其他内容省略
volumeMounts:
- mountPath: /var/lib/kube-proxy
name: kube-proxy
..... #其他内容省略
volumes:
- configMap:
defaultMode: 420
name: kube-proxy
name: kube-proxy
]# kubectl get cm -n kube-system
NAME DATA AGE
coredns 1 5h18m
extension-apiserver-authentication 6 5h18m
kube-proxy 2 5h18m
kube-root-ca.crt 1 5h18m
kubeadm-config 2 5h18m
kubelet-config-1.20 1 5h18m
]# kubectl edit cm kube-proxy -n kube-system
#在编辑模式中找到以下字段,删除后保存退出
featureGates:
SupportIPVSProxyMode: true
然后将删除所有kube-proxy进行重启,查看pod运行情况
]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
... #其他内容省略
kube-proxy-24tkb 1/1 Running 0 122m
kube-proxy-9cl7j 1/1 Running 0 123m
kube-proxy-cxbg5 1/1 Running 0 123m
kube-proxy-cxgqk 1/1 Running 0 123m
kube-proxy-hkq54 1/1 Running 0 123m
kube-proxy-ttdqb 1/1 Running 0 122m
在服务器上安装ipvsadm,查看ipvs模式是否启用成功
]# yum install ipvsadm -y
]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.96.0.1:443 rr
-> 和谐ip:6443 Masq 1 0 0
-> 和谐ip:6443 Masq 1 0 0
-> 和谐ip:6443 Masq 1 0 0
TCP 10.96.0.10:53 rr
TCP 10.96.0.10:9153 rr
UDP 10.96.0.10:53 rr
