kubernetes:[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz‘ failed

[kubelet-check] The HTTP call equal to ‘curl -sSL http://localhost:10248/healthz’ failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.

背景：
搭建k8s高可用集群，由于自己瞎操作，在初始化master（master-1）时操作失败，于是又换了一个master（master-2）进行初始化，但是又由于其他问题导致失败，于是又换了一台master(master-3)进行初始化，终于成功。于是将前两台的master加入进来，出现以下报错：

[root@kube-master-2 ~]#  kubeadm join 192.168.186.100:6444 --token abcdef.0123456789abcdef     --discovery-token-ca-cert-hash sha256:96d22a2b958829a6c3039cc4c6f18a2fc8a508486dd9c1d9977d0ccca3b4eb95     --control-plane --certificate-key d8e791cb9b567cec3c2a6cb2f8a1ce197cf108fdfee3e6c8945eb20ebdbc8930
--snip--
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
[kubelet-check] It seems like the kubelet isn't running or healthy.
[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get http://localhost:10248/healthz: dial tcp [::1]:10248: connect: connection refused.
^C

大概的原因应该是由于之前操作初始化的时候导致了环境不干净，造成后面加入集群时出现了问题。

解决：
[root@kube-master-2 ~]# kubeadm reset
[root@kube-master-2 ~]# rm -rf /etc/cni/net.d
[root@kube-master-2 ~]# rm -rf $HOME/.kube/config
[root@kube-master-2 ~]# rm -rf /etc/kubernetes/
然后再次执行加入集群命令

[root@kube-master-1 ~]# kubeadm join 192.168.186.100:6444 --token abcdef.0123456789abcdef     --discovery-token-ca-cert-hash sha256:96d22a2b958829a6c3039cc4c6f18a2fc8a508486dd9c1d9977d0ccca3b4eb95     --control-plane --certificate-key d8e791cb9b567cec3c2a6cb2f8a1ce197cf108fdfee3e6c8945eb20ebdbc8930
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[preflight] Running pre-flight checks before initializing the new control plane instance
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[download-certs] Downloading the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
error execution phase control-plane-prepare/download-certs: error downloading certs: error downloading the secret: Secret "kubeadm-certs" was not found in the "kube-system" Namespace. This Secret might have expired. Please, run `kubeadm init phase upload-certs --upload-certs` on a control plane to generate a new one
To see the stack trace of this error execute with --v=5 or higher

如果出现以上错误，就需要去初始化成功的master重新生成验证消息

[root@kube-master-3 k8s]# kubeadm init phase upload-certs --upload-certs
I0304 15:16:59.221639   95329 version.go:252] remote version is much newer: v1.20.4; falling back to: stable-1.18
W0304 15:17:00.282978   95329 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[upload-certs] Storing the certificates in Secret "kubeadm-certs" in the "kube-system" Namespace
[upload-certs] Using certificate key:
0db707eb72d2282b642cc2f6fe057ae3963cf380a351cc932bfcd424963ce837

然后替换刚刚的验证消息继续执行命令即可。

以上解决问题的过程有点粗糙，估计适用范围不大，可以多多交流。