kubesphere & K8S & Kubeedge 安装配置菜鸟级指导

kubesphere & K8S & Kubeedge

安装配置：一键安装

kubeKey安装 kubesphere&kubernertest:

curl -sfL https://get-kk.kubesphere.io | VERSION=v2.2.1 sh -
chmod +x kk
apt-get update 
apt install conntrack && apt install socat
./kk create cluster --with-kubernetes v1.22.0 --with-kubesphere v3.3.0

PS. 目前：用kk工具管理集群是最方便的；

K8S集群直接创建网络方面有问题，storageClass没有办法配置，最方便的方法还是kk管理集群

#从已有集群创建文件
./kk create config --from-cluster
./kk create config -f ~/cluster_sed.yam
#目前最优的方案是先创建集群 && add 节点
./kk create cluster --with-kubernetes v1.22.0 --with-kubesphere v3.3.0 && ./kk add nodes -f cluster_set.yaml
#检查集群初始化是否完成
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

#删除集群
./kk delete cluster

#！/bin/cluster_set.yaml
apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  ##You should complete the ssh information of the hosts
  - {name: 4core-01, address: 158.39.74.172, internalAddress: 158.39.74.172}
  roleGroups:
    etcd:
    - 4core-01
    master:
    - 4core-01
    worker:
    - 4core-02
  controlPlaneEndpoint:
    ##Internal loadbalancer for apiservers
    #internalLoadbalancer: haproxy

    ##If the external loadbalancer was used, 'address' should be set to loadbalancer's ip.
    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.22.0
    clusterName: cluster.local
    proxyMode: ipvs
    masqueradeAll: false
    maxPods: 110
    nodeCidrMaskSize: 24
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
  registry:
    privateRegistry: ""

把公钥写入authoca文件中，集群会默认接通

ssh-keygen -t rsa
#生成密钥，私钥放在本地，公钥写进autho文件中

配置各种应用：

kubectl edit cc -n kubesphere-system
. opentrix
. kubeedge
. metrics-service
. dev

错误排查：(主要问题多是资源不足)

        "1. check the storage configuration and storage server",
        "2. make sure the DNS address in /etc/resolv.conf is available",
        "3. execute 'kubectl logs -n kubesphere-system -l job-name=minio-make-bucket-job' to watch logs",
        "4. execute 'helm -n kubesphere-system uninstall ks-minio && kubectl -n kubesphere-system delete job minio-make-bucket-job'",
        "5. Restart the installer pod in kubesphere-system namespace"

部署kubeedge

开启kubeedge选项 -> 加入节点

端口转发规则配置：

#iptable要设置在edge host上，将本地10002 port 转发到cloudport的 ：30002port上
iptables -t nat -A OUTPUT -p tcp --dport 10000 -j DNAT --to $CLOUDCOREIPS:30000
iptables -t nat -A OUTPUT -p tcp --dport 10001 -j DNAT --to $CLOUDCOREIPS:30001
iptables -t nat -A OUTPUT -p tcp --dport 10002 -j DNAT --to $CLOUDCOREIPS:30002
iptables -t nat -A OUTPUT -p tcp --dport 10350 -j DNAT --to $CLOUDCOREIPS:10003
iptables -t nat -A OUTPUT -p tcp --dport 10004 -j DNAT --to $CLOUDCOREIPS:30004

arch=$(uname -m); if [[ $arch != x86_64 ]]; then arch='arm64'; fi;  curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.9.2/$arch/keadm-v1.9.2-linux-$arch.tar.gz  && tar xvf keadm-v1.9.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.9.2 --region=zh --cloudcore-ipport=158.39.74.211:30000 --quicport 30001 --certport 30002 --tunnelport 30004 --edgenode-name 2core-00 --edgenode-ip 158.39.201.145 --token 8746be082ce99bbc8853f7f7c3f72e0caad20d6bc170a1de607f1e92e1bff682.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2NTk1MTU3MjR9.jhtxb6_42XhQztFgvhjudyRcWbSknAR1E2gswjDIaEI --with-edge-taint

踩坑：

1. 要使得边缘节点能够被master节点监控，要修改edge节点的edgecore.service参数

注意！ ipstable端口转发如果没开启规则，则要修改server端口号为30004否则监听不到

#vim /etc/kubeedge/config/edgecore.yaml
edgeStream:
enable: true #将“false”更改为“true”。
handshakeTimeout: 30
readDeadline: 15
server: xx.xxx.xxx.xxx:10004 #如果没有添加端口转发，将端口修改为30004。
tlsTunnelCAFile: /etc/kubeedge/ca/rootCA.crt
tlsTunnelCertFile: /etc/kubeedge/certs/server.crt
tlsTunnelPrivateKeyFile: /etc/kubeedge/certs/server.key
writeDeadline: 15

edged:
    cgroupDriver: cgroupfs
    cgroupRoot: ""
    cgroupsPerQOS: true
    clusterDNS: "169.254.96.16"
    clusterDomain: "cluster.local"
    cniBinDir: /opt/cni/bin
    cniCacheDirs: /var/lib/cni/cache
    cniConfDir: /etc/cni/net.d
    concurrentConsumers: 5
    devicePluginEnabled: false
    dockerAddress: unix:///var/run/docker.sock
    edgedMemoryCapacity: 7852396000
    enable: true
    enableMetrics: true

systemctl restart edgecore.service

2. 尚不清楚 cfgroup是否会对系统构成影响

sudo tee /etc/docker/daemon.json <<EOF 
{
"exec-opts": ["native.cgroupdriver=cgroupfs"],
"registry-mirrors":["http://hub-mirror.c.163.com"]
}

EOF

systemctl daemon-reload

service docker restart 

docker info | grep -i cgroup

3. Edgemesh 配置

   #edmesh 可以用商店部署，注意设置tolerations
   #agent
     tolerations: 
       - key: "node-role.kubernetes.io/edge"
         operator: "Exists"
         effect: "NoSchedule"
   

问题记录：

edgemesh网络链接有问题，adress找不到？ woker节点上的agen倒是没问题

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-iXGcKG9S-1660405035532)(https://s2.loli.net/2022/08/04/t3X2OKkq4melA9N.png)]