程序员经验分享-hwhale

Microk8s、MetalLB、ingress-nginx - 如何路由外部流量？

2024-04-13

Kubernetes/Ubuntu 新手来了！

我正在使用单个 Raspberry Pi 设置 k8s 集群（希望将来有更多）。我用的是microk8sv1.18.8和 Ubuntu 服务器20.04.1 LTS (GNU/Linux 5.4.0-1018-raspi aarch64).

我正在尝试访问端口上的一项 k8s 服务80，但我无法正确设置它。我还设置了用于访问该服务的静态 IP 地址，并将流量从路由器路由到该服务的 IP 地址。

我想知道我做错了什么，或者是否有更好的方法来完成我想做的事情！

我正在执行的步骤：

  1. 我已经跑了microk8s enable dns metallb。我给出的 MetalLB IP 地址未被 DHCP 服务器处理（192.168.0.90-192.168.0.99).
  2. 我已经安装了ingress-nginx通过跑步kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/baremetal/deploy.yaml。这创建了一个NodePort服务于ingress-nginx-controller，这不适用于 MetalLB。如上所述here https://medium.com/better-programming/how-to-expose-your-services-with-kubernetes-ingress-7f34eb6c9b5a，我编辑spec.type的服务来自NodePort to LoadBalancer通过跑步kubectl edit service ingress-nginx-controller -n ingress-nginx。 MetalLB然后分配IP192.168.0.90到服务。
  3. 然后我应用以下配置文件：
apiVersion: v1
kind: Service
metadata:
  name: wow-ah-api-service
  namespace: develop
spec:
  selector:
    app: wow-ah-api
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  # Unique key of the Deployment instance
  name: wow-ah-api
  namespace: develop
spec:
  # 3 Pods should exist at all times.
  replicas: 3
  selector:
    matchLabels:
      app: wow-ah-api
  template:
    metadata:
      namespace: develop
      labels:
        # Apply this label to pods and default
        # the Deployment label selector to this value
        app: wow-ah-api
    spec:
      imagePullSecrets:
        - name: some-secret
      containers:
        - name: wow-ah-api
          # Run this image
          image: some-image
          imagePullPolicy: Always
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: wow-ah-api-ingress
  namespace: develop
spec:
  backend:
    serviceName: wow-ah-api-service
    servicePort: 3000

这些是我看到的一些输出：

microk8s kubectl get all --all-namespaces

NAMESPACE        NAME                                            READY   STATUS      RESTARTS   AGE
develop          pod/wow-ah-api-6c4bff88f9-2x48v                 1/1     Running     4          4h21m
develop          pod/wow-ah-api-6c4bff88f9-ccw9z                 1/1     Running     4          4h21m
develop          pod/wow-ah-api-6c4bff88f9-rd6lp                 1/1     Running     4          4h21m
ingress-nginx    pod/ingress-nginx-admission-create-mnn8g        0/1     Completed   0          4h27m
ingress-nginx    pod/ingress-nginx-admission-patch-x5r6d         0/1     Completed   1          4h27m
ingress-nginx    pod/ingress-nginx-controller-7896b4fbd4-nglsd   1/1     Running     4          4h27m
kube-system      pod/coredns-588fd544bf-576x5                    1/1     Running     4          4h26m
metallb-system   pod/controller-5f98465b6b-hcj9g                 1/1     Running     4          4h23m
metallb-system   pod/speaker-qc9pc                               1/1     Running     4          4h23m

NAMESPACE       NAME                                         TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                      AGE
default         service/kubernetes                           ClusterIP      10.152.183.1     <none>         443/TCP                      21h
develop         service/wow-ah-api-service                   ClusterIP      10.152.183.88    <none>         80/TCP                       4h21m
ingress-nginx   service/ingress-nginx-controller             LoadBalancer   10.152.183.216   192.168.0.90   80:32151/TCP,443:30892/TCP   4h27m
ingress-nginx   service/ingress-nginx-controller-admission   ClusterIP      10.152.183.41    <none>         443/TCP                      4h27m
kube-system     service/kube-dns                             ClusterIP      10.152.183.10    <none>         53/UDP,53/TCP,9153/TCP       4h26m

NAMESPACE        NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
metallb-system   daemonset.apps/speaker   1         1         1       1            1           beta.kubernetes.io/os=linux   4h23m

NAMESPACE        NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
develop          deployment.apps/wow-ah-api                 3/3     3            3           4h21m
ingress-nginx    deployment.apps/ingress-nginx-controller   1/1     1            1           4h27m
kube-system      deployment.apps/coredns                    1/1     1            1           4h26m
metallb-system   deployment.apps/controller                 1/1     1            1           4h23m

NAMESPACE        NAME                                                  DESIRED   CURRENT   READY   AGE
develop          replicaset.apps/wow-ah-api-6c4bff88f9                 3         3         3       4h21m
ingress-nginx    replicaset.apps/ingress-nginx-controller-7896b4fbd4   1         1         1       4h27m
kube-system      replicaset.apps/coredns-588fd544bf                    1         1         1       4h26m
metallb-system   replicaset.apps/controller-5f98465b6b                 1         1         1       4h23m

NAMESPACE       NAME                                       COMPLETIONS   DURATION   AGE
ingress-nginx   job.batch/ingress-nginx-admission-create   1/1           27s        4h27m
ingress-nginx   job.batch/ingress-nginx-admission-patch    1/1           29s        4h27m

microk8s kubectl get ingress --all-namespaces

NAMESPACE   NAME                 CLASS    HOSTS   ADDRESS         PORTS   AGE
develop     wow-ah-api-ingress   <none>   *       192.168.0.236   80      4h23m

我一直认为这可能与我的 iptables 配置有关，但我不确定如何配置它们以与 microk8s 一起使用。

sudo iptables -L

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
KUBE-EXTERNAL-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere            

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
KUBE-FORWARD  all  --  anywhere             anywhere             /* kubernetes forwarding rules */
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
ACCEPT     all  --  10.1.0.0/16          anywhere             /* generated for MicroK8s pods */
ACCEPT     all  --  anywhere             10.1.0.0/16          /* generated for MicroK8s pods */
ACCEPT     all  --  10.1.0.0/16          anywhere            
ACCEPT     all  --  anywhere             10.1.0.0/16         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
KUBE-SERVICES  all  --  anywhere             anywhere             ctstate NEW /* kubernetes service portals */
KUBE-FIREWALL  all  --  anywhere             anywhere            

Chain KUBE-EXTERNAL-SERVICES (1 references)
target     prot opt source               destination         

Chain KUBE-FIREWALL (2 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
DROP       all  -- !localhost/8          localhost/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-FORWARD (1 references)
target     prot opt source               destination         
DROP       all  --  anywhere             anywhere             ctstate INVALID
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere             /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED

Chain KUBE-KUBELET-CANARY (0 references)
target     prot opt source               destination         

Chain KUBE-PROXY-CANARY (0 references)
target     prot opt source               destination         

Chain KUBE-SERVICES (3 references)
target     prot opt source               destination

更新#1

metallb ConfigMap (microk8s kubectl edit ConfigMap/config -n metallb-system)

apiVersion: v1
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 192.168.0.90-192.168.0.99
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"config":"address-pools:\n- name: default\n  protocol: layer2\n  addresses:\n  - 192.168.0.90-192.168.0.99\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"config","namespace":"metallb-system"}}
  creationTimestamp: "2020-09-19T21:18:45Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:config: {}
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
    manager: kubectl
    operation: Update
    time: "2020-09-19T21:18:45Z"
  name: config
  namespace: metallb-system
  resourceVersion: "133422"
  selfLink: /api/v1/namespaces/metallb-system/configmaps/config
  uid: 774f6a73-b1e1-4e26-ba73-ef71bc2e1060

如果您能给我任何帮助，我将不胜感激！


简短回答：

  1. 您只需要（并且可能拥有）一个 IP 地址。您必须可以从 Microk8s 机器外部对其执行 ping 操作。
  2. 这是错误。去掉这一步

长答案举例：

清洁 Microk8s。只有一个公共 IP（或本地计算机 IP。在您的用例中，我将使用 192.168.0.90）。

你如何测试？例如

curl -H "Host: blue.nginx.example.com" http://PUBLIC_IP

从机器外部。

运行测试。它一定会失败。

启用 microk8s dns 和 ingress

microk8s.enable dns ingress

运行测试。失败？

如果是同样的错误那么：你需要 metallb

  • 拥有互联网公网IP

    microk8s.启用金属：$(curl ipinfo.io/ip)-$(curl ipinfo.io/ip)

  • 局域网IP 192.168.0.90

    microk8s.启用金属：192.168.0.90-192.168.0.90

再次运行测试

如果测试未返回 503 或 404，则：您无法执行后续步骤。也许您有网络问题或防火墙过滤器。

入口层

我们的测试到达了 Microk8s Ingress 控制器。他不知道该怎么办并返回 404 错误（有时是 503）。

没关系。下一步！

我将使用一个例子https://youtu.be/A_PjjCM1eLA?t=984 https://youtu.be/A_PjjCM1eLA?t=984 16:24

[ Kube 32 ] 在 kubernetes 裸机集群上设置 Traefik Ingress

设置 kubectl 别名

alias kubectl=microk8s.kubectl
Deploy apps 
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/nginx-deploy-main.yaml
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/nginx-deploy-blue.yaml
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/nginx-deploy-green.yaml
Expose apps in the internal cluster network. ClusterIP by default. 
kubectl expose deploy nginx-deploy-main --port 80
kubectl expose deploy nginx-deploy-blue --port 80
kubectl expose deploy nginx-deploy-green --port 80

运行测试。它还不起作用...

Ingress rule example: how to delivery by host name

配置主机 nginx.example.com、blue.nginx.example.com 和 green.nginx.example.com 并将请求分发到公开的部署：

kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/ingress-resource-2.yaml

运行此测试：

curl -H "Host: blue.nginx.example.com" http://PUBLIC_IP

现在你会得到类似的回应

<h1>I am <font color=blue>BLUE</font></h1>

你可以玩

curl -H "Host: nginx.example.com" http://PUBLIC_IP
curl -H "Host: blue.nginx.example.com" http://PUBLIC_IP
curl -H "Host: green.nginx.example.com" http://PUBLIC_IP

结论：

  • 我们只有 1 个 IP 地址和多台主机。
  • 我们有 3 个不同的服务使用同一端口。
  • 请求分发是使用 Ingress 完成的。
本文内容由网友自发贡献，版权归原作者所有，本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容，请联系:hwhale#tublm.com(使用前将#替换为@)

Kubernetes

UbuntuServer

raspberrypi4

microk8s

metallb

Microk8s、MetalLB、ingress-nginx - 如何路由外部流量？ 的相关文章

随机推荐

热门标签