Istio 直接 Pod 到 Pod 通信

我在使用 Istio 部署的 Pod 与 Pod 进行通信时遇到问题？我实际上需要它与 Istio 合作进行 Hazelcast 发现 https://github.com/hazelcast/hazelcast-kubernetes/issues/118，但我会尝试在这里概括这个问题。

让我们在 Kubernetes 上部署一个示例 hello world 服务。该服务在端口 8000 上回复 HTTP 请求。

$ kubectl create deployment nginx --image=crccheck/hello-world

创建的 Pod 分配了一个内部 IP：

$ kubectl get pods -o wide
NAME                               READY   STATUS    RESTARTS   AGE   IP           NODE                                                  NOMINATED NODE
hello-deployment-84d876dfd-s6r5w   1/1     Running   0          8m    10.20.3.32   gke-rafal-test-istio-1-0-default-pool-91f437a3-cf5d   <none>

在工作curl.yaml，我们可以直接使用Pod IP。

apiVersion: batch/v1
kind: Job
metadata:
  name: curl
spec:
  template:
    spec:
      containers:
      - name: curl
        image: byrnedo/alpine-curl
        command: ["curl",  "10.20.3.32:8000"]
      restartPolicy: Never
  backoffLimit: 4

在没有 Istio 的情况下运行作业效果很好。

$ kubectl apply -f curl.yaml
$ kubectl logs pod/curl-pptlm
...
Hello World
...

但是，当我尝试对 Istio 执行相同操作时，它不起作用。 HTTP 请求被 Envoy 阻止。

$ kubectl apply -f <(istioctl kube-inject -f curl.yaml)
$ kubectl logs pod/curl-s2bj6 curl
...
curl: (7) Failed to connect to 10.20.3.32 port 8000: Connection refused

我尝试过服务条目、MESH_INTERNAL 和 MESH_EXTERNAL，但没有成功。如何绕过 Envoy 直接调用 Pod？

编辑：输出istioctl kube-inject -f curl.yaml.

$ istioctl kube-inject -f curl.yaml
apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  name: curl
spec:
  backoffLimit: 4
  template:
    metadata:
      annotations:
        sidecar.istio.io/status: '{"version":"dbf2d95ff300e5043b4032ed912ac004974947cdd058b08bade744c15916ba6a","initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["istio-envoy","istio-certs"],"imagePullSecrets":null}'
      creationTimestamp: null
    spec:
      containers:
      - command:
        - curl
        - 10.16.2.34:8000/
        image: byrnedo/alpine-curl
        name: curl
        resources: {}
      - args:
        - proxy
        - sidecar
        - --domain
        - $(POD_NAMESPACE).svc.cluster.local
        - --configPath
        - /etc/istio/proxy
        - --binaryPath
        - /usr/local/bin/envoy
        - --serviceCluster
        - curl.default
        - --drainDuration
        - 45s
        - --parentShutdownDuration
        - 1m0s
        - --discoveryAddress
        - istio-pilot.istio-system:15010
        - --zipkinAddress
        - zipkin.istio-system:9411
        - --connectTimeout
        - 10s
        - --proxyAdminPort
        - "15000"
        - --concurrency
        - "2"
        - --controlPlaneAuthPolicy
        - NONE
        - --statusPort
        - "15020"
        - --applicationPorts
        - ""
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: INSTANCE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: ISTIO_META_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: ISTIO_META_CONFIG_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: ISTIO_META_INTERCEPTION_MODE
          value: REDIRECT
        image: docker.io/istio/proxyv2:1.1.1
        imagePullPolicy: IfNotPresent
        name: istio-proxy
        ports:
        - containerPort: 15090
          name: http-envoy-prom
          protocol: TCP
        readinessProbe:
          failureThreshold: 30
          httpGet:
            path: /healthz/ready
            port: 15020
          initialDelaySeconds: 1
          periodSeconds: 2
        resources:
          limits:
            cpu: "2"
            memory: 128Mi
          requests:
            cpu: 100m
            memory: 128Mi
        securityContext:
          readOnlyRootFilesystem: true
          runAsUser: 1337
        volumeMounts:
        - mountPath: /etc/istio/proxy
          name: istio-envoy
        - mountPath: /etc/certs/
          name: istio-certs
          readOnly: true
      initContainers:
      - args:
        - -p
        - "15001"
        - -u
        - "1337"
        - -m
        - REDIRECT
        - -i
        - '*'
        - -x
        - ""
        - -b
        - ""
        - -d
        - "15020"
        image: docker.io/istio/proxy_init:1.1.1
        imagePullPolicy: IfNotPresent
        name: istio-init
        resources:
          limits:
            cpu: 100m
            memory: 50Mi
          requests:
            cpu: 10m
            memory: 10Mi
        securityContext:
          capabilities:
            add:
            - NET_ADMIN
      restartPolicy: Never
      volumes:
      - emptyDir:
          medium: Memory
        name: istio-envoy
      - name: istio-certs
        secret:
          optional: true
          secretName: istio.default
status: {}
---

当带有 istio sidecar 的 pod 启动时，会发生以下情况

1. init 容器会更改 iptables 规则，以便所有传出 tcp 流量都路由到端口 15001 上的 sidecar 容器 (istio-proxy)。

2. Pod 的容器并行启动（curl and istio-proxy)

如果你的curl容器之前执行过istio-proxy监听端口 15001，您会收到错误。

我使用 sleep 命令启动了这个容器，在容器中执行 exec-d，curl 工作了。

$ kubectl apply -f <(istioctl kube-inject -f curl-pod.yaml)

$ k exec -it -n noistio curl -c curl bash
[root@curl /]# curl 172.16.249.198:8000
<xmp>
Hello World


                                       ##         .
                                 ## ## ##        ==
                              ## ## ## ## ##    ===
                           /""""""""""""""""\___/ ===
                      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
                           \______ o          _,/
                            \      \       _,'
                             `'--.._\..--''
</xmp>
[root@curl /]# 

卷曲pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: curl
spec:
  containers:
  - name: curl
    image: centos
    command: ["sleep",  "3600"]