如何为 Kubernetes 中的 Flink 应用程序自定义资源提供 Vault 密钥

我想为 Kubernetes 集群中运行的 Apache Flink 作业提供来自 Hashicorp Vault 的机密。 这些积分将用于访问状态后端以进行检查点和保存点。例如，状态后端可以是 Minio S3 存储。 有人可以提供一个工作示例吗Flink应用算子 https://github.com/lyft/flinkk8soperator/blob/master/docs/crd.md请给出以下设置？

用户名和密码（或访问密钥）的保管库机密：

vault kv put vvp/storage/config username=user password=secret
vault kv put vvp/storage/config access-key=minio secret-key=minio123

Flink 应用程序自定义资源的 k8s 清单：

apiVersion: flink.k8s.io/v1beta1
kind: FlinkApplication
metadata:
  name: processor
  namespace: default
spec:
  image: stream-processor:0.1.0
  deleteMode: None

  template:
    metadata:
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: vvp-flink-job
        vault.hashicorp.com/agent-inject-secret-storage-config.txt: vvp/data/storage/config

  flinkConfig:
    taskmanager.memory.flink.size: 1024mb
    taskmanager.heap.size: 200
    taskmanager.network.memory.fraction: 0.1
    taskmanager.network.memory.min: 10mb
    web.upload.dir: /opt/flink
  jobManagerConfig:
    resources:
      requests:
        memory: "1280Mi"
        cpu: "0.1"
    replicas: 1
  taskManagerConfig:
    taskSlots: 2
    resources:
      requests:
        memory: "1280Mi"
        cpu: "0.1"
  flinkVersion: "1.14.2"
  jarName: "stream-processor-1.0-SNAPSHOT.jar"
  parallelism: 3
  entryClass: "org.StreamingJob"
  programArgs: >
    --name value

flink应用程序的Docker文件：

FROM maven:3.8.4-jdk-11 AS build
ARG revision

WORKDIR /
COPY    src /src
COPY    pom.xml /

RUN mvn -B -Drevision=${revision} package

# runtime
FROM flink:1.14.2-scala_2.12-java11

ENV FLINK_HOME=/opt/flink

ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 6123 8081
CMD ["help"]

flink-config.yaml 包含以下示例：

# state.backend: filesystem

# Directory for checkpoints filesystem, when using any of the default bundled
# state backends.
#
# state.checkpoints.dir: hdfs://namenode-host:port/flink-checkpoints

# Default target directory for savepoints, optional.
#
# state.savepoints.dir: hdfs://namenode-host:port/flink-savepoints

最终目标是替换硬编码的秘密或以某种方式从保险库中设置它们：

state.backend: filesystem
s3.endpoint: http://minio:9000
s3.path.style.access: true
s3.access-key: minio
s3.secret-key: minio123

谢谢。

设置 Vault 变量后

可以在部署中添加注释，将变量从Vault中取出到部署中

annotations:
        vault.hashicorp.com/agent-image: <Agent image>
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-inject-secret-secrets: kv/<Path-of-secret>
        vault.hashicorp.com/agent-inject-template-secrets: |2

          {{- with secret "kv/<Path-of-secret>" -}}

          #!/bin/sh
          set -e

          {{- range $key, $value := .Data.data }}
          export {{ $key }}={{ $value }}
          {{- end }}

          exec "$@"
          {{- end }}
        vault.hashicorp.com/auth-path: auth/<K8s cluster for auth>
        vault.hashicorp.com/role: app

这将在您的 POD 中创建该文件。

当你的应用程序运行时，它应该首先执行这个文件，环境变量将被注入到 POD 中。

因此，Vault 注释将创建一个与您获得的文件相同的文件txt但相反，我们会这样做

{{- range $key, $value := .Data.data }}
       export {{ $key }}={{ $value }}
{{- end }}

它将保留并注入export在键和值之前。现在该文件是一种 shell 脚本，一旦它在应用程序启动时执行，它将向OS level.

将此文件保留在 repo 中并将其添加到 Docker 中./bin/runapp

#!/bin/bash
if [ -f '/vault/secrets/secrets' ]; then
  source '/vault/secrets/secrets'
fi
node <path-insnide-docker>/index.js #Sorry dont know scala or Java

包.json

"start": "./bin/runapp",

Dockerfile

ADD ./bin/runapp ./
EXPOSE 4444
CMD ["npm", "start"]

您的保险库注入文件将类似于 pod 内的内容/vault/secrets/secrets或您配置的路径。

#!/bin/sh
set -e
export development=false
export production=true
exec "$@"