使用 NLB 和 IAM 身份验证公开 AWS MSK - 主机名验证失败

我们正在努力让 Amazon MSK (Kafka) 与 IAM 身份验证配合使用，然后使用 aws kafka 广告侦听器中的更改通过 DNS 公开访问它。为了实现这一点，我们遵循如下相同的基础设施计划，但我们不使用接口端点，而是使用网络负载平衡器：https://aws.amazon.com/blogs/big-data/how-goldman-sachs-builds-cross-account-connectivity-to-their-amazon-msk-clusters-with-aws-privatelink/

有趣的是，相同的基础设施使用 SASL/SCRAM 身份验证可以完美运行，但不能使用 IAM 身份验证。您是否有关于可公开访问的 AWS MSK 和 IAM 身份验证问题的任何信息？

基本上，我们遵循指南中的想法，特别是模式 2：在所有 MSK 代理面前使用单个共享接口端点，但使用 IAM 身份验证 使用您的 AWS MSK IAM 指南，我们已使用内部 dns 代理地址成功与我们的代理进行通信。当我们稍后根据上面的指南更改广告侦听器时，我们无法与代理通信并收到错误消息：

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SaslAuthenticationException: [9d5b944c-df83-4573-9979-4d121f49a533]: Hostname verification failed
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:104)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
at kafka.admin.ConfigCommand$.getResourceConfig(ConfigCommand.scala:552)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4(ConfigCommand.scala:512)
at kafka.admin.ConfigCommand$.$anonfun$describeResourceConfig$4$adapted(ConfigCommand.scala:504)
at scala.collection.immutable.List.foreach(List.scala:431)
at kafka.admin.ConfigCommand$.describeResourceConfig(ConfigCommand.scala:504)
at kafka.admin.ConfigCommand$.describeConfig(ConfigCommand.scala:484)
at kafka.admin.ConfigCommand$.processCommand(ConfigCommand.scala:304)
at kafka.admin.ConfigCommand$.main(ConfigCommand.scala:97)
at kafka.admin.ConfigCommand.main(ConfigCommand.scala)
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: [9d5b944c-df83-4573-9979-4d121f49a533]: Hostname verification failed

所以事实证明这是不受支持的，以下是来自 AWS 支持的消息：

Dear Customer,
 
Thank you for you patience while I investigate this issue.
 
After going through our internal resources, I would like to inform you that unfortunately IAM authentication against cluster using a custom domain name through intermediate NLB is not supported as of now. 
 
Also, I could confirm that there is an already existing feature request for this and it is indeed in the backlog of our MSK service team. As you may understand, any new functionality addition goes through regressive testing and analysis to determine feasibility and ensure the stability of the service. It is for this reason that we cannot provide a timeline on when this feature would be available. I sincerely apologise on behalf of AWS for the inconvenience caused. I appreciate your understanding and patience with us as we grow the service. 
 
In the meantime, I would suggest you to keep an eye on our What's New page[1] and AWS Blogs[2] for updates on the latest announcements.
 
In case you require any further assistance kindly feel free to reach out to me and I will be happy to assist you with the same.
 
Stay safe and Have a nice day!