docker-compose部署emqx集群配置带mysql授权认证

2023-05-16

EMQX 是一款大规模可弹性伸缩的云原生分布式物联网 MQTT 消息服务器。

作为全球最具扩展性的 MQTT 消息服务器，EMQX 提供了高效可靠海量物联网设备连接，能够高性能实时移动与处理消息和事件流数据，帮助您快速构建关键业务的物联网平台与应用。

产品详细介绍：产品概览 | EMQX 5.0 文档https://www.emqx.io/docs/zh/v5.0/

前言

EMQX 是目前全球市场广泛应用的百万级开源 MQTT 消息服务器，全球市场(西欧、北美、印度、中国)累积超 5000 家企业用户，产品环境下部署超 1 万节点，累计下载量超过 50 万，承载 MQTT 连接超 3000 万线

EMQ X 可以作为智能硬件、智能家居、物联网、车联网应用的百万级设备接入平台

基础配置就不介绍了，去官方看详细的配置，本文主要介绍下通过docker-compose的方式部署

docker已安装
docker-compose已安装

一、单机集群（伪集群）

创建 docker-compose.yaml

version: '3'

services:
  emqx1:
    image: emqx/emqx:4.3.10
    restart: always
    container_name: emqx1-borker1
    deploy:
      resources:
        limits:
          cpus: "0.5"
          memory: 1G
        reservations:
          memory: 1G
    environment:
    - "EMQX_NAME=emqx"
    - "EMQX_HOST=node1.emqx.io"
    - "EMQX_CLUSTER__DISCOVERY=static"
    - "EMQX_CLUSTER__STATIC__SEEDS=emqx@node1.emqx.io, emqx@node2.emqx.io"
    - "EMQX_ALLOW_ANONYMOUS=false"
    - "EMQX_ACL_NOMATCH=deny"
    - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid"
    - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true"
    - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s"
    - "EMQX_AUTH__MYSQL__SERVER=192.168.1.1:3306"
    - "EMQX_AUTH__MYSQL__POOL=8"
    - "EMQX_AUTH__MYSQL__USERNAME=root"
    - "EMQX_AUTH__MYSQL__PASSWORD=root"
    - "EMQX_AUTH__MYSQL__DATABASE=emqx"
    - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=5s"
    - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256"
    - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password as password from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'"
    volumes:
    - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z
    ports:
      - 1883:1883
      - 8081:8081
      - 8083:8083
      - 8084:8084
      - 8883:8883
      - 18083:18083
    networks:
      emqx-bridge:
        aliases:
        - node1.emqx.io

  emqx2:
    image: emqx/emqx:4.3.10
    environment:
    - "EMQX_NAME=emqx"
    - "EMQX_HOST=node2.emqx.io"
    - "EMQX_CLUSTER__DISCOVERY=static"
    - "EMQX_CLUSTER__STATIC__SEEDS=emqx@node1.emqx.io, emqx@node2.emqx.io"
    - "EMQX_ALLOW_ANONYMOUS=false"
    - "EMQX_ACL_NOMATCH=deny"
    - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid"
    - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true"
    - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s"
    - "EMQX_AUTH__MYSQL__SERVER=192.168.1.1:3306"
    - "EMQX_AUTH__MYSQL__POOL=8"
    - "EMQX_AUTH__MYSQL__USERNAME=root"
    - "EMQX_AUTH__MYSQL__PASSWORD=root"
    - "EMQX_AUTH__MYSQL__DATABASE=emqx"
    - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=15s"
    - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256"
    - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password_hash as password from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u'"
    - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'"
    volumes:
    - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z
    ports:
      - 1884:1883
    networks:
      emqx-bridge:
        aliases:
        - node2.emqx.io

networks:
  emqx-bridge:
    driver: bridge

loaded_plugins文件：用于替换插件启用配置（启用`emqx_auth_mysql`插件）

{emqx_management, true}.
{emqx_recon, true}.
{emqx_retainer, true}.
{emqx_dashboard, true}.
{emqx_telemetry, true}.
{emqx_rule_engine, true}.
{emqx_bridge_mqtt, false}.
{emqx_auth_mysql, true}.

mysql授权认证表：

CREATE TABLE `iot_mqtt_acl` (
  `id` int NOT NULL AUTO_INCREMENT COMMENT '主键',
  `username` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt用户名',
  `topic` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'Topic Filter',
  `access` int DEFAULT NULL COMMENT '访问权限 1: subscribe, 2: publish, 3: pubsub',
  `create_time` datetime DEFAULT NULL COMMENT '创建日期',
  `del_flag` int DEFAULT '0' COMMENT '是否删除 0未删除 1删除',
  PRIMARY KEY (`id`) USING BTREE,
  KEY `idx_username` (`username`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC COMMENT='mqtt的ACL规则表';

CREATE TABLE `iot_mqtt_user` (
  `id` int NOT NULL AUTO_INCREMENT COMMENT '主键',
  `username` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt用户名',
  `password` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt加密的密码',
  `login_pwd` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT 'mqtt明文的密码',
  `is_superuser` tinyint(1) DEFAULT NULL COMMENT '是否超级用户(1-是,0-否)',
  `create_by` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL COMMENT '创建人',
  `create_time` datetime DEFAULT NULL COMMENT '创建日期',
  `update_by` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL COMMENT '更新人',
  `update_time` datetime DEFAULT NULL COMMENT '更新日期',
  `del_flag` int DEFAULT '0' COMMENT '是否删除 0未删除 1删除',
  PRIMARY KEY (`id`) USING BTREE,
  KEY `idx_username` (`username`) USING BTREE,
  KEY `idx_sn` (`sn`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC COMMENT='mqtt用户表';

关于日志挂载

可以在yaml文件中配置 ./logs:/opt/emqx/log，说明我们把日志挂载出来了
但是这里有个问题：容器中使用的并不是root用户，如果我们用root用户启动容器，则docker在宿主机上创建的logs目录的属主是 root，那么容器中的服务会没有权限写，有如下两个解决方案：

给logs目录777 权限（不推荐）
修改logs目录属主（推荐）
一般容器中的用户会映射到宿主机中用户ID是 1001这个用户。为了保险我们可以自己实测：
1）创建logs目录给777 权限
2）启动服务，查看容器在logs目录下创建的文件属主（比如看到是用户A）
3）停止容器，删除logs目录
4）重新创建logs目录，将主、组都修改为用户A

启动： docker-compose up -d

注意：最开始没有在docker-compose文件中没有配置限制内容大小等，后面加的；如果使用docker-compose up会有问题(docker-compose 默认的有deploy配置)，可以使用这个命令启动:

docker-compose不支持deploy节点配置启动命令:
docker-compose --compatibility up -d

查看启动的容器：

web访问
URL： http://IP:18083

emqx如果不需要授权的可以干掉授权部分的配置(参考：使用 MySQL 的密码认证 | EMQX 5.0 文档)

二、多态机子集群部署

1. 环境说明（IP是阿里云内网IP，这里只是展示，请自行切换）

节点	IP
emqx01	192.168.1.1
emqx02	192.168.1.2

emqx集群部署

2.1 emqx01

网络需要使用host模式

docker-compose.yml 文件

version: '3'

services:
  emqx1:
    image: emqx/emqx:4.3.10
    restart: always
    container_name: emqx1-borker
    deploy:
      resources:
        limits:
          cpus: "0.5"
          memory: 1G
        reservations:
          memory: 1G
    environment:
    - "EMQX_NAME=emqx"
    - "EMQX_HOST=192.168.1.1"
    - "EMQX_NODE_NMAE=emqx@192.168.1.1"
    - "EMQX_CLUSTER__DISCOVERY=static"
    - "EMQX_CLUSTER__STATIC__SEEDS=emqx@192.168.1.1,emqx@192.168.1.2"
    - "EMQX_CLUSTER_AUTOHEAL=on"
    - "EMQX_ALLOW_ANONYMOUS=false" 
    - "EMQX_ACL_NOMATCH=deny"
    - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid"
    - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true"
    - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s"
    - "EMQX_AUTH__MYSQL__SERVER=域名/IP:3306"
    - "EMQX_AUTH__MYSQL__POOL=8"
    - "EMQX_AUTH__MYSQL__USERNAME=root"
    - "EMQX_AUTH__MYSQL__PASSWORD=xxxxx"
    - "EMQX_AUTH__MYSQL__DATABASE=emqx"
    - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=15s"
    - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256"
    - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password as password from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'"
    network_mode: host
    volumes:
    - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z
    healthcheck:
      test: ["CMD", "/opt/emqx/bin/emqx_ctl", "status"]
      interval: 5s
      timeout: 25s
      retries: 5

启动

[root@XXXXXX emqx01]# docker-compose --compatibility up -d

查看结果

[root@xxxx emqx01]# docker-compose ps
     Name                   Command                  State       Ports
----------------------------------------------------------------------
emqx1_broker   /usr/bin/docker-entrypoint ...   Up (healthy)

2.2 emqx02

网络需要使用host模式

docker-compose.yml 文件

version: '3'

services:
  emqx1:
    image: emqx/emqx:4.3.10
    restart: always
    container_name: emqx2-borker
    deploy:
      resources:
        limits:
          cpus: "0.5"
          memory: 1G
        reservations:
          memory: 1G
    environment:
    - "EMQX_NAME=emqx"
    - "EMQX_HOST=192.168.1.1"
    - "EMQX_NODE_NMAE=emqx@192.168.1.2"
    - "EMQX_CLUSTER__DISCOVERY=static"
    - "EMQX_CLUSTER__STATIC__SEEDS=emqx@192.168.1.1,emqx@192.168.1.2"
    - "EMQX_CLUSTER_AUTOHEAL=on"
    - "EMQX_ALLOW_ANONYMOUS=false" 
    - "EMQX_ACL_NOMATCH=deny"
    - "EMQX_BROKER__SHARED_SUBSCRIPTION_STRATEGY=hash_clientid"
    - "EMQX_BROKER__SHARED_DISPATCH_ACK_ENABLED=true"
    - "EMQX_ZONE__EXTERNAL__IDLE_TIMEOUT=60s"
    - "EMQX_AUTH__MYSQL__SERVER=域名/IP:3306"
    - "EMQX_AUTH__MYSQL__POOL=8"
    - "EMQX_AUTH__MYSQL__USERNAME=root"
    - "EMQX_AUTH__MYSQL__PASSWORD=xxxxx"
    - "EMQX_AUTH__MYSQL__DATABASE=emqx"
    - "EMQX_AUTH__MYSQL__QUERY_TIMEOUT=15s"
    - "EMQX_AUTH__MYSQL__PASSWORD_HASH=sha256"
    - "EMQX_AUTH__MYSQL__AUTH_QUERY=select password as password from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__SUPER_QUERY=SELECT is_superuser from mqtt_user where username = '%u' limit 1"
    - "EMQX_AUTH__MYSQL__ACL_QUERY=select 0 as allow, null as ipaddr, '%u' as username, null as clientid, 3 as access, '/#' as topic union select 1 as allow, null as ipaddr, username, null as clientid, access, topic from mqtt_acl WHERE username = '%u'"
    network_mode: host
    volumes:
    - /usr/share/zoneinfo/Asia/Shanghai:/etc/localtime
    - ./loaded_plugins:/opt/emqx/data/loaded_plugins:z
    healthcheck:
      test: ["CMD", "/opt/emqx/bin/emqx_ctl", "status"]
      interval: 5s
      timeout: 25s
      retries: 5

启动

[root@XXXXXX emqx02]# docker-compose --compatibility up -d

查看结果

[root@xxxx emqx02]#  docker-compose ps
     Name                   Command                  State       Ports
----------------------------------------------------------------------
emqx2_broker   /usr/bin/docker-entrypoint ...   Up (healthy)

WEB查看结果：

IP1:18083

IP2:18083

总结

提示：集群和集群后同一个订阅消息花了几天时间，最后才发现是阿里云的内网端口的问题，明明是同一个地区，同一个专有网络，结果发现内网端口不通；最后才知道因为两台服务器安全组不一样。

进入容器查看集群状态： docker exec -it emqx1-borker sh -c "emqx_ctl cluster status"

同一个集群下的客户端，订阅相同主题缺无法收到消息：

4.0+版本，检查5369端口开放

集群方式介绍：

设置集群方式（manual） 手动集群方式是 EMQ X 的默认集群方式。
请在配置文件emqx/etc/emqx.conf中找到以下行并做相应修改：
cluster.discovery = manual 
#-manual：手动加入命令 
#-static：静态节点列表 
#-mcast：IP组播 
#-dns：DNS自动集群 
#-etcd：etcd自动集群 
#-k8s：Kubernetes

最开始集群未成功：还是端口的问题，导致我进容器手动添加：不推荐使用

$ ./bin/emqx_ctl cluster join emqx@s1.emqx.io
Join the cluster successfully.
Cluster status: [{running_nodes,['emqx@s1.emqx.io','emqx@s2.emqx.io']}]

好了：以上就是今天要讲的内容，记录下自己踩过的坑

EMQX产品优势：

开放源码：基于 Apache 2.0 许可证完全开源，自 2013 年起 200+ 开源版本迭代。
MQTT 5.0：100% 支持 MQTT 5.0 和 3.x 协议标准，更好的伸缩性、安全性和可靠性。
海量连接：单节点支持 500 万 MQTT 设备连接，集群可扩展至 1 亿并发 MQTT 连接。
高性能：单节点支持每秒实时接收、移动、处理与分发数百万条的 MQTT 消息。
低时延：基于 Erlang/OTP 软实时的运行时系统设计，消息分发与投递时延低于 1 毫秒。
高可用：采用 Masterless 的大规模分布式集群架构，实现系统高可用和水平扩展

本文内容由网友自发贡献，版权归原作者所有，本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容，请联系:hwhale#tublm.com(使用前将#替换为@)