一、Mysql-mmm集群技术概述；
概述：MMM（Master-Master replication manager for MySQL）是一套支持双主故障切换和双主日常管理的脚本程序。MMM使用Perl语言开发，主要用来监控和管理MySQL Master-Master（双主）复制，可以说是mysql主主复制管理器。
虽然叫做双主复制，但是业务上同一时刻只允许对一个主进行写入，另一台备选主上提供部分读服务，以加速在主主切换时刻备选主的预热，可以说MMM这套脚本程序一方面实现了故障切换的功能，另一方面其内部附加的工具脚本也可以实现多个slave的read负载均衡。
MMM提供了自动和手动两种方式移除一组服务器中复制延迟较高的服务器的虚拟ip，同时它还可以备份数据，实现两节点之间的数据同步等。由于MMM无法完全的保证数据一致性，所以MMM适用于对数据的一致性要求不是很高，但是又想最大程度的保证业务可用性的场景。
二、Mysql-mmm优缺点；
优点：高可用性，扩展性好，出现故障自动切换，对于主主同步，在同一时间内只提供一台数据库写操作，保证数据的一致性。
缺点：Monitor节点是单点，可以结合Keepalived实现高可用。
三、Mysql-mmm内部工作架构；
进程类型：
mmm_mond：监控进程，负责所有的监控工作，决定和处理所有节点角色活动。此脚本需要在监管机上运行；
mmm_agentd：运行在每个mysql服务器上（Master和Slave）的代理进程，完成监控的探针工作和执行简单的远端服务设置。此脚本需要在被监管机上运行；
mmm_control：一个简单的脚本，提供管理mmm_mond进程的命令；
工作原理：
mysql-mmm的监管端会提供多个虚拟IP（VIP），包括一个可写VIP，多个可读VIP；
通过监管的管理，这些IP会绑定在可用mysql之上；
当某一台mysql宕机时，监管会将VIP迁移至其他mysql；
案例搭建Mysql-mmm+mysql 5.6双主高可用集群；
案例环境：
| 系统 |
IP地址 |
主机名 |
所需软件 |
| Centos 7.4 1708 64bit |
192.168.100.1 |
master1 |
mysql-5.6.36.tar.gz mysql-mmm mysql-mmm-agent mysql-tools |
| Centos 7.4 1708 64bit |
192.168.100.3 |
master2 |
mysql-5.6.36.tar.gz mysql-mmm mysql-mmm-agent mysql-tools |
| Centos 7.4 1708 64bit |
192.168.100.5 |
slave1 |
mysql-5.6.36.tar.gz mysql-mmm mysql-mmm-agent mysql-tools |
| Centos 7.4 1708 64bit |
192.168.100.6 |
slave2 |
mysql-5.6.36.tar.gz mysql-mmm mysql-mmm-agent mysql-tools |
| Centos 7.4 1708 64bit |
192.168.100.2 |
monitor |
mysql-5.6.36.tar.gz mysql-mmm mysql-mmm-monitor mysql-tools |
| Centos 7.4 1708 64bit |
192.168.100.2 |
client |
mysql |
案例步骤：
实验过程
部署master1节点的ntp服务以及域名解析
[root@master1 ~]# cat <<END>>/etc/hosts
> 192.168.100.1 master1
> 192.168.100.3 master2
> 192.168.100.5 slave1
> 192.168.100.6 slave2
> 192.168.100.2 monitor
> END
[root@master1 ~]# yum -y install ntp
[root@master1 ~]# sed -i '/^server/s/^/#/g' /etc/ntp.conf
[root@master1 ~]# cat <<END >>/etc/ntp.conf
> server 127.127.1.0
> fudge 127.127.1.0 stratum 8
> END
[root@master1 ~]# systemctl enable ntpd --now
配置master2、slave1、slave2、monitor节点同步ntp时间及域名解析（在此只列举master2单台主机配置）；
[root@master1 ~]# scp /etc/hosts root@192.168.100.3:/etc/hosts
[root@master2 ~]# yum -y install ntpdate
[root@master2 ~]# /usr/sbin/ntpdate 192.168.100.1
14 Feb 13:52:55 ntpdate[62867]: adjust time server 192.168.100.1 offset -0.000655 sec
[root@master2 ~]# echo "/usr/sbin/ntpdate 192.168.100.1">>/etc/rc.local
[root@master2 ~]# chmod +x /etc/rc.local
分别在master1、master2、slave1、slave2节点上安装mysql服务（在此只列举master1单台主机配置）
[root@master1 ~]# yum -y install ncurses cmake
[root@master1 ~]# mount /dev/cdrom /mnt/
[root@master1 ~]# rpm -ivh /mnt/Packages/ncurses-devel-5.9-13.20130511.el7.x86_64.rpm --nodeps
[root@master1 ~]# tar zxvf mysql-5.6.36.tar.gz -C /usr/src/
[root@master1 ~]# cd /usr/src/mysql-5.6.36/
[root@master2 mysql-5.6.36]# cmake -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci -DWITH_INNOBASE_STORAGE_ENGINE=1 -DWITH_ARCHIVE_STORAGE_ENGINE=1 -DWITH_BLACKHOLE_STORAGE_ENGINE=1 -DENABLE_DOWNLOADS=1
[root@master1 mysql-5.6.36]# make
[root@master1 mysql-5.6.36]# make install
[root@master1 mysql-5.6.36]# cd
[root@master1 ~]# cp /usr/src/mysql-5.6.36/support-files/mysql.server /etc/init.d/
[root@master1 ~]# chmod +x /etc/init.d/mysql.server
[root@master1 ~]# cat <<END >>/usr/lib/systemd/system/mysqld.service
[Unit]
Description=mysqldapi
After=network.target
[Service]
Type=forking
PIDFile=/usr/local/mysql/logs/mysqld.pid
ExecStart=/etc/init.d/mysql.server start
ExecReload=/etc/init.d/mysql.server restart
ExecStop=/etc/init.d/mysql.server stop
PrivateTmp=Flase
[Install]
WantedBy=multi-user.target
END
[root@master1 ~]# echo "export PATH=$PATH:/usr/local/mysql/bin/" >>/etc/profile
[root@master1 ~]# source /etc/profile
[root@master1 ~]# groupadd mysql
[root@master1 ~]# useradd -g mysql mysql
[root@master1 ~]# cat <<END >/etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
port = 3306
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
character_set_server=utf8
init_connect='SET NAMES utf8'
log-error=/usr/local/mysql/logs/mysqld.log
pid-file=/usr/local/mysql/logs/mysqld.pid
skip-name-resolve
END
[root@master1 ~]# mkdir /usr/local/mysql/logs
[root@master1 ~]# chown mysql:mysql /usr/local/mysql/ -R
[root@master1 ~]# /usr/local/mysql/scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data ##初始化mysql
[root@master1 ~]# systemctl start mysqld
[root@master1 ~]# systemctl enable mysqld
Created symlink from /etc/systemd/system/multi-user.target.wants/mysqld.service to /usr/lib/systemd/system/mysqld.service.
[root@master1 ~]# netstat -utpln |grep 3306
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 31481/mysqld
[root@master1 ~]# mysqladmin -uroot password 123123
Warning: Using a password on the command line interface can be insecure.
[root@master1 ~]# mysql -uroot -p123123
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
mysql> exit
配置master1实现双主复制；
[root@master1 ~]# cat <<END >>/etc/my.cnf
server-id=1
log-bin=mysql-bin
log-slave-updates
sync_binlog=1
auto_increment_increment=2
auto_increment_offset=1
relay-log=relay1-log-bin
relay-log-index=slave-relay1-bin.index
END
[root@master1 ~]# systemctl restart mysqld
[root@master1 ~]# mysql -uroot -p123123
mysql> show master status;
mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 | 154 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
mysql> grant replication slave on *.* to 'master'@'192.168.100.%' identified by '123123';
mysql> flush privileges;
mysql> exit;
配置master2实现双主复制；
[root@master2 ~]# cat <<END >>/etc/my.cnf
server-id=2
log-bin=mysql-bin
log-slave-updates
sync_binlog=1
auto_increment_increment=2
auto_increment_offset=1
relay-log=relay2-log-bin
relay-log-index=slave-relay2-bin.index
END
[root@master2 ~]# systemctl restart mysqld
[root@master2 ~]# mysql -uroot -p123123
mysql> change master to master_host='192.168.100.1',master_user='master',master_password='123123',master_log_file='mysql-bin.000001',master_log_pos=154;
Query OK, 0 rows affected, 2 warnings (0.00 sec)
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)
mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 | 154 | | | |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)
mysql> grant replication slave on *.* to 'master'@'192.168.100.%' identified by '123123';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.01 sec)
mysql> exit
在master1
mysql> change master to master_host='192.168.100.3',master_user='master',master_password='123123',master_log_file='mysql-bin.000001',master_log_pos=154;
mysql> start slave;
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
只要这俩是yes，双主就完成了
配置slave1和slave2实现主从复制（在此只列举slave1单台主机配置）；
[root@slave1 ~]# cat <<END >>/etc/my.cnf
> server-id=4
> relay-log=relay4-log-bin
> relay-log-index=slave-relay4-bin.index
> END
mysql> start slave;
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
只要这俩是yes，主从就完成了
测试：
在主服务器上创建lyh，从服务器是有同步
分别在master1、master2、slave1、slave2 monitor五个节点安装mysql-mmm软件程序并且配置agent指定本机的节点名称（在此只列举master1单台主机配置）；
配置本地yum源，上传Mysql_mmm_rpm文件夹到master1的/root下
[root@master1 ~]# yum -y install mysql-mmm mysql-mmm-agent mysql-mmm-tools
[root@master1 ~]# vi /etc/mysql-mmm/mmm_agent.conf
include mmm_common.conf
# The 'this' variable refers to this server. Proper operation requires
# that 'this' server (db1 by default), as well as all other servers, have the
# proper IP addresses set in mmm_common.conf.
this db1
:wq
[root@master2 ~]# egrep -v "^#|^$" /etc/mysql-mmm/mmm_agent.conf
include mmm_common.conf
this db2
[root@slave1 ~]# egrep -v "^#|^$" /etc/mysql-mmm/mmm_agent.conf
include mmm_common.conf
this db3
[root@slave2 ~]# egrep -v "^#|^$" /etc/mysql-mmm/mmm_agent.conf
include mmm_common.conf
this db4
在master1节点上授权monitor节点连接数据库集群；
[root@master1 ~]# mysql -uroot -p123123
mysql> grant replication client on *.* to 'mmm_monitor'@'192.168.100.%' identified by 'monitor';
Query OK, 0 rows affected (0.00 sec)
mysql> grant super,replication client,process on *.* to 'mmm_agent'@'192.168.100.%' identified by 'agent';
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
mysql> exit
安装monitor节点上的mysql-mmm软件程序；
[root@monitor ~]# yum -y install mysql-mmm mysql-mmm-tools mysql-mmm-monitor
配置 monitor节点上的mysql-mmm的配置文件并且复制到各个mysql节点；
[root@monitor ~]# vi /etc/mysql-mmm/mmm_common.conf
active_master_role writer ##指定活跃角色为写角色
<host default>
cluster_interface eth0 ##承载的网卡
pid_path /var/run/mysql-mmm/mmm_agentd.pid
bin_path /usr/libexec/mysql-mmm/ ##可执行文件路径
replication_user master ##mysql集群主从复制账户
replication_password 123123
agent_user mmm_agent ##monitor连接集群的账户
agent_password agent
</host>
<host db1>
ip 192.168.100.1
mode master
peer db2 ##指定对立的节点名
</host>
<host db2>
ip 192.168.100.3
mode master
peer db1
</host>
<host db3>
ip 192.168.100.5
mode slave
</host>
<host db4>
ip 192.168.100.6
mode slave
</host>
<role writer>
hosts db1, db2
ips 192.168.100.10
mode exclusive ##同一时间存在单个主
</role>
<role reader>
hosts db3, db4
ips 192.168.100.251, 192.168.100.252
mode balanced ##轮询
</role>
[root@monitor ~]# vi /etc/mysql-mmm/mmm_mon.conf
include mmm_common.conf
<monitor>
ip 127.0.0.1
pid_path /var/run/mysql-mmm/mmm_mond.pid
bin_path /usr/libexec/mysql-mmm
status_path /var/lib/mysql-mmm/mmm_mond.status
ping_ips 192.168.100.1, 192.168.100.3, 192.168.100.5, 192.168.100.6 ##指定监听的所有节点ip
auto_set_online 60 ##判定其online的时间，超过60s认为其down
</monitor>
<host default>
monitor_user mmm_monitor ##monitor的工作用户
monitor_password monitor
</host>
debug 0
[root@monitor ~]# for i in 1 3 5 6;do scp /etc/mysql-mmm/mmm_common.conf root@192.168.100.$i:/etc/mysql-mmm/; done ##将配置文件复制到mysql节点
启动master1、master2、slave1、slave2四个节点上的mysql-mmm服务（在此只列举master1单台主机配置）；
[root@master1 ~]# systemctl daemon-reload
[root@master1 ~]# systemctl start mysql-mmm-agent
[root@master1 ~]# netstat -utpln |grep mmm
tcp 0 0 192.168.100.101:9989 0.0.0.0:* LISTEN 32670/mmm_agentd
启动monitor节点上的mysql-mmm服务并查看集群状态；
[root@monitor ~]# systemctl daemon-reload
[root@monitor ~]# systemctl start mysql-mmm-monitor
[root@monitor ~]# netstat -utpln |grep mmm
tcp 0 0 127.0.0.1:9988 0.0.0.0:* LISTEN 15266/mmm_mond
测试
[root@monitor ~]# mmm_control show
# Warning: agent on host db4 is not reachable
db1(192.168.100.1) master/ONLINE. Roles: writer(192.168.100.250)
db2(192.168.100.3） master/ONLINE. Roles:
db3(192.168.100.5) slave/ONLINE. Roles: reader(192.168.100.251)
db4(192.168.100.6) slave/ONLINE. Roles: reader(192.168.100.252)
