背景:mongodb集群搭建方式有三种,1、主从(官方已经不推荐),2、副本集,3、分片。这里介绍如何通过分片sharding方式搭建mongodb集群。sharding集群方式也基于副本集,在搭建过程中,需要对分片和配置节点做副本集。最后将做好的副本集的分片加入到路由节点,构成集群。
sharding方式的集群中,有三类角色,分别是shard,config,router。如下图所示。
![](https://img-blog.csdn.net/20180629153220189?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaW5pZmk=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70)
shard:分片节点,存储数据。
config:配置节点,不会存储数据,会存储元数据信息,比如片键的范围。
router:路由节点,是mongo集群与外部客户端连接的入口,他提供mongos客户端,对客户端透明,让客户端感觉使用单节点数据库。
环境介绍:
这里设计的集群如下节点分别如下:
router : 127.0.0.1:27017
config : config/127.0.0.1:27117,127.0.0.1:27217,127.0.0.1:27317
shard : shard1/127.0.0.1:27018,127.0.0.1:27019 shard2/127.0.0.1:27118,127.0.0.1:27119
搭建步骤:
mongodb下载地址:
http://downloads.mongodb.org/linux/mongodb-linux-x86_64-3.4.0.tgz?_ga=2.55492392.2036626358.1530260071-2147164267.1528775467
如果想要下载其他版本,可以在这里选择:http://dl.mongodb.org/dl/linux。
一、设置三类配置文件,分片节点,配置节点,路由节点。
shard01.conf
port=27018
fork=true
dbpath=/home/hadoop/software/mongodb-3.4/data/shard01
logpath=/home/hadoop/software/mongodb-3.4/logs/shard01.log
logappend=true
bind_ip=127.0.0.1
replSet=shard1
shardsvr=true
shard02.conf配置文件和shard02.conf配置文件的区别就是端口改为了27019,另外数据存储路径和日志路径均和文件名一致。
shard21.conf
port=27118
fork=true
dbpath=/home/hadoop/software/mongodb-3.4/data/shard21
logpath=/home/hadoop/software/mongodb-3.4/logs/shard21.log
logappend=true
bind_ip=127.0.0.1
replSet=shard2
shardsvr=true
shard22.conf配置文件和shard21.conf配置文件的区别就是端口改为了27119,另外数据存储路径和日志路径均和文件名一致。
config.conf
port=27117
fork=true
dbpath=/home/hadoop/software/mongodb-3.4/data/config
logpath=/home/hadoop/software/mongodb-3.4/logs/config.log
logappend=true
bind_ip=127.0.0.1
replSet=config
configsvr=true
config2.conf,config3.conf配置文件和config.conf配置文件的区别就是端口改为了27217和27317,另外数据存储路径和日志路径均和文件名一致。
router.conf
port=27017
configdb=config/127.0.0.1:27117,127.0.0.1:27217,127.0.0.1:27317
logpath=/home/hadoop/software/mongodb-3.4/logs/router.log
fork=true
logappend=true
关于配置文件说明:
shard01.conf,shard02.conf是shard1分片的配置。
shard21.conf,shard22.conf是shard2分片的配置。
config.conf,config2.conf,config3.conf是config节点配置。
二、启动分片节点和配置节点。
启动脚本内容:
#!/bin/sh
rm -rf data/*/*
rm -f logs/*
bin/mongod -f conf/shard01.conf
bin/mongod -f conf/shard02.conf
bin/mongod -f conf/shard21.conf
bin/mongod -f conf/shard22.conf
bin/mongod -f conf/config.conf
bin/mongod -f conf/config2.conf
bin/mongod -f conf/config3.conf
启动之后,显示如下信息,表明启动成功。
[root@server mongodb-3.4]# ./start_mongod.sh
about to fork child process, waiting until server is ready for connections.
forked process: 4870
child process started successfully, parent exiting
about to fork child process, waiting until server is ready for connections.
forked process: 4895
child process started successfully, parent exiting
about to fork child process, waiting until server is ready for connections.
forked process: 4920
child process started successfully, parent exiting
about to fork child process, waiting until server is ready for connections.
forked process: 4944
child process started successfully, parent exiting
about to fork child process, waiting until server is ready for connections.
forked process: 4970
child process started successfully, parent exiting
about to fork child process, waiting until server is ready for connections.
forked process: 5002
child process started successfully, parent exiting
about to fork child process, waiting until server is ready for connections.
forked process: 5035
child process started successfully, parent exiting
三、配置节点构成副本集。
> var config = {_id:'config',members:[{_id:0,host:'127.0.0.1:27117'},{_id:1,host:'127.0.0.1:27217'},{_id:2,host:'127.0.0.1:27317'}]}
> rs.initiate(config)
{ "ok" : 1 }
config:SECONDARY>
config:SECONDARY>
config:PRIMARY> exit
配置分片之后,等待10S左右,当前节点会变为primary节点。
四、分片节点构成副本集。
登录127.0.0.1:27018
bin/mongo --port 27018
> var config = {_id:'shard1',members:[{_id:0,host:'127.0.0.1:27018'},{_id:1,host:'127.0.0.1:27019'}]}
> rs.initiate(config)
{ "ok" : 1 }
shard1:SECONDARY>
shard1:SECONDARY>
shard1:PRIMARY>
接着登录127.0.0.1:27118
bin/mongo --port 27118
> var config = {_id:'shard2',members:[{_id:0,host:'127.0.0.1:27118'},{_id:1,host:'127.0.0.1:27119'}]}
> rs.initiate(config)
{ "ok" : 1 }
shard2:SECONDARY>
shard2:PRIMARY>
两个分片shard1,shard2设置成功之后,等待10s左右就会发现当前节点默认变为了当前分片的primary节点。
五、启动路由节点,并增加分片。
启动脚本内容:
#!/bin/sh
bin/mongos -f conf/router.conf
增加分片格式如下:"shard1/127.0.0.1:27018,127.0.0.1:27019"。
[root@server mongodb-3.4]# ./start_mongos.sh
about to fork child process, waiting until server is ready for connections.
forked process: 5928
child process started successfully, parent exiting
[root@server mongodb-3.4]# bin/mongo --port 27017
MongoDB shell version v3.4.0
connecting to: mongodb://127.0.0.1:27017/
MongoDB server version: 3.4.0
Server has startup warnings:
2018-06-29T14:51:29.383+0800 I CONTROL [main]
2018-06-29T14:51:29.383+0800 I CONTROL [main] ** WARNING: Access control is not enabled for the database.
2018-06-29T14:51:29.383+0800 I CONTROL [main] ** Read and write access to data and configuration is unrestricted.
2018-06-29T14:51:29.383+0800 I CONTROL [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-06-29T14:51:29.384+0800 I CONTROL [main]
mongos> sh.addShard("shard1/127.0.0.1:27018,127.0.0.1:27019")
{ "shardAdded" : "shard1", "ok" : 1 }
mongos> sh.addShard("shard2/127.0.0.1:27118,127.0.0.1:27119")
{ "shardAdded" : "shard2", "ok" : 1 }
数据库集群状态截图:
![](https://img-blog.csdn.net/20180629160330501?watermark/2/text/aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ZlaW5pZmk=/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70)
六、设置数据库启用分片,启用索引。
mongos> use admin
switched to db admin
mongos> sh.enableSharding("push")
{ "ok" : 1 }
mongos> sh.shardCollection("push.user",{name:1})
{ "collectionsharded" : "push.user", "ok" : 1 }
七、写入数据测试。
mongos> for(var i=1;i<=2000000;i++){db.user.save({_id:i,name:"user-"+i,age:18})}
WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 2000000 })
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5b35d5a5d35df9180e59a17e")
}
shards:
{ "_id" : "shard1", "host" : "shard1/127.0.0.1:27018,127.0.0.1:27019", "state" : 1 }
{ "_id" : "shard2", "host" : "shard2/127.0.0.1:27118,127.0.0.1:27119", "state" : 1 }
active mongoses:
"3.4.0" : 1
autosplit:
Currently enabled: yes
balancer:
Currently enabled: yes
Currently running: no
Balancer lock taken at Fri Jun 29 2018 14:46:03 GMT+0800 (CST) by ConfigServer:Balancer
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
5 : Success
databases:
{ "_id" : "push", "primary" : "shard1", "partitioned" : true }
push.user
shard key: { "name" : 1 }
unique: false
balancing: true
chunks:
shard1 6
shard2 7
{ "name" : { "$minKey" : 1 } } -->> { "name" : "user-10" } on : shard1 Timestamp(5, 0)
{ "name" : "user-10" } -->> { "name" : "user-1142445" } on : shard1 Timestamp(6, 0)
{ "name" : "user-1142445" } -->> { "name" : "user-1284894" } on : shard2 Timestamp(6, 1)
{ "name" : "user-1284894" } -->> { "name" : "user-1509894" } on : shard2 Timestamp(5, 2)
{ "name" : "user-1509894" } -->> { "name" : "user-1734895" } on : shard2 Timestamp(5, 3)
{ "name" : "user-1734895" } -->> { "name" : "user-191792" } on : shard2 Timestamp(5, 4)
{ "name" : "user-191792" } -->> { "name" : "user-217119" } on : shard2 Timestamp(4, 7)
{ "name" : "user-217119" } -->> { "name" : "user-245186" } on : shard2 Timestamp(4, 4)
{ "name" : "user-245186" } -->> { "name" : "user-390373" } on : shard1 Timestamp(4, 1)
{ "name" : "user-390373" } -->> { "name" : "user-5636" } on : shard1 Timestamp(3, 3)
{ "name" : "user-5636" } -->> { "name" : "user-81307" } on : shard1 Timestamp(3, 4)
{ "name" : "user-81307" } -->> { "name" : "user-9" } on : shard1 Timestamp(2, 4)
{ "name" : "user-9" } -->> { "name" : { "$maxKey" : 1 } } on : shard2 Timestamp(2, 0)
这里在集群上插入200W条数据。然后查看数据分布情况,发现两个分片shard1,shard2上均有数据。
在shard1分片上验证数据分布。
shard1:PRIMARY> show dbs;
admin 0.000GB
local 0.046GB
push 0.032GB
shard1:PRIMARY> use push
switched to db push
shard1:PRIMARY> db.user.count()
885844
在shard2分片上验证数据分布。
shard2:PRIMARY> show dbs;
admin 0.000GB
local 0.055GB
push 0.042GB
shard2:PRIMARY> use push
switched to db push
shard2:PRIMARY> db.user.count()
1114156
本文内容由网友自发贡献,版权归原作者所有,本站不承担相应法律责任。如您发现有涉嫌抄袭侵权的内容,请联系:hwhale#tublm.com(使用前将#替换为@)