MongoDB运维与开发（六）---MongoDB集群(5)

//

MongoDB运维与开发（六）---MongoDB集群(5)

//

关于MongoDB的集群运维，之前的文章已经说了很多内容了，这块儿知识点比较多，由于是每天抽空写文章，所以每天能说到的点有限，慢慢一点一点搞懂它，今天我们来看MongDB集群里面的分片相关知识。

关于分片和副本集的概念，之前的文章中也讲过，上一篇文章中讲了MongoS和ConfigDB的配置方法，今天来看如何在MongoS上添加Mongod分片。

01

初始化分片

通常情况下，我们配置好了mongos和configDB之后，需要在Mongos上面配置分片节点，配置分片节点的之前，需要有几项准备工作：

1、启动分片节点副本集

2、在mongos的配置文件中写清除configDB的地址

3、保证mongos可以同时访问configDB和分片节点

然后，我们通过下面的命令来配置初始化添加一个分片节点副本集：

use admin
db.runCommand({addShard:"sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020"})

可以看到，我们添加了一个名字叫做sharding_yeyz的副本集，它作为Mongos的分片加入到这个集群中，它包含27018、27019、27020三个mongod实例，架构图如下：

02

在将要分片的集合上面创建索引

mongos> use test
switched to db test
mongos> db.test0.ensureIndex({_id:"hashed"})
{
        "raw" : {
                "sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020" : {
                        "createdCollectionAutomatically" : false,
                        "numIndexesBefore" : 1,
                        "numIndexesAfter" : 2,
                        "ok" : 1
                }
        },
        "ok" : 1,
        "operationTime" : Timestamp(1605690613, 4),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1605690613, 4),
                "signature" : {
                        "hash" : BinData(0,"9ThfDRTC/X/GSuLbe8E7TjXLP2I="),
                        "keyId" : NumberLong("6894922308364795934")
                }
        }
}

如上面的命令，我们在test库下面的test0集合上的_id字段上创建hash索引，也就是：

db.test0.ensureIndex({_id:"hashed"})

03

对db和collection启用分片

首先我们通过下面的命令，对test库启用分片：

mongos> sh.enableSharding("test")
{
        "ok" : 1,
        "operationTime" : Timestamp(1605690781, 3),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1605690781, 3),
                "signature" : {
                        "hash" : BinData(0,"YrrrniMbOp8Vfjkf8cYulX+n9oA="),
                        "keyId" : NumberLong("6894922308364795934")
                }
        }
}

可以看到，命令sh.enableSharding对库test启用了分片。

然后对test0这个collection启用分片：

mongos> sh.shardCollection("test.test0",{_id:"hashed"})
{
        "collectionsharded" : "test.test0",
        "collectionUUID" : UUID("a1bafe79-c74e-4454-ae98-ce1b175cda2c"),
        "ok" : 1,
        "operationTime" : Timestamp(1605690845, 10),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1605690845, 10),
                "signature" : {
                        "hash" : BinData(0,"PIttL1zPmNcjjGZ1BpBIZVrhT6I="),
                        "keyId" : NumberLong("6894922308364795934")
                }
        }
}

04

插入数据，观察分片状态

我们循环插入number=1、2、3~1000的数字，并观察当前分片状态如下：

### 循环程序，插入i=1、2、3、4...1000到test.test0这个集合
mongos> for (var i=1 ;i<=1000 ; i++){ db.test1.insert({"number":i})}
WriteResult({ "nInserted" : 1 })

### 使用printShardingStatus查看分片状态
mongos> db.printShardingStatus()
--- Sharding Status ---
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("5fafaf4f5785d9965548f687")
  }
  shards:    # 涉及的分片只有sharding_yeyz
        {  "_id" : "sharding_yeyz",  "host" : "sharding_yeyz/127.0.0.1:27018,127.0.0.1:27019,127.0.0.1:27020",  "state" : 1 }
  active mongoses:
        "4.0.6" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  1
        Last reported error:  Could not find host matching read preference { mode: "primary" } for set sharding_yeyz
        Time of Reported error:  Wed Nov 18 2020 17:08:14 GMT+0800 (CST)
        Migration Results for the last 24 hours:
                No recent migrations
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                sharding_yeyz   1
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : sharding_yeyz Timestamp(1, 0)
        {  "_id" : "test",  "primary" : "sharding_yeyz",  "partitioned" : true,  "version" : {  "uuid" : UUID("f1d54510-2675-45e1-b61b-2eb0fe015f2d"),  "lastMod" : 1 } }
                test.test0
                        shard key: { "_id" : "hashed" }
                        unique: false
                        balancing: true
                        chunks:  # 两个数据chunk都在sharding_yeyz上
                                sharding_yeyz   2 
                        { "_id" : { "$minKey" : 1 } } -->> { "_id" : NumberLong(0) } on : sharding_yeyz Timestamp(1, 0)
                        { "_id" : NumberLong(0) } -->> { "_id" : { "$maxKey" : 1 } } on : sharding_yeyz Timestamp(1, 1)

从分片状态可以看到，当前只存在一个分片sharding_yeyz,所有的数据都在该分片上，然后我们查询一下数据：

sharding_yeyz:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99a"), "number" : 1 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99b"), "number" : 2 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99c"), "number" : 3 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99d"), "number" : 4 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99e"), "number" : 5 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99f"), "number" : 6 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a0"), "number" : 7 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a1"), "number" : 8 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a2"), "number" : 9 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a3"), "number" : 10 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a4"), "number" : 11 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a5"), "number" : 12 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a6"), "number" : 13 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a7"), "number" : 14 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a8"), "number" : 15 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a9"), "number" : 16 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9aa"), "number" : 17 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ab"), "number" : 18 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ac"), "number" : 19 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ad"), "number" : 20 }
Type "it" for more
sharding_yeyz:PRIMARY> db.test0.count()
1000

可以看到所有数据都在，数据总条数是1000条，和我们设想的结果一致。

05

加入一个新的分片

此时我们加入一个新的分片副本集，名字叫做sharding_yeyz1,它由27024、27025、27026三个mongod实例组成，如下：

db.runCommand({addShard:"sharding_yeyz1/127.0.0.1:27024,127.0.0.1:27025,127.0.0.1:27026"})
{
        "shardAdded" : "sharding_yeyz1",
        "ok" : 1,
        "operationTime" : Timestamp(1605697822, 2),
        "$clusterTime" : {
                "clusterTime" : Timestamp(1605697822, 3),
                "signature" : {
                        "hash" : BinData(0,"wpspZ+ILVa7R5Zybhks+3da9N4g="),
                        "keyId" : NumberLong("6894922308364795934")
                }
        }
}

数据的架构变成了：

此时我们重新对test库下面的test1集合启用分片，并插入1~1000的数据，如下：

启用分片
sh.shardCollection("test.test1",{_id:"hashed"})

插入数据
for (var i=1 ;i<=1000 ; i++){ db.test1.insert({"number":i})}

来看sharding_yeyz和sharding_yeyz1上test.test1这个集合的查询结果：

sharding_yeyz:PRIMARY> db.test1.find()
{ "_id" : ObjectId("5fb50645eebfcce3b111e552"), "number" : 1 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e553"), "number" : 2 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e554"), "number" : 3 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e555"), "number" : 4 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e556"), "number" : 5 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e557"), "number" : 6 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55b"), "number" : 10 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55e"), "number" : 13 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e563"), "number" : 18 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e565"), "number" : 20 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e566"), "number" : 21 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e568"), "number" : 23 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e569"), "number" : 24 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56a"), "number" : 25 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56b"), "number" : 26 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56d"), "number" : 28 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56e"), "number" : 29 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e573"), "number" : 34 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e576"), "number" : 37 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e577"), "number" : 38 }
Type "it" for more
sharding_yeyz:PRIMARY> db.test1.count()
519
--------------------
sharding_yeyz1:PRIMARY> db.test1.find()
{ "_id" : ObjectId("5fb50645eebfcce3b111e558"), "number" : 7 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e559"), "number" : 8 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55a"), "number" : 9 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55c"), "number" : 11 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55d"), "number" : 12 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e55f"), "number" : 14 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e560"), "number" : 15 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e561"), "number" : 16 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e562"), "number" : 17 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e564"), "number" : 19 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e567"), "number" : 22 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56c"), "number" : 27 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e56f"), "number" : 30 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e570"), "number" : 31 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e571"), "number" : 32 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e572"), "number" : 33 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e574"), "number" : 35 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e575"), "number" : 36 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e57a"), "number" : 41 }
{ "_id" : ObjectId("5fb50645eebfcce3b111e57b"), "number" : 42 }
Type "it" for more
sharding_yeyz1:PRIMARY> db.test1.count()
481

可以看到，sharding_yeyz保存有519条数据，而sharding_yeyz1保存有481条数据，并不是严格的按照50%的比例分配的，而且number的序号，也不是严格按照奇数偶数来分的。

还有一个问题？已经分配了的test0集合，1000条记录原本都在sharding_yeyz这个分片上，加入了sharding_yeyz1，这1000条记录会不会重新分配到第二个分片上呢？我们来看实际结果：

------------原始分片------------
sharding_yeyz:PRIMARY> db.test0.count()
492
sharding_yeyz:PRIMARY>
sharding_yeyz:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99b"), "number" : 2 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99d"), "number" : 4 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99f"), "number" : 6 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a1"), "number" : 8 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a2"), "number" : 9 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a3"), "number" : 10 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a4"), "number" : 11 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a5"), "number" : 12 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a6"), "number" : 13 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a7"), "number" : 14 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ae"), "number" : 21 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b0"), "number" : 23 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b2"), "number" : 25 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b3"), "number" : 26 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b4"), "number" : 27 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b5"), "number" : 28 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b6"), "number" : 29 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b7"), "number" : 30 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b9"), "number" : 32 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bd"), "number" : 36 }
Type "it" for more


----------------新增分片------------
sharding_yeyz1:PRIMARY> db.test0.count()
508
sharding_yeyz1:PRIMARY> db.test0.find()
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99a"), "number" : 1 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99c"), "number" : 3 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d99e"), "number" : 5 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a0"), "number" : 7 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a8"), "number" : 15 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9a9"), "number" : 16 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9aa"), "number" : 17 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ab"), "number" : 18 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ac"), "number" : 19 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ad"), "number" : 20 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9af"), "number" : 22 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b1"), "number" : 24 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9b8"), "number" : 31 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9ba"), "number" : 33 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bb"), "number" : 34 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bc"), "number" : 35 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9be"), "number" : 37 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9bf"), "number" : 38 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9c1"), "number" : 40 }
{ "_id" : ObjectId("5fb4e78beebfcce3b111d9c4"), "number" : 43 }

从结果中不难看出来，原来test0这个集合上面的1000条记录，也都从分片1上相对均匀的分配到了分片2上面，而分片1上的数据进行了清理，不再是之前的1000条。

到这里，大家对MongoDB的分片应该有了如下认识：

1、分片是自动的

2、分片不是绝对均匀的

后续的文章，我们会说明分片受哪些因素的影响。

今天就到这里吧。