[MongoDB] Shard 환경 구성하기 (2)

몽고디비에서 Shard 1개 ( = ReplicaSet 1개)를 구성하였다면 이제는 n개의 Shard를 구성하여 mongos를 통하여 연결해 본다

1. Shard n개 구성하기

  - Shard Server 1개에 mongod 1개만 만듦 (mongod n개 구성은 다음 블로깅에서)

  - db11, db22, db33 디렉토리를 미리 만들어 놓는다

/mongodb_2.4.5> mongod --shardsvr --dbpath /mongodb_2.4.5/db11 --port 10000
Sat Jul 27 15:15:40.854 [initandlisten] MongoDB starting : pid=71105 port=10000 dbpath=/mongodb_2.4.5/db11 64-bit host=nulpulum-mac-13-retina.local
Sat Jul 27 15:15:41.141 [initandlisten] command local.$cmd command: { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0  reslen:37 271ms
Sat Jul 27 15:15:41.141 [websvr] admin web console waiting for connections on port 11000
Sat Jul 27 15:15:41.142 [initandlisten] waiting for connections on port 10000

/mongodb_2.4.5> mongod --shardsvr --dbpath /mongodb_2.4.5/db22 --port 20000
Sat Jul 27 15:16:38.842 [initandlisten] MongoDB starting : pid=71106 port=20000 dbpath=/mongodb_2.4.5/db22 64-bit host=nulpulum-mac-13-retina.local
Sat Jul 27 15:16:39.105 [websvr] admin web console waiting for connections on port 21000
Sat Jul 27 15:16:39.105 [initandlisten] waiting for connections on port 20000

/mongodb_2.4.5> mongod --shardsvr --dbpath /mongodb_2.4.5/db33 --port 30000
Sat Jul 27 15:18:04.778 [initandlisten] MongoDB starting : pid=71120 port=30000 dbpath=/mongodb_2.4.5/db33 64-bit host=nulpulum-mac-13-retina.local
Sat Jul 27 15:18:05.031 [websvr] admin web console waiting for connections on port 31000
Sat Jul 27 15:18:05.031 [initandlisten] waiting for connections on port 30000

2. Config 환경 구성하기

   - config1, config2, config3 디렉토리를 미리 만들어 놓는다

/mongodb_2.4.5> mongod --help
.. 중략 ..
Master/slave options (old; use replica sets instead):
  --master              master mode
  --slave               slave mode
  --source arg          when slave: specify master as <server:port>
  --only arg            when slave: specify a single database to replicate
  --slavedelay arg      specify delay (in seconds) to be used when applying
                        master ops to slave
  --autoresync          automatically resync if slave data is stale

Replica set options:
  --replSet arg           arg is <setname>[/<optionalseedhostlist>]
  --replIndexPrefetch arg specify index prefetching behavior (if secondary)
                          [none|_id_only|all]

Sharding options:
  --configsvr           declare this is a config db of a cluster; default port
                        27019; default dir /data/configdb
  --shardsvr            declare this is a shard db of a cluster; default port
                        27018

  - config 서버 기동하기

/mongodb_2.4.5> mongod --configsvr --dbpath /mongodb_2.4.5/config1 --port 40000
Sat Jul 27 15:22:12.404 [initandlisten] MongoDB starting : pid=71141 port=40000 dbpath=/mongodb_2.4.5/config1 master=1 64-bit host=nulpulum-mac-13-retina.local
Sat Jul 27 15:22:12.544 [initandlisten] creating replication oplog of size: 5MB...
Sat Jul 27 15:22:12.547 [initandlisten] ******
Sat Jul 27 15:22:12.547 [websvr] admin web console waiting for connections on port 41000
Sat Jul 27 15:22:12.547 [initandlisten] waiting for connections on port 40000

3. Mongos로 Shard 제어하기

  - 명령어 : mongos <config server>

/mongodb_2.4.5> mongos --help
.. 중략 ..
Sharding options:
  --configdb arg      1 or 3 comma separated config servers
  --localThreshold arg 

                              ping time (in ms) for a node to be considered local (default 15ms)
  --test                   just run unit tests
  --upgrade             upgrade meta data version
  --chunkSize arg     maximum amount of data per chunk (기본 : 64kbytes)
  --ipv6                   enable IPv6 support (disabled by default)
  --jsonp                 allow JSONP access via http (has security implications)
  --noscripting          disable scripting engine

  - mongos 연결하기

/mongodb_2.4.5> mongos --configdb localhost:40000 --chunkSize 1 --port 50000
Sat Jul 27 15:31:45.926 [mongosMain] options: { chunkSize: 1, configdb: "localhost:40000", port: 50000 }
Sat Jul 27 15:31:45.929 [Balancer] about to contact config servers and shards
Sat Jul 27 15:31:45.929 [websvr] admin web console waiting for connections on port 51000
Sat Jul 27 15:31:45.929 [Balancer] config servers and shards contacted successfully
Sat Jul 27 15:31:45.929 [mongosMain] waiting for connections on port 50000
Sat Jul 27 15:31:45.932 [Balancer] distributed lock 'balancer/nulpulum-mac-13-retina.local:50000:1374906705:16807' acquired, ts : 51f36951133d3c7f4fe7b076
Sat Jul 27 15:31:45.932 [Balancer] distributed lock 'balancer/nulpulum-mac-13-retina.local:50000:1374906705:16807' unlocked.

4. REPL로 데이터 제어하기

  - mongo로 들어가면 mongos 프롬프트가 나온다

mongodb_2.4.5> mongo localhost:50000
MongoDB shell version: 2.4.5
connecting to: localhost:50000/test
mongos>

mongos> show dbs
admin    (empty)
config    0.046875GB

  - admin db로 들어가서 config server에 shard환경을 구성한다

mongos> show dbs
admin    (empty)
config    0.046875GB
mongos> use admin
switched to db admin
mongos> show collections
mongos> db
admin
mongos> db.runCommand({ addShard: 'localhost:10000' });
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> db.runCommand({ addShard: 'localhost:20000' });
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> db.runCommand({ addShard: 'localhost:30000' });
{ "shardAdded" : "shard0002", "ok" : 1 }

  - 동적으로 mongod를 Shard에 포함시키기

/////////////// --shardsvr 옵션없이 mongod를 띄운다

/mongodb_2.4.5> mongod --dbpath /mongodb_2.4.5/db111 --port 60000
Sat Jul 27 15:43:48.510 [initandlisten] MongoDB starting : pid=71193 port=60000 dbpath=/mongodb_2.4.5/db111 64-bit host=nulpulum-mac-13-retina.local
Sat Jul 27 15:43:48.542 [websvr] admin web console waiting for connections on port 61000
Sat Jul 27 15:43:48.542 [initandlisten] waiting for connections on port 60000
Sat Jul 27 15:43:57.418 [initandlisten] connection accepted from 127.0.0.1:51723 #1 (1 connection now open)

/////////////// runCommand를 수행한다

mongos> show collections
mongos> db.runCommand({ addShard: 'localhost:60000' });
{ "shardAdded" : "shard0003", "ok" : 1 }

/////////////// mongos 의 system out 출력값
Sat Jul 27 15:43:57.419 [conn1] going to add shard: { _id: "shard0003", host: "localhost:60000" }

5. 데이터 Sharding 하기

  - mongos를 통하여 데이터를 Sharding하여 나누어서 넣는 방법 : 한곳으로 데이터가 들어갈때 트래픽이 감당하기 힘들때 다른 샤드로 데이터를 보낸다 (10gen)

mongos> use dowonDB
switched to db dowonDB
mongos> db
dowonDB
mongos> db.person.save({age:1, name:'youngsik', address:'seoul'});
mongos> db.person.find();
{ "_id" : ObjectId("51f36cfb065189f0c02620d4"), "age" : 1, "name" : "youngsik", "address" : "seoul" }
mongos> use admin
switched to db admin
mongos> db
admin
mongos> db.runCommand({ enablesharding: 'dowonDB' })
{ "ok" : 1 }
mongos> db.runCommand({ shardcollection: 'dowonDB.person', key:{_id:1} })
{ "collectionsharded" : "dowonDB.person", "ok" : 1 }

  - 데이터 넣어서 샤드 점검하기 : 여러 mongod 서버로 데이터가 나뉘어서 들어갈 것이다. 즉, mongos가 로드발샌싱하여 데이터를 라우팅하여 줄 것이다. (5백만건 넣기)

mongos> use dowonDB
switched to db dowonDB
mongos> db
dowonDB
mongos> for(var i=0; i < 5000000 ; i++) { db.person.save({ age: i, name:i+9, address: i+1000}); };

mongos> db.person.find().size();
5000001

mongos> db.person.find().skip(1000000).limit(10);
{ "_id" : ObjectId("51f36fc4065189f0c037d1d1"), "age" : 1159420, "name" : 1159429, "address" : 1160420 }
{ "_id" : ObjectId("51f36f8d065189f0c02b7e4c"), "age" : 351607, "name" : 351616, "address" : 352607 }
{ "_id" : ObjectId("51f37031065189f0c04dc534"), "age" : 2597983, "name" : 2597992, "address" : 2598983 }
{ "_id" : ObjectId("51f36fab065189f0c032416e"), "age" : 794777, "name" : 794786, "address" : 795777 }
{ "_id" : ObjectId("51f36fc4065189f0c037d1d2"), "age" : 1159421, "name" : 1159430, "address" : 1160421 }
{ "_id" : ObjectId("51f36f8d065189f0c02b7e4d"), "age" : 351608, "name" : 351617, "address" : 352608 }
{ "_id" : ObjectId("51f37031065189f0c04dc535"), "age" : 2597984, "name" : 2597993, "address" : 2598984 }
{ "_id" : ObjectId("51f36fab065189f0c032416f"), "age" : 794778, "name" : 794787, "address" : 795778 }
{ "_id" : ObjectId("51f36fc4065189f0c037d1d3"), "age" : 1159422, "name" : 1159431, "address" : 1160422 }
{ "_id" : ObjectId("51f36f8d065189f0c02b7e4e"), "age" : 351609, "name" : 351618, "address" : 352609 }

  - db11, db22, db33, db111 각 디렉토리에 1Gbytes 가량의 데이터가 쌓여진다.

<참조>

  - 10Gen ReplicaSet 프리젠테이션