블로그 이미지
윤영식
Frontend Application Architecter, Full Stacker, KnowHow Dispenser and Bike Rider

Publication

Statistics Graph

Recent Comment

2013.01.05 16:26 MongoDB/Prototyping

MongoDB에서 Replica Set을 구성하였을 때 mongodb를 접속하는 client에 대하여 routing 서비스를 하는 mongos(몽고스)에 대해 알아보자.



1. 구성 아키텍쳐

  - Shard 한개에 mongod 한개를 맵핑한다

  - config 한개에 mongod 한개만 사용한다



2) 구성하기 

  - shard1, shard2, shard3, config, mongos를 간단히 만들어 보자 (shard를 replica set으로 구성은 배재, 참조)

  - Shard server 3개 띄우기 : sdb01, sdb02, sdb03 디렉토리는 미리 만들어 놓는다. (기동은 개별 콘솔에서 실행함)

$ ./mongod --shardsvr --dbpath /home/dowon/MongoDB/sdb01 --port 10000

Fri Jan  4 22:25:03 

Fri Jan  4 22:25:03 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.

.. 중략 ...

Fri Jan  4 22:25:03 [initandlisten] build info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49

Fri Jan  4 22:25:03 [initandlisten] options: { dbpath: "/home/dowon/MongoDB/sdb01", port: 10000, shardsvr: true }

Fri Jan  4 22:25:03 [websvr] admin web console waiting for connections on port 11000


./mongod --shardsvr --dbpath /home/dowon/MongoDB/sdb02 --port 20000
./mongod --shardsvr --dbpath /home/dowon/MongoDB/sdb03 --port 30000

////////////////////////////
// config srever를 띄웠다. 
$ ./mongod --configsvr --dbpath /home/dowon/MongoDB/sdb04c --port 40000
Fri Jan  4 22:29:47 
Fri Jan  4 22:29:47 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.
Fri Jan  4 22:29:47 
Fri Jan  4 22:29:47 [initandlisten] MongoDB starting : pid=8201 port=40000 dbpath=/home/dowon/MongoDB/sdb04c 32-bit host=localhost.localdomain
Fri Jan  4 22:29:48 [initandlisten] waiting for connections on port 40000
Fri Jan  4 22:29:48 [websvr] admin web console waiting for connections on port 41000

////////////////////////////
// mongos 띄우기 : chunksize 정하지 않으며 1Mbytes이다. 
$ ./mongos --configdb localhost:40000 --chunkSize 1
Fri Jan  4 22:32:04 warning: running with 1 config server should be done only for testing purposes and is not recommended for production
Fri Jan  4 22:32:04 [mongosMain] MongoS version 2.2.2 starting: pid=8239 port=27017 32-bit host=localhost.localdomain (--help for usage)
Fri Jan  4 22:32:04 [mongosMain] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Fri Jan  4 22:32:04 [mongosMain] build info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Fri Jan  4 22:32:11 [Balancer] distributed lock 'balancer/localhost.localdomain:27017:1357367525:1804289383' unlocked. 
Fri Jan  4 22:32:17 [Balancer] distributed lock 'balancer/localhost.localdomain:27017:1357367525:1804289383' acquired, ts : 50e7c8f1b1a36d7262b31138


3) mongos 에서 환경 구성하기

  - mongo shell로 들어가서 shard 환경을 만든다 : 자동으로 mongos로 들어간다 (프롬프트 명칭이 mongos가 됨)

$ ./mongo

MongoDB shell version: 2.2.2

connecting to: test

mongos> use admin

switched to db admin

mongos> db

admin

mongos> show dbs

config 0.046875GB

mongos> use peopleDB
switched to db peopleDB
// config 셋팅을 하지 않고 사용하면 다음과 같은 에러가 발생한다. 
mongos> db.people.save({_id:1, age:11, name:'dowon1', sex:false});
Fri Jan  4 22:38:03 uncaught exception: error {
"$err" : "error creating initial database config information :: caused by :: can't find a shard to put new db on",
"code" : 10185
}
mongos> 

/////////////////////////
// 다시 mongo로 들어온다. 
// 그리고 shard에 서버를 추가해 준다.
$ ./mongo
MongoDB shell version: 2.2.2
connecting to: test
mongos> db
test
mongos> use admin
switched to db admin
mongos> db
admin
mongos> db.runCommand({addshard:'localhost:10000'});
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> db.runCommand({addshard:'localhost:20000'});
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> db.runCommand({addshard:'localhost:30000'});
{ "shardAdded" : "shard0002", "ok" : 1 }

////////////////////////////////
// mongos를 통하여 CRUD를 한다
// 과연 입력한 데이터는 Shard가 되었을까? 
mongos> 
mongos> use testShard
switched to db testShard
mongos> db
testShard
mongos> db.people.save({_id:1, age:11, name:'dowon1', sex:false});
mongos> db.people.save({_id:2, age:22, name:'dowon2', sex:true});
mongos> db.people.save({_id:3, age:33, name:'dowon3', sex:false});
mongos> db.people.find();
{ "_id" : 1, "age" : 11, "name" : "dowon1", "sex" : false }
{ "_id" : 2, "age" : 22, "name" : "dowon2", "sex" : true }
{ "_id" : 3, "age" : 33, "name" : "dowon3", "sex" : false }

/////////////////////////////////////
// 결과 확인 : auto shard가 안되어 있다
// sdb01에만 데이터가 저장되어 있다. 이유 : config server를 가동시키지 않았기 때문이다.
[dowon@localhost MongoDB]$ cd sdb01
[dowon@localhost sdb01]$ ll
total 65548
-rwxrwxr-x. 1 dowon dowon        5 Jan  4 22:25 mongod.lock
-rw-------. 1 dowon dowon 16777216 Jan  4 22:43 testShard.0
-rw-------. 1 dowon dowon 33554432 Jan  4 22:42 testShard.1
-rw-------. 1 dowon dowon 16777216 Jan  4 22:43 testShard.ns
drwxrwxr-x. 2 dowon dowon     4096 Jan  4 22:42 _tmp
[dowon@localhost sdb01]$ cd ../sdb02
[dowon@localhost sdb02]$ ll
total 4
-rwxrwxr-x. 1 dowon dowon 5 Jan  4 22:25 mongod.lock
[dowon@localhost sdb02]$ cd ../sdb03
[dowon@localhost sdb03]$ ll
total 4
-rwxrwxr-x. 1 dowon dowon 5 Jan  4 22:25 mongod.lock


4) Shard 테스트를 위한 데이터 만들고 확인하기 

  - shard 연결 환경을 설정한다. (참조 동영상, Shard Guide blog)

////////////////////////////////

// 10000 port mongodb로 들어가자

// 이것은 10000 port의 mongodb로 direct 들어가는 것이다. 

$ ./mongo localhost:10000

MongoDB shell version: 2.2.2

connecting to: localhost:10000/test

> show dbs

local (empty)

testShard 0.0625GB

> use testShard

switched to db testShard

> db

testShard

> show collections

people

system.indexes

> db.people.find();

{ "_id" : 1, "age" : 11, "name" : "dowon1", "sex" : false }

{ "_id" : 2, "age" : 22, "name" : "dowon2", "sex" : true }

{ "_id" : 3, "age" : 33, "name" : "dowon3", "sex" : false }


////////////////////////////
// 20000, 30000 port로 mongos를 통하지 않고 mongodb로 바로 접속하면 
// testShard DB는 없다. 
$ ./mongo localhost:20000
MongoDB shell version: 2.2.2
connecting to: localhost:20000/test
> show dbs
local (empty)

/////////////////////////////////
// mongos로 들어간다. use admin 
// Sharding 환경설정하기
$ ./mongo
MongoDB shell version: 2.2.2
connecting to: test
mongos> dbs
Fri Jan  4 22:54:15 ReferenceError: dbs is not defined (shell):1
mongos> show dbs
config 0.046875GB
testShard 0.0625GB
mongos> use admin
switched to db admin
mongos> db
admin
mongos> db.runCommand({enablesharding:'testShard'});
{ "ok" : 1 }
mongos> db.runCommand({shardcollection:'people', key:{_id:1}});
{ "ok" : 0, "errmsg" : "bad ns[people]" } <-- namespace 에러 
mongos> db.runCommand({shardcollection:'testShard.people', key:{_id:1}});  // _id:1 의 1은 sorting
{ "collectionsharded" : "testShard.people", "ok" : 1 }

//////////////////////////////////////////////
// people collection에 데이터를 넣어보자
mongos> for(var i=6 ; i<100000 ; i++) {
      db.people.save({_id:i, age:44, name:'dowon'+i, sex:true});
   }

//////////////////////////////////////////////////////////////////////////
// 다른 mongo에서 show dbs를 해보면 testShard DB가 생성되었음을 볼 수 있다. 
// chunkSize 1Mbytes 이상이면 sharding이 이루어진다. 
[dowon@localhost mongodb]$ ./mongo localhost:20000
MongoDB shell version: 2.2.2
connecting to: localhost:20000/test
> show dbs
local (empty)
> show dbs
admin (empty)
local (empty)
testShard 0.0625GB
> use testShard
switched to db testShard
> db.people.find();
{ "_id" : 54053, "age" : 44, "name" : "dowon54053", "sex" : true } // 54053 번부터 Sharding 되었다. 
{ "_id" : 54054, "age" : 44, "name" : "dowon54054", "sex" : true }
{ "_id" : 54055, "age" : 44, "name" : "dowon54055", "sex" : true }
{ "_id" : 54056, "age" : 44, "name" : "dowon54056", "sex" : true }
{ "_id" : 54057, "age" : 44, "name" : "dowon54057", "sex" : true }

[dowon@localhost mongodb]$ ./mongo localhost:30000
MongoDB shell version: 2.2.2
connecting to: localhost:30000/test
local (empty)
> show dbs
admin (empty)
local (empty)
testShard 0.0625GB
> use testShard
switched to db testShard
> db.people.find();
{ "_id" : 22200, "age" : 44, "name" : "dowon22200", "sex" : true } // 22200 번부터 Sharding 되었다. 
{ "_id" : 22201, "age" : 44, "name" : "dowon22201", "sex" : true }
{ "_id" : 22202, "age" : 44, "name" : "dowon22202", "sex" : true }
{ "_id" : 22203, "age" : 44, "name" : "dowon22203", "sex" : true }

< 결론 > 
Sharding 시켜놓은 서버로 ChunkSize 크기에 따라 데이터가 분산되어 저장된다. 

 

* 요약 (명령 참조)

  - mongo admin 으로 들어간다

  - runCommand 통하여 세가지 설정을 수행한다. 

  - chunkSize에 따라서 Sharding이 이루어진다.


* Sharding 개념 다시 정리 하기


* MongoDB의 철학은?

데이터와 데이터로직의 분리 

 - 데이터 : JSON (실제 BSON : binary json 형태로 저장됨)

 - 데이터로직 : JavaScript


<참조>

  - 10gen 설정 튜토리얼

저작자 표시 비영리 변경 금지
신고
posted by peter yun 윤영식