블로그 이미지
윤영식
Frontend Application Architecter, Full Stacker, KnowHow Dispenser and Bike Rider

Publication

Statistics Graph

Recent Comment

2013.07.27 14:56 MongoDB/Prototyping

몽고디비에서 Replica Set 환경을 구성해 본다. ReplicaSet을 하나의 Shard로 만들어서 여러개의 Shard가 구성되면 하기와 같이 routing은 mongos가 해준다




1. Replica Set 만들기

  - 옵션 : --replSet 

  - 디렉토리 만들어 놓기 : db1, db2, db3을 미리 만들어 놓는다

mongodb_2.4.5> mongod --replSet dowonSet --port  20000  -dbpath /mongodb_2.4.5/db1
Sat Jul 27 14:11:58.007 [initandlisten] MongoDB starting : pid=71010 port=20000 dbpath=/mongodb_2.4.5/db1 64-bit host=nulpulum-mac-13-retina.local
Sat Jul 27 14:11:58.299 [websvr] admin web console waiting for connections on port 21000
Sat Jul 27 14:11:58.300 [initandlisten] waiting for connections on port 20000
Sat Jul 27 14:12:08.302 [rsStart] replSet can't get local.system.replset config from self or any seed (EMPTYCONFIG)


mongodb_2.4.5> mongod --replSet dowonSet --port  30000  -dbpath /mongodb_2.4.5/db2
Sat Jul 27 14:32:08.934 [initandlisten] MongoDB starting : pid=71062 port=30000 dbpath=/mongodb_2.4.5/db2 64-bit host=nulpulum-mac-13-retina.local


/mongodb_2.4.5> mongod --replSet dowonSet --port  40000  -dbpath /mongodb_2.4.5/db3
Sat Jul 27 14:32:16.317 [initandlisten] MongoDB starting : pid=71063 port=40000 dbpath=/mongodb_2.4.5/db3 64-bit host=nulpulum-mac-13-retina.local



2. Replica Set 환경설정하기

  - mongo REPL 로 연결

  - config 환경 구성

/mongodb_2.4.5> mongo localhost:20000
MongoDB shell version: 2.4.5
connecting to: localhost:20000/test
> var config={_id:'dowonSet', members:[{_id:0, host:'localhost:20000'}, {_id:1, host:'localhost:30000'}, {_id:2, host:'localhost:40000'}] };


///////////// Replica set help 보기

> rs.help();
    rs.status()                         { replSetGetStatus : 1 } checks repl set status
    rs.initiate()                         { replSetInitiate : null } initiates set with default settings
    rs.initiate(cfg)                   { replSetInitiate : cfg } initiates set with configuration cfg
    rs.conf()                           get the current configuration object from local.system.replset
    rs.reconfig(cfg)                  updates the configuration of a running replica set with cfg (disconnects)
    rs.add(hostportstr)             add a new member to the set with default attributes (disconnects)
    rs.add(membercfgobj)        add a new member to the set with extra attributes (disconnects)
    rs.addArb(hostportstr)        add a new member which is arbiterOnly:true (disconnects)
    rs.stepDown([secs])          step down as primary (momentarily) (disconnects)
    rs.syncFrom(hostportstr)    make a secondary to sync from the given member
    rs.freeze(secs)                  make a node ineligible to become primary for the time specified
    rs.remove(hostportstr)       remove a host from the replica set (disconnects)
    rs.slaveOk()                      shorthand for db.getMongo().setSlaveOk()

    db.isMaster()                     check who is primary
    db.printReplicationInfo()       check oplog size and time range

    reconfiguration helpers disconnect from the database so the shell will display
    an error, even if the command succeeds.
    see also http://<mongod_host>:28017/_replSet for additional diagnostic info


///////////// 환경 적용하기

> rs.initiate(config);
{
    "info" : "Config now saved locally.  Should come online in about a minute.",
    "ok" : 1
}


///////////// Enter를 치면 프롬프트가 변경되어 나온다

dowonSet:PRIMARY>
dowonSet:PRIMARY> show dbs
local    0.328125GB
dowonSet:PRIMARY> use dowonDB
switched to db dowonDB
dowonSet:PRIMARY> db
dowonDB
dowonSet:PRIMARY>



3. REPL 조작하기

  - PRIMARY에서 데이터 조작하기

dowonSet:PRIMARY> db.person.save({age:23, name:'youngsik', title:'hi'});
dowonSet:PRIMARY> show collections
person
system.indexes
dowonSet:PRIMARY> db.person.find();
{ "_id" : ObjectId("51f35c76076489dd83ccb46d"), "age" : 23, "name" : "youngsik", "title" : "hi" }


  - SECONDARY 들어가 보기 : PRIMARY가 다운되면 show collections에서 에러가 발생하지 않고 자신이 PRIMARY로 된다

    PRIMARY 1개 + SECONDARY 2개 + 여분 n개

/mongodb_2.4.5> mongo localhost:30000
MongoDB shell version: 2.4.5
connecting to: localhost:30000/test
dowonSet:SECONDARY> show dbs
dowonDB    0.203125GB
local    0.328125GB
test    (empty)
dowonSet:SECONDARY> use dowonDB
switched to db dowonDB
dowonSet:SECONDARY> db
dowonDB
dowonSet:SECONDARY> show collections
Sat Jul 27 14:42:38.244 JavaScript execution failed: error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:L128

///////////// 20000 번 포트의 mongod를 강제로 다운시켰을 때 : 40000 포트의 mongod가 PRIMARY가 된다

/mongodb_2.4.5> mongo localhost:40000
MongoDB shell version: 2.4.5
connecting to: localhost:40000/test
dowonSet:PRIMARY>
dowonSet:PRIMARY> use dowonDB
switched to db dowonDB
dowonSet:PRIMARY> show collections
person
system.indexes
dowonSet:PRIMARY> db.person.find();
{ "_id" : ObjectId("51f35c76076489dd83ccb46d"), "age" : 23, "name" : "youngsik", "title" : "hi" }


  - SECONDARY로 들어가서 show collections가 안될 경우 rs.slaveOk() 명령수행

/mongodb_2.4.5> mongo localhost:30000
MongoDB shell version: 2.4.5
connecting to: localhost:30000/test
dowonSet:SECONDARY>
dowonSet:SECONDARY> show dbs
dowonDB    0.203125GB
local    0.328125GB
test    (empty)
dowonSet:SECONDARY> use dowonDB
switched to db dowonDB
dowonSet:SECONDARY> db
dowonDB
dowonSet:SECONDARY> show collections
Sat Jul 27 14:48:38.663 JavaScript execution failed: error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:L128
dowonSet:SECONDARY> rs.slaveOk();
dowonSet:SECONDARY> show collections
person
system.indexes
dowonSet:SECONDARY> db.person.find();
{ "_id" : ObjectId("51f35c76076489dd83ccb46d"), "age" : 23, "name" : "youngsik", "title" : "hi" }


현재까지 Shard 1개를 만들었다. (구현되지 않는 것은 거짓말이다)



<참조>

  - 10gen Sharding 메뉴얼

저작자 표시 비영리 변경 금지
신고
posted by peter yun 윤영식
2013.01.05 16:26 MongoDB/Prototyping

MongoDB에서 Replica Set을 구성하였을 때 mongodb를 접속하는 client에 대하여 routing 서비스를 하는 mongos(몽고스)에 대해 알아보자.



1. 구성 아키텍쳐

  - Shard 한개에 mongod 한개를 맵핑한다

  - config 한개에 mongod 한개만 사용한다



2) 구성하기 

  - shard1, shard2, shard3, config, mongos를 간단히 만들어 보자 (shard를 replica set으로 구성은 배재, 참조)

  - Shard server 3개 띄우기 : sdb01, sdb02, sdb03 디렉토리는 미리 만들어 놓는다. (기동은 개별 콘솔에서 실행함)

$ ./mongod --shardsvr --dbpath /home/dowon/MongoDB/sdb01 --port 10000

Fri Jan  4 22:25:03 

Fri Jan  4 22:25:03 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.

.. 중략 ...

Fri Jan  4 22:25:03 [initandlisten] build info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49

Fri Jan  4 22:25:03 [initandlisten] options: { dbpath: "/home/dowon/MongoDB/sdb01", port: 10000, shardsvr: true }

Fri Jan  4 22:25:03 [websvr] admin web console waiting for connections on port 11000


./mongod --shardsvr --dbpath /home/dowon/MongoDB/sdb02 --port 20000
./mongod --shardsvr --dbpath /home/dowon/MongoDB/sdb03 --port 30000

////////////////////////////
// config srever를 띄웠다. 
$ ./mongod --configsvr --dbpath /home/dowon/MongoDB/sdb04c --port 40000
Fri Jan  4 22:29:47 
Fri Jan  4 22:29:47 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.
Fri Jan  4 22:29:47 
Fri Jan  4 22:29:47 [initandlisten] MongoDB starting : pid=8201 port=40000 dbpath=/home/dowon/MongoDB/sdb04c 32-bit host=localhost.localdomain
Fri Jan  4 22:29:48 [initandlisten] waiting for connections on port 40000
Fri Jan  4 22:29:48 [websvr] admin web console waiting for connections on port 41000

////////////////////////////
// mongos 띄우기 : chunksize 정하지 않으며 1Mbytes이다. 
$ ./mongos --configdb localhost:40000 --chunkSize 1
Fri Jan  4 22:32:04 warning: running with 1 config server should be done only for testing purposes and is not recommended for production
Fri Jan  4 22:32:04 [mongosMain] MongoS version 2.2.2 starting: pid=8239 port=27017 32-bit host=localhost.localdomain (--help for usage)
Fri Jan  4 22:32:04 [mongosMain] git version: d1b43b61a5308c4ad0679d34b262c5af9d664267
Fri Jan  4 22:32:04 [mongosMain] build info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Fri Jan  4 22:32:11 [Balancer] distributed lock 'balancer/localhost.localdomain:27017:1357367525:1804289383' unlocked. 
Fri Jan  4 22:32:17 [Balancer] distributed lock 'balancer/localhost.localdomain:27017:1357367525:1804289383' acquired, ts : 50e7c8f1b1a36d7262b31138


3) mongos 에서 환경 구성하기

  - mongo shell로 들어가서 shard 환경을 만든다 : 자동으로 mongos로 들어간다 (프롬프트 명칭이 mongos가 됨)

$ ./mongo

MongoDB shell version: 2.2.2

connecting to: test

mongos> use admin

switched to db admin

mongos> db

admin

mongos> show dbs

config 0.046875GB

mongos> use peopleDB
switched to db peopleDB
// config 셋팅을 하지 않고 사용하면 다음과 같은 에러가 발생한다. 
mongos> db.people.save({_id:1, age:11, name:'dowon1', sex:false});
Fri Jan  4 22:38:03 uncaught exception: error {
"$err" : "error creating initial database config information :: caused by :: can't find a shard to put new db on",
"code" : 10185
}
mongos> 

/////////////////////////
// 다시 mongo로 들어온다. 
// 그리고 shard에 서버를 추가해 준다.
$ ./mongo
MongoDB shell version: 2.2.2
connecting to: test
mongos> db
test
mongos> use admin
switched to db admin
mongos> db
admin
mongos> db.runCommand({addshard:'localhost:10000'});
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> db.runCommand({addshard:'localhost:20000'});
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> db.runCommand({addshard:'localhost:30000'});
{ "shardAdded" : "shard0002", "ok" : 1 }

////////////////////////////////
// mongos를 통하여 CRUD를 한다
// 과연 입력한 데이터는 Shard가 되었을까? 
mongos> 
mongos> use testShard
switched to db testShard
mongos> db
testShard
mongos> db.people.save({_id:1, age:11, name:'dowon1', sex:false});
mongos> db.people.save({_id:2, age:22, name:'dowon2', sex:true});
mongos> db.people.save({_id:3, age:33, name:'dowon3', sex:false});
mongos> db.people.find();
{ "_id" : 1, "age" : 11, "name" : "dowon1", "sex" : false }
{ "_id" : 2, "age" : 22, "name" : "dowon2", "sex" : true }
{ "_id" : 3, "age" : 33, "name" : "dowon3", "sex" : false }

/////////////////////////////////////
// 결과 확인 : auto shard가 안되어 있다
// sdb01에만 데이터가 저장되어 있다. 이유 : config server를 가동시키지 않았기 때문이다.
[dowon@localhost MongoDB]$ cd sdb01
[dowon@localhost sdb01]$ ll
total 65548
-rwxrwxr-x. 1 dowon dowon        5 Jan  4 22:25 mongod.lock
-rw-------. 1 dowon dowon 16777216 Jan  4 22:43 testShard.0
-rw-------. 1 dowon dowon 33554432 Jan  4 22:42 testShard.1
-rw-------. 1 dowon dowon 16777216 Jan  4 22:43 testShard.ns
drwxrwxr-x. 2 dowon dowon     4096 Jan  4 22:42 _tmp
[dowon@localhost sdb01]$ cd ../sdb02
[dowon@localhost sdb02]$ ll
total 4
-rwxrwxr-x. 1 dowon dowon 5 Jan  4 22:25 mongod.lock
[dowon@localhost sdb02]$ cd ../sdb03
[dowon@localhost sdb03]$ ll
total 4
-rwxrwxr-x. 1 dowon dowon 5 Jan  4 22:25 mongod.lock


4) Shard 테스트를 위한 데이터 만들고 확인하기 

  - shard 연결 환경을 설정한다. (참조 동영상, Shard Guide blog)

////////////////////////////////

// 10000 port mongodb로 들어가자

// 이것은 10000 port의 mongodb로 direct 들어가는 것이다. 

$ ./mongo localhost:10000

MongoDB shell version: 2.2.2

connecting to: localhost:10000/test

> show dbs

local (empty)

testShard 0.0625GB

> use testShard

switched to db testShard

> db

testShard

> show collections

people

system.indexes

> db.people.find();

{ "_id" : 1, "age" : 11, "name" : "dowon1", "sex" : false }

{ "_id" : 2, "age" : 22, "name" : "dowon2", "sex" : true }

{ "_id" : 3, "age" : 33, "name" : "dowon3", "sex" : false }


////////////////////////////
// 20000, 30000 port로 mongos를 통하지 않고 mongodb로 바로 접속하면 
// testShard DB는 없다. 
$ ./mongo localhost:20000
MongoDB shell version: 2.2.2
connecting to: localhost:20000/test
> show dbs
local (empty)

/////////////////////////////////
// mongos로 들어간다. use admin 
// Sharding 환경설정하기
$ ./mongo
MongoDB shell version: 2.2.2
connecting to: test
mongos> dbs
Fri Jan  4 22:54:15 ReferenceError: dbs is not defined (shell):1
mongos> show dbs
config 0.046875GB
testShard 0.0625GB
mongos> use admin
switched to db admin
mongos> db
admin
mongos> db.runCommand({enablesharding:'testShard'});
{ "ok" : 1 }
mongos> db.runCommand({shardcollection:'people', key:{_id:1}});
{ "ok" : 0, "errmsg" : "bad ns[people]" } <-- namespace 에러 
mongos> db.runCommand({shardcollection:'testShard.people', key:{_id:1}});  // _id:1 의 1은 sorting
{ "collectionsharded" : "testShard.people", "ok" : 1 }

//////////////////////////////////////////////
// people collection에 데이터를 넣어보자
mongos> for(var i=6 ; i<100000 ; i++) {
      db.people.save({_id:i, age:44, name:'dowon'+i, sex:true});
   }

//////////////////////////////////////////////////////////////////////////
// 다른 mongo에서 show dbs를 해보면 testShard DB가 생성되었음을 볼 수 있다. 
// chunkSize 1Mbytes 이상이면 sharding이 이루어진다. 
[dowon@localhost mongodb]$ ./mongo localhost:20000
MongoDB shell version: 2.2.2
connecting to: localhost:20000/test
> show dbs
local (empty)
> show dbs
admin (empty)
local (empty)
testShard 0.0625GB
> use testShard
switched to db testShard
> db.people.find();
{ "_id" : 54053, "age" : 44, "name" : "dowon54053", "sex" : true } // 54053 번부터 Sharding 되었다. 
{ "_id" : 54054, "age" : 44, "name" : "dowon54054", "sex" : true }
{ "_id" : 54055, "age" : 44, "name" : "dowon54055", "sex" : true }
{ "_id" : 54056, "age" : 44, "name" : "dowon54056", "sex" : true }
{ "_id" : 54057, "age" : 44, "name" : "dowon54057", "sex" : true }

[dowon@localhost mongodb]$ ./mongo localhost:30000
MongoDB shell version: 2.2.2
connecting to: localhost:30000/test
local (empty)
> show dbs
admin (empty)
local (empty)
testShard 0.0625GB
> use testShard
switched to db testShard
> db.people.find();
{ "_id" : 22200, "age" : 44, "name" : "dowon22200", "sex" : true } // 22200 번부터 Sharding 되었다. 
{ "_id" : 22201, "age" : 44, "name" : "dowon22201", "sex" : true }
{ "_id" : 22202, "age" : 44, "name" : "dowon22202", "sex" : true }
{ "_id" : 22203, "age" : 44, "name" : "dowon22203", "sex" : true }

< 결론 > 
Sharding 시켜놓은 서버로 ChunkSize 크기에 따라 데이터가 분산되어 저장된다. 

 

* 요약 (명령 참조)

  - mongo admin 으로 들어간다

  - runCommand 통하여 세가지 설정을 수행한다. 

  - chunkSize에 따라서 Sharding이 이루어진다.


* Sharding 개념 다시 정리 하기


* MongoDB의 철학은?

데이터와 데이터로직의 분리 

 - 데이터 : JSON (실제 BSON : binary json 형태로 저장됨)

 - 데이터로직 : JavaScript


<참조>

  - 10gen 설정 튜토리얼

저작자 표시 비영리 변경 금지
신고
posted by peter yun 윤영식
prev 1 next