AIMDek Technologies AIMDek Technologies AIMDek Technologies AIMDek Technologies
  • SERVICES
    • Enterprise Applications
      • Enterprise Portals
      • Web Application
      • E-Commerce Portal Development
    • Liferay
    • Integration Services
    • UX UI Design
    • Mobile App Development Service
    • Consulting Services
    • Migrations and Upgrades
    • QA & Testing
    • Cloud Infrastructure
  • SOLUTIONS
    • Healthcare IT
      • EHR Software
      • HMS Software
      • Patient Portal
      • Healthcare Mobility
    • Mobility Solutions
    • PLM Solutions
      • ARAS PLM
      • Oracle PLM
    • Robotic Process Automation
    • Portal Solutions
      • Intranet Solution
      • Partner Portal
      • Collaboration Portal
    • ECM Solution
    • ERP Solutions
  • PARTNERSHIP
    • Microsoft
    • Salesforce
  • RESOURCES
    • Blogs
    • Webinars
    • Case Studies
  • COMPANY
    • About Us
    • Industries
    • Our Infrastructure
    • Our culture
    • Careers
  • CONTACT US
AIMDek Technologies AIMDek Technologies
  • SERVICES
    • Enterprise Applications
      • Enterprise Portals
      • Web Application
      • E-Commerce Portal Development
    • Liferay
    • Integration Services
    • UX UI Design
    • Mobile App Development Service
    • Consulting Services
    • Migrations and Upgrades
    • QA & Testing
    • Cloud Infrastructure
  • SOLUTIONS
    • Healthcare IT
      • EHR Software
      • HMS Software
      • Patient Portal
      • Healthcare Mobility
    • Mobility Solutions
    • PLM Solutions
      • ARAS PLM
      • Oracle PLM
    • Robotic Process Automation
    • Portal Solutions
      • Intranet Solution
      • Partner Portal
      • Collaboration Portal
    • ECM Solution
    • ERP Solutions
  • PARTNERSHIP
    • Microsoft
    • Salesforce
  • RESOURCES
    • Blogs
    • Webinars
    • Case Studies
  • COMPANY
    • About Us
    • Industries
    • Our Infrastructure
    • Our culture
    • Careers
  • CONTACT US
Jun 23
Managing the Data Volume with MongoDB

Managing the Data Volume with MongoDB

  • 23/06/2017
  • Avakash Dekavadiya(Director & Head - Global Sales)

Table of Contents

  • MongoDB
    • MongoDB is a cross-platform and schema less database. It provides high performance, high availability, and easy scalability. It works on concept of collection and document.Collection is equivalent of a RDBMS table while a document is a set of key-value pairs, just like tuple/row in RDBMS.It stores data in BSON which is binary JSON.
  • Sharding
    • Sharding is the mechanism of storing data across various machines to meet the needs of  growing data. A single machine may not be enough to store data when data grows exponentially. Sharding solves this growing data problem with horizontal scaling. It connects multiple machines to support the data volume and address the challenge of multiple read and write operations simultaneously.
  • Types of Server 
    • Shard Server :- A shard is a MongoDB physical instance which stores data records. Shard server can be either a single MongoDB instance or a replica set.
    • Config Server :- Config servers are important MongoDB instances which store the metadata for a sharded cluster. It maps App server to Shard server. So, it is recommended to use three config servers for production deployments.
    • App Server :- It connects Users with Shard server using metadata of Config server which decide which Shard server to connect. App server also executes read/write operation on sharded clusters.If you want to divide client request, then use multiple App server which is more reliable for robust applications. Multiple users can access simultaneously.
  • Fig.1 Sharding Server
  • Replication
    • Replication provides redundancy and increases data availability.There are two kinds of sever in it: primary and secondary. We can write data into primary and only read from secondary and acts during Automatic Failover. When a primary does not communicate with the other members of the set for more than 10 seconds then secondary node becomes primary node automatically.So replication helps our application to be redundant without failure.
  • Fig. 2 Replication
  • Sharding and Replication
    • Sharding and Replication both are combined to harness the benefits of both the features of MongoDB. Sharding addresses data growth and Replication provides for automatic failure recovery.
    • To configure Sharding and Replication in local, we will demonstrate with one App server, one Config server, two Shard servers and one Replica server corresponding to each primary server. For production environment, it is ideal to use more than one App server to distribute client side requests. More than three Config servers are required in production to avoid single point failure. If the Config server is inaccessible, the cluster is not accessible. More than two shard must be configured or add more shards according to your application. It is recommended to use at least two replica set for each primary node in production for failure recovery.
  • Fig. 3 Sharding and Replication
  • Configuring Sharding and Replication Servers
    • Create Config server
    • mongod –configsvr –port 27010  –dbpath
    • D:/Mongo_Cluster_Example/configDb
    • Create App server
    • mongod -configdb localhost:27010 –port 27011
    • Create Shard 1
    • MongoDB -shardsvr –replSet shard1 –port 27012 –dbpath D:/Mongo_Cluster_Example/db1
    • Create Shard 2
    • mongod -shardsvr –replSet shard 2 –port 27014 –dbpath D:/Mongo_Cluster_Example/db2
    • Create ReplSet 1
    • mongod –replSet shard1 –port 27013 –dbpath D:/Mongo_Cluster_Example/db1A
    • Create ReplSet 2
    • mongod –replSet shard 2 –port 27015 –dbpath D:/Mongo_Cluster_Example/db2A
    • Connect replica set 1 to primary shard
    • mongo –host localhost –port 27012 config = {_id: “shard1”, members: [ { _id: 0, host: “localhost:27012”} , { _id: 1, host: “localhost:27013”} ] } rs.initiate(config) rs.status()
    • Connect replica set 2 to primary shard
    • mongo –host localhost –port 27014 config = {_id: “shard2”, members: [ { _id: 0, host: “localhost:27014”} , { _id: 1, host: “localhost:27015”} ] } rs.initiate(config) rs.status()
    • Set App Server metadata
    • mongo –port 27011 –host localhost sh.addShard( “shard1/localhost:27012,localhost:27013” ) sh.addShard( “shard2/localhost:27014,localhost:27015” ) use mongodb; db.createCollection(“clustering”) sh.enableSharding(“mongodb”) sh.shardCollection(“mongodb.clustering”, {“_id” : 1}) for ( i = 1; i < 999; i++ ) {  db.clustering.insert({ type: “sharding”, score : i});  db.clustering.insert({ type: “replication”, score : i }); } sh.status()
    • Check in shard 1 PRIMARY
    • mongo –host localhost –port 27012 use mongodb db.clustering.find().sort({_id : 1}).limit(1).pretty()
    • Check in shard 1 SECONDARY
    • mongo –host localhost –port 27013 rs.slaveOk() use mongodb db.clustering.find().sort({_id : 1}).limit(1).pretty()
    • Check in shard 2 PRIMARY
    • mongo –host localhost –port 27014 use mongodb db.clustering.find().sort({_id : 1}).limit(1).pretty()
    • Check in shard 2 SECONDARY
    • mongo –host localhost –port 27015 rs.slaveOk() use mongodb db.clustering.find().sort({_id : 1}).limit(1).pretty()
    • Here, Step 1 to 6 are for connecting all necessary nodes. Steps 7 , 8 and 9 for connecting them together. Step 10 to 13 are for testing your configuration. You will find same data in shard1 and it’s replica set, similarly for shard 2 and it’s replica set. Also data which is inserted through App server are distributed across both shard servers.
    • Concept of connected devices and better internet services are creating volumes of data every nano second. It’s imperative for organizations to ensure quick access to data while maintaining server uptime across 24 hours. MongoDB comes to rescue with its sharding and replication features that addresses both the issues: quick access and server uptime.
    • Our team of experts guide you to set up data servers to manage your ever growing data needs. Call us to know more about our MongoDB expertise.
Post Views: 733

Share On : [feather_share show="twitter, facebook,linkedin" hide="reddit, pinterest, google_plus, tumblr, mail"]

MongoDB

MongoDB is a cross-platform and schema less database. It provides high performance, high availability, and easy scalability. It works on concept of collection and document.Collection is equivalent of a RDBMS table while a document is a set of key-value pairs, just like tuple/row in RDBMS.It stores data in BSON which is binary JSON.

Sharding

Sharding is the mechanism of storing data across various machines to meet the needs of  growing data. A single machine may not be enough to store data when data grows exponentially. Sharding solves this growing data problem with horizontal scaling. It connects multiple machines to support the data volume and address the challenge of multiple read and write operations simultaneously.

Types of Server 

  1. Shard Server :- A shard is a MongoDB physical instance which stores data records. Shard server can be either a single MongoDB instance or a replica set.

  2. Config Server :- Config servers are important MongoDB instances which store the metadata for a sharded cluster. It maps App server to Shard server. So, it is recommended to use three config servers for production deployments.

  3. App Server :- It connects Users with Shard server using metadata of Config server which decide which Shard server to connect. App server also executes read/write operation on sharded clusters.If you want to divide client request, then use multiple App server which is more reliable for robust applications. Multiple users can access simultaneously.

Sharding Server

Fig.1 Sharding Server

Replication

Replication provides redundancy and increases data availability.There are two kinds of sever in it: primary and secondary. We can write data into primary and only read from secondary and acts during Automatic Failover. When a primary does not communicate with the other members of the set for more than 10 seconds then secondary node becomes primary node automatically.So replication helps our application to be redundant without failure.

Sharding Server

Fig. 2 Replication

Sharding and Replication

Sharding and Replication both are combined to harness the benefits of both the features of MongoDB. Sharding addresses data growth and Replication provides for automatic failure recovery.

To configure Sharding and Replication in local, we will demonstrate with one App server, one Config server, two Shard servers and one Replica server corresponding to each primary server. For production environment, it is ideal to use more than one App server to distribute client side requests. More than three Config servers are required in production to avoid single point failure. If the Config server is inaccessible, the cluster is not accessible. More than two shard must be configured or add more shards according to your application. It is recommended to use at least two replica set for each primary node in production for failure recovery.

Sharding + Replication

Fig. 3 Sharding and Replication

Configuring Sharding and Replication Servers

  1. Create Config server

    1. mongod –configsvr –port 27010  –dbpath

    2. D:/Mongo_Cluster_Example/configDb

  2. Create App server

    1. mongod -configdb localhost:27010 –port 27011

  3. Create Shard 1

    1. MongoDB -shardsvr –replSet shard1 –port 27012 –dbpath D:/Mongo_Cluster_Example/db1

  4. Create Shard 2

    1. mongod -shardsvr –replSet shard 2 –port 27014 –dbpath D:/Mongo_Cluster_Example/db2

  5. Create ReplSet 1

    1. mongod –replSet shard1 –port 27013 –dbpath D:/Mongo_Cluster_Example/db1A

  6. Create ReplSet 2

    1. mongod –replSet shard 2 –port 27015 –dbpath D:/Mongo_Cluster_Example/db2A

  7. Connect replica set 1 to primary shard

    1. mongo –host localhost –port 27012
      config = {_id: “shard1”, members: [ { _id: 0, host: “localhost:27012”} , { _id: 1, host: “localhost:27013”} ] }
      rs.initiate(config)
      rs.status()

  8. Connect replica set 2 to primary shard

    1. mongo –host localhost –port 27014
      config = {_id: “shard2”, members: [ { _id: 0, host: “localhost:27014”} , { _id: 1, host: “localhost:27015”} ] }
      rs.initiate(config)
      rs.status()

  9. Set App Server metadata

    1. mongo –port 27011 –host localhost
      sh.addShard( “shard1/localhost:27012,localhost:27013” )
      sh.addShard( “shard2/localhost:27014,localhost:27015” )
      use mongodb;
      db.createCollection(“clustering”)
      sh.enableSharding(“mongodb”)
      sh.shardCollection(“mongodb.clustering”, {“_id” : 1})
      for ( i = 1; i < 999; i++ ) {
       db.clustering.insert({ type: “sharding”, score : i});
       db.clustering.insert({ type: “replication”, score : i });
      }
      sh.status()

  10. Check in shard 1 PRIMARY

    1. mongo –host localhost –port 27012
      use mongodb
      db.clustering.find().sort({_id : 1}).limit(1).pretty()

  11. Check in shard 1 SECONDARY

    1. mongo –host localhost –port 27013
      rs.slaveOk()
      use mongodb
      db.clustering.find().sort({_id : 1}).limit(1).pretty()

  12. Check in shard 2 PRIMARY

    1. mongo –host localhost –port 27014
      use mongodb
      db.clustering.find().sort({_id : 1}).limit(1).pretty()

  13. Check in shard 2 SECONDARY

    1. mongo –host localhost –port 27015
      rs.slaveOk()
      use mongodb
      db.clustering.find().sort({_id : 1}).limit(1).pretty()

Here, Step 1 to 6 are for connecting all necessary nodes. Steps 7 , 8 and 9 for connecting them together. Step 10 to 13 are for testing your configuration. You will find same data in shard1 and it’s replica set, similarly for shard 2 and it’s replica set. Also data which is inserted through App server are distributed across both shard servers.

Concept of connected devices and better internet services are creating volumes of data every nano second. It’s imperative for organizations to ensure quick access to data while maintaining server uptime across 24 hours. MongoDB comes to rescue with its sharding and replication features that addresses both the issues: quick access and server uptime.

Our team of experts guide you to set up data servers to manage your ever growing data needs. Call us to know more about our MongoDB expertise.

Related

Spread the love
    

Comments are closed.

Subscribe for more interesting blogs


    AIMDek Logo
    canda flag icon

    CANADA
    ONTARIO

      +1 437-999-0755

    usa flag icon

    USA
    TEXAS

      +1 956-506-1010

    india flag logo

    INDIA
    AHMEDABAD

      +91 9879856334

    QUICK LINKS:
    About Us
    Service
    Solutions
    Case Study
    Our Team
    Contact Us

    CONNECT WITH US:
    marketing@aimdek.com

      (437)-999-0755

    linkedin icon
    facebook icon
    tweeter icon
    skype icon
    youtube icon
    instagram icon

    REVIEWED ON




    REVIEWED ON




    PARTNERS

    ISO

    CERTIFIED
    COMPANY

    @ 2021 AIMDek Technologies Private Limited. All rights reserved