What is the MongoDB Sharding?
MongoDB Sharding provides a mechanism to distribute the dataset on multiple nodes also called shards. MongoDB uses sharding to deploy large datasets and support high throughput operations. The challenges with database servers which are having larger datasets are high query rate processing that exhausted the CPU capacity of the server, the working set size exceeds the physical memory and the read/write throughput exceeds I/O operation.
These issues are resolved in MongoDB by using Horizontal Scaling that is also called Scale-Out. MongoDB supports Horizontal Scaling that divides the large datasets over multiple servers and, adds additional servers as per the requirement in the shard. In the MongoDB shards, there can be multiple servers and the capacity and overall speed of a single server may not be that high but datasets are distributed on multiple servers so each server performs sub-task of the overall workload and together it delivers the best performance compared to a single high configured server. To expand the MongoDB shard's capacity, the requirement is to add the additional server in the shard that is lower in cost compared to the high-end hardware for a single machine.
The following diagram represents the components of MongoDB Sharding also shows the interaction between each component of MongoDB Sharding in a sharded cluster.
Components of MongoDB Sharding
MongoDB Sharded cluster consists of the following three components.
- Routers(mongos)
- Config Server Replica Sets
- Shards
Let us see each Component of the MongoDB Sharded cluster in detail.
1. Routers(mongos)
The MongoDB Routers(mongos) instances are used to route the client application queries and write operations on the appropriate shards in MongoDB sharded cluster. Mongos reads the metadata information from the Configuration server to know the detail of datasets such as which data is present on which shards and this was it cache the metadata information and then route the queries and write operation on the relevant shards. It does not have any persistent state hence it consumes very few system resources.
1.1 MongoDB Cluster Request Routing
MongoDB Routers(mongos) checks the below points before routing the query to the cluster.
- It checks the detail of shards from metadata and then decides which shard should receive the query request.
- Once the target shards are identified then it creates a cursor on all identified shards.
There are two types of operations performed to route the client query, Broadcast Operations, and Targeted Operations.
Let us see each operation in the below section.
a. Broadcast Operations
In the Broadcast Operations, Mongos instance sends the query request to all shards to check which shard has the data. Once it receives the response from all shards it sends the result to the client application. The Broadcast Operations response is dependent on the network latency, the load on an individual shard, and the number of the document which are returning in a query response.
The methods such as updateMany() and deleteMany() triggers the Broadcast Operations.
In the following figure, we can see that the client query request is routing to all shards of the cluster.
b. Targeted Operations
In the Targeted Operations, the Mongos instances use the shard key value to determine the chunk of data. The operation such as insertOne(),updateOne(), replaceOne() and deleteOne() are the Targeted Operations.
In the following figure, we can see that if a shard key{a:"zl"} is included in the query then it is routed to the specific shard or set of shards.
2. Config Server Replica Sets
The Config Servers are used to store the metadata information of the sharded cluster. The metadata information contains the detail of all datasets and chunks on the cluster. This information is used by the Mongos process to route read and writes operations on the specific shard. The Config Server also stores the Role-Based Access Control or internal authentication of the cluster. Each sharded cluster has its Config Servers.
3. Shards
MongoDB shard is a single instance of MongoDB cluster that is used to store the subset of datasets. It is deployed as a replica set to provides the redundancy and high availability of data. For example, if we have a dataset of 10 GB and we have 5 shards then the data should be divided into 2 GB per shard.
In a Sharded cluster, there is a primary shard for each database that holds the un-sharded collections of that database. It has no relation with the Primary in the replica set.
Benefits of MongoDB Sharding
Using MongoDB sharding we can scale the database size to handle unexpected high loads. MongoDB sharding provides the capability to increase the read/write throughput for client queries, high availability of the system by replicating data, and storage capacity by adding nodes.
Let us see the benefits of MongoDB sharding in the below section.
Increased Read And Write Throughput
The performance of reading and writing operations is increased if the datasets are distributed properly on shards based on the shard key.
Increased Storage Capacity
The storage capacity is increased when we add the shards in the MongoDB cluster. The MongoDB scalability can be near infinitive.
High Availability
MongoDB provides the high availability of datasets by replication subset of data so if there is an issue with a shard in that case the cluster will still accept the read and write operation.
Shard Keys
MongoDB Shard key is the index filed which decides how the distribution of documents will take place in the shards cluster. MongoDB divides the Shard key into a range of Shard key values and each range is connected with the chunk. So MongoDB distributes each Chunk evenly on Shards as mentioned in the below figure.
Characteristics of Perfect Shard Key
The perfect Shard Key should have the following characteristics.
- The inserts, updates, and deletes operations should be distributed evenly on all shards of the Cluster.
- All Queries should be distributed evenly on all shards of the Cluster.
- An operation like Insert, Update, Delete should be sent to only those shards which are having Data. The Shards which are not having data should not receive the request.
- The Query operation also sent to those Shards which are holding the data.
Shard Key Considerations
The following are the criteria for a good Shard Key.
- Cardinality
- Write Distribution
- Read Distribution
- Read Targeting
- Read Locality
Chunks
MongoDB Chunks are the group of logical documents which are moved on Shards depending upon the sharding range mapping. The Chunk has an upper range and a lower range based on the Shard Key, so if the Chunk reaches the defined size then the Chunk is split into two Chunks and the Balancer is used to balance the Chunks across the Shards.
