
在NoSQL数据库领域,MongoDB无疑是文档型数据库的标杆之作。它是一个基于分布式文件存储的NoSQL数据库,旨在为Web应用提供可扩展的高性能数据存储解决方案,凭借灵活的存储模式、优异的性能和强大的扩展能力,成为互联网、大数据等场景下的首选数据库之一。很多开发者日常使用MongoDB进行数据存储、查询,但对其核心功能背后的架构设计、算法支撑却了解不深。今天这篇博客,就带大家从“是什么(功能特性)”到“为什么(架构算法)”,全面拆解MongoDB的核心逻辑,帮你深入了解这款数据库。
Learn and share.

在NoSQL数据库领域,MongoDB无疑是文档型数据库的标杆之作。它是一个基于分布式文件存储的NoSQL数据库,旨在为Web应用提供可扩展的高性能数据存储解决方案,凭借灵活的存储模式、优异的性能和强大的扩展能力,成为互联网、大数据等场景下的首选数据库之一。很多开发者日常使用MongoDB进行数据存储、查询,但对其核心功能背后的架构设计、算法支撑却了解不深。今天这篇博客,就带大家从“是什么(功能特性)”到“为什么(架构算法)”,全面拆解MongoDB的核心逻辑,帮你深入了解这款数据库。
还是蛮简单的,驱动把任务全部做掉了
package com.djhu.mongodb.test;
import java.util.Arrays;
import org.bson.Document;
import com.mongodb.MongoClient;
import com.mongodb.ServerAddress;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
public class ReplTest
{
private static void testInsert()
{
MongoClient mongoClient = new MongoClient(Arrays.asList(
new ServerAddress("172.16.172.4", 27017),
new ServerAddress("172.16.172.4", 27018),
new ServerAddress("172.16.172.4", 27019)));
MongoDatabase db = mongoClient.getDatabase("test");
MongoCollection collection = db.getCollection("person");
Document doc = new Document();
doc.put("name", "tuzi");
doc.put("age", 27);
doc.put("sex", "Female");
collection.insertOne(doc);
}
public static void main(String[] args)
{
testInsert();
}
}
如果遇到下面的错误,是因为用了localhost作为replSet的地址,重新config一下就好了
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:50)
at com.mongodb.connection.SocketStream.open(SocketStream.java:58)
... 3 more
Exception in thread "main" com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches PrimaryServerSelector. Client view of cluster state is {type=REPLICA_SET, servers=[{address=localhost:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused: connect}}, {address=localhost:27018, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused: connect}}, {address=localhost:27019, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused: connect}}, {address=localhost:27020, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused: connect}}]
at com.mongodb.connection.BaseCluster.createTimeoutException(BaseCluster.java:370)
at com.mongodb.connection.BaseCluster.selectServer(BaseCluster.java:101)
at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:75)
at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:71)
at com.mongodb.binding.ClusterBinding.getWriteConnectionSource(ClusterBinding.java:68)
at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:175)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:141)
at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:72)
at com.mongodb.Mongo.execute(Mongo.java:747)
at com.mongodb.Mongo$2.execute(Mongo.java:730)
at com.mongodb.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:482)
at com.mongodb.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:277)
at com.djhu.mongodb.test.ReplTest.testInsert(ReplTest.java:28)
at com.djhu.mongodb.test.ReplTest.main(ReplTest.java:33)
1、被引用数据
db.address.insert({"city":"shanghai","street":"huaihai road","no":"101"})
db.address.insert({"city":"beijing","street":"taipingqiao road","no":"102"})
2、引用数据
db.persons.insert({"name":"joe","address":{"$ref":"address","$id": ObjectId("55f522e96811e30fd403e83d"),"$db": "test"},"age":20,"sex":"male"})
db.persons.insert({"name":"leo","address":{"$ref":"address","$id": ObjectId("55f522f46811e30fd403e83e"),"$db": "test"},"age":21,"sex":"male"})
3、查询被引用数据
var user = db.persons.findOne({"name":"joe"})
var addressRef = user.address
db[addressRef.$ref].findOne({"_id":(addressRef.$id)})
1、备份
mongodump --host localhost --port 27027 -d test -c patient -o D:\Database\MongoDB3\backup
2、还原
mongorestore --host localhost --port 27027 --drop -d test D:\Database\MongoDB3\backup
MongoDB中存储的文档必须有一个”_id”键。这个键的值可以是任何类型的,默认是个ObjectId对象。该_id用来确保集合里面每个文档都能被唯一标识,并用来在多个服务器上同步数据。
ObjectId是一个12字节BSON类型数据,格式如下:
前4个字节表示时间戳
接下来的3个字节是机器标识码
接的两个字节由进程id组成(PID)
最后三个字节是随机数
myObjectId = ObjectId() myObjectId.getTimestamp()
0、数据准备
for(var i=0;i<10000;i++){
var patid="pat"+i;
var patname="name"+i;
var sex="M";
var age=parseInt(100*Math.random(i));
db.patient.insert({"patid":patid,"patname":patname,"sex":sex,"age":age,address:{"city":"shanghai","street":"huaihai road"}});
}
1、count
db.patient.count({"age":12})
2、distinct
db.patient.distinct("age")
3、min, max, sum,avg
db.patient.aggregate([{$group:{_id:"$item",maxAge:{$max:"$age"}}}])
db.patient.aggregate([{$group:{_id:"$item",minAge:{$min:"$age"}}}])
db.patient.aggregate([{$group:{_id:"$item",sumAge:{$sum:"$age"}}}])
db.patient.aggregate([{$group:{_id:"$item",avgAge:{$avg:"$age"}}}])
4、group
db.patient.group({
"key":{"age":true},
"initial":{"patids":[]},
"reduce":function(item,out){out.patids.push(item.patid);},
"finalize":function(out){out.count=out.patids.length;},
"condition":{"age":{$lte:18}}
})
5、map reduce
map=function(){emit(this.age,1);}
reduce=function(key,values){return values.length;}
mropt={"out":"mrresult"}
db.patient.mapReduce(map,reduce,mropt).find()
0、数据准备
for(var i=0;i<10000;i++){
var patid="pat"+i;
var patname="name"+i;
var sex="M";
var age=parseInt(100*Math.random(i));
db.patient.insert({"patid":patid,"patname":patname,"sex":sex,"age":age,address:{"city":"shanghai","street":"huaihai road"}});
}
1、默认为全局更新
db.patient.find({"patid":"pat100"})
db.patient.update({"patid":"pat100"},{"patid":"pat100","sex":"F"})
db.patient.find({"patid":"pat100"})
2、局部更新$set
db.patient.find({"patid":"pat101"})
db.patient.update({"patid":"pat101"},{$set:{"sex":"F"}})
db.patient.find({"patid":"pat101"})
3、局部更新$inc
db.patient.find({"patid":"pat102"})
db.patient.update({"patid":"pat102"},{$inc:{"age":-100}})
db.patient.find({"patid":"pat102"})
4、批量更新
db.patient.find({"age":10})
db.patient.update({"age":10},{$set:{"age":11}})
db.patient.find({"age":10})
db.patient.update({"age":10},{$set:{"age":11}},false,true)
db.patient.find({"age":10})
5、更新时,没有匹配则插入
db.patient.find({"patid":"pidx001"})
db.patient.update({"patid":"pidx001"},{"patid":"pidx001","sex":"F"},true)
db.patient.find({"patid":"pidx001"})
0、数据准备
for(var i=0;i<10000;i++){
var patid="pat"+i;
var patname="name"+i;
var sex="M";
var age=parseInt(100*Math.random(i));
db.patient.insert({"patid":patid,"patname":patname,"sex":sex,"age":age,address:{"city":"shanghai","street":"huaihai road"}});
}
1、比较运算符
| 运算符 | 操作符 |
| > | $gt |
| >= | $gte |
| < | $lt |
| <= | $lte |
| != | $ne |
| = | $eq 或 空 |
db.patient.find({"age":20})
db.patient.find({"age":{$eq:20}})
db.patient.find({"age":{$ne:20}})
db.patient.find({"age":{$gt:20}})
db.patient.find({"age":{$gte:20}})
db.patient.find({"age":{$lt:20}})
db.patient.find({"age":{$lte:20}})
2、逻辑运算符
| 运算符 | 操作符 |
| And | $and |
| Or | $or |
db.patient.find({$and:[{"age":10},{"age":11}]})
db.patient.find({$or:[{"age":10},{"age":11}]})
3、IN 与 Not IN
| 运算符 | 操作符 |
| In | $in |
| not in | nin |
db.patient.find({"age":{$in:[10,11]}})
db.patient.find({"age":{$nin:[10,11]}})
4、where
db.patient.find({$where:function(){return this.patid=='pat1000'}})
db.patient.find({$where:function(){return this.patid=='pat1000' || this.age==1}})
5、正则表达式
#patid以0做结尾
db.patient.find({"patid":/0$/})
#patid以pat开头
db.patient.find({"patid":{$regex:"^pat"}})
#patid以pat开头,切不区分pat大小写
db.patient.find({"patid":{$regex:"^pat",$options:"$i"}})
#patid以pat1做开头,age为10
db.patient.find({"patid":/^pat1/,age:10})
6、分页与排序
db.patient.find({"patid":/0$/}).count()
db.patient.find({"patid":/0$/}).limit(10)
db.patient.find({"patid":/0$/}).skip(10).limit(10)
db.patient.find({"patid":/0$/}).sort({"age":1})
db.patient.find({"patid":/0$/}).sort({"age":-1})
7、between是由min max来实现的
#需要age字段的索引哦
db.patient.find({"patid":/0$/}).min({"age":10}).max({"age":20})
8、全文检索
#建立text索引后,mongo会帮你分词,一个collection只能建立一个text索引
#在patname字段建立全文索引
db.patient.ensureIndex({"address.street":"text"})
#在全部字段建立全文索引
db.patient.ensureIndex({"$**": "text"})
#进行简单查询
db.patient.find({$text:{$search:"huaihai"}})
MongoDB分片存储结构如下图所示:

#启动配置服务 mongod --dbpath=D:\Database\MongoDB3\slice\config --port 27018 #启动mongos mongos --port 27017 --configdb=localhost:27018 #启动mongo分片服务 mongod --dbpath=D:\Database\MongoDB3\slice\slice1 --port 27019 mongod --dbpath=D:\Database\MongoDB3\slice\slice2 --port 27020
mongo localhost:27017
#增加节点
sh.addShard("localhost:27019")
sh.addShard("localhost:27020")
#开启分片
sh.enableSharding("test")
#配置collection
sh.shardCollection("test.patient",{"patid":1},true)
#查看状态
sh.status()
#批量插入数据
#单独连一个mongo查一下试试:)
for(var i=0;i<10000;i++){
var patid="pat"+i;
var patname="name"+i;
var sex="M";
var age=parseInt(100*Math.random(i));
db.patient.insert({"patid":patid,"patname":patname,"sex":sex,"age":age,address:{"city":"shanghai","street":"huaihai road"}});
}
for(var i=0;i<100000;i++){
var patid="pat"+i;
var patname="name"+i;
var sex="M";
var age=parseInt(100*Math.random(i));
db.patient.insert({"patid":patid,"patname":patname,"sex":sex,"age":age});
}
#普通索引
db.patient.ensureIndex({"age":1})
#唯一索引
db.patient.ensureIndex({"patid":1},{"unique":true})
#复合索引
db.patient.ensureIndex({"patname":1,"age":1})
#解释执行计划
db.patient.find({"patid":"pat1000"}).explain();
db.patient.find({"age":99}).explain();
#指定索引进行查询
db.patient.find({"age":99}).hint({"patname":1,"age":1}).explain();
#枚举索引
db.patient.getIndexes();
#删除索引
db.patient.dropIndex("age_1");