Tuesday 15 August 2017

NodeJs - Persistence with MongoDB and Mongoskin

MONGO DB:

NoSQL databases, also called non-relational databases, are more horizontally scalable, usually open source, and better suited for distributed systems. NoSQL databases deal routinely with larger data sizes than traditional ones. The key distinction in implementation comes from the fact that relationships between database entities are not stored in the database itself (no more join queries); they are moved to the application or object-relational mapping (ORM) levels—in our case, to Node.js code. Another good reason to use NoSQL databases is that, because they are schemaless, they are perfect for prototyping and Agile iterations (more pushes!).


  • install mongodb on ubuntu

run below commands from terminal:


it000483@IND-PUN-LAP-183:~$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
it000483@IND-PUN-LAP-183:~$ echo "deb http://repo.mongodb.org/apt/ubuntu "$(lsb_release -sc)"/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
it000483@IND-PUN-LAP-183:~$ sudo apt-get update

it000483@IND-PUN-LAP-183:~$ sudo apt-get install -y mongodb

create mongo db service 

it000483@IND-PUN-LAP-183:/lib/systemd/system$ cd /lib/systemd/system

it000483@IND-PUN-LAP-183:/lib/systemd/system$ vim mongod.service


mongod.service
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual

[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --quiet --config --auth /etc/mongod.conf

[Install]
WantedBy=multi-user.target


Now update the systemd service with command below:

it000483@IND-PUN-LAP-183:/lib/systemd/system$ systemctl daemon-reload


it000483@IND-PUN-LAP-183:/etc/init$ systemctl start mongod


to check the mongodb process status on port# 27017

it000483@IND-PUN-LAP-183:/etc/init$ netstat -plntu

tcp        0      0 127.0.0.1:27017         0.0.0.0:*               LISTEN      -            


to start mongo db shell:

it000483@IND-PUN-LAP-183:/etc/init$ mongo
> use admin
to create user with role:
> db.createUser({user:"admin", pwd:"admin", roles:[{role:"root", db:"admin"}]})

to exit from mongo shell
>exit

it000483@IND-PUN-LAP-183:/etc/init$ sudo service mongod restart
login to mongo db with user:

it000483@IND-PUN-LAP-183:/etc/init$ mongo -u admin -p admin --authenticationDatabase admin

To start and see console  the status of port and db path:


it000483@IND-PUN-LAP-183:/data/db$ sudo mongod

mongod --help for help and startup options
2017-08-16T02:34:54.100+0530 [initandlisten] MongoDB starting : pid=2604 port=27017 dbpath=/data/db 64-bit host=IND-PUN-LAP-183
2017-08-16T02:34:54.100+0530 [initandlisten] db version v2.6.10
2017-08-16T02:34:54.100+0530 [initandlisten] git version: nogitversion
2017-08-16T02:34:54.100+0530 [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
2017-08-16T02:34:54.100+0530 [initandlisten] build info: Linux lgw01-12 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 BOOST_LIB_VERSION=1_58
2017-08-16T02:34:54.100+0530 [initandlisten] allocator: tcmalloc
2017-08-16T02:34:54.100+0530 [initandlisten] options: {}
2017-08-16T02:34:54.141+0530 [initandlisten] journal dir=/data/db/journal
2017-08-16T02:34:54.141+0530 [initandlisten] recover : no journal files present, no recovery needed
2017-08-16T02:34:54.218+0530 [initandlisten] waiting for connections on port 27017

NOTE: MongoDB server is using two different ports (27017 and 28017): one is primary (native)
for communications with apps; the other is a web-based GUI for monitoring/statistics.

MongoDB Shell in Detail

> help: prints a list of available commands
> show dbs: prints the names of the databases on the database server to which the console is
connected (by default, localhost:27017; but, if we pass params to mongo, we can connect to any
remote instance)
> use db_name: switches to db_name
> show collections: prints a list of collections in the selected database
> db.collection_name.find(query);: finds all items matching query
> db.collection_name.findOne(query);: finds one item that matches query
> db.collection_name.insert(document): adds a document to the collection_name
collection
> db.collection_name.save(document);: saves a document in the collection_name
collection—a shorthand of upsert (no _id) or insert (with _id)
> db.collection_name.update(query,{$set: data});: updates items that match query in
the collection_name collection with data object values
> db.collection_name.remove(query); removes all items from collection_name that
match query criteria
> printjson(document);: prints the variable document


DB Query:

> db.test.save( { a: 1 } )
> db.test.find()

> db.test.save( { a: 1 } )
WriteResult({ "nInserted" : 1 })
> db.test.find()
{ "_id" : ObjectId("599365cfdca57436f0ba6ef4"), "a" : 1 }



Native MongoDB Driver for Node.js Example


it000483@IND-PUN-LAP-183:~/vuclip/eclipse-workspace/nodejs_jade_handlers$ sudo npm install mongodb@1.3.23 -save



create mongo-insert.js for demo:

var mongo=require('mongodb');
var dbHost = '127.0.0.1';
var dbPort = 27017;

var Db = mongo.Db;
var Connection = mongo.Connection;
var Server = mongo.Server;
var db = new Db ('local', new Server(dbHost, dbPort), {safe:true});

db.open(function(error, dbConnection){
if (error) {
console.error(error);
process.exit(1);
}
console.log('db state: ', db._state);
item = {
name: 'Sachin'
}
dbConnection.collection('test').insert(item, function(error, item){
if (error) {
console.error(error);
process.exit(1);
}
console.info('created/inserted: ', item);
db.close();
process.exit(0);
});
});




Run with below command:


it000483@IND-PUN-LAP-183:~/vuclip/eclipse-workspace/nodejs_jade_handlers$ node mongo-insert.js 
Failed to load c++ bson extension, using pure JS version
db state:  connected
created/inserted:  [ { name: 'Sachin', _id: 599369bec4763760105a9fff } ]







create mongo-retrieve.js for demo:



var mongo=require('mongodb');

var dbHost = '127.0.0.1';
var dbPort = 27017;

var Db = mongo.Db;
var Connection = mongo.Connection;
var Server = mongo.Server;
var db = new Db ('local', new Server(dbHost, dbPort), {safe:true});

db.open(function(error, dbConnection){
if (error) {
console.error(error);
process.exit(1);
}
console.log('db state: ', db._state);

dbConnection.collection('test').findOne({}, function(error, item){
if (error) {
console.error(error);
process.exit(1);
}
console.info('retrived: ', item);
db.close();
process.exit(0);
});
});



Now run with below commands:

it000483@IND-PUN-LAP-183:~/vuclip/eclipse-workspace/nodejs_jade_handlers$ node mongo-retrieve.js 
Failed to load c++ bson extension, using pure JS version
db state:  connected
retrived:  { _id: 599369bec4763760105a9fff, name: 'Sachin' }




To modify the stored item  and retrieve item  with Id:

dbConnection.collection('test').findOne({}, function(error, item){
if (error) {
console.error(error);
process.exit(1);
}
console.info('retrived before: ', item);
console.log('Modify the item object');

item.text= 'added test to item123';
var id=item._id.toString();
console.info('before saving: ', item);
dbConnection.collection('test').save(item, function (error, item) {
if(error){
console.error(error);
process.exit(1);
}
console.info('item inserted with modification');
});

//retrive with id
dbConnection.collection('test').find({_id: new mongo.ObjectID(id)}).toArray(function (error, items) {
if(error){
console.error(error);
process.exit(1);
}
console.info('Items:', items);
process.exit(0);
});


});


GUI client for Mongo DB in eclipse:

Go to eclipse marketplace:
install monja db plugin.





  • Using MongoSkin:

install mongoskin dependency:

it000483@IND-PUN-LAP-183:~/vuclip/eclipse-workspace/nodejs_jade_handlers$ sudo npm install mongoskin@0.6.1 -save

We can also create our own methods on collections. This might be useful when implementing a model-
view-controller-like (MVC-like) architecture by incorporating app-specific logic into these custom methods:

db.bind('messages', {
findOneAndAddText : function (text, fn) {
db.collection('messages').findOne({}, function(error, item){
if (error) {
console.error(error);
process.exit(1);
}
console.info('findOne: ', item);
item.text = text;
var id = item._id.toString(); // we can store ID in a string
console.info('before saving: ', item);
db.collection('messages').save(item, function(error, count){
console.info('save: ', count);
return fn(count, id);
});
})
}
});

db.collection('messages').findOneAndAddText('hi', function(count, id){
db.collection('messages').find({
_id: db.collection('messages').id(id)
}).toArray(function(error, items){
console.info("find: ", items);
db.close();
process.exit(0);
});
});


Mongoskin–only methods:

findItems(..., callback): finds elements and returns an array instead of a cursor
findEach(..., callback): iterates through each found element
findById(id, ..., callback): finds by _id in a string format
updateById(_id, ..., callback): updates an element with a matching _id
removeById(_id, ..., callback): removes an element with a matching _id

Alternatives to the native MongoDB driver and Mongoskin include:

mongoose (http://mongoosejs.com/): an asynchronous JavaScript driver with optional
support for modelling
mongolia (https://github.com/masylum/mongolia): a lightweight MongoDB ORM/driver
wrapper
monk (https://github.com/LearnBoost/monk): a tiny layer that provides simple yet
substantial usability improvements for MongoDB use within Node.js




  • Adding Persistence WIth Mongo DB





























No comments:

Post a Comment

Extract error records while inserting into db table using JDBCIO apache beam in java

 I was inserting data into postgres db using apache beam pipeline. it works perfectly with JdbcIO write of apache beam library. But, now, i ...