Monday 25 May 2015

Getting started with Redis


        Redis is a really fast and useful NoSQL database. It's used by many large web companies like StackOverflow, Twitter and Instagram.
       
        But if you are reading about it for the first time it can be pretty confusing about what Redis does and how can it be used. Also why do we need to use Redis instead of relational databases like MySQL or SQLServer. This post is meant to give you a brief introduction about Redis and how it differs from relational databases and NoSQL.
       
        Redis is an Open Source database that stores data in the form of a key value pair. Each type of data is associated with a key. You might think of Redis as a dictionary in C# or Map in Java, but the value in Redis is a structured data type consisting of lists, set and hashes.
        The key point about Redis is it stores data in the main memory i.e in the RAM of the computer it is running on. You can persist the data to disk, but the in-memory dataset and the key value pair is what makes Redis extremely fast.

        Redis is built with LUA support. If you are totally new to LUA, you can read about it at http://www.lua.org/start.html. Basically it's a lightweight scripting language. You don't necessarily need to know LUA for working with Redis, but Redis provides support for lua scripts similar to stored procedures for SQL.

        Also Redis can be replicated i.e you can have Redis installed on different machines to read data and a master controlling these nodes. It is highly recommended to deploy Redis on linux machines.

Note: As per the Redis official website, there is no official support for Windows builds, but Microsoft maintains a Win-64 port of Redis.

        You might feel this post being too theoretical but believe me these points will help you decide the usage of Redis in your scenario. So just take a deep breath and continue reading.

        Next we will talk about Sentinel. Pay some attention here because it's kinda senti! Sentinel is a feature that provides an automatic failover mechanism. It's a built-in feature in v2.6+. It automatically promotes a slave to replace a failing master, also reconfigures other slaves to use the new master and informs about it to the client applications. Thus it provides high availability. 
        
       Generally when you're working with a database it's always difficult to monitor and configure it. But Redis Sentinel provides with tasks such as monitoring, notifications and acts as a configuration provider for clients. All in one! Isn't it???

Note: A prerequisite for using Sentinel for automatic failover is to have a master-slave replication setup

         Now lets talk more about Redis. Redis is a NoSQL database as you must be knowing till now i.e it has no schema and is much more flexible in its structure. But Redis actually is quite a bit different from other NoSQL databases such as mongodb or ravendb. These databases focus is to create documents which can be persisted on disk and can be indexed and hence these are known as document databases. 
       On the other hand, Redis stores its data using keys and the data stores can be in the form of different data structures not just a document. There is no kind of indexing in Redis. Offcourse you can implement it yourself but Redis does not provide any support as such. Redis only lets you get data by specifying the key. But it's very fast speed and less overhead over less query capability makes it suitable for certain conditions.





Sunday 24 May 2015

Mongoose tutorial with node.js and mongodb

In recent times NoSQL has gained popularity and of these mongodb is the clear favorite. But for people using relational databases with different object-relational models(ORMs) for some time might find it difficult to switch from schema-based to schema less approach. You may have tried to use mongodb but faced problems due to its lack of structure or schema.

    There are official drivers present for mongodb in different languages such as C#, C++, Java, Python, Ruby, Perl, PHP and Node.js on the mongodb website. This is how native mongodb driver for node.js can be used to connect to mongodb:

var MongoClient = require('mongo').MongoClient;

MongoClient.connect("mongodb://localhost:27017//somedb", function(err, db){
   if(!err){
      console.log("Connection Successful");
   }
});

So why not use native mongodb driver for your application, well sometimes you should but however if you need validations, associations and other high-level data modeling functions, then an Object Document Mapper may be helpful.

According to mongoose website it provides a straight-forward, schema-based solution to modeling your application data and includes built-in type casting, validation, query building, business logic hooks and more, out of the box.

Now let's take a small look at the mongoose schema. Suppose we want to store customer information in mongodb. The schema might look as below:

var mongoose = require("mongoose");

var Schema = mongoose.Schema();
var customerSchema = new Schema({
                 name: String,
                 address: String,
                 phone: Number,
                 createdOn: Date,
                 isActive: Boolean
});

First we require the mongoose to gain access to the schema object. Next we created a simple flat customer schema. The schema takes the field name and the schema type. The schema types present in mongoose are String, Date, Number, Boolean, ObjectId, Buffer(similar to Object in javascript), Array and Mixed. When you need a flexibility of data types to be stored in a single field, you can use Mixed data type for that. Let's look at a complex schema for customer information storage.

 var Schema = mongoose.Schema();

var addressSchema = new Schema({
                  city: String,
                  state: String,
                  country: String,
                  postalCode: String
});

var customerSchema = new Schema({
                 name: {
                      first: String,
                      last: String
                 },
                 address: addressSchema,
                 phone: Number,
                 createdOn: { type: Date, default: Date.now},
                 isActive: { type: Boolean, default: true}
});

Here, first we dissolved the name property into first and last fields, address becomes a nested schema and it's always a good practice to define the nested schemas separately. Mongoose also provides us with the facility to define the default value for a field. For createdOn and isActive fields we have defined an object instead of a schema type containing the type and the default value for that field. 

We can also add fields to the schema using the add method of schema object. For example,

var customerSchema = new Schema;
customerSchema.add({ name: String});

This allows you to define the schema dynamically depending on some of the known conditions.Next we will connect to the test db in mongo using the mongoose.

mongoose.connect('mongodb://localhost:27017/test');

Next we will create a model using the schema previously defined. The model function takes first argument as the model name and schema as the second. It also contains an optional third argument wherein you can name the collection to be used for this model. We can create multiple models from the schema definition. 

var Customer = mongoose.model('customer', customerSchema);

customerSchema.add({ premiumCode: Number});
var premiumCust = mongoose.model('premiumCustomer', customerSchema);

A document in mongoose is nothing but an instance of a model. This mongoose document is converted into a mongodb document which can then be stored in mongodb.

                   schema --> model --> mongoose document --> mongodb document


Taking our customer model, we now create an instance of the model below:

var Customer = mongoose.model('customer', customerSchema);
var abd = new Customer({
            name: 'AB deVilliers',
            address: 'South Africa',
            phone: 1233567890,
            isActive: true
});

Here as you can see 'abd' is an instance of the Customer model derived from the customerSchema. Next to save this document to mongodb call the save function on the document.

abd.save();

Optionally, the save function also takes a callback as an argument. This can be useful to debug the errors. The save() function with a callback is shown below:

abd.save(function (err) {
         if (err) return console.log(err);
});

Now to retrieve a document from the mongodb, mongoose provides with four most commonly used functions with the model object. These are find, findById, findOne, and where.
The signature of these methods can be seen below:

Model.find(conditions, [fields], [options], [callback])
Model.findOne(conditions, [fields], [options], [callback])
Model.findById(id, [fields], [options], [callback])
Model.where(path, [val])

Remember the Customer variable which holds the model object. We will use this variable to retrieve the documents from mongodb. Here we are using a callback which contains the retrieved data or errors if any.

//This returns all the documents from the collection that was used for the model.

Customer.find(function(err, results){
              if(err) throw err;
              console.log(results);
});


Note: If you don't give the collection name while creating your model, the model name appended with an 's' is taken as the default collection name. For example, in our case the model name is customer and thus it will be assigned to the default 'customers' collection.


//In this case we are passing in some query conditions to narrow down the results.

Customer.find({ name: 'Sachin' }, function(err, results){
});

//This will only return the specified fields from the document.

Customer.find({ name: 'Sachin' }, 'name address' function(err, results){
});


Note: To remove a field from the results, prefix the field name with a '-' sign.


//The find method without any arguments returns only the query object. Then we can call the exec method on the query object to get the results in the form of a callback.

var query = Customer.find();
query.exec(function(err, results){
});

Similarly for the findOne method, as the name suggests it will only return the first matched document.

//Returns the first document that has the name value of Sachin

Customer.findOne({ name: 'Sachin' }, function(err, results){
});

If you want to get any document based on its id, you can use the findById method.

//Returns the document with the specified id field

var id = '3485fgwtr4cnm8175890';
Customer.findById(id, function(err, results){
});


Note: We can also use the mongodb comparison query operators while retrieving the documents
Customer.find({ age: { $gte: 12 }}, function(err, results){
});


The where method works as follows. It expects a field path to be queried against and the query value
Customer.where('age').gte(12).exec(function(err, results){
              if(err) throw err;
              console.log(results);
})

You can also chain your query for retrieving documents such as:

Customer.where('age').gte(12).where('name', 'Rahul').exec(function(err, results){
              if(err) throw err;
              console.log(results);
})

Note: The path in the method signature signifies the field we are querying against. This can also be a nested field like name.first or address.city


I hope after reading this you will feel comfortable with using mongoose and mongodb.
In case you've got any doubts do comment below and I'll be happy to help :)