Skip to main content

MongoDB and Hadoop

Traditional relational databases were ruling the roost until datasets were being reckoned in megabytes and gigabytes. However, as organizations around the world kept growing, a tsunami called “Big Data” rendered the old technologies unfeasible.
When it came to data storage and retrieval, these technologies simply crumbled under the burden of such colossal amounts of data. Thanks to Hadoop, Hive and Hbase, these popular technologies now have the capability of handling large sets of raw unstructured data, efficiently, as well as economically.
Another aftermath of the above problems was the parallel advent of “Not Only SQL” or NoSQL databases. The primary advantage of the NoSQL databases is their mechanism that facilitates the storage and retrieval of data in the loser consistency model along with added benefits like horizontal scaling, better availability and quicker access.
With its implementation in over five hundred top notch organizations across the globe, MongoDB certainly has emerged as the most popular NoSql databases amongst all. In the absence of a concrete survey, it might be a bit difficult to assess the percentage of adoption and penetration of MongoDB. However, there are various metrics like Google searches and the number of employment opportunities for Hadoop and MongoDB professionals that give a good idea of the popularity of these technologies.
Based on its Google search volume, it was found that MongoDB ranked first and was three times more popular than the next prevailing technology. When it came to comparing with the least prevailing database, MongoDB fared 10 times better.
A survey of profiles of IT professionals on LinkedIn revealed that percentage of professionals skilled in MongoDB was almost 50% as compared to other NoSQL skilled professionals. When it comes down to acceptance levels, MongoDB equals the sum of next 3 NoSQL databases put together. Rackspace, one of the pioneers to adopt MongoDB for their cloud solutions affirms ““MongoDB is the de facto choice for NoSQL applications”.
The reasons that MongoDB is being widely adopted by developers follow:
  • 1.MongoDB enhances productivity and it is easy to get started and to use.
  • 2.Owing to the removal of schema barrier, developers can now concentrate on developing applications rather than databases.
  • 3.MongoDB offers extensive support for an array of languages like C#, C, C++, Node.js, Scala, Javascript and Objective-C. These languages are pertinent to the future of the web.
Understanding how MongoDB teams up with Hadoop and Big Data technologies?
Of late, Technologists at MongoDB have successfully developed a MongoDB connector for Hadoop that facilitates enhanced integration combined with ease in execution of various tasks as below:
  • The MongoDB-Hadoop connector uses the authority of Hadoop’s MapReduce to live application data in MongoDB by extracting values from Big Data – speedily as well as efficiently.
  • The MongoDB-Hadoop connector projects it as ‘Hadoop compatible file system’ and MapReduce jobs can now be read directly from MongoDB, without being copied to the HDFS. Thus, doing away with the necessity of transferring terabytes of data across the network.
  • The “necessity” of scanning entire collections has been eliminated as MapReduce jobs can pass queries by means of filters and can harness MongoDB’s indexing abilities like text search, compound, array, Geo-spatial and sparse indexes.
  • Reading and writing back results from Hadoop jobs back to MongoDB in order to support queries and real time operational processes.
Scope of application - Hadoop and MongoDB
In context to Big Data stacks, MongoDB and Hadoop have the following scopes of application:
  • 1)MongoDB is used for the operational part – as a real time data store.
  • 2)Hadoop is used primarily for offline analysis and processing of batch data.
Scope of usage in Batch Aggregation

Image Credit: http://mobicon.tistory.com
When it comes to analyzing data, the inbuilt aggregation features incorporated in MongoDBhold good in the majority of situations. However, there are cases that require a higher degree of data aggregation. Under such circumstances, Hadoop provides a powerful support for complex analytics.
  • a)Hadoop, by means of single or multiple MapReduce jobs processes the data extracted from MongoDB. It is also possible to pull data from other locations in these MapReduce jobs in order to formulate a multi data solution.
  • b)The results received from MapReduce jobs can be written back to MongoDB and they can be used for analysis and querying as and when required.
  • c)MongoDB applications can thus make use of the data from batch analytics with a view of handing over to the end user or to facilitate other features down the line.
Scope of usage in Data Warehousing
In a usual production environment, application data with their specific functionality and language may exist in more than one data store. Under such complex situations, Hadoop is used as an integrated source for data - as well as a data warehouse.
  • a)MapReduce jobs transfer MongoDB data to Hadoop.
  • b)As soon as the data from MongoDB and other sources is available in Hadoop, the datasets can be queried.
  • c)At this stage data analysts can opt to use Pig or MapReduce for querying large datasets that includes data from MongoDB.
Owing to the above, MongoDB has emerged as the most preferred choice of developers. From the perspective of NoSQL databases, engineers at MongoDB have successfully integrated it with Hadoop. The MongoDB Hadoop permutation is extremely effective in solving quite a few architectural problems pertaining to data warehousing, processing, data retrieval and aggregating. 
source : http://www.dezyre.com/article/mongodb-and-hadoop/81#.VPFJWbPF8VE .
Labels:

Comments

Post a Comment

Popular posts from this blog

The First 10 People Who Sign up On Facebook

The First 10 People Who Sign up On Facebook 10. Zach Bercu sphotos-b.xx.fbcdn.net “The past eight years have been extraordinary,” Bercu said. A graduate of Emory’s medical school, Bercu spent a year in Israel, where he became fluent in Hebrew. He completed his residency in New York, part of the last intern class at St. Vincent’s, whose “hospital infrastructure crumbled around me,” he remembered of the facility, which closed in 2010. Now a resident at Mount Sinai in radiology, Bercu plans to complete a fellowship in interventional radiology, a form of “micro-surgery.” From his undergraduate years, “whether through Facebook or in person,” Bercu says he “took with me some of the greatest friendships one could have.” 9. Manuel Antonio Aguilar publicogt.com Aguilar calls himself a social entrepreneur “focused on the base o...
Post a Comment
Read more

Do not automatically mark mail as read in Outlook

You may know already, but few of my friends facing the below confusion , so thought of sharing with you. When quickly going through my emails, often we haven’t actually really read the emails but Outlook already marks them as read. This is quite confusing unless we mark them as unread again which takes more time and defeats the purpose of quickly going through our emails. The options for how and when Outlook should mark emails as read can be found in; Outlook 2007 and previous Tools-> Options…-> tab Other-> button Reading Pane… Outlook 2010, Outlook 2013 and Outlook 2016 tab View-> Reading Pane-> Options… The default is set to mark items as read when you select another email. When you disable this option, all mails remain as unread unless you manually mark it as read or double click the message to open it in its own window. Once you uncheck  “mark items as read when selection changes”, the mail will change to read only w...
Post a Comment
Read more

Money Making Apps For Your Android Smartphone

Money Making Apps For Your Android Smartphone Article Category: Manikandan A little while ago, we put together a list of money making apps for iPhone owners . This has quickly become one of our most popular articles and, from the emails we’ve received, readers seem impressed by the options they have. But what about all you owners of Android smartphones ? Surely there must be some apps that can make you money ? Yes, there are. Here’s seven of the best: MintCoins Have you ever thought it would be cool to earn money from doing something as simple as downloading an app? Well, with " MintCoins ", that’s exactly what you get. But, wait! That’s not all; there are a huge number of tasks that, once completed, earn you extra cash. The time honoured offer of cash for completing a survey is one of the standard tasks you’ll be given. But MintCoins has a range of other ways to make extra money with little to no effort. Want ...
Post a Comment
Read more