Skip to main content

Apache Hadoop

  • Apache Hadoop is an open-source software framework written in Java for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware.
  •  All the modules in Hadoop are designed with a fundamental assumption that hardware failures (of individual machines, or racks of machines) are commonplace and thus should be automatically handled in software by the framework. 
  • The core of Apache Hadoop consists of a storage part (Hadoop Distributed File System (HDFS)) and a processing part (MapReduce). 
  •   To process the data, Hadoop MapReduce transfers packaged code for nodes to process in parallel, based on the data each node needs to process.  

The base Apache Hadoop framework is composed of the following modules:
    • Hadoop Common – contains libraries and utilities needed by other Hadoop modules;
    • Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster;
    • Hadoop YARN – a resource-management platform responsible for managing computing resources in clusters and using them for scheduling of users' applications
    • Hadoop MapReduce – a programming model for large scale data processing.

    The Hadoop framework itself is mostly written in the Java programming language, with some native code in C and command line utilities written as Shell script. For end-users, though MapReduce Java code is common, any programming language can be used with "Hadoop Streaming" to implement the "map" and "reduce" parts of the user's program. Other related projects expose other higher-level user interfaces.
    Labels:

    Comments

    Post a Comment

    Popular posts from this blog

    Cygwin Install Tutorial

    1. Introduction OPEN-R requires a Unix/Linux like environment.   To work with it on Windows, the Cygwin package can be used.  It's based on Linux, and allows Linux programs (for the most part) to work under Windows.    You'll need at least 200 MBytes of free disk space available to download and install Cygwin. 2. Download Installer Download the following Cygwin setup program:   http://www.cygwin.com/setup.exe   (~250 KBytes).   Save it to your desktop, and launch once ready. 3. Cygwin Setup After launching the setup program, you'll see this screen.  Click the  Next  button...      4. Installation Type Select "Install from Internet".   Click the  Next  button... 5. Installation Directory The defaults are recommended.   Cygwin software packages will install to "C:\Cygwin". Click the  Next  button... . ...
    Post a Comment
    Read more

    Hear a PDF Instead of Reading It...

    Hear a PDF Instead of Reading It Did you know you can listen to any PDF instead of reading it? It's possible with Adobe Reader 6.0+. Here's the short cut: CTRL+SHIFT+B : This allows you to hear the entire Document (or, you can use View—>Read Out Loud—>Read to the End of Document ). CTRL+SHIFT+V : This allows you to hear just the page you are viewing (or, you can use View—&GtRead Out Loud—>Read This Page ). If the voice is too fast, adjust the speed by going to Control Panel—>(in search box enter "SPEECH" and then click change text to speech setting)  —>Voice Speed—>Slow . You can listen to any PDF instead of reading with Adobe Reader 7.0 or 6.0, and the short cut is: Ctrl+shift+b - to hear the entire Document Ctrl+shift+v - to hear the page Ctrl+shift+c - to resume Ctrl+shift+e - to stop Open any PDF File and test.... its unbelievable..! 
    Post a Comment
    Read more

    The 7 Best Chrome Apps of 2014

    The 7 Best Chrome Apps of 2014 With a brand new year looming large, it’s time for a quick look back at the rear view mirror before pressing on ahead — and what a view 2014 holds for Chrome fans.  We’ve seen native 64-bit builds of the browser land on both Mac OS X and Windows, savoured the introduction of nifty new features like tab audio indicators, and reaped the benefits of developers getting access to new APIs, design guidelines and features. The past 12 months have also seen a great many changes in the world of Chrome Apps. In this post we look back at some of the very best new and updated Chrome Apps from 2014. Sunrise (New) Sunrise  built up a huge following on iOS, making its arrival on Android and Chrome OS ( in May ) all the more anticipated. Kitted out with all kinds of handy organisational wizardry, support for multiple service providers, including Google Calendar, Outlook, Facebook and Todois...
    Post a Comment
    Read more