Skip to main content

Apache Hadoop

  • Apache Hadoop is an open-source software framework written in Java for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware.
  •  All the modules in Hadoop are designed with a fundamental assumption that hardware failures (of individual machines, or racks of machines) are commonplace and thus should be automatically handled in software by the framework. 
  • The core of Apache Hadoop consists of a storage part (Hadoop Distributed File System (HDFS)) and a processing part (MapReduce). 
  •   To process the data, Hadoop MapReduce transfers packaged code for nodes to process in parallel, based on the data each node needs to process.  

The base Apache Hadoop framework is composed of the following modules:
    • Hadoop Common – contains libraries and utilities needed by other Hadoop modules;
    • Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster;
    • Hadoop YARN – a resource-management platform responsible for managing computing resources in clusters and using them for scheduling of users' applications
    • Hadoop MapReduce – a programming model for large scale data processing.

    The Hadoop framework itself is mostly written in the Java programming language, with some native code in C and command line utilities written as Shell script. For end-users, though MapReduce Java code is common, any programming language can be used with "Hadoop Streaming" to implement the "map" and "reduce" parts of the user's program. Other related projects expose other higher-level user interfaces.
    Labels:

    Comments

    Post a Comment

    Popular posts from this blog

    The First 10 People Who Sign up On Facebook

    The First 10 People Who Sign up On Facebook 10. Zach Bercu sphotos-b.xx.fbcdn.net “The past eight years have been extraordinary,” Bercu said. A graduate of Emory’s medical school, Bercu spent a year in Israel, where he became fluent in Hebrew. He completed his residency in New York, part of the last intern class at St. Vincent’s, whose “hospital infrastructure crumbled around me,” he remembered of the facility, which closed in 2010. Now a resident at Mount Sinai in radiology, Bercu plans to complete a fellowship in interventional radiology, a form of “micro-surgery.” From his undergraduate years, “whether through Facebook or in person,” Bercu says he “took with me some of the greatest friendships one could have.” 9. Manuel Antonio Aguilar publicogt.com Aguilar calls himself a social entrepreneur “focused on the base o...
    Post a Comment
    Read more

    Money Making Apps For Your Android Smartphone

    Money Making Apps For Your Android Smartphone Article Category: Manikandan A little while ago, we put together a list of money making apps for iPhone owners . This has quickly become one of our most popular articles and, from the emails we’ve received, readers seem impressed by the options they have. But what about all you owners of Android smartphones ? Surely there must be some apps that can make you money ? Yes, there are. Here’s seven of the best: MintCoins Have you ever thought it would be cool to earn money from doing something as simple as downloading an app? Well, with " MintCoins ", that’s exactly what you get. But, wait! That’s not all; there are a huge number of tasks that, once completed, earn you extra cash. The time honoured offer of cash for completing a survey is one of the standard tasks you’ll be given. But MintCoins has a range of other ways to make extra money with little to no effort. Want ...
    Post a Comment
    Read more

    Enable USB 3.0 in the VirtualBox Manager

    Enable USB 3.0 in the VirtualBox Manager.  If you use the Pen Drive which is USB 3.0 in the virtual box it may not be detected in the VM machine. To make it detected in the VM machine, you need to enable the USB 3.0 in the Virtual box manager. * Open the VirtualBox Manager.  * Click on the Setting to the machine you need to enable the USB 3.0. * Click on the USB Icon from the setting Page. * Check for the USB 3.0 Radio Button. Install Oracle Extension Pack to enable USB 3.0 in VirtualBox manager. To enable the USB support (USB 2.0 or 3,0) on the Oracle VM VirtualBox Guest machine, you must install Oracle VM Extension Pack on the Host machine. Shutdown the Guest machine. Download Oracle VM VirtualBox Extension Pack . When download is completed, double click to launch the " Oracle_VM_VirtualBox_Extension_Pack-5.x.x.x.vbox-extpack " installer.  Follow the on screen instructions to install the Extension Pack on your computer. After Completing Instruction restart the oracle ...
    Post a Comment
    Read more