Skip to main content

Apache Hadoop

  • Apache Hadoop is an open-source software framework written in Java for distributed storage and distributed processing of very large data sets on computer clusters built from commodity hardware.
  •  All the modules in Hadoop are designed with a fundamental assumption that hardware failures (of individual machines, or racks of machines) are commonplace and thus should be automatically handled in software by the framework. 
  • The core of Apache Hadoop consists of a storage part (Hadoop Distributed File System (HDFS)) and a processing part (MapReduce). 
  •   To process the data, Hadoop MapReduce transfers packaged code for nodes to process in parallel, based on the data each node needs to process.  

The base Apache Hadoop framework is composed of the following modules:
    • Hadoop Common – contains libraries and utilities needed by other Hadoop modules;
    • Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster;
    • Hadoop YARN – a resource-management platform responsible for managing computing resources in clusters and using them for scheduling of users' applications
    • Hadoop MapReduce – a programming model for large scale data processing.

    The Hadoop framework itself is mostly written in the Java programming language, with some native code in C and command line utilities written as Shell script. For end-users, though MapReduce Java code is common, any programming language can be used with "Hadoop Streaming" to implement the "map" and "reduce" parts of the user's program. Other related projects expose other higher-level user interfaces.

    Comments

    Popular posts from this blog

    4G

    4G benifits: At the end of October, EE launched the UK’s first superfast 4G broadband services for consumers and businesses in 11 cities. But what benefits would business have from accessing new found speeds of up to 40Mbps? Firstly there is the extra productivity on hand for employees. With the increasing number of businesses adopting mobile working policies, there has been a much bigger focus on connectivity on the move. These tools can cut costs for a company, reducing the need for employees to travel, as well as cutting necessary office space as more people work from home. With a strong 4G network, these solutions can be brought to life in a way that current 3G connections or ad hoc Wi-Fi just cannot serve. First tests of EE 4G network show inconsistent coverage Following the launch of 4G in October, first tests of EE's 4G network showed less than half of Manchester city centre was covered by the high-speed mobile network and 4G connections were inconsistent. Only ...

    The First 10 People Who Sign up On Facebook

    The First 10 People Who Sign up On Facebook 10. Zach Bercu sphotos-b.xx.fbcdn.net “The past eight years have been extraordinary,” Bercu said. A graduate of Emory’s medical school, Bercu spent a year in Israel, where he became fluent in Hebrew. He completed his residency in New York, part of the last intern class at St. Vincent’s, whose “hospital infrastructure crumbled around me,” he remembered of the facility, which closed in 2010. Now a resident at Mount Sinai in radiology, Bercu plans to complete a fellowship in interventional radiology, a form of “micro-surgery.” From his undergraduate years, “whether through Facebook or in person,” Bercu says he “took with me some of the greatest friendships one could have.” 9. Manuel Antonio Aguilar publicogt.com Aguilar calls himself a social entrepreneur “focused on the base o...

    Your current security settings do not allow this file to be downloaded' in Internet Explorer

    I faced the below issue while i am trying to download the software from one the VM machine that i am Working for my project in my office. Then u found the below solution. Instructions You can receive the following message when trying to download a file in Internet Explorer: ------------------------------------------------------------------------------------------------------- Security Alert Your current security settings do not allow this file to be downloaded. ------------------------------------------------------------------------------------------------------- To remove this message and allow downloading of files, follow the steps based on your operating system. Windows 10 Click Search, search for Control Panel, and then click Control Panel. Click Network and Internet > Internet Options. On the Security tab, click Custom level. Scroll down in the list to the Downloads section, and set all download options to Enabled. Click OK. Windows 8 Right-click i...