Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets

Hrishikesh Gadre, Ivan Rodero, Manish Parashar

Research output: Contribution to journalConference articlepeer-review

6 Scopus citations

Abstract

In this paper, we investigate real-world scenarios in which MapReduce programming model and specifically Hadoop framework could be used for processing large-scale, geographically scattered datasets. We propose an Adaptive Reduce Task Scheduling (ARTS) algorithm and evaluate it on a distributed Hadoop cluster involving multiple datacenters as well as the on a shared Hadoop cluster. The evaluation demonstrates that the ARTS algorithm outperforms the default Reduce phase scheduling algorithm in Hadoop framework.

Original languageEnglish (US)
Pages (from-to)116-118
Number of pages3
JournalPerformance Evaluation Review
Volume39
Issue number3
DOIs
StatePublished - Dec 21 2011
EventGreenMetrics Workshop 2011 - San Jose, United States
Duration: Jun 7 2011 → …

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Keywords

  • Data center
  • Data processing
  • Distributed
  • MapReduce

Fingerprint

Dive into the research topics of 'Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets'. Together they form a unique fingerprint.

Cite this