Mohamed, SH orcid.org/0000-0002-6234-1906, El-Gorashi, TEH and Elmirghani, JHM (2017) On the Energy Efficiency of MapReduce Shuffling Operations in Data Centers. In: ICTON 2017. 19th International Conference on Transparent Optical Networks (ICTON 2017), 02-06 Jul 2017, Girona, Catalonia, Spain. IEEE ISBN 978-1-5386-0859-3
Abstract
This paper aims to quantitatively measure the impact of different data centers networking topologies on the performance and energy efficiency of shuffling operations in MapReduce. Mixed Integer Linear Programming (MILP) models are utilized to optimize the shuffling in several data center topologies with electronic, hybrid, and all-optical switching while maximizing the throughput and reducing the power consumption. The results indicate that the networking topology has a significant impact on the performance of MapReduce. They also indicate that with comparable performance, optical-based data centers can achieve an average of 54% reduction in the energy consumption when compared to electronic switching data centers.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2017 IEEE. This is an author produced version of a paper published in the proceedings of ICTON 2017. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Uploaded in accordance with the publisher’s self-archiving policy. |
Keywords: | Data Center Networking (DCN); MapReduce; energy efficiency; completion time |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Electronic & Electrical Engineering (Leeds) > Institute of Communication & Power Networks (Leeds) |
Funding Information: | Funder Grant number EPSRC EP/H040536/1 EPSRC EP/K016873/1 |
Depositing User: | Symplectic Publications |
Date Deposited: | 24 May 2017 12:27 |
Last Modified: | 22 Nov 2017 01:33 |
Status: | Published |
Publisher: | IEEE |
Identification Number: | 10.1109/ICTON.2017.8024733 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:116808 |