Yang, R, Ouyang, X, Chen, Y et al. (2 more authors) (2018) Intelligent Resource Scheduling at Scale: a Machine Learning Perspective. In: IEEE International Symposium on Service Oriented System Engineering. 2018 IEEE SOSE, 26-29 Mar 2018, Bamberg, Germany. IEEE , pp. 132-141. ISBN 978-1-5386-5207-7
Abstract
Resource scheduling in a computing system addresses the problem of packing tasks with multi-dimensional resource requirements and non-functional constraints. The exhibited heterogeneity of workload and server characteristics in Cloud-scale or Internet-scale systems is adding further complexity and new challenges to the problem. Compared with,,,, existing solutions based on ad-hoc heuristics, Machine Learning (ML) has the potential to improve further the efficiency of resource management in large-scale systems. In this paper we,,,, will describe and discuss how ML could be used to understand automatically both workloads and environments, and to help to cope with scheduling-related challenges such as consolidating co-located workloads, handling resource requests, guaranteeing application's QoSs, and mitigating tailed stragglers. We will introduce a generalized ML-based solution to large-scale resource scheduling and demonstrate its effectiveness through a case study that deals with performance-centric node classification and straggler mitigation. We believe that an MLbased method will help to achieve architectural optimization and efficiency improvement.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |
Keywords: | Resource Scheduling; machine learning; resource management; straggler |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Funding Information: | Funder Grant number EPSRC EP/K503836/1 |
Depositing User: | Symplectic Publications |
Date Deposited: | 02 May 2018 10:53 |
Last Modified: | 13 Jul 2018 13:35 |
Status: | Published |
Publisher: | IEEE |
Identification Number: | 10.1109/SOSE.2018.00025 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:130350 |