Exploiting heterogeneity for tail latency and energy effiiciency

Md E. Haque, Yuxiong He, Sameh Elnikety, Thu D. Nguyen, Ricardo Bianchini, Kathryn S. McKinley

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Scopus citations


Interactive service providers have strict requirements on highpercentile (tail) latency to meet user expectations. If providers meet tail latency targets with less energy, they increase profits, because energy is a significant operating expense. Unfortunately, optimizing tail latency and energy are typically conflicting goals. Our work resolves this conflict by exploiting servers with per-core Dynamic Voltage and Frequency Scaling (DVFS) and Asymmetric Multicore Processors (AMPs). We introduce the Adaptive Slow-to-Fast scheduling framework, which matches the heterogeneity of the workload - a mix of short and long requests - to the heterogeneity of the hardware - cores running at different speeds. The scheduler prioritizes long requests to faster cores by exploiting the insight that long requests reveal themselves. We use control theory to design threshold-based scheduling policies that use individual request progress, load, competition, and latency targets to optimize performance and energy. We configure our framework to optimize Energy Efficiency for a given Tail Latency (EETL) for both DVFS and AMP. In this framework, each request self-schedules, starting on a slow core and then migrating itself to faster cores. At high load, when a desired AMP core speed s is not available for a request but a faster core is, the longest request on an s core type migrates early to make room for the other request. Compared to per-core DVFS systems, EETL for AMPs delivers the same tail latency, reduces energy by 18% to 50%, and improves capacity (throughput) by 32% to 82%. We demonstrate that our framework effectively exploits dynamic DVFS and static AMP heterogeneity to reduce provisioning and operational costs for interactive services.

Original languageEnglish (US)
Title of host publicationMICRO 2017 - 50th Annual IEEE/ACM International Symposium on Microarchitecture Proceedings
PublisherIEEE Computer Society
Number of pages14
ISBN (Electronic)9781450349529
StatePublished - Oct 14 2017
Event50th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2017 - Cambridge, United States
Duration: Oct 14 2017Oct 18 2017

Publication series

NameProceedings of the Annual International Symposium on Microarchitecture, MICRO
VolumePart F131207
ISSN (Print)1072-4451


Other50th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO 2017
Country/TerritoryUnited States

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture


  • Energy efficiency
  • Heterogeneous processors
  • Tail latency


Dive into the research topics of 'Exploiting heterogeneity for tail latency and energy effiiciency'. Together they form a unique fingerprint.

Cite this