Phoenix: Memory Speed HPC I/O with NVM

Pradeep Fernando, Sudarsun Kannan, Ada Gavrilovska, Karsten Schwan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Scopus citations


In order to bridge the gap between the applications' I/O needs on future exascale platforms, and thecapabilities of conventional memory and storage technologies, HPC system designs started integrating components based onemerging non-volatile memory technologies. Non-volatile memory (NVRAM) provides persistent storage at close to memoryspeeds, with good capacity scaling, leading to opportunitiesto accelerate I/O in exascale machines. However, naive use of NVRAM devices with current software stacks, exposes newbottlenecks due to the limited device bandwidth and slowerdevice access times compared to DRAM. To address this, we propose Phoenix (PHX), an NVRAM-bandwidth aware object store for persistent objects. PHXachieves efficiency through use of memory-centric objectinterfaces and device access stack specialized for NVRAM. Furthermore, PHX deals with the limited PCM bandwidththrough simultaneous use of NVRAM and DRAM devices, thus increasing the effective data movement bandwidth. Thisleads to reduction in the time length of the critical path I/Ooperations associated with the slow NVM device. To continueguaranteeing adequate reliability for the persistent objects, DRAM-resident object state is replicated across peer nodes'memory, accessible through high-bandwidth interconnects. Fur-thermore PHX minimizes the data movement overheads dueto additional data copies, by using a cost model that considersdevice bandwidths, remote storage distance and energy costs. Experimental analysis using real-world HPC applications onemulated NVRAM hardware shows that Phoenix's controlleduse of node-local and remote-node memory bandwidth, delivers up to × 1.2×, × 2× and × 12× speed-up for checkpoint I/Ofor the S3D, CM1 and GTC HPC applications, respectively. Furthermore PHX reduces total simulation checkpoint over-head of GTC up to × 18%.

Original languageEnglish (US)
Title of host publicationProceedings - 23rd IEEE International Conference on High Performance Computing, HiPC 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages11
ISBN (Electronic)9781509054114
StatePublished - Feb 1 2017
Externally publishedYes
Event23rd IEEE International Conference on High Performance Computing, HiPC 2016 - Hyderabad, India
Duration: Dec 19 2016Dec 22 2016

Publication series

NameProceedings - 23rd IEEE International Conference on High Performance Computing, HiPC 2016


Other23rd IEEE International Conference on High Performance Computing, HiPC 2016

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture


  • Checkpoints
  • HPC I/O
  • NVM


Dive into the research topics of 'Phoenix: Memory Speed HPC I/O with NVM'. Together they form a unique fingerprint.

Cite this