Near-Storage Processing for Solid State Drive Based Recommendation Inference with SmartSSDs®

Mohammadreza Soltaniyeh, Veronica Lagrange Moutinho Dos Reis, Matt Bryson, Xuebin Yao, Richard P. Martin, Santosh Nagarakatte

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

Deep learning-based recommendation systems are extensively deployed in numerous internet services, including social media, entertainment services, and search engines, to provide users with the most relevant and personalized content. Production scale deep learning models consist of large embedding tables with billions of parameters. DRAM-based recommendation systems incur a high infrastructure cost and limit the size of the deployed models. Recommendation systems based on solid-state drives (SSDs) are a promising alternative for DRAM-based systems. Systems based on SSDs can offer ample storage required for deep learning models with large embedding tables. This paper proposes SmartRec, an inference engine for deep learning-based recommendation systems that utilizes Samsung SmartSSD, an SSD with an on-board FPGA that can process data in-situ. We evaluate SmartRec with state-of-the-art recommendation models from Facebook and compare its performance and energy efficiency to a DRAM-based system on a CPU. We show SmartRec improves the energy efficiency of the recommendation inference task up to 10x in comparison to the baseline CPU implementation. In addition, we propose a novel application-specific caching system for SmartSSDs that allows the kernel on the FPGA to use its DRAM as a cache to minimize high latency SSD accesses. Finally, we demonstrate the scalability of our design by offloading the computation to multiple SmartSSDs to further improve performance.

Original languageEnglish (US)
Title of host publicationICPE 2022 - Proceedings of the 2022 ACM/SPEC International Conference on Performance Engineering
PublisherAssociation for Computing Machinery, Inc
Pages177-186
Number of pages10
ISBN (Electronic)9781450391436
DOIs
StatePublished - Apr 9 2022
Event13th Annual ACM/SPEC International Conference on Performance Engineering, ICPE 2022 - Virtual, Online, China
Duration: Apr 9 2022Apr 13 2022

Publication series

NameICPE 2022 - Proceedings of the 2022 ACM/SPEC International Conference on Performance Engineering

Conference

Conference13th Annual ACM/SPEC International Conference on Performance Engineering, ICPE 2022
Country/TerritoryChina
CityVirtual, Online
Period4/9/224/13/22

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Hardware and Architecture
  • Software

Keywords

  • deep learning
  • fpga
  • near-storage computation
  • recommendation systems
  • smartssd

Fingerprint

Dive into the research topics of 'Near-Storage Processing for Solid State Drive Based Recommendation Inference with SmartSSDs®'. Together they form a unique fingerprint.

Cite this