Designing reconfigurable large-scale deep learning systems using stochastic computing

Ao Ren, Zhe Li, Yanzhi Wang, Qinru Qiu, Bo Yuan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Scopus citations

Abstract

Deep Learning, as an important branch of machine learning and neural network, is playing an increasingly important role in a number of fields like computer vision, natural language processing, etc. However, large-scale deep learning systems mainly operate in high-performance server clusters, thus restricting the application extensions to personal or mobile devices. The solution proposed in this paper is taking advantage of the fantastic features of stochastic computing methods. Stochastic computing is a type of data representation and processing technique, which uses a binary bit stream to represent a probability number (by counting the number of ones in this bit stream). In the stochastic computing area, some key arithmetic operations such as additions or multiplications can be implemented with very simple components like AND gates or multiplexers, respectively. Thus it provides an immense design space for integrating a large amount of neurons and enabling fully parallel and scalable hardware implementations of large-scale deep learning systems. In this paper, we present a reconfigurable large-scale deep learning system based on stochastic computing technologies, including the design of the neuron, the convolution function, the back-propagation function and some other basic operations. And the network-on-chip technique is also proposed in this paper to achieve the goal of implementing a large-scale hardware system. Our experiments validate the functionality of reconfigurable deep learning systems using stochastic computing, and demonstrate that when the bit streams are set to be 8192 bits, classification of MNIST digits by stochastic computing can perform as low error rate as that by normal arithmetic operations.

Original languageEnglish (US)
Title of host publication2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - Conference Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509013708
DOIs
StatePublished - Nov 8 2016
Externally publishedYes
Event2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - San Diego, United States
Duration: Oct 17 2016Oct 19 2016

Publication series

Name2016 IEEE International Conference on Rebooting Computing, ICRC 2016 - Conference Proceedings

Other

Other2016 IEEE International Conference on Rebooting Computing, ICRC 2016
Country/TerritoryUnited States
CitySan Diego
Period10/17/1610/19/16

All Science Journal Classification (ASJC) codes

  • Hardware and Architecture

Keywords

  • Stochastic computing
  • deep learning
  • large-scale
  • neuron
  • reconfigurable

Fingerprint

Dive into the research topics of 'Designing reconfigurable large-scale deep learning systems using stochastic computing'. Together they form a unique fingerprint.

Cite this