TY - GEN
T1 - Neural Network Design via Voltage-based Resistive Processing Unit and Diode Activation Function - A New Architecture
AU - Hsieh, Yung Ting
AU - Anjum, Khizar
AU - Huang, Songjun
AU - Kulkarni, Indraneel
AU - Pompili, Dario
N1 - Funding Information:
Acknowledgements: This work is supported by the NSF RTML Award No. 1937403.
Publisher Copyright:
© 2021 IEEE.
PY - 2021/8/9
Y1 - 2021/8/9
N2 - In recent years, the architecture based on Resistive Processing Unit (RPU) has become a hot topic due to its potential to accelerate training of a Neural Network (NN). However, attempts to realize the RPU concept based on non-volatile memory technology face a myriad of technological and physical constraints. The theoretical concept of crossbar array is nearly impossible to implement in the real world without certain tweaks. Hence, we propose an Voltage output Complementary Metal Oxide Semiconductor (CMOS)-based RPU design VRPU, which is used to build a neural network. We also introduce a novel diode-based circuit to behave as a non-linear activation function, which consists of a single Diode (D) and Diode Pair (DP). The proposed VRPU design when tested with MNIST dataset for hidden layer and output layer combinations of ReLU+Sigmoid, D+Sigmoid, ReLU+DP, D+DP (low temperature) and D+DP (high temperature) resulted in accuracies of 94.29%, 95.90%, 95.53%, 96.75% and 96.57% respectively corroborating the merits of the proposed design.
AB - In recent years, the architecture based on Resistive Processing Unit (RPU) has become a hot topic due to its potential to accelerate training of a Neural Network (NN). However, attempts to realize the RPU concept based on non-volatile memory technology face a myriad of technological and physical constraints. The theoretical concept of crossbar array is nearly impossible to implement in the real world without certain tweaks. Hence, we propose an Voltage output Complementary Metal Oxide Semiconductor (CMOS)-based RPU design VRPU, which is used to build a neural network. We also introduce a novel diode-based circuit to behave as a non-linear activation function, which consists of a single Diode (D) and Diode Pair (DP). The proposed VRPU design when tested with MNIST dataset for hidden layer and output layer combinations of ReLU+Sigmoid, D+Sigmoid, ReLU+DP, D+DP (low temperature) and D+DP (high temperature) resulted in accuracies of 94.29%, 95.90%, 95.53%, 96.75% and 96.57% respectively corroborating the merits of the proposed design.
KW - Diode-based Activation Function
KW - Memristor Crossbar Array
KW - Neural Network
KW - Resistive Processing Unit
UR - http://www.scopus.com/inward/record.url?scp=85115656900&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85115656900&partnerID=8YFLogxK
U2 - 10.1109/MWSCAS47672.2021.9531917
DO - 10.1109/MWSCAS47672.2021.9531917
M3 - Conference contribution
AN - SCOPUS:85115656900
T3 - Midwest Symposium on Circuits and Systems
SP - 59
EP - 62
BT - 2021 IEEE International Midwest Symposium on Circuits and Systems, MWSCAS 2021 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2021 IEEE International Midwest Symposium on Circuits and Systems, MWSCAS 2021
Y2 - 9 August 2021 through 11 August 2021
ER -