ABS: Scanning neural networks for back-doors by artificial brain stimulation

Yingqi Liu, Shiqing Ma, Wen Chuan Lee, Yousra Aafer, Guanhong Tao, Xiangyu Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

123 Scopus citations


This paper presents a technique to scan neural network based AI models to determine if they are trojaned. Pre-trained AI models may contain back-doors that are injected through training or by transforming inner neuron weights. These trojaned models operate normally when regular inputs are provided, and mis-classify to a specific output label when the input is stamped with some special pattern called trojan trigger. We develop a novel technique that analyzes inner neuron behaviors by determining how output activations change when we introduce different levels of stimulation to a neuron. The neurons that substantially elevate the activation of a particular output label regardless of the provided input is considered potentially compromised. Trojan trigger is then reverse-engineered through an optimization procedure using the stimulation analysis results, to confirm that a neuron is truly compromised. We evaluate our system ABS on 177 trojaned models that are trojaned with various attack methods that target both the input space and the feature space, and have various trojan trigger sizes and shapes, together with 144 benign models that are trained with different data and initial weight values. These models belong to 7 different model structures and 6 different datasets, including some complex ones such as ImageNet, VGG-Face and ResNet110. Our results show that ABS is highly effective, can achieve over 90% detection rate for most cases (and many 100%), when only one input sample is provided for each output label. It substantially out-performs the state-of-the-art technique Neural Cleanse that requires a lot of input samples and small trojan triggers to achieve good performance.

Original languageEnglish (US)
Title of host publicationCCS 2019 - Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security
PublisherAssociation for Computing Machinery
Number of pages18
ISBN (Electronic)9781450367479
StatePublished - Nov 6 2019
Event26th ACM SIGSAC Conference on Computer and Communications Security, CCS 2019 - London, United Kingdom
Duration: Nov 11 2019Nov 15 2019

Publication series

NameProceedings of the ACM Conference on Computer and Communications Security
ISSN (Print)1543-7221


Conference26th ACM SIGSAC Conference on Computer and Communications Security, CCS 2019
Country/TerritoryUnited Kingdom

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications


  • AI trojan attacks
  • Artificial brain stimulation
  • Deep learning system


Dive into the research topics of 'ABS: Scanning neural networks for back-doors by artificial brain stimulation'. Together they form a unique fingerprint.

Cite this