Correlations between deep neural network model coverage criteria and model quality

Shenao Yan, Guanhong Tao, Xuwei Liu, Juan Zhai, Shiqing Ma, Lei Xu, Xiangyu Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

Inspired by the great success of using code coverage as guidance in software testing, a lot of neural network coverage criteria have been proposed to guide testing of neural network models (e.g., model accuracy under adversarial attacks). However, while the monotonic relation between code coverage and software quality has been supported by many seminal studies in software engineering, it remains largely unclear whether similar monotonicity exists between neural network model coverage and model quality. This paper sets out to answer this question. Specifically, this paper studies the correlation between DNN model quality and coverage criteria, effects of coverage guided adversarial example generation compared with gradient decent based methods, effectiveness of coverage based retraining compared with existing adversarial training, and the internal relationships among coverage criteria.

Original languageEnglish (US)
Title of host publicationESEC/FSE 2020 - Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering
EditorsPrem Devanbu, Myra Cohen, Thomas Zimmermann
PublisherAssociation for Computing Machinery, Inc
Pages775-787
Number of pages13
ISBN (Electronic)9781450370431
DOIs
StatePublished - Nov 8 2020
Event28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020 - Virtual, Online, United States
Duration: Nov 8 2020Nov 13 2020

Publication series

NameESEC/FSE 2020 - Proceedings of the 28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Conference

Conference28th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020
Country/TerritoryUnited States
CityVirtual, Online
Period11/8/2011/13/20

All Science Journal Classification (ASJC) codes

  • Software

Keywords

  • Deep Neural Networks
  • Software Testing

Fingerprint

Dive into the research topics of 'Correlations between deep neural network model coverage criteria and model quality'. Together they form a unique fingerprint.

Cite this