Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition

Zhennan Yan, Yiqiang Zhan, Zhigang Peng, Shu Liao, Yoshihisa Shinagawa, Shaoting Zhang, Dimitris N. Metaxas, Xiang Sean Zhou

Research output: Contribution to journalArticlepeer-review

115 Scopus citations


In general image recognition problems, discriminative information often lies in local image patches. For example, most human identity information exists in the image patches containing human faces. The same situation stays in medical images as well. 'Bodypart identity' of a transversal slice - which bodypart the slice comes from - is often indicated by local image information, e.g., a cardiac slice and an aorta arch slice are only differentiated by the mediastinum region. In this work, we design a multi-stage deep learning framework for image classification and apply it on bodypart recognition. Specifically, the proposed framework aims at: 1) discover the local regions that are discriminative and non-informative to the image classification problem, and 2) learn a image-level classifier based on these local regions. We achieve these two tasks by the two stages of learning scheme, respectively. In the pre-train stage, a convolutional neural network (CNN) is learned in a multi-instance learning fashion to extract the most discriminative and and non-informative local patches from the training slices. In the boosting stage, the pre-learned CNN is further boosted by these local patches for image classification. The CNN learned by exploiting the discriminative local appearances becomes more accurate than those learned from global image context. The key hallmark of our method is that it automatically discovers the discriminative and non-informative local patches through multi-instance deep learning. Thus, no manual annotation is required. Our method is validated on a synthetic dataset and a large scale CT dataset. It achieves better performances than state-of-the-art approaches, including the standard deep CNN.

Original languageEnglish (US)
Article number7398101
Pages (from-to)1332-1343
Number of pages12
JournalIEEE Transactions on Medical Imaging
Issue number5
StatePublished - May 2016

All Science Journal Classification (ASJC) codes

  • Software
  • Radiological and Ultrasound Technology
  • Computer Science Applications
  • Electrical and Electronic Engineering


  • CNN
  • discriminative local information discovery
  • multi-instance
  • multi-stage

Fingerprint Dive into the research topics of 'Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition'. Together they form a unique fingerprint.

Cite this