Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes

Dong Yang, Tao Xiong, Daguang Xu, S. Kevin Zhou, Zhoubing Xu, Mingqing Chen, Jin Hyeong Park, Sasa Grbic, Trac D. Tran, Sang Peter Chin, Dimitri Metaxas, Dorin Comaniciu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

11 Citations (Scopus)

Abstract

Automatic vertebra localization and identification in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. In this paper, we propose an automatic and efficient algorithm to localize and label the vertebra centroids in 3D CT volumes. First, a deep image-to-image network (DI2IN) is deployed to initialize vertebra locations, employing the convolutional encoder-decoder architecture. Next, the centroid probability maps from DI2IN are modeled as a sequence according to the spatial relationship of vertebrae, and evolved with the convolutional long short-term memory (ConvLSTM) model. Finally, the landmark positions are further refined and regularized by another neural network with a learned shape basis. The whole pipeline can be conducted in the end-to-end manner. The proposed method outperforms other state-of-the-art methods on a public database of 302 spine CT volumes with various pathologies. To further boost the performance and validate that large labeled training data can benefit the deep learning algorithms, we leverage the knowledge of additional 1000 3D CT volumes from different patients. Our experimental results show that training with a large database improves the performance of proposed framework by a large margin and achieves an identification rate of 89%.

Original languageEnglish (US)
Title of host publicationMedical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings
EditorsLena Maier-Hein, Alfred Franz, Pierre Jannin, Simon Duchesne, Maxime Descoteaux, D. Louis Collins
PublisherSpringer Verlag
Pages498-506
Number of pages9
ISBN (Print)9783319661780
DOIs
StatePublished - Jan 1 2017
Event20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017 - Quebec City, Canada
Duration: Sep 11 2017Sep 13 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10435 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017
CountryCanada
CityQuebec City
Period9/11/179/13/17

Fingerprint

Recurrent Networks
Labeling
Centroid
Pathology
Learning algorithms
Labels
Spine
Pipelines
Memory Model
3D Image
Medical Image
Landmarks
Encoder
Neural networks
Planning
Leverage
Margin
Learning Algorithm
Efficient Algorithms
Neural Networks

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Yang, D., Xiong, T., Xu, D., Zhou, S. K., Xu, Z., Chen, M., ... Comaniciu, D. (2017). Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes. In L. Maier-Hein, A. Franz, P. Jannin, S. Duchesne, M. Descoteaux, & D. L. Collins (Eds.), Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings (pp. 498-506). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10435 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-66179-7_57
Yang, Dong ; Xiong, Tao ; Xu, Daguang ; Zhou, S. Kevin ; Xu, Zhoubing ; Chen, Mingqing ; Park, Jin Hyeong ; Grbic, Sasa ; Tran, Trac D. ; Chin, Sang Peter ; Metaxas, Dimitri ; Comaniciu, Dorin. / Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes. Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings. editor / Lena Maier-Hein ; Alfred Franz ; Pierre Jannin ; Simon Duchesne ; Maxime Descoteaux ; D. Louis Collins. Springer Verlag, 2017. pp. 498-506 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e17f0747550e4ce0850957afe77511a5,
title = "Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes",
abstract = "Automatic vertebra localization and identification in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. In this paper, we propose an automatic and efficient algorithm to localize and label the vertebra centroids in 3D CT volumes. First, a deep image-to-image network (DI2IN) is deployed to initialize vertebra locations, employing the convolutional encoder-decoder architecture. Next, the centroid probability maps from DI2IN are modeled as a sequence according to the spatial relationship of vertebrae, and evolved with the convolutional long short-term memory (ConvLSTM) model. Finally, the landmark positions are further refined and regularized by another neural network with a learned shape basis. The whole pipeline can be conducted in the end-to-end manner. The proposed method outperforms other state-of-the-art methods on a public database of 302 spine CT volumes with various pathologies. To further boost the performance and validate that large labeled training data can benefit the deep learning algorithms, we leverage the knowledge of additional 1000 3D CT volumes from different patients. Our experimental results show that training with a large database improves the performance of proposed framework by a large margin and achieves an identification rate of 89{\%}.",
author = "Dong Yang and Tao Xiong and Daguang Xu and Zhou, {S. Kevin} and Zhoubing Xu and Mingqing Chen and Park, {Jin Hyeong} and Sasa Grbic and Tran, {Trac D.} and Chin, {Sang Peter} and Dimitri Metaxas and Dorin Comaniciu",
year = "2017",
month = "1",
day = "1",
doi = "10.1007/978-3-319-66179-7_57",
language = "English (US)",
isbn = "9783319661780",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "498--506",
editor = "Lena Maier-Hein and Alfred Franz and Pierre Jannin and Simon Duchesne and Maxime Descoteaux and Collins, {D. Louis}",
booktitle = "Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings",
address = "Germany",

}

Yang, D, Xiong, T, Xu, D, Zhou, SK, Xu, Z, Chen, M, Park, JH, Grbic, S, Tran, TD, Chin, SP, Metaxas, D & Comaniciu, D 2017, Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes. in L Maier-Hein, A Franz, P Jannin, S Duchesne, M Descoteaux & DL Collins (eds), Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10435 LNCS, Springer Verlag, pp. 498-506, 20th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2017, Quebec City, Canada, 9/11/17. https://doi.org/10.1007/978-3-319-66179-7_57

Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes. / Yang, Dong; Xiong, Tao; Xu, Daguang; Zhou, S. Kevin; Xu, Zhoubing; Chen, Mingqing; Park, Jin Hyeong; Grbic, Sasa; Tran, Trac D.; Chin, Sang Peter; Metaxas, Dimitri; Comaniciu, Dorin.

Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings. ed. / Lena Maier-Hein; Alfred Franz; Pierre Jannin; Simon Duchesne; Maxime Descoteaux; D. Louis Collins. Springer Verlag, 2017. p. 498-506 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10435 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes

AU - Yang, Dong

AU - Xiong, Tao

AU - Xu, Daguang

AU - Zhou, S. Kevin

AU - Xu, Zhoubing

AU - Chen, Mingqing

AU - Park, Jin Hyeong

AU - Grbic, Sasa

AU - Tran, Trac D.

AU - Chin, Sang Peter

AU - Metaxas, Dimitri

AU - Comaniciu, Dorin

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Automatic vertebra localization and identification in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. In this paper, we propose an automatic and efficient algorithm to localize and label the vertebra centroids in 3D CT volumes. First, a deep image-to-image network (DI2IN) is deployed to initialize vertebra locations, employing the convolutional encoder-decoder architecture. Next, the centroid probability maps from DI2IN are modeled as a sequence according to the spatial relationship of vertebrae, and evolved with the convolutional long short-term memory (ConvLSTM) model. Finally, the landmark positions are further refined and regularized by another neural network with a learned shape basis. The whole pipeline can be conducted in the end-to-end manner. The proposed method outperforms other state-of-the-art methods on a public database of 302 spine CT volumes with various pathologies. To further boost the performance and validate that large labeled training data can benefit the deep learning algorithms, we leverage the knowledge of additional 1000 3D CT volumes from different patients. Our experimental results show that training with a large database improves the performance of proposed framework by a large margin and achieves an identification rate of 89%.

AB - Automatic vertebra localization and identification in 3D medical images plays an important role in many clinical tasks, including pathological diagnosis, surgical planning and postoperative assessment. In this paper, we propose an automatic and efficient algorithm to localize and label the vertebra centroids in 3D CT volumes. First, a deep image-to-image network (DI2IN) is deployed to initialize vertebra locations, employing the convolutional encoder-decoder architecture. Next, the centroid probability maps from DI2IN are modeled as a sequence according to the spatial relationship of vertebrae, and evolved with the convolutional long short-term memory (ConvLSTM) model. Finally, the landmark positions are further refined and regularized by another neural network with a learned shape basis. The whole pipeline can be conducted in the end-to-end manner. The proposed method outperforms other state-of-the-art methods on a public database of 302 spine CT volumes with various pathologies. To further boost the performance and validate that large labeled training data can benefit the deep learning algorithms, we leverage the knowledge of additional 1000 3D CT volumes from different patients. Our experimental results show that training with a large database improves the performance of proposed framework by a large margin and achieves an identification rate of 89%.

UR - http://www.scopus.com/inward/record.url?scp=85029477002&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85029477002&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-66179-7_57

DO - 10.1007/978-3-319-66179-7_57

M3 - Conference contribution

AN - SCOPUS:85029477002

SN - 9783319661780

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 498

EP - 506

BT - Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings

A2 - Maier-Hein, Lena

A2 - Franz, Alfred

A2 - Jannin, Pierre

A2 - Duchesne, Simon

A2 - Descoteaux, Maxime

A2 - Collins, D. Louis

PB - Springer Verlag

ER -

Yang D, Xiong T, Xu D, Zhou SK, Xu Z, Chen M et al. Deep image-to-image recurrent network with shape basis learning for automatic vertebra labeling in large-scale 3D CT volumes. In Maier-Hein L, Franz A, Jannin P, Duchesne S, Descoteaux M, Collins DL, editors, Medical Image Computing and Computer Assisted Intervention − MICCAI 2017 - 20th International Conference, Proceedings. Springer Verlag. 2017. p. 498-506. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-66179-7_57