TY - JOUR
T1 - Comparing workflow application designs for high resolution satellite image analysis
AU - Al-Saadi, Aymen
AU - Paraskevakos, Ioannis
AU - Gonçalves, Bento Collares
AU - Lynch, Heather J.
AU - Jha, Shantenu
AU - Turilli, Matteo
N1 - Funding Information:
Shantenu was the recipient of the inaugural Chancellor’s Excellence in Research (2016) for his cyberinfrastructure contributions to computational science. He was also awarded a Rutgers Board of Trustees Fellowship for Scholarly Excellence (2014). He is a recipient of the NSF CAREER Award (2013) and several best paper prizes at SC’xy and ISC’xy. His current research has been funded by multiple NSF awards and US Department of Energy (DoE); his work has also been funded by US National Institute for Health (NIH), and the UK EPSRC.
Funding Information:
We thank Andre Merzky (Rutgers) and Brad Spitzbart (Stony Brook) for useful discussions. This work is funded by NSF EarthCube Award, United States of America Number 1740572. Computational resources were provided by NSF XRAC awards, United States of AmericaTG-MCB090174. Geospatial support for this work provided by the Polar Geospatial Center, United States of America under NSF-OPP awards 1043681 and 1559691. We thank the PSC Bridges PI and Support Staff for supporting this work through resource reservations. Aymen Alsaadi, Ioannis Paraskevakos and Matteo Turilli contributed equally to all section of the paper. Bento Collares Gon?alves contributed to Section 3.
Funding Information:
We thank Andre Merzky (Rutgers) and Brad Spitzbart (Stony Brook) for useful discussions. This work is funded by NSF EarthCube Award, United States of America Number 1740572 . Computational resources were provided by NSF XRAC awards, United States of America TG-MCB090174 . Geospatial support for this work provided by the Polar Geospatial Center, United States of America under NSF-OPP awards 1043681 and 1559691 . We thank the PSC Bridges PI and Support Staff for supporting this work through resource reservations. Aymen Alsaadi, Ioannis Paraskevakos and Matteo Turilli contributed equally to all section of the paper. Bento Collares Gonçalves contributed to Section 3 .
Publisher Copyright:
© 2021
PY - 2021/11
Y1 - 2021/11
N2 - Very High Resolution satellite and aerial imagery are used to monitor and conduct large scale surveys of ecological systems. Convolutional Neural Networks have successfully been employed to analyze such imagery to detect large animals and salient features. As the datasets increase in volume and number of images, utilizing High Performance Computing resources becomes necessary. In this paper, we investigate three task-parallel, data-driven workflow designs to support imagery analysis pipelines with heterogeneous tasks on high performance computing platforms. We analyze the capabilities of each design when processing 3097 and 1575 images for two distinct use cases, for a total of 4,672 satellite and aerial images and 8.35 TB of data. We experimentally model the execution time of the tasks of the image processing pipelines. We perform experiments to characterize resource utilization, total time to completion and overheads of each design. Our analysis shows which design is best suited to scientific pipelines with similar characteristics.
AB - Very High Resolution satellite and aerial imagery are used to monitor and conduct large scale surveys of ecological systems. Convolutional Neural Networks have successfully been employed to analyze such imagery to detect large animals and salient features. As the datasets increase in volume and number of images, utilizing High Performance Computing resources becomes necessary. In this paper, we investigate three task-parallel, data-driven workflow designs to support imagery analysis pipelines with heterogeneous tasks on high performance computing platforms. We analyze the capabilities of each design when processing 3097 and 1575 images for two distinct use cases, for a total of 4,672 satellite and aerial images and 8.35 TB of data. We experimentally model the execution time of the tasks of the image processing pipelines. We perform experiments to characterize resource utilization, total time to completion and overheads of each design. Our analysis shows which design is best suited to scientific pipelines with similar characteristics.
KW - Computational modeling
KW - Image analysis
KW - Runtime
KW - Scientific workflows
KW - Task-parallel
UR - http://www.scopus.com/inward/record.url?scp=85108114271&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85108114271&partnerID=8YFLogxK
U2 - 10.1016/j.future.2021.04.023
DO - 10.1016/j.future.2021.04.023
M3 - Article
AN - SCOPUS:85108114271
SN - 0167-739X
VL - 124
SP - 315
EP - 329
JO - Future Generation Computer Systems
JF - Future Generation Computer Systems
ER -