TY - GEN
T1 - Ensemble Toolkit
T2 - 45th International Conference on Parallel Processing, ICPP 2016
AU - Balasubramanian, Vivekanandan
AU - Treikalis, Antons
AU - Weidner, Ole
AU - Jha, Shantenu
N1 - Publisher Copyright:
© 2016 IEEE.
PY - 2016/9/21
Y1 - 2016/9/21
N2 - There are many science applications that requirescalable task-level parallelism, support for flexible execution andcoupling of ensembles of simulations. Most high-performancesystem software and middleware, however, are designed tosupport the execution and optimization of single tasks. Motivatedby the missing capabilities of these computing systems and theincreasing importance of task-level parallelism, we introduce theEnsemble toolkit which has the following application developmentfeatures: (i) abstractions that enable the expression of ensemblesas primary entities, and (ii) support for ensemble-based executionpatterns that capture the majority of application scenarios. Ensemble toolkit uses a scalable pilot-based runtime systemthat decouples workload execution and resource managementdetails from the expression of the application, and enables theefficient and dynamic execution of ensembles on heterogeneouscomputing resources. We investigate three execution patterns andcharacterize the scalability and overhead of Ensemble toolkit forthese patterns. We investigate scaling properties for up to O(1000)concurrent ensembles and O(1000) cores and find linear weakand strong scaling behaviour.
AB - There are many science applications that requirescalable task-level parallelism, support for flexible execution andcoupling of ensembles of simulations. Most high-performancesystem software and middleware, however, are designed tosupport the execution and optimization of single tasks. Motivatedby the missing capabilities of these computing systems and theincreasing importance of task-level parallelism, we introduce theEnsemble toolkit which has the following application developmentfeatures: (i) abstractions that enable the expression of ensemblesas primary entities, and (ii) support for ensemble-based executionpatterns that capture the majority of application scenarios. Ensemble toolkit uses a scalable pilot-based runtime systemthat decouples workload execution and resource managementdetails from the expression of the application, and enables theefficient and dynamic execution of ensembles on heterogeneouscomputing resources. We investigate three execution patterns andcharacterize the scalability and overhead of Ensemble toolkit forthese patterns. We investigate scaling properties for up to O(1000)concurrent ensembles and O(1000) cores and find linear weakand strong scaling behaviour.
UR - http://www.scopus.com/inward/record.url?scp=84991020726&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84991020726&partnerID=8YFLogxK
U2 - 10.1109/ICPP.2016.59
DO - 10.1109/ICPP.2016.59
M3 - Conference contribution
AN - SCOPUS:84991020726
T3 - Proceedings of the International Conference on Parallel Processing
SP - 458
EP - 463
BT - Proceedings - 45th International Conference on Parallel Processing, ICPP 2016
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 16 August 2016 through 19 August 2016
ER -