In-situ feature-based objects tracking for data-intensive scientific and enterprise analytics workflows

Solomon Lasluisa, Fan Zhang, Tong Jin, Ivan Rodero, Hoang Bui, Manish Parashar

Research output: Contribution to journalArticle

4 Scopus citations

Abstract

Emerging scientific simulations on leadership class systems are generating huge amounts of data and processing this data in an efficient and timely manner is critical for generating insights from the simulations. However, the increasing gap between computation and disk I/O speeds makes traditional data analytics pipelines based on post-processing cost prohibitive and often infeasible. In this paper, we investigate an alternate approach that aims to bring the analytics closer to the data using in-situ execution of data analysis operations. Specifically, we present the design, implementation and evaluation of a framework that can support in-situ feature-based objects tracking on distributed scientific datasets. Central to this framework is a scalable decentralized and online clustering, a cluster tracking algorithm, which executes in-situ (on different cores) in parallel with the simulation processes, and retrieves data from the simulations directly via on-chip shared memory. The results from our experimental evaluation demonstrate that the in-situ approach significantly reduces the cost of data movement, that the presented framework can support scalable feature-based objects tracking, and that it can be effectively used for in-situ analytics in large scale simulations.

Original languageEnglish (US)
Pages (from-to)29-40
Number of pages12
JournalCluster Computing
Volume18
Issue number1
DOIs
StatePublished - Jan 1 2015

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications

Keywords

  • Feature-based objects tracking
  • Scalable in-situ data analytics
  • Scientific data analysis
  • Simulations workflows

Fingerprint Dive into the research topics of 'In-situ feature-based objects tracking for data-intensive scientific and enterprise analytics workflows'. Together they form a unique fingerprint.

  • Cite this