We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

astro-ph.IM

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Astrophysics > Instrumentation and Methods for Astrophysics

Title: Astronomical Pipeline Provenance: A Use Case Evaluation

Abstract: In this decade astronomy is undergoing a paradigm shift to handle data from next generation observatories such as the Square Kilometre Array (SKA) or the Vera C. Rubin Observatory (LSST). Producing real time data streams of up to 10 TB/s and data products of the order of 600 Pbytes/year, the SKA will be the biggest civil data producing machine of the world that demands novel solutions on how these data volumes can be stored and analysed. Through the use of complex, automated pipelines the provenance of this real time data processing is key to establish confidence within the system, its final data products, and ultimately its scientific results.
The intention of this paper is to lay the foundation for making an automated provenance generation tool for astronomical/data-processing pipelines. We therefore present a use case analysis, specific to the astronomical needs which addresses the issues of trust and reproducibility as well as other ulterior use cases which are of interest to astronomers. This analysis is subsequently used as the basis to discuss the requirements, challenges, and opportunities involved in designing both the tool and the associated provenance model.
Comments: 9 pages, 1 table
Subjects: Instrumentation and Methods for Astrophysics (astro-ph.IM); Information Retrieval (cs.IR)
ACM classes: H.3.3; J.2
Journal reference: 13th International Workshop on Theory and Practice of Provenance (2021) 54
Cite as: arXiv:2109.10759 [astro-ph.IM]
  (or arXiv:2109.10759v1 [astro-ph.IM] for this version)

Submission history

From: Michael Johnson Dr [view email]
[v1] Wed, 22 Sep 2021 14:25:17 GMT (54kb)

Link back to: arXiv, form interface, contact.