We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Distributed, Parallel, and Cluster Computing

Title: Real-Time XFEL Data Analysis at SLAC and NERSC: a Trial Run of Nascent Exascale Experimental Data Analysis

Authors: Johannes P. Blaschke (1), Aaron S. Brewster (2), Daniel W. Paley (2), Derek Mendez (2), Asmit Bhowmick (2), Nicholas K. Sauter (2), Wilko Kröger (3), Murali Shankar (3), Bjoern Enders (1), Deborah Bard (1) ((1) National Energy Research Scientific Computing Center, Lawrence Berkeley National Laboratory, USA, (2) Molecular Biophysics and Integrated Bioimaging Division, Lawrence Berkeley National Laboratory, USA, (3) SLAC National Accelerator Laboratory, USA)
Abstract: X-ray scattering experiments using Free Electron Lasers (XFELs) are a powerful tool to determine the molecular structure and function of unknown samples (such as COVID-19 viral proteins). XFEL experiments are a challenge to computing in two ways: i) due to the high cost of running XFELs, a fast turnaround time from data acquisition to data analysis is essential to make informed decisions on experimental protocols; ii) data collection rates are growing exponentially, requiring new scalable algorithms. Here we report our experiences analyzing data from two experiments at the Linac Coherent Light Source (LCLS) during September 2020. Raw data were analyzed on NERSC's Cori XC40 system, using the Superfacility paradigm: our workflow automatically moves raw data between LCLS and NERSC, where it is analyzed using the software package CCTBX. We achieved real time data analysis with a turnaround time from data acquisition to full molecular reconstruction in as little as 10 min -- sufficient time for the experiment's operators to make informed decisions. By hosting the data analysis on Cori, and by automating LCLS-NERSC interoperability, we achieved a data analysis rate which matches the data acquisition rate. Completing data analysis with 10 mins is a first for XFEL experiments and an important milestone if we are to keep up with data collection trends.
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Data Analysis, Statistics and Probability (physics.data-an)
Cite as: arXiv:2106.11469 [cs.DC]
  (or arXiv:2106.11469v2 [cs.DC] for this version)

Submission history

From: Johannes Blaschke P [view email]
[v1] Tue, 22 Jun 2021 01:09:12 GMT (7637kb,D)
[v2] Mon, 16 Aug 2021 22:03:26 GMT (7640kb,D)

Link back to: arXiv, form interface, contact.