Title: Investigating usability of MSstatsQC software

Abstract: MSstatsQC [3] is an open-source software that provides longitudinal system suitability monitoring tools in the form of control charts for proteomic experiments. It includes simultaneous tools for the mean and dispersion of suitability metrics and presents alternative methods of monitoring through different tabs that are designed in the interface. This research focuses on investigating the usability of MSstatsQC software and the interpretability of the designed plots. In this study, we ask 4 test users, from the proteomics field, to complete a series of tasks and questionnaires. The tasks are designed to test the usability of the software in terms of importing data files, selecting appropriate metrics, guide set, and peptides, and finally creating decision rules (tasks 1 and 3 in appendix). The questionnaires ask about interpretability of the plots including control charts, box plots, heat maps, river plots, and radar plots (tasks 1 and 4 in appendix). The goal of the questions is to determine if the test users understand the plots and can interpret them. Results show limitations in usability and plot interpretability, especially in the data import section. We suggest the following modifications. I) providing conspicuous guides close to the window related to up-loading a datafile as well as providing error messages that pop-up when the data set has a wrong format II) providing plot descriptions, hints to interpret plots, plot titles and appropriate axis labels, and, III) Numbering tabs to show the flow of procedures in the software.