We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.SE

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Software Engineering

Title: Towards better data discovery and collection with flow-based programming

Abstract: Despite huge successes reported by the field of machine learning, such as voice assistants or self-driving cars, businesses still observe very high failure rate when it comes to deployment of ML in production. We argue that part of the reason is infrastructure that was not designed for data-oriented activities. This paper explores the potential of flow-based programming (FBP) for simplifying data discovery and collection in software systems. We compare FBP with the currently prevalent service-oriented paradigm to assess characteristics of each paradigm in the context of ML deployment. We develop a data processing application, formulate a subsequent ML deployment task, and measure the impact of the task implementation within both programming paradigms. Our main conclusion is that FBP shows great potential for providing data-centric infrastructural benefits for deployment of ML. Additionally, we provide an insight into the current trend that prioritizes model development over data quality management.
Comments: Extended version. Short version is accepted to Data-Centric AI Workshop, NeurIPS 2021
Subjects: Software Engineering (cs.SE); Machine Learning (cs.LG)
Cite as: arXiv:2108.04105 [cs.SE]
  (or arXiv:2108.04105v2 [cs.SE] for this version)

Submission history

From: Andrei Paleyes [view email]
[v1] Mon, 9 Aug 2021 15:06:02 GMT (1465kb,D)
[v2] Mon, 25 Oct 2021 21:16:47 GMT (471kb,D)

Link back to: arXiv, form interface, contact.