Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: PClean: Bayesian Data Cleaning at Scale with Domain-Specific Probabilistic Programming
(Submitted on 23 Jul 2020 (v1), last revised 19 Nov 2022 (this version, v5))
Abstract: Data cleaning is naturally framed as probabilistic inference in a generative model of ground-truth data and likely errors, but the diversity of real-world error patterns and the hardness of inference make Bayesian approaches difficult to automate. We present PClean, a probabilistic programming language (PPL) for leveraging dataset-specific knowledge to automate Bayesian cleaning. Compared to general-purpose PPLs, PClean tackles a restricted problem domain, enabling three modeling and inference innovations: (1) a non-parametric model of relational database instances, which users' programs customize; (2) a novel sequential Monte Carlo inference algorithm that exploits the structure of PClean's model class; and (3) a compiler that generates near-optimal SMC proposals and blocked-Gibbs rejuvenation kernels based on the user's model and data. We show empirically that short (< 50-line) PClean programs can: be faster and more accurate than generic PPL inference on data-cleaning benchmarks; match state-of-the-art data-cleaning systems in terms of accuracy and runtime (unlike generic PPL inference in the same runtime); and scale to real-world datasets with millions of records.
Submission history
From: Alexander Lew [view email][v1] Thu, 23 Jul 2020 08:01:47 GMT (290kb,D)
[v2] Fri, 7 Aug 2020 05:19:18 GMT (292kb,D)
[v3] Sun, 25 Oct 2020 21:07:53 GMT (313kb,D)
[v4] Tue, 27 Oct 2020 18:41:52 GMT (313kb,D)
[v5] Sat, 19 Nov 2022 00:20:31 GMT (3184kb,D)
Link back to: arXiv, form interface, contact.