We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Statistics > Methodology

Title: A Practical Approach to Proper Inference with Linked Data

Abstract: Entity resolution (ER), comprising record linkage and de-duplication, is the process of merging noisy databases in the absence of unique identifiers to remove duplicate entities. One major challenge of analysis with linked data is identifying a representative record among determined matches to pass to an inferential or predictive task, referred to as the \emph{downstream task}. Additionally, incorporating uncertainty from ER in the downstream task is critical to ensure proper inference. To bridge the gap between ER and the downstream task in an analysis pipeline, we propose five methods to choose a representative (or canonical) record from linked data, referred to as canonicalization. Our methods are scalable in the number of records, appropriate in general data scenarios, and provide natural error propagation via a Bayesian canonicalization stage. The proposed methodology is evaluated on three simulated data sets and one application -- determining the relationship between demographic information and party affiliation in voter registration data from the North Carolina State Board of Elections. We first perform Bayesian ER and evaluate our proposed methods for canonicalization before considering the downstream tasks of linear and logistic regression. Bayesian canonicalization methods are empirically shown to improve downstream inference in both settings through prediction and coverage.
Comments: 31 pages, 2 figures
Subjects: Methodology (stat.ME); Databases (cs.DB); Machine Learning (cs.LG)
Cite as: arXiv:1810.01538 [stat.ME]
  (or arXiv:1810.01538v2 [stat.ME] for this version)

Submission history

From: Andee Kaplan [view email]
[v1] Tue, 2 Oct 2018 22:55:58 GMT (974kb,D)
[v2] Tue, 8 Feb 2022 22:01:17 GMT (6083kb,D)

Link back to: arXiv, form interface, contact.