References & Citations
Computer Science > Machine Learning
Title: Near-optimal Bayesian Active Learning with Correlated and Noisy Tests
(Submitted on 24 May 2016 (v1), last revised 11 Jul 2016 (this version, v2))
Abstract: We consider the Bayesian active learning and experimental design problem, where the goal is to learn the value of some unknown target variable through a sequence of informative, noisy tests. In contrast to prior work, we focus on the challenging, yet practically relevant setting where test outcomes can be conditionally dependent given the hidden target variable. Under such assumptions, common heuristics, such as greedily performing tests that maximize the reduction in uncertainty of the target, often perform poorly. In this paper, we propose ECED, a novel, computationally efficient active learning algorithm, and prove strong theoretical guarantees that hold with correlated, noisy tests. Rather than directly optimizing the prediction error, at each step, ECED picks the test that maximizes the gain in a surrogate objective, which takes into account the dependencies between tests. Our analysis relies on an information-theoretic auxiliary function to track the progress of ECED, and utilizes adaptive submodularity to attain the near-optimal bound. We demonstrate strong empirical performance of ECED on two problem instances, including a Bayesian experimental design task intended to distinguish among economic theories of how people make risky decisions, and an active preference learning task via pairwise comparisons.
Submission history
From: Yuxin Chen [view email][v1] Tue, 24 May 2016 08:25:27 GMT (3161kb,D)
[v2] Mon, 11 Jul 2016 06:47:19 GMT (3162kb,D)
Link back to: arXiv, form interface, contact.