We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: A Bayesian/Information Theoretic Model of Bias Learning

Abstract: In this paper the problem of learning appropriate bias for an environment of related tasks is examined from a Bayesian perspective. The environment of related tasks is shown to be naturally modelled by the concept of an {\em objective} prior distribution. Sampling from the objective prior corresponds to sampling different learning tasks from the environment. It is argued that for many common machine learning problems, although we don't know the true (objective) prior for the problem, we do have some idea of a set of possible priors to which the true prior belongs. It is shown that under these circumstances a learner can use Bayesian inference to learn the true prior by sampling from the objective prior. Bounds are given on the amount of information required to learn a task when it is simultaneously learnt with several other tasks. The bounds show that if the learner has little knowledge of the true prior, and the dimensionality of the true prior is small, then sampling multiple tasks is highly advantageous.
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Journal reference: COLT 96 Proceedings of the ninth annual conference on Computational learning theory (1996) Pages 77-88
DOI: 10.1145/238061.238071
Cite as: arXiv:1911.06129 [cs.LG]
  (or arXiv:1911.06129v1 [cs.LG] for this version)

Submission history

From: Jonathan Baxter [view email]
[v1] Thu, 14 Nov 2019 14:34:58 GMT (25kb)

Link back to: arXiv, form interface, contact.