We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Actively Learning Deep Neural Networks with Uncertainty Sampling Based on Sum-Product Networks

Abstract: Active learning is popular approach for reducing the amount of data in training deep neural network model. Its success hinges on the choice of an effective acquisition function, which ranks not yet labeled data points according to their expected informativeness. In uncertainty sampling, the uncertainty that the current model has about a point's class label is the main criterion for this type of ranking. This paper proposes a new approach to uncertainty sampling in training a Convolutional Neural Network (CNN). The main idea is to use feature representation extracted extracted by the CNN as data for training a Sum-Product Network (SPN). Since SPNs are typically used for estimating the distribution of a dataset, they are well suited to the task of estimating class probabilities that can be used directly by standard acquisition functions such as max entropy and variational ratio. Moreover, we enhance these acquisition functions by weights calculated with the help of the SPN model; these weights make the acquisition function more sensitive to the diversity of conceivable class labels for data points. The effectiveness of our method is demonstrated in an experimental study on the MNIST, Fashion-MNIST and CIFAR-10 datasets, where we compare it to the state-of-the-art methods MC Dropout and Bayesian Batch.
Comments: 15 pages,9 figures, 4 tables
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)
MSC classes: 62P30
ACM classes: I.5.0; I.4.10
Cite as: arXiv:2206.09798 [cs.LG]
  (or arXiv:2206.09798v1 [cs.LG] for this version)

Submission history

From: Mohamadsadegh Khosravani [view email]
[v1] Mon, 20 Jun 2022 14:28:19 GMT (18344kb,D)

Link back to: arXiv, form interface, contact.