We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Confidence Calibration for Domain Generalization under Covariate Shift

Abstract: Existing calibration algorithms address the problem of covariate shift via unsupervised domain adaptation. However, these methods suffer from the following limitations: 1) they require unlabeled data from the target domain, which may not be available at the stage of calibration in real-world applications and 2) their performance depends heavily on the disparity between the distributions of the source and target domains. To address these two limitations, we present novel calibration solutions via domain generalization. Our core idea is to leverage multiple calibration domains to reduce the effective distribution disparity between the target and calibration domains for improved calibration transfer without needing any data from the target domain. We provide theoretical justification and empirical experimental results to demonstrate the effectiveness of our proposed algorithms. Compared against state-of-the-art calibration methods designed for domain adaptation, we observe a decrease of 8.86 percentage points in expected calibration error or, equivalently, an increase of 35 percentage points in improvement ratio for multi-class classification on the Office-Home dataset.
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
Journal reference: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 8958-8967
Cite as: arXiv:2104.00742 [cs.LG]
  (or arXiv:2104.00742v2 [cs.LG] for this version)

Submission history

From: Yunye Gong [view email]
[v1] Thu, 1 Apr 2021 19:31:54 GMT (317kb,D)
[v2] Thu, 19 Aug 2021 20:22:14 GMT (320kb,D)

Link back to: arXiv, form interface, contact.