References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives
(Submitted on 27 Jan 2022)
Abstract: This paper introduces Ranking Info Noise Contrastive Estimation (RINCE), a new member in the family of InfoNCE losses that preserves a ranked ordering of positive samples. In contrast to the standard InfoNCE loss, which requires a strict binary separation of the training pairs into similar and dissimilar samples, RINCE can exploit information about a similarity ranking for learning a corresponding embedding space. We show that the proposed loss function learns favorable embeddings compared to the standard InfoNCE whenever at least noisy ranking information can be obtained or when the definition of positives and negatives is blurry. We demonstrate this for a supervised classification task with additional superclass labels and noisy similarity scores. Furthermore, we show that RINCE can also be applied to unsupervised training with experiments on unsupervised representation learning from videos. In particular, the embedding yields higher classification accuracy, retrieval rates and performs better in out-of-distribution detection than the standard InfoNCE loss.
Submission history
From: Nadine Behrmann [view email][v1] Thu, 27 Jan 2022 18:55:32 GMT (11442kb,D)
Link back to: arXiv, form interface, contact.