We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IR

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Information Retrieval

Title: Collective Relevance Labeling for Passage Retrieval

Abstract: Deep learning for Information Retrieval (IR) requires a large amount of high-quality query-document relevance labels, but such labels are inherently sparse. Label smoothing redistributes some observed probability mass over unobserved instances, often uniformly, uninformed of the true distribution. In contrast, we propose knowledge distillation for informed labeling, without incurring high computation overheads at evaluation time. Our contribution is designing a simple but efficient teacher model which utilizes collective knowledge, to outperform state-of-the-arts distilled from a more complex teacher model. Specifically, we train up to x8 faster than the state-of-the-art teacher, while distilling the rankings better. Our code is publicly available at this https URL
Comments: NAACL 2022
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2205.03273 [cs.IR]
  (or arXiv:2205.03273v2 [cs.IR] for this version)

Submission history

From: Jihyuk Kim [view email]
[v1] Fri, 6 May 2022 14:47:15 GMT (1444kb,D)
[v2] Mon, 9 May 2022 01:40:07 GMT (1444kb,D)

Link back to: arXiv, form interface, contact.