We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches

Abstract: Text classification of unseen classes is a challenging Natural Language Processing task and is mainly attempted using two different types of approaches. Similarity-based approaches attempt to classify instances based on similarities between text document representations and class description representations. Zero-shot text classification approaches aim to generalize knowledge gained from a training task by assigning appropriate labels of unknown classes to text documents. Although existing studies have already investigated individual approaches to these categories, the experiments in literature do not provide a consistent comparison. This paper addresses this gap by conducting a systematic evaluation of different similarity-based and zero-shot approaches for text classification of unseen classes. Different state-of-the-art approaches are benchmarked on four text classification datasets, including a new dataset from the medical domain. Additionally, novel SimCSE and SBERT-based baselines are proposed, as other baselines used in existing work yield weak classification results and are easily outperformed. Finally, the novel similarity-based Lbl2TransformerVec approach is presented, which outperforms previous state-of-the-art approaches in unsupervised text classification. Our experiments show that similarity-based approaches significantly outperform zero-shot approaches in most cases. Additionally, using SimCSE or SBERT embeddings instead of simpler text representations increases similarity-based classification results even further.
Comments: Accepted to 6th International Conference on Natural Language Processing and Information Retrieval (NLPIR '22)
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
ACM classes: I.2.7
Cite as: arXiv:2211.16285 [cs.CL]
  (or arXiv:2211.16285v2 [cs.CL] for this version)

Submission history

From: Tim Schopf [view email]
[v1] Tue, 29 Nov 2022 15:14:47 GMT (8247kb,D)
[v2] Tue, 31 Jan 2023 08:51:19 GMT (8248kb,D)

Link back to: arXiv, form interface, contact.