References & Citations
Computer Science > Computation and Language
Title: PTR: Prompt Tuning with Rules for Text Classification
(Submitted on 24 May 2021 (v1), last revised 15 Sep 2021 (this version, v3))
Abstract: Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Prompt tuning has achieved promising results on some few-class classification tasks such as sentiment classification and natural language inference. However, manually designing lots of language prompts is cumbersome and fallible. For those auto-generated prompts, it is also expensive and time-consuming to verify their effectiveness in non-few-shot scenarios. Hence, it is still challenging for prompt tuning to address many-class classification tasks. To this end, we propose prompt tuning with rules (PTR) for many-class text classification and apply logic rules to construct prompts with several sub-prompts. In this way, PTR is able to encode prior knowledge of each class into prompt tuning. We conduct experiments on relation classification, a typical and complicated many-class classification task, and the results show that PTR can significantly and consistently outperform existing state-of-the-art baselines. This indicates that PTR is a promising approach to take advantage of both human prior knowledge and PLMs for those complicated classification tasks.
Submission history
From: Han Xu [view email][v1] Mon, 24 May 2021 13:24:02 GMT (308kb,D)
[v2] Mon, 31 May 2021 03:32:06 GMT (338kb,D)
[v3] Wed, 15 Sep 2021 09:12:35 GMT (446kb,D)
Link back to: arXiv, form interface, contact.