We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection

Abstract: Pre-trained models have proved to be powerful in enhancing task-oriented dialog systems. However, current pre-training methods mainly focus on enhancing dialog understanding and generation tasks while neglecting the exploitation of dialog policy. In this paper, we propose GALAXY, a novel pre-trained dialog model that explicitly learns dialog policy from limited labeled dialogs and large-scale unlabeled dialog corpora via semi-supervised learning. Specifically, we introduce a dialog act prediction task for policy optimization during pre-training and employ a consistency regularization term to refine the learned representation with the help of unlabeled dialogs. We also implement a gating mechanism to weigh suitable unlabeled dialog samples. Empirical results show that GALAXY substantially improves the performance of task-oriented dialog systems, and achieves new state-of-the-art results on benchmark datasets: In-Car, MultiWOZ2.0 and MultiWOZ2.1, improving their end-to-end combined scores by 2.5, 5.3 and 5.5 points, respectively. We also show that GALAXY has a stronger few-shot ability than existing models under various low-resource settings.
Comments: 7 pages, 5 figures. Accepted by AAAI 2022
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2111.14592 [cs.CL]
  (or arXiv:2111.14592v7 [cs.CL] for this version)

Submission history

From: Yinpei Dai [view email]
[v1] Mon, 29 Nov 2021 15:24:36 GMT (5811kb,D)
[v2] Wed, 1 Dec 2021 02:15:16 GMT (5828kb,D)
[v3] Tue, 7 Dec 2021 02:39:31 GMT (5828kb,D)
[v4] Thu, 9 Dec 2021 13:43:16 GMT (5829kb,D)
[v5] Mon, 13 Dec 2021 05:47:06 GMT (5829kb,D)
[v6] Mon, 27 Dec 2021 09:35:34 GMT (5829kb,D)
[v7] Thu, 6 Jan 2022 09:44:18 GMT (5829kb,D)

Link back to: arXiv, form interface, contact.