References & Citations
Computer Science > Artificial Intelligence
Title: Countering Language Drift with Seeded Iterated Learning
(Submitted on 28 Mar 2020 (v1), last revised 24 Aug 2020 (this version, v3))
Abstract: Pretraining on human corpus and then finetuning in a simulator has become a standard pipeline for training a goal-oriented dialogue agent. Nevertheless, as soon as the agents are finetuned to maximize task completion, they suffer from the so-called language drift phenomenon: they slowly lose syntactic and semantic properties of language as they only focus on solving the task. In this paper, we propose a generic approach to counter language drift called Seeded iterated learning (SIL). We periodically refine a pretrained student agent by imitating data sampled from a newly generated teacher agent. At each time step, the teacher is created by copying the student agent, before being finetuned to maximize task completion. SIL does not require external syntactic constraint nor semantic knowledge, making it a valuable task-agnostic finetuning protocol. We evaluate SIL in a toy-setting Lewis Game, and then scale it up to the translation game with natural language. In both settings, SIL helps counter language drift as well as it improves the task completion compared to baselines.
Submission history
From: Yuchen Lu [view email][v1] Sat, 28 Mar 2020 03:45:31 GMT (3728kb,D)
[v2] Mon, 6 Apr 2020 03:24:46 GMT (3728kb,D)
[v3] Mon, 24 Aug 2020 19:26:54 GMT (7230kb,D)
Link back to: arXiv, form interface, contact.