We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Adversarial Mixing Policy for Relaxing Locally Linear Constraints in Mixup

Abstract: Mixup is a recent regularizer for current deep classification networks. Through training a neural network on convex combinations of pairs of examples and their labels, it imposes locally linear constraints on the model's input space. However, such strict linear constraints often lead to under-fitting which degrades the effects of regularization. Noticeably, this issue is getting more serious when the resource is extremely limited. To address these issues, we propose the Adversarial Mixing Policy (AMP), organized in a min-max-rand formulation, to relax the Locally Linear Constraints in Mixup. Specifically, AMP adds a small adversarial perturbation to the mixing coefficients rather than the examples. Thus, slight non-linearity is injected in-between the synthetic examples and synthetic labels. By training on these data, the deep networks are further regularized, and thus achieve a lower predictive error rate. Experiments on five text classification benchmarks and five backbone models have empirically shown that our methods reduce the error rate over Mixup variants in a significant margin (up to 31.3%), especially in low-resource conditions (up to 17.5%).
Comments: This paper is accepted to appear in the main conference of EMNLP2021
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
MSC classes: NLP
Cite as: arXiv:2109.07177 [cs.CL]
  (or arXiv:2109.07177v1 [cs.CL] for this version)

Submission history

From: Guang Liu [view email]
[v1] Wed, 15 Sep 2021 09:31:59 GMT (460kb,D)

Link back to: arXiv, form interface, contact.