We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: On the Unintended Social Bias of Training Language Generation Models with Data from Local Media

Abstract: There are concerns that neural language models may preserve some of the stereotypes of the underlying societies that generate the large corpora needed to train these models. For example, gender bias is a significant problem when generating text, and its unintended memorization could impact the user experience of many applications (e.g., the smart-compose feature in Gmail).
In this paper, we introduce a novel architecture that decouples the representation learning of a neural model from its memory management role. This architecture allows us to update a memory module with an equal ratio across gender types addressing biased correlations directly in the latent space. We experimentally show that our approach can mitigate the gender bias amplification in the automatic generation of articles news while providing similar perplexity values when extending the Sequence2Sequence architecture.
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:1911.00461 [cs.CL]
  (or arXiv:1911.00461v1 [cs.CL] for this version)

Submission history

From: Omar U. Florez [view email]
[v1] Fri, 1 Nov 2019 16:52:02 GMT (241kb,D)

Link back to: arXiv, form interface, contact.