Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: On the Robustness of Text Vectorizers
(Submitted on 9 Mar 2023 (v1), last revised 12 Jun 2023 (this version, v2))
Abstract: A fundamental issue in machine learning is the robustness of the model with respect to changes in the input. In natural language processing, models typically contain a first embedding layer, transforming a sequence of tokens into vector representations. While the robustness with respect to changes of continuous inputs is well-understood, the situation is less clear when considering discrete changes, for instance replacing a word by another in an input sentence. Our work formally proves that popular embedding schemes, such as concatenation, TF-IDF, and Paragraph Vector (a.k.a. doc2vec), exhibit robustness in the H\"older or Lipschitz sense with respect to the Hamming distance. We provide quantitative bounds for these schemes and demonstrate how the constants involved are affected by the length of the document. These findings are exemplified through a series of numerical examples.
Submission history
From: Damien Garreau [view email][v1] Thu, 9 Mar 2023 16:37:37 GMT (894kb,D)
[v2] Mon, 12 Jun 2023 12:55:48 GMT (923kb,D)
Link back to: arXiv, form interface, contact.