We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AI

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: Text Guide: Improving the quality of long text classification by a text selection method based on feature importance

Authors: Krzysztof Fiok (1), Waldemar Karwowski (1), Edgar Gutierrez (1) (2), Mohammad Reza Davahli (1), Maciej Wilamowski (3), Tareq Ahram (1), Awad Al-Juaid (4), Jozef Zurada (5) ((1) Department of Industrial Engineering and Management Systems, University of Central Florida, USA, (2) Center for Latin-American Logistics Innovation, LOGyCA, Bogota, Colombia (3) Faculty of Economic Sciences, University of Warsaw, Warsaw, Poland (4) Department of Industrial Engineering, College of Engineering, Taif University, Saudi Arabia (5) Business School, University of Louisville, USA)
Abstract: The performance of text classification methods has improved greatly over the last decade for text instances of less than 512 tokens. This limit has been adopted by most state-of-the-research transformer models due to the high computational cost of analyzing longer text instances. To mitigate this problem and to improve classification for longer texts, researchers have sought to resolve the underlying causes of the computational cost and have proposed optimizations for the attention mechanism, which is the key element of every transformer model. In our study, we are not pursuing the ultimate goal of long text classification, i.e., the ability to analyze entire text instances at one time while preserving high performance at a reasonable computational cost. Instead, we propose a text truncation method called Text Guide, in which the original text length is reduced to a predefined limit in a manner that improves performance over naive and semi-naive approaches while preserving low computational costs. Text Guide benefits from the concept of feature importance, a notion from the explainable artificial intelligence domain. We demonstrate that Text Guide can be used to improve the performance of recent language models specifically designed for long text classification, such as Longformer. Moreover, we discovered that parameter optimization is the key to Text Guide performance and must be conducted before the method is deployed. Future experiments may reveal additional benefits provided by this new method.
Comments: This is the reviewed and accepted for publication version of the article by the IEEE Access Journal. One of the important modifications is publication of the code along with the paper. The code can be used to apply Text Guide to a data set of ones choice. The code is available at: this https URL
Subjects: Artificial Intelligence (cs.AI)
DOI: 10.1109/ACCESS.2021.3099758
Cite as: arXiv:2104.07225 [cs.AI]
  (or arXiv:2104.07225v2 [cs.AI] for this version)

Submission history

From: Krzysztof Fiok [view email]
[v1] Thu, 15 Apr 2021 04:10:08 GMT (989kb)
[v2] Mon, 25 Oct 2021 08:24:33 GMT (784kb)

Link back to: arXiv, form interface, contact.