We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: NLP-IIS@UT at SemEval-2021 Task 4: Machine Reading Comprehension using the Long Document Transformer

Abstract: This paper presents a technical report of our submission to the 4th task of SemEval-2021, titled: Reading Comprehension of Abstract Meaning. In this task, we want to predict the correct answer based on a question given a context. Usually, contexts are very lengthy and require a large receptive field from the model. Thus, common contextualized language models like BERT miss fine representation and performance due to the limited capacity of the input tokens. To tackle this problem, we used the Longformer model to better process the sequences. Furthermore, we utilized the method proposed in the Longformer benchmark on Wikihop dataset which improved the accuracy on our task data from 23.01% and 22.95% achieved by the baselines for subtask 1 and 2, respectively, to 70.30% and 64.38%.
Comments: 6 pages, 1 figure. Accepted in SemEval2021
Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR); Information Theory (cs.IT); Machine Learning (cs.LG)
Cite as: arXiv:2105.03775 [cs.CL]
  (or arXiv:2105.03775v1 [cs.CL] for this version)

Submission history

From: Hossein Basafa [view email]
[v1] Sat, 8 May 2021 20:48:32 GMT (52kb,D)

Link back to: arXiv, form interface, contact.