References & Citations
Computer Science > Computation and Language
Title: Ensemble Learning-Based Approach for Improving Generalization Capability of Machine Reading Comprehension Systems
(Submitted on 1 Jul 2021 (v1), last revised 15 Jul 2021 (this version, v2))
Abstract: Machine Reading Comprehension (MRC) is an active field in natural language processing with many successful developed models in recent years. Despite their high in-distribution accuracy, these models suffer from two issues: high training cost and low out-of-distribution accuracy. Even though some approaches have been presented to tackle the generalization problem, they have high, intolerable training costs. In this paper, we investigate the effect of ensemble learning approach to improve generalization of MRC systems without retraining a big model. After separately training the base models with different structures on different datasets, they are ensembled using weighting and stacking approaches in probabilistic and non-probabilistic settings. Three configurations are investigated including heterogeneous, homogeneous, and hybrid on eight datasets and six state-of-the-art models. We identify the important factors in the effectiveness of ensemble methods. Also, we compare the robustness of ensemble and fine-tuned models against data distribution shifts. The experimental results show the effectiveness and robustness of the ensemble approach in improving the out-of-distribution accuracy of MRC systems, especially when the base models are similar in accuracies.
Submission history
From: Razieh Baradaran [view email][v1] Thu, 1 Jul 2021 11:11:17 GMT (270kb,D)
[v2] Thu, 15 Jul 2021 17:28:31 GMT (270kb,D)
Link back to: arXiv, form interface, contact.