Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: PoWER-BERT: Accelerating BERT Inference via Progressive Word-vector Elimination
(Submitted on 24 Jan 2020 (v1), last revised 8 Sep 2020 (this version, v5))
Abstract: We develop a novel method, called PoWER-BERT, for improving the inference time of the popular BERT model, while maintaining the accuracy. It works by: a) exploiting redundancy pertaining to word-vectors (intermediate encoder outputs) and eliminating the redundant vectors. b) determining which word-vectors to eliminate by developing a strategy for measuring their significance, based on the self-attention mechanism. c) learning how many word-vectors to eliminate by augmenting the BERT model and the loss function. Experiments on the standard GLUE benchmark shows that PoWER-BERT achieves up to 4.5x reduction in inference time over BERT with <1% loss in accuracy. We show that PoWER-BERT offers significantly better trade-off between accuracy and inference time compared to prior methods. We demonstrate that our method attains up to 6.8x reduction in inference time with <1% loss in accuracy when applied over ALBERT, a highly compressed version of BERT. The code for PoWER-BERT is publicly available at this https URL
Submission history
From: Saurabh Goyal [view email][v1] Fri, 24 Jan 2020 11:36:12 GMT (4624kb,D)
[v2] Wed, 17 Jun 2020 20:53:56 GMT (4671kb,D)
[v3] Sat, 11 Jul 2020 12:22:08 GMT (4671kb,D)
[v4] Tue, 21 Jul 2020 18:35:54 GMT (4671kb,D)
[v5] Tue, 8 Sep 2020 14:11:33 GMT (4671kb,D)
Link back to: arXiv, form interface, contact.