Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Beam Tree Recursive Cells
(Submitted on 31 May 2023 (v1), last revised 20 Jun 2023 (this version, v3))
Abstract: We propose Beam Tree Recursive Cell (BT-Cell) - a backpropagation-friendly framework to extend Recursive Neural Networks (RvNNs) with beam search for latent structure induction. We further extend this framework by proposing a relaxation of the hard top-k operators in beam search for better propagation of gradient signals. We evaluate our proposed models in different out-of-distribution splits in both synthetic and realistic data. Our experiments show that BTCell achieves near-perfect performance on several challenging structure-sensitive synthetic tasks like ListOps and logical inference while maintaining comparable performance in realistic data against other RvNN-based models. Additionally, we identify a previously unknown failure case for neural models in generalization to unseen number of arguments in ListOps. The code is available at: this https URL
Submission history
From: Jishnu Ray Chowdhury [view email][v1] Wed, 31 May 2023 16:20:04 GMT (201kb,D)
[v2] Thu, 1 Jun 2023 14:42:16 GMT (201kb,D)
[v3] Tue, 20 Jun 2023 11:50:33 GMT (205kb,D)
Link back to: arXiv, form interface, contact.