Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Matrix Shuffle-Exchange Networks for Hard 2D Tasks
(Submitted on 29 Jun 2020 (v1), last revised 5 Oct 2020 (this version, v2))
Abstract: Convolutional neural networks have become the main tools for processing two-dimensional data. They work well for images, yet convolutions have a limited receptive field that prevents its applications to more complex 2D tasks. We propose a new neural model, called Matrix Shuffle-Exchange network, that can efficiently exploit long-range dependencies in 2D data and has comparable speed to a convolutional neural network. It is derived from Neural Shuffle-Exchange network and has $\mathcal{O}( \log{n})$ layers and $\mathcal{O}( n^2 \log{n})$ total time and space complexity for processing a $n \times n$ data matrix. We show that the Matrix Shuffle-Exchange network is well-suited for algorithmic and logical reasoning tasks on matrices and dense graphs, exceeding convolutional and graph neural network baselines. Its distinct advantage is the capability of retaining full long-range dependency modelling when generalizing to larger instances - much larger than could be processed with models equipped with a dense attention mechanism.
Submission history
From: Emīls Ozoliņš [view email][v1] Mon, 29 Jun 2020 09:38:54 GMT (1362kb,D)
[v2] Mon, 5 Oct 2020 08:55:35 GMT (391kb,D)
Link back to: arXiv, form interface, contact.