Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Recurrent Graph Tensor Networks: A Low-Complexity Framework for Modelling High-Dimensional Multi-Way Sequence
(Submitted on 18 Sep 2020 (v1), last revised 11 May 2021 (this version, v5))
Abstract: Recurrent Neural Networks (RNNs) are among the most successful machine learning models for sequence modelling, but tend to suffer from an exponential increase in the number of parameters when dealing with large multidimensional data. To this end, we develop a multi-linear graph filter framework for approximating the modelling of hidden states in RNNs, which is embedded in a tensor network architecture to improve modelling power and reduce parameter complexity, resulting in a novel Recurrent Graph Tensor Network (RGTN). The proposed framework is validated through several multi-way sequence modelling tasks and benchmarked against traditional RNNs. By virtue of the domain aware information processing of graph filters and the expressive power of tensor networks, we show that the proposed RGTN is capable of not only out-performing standard RNNs, but also mitigating the Curse of Dimensionality associated with traditional RNNs, demonstrating superior properties in terms of performance and complexity.
Submission history
From: Yao Lei Xu [view email][v1] Fri, 18 Sep 2020 10:13:36 GMT (934kb,D)
[v2] Wed, 30 Sep 2020 14:00:01 GMT (921kb,D)
[v3] Sat, 17 Oct 2020 14:18:30 GMT (921kb,D)
[v4] Wed, 11 Nov 2020 12:35:31 GMT (919kb,D)
[v5] Tue, 11 May 2021 12:40:23 GMT (1387kb,D)
Link back to: arXiv, form interface, contact.