We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Oversquashing in GNNs through the lens of information contraction and graph expansion

Abstract: The quality of signal propagation in message-passing graph neural networks (GNNs) strongly influences their expressivity as has been observed in recent works. In particular, for prediction tasks relying on long-range interactions, recursive aggregation of node features can lead to an undesired phenomenon called "oversquashing". We present a framework for analyzing oversquashing based on information contraction. Our analysis is guided by a model of reliable computation due to von Neumann that lends a new insight into oversquashing as signal quenching in noisy computation graphs. Building on this, we propose a graph rewiring algorithm aimed at alleviating oversquashing. Our algorithm employs a random local edge flip primitive motivated by an expander graph construction. We compare the spectral expansion properties of our algorithm with that of an existing curvature-based non-local rewiring strategy. Synthetic experiments show that while our algorithm in general has a slower rate of expansion, it is overall computationally cheaper, preserves the node degrees exactly and never disconnects the graph.
Comments: 8 pages, 5 figures; Accepted at the 58th Annual Allerton Conference on Communication, Control, and Computing
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)
Cite as: arXiv:2208.03471 [cs.LG]
  (or arXiv:2208.03471v1 [cs.LG] for this version)

Submission history

From: Pradeep Kr. Banerjee [view email]
[v1] Sat, 6 Aug 2022 08:44:39 GMT (919kb,D)

Link back to: arXiv, form interface, contact.