We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.IT

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Computer Science > Information Theory

Title: Memory AMP

Abstract: Approximate message passing (AMP) is a low-cost iterative parameter-estimation technique for certain high-dimensional linear systems with non-Gaussian distributions. AMP only applies to independent identically distributed (IID) transform matrices, but may become unreliable (e.g., perform poorly or even diverge) for other matrix ensembles, especially for ill-conditioned ones. To solve this issue, orthogonal/vector AMP (OAMP/VAMP) was proposed for general right-unitarily-invariant matrices. However, the Bayes-optimal OAMP/VAMP (BO-OAMP/VAMP) requires a high-complexity linear minimum mean square error (MMSE) estimator. This prevents OAMP/VAMP from being used in large-scale systems.
To address the drawbacks of AMP and BO-OAMP/VAMP, this paper offers a memory AMP (MAMP) framework based on the orthogonality principle, which ensures that estimation errors in MAMP are asymptotically IID Gaussian. To realize the required orthogonality for MAMP, we provide an orthogonalization procedure for the local memory estimators. In addition, we propose a Bayes-optimal MAMP (BO-MAMP), in which a long-memory matched filter is used for interference suppression. The complexity of BO-MAMP is comparable to AMP. To asymptotically characterize the performance of BO-MAMP, a state evolution is derived. The relaxation parameters and damping vector in BO-MAMP are optimized based on state evolution. Most crucially, the state evolution of the optimized BO-MAMP converges to the same fixed point as that of the high-complexity BO-OAMP/VAMP for all right-unitarily-invariant matrices, and achieves the Bayes optimal MSE predicted by the replica method if its state evolution has a unique fixed point. Finally, simulations are provided to verify the theoretical results' validity and accuracy.
Comments: Accepted by IEEE Trans. on Information Theory. [Memory AMP inherits the strengths of AMP and OAMP/VAMP such as low complexity, Bayes optimality and applicability to unitarily-inavariant matrices while avoiding the weakness of AMP (e.g. limited to IID matrices), OAMP/VAMP (e.g. needs high-complexity LMMSE) and CAMP (e.g., Fails to converge in high condition numbers).]
Subjects: Information Theory (cs.IT); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Signal Processing (eess.SP); Statistics Theory (math.ST)
Cite as: arXiv:2012.10861 [cs.IT]
  (or arXiv:2012.10861v7 [cs.IT] for this version)

Submission history

From: Lei Liu [view email]
[v1] Sun, 20 Dec 2020 07:42:15 GMT (202kb,D)
[v2] Wed, 10 Feb 2021 14:31:38 GMT (305kb,D)
[v3] Thu, 3 Jun 2021 04:08:05 GMT (307kb,D)
[v4] Thu, 5 Aug 2021 04:37:16 GMT (686kb,D)
[v5] Thu, 30 Dec 2021 10:46:19 GMT (1348kb,D)
[v6] Thu, 26 May 2022 06:38:06 GMT (1364kb,D)
[v7] Thu, 23 Jun 2022 02:58:34 GMT (1144kb)

Link back to: arXiv, form interface, contact.