Title: Tradeoffs between Convergence Speed and Reconstruction Accuracy in Inverse Problems

Abstract: Solving inverse problems with iterative algorithms such as stochastic gradient descent is a popular technique, especially for large data. In applications, due to time constraints, the number of iterations one may apply is usually limited, consequently limiting the accuracy achievable by certain methods. Given a reconstruction error one is willing to tolerate, an important question is whether it is possible to modify the original iterations to obtain a faster convergence to a minimizer with the allowed error. Relying on recent recovery techniques developed for settings in which the desired signal belongs to some low-dimensional set, we show that using a coarse estimate of this set leads to faster convergence to an error related to the accuracy of the set approximation. Our theory ties to recent advances in sparse recovery, compressed sensing and deep learning. In particular, it provides an explanation for the successful approximation of the ISTA solution by neural networks with layers representing iterations.