We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.AI

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: Implicit Regularization in Deep Tensor Factorization

Authors: Paolo Milanesi (QARMA), Hachem Kadri (LIS, QARMA, AMU SCI), Stéphane Ayache (QARMA), Thierry Artières (QARMA)
Abstract: Attempts of studying implicit regularization associated to gradient descent (GD) have identified matrix completion as a suitable test-bed. Late findings suggest that this phenomenon cannot be phrased as a minimization-norm problem, implying that a paradigm shift is required and that dynamics has to be taken into account. In the present work we address the more general setup of tensor completion by leveraging two popularized tensor factorization, namely Tucker and TensorTrain (TT). We track relevant quantities such as tensor nuclear norm, effective rank, generalized singular values and we introduce deep Tucker and TT unconstrained factorization to deal with the completion task. Experiments on both synthetic and real data show that gradient descent promotes solution with low-rank, and validate the conjecture saying that the phenomenon has to be addressed from a dynamical perspective.
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
Journal reference: International Joint Conference on Neural Networks (IJCNN), Jul 2021, Online, China
Cite as: arXiv:2105.01346 [cs.AI]
  (or arXiv:2105.01346v1 [cs.AI] for this version)

Submission history

From: Hachem Kadri [view email]
[v1] Tue, 4 May 2021 07:48:40 GMT (310kb,D)

Link back to: arXiv, form interface, contact.