We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CC

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computational Complexity

Title: Algorithmic Complexities in Backpropagation and Tropical Neural Networks

Authors: Ozgur Ceyhan
Abstract: In this note, we propose a novel technique to reduce the algorithmic complexity of neural network training by using matrices of tropical real numbers instead of matrices of real numbers. Since the tropical arithmetics replaces multiplication with addition, and addition with max, we theoretically achieve several order of magnitude better constant factors in time complexities in the training phase. The fact that we replace the field of real numbers with the tropical semiring of real numbers and yet achieve the same classification results via neural networks come from deep results in topology and analysis, which we verify in our note. We then explore artificial neural networks in terms of tropical arithmetics and tropical algebraic geometry, and introduce the multi-layered tropical neural networks as universal approximators. After giving a tropical reformulation of the backpropagation algorithm, we verify the algorithmic complexity is substantially lower than the usual backpropagation as the tropical arithmetic is free of the complexity of usual multiplication.
Comments: This note is a summary of the lecture given at S\'eminaire "Fables G\'eom\'etriques" in University of Geneva on December 9, 2016
Subjects: Computational Complexity (cs.CC); Machine Learning (cs.LG); Algebraic Geometry (math.AG)
Cite as: arXiv:2101.00717 [cs.CC]
  (or arXiv:2101.00717v1 [cs.CC] for this version)

Submission history

From: Ozgur Ceyhan [view email]
[v1] Sun, 3 Jan 2021 22:19:17 GMT (12kb)

Link back to: arXiv, form interface, contact.