Computational Complexity

New submissions

• New submissions
• Cross-lists
• Replacements

New submissions for Wed, 18 May 22

[1]  arXiv:2205.08032 [pdf, ps, other]

Title: On Algebraic Constructions of Neural Networks with Small Weights

Authors: Kordag Mehmet Kilic, Jin Sima, Jehoshua Bruck

Subjects: Computational Complexity (cs.CC); Discrete Mathematics (cs.DM); Information Theory (cs.IT); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)

Neural gates compute functions based on weighted sums of the input variables. The expressive power of neural gates (number of distinct functions it can compute) depends on the weight sizes and, in general, large weights (exponential in the number of inputs) are required. Studying the trade-offs among the weight sizes, circuit size and depth is a well-studied topic both in circuit complexity theory and the practice of neural computation. We propose a new approach for studying these complexity trade-offs by considering a related algebraic framework. Specifically, given a single linear equation with arbitrary coefficients, we would like to express it using a system of linear equations with smaller (even constant) coefficients. The techniques we developed are based on Siegel's Lemma for the bounds, anti-concentration inequalities for the existential results and extensions of Sylvester-type Hadamard matrices for the constructions.
We explicitly construct a constant weight, optimal size matrix to compute the EQUALITY function (checking if two integers expressed in binary are equal). Computing EQUALITY with a single linear equation requires exponentially large weights. In addition, we prove the existence of the best-known weight size (linear) matrices to compute the COMPARISON function (comparing between two integers expressed in binary). In the context of the circuit complexity theory, our results improve the upper bounds on the weight sizes for the best-known circuit sizes for EQUALITY and COMPARISON.

Cross-lists for Wed, 18 May 22

[2]  arXiv:2205.07528 (cross-list from math.RA) [pdf, ps, other]

Title: The Smallest Hard Trees

Authors: Manuel Bodirsky, Jakub Bulin, Florian Starke, Michael Wernthaler

Subjects: Rings and Algebras (math.RA); Computational Complexity (cs.CC)

We find an orientation of a tree with 20 vertices such that the corresponding fixed-template constraint satisfaction problem (CSP) is NP-complete, and prove that for every orientation of a tree with fewer vertices the corresponding CSP can be solved in polynomial time. We also compute the smallest tree that is NL-hard (assuming L is not NL), the smallest tree that cannot be solved by arc consistency, and the smallest tree that cannot be solved by Datalog. Our experimental results also support a conjecture of Bulin concerning a question of Hell, Nesetril and Zhu, namely that "easy trees lack the ability to count". Most proofs are computer-based and make use of the most recent universal-algebraic theory about the complexity of finite-domain CSPs. However, further ideas are required because of the huge number of orientations of trees. In particular, we use the well-known fact that it suffices to study orientations of trees that are cores and show how to efficiently decide whether a given orientation of a tree is a core using the arc-consistency procedure. Moreover, we present a method to generate orientations of trees that are cores that works well in practice. In this way we found interesting examples for the open research problem to classify finite-domain CSPs in NL.

Replacements for Wed, 18 May 22

[3]  arXiv:2205.02374 (replaced) [pdf, other]

Title: The composition complexity of majority

Authors: Victor Lecomte, Prasanna Ramakrishnan, Li-Yang Tan

Comments: to appear in CCC 2022. Fixed typos, updated references

Subjects: Computational Complexity (cs.CC)

• New submissions
• Cross-lists
• Replacements