Neural and Evolutionary Computing
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Wed, 23 Sep 20
 [1] arXiv:2009.10199 [pdf, other]

Title: Evolutionary Architecture Search for Graph Neural NetworksComments: 11 pagesSubjects: Neural and Evolutionary Computing (cs.NE)
Automated machine learning (AutoML) has seen a resurgence in interest with the boom of deep learning over the past decade. In particular, Neural Architecture Search (NAS) has seen significant attention throughout the AutoML research community, and has pushed forward the stateoftheart in a number of neural models to address gridlike data such as texts and images. However, very litter work has been done about Graph Neural Networks (GNN) learning on unstructured network data. Given the huge number of choices and combinations of components such as aggregator and activation function, determining the suitable GNN structure for a specific problem normally necessitates tremendous expert knowledge and laborious trails. In addition, the slight variation of hyper parameters such as learning rate and dropout rate could dramatically hurt the learning capacity of GNN. In this paper, we propose a novel AutoML framework through the evolution of individual models in a large GNN architecture space involving both neural structures and learning parameters. Instead of optimizing only the model structures with fixed parameter settings as existing work, an alternating evolution process is performed between GNN structures and learning parameters to dynamically find the best fit of each other. To the best of our knowledge, this is the first work to introduce and evaluate evolutionary architecture search for GNN models. Experiments and validations demonstrate that evolutionary NAS is capable of matching existing stateoftheart reinforcement learning approaches for both the semisupervised transductive and inductive node representation learning and classification.
 [2] arXiv:2009.10460 [pdf, other]

Title: Multithreaded Memory Efficient Crossover in C++ for Generational Genetic ProgrammingAuthors: W. B. LangdonComments: 11 pages, 3 figuresSubjects: Neural and Evolutionary Computing (cs.NE)
C++ code snippets from a multicore parallel memoryefficient crossover for genetic programming are given. They may be adapted for separate generation evolutionary algorithms where large chromosomes or small RAM require no more than M + (2 times nthreads) simultaneously active individuals.
 [3] arXiv:2009.10520 [pdf, other]

Title: Complex Vehicle Routing with Memory Augmented Neural NetworksComments: Presented at 3RD IEEE International Conference on Industrial CyberPhysical Systems, 2020Subjects: Neural and Evolutionary Computing (cs.NE)
Complex reallife routing challenges can be modeled as variations of wellknown combinatorial optimization problems. These routing problems have long been studied and are difficult to solve at scale. The particular setting may also make exact formulation difficult. Deep Learning offers an increasingly attractive alternative to traditional solutions, which mainly revolve around the use of various heuristics. Deep Learning may provide solutions which are less timeconsuming and of higher quality at large scales, as it generally does not need to generate solutions in an iterative manner, and Deep Learning models have shown a surprising capacity for solving complex tasks in recent years. Here we consider a particular variation of the Capacitated Vehicle Routing (CVRP) problem and investigate the use of Deep Learning models with explicit memory components. Such memory components may help in gaining insight into the model's decisions as the memory and operations on it can be directly inspected at any time, and may assist in scaling the method to such a size that it becomes viable for industry settings.
 [4] arXiv:2009.10685 [pdf, other]

Title: Tensor Programs III: Neural Matrix LawsAuthors: Greg YangSubjects: Neural and Evolutionary Computing (cs.NE); Probability (math.PR)
In a neural network (NN), \emph{weight matrices} linearly transform inputs into \emph{preactivations} that are then transformed nonlinearly into \emph{activations}. A typical NN interleaves multitudes of such linear and nonlinear transforms to express complex functions. Thus, the (pre)activations depend on the weights in an intricate manner. We show that, surprisingly, (pre)activations of a randomly initialized NN become \emph{independent} from the weights as the NN's widths tend to infinity, in the sense of \emph{asymptotic freeness} in random matrix theory. We call this the \emph{Free Independence Principle (FIP)}, which has these consequences: 1) It rigorously justifies the calculation of asymptotic Jacobian singular value distribution of an NN in Pennington et al. [36,37], essential for training ultradeep NNs [48]. 2) It gives a new justification of \emph{gradient independence assumption} used for calculating the \emph{Neural Tangent Kernel} of a neural network. FIP and these results hold for any neural architecture. We show FIP by proving a Master Theorem for any Tensor Program, as introduced in Yang [50,51], generalizing the Master Theorems proved in those works. As warmup demonstrations of this new Master Theorem, we give new proofs of the semicircle and MarchenkoPastur laws, which benchmarks our framework against these fundamental mathematical results.
Crosslists for Wed, 23 Sep 20
 [5] arXiv:2009.10214 (crosslist from cs.LG) [pdf, other]

Title: DISPATCH: Design Space Exploration of CyberPhysical SystemsComments: 14 pagesSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Systems and Control (eess.SY)
Design of Cyberphysical systems (CPSs) is a challenging task that involves searching over a large search space of various CPS configurations and possible values of components composing the system. Hence, there is a need for sampleefficient CPS design space exploration to select the system architecture and component values that meet the target system requirements. We address this challenge by formulating CPS design as a multiobjective optimization problem and propose DISPATCH, a twostep methodology for sampleefficient search over the design space. First, we use a genetic algorithm to search over discrete choices of system component values for architecture search and component selection or only component selection and terminate the algorithm even before meeting the system requirements, thus yielding a coarse design. In the second step, we use an inverse design to search over a continuous space to finetune the component values and meet the diverse set of system requirements. We use a neural network as a surrogate function for the inverse design of the system. The neural network, converted into a mixedinteger linear program, is used for active learning to sample component values efficiently in a continuous search space. We illustrate the efficacy of DISPATCH on electrical circuit benchmarks: twostage and threestage transimpedence amplifiers. Simulation results show that the proposed methodology improves sample efficiency by 514x compared to a prior synthesis method that relies on reinforcement learning. It also synthesizes circuits with the best performance (highest bandwidth/lowest area) compared to designs synthesized using reinforcement learning, Bayesian optimization, or humans.
Replacements for Wed, 23 Sep 20
 [6] arXiv:1903.10410 (replaced) [pdf, ps, other]

Title: A Conceptual BioInspired Framework for the Evolution of Artificial General IntelligenceComments: 7 pages, 2 figures, accepted to "The 3rd Special Session on Biologically Inspired Parallel and Distributed Computing, Algorithms and Solutions" (BICAS 2020)Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
 [7] arXiv:1605.05359 (replaced) [pdf, other]

Title: Option Discovery in Hierarchical Reinforcement Learning using SpatioTemporal ClusteringComments: Revised version of ICML 16 Abstraction in Reinforcement Learning workshop paperSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)
 [8] arXiv:1605.05365 (replaced) [pdf, other]

Title: Dynamic Frame skip Deep Q NetworkComments: IJCAI 2016 Workshop on Deep Reinforcement Learning: Frontiers and Challenges; 6 pages, 8 figuresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
 [9] arXiv:1702.06054 (replaced) [pdf, other]

Title: Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement LearningComments: 24 pagesSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
 [10] arXiv:1912.08881 (replaced) [pdf, other]

Title: Pruning by Explaining: A Novel Criterion for Deep Neural Network PruningAuthors: SeulKi Yeom, Philipp Seegerer, Sebastian Lapuschkin, Alexander Binder, Simon Wiedemann, KlausRobert Müller, Wojciech SamekComments: 29 pages, 11 figures, 6 tablesSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
 [11] arXiv:2008.04245 (replaced) [pdf, other]

Title: TinySpeech: Attention Condensers for Deep Speech Recognition Neural Networks on Edge DevicesComments: 9 pagesSubjects: Audio and Speech Processing (eess.AS); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
 [12] arXiv:2009.08841 (replaced) [pdf, other]

Title: On the spatiotemporal behavior in biologymimicking computing systemsComments: 33 pages, 6 figuresSubjects: Emerging Technologies (cs.ET); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2009, contact, help (Access key information)