Neural and Evolutionary Computing
New submissions
[ showing up to 2000 entries per page: fewer  more ]
New submissions for Wed, 25 Nov 20
 [1] arXiv:2011.11705 [pdf, other]

Title: DeepClimGAN: A HighResolution Climate Data GeneratorComments: Presented at NeurIPS 2019 Workshop Tackling Climate Change with Machine LearningSubjects: Neural and Evolutionary Computing (cs.NE)
Earth system models (ESMs), which simulate the physics and chemistry of the global atmosphere, land, and ocean, are often used to generate future projections of climate change scenarios. These models are far too computationally intensive to run repeatedly, but limited sets of runs are insufficient for some important applications, like adequately sampling distribution tails to characterize extreme events. As a compromise, emulators are substantially less expensive but may not have all of the complexity of an ESM. Here we demonstrate the use of a conditional generative adversarial network (GAN) to act as an ESM emulator. In doing so, we gain the ability to produce daily weather data that is consistent with what ESM might output over any chosen scenario. In particular, the GAN is aimed at representing a joint probability distribution over space, time, and climate variables, enabling the study of correlated extreme events, such as floods, droughts, or heatwaves.
 [2] arXiv:2011.12012 [pdf, other]

Title: A More Biologically Plausible Local Learning Rule for ANNsAuthors: Shashi Kant GuptaComments: 8 pages (4 main + 1 reference + 3 supplementary)Subjects: Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (qbio.NC)
The backpropagation algorithm is often debated for its biological plausibility. However, various learning methods for neural architecture have been proposed in search of more biologically plausible learning. Most of them have tried to solve the "weight transport problem" and try to propagate errors backward in the architecture via some alternative methods. In this work, we investigated a slightly different approach that uses only the local information which captures spike timing information with no propagation of errors. The proposed learning rule is derived from the concepts of spike timing dependant plasticity and neuronal association. A preliminary evaluation done on the binary classification of MNIST and IRIS datasets with two hidden layers shows comparable performance with backpropagation. The model learned using this method also shows a possibility of better adversarial robustness against the FGSM attack compared to the model learned through backpropagation of crossentropy loss. The local nature of learning gives a possibility of large scale distributed and parallel learning in the network. And finally, the proposed method is a more biologically sound method that can probably help in understanding how biological neurons learn different abstractions.
Crosslists for Wed, 25 Nov 20
 [3] arXiv:2011.11710 (crosslist from qbio.NC) [pdf, other]

Title: Naturalgradient learning for spiking neuronsComments: Joint senior authorship: Walter M. Senn and Mihai A. PetroviciSubjects: Neurons and Cognition (qbio.NC); Neural and Evolutionary Computing (cs.NE); Differential Geometry (math.DG); Computation (stat.CO)
In many normative theories of synaptic plasticity, weight updates implicitly depend on the chosen parametrization of the weights. This problem relates, for example, to neuronal morphology: synapses which are functionally equivalent in terms of their impact on somatic firing can differ substantially in spine size due to their different positions along the dendritic tree. Classical theories based on Euclidean gradient descent can easily lead to inconsistencies due to such parametrization dependence. The issues are solved in the framework of Riemannian geometry, in which we propose that plasticity instead follows natural gradient descent. Under this hypothesis, we derive a synaptic learning rule for spiking neurons that couples functional efficiency with the explanation of several welldocumented biological phenomena such as dendritic democracy, multiplicative scaling and heterosynaptic plasticity. We therefore suggest that in its search for functional synaptic plasticity, evolution might have come up with its own version of natural gradient descent.
 [4] arXiv:2011.11715 (crosslist from cs.CL) [pdf, other]

Title: Multitask Language Modeling for Improving Speech Recognition of Rare WordsAuthors: ChaoHan Huck Yang, Linda Liu, Ankur Gandhe, Yile Gu, Anirudh Raju, Denis Filimonov, Ivan BulykoComments: Submitted to ICASSP 2021Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Endtoend automatic speech recognition (ASR) systems are increasingly popular due to their relative architectural simplicity and competitive performance. However, even though the average accuracy of these systems may be high, the performance on rare content words often lags behind hybrid ASR systems. To address this problem, secondpass rescoring is often applied. In this paper, we propose a secondpass system with multitask learning, utilizing semantic targets (such as intent and slot prediction) to improve speech recognition performance. We show that our rescoring model with trained with these additional tasks outperforms the baseline rescoring model, trained with only the language modeling task, by 1.4% on a general test and by 2.6% on a rare word test set in term of worderrorrate relative (WERR).
 [5] arXiv:2011.11944 (crosslist from cs.LG) [pdf, other]

Title: Hyper parameter estimation method with particle swarm optimizationSubjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Particle swarm optimization (PSO) method cannot be directly used in the problem of hyperparameter estimation since the mathematical formulation of the mapping from hyperparameters to loss function or generalization accuracy is unclear. Bayesian optimization (BO) framework is capable of converting the optimization of the hyperparameters into the optimization of an acquisition function. The acquisition function is nonconvex and multipeak. So the problem can be better solved by the PSO. The proposed method in this paper uses the particle swarm method to optimize the acquisition function in the BO framework to get better hyperparameters. The performances of proposed method in both of the classification and regression models are evaluated and demonstrated. The results on several benchmark problems are improved.
 [6] arXiv:2011.12043 (crosslist from cs.LG) [pdf, other]

Title: Efficient Sampling for PredictorBased Neural Architecture SearchAuthors: Lukas Mauch, Stephen Tiedemann, Javier Alonso Garcia, Bac Nguyen Cong, Kazuki Yoshiyama, Fabien Cardinaux, Thomas KempSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
Recently, predictorbased algorithms emerged as a promising approach for neural architecture search (NAS). For NAS, we typically have to calculate the validation accuracy of a large number of Deep Neural Networks (DNNs), what is computationally complex. Predictorbased NAS algorithms address this problem. They train a proxy model that can infer the validation accuracy of DNNs directly from their network structure. During optimization, the proxy can be used to narrow down the number of architectures for which the true validation accuracy must be computed, what makes predictorbased algorithms sample efficient. Usually, we compute the proxy for all DNNs in the network search space and pick those that maximize the proxy as candidates for optimization. However, that is intractable in practice, because the search spaces are often very large and contain billions of network architectures. The contributions of this paper are threefold: 1) We define a sample efficiency gain to compare different predictorbased NAS algorithms. 2) We conduct experiments on the NASBench101 dataset and show that the sample efficiency of predictorbased algorithms decreases dramatically if the proxy is only computed for a subset of the search space. 3) We show that if we choose the subset of the search space on which the proxy is evaluated in a smart way, the sample efficiency of the original predictorbased algorithm that has access to the full search space can be regained. This is an important step to make predictorbased NAS algorithms useful, in practice.
Replacements for Wed, 25 Nov 20
 [7] arXiv:2007.05785 (replaced) [pdf, other]

Title: Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural NetworksSubjects: Neural and Evolutionary Computing (cs.NE); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
 [8] arXiv:1910.02600 (replaced) [pdf, other]

Title: Deep Evidential RegressionComments: Code available on: this https URLJournalref: Advances in Neural Information Processing Systems (NeurIPS) 2020Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
 [9] arXiv:2010.01729 (replaced) [pdf, other]

Title: Revisiting Batch Normalization for Training Lowlatency Deep Spiking Neural Networks from ScratchSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
[ showing up to 2000 entries per page: fewer  more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2011, contact, help (Access key information)