Neural and Evolutionary Computing
New submissions
[ showing up to 500 entries per page: fewer | more ]
New submissions for Thu, 18 Apr 24
- [1] arXiv:2404.10861 [pdf, other]
-
Title: Trackable Agent-based Evolution Models at Wafer ScaleSubjects: Neural and Evolutionary Computing (cs.NE); Distributed, Parallel, and Cluster Computing (cs.DC)
Continuing improvements in computing hardware are poised to transform capabilities for in silico modeling of cross-scale phenomena underlying major open questions in evolutionary biology and artificial life, such as transitions in individuality, eco-evolutionary dynamics, and rare evolutionary events. Emerging ML/AI-oriented hardware accelerators, like the 850,000 processor Cerebras Wafer Scale Engine (WSE), hold particular promise. However, practical challenges remain in conducting informative evolution experiments that efficiently utilize these platforms' large processor counts. Here, we focus on the problem of extracting phylogenetic information from agent-based evolution on the WSE platform. This goal drove significant refinements to decentralized in silico phylogenetic tracking, reported here. These improvements yield order-of-magnitude performance improvements. We also present an asynchronous island-based genetic algorithm (GA) framework for WSE hardware. Emulated and on-hardware GA benchmarks with a simple tracking-enabled agent model clock upwards of 1 million generations a minute for population sizes reaching 16 million agents. We validate phylogenetic reconstructions from these trials and demonstrate their suitability for inference of underlying evolutionary conditions. In particular, we demonstrate extraction, from wafer-scale simulation, of clear phylometric signals that differentiate runs with adaptive dynamics enabled versus disabled. Together, these benchmark and validation trials reflect strong potential for highly scalable agent-based evolution simulation that is both efficient and observable. Developed capabilities will bring entirely new classes of previously intractable research questions within reach, benefiting further explorations within the evolutionary biology and artificial life communities across a variety of emerging high-performance computing platforms.
- [2] arXiv:2404.11239 [pdf, ps, other]
-
Title: Runtime Analysis of a Multi-Valued Compact Genetic Algorithm on Generalized OneMaxSubjects: Neural and Evolutionary Computing (cs.NE)
A class of metaheuristic techniques called estimation-of-distribution algorithms (EDAs) are employed in optimization as more sophisticated substitutes for traditional strategies like evolutionary algorithms. EDAs generally drive the search for the optimum by creating explicit probabilistic models of potential candidate solutions through repeated sampling and selection from the underlying search space.
Most theoretical research on EDAs has focused on pseudo-Boolean optimization. Jedidia et al. (GECCO 2023) proposed the first EDAs for optimizing problems involving multi-valued decision variables. By building a framework, they have analyzed the runtime of a multi-valued UMDA on the r-valued LeadingOnes function. Using their framework, here we focus on the multi-valued compact genetic algorithm (r-cGA) and provide a first runtime analysis of a generalized OneMax function.
To prove our results, we investigate the effect of genetic drift and progress of the probabilistic model towards the optimum. After finding the right algorithm parameters, we prove that the r-cGA solves this r-valued OneMax problem efficiently. We show that with high probability, the runtime bound is O(r2 n log2 r log3 n). At the end of experiments, we state one conjecture related to the expected runtime of another variant of multi-valued OneMax function. - [3] arXiv:2404.11433 [pdf, other]
-
Title: Runtime Analyses of NSGA-III on Many-Objective ProblemsComments: To appear at GECCO 2024Subjects: Neural and Evolutionary Computing (cs.NE)
NSGA-II and NSGA-III are two of the most popular evolutionary multi-objective algorithms used in practice. While NSGA-II is used for few objectives such as 2 and 3, NSGA-III is designed to deal with a larger number of objectives. In a recent breakthrough, Wietheger and Doerr (IJCAI 2023) gave the first runtime analysis for NSGA-III on the 3-objective OneMinMax problem, showing that this state-of-the-art algorithm can be analyzed rigorously. We advance this new line of research by presenting the first runtime analyses of NSGA-III on the popular many-objective benchmark problems mLOTZ, mOMM, and mCOCZ, for an arbitrary constant number $m$ of objectives. Our analysis provides ways to set the important parameters of the algorithm: the number of reference points and the population size, so that a good performance can be guaranteed. We show how these parameters should be scaled with the problem dimension, the number of objectives and the fitness range. To our knowledge, these are the first runtime analyses for NSGA-III for more than 3 objectives.
- [4] arXiv:2404.11496 [pdf, ps, other]
-
Title: Runtime Analysis of Evolutionary Diversity Optimization on the Multi-objective (LeadingOnes, TrailingZeros) ProblemSubjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
The diversity optimization is the class of optimization problems, in which we aim at finding a diverse set of good solutions. One of the frequently used approaches to solve such problems is to use evolutionary algorithms which evolve a desired diverse population. This approach is called evolutionary diversity optimization (EDO).
In this paper, we analyse EDO on a 3-objective function LOTZ$_k$, which is a modification of the 2-objective benchmark function (LeadingOnes, TrailingZeros). We prove that the GSEMO computes a set of all Pareto-optimal solutions in $O(kn^3)$ expected iterations. We also analyze the runtime of the GSEMO$_D$ (a modification of the GSEMO for diversity optimization) until it finds a population with the best possible diversity for two different diversity measures, the total imbalance and the sorted imbalances vector. For the first measure we show that the GSEMO$_D$ optimizes it asymptotically faster than it finds a Pareto-optimal population, in $O(kn^2\log(n))$ expected iterations, and for the second measure we show an upper bound of $O(k^2n^3\log(n))$ expected iterations. We complement our theoretical analysis with an empirical study, which shows a very similar behavior for both diversity measures that is close to the theory predictions.
Cross-lists for Thu, 18 Apr 24
- [5] arXiv:2404.10824 (cross-list from cs.LG) [pdf, other]
-
Title: Decoupled Weight Decay for Any $p$ NormSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Optimization and Control (math.OC)
With the success of deep neural networks (NNs) in a variety of domains, the computational and storage requirements for training and deploying large NNs have become a bottleneck for further improvements. Sparsification has consequently emerged as a leading approach to tackle these issues. In this work, we consider a simple yet effective approach to sparsification, based on the Bridge, or $L_p$ regularization during training. We introduce a novel weight decay scheme, which generalizes the standard $L_2$ weight decay to any $p$ norm. We show that this scheme is compatible with adaptive optimizers, and avoids the gradient divergence associated with $0<p<1$ norms. We empirically demonstrate that it leads to highly sparse networks, while maintaining generalization performance comparable to standard $L_2$ regularization.
- [6] arXiv:2404.10854 (cross-list from q-bio.PE) [pdf, other]
-
Title: Methods to Estimate Cryptic Sequence ComplexityAuthors: Matthew Andres MorenoSubjects: Populations and Evolution (q-bio.PE); Neural and Evolutionary Computing (cs.NE)
Complexity is a signature quality of interest in artificial life systems. Alongside other dimensions of assessment, it is common to quantify genome sites that contribute to fitness as a complexity measure. However, limitations to the sensitivity of fitness assays in models with implicit replication criteria involving rich biotic interactions introduce the possibility of difficult-to-detect ``cryptic'' adaptive sites, which contribute small fitness effects below the threshold of individual detectability or involve epistatic redundancies. Here, we propose three knockout-based assay procedures designed to quantify cryptic adaptive sites within digital genomes. We report initial tests of these methods on a simple genome model with explicitly configured site fitness effects. In these limited tests, estimation results reflect ground truth cryptic sequence complexities well. Presented work provides initial steps toward development of new methods and software tools that improve the resolution, rigor, and tractability of complexity analyses across alife systems, particularly those requiring expensive in situ assessments of organism fitness.
- [7] arXiv:2404.11229 (cross-list from physics.comp-ph) [pdf, other]
-
Title: Mean field initialization of the Annealed Importance Sampling algorithm for an efficient evaluation of the Partition Function of Restricted Boltzmann MachinesComments: arXiv admin note: text overlap with arXiv:2007.11926Subjects: Computational Physics (physics.comp-ph); Neural and Evolutionary Computing (cs.NE)
Probabilistic models in physics often require from the evaluation of normalized Boltzmann factors, which in turn implies the computation of the partition function Z. Getting the exact value of Z, though, becomes a forbiddingly expensive task as the system size increases. This problem is also present in probabilistic learning models such as the Restricted Boltzmann Machine (RBM), where the situation is even worse as the exact learning rules implies the computation of Z at each iteration. A possible way to tackle this problem is to use the Annealed Importance Sampling (AIS) algorithm, which provides a tool to stochastically estimate the partition function of the system. So far, the standard application of the AIS algorithm starts from the uniform probability distribution and uses a large number of Monte Carlo steps to obtain reliable estimations of Z following an annealing process. In this work we show that both the quality of the estimation and the cost of the computation can be significantly improved by using a properly selected mean-field starting probability distribution. We perform a systematic analysis of AIS in both small- and large-sized problems, and compare the results to exact values in problems where these are known. As a result of our systematic analysis, we propose two successful strategies that work well in all the problems analyzed. We conclude that these are good starting points to estimate the partition function with AIS with a relatively low computational cost.
Replacements for Thu, 18 Apr 24
- [8] arXiv:2404.04587 (replaced) [pdf, other]
-
Title: Neuroevolving Electronic Dynamical NetworksAuthors: Derek WhitleyComments: 8 pages, 3 figuresSubjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)
- [9] arXiv:2404.08161 (replaced) [pdf, ps, other]
-
Title: R2 Indicator and Deep Reinforcement Learning Enhanced Adaptive Multi-Objective Evolutionary AlgorithmSubjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
- [10] arXiv:2106.13082 (replaced) [pdf, ps, other]
-
Title: On the relationship between predictive coding and backpropagationAuthors: Robert RosenbaumSubjects: Neurons and Cognition (q-bio.NC); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
- [11] arXiv:2307.03798 (replaced) [pdf, other]
-
Title: Fooling Contrastive Language-Image Pre-trained Models with CLIPMasterPrintsComments: This work was supported by a research grant (40575) from VILLUM FONDENSubjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
- [12] arXiv:2312.00425 (replaced) [pdf, other]
-
Title: Retina : Low-Power Eye Tracking with Event Camera and Spiking HardwareSubjects: Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)
- [13] arXiv:2403.09863 (replaced) [pdf, other]
-
Title: Towards White Box Deep LearningAuthors: Maciej SatkiewiczComments: 16 pages, 12 figures, independent research, v5 changes: Expanded Abstract and Related Work section; minor wording improvementsSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
[ showing up to 500 entries per page: fewer | more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2404, contact, help (Access key information)