Neural and Evolutionary Computing

New submissions

• New submissions
• Cross-lists
• Replacements

New submissions for Mon, 29 May 23

[1]  arXiv:2305.16594 [pdf, other]

Title: A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks

Authors: Xinyi Chen, Qu Yang, Jibin Wu, Haizhou Li, Kay Chen Tan

Subjects: Neural and Evolutionary Computing (cs.NE)

The biological neural systems evolved to adapt to ecological environment for efficiency and effectiveness, wherein neurons with heterogeneous structures and rich dynamics are optimized to accomplish complex cognitive tasks. Most of the current research of biologically inspired spiking neural networks (SNNs) are, however, grounded on a homogeneous neural coding scheme, which limits their overall performance in terms of accuracy, latency, efficiency, and robustness, etc. In this work, we argue that one should holistically design the network architecture to incorporate diverse neuronal functions and neural coding schemes for best performance. As an early attempt in this research direction, we put forward a hybrid neural coding framework that integrates multiple neural coding schemes discovered in neuroscience. We demonstrate that the proposed hybrid coding scheme achieves a comparable accuracy with the state-of-the-art SNNs with homogeneous neural coding on CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets with less than eight time steps and at least 3.90x fewer computations. Furthermore, we demonstrate accurate, rapid, and robust sound source localization on SoClas dataset. This study yields valuable insights into the performance of various hybrid neural coding designs and hold significant implications for designing high performance SNNs.

[2]  arXiv:2305.16870 [pdf, other]

Title: Non-Elitist Evolutionary Multi-Objective Optimisation: Proof-of-Principle Results

Authors: Zimin Liang, Miqing Li, Per Kristian Lehre

Subjects: Neural and Evolutionary Computing (cs.NE)

Elitism, which constructs the new population by preserving best solutions out of the old population and newly-generated solutions, has been a default way for population update since its introduction into multi-objective evolutionary algorithms (MOEAs) in the late 1990s. In this paper, we take an opposite perspective to conduct the population update in MOEAs by simply discarding elitism. That is, we treat the newly-generated solutions as the new population directly (so that all selection pressure comes from mating selection). We propose a simple non-elitist MOEA (called NE-MOEA) that only uses Pareto dominance sorting to compare solutions, without involving any diversity-related selection criterion. Preliminary experimental results show that NE-MOEA can compete with well-known elitist MOEAs (NSGA-II, SMS-EMOA and NSGA-III) on several combinatorial problems. Lastly, we discuss limitations of the proposed non-elitist algorithm and suggest possible future research directions.

[3]  arXiv:2305.16873 [pdf, other]

Title: Efficient Decoding of Compositional Structure in Holistic Representations

Authors: Denis Kleyko, Connor Bybee, Ping-Chen Huang, Christopher J. Kymn, Bruno A. Olshausen, E. Paxon Frady, Friedrich T. Sommer

Comments: 28 pages, 5 figures

Journal-ref: Neural Computation, 2023

Subjects: Neural and Evolutionary Computing (cs.NE); Information Retrieval (cs.IR); Information Theory (cs.IT)

We investigate the task of retrieving information from compositional distributed representations formed by Hyperdimensional Computing/Vector Symbolic Architectures and present novel techniques which achieve new information rate bounds. First, we provide an overview of the decoding techniques that can be used to approach the retrieval task. The techniques are categorized into four groups. We then evaluate the considered techniques in several settings that involve, e.g., inclusion of external noise and storage elements with reduced precision. In particular, we find that the decoding techniques from the sparse coding and compressed sensing literature (rarely used for Hyperdimensional Computing/Vector Symbolic Architectures) are also well-suited for decoding information from the compositional distributed representations. Combining these decoding techniques with interference cancellation ideas from communications improves previously reported bounds (Hersche et al., 2021) of the information rate of the distributed representations from 1.20 to 1.40 bits per dimension for smaller codebooks and from 0.60 to 1.26 bits per dimension for larger codebooks.

[4]  arXiv:2305.16956 [pdf, other]

Title: Local Search, Semantics, and Genetic Programming: a Global Analysis

Authors: Fabio Anselmi, Mauro Castelli, Alberto d'Onofrio, Luca Manzoni, Luca Mariot, Martina Saletta

Comments: 22 pages, 4 figures, 1 table

Subjects: Neural and Evolutionary Computing (cs.NE)

Geometric Semantic Geometric Programming (GSGP) is one of the most prominent Genetic Programming (GP) variants, thanks to its solid theoretical background, the excellent performance achieved, and the execution time significantly smaller than standard syntax-based GP. In recent years, a new mutation operator, Geometric Semantic Mutation with Local Search (GSM-LS), has been proposed to include a local search step in the mutation process based on the idea that performing a linear regression during the mutation can allow for a faster convergence to good-quality solutions. While GSM-LS helps the convergence of the evolutionary search, it is prone to overfitting. Thus, it was suggested to use GSM-LS only for a limited number of generations and, subsequently, to switch back to standard geometric semantic mutation. A more recently defined variant of GSGP (called GSGP-reg) also includes a local search step but shares similar strengths and weaknesses with GSM-LS. Here we explore multiple possibilities to limit the overfitting of GSM-LS and GSGP-reg, ranging from adaptive methods to estimate the risk of overfitting at each mutation to a simple regularized regression. The results show that the method used to limit overfitting is not that important: providing that a technique to control overfitting is used, it is possible to consistently outperform standard GSGP on both training and unseen data. The obtained results allow practitioners to better understand the role of local search in GSGP and demonstrate that simple regularization strategies are effective in controlling overfitting.

Cross-lists for Mon, 29 May 23

[5]  arXiv:2305.16347 (cross-list from cs.LG) [pdf, other]

Title: Prompt Evolution for Generative AI: A Classifier-Guided Approach

Authors: Melvin Wong, Yew-Soon Ong, Abhishek Gupta, Kavitesh K. Bali, Caishun Chen

Comments: To appear in Proceedings of the 2023 IEEE Conference on Artificial Intelligence (CAI'23)

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)

Synthesis of digital artifacts conditioned on user prompts has become an important paradigm facilitating an explosion of use cases with generative AI. However, such models often fail to connect the generated outputs and desired target concepts/preferences implied by the prompts. Current research addressing this limitation has largely focused on enhancing the prompts before output generation or improving the model's performance up front. In contrast, this paper conceptualizes prompt evolution, imparting evolutionary selection pressure and variation during the generative process to produce multiple outputs that satisfy the target concepts/preferences better. We propose a multi-objective instantiation of this broader idea that uses a multi-label image classifier-guided approach. The predicted labels from the classifiers serve as multiple objectives to optimize, with the aim of producing diversified images that meet user preferences. A novelty of our evolutionary algorithm is that the pre-trained generative model gives us implicit mutation operations, leveraging the model's stochastic generative capability to automate the creation of Pareto-optimized images more faithful to user preferences.

[6]  arXiv:2305.16497 (cross-list from cs.LG) [pdf, other]

Title: AD-NEV: A Scalable Multi-level Neuroevolution Framework for Multivariate Anomaly Detection

Authors: Marcin Pietron, Dominik Zurek, Kamil Faber, Roberto Corizzo

Comments: submitted to IEEE TNNLS

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

Anomaly detection tools and methods present a key capability in modern cyberphysical and failure prediction systems. Despite the fast-paced development in deep learning architectures for anomaly detection, model optimization for a given dataset is a cumbersome and time consuming process. Neuroevolution could be an effective and efficient solution to this problem, as a fully automated search method for learning optimal neural networks, supporting both gradient and non-gradient fine tuning. However, existing methods mostly focus on optimizing model architectures without taking into account feature subspaces and model weights. In this work, we propose Anomaly Detection Neuroevolution (AD-NEv) - a scalable multi-level optimized neuroevolution framework for multivariate time series anomaly detection. The method represents a novel approach to synergically: i) optimize feature subspaces for an ensemble model based on the bagging technique; ii) optimize the model architecture of single anomaly detection models; iii) perform non-gradient fine-tuning of network weights. An extensive experimental evaluation on widely adopted multivariate anomaly detection benchmark datasets shows that the models extracted by AD-NEv outperform well-known deep learning architectures for anomaly detection. Moreover, results show that AD-NEv can perform the whole process efficiently, presenting high scalability when multiple GPUs are available.

[7]  arXiv:2305.16625 (cross-list from cs.LG) [pdf, other]

Title: Set-based Neural Network Encoding

Authors: Bruno Andreis, Soro Bedionita, Sung Ju Hwang

Comments: 13 pages

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

We propose an approach to neural network weight encoding for generalization performance prediction that utilizes set-to-set and set-to-vector functions to efficiently encode neural network parameters. Our approach is capable of encoding neural networks in a modelzoo of mixed architecture and different parameter sizes as opposed to previous approaches that require custom encoding models for different architectures. Furthermore, our \textbf{S}et-based \textbf{N}eural network \textbf{E}ncoder (SNE) takes into consideration the hierarchical computational structure of neural networks by utilizing a layer-wise encoding scheme that culminates to encoding all layer-wise encodings to obtain the neural network encoding vector. Additionally, we introduce a \textit{pad-chunk-encode} pipeline to efficiently encode neural network layers that is adjustable to computational and memory constraints. We also introduce two new tasks for neural network generalization performance prediction: cross-dataset and cross-architecture. In cross-dataset performance prediction, we evaluate how well performance predictors generalize across modelzoos trained on different datasets but of the same architecture. In cross-architecture performance prediction, we evaluate how well generalization performance predictors transfer to modelzoos of different architecture. Experimentally, we show that SNE outperforms the relevant baselines on the cross-dataset task and provide the first set of results on the cross-architecture task.

[8]  arXiv:2305.16748 (cross-list from cs.RO) [pdf, other]

Title: A Decentralized Spike-based Learning Framework for Sequential Capture in Discrete Perimeter Defense Problem

Authors: Mohammed Thousif, Shridhar Velhal, Suresh Sundaram, Shirin Dora

Comments: Mohammed Thousif and Shridhar Velhal has equal contribution to this manuscript

Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multiagent Systems (cs.MA); Neural and Evolutionary Computing (cs.NE); Systems and Control (eess.SY)

This paper proposes a novel Decentralized Spike-based Learning (DSL) framework for the discrete Perimeter Defense Problem (d-PDP). A team of defenders is operating on the perimeter to protect the circular territory from radially incoming intruders. At first, the d-PDP is formulated as a spatio-temporal multi-task assignment problem (STMTA). The problem of STMTA is then converted into a multi-label learning problem to obtain labels of segments that defenders have to visit in order to protect the perimeter. The DSL framework uses a Multi-Label Classifier using Synaptic Efficacy Function spiking neuRON (MLC-SEFRON) network for deterministic multi-label learning. Each defender contains a single MLC-SEFRON network. Each MLC-SEFRON network is trained independently using input from its own perspective for decentralized operations. The input spikes to the MLC-SEFRON network can be directly obtained from the spatio-temporal information of defenders and intruders without any extra pre-processing step. The output of MLC-SEFRON contains the labels of segments that a defender has to visit in order to protect the perimeter. Based on the multi-label output from the MLC-SEFRON a trajectory is generated for a defender using a Consensus-Based Bundle Algorithm (CBBA) in order to capture the intruders. The target multi-label output for training MLC-SEFRON is obtained from an expert policy. Also, the MLC-SEFRON trained for a defender can be directly used for obtaining labels of segments assigned to another defender without any retraining. The performance of MLC-SEFRON has been evaluated for full observation and partial observation scenarios of the defender. The overall performance of the DSL framework is then compared with expert policy along with other existing learning algorithms. The scalability of the DSL has been evaluated using an increasing number of defenders.

Replacements for Mon, 29 May 23

[9]  arXiv:2211.09498 (replaced) [pdf, other]

Title: Automatic Construction of Parallel Algorithm Portfolios for Multi-objective Optimization

Authors: Xiasheng Ma, Shengcai Liu, Wenjing Hong

Subjects: Neural and Evolutionary Computing (cs.NE)

[10]  arXiv:2204.10438 (replaced) [pdf, other]

Title: EVOTER: Evolution of Transparent Explainable Rule-sets

Authors: Hormoz Shahrzad, Babak Hodjat, Risto Miikkulainen

Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)

[11]  arXiv:2211.16550 (replaced) [pdf, other]

Title: Soft Alignment Objectives for Robust Adaptation of Language Generation

Authors: Michal Štefánik, Marek Kadlčík, Petr Sojka

Comments: Annual Meeting of The ACL 2023: Main conference long paper

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

[12]  arXiv:2211.17179 (replaced) [pdf, ps, other]

Title: Investigation of Proper Orthogonal Decomposition for Echo State Networks

Authors: Jean Panaioti Jordanou, Eric Aislan Antonelo, Eduardo Camponogara, Eduardo Gildin

Comments: Accepted in Neurocomputing, not yet published

Subjects: Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Systems and Control (eess.SY)

[13]  arXiv:2212.07431 (replaced) [pdf, other]

Title: Simulator-Based Self-Supervision for Learned 3D Tomography Reconstruction

Authors: Onni Kosomaa, Samuli Laine, Tero Karras, Miika Aittala, Jaakko Lehtinen

Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)

• New submissions
• Cross-lists
• Replacements