We gratefully acknowledge support from
the Simons Foundation and member institutions.

Artificial Intelligence

New submissions

[ total of 195 entries: 1-195 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Wed, 19 Jan 22

[1]  arXiv:2201.05651 [pdf, other]
Title: CLUE: Contextualised Unified Explainable Learning of User Engagement in Video Lectures
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Predicting contextualised engagement in videos is a long-standing problem that has been popularly attempted by exploiting the number of views or the associated likes using different computational methods. The recent decade has seen a boom in online learning resources, and during the pandemic, there has been an exponential rise of online teaching videos without much quality control. The quality of the content could be improved if the creators could get constructive feedback on their content. Employing an army of domain expert volunteers to provide feedback on the videos might not scale. As a result, there has been a steep rise in developing computational methods to predict a user engagement score that is indicative of some form of possible user engagement, i.e., to what level a user would tend to engage with the content. A drawback in current methods is that they model various features separately, in a cascaded approach, that is prone to error propagation. Besides, most of them do not provide crucial explanations on how the creator could improve their content. In this paper, we have proposed a new unified model, CLUE for the educational domain, which learns from the features extracted from freely available public online teaching videos and provides explainable feedback on the video along with a user engagement score. Given the complexity of the task, our unified framework employs different pre-trained models working together as an ensemble of classifiers. Our model exploits various multi-modal features to model the complexity of language, context agnostic information, textual emotion of the delivered content, animation, speaker's pitch and speech emotions. Under a transfer learning setup, the overall model, in the unified space, is fine-tuned for downstream applications.

[2]  arXiv:2201.05658 [pdf, other]
Title: Sequence-to-Sequence Models for Extracting Information from Registration and Legal Documents
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

A typical information extraction pipeline consists of token- or span-level classification models coupled with a series of pre- and post-processing scripts. In a production pipeline, requirements often change, with classes being added and removed, which leads to nontrivial modifications to the source code and the possible introduction of bugs. In this work, we evaluate sequence-to-sequence models as an alternative to token-level classification methods for information extraction of legal and registration documents. We finetune models that jointly extract the information and generate the output already in a structured format. Post-processing steps are learned during training, thus eliminating the need for rule-based methods and simplifying the pipeline. Furthermore, we propose a novel method to align the output with the input text, thus facilitating system inspection and auditing. Our experiments on four real-world datasets show that the proposed method is an alternative to classical pipelines.

[3]  arXiv:2201.05673 [pdf, other]
Title: Adaptive Information Belief Space Planning
Subjects: Artificial Intelligence (cs.AI); Robotics (cs.RO)

Reasoning about uncertainty is vital in many real-life autonomous systems. However, current state-of-the-art planning algorithms cannot either reason about uncertainty explicitly, or do so with a high computational burden. Here, we focus on making informed decisions efficiently, using reward functions that explicitly deal with uncertainty. We formulate an approximation, namely an abstract observation model, that uses an aggregation scheme to alleviate computational costs. We derive bounds on the expected information-theoretic reward function and, as a consequence, on the value function. We then propose a method to refine aggregation to achieve identical action selection with a fraction of the computational time.

[4]  arXiv:2201.05710 [pdf, other]
Title: Specifying and Reasoning about CPS through the Lens of the NIST CPS Framework
Comments: Under consideration in Theory and Practice of Logic Programming (TPLP)
Subjects: Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)

This paper introduces a formal definition of a Cyber-Physical System (CPS) in the spirit of the CPS Framework proposed by the National Institute of Standards and Technology (NIST). It shows that using this definition, various problems related to concerns in a CPS can be precisely formalized and implemented using Answer Set Programming (ASP). These include problems related to the dependency or conflicts between concerns, how to mitigate an issue, and what the most suitable mitigation strategy for a given issue would be. It then shows how ASP can be used to develop an implementation that addresses the aforementioned problems. The paper concludes with a discussion of the potentials of the proposed methodologies.

[5]  arXiv:2201.05818 [pdf]
Title: Deciding Not To Decide
Comments: 22 pages, 15 figures
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Sometimes unexpected, novel, unconceivable events enter our lives. The cause-effect mappings that usually guide our behaviour are destroyed. Surprised and shocked by possibilities that we had never imagined, we are unable to make any decision beyond mere routine. Among them there are decisions, such as making investments, that are essential for the long-term survival of businesses as well as the economy at large. We submit that the standard machinery of utility maximization does not apply, but we propose measures inspired by scenario planning and graph analysis, pointing to solutions being explored in machine learning.

[6]  arXiv:2201.05910 [pdf]
Title: An Automatic Ontology Generation Framework with An Organizational Perspective
Comments: Proceedings of the 53rd Hawaii International Conference on System Sciences | 2020
Subjects: Artificial Intelligence (cs.AI)

Ontologies have been known for their semantic representation of knowledge. ontologies cannot automatically evolve to reflect updates that occur in respective domains. To address this limitation, researchers have called for automatic ontology generation from unstructured text corpus. Unfortunately, systems that aim to generate ontologies from unstructured text corpus are domain-specific and require manual intervention. In addition, they suffer from uncertainty in creating concept linkages and difficulty in finding axioms for the same concept. Knowledge Graphs (KGs) has emerged as a powerful model for the dynamic representation of knowledge. However, KGs have many quality limitations and need extensive refinement. This research aims to develop a novel domain-independent automatic ontology generation framework that converts unstructured text corpus into domain consistent ontological form. The framework generates KGs from unstructured text corpus as well as refine and correct them to be consistent with domain ontologies. The power of the proposed automatically generated ontology is that it integrates the dynamic features of KGs and the quality features of ontologies.

[7]  arXiv:2201.06202 [pdf, other]
Title: Neighboring Backdoor Attacks on Graph Convolutional Network
Comments: 12 pages
Subjects: Artificial Intelligence (cs.AI)

Backdoor attacks have been widely studied to hide the misclassification rules in the normal models, which are only activated when the model is aware of the specific inputs (i.e., the trigger). However, despite their success in the conventional Euclidean space, there are few studies of backdoor attacks on graph structured data. In this paper, we propose a new type of backdoor which is specific to graph data, called neighboring backdoor. Considering the discreteness of graph data, how to effectively design the triggers while retaining the model accuracy on the original task is the major challenge. To address such a challenge, we set the trigger as a single node, and the backdoor is activated when the trigger node is connected to the target node. To preserve the model accuracy, the model parameters are not allowed to be modified. Thus, when the trigger node is not connected, the model performs normally. Under these settings, in this work, we focus on generating the features of the trigger node. Two types of backdoors are proposed: (1) Linear Graph Convolution Backdoor which finds an approximation solution for the feature generation (can be viewed as an integer programming problem) by looking at the linear part of GCNs. (2) Variants of existing graph attacks. We extend current gradient-based attack methods to our backdoor attack scenario. Extensive experiments on two social networks and two citation networks datasets demonstrate that all proposed backdoors can achieve an almost 100\% attack success rate while having no impact on predictive accuracy.

[8]  arXiv:2201.06248 [pdf]
Title: Using Machine Learning Based Models for Personality Recognition
Subjects: Artificial Intelligence (cs.AI)

Personality can be defined as the combination of behavior, emotion, motivation, and thoughts that aim at describing various aspects of human behavior based on a few stable and measurable characteristics. Considering the fact that our personality has a remarkable influence in our daily life, automatic recognition of a person's personality attributes can provide many essential practical applications in various aspects of cognitive science. deep learning based method for the task of personality recognition from text is proposed in this paper. Among various deep neural networks, Convolutional Neural Networks (CNN) have demonstrated profound efficiency in natural language processing and especially personality detection. Owing to the fact that various filter sizes in CNN may influence its performance, we decided to combine CNN with AdaBoost, a classical ensemble algorithm, to consider the possibility of using the contribution of various filter lengths and gasp their potential in the final classification via combining various classifiers with respective filter size using AdaBoost. Our proposed method was validated on the Essay dataset by conducting a series of experiments and the empirical results demonstrated the superiority of our proposed method compared to both machine learning and deep learning methods for the task of personality recognition.

[9]  arXiv:2201.06254 [pdf, other]
Title: Exploit Customer Life-time Value with Memoryless Experiments
Subjects: Artificial Intelligence (cs.AI)

As a measure of the long-term contribution produced by customers in a service or product relationship, life-time value, or LTV, can more comprehensively find the optimal strategy for service delivery. However, it is challenging to accurately abstract the LTV scene, model it reasonably, and find the optimal solution. The current theories either cannot precisely express LTV because of the single modeling structure, or there is no efficient solution. We propose a general LTV modeling method, which solves the problem that customers' long-term contribution is difficult to quantify while existing methods, such as modeling the click-through rate, only pursue the short-term contribution. At the same time, we also propose a fast dynamic programming solution based on a mutated bisection method and the memoryless repeated experiments assumption. The model and method can be applied to different service scenarios, such as the recommendation system. Experiments on real-world datasets confirm the effectiveness of the proposed model and optimization method. In addition, this whole LTV structure was deployed at a large E-commerce mobile phone application, where it managed to select optimal push message sending time and achieved a 10\% LTV improvement.

[10]  arXiv:2201.06268 [pdf, other]
Title: Continual Transformers: Redundancy-Free Attention for Online Inference
Comments: 10 pages, 6 figures, 8 tables
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

Transformers are attention-based sequence transduction models, which have found widespread success in Natural Language Processing and Computer Vision applications. Yet, Transformers in their current form are inherently limited to operate on whole token sequences rather than on one token at a time. Consequently, their use during online inference entails considerable redundancy due to the overlap in successive token sequences. In this work, we propose novel formulations of the Scaled Dot-Product Attention, which enable Transformers to perform efficient online token-by-token inference in a continual input stream. Importantly, our modification is purely to the order of computations, while the produced outputs and learned weights are identical to those of the original Multi-Head Attention. To validate our approach, we conduct experiments on visual, audio, and audio-visual classification and detection tasks, i.e. Online Action Detection on THUMOS14 and TVSeries and Online Audio Classification on GTZAN, with remarkable results. Our continual one-block transformers reduce the floating point operations by respectively 63.5x and 51.5x in the Online Action Detection and Audio Classification experiments at similar predictive performance.

[11]  arXiv:2201.06274 [pdf, other]
Title: Detecting danger in gridworlds using Gromov's Link Condition
Comments: 16 pages, 11 figures, 3 appendices
Subjects: Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Combinatorics (math.CO); Geometric Topology (math.GT); Metric Geometry (math.MG)

Gridworlds have been long-utilised in AI research, particularly in reinforcement learning, as they provide simple yet scalable models for many real-world applications such as robot navigation, emergent behaviour, and operations research. We initiate a study of gridworlds using the mathematical framework of reconfigurable systems and state complexes due to Abrams, Ghrist & Peterson. State complexes represent all possible configurations of a system as a single geometric space, thus making them conducive to study using geometric, topological, or combinatorial methods. The main contribution of this work is a modification to the original Abrams, Ghrist & Peterson setup which we believe is more naturally-suited to the context of gridworlds. With this modification, the state complexes may exhibit geometric defects (failure of Gromov's Link Condition), however, we argue that these failures can indicate undesirable or dangerous states in the gridworld. Our results provide a novel method for seeking guaranteed safety limitations in discrete task environments with single or multiple agents, and offer potentially useful geometric and topological information for incorporation in or analysis of machine learning systems.

[12]  arXiv:2201.06276 [pdf]
Title: Railway Operation Rescheduling System via Dynamic Simulation and Reinforcement Learning
Comments: English translated version is placed at first and the original Japanese version follows. 4 pages and 5 figures in the original manuscript. Proceedings of the 28th jointed railway technology symposium (J-RAIL 2021)
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Optimization and Control (math.OC)

The number of railway service disruptions has been increasing owing to intensification of natural disasters. In addition, abrupt changes in social situations such as the COVID-19 pandemic require railway companies to modify the traffic schedule frequently. Therefore, automatic support for optimal scheduling is anticipated. In this study, an automatic railway scheduling system is presented. The system leverages reinforcement learning and a dynamic simulator that can simulate the railway traffic and passenger flow of a whole line. The proposed system enables rapid generation of the traffic schedule of a whole line because the optimization process is conducted in advance as the training. The system is evaluated using an interruption scenario, and the results demonstrate that the system can generate optimized schedules of the whole line in a few minutes.

[13]  arXiv:2201.06378 [pdf, other]
Title: Self-Supervised Anomaly Detection by Self-Distillation and Negative Sampling
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

Detecting whether examples belong to a given in-distribution or are Out-Of-Distribution (OOD) requires identifying features specific to the in-distribution. In the absence of labels, these features can be learned by self-supervised techniques under the generic assumption that the most abstract features are those which are statistically most over-represented in comparison to other distributions from the same domain. In this work, we show that self-distillation of the in-distribution training set together with contrasting against negative examples derived from shifting transformation of auxiliary data strongly improves OOD detection. We find that this improvement depends on how the negative samples are generated. In particular, we observe that by leveraging negative samples, which keep the statistics of low-level features while changing the high-level semantics, higher average detection performance is obtained. Furthermore, good negative sampling strategies can be identified from the sensitivity of the OOD detection score. The efficiency of our approach is demonstrated across a diverse range of OOD detection problems, setting new benchmarks for unsupervised OOD detection in the visual domain.

[14]  arXiv:2201.06386 [pdf, other]
Title: Visual Identification of Problematic Bias in Large Label Spaces
Subjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)

While the need for well-trained, fair ML systems is increasing ever more, measuring fairness for modern models and datasets is becoming increasingly difficult as they grow at an unprecedented pace. One key challenge in scaling common fairness metrics to such models and datasets is the requirement of exhaustive ground truth labeling, which cannot always be done. Indeed, this often rules out the application of traditional analysis metrics and systems. At the same time, ML-fairness assessments cannot be made algorithmically, as fairness is a highly subjective matter. Thus, domain experts need to be able to extract and reason about bias throughout models and datasets to make informed decisions. While visual analysis tools are of great help when investigating potential bias in DL models, none of the existing approaches have been designed for the specific tasks and challenges that arise in large label spaces. Addressing the lack of visualization work in this area, we propose guidelines for designing visualizations for such large label spaces, considering both technical and ethical issues. Our proposed visualization approach can be integrated into classical model and data pipelines, and we provide an implementation of our techniques open-sourced as a TensorBoard plug-in. With our approach, different models and datasets for large label spaces can be systematically and visually analyzed and compared to make informed fairness assessments tackling problematic bias.

[15]  arXiv:2201.06401 [pdf, other]
Title: Spatial State-Action Features for General Games
Comments: Submitted for peer review
Subjects: Artificial Intelligence (cs.AI)

In many board games and other abstract games, patterns have been used as features that can guide automated game-playing agents. Such patterns or features often represent particular configurations of pieces, empty positions, etc., which may be relevant for a game's strategies. Their use has been particularly prevalent in the game of Go, but also many other games used as benchmarks for AI research. Simple, linear policies of such features are unlikely to produce state-of-the-art playing strength like the deep neural networks that have been more commonly used in recent years do. However, they typically require significantly fewer resources to train, which is paramount for large-scale studies of hundreds to thousands of distinct games. In this paper, we formulate a design and efficient implementation of spatial state-action features for general games. These are patterns that can be trained to incentivise or disincentivise actions based on whether or not they match variables of the state in a local area around action variables. We provide extensive details on several design and implementation choices, with a primary focus on achieving a high degree of generality to support a wide variety of different games using different board geometries or other graphs. Secondly, we propose an efficient approach for evaluating active features for any given set of features. In this approach, we take inspiration from heuristics used in problems such as SAT to optimise the order in which parts of patterns are matched and prune unnecessary evaluations. An empirical evaluation on 33 distinct games in the Ludii general game system demonstrates the efficiency of this approach in comparison to a naive baseline, as well as a baseline based on prefix trees.

[16]  arXiv:2201.06406 [pdf]
Title: Deep Learning-based Quality Assessment of Clinical Protocol Adherence in Fetal Ultrasound Dating Scans
Comments: 13 pages, 2 figures, 3 tables. Proceedings of Machine Learning Research, Under Review. Full Paper MIDL 2022 submission
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

To assess fetal health during pregnancy, doctors use the gestational age (GA) calculation based on the Crown Rump Length (CRL) measurement in order to check for fetal size and growth trajectory. However, GA estimation based on CRL, requires proper positioning of calipers on the fetal crown and rump view, which is not always an easy plane to find, especially for an inexperienced sonographer. Finding a slightly oblique view from the true CRL view could lead to a different CRL value and therefore incorrect estimation of GA. This study presents an AI-based method for a quality assessment of the CRL view by verifying 7 clinical scoring criteria that are used to verify the correctness of the acquired plane. We show how our proposed solution achieves high accuracy on the majority of the scoring criteria when compared to an expert. We also show that if such scoring system is used, it helps identify poorly acquired images accurately and hence may help sonographers acquire better images which could potentially lead to a better assessment of conditions such as Intrauterine Growth Restriction (IUGR).

[17]  arXiv:2201.06409 [pdf, other]
Title: Search and Score-Based Waterfall Auction Optimization
Subjects: Artificial Intelligence (cs.AI)

Online advertising is a major source of income for many online companies. One common approach is to sell online advertisements via waterfall auctions, where a publisher makes sequential price offers to ad networks. The publisher controls the order and prices of the waterfall and by that aims to maximize his revenue. In this work, we propose a methodology to learn a waterfall strategy from historical data by wisely searching in the space of possible waterfalls and selecting the one leading to the highest revenue. The contribution of this work is twofold; First, we propose a novel method to estimate the valuation distribution of each user with respect to each ad network. Second, we utilize the valuation matrix to score our candidate waterfalls as part of a procedure that iteratively searches in local neighborhoods. Our framework guarantees that the waterfall revenue improves between iterations until converging to a local optimum. Real-world demonstrations are provided to show that the proposed method improves the total revenue of real-world waterfalls compared to manual expert optimization. Finally, the code and the data are available here.

[18]  arXiv:2201.06494 [pdf, other]
Title: AugLy: Data Augmentations for Robustness
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

We introduce AugLy, a data augmentation library with a focus on adversarial robustness. AugLy provides a wide array of augmentations for multiple modalities (audio, image, text, & video). These augmentations were inspired by those that real users perform on social media platforms, some of which were not already supported by existing data augmentation libraries. AugLy can be used for any purpose where data augmentations are useful, but it is particularly well-suited for evaluating robustness and systematically generating adversarial attacks. In this paper we present how AugLy works, benchmark it compared against existing libraries, and use it to evaluate the robustness of various state-of-the-art models to showcase AugLy's utility. The AugLy repository can be found at https://github.com/facebookresearch/AugLy.

[19]  arXiv:2201.06505 [pdf]
Title: Data Harmonisation for Information Fusion in Digital Healthcare: A State-of-the-Art Systematic Review, Meta-Analysis and Future Research Directions
Comments: 54 pages, 14 figures, accepted by the Information Fusion journal
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

Removing the bias and variance of multicentre data has always been a challenge in large scale digital healthcare studies, which requires the ability to integrate clinical features extracted from data acquired by different scanners and protocols to improve stability and robustness. Previous studies have described various computational approaches to fuse single modality multicentre datasets. However, these surveys rarely focused on evaluation metrics and lacked a checklist for computational data harmonisation studies. In this systematic review, we summarise the computational data harmonisation approaches for multi-modality data in the digital healthcare field, including harmonisation strategies and evaluation metrics based on different theories. In addition, a comprehensive checklist that summarises common practices for data harmonisation studies is proposed to guide researchers to report their research findings more effectively. Last but not least, flowcharts presenting possible ways for methodology and metric selection are proposed and the limitations of different methods have been surveyed for future research.

[20]  arXiv:2201.06655 [pdf, other]
Title: Multi-winner Approval Voting Goes Epistemic
Subjects: Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)

Epistemic voting interprets votes as noisy signals about a ground truth. We consider contexts where the truth consists of a set of objective winners, knowing a lower and upper bound on its cardinality. A prototypical problem for this setting is the aggre-gation of multi-label annotations with prior knowledge on the size of the ground truth. We posit noisemodels, for which we define rules that output an optimal set of winners. We report on experiments on multi-label annotations (which we collected).

[21]  arXiv:2201.06692 [pdf, ps, other]
Title: Explainable Decision Making with Lean and Argumentative Explanations
Comments: JAIR submission. 74 pages (50 excluding proofs, appendix, and references)
Subjects: Artificial Intelligence (cs.AI)

It is widely acknowledged that transparency of automated decision making is crucial for deployability of intelligent systems, and explaining the reasons why some decisions are "good" and some are not is a way to achieving this transparency. We consider two variants of decision making, where "good" decisions amount to alternatives (i) meeting "most" goals, and (ii) meeting "most preferred" goals. We then define, for each variant and notion of "goodness" (corresponding to a number of existing notions in the literature), explanations in two formats, for justifying the selection of an alternative to audiences with differing needs and competences: lean explanations, in terms of goals satisfied and, for some notions of "goodness", alternative decisions, and argumentative explanations, reflecting the decision process leading to the selection, while corresponding to the lean explanations. To define argumentative explanations, we use assumption-based argumentation (ABA), a well-known form of structured argumentation. Specifically, we define ABA frameworks such that "good" decisions are admissible ABA arguments and draw argumentative explanations from dispute trees sanctioning this admissibility. Finally, we instantiate our overall framework for explainable decision-making to accommodate connections between goals and decisions in terms of decision graphs incorporating defeasible and non-defeasible information.

[22]  arXiv:2201.06759 [pdf, other]
Title: Knowledge Sharing via Domain Adaptation in Customs Fraud Detection
Comments: AAAI2022, Special track on AI for Social Impact
Subjects: Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG)

Knowledge of the changing traffic is critical in risk management. Customs offices worldwide have traditionally relied on local resources to accumulate knowledge and detect tax fraud. This naturally poses countries with weak infrastructure to become tax havens of potentially illicit trades. The current paper proposes DAS, a memory bank platform to facilitate knowledge sharing across multi-national customs administrations to support each other. We propose a domain adaptation method to share transferable knowledge of frauds as prototypes while safeguarding the local trade information. Data encompassing over 8 million import declarations have been used to test the feasibility of this new system, which shows that participating countries may benefit up to 2-11 times in fraud detection with the help of shared knowledge. We discuss implications for substantial tax revenue potential and strengthened policy against illicit trades.

[23]  arXiv:2201.06771 [pdf, other]
Title: TaxoCom: Topic Taxonomy Completion with Hierarchical Discovery of Novel Topic Clusters
Comments: 11 pages, 7 figures
Subjects: Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

Topic taxonomies, which represent the latent topic (or category) structure of document collections, provide valuable knowledge of contents in many applications such as web search and information filtering. Recently, several unsupervised methods have been developed to automatically construct the topic taxonomy from a text corpus, but it is challenging to generate the desired taxonomy without any prior knowledge. In this paper, we study how to leverage the partial (or incomplete) information about the topic structure as guidance to find out the complete topic taxonomy. We propose a novel framework for topic taxonomy completion, named TaxoCom, which recursively expands the topic taxonomy by discovering novel sub-topic clusters of terms and documents. To effectively identify novel topics within a hierarchical topic structure, TaxoCom devises its embedding and clustering techniques to be closely-linked with each other: (i) locally discriminative embedding optimizes the text embedding space to be discriminative among known (i.e., given) sub-topics, and (ii) novelty adaptive clustering assigns terms into either one of the known sub-topics or novel sub-topics. Our comprehensive experiments on two real-world datasets demonstrate that TaxoCom not only generates the high-quality topic taxonomy in terms of term coherency and topic coverage but also outperforms all other baselines for a downstream task.

[24]  arXiv:2201.06779 [pdf, other]
Title: Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records
Subjects: Artificial Intelligence (cs.AI)

Disease risk prediction has attracted increasing attention in the field of modern healthcare, especially with the latest advances in artificial intelligence (AI). Electronic health records (EHRs), which contain heterogeneous patient information, are widely used in disease risk prediction tasks. One challenge of applying AI models for risk prediction lies in generating interpretable evidence to support the prediction results while retaining the prediction ability. In order to address this problem, we propose the method of jointly embedding words and labels whereby attention modules learn the weights of words from medical notes according to their relevance to the names of risk prediction labels. This approach boosts interpretability by employing an attention mechanism and including the names of prediction tasks in the model. However, its application is only limited to the handling of textual inputs such as medical notes. In this paper, we propose a label dependent attention model LDAM to 1) improve the interpretability by exploiting Clinical-BERT (a biomedical language model pre-trained on a large clinical corpus) to encode biomedically meaningful features and labels jointly; 2) extend the idea of joint embedding to the processing of time-series data, and develop a multi-modal learning framework for integrating heterogeneous information from medical notes and time-series health status indicators. To demonstrate our method, we apply LDAM to the MIMIC-III dataset to predict different disease risks. We evaluate our method both quantitatively and qualitatively. Specifically, the predictive power of LDAM will be shown, and case studies will be carried out to illustrate its interpretability.

[25]  arXiv:2201.06783 [pdf, other]
Title: Label-dependent and event-guided interpretable disease risk prediction using EHRs
Subjects: Artificial Intelligence (cs.AI)

Electronic health records (EHRs) contain patients' heterogeneous data that are collected from medical providers involved in the patient's care, including medical notes, clinical events, laboratory test results, symptoms, and diagnoses. In the field of modern healthcare, predicting whether patients would experience any risks based on their EHRs has emerged as a promising research area, in which artificial intelligence (AI) plays a key role. To make AI models practically applicable, it is required that the prediction results should be both accurate and interpretable. To achieve this goal, this paper proposed a label-dependent and event-guided risk prediction model (LERP) to predict the presence of multiple disease risks by mainly extracting information from unstructured medical notes. Our model is featured in the following aspects. First, we adopt a label-dependent mechanism that gives greater attention to words from medical notes that are semantically similar to the names of risk labels. Secondly, as the clinical events (e.g., treatments and drugs) can also indicate the health status of patients, our model utilizes the information from events and uses them to generate an event-guided representation of medical notes. Thirdly, both label-dependent and event-guided representations are integrated to make a robust prediction, in which the interpretability is enabled by the attention weights over words from medical notes. To demonstrate the applicability of the proposed method, we apply it to the MIMIC-III dataset, which contains real-world EHRs collected from hospitals. Our method is evaluated in both quantitative and qualitative ways.

[26]  arXiv:2201.06786 [pdf, other]
Title: Unsupervised Multimodal Word Discovery based on Double Articulation Analysis with Co-occurrence cues
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Robotics (cs.RO)

Human infants acquire their verbal lexicon from minimal prior knowledge of language based on the statistical properties of phonological distributions and the co-occurrence of other sensory stimuli. In this study, we propose a novel fully unsupervised learning method discovering speech units by utilizing phonological information as a distributional cue and object information as a co-occurrence cue. The proposed method can not only (1) acquire words and phonemes from speech signals using unsupervised learning, but can also (2) utilize object information based on multiple modalities (i.e., vision, tactile, and auditory) simultaneously. The proposed method is based on the Nonparametric Bayesian Double Articulation Analyzer (NPB-DAA) discovering phonemes and words from phonological features, and Multimodal Latent Dirichlet Allocation (MLDA) categorizing multimodal information obtained from objects. In the experiment, the proposed method showed higher word discovery performance than the baseline methods. In particular, words that expressed the characteristics of the object (i.e., words corresponding to nouns and adjectives) were segmented accurately. Furthermore, we examined how learning performance is affected by differences in the importance of linguistic information. When the weight of the word modality was increased, the performance was further improved compared to the fixed condition.

[27]  arXiv:2201.06863 [pdf, other]
Title: Programmatic Policy Extraction by Iterative Local Search
Subjects: Artificial Intelligence (cs.AI)

Reinforcement learning policies are often represented by neural networks, but programmatic policies are preferred in some cases because they are more interpretable, amenable to formal verification, or generalize better. While efficient algorithms for learning neural policies exist, learning programmatic policies is challenging. Combining imitation-projection and dataset aggregation with a local search heuristic, we present a simple and direct approach to extracting a programmatic policy from a pretrained neural policy. After examining our local search heuristic on a programming by example problem, we demonstrate our programmatic policy extraction method on a pendulum swing-up problem. Both when trained using a hand crafted expert policy and a learned neural policy, our method discovers simple and interpretable policies that perform almost as well as the original.

[28]  arXiv:2201.07032 [pdf, other]
Title: The Mathematics of Comparing Objects
Subjects: Artificial Intelligence (cs.AI); Rings and Algebras (math.RA)

`After reading two different crime stories, an artificial intelligence concludes that in both stories the police has found the murderer just by random.'' -- To what extend and under which assumptions this is a description of a realistic scenario?

[29]  arXiv:2201.07040 [pdf]
Title: Benchmark datasets driving artificial intelligence development fail to capture the needs of medical professionals
Subjects: Artificial Intelligence (cs.AI)

Publicly accessible benchmarks that allow for assessing and comparing model performances are important drivers of progress in artificial intelligence (AI). While recent advances in AI capabilities hold the potential to transform medical practice by assisting and augmenting the cognitive processes of healthcare professionals, the coverage of clinically relevant tasks by AI benchmarks is largely unclear. Furthermore, there is a lack of systematized meta-information that allows clinical AI researchers to quickly determine accessibility, scope, content and other characteristics of datasets and benchmark datasets relevant to the clinical domain.
To address these issues, we curated and released a comprehensive catalogue of datasets and benchmarks pertaining to the broad domain of clinical and biomedical natural language processing (NLP), based on a systematic review of literature and online resources. A total of 450 NLP datasets were manually systematized and annotated with rich metadata, such as targeted tasks, clinical applicability, data types, performance metrics, accessibility and licensing information, and availability of data splits. We then compared tasks covered by AI benchmark datasets with relevant tasks that medical practitioners reported as highly desirable targets for automation in a previous empirical study.
Our analysis indicates that AI benchmarks of direct clinical relevance are scarce and fail to cover most work activities that clinicians want to see addressed. In particular, tasks associated with routine documentation and patient data administration workflows are not represented despite significant associated workloads. Thus, currently available AI benchmarks are improperly aligned with desired targets for AI automation in clinical settings, and novel benchmarks should be created to fill these gaps.

[30]  arXiv:2201.07050 [pdf, other]
Title: Combining Fast and Slow Thinking for Human-like and Efficient Navigation in Constrained Environments
Comments: arXiv admin note: substantial text overlap with arXiv:2110.01834
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Current AI systems lack several important human capabilities, such as adaptability, generalizability, self-control, consistency, common sense, and causal reasoning. We believe that existing cognitive theories of human decision making, such as the thinking fast and slow theory, can provide insights on how to advance AI systems towards some of these capabilities. In this paper, we propose a general architecture that is based on fast/slow solvers and a metacognitive component. We then present experimental results on the behavior of an instance of this architecture, for AI systems that make decisions about navigating in a constrained environment. We show how combining the fast and slow decision modalities allows the system to evolve over time and gradually pass from slow to fast thinking with enough experience, and that this greatly helps in decision quality, resource consumption, and efficiency.

[31]  arXiv:2201.07125 [pdf, other]
Title: WATCH: Wasserstein Change Point Detection for High-Dimensional Time Series Data
Journal-ref: 2021 IEEE International Conference on Big Data (Big Data)
Subjects: Artificial Intelligence (cs.AI)

Detecting relevant changes in dynamic time series data in a timely manner is crucially important for many data analysis tasks in real-world settings. Change point detection methods have the ability to discover changes in an unsupervised fashion, which represents a desirable property in the analysis of unbounded and unlabeled data streams. However, one limitation of most of the existing approaches is represented by their limited ability to handle multivariate and high-dimensional data, which is frequently observed in modern applications such as traffic flow prediction, human activity recognition, and smart grids monitoring. In this paper, we attempt to fill this gap by proposing WATCH, a novel Wasserstein distance-based change point detection approach that models an initial distribution and monitors its behavior while processing new data points, providing accurate and robust detection of change points in dynamic high-dimensional data. An extensive experimental evaluation involving a large number of benchmark datasets shows that WATCH is capable of accurately identifying change points and outperforming state-of-the-art methods.

Cross-lists for Wed, 19 Jan 22

[32]  arXiv:2201.05624 (cross-list from cs.LG) [pdf, other]
Title: Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Numerical Analysis (math.NA); Data Analysis, Statistics and Probability (physics.data-an)

Physic-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations, like Partial Differential Equations (PDE), as a component of the neural network itself. PINNs are nowadays used to solve PDEs, fractional equations, and integral-differential equations. This novel methodology has arisen as a multi-task learning framework in which a NN must fit observed data while reducing a PDE residual. This article provides a comprehensive review of the literature on PINNs: while the primary goal of the study was to characterize these networks and their related advantages and disadvantages, the review also attempts to incorporate publications on a larger variety of issues, including physics-constrained neural networks (PCNN), where the initial or boundary conditions are directly embedded in the NN structure rather than in the loss functions. The study indicates that most research has focused on customizing the PINN through different activation functions, gradient optimization techniques, neural network structures, and loss function structures. Despite the wide range of applications for which PINNs have been used, by demonstrating their ability to be more feasible in some contexts than classical numerical techniques like Finite Element Method (FEM), advancements are still possible, most notably theoretical issues that remain unresolved.

[33]  arXiv:2201.05629 (cross-list from cs.LG) [pdf, other]
Title: Zero-Shot Machine Unlearning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

With the introduction of new privacy regulations, machine unlearning is becoming an emerging research problem due to an increasing need for regulatory compliance required for machine learning (ML) applications. Modern privacy regulations grant citizens the right to be forgotten by products, services and companies. This necessitates deletion of data not only from storage archives but also from ML model. The right to be forgotten requests come in the form of removal of a certain set or class of data from the already trained ML model. Practical considerations preclude retraining of the model from scratch minus the deleted data. The few existing studies use the whole training data, or a subset of training data, or some metadata stored during training to update the model weights for unlearning. However, strict regulatory compliance requires time-bound deletion of data. Thus, in many cases, no data related to the training process or training samples may be accessible even for the unlearning purpose. We therefore ask the question: is it possible to achieve unlearning with zero training samples? In this paper, we introduce the novel problem of zero-shot machine unlearning that caters for the extreme but practical scenario where zero original data samples are available for use. We then propose two novel solutions for zero-shot machine unlearning based on (a) error minimizing-maximizing noise and (b) gated knowledge transfer. We also introduce a new evaluation metric, Anamnesis Index (AIN) to effectively measure the quality of the unlearning method. The experiments show promising results for unlearning in deep learning models on benchmark vision data-sets. The source code will be made publicly available.

[34]  arXiv:2201.05632 (cross-list from cs.NI) [pdf, other]
Title: OrchestRAN: Network Automation through Orchestrated Intelligence in the Open RAN
Journal-ref: IEEE INFOCOM 2022
Subjects: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI)

The next generation of cellular networks will be characterized by softwarized, open, and disaggregated architectures exposing analytics and control knobs to enable network intelligence. How to realize this vision, however, is largely an open problem. In this paper, we take a decisive step forward by presenting and prototyping OrchestRAN, a novel orchestration framework that embraces and builds upon the Open RAN paradigm to provide a practical solution to these challenges. OrchestRAN has been designed to execute in the non-real-time RAN Intelligent Controller (RIC) and allows Network Operators (NOs) to specify high-level control/inference objectives (i.e., adapt scheduling, and forecast capacity in near-real-time for a set of base stations in Downtown New York). OrchestRAN automatically computes the optimal set of data-driven algorithms and their execution location to achieve intents specified by the NOs while meeting the desired timing requirements. We show that the problem of orchestrating intelligence in Open RAN is NP-hard, and design low-complexity solutions to support real-world applications. We prototype OrchestRAN and test it at scale on Colosseum. Our experimental results on a network with 7 base stations and 42 users demonstrate that OrchestRAN is able to instantiate data-driven services on demand with minimal control overhead and latency.

[35]  arXiv:2201.05646 (cross-list from cs.IR) [pdf, other]
Title: ULTRA: A Data-driven Approach for Recommending Team Formation in Response to Proposal Calls
Comments: 7 pages
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

We introduce an emerging AI-based approach and prototype system for assisting team formation when researchers respond to calls for proposals from funding agencies. This is an instance of the general problem of building teams when demand opportunities come periodically and potential members may vary over time. The novelties of our approach are that we: (a) extract technical skills needed about researchers and calls from multiple data sources and normalize them using Natural Language Processing (NLP) techniques, (b) build a prototype solution based on matching and teaming based on constraints, (c) describe initial feedback about system from researchers at a University to deploy, and (d) create and publish a dataset that others can use.

[36]  arXiv:2201.05647 (cross-list from cs.LG) [pdf, other]
Title: Tools and Practices for Responsible AI Engineering
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)

Responsible Artificial Intelligence (AI) - the practice of developing, evaluating, and maintaining accurate AI systems that also exhibit essential properties such as robustness and explainability - represents a multifaceted challenge that often stretches standard machine learning tooling, frameworks, and testing methods beyond their limits. In this paper, we present two new software libraries - hydra-zen and the rAI-toolbox - that address critical needs for responsible AI engineering. hydra-zen dramatically simplifies the process of making complex AI applications configurable, and their behaviors reproducible. The rAI-toolbox is designed to enable methods for evaluating and enhancing the robustness of AI-models in a way that is scalable and that composes naturally with other popular ML frameworks. We describe the design principles and methodologies that make these tools effective, including the use of property-based testing to bolster the reliability of the tools themselves. Finally, we demonstrate the composability and flexibility of the tools by showing how various use cases from adversarial robustness and explainable AI can be concisely implemented with familiar APIs.

[37]  arXiv:2201.05700 (cross-list from cs.CL) [pdf, other]
Title: Cost-Effective Training in Low-Resource Neural Machine Translation
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

While Active Learning (AL) techniques are explored in Neural Machine Translation (NMT), only a few works focus on tackling low annotation budgets where a limited number of sentences can get translated. Such situations are especially challenging and can occur for endangered languages with few human annotators or having cost constraints to label large amounts of data. Although AL is shown to be helpful with large budgets, it is not enough to build high-quality translation systems in these low-resource conditions. In this work, we propose a cost-effective training procedure to increase the performance of NMT models utilizing a small number of annotated sentences and dictionary entries. Our method leverages monolingual data with self-supervised objectives and a small-scale, inexpensive dictionary for additional supervision to initialize the NMT model before applying AL. We show that improving the model using a combination of these knowledge sources is essential to exploit AL strategies and increase gains in low-resource conditions. We also present a novel AL strategy inspired by domain adaptation for NMT and show that it is effective for low budgets. We propose a new hybrid data-driven approach, which samples sentences that are diverse from the labelled data and also most similar to unlabelled data. Finally, we show that initializing the NMT model and further using our AL strategy can achieve gains of up to $13$ BLEU compared to conventional AL methods.

[38]  arXiv:2201.05729 (cross-list from cs.CV) [pdf, other]
Title: CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multimedia (cs.MM)

Contrastive language-image pretraining (CLIP) links vision and language modalities into a unified embedding space, yielding the tremendous potential for vision-language (VL) tasks. While early concurrent works have begun to study this potential on a subset of tasks, important questions remain: 1) What is the benefit of CLIP on unstudied VL tasks? 2) Does CLIP provide benefit in low-shot or domain-shifted scenarios? 3) Can CLIP improve existing approaches without impacting inference or pretraining complexity? In this work, we seek to answer these questions through two key contributions. First, we introduce an evaluation protocol that includes Visual Commonsense Reasoning (VCR), Visual Entailment (SNLI-VE), and Visual Question Answering (VQA), across a variety of data availability constraints and conditions of domain shift. Second, we propose an approach, named CLIP Targeted Distillation (CLIP-TD), to intelligently distill knowledge from CLIP into existing architectures using a dynamically weighted objective applied to adaptively selected tokens per instance. Experiments demonstrate that our proposed CLIP-TD leads to exceptional gains in the low-shot (up to 51.9%) and domain-shifted (up to 71.3%) conditions of VCR, while simultaneously improving performance under standard fully-supervised conditions (up to 2%), achieving state-of-art performance on VCR compared to other single models that are pretrained with image-text data only. On SNLI-VE, CLIP-TD produces significant gains in low-shot conditions (up to 6.6%) as well as fully supervised (up to 3%). On VQA, CLIP-TD provides improvement in low-shot (up to 9%), and in fully-supervised (up to 1.3%). Finally, CLIP-TD outperforms concurrent works utilizing CLIP for finetuning, as well as baseline naive distillation approaches. Code will be made available.

[39]  arXiv:2201.05737 (cross-list from math.OC) [pdf, other]
Title: A unified algorithm framework for mean-variance optimization in discounted Markov decision processes
Subjects: Optimization and Control (math.OC); Artificial Intelligence (cs.AI)

This paper studies the risk-averse mean-variance optimization in infinite-horizon discounted Markov decision processes (MDPs). The involved variance metric concerns reward variability during the whole process, and future deviations are discounted to their present values. This discounted mean-variance optimization yields a reward function dependent on a discounted mean, and this dependency renders traditional dynamic programming methods inapplicable since it suppresses a crucial property -- time consistency. To deal with this unorthodox problem, we introduce a pseudo mean to transform the untreatable MDP to a standard one with a redefined reward function in standard form and derive a discounted mean-variance performance difference formula. With the pseudo mean, we propose a unified algorithm framework with a bilevel optimization structure for the discounted mean-variance optimization. The framework unifies a variety of algorithms for several variance-related problems including, but not limited to, risk-averse variance and mean-variance optimizations in discounted and average MDPs. Furthermore, the convergence analyses missing from the literature can be complemented with the proposed framework as well. Taking the value iteration as an example, we develop a discounted mean-variance value iteration algorithm and prove its convergence to a local optimum with the aid of a Bellman local-optimality equation. Finally, we conduct a numerical experiment on portfolio management to validate the proposed algorithm.

[40]  arXiv:2201.05745 (cross-list from cs.LG) [pdf, other]
Title: Deep Optimal Transport on SPD Manifolds for Domain Adaptation
Authors: Ce Ju, Cuntai Guan
Comments: 10 pages; 7 figures
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Signal Processing (eess.SP)

The domain adaption (DA) problem on symmetric positive definite (SPD) manifolds has raised interest in the machine learning community because of the growing potential for the SPD-matrix representations across many non-stationary applicable scenarios. This paper generalizes the joint distribution adaption (JDA) to align the source and target domains on SPD manifolds and proposes a deep network architecture, Deep Optimal Transport (DOT), using the generalized JDA and the existing deep network architectures on SPD manifolds. The specific architecture in DOT enables it to learn an approximate optimal transport (OT) solution to the DA problems on SPD manifolds. In the experiments, DOT exhibits a 2.32% and 2.92% increase on the average accuracy in two highly non-stationary cross-session scenarios in brain-computer interfaces (BCIs), respectively. The visualizational results of the source and target domains before and after the transformation also demonstrate the validity of DOT.

[41]  arXiv:2201.05748 (cross-list from cs.LG) [pdf, other]
Title: Concise Logarithmic Loss Function for Robust Training of Anomaly Detection Model
Authors: YeongHyeon Park
Comments: 4 pages, 3 figures, 2 tables
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Recently, deep learning-based algorithms are widely adopted due to the advantage of being able to establish anomaly detection models without or with minimal domain knowledge of the task. Instead, to train the artificial neural network more stable, it should be better to define the appropriate neural network structure or the loss function. For the training anomaly detection model, the mean squared error (MSE) function is adopted widely. On the other hand, the novel loss function, logarithmic mean squared error (LMSE), is proposed in this paper to train the neural network more stable. This study covers a variety of comparisons from mathematical comparisons, visualization in the differential domain for backpropagation, loss convergence in the training process, and anomaly detection performance. In an overall view, LMSE is superior to the existing MSE function in terms of strongness of loss convergence, anomaly detection performance. The LMSE function is expected to be applicable for training not only the anomaly detection model but also the general generative neural network.

[42]  arXiv:2201.05756 (cross-list from cs.LG) [pdf, other]
Title: Block Policy Mirror Descent
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC)

In this paper, we present a new class of policy gradient (PG) methods, namely the block policy mirror descent (BPMD) methods for solving a class of regularized reinforcement learning (RL) problems with (strongly) convex regularizers. Compared to the traditional PG methods with batch update rule, which visit and update the policy for every state, BPMD methods have cheap per-iteration computation via a partial update rule that performs the policy update on a sampled state. Despite the nonconvex nature of the problem and a partial update rule, BPMD methods achieve fast linear convergence to the global optimality. We further extend BPMD methods to the stochastic setting, by utilizing stochastic first-order information constructed from samples. We establish $\cO(1/\epsilon)$ (resp. $\cO(1/\epsilon^2)$) sample complexity for the strongly convex (resp. non-strongly convex) regularizers, with different procedures for constructing the stochastic first-order information, where $\epsilon$ denotes the target accuracy. To the best of our knowledge, this is the first time that block coordinate descent methods have been developed and analyzed for policy optimization in reinforcement learning.

[43]  arXiv:2201.05760 (cross-list from cs.LG) [pdf]
Title: Big Data Application for Network Level Travel Time Prediction
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Travel time is essential in advanced traveler information systems (ATIS). This paper used the big data analytics engines Apache Spark and Apache MXNet for data processing and modeling. The efficiency gain was evaluated by comparing it with popular data science and deep learning frameworks. The hierarchical feature pooling is explored for both between layer and the output layer LSTM (Long-Short-Term-Memory). The designed hierarchical LSTM (hiLSTM) model can consider the dependencies at a different time scale to capture the spatial-temporal correlations from network-level corridor travel time. A self-attention module is then used to connect temporal and spatial features to the fully connected layers, predicting travel time for all corridors instead of a single link/route. Seasonality and autocorrelation were performed to explore the trend of time-varying data. The case study shows that the Hierarchical LSTM with Attention (hiLSTMat) model gives the best result and outperforms baseline models. The California Bay Area corridor travel time dataset covering four-year periods was published from Caltrans Performance Measurement System (PeMS) system.

[44]  arXiv:2201.05773 (cross-list from stat.ME) [pdf, other]
Title: Automated causal inference in application to randomized controlled clinical trials
Comments: Submitted to Nature Machine Intelligence. The code is publicly available via this https URL
Subjects: Methodology (stat.ME); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Randomized controlled trials (RCTs) are considered as the gold standard for testing causal hypotheses in the clinical domain. However, the investigation of prognostic variables of patient outcome in a hypothesized cause-effect route is not feasible using standard statistical methods. Here, we propose a new automated causal inference method (AutoCI) built upon the invariant causal prediction (ICP) framework for the causal re-interpretation of clinical trial data. Compared to existing methods, we show that the proposed AutoCI allows to efficiently determine the causal variables with a clear differentiation on two real-world RCTs of endometrial cancer patients with mature outcome and extensive clinicopathological and molecular data. This is achieved via suppressing the causal probability of non-causal variables by a wide margin. In ablation studies, we further demonstrate that the assignment of causal probabilities by AutoCI remain consistent in the presence of confounders. In conclusion, these results confirm the robustness and feasibility of AutoCI for future applications in real-world clinical analysis.

[45]  arXiv:2201.05775 (cross-list from cs.CV) [pdf, other]
Title: Explainability Tools Enabling Deep Learning in Future In-Situ Real-Time Planetary Explorations
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Deep learning (DL) has proven to be an effective machine learning and computer vision technique. DL-based image segmentation, object recognition and classification will aid many in-situ Mars rover tasks such as path planning and artifact recognition/extraction. However, most of the Deep Neural Network (DNN) architectures are so complex that they are considered a 'black box'. In this paper, we used integrated gradients to describe the attributions of each neuron to the output classes. It provides a set of explainability tools (ET) that opens the black box of a DNN so that the individual contribution of neurons to category classification can be ranked and visualized. The neurons in each dense layer are mapped and ranked by measuring expected contribution of a neuron to a class vote given a true image label. The importance of neurons is prioritized according to their correct or incorrect contribution to the output classes and suppression or bolstering of incorrect classes, weighted by the size of each class. ET provides an interface to prune the network to enhance high-rank neurons and remove low-performing neurons. ET technology will make DNNs smaller and more efficient for implementation in small embedded systems. It also leads to more explainable and testable DNNs that can make systems easier for Validation \& Verification. The goal of ET technology is to enable the adoption of DL in future in-situ planetary exploration missions.

[46]  arXiv:2201.05793 (cross-list from cs.CL) [pdf, other]
Title: A Benchmark for Generalizable and Interpretable Temporal Question Answering over Knowledge Bases
Comments: 7 pages, 2 figures, 7 tables. arXiv admin note: substantial text overlap with arXiv:2109.13430
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Knowledge Base Question Answering (KBQA) tasks that involve complex reasoning are emerging as an important research direction. However, most existing KBQA datasets focus primarily on generic multi-hop reasoning over explicit facts, largely ignoring other reasoning types such as temporal, spatial, and taxonomic reasoning. In this paper, we present a benchmark dataset for temporal reasoning, TempQA-WD, to encourage research in extending the present approaches to target a more challenging set of complex reasoning tasks. Specifically, our benchmark is a temporal question answering dataset with the following advantages: (a) it is based on Wikidata, which is the most frequently curated, openly available knowledge base, (b) it includes intermediate sparql queries to facilitate the evaluation of semantic parsing based approaches for KBQA, and (c) it generalizes to multiple knowledge bases: Freebase and Wikidata. The TempQA-WD dataset is available at https://github.com/IBM/tempqa-wd.

[47]  arXiv:2201.05799 (cross-list from cs.LG) [pdf, ps, other]
Title: Hyperplane bounds for neural feature mappings
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Deep learning methods minimise the empirical risk using loss functions such as the cross entropy loss. When minimising the empirical risk, the generalisation of the learnt function still depends on the performance on the training data, the Vapnik-Chervonenkis(VC)-dimension of the function and the number of training examples. Neural networks have a large number of parameters, which correlates with their VC-dimension that is typically large but not infinite, and typically a large number of training instances are needed to effectively train them.
In this work, we explore how to optimize feature mappings using neural network with the intention to reduce the effective VC-dimension of the hyperplane found in the space generated by the mapping. An interpretation of the results of this study is that it is possible to define a loss that controls the VC-dimension of the separating hyperplane. We evaluate this approach and observe that the performance when using this method improves when the size of the training set is small.

[48]  arXiv:2201.05809 (cross-list from cs.LG) [pdf, other]
Title: Weighting and Pruning based Ensemble Deep Random Vector Functional Link Network for Tabular Data Classification
Comments: 8 tables, 8 figures, 31 pages. This paper has already been reviewed by many reviewers and can be easily integrated with ELM
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)

In this paper, we first introduce batch normalization to the edRVFL network. This re-normalization method can help the network avoid divergence of the hidden features. Then we propose novel variants of Ensemble Deep Random Vector Functional Link (edRVFL). Weighted edRVFL (WedRVFL) uses weighting methods to give training samples different weights in different layers according to how the samples were classified confidently in the previous layer thereby increasing the ensemble's diversity and accuracy. Furthermore, a pruning-based edRVFL (PedRVFL) has also been proposed. We prune some inferior neurons based on their importance for classification before generating the next hidden layer. Through this method, we ensure that the randomly generated inferior features will not propagate to deeper layers. Subsequently, the combination of weighting and pruning, called Weighting and Pruning based Ensemble Deep Random Vector Functional Link Network (WPedRVFL), is proposed. We compare their performances with other state-of-the-art deep feedforward neural networks (FNNs) on 24 tabular UCI classification datasets. The experimental results illustrate the superior performance of our proposed methods.

[49]  arXiv:2201.05843 (cross-list from eess.SY) [pdf, other]
Title: Cooperative Multi-Agent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control
Comments: 10 pages, 6 figures, Accepted for publication in IEEE Transactions on Industrial Informatics (TII)
Subjects: Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO)

CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments. This paper creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimize overlapping and shadow areas for a reliable surveillance system. However, the operation of UAVs is subject to high uncertainty, necessitating autonomous recovery systems. This work develops a multi-agent deep reinforcement learning-based management scheme for reliable industry surveillance in smart city applications. The core idea this paper employs is autonomously replenishing the UAV's deficient network requirements with communications. Via intensive simulations, our proposed algorithm outperforms the state-of-the-art algorithms in terms of surveillance coverage, user support capability, and computational costs.

[50]  arXiv:2201.05845 (cross-list from eess.AS) [pdf, other]
Title: Recent Progress in the CUHK Dysarthric Speech Recognition System
Subjects: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Sound (cs.SD)

Despite the rapid progress of automatic speech recognition (ASR) technologies in the past few decades, recognition of disordered speech remains a highly challenging task to date. Disordered speech presents a wide spectrum of challenges to current data intensive deep neural networks (DNNs) based ASR technologies that predominantly target normal speech. This paper presents recent research efforts at the Chinese University of Hong Kong (CUHK) to improve the performance of disordered speech recognition systems on the largest publicly available UASpeech dysarthric speech corpus. A set of novel modelling techniques including neural architectural search, data augmentation using spectra-temporal perturbation, model based speaker adaptation and cross-domain generation of visual features within an audio-visual speech recognition (AVSR) system framework were employed to address the above challenges. The combination of these techniques produced the lowest published word error rate (WER) of 25.21% on the UASpeech test set 16 dysarthric speakers, and an overall WER reduction of 5.4% absolute (17.6% relative) over the CUHK 2018 dysarthric speech recognition system featuring a 6-way DNN system combination and cross adaptation of out-of-domain normal speech data trained systems. Bayesian model adaptation further allows rapid adaptation to individual dysarthric speakers to be performed using as little as 3.06 seconds of speech. The efficacy of these techniques were further demonstrated on a CUDYS Cantonese dysarthric speech recognition task.

[51]  arXiv:2201.05877 (cross-list from cs.LG) [pdf, other]
Title: A Framework for Pedestrian Sub-classification and Arrival Time Prediction at Signalized Intersection Using Preprocessed Lidar Data
Comments: 15 pages, 11 figures, 4 tables
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

The mortality rate for pedestrians using wheelchairs was 36% higher than the overall population pedestrian mortality rate. However, there is no data to clarify the pedestrians' categories in both fatal and nonfatal accidents, since police reports often do not keep a record of whether a victim was using a wheelchair or has a disability. Currently, real-time detection of vulnerable road users using advanced traffic sensors installed at the infrastructure side has a great potential to significantly improve traffic safety at the intersection. In this research, we develop a systematic framework with a combination of machine learning and deep learning models to distinguish disabled people from normal walk pedestrians and predict the time needed to reach the next side of the intersection. The proposed framework shows high performance both at vulnerable user classification and arrival time prediction accuracy.

[52]  arXiv:2201.05906 (cross-list from q-fin.TR) [pdf]
Title: Profitable Strategy Design by Using Deep Reinforcement Learning for Trades on Cryptocurrency Markets
Subjects: Trading and Market Microstructure (q-fin.TR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Deep Reinforcement Learning solutions have been applied to different control problems with outperforming and promising results. In this research work we have applied Proximal Policy Optimization, Soft Actor-Critic and Generative Adversarial Imitation Learning to strategy design problem of three cryptocurrency markets. Our input data includes price data and technical indicators. We have implemented a Gym environment based on cryptocurrency markets to be used with the algorithms. Our test results on unseen data shows a great potential for this approach in helping investors with an expert system to exploit the market and gain profit. Our highest gain for an unseen 66 day span is 4850 US dollars per 10000 US dollars investment. We also discuss on how a specific hyperparameter in the environment design can be used to adjust risk in the generated strategies.

[53]  arXiv:2201.05929 (cross-list from cs.DB) [pdf]
Title: Characterizing Big Data Management
Comments: volume 12, 2015
Journal-ref: Issues in Informing Science and Information Technology 2015
Subjects: Databases (cs.DB); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Applications (stat.AP)

Big data management is a reality for an increasing number of organizations in many areas and represents a set of challenges involving big data modeling, storage and retrieval, analysis and visualization. However, technological resources, people and processes are crucial to facilitate the management of big data in any kind of organization, allowing information and knowledge from a large volume of data to support decision-making. Big data management can be supported by these three dimensions: technology, people and processes. Hence, this article discusses these dimensions: the technological dimension that is related to storage, analytics and visualization of big data; the human aspects of big data; and, in addition, the process management dimension that involves in a technological and business approach the aspects of big data management.

[54]  arXiv:2201.05951 (cross-list from cs.CV) [pdf, ps, other]
Title: Global Regular Network for Writer Identification
Authors: Shiyu Wang
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Writer identification has practical applications for forgery detection and forensic science. Most models based on deep neural networks extract features from character image or sub-regions in character image, which ignoring features contained in page-region image. Our proposed global regular network (GRN) pays attention to these features. GRN network consists of two branches: one branch takes page handwriting as input to extract global features, and the other takes word handwriting as input to extract local features. Global features and local features merge in a global residual way to form overall features of the handwriting. The proposed GRN has two attributions: one is adding a branch to extract features contained in page; the other is using residual attention network to extract local feature. Experiments demonstrate the effectiveness of both strategies. On CVL dataset, our models achieve impressive 99.98% top-1 accuracy and 100% top-5 accuracy with shorter training time and fewer network parameters, which exceeded the state-of-the-art structure. The experiment shows the powerful ability of the network in the field of writer identification. The source code is available at https://github.com/wangshiyu001/GRN.

[55]  arXiv:2201.05962 (cross-list from cs.LG) [pdf]
Title: Enhancement of Healthcare Data Performance Metrics using Neural Network Machine Learning Algorithms
Comments: 8 pages, 18 fingures
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

Patients are often encouraged to make use of wearable devices for remote collection and monitoring of health data. This adoption of wearables results in a significant increase in the volume of data collected and transmitted. The battery life of the devices is then quickly diminished due to the high processing requirements of the devices. Given the importance attached to medical data, it is imperative that all transmitted data adhere to strict integrity and availability requirements. Reducing the volume of healthcare data for network transmission may improve sensor battery life without compromising accuracy. There is a trade-off between efficiency and accuracy which can be controlled by adjusting the sampling and transmission rates. This paper demonstrates that machine learning can be used to analyse complex health data metrics such as the accuracy and efficiency of data transmission to overcome the trade-off problem. The study uses time series nonlinear autoregressive neural network algorithms to enhance both data metrics by taking fewer samples to transmit. The algorithms were tested with a standard heart rate dataset to compare their accuracy and efficiency. The result showed that the Levenbery-Marquardt algorithm was the best performer with an efficiency of 3.33 and accuracy of 79.17%, which is similar to other algorithms accuracy but demonstrates improved efficiency. This proves that machine learning can improve without sacrificing a metric over the other compared to the existing methods with high efficiency.

[56]  arXiv:2201.05972 (cross-list from cs.CV) [pdf, other]
Title: Sparse Cross-scale Attention Network for Efficient LiDAR Panoptic Segmentation
Comments: Accepted by the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22)
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)

Two major challenges of 3D LiDAR Panoptic Segmentation (PS) are that point clouds of an object are surface-aggregated and thus hard to model the long-range dependency especially for large instances, and that objects are too close to separate each other. Recent literature addresses these problems by time-consuming grouping processes such as dual-clustering, mean-shift offsets, etc., or by bird-eye-view (BEV) dense centroid representation that downplays geometry. However, the long-range geometry relationship has not been sufficiently modeled by local feature learning from the above methods. To this end, we present SCAN, a novel sparse cross-scale attention network to first align multi-scale sparse features with global voxel-encoded attention to capture the long-range relationship of instance context, which can boost the regression accuracy of the over-segmented large objects. For the surface-aggregated points, SCAN adopts a novel sparse class-agnostic representation of instance centroids, which can not only maintain the sparsity of aligned features to solve the under-segmentation on small objects, but also reduce the computation amount of the network through sparse convolution. Our method outperforms previous methods by a large margin in the SemanticKITTI dataset for the challenging 3D PS task, achieving 1st place with a real-time inference speed.

[57]  arXiv:2201.06014 (cross-list from cs.MA) [pdf, other]
Title: Standby-Base Deadlock Avoidance Method Standby-Based Deadlock Avoidance Method for Multi-Agent Pickup and Delivery Tasks
Comments: Extended version of paper accepted at AAMAS 2022
Subjects: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Robotics (cs.RO)

The multi-agent pickup and delivery (MAPD) problem, in which multiple agents iteratively carry materials without collisions, has received significant attention. However, many conventional MAPD algorithms assume a specifically designed grid-like environment, such as an automated warehouse. Therefore, they have many pickup and delivery locations where agents can stay for a lengthy period, as well as plentiful detours to avoid collisions owing to the freedom of movement in a grid. By contrast, because a maze-like environment such as a search-and-rescue or construction site has fewer pickup/delivery locations and their numbers may be unbalanced, many agents concentrate on such locations resulting in inefficient operations, often becoming stuck or deadlocked. Thus, to improve the transportation efficiency even in a maze-like restricted environment, we propose a deadlock avoidance method, called standby-based deadlock avoidance (SBDA). SBDA uses standby nodes determined in real-time using the articulation-point-finding algorithm, and the agent is guaranteed to stay there for a finite amount of time. We demonstrated that our proposed method outperforms a conventional approach. We also analyzed how the parameters used for selecting standby nodes affect the performance.

[58]  arXiv:2201.06021 (cross-list from cs.GT) [pdf, other]
Title: Rawlsian Fairness in Online Bipartite Matching: Two-sided, Group, and Individual
Subjects: Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI); Data Structures and Algorithms (cs.DS)

Online bipartite-matching platforms are ubiquitous and find applications in important areas such as crowdsourcing and ridesharing. In the most general form, the platform consists of three entities: two sides to be matched and a platform operator that decides the matching. The design of algorithms for such platforms has traditionally focused on the operator's (expected) profit. Recent reports have shown that certain demographic groups may receive less favorable treatment under pure profit maximization. As a result, a collection of online matching algorithms have been developed that give a fair treatment guarantee for one side of the market at the expense of a drop in the operator's profit. In this paper, we generalize the existing work to offer fair treatment guarantees to both sides of the market simultaneously, at a calculated worst case drop to operator profit. We consider group and individual Rawlsian fairness criteria. Moreover, our algorithms have theoretical guarantees and have adjustable parameters that can be tuned as desired to balance the trade-off between the utilities of the three sides. We also derive hardness results that give clear upper bounds over the performance of any algorithm.

[59]  arXiv:2201.06025 (cross-list from cs.CL) [pdf, other]
Title: COLD: A Benchmark for Chinese Offensive Language Detection
Comments: 10 pages
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Offensive language detection and prevention becomes increasing critical for maintaining a healthy social platform and the safe deployment of language models. Despite plentiful researches on toxic and offensive language problem in NLP, existing studies mainly focus on English, while few researches involve Chinese due to the limitation of resources. To facilitate Chinese offensive language detection and model evaluation, we collect COLDataset, a Chinese offensive language dataset containing 37k annotated sentences. With this high-quality dataset, we provide a strong baseline classifier, COLDetector, with 81% accuracy for offensive language detection. Furthermore, we also utilize the proposed \textsc{COLDetector} to study output offensiveness of popular Chinese language models (CDialGPT and CPM). We find that (1) CPM tends to generate more offensive output than CDialGPT, and (2) certain type of prompts, like anti-bias sentences, can trigger offensive outputs more easily.Altogether, our resources and analyses are intended to help detoxify the Chinese online communities and evaluate the safety performance of generative language models. Disclaimer: The paper contains example data that may be considered profane, vulgar, or offensive.

[60]  arXiv:2201.06026 (cross-list from cs.LG) [pdf, other]
Title: Toward Among-Device AI from On-Device AI with Stream Pipelines
Comments: to appear in ICSE 2022 SEIP (preprint)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)

Modern consumer electronic devices often provide intelligence services with deep neural networks. We have started migrating the computing locations of intelligence services from cloud servers (traditional AI systems) to the corresponding devices (on-device AI systems). On-device AI systems generally have the advantages of preserving privacy, removing network latency, and saving cloud costs. With the emergent of on-device AI systems having relatively low computing power, the inconsistent and varying hardware resources and capabilities pose difficulties. Authors' affiliation has started applying a stream pipeline framework, NNStreamer, for on-device AI systems, saving developmental costs and hardware resources and improving performance. We want to expand the types of devices and applications with on-device AI services products of both the affiliation and second/third parties. We also want to make each AI service atomic, re-deployable, and shared among connected devices of arbitrary vendors; we now have yet another requirement introduced as it always has been. The new requirement of "among-device AI" includes connectivity between AI pipelines so that they may share computing resources and hardware capabilities across a wide range of devices regardless of vendors and manufacturers. We propose extensions of the stream pipeline framework, NNStreamer, for on-device AI so that NNStreamer may provide among-device AI capability. This work is a Linux Foundation (LF AI and Data) open source project accepting contributions from the general public.

[61]  arXiv:2201.06028 (cross-list from cs.CL) [pdf, other]
Title: Natural Language Deduction through Search over Statement Compositions
Comments: 10 pages, 4 figures
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence entails a hypothesis. Existing methods primarily focus on end-to-end discriminative versions of this task, but less work has treated the generative version in which a model searches over the space of entailed statements to derive the hypothesis. We propose a system for natural language deduction that decomposes the task into separate steps coordinated by best-first search, producing a tree of intermediate conclusions that faithfully reflects the system's reasoning process. Our experiments demonstrate that the proposed system can better distinguish verifiable hypotheses from unverifiable ones and produce natural language explanations that are more internally consistent than those produced by an end-to-end T5 model.

[62]  arXiv:2201.06030 (cross-list from cs.CV) [pdf]
Title: Fully Convolutional Change Detection Framework with Generative Adversarial Network for Unsupervised, Weakly Supervised and Regional Supervised Change Detection
Comments: 13 pages, 19 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)

Deep learning for change detection is one of the current hot topics in the field of remote sensing. However, most end-to-end networks are proposed for supervised change detection, and unsupervised change detection models depend on traditional pre-detection methods. Therefore, we proposed a fully convolutional change detection framework with generative adversarial network, to conclude unsupervised, weakly supervised, regional supervised, and fully supervised change detection tasks into one framework. A basic Unet segmentor is used to obtain change detection map, an image-to-image generator is implemented to model the spectral and spatial variation between multi-temporal images, and a discriminator for changed and unchanged is proposed for modeling the semantic changes in weakly and regional supervised change detection task. The iterative optimization of segmentor and generator can build an end-to-end network for unsupervised change detection, the adversarial process between segmentor and discriminator can provide the solutions for weakly and regional supervised change detection, the segmentor itself can be trained for fully supervised task. The experiments indicate the effectiveness of the propsed framework in unsupervised, weakly supervised and regional supervised change detection. This paper provides theorical definitions for unsupervised, weakly supervised and regional supervised change detection tasks, and shows great potentials in exploring end-to-end network for remote sensing change detection.

[63]  arXiv:2201.06035 (cross-list from cs.IR) [pdf, other]
Title: Sequential Recommendation via Stochastic Self-Attention
Comments: Accepted by TheWebConf 2022
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Sequential recommendation models the dynamics of a user's previous behaviors in order to forecast the next item, and has drawn a lot of attention. Transformer-based approaches, which embed items as vectors and use dot-product self-attention to measure the relationship between items, demonstrate superior capabilities among existing sequential methods. However, users' real-world sequential behaviors are \textit{\textbf{uncertain}} rather than deterministic, posing a significant challenge to present techniques. We further suggest that dot-product-based approaches cannot fully capture \textit{\textbf{collaborative transitivity}}, which can be derived in item-item transitions inside sequences and is beneficial for cold start items. We further argue that BPR loss has no constraint on positive and sampled negative items, which misleads the optimization. We propose a novel \textbf{STO}chastic \textbf{S}elf-\textbf{A}ttention~(STOSA) to overcome these issues. STOSA, in particular, embeds each item as a stochastic Gaussian distribution, the covariance of which encodes the uncertainty. We devise a novel Wasserstein Self-Attention module to characterize item-item position-wise relationships in sequences, which effectively incorporates uncertainty into model training. Wasserstein attentions also enlighten the collaborative transitivity learning as it satisfies triangle inequality. Moreover, we introduce a novel regularization term to the ranking loss, which assures the dissimilarity between positive and the negative items. Extensive experiments on five real-world benchmark datasets demonstrate the superiority of the proposed model over state-of-the-art baselines, especially on cold start items. The code is available in \url{https://github.com/zfan20/STOSA}.

[64]  arXiv:2201.06044 (cross-list from cs.SE) [pdf, ps, other]
Title: A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning
Comments: Accepted for publication in ACM Transactions on Software Engineering and Methodology. Additional material available from a GitHub repository: this https URL
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)

Most software companies have extensive test suites and re-run parts of them continuously to ensure recent changes have no adverse effects. Since test suites are costly to execute, industry needs methods for test case prioritisation (TCP). Recently, TCP methods use machine learning (ML) to exploit the information known about the system under test (SUT) and its test cases. However, the value added by ML-based TCP methods should be critically assessed with respect to the cost of collecting the information. This paper analyses two decades of TCP research, and presents a taxonomy of 91 information attributes that have been used. The attributes are classified with respect to their information sources and the characteristics of their extraction process. Based on this taxonomy, TCP methods validated with industrial data and those applying ML are analysed in terms of information availability, attribute combination and definition of data features suitable for ML. Relying on a high number of information attributes, assuming easy access to SUT code and simplified testing environments are identified as factors that might hamper industrial applicability of ML-based TCP. The TePIA taxonomy provides a reference framework to unify terminology and evaluate alternatives considering the cost-benefit of the information attributes.

[65]  arXiv:2201.06061 (cross-list from cs.CV) [pdf, other]
Title: PETS-SWINF: A regression method that considers images with metadata based Neural Network for pawpularity prediction on 2021 Kaggle Competition "PetFinder.my"
Comments: 8 pages, 10 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Millions of stray animals suffer on the streets or are euthanized in shelters every day around the world. In order to better adopt stray animals, scoring the pawpularity (cuteness) of stray animals is very important, but evaluating the pawpularity of animals is a very labor-intensive thing. Consequently, there has been an urgent surge of interest to develop an algorithm that scores pawpularity of animals. However, the dataset in Kaggle not only has images, but also metadata describing images. Most methods basically focus on the most advanced image regression methods in recent years, but there is no good method to deal with the metadata of images. To address the above challenges, the paper proposes an image regression model called PETS-SWINF that considers metadata of the images. Our results based on a dataset of Kaggle competition, "PetFinder.my", show that PETS-SWINF has an advantage over only based images models. Our results shows that the RMSE loss of the proposed model on the test dataset is 17.71876 but 17.76449 without metadata. The advantage of the proposed method is that PETS-SWINF can consider both low-order and high-order features of metadata, and adaptively adjust the weights of the image model and the metadata model. The performance is promising as our leadboard score is ranked 15 out of 3545 teams (Gold medal) currently for 2021 Kaggle competition on the challenge "PetFinder.my".

[66]  arXiv:2201.06070 (cross-list from cs.CV) [pdf, other]
Title: ALA: Adversarial Lightness Attack via Naturalness-aware Regularizations
Comments: 8 pages
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Most researchers have tried to enhance the robustness of deep neural networks (DNNs) by revealing and repairing the vulnerability of DNNs with specialized adversarial examples. Parts of the attack examples have imperceptible perturbations restricted by Lp norm. However, due to their high-frequency property, the adversarial examples usually have poor transferability and can be defensed by denoising methods. To avoid the defects, some works make the perturbations unrestricted to gain better robustness and transferability. However, these examples usually look unnatural and alert the guards. To generate unrestricted adversarial examples with high image quality and good transferability, in this paper, we propose Adversarial Lightness Attack (ALA), a white-box unrestricted adversarial attack that focuses on modifying the lightness of the images. The shape and color of the samples, which are crucial to human perception, are barely influenced. To obtain adversarial examples with high image quality, we craft a naturalness-aware regularization. To achieve stronger transferability, we propose random initialization and non-stop attack strategy in the attack procedure. We verify the effectiveness of ALA on two popular datasets for different tasks (i.e., ImageNet for image classification and Places-365 for scene recognition). The experiments show that the generated adversarial examples have both strong transferability and high image quality. Besides, the adversarial examples can also help to improve the standard trained ResNet50 on defending lightness corruption.

[67]  arXiv:2201.06081 (cross-list from cs.GT) [pdf, other]
Title: Bayesian Promised Persuasion: Dynamic Forward-Looking Multiagent Delegation with Informational Burning
Subjects: Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI); Systems and Control (eess.SY)

This work studies a dynamic mechanism design problem in which a principal delegates decision makings to a group of privately-informed agents without the monetary transfer or burning. We consider that the principal privately possesses complete knowledge about the state transitions and study how she can use her private observation to support the incentive compatibility of the delegation via informational burning, a process we refer to as the looking-forward persuasion. The delegation mechanism is formulated in which the agents form belief hierarchies due to the persuasion and play a dynamic Bayesian game. We propose a novel randomized mechanism, known as Bayesian promised delegation (BPD), in which the periodic incentive compatibility is guaranteed by persuasions and promises of future delegations. We show that the BPD can achieve the same optimal social welfare as the original mechanism in stationary Markov perfect Bayesian equilibria. A revelation-principle-like design regime is established to show that the persuasion with belief hierarchies can be fully characterized by correlating the randomization of the agents' local BPD mechanisms with the persuasion as a direct recommendation of the future promises.

[68]  arXiv:2201.06118 (cross-list from cs.LG) [pdf, other]
Title: DeepCreativity: Measuring Creativity with Deep Learning Techniques
Comments: 12 pages, 2 figures
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

Measuring machine creativity is one of the most fascinating challenges in Artificial Intelligence. This paper explores the possibility of using generative learning techniques for automatic assessment of creativity. The proposed solution does not involve human judgement, it is modular and of general applicability. We introduce a new measure, namely DeepCreativity, based on Margaret Boden's definition of creativity as composed by value, novelty and surprise. We evaluate our methodology (and related measure) considering a case study, i.e., the generation of 19th century American poetry, showing its effectiveness and expressiveness.

[69]  arXiv:2201.06147 (cross-list from cs.LG) [pdf, other]
Title: Data augmentation through multivariate scenario forecasting in Data Centers using Generative Adversarial Networks
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

The Cloud paradigm is at a critical point in which the existing energy-efficiency techniques are reaching a plateau, while the computing resources demand at Data Center facilities continues to increase exponentially. The main challenge in achieving a global energy efficiency strategy based on Artificial Intelligence is that we need massive amounts of data to feed the algorithms. Nowadays, any optimization strategy must begin with data. However, companies with access to these large amounts of data decide not to share them because it could compromise their security. This paper proposes a time-series data augmentation methodology based on synthetic scenario forecasting within the Data Center. For this purpose, we will implement a powerful generative algorithm: Generative Adversarial Networks (GANs). The use of GANs will allow us to handle multivariate data and data from different natures (e.g., categorical). On the other hand, adapting Data Centers' operational management to the occurrence of sporadic anomalies is complicated due to the reduced frequency of failures in the system. Therefore, we also propose a methodology to increase the generated data variability by introducing on-demand anomalies. We validated our approach using real data collected from an operating Data Center, successfully obtaining forecasts of random scenarios with several hours of prediction. Our research will help to optimize the energy consumed in Data Centers, although the proposed methodology can be employed in any similar time-series-like problem.

[70]  arXiv:2201.06180 (cross-list from eess.SY) [pdf, other]
Title: Nonlinear Control Allocation: A Learning Based Approach
Subjects: Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Optimization and Control (math.OC)

Modern aircraft are designed with redundant control effectors to cater for fault tolerance and maneuverability requirements. This leads to an over-actuated aircraft which requires a control allocation scheme to distribute the control commands among effectors. Traditionally, optimization based control allocation schemes are used; however, for nonlinear allocation problems these methods require large computational resources. In this work, a novel ANN based nonlinear control allocation scheme is proposed. To start, a general nonlinear control allocation problem is posed in a different perspective to seek a function which maps desired moments to control effectors. Few important results on stability and performance of nonlinear allocation schemes in general and this ANN based allocation scheme, in particular, are presented. To demonstrate the efficacy of the proposed scheme, it is compared with standard quadratic programming based method for control allocation.

[71]  arXiv:2201.06192 (cross-list from cs.CV) [pdf, other]
Title: Fooling the Eyes of Autonomous Vehicles: Robust Physical Adversarial Examples Against Traffic Sign Recognition Systems
Comments: 17 pages, 15 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Adversarial Examples (AEs) can deceive Deep Neural Networks (DNNs) and have received a lot of attention recently. However, majority of the research on AEs is in the digital domain and the adversarial patches are static, which is very different from many real-world DNN applications such as Traffic Sign Recognition (TSR) systems in autonomous vehicles. In TSR systems, object detectors use DNNs to process streaming video in real time. From the view of object detectors, the traffic sign`s position and quality of the video are continuously changing, rendering the digital AEs ineffective in the physical world.
In this paper, we propose a systematic pipeline to generate robust physical AEs against real-world object detectors. Robustness is achieved in three ways. First, we simulate the in-vehicle cameras by extending the distribution of image transformations with the blur transformation and the resolution transformation. Second, we design the single and multiple bounding boxes filters to improve the efficiency of the perturbation training. Third, we consider four representative attack vectors, namely Hiding Attack, Appearance Attack, Non-Target Attack and Target Attack.
We perform a comprehensive set of experiments under a variety of environmental conditions, and considering illuminations in sunny and cloudy weather as well as at night. The experimental results show that the physical AEs generated from our pipeline are effective and robust when attacking the YOLO v5 based TSR system. The attacks have good transferability and can deceive other state-of-the-art object detectors. We launched HA and NTA on a brand-new 2021 model vehicle. Both attacks are successful in fooling the TSR system, which could be a life-threatening case for autonomous vehicles. Finally, we discuss three defense mechanisms based on image preprocessing, AEs detection, and model enhancing.

[72]  arXiv:2201.06213 (cross-list from cs.LG) [pdf, other]
Title: An Improved Reinforcement Learning Algorithm for Learning to Branch
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC)

Most combinatorial optimization problems can be formulated as mixed integer linear programming (MILP), in which branch-and-bound (B\&B) is a general and widely used method. Recently, learning to branch has become a hot research topic in the intersection of machine learning and combinatorial optimization. In this paper, we propose a novel reinforcement learning-based B\&B algorithm. Similar to offline reinforcement learning, we initially train on the demonstration data to accelerate learning massively. With the improvement of the training effect, the agent starts to interact with the environment with its learned policy gradually. It is critical to improve the performance of the algorithm by determining the mixing ratio between demonstration and self-generated data. Thus, we propose a prioritized storage mechanism to control this ratio automatically. In order to improve the robustness of the training process, a superior network is additionally introduced based on Double DQN, which always serves as a Q-network with competitive performance. We evaluate the performance of the proposed algorithm over three public research benchmarks and compare it against strong baselines, including three classical heuristics and one state-of-the-art imitation learning-based branching algorithm. The results show that the proposed algorithm achieves the best performance among compared algorithms and possesses the potential to improve B\&B algorithm performance continuously.

[73]  arXiv:2201.06216 (cross-list from math.OC) [pdf, other]
Title: Learning to Reformulate for Linear Programming
Subjects: Optimization and Control (math.OC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

It has been verified that the linear programming (LP) is able to formulate many real-life optimization problems, which can obtain the optimum by resorting to corresponding solvers such as OptVerse, Gurobi and CPLEX. In the past decades, a serial of traditional operation research algorithms have been proposed to obtain the optimum of a given LP in a fewer solving time. Recently, there is a trend of using machine learning (ML) techniques to improve the performance of above solvers. However, almost no previous work takes advantage of ML techniques to improve the performance of solver from the front end, i.e., the modeling (or formulation). In this paper, we are the first to propose a reinforcement learning-based reformulation method for LP to improve the performance of solving process. Using an open-source solver COIN-OR LP (CLP) as an environment, we implement the proposed method over two public research LP datasets and one large-scale LP dataset collected from practical production planning scenario. The evaluation results suggest that the proposed method can effectively reduce both the solving iteration number ($25\%\downarrow$) and the solving time ($15\%\downarrow$) over above datasets in average, compared to directly solving the original LP instances.

[74]  arXiv:2201.06219 (cross-list from cs.CL) [pdf, other]
Title: An Empirical Study on the Overlapping Problem of Open-Domain Dialogue Datasets
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Open-domain dialogue systems aim to converse with humans through text, and its research has heavily relied on benchmark datasets. In this work, we first identify the overlapping problem in DailyDialog and OpenSubtitles, two popular open-domain dialogue benchmark datasets. Our systematic analysis then shows that such overlapping can be exploited to obtain fake state-of-the-art performance. Finally, we address this issue by cleaning these datasets and setting up a proper data processing procedure for future research.

[75]  arXiv:2201.06220 (cross-list from cs.CV) [pdf]
Title: Face Detection in Extreme Conditions: A Machine-learning Approach
Comments: 6 pages, 9 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Face detection in unrestricted conditions has been a trouble for years due to various expressions, brightness, and coloration fringing. Recent studies show that deep learning knowledge of strategies can acquire spectacular performance inside the identification of different gadgets and patterns. This face detection in unconstrained surroundings is difficult due to various poses, illuminations, and occlusions. Figuring out someone with a picture has been popularized through the mass media. However, it's miles less sturdy to fingerprint or retina scanning. The latest research shows that deep mastering techniques can gain mind-blowing performance on those two responsibilities. In this paper, I recommend a deep cascaded multi-venture framework that exploits the inherent correlation among them to boost up their performance. In particular, my framework adopts a cascaded shape with 3 layers of cautiously designed deep convolutional networks that expect face and landmark region in a coarse-to-fine way. Besides, within the gaining knowledge of the procedure, I propose a new online tough sample mining method that can enhance the performance robotically without manual pattern choice.

[76]  arXiv:2201.06224 (cross-list from cs.IR) [pdf, other]
Title: Unintended Bias in Language Model-drivenConversational Recommendation
Comments: 12 pages, 7 figures
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)

Conversational Recommendation Systems (CRSs) have recently started to leverage pretrained language models (LM) such as BERTfor their ability to semantically interpret a wide range of preference statement variations. However, pretrained LMs are well-known to be prone to intrinsic biases in their training data, which may be exacerbated by biases embedded in domain-specific language data(e.g., user reviews) used to fine-tune LMs for CRSs. We study are recently introduced LM-driven recommendation backbone (termedLMRec) of a CRS to investigate how unintended bias i.e., language variations such as name references or indirect indicators of sexual orientation or location that should not affect recommendations manifests in significantly shifted price and category distributions of restaurant recommendations. The alarming results we observe strongly indicate that LMRec has learned to reinforce harmful stereotypes through its recommendations. For example, offhand mention of names associated with the black community significantly lowers the price distribution of recommended restaurants, while offhand mentions of common male-associated names lead to an increase in recommended alcohol-serving establishments. These and many related results presented in this work raise a red flag that advances in the language handling capability of LM-drivenCRSs do not come without significant challenges related to mitigating unintended bias in future deployed CRS assistants with a potential reach of hundreds of millions of end users.

[77]  arXiv:2201.06225 (cross-list from cs.CL) [pdf, other]
Title: ICLEA: Interactive Contrastive Learning for Self-supervised Entity Alignment
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Self-supervised entity alignment (EA) aims to link equivalent entities across different knowledge graphs (KGs) without seed alignments. The current SOTA self-supervised EA method draws inspiration from contrastive learning, originally designed in computer vision based on instance discrimination and contrastive loss, and suffers from two shortcomings. Firstly, it puts unidirectional emphasis on pushing sampled negative entities far away rather than pulling positively aligned pairs close, as is done in the well-established supervised EA. Secondly, KGs contain rich side information (e.g., entity description), and how to effectively leverage those information has not been adequately investigated in self-supervised EA. In this paper, we propose an interactive contrastive learning model for self-supervised EA. The model encodes not only structures and semantics of entities (including entity name, entity description, and entity neighborhood), but also conducts cross-KG contrastive learning by building pseudo-aligned entity pairs. Experimental results show that our approach outperforms previous best self-supervised results by a large margin (over 9% average improvement) and performs on par with previous SOTA supervised counterparts, demonstrating the effectiveness of the interactive contrastive learning for self-supervised EA.

[78]  arXiv:2201.06227 (cross-list from cs.LG) [pdf, other]
Title: Efficient DNN Training with Knowledge-Guided Layer Freezing
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)

Training deep neural networks (DNNs) is time-consuming. While most existing solutions try to overlap/schedule computation and communication for efficient training, this paper goes one step further by skipping computing and communication through DNN layer freezing. Our key insight is that the training progress of internal DNN layers differs significantly, and front layers often become well-trained much earlier than deep layers. To explore this, we first introduce the notion of training plasticity to quantify the training progress of internal DNN layers. Then we design KGT, a knowledge-guided DNN training system that employs semantic knowledge from a reference model to accurately evaluate individual layers' training plasticity and safely freeze the converged ones, saving their corresponding backward computation and communication. Our reference model is generated on the fly using quantization techniques and runs forward operations asynchronously on available CPUs to minimize the overhead. In addition, KGT caches the intermediate outputs of the frozen layers with prefetching to further skip the forward computation. Our implementation and testbed experiments with popular vision and language models show that KGT achieves 19%-43% training speedup w.r.t. the state-of-the-art without sacrificing accuracy.

[79]  arXiv:2201.06230 (cross-list from cs.CL) [pdf, other]
Title: Generalizable Neuro-symbolic Systems for Commonsense Question Answering
Comments: In Pascal Hitzler, Md Kamruzzaman Sarker (eds.), Neuro-Symbolic Artificial Intelligence: The State of the Art. Frontiers in Artificial Intelligence and Applications Vol. 342, IOS Press, Amsterdam, 2022. arXiv admin note: text overlap with arXiv:2003.04707
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

This chapter illustrates how suitable neuro-symbolic models for language understanding can enable domain generalizability and robustness in downstream tasks. Different methods for integrating neural language models and knowledge graphs are discussed. The situations in which this combination is most appropriate are characterized, including quantitative evaluation and qualitative error analysis on a variety of commonsense question answering benchmark datasets.

[80]  arXiv:2201.06286 (cross-list from cs.CL) [pdf, other]
Title: MuLVE, A Multi-Language Vocabulary Evaluation Data Set
Comments: Submitted to LREC 2022
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Vocabulary learning is vital to foreign language learning. Correct and adequate feedback is essential to successful and satisfying vocabulary training. However, many vocabulary and language evaluation systems perform on simple rules and do not account for real-life user learning data. This work introduces Multi-Language Vocabulary Evaluation Data Set (MuLVE), a data set consisting of vocabulary cards and real-life user answers, labeled indicating whether the user answer is correct or incorrect. The data source is user learning data from the Phase6 vocabulary trainer. The data set contains vocabulary questions in German and English, Spanish, and French as target language and is available in four different variations regarding pre-processing and deduplication. We experiment to fine-tune pre-trained BERT language models on the downstream task of vocabulary evaluation with the proposed MuLVE data set. The results provide outstanding results of > 95.5 accuracy and F2-score. The data set is available on the European Language Grid.

[81]  arXiv:2201.06289 (cross-list from cs.CV) [pdf, other]
Title: The CLEAR Benchmark: Continual LEArning on Real-World Imagery
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Continual learning (CL) is widely regarded as crucial challenge for lifelong AI. However, existing CL benchmarks, e.g. Permuted-MNIST and Split-CIFAR, make use of artificial temporal variation and do not align with or generalize to the real-world. In this paper, we introduce CLEAR, the first continual image classification benchmark dataset with a natural temporal evolution of visual concepts in the real world that spans a decade (2004-2014). We build CLEAR from existing large-scale image collections (YFCC100M) through a novel and scalable low-cost approach to visio-linguistic dataset curation. Our pipeline makes use of pretrained vision-language models (e.g. CLIP) to interactively build labeled datasets, which are further validated with crowd-sourcing to remove errors and even inappropriate images (hidden in original YFCC100M). The major strength of CLEAR over prior CL benchmarks is the smooth temporal evolution of visual concepts with real-world imagery, including both high-quality labeled data along with abundant unlabeled samples per time period for continual semi-supervised learning. We find that a simple unsupervised pre-training step can already boost state-of-the-art CL algorithms that only utilize fully-supervised data. Our analysis also reveals that mainstream CL evaluation protocols that train and test on iid data artificially inflate performance of CL system. To address this, we propose novel "streaming" protocols for CL that always test on the (near) future. Interestingly, streaming protocols (a) can simplify dataset curation since today's testset can be repurposed for tomorrow's trainset and (b) can produce more generalizable models with more accurate estimates of performance since all labeled data from each time-period is used for both training and testing (unlike classic iid train-test splits).

[82]  arXiv:2201.06317 (cross-list from cs.NE) [pdf, other]
Title: Language Model-Based Paired Variational Autoencoders for Robotic Language Learning
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Robotics (cs.RO)

Human infants learn language while interacting with their environment in which their caregivers may describe the objects and actions they perform. Similar to human infants, artificial agents can learn language while interacting with their environment. In this work, first, we present a neural model that bidirectionally binds robot actions and their language descriptions in a simple object manipulation scenario. Building on our previous Paired Variational Autoencoders (PVAE) model, we demonstrate the superiority of the variational autoencoder over standard autoencoders by experimenting with cubes of different colours, and by enabling the production of alternative vocabularies. Additional experiments show that the model's channel-separated visual feature extraction module can cope with objects of different shapes. Next, we introduce PVAE-BERT, which equips the model with a pretrained large-scale language model, i.e., Bidirectional Encoder Representations from Transformers (BERT), enabling the model to go beyond comprehending only the predefined descriptions that the network has been trained on; the recognition of action descriptions generalises to unconstrained natural language as the model becomes capable of understanding unlimited variations of the same descriptions. Our experiments suggest that using a pretrained language model as the language encoder allows our approach to scale up for real-world scenarios with instructions from human users.

[83]  arXiv:2201.06321 (cross-list from cs.LG) [pdf, other]
Title: Landscape of Neural Architecture Search across sensors: how much do they differ ?
Comments: This work is under review for a conference publication
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)

With the rapid rise of neural architecture search, the ability to understand its complexity from the perspective of a search algorithm is desirable. Recently, Traor\'e et al. have proposed the framework of Fitness Landscape Footprint to help describe and compare neural architecture search problems. It attempts at describing why a search strategy might be successful, struggle or fail on a target task. Our study leverages this methodology in the context of searching across sensors, including sensor data fusion. In particular, we apply the Fitness Landscape Footprint to the real-world image classification problem of So2Sat LCZ42, in order to identify the most beneficial sensor to our neural network hyper-parameter optimization problem. From the perspective of distributions of fitness, our findings indicate a similar behaviour of the search space for all sensors: the longer the training time, the larger the overall fitness, and more flatness in the landscapes (less ruggedness and deviation). Regarding sensors, the better the fitness they enable (Sentinel-2), the better the search trajectories (smoother, higher persistence). Results also indicate very similar search behaviour for sensors that can be decently fitted by the search space (Sentinel-2 and fusion).

[84]  arXiv:2201.06336 (cross-list from cs.LG) [pdf, other]
Title: Fair Group-Shared Representations with Normalizing Flows
Comments: ICLR-21 Workshop on Responsible AI
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

The issue of fairness in machine learning stems from the fact that historical data often displays biases against specific groups of people which have been underprivileged in the recent past, or still are. In this context, one of the possible approaches is to employ fair representation learning algorithms which are able to remove biases from data, making groups statistically indistinguishable. In this paper, we instead develop a fair representation learning algorithm which is able to map individuals belonging to different groups in a single group. This is made possible by training a pair of Normalizing Flow models and constraining them to not remove information about the ground truth by training a ranking or classification model on top of them. The overall, ``chained'' model is invertible and has a tractable Jacobian, which allows to relate together the probability densities for different groups and ``translate'' individuals from one group to another. We show experimentally that our methodology is competitive with other fair representation learning algorithms. Furthermore, our algorithm achieves stronger invariance w.r.t. the sensitive attribute.

[85]  arXiv:2201.06348 (cross-list from cs.CL) [pdf]
Title: Chatbot System Architecture
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

The conversational agents is one of the most interested topics in computer science field in the recent decade. Which can be composite from more than one subject in this field, which you need to apply Natural Language Processing Concepts and some Artificial Intelligence Techniques such as Deep Learning methods to make decision about how should be the response. This paper is dedicated to discuss the system architecture for the conversational agent and explain each component in details.

[86]  arXiv:2201.06398 (cross-list from cs.DC) [pdf, other]
Title: Efficient Data-Plane Memory Scheduling for In-Network Aggregation
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)

As the scale of distributed training grows, communication becomes a bottleneck. To accelerate the communication, recent works introduce In-Network Aggregation (INA), which moves the gradients summation into network middle-boxes, e.g., programmable switches to reduce the traffic volume. However, switch memory is scarce compared to the volume of gradients transmitted in distributed training. Although literature applies methods like pool-based streaming or dynamic sharing to tackle the mismatch, switch memory is still a potential performance bottleneck. Furthermore, we observe the under-utilization of switch memory due to the synchronization requirement for aggregator deallocation in recent works. To improve the switch memory utilization, we propose ESA, an $\underline{E}$fficient Switch Memory $\underline{S}$cheduler for In-Network $\underline{A}$ggregation. At its cores, ESA enforces the preemptive aggregator allocation primitive and introduces priority scheduling at the data-plane, which improves the switch memory utilization and average job completion time (JCT). Experiments show that ESA can improve the average JCT by up to $1.35\times$.

[87]  arXiv:2201.06427 (cross-list from cs.CV) [pdf, other]
Title: Masked Faces with Faced Masks
Comments: 8 pages
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Modern face recognition systems (FRS) still fall short when the subjects are wearing facial masks, a common theme in the age of respiratory pandemics. An intuitive partial remedy is to add a mask detector to flag any masked faces so that the FRS can act accordingly for those low-confidence masked faces. In this work, we set out to investigate the potential vulnerability of such FRS, equipped with a mask detector, on large-scale masked faces. As existing face recognizers and mask detectors have high performance in their respective tasks, it is a challenge to simultaneously fool them and preserve the transferability of the attack. To this end, we devise realistic facial masks that exhibit partial face patterns (i.e., faced masks) and stealthily add adversarial textures that can not only lead to significant performance deterioration of the SOTA deep learning-based FRS, but also remain undetected by the SOTA facial mask detector, thus successfully fooling both systems at the same time. The proposed method unveils the vulnerability of the FRS when dealing with masked faces wearing faced masks.

[88]  arXiv:2201.06444 (cross-list from cs.LG) [pdf, other]
Title: Black-box error diagnosis in deep neural networks: a survey of tools
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Software Engineering (cs.SE)

The application of Deep Neural Networks (DNNs) to a broad variety of tasks demands methods for coping with the complex and opaque nature of these architectures. The analysis of performance can be pursued in two ways. On one side, model interpretation techniques aim at "opening the box" to assess the relationship between the input, the inner layers, and the output. For example, saliency and attention models exploit knowledge of the architecture to capture the essential regions of the input that have the most impact on the inference process and output. On the other hand, models can be analysed as "black boxes", e.g., by associating the input samples with extra annotations that do not contribute to model training but can be exploited for characterizing the model response. Such performance-driven meta-annotations enable the detailed characterization of performance metrics and errors and help scientists identify the features of the input responsible for prediction failures and focus their model improvement efforts. This paper presents a structured survey of the tools that support the "black box" analysis of DNNs and discusses the gaps in the current proposals and the relevant future directions in this research field.

[89]  arXiv:2201.06468 (cross-list from cs.LG) [pdf, other]
Title: Chaining Value Functions for Off-Policy Learning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)

To accumulate knowledge and improve its policy of behaviour, a reinforcement learning agent can learn `off-policy' about policies that differ from the policy used to generate its experience. This is important to learn counterfactuals, or because the experience was generated out of its own control. However, off-policy learning is non-trivial, and standard reinforcement-learning algorithms can be unstable and divergent.
In this paper we discuss a novel family of off-policy prediction algorithms which are convergent by construction. The idea is to first learn on-policy about the data-generating behaviour, and then bootstrap an off-policy value estimate on this on-policy estimate, thereby constructing a value estimate that is partially off-policy. This process can be repeated to build a chain of value functions, each time bootstrapping a new estimate on the previous estimate in the chain. Each step in the chain is stable and hence the complete algorithm is guaranteed to be stable. Under mild conditions this comes arbitrarily close to the off-policy TD solution when we increase the length of the chain. Hence it can compute the solution even in cases where off-policy TD diverges.
We prove that the proposed scheme is convergent and corresponds to an iterative decomposition of the inverse key matrix. Furthermore it can be interpreted as estimating a novel objective -- that we call a `k-step expedition' -- of following the target policy for finitely many steps before continuing indefinitely with the behaviour policy. Empirically we evaluate the idea on challenging MDPs such as Baird's counter example and observe favourable results.

[90]  arXiv:2201.06499 (cross-list from cs.CL) [pdf, ps, other]
Title: RuMedBench: A Russian Medical Language Understanding Benchmark
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

The paper describes the open Russian medical language understanding benchmark covering several task types (classification, question answering, natural language inference, named entity recognition) on a number of novel text sets. Given the sensitive nature of the data in healthcare, such a benchmark partially closes the problem of Russian medical dataset absence. We prepare the unified format labeling, data split, and evaluation metrics for new tasks. The remaining tasks are from existing datasets with a few modifications. A single-number metric expresses a model's ability to cope with the benchmark. Moreover, we implement several baseline models, from simple ones to neural networks with transformer architecture, and release the code. Expectedly, the more advanced models yield better performance, but even a simple model is enough for a decent result in some tasks. Furthermore, for all tasks, we provide a human evaluation. Interestingly the models outperform humans in the large-scale classification tasks. However, the advantage of natural intelligence remains in the tasks requiring more knowledge and reasoning.

[91]  arXiv:2201.06500 (cross-list from cs.LG) [pdf, other]
Title: Growing Neural Network with Shared Parameter
Authors: Ruilin Tong
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

We propose a general method for growing neural network with shared parameter by matching trained network to new input. By leveraging Hoeffding's inequality, we provide a theoretical base for improving performance by adding subnetwork to existing network. With the theoretical base of adding new subnetwork, we implement a matching method to apply trained subnetwork of existing network to new input. Our method has shown the ability to improve performance with higher parameter efficiency. It can also be applied to trans-task case and realize transfer learning by changing the combination of subnetworks without training on new task.

[92]  arXiv:2201.06517 (cross-list from cs.SI) [pdf]
Title: Demographic Confounding Causes Extreme Instances of Lifestyle Politics on Facebook
Comments: 29 pages (27 body, 2 supplemental material), 14 figures (12 body, 2 supplemental material), 2 tables
Subjects: Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR); Physics and Society (physics.soc-ph)

Lifestyle politics emerge when activities that have no substantive relevance to ideology become politically aligned and polarized. Homophily and social influence are able generate these fault lines on their own; however, social identities from demographics may serve as coordinating mechanisms through which lifestyle politics are mobilized are spread. Using a dataset of 137,661,886 observations from 299,327 Facebook interests aggregated across users of different racial/ethnic, education, age, gender, and income demographics, we find that the most extreme instances of lifestyle politics are those which are highly confounded by demographics such as race/ethnicity (e.g., Black artists and performers). After adjusting political alignment for demographic effects, lifestyle politics decreased by 27.36% toward the political "center" and demographically confounded interests were no longer among the most polarized interests. Instead, after demographic deconfounding, we found that the most liberal interests included electric cars, Planned Parenthood, and liberal satire while the most conservative interests included the Republican Party and conservative commentators. We validate our measures of political alignment and lifestyle politics using the General Social Survey and find similar demographic entanglements with lifestyle politics existed before social media such as Facebook were ubiquitous, giving us strong confidence that our results are not due to echo chambers or filter bubbles. Likewise, since demographic characteristics exist prior to ideological values, we argue that the demographic confounding we observe is causally responsible for the extreme instances of lifestyle politics that we find among the aggregated interests. We conclude our paper by relating our results to Simpson's paradox, cultural omnivorousness, and network autocorrelation.

[93]  arXiv:2201.06523 (cross-list from cs.LG) [pdf]
Title: Patterns of near-crash events in a naturalistic driving dataset: applying rules mining
Journal-ref: Accident Analysis & Prevention (2021)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Databases (cs.DB); Information Retrieval (cs.IR)

This study aims to explore the associations between near-crash events and road geometry and trip features by investigating a naturalistic driving dataset and a corresponding roadway inventory dataset using an association rule mining method.

[94]  arXiv:2201.06539 (cross-list from cs.RO) [pdf, other]
Title: Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning
Comments: IEEE Robotics and Automation Letters (RA-L)
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)

It can be difficult to autonomously produce driver behavior so that it appears natural to other traffic participants. Through Inverse Reinforcement Learning (IRL), we can automate this process by learning the underlying reward function from human demonstrations. We propose a new IRL algorithm that learns a goal-conditioned spatiotemporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task without any hand-designing or hand-tuning of the cost function. We evaluate our proposed Goal-conditioned SpatioTemporal Zeroing Maximum Entropy Deep IRL (GSTZ)-MEDIRL framework together with MPC in the CARLA simulator for autonomous driving, lane keeping, and lane changing tasks in a challenging dense traffic highway scenario. Our proposed methods show higher success rates compared to other baseline methods including behavior cloning, state-of-the-art RL policies, and MPC with a learning-based behavior prediction model.

[95]  arXiv:2201.06556 (cross-list from cs.SI) [pdf]
Title: Millions of Co-purchases and Reviews Reveal the Spread of Polarization and Lifestyle Politics across Online Markets
Comments: 25 pages (21 body, 4 supplemental material), 10 figures (4 body, 6 supplemental material), 5 tables (3 body, 2 supplemental material)
Subjects: Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR); Physics and Society (physics.soc-ph)

Polarization in America has reached a high point as markets are also becoming polarized. Existing research, however, focuses on specific market segments and products and has not evaluated this trend's full breadth. If such fault lines do spread into other segments that are not explicitly political, it would indicate the presence of lifestyle politics -- when ideas and behaviors not inherently political become politically aligned through their connections with explicitly political things. We study the pervasiveness of polarization and lifestyle politics over different product segments in a diverse market and test the extent to which consumer- and platform-level network effects and morality may explain lifestyle politics. Specifically, using graph and language data from Amazon (82.5M reviews of 9.5M products and product and category metadata from 1996-2014), we sample 234.6 million relations among 21.8 million market entities to find product categories that are most politically relevant, aligned, and polarized. We then extract moral values present in reviews' text and use these data and other reviewer-, product-, and category-level data to test whether individual- and platform- level network factors explain lifestyle politics better than products' implicit morality. We find pervasive lifestyle politics. Cultural products are 4 times more polarized than any other segment, products' political attributes have up to 3.7 times larger associations with lifestyle politics than author-level covariates, and morality has statistically significant but relatively small correlations with lifestyle politics. Examining lifestyle politics in these contexts helps us better understand the extent and root of partisan differences, why Americans may be so polarized, and how this polarization affects market systems.

[96]  arXiv:2201.06573 (cross-list from cs.CL) [pdf, other]
Title: PerPaDa: A Persian Paraphrase Dataset based on Implicit Crowdsourcing Data Collection
Comments: Submitted to LREC 2022
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

In this paper we introduce PerPaDa, a Persian paraphrase dataset that is collected from users' input in a plagiarism detection system. As an implicit crowdsourcing experience, we have gathered a large collection of original and paraphrased sentences from Hamtajoo; a Persian plagiarism detection system, in which users try to conceal cases of text re-use in their documents by paraphrasing and re-submitting manuscripts for analysis. The compiled dataset contains 2446 instances of paraphrasing. In order to improve the overall quality of the collected data, some heuristics have been used to exclude sentences that don't meet the proposed criteria. The introduced corpus is much larger than the available datasets for the task of paraphrase identification in Persian. Moreover, there is less bias in the data compared to the similar datasets, since the users did not try some fixed predefined rules in order to generate similar texts to their original inputs.

[97]  arXiv:2201.06578 (cross-list from cs.CV) [pdf, other]
Title: Collapse by Conditioning: Training Class-conditional GANs with Limited Data
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Class-conditioning offers a direct means of controlling a Generative Adversarial Network (GAN) based on a discrete input variable. While necessary in many applications, the additional information provided by the class labels could even be expected to benefit the training of the GAN itself. Contrary to this belief, we observe that class-conditioning causes mode collapse in limited data settings, where unconditional learning leads to satisfactory generative ability. Motivated by this observation, we propose a training strategy for conditional GANs (cGANs) that effectively prevents the observed mode-collapse by leveraging unconditional learning. Our training strategy starts with an unconditional GAN and gradually injects conditional information into the generator and the objective function. The proposed method for training cGANs with limited data results not only in stable training but also in generating high-quality images, thanks to the early-stage exploitation of the shared information across classes. We analyze the aforementioned mode collapse problem in comprehensive experiments on four datasets. Our approach demonstrates outstanding results compared with state-of-the-art methods and established baselines. The code is available at: https://github.com/mshahbazi72/transitional-cGAN

[98]  arXiv:2201.06619 (cross-list from cs.MA) [pdf, other]
Title: Planning Not to Talk: Multiagent Systems that are Robust to Communication Loss
Comments: Accepted at AAMAS 2022
Subjects: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI)

In a cooperative multiagent system, a collection of agents executes a joint policy in order to achieve some common objective. The successful deployment of such systems hinges on the availability of reliable inter-agent communication. However, many sources of potential disruption to communication exist in practice, such as radio interference, hardware failure, and adversarial attacks. In this work, we develop joint policies for cooperative multiagent systems that are robust to potential losses in communication. More specifically, we develop joint policies for cooperative Markov games with reach-avoid objectives. First, we propose an algorithm for the decentralized execution of joint policies during periods of communication loss. Next, we use the total correlation of the state-action process induced by a joint policy as a measure of the intrinsic dependencies between the agents. We then use this measure to lower-bound the performance of a joint policy when communication is lost. Finally, we present an algorithm that maximizes a proxy to this lower bound in order to synthesize minimum-dependency joint policies that are robust to communication loss. Numerical experiments show that the proposed minimum-dependency policies require minimal coordination between the agents while incurring little to no loss in performance; the total correlation value of the synthesized policy is one fifth of the total correlation value of the baseline policy which does not take potential communication losses into account. As a result, the performance of the minimum-dependency policies remains consistently high regardless of whether or not communication is available. By contrast, the performance of the baseline policy decreases by twenty percent when communication is lost.

[99]  arXiv:2201.06626 (cross-list from math.NA) [pdf, other]
Title: Closed-Loop ACAS Xu NNCS is Unsafe: Quantized State Backreachability for Verification
Subjects: Numerical Analysis (math.NA); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)

ACAS Xu is an air-to-air collision avoidance system designed for unmanned aircraft that issues horizontal turn advisories to avoid an intruder aircraft. Due the use of a large lookup table in the design, a neural network compression of the policy was proposed. Analysis of this system has spurred a significant body of research in the formal methods community on neural network verification. While many powerful methods have been developed, most work focuses on open-loop properties of the networks, rather than the main point of the system -- collision avoidance -- which requires closed-loop analysis.
In this work, we develop a technique to verify a closed-loop approximation of ACAS Xu using state quantization and backreachability. We use favorable assumptions for the analysis -- perfect sensor information, instant following of advisories, ideal aircraft maneuvers and an intruder that only flies straight. When the method fails to prove the system is safe, we refine the quantization parameters until generating counterexamples where the original (non-quantized) system also has collisions.

[100]  arXiv:2201.06653 (cross-list from cs.LG) [pdf, other]
Title: Data-Centric Machine Learning in the Legal Domain
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Machine learning research typically starts with a fixed data set created early in the process. The focus of the experiments is finding a model and training procedure that result in the best possible performance in terms of some selected evaluation metric. This paper explores how changes in a data set influence the measured performance of a model. Using three publicly available data sets from the legal domain, we investigate how changes to their size, the train/test splits, and the human labelling accuracy impact the performance of a trained deep learning classifier. We assess the overall performance (weighted average) as well as the per-class performance. The observed effects are surprisingly pronounced, especially when the per-class performance is considered. We investigate how "semantic homogeneity" of a class, i.e., the proximity of sentences in a semantic embedding space, influences the difficulty of its classification. The presented results have far reaching implications for efforts related to data collection and curation in the field of AI & Law. The results also indicate that enhancements to a data set could be considered, alongside the advancement of the ML models, as an additional path for increasing classification performance on various tasks in AI & Law. Finally, we discuss the need for an established methodology to assess the potential effects of data set properties.

[101]  arXiv:2201.06700 (cross-list from cs.NE) [pdf, other]
Title: Benchmarking Subset Selection from Large Candidate Solution Sets in Evolutionary Multi-objective Optimization
Comments: This paper is currently under review
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)

In the evolutionary multi-objective optimization (EMO) field, the standard practice is to present the final population of an EMO algorithm as the output. However, it has been shown that the final population often includes solutions which are dominated by other solutions generated and discarded in previous generations. Recently, a new EMO framework has been proposed to solve this issue by storing all the non-dominated solutions generated during the evolution in an archive and selecting a subset of solutions from the archive as the output. The key component in this framework is the subset selection from the archive which usually stores a large number of candidate solutions. However, most studies on subset selection focus on small candidate solution sets for environmental selection. There is no benchmark test suite for large-scale subset selection. This paper aims to fill this research gap by proposing a benchmark test suite for subset selection from large candidate solution sets, and comparing some representative methods using the proposed test suite. The proposed test suite together with the benchmarking studies provides a baseline for researchers to understand, use, compare, and develop subset selection methods in the EMO field.

[102]  arXiv:2201.06703 (cross-list from cs.ET) [pdf, other]
Title: Design Space Exploration of Dense and Sparse Mapping Schemes for RRAM Architectures
Comments: Accepted at 2022 IEEE International Symposium on Circuits and Systems (ISCAS)
Subjects: Emerging Technologies (cs.ET); Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR)

The impact of device and circuit-level effects in mixed-signal Resistive Random Access Memory (RRAM) accelerators typically manifest as performance degradation of Deep Learning (DL) algorithms, but the degree of impact varies based on algorithmic features. These include network architecture, capacity, weight distribution, and the type of inter-layer connections. Techniques are continuously emerging to efficiently train sparse neural networks, which may have activation sparsity, quantization, and memristive noise. In this paper, we present an extended Design Space Exploration (DSE) methodology to quantify the benefits and limitations of dense and sparse mapping schemes for a variety of network architectures. While sparsity of connectivity promotes less power consumption and is often optimized for extracting localized features, its performance on tiled RRAM arrays may be more susceptible to noise due to under-parameterization, when compared to dense mapping schemes. Moreover, we present a case study quantifying and formalizing the trade-offs of typical non-idealities introduced into 1-Transistor-1-Resistor (1T1R) tiled memristive architectures and the size of modular crossbar tiles using the CIFAR-10 dataset.

[103]  arXiv:2201.06707 (cross-list from cs.NE) [pdf, other]
Title: Learning to Approximate: Auto Direction Vector Set Generation for Hypervolume Contribution Approximation
Comments: This paper is currently under review
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)

Hypervolume contribution is an important concept in evolutionary multi-objective optimization (EMO). It involves in hypervolume-based EMO algorithms and hypervolume subset selection algorithms. Its main drawback is that it is computationally expensive in high-dimensional spaces, which limits its applicability to many-objective optimization. Recently, an R2 indicator variant (i.e., $R_2^{\text{HVC}}$ indicator) is proposed to approximate the hypervolume contribution. The $R_2^{\text{HVC}}$ indicator uses line segments along a number of direction vectors for hypervolume contribution approximation. It has been shown that different direction vector sets lead to different approximation quality. In this paper, we propose \textit{Learning to Approximate (LtA)}, a direction vector set generation method for the $R_2^{\text{HVC}}$ indicator. The direction vector set is automatically learned from training data. The learned direction vector set can then be used in the $R_2^{\text{HVC}}$ indicator to improve its approximation quality. The usefulness of the proposed LtA method is examined by comparing it with other commonly-used direction vector set generation methods for the $R_2^{\text{HVC}}$ indicator. Experimental results suggest the superiority of LtA over the other methods for generating high quality direction vector sets.

[104]  arXiv:2201.06717 (cross-list from cs.LG) [pdf, other]
Title: GTrans: Spatiotemporal Autoregressive Transformer with Graph Embeddings for Nowcasting Extreme Events
Authors: Bo Feng, Geoffrey Fox
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Spatiotemporal time series nowcasting should preserve temporal and spatial dynamics in the sense that generated new sequences from models respect the covariance relationship from history. Conventional feature extractors are built with deep convolutional neural networks (CNN). However, CNN models have limits to image-like applications where data can be formed with high-dimensional arrays. In contrast, applications in social networks, road traffic, physics, and chemical property prediction where data features can be organized with nodes and edges of graphs. Transformer architecture is an emerging method for predictive models, bringing high accuracy and efficiency due to attention mechanism design. This paper proposes a spatiotemporal model, namely GTrans, that transforms data features into graph embeddings and predicts temporal dynamics with a transformer model. According to our experiments, we demonstrate that GTrans can model spatial and temporal dynamics and nowcasts extreme events for datasets. Furthermore, in all the experiments, GTrans can achieve the highest F1 and F2 scores in binary-class prediction tests than the baseline models.

[105]  arXiv:2201.06721 (cross-list from cs.CL) [pdf, other]
Title: Selecting and combining complementary feature representations and classifiers for hate speech detection
Comments: acceped for publication on the Online Social Networks and Media (OSNEM) journal
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI)

Hate speech is a major issue in social networks due to the high volume of data generated daily. Recent works demonstrate the usefulness of machine learning (ML) in dealing with the nuances required to distinguish between hateful posts from just sarcasm or offensive language. Many ML solutions for hate speech detection have been proposed by either changing how features are extracted from the text or the classification algorithm employed. However, most works consider only one type of feature extraction and classification algorithm. This work argues that a combination of multiple feature extraction techniques and different classification models is needed. We propose a framework to analyze the relationship between multiple feature extraction and classification techniques to understand how they complement each other. The framework is used to select a subset of complementary techniques to compose a robust multiple classifiers system (MCS) for hate speech detection. The experimental study considering four hate speech classification datasets demonstrates that the proposed framework is a promising methodology for analyzing and designing high-performing MCS for this task. MCS system obtained using the proposed framework significantly outperforms the combination of all models and the homogeneous and heterogeneous selection heuristics, demonstrating the importance of having a proper selection scheme. Source code, figures, and dataset splits can be found in the GitHub repository: https://github.com/Menelau/Hate-Speech-MCS.

[106]  arXiv:2201.06740 (cross-list from cs.CV) [pdf, other]
Title: Convolutional Cobweb: A Model of Incremental Learning from 2D Images
Comments: 14 pages, 6 figures, Presented at Advances in Cognitive Systems 2021
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

This paper presents a new concept formation approach that supports the ability to incrementally learn and predict labels for visual images. This work integrates the idea of convolutional image processing, from computer vision research, with a concept formation approach that is based on psychological studies of how humans incrementally form and use concepts. We experimentally evaluate this new approach by applying it to an incremental variation of the MNIST digit recognition task. We compare its performance to Cobweb, a concept formation approach that does not support convolutional processing, as well as two convolutional neural networks that vary in the complexity of their convolutional processing. This work represents a first step towards unifying modern computer vision ideas with classical concept formation research.

[107]  arXiv:2201.06757 (cross-list from cs.CL) [pdf, ps, other]
Title: Dilated Convolutional Neural Networks for Lightweight Diacritics Restoration
Comments: 7 pages, 2 figures
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Diacritics restoration has become a ubiquitous task in the Latin-alphabet-based English-dominated Internet language environment. In this paper, we describe a small footprint 1D dilated convolution-based approach which operates on a character-level. We find that solutions based on 1D dilated convolutional neural networks are competitive alternatives to models based on recursive neural networks or linguistic modeling for the task of diacritics restoration. Our solution surpasses the performance of similarly sized models and is also competitive with larger models. A special feature of our solution is that it even runs locally in a web browser. We also provide a working example of this browser-based implementation. Our model is evaluated on different corpora, with emphasis on the Hungarian language. We performed comparative measurements about the generalization power of the model in relation to three Hungarian corpora. We also analyzed the errors to understand the limitation of corpus-based self-supervised training.

[108]  arXiv:2201.06776 (cross-list from cs.CV) [pdf, other]
Title: Pruning-aware Sparse Regularization for Network Pruning
Comments: 11 pages, 5 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Structural neural network pruning aims to remove the redundant channels in the deep convolutional neural networks (CNNs) by pruning the filters of less importance to the final output accuracy. To reduce the degradation of performance after pruning, many methods utilize the loss with sparse regularization to produce structured sparsity. In this paper, we analyze these sparsity-training-based methods and find that the regularization of unpruned channels is unnecessary. Moreover, it restricts the network's capacity, which leads to under-fitting. To solve this problem, we propose a novel pruning method, named MaskSparsity, with pruning-aware sparse regularization. MaskSparsity imposes the fine-grained sparse regularization on the specific filters selected by a pruning mask, rather than all the filters of the model. Before the fine-grained sparse regularization of MaskSparity, we can use many methods to get the pruning mask, such as running the global sparse regularization. MaskSparsity achieves 63.03%-FLOPs reduction on ResNet-110 by removing 60.34% of the parameters, with no top-1 accuracy loss on CIFAR-10. On ILSVRC-2012, MaskSparsity reduces more than 51.07% FLOPs on ResNet-50, with only a loss of 0.76% in the top-1 accuracy.
The code is released at https://github.com/CASIA-IVA-Lab/MaskSparsity. Moreover, we have integrated the code of MaskSparity into a PyTorch pruning toolkit, EasyPruner, at https://gitee.com/casia_iva_engineer/easypruner.

[109]  arXiv:2201.06843 (cross-list from cs.DC) [pdf, other]
Title: Surrogate-assisted distributed swarm optimisation for computationally expensive models
Comments: under review
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)

Advances in parallel and distributed computing have enabled efficient implementation of the distributed swarm and evolutionary algorithms for complex and computationally expensive models. Evolutionary algorithms provide gradient-free optimisation which is beneficial for models that do not have such information available, for instance, geoscientific landscape evolution models. However, such models are so computationally expensive that even distributed swarm and evolutionary algorithms with the power of parallel computing struggle. We need to incorporate efficient strategies such as surrogate assisted optimisation that further improves their performance; however, this becomes a challenge given parallel processing and inter-process communication for implementing surrogate training and prediction. In this paper, we implement surrogate-based estimation of fitness evaluation in distributed swarm optimisation over a parallel computing architecture. Our results demonstrate very promising results for benchmark functions and geoscientific landscape evolution models. We obtain a reduction in computationally time while retaining optimisation solution accuracy through the use of surrogates in a parallel computing environment.

[110]  arXiv:2201.06857 (cross-list from cs.CV) [pdf, other]
Title: RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Recently, self-supervised vision transformers have attracted unprecedented attention for their impressive representation learning ability. However, the dominant method, contrastive learning, mainly relies on an instance discrimination pretext task, which learns a global understanding of the image. This paper incorporates local feature learning into self-supervised vision transformers via Reconstructive Pre-training (RePre). Our RePre extends contrastive frameworks by adding a branch for reconstructing raw image pixels in parallel with the existing contrastive objective. RePre is equipped with a lightweight convolution-based decoder that fuses the multi-hierarchy features from the transformer encoder. The multi-hierarchy features provide rich supervisions from low to high semantic information, which are crucial for our RePre. Our RePre brings decent improvements on various contrastive frameworks with different vision transformer architectures. Transfer performance in downstream tasks outperforms supervised pre-training and state-of-the-art (SOTA) self-supervised counterparts.

[111]  arXiv:2201.06860 (cross-list from cs.SE) [pdf]
Title: A Knowledge-driven Business Process Analysis Canvas
Comments: 15 pages, 3 fugures, 4 tables
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)

Business process (BP) analysis represents a first key phase of information system development. It consists in the gathering of domain knowledge and its organization to be later used in the software development, and beyond (e.g., for Business Process Reengineering). The quality of the developed information system largely depends on how the BP analysis has been carried out and the quality of the produced requirement specification documents. Despite the fact that the issue is on the table for decades, business process analysis is still a critical phase of information systems development. One promising strategy is an early and more important involvement of business experts in the BP analysis. This paper presents a methodology that aims at an early involvement of business experts while providing a formal grounding that guarantees the quality of the produced specifications. To this end, we propose the Business Process Analysis Canvas, a knowledge framework organized in eight knowledge sections aimed at supporting the business expert in carrying out the analysis, eventually yielding a BP analysis Ontology.

[112]  arXiv:2201.06877 (cross-list from cs.NE) [pdf, other]
Title: Frequent Itemset-driven Search for Finding Minimum Node Separators in Complex Networks
Comments: 13 pages, 4 figures
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)

Finding an optimal set of critical nodes in a complex network has been a long-standing problem in the fields of both artificial intelligence and operations research. Potential applications include epidemic control, network security, carbon emission monitoring, emergence response, drug design, and vulnerability assessment. In this work, we consider the problem of finding a minimal node separator whose removal separates a graph into multiple different connected components with fewer than a limited number of vertices in each component. To solve it, we propose a frequent itemset-driven search approach, which integrates the concept of frequent itemset mining in data mining into the well-known memetic search framework. Starting from a high-quality population built by the solution construction and population repair procedures, it iteratively employs the frequent itemset recombination operator (to generate promising offspring solution based on itemsets that frequently occur in high-quality solutions), tabu search-based simulated annealing (to find high-quality local optima), population repair procedure (to modify the population), and rank-based population management strategy (to guarantee a healthy population). Extensive evaluations on 50 widely used benchmark instances show that it significantly outperforms state-of-the-art algorithms. In particular, it discovers 29 new upper bounds and matches 18 previous best-known bounds. Finally, experimental analyses are performed to confirm the effectiveness of key algorithmic modules of the proposed method.

[113]  arXiv:2201.06885 (cross-list from cs.CL) [pdf, other]
Title: Mining Fine-grained Semantics via Graph Neural Networks for Evidence-based Fake News Detection
Comments: Accepted by TheWebConf 2022
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

The prevalence and perniciousness of fake news has been a critical issue on the Internet, which stimulates the development of automatic fake news detection in turn. In this paper, we focus on the evidence-based fake news detection, where several evidences are utilized to probe the veracity of news (i.e., a claim). Most previous methods first employ sequential models to embed the semantic information and then capture the claim-evidence interaction based on different attention mechanisms. Despite their effectiveness, they still suffer from two main weaknesses. Firstly, due to the inherent drawbacks of sequential models, they fail to integrate the relevant information that is scattered far apart in evidences for veracity checking. Secondly, they neglect much redundant information contained in evidences that may be useless or even harmful. To solve these problems, we propose a unified Graph-based sEmantic sTructure mining framework, namely GET in short. Specifically, different from the existing work that treats claims and evidences as sequences, we model them as graph-structured data and capture the long-distance semantic dependency among dispersed relevant snippets via neighborhood propagation. After obtaining contextual semantic information, our model reduces information redundancy by performing graph structure learning. Finally, the fine-grained semantic representations are fed into the downstream claim-evidence interaction module for predictions. Comprehensive experiments have demonstrated the superiority of GET over the state-of-the-arts.

[114]  arXiv:2201.06907 (cross-list from cs.CL) [pdf]
Title: Improve Sentence Alignment by Divide-and-conquer
Authors: Wu Zhang
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

In this paper, we introduce a divide-and-conquer algorithm to improve sentence alignment speed. We utilize external bilingual sentence embeddings to find accurate hard delimiters for the parallel texts to be aligned. We use Monte Carlo simulation to show experimentally that using this divide-and-conquer algorithm, we can turn any quadratic time complexity sentence alignment algorithm into an algorithm with average time complexity of O(NlogN). On a standard OCR-generated dataset, our method improves the Bleualign baseline by 3 F1 points. Besides, when computational resources are restricted, our algorithm is faster than Vecalign in practice.

[115]  arXiv:2201.06912 (cross-list from cs.SE) [pdf]
Title: Digital Twin: From Concept to Practice
Comments: To be published in ASCE Journal of Management in Engineering. The published version will be available at: this https URL)ME.1943-5479.0001034
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)

Recent technological developments and advances in Artificial Intelligence (AI) have enabled sophisticated capabilities to be a part of Digital Twin (DT), virtually making it possible to introduce automation into all aspects of work processes. Given these possibilities that DT can offer, practitioners are facing increasingly difficult decisions regarding what capabilities to select while deploying a DT in practice. The lack of research in this field has not helped either. It has resulted in the rebranding and reuse of emerging technological capabilities like prediction, simulation, AI, and Machine Learning (ML) as necessary constituents of DT. Inappropriate selection of capabilities in a DT can result in missed opportunities, strategic misalignments, inflated expectations, and risk of it being rejected as just hype by the practitioners. To alleviate this challenge, this paper proposes the digitalization framework, designed and developed by following a Design Science Research (DSR) methodology over a period of 18 months. The framework can help practitioners select an appropriate level of sophistication in a DT by weighing the pros and cons for each level, deciding evaluation criteria for the digital twin system, and assessing the implications of the selected DT on the organizational processes and strategies, and value creation. Three real-life case studies illustrate the application and usefulness of the framework.

[116]  arXiv:2201.06924 (cross-list from cs.CY) [pdf, other]
Title: A Synthetic Prediction Market for Estimating Confidence in Published Work
Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG); Multiagent Systems (cs.MA)

Explainably estimating confidence in published scholarly work offers opportunity for faster and more robust scientific progress. We develop a synthetic prediction market to assess the credibility of published claims in the social and behavioral sciences literature. We demonstrate our system and detail our findings using a collection of known replication projects. We suggest that this work lays the foundation for a research agenda that creatively uses AI for peer review.

[117]  arXiv:2201.06941 (cross-list from cs.CY) [pdf, other]
Title: Incremental Knowledge Tracing from Multiple Schools
Comments: In AAAI22 AI4EDU Workshop
Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Knowledge tracing is the task of predicting a learner's future performance based on the history of the learner's performance. Current knowledge tracing models are built based on an extensive set of data that are collected from multiple schools. However, it is impossible to pool learner's data from all schools, due to data privacy and PDPA policies. Hence, this paper explores the feasibility of building knowledge tracing models while preserving the privacy of learners' data within their respective schools. This study is conducted using part of the ASSISTment 2009 dataset, with data from multiple schools being treated as separate tasks in a continual learning framework. The results show that learning sequentially with the Self Attentive Knowledge Tracing (SAKT) algorithm is able to achieve considerably similar performance to that of pooling all the data together.

[118]  arXiv:2201.06947 (cross-list from cs.NI) [pdf, other]
Title: AI-Aided Integrated Terrestrial and Non-Terrestrial 6G Solutions for Sustainable Maritime Networking
Comments: The article has been accepted for publication in IEEE Network
Subjects: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI)

The maritime industry is experiencing a technological revolution that affects shipbuilding, operation of both seagoing and inland vessels, cargo management, and working practices in harbors. This ongoing transformation is driven by the ambition to make the ecosystem more sustainable and cost-efficient. Digitalization and automation help achieve these goals by transforming shipping and cruising into a much more cost- and energy-efficient, and decarbonized industry segment. The key enablers in these processes are always-available connectivity and content delivery services, which can not only aid shipping companies in improving their operational efficiency and reducing carbon emissions but also contribute to enhanced crew welfare and passenger experience. Due to recent advancements in integrating high-capacity and ultra-reliable terrestrial and non-terrestrial networking technologies, ubiquitous maritime connectivity is becoming a reality. To cope with the increased complexity of managing these integrated systems, this article advocates the use of artificial intelligence and machine learning-based approaches to meet the service requirements and energy efficiency targets in various maritime communications scenarios.

[119]  arXiv:2201.06952 (cross-list from cs.CY) [pdf, other]
Title: Fairness Score and Process Standardization: Framework for Fairness Certification in Artificial Intelligence Systems
Comments: 15 pages, 4 figures
Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Decisions made by various Artificial Intelligence (AI) systems greatly influence our day-to-day lives. With the increasing use of AI systems, it becomes crucial to know that they are fair, identify the underlying biases in their decision-making, and create a standardized framework to ascertain their fairness. In this paper, we propose a novel Fairness Score to measure the fairness of a data-driven AI system and a Standard Operating Procedure (SOP) for issuing Fairness Certification for such systems. Fairness Score and audit process standardization will ensure quality, reduce ambiguity, enable comparison and improve the trustworthiness of the AI systems. It will also provide a framework to operationalise the concept of fairness and facilitate the commercial deployment of such systems. Furthermore, a Fairness Certificate issued by a designated third-party auditing agency following the standardized process would boost the conviction of the organizations in the AI systems that they intend to deploy. The Bias Index proposed in this paper also reveals comparative bias amongst the various protected attributes within the dataset. To substantiate the proposed framework, we iteratively train a model on biased and unbiased data using multiple datasets and check that the Fairness Score and the proposed process correctly identify the biases and judge the fairness.

[120]  arXiv:2201.06953 (cross-list from cs.CY) [pdf, other]
Title: Knowledge Tracing: A Survey
Subjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Humans ability to transfer knowledge through teaching is one of the essential aspects for human intelligence. A human teacher can track the knowledge of students to customize the teaching on students needs. With the rise of online education platforms, there is a similar need for machines to track the knowledge of students and tailor their learning experience. This is known as the Knowledge Tracing (KT) problem in the literature. Effectively solving the KT problem would unlock the potential of computer-aided education applications such as intelligent tutoring systems, curriculum learning, and learning materials' recommendation. Moreover, from a more general viewpoint, a student may represent any kind of intelligent agents including both human and artificial agents. Thus, the potential of KT can be extended to any machine teaching application scenarios which seek for customizing the learning experience for a student agent (i.e., a machine learning model). In this paper, we provide a comprehensive and systematic review for the KT literature. We cover a broad range of methods starting from the early attempts to the recent state-of-the-art methods using deep learning, while highlighting the theoretical aspects of models and the characteristics of benchmark datasets. Besides these, we shed light on key modelling differences between closely related methods and summarize them in an easy-to-understand format. Finally, we discuss current research gaps in the KT literature and possible future research and application directions.

[121]  arXiv:2201.06961 (cross-list from eess.SY) [pdf, other]
Title: AI for Closed-Loop Control Systems --- New Opportunities for Modeling, Designing, and Tuning Control Systems
Comments: 11 pages, 7 figures
Subjects: Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Control Systems, particularly closed-loop control systems (CLCS), are frequently used in production machines, vehicles, and robots nowadays. CLCS are needed to actively align actual values of a process to a given reference or set values in real-time with a very high precession. Yet, artificial intelligence (AI) is not used to model, design, optimize, and tune CLCS. This paper will highlight potential AI-empowered and -based control system designs and designing procedures, gathering new opportunities and research direction in the field of control system engineering. Therefore, this paper illustrates which building blocks within the standard block diagram of CLCS can be replaced by AI, i.e., artificial neuronal networks (ANN). Having processes with real-time contains and functional safety in mind, it is discussed if AI-based controller blocks can cope with these demands. By concluding the paper, the pros and cons of AI-empowered as well as -based CLCS designs are discussed, and possible research directions for introducing AI in the domain of control system engineering are given.

[122]  arXiv:2201.06964 (cross-list from cs.LG) [pdf, ps, other]
Title: There is a Singularity in the Loss Landscape
Authors: Mark Lowell
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Despite the widespread adoption of neural networks, their training dynamics remain poorly understood. We show experimentally that as the size of the dataset increases, a point forms where the magnitude of the gradient of the loss becomes unbounded. Gradient descent rapidly brings the network close to this singularity in parameter space, and further training takes place near it. This singularity explains a variety of phenomena recently observed in the Hessian of neural network loss functions, such as training on the edge of stability and the concentration of the gradient in a top subspace. Once the network approaches the singularity, the top subspace contributes little to learning, even though it constitutes the majority of the gradient.

[123]  arXiv:2201.06972 (cross-list from cs.SI) [pdf, other]
Title: Representation Learning on Heterostructures via Heterogeneous Anonymous Walks
Comments: 13 pages, 6 figures, 5 tables
Subjects: Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI)

Capturing structural similarity has been a hot topic in the field of network embedding recently due to its great help in understanding the node functions and behaviors. However, existing works have paid very much attention to learning structures on homogeneous networks while the related study on heterogeneous networks is still a void. In this paper, we try to take the first step for representation learning on heterostructures, which is very challenging due to their highly diverse combinations of node types and underlying structures. To effectively distinguish diverse heterostructures, we firstly propose a theoretically guaranteed technique called heterogeneous anonymous walk (HAW) and its variant coarse HAW (CHAW). Then, we devise the heterogeneous anonymous walk embedding (HAWE) and its variant coarse HAWE in a data-driven manner to circumvent using an extremely large number of possible walks and train embeddings by predicting occurring walks in the neighborhood of each node. Finally, we design and apply extensive and illustrative experiments on synthetic and real-world networks to build a benchmark on heterostructure learning and evaluate the effectiveness of our methods. The results demonstrate our methods achieve outstanding performance compared with both homogeneous and heterogeneous classic methods, and can be applied on large-scale networks.

[124]  arXiv:2201.06974 (cross-list from cs.CV) [pdf, other]
Title: Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation
Comments: 24 pages, 9 figures, 6 tables, under submission
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)

Deep neural networks are typically trained in a single shot for a specific task and data distribution, but in real world settings both the task and the domain of application can change. The problem becomes even more challenging in dense predictive tasks, such as semantic segmentation, and furthermore most approaches tackle the two problems separately. In this paper we introduce the novel task of coarse-to-fine learning of semantic segmentation architectures in presence of domain shift. We consider subsequent learning stages progressively refining the task at the semantic level; i.e., the finer set of semantic labels at each learning step is hierarchically derived from the coarser set of the previous step. We propose a new approach (CCDA) to tackle this scenario. First, we employ the maximum squares loss to align source and target domains and, at the same time, to balance the gradients between well-classified and harder samples. Second, we introduce a novel coarse-to-fine knowledge distillation constraint to transfer network capabilities acquired on a coarser set of labels to a set of finer labels. Finally, we design a coarse-to-fine weight initialization rule to spread the importance from each coarse class to the respective finer classes. To evaluate our approach, we design two benchmarks where source knowledge is extracted from the GTA5 dataset and it is transferred to either the Cityscapes or the IDD datasets, and we show how it outperforms the main competitors.

[125]  arXiv:2201.06993 (cross-list from cs.NE) [pdf, other]
Title: FPGA-optimized Hardware acceleration for Spiking Neural Networks
Comments: 6 pages, 5 figures
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)

Artificial intelligence (AI) is gaining success and importance in many different tasks. The growing pervasiveness and complexity of AI systems push researchers towards developing dedicated hardware accelerators. Spiking Neural Networks (SNN) represent a promising solution in this sense since they implement models that are more suitable for a reliable hardware design. Moreover, from a neuroscience perspective, they better emulate a human brain. This work presents the development of a hardware accelerator for an SNN, with off-line training, applied to an image recognition task, using the MNIST as the target dataset. Many techniques are used to minimize the area and to maximize the performance, such as the replacement of the multiplication operation with simple bit shifts and the minimization of the time spent on inactive spikes, useless for the update of neurons' internal state. The design targets a Xilinx Artix-7 FPGA, using in total around the 40% of the available hardware resources and reducing the classification time by three orders of magnitude, with a small 4.5% impact on the accuracy, if compared to its software, full precision counterpart.

[126]  arXiv:2201.07013 (cross-list from cs.LG) [pdf, other]
Title: Deep Cervix Model Development from Heterogeneous and Partially Labeled Image Datasets
Comments: 5 Pages
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Cervical cancer is the fourth most common cancer in women worldwide. The availability of a robust automated cervical image classification system can augment the clinical care provider's limitation in traditional visual inspection with acetic acid (VIA). However, there are a wide variety of cervical inspection objectives which impact the labeling criteria for criteria-specific prediction model development. Moreover, due to the lack of confirmatory test results and inter-rater labeling variation, many images are left unlabeled.
Motivated by these challenges, we propose a self-supervised learning (SSL) based approach to produce a pre-trained cervix model from unlabeled cervical images. The developed model is further fine-tuned to produce criteria-specific classification models with the available labeled images. We demonstrate the effectiveness of the proposed approach using two cervical image datasets. Both datasets are partially labeled and labeling criteria are different. The experimental results show that the SSL-based initialization improves classification performance (Accuracy: 2.5% min) and the inclusion of images from both datasets during SSL further improves the performance (Accuracy: 1.5% min). Further, considering data-sharing restrictions, we experimented with the effectiveness of Federated SSL and find that it can improve performance over the SSL model developed with just its images. This justifies the importance of SSL-based cervix model development. We believe that the present research shows a novel direction in developing criteria-specific custom deep models for cervical image classification by combining images from different sources unlabeled and/or labeled with varying criteria, and addressing image access restrictions.

[127]  arXiv:2201.07016 (cross-list from cs.LG) [pdf, other]
Title: Accelerating Representation Learning with View-Consistent Dynamics in Data-Efficient Reinforcement Learning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Learning informative representations from image-based observations is of fundamental concern in deep Reinforcement Learning (RL). However, data-inefficiency remains a significant barrier to this objective. To overcome this obstacle, we propose to accelerate state representation learning by enforcing view-consistency on the dynamics. Firstly, we introduce a formalism of Multi-view Markov Decision Process (MMDP) that incorporates multiple views of the state. Following the structure of MMDP, our method, View-Consistent Dynamics (VCD), learns state representations by training a view-consistent dynamics model in the latent space, where views are generated by applying data augmentation to states. Empirical evaluation on DeepMind Control Suite and Atari-100k demonstrates VCD to be the SoTA data-efficient algorithm on visual control tasks.

[128]  arXiv:2201.07034 (cross-list from cs.SE) [pdf, other]
Title: A Semantic Web Technology Index
Comments: 11 pages, four figures, 2 tables
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)

Semantic Web (SW) technology has been widely applied to many domains such as medicine, health care, finance, geology. At present, researchers mainly rely on their experience and preferences to develop and evaluate the work of SW technology. Although the general architecture (e.g., Tim Berners-Lee's Semantic Web Layer Cake) of SW technology was proposed many years ago and has been well-known, it still lacks a concrete guideline for standardizing the development of SW technology. In this paper, we propose an SW technology index to standardize the development for ensuring that the work of SW technology is designed well and to quantitatively evaluate the quality of the work in SW technology. This index consists of 10 criteria that quantify the quality as a score of 0 ~ 10. We address each criterion in detail for a clear explanation from three aspects: 1) what is the criterion? 2) why do we consider this criterion and 3) how do the current studies meet this criterion? Finally, we present the validation of this index by providing some examples of how to apply the index to the validation cases. We conclude that the index is a useful standard to guide and evaluate the work in SW technology.

[129]  arXiv:2201.07082 (cross-list from cs.RO) [pdf, other]
Title: Inducing Structure in Reward Learning by Learning Features
Comments: 24 pages, 22 figures, accepted to the International Journal of Robotics Research. arXiv admin note: text overlap with arXiv:2006.13208
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)

Reward learning enables robots to learn adaptable behaviors from human input. Traditional methods model the reward as a linear function of hand-crafted features, but that requires specifying all the relevant features a priori, which is impossible for real-world tasks. To get around this issue, recent deep Inverse Reinforcement Learning (IRL) methods learn rewards directly from the raw state but this is challenging because the robot has to implicitly learn the features that are important and how to combine them, simultaneously. Instead, we propose a divide and conquer approach: focus human input specifically on learning the features separately, and only then learn how to combine them into a reward. We introduce a novel type of human input for teaching features and an algorithm that utilizes it to learn complex features from the raw state space. The robot can then learn how to combine them into a reward using demonstrations, corrections, or other reward learning frameworks. We demonstrate our method in settings where all features have to be learned from scratch, as well as where some of the features are known. By first focusing human input specifically on the feature(s), our method decreases sample complexity and improves generalization of the learned reward over a deepIRL baseline. We show this in experiments with a physical 7DOF robot manipulator, as well as in a user study conducted in a simulated environment.

[130]  arXiv:2201.07092 (cross-list from cs.LG) [pdf, other]
Title: K-nearest Multi-agent Deep Reinforcement Learning for Collaborative Tasks with a Variable Number of Agents
Journal-ref: 2021 IEEE International Conference on Big Data (Big Data)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)

Traditionally, the performance of multi-agent deep reinforcement learning algorithms are demonstrated and validated in gaming environments where we often have a fixed number of agents. In many industrial applications, the number of available agents can change at any given day and even when the number of agents is known ahead of time, it is common for an agent to break during the operation and become unavailable for a period of time. In this paper, we propose a new deep reinforcement learning algorithm for multi-agent collaborative tasks with a variable number of agents. We demonstrate the application of our algorithm using a fleet management simulator developed by Hitachi to generate realistic scenarios in a production site.

[131]  arXiv:2201.07096 (cross-list from cs.SE) [pdf, other]
Title: Lifelong Dynamic Optimization for Self-Adaptive Systems: Fact or Fiction?
Authors: Tao Chen
Comments: The paper has been accepted as a full technical paper at the 29th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2022)
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)

When faced with changing environment, highly configurable software systems need to dynamically search for promising adaptation plan that keeps the best possible performance, e.g., higher throughput or smaller latency -- a typical planning problem for self-adaptive systems (SASs). However, given the rugged and complex search landscape with multiple local optima, such a SAS planning is challenging especially in dynamic environments. In this paper, we propose LiDOS, a lifelong dynamic optimization framework for SAS planning. What makes LiDOS unique is that to handle the "dynamic", we formulate the SAS planning as a multi-modal optimization problem, aiming to preserve the useful information for better dealing with the local optima issue under dynamic environment changes. This differs from existing planners in that the "dynamic" is not explicitly handled during the search process in planning. As such, the search and planning in LiDOS run continuously over the lifetime of SAS, terminating only when it is taken offline or the search space has been covered under an environment.
Experimental results on three real-world SASs show that the concept of explicitly handling dynamic as part of the search in the SAS planning is effective, as LiDOS outperforms its stationary counterpart overall with up to 10x improvement. It also achieves better results in general over state-of-the-art planners and with 1.4x to 10x speedup on generating promising adaptation plans.

[132]  arXiv:2201.07106 (cross-list from cs.CV) [pdf, other]
Title: Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures
Comments: SPIE Medical Imaging 2022 (Oral)
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Lesions or organ boundaries visible through medical imaging data are often ambiguous, thus resulting in significant variations in multi-reader delineations, i.e., the source of aleatoric uncertainty. In particular, quantifying the inter-observer variability of manual annotations with Magnetic Resonance (MR) Imaging data plays a crucial role in establishing a reference standard for various diagnosis and treatment tasks. Most segmentation methods, however, simply model a mapping from an image to its single segmentation map and do not take the disagreement of annotators into consideration. In order to account for inter-observer variability, without sacrificing accuracy, we propose a novel variational inference framework to model the distribution of plausible segmentation maps, given a specific MR image, which explicitly represents the multi-reader variability. Specifically, we resort to a latent vector to encode the multi-reader variability and counteract the inherent information loss in the imaging data. Then, we apply a variational autoencoder network and optimize its evidence lower bound (ELBO) to efficiently approximate the distribution of the segmentation map, given an MR image. Experimental results, carried out with the QUBIQ brain growth MRI segmentation datasets with seven annotators, demonstrate the effectiveness of our approach.

[133]  arXiv:2201.07139 (cross-list from cs.LG) [pdf, ps, other]
Title: Safe Online Bid Optimization with Return-On-Investment and Budget Constraints subject to Uncertainty
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

In online marketing, the advertisers' goal is usually a tradeoff between achieving high volumes and high profitability. The companies' business units customarily address this tradeoff by maximizing the volumes while guaranteeing a lower bound to the Return On Investment (ROI). This paper investigates combinatorial bandit algorithms for the bid optimization of advertising campaigns subject to uncertain budget and ROI constraints. We study the nature of both the optimization and learning problems. In particular, when focusing on the optimization problem without uncertainty, we show that it is inapproximable within any factor unless P=NP, and we provide a pseudo-polynomial-time algorithm that achieves an optimal solution. When considering uncertainty, we prove that no online learning algorithm can violate the (ROI or budget) constraints during the learning process a sublinear number of times while guaranteeing a sublinear pseudo-regret. Thus, we provide an algorithm, namely GCB, guaranteeing sublinear regret at the cost of a potentially linear number of constraints violations. We also design its safe version, namely GCB_{safe}, guaranteeing w.h.p. a constant upper bound on the number of constraints violations at the cost of a linear pseudo-regret. More interestingly, we provide an algorithm, namely GCB_{safe}(\psi,\phi), guaranteeing both sublinear pseudo-regret and safety w.h.p. at the cost of accepting tolerances \psi and \phi in the satisfaction of the ROI and budget constraints, respectively. This algorithm actually mitigates the risks due to the constraints violations without precluding the convergence to the optimal solution. Finally, we experimentally compare our algorithms in terms of pseudo-regret/constraint-violation tradeoff in settings generated from real-world data, showing the importance of adopting safety constraints in practice and the effectiveness of our algorithms.

[134]  arXiv:2201.07207 (cross-list from cs.LG) [pdf, other]
Title: Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents
Comments: Project website at this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

Can world knowledge learned by large language models (LLMs) be used to act in interactive environments? In this paper, we investigate the possibility of grounding high-level tasks, expressed in natural language (e.g. "make breakfast"), to a chosen set of actionable steps (e.g. "open fridge"). While prior work focused on learning from explicit step-by-step examples of how to act, we surprisingly find that if pre-trained LMs are large enough and prompted appropriately, they can effectively decompose high-level tasks into low-level plans without any further training. However, the plans produced naively by LLMs often cannot map precisely to admissible actions. We propose a procedure that conditions on existing demonstrations and semantically translates the plans to admissible actions. Our evaluation in the recent VirtualHome environment shows that the resulting method substantially improves executability over the LLM baseline. The conducted human evaluation reveals a trade-off between executability and correctness but shows a promising sign towards extracting actionable knowledge from language models. Website at https://huangwl18.github.io/language-planner

Replacements for Wed, 19 Jan 22

[135]  arXiv:2101.00058 (replaced) [pdf, ps, other]
Title: Conflict-driven Inductive Logic Programming
Authors: Mark Law
Comments: Under consideration in Theory and Practice of Logic Programming (TPLP)
Subjects: Artificial Intelligence (cs.AI)
[136]  arXiv:2103.10453 (replaced) [pdf, other]
Title: Massively parallel hybrid search for the partial Latin square extension problem
Subjects: Artificial Intelligence (cs.AI)
[137]  arXiv:2103.15059 (replaced) [pdf, other]
Title: A Comprehensive Survey on Knowledge Graph Entity Alignment via Representation Learning
Subjects: Artificial Intelligence (cs.AI)
[138]  arXiv:2104.02137 (replaced) [pdf, other]
Title: ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualities
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
[139]  arXiv:2107.12979 (replaced) [pdf, other]
Title: Predictive Coding: a Theoretical and Experimental Review
Comments: 27/07/21 initial upload; 14/01/22 maths fix
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Neurons and Cognition (q-bio.NC)
[140]  arXiv:2108.00056 (replaced) [pdf, other]
Title: Procedural Generation of 3D Maps with Snappable Meshes
Subjects: Artificial Intelligence (cs.AI); Graphics (cs.GR); Human-Computer Interaction (cs.HC)
[141]  arXiv:2112.02682 (replaced) [pdf, other]
Title: BERTMap: A BERT-based Ontology Alignment System
Comments: Full version (with appendix) of the accepted paper in 36th AAAI Conference on Artificial Intelligence 2022
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
[142]  arXiv:2112.11805 (replaced) [pdf, other]
Title: Neural-Symbolic Integration for Interactive Learning and Conceptual Grounding
Comments: Corrected references
Journal-ref: 1st Workshop on Human and Machine Decisions (WHMD 2021), NeurIPS 2021
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[143]  arXiv:2004.10489 (replaced) [pdf, ps, other]
Title: Differential evolution outside the box
Journal-ref: Information Sciences, 581:587-604, 2021
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
[144]  arXiv:2008.11707 (replaced) [pdf, other]
Title: Bandit Data-Driven Optimization
Comments: This is the complete version of the paper. A version of this paper is also published at AAAI-22
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (stat.ML)
[145]  arXiv:2009.13472 (replaced) [pdf, other]
Title: Targeted VAE: Variational and Targeted Learning for Causal Inference
Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[146]  arXiv:2010.03950 (replaced) [pdf, other]
Title: Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning
Journal-ref: Journal of Artificial Intelligence Research 73 (2022) 173-208
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[147]  arXiv:2010.14282 (replaced) [pdf, other]
Title: Active Learning for Human-in-the-Loop Customs Inspection
Comments: To Appear at IEEE TKDE
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
[148]  arXiv:2011.09257 (replaced) [pdf, other]
Title: Inspecting state of the art performance and NLP metrics in image-based medical report generation
Comments: 3 pages, 1 figure, 1 table. Accepted in LatinX in AI workshop at NeurIPS 2020. (v3 updated ack)
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
[149]  arXiv:2012.06337 (replaced) [pdf, other]
Title: Privacy and Robustness in Federated Learning: Attacks and Defenses
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
[150]  arXiv:2012.10773 (replaced) [pdf, other]
Title: Forming Human-Robot Cooperation for Tasks with General Goal using Evolutionary Value Learning
Comments: Accepted by RAL
Journal-ref: in IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 762-769, April 2022
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
[151]  arXiv:2101.10904 (replaced) [pdf, other]
Title: Untargeted Poisoning Attack Detection in Federated Learning via Behavior Attestation
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[152]  arXiv:2101.11881 (replaced) [pdf, other]
Title: Deep learning via LSTM models for COVID-19 infection forecasting in India
Comments: PLOS One, 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Applications (stat.AP); Machine Learning (stat.ML)
[153]  arXiv:2102.03314 (replaced) [pdf, other]
Title: On Utility and Privacy in Synthetic Genomic Data
Comments: Published in the Proceedings of the 29th Network and Distributed System Security Symposium (NDSS 2022)
Subjects: Genomics (q-bio.GN); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
[154]  arXiv:2103.09071 (replaced) [pdf, ps, other]
Title: Map completion from partial observation using the global structure of multiple environmental maps
Comments: Accepted to Advanced Robotics
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
[155]  arXiv:2104.03986 (replaced) [pdf, other]
Title: Deep Indexed Active Learning for Matching Heterogeneous Entity Representations
Comments: VLDB 2022
Subjects: Databases (cs.DB); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
[156]  arXiv:2104.08682 (replaced) [pdf, other]
Title: Rethinking Network Pruning -- under the Pre-train and Fine-tune Paradigm
Comments: 7 pages, 6 figures, 1 table
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
[157]  arXiv:2104.08813 (replaced) [pdf, ps, other]
Title: CNN aided Weighted Interpolation for Channel Estimation in Vehicular Communications
Comments: 16 pages
Journal-ref: IEEE Transactions on Vehicular Technology ( Early Access ) 2021
Subjects: Information Theory (cs.IT); Artificial Intelligence (cs.AI)
[158]  arXiv:2105.01099 (replaced) [pdf, other]
Title: Reinforcement Learning for Ridesharing: An Extended Survey
Comments: This survey has been significantly expanded and refined over the conference version presented at IEEE ITSC 2021
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[159]  arXiv:2105.03962 (replaced) [pdf, other]
Title: Stochastic Multi-Armed Bandits with Control Variates
Comments: Accepted to NeurIPS 2021
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
[160]  arXiv:2106.02634 (replaced) [pdf, other]
Title: Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering
Comments: First two authors contributed equally. Project website: this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Graphics (cs.GR); Machine Learning (cs.LG); Multimedia (cs.MM)
[161]  arXiv:2106.06039 (replaced) [pdf, other]
Title: Neural Predicting Higher-order Patterns in Temporal Networks
Subjects: Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI)
[162]  arXiv:2107.00710 (replaced) [pdf, other]
Title: Long-Short Ensemble Network for Bipolar Manic-Euthymic State Recognition Based on Wrist-worn Sensors
Comments: Submitted for peer-review. 10 pages + 2. 2 Figures and 1 table
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
[163]  arXiv:2107.01035 (replaced) [pdf]
Title: Molecular distance matrix prediction based on graph convolutional networks
Comments: 17 pages, 4 figures, 6 tables
Subjects: Chemical Physics (physics.chem-ph); Artificial Intelligence (cs.AI)
[164]  arXiv:2107.05445 (replaced) [pdf, other]
Title: Disentangling Transfer and Interference in Multi-Domain Learning
Comments: AAAI 2022 PracticalDL Workshop
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[165]  arXiv:2107.05798 (replaced) [pdf, other]
Title: Cautious Policy Programming: Exploiting KL Regularization in Monotonic Policy Improvement for Reinforcement Learning
Comments: 15 pages. arXiv admin note: text overlap with arXiv:2008.10806
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[166]  arXiv:2107.05948 (replaced) [pdf, other]
Title: On Designing Good Representation Learning Models
Comments: 14 pages
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
[167]  arXiv:2107.09807 (replaced) [pdf]
Title: Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer
Comments: Accepted for publication by The Journal of Supercomputing
Subjects: Multiagent Systems (cs.MA); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[168]  arXiv:2107.12806 (replaced) [pdf, other]
Title: Towards Industrial Private AI: A two-tier framework for data and model security
Comments: 9 pages, 4 figures, 2 tables, Magazine article
Journal-ref: IEEE Wireless Communications 2022
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
[169]  arXiv:2108.01662 (replaced) [pdf, other]
Title: Uniform Sampling over Episode Difficulty
Comments: NeurIPS'21 camera ready
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
[170]  arXiv:2108.06227 (replaced) [pdf, other]
Title: SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[171]  arXiv:2108.10293 (replaced) [pdf, other]
Title: A Simplicial Model for $KB4_n$: Epistemic Logic with Agents that May Die
Subjects: Logic in Computer Science (cs.LO); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Multiagent Systems (cs.MA); Logic (math.LO)
[172]  arXiv:2110.06968 (replaced) [pdf, other]
Title: Interpretable AI forecasting for numerical relativity waveforms of quasi-circular, spinning, non-precessing binary black hole mergers
Comments: 15 pages, 7 figures, 2 appendices, 1 interactive visualization at this https URL
Journal-ref: Phys. Rev. D 105, 024024 (2022)
Subjects: General Relativity and Quantum Cosmology (gr-qc); Instrumentation and Methods for Astrophysics (astro-ph.IM); Artificial Intelligence (cs.AI)
[173]  arXiv:2110.07701 (replaced) [pdf, other]
Title: Exposing Query Identification for Search Transparency
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[174]  arXiv:2110.14755 (replaced) [pdf, other]
Title: Algorithmic encoding of protected characteristics in image-based models for disease detection
Comments: Code available on this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
[175]  arXiv:2110.15122 (replaced) [pdf, other]
Title: CAFE: Catastrophic Data Leakage in Vertical Federated Learning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[176]  arXiv:2111.00289 (replaced) [pdf, other]
Title: Intrusion Prevention through Optimal Stopping
Comments: Preprint; Submitted to IEEE for review. Minor text updates 17/1 2022. arXiv admin note: substantial text overlap with arXiv:2106.07160
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Networking and Internet Architecture (cs.NI)
[177]  arXiv:2111.00876 (replaced) [pdf, other]
Title: On the Expressivity of Markov Reward
Comments: Accepted to NeurIPS 2021
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[178]  arXiv:2111.01531 (replaced) [pdf, other]
Title: Generating synthetic transactional profiles
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[179]  arXiv:2111.01853 (replaced) [pdf, other]
Title: Recursive Bayesian Networks: Generalising and Unifying Probabilistic Context-Free Grammars and Dynamic Bayesian Networks
Comments: In: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021); Code: this https URL; Comments: corrected typo in outside probabilities: {\alpha}(y) --> {\alpha}(x); corrected typo in Appendix C.2.1; updated references
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)
[180]  arXiv:2111.04095 (replaced) [pdf, other]
Title: Iterative Causal Discovery in the Possible Presence of Latent Confounders and Selection Bias
Comments: 35th Conference on Neural Information Processing Systems (NeurIPS 2021). arXiv admin note: text overlap with arXiv:2012.07513
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Methodology (stat.ME); Machine Learning (stat.ML)
[181]  arXiv:2111.05498 (replaced) [pdf, other]
Title: Attention Approximates Sparse Distributed Memory
Journal-ref: 35th Conference on Neural Information Processing Systems (NeurIPS 2021)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[182]  arXiv:2111.08749 (replaced) [pdf, other]
Title: SMACE: A New Method for the Interpretability of Composite Decision Systems
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[183]  arXiv:2111.10635 (replaced) [pdf, other]
Title: HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments
Comments: 14 pages, 11 figures, 2 tables
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Systems and Control (eess.SY)
[184]  arXiv:2112.00427 (replaced) [pdf]
Title: Research on Event Accumulator Settings for Event-Based SLAM
Comments: arXiv admin note: text overlap with arXiv:2008.05749 by other authors
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
[185]  arXiv:2112.04674 (replaced) [pdf, other]
Title: DualFormer: Local-Global Stratified Transformer for Efficient Video Recognition
Comments: Preprint
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
[186]  arXiv:2112.08588 (replaced) [pdf, other]
Title: Learning to acquire novel cognitive tasks with evolution, plasticity and meta-meta-learning
Authors: Thomas Miconi
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
[187]  arXiv:2112.12591 (replaced) [pdf, other]
Title: Black-Box Testing of Deep Neural Networks through Test Case Diversity
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[188]  arXiv:2201.00016 (replaced) [pdf, other]
Title: TransLog: A Unified Transformer-based Framework for Log Anomaly Detection
Comments: 6 pages
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[189]  arXiv:2201.01305 (replaced) [pdf, other]
Title: Augmenting astrophysical scaling relations with machine learning : application to reducing the SZ flux-mass scatter
Comments: Minor updates to Figs. 4 & 8. Added Fig.10. The code and data associated with this paper are available at this https URL
Subjects: Cosmology and Nongalactic Astrophysics (astro-ph.CO); Astrophysics of Galaxies (astro-ph.GA); Instrumentation and Methods for Astrophysics (astro-ph.IM); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[190]  arXiv:2201.02373 (replaced) [pdf, other]
Title: Mirror Learning: A Unifying Framework of Policy Optimisation
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[191]  arXiv:2201.02946 (replaced) [pdf, other]
Title: Resolving Camera Position for a Practical Application of Gaze Estimation on Edge Devices
Comments: 6 pages, 11 figures, conference paper
Journal-ref: ICAIIC 2022 (The 4th International Conference on Artificial Intelligence in Information and Communication February 21 (Mon.) ~ 24 (Thur.), 2022, Guam, USA & Virtual Conference)
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
[192]  arXiv:2201.04194 (replaced) [pdf, other]
Title: Neural Capacitance: A New Perspective of Neural Network Selection via Edge Dynamics
Comments: 19 pages, 7 figures, neural architecture search, mean-field
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
[193]  arXiv:2201.04266 (replaced) [pdf, ps, other]
Title: Safe Equilibrium
Authors: Sam Ganzfried
Subjects: Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Multiagent Systems (cs.MA); Theoretical Economics (econ.TH)
[194]  arXiv:2201.04876 (replaced) [pdf, other]
Title: Towards a Reference Software Architecture for Human-AI Teaming in Smart Manufacturing
Comments: Conference: ICSE-NIER 2022 - The 44th International Conference on Software Engineering, 5 pages, 1 figure
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
[195]  arXiv:2201.05156 (replaced) [html]
Title: Proceedings of the 4th Workshop on Online Recommender Systems and User Modeling -- ORSUM 2021
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[ total of 195 entries: 1-195 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2201, contact, help  (Access key information)