We gratefully acknowledge support from
the Simons Foundation and member institutions.

Information Retrieval

New submissions

[ total of 51 entries: 1-51 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Tue, 16 Apr 24

[1]  arXiv:2404.08660 [pdf, other]
Title: How Does Message Passing Improve Collaborative Filtering?
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)

Collaborative filtering (CF) has exhibited prominent results for recommender systems and been broadly utilized for real-world applications. A branch of research enhances CF methods by message passing used in graph neural networks, due to its strong capabilities of extracting knowledge from graph-structured data, like user-item bipartite graphs that naturally exist in CF. They assume that message passing helps CF methods in a manner akin to its benefits for graph-based learning tasks in general. However, even though message passing empirically improves CF, whether or not this assumption is correct still needs verification. To address this gap, we formally investigate why message passing helps CF from multiple perspectives and show that many assumptions made by previous works are not entirely accurate. With our curated ablation studies and theoretical analyses, we discover that (1) message passing improves the CF performance primarily by additional representations passed from neighbors during the forward pass instead of additional gradient updates to neighbor representations during the model back-propagation and (ii) message passing usually helps low-degree nodes more than high-degree nodes. Utilizing these novel findings, we present Test-time Aggregation for CF, namely TAG-CF, a test-time augmentation framework that only conducts message passing once at inference time. The key novelty of TAG-CF is that it effectively utilizes graph knowledge while circumventing most of notorious computational overheads of message passing. Besides, TAG-CF is extremely versatile can be used as a plug-and-play module to enhance representations trained by different CF supervision signals. Evaluated on six datasets, TAG-CF consistently improves the recommendation performance of CF methods without graph by up to 39.2% on cold users and 31.7% on all users, with little to no extra computational overheads.

[2]  arXiv:2404.08662 [pdf, other]
Title: FewUser: Few-Shot Social User Geolocation via Contrastive Learning
Comments: 17 pages, 3 figures, 8 tables, submitted to ECML-PKDD 2024 for review
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG); Social and Information Networks (cs.SI)

To address the challenges of scarcity in geotagged data for social user geolocation, we propose FewUser, a novel framework for Few-shot social User geolocation. We incorporate a contrastive learning strategy between users and locations to improve geolocation performance with no or limited training data. FewUser features a user representation module that harnesses a pre-trained language model (PLM) and a user encoder to process and fuse diverse social media inputs effectively. To bridge the gap between PLM's knowledge and geographical data, we introduce a geographical prompting module with hard, soft, and semi-soft prompts, to enhance the encoding of location information. Contrastive learning is implemented through a contrastive loss and a matching loss, complemented by a hard negative mining strategy to refine the learning process. We construct two datasets TwiU and FliU, containing richer metadata than existing benchmarks, to evaluate FewUser and the extensive experiments demonstrate that FewUser significantly outperforms state-of-the-art methods in both zero-shot and various few-shot settings, achieving absolute improvements of 26.95\% and \textbf{41.62\%} on TwiU and FliU, respectively, with only one training sample per class. We further conduct a comprehensive analysis to investigate the impact of user representation on geolocation performance and the effectiveness of FewUser's components, offering valuable insights for future research in this area.

[3]  arXiv:2404.08664 [pdf, other]
Title: Identifying Banking Transaction Descriptions via Support Vector Machine Short-Text Classification Based on a Specialized Labelled Corpus
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Computation and Language (cs.CL); Machine Learning (cs.LG)

Short texts are omnipresent in real-time news, social network commentaries, etc. Traditional text representation methods have been successfully applied to self-contained documents of medium size. However, information in short texts is often insufficient, due, for example, to the use of mnemonics, which makes them hard to classify. Therefore, the particularities of specific domains must be exploited. In this article we describe a novel system that combines Natural Language Processing techniques with Machine Learning algorithms to classify banking transaction descriptions for personal finance management, a problem that was not previously considered in the literature. We trained and tested that system on a labelled dataset with real customer transactions that will be available to other researchers on request. Motivated by existing solutions in spam detection, we also propose a short text similarity detector to reduce training set size based on the Jaccard distance. Experimental results with a two-stage classifier combining this detector with a SVM indicate a high accuracy in comparison with alternative approaches, taking into account complexity and computing time. Finally, we present a use case with a personal finance application, CoinScrap, which is available at Google Play and App Store.

[4]  arXiv:2404.08665 [pdf, other]
Title: Targeted aspect-based emotion analysis to detect opportunities and precaution in financial Twitter messages
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG); Social and Information Networks (cs.SI); Trading and Market Microstructure (q-fin.TR)

Microblogging platforms, of which Twitter is a representative example, are valuable information sources for market screening and financial models. In them, users voluntarily provide relevant information, including educated knowledge on investments, reacting to the state of the stock markets in real-time and, often, influencing this state. We are interested in the user forecasts in financial, social media messages expressing opportunities and precautions about assets. We propose a novel Targeted Aspect-Based Emotion Analysis (TABEA) system that can individually discern the financial emotions (positive and negative forecasts) on the different stock market assets in the same tweet (instead of making an overall guess about that whole tweet). It is based on Natural Language Processing (NLP) techniques and Machine Learning streaming algorithms. The system comprises a constituency parsing module for parsing the tweets and splitting them into simpler declarative clauses; an offline data processing module to engineer textual, numerical and categorical features and analyse and select them based on their relevance; and a stream classification module to continuously process tweets on-the-fly. Experimental results on a labelled data set endorse our solution. It achieves over 90% precision for the target emotions, financial opportunity, and precaution on Twitter. To the best of our knowledge, no prior work in the literature has addressed this problem despite its practical interest in decision-making, and we are not aware of any previous NLP nor online Machine Learning approaches to TABEA.

[5]  arXiv:2404.08668 [pdf, other]
Title: A Comprehensive Survey on AI-based Methods for Patents
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)

Recent advancements in Artificial Intelligence (AI) and machine learning have demonstrated transformative capabilities across diverse domains. This progress extends to the field of patent analysis and innovation, where AI-based tools present opportunities to streamline and enhance important tasks in the patent cycle such as classification, retrieval, and valuation prediction. This not only accelerates the efficiency of patent researchers and applicants but also opens new avenues for technological innovation and discovery. Our survey provides a comprehensive summary of recent AI tools in patent analysis from more than 40 papers from 26 venues between 2017 and 2023. Unlike existing surveys, we include methods that work for patent image and text data. Furthermore, we introduce a novel taxonomy for the categorization based on the tasks in the patent life cycle as well as the specifics of the AI methods. This survey aims to serve as a resource for researchers, practitioners, and patent offices in the domain of AI-powered patent analysis.

[6]  arXiv:2404.08669 [pdf, ps, other]
Title: Combining PatternRank with Huffman Coding: A Novel Compression Algorithm
Subjects: Information Retrieval (cs.IR); Databases (cs.DB)

The escalating volume of data involved in Android backup packages necessitates an innovative approach to compression beyond traditional methods like GZIP, which may not fully exploit the redundancy inherent in Android backups, particularly those containing extensive XML data. This paper introduces the PatternRank algorithm, a novel compression strategy specifically designed for Android backups. PatternRank leverages pattern recognition and ranking, combined with Huffman coding, to efficiently compress data by identifying and replacing frequent, longer patterns with shorter codes. We detail two versions of the PatternRank algorithm: the original version focuses on dynamic pattern extraction and ranking, while the second version incorporates a pre-defined dictionary optimized for the common patterns found in Android backups, particularly within XML files. This tailored approach ensures that PatternRank not only outperforms traditional compression methods in terms of compression ratio and speed but also remains highly effective when dealing with the specific challenges posed by Android backup data. Our analysis includes a comparative study of compression performance across GZIP, PatternRank v1, PatternRank v2, and a combined PatternRank-Huffman method, highlighting the superior efficiency and potential of PatternRank in managing the growing data demands of Android backup packages. Through this exploration, we underscore the significance of adopting pattern-based compression algorithms in optimizing data storage and transmission in the mobile domain.

[7]  arXiv:2404.08671 [pdf, other]
Title: Navigating the Evaluation Funnel to Optimize Iteration Speed for Recommender Systems
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)

Over the last decades has emerged a rich literature on the evaluation of recommendation systems. However, less is written about how to efficiently combine different evaluation methods from this rich field into a single efficient evaluation funnel. In this paper we aim to build intuition for how to choose evaluation methods, by presenting a novel framework that simplifies the reasoning around the evaluation funnel for a recommendation system. Our contribution is twofold. First we present our framework for how to decompose the definition of success to construct efficient evaluation funnels, focusing on how to identify and discard non-successful iterations quickly. We show that decomposing the definition of success into smaller necessary criteria for success enables early identification of non-successful ideas. Second, we give an overview of the most common and useful evaluation methods, discuss their pros and cons, and how they fit into, and complement each other in, the evaluation process. We go through so-called offline and online evaluation methods such as counterfactual logging, validation, verification, A/B testing, and interleaving. The paper concludes with some general discussion and advice on how to design an efficient evaluation process for recommender systems.

[8]  arXiv:2404.08672 [pdf, other]
Title: Taxonomy and Analysis of Sensitive User Queries in Generative AI Search
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Machine Learning (cs.LG)

Although there has been a growing interest among industries to integrate generative LLMs into their services, limited experiences and scarcity of resources acts as a barrier in launching and servicing large-scale LLM-based conversational services. In this paper, we share our experiences in developing and operating generative AI models within a national-scale search engine, with a specific focus on the sensitiveness of user queries. We propose a taxonomy for sensitive search queries, outline our approaches, and present a comprehensive analysis report on sensitive queries from actual users.

[9]  arXiv:2404.08675 [pdf, other]
Title: RecGPT: Generative Personalized Prompts for Sequential Recommendation via ChatGPT Training Paradigm
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

ChatGPT has achieved remarkable success in natural language understanding. Considering that recommendation is indeed a conversation between users and the system with items as words, which has similar underlying pattern with ChatGPT, we design a new chat framework in item index level for the recommendation task. Our novelty mainly contains three parts: model, training and inference. For the model part, we adopt Generative Pre-training Transformer (GPT) as the sequential recommendation model and design a user modular to capture personalized information. For the training part, we adopt the two-stage paradigm of ChatGPT, including pre-training and fine-tuning. In the pre-training stage, we train GPT model by auto-regression. In the fine-tuning stage, we train the model with prompts, which include both the newly-generated results from the model and the user's feedback. For the inference part, we predict several user interests as user representations in an autoregressive manner. For each interest vector, we recall several items with the highest similarity and merge the items recalled by all interest vectors into the final result. We conduct experiments with both offline public datasets and online A/B test to demonstrate the effectiveness of our proposed method.

[10]  arXiv:2404.08677 [pdf, other]
Title: PMG : Personalized Multimodal Generation with Large Language Models
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

The emergence of large language models (LLMs) has revolutionized the capabilities of text comprehension and generation. Multi-modal generation attracts great attention from both the industry and academia, but there is little work on personalized generation, which has important applications such as recommender systems. This paper proposes the first method for personalized multimodal generation using LLMs, showcases its applications and validates its performance via an extensive experimental study on two datasets. The proposed method, Personalized Multimodal Generation (PMG for short) first converts user behaviors (e.g., clicks in recommender systems or conversations with a virtual assistant) into natural language to facilitate LLM understanding and extract user preference descriptions. Such user preferences are then fed into a generator, such as a multimodal LLM or diffusion model, to produce personalized content. To capture user preferences comprehensively and accurately, we propose to let the LLM output a combination of explicit keywords and implicit embeddings to represent user preferences. Then the combination of keywords and embeddings are used as prompts to condition the generator. We optimize a weighted sum of the accuracy and preference scores so that the generated content has a good balance between them. Compared to a baseline method without personalization, PMG has a significant improvement on personalization for up to 8% in terms of LPIPS while retaining the accuracy of generation.

[11]  arXiv:2404.08678 [pdf, other]
Title: Information Retrieval with Entity Linking
Authors: Dahlia Shehata
Subjects: Information Retrieval (cs.IR)

Despite the advantages of their low-resource settings, traditional sparse retrievers depend on exact matching approaches between high-dimensional bag-of-words (BoW) representations of both the queries and the collection. As a result, retrieval performance is restricted by semantic discrepancies and vocabulary gaps. On the other hand, transformer-based dense retrievers introduce significant improvements in information retrieval tasks by exploiting low-dimensional contextualized representations of the corpus. While dense retrievers are known for their relative effectiveness, they suffer from lower efficiency and lack of generalization issues, when compared to sparse retrievers. For a lightweight retrieval task, high computational resources and time consumption are major barriers encouraging the renunciation of dense models despite potential gains. In this work, I propose boosting the performance of sparse retrievers by expanding both the queries and the documents with linked entities in two formats for the entity names: 1) explicit and 2) hashed. A zero-shot end-to-end dense entity linking system is employed for entity recognition and disambiguation to augment the corpus. By leveraging the advanced entity linking methods, I believe that the effectiveness gap between sparse and dense retrievers can be narrowed. Experiments are conducted on the MS MARCO passage dataset using the original qrel set, the re-ranked qrels favoured by MonoT5 and the latter set further re-ranked by DuoT5. Since I am concerned with the early stage retrieval in cascaded ranking architectures of large information retrieval systems, the results are evaluated using recall@1000. The suggested approach is also capable of retrieving documents for query subsets judged to be particularly difficult in prior work.

[12]  arXiv:2404.08682 [pdf, ps, other]
Title: Access to Library Information Resources by University Students during COVID-19 Pandemic in Africa: A Systematic Literature Review
Subjects: Information Retrieval (cs.IR)

The study examined access to library information resources by university students during the outbreak of the COVID-19 pandemic. The study investigated measures that were adopted by academic libraries for smooth delivery of library information resources to their patrons. It also identified technological tools that were employed by libraries to facilitate access to library information resources. We also investigated the challenges faced by students in accessing library information resources. A systematic literature review approach using PRISMA guidelines was employed to investigate the relevant literature on the subject. The keyword search strategy was employed to search for relevant literature from four scholarly databases Scopus, emerald, Research4life, and Google Scholar. In this study, 23 studies that fulfilled the criteria were included. The findings revealed that the majority of the reviewed studies indicate that, during the COVID-19 pandemic many academic libraries in Africa adopted different approaches to facilitate access to library information resources by university students including expanding access to electronic resources off-campus, virtual reference services, circulation and lending services. To support access to different library services and information resources academic libraries in Africa used various digital technological tools like social media, library websites, email and video conferencing. Moreover, the study revealed that limited access to internet services and ICT devices, inadequate electronic library collection and inadequate digital and information literacy were the major challenges faced by patrons during the pandemic. This study recommends investment in ICT infrastructures and expanding electronic resource collections which are vital resources in the digital era.

[13]  arXiv:2404.08687 [pdf, other]
Title: A Survey of Reasoning for Substitution Relationships: Definitions, Methods, and Directions
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)

Substitute relationships are fundamental to people's daily lives across various domains. This study aims to comprehend and predict substitute relationships among products in diverse fields, extensively analyzing the application of machine learning algorithms, natural language processing, and other technologies. By comparing model methodologies across different domains, such as defining substitutes, representing and learning substitute relationships, and substitute reasoning, this study offers a methodological foundation for delving deeper into substitute relationships. Through ongoing research and innovation, we can further refine the personalization and accuracy of substitute recommendation systems, thus advancing the development and application of this field.

[14]  arXiv:2404.08691 [pdf, ps, other]
Title: Enhancing Adaptive Video Streaming through Fuzzy Logic-Based Content Recommendation Systems: A Comprehensive Review and Future Directions
Authors: Koffka Khan
Comments: 7 pages
Subjects: Information Retrieval (cs.IR)

As the demand for high-quality video content continues to rise, adaptive video streaming plays a pivotal role in delivering an optimal viewing experience. However, traditional content recommendation systems face challenges in dynamically adapting to users' preferences, content features, and contextual information. This review paper explores the integration of fuzzy logic into content recommendation systems for adaptive video streaming. Fuzzy logic, known for handling uncertainty and imprecision, provides a promising framework for modeling and accommodating the dynamic nature of user preferences and contextual factors. The paper discusses the evolution of adaptive video streaming, reviews traditional content recommendation algorithms, and introduces fuzzy logic as a solution to enhance the adaptability of these systems. Through a comprehensive exploration of case studies and applications, the effectiveness of fuzzy logic in improving user satisfaction and system performance is highlighted. The review also addresses challenges associated with the integration of fuzzy logic and suggests future research directions to further advance this approach. The proposed framework offers insights into a dynamic and context-aware content recommendation system, contributing to the evolution of adaptive video streaming technologies.

[15]  arXiv:2404.08692 [pdf, other]
Title: Apollonion: Profile-centric Dialog Agent
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

The emergence of Large Language Models (LLMs) has innovated the development of dialog agents. Specially, a well-trained LLM, as a central process unit, is capable of providing fluent and reasonable response for user's request. Besides, auxiliary tools such as external knowledge retrieval, personalized character for vivid response, short/long-term memory for ultra long context management are developed, completing the usage experience for LLM-based dialog agents. However, the above-mentioned techniques does not solve the issue of \textbf{personalization from user perspective}: agents response in a same fashion to different users, without consideration of their features, such as habits, interests and past experience. In another words, current implementation of dialog agents fail in ``knowing the user''. The capacity of well-description and representation of user is under development. In this work, we proposed a framework for dialog agent to incorporate user profiling (initialization, update): user's query and response is analyzed and organized into a structural user profile, which is latter served to provide personal and more precise response. Besides, we proposed a series of evaluation protocols for personalization: to what extend the response is personal to the different users.
The framework is named as \method{}, inspired by inscription of ``Know Yourself'' in the temple of Apollo (also known as \method{}) in Ancient Greek. Few works have been conducted on incorporating personalization into LLM, \method{} is a pioneer work on guiding LLM's response to meet individuation via the application of dialog agents, with a set of evaluation methods for measurement in personalization.

[16]  arXiv:2404.08694 [pdf, ps, other]
Title: Musical Listening Qualia: A Multivariate Approach
Journal-ref: Auditory Perception & Cognition, 2022, 5 (1-2), pp.46-75.
Subjects: Information Retrieval (cs.IR); Methodology (stat.ME)

French and American participants listened to new music stimuli and evaluated the stimuli using either adjectives or quantitative musical dimensions. Results were analyzed using correspondence analysis (CA), hierarchical cluster analysis (HCA), multiple factor analysis (MFA), and partial least squares correlation (PLSC). French and American listeners differed when they described the musical stimuli using adjectives, but not when using the quantitative dimensions. The present work serves as a case study in research methodology that allows for a balance between relaxing experimental control and maintaining statistical rigor.

[17]  arXiv:2404.08796 [pdf, other]
Title: The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation
Comments: 10 pages
Subjects: Information Retrieval (cs.IR)

Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs). Some PLM-based SR models directly use PLM to encode user historical behavior's text sequences to learn user representations, while there is seldom an in-depth exploration of the capability and suitability of PLM in behavior sequence modeling. In this work, we first conduct extensive model analyses between PLMs and PLM-based SR models, discovering great underutilization and parameter redundancy of PLMs in behavior sequence modeling. Inspired by this, we explore different lightweight usages of PLMs in SR, aiming to maximally stimulate the ability of PLMs for SR while satisfying the efficiency and usability demands of practical systems. We discover that adopting behavior-tuned PLMs for item initializations of conventional ID-based SR models is the most economical framework of PLM-based SR, which would not bring in any additional inference cost but could achieve a dramatic performance boost compared with the original version. Extensive experiments on five datasets show that our simple and universal framework leads to significant improvement compared to classical SR and SOTA PLM-based SR models without additional inference costs.

[18]  arXiv:2404.08829 [pdf, other]
Title: Measuring the Predictability of Recommender Systems using Structural Complexity Metrics
Comments: Accepted at WWW-24 Workshop: DCAI Data-centric Artificial Intelligence
Subjects: Information Retrieval (cs.IR); Information Theory (cs.IT); Machine Learning (cs.LG)

Recommender systems (RS) are central to the filtering and curation of online content. These algorithms predict user ratings for unseen items based on past preferences. Despite their importance, the innate predictability of RS has received limited attention. This study introduces data-driven metrics to measure the predictability of RS based on the structural complexity of the user-item rating matrix. A low predictability score indicates complex and unpredictable user-item interactions, while a high predictability score reveals less complex patterns with predictive potential. We propose two strategies that use singular value decomposition (SVD) and matrix factorization (MF) to measure structural complexity. By perturbing the data and evaluating the prediction of the perturbed version, we explore the structural consistency indicated by the SVD singular vectors. The assumption is that a random perturbation of highly structured data does not change its structure. Empirical results show a high correlation between our metrics and the accuracy of the best-performing prediction algorithms on real data sets.

[19]  arXiv:2404.08847 [pdf, other]
Title: LazyDP: Co-Designing Algorithm-Software for Scalable Training of Differentially Private Recommendation Models
Journal-ref: Published at 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS-29), 2024
Subjects: Information Retrieval (cs.IR); Cryptography and Security (cs.CR); Machine Learning (cs.LG)

Differential privacy (DP) is widely being employed in the industry as a practical standard for privacy protection. While private training of computer vision or natural language processing applications has been studied extensively, the computational challenges of training of recommender systems (RecSys) with DP have not been explored. In this work, we first present our detailed characterization of private RecSys training using DP-SGD, root-causing its several performance bottlenecks. Specifically, we identify DP-SGD's noise sampling and noisy gradient update stage to suffer from a severe compute and memory bandwidth limitation, respectively, causing significant performance overhead in training private RecSys. Based on these findings, we propose LazyDP, an algorithm-software co-design that addresses the compute and memory challenges of training RecSys with DP-SGD. Compared to a state-of-the-art DP-SGD training system, we demonstrate that LazyDP provides an average 119x training throughput improvement while also ensuring mathematically equivalent, differentially private RecSys models to be trained.

[20]  arXiv:2404.08860 [pdf, other]
Title: Improving Technical "How-to" Query Accuracy with Automated Search Results Verification and Reranking
Comments: 12 pages, 2 columns, 3 figures
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)

Many people use search engines to find online guidance to solve computer or mobile device problems. Users frequently encounter challenges in identifying effective solutions from search results, often wasting time trying ineffective solutions that seem relevant yet fail to solve the real problems. This paper introduces a novel approach to improving the accuracy and relevance of online technical support search results through automated search results verification and reranking. Taking "How-to" queries specific to on-device execution as a starting point, we first developed a solution that allows an AI agent to interpret and execute step-by-step instructions in the search results in a controlled Android environment. We further integrated the agent's findings into a reranking mechanism that orders search results based on the success indicators of the tested solutions.
The paper details the architecture of our solution and a comprehensive evaluation of the system through a series of tests across various application domains. The results demonstrate a significant improvement in the quality and reliability of the top-ranked results. Our findings suggest a paradigm shift in how search engine ranking for online technical support help can be optimized, offering a scalable and automated solution to the pervasive challenge of finding effective and reliable online help.

[21]  arXiv:2404.08869 [pdf, other]
Title: Misinformation Resilient Search Rankings with Webgraph-based Interventions
Subjects: Information Retrieval (cs.IR); Social and Information Networks (cs.SI)

The proliferation of unreliable news domains on the internet has had wide-reaching negative impacts on society. We introduce and evaluate interventions aimed at reducing traffic to unreliable news domains from search engines while maintaining traffic to reliable domains. We build these interventions on the principles of fairness (penalize sites for what is in their control), generality (label/fact-check agnostic), targeted (increase the cost of adversarial behavior), and scalability (works at webscale). We refine our methods on small-scale webdata as a testbed and then generalize the interventions to a large-scale webgraph containing 93.9M domains and 1.6B edges. We demonstrate that our methods penalize unreliable domains far more than reliable domains in both settings and we explore multiple avenues to mitigate unintended effects on both the small-scale and large-scale webgraph experiments. These results indicate the potential of our approach to reduce the spread of misinformation and foster a more reliable online information ecosystem. This research contributes to the development of targeted strategies to enhance the trustworthiness and quality of search engine results, ultimately benefiting users and the broader digital community.

[22]  arXiv:2404.08887 [pdf, other]
Title: Countering Mainstream Bias via End-to-End Adaptive Local Learning
Comments: ECIR 2024
Journal-ref: In European Conference on Information Retrieval 2024, vol 14612 (pp. 75-89)
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)

Collaborative filtering (CF) based recommendations suffer from mainstream bias -- where mainstream users are favored over niche users, leading to poor recommendation quality for many long-tail users. In this paper, we identify two root causes of this mainstream bias: (i) discrepancy modeling, whereby CF algorithms focus on modeling mainstream users while neglecting niche users with unique preferences; and (ii) unsynchronized learning, where niche users require more training epochs than mainstream users to reach peak performance. Targeting these causes, we propose a novel end-To-end Adaptive Local Learning (TALL) framework to provide high-quality recommendations to both mainstream and niche users. TALL uses a loss-driven Mixture-of-Experts module to adaptively ensemble experts to provide customized local models for different users. Further, it contains an adaptive weight module to synchronize the learning paces of different users by dynamically adjusting weights in the loss. Extensive experiments demonstrate the state-of-the-art performance of the proposed model. Code and data are provided at \url{https://github.com/JP-25/end-To-end-Adaptive-Local-Leanring-TALL-}

[23]  arXiv:2404.08896 [pdf, other]
Title: Approximate Cluster-Based Sparse Document Retrieval with Segmented Maximum Term Weights
Subjects: Information Retrieval (cs.IR)

This paper revisits cluster-based retrieval that partitions the inverted index into multiple groups and skips the index partially at cluster and document levels during online inference using a learned sparse representation. It proposes an approximate search scheme with two parameters to control the rank-safeness competitiveness of pruning with segmented maximum term weights within each cluster. Cluster-level maximum weight segmentation allows an improvement in the rank score bound estimation and threshold-based pruning to be approximately adaptive to bound estimation tightness, resulting in better relevance and efficiency. The experiments with MS MARCO passage ranking and BEIR datasets demonstrate the usefulness of the proposed scheme with a comparison to the baselines. This paper presents the design of this approximate retrieval scheme with rank-safeness analysis, compares clustering and segmentation options, and reports evaluation results.

[24]  arXiv:2404.08940 [pdf, other]
Title: Introducing Super RAGs in Mistral 8x7B-v1
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL); Machine Learning (cs.LG)

The relentless pursuit of enhancing Large Language Models (LLMs) has led to the advent of Super Retrieval-Augmented Generation (Super RAGs), a novel approach designed to elevate the performance of LLMs by integrating external knowledge sources with minimal structural modifications. This paper presents the integration of Super RAGs into the Mistral 8x7B v1, a state-of-the-art LLM, and examines the resultant improvements in accuracy, speed, and user satisfaction. Our methodology uses a fine-tuned instruct model setup and a cache tuning fork system, ensuring efficient and relevant data retrieval. The evaluation, conducted over several epochs, demonstrates significant enhancements across all metrics. The findings suggest that Super RAGs can effectively augment LLMs, paving the way for more sophisticated and reliable AI systems. This research contributes to the field by providing empirical evidence of the benefits of Super RAGs and offering insights into their potential applications.

[25]  arXiv:2404.09091 [pdf, other]
Title: Semantic In-Domain Product Identification for Search Queries
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)

Accurate explicit and implicit product identification in search queries is critical for enhancing user experiences, especially at a company like Adobe which has over 50 products and covers queries across hundreds of tools. In this work, we present a novel approach to training a product classifier from user behavioral data. Our semantic model led to >25% relative improvement in CTR (click through rate) across the deployed surfaces; a >50% decrease in null rate; a 2x increase in the app cards surfaced, which helps drive product visibility.

[26]  arXiv:2404.09253 [pdf, other]
Title: Competitive Retrieval: Going Beyond the Single Query
Subjects: Information Retrieval (cs.IR); Computer Science and Game Theory (cs.GT)

Previous work on the competitive retrieval setting focused on a single-query setting: document authors manipulate their documents so as to improve their future ranking for a given query. We study a competitive setting where authors opt to improve their document's ranking for multiple queries. We use game theoretic analysis to prove that equilibrium does not necessarily exist. We then empirically show that it is more difficult for authors to improve their documents' rankings for multiple queries with a neural ranker than with a state-of-the-art feature-based ranker. We also present an effective approach for predicting the document most highly ranked in the next induced ranking.

[27]  arXiv:2404.09473 [pdf, other]
Title: Exploring the Nexus Between Retrievability and Query Generation Strategies
Comments: Accepted at ECIR 2024
Subjects: Information Retrieval (cs.IR)

Quantifying bias in retrieval functions through document retrievability scores is vital for assessing recall-oriented retrieval systems. However, many studies investigating retrieval model bias lack validation of their query generation methods as accurate representations of retrievability for real users and their queries. This limitation results from the absence of established criteria for query generation in retrievability assessments. Typically, researchers resort to using frequent collocations from document corpora when no query log is available. In this study, we address the issue of reproducibility and seek to validate query generation methods by comparing retrievability scores generated from artificially generated queries to those derived from query logs. Our findings demonstrate a minimal or negligible correlation between retrievability scores from artificial queries and those from query logs. This suggests that artificially generated queries may not accurately reflect retrievability scores as derived from query logs. We further explore alternative query generation techniques, uncovering a variation that exhibits the highest correlation. This alternative approach holds promise for improving reproducibility when query logs are unavailable.

[28]  arXiv:2404.09520 [pdf, other]
Title: UniSAR: Modeling User Transition Behaviors between Search and Recommendation
Comments: Accepted by SIGIR 2024
Subjects: Information Retrieval (cs.IR)

Nowadays, many platforms provide users with both search and recommendation services as important tools for accessing information. The phenomenon has led to a correlation between user search and recommendation behaviors, providing an opportunity to model user interests in a fine-grained way. Existing approaches either model user search and recommendation behaviors separately or overlook the different transitions between user search and recommendation behaviors. In this paper, we propose a framework named UniSAR that effectively models the different types of fine-grained behavior transitions for providing users a Unified Search And Recommendation service. Specifically, UniSAR models the user transition behaviors between search and recommendation through three steps: extraction, alignment, and fusion, which are respectively implemented by transformers equipped with pre-defined masks, contrastive learning that aligns the extracted fine-grained user transitions, and cross-attentions that fuse different transitions. To provide users with a unified service, the learned representations are fed into the downstream search and recommendation models. Joint learning on both search and recommendation data is employed to utilize the knowledge and enhance each other. Experimental results on two public datasets demonstrated the effectiveness of UniSAR in terms of enhancing both search and recommendation simultaneously. The experimental analysis further validates that UniSAR enhances the results by successfully modeling the user transition behaviors between search and recommendation.

[29]  arXiv:2404.09578 [pdf, other]
Title: Recall-Augmented Ranking: Enhancing Click-Through Rate Prediction Accuracy with Cross-Stage Data
Comments: 4 pages, accepted by WWW 2024 Short Track
Subjects: Information Retrieval (cs.IR)

Click-through rate (CTR) prediction plays an indispensable role in online platforms. Numerous models have been proposed to capture users' shifting preferences by leveraging user behavior sequences. However, these historical sequences often suffer from severe homogeneity and scarcity compared to the extensive item pool. Relying solely on such sequences for user representations is inherently restrictive, as user interests extend beyond the scope of items they have previously engaged with. To address this challenge, we propose a data-driven approach to enrich user representations. We recognize user profiling and recall items as two ideal data sources within the cross-stage framework, encompassing the u2u (user-to-user) and i2i (item-to-item) aspects respectively. In this paper, we propose a novel architecture named Recall-Augmented Ranking (RAR). RAR consists of two key sub-modules, which synergistically gather information from a vast pool of look-alike users and recall items, resulting in enriched user representations. Notably, RAR is orthogonal to many existing CTR models, allowing for consistent performance improvements in a plug-and-play manner. Extensive experiments are conducted, which verify the efficacy and compatibility of RAR against the SOTA methods.

[30]  arXiv:2404.09709 [pdf, other]
Title: Scenario-Adaptive Fine-Grained Personalization Network: Tailoring User Behavior Representation to the Scenario Context
Comments: Accepted by SIGIR 2024, 10 pages, 5 figures, 5 tables
Journal-ref: SIGIR 2024
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)

Existing methods often adjust representations adaptively only after aggregating user behavior sequences. This coarse-grained approach to re-weighting the entire user sequence hampers the model's ability to accurately model the user interest migration across different scenarios. To enhance the model's capacity to capture user interests from historical behavior sequences in each scenario, we develop a ranking framework named the Scenario-Adaptive Fine-Grained Personalization Network (SFPNet), which designs a kind of fine-grained method for multi-scenario personalized recommendations. Specifically, SFPNet comprises a series of blocks named as Scenario-Tailoring Block, stacked sequentially. Each block initially deploys a parameter personalization unit to integrate scenario information at a coarse-grained level by redefining fundamental features. Subsequently, we consolidate scenario-adaptively adjusted feature representations to serve as context information. By employing residual connection, we incorporate this context into the representation of each historical behavior, allowing for context-aware fine-grained customization of the behavior representations at the scenario-level, which in turn supports scenario-aware user interest modeling.

[31]  arXiv:2404.09889 [pdf, other]
Title: Is Table Retrieval a Solved Problem? Join-Aware Multi-Table Retrieval
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Retrieving relevant tables containing the necessary information to accurately answer a given question over tables is critical to open-domain question-answering (QA) systems. Previous methods assume the answer to such a question can be found either in a single table or multiple tables identified through question decomposition or rewriting. However, neither of these approaches is sufficient, as many questions require retrieving multiple tables and joining them through a join plan that cannot be discerned from the user query itself. If the join plan is not considered in the retrieval stage, the subsequent steps of reasoning and answering based on those retrieved tables are likely to be incorrect. To address this problem, we introduce a method that uncovers useful join relations for any query and database during table retrieval. We use a novel re-ranking method formulated as a mixed-integer program that considers not only table-query relevance but also table-table relevance that requires inferring join relationships. Our method outperforms the state-of-the-art approaches for table retrieval by up to 9.3% in F1 score and for end-to-end QA by up to 5.4% in accuracy.

Cross-lists for Tue, 16 Apr 24

[32]  arXiv:2404.08680 (cross-list from cs.CL) [pdf, other]
Title: Automating Research Synthesis with Domain-Specific Large Language Model Fine-Tuning
Subjects: Computation and Language (cs.CL); Digital Libraries (cs.DL); Information Retrieval (cs.IR)

This research pioneers the use of fine-tuned Large Language Models (LLMs) to automate Systematic Literature Reviews (SLRs), presenting a significant and novel contribution in integrating AI to enhance academic research methodologies. Our study employed the latest fine-tuning methodologies together with open-sourced LLMs, and demonstrated a practical and efficient approach to automating the final execution stages of an SLR process that involves knowledge synthesis. The results maintained high fidelity in factual accuracy in LLM responses, and were validated through the replication of an existing PRISMA-conforming SLR. Our research proposed solutions for mitigating LLM hallucination and proposed mechanisms for tracking LLM responses to their sources of information, thus demonstrating how this approach can meet the rigorous demands of scholarly research. The findings ultimately confirmed the potential of fine-tuned LLMs in streamlining various labor-intensive processes of conducting literature reviews. Given the potential of this approach and its applicability across all research domains, this foundational study also advocated for updating PRISMA reporting guidelines to incorporate AI-driven processes, ensuring methodological transparency and reliability in future SLRs. This study broadens the appeal of AI-enhanced tools across various academic and research fields, setting a new standard for conducting comprehensive and accurate literature reviews with more efficiency in the face of ever-increasing volumes of academic studies.

[33]  arXiv:2404.08695 (cross-list from cs.CL) [pdf, other]
Title: Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Models
Comments: DASFAA 2024 Accepted
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

Efficient knowledge management plays a pivotal role in augmenting both the operational efficiency and the innovative capacity of businesses and organizations. By indexing knowledge through vectorization, a variety of knowledge retrieval methods have emerged, significantly enhancing the efficacy of knowledge management systems. Recently, the rapid advancements in generative natural language processing technologies paved the way for generating precise and coherent answers after retrieving relevant documents tailored to user queries. However, for enterprise knowledge bases, assembling extensive training data from scratch for knowledge retrieval and generation is a formidable challenge due to the privacy and security policies of private data, frequently entailing substantial costs. To address the challenge above, in this paper, we propose EKRG, a novel Retrieval-Generation framework based on large language models (LLMs), expertly designed to enable question-answering for Enterprise Knowledge bases with limited annotation costs. Specifically, for the retrieval process, we first introduce an instruction-tuning method using an LLM to generate sufficient document-question pairs for training a knowledge retriever. This method, through carefully designed instructions, efficiently generates diverse questions for enterprise knowledge bases, encompassing both fact-oriented and solution-oriented knowledge. Additionally, we develop a relevance-aware teacher-student learning strategy to further enhance the efficiency of the training process. For the generation process, we propose a novel chain of thought (CoT) based fine-tuning method to empower the LLM-based generator to adeptly respond to user questions using retrieved documents. Finally, extensive experiments on real-world datasets have demonstrated the effectiveness of our proposed framework.

[34]  arXiv:2404.08720 (cross-list from cs.LG) [pdf, other]
Title: Exploring Contrastive Learning for Long-Tailed Multi-Label Text Classification
Comments: 14 pages, 2 figures
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Information Retrieval (cs.IR)

Learning an effective representation in multi-label text classification (MLTC) is a significant challenge in NLP. This challenge arises from the inherent complexity of the task, which is shaped by two key factors: the intricate connections between labels and the widespread long-tailed distribution of the data. To overcome this issue, one potential approach involves integrating supervised contrastive learning with classical supervised loss functions. Although contrastive learning has shown remarkable performance in multi-class classification, its impact in the multi-label framework has not been thoroughly investigated. In this paper, we conduct an in-depth study of supervised contrastive learning and its influence on representation in MLTC context. We emphasize the importance of considering long-tailed data distributions to build a robust representation space, which effectively addresses two critical challenges associated with contrastive learning that we identify: the "lack of positives" and the "attraction-repulsion imbalance". Building on this insight, we introduce a novel contrastive loss function for MLTC. It attains Micro-F1 scores that either match or surpass those obtained with other frequently employed loss functions, and demonstrates a significant improvement in Macro-F1 scores across three multi-label datasets.

[35]  arXiv:2404.08886 (cross-list from cs.CV) [pdf, other]
Title: EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLM
Comments: Accepted by NAACL 2024 Industry Track
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)

In e-commerce, accurately extracting product attribute values from multimodal data is crucial for improving user experience and operational efficiency of retailers. However, previous approaches to multimodal attribute value extraction often struggle with implicit attribute values embedded in images or text, rely heavily on extensive labeled data, and can easily confuse similar attribute values. To address these issues, we introduce EIVEN, a data- and parameter-efficient generative framework that pioneers the use of multimodal LLM for implicit attribute value extraction. EIVEN leverages the rich inherent knowledge of a pre-trained LLM and vision encoder to reduce reliance on labeled data. We also introduce a novel Learning-by-Comparison technique to reduce model confusion by enforcing attribute value comparison and difference identification. Additionally, we construct initial open-source datasets for multimodal implicit attribute value extraction. Our extensive experiments reveal that EIVEN significantly outperforms existing methods in extracting implicit attribute values while requiring less labeled data.

[36]  arXiv:2404.09077 (cross-list from cs.CL) [pdf, other]
Title: CuriousLLM: Elevating Multi-Document QA with Reasoning-Infused Knowledge Graph Prompting
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG)

In the field of Question Answering (QA), unifying large language models (LLMs) with external databases has shown great success. However, these methods often fall short in providing the advanced reasoning needed for complex QA tasks. To address these issues, we improve over a novel approach called Knowledge Graph Prompting (KGP), which combines knowledge graphs with a LLM-based agent to improve reasoning and search accuracy. Nevertheless, the original KGP framework necessitates costly fine-tuning with large datasets yet still suffers from LLM hallucination. Therefore, we propose a reasoning-infused LLM agent to enhance this framework. This agent mimics human curiosity to ask follow-up questions to more efficiently navigate the search. This simple modification significantly boosts the LLM performance in QA tasks without the high costs and latency associated with the initial KGP framework. Our ultimate goal is to further develop this approach, leading to more accurate, faster, and cost-effective solutions in the QA domain.

[37]  arXiv:2404.09286 (cross-list from cs.CR) [pdf, ps, other]
Title: Artificial Intelligence enhanced Security Problems in Real-Time Scenario using Blowfish Algorithm
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR)

In a nutshell, "the cloud" refers to a collection of interconnected computing resources made possible by an extensive, real-time communication network like the internet. Because of its potential to reduce processing costs, the emerging paradigm of cloud computing has recently attracted a large number of academics. The exponential expansion of cloud computing has made the rapid expansion of cloud services very remarkable. Ensuring the security of personal information in today's interconnected world is no easy task. These days, security is really crucial. Models of security that are relevant to cloud computing include confidentiality, authenticity, accessibility, data integrity, and recovery. Using the Hybrid Encryption this study, we cover all the security issues and leaks in cloud infrastructure.

[38]  arXiv:2404.09980 (cross-list from cs.CL) [pdf, other]
Title: Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems
Comments: Accepted at NAACL 2024 Findings
Subjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR)

Crowdsourced labels play a crucial role in evaluating task-oriented dialogue systems (TDSs). Obtaining high-quality and consistent ground-truth labels from annotators presents challenges. When evaluating a TDS, annotators must fully comprehend the dialogue before providing judgments. Previous studies suggest using only a portion of the dialogue context in the annotation process. However, the impact of this limitation on label quality remains unexplored. This study investigates the influence of dialogue context on annotation quality, considering the truncated context for relevance and usefulness labeling. We further propose to use large language models (LLMs) to summarize the dialogue context to provide a rich and short description of the dialogue context and study the impact of doing so on the annotator's performance. Reducing context leads to more positive ratings. Conversely, providing the entire dialogue context yields higher-quality relevance ratings but introduces ambiguity in usefulness ratings. Using the first user utterance as context leads to consistent ratings, akin to those obtained using the entire dialogue, with significantly reduced annotation effort. Our findings show how task design, particularly the availability of dialogue context, affects the quality and consistency of crowdsourced evaluation labels.

Replacements for Tue, 16 Apr 24

[39]  arXiv:2009.10128 (replaced) [pdf, ps, other]
Title: Claraprint: a chord and melody based fingerprint for western classical music cover detection
Authors: Mickaël Arcos
Subjects: Information Retrieval (cs.IR)
[40]  arXiv:2305.13859 (replaced) [pdf, other]
Title: Generative Retrieval via Term Set Generation
Subjects: Information Retrieval (cs.IR)
[41]  arXiv:2307.02046 (replaced) [pdf, other]
Title: Recommender Systems in the Era of Large Language Models (LLMs)
Comments: Accepted by IEEE TKDE
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
[42]  arXiv:2309.10435 (replaced) [pdf, other]
Title: Reformulating Sequential Recommendation: Learning Dynamic User Interest with Content-enriched Language Modeling
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL)
[43]  arXiv:2312.09591 (replaced) [pdf, other]
Title: Explicitly Integrating Judgment Prediction with Legal Document Retrieval: A Law-Guided Generative Approach
Comments: Accepted by SIGIR'2024
Subjects: Information Retrieval (cs.IR)
[44]  arXiv:2401.12798 (replaced) [pdf, other]
Title: Gradient Flow of Energy: A General and Efficient Approach for Entity Alignment Decoding
Subjects: Information Retrieval (cs.IR); Computation and Language (cs.CL)
[45]  arXiv:2403.17416 (replaced) [pdf, other]
Title: AFDGCF: Adaptive Feature De-correlation Graph Collaborative Filtering for Recommendations
Comments: Accepted by SIGIR2024
Subjects: Information Retrieval (cs.IR)
[46]  arXiv:2404.04264 (replaced) [pdf, other]
Title: Logic Query of Thoughts: Guiding Large Language Models to Answer Complex Logic Queries with Knowledge Graphs
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
[47]  arXiv:2404.07494 (replaced) [pdf, other]
Title: Adaptive Fair Representation Learning for Personalized Fairness in Recommendations via Information Alignment
Comments: This paper has been accepted by SIGIR '24
Subjects: Information Retrieval (cs.IR)
[48]  arXiv:2404.07581 (replaced) [pdf, other]
Title: M-scan: A Multi-Scenario Causal-driven Adaptive Network for Recommendation
Comments: This paper has been accepted by WWW'24
Subjects: Information Retrieval (cs.IR)
[49]  arXiv:2404.08361 (replaced) [pdf, other]
Title: Large-Scale Multi-Domain Recommendation: an Automatic Domain Feature Extraction and Personalized Integration Framework
Comments: 8 pages
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)
[50]  arXiv:2306.01163 (replaced) [pdf, other]
Title: A Multi-Modal Latent-Features based Service Recommendation System for the Social Internet of Things
Comments: Published in IEEE Transactions on Computational Social Systems
Subjects: Social and Information Networks (cs.SI); Information Retrieval (cs.IR)
[51]  arXiv:2308.11767 (replaced) [pdf, other]
Title: Detection of ChatGPT Fake Science with the xFakeSci Learning Algorithm
Comments: 18 pages, 8 figures, 8 tables, 5 algorithms
Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)
[ total of 51 entries: 1-51 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2404, contact, help  (Access key information)