Performance

New submissions

• New submissions
• Cross-lists

New submissions for Wed, 24 Apr 24

[1]  arXiv:2404.14926 [pdf, ps, other]

Title: Towards self-optimization of publish/subscribe IoT systems using continuous performance monitoring

Authors: Mohammed DjahafiI, Nabila Salmi

Comments: 13 pages, 7 figures. Conference: 5th International Conference on Networks, Blockchain and Internet of Things (NBIoT 2024). Published by Computer Science Conference Proceedings in Computer Science & Information Technology (CS & IT)

Journal-ref: Computer Science & Information Technology (CS & IT) 2024

Subjects: Performance (cs.PF)

Today, more and more embedded devices are being connected through a network, generally Internet, offering users different services. This concept refers to Internet of Things (IoT), bringing information and control capabilities in many fields like medicine, smart homes, home security, etc. Main drawbacks of IoT environment are its dependency on Internet connectivity and need continuous devices power. These dependencies may affect system performances, namely request processing response times. In this context, we propose in this paper a continuous performance monitoring methodology, applied on IoT systems based on Publish/subscribe communication model. Our approach assesses performances using Stochastic Petri net modeling, and self-optimizes whenever poor performances are detected. Our approach relies on a Stochastic Petri nets modelling and analysis to assess performances. We target improving performances, in particular response times, by online modification of influencing factors.

Cross-lists for Wed, 24 Apr 24

[2]  arXiv:2404.15153 (cross-list from cs.CL) [pdf, other]

Title: Expert Router: Orchestrating Efficient Language Model Inference through Prompt Classification

Authors: Josef Pichlmeier, Philipp Ross, Andre Luckow

Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Performance (cs.PF)

Large Language Models (LLMs) have experienced widespread adoption across scientific and industrial domains due to their versatility and utility for diverse tasks. Nevertheless, deploying and serving these models at scale with optimal throughput and latency remains a significant challenge, primarily because of the high computational and memory demands associated with LLMs. To tackle this limitation, we introduce Expert Router, a system designed to orchestrate multiple expert models efficiently, thereby enhancing scalability. Expert Router is a parallel inference system with a central routing gateway that distributes incoming requests using a clustering method. This approach effectively partitions incoming requests among available LLMs, maximizing overall throughput. Our extensive evaluations encompassed up to 1,000 concurrent users, providing comprehensive insights into the system's behavior from user and infrastructure perspectives. The results demonstrate Expert Router's effectiveness in handling high-load scenarios and achieving higher throughput rates, particularly under many concurrent users.

• New submissions
• Cross-lists