We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.NI

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Networking and Internet Architecture

Title: LEAF: Navigating Concept Drift in Cellular Networks

Abstract: Operational networks commonly rely on machine learning models for many tasks, including detecting anomalies, inferring application performance, and forecasting demand. Yet, unfortunately, model accuracy can degrade due to concept drift, whereby the relationship between the features and the target prediction changes due to reasons ranging from software upgrades to seasonality to changes in user behavior. Mitigating concept drift is thus an essential part of operationalizing machine learning models, and yet despite its importance, concept drift has not been extensively explored in the context of networking -- or regression models in general. Thus, it is not well-understood how to detect or mitigate it for many common network management tasks that currently rely on machine learning models. Unfortunately, as we show, concept drift cannot be sufficiently mitigated by frequently retraining models using newly available data, and doing so can even degrade model accuracy further. In this paper, we characterize concept drift in a large cellular network for a major metropolitan area in the United States. We find that concept drift occurs across many important key performance indicators (KPIs), independently of the model, training set size, and time interval -- thus necessitating practical approaches to detect, explain, and mitigate it. To do so, we develop Local Error Approximation of Features (LEAF). LEAF detects drift; explains features and time intervals that most contribute to drift; and mitigates drift using forgetting and over-sampling. We evaluate LEAF against industry-standard mitigation approaches with more than four years of cellular KPI data. Our initial tests with a major cellular provider in the US show that LEAF is effective on a variety of KPIs and models. LEAF consistently outperforms periodic and triggered retraining while reducing costly retraining operations.
Subjects: Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG); Performance (cs.PF)
Cite as: arXiv:2109.03011 [cs.NI]
  (or arXiv:2109.03011v4 [cs.NI] for this version)

Submission history

From: Shinan Liu [view email]
[v1] Tue, 7 Sep 2021 11:57:07 GMT (15740kb,D)
[v2] Sat, 12 Feb 2022 14:54:49 GMT (39889kb,D)
[v3] Tue, 15 Feb 2022 16:21:04 GMT (39889kb,D)
[v4] Thu, 11 Aug 2022 19:33:23 GMT (37448kb,D)

Link back to: arXiv, form interface, contact.