References & Citations
Statistics > Methodology
Title: A Fast and Efficient Change-point Detection Framework based on Approximate $k$-Nearest Neighbor Graphs
(Submitted on 24 Jun 2020 (v1), last revised 22 Mar 2022 (this version, v3))
Abstract: Change-point analysis is thriving in this big data era to address problems arising in many fields where massive data sequences are collected to study complicated phenomena over time. It plays an important role in processing these data by segmenting a long sequence into homogeneous parts for follow-up studies. The task requires the method to be able to process large datasets quickly and deal with various types of changes for high-dimensional data. We propose a new approach making use of approximate $k$-nearest neighbor information from the observations, and derive an analytic formula to control the type I error. The time complexity of our proposed method is $O\left(dn(\log n+k \log d)+nk^2\right)$ for an $n$-length sequence of $d$-dimensional data. The test statistic we consider incorporates a useful pattern for moderate- to high- dimensional data so that the proposed method could detect various types of changes in the sequence. The new approach is also asymptotic distribution free, facilitating its usage for a broader community. We apply our method to fMRI datasets and Neuropixels datasets to illustrate its effectiveness.
Submission history
From: Hao Chen [view email][v1] Wed, 24 Jun 2020 03:25:57 GMT (2058kb,D)
[v2] Sun, 20 Jun 2021 06:08:03 GMT (3232kb,D)
[v3] Tue, 22 Mar 2022 06:22:54 GMT (4431kb,D)
Link back to: arXiv, form interface, contact.