We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Attentive Neural Controlled Differential Equations for Time-series Classification and Forecasting

Abstract: Neural networks inspired by differential equations have proliferated for the past several years. Neural ordinary differential equations (NODEs) and neural controlled differential equations (NCDEs) are two representative examples of them. In theory, NCDEs provide better representation learning capability for time-series data than NODEs. In particular, it is known that NCDEs are suitable for processing irregular time-series data. Whereas NODEs have been successfully extended after adopting attention, however, it had not been studied yet how to integrate attention into NCDEs. To this end, we present the method of Attentive Neural Controlled Differential Equations (ANCDEs) for time-series classification and forecasting, where dual NCDEs are used: one for generating attention values, and the other for evolving hidden vectors for a downstream machine learning task. We conduct experiments with three real-world time-series datasets and 10 baselines. After dropping some values, we also conduct irregular time-series experiments. Our method consistently shows the best accuracy in all cases by non-trivial margins. Our visualizations also show that the presented attention mechanism works as intended by focusing on crucial information.
Comments: Accepted in ICDM 2021, Corrected typos
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
Cite as: arXiv:2109.01876 [cs.LG]
  (or arXiv:2109.01876v3 [cs.LG] for this version)

Submission history

From: Noseong Park [view email]
[v1] Sat, 4 Sep 2021 14:17:01 GMT (6026kb,D)
[v2] Fri, 12 Nov 2021 08:43:26 GMT (7093kb,D)
[v3] Tue, 16 Nov 2021 06:34:50 GMT (7277kb,D)

Link back to: arXiv, form interface, contact.