Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Geometric Insights into Support Vector Machine Behavior using the KKT Conditions
(Submitted on 3 Apr 2017 (v1), last revised 10 Oct 2018 (this version, v2))
Abstract: The support vector machine (SVM) is a powerful and widely used classification algorithm. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM. These insights provide perhaps unexpected relationships between SVM and two other linear classifiers: the mean difference and the maximal data piling direction. For example, we show that in many cases SVM can be viewed as a cropped version of these classifiers. By carefully exploring these connections we show how SVM tuning behavior is affected by characteristics including: balanced vs. unbalanced classes, low vs. high dimension, separable vs. non-separable data. These results provide further insights into tuning SVM via cross-validation by explaining observed pathological behavior and motivating improved cross-validation methodology. Finally, we also provide new results on the geometry of complete data piling directions in high dimensional space.
Submission history
From: Iain Carmichael [view email][v1] Mon, 3 Apr 2017 19:08:54 GMT (2156kb,D)
[v2] Wed, 10 Oct 2018 13:43:27 GMT (2627kb,D)
Link back to: arXiv, form interface, contact.