Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: LP-SparseMAP: Differentiable Relaxed Optimization for Sparse Structured Prediction
(Submitted on 13 Jan 2020 (v1), last revised 5 Aug 2020 (this version, v3))
Abstract: Structured prediction requires manipulating a large number of combinatorial structures, e.g., dependency trees or alignments, either as latent or output variables. Recently, the SparseMAP method has been proposed as a differentiable, sparse alternative to maximum a posteriori (MAP) and marginal inference. SparseMAP returns a combination of a small number of structures, a desirable property in some downstream applications. However, SparseMAP requires a tractable MAP inference oracle. This excludes, e.g., loopy graphical models or factor graphs with logic constraints, which generally require approximate inference. In this paper, we introduce LP-SparseMAP, an extension of SparseMAP that addresses this limitation via a local polytope relaxation. LP-SparseMAP uses the flexible and powerful domain specific language of factor graphs for defining and backpropagating through arbitrary hidden structure, supporting coarse decompositions, hard logic constraints, and higher-order correlations. We derive the forward and backward algorithms needed for using LP-SparseMAP as a hidden or output layer. Experiments in three structured prediction tasks show benefits compared to SparseMAP and Structured SVM.
Submission history
From: Vlad Niculae [view email][v1] Mon, 13 Jan 2020 18:16:13 GMT (67kb,D)
[v2] Mon, 29 Jun 2020 18:05:12 GMT (80kb,D)
[v3] Wed, 5 Aug 2020 15:36:49 GMT (90kb,D)
Link back to: arXiv, form interface, contact.