We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Energy-Based Models for Code Generation under Compilability Constraints

Abstract: Neural language models can be successfully trained on source code, leading to applications such as code completion. However, their versatile autoregressive self-supervision objective overlooks important global sequence-level features that are present in the data such as syntactic correctness or compilability. In this work, we pose the problem of learning to generate compilable code as constraint satisfaction. We define an Energy-Based Model (EBM) representing a pre-trained generative model with an imposed constraint of generating only compilable sequences. We then use the KL-Adaptive Distributional Policy Gradient algorithm (Khalifa et al., 2021) to train a generative model approximating the EBM. We conduct experiments showing that our proposed approach is able to improve compilability rates without sacrificing diversity and complexity of the generated samples.
Comments: Accepted for the First Workshop on Natural Language Processing for Programming, ACL 2021
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Neural and Evolutionary Computing (cs.NE); Software Engineering (cs.SE)
ACM classes: I.2.2; I.2.7; I.2.6; I.5.1
Cite as: arXiv:2106.04985 [cs.LG]
  (or arXiv:2106.04985v1 [cs.LG] for this version)

Submission history

From: Tomek Korbak [view email]
[v1] Wed, 9 Jun 2021 11:06:32 GMT (5331kb,D)

Link back to: arXiv, form interface, contact.