We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.SE

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Software Engineering

Title: AixBench: A Code Generation Benchmark Dataset

Authors: Yiyang Hao (1), Ge Li (2), Yongqiang Liu (1), Xiaowei Miao (1), He Zong (1), Siyuan Jiang (1), Yang Liu (1), He Wei (1) ((1) aiXcoder, (2) Peking University)
Abstract: We present a benchmark dataset for evaluating method-level code generation task. The benchmark contains a dataset of 175 samples for automated evaluation and a dataset of 161 samples for manual evaluation. We also present a new metric for automatically evaluating the correctness of the generated code, and a set of criteria to manually evaluating the overall quality of the generated code.
Subjects: Software Engineering (cs.SE)
Cite as: arXiv:2206.13179 [cs.SE]
  (or arXiv:2206.13179v2 [cs.SE] for this version)

Submission history

From: Yiyang Hao [view email]
[v1] Mon, 27 Jun 2022 10:44:48 GMT (202kb,D)
[v2] Thu, 21 Jul 2022 02:55:15 GMT (202kb,D)

Link back to: arXiv, form interface, contact.