We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: PhysNLU: A Language Resource for Evaluating Natural Language Understanding and Explanation Coherence in Physics

Abstract: In order for language models to aid physics research, they must first encode representations of mathematical and natural language discourse which lead to coherent explanations, with correct ordering and relevance of statements. We present a collection of datasets developed to evaluate the performance of language models in this regard, which measure capabilities with respect to sentence ordering, position, section prediction, and discourse coherence. Analysis of the data reveals equations and sub-disciplines which are most common in physics discourse, as well as the sentence-level frequency of equations and expressions. We present baselines that demonstrate how contemporary language models are challenged by coherence related tasks in physics, even when trained on mathematical natural language objectives.
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2201.04275 [cs.CL]
  (or arXiv:2201.04275v2 [cs.CL] for this version)

Submission history

From: Jordan Meadows [view email]
[v1] Wed, 12 Jan 2022 02:32:40 GMT (302kb,D)
[v2] Mon, 9 May 2022 00:08:14 GMT (302kb,D)

Link back to: arXiv, form interface, contact.