Current browse context:
cs.HC
Change to browse by:
References & Citations
Computer Science > Human-Computer Interaction
Title: Parachute: Evaluating Interactive Human-LM Co-writing Systems
(Submitted on 11 Mar 2023 (v1), last revised 25 Mar 2023 (this version, v2))
Abstract: A surge of advances in language models (LMs) has led to significant interest in using LMs to build co-writing systems, in which humans and LMs interactively contribute to a shared writing artifact. However, there is a lack of studies assessing co-writing systems in interactive settings. We propose a human-centered evaluation framework, Parachute, for interactive co-writing systems. Parachute showcases an integrative view of interaction evaluation, where each evaluation aspect consists of categorized practical metrics. Furthermore, we present Parachute with a use case to demonstrate how to evaluate and compare co-writing systems using Parachute.
Submission history
From: Hua Shen [view email][v1] Sat, 11 Mar 2023 07:30:52 GMT (595kb,D)
[v2] Sat, 25 Mar 2023 01:40:34 GMT (596kb,D)
Link back to: arXiv, form interface, contact.