We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Methodology

Title: Pigeonhole Design: Balancing Sequential Experiments from an Online Matching Perspective

Abstract: Practitioners and academics have long appreciated the benefits that experimentation brings to firms. For online web-facing firms, however, it still remains challenging in balancing covariate information when experimental units arrive sequentially in online field experiments. In this paper, we study a novel online experimental design problem, which we refer to as the "Online Blocking Problem." In this problem, experimental units with heterogeneous covariate information arrive sequentially and must be immediately assigned into either the control or the treatment group, with an objective of minimizing the total discrepancy, which is defined as the minimum weight perfect matching between the two groups. To solve this problem, we propose a novel experimental design approach, which we refer to as the "Pigeonhole Design." The pigeonhole design first partitions the covariate space into smaller spaces, which we refer to as pigeonholes, and then, when the experimental units arrive at each pigeonhole, balances the number of control and treatment units for each pigeonhole. We analyze the theoretical performance of the pigeonhole design and show its effectiveness by comparing against two well-known benchmark designs: the match-pair design and the completely randomized design. We conduct extensive simulations to study the numerical performance of the different designs and conclude with practical suggestions.
Subjects: Methodology (stat.ME); Econometrics (econ.EM)
Cite as: arXiv:2201.12936 [stat.ME]
  (or arXiv:2201.12936v3 [stat.ME] for this version)

Submission history

From: Jinglong Zhao [view email]
[v1] Sun, 30 Jan 2022 23:14:24 GMT (428kb,D)
[v2] Fri, 4 Feb 2022 00:46:45 GMT (448kb,D)
[v3] Tue, 8 Mar 2022 04:50:18 GMT (448kb,D)

Link back to: arXiv, form interface, contact.