Current browse context:
stat.AP
Change to browse by:
References & Citations
Statistics > Applications
Title: How to Deal With Ratio Metrics When Accounting for Intra-User Correlation in A/B Testing
(Submitted on 8 Nov 2019 (this version), latest version 23 Jul 2020 (v2))
Abstract: We consider the A/B testing problem at the presence of correlation among observations coming from the same user. Furthermore, users may come from various segments where levels of correlation are different. A uniformly minimum-variance unbiased estimator of the population mean, called correlation-adjusted mean, is proposed to account for such correlation structure. It is proved theoretically and numerically better than the usual naive mean estimator and normalized mean estimator (taking average within users and then across users). The correlation-adjusted mean method is still unbiased but has reduced variance so it gains additional power. Several simulation studies are designed to show the estimation accuracy of the correlation structure, effectiveness in reducing variance, and capability of obtaining more power. An application to the eBay data is conducted to conclude this paper.
Submission history
From: Keyu Nie [view email][v1] Fri, 8 Nov 2019 21:40:42 GMT (184kb,D)
[v2] Thu, 23 Jul 2020 00:59:02 GMT (182kb,D)
Link back to: arXiv, form interface, contact.