在线试验的最优设计
Optimal Design of Online Experiments
-
摘要: 在线试验设计在产品的开发与有效性验证中扮演着重要的角色.一个好的在线试验设计需要满足两个重要特性:当试验中没有活跃因子时,可以及时暂停试验;当试验中有活跃因子时,试验的安排可以能够让研究者更准确地估准处理效应或者输出的响应曲面.本文将以线性模型的多因子试验设计为例,讨论在线试验设计框架下的试验安排问题.传统的在线试验设计中,基于p值构造的停时规则会导致第一类错误无法控制.本文首先对传统在线试验设计方法进行了优化,通过持续监控e值来决定试验的终止时机,能够有效控制第一类错误.其次,本文构建了设计准则来寻找在线试验设计框架下的最优试验设计,并在数值模拟实验中与传统的设计方法进行了比较.Abstract: The design of online experiments plays an important role in product development and effcacy validation. A good online experimental design should meet two essential criteria: when there are no active factors, the experiment should be stopped in time; when there are active factors, the arrangement should allow researchers to accurately estimate the treatment effects or the response surface. In this paper, we investigate the problem of experiment arrangement under the framework of online multi-factorial design using linear models. Traditional online experiment designs based on p-values may fail to control the type I error rate. We first optimize the classical online design method by using continuously monitored e-values to determine the stopping time, which effectively controls the type I error. Then, we construct design criteria to find the optimal design under the online experimental framework, and compare the proposed approach with traditional methods via simulation studies.
下载: