Optimal Design of Online Experiments

The design of online experiments plays an important role in product development and effcacy validation. A good online experimental design should meet two essential criteria: when there are no active factors, the experiment should be stopped in time; when there are active factors, the arrangement should allow researchers to accurately estimate the treatment effects or the response surface. In this paper, we investigate the problem of experiment arrangement under the framework of online multi-factorial design using linear models. Traditional online experiment designs based on p-values may fail to control the type I error rate. We first optimize the classical online design method by using continuously monitored e-values to determine the stopping time, which effectively controls the type I error. Then, we construct design criteria to find the optimal design under the online experimental framework, and compare the proposed approach with traditional methods via simulation studies.