TL;DR: LLM predictions can stand in for human outcomes in A/B tests, but only by assumption, not by design. In an application to the Upworthy dataset with thousands of A/B tests, calibrating LLM outputs against human data recovered the treatment effect, but only using a specific set of methods. The conditions that make this work cannot be verified for new treatments, and they become less plausible the further the new treatment is from past experiments. The promise is least justified precisely when it offers the most benefit.The promise is straightforward: run the experiment on a model instead of on users, get results in hours instead of weeks, and skip the traffic allocation entirely. But most proposals for replacing humans with LLM in A/B tests skip the statistical question that makes experiments valid in the first place: under what conditions does the experiment identify the treatment effect of interest?Randomized experiments are considered the gold-standard because they causally identify the treatment effect by design. Replacing real user responses with LLM-generated predictions removes that guarantee. Identification then holds only by assumption. We wrote a paper that formalizes what those assumptions are, using surrogate endpoint theory from biostatistics. Our idea is simple: If a surrogate outcome captures everything about a treatment that matters for the outcome, then, clearly, the surrogate is a valid proxy and experimenting on it will give the correct result. In clinical trials, biomarkers from labs are commonly used as fast and cheap surrogates for the clinical result. For A/B tests in digital environments, LLM predictions have become attractive candidates as surrogates for user responses. The promise is that LLM predictions sidesteps the effort, time, and opportunity costs of A/B testing.Raw LLM predictions are biased, not just noisyWe empirically evaluated the promise of LLM-based A/B testing using the Upworthy Research Archive, the largest open-access dataset on A/B tests currently available. The dataset contains click-through rates for variants of news headlines across thousands of A/B tests. We prompted gpt-4o-mini to predict the click-through rate of a typical user for each headline, separately for the treatment and control variants. Using those raw predictions in a standard experimental analysis recovered only 39% of the observed human treatment effect. If you used those predictions as if they were human data, you would conclude that treatments are less than half as effective as they actually are.This is not just random noise. The bias is systematic and directional: LLM outcomes attenuate treatment effects toward zero, making treatments look less effective than they are. A/B tests are used to learn what users value for making better product decisions. If LLM-based experiments are used across many features and product areas, that attenuation causes the organization to underestimate the value they bring to users, and potentially make incorrect shipping decisions.Two conditions make LLM outputs valid surrogates Our paper formalizes two conditions under which LLM predictions can be used to identify a human average treatment effect.Surrogacy. This assumption requires that the LLM output fully mediates the treatment effect on the human outcome. After accounting for the LLM's prediction and any covariates capturing baseline characteristics independent of treatment, the assignment of a user to the treatment or control condition tells you nothing additional about what the user would do. In plain language: the LLM captures everything about the treatment that matters for the human response. This assumption is often implicitly assumed but rarely spelled out in LLM-based A/B testing, nor is it commonly validated.