Researchers at the Wharton School at the University of Pennsylvania tested how consistently AI shopping agents recommend products when the search process changes. Even tiny shifts in context swung purchase decisions by wide margins.
The team tested six current models, both mini variants and frontier-level, tasking each one to act as a personal shopping assistant picking a fitness watch from a fixed product grid. They used the ACES simulator (Agentic e-Commerce Simulator), which shows the AI agent a screenshot of a product page. The agent analyzes the image, optionally pulls in recommendation sources, and then picks a product.
This is how the models received product information in the study. | Image: Kumar et al.
A single source is enough to flip the recommendation
Even without external sources, the models showed different baseline preferences. But when the agent saw just one external source before the product page, recommendations shifted dramatically in some cases.
