Surveys and polls help societies understand what people think about issues in politics, health, education and much more. But fewer people these days tend to respond, so pollsters have to reach out more widely, which raises cost considerably. One survey provider prices a 10 minute survey of 1,000 people in the tens of thousands of dollars.
Could AI models stand in for hundreds or thousands of people, emulating the range of answers humans would provide? This practice, known as synthetic surveys or silicon sampling, is already happening, and it’s far less expensive. But are the results trustworthy?
I am a machine learning researcher. I study large language models and their uses in medicine and science. These systems change constantly as companies update them. Different prompts, settings and model versions can produce very different answers to questions. That trait can make models difficult to use reliably in social science research, but it can help simulate replies of many humans, what researchers call “synthetic respondents.”
To create 10,000 answers from ChatGPT, for example, a pollster would prompt the model with some basic respondent demographics and context, such as “You are a young college-going urban voter with conservative political views. Respond to the following questions.” Researchers can change the demographic settings to elicit many different responses from ChatGPT for the same query.
