Synthetic users: the complete guide

A synthetic user is an AI-generated profile that stands in for a real research participant. You can ask it questions, run a concept past it, or use it to pressure-test a document before taking that work to real people.

The most important distinction is whether the synthetic user is grounded or ungrounded. A grounded synthetic user is built from research you already own, with claims tied back to real interviews, surveys, support conversations, or other customer evidence. An ungrounded one starts with a prompt and fills in the gaps from the model's training data.

The question everyone asks is how accurate synthetic users are. The problem is that there is no stable human baseline to measure them against. People do not give perfectly consistent answers themselves, so a single accuracy percentage can sound much more definitive than the evidence really is. In a Stanford-led study of 1,052 people, participants reproduced their own prior answers about 81% of the time, while synthetic replicas matched the original answers about 68% of the time. The often-quoted 85% figure comes from normalizing that 68% against the 81% human self-consistency rate.

The better question is what synthetic users are reliably useful for. The strongest use cases are early and low-stakes: rehearsing a discussion guide, pressure-testing a document, surfacing gaps in existing research, and keeping known customer perspectives present in a conversation. Ground them in your own evidence, require a citation for every meaningful claim, and do not let the synthetic user become the decision-maker.

A synthetic user is a profile generated by a large language model that responds as if it were a real user.

What makes one useful is not how convincingly it sounds human. It is where the profile comes from.

A grounded synthetic user is built from evidence you already have, such as interview transcripts, survey responses, support tickets, sales calls, and product research. Its attributes and responses can be traced back to that source material, and where the evidence runs out, the profile should be able to say so.

An ungrounded synthetic user starts with a description such as: act as a 34-year-old product manager at a mid-market fintech company.

A general-purpose model can generate a plausible response from there. But that answer reflects patterns in the model's training data, not evidence from your own customers.

Both get called synthetic users. They are not the same thing.

The practical difference shows up quickly in language. A general-purpose model tends to produce polished, generic answers. A synthetic user grounded in your own research repository can reflect the language your customers actually use, including the different words roles or segments use to describe the same problem.

Several terms get used interchangeably in this space, which makes conversations about synthetic research harder than they need to be.

Accuracy sounds like the obvious metric. The problem is that you need a stable ground truth to measure it against, and human responses are not perfectly stable.

Great Question originally designed a head-to-head experiment to compare synthetic and real participants. The plan was straightforward: give a synthetic panel and a matched group of real senior user experience researchers the same concepts, documents, and questions, then compare the themes each group produced.

The problem appeared as soon as the panels disagreed.

If the synthetic users raised an issue the humans did not, was that a useful pattern the humans happened to miss, or was it fabrication? If the real participants raised something the synthetic panel missed, was that a failure of the model or simply an unusual experience from one participant?

The comparison could show differences, but not which side represented the truth.

Sentiment and idea overlap do not completely solve the problem either. You can create scoring rules to measure how similar two sets of answers are, but those rules introduce judgment about what counts as the same idea.

One of the most frequently cited pieces of evidence for synthetic users comes from research involving 1,052 people and their AI-generated replicas. The headline number is often presented as 85% accuracy.

The underlying numbers tell a more useful story:

There is no single synthetic-user workflow. At least four distinct approaches are useful, and each solves a different problem.

Start with what you are trying to represent.

Synthetic users work much better when the source material lives in a structured UX research repository rather than a folder of transcripts. The reason is technical.

Long transcripts contain a lot of information, and large language models do not always use every part of a long input equally well. Research on long-context models has shown that information buried in the middle of large inputs can be harder for models to retrieve effectively, a problem often described as lost in the middle.

Simply pasting dozens of transcripts into a prompt does not guarantee the right evidence will shape the answer.

A stronger system uses retrieval-augmented generation. Instead of asking the model to reason over everything at once, the system retrieves the pieces of evidence most relevant to the question and gives the model a smaller, more focused body of material to work with.

The infrastructure underneath matters.

Great Question tested reproducibility by building profiles from the same small survey five separate times using the same underlying data and prompts.

Some things stayed remarkably stable. The strongest segment appeared across all five runs. Distinctive verbatim phrases were repeatedly attributed to the correct respondents. Participants who were clearly outside the target segment were consistently excluded.

Other things moved. The number of profiles generated varied. Confidence ratings on weaker clusters changed between runs. The thinnest segment fractured differently depending on how aggressively each run grouped people together.

That pattern is more useful than a blanket statement that synthetic users are either reproducible or not.

You do not need a formal validation study every time you ask a synthetic user a question. A few simple checks go a long way.

Grounding synthetic users in real customer language makes them much more convincing. That creates another problem: convincing is not the same as true.

One company experimenting with synthetic personas found that early versions built from persona summaries sounded generic. Adding real interview language made different roles sound noticeably more distinct. That was an improvement, but it also made the personas easier to over-trust.

Real people contradict themselves. They change their minds. They misunderstand questions. They have bad days. They care intensely about one issue and barely notice another. Synthetic users tend to smooth away some of that messiness.

The more human a synthetic user sounds, the easier it becomes to forget that you are interacting with a model of the research rather than another participant.

That is why trust but verify is not enough on its own. You need structural guardrails such as citations, evidence counts, confidence labels, and clear boundaries around where synthetic users are allowed to influence decisions.

A useful governance model is to divide use cases by the cost of being wrong, not simply by research method.

If you keep one operational rule from this guide, make it this: no citation, no claim.

Citations are what separate a grounded synthetic user from a plausible character. Every meaningful assertion should connect back to the interview, survey response, support conversation, or other customer evidence behind it. Anything without support should be labeled as low confidence or unknown.

This matters because the most dangerous failure mode is not always an obvious hallucination. It is confident thinness: a polished answer built from weak evidence that sounds exactly like one built from strong evidence.

A good AI analysis and synthesis workflow keeps the claim and source connected so a researcher can inspect the evidence rather than trusting the summary.

There is a larger implication behind all of this.

Traditional research often works one question at a time. A study is designed around a decision, participants are recruited, findings are synthesized, and the team moves on.

Synthetic users make the accumulated evidence itself more reusable. That shifts some of the researcher's value toward building and maintaining the evidence base that future questions will draw from.

The repository stops being administrative infrastructure and becomes part of the research product. That means maintaining clean metadata, preserving source material, curating validated insights, filling meaningful gaps in customer evidence, setting governance around AI use, and making uncertainty visible.

Great Question's AI features, MCP integration, and research repository are built around that broader idea: research as a reusable evidence system rather than a stack of individual reports. The AI-native research guide covers the wider shift.

It also leaves an important question open. What does the research role become when more people interact with customer evidence through AI instead of opening a research report themselves?

Synthetic users are most useful when you treat them as a rehearsal and synthesis tool, not a replacement participant.

They cannot be reduced to one clean universal accuracy score because the human baseline itself moves. What you can evaluate is whether the profile is grounded, whether its claims are traceable, whether its strongest patterns reproduce across runs, and whether you're using it in a situation where being wrong is cheap enough.

Use synthetic users to sharpen a discussion guide before a real interview. Use them to pressure-test a document. Use them to keep existing customer evidence present while your team debates an idea.

Do not use them to avoid talking to people. The better synthetic users get at sounding human, the more important that distinction becomes.