Correction, 2026-10-02: this page still presented a vendor result and some of our own judgments as facts. The Aaru figure (a median Spearman correlation of 0.90 with EY's wealth survey) is EY's own account of a project it ran with Aaru, not an independent result; it is now quoted and labelled that way, with the divergences EY's write-up also reports. The Verasight errors of 27 to 33 points are the four worst examples the report names, not typical results (its overall error was 14.5 percentage points), and the Paglieri paper says persona approaches "often" favor the most probable answer, not that most of them do. We had also written, without any measurement, that ranking hooks is where "the calibration holds up", that synthetic audiences "will not" flag a cultural misread, that they "systematically underweight the tails" and that a panel regressing to the mean "will systematically rate your safest variant highest"; these are now cautions tied to the research. The Park paper now carries its current title, the cost answer now uses the Kling tool's default (86 credits for 5 seconds with sound) and says Kling v3 needs a start image, and the page now says that 8frame does not run synthetic audiences.
Correction, 2026-10-01: the first version of this page gave the Stanford study's result as 85% normalized accuracy, 14 points above persona-only agents, which does not match the current version of the paper: it reports 83% for interview-based agents, 86% combined, against 74% for demographics-only agents. It said Aaru works for Fortune 500 clients (EY's page does not say so), summarised a review of 63 papers as finding "poor ecological validity throughout" (it found that validity varies considerably), stretched the Paglieri paper beyond its abstract, and gave an "80 to 95 percent" accuracy range across platforms that we cannot source. It also missed Simile's later Series B. All corrected below, with sources.
The pitch for synthetic audiences is seductive: before you spend a dollar on media, run your creative past a panel of AI personas calibrated to your target segment and get back predicted reactions in minutes. When AI generation makes 50 variants of one brief cheap to produce, the obvious next question is which of the 50 to actually launch, and a synthetic pretest promises to answer it without burning test budget. The funding is real, and some published results against real survey data look strong, though the commercial figure cited below comes from a vendor's client, not from an independent test. It is also easy to trust too much. This piece is about where synthetic pretesting can earn a place in a creative workflow and where leaning on it can quietly cost you.
What a synthetic audience actually is
A synthetic audience is a panel of AI-generated personas, built to simulate the responses of a real target segment, that you can query the way you'd query a survey panel or a focus group, except there are no humans in it. You describe your target, the system generates or retrieves personas matching that profile, and you show them creative and collect predicted reactions, preferences, and rankings. The panel predicts responses rather than measuring them.
Investors have backed the idea. Simile, whose founding team "introduced the original concepts of generative agents" according to its lead investor, Index Ventures, announced $100 million in funding led by Index Ventures in February 2026, with Bain Capital Ventures, Andrej Karpathy and Fei-Fei Li among the backers. On 2026-07-30 it announced a Series B of more than $200 million at a $2 billion post-money valuation, co-led by Greenoaks and Index Ventures.
For Aaru, another AI simulation company, the figure on record is a client's claim, not an independent result. In a write-up published on 2025-10-07, EY, which ran the test with Aaru, says: "Using Aaru's proprietary multi-agent infrastructure, EY recreated its Global Wealth Management Survey, which normally spans 3,600 affluent investors across 30+ markets and takes six months to complete. The simulation was completed in one day and evaluated across 53 single-select questions, achieving a median Spearman correlation of 0.90." The same page gives the average difference per question as 7.1 percentage points and also reports "notable divergences", including "a -37.82% correlation" on inheritance planning. Read it as EY's account of its own project with the vendor.
The academic work behind data-grounded agents is a study by Joon Sung Park and colleagues, who built agents of 1,052 Americans from two-hour interviews, from surveys, or from both. In the current version of the paper, on held-out General Social Survey items, the agents reached 83% (interviews only), 82% (surveys only) and 86% (both) of participants' own two-week test-retest consistency, against 74% for agents given demographics only.
Those numbers sound strong. The trap is assuming they carry over to your specific creative decision: they were measured on survey questions, not on reactions to a new ad.
What it can predict, and what it can't
The published evidence is strongest on survey-style questions whose answers track who the respondent is. Verasight's January 2026 report puts it plainly: questions "heavily discussed in public discourse and well-represented in LLM training data, particularly political attitudes, are easier to impute than questions about personal behaviors and preferences that lack strong demographic correlates", because "an LLM has few demographic proxies for what type of consumer prefers a pumpkin spice latte to a regular cup of coffee." Reactions to a new ad sit closer to the pumpkin-spice end. So if you use a synthetic panel, ask it relative questions about a batch (which of these five hooks, which framing reads as relevant, how a concept ranks against the alternatives) and treat the answer as a way to decide what gets a real test, not as a forecast of how any single ad will perform.
The same report shows where it breaks. On multi-response questions, 31% of the options were never selected by the LLM at all, and the authors "do not recommend using LLMs to generate data for multi-response questions". The worst-performing questions were about personal experiences and behavioral preferences that "lack strong demographic predictors", with mean absolute errors of 27% to 33% on the four examples the report names; across the response options of its single-answer questions, the error averaged 14.5 percentage points, and the report found "a systematic tendency to regress predictions toward the mean". Paglieri and colleagues (2026) note that most persona-generation approaches need detailed data about the target population and often prioritize "density matching (replicating what is most probable) rather than support coverage (spanning what is possible), leaving long-tail behaviors underexplored." And a review of 63 peer-reviewed studies published between 2023 and 2025 in leading NLP and AI venues, by Batzner and colleagues, found that the representativeness and ecological validity of synthetic personas "vary considerably between studies", that the task and the population of interest are often underspecified, and that only 35% of the studies discussed how representative their personas were.
Translated into creative decisions: a panel that pulls answers toward the most probable one can help you sort a batch, but don't expect it to surprise you. It can underweight unusual reactions, and an unusual idea may be exactly what you are testing. Don't count on it to flag an offense or a cultural misread that a real person would catch.
The pretest-then-spend workflow
Used as a filter rather than an oracle, synthetic pretesting slots into an AI creative pipeline.
Generate wide. Start from a batch, one brief into 40 to 50 variants across hooks, visuals, and framings, using the AI ad variant testing workflow. Volume is the input: the pretest needs something to filter.
Pretest to rank, not to decide. Run the batch past your synthetic audience to rank and cluster. The goal is to cut the obvious losers and surface 10 to 15 candidates for a real test, not to crown a single winner.
Spend on the survivors, in-market. Take the shortlist to a real, small-budget in-market test on Meta or TikTok. This is the step nobody gets to skip. Pair every synthetic prediction with a real in-market check: the synthetic layer narrows the field cheaply, the live layer supplies ground truth.
Feed results back. Whatever wins in-market becomes calibration signal. Over time you learn how well your synthetic panel's rankings predicted your live outcomes, which tells you how much to trust it next time. That feedback loop is how you find out whether the panel is a filter or a crutch for your category.
The workflow's whole value is sequencing: synthetic to filter cheaply, live to confirm truthfully. Invert that order, or drop the live step, and you've traded a validated decision for a confident guess.
The skepticism section
A few things to hold onto so the tool doesn't drift from filter to authority.
Calibration is not truth. EY's reported 0.90 median correlation with an existing survey, or the 83% to 86% of test-retest consistency in the Park paper, describes how well models tracked answers to survey questions. It says nothing direct about how your specific new creative will do, especially if that creative is doing something unusual.
The averaging bias can cut against what you want from creative. Verasight found a "systematic tendency to regress predictions toward the mean" on survey answers; if a panel does the same with ads, it will lean toward the safest variant. Before you cut a variant only because the panel ranked it low, check whether it ranked low because it is unusual, and consider keeping one or two outliers in the live test.
Ask what a figure measures. Vendors and their clients report their own accuracy figures, and "a 0.90 correlation" can mean very different things depending on the benchmark: EY's figure is a rank correlation (Spearman) across 53 survey questions, not a hit rate on ad decisions. Ask what a number was measured against before you weight it in a decision.
None of this makes synthetic audiences a bad tool. It makes them a filter with blind spots, which can still be useful when you're choosing which 12 of 50 variants deserve real spend.
What this means for brand teams
The honest framing is that synthetic pretesting compresses the front of the funnel, not the whole thing. It can make the "which of these should we even test" question cheaper and faster, which matters when AI generation makes 50 variants easy to produce and impossible to test all at once.
What it does not do is replace in-market testing, and any pitch that says otherwise is selling past the research. Used as a triage layer, it fits: generate wide, pretest to shortlist, spend to confirm, calibrate over time. Used as a decision, it lets a confident-sounding prediction replace the part where reality gets a vote.
FAQ
How accurate are synthetic audiences?
It depends on the question, and the commercial figure here is a client's claim about a vendor. EY says its replication of a wealth survey with Aaru reached a median Spearman correlation of 0.90 (EY, 2025-10-07). In the Park paper, data-grounded agents of 1,052 Americans reached 83% to 86% of participants' own test-retest consistency on General Social Survey items. Verasight found much larger errors on multi-response questions and on personal behaviors with weak demographic predictors. Ad creative reactions are closer to the second kind.
Can a synthetic audience replace an A/B test?
No. Use it to rank and shortlist variants cheaply, then spend a small in-market budget on the shortlist. Feed the live results back to learn how well the synthetic rankings predict your outcomes.
Does 8frame run synthetic audiences?
No. 8frame generates the image and video variants; the synthetic panel is a separate tool.
What does generating 50 variants to pretest cost?
On 8frame, 50 Nano Banana 2 stills at 1K are 550 credits (11 each). Video costs more: 50 five-second Kling v3 Standard clips are 4,300 credits with sound (86 each, the Kling tool's default) or 2,850 silent (57 each), and Kling v3 needs a start image for every clip, such as one of those stills. A credit is about $0.01 at pack rate.
Sources
- Simile $100M funding and investors: Simile, "The Simulation Company" (published 2026-02-12) and Index Ventures, "Life, the Universe, and Simile: Leading Simile's Series A", February 12, 2026 ("the founding team introduced the original concepts of generative agents"), checked 2026-10-02
- Simile Series B: Simile, "Announcing Simile's Series B" (published 2026-07-30; "over $200 million at a $2 billion post-money valuation, co-led by Greenoaks and Index Ventures"), checked 2026-10-02
- Aaru and the EY survey replication, EY's own account of a project it ran with Aaru (median Spearman correlation of 0.90 across 53 single-select questions, 3,600 investors, 7.1 percentage points average difference, "a -37.82% correlation" on inheritance planning): EY, published 2025-10-07, checked 2026-10-02
- Park et al., "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals" (first posted in November 2024 as "Generative Agent Simulations of 1,000 People"; current version revised 2026-06-28; 83%, 82% and 86% for interview-only, survey-only and combined agents vs 74% for demographics-only): arXiv 2411.10109, checked 2026-10-02
- Verasight, "Can Large Language Models Replicate Survey Data Across Topics?" (G. Elliott Morris and Benjamin Leff, 2026-01-13; 31% of multi-response options never selected; worst examples named at 33%, 30%, 29% and 27% mean absolute error; overall 14.5 percentage points; the pumpkin spice latte example): verasight.io, checked 2026-10-02
- Paglieri et al., "Persona Generators: Generating Diverse Synthetic Personas for Arbitrary Contexts" (submitted 2026-02-03): arXiv 2602.03545, checked 2026-10-02
- Batzner et al., "Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency" (63 peer-reviewed studies, 2023 to 2025; only 35% discussed representativeness): arXiv 2512.00461, checked 2026-10-02
- 8frame Nano Banana 2 and Kling v3 Standard prices, the Kling tool's default, Kling v3's start-image requirement and the absence of any synthetic-audience tool: the 8frame app, checked 2026-10-02
Synthetic pretesting needs a wide batch to filter. Generate the variants on 8frame, then shortlist and spend. Start with the AI ad variant testing workflow, and see what is a synthetic audience for the fundamentals.
Try it without signing up: the 8frame canvas opens on a real board. The canvas is free and unlimited; generation is paid from $19/month.