Can AI Replace Human Respondents In Qualitative Research?
Like most industries these days, market research is no stranger to AI with its broad applications including the employment of synthetic respondents, which are individual profiles constructed by Large Language Models (LLMs) from real or simulated data. They offer fast, cheap, and scalable synthetic data that closely mimics how human participants would respond, a boon for quantitative research. But can synthetic respondents be just as effective in qualitative research? Can AI-powered profiles fully take over the role of human respondents in market research?
Image: Diana
Synthetic Respondents and Qualitative Research
L&E Research recently hosted a webinar sharing their findings and observations testing synthetic respondents across a variety of qualitative research tasks. They shared that AI characteristically produces quick, structured, and consistent surface-level insights. It does well with detecting macro trends in usage or preferences, concept screening if you need to compare multiple ideas at scale, and spot issues with survey testing. It is also capable of gap-filling or simulating missing segments from known data, as well as bulk analysis for summarizing large open-ends quickly.
The key takeaway L&E found is that AI can describe what people do, but it falls short of telling why people do it. AI fundamentally excels in following patterns, but it would struggle with finding out the emotional driver, the motivation behind certain responses. AI can match logic but it won’t be able to fill in tone, nuance nor context like human insight and experience can.
Most AI models are also built on public data and may not have access to knowing how real people would respond to certain questions. When the engineers tried to influence AI agents in the direction of how real participants would respond, it rejected this notion and firmly stood by the perspective formed from the vastness of public data.
Additionally, AI can be absolutely and confidently wrong. Synthetic data can look convincingly human but since AI relies on patterns instead of experience, the air of confidence it puts up doesn’t guarantee accuracy.
Of course, the hosts added a disclaimer that this is where synthetic respondents are at right now, as no one could tell how things could possibly be so much different in the years to come. But the continued utilization of AI in market research- or any other industry, for that matter- is inevitable thanks to the operational and executionary efficiency it grants, and that is enough reason to continue studying and developing synthetic respondents.
Image: Ron Lach
Why The Human Factor Matters
In market research, emotions matter and context counts. AI can prove to be a powerful partner but it is no replacement for lived insight or validation. Human researchers are simply going to remain essential.
AI’s inherent structure and consistency is representative of its pursuit of perfection; however, humans aren’t perfect, or simple. Humans are emotional and oftentimes, irrational. AI participants would respond based on their perfect approximation of how a human being would, but the synthetic logic behind that would be narrower and more consistent, as it discounts the fact that humans are imperfect.
Humans also bring incredible complexity and a broader range of perception to the table. We can contradict ourselves, and this would be natural. One human participant’s perception and experiences could inform the difference in how they respond from the next, while synthetic data would be uniformly shaped by congruence and invariability, no matter how much effort or work is put into making AI come close to mimicking humanlike responses.
The complexity, variability, and randomness of human nature is desirable in qualitative research. The engineers recognized this and cautioned about overly guiding or influencing randomness in AI that it “will hard-code your picture of randomness to the point where it is no longer random.”
AI can quickly give you bulk analysis but you might not want to rush in bringing it to your stakeholders, as they would question and challenge the quality and reliability of synthetic data. Human insight continues to be vital and irreplaceable when it comes to trust, nuance, and real-world complexity in market research.
Image: Kathrine Birch
The Hybrid Approach
At the end of it all, the hosts made a point that the webinar wasn’t meant to scare people away from synthetic data but rather bring a valid conversation on when it makes sense to take advantage or steer clear of AI-generated personas. In fact, they recommended utilizing a hybrid approach of employing virtual respondents and recruiting human participants, striking a delicate balance between synthesis and empathy.
Synthetic data would be great during the early exploratory stages of market research when you want to get an initial pulse check, something quick and good enough before getting people involved. But once you’re at the point when you need to uncover the emotional driver behind responses and decisions, understand or predict behaviors, or even gain a bit more confidence and trust in your findings, that’s when you bring in your human respondents.
This all aligns not only with a recent growing trend of companies coming around from the AI hype of the last few years but also with our stance on the appropriate use of AI, where we advocate for the responsible and ethical use of artificial intelligence. Instead of handing AI complete reins over all aspects of a business- or in this case, all stages of research work- we at Nimbus Online encourage the thoughtful and practical application of artificial intelligence in combination with or enhanced by human experience, values and discretion.
To find out how our brand of inspired and enlightened human thinking can help you with your market research needs, please contact us here.
Additional Reading:
Can Synthetic Respondents Take Over Surveys?