The Background:
Qualitative and quantitative research each have their place in product development. Early on, qualitative methods, like focus groups and one-on-one interviews, help brands explore consumer reactions to initial product concepts or prototypes. Follow-up quantitative research then provides the numbers needed to proceed confidently through the remaining stages of the process to the final launch decision. When done properly, this research produces results directly related to key performance indicators and enables progress in line with a company’s action standards.
Still, even during quantitative surveys, brands benefit from hearing the voice of the consumer. Collecting reactions in a consumer’s own language delivers insights beyond the numerical counts and averages of the responses to closed-ended questions. To get this blend of qualitative and quantitative information, surveys usually have several open-ended questions, such as “What did you like about this product?”. To take it a step further, they may select a small subgroup of participants (such as a focus group) for exit interviews after the quantitative phase of the research.
Breakthroughs in conversational AI unlock a third opportunity. P&K can now incorporate smart probing to open-ended survey questions, having AI guide a qualitative exploration after the quantitative phases. In this case study, P&K demonstrates the power and advantages of including conversational AI in quantitative studies.
The Challenge:
A confectionary company sought to benchmark its breath mint product to that of a leading competitor. The company asked P&K to design research that would provide insight into how to enhance both the product and its packaging.
The P&K Solution:
P&K Research saw an opportunity to leverage AI in this research. In standard self-administered surveys, open-ended questions by themselves offer limited learning. People in a food or beverage study asked “What did you like about this product?” might simply say “The taste.” However, an AI-driven chatbot can follow up with a probe, such as “What did you like about the taste?” Depending on the reply (“It was not too sweet”), the AI can dig deeper (“Why is the sweetness important to you?”) or ask if any other attributes were appealing. With AI, we can seamlessly weave open-ended probes throughout a quantitative survey, providing a more in-depth understanding of consumer opinions and feelings about the product experience.
For the breath mint study, P&K recruited 200 category users – split between client and competitive brand users – for a central location test. Respondents evaluated samples of each unbranded breath mint in sequential order, with timed intervals for in-mouth experience and a between-sample wait to minimize sensory carryover.
The self-administered survey asked about overall liking and product perceptions on key attributes. After trying both samples, users answered a preference question. The survey then used AI-driven prompts to explore the reasons behind their preference in a conversational manner. Each prompt builds on past answers, asking questions such as:
- Why did you prefer this product?
- Other than [insert prior response], what other reasons do you have for preferring it?
- What would you change to improve your preferred product?
- What else would you want changed?
If someone had no preference, prompts also examined that person’s reasoning.
In the final phase of the survey, respondents received packages of the client and competitor products. Participants rated each package on dimensions such as overall appeal and functionality. Likes and dislikes were probed in detail with conversational AI.
The Outcome:
Qualitative AI was able to uncover reasons behind consumers’ preferences and successfully replaced exit interviews with guided one-on-one conversations based on a series of prompts and probes. And while researchers traditionally conduct exit interviews with a subset of respondents, AI made it possible for us to collect qualitative data at scale – with the entire sample of participants!
The findings laid out a clear picture of the strengths and opportunities for product and package improvement. The voice of the consumer elicited by the conversational AI yielded nuanced insights that expanded on the quantitative ratings and guided the client on the steps for improving their product and package.