Large consumer-facing companies are increasingly using generative AI to search, summarise and analyse customer and market research, but new tools are not removing long-standing organisational barriers to using insights effectively, according to researchers and industry leaders.
Companies including Procter & Gamble, PepsiCo and Novartis have adopted systems that combine internal customer data with the language and reasoning capabilities of large language models. These approaches often use retrieval-augmented generation, or RAG, which allows an AI system to draw on a company’s own documents and data rather than relying only on the material on which the model was originally trained.
The systems are being used to help employees find answers in large stores of market research, sales feedback, social media comments, customer service records, purchase patterns, interview transcripts, focus group results and other material. In large organisations, staff may know that relevant insight exists but not where it was produced, who owns it or how it was stored.
Thomas H. Davenport, of Babson College and the MIT Initiative on the Digital Economy, and Viktor Dörfler, professor of AI strategy at the University of Strathclyde Business School, examined how customer and market insight leaders are using the technology. Their work drew on discussions with specialists and leaders at eight consumer-oriented companies, several generative AI vendors and a market research agency using AI-based qualitative research tools.
Their findings suggest that generative AI can make customer knowledge easier to retrieve and interpret, but that companies risk repeating mistakes made during earlier waves of knowledge management if they focus only on storage and access. Earlier tools such as Lotus Notes and Microsoft SharePoint widened access to information, but many organisations still struggled with poor classification, siloed teams, duplicated work and weak collaboration with external agencies.
The same issues remain relevant with generative AI. Companies need to manage the full flow of knowledge, from how insights are created and analysed to how they are stored, governed, tagged and reused. A customer insights leader cited in the research said: “AI is only as useful as the data it learns from.”
Some companies are building their own systems, while others are using external software vendors or combining the two approaches. P&G uses vendor-supplied software for knowledge storage and access, but has developed its own system for AI-based analysis and categorisation. Kirti Singh, P&G’s chief analytics, insights and media officer, said this helped the company obtain “sharp, pointed answers from GenAI” rather than only links to documents.
Vendors in the market vary in focus. Some concentrate on storing and retrieving insight documents, with features such as automated curation, tagging, synthesis and prompt-based answers. Others specialise in analysing qualitative data from interviews, focus groups and other unstructured material, while some focus on rapid testing of consumer responses to advertising. The researchers said these categories are beginning to overlap as vendors move towards broader platforms for managing the customer insight process.
Novartis is cited as an example of a company that has used generative AI to improve insight storage and retrieval. Working with an external vendor, it developed a customer and market insights system called Sherlock for its consumer business. Users can ask questions and receive answers linked to specific lines of text or time stamps in video. The system also includes expert-curated microsites, called Knowledge Zones, for particular topics such as packaging.
Novartis requires users adding content to follow governance rules covering document formats and quality. Research vendors can upload project deliverables directly into the system. According to the research, Sherlock helped the company avoid duplicated spending on insight services and find relevant material more quickly. Novartis saved more than $29 million in primary market research costs in one year.
Generative AI is also changing qualitative research, an area that has traditionally relied heavily on manual coding, spreadsheets and researcher interpretation. Tracy Tuten, who leads qualitative research at Illuminas, now part of Radius Insights, uses AI software to analyse interviews and focus group material through natural language prompts. The system can transcribe audio and video, summarise findings, identify themes and compare responses across audience segments.
Tuten said a large qualitative project, such as a global study with more than 30 interviews, might previously have taken six weeks to analyse but can now be synthesised in a day. The researchers said specialised tools appear better suited to this work than generic AI chatbots, although they stressed that AI augments researchers rather than replacing them.
PepsiCo has also made extensive use of software for customer and market knowledge, including structured and unstructured data. Its Ask Ada marketing research platform was described by Stephan Gans, senior vice president and chief consumer insights and analytics officer, as part of a wider transformation of the company’s customer insights function. Gans said the platform had reduced PepsiCo’s dependence on external agencies and consultants.
However, he also said strategic marketing judgement remains a human responsibility. “Raising the bar on marketing and innovation effectiveness to fuel commercial excellence will become increasingly automated,” Gans said. “Leading the understanding of consumer demand is much more strategic and still requires humans.”
The research identified several obstacles that can limit the impact of AI systems. One global consumer goods company had acquired an AI-based customer knowledge tool but struggled to improve global access because its country units used different names and classifications for brands, categories and distribution approaches. The head of knowledge management for insights at the company said it had “pockets of knowledge” that were “very incoherent”.
PepsiCo took a different approach after Gans became chief consumer insights and analytics officer in 2017. With support from the chief marketing officer, he created a Global Insights Council of regional and central insights leaders to encourage a more unified approach to research and knowledge sharing. Customer insights are now more closely integrated into the company’s innovation work.
The findings also underline the importance of demand for insight inside a company. At P&G, Singh linked the use of AI to the company’s long-running emphasis on consumer understanding. “At the heart of everything we do is the consumer,” she said, adding that P&G combines experimental science, behavioural science, data science and technology platform knowledge to understand customers.
The researchers said companies also need clear rules for working with external agencies. If agencies retain control over research outputs and learning, internal teams may become dependent on outside support. Gans argued that client companies should own research results and insights produced on their behalf, and that employees should apply lessons from research in future campaigns.
Despite rapid advances in software, the researchers concluded that generative AI cannot by itself solve problems created by inconsistent data, fragmented governance or a lack of interest in customer insight. Companies still need people to standardise information, curate material, ask useful questions and act on the results.
As Singh put it, P&G is using AI to build on existing strengths rather than replace established methods. She said the company had brought together its tradition of understanding customers with newer AI tools, “not replacing, for example, customer home visits, but augmenting them with AI”.
