Leveraging AI for market research - Not out of a job...yet
Updated: Jul 9
In today's data-driven world, businesses and researchers rely on surveys to gather valuable insights from their target audience. But in case you haven't heard we are at the start of a revolution. Artificial Intelligence (AI) is dominating the news & socials as the next wave of technological advancement but it has come with some scary headlines: 'Artificial intelligence could lead to extinction, experts warn' in a BBC article. This isn't limited to humankind but also market research (okay may not be as equally important!). Government body UK Research and Innovation (UKRI) announced there has been a suite of AI investments bringing together academia and the business world as Rishi Sunak’s government is looking to position itself as an AI world leader.
Although there is investment, Ray Poynter, President Elect of ESOMAR says: "AI will take many people’s jobs. It will probably be five years before we really notice it, and in ten years, there will have been a major change." He notes that jobs will likely shift from agency side to client side as self-serve market research solutions become more user friendly and accessible. I even used AI to help me write this blog (you can probably see the 2 competing styles), although I have proof-read, fact-checked, amended and added to it's initial generation (I'm no expert in copywriting as you can see also)- it's important I have an input and a sense of 'Elliot' in content I am sharing.
Ray continues in his article: "Most non-researchers, such as brand managers or product designers, can only go so far with self-serve platforms. The key limitation is knowing what is possible and knowing how to turn the results into an engaging story. "Developments in AI suggest that within five years, AI will be able to have a natural language conversation with, for example, a brand manager, suggest and implement a research design, and then present the findings and recommendations. The thinking in this area is around ‘virtual agents’. I think we will see this develop over the next five years, but I suspect it will be closer to ten years before this reduces the number of consultants and presenters the insights sector will need." I better make the most of the next 5-10 years then!
I'm not quite panicking yet as market research still requires good input, project management and the ability to join up other data sources for the 'bigger picture' (although AI could well be doing that too). AI doesn't however, as far as I can tell, help with visual storytelling especially in conventional PowerPoint reporting (I'm still a fan) or dashboards. And AI could be moving faster than some organisations can cope with. I've explored some AI survey tools such as BlockSurvey and ones offered by Hotjar. And the existing templates & question types are in my view more than appropriate and overall output can be as good what you input in.
However questions still need to be 'tidied up' - when I tested the tool on BlockSurvey, the NPS question didn't have the right scale (only 5 points), the gender question didn't have a MRS recommended option 'Prefer to self-describe', and it made open-ended questions mandatory which could put off some respondents. I've also tried Chatsonic but there's a word limit there on the basic account and in my view doesn't lend itself to surveys other than perhaps an initial starting point. These are changes easily remedied on the platform but is a reminder at this point, they can't be solely relied upon.
Nevertheless we can all agree that crafting effective survey questions is a crucial aspect of ensuring accurate and actionable data. So let's explore how AI can assist in this area and unlock deeper insights for your research endeavours.
Stuck for ideas: AI is a great starting point. Even for novices, early prompting in Chat GPT or BlockSurvey can generate a good structure for a survey. It won't be perfect but it won't be bad either. And remember it is only as good as the prompts you put in.
Automated Analysis of Existing Questions: AI-powered tools can analyse vast amounts of existing survey questions to identify patterns, common themes, and best practices. By studying successful question structures, wording, and response options, AI algorithms can suggest improvements and provide insights into creating more effective questions.
Natural Language Processing (NLP): NLP techniques enable AI systems to understand and interpret human language. AI algorithms can analyse open-ended responses from previous surveys, identify key themes, and suggest potential question topics. This helps in ensuring the inclusion of relevant and comprehensive questions that capture the true sentiment and opinions of the respondents.
Survey Personalisation: AI algorithms can utilise demographic and behavioural data to personalise survey questions based on individual respondents. By tailoring questions to their specific interests, experiences, or preferences, AI-driven surveys can increase engagement and response rates, leading to more accurate and insightful data.
Adaptive Questioning: AI-powered survey platforms can dynamically adjust subsequent questions based on previous responses. By adapting the survey flow in real-time, AI ensures a more personalised and efficient respondent experience. This adaptability reduces respondent fatigue.
Sentiment Analysis: AI can go beyond simple response analysis by employing sentiment analysis techniques. By understanding the sentiment expressed in respondents' answers, AI algorithms can identify positive or negative trends, detect subtle nuances, and extract meaningful insights from unstructured data. Again this should be 'sense-checked' by individuals.
Conclusion: Incorporating AI into the survey question creation process offers significant advantages for researchers and businesses alike. From automated analysis of existing questions to personalised and adaptive surveys, AI-driven tools empower organisations to obtain deeper insights and make data-driven decisions. Leveraging AI in survey creation enhances accuracy, and can improve respondent engagement (it's very good at keeping surveys short!). As AI continues to evolve, its potential to revolutionise the field of survey research grows, enabling us to extract richer and more valuable information from the voices of respondents.
Remember, while AI can assist in survey question creation, human expertise remains invaluable (at the moment) in designing surveys that align with research goals and ensure the ethical treatment of data (adhering to GDPR, ICO & MRS Code of conduct) and respondents. So I am not out of the job just yet!