We are thrilled to share that FlexMR and the UK Health and Security Agency (UKHSA) have collaborated on a professional study that has been published in the International Journal of Market Research (IJMR). Christopher Martin and Edward Hill, Chief Marketing Officer and Insight Manager from FlexMR, and UKHSA’s Lee Chan, Virginia Bertelli, Catherine White, and Lewis Wooding are the authors responsible for conducting an experiment to test whether cognitive pre-tests with artificial intelligence interview tools could carry out effective and sensitive pre-testing at speed.
Designing and operationalizing valid surveys during public health emergencies and similarly fast changing situations can be a challenge. In this research note, FlexMR and UKHSA present an exploration of whether an approach to cognitive pre-test interviewing through an online survey platform augmented by the use of AI prompts could help achieve robust pre-testing at pace, in a sensitive and effective manner.
In this study, Chan et all. integrated a Large Language Model (LLM) tool into an online survey system on FlexMR’s InsightHub platform and configured it to generate qualitative prompts to better understand how a sample of forty-six respondents understood and approached answering a series of questions about mpox.
Overall, the results of this study indicate using an LLM to conduct pre-test cognitive interviewing did not fully meet the need to conduct these activities to the standard needed by those working in quickly evolving situations such as health security incidents. However, this investigation of an untrained LLM with limited capability should be understood as an early step towards discovering the potential to cognitive pre-test survey questions using LLMs.
Read the full study in the International Journal of Market Research by Sage Journals to find out more about the study and the results in full.
Founded in 1958, the aim of the International Journal of Market Research (IJMR) is to publish original research addressing key challenges in market research and insight. The IJMR is first and foremost an applied journal that seeks to encourage a dialogue between practitioners, academics, and those in the business or public sectors who rely on research and insight to make effective decisions, deliver impact and drive change. IJMR publishes theoretical and empirical papers from both established and emerging scholars, and from a wide range of methodological backgrounds.
UK Health Security Agency (UKHSA) prevents, prepares for and responds to infectious diseases, and environmental hazards, to keep all our communities safe, save lives and protect livelihoods. We provide scientific and operational leadership, working with local, national and international partners to protect the public’s health and build the nation’s health security capability. UKHSA is a research active organisation. Research, and the translation of its findings into innovations both inform and support public health policy, practice and services.