Stafford, Lucy, Preston, Catherine Elizabeth Jane orcid.org/0000-0001-7158-5382 and Pike, Alex orcid.org/0000-0003-1972-5530 (2024) Participant Use of Artificial Intelligence in Online Focus Groups:An Experiential Account. International Journal of Qualitative Methods. ISSN 1609-4069
Abstract
Large language models (LLMs), one application of artificial intelligence, experienced a surge in users between 2022–2023. During this time, we were conducting online focus groups in which participants insisted on responding using the chat box feature. Based on several chat box responses, we became concerned they were LLM generated. Out of the 42 participants who typed a chat box response during a focus group, we identify 9 as potentially providing LLM generated answers and present their responses with the highest similarity score to an LLM answer. Given the growth and improvement in LLMs, we believe that this issue is likely to increase in frequency. In response to this, in this article we reflect on (1) strategies to prevent participants from using LLMs, (2) indicators LLMs may be being used, (3) the fallibility of identifying LLM generated responses, (4) philosophical frameworks that may permit LLM responses to be incorporated into analyses, and (5) procedures researchers may follow to evaluate the influence of LLM responses on their results.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © The Author(s) 2024 |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Psychology (York) |
Depositing User: | Pure (York) |
Date Deposited: | 02 Nov 2024 01:27 |
Last Modified: | 31 Dec 2024 00:16 |
Published Version: | https://doi.org/10.1177/16094069241286417 |
Status: | Published |
Refereed: | Yes |
Identification Number: | 10.1177/16094069241286417 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:219171 |
Download
Description: Participant Use of Artificial Intelligence in Online Focus Groups: An Experiential Account
Licence: CC-BY 2.5