I agree that AI programs like ChatGPT can come in handy for qualitative data analysis, particularly with pre-processing, categorization, sentiment analysis, topic modeling, summarization, and keyword extraction. These tools could help uncover insights, identify themes, and render context from unstructured text sources. That said, human expertise is crucial to shun biases and ensure accurate data interpretation. Here is an article that might be of interest.

Hassani, H., & Silva, E. S. (2023). The role of ChatGPT in data science: How AI-assisted conversational interfaces are revolutionizing the field. Big Data and Cognitive Computing, 7(2), Article 2. https://doi.org/10.3390/bdcc7020062

Best,

Mohialdeen Alotumi First, thank you for the reference to the Hassani & Silva article -- this is such a rapidly moving and interdisciplinary filed that it is difficult to keep track of all the literature.

Beyond that, I would point to the disruptive nature of ChatGPT. You refer to the role of AI in "pre-processing, categorization, sentiment analysis, topic modeling, summarization, and keyword extraction," but my experience is that these processes all fall short of what ChatGPT can offer. Thus, the history of programs for the analysis of qualitative data began with "mark and retrieve" versions of manual coding, followed by a short period of experimentation with sentiment analysis, keyword extraction, etc. More recently both ATLAS.ti and MAXQDA have released beta versions of their programs that incorporate ChatGPT (although the two implementations are very different).

So, I foresee major changes in the standard approach to qualitative data analysis, based on ChatGPT and the related programs that are sure to follow.

I do not know. my main concern with using ChatGPT and other tools(I use it by the way) is the long-term impact on our capability of thinking and processing information.

I think the immediate benefit of ChatGPT is to reduce all the effort involved in manual coding to locate the core content in the data. Beyond that, critical thinking will always be necessary to interpret the meaning of that basic content.

very true that is a very good point. and adding to that I think now the research topic would be broader and more complex. and more comprehensive than before. since now one researcher could handle a big chunk of information. thank you for sharing

David L Morgan, I agree with you to the end. In my Turkish conceptual work, I gave examples from ChatGPT in the sections related to qualitative data analysis, hypothesis formulation and testing, survey, design and analysis of experiments, computational social sciences, and I mentioned that people who will conduct research in social sciences will make great contributions to their research. I guess nothing will ever be the same.

Article Sosyal bilimler araştırmaları için ChatGPT potansiyelinin aç...

I have concerns about all this regarding the "trasnparency" of the criteria used by ChatGPT to make its analysis. I have made a couple of informal experiments myself, for example, asking ChatGPT if it can detect any kind of sentiments in some sample texts. The AI everytime provides an answer. However, I have not been able to get the AI to explicitly state its criteria. Maybe I did not formulate the right questions, or maybe not. I think there is a risk in using the AI assessments as some kind of black box because, even if we use our "critical thinking" (whatever it is), we will be using it over results that we are not sure how were obtained. I mean, there is no possibility (again, maybe I did not devote enough time to it) to argue or discuss with ChatGPT because everytime you object to one of its answers, he just apologizes (somewhat sonsensically) and reformulates or revises its answer or it just states that he is unable to give further details because it is "just" an AI.

Antonio M. Oller-Marcén This a general limitation with programs like ChatGPT. The answers they supply are based on their training with very large data sets, so the basis for those answers does not take the form of any specifiable algorithm or other explicit criteria. There are several other more traditional programs that perform sentiment analysis, such as NVivo, but I have no idea how much access this give you to their decision making criteria.

I personally do not think that this "black box" aspect of ChatGPT is a problem, due to the power of the program. You do need to be familiar with your data in order to evaluate the response you get from the program, but I personally have found these responses to be strikingly on target, regardless of the underlying process that produced them.

From a tactical point of view, I often reformat my queries in more than one way to compare the responses. The goal is to find out as much you can about your data, rather than to create one "perfect" query.

AS far as I am concerned, if I can produce essentially the same results by querying my data for 2 hours with ChatGPT versus spending 20 hours coding it by hand, the choice is obvious.

David L Morgan, but ir oder to know that you "can produce essentially the same results by querying my data for 2 hours with ChatGPT versus spending 20 hours coding it by hand" you must have both results to compare. It will not be tipically the case unless you are revisiting data from previous studies. Furthermore, how do you know that ChatGPT has not used your studies during its training and that this is the reason why the results are coherent?

In my current work, I have indeed been "revisiting data from previous studies," and that is the source of my positive reactions to ChatGPT. At this point, I believe that we are still at the point of doing the kind of methodological work that will justify the more widespread use of ChatGPT in qualitative analysis.

At present, the dominant approach to analyzing complete qualitative data sets is through coding, so I anticipate that the early methodological work with ChatGPT will compare and contrast its results with coding. But if you are doing sentiment analysis, you may not be examining "complete" data sets, which would open up a different set of opportunities and challenges.

As for ChatGPT having possibly used my studies, in this case, I worked with unpublished data. Even so, ChatGPT works via the synthesis of huge amounts of data to build its algorithms, instead of "remembering" each item that was part of its training.

Interesting

With huge interest and a rapid increase in using and implementing different kinds of artificial intelligence programs around the world, ChatGPT is considered one of the most important programs and needs high focus and intense study. It would be appreciated if you could please let me know about the details of your plan and the next step to proceed with this project.

I am interested and looking forward to a short online meeting.

I doubt very much how data protection and anonymity can be assured when passing one's interviews through ChatGPT. Any idea on this?

Lorena Molnar ChatGPT does not automatically use material that you analyze using the program. Instead, the program's training is periodically updated through "scraping" vast amounts of data from the internet as a whole.

In particular, I used ChatGPT which was trained on material that was available prior to Sept. 2021.

Lorena Molnar , in order to work with confidential or "sensitive" data, you need a "private", isolated copy of ChatGPT. Institutions like "national level health organizations" can do this. It is probably a matter of time before companies like Google and Amazon will offer "private" versions of ChatGPT-like AI systems to customers.

so i use ChatGPT to analyse my data, should I mention this reference:Article The Role of ChatGPT in Data Science: How AI-Assisted Convers...

Ahmad azmi Esmaeil Whether or not you cite an article depends on how relevant it is to your own research. If you are doing something similar, then you should indeed cite it.

thank you David L Morgan . i really appreciate your help and your answer.

Ahmad azmi Esmaeil : Imho more important to mention that you used ChatGPT to process data. You should only mention the article, if you are using results from that article, (such as specific methods) in your own research.

ChatGPT consistently provides biased answers (due to bias in training), and it often provides generalizations, which do not hold. Remember that ChatGPT is a computer program. It has no UNDERSTANDING of anything whatsoever, it can only retrieve and generalize data it was thought previously.

ChatGPT can not "create" anything, it can only copy and interpolate. Finally, as a computer program, it is incapable of critical thinking, because of the before mentioned limitations.

Because ChatGPT is incapable of (critical) thinking, it may only be used to "analyze" and "summarize" things, but it can not draw any conclusions (correctly).

I don't know the nature of your research, but I would be extremely careful to use a deceptive tool as ChatGPT in research.

When we start to see dialogues between people like:

A: What makes you think your research results are correct?

B: Because ChatGPT said so....

.....THAT will be the end of science as we know it...... Imho. even this (sad) moment is inevitable.

thank you for your note Erik Plesner . personally, the way I used ChatGPT was like this. I had quotations that I had already coded regarding certain questions ( that represent a theme). then I told chatGPT" I am trying to answer this question. can provide me with themes from these quotations?". then I provide all the quotations I had from my research participants. then it provides themes and explanations for each one. I did not use it in the exact way the ChatGPT provided for me. but it helps me to manage my data.

Ahmad azmi Esmaeil Imho this way to use ChatGPT is not "safe", because ChatGPT will readily present conclusions, which do not hold, as if they were facts. Make sure you double-check all "conclusions" ChatGPT provides manually before using them in your research. After-all, you will have to put your own name behind the research results.... Seems like you did that already, so best of luck with your research!

Erik Plesner it is very true. ChatGPT can be easily abused. and frankly, the study that I used ChatGPT to help me with was about the impact of ChatGPT on the future of higher education. so i tried to i do the writing by myself then I asked ChatGPT to correct my English and improve. i think this is one of the healthy approaches to use ChatGPT. it, however, brings up the issue of not improving your English in the first place. so you can write your answer properly. and I believe that mere reliance on ChatGPT to improve your English will eventually degrade your fluency which I believe would ultimately degrade your thinking as a consequence. and i really wish that i did not have to. but i see that almost everyone already use it. so i believe journals would have higher standard in term of their acceptance because of the sudden improvement of the English language. so i had to cope with that( which again is not the best justification for using ChaTGPT)

Ahmad azmi Esmaeil I agree to all of your observations. Imho the main difference between ChatGPT and Google (or other systems similar to either of these) is: The search engine will present you facts with sources and leave it to humans to judge the relevance of the information and draw conclusions. ChatGPT and similar AI systems will present you "facts" which it made up based on biased and incomplete training of a neural network. These "facts" may or many not be true, and come without any source information, and any possibility to verify the credibility of the sources.

This renders (imho) ChatGPT useless in almost any research application. There are far better tools for improving the English language, such as "Grammarly". Try to google that. Grammarly is also an educational tool, which teaches you English at the same time as improving your documents. It literally allows you to "learn from your mistakes".

Erik Plesner it is very true. but when you know that you could just improve the whole your article. like someone who writes and develops much better than you. and all that by mere prompts and copy-paste. it is really challenging to resist that.

i do firmly believe that our life should be easy till a certain limit. more than this we would end up harming ourselves. like now when people replace real social interaction with more convenient and easy ones on social media. And I believe ChatGPT have exceeded this limit. people Should not be given access to such a tool. because many people would follow pray into the laziness nature of human beings. which in the case of ChatGPT would not be very great. thank you again for your respond. it really opened my eyes.

there are easier ways to do coding in qualitative research. save your interview script into a pdf file then enter it into https://www.chatpdf.com/, ask for the topic you want to identify... for example "find a sentence that discusses xxx" look at the results... wow. Hope it is useful

I agree that ChatGPT can be useful for coding. My strategy is to start by asking relatively general questions and then use the results to ask more specific questions. At some point the outcome is basically a hierarchicaly organized code book.

Analyzing qualitative data with artificial intelligence programs like ChatGPT offers several advantages. These programs can assist in data coding, categorization, and thematic analysis, streamlining the process and reducing human bias. ChatGPT can parse large volumes of text data quickly, making it efficient for handling extensive qualitative datasets. Additionally, it can uncover subtle patterns, emerging themes, and sentiments within the text, providing valuable insights that may be missed by manual analysis alone. However, it's essential to exercise caution when using AI for qualitative analysis, as it may not fully grasp the context, cultural nuances, or the intricacies of human emotions and behaviors. Researchers should always complement AI-driven analysis with human interpretation and validation to ensure the accuracy and reliability of findings.

Sounds like ChatGPT to me

Dear colleague,

I am pleased to hear that you have been successfully experimenting with artificial intelligence for qualitative data analysis. The ability of AI to quickly process large sets of textual data certainly has great potential to streamline and enhance the qualitative research process.

I agree that tools like ChatGPT can be very useful for initial coding and identification of key concepts and themes in the data. By asking direct questions, the AI can efficiently detect patterns and extract the core elements, saving valuable time compared to manual approaches.

However, you are right to point out that AI cannot fully replace human analysis and interpretation. A deep understanding of the data and research context is required to truly interpret findings and draw out deeper meanings. The AI is a tool to accelerate the process, not a substitute for analytical insight.

Overall, I see great potential for AI as a complementary technique within the qualitative research workflow. When applied properly, it can improve efficiency without compromising rigor. I would be interested to hear more about your experiences as you continue exploring the use of AI in this domain. Please keep me updated on any new findings or perspectives.

Sincerely,

Gregory celis

Stay tuned, I have written a nice research article on this regard, and will submit it soon David L Morgan