Artificial Intelligence in Sociological Research: Experience of Using It for Interview Processing and Analysis

Voroshilova Anastasiia Igorevna ORCID: 0009-0003-2148-7823 Graduate student; Department of Applied Sociology; Ural Federal University named after the first President of Russia B. N. Yeltsin 19 Mira Street, Yekaterinburg, Sverdlovsk Region, 620062, Russia 79024450663@yandex.ru

Introduction

Artificial intelligence (AI) is one of the most significant phenomena of our time, reflecting the rapid development of technological achievements and human intellectual activity. At its core, AI is a set of logical and algorithmic rules designed to simulate cognitive processes, which allows it to effectively perform tasks that traditionally require human intelligence.

In a rapidly changing world, the question of the permissibility of using AI has become the subject of extensive discussions covering both scientific and public spheres.

On the one hand, the introduction of AI in various fields of human activity promises significant advantages: automation of routine processes, increased productivity and optimization of decision-making. These aspects, among other things, help to free up time resources, allowing people to focus on more creative and complex tasks. On the other hand, this process is fraught with many risks and challenges. The main problems associated with the use of AI include the threat of loss of professional skills, unreliability of data due to algorithmic bias, and the lack of responsibility for automated solutions.

The purpose of this article is to analyze and describe the experience of using artificial intelligence (AI), namely GPT-4o mini, as part of a sociological study to process and analyze qualitative data obtained during semi-formal interviews. The subject of the study is artificial intelligence in sociological research in the context of the experience of using interviews for processing and analysis.

The relevance of reflecting on such research experience is due to the increasing and increasing role of artificial intelligence in modern scientific research, as well as the obvious need to evaluate the interaction between a researcher and AI.

Literature review

Today, artificial intelligence can be attributed to the phenomenon of "second nature", which is a product of natural human development that creates and changes the world around it ^[1]. Rezaev A.V., Starikova V.S. and Tregubov N.D. define AI as "a set of rational-logical, formalized rules" created and programmed by a person, organizing processes that facilitate the imitation of intellectual activity and the implementation of purposeful actions ^[2].

The topic of the permissibility of using artificial intelligence, like any significant technological achievement, has been at the center of discussions characterized by a variety of assessments, polar opinions and forecasts. The positive aspects of the large-scale implementation of AI are largely based on arguments such as facilitating complex, risky, and routine tasks, as well as increasing free time for humans by performing these routine tasks ^[3]. Given the possibility of natural language processing, technologies in this field will only develop every year ^[4]. However, along with the positive effects, the characteristics reflecting the dilemmas of using AI are being criticized by both the public and the scientific community.

The contradictory nature of AI is confirmed by research conducted by VTsIOM among Russians. The research data show that respondents named among the key negative consequences of the development of AI technologies "the risk of fraudulent actions, data theft (62%), the use of AI for selfish purposes (61%), the lack of responsibility for decisions (58%), the risk of making erroneous decisions (54%)." Among the positive effects are "reduction of time spent on routine and monotonous work (65% each), increased labor productivity (65%), easier solving of complex tasks (58%), increased comfort of life (58%), increased safety (57%)" ^[5].

The use of AI in the scientific field also carries a number of negative consequences. First, one of the main aspects is the threat of loss of professional skills among researchers and researchers. Automation of analytical processes can lead to a decrease in the role of human intelligence in interpreting data, which, in turn, can affect creativity and critical thinking.

Secondly, the use of AI in scientific research may contribute to problems with the reliability and quality of data. Algorithms can be subject to bias, which can distort the results of the analysis and lead to incorrect conclusions. This highlights the need for stricter verification of the results obtained.

Thirdly, there is a danger of dependence on technology. Excessive use of AI can lead to a decrease in researchers' motivation to master fundamental knowledge and methods, which can become an obstacle to further scientific discoveries.

However, despite these disadvantages of AI, recently there have been more and more works aimed at understanding the experience of using AI in research activities. There is an opinion that with the help of AI it is possible to optimize scientific activities, and, in this regard, this topic is becoming more and more relevant ^[6]. The use of AI-based tools, such as algorithms for processing natural language and its subsequent transcription, allows researchers to automate some of the analysis procedures, which can facilitate faster interpretation of data and identify relationships that might otherwise go unnoticed using traditional methods.

For example, ChatGPT has already been tested by foreign researchers to analyze qualitative data and has shown itself to be quite capable of isolating key aspects of interviews. However, his answers were biased and sometimes meaningless ^[7]. However, the issue of potential bias in qualitative data analysis is somewhat different, since the data for which information is requested is provided by the researcher, and the biases that arise in this case will be those biases that were inherent in the original dataset. For example, if the statements of the participants contain racist or sexist language and assumptions, then this can be reproduced in the AI responses. One way to avoid such biases is to rely on queries that essentially ask, "Analyze the interview fragments. What do these participants say about..." rather than asking for more interpretative reflections. Obviously, at the moment, using ChatGPT in the analysis of qualitative data cannot be considered as an "ideal" tool. Rather, it should be considered as an additional tool that needs careful verification by the researcher. Thus, the use of AI "acts as an indicator of how perfect each of us can be, as well as limited" ^[3]. In general, the introduction of AI technologies into the practice of sociological research, in particular the practice of qualitative research, carries not only potential, but also challenges, such as the need to take into account ethnic, cultural and social contexts in the interpretation of data.

Within the framework of our article, it is necessary to distinguish between the concepts of artificial intelligence and GPT-4o mini. It would be wrong to say that artificial intelligence (AI) and the GPT-4o mini are one and the same thing. AI is a broad term that covers any system capable of performing tasks that require "smart" actions, such as learning, pattern recognition, decision making, etc. GPT-4o mini (Generative Pre-trained Transformer 4) is a specific AI architecture and model developed by OpenAI, which specializes in processing natural language and text generation. Thus, the GPT-4o mini is one of the many AI applications, but it does not cover all its aspects. Since this article is devoted to the experience of using the GPT-4o mini in relation to qualitative data processing, issues related to the processing and analysis of quantitative data will not be addressed.

Data and methods

The analysis is based on the materials of the study "Marketing ethics as a field of manipulation of consumer behavior", conducted in November-December 2024 using GPT-4o-mini. As indicated on the OpenAI manufacturer's website, the GPT-4o mini is a small model that allows you to perform a wide range of tasks. She is able to cope with reasoning tasks, knows mathematics. The descriptive method, the categorization method and the analysis method were used as the methodology of the subject area of research in this article.

In the course of the study, a number of semi–formal interviews were conducted with marketing experts on the problem of identifying the boundaries of marketing ethics and manipulative practices, as well as their impact on consumer behavior (n=12). The interview data was processed using a GPT-4o mini.

Results and discussion

Transcription

The first stage where the GPT-4o mini can be useful is transcribing interviews. Transcribing interviews is an important step in sociological research, regardless of the recording method, contributing to several key aspects of the analytical process. First, transcription ensures the accuracy and reproducibility of the data, which is crucial for verifying the results of the study. Secondly, this process facilitates a detailed analysis of informants' verbal statements, making it easier to identify key themes and patterns during qualitative analysis. It is important not only to write a written text, but also to note all the details, such as interjections and emotional reactions of the informant. Since transcription takes a long time, it seems reasonable to delegate this task to the GPT-4o mini.

Transcription is carried out according to the following principle: the user needs to provide a video or audio recording of the interview, the promt is set. Soon, the GPT-4o mini returns a text file with a transcript of the interview. Based on the results of transcription of 12 interviews, we identified the following problems. Firstly, the GPT-4o mini distorts some words. For example, instead of the term "brandbook" uttered by the informant, the word "brandbook" appeared in the transcription, and instead of "<...> we are talking about the marketing department <...>" GPT-4o mini suggested "<...> we are talking about the business of marketing <...>». Sometimes AI combines and doubles words, for example, "that is, there is some kind of third-party assessment." Spelling errors are also common, such as replacing the word "hyperbolization" with "hyperpolization." It is often necessary to listen to the interview to understand the distorted word. An example is the case when an informant's phrase was found in the transcript: "well, no matter how you say it directly, you never know who will agree to it." Without listening to the interview again, it's hard to guess that we are talking about the slang expression "steaming." It is worth noting that spelling errors occurred at least 4 times in each text in all 12 transcripts, and the largest number of spelling errors in one text in our experiment was 37.

In most cases, the GPT-4o mini successfully performs the task of separating the interviewer's and informant's cues: out of 12 interviews, 11 transcripts were suitable for work. All of them contain errors, but in general it does not create difficulties for understanding.:

"Participant 00

[00:31:45]-[00:31:56] The question is, how much, in your opinion…Are consumers aware that marketers use such manipulative techniques?

[00:31:56]-[00:32:12] Yes, yes, yes. This is definitely a two-way game. They know perfectly well that normal people consider it all, they find it in the husk. We always say "more details here." Or there's a "call me here." Well, that is, discuss it. I think everyone understands everything perfectly. You have to be…I say, even though they are young, our target audience, but already when you open a restaurant, you have already gone to the lair, someone else is there, people are already beaten with their lives. I think everyone understands everything. Well, I'd like to believe that" (OpenAI, 2025).

The GPT-4o mini correctly separates the phrases of the interviewer and the informant, while timecodes are present; however, all sentences are not separated by spaces, and words are still misspelled.

It is worth noting that the GPT-4o mini is best suited for interviews in which the voices of the interviewer and the informant have noticeable differences in timbre and pitch. If the informant and the interviewer are of the same gender, have approximately the same age, and have a similar timbre of voice, the GPT system has difficulty determining who is speaking, which leads to combining all the cues into one. This happened in an interview.

"[00:10:51]-[00:11:07] Now, Ethics is probably about that. Yes, don't broadcast offensive things. Okay, I get it. But if we narrow this concept down to the field of marketing, will it mean the same thing or is there a different specificity? Well, that's exactly what you mean in communications, that is, well, in advertising, yes, and so on. Yes, yes, yes. In general, the same thing, but the plus is probably added here, well, from the point of view, it seems to me ..." (OpenAI, 2025).

Here we see that there is an interviewer's question in the middle of the text, but since the pace of the conversation was quite fluid and the voices were similar to each other, the GPT-4o mini could not cope with the separation of cues.

Special attention should be paid to the problem of punctuation in the text generated by the GPT-4o mini. Often, the system is not able to correctly determine where a phrase begins and where it ends. As a result, there is a tendency for the "abrupt" wording of the sentence, which significantly complicates the perception of the text. In addition, short pauses in the informant's speech are often interpreted as the end of a sentence, which leads to a loss of semantic integrity and a decrease in the quality of the final transcript. This suggests the need for careful verification of AI-generated text, especially in the context of qualitative research, where understanding and interpreting text plays a critical role. Thus, inadequate punctuation can significantly distort the real meaning of what is said and create difficulties for subsequent data analysis.

As mentioned earlier, the emotions and non-verbal reactions of the informant during the interview are no less important than what he says. However, the GPT-4o mini is based on algorithms that work with text data and does not have the ability to take into account non-verbal signals such as intonation or laughter, which can be indicators of the informant's emotional state. Expressions of anxiety, joy, or irony are often expressed not only in words, but also in intonation, which may be overlooked in automatic transcription. For example, in the fragment below, the informant was humming a tune from an advertisement. The informant sang part of the melody without words, while the transcript reflected only the moment where the words were. The important thing here is how the informant did it, moving his body and head, smiling, showing with all his appearance how deeply this advertisement was stuck in his head. This is definitely something that may prompt the researcher to some reflections.

"[00:10:51]-[00:11:07] I'll give you a drink now, and you'll remember. Where are the meat patties and the grill. You understand, right? Well, that's illegal. Just get into my head and stay there for 20 years, right? That is, it is good marketing" (OpenAI, 2025).

Another significant problem that we have encountered is the presence of low-quality audio in interview recordings, whether audio or video. Although this sound is distinguishable by the human ear, it is difficult for the GPT-4o mini. As a result, the system often ignores these fragments and continues transcribing from the part where the text becomes quite clear and understandable again. This once again helps to understand that any transcript needs careful human revision.

This model demonstrates efficiency in automating the transcription process, which significantly saves researchers time and facilitates faster data processing. However, at the same time, the transcription results reveal many shortcomings, including word distortions, spelling errors and insufficient punctuation, which affects the clarity and interpretability of the final text.

Special attention should be paid to the fact that the GPT-4o mini is unable to take into account non-verbal signals and emotional nuances that can significantly enrich the semantic contexts of respondents' statements. This highlights the need for manual verification and refinement of automatically generated texts, especially in the context of qualitative research, where nuances of perception and expression play a critical role. Thus, although the GPT-4o mini can serve as a useful tool in the context of sociological research, its results require careful correction and analysis by researchers.

Analysis

Before trying out the GPT-4o mini for interview analysis, we asked him how he could help us with data analysis. He identified 6 functions, but the following seemed to be the most interesting:

1. Highlighting key topics. Here, the GPT-4o mini offers to help "identify key themes and patterns" (OpenAI, 2025);
2. Identifying similarities and differences;
3. Grouping of data. It implies combining quotes from informants on topics and categories identified by ChatGPT itself or set by the researcher in advance.;
4. Generation of conclusions. Assistance in forming intermediate or final conclusions.

To test the capabilities of the GPT-4o mini, he was consistently provided with the files of several interviews and was given a prompt: "Highlight the key topics that were discussed in these interviews." As a result, a list of 5 key topics was generated. In general, we can say that all the highlighted topics are extremely superficial and do not provide any new analytical information. Most of the topics provided by the GPT-4o mini relate to secondary aspects of the interview and do not reflect the depth and complexity of the discussions. This may indicate certain limitations in the AI's ability to analyze deeper, contextually rich topics that require a more nuanced understanding of the subject matter and the emotional burden of the informants' statements.

Another task for the GPT-4o mini was a pre-prepared list of informants' answers to the same question. The wording of the assignment was as follows: "Analyze the informants' answers to the question. What do the answers have in common? What differences do you see?" However, there is an obvious flaw in this formulation: we configure the GPT-4o mini to ensure that there are definitely differences and similarities in the formulations of the informants. Despite this, before starting the analysis, we first made sure that such characteristics actually occur. The GPT-4o mini performed much better in this task. Firstly, the system accurately identified the common meanings present in all interviews, although it should be noted that the depth of analysis was clearly insufficient. Secondly, the GPT-4o mini used special terms and, in its comments on the differences, indicated the number of informants who identified certain aspects. For example, he formulated his thoughts as follows: "one informant highlights the need ..." or "one of the respondents notes ..." (OpenAI, 2025).

Grouping data, according to GPT-4o mini, is a process of highlighting the main topics of an interview, followed by classifying informants' quotes according to these topics. After uploading 12 interviews and asking a question: "Analyze 12 interviews. Highlight the main topics in them and group quotes from interviews that are appropriate in meaning," the following results were obtained. Firstly, there are only three main topics, of which two are secondary. Secondly, the GPT-4o mini failed to distinguish between the interviewer's questions and the informants' answers, reflecting the corresponding question as a quote in each category. Thirdly, there were only two quotes in each of the three categories: one of them is a question, and the other is an answer. However, it is not specified from which interview this question was taken. There are also spelling errors in the text. Here is an example of one of the categories:

"Ethics in marketing:

• "You can't give the end user false thoughts, false information about the product."
• "How popular is it with marketers today to meet any ethical expectations at all?"" (OpenAI, 2025).

These examples highlight a number of limitations in the current GPT-4o mini approach to quality data analysis. The system demonstrates insufficient ability to distinguish between types of utterances, which can lead to a distortion of the analysis. Moreover, the lack of an indication of the source makes it difficult to interpret the results.

When performing tasks to identify common meanings and differences in the responses of informants, the system showed a low degree of accuracy, and the depth of analysis remained clearly limited. Problems related to identifying the sources of statements and correcting spelling errors further complicate the interpretation of data and can undermine confidence in the conclusions.

Generating outputs

The last point to check is the generation of intermediate and final conclusions. First of all, it is worth clarifying with the GPT-4o mini what the formulation of interim and final conclusions means for it. Interim conclusions are understood as conclusions formulated at various stages of the study and having a "temporary, partial character, reflecting the preliminary results of the analysis" (OpenAI, 2025). The final conclusions, in turn, summarize the entire study and summarize all the collected data, presenting a complete picture. After asking to analyze the interview and give interim and final conclusions, the following response was received:

"To analyze the interview and form interim and final conclusions, I will highlight several key themes and patterns that can be observed in the materials provided. Please note that the conclusions may be conditional, since more detailed information would be needed for this analysis" (OpenAI, 2025).

It is important to note that the chat itself warns that the conclusions may be conditional and more detailed information will be required for analysis. Next, a list of topics was presented again, mostly of secondary importance, which, in our opinion, does not deepen the analysis.

The system's ability to generate conclusions is limited, which leads to an emphasis on secondary aspects of the study and, consequently, negatively affects the analytical value of the result. Recalling the need for more information for reliable analysis, the GPT-4o mini emphasizes the importance of contextual saturation of data in the inference process. Thus, despite the practical application of AI as a tool for automating scientific analysis, its limitations and the importance of human involvement should be taken into account in order to achieve more accurate and in-depth results in sociological research.

Conclusion

Thus, the results of the analysis indicate that, despite certain limitations in the depth of interpretation, the GPT-4o mini model can be useful for researchers in a number of aspects. First of all, it concerns the automation of the transcription process. It is important to note that any transcript of an interview performed using artificial intelligence must be carefully checked by the researcher, since the model is sometimes unable to correctly distinguish, for example, the interviewer's questions and the informants' answers. There are also cases of spelling errors, which can negatively affect the quality of the final data.

During the study of the GPT-4o mini's capabilities for analyzing qualitative data, significant limitations were identified regarding both the depth of analysis and the ability to distinguish between types of utterances. Despite the designated functions, such as highlighting key topics and grouping data, the analysis results turn out to be superficial and do not provide analytically valuable information, which highlights the lack of contextual sensitivity of AI. These aspects indicate the need for careful proofreading and analysis of the results by the researcher himself.

Nevertheless, using the GPT-4o mini significantly saves the researcher's time, allowing them to focus on a deeper analysis of the data obtained. It is important to note that as an assistant in data analysis, the GPT-4o mini showed mediocre results, limited to superficial conclusions. Overall, the results highlight the potential of the GPT-4o mini as a tool for automating text data processing. At the same time, they reveal the fact that a full-fledged analysis of high-quality information requires human input.