From reflex to meaning: What LLMs do in qualitative analysis
Dr Susanne Friese · · Methodological reflections
From Reflex to Meaning: What LLMs Really Do in Qualitative Analysis
Many of you probably know that large language models (LLMs) work by recognizing statistical patterns in text—like a musician who’s heard the same chord progression a thousand times. Just as a pianist reflexively follows Dm7 → G7 with Cmaj7, a language model sees “peanut butter and…” and confidently predicts “jelly.” That’s not intelligence—it’s reflex.
This reflexive pattern recognition forms the backbone of everything LLMs do: token by token, they calculate which words most likely follow. But from that low-level reflex, we can build more complex tasks—like summarization. Summarizing pushes the model further: it has to find what matters, decide what to leave out, and express key ideas concisely. It’s no longer just continuing the tune; now it’s arranging the music for solo piano, choosing which parts to highlight.
Go one step higher, and you reach theme generation. Here the model is scanning across paragraphs and documents for ideas that echo and repeat—emotional undercurrents, shared metaphors, abstract values. Sentences like “migrants leaving home,” “workers packing up,” and “birds flying south” may look different on the surface, but they cluster together because they mean something similar. The model groups those shared meanings and gives them a name like “longing for home”. It’s like a music critic distilling the feeling of an entire piece.
Why This Matters for Coding In the world of qualitative analysis, many researchers are now experimenting with using generative AI to replicate code-and-theme workflows, especially Braun & Clarke’s Thematic Analysis. At first glance, it seems promising: LLMs can label segments quickly, sort them by similarity, and even generate theme names. But under the hood, this requires a technically complex pipeline—breaking up the text into chunks, assigning labels, merging duplicates, calculating vector distances, and clustering overlapping codes. Most of this isn’t something a qualitative researcher can do with just a prompt in ChatGPT.
This process begins with pattern recognition, i.e. identifying the underlying idea or gist of each segment. This is followed by a form of micro-summarization, where the model assigns a short label that captures the main point of the chunk. Unlike the iterative, back-and-forth process used by human coders who continuously revise and refine their code labels in light of new data, the model processes each chunk in a linear pass. Once all chunks are labelled, the resulting list of codes typically includes many near-duplicates or semantically overlapping terms. These then require post-processing: measuring vector similarities between labels, clustering them based on proximity in embedding space, and generating umbrella terms. At this stage, pattern recognition operates again—now at the level of labels—along with summarization to formulate more abstract or inclusive code names.
The next step after coding usually is to cluster codes in order to generate themes (Anakok et al., 2025; De Paoli, 2024; Goyanes et al., 2024; Naeem et al., 2025; Nguyen Trung, 2024; Turobov et al., 2024; Zhang et al., 2024; Gamildien et al., 2023). In essence, this means clustering summaries of summaries, rather than developing themes with an awareness of the underlying data.
So Why Do Researchers Still Want to Code? Why are we bending AI into old frameworks rather than exploring new ones? Because codebooks feel familiar. Supervisors, reviewers, and ethics boards understand them. They offer a reassuring sense of structure. But sticking with the familiar has its costs. First of all, it is not utilising the real strength of the new technology. Any detail, contradiction, or subtlety that doesn’t make it into a code label during the initial chunk-level pass is effectively excluded from all later steps. If it wasn't coded, it won’t be merged, clustered, or show up in a theme. As seen in papers like De Paoli (2024), themes are often constructed by clustering code descriptions—not by revisiting the full text. This means themes aren’t grounded in the original context but in pre-parsed and compressed representations. There’s no “re-reading” of the data to refine or test the theme.
Human coders revisit and revise their codes in light of new data. LLMs, when forced into this process, don’t. Each chunk is processed in isolation during coding. There’s no back-and-forth, and the model doesn't “remember” what it previously labelled unless explicitly prompted. Thus, the coding process may feel structured and “methodologically familiar,” but all what it does creates an illusion of thoroughness.
Instead of forcing AI into legacy workflows, let’s use it in ways that are more natural to LLMs. You can directly jump to theme extraction, instead of painstakingly building themes via codes which are not fully grounded it data. Then take it from there and engage in conversation – LLMs natural territory. Ask questions about suggested themes, dig deeper, retrieve detail and develop an understanding.
Instead of forcing AI into legacy workflows, we should use it in ways that align with how LLMs naturally operate. Rather than replicating traditional coding through elaborate pipelines—which LLMs don’t actually need—you can jump straight to theme generation, grounded in the full dataset. From there, let the model do what it does best: engage in conversation. Ask questions about the suggested themes, dig deeper, retrieve supporting context, and refine your understanding through interactive exploration.
This doesn’t mean giving up rigor—it means applying it in a new way as I have described in my recent article: Conversational Analysis with AI - CA to the Power of AI: Rethinking Coding in Qualitative Analysis
We’re standing at the edge of a new path. Let’s stop coding—and start talking.
Warm regards, Susanne
References Anakok, I., Katz, A., Chew, K. J., & Matusovich, H. (2025).Leveraging Generative Text Models and Natural Language Processing to Perform Traditional Thematic Data Analysis
doi.org/10.1177/16094069251338898 . International Journal of Qualitative Methods, 24.
De Paoli, S. (2024). Further Explorations on the Use of Large Language Models for Thematic Analysis. Open-Ended Prompts, Better Terminologies and Thematic Maps.
doi.org/10.17169/fqs-25.3.4196 Forum Qualitative Sozialforschung Forum: Qualitative Social Research, 25(3).
Gamieldien, Yasir and Case, Jennifer M. and Katz, Andrew (2023). Advancing Qualitative Analysis: An Exploration of the Potential of Generative AI and NLP in Thematic Coding
ssrn.com/abstract=4487768 (June 21, 2023).
Goyanes, M., Lopezosa, c., & Jordá, B. (2024, March 1). Thematic Analysis of Interview Data with ChatGPT: Designing and Testing a Reliable Research Protocol for Qualitative Research
doi.org/10.31235/osf.io/8mr2f_v1 .
Nyaaba, Matthew and Min, SungEun and Abiswin Apam, Mary and Acheampong, Kwame Owoahene and Dwamena, Emmanual and Zhai, Xiaoming (2025). Optimizing Generative AI's Accuracy and Transparency in Inductive Thematic Analysis: A Human-AI Comparison
ssrn.com/abstract=5174910 (March 11, 2025).
Nguyen-Trung, K. (2025). ChatGPT in Thematic Analysis: Can AI become a research assistant in qualitative research?
doi.org/10.31219/osf.io/vefwc_v2 . (January 30, 2025)
Turobov, A., Coyle, D., & Harding, V. (2024). Using ChatGPT for Thematic Analysis
arxiv.org/abs/2405.08828 . ArXiv, abs/2405.08828.
Zhang, H., Wu, C., Xie, J., Lyu, Y., Cai, J., & Carroll, J.M. (2023). Redefining Qualitative Analysis in the AI Era: Utilizing ChatGPT for Efficient Thematic Analysis
arxiv.org/abs/2309.10771 . ArXiv, abs/2309.10771.