I tested NotebookLM for qualitative analysis
NotebookLM has attracted serious attention among qualitative researchers. Its core feature — source-grounded responses with citeable passages — addresses a genuine limitation of general-purpose chatbots: the tendency to generate fluent but ungrounded claims. For researchers who have watched language models confidently produce interpretations with no visible connection to their data, source-grounding sounds like progress.
Just as a heads-up: Knowing that people use if for qualitative data analysis, I recently took a closer look from this angle. I use NotebookLM for all kinds of things, but qualtiative analysis.
After 5 minutes, I dismissed it from a procedural point of view (of course knowing what we built with QInsights). Just a few pointers: • you have to add project context to the prompt • it cannot distinguish between respondent and interviewer • there is no place where you can add sociodemographics (workaround, on top of each interview transcript, but this does not allow for systematic analysis. • data security - you are on US servers with NotebookLM • and some more (see below)
Despite all of these shortcomings, I wanted to test how well it can analyse data.
The experiment The dataset: four qualitative interviews on friendship formation, material I know well. After uploading the transcripts, I copied the project description into the prompt and then asked:
What was the first impression when they met the person who later became a friend?
NotebookLM responded with a thematic synthesis. Its opening sentence: Feelings of relief in daunting environments often shaped the participants' very first impressions of people who would become their friends.
That is a conclusion, not a starting point. In qualitative analysis, the sequence matters. Interpretation follows from description. NotebookLM immediately developed themes:
• Feelings of relief in daunting environments • Intrigue and curiosity • immediate sense of shared identity • humorous icebreakers or unexpected role reversals
nicely embedded in prose that reads very well. However, this does not open analysis, it foreclose it.
Then I add the following instruction to the same prompt: Be analytical, compare across cases
Based on this prompt, the answer changed, not a deeper version of the first, but a structurally different analysis: different themes, different evidence, different passages retrieved from the same transcripts. Moments absent from the first answer appeared in the second.
NotebookLM for qualitative data analysis
What context does Presenting this experiment, someone asked me whether I provided a role description. I thought that it was obvious from the prompt that the task was qualitative analysis. I gave NotebookLM the prompt again and asked whether this clarified its role. NotebookLM responded:
"Based on the project description you provided, it is abundantly clear that my role is to act as a qualitative researcher and data analyst. In fact, the description perfectly encapsulates the methodological approach we have been taking together throughout our conversation."
So I thought that maybe the project description triggered it to jump to conclusions immediatly.
I created a new project providing a narrower project description — just the opening sentence, stating these were qualitative interviews on friendship formation. This produced a markedly different first answer.
The answer was now case by case and descriptive. Thus, the richer project description including words like uncovering patterns, had apparently triggered the interpretive mode.
I now added the additional instructions to the prompt again: 'Be analytical, compare across cases'. The answer was theme-based as before, however now the themes were:
• The Power of Shared Identity and Circumstance • Friction and Misperception (The "Slow Burn") • Contextual Roles and Boundaries (The "Functional" Impression) • Romantic Miscalibration
Compare those to the above listed themes. Would you expect that these reulsts are based on the same data set and the same question - and in both cases source-grounded?
What is a possible explanation for this? NotebookLM is a generic source-grounded retrieval tool (RAG). Text fragments are retrieved and placed in the context window. There is no analytic scaffolding guiding what gets retrieved, in what order, for what methodological purpose. The model's attention mechanism and its training defaults — toward thematic synthesis — fill that gap.
The experiment showed that using a source-grounded tool is not the only requirement for being useful for qualitative data analysis.
QInsights, the tool I am building, is also build as RAG system, however a methodologically informed RAG. The retrieval is structured around the analytic workflow — project context, document context, case-level organization. The researcher moves deliberately through stages.
Those four interviews I used in NotebookLM are my test interviews, so I know them very well. And I also run the test in QInsights.
QInsights went case by case — describing what each participant actually said — before drawing any cross-case conclusions. Also when I changed the prompt, the answer remained formulated close to the data without premature conclusions.
Certainly you can also do interpretive work with QInsights, but together with the AI assistant - and not as an answer to the first question of a question set that was intended to explore a the larger question of friendship formation.
NotebookLM produced an answer that looked like ready to be put into a report. In NotebookLM, you can create a note from each answer ready to be pasted into a report. If you don't, it is gone the next time you open NotebookLM. The only thing that remains is 1 chat window - within this window you have to complete your entire analysis. Or alternatively, delete the chat and start a new one with no possibility to going back, thinking and reflecting on previous answers.
NotebookLM costs 20 EUR per month. What you get is an answer to a question about your documents. What you do not get is a tool for qualitative data analysis.