AI’s role in qualitative research: Rethinking the tools

This is at least the plan—to share some thoughts and ideas about what I've learned throughout the week on using AI in qualitative research, along with any exciting announcements.

There's plenty to discuss, and it's been a few weeks since my last newsletter, so brace yourselves; this might be a long one.

Fierce debates have flared up again on an email list, this time focusing not on the usual suspects—bias, hallucination, and training data—but on attempting to use chatGPT as a stand-in for the procedural steps in various analytical approaches like content analysis, grounded theory, or thematic analysis. You name it.

From what's been shared, the expectation seems to be: "Alright, chatGPT, let's see if you're as good as they say. Can you code my data or pinpoint and interpret the right metaphors?"

Well, guess what? chatGPT falls short miserably. Why don't people first investigate what generative AI is actually capable of, using its strengths within known limits to make it a valuable ally, instead of misusing it like a hammer to drive in a screw—and then complaining when it doesn't work?

And why is it always chatGPT from OpenAI? Aren't there other models worth trying? Perhaps the conversational assistant from Claude might better suit certain needs.

When you ask which model they've used, many say, "I tried chatGPT a few months back." Which version? Paid or free? It might have been gpt-3.5. Remember, AI technology is racing ahead. Are you really basing your judgments on outdated tech?

This compounds the problem, along with asking the wrong questions from the start. As I've mentioned countless times, we need to rethink our analytic processes to leverage this new tech to our advantage. Purpose-built apps for qualitative analysis, like QInsights, are designed for this very challenge. chatGPT, Claude, Gemini, Lama—they're all generalists.

A recent LinkedIn discussion questioned whether using general AI actually saves time. The consensus? Not really, because you end up double-checking everything. Time that could be spent on quality control is wasted because these tools weren't built for qualitative analysis, and you can never be too sure they're not fabricating answers.

When using a purpose-built app like QInsights

qinsights.ai/ answers are strictly based on the documents you've uploaded, and it even shows you where in the document the answers are coming from, saving you the hunt.

I know most of you haven't dared to try it yet. It's new, innovative, and yes, a bit scary. Are there any academic articles on it? Not yet—it's too new, and publishing takes time (though I know some people are working on it). It requires the bold to venture into uncharted territory, challenging the establishment that lacks the vision for the future of analysis. And understandably so, as it could mean they are essentially sawing off the legs of the chairs they're sitting on.