What vibe coding taught me about qualitative analysis with AI
Dr Susanne Friese · · The journey
Something else today - vibe coding
I am non-technical and also not very interested in learning Python or similar languages. But I am always interested in trying out all things AI-related. You have probably heard about vibe coding. I always wanted to try it but needed a proper use case. A few weeks ago, ChatGPT sent me a message that you can now also build websites with ChatGPT Work. This was my trigger.
So I started — in dialogue with ChatGPT.
As we already had a website, I had a good starting point. I could ask to copy what we already had and then take it from there, making the changes I wanted to implement.
Everything you need to know along the way about hosting, setting the site live, connecting a domain, and so on, you can ask. But before you get there, you can first work on your homepage, all the subpages, the structure you have in mind, setting up a contact form, a privacy page, etc.
You can upload images and tell it where to place them, change colours and design elements, decide where you want the menu, and so on. You do all of this in natural language.
You still need to know what to ask If you want your webpage to be found by Google, it does help if you know a bit about SEO. Then you know to ask whether page descriptions have been added, whether there is a sitemap, whether the pages can be indexed, etc. Some of it was already in place, but it was good to check on it and improve.
This is not much different from other uses of AI. You get a better result if you bring some knowledge to the table. This helps you ask the right questions, notice what is missing, and provide better instructions.
And this is where I started seeing parallels with qualitative data analysis.
It makes a difference whether you work with an AI assistant knowing your methodological approach, or whether you simply rely on whatever the AI assistant knows from its training data. And you need enough knowledge yourself to evaluate whether what comes back from the AI actually makes sense.
From conversation to a live website Back to the website. You can create a website with ChatGPT now because OpenAI provides a server for previewing the pages you create. But this is not necessarily where you want your website to live. You probably want to host it with a provider of your choice.
How this works and what you need to do, you can also talk through with your AI assistant. It will guide you through setting up a GitHub repository, suggest a host that reads your website from that repository, and eventually help you automate the entire process.
And yes, you do feel a bit proud in the end having mastered all that: being the owner of maybe your first GitHub repository, setting up a space for hosting your website, sending data from your repository to the host, seeing it deployed at a draft URL, then buying and linking your own domain.
In the end, it feels a bit like magic. You want to make some changes to your website and talk them through with ChatGPT. When everything looks right, you say: Send it to GitHub now. From there, everything is automated, and a minute later you see the changes live on your website.
There can be issues. But then you send the error report to ChatGPT, it fixes the problem, sends the corrected version again, and lets you know when it is done.
What should be automated — and what shouldn’t I am sharing this with you because qualitative data analysis with AI is, in some respects, not so different from vibe coding a webpage.
The similarity is not simply that both use natural language. It is about deciding which parts of a process can sensibly be automated and which parts require human input and judgment.
In my website workflow, one part can be automated completely: updating the live website. It is the same technical process every time. Once I have approved a change, the files go to GitHub, the hosting provider picks them up, and the website is redeployed. There is little value in me carrying out those repetitive technical steps manually each time.
What cannot be automated in the same way is deciding what I want to change. If I want to rewrite a text, add a new blog post, restructure a page, or change the design, I need to provide input. I need to look at what the AI produces, decide whether I like it, modify my instructions, inspect another version, and continue until I am satisfied with the outcome. That is the part where my judgment matters.
When it comes to qualitative data analysis with AI, there are also parts that can be automated. Speaker recognition is one example. This is a process that can be clearly described and works essentially the same way every time a transcript is uploaded. Therefore, we have programmed this part into QInsights, so you don't have to do it manually.
Generating a theme analysis and pulling references for you to inspect the results in the context of the source data is another process that can be specified and repeated. So this is another candidate for automation. The way Grid Analysis is structured would be a third example.
But what you want to know from your data will be different every time. Which question matters, what you find surprising, which comparison becomes relevant, whether an interpretation seems too smooth, whether you want to pursue a contradiction, or whether something in one interview changes how you read the others — these are not repetitive technical steps. This is why this part is not automated. It is left to you to interact in dialogue with the AI assistant and your data.
The AI should not decide what the result looks like Vibe coding does not mean that the AI decides what website I should have. Only if you just tell it: Build a website for me (plus a little bit of input). If you have not idea what you want, it does produce something, which looks decent - but will it truly reflect you if this is your personal website? Very likely not. It will be a website presenting the average of how personal web pages usually look like.
Same with AI-supported qualitative analysis. You also do not want an AI assistant to decide what the analysis should look like. You will get an analysis that looks decent from the outside. If you take a closer look, however, it will often be rather generic, smooth over contradictions, and reproduce the kinds of patterns that are most likely rather than the ones that are most meaningful for your particular research question.
What this has to do with Hybrid Intelligence Yesterday, I was working on a chapter for a book on Hybrid Intelligence, responding to reviewer comments. For that chapter, I had done a literature review on the current state of using AI in qualitative data analysis. Hybrid Intelligence is generally understood as the deliberate combination of human and artificial intelligence, so that complementary capabilities contribute to outcomes that neither could achieve as effectively alone.
Looking at qualitative research through the lens of Hybrid Intelligence, the field is still quite far away from this. A large part of the current work is still concerned with automating coding, or asking AI to approximate human coding and then positioning researchers as validators of its performance.
A smaller but growing part is looking for new ways of analysing qualitative data with AI — ways that are much closer to the kind of interaction I experienced when vibe coding a website: the AI takes over technical and repetitive work, while the human sets direction, evaluates what is produced, changes course, and decides what the final result should be.
More about that next week.