Open-ended questions are wonderful right up until you find yourself staring at a spreadsheet with two thousand rows. Useful observations, “everything is fine,” and things you never asked about, all mixed together.
First, you need categories. Then you assign the responses to them. Then you discover that some categories are barely used, while others should have been split in two. And you start again.
Somewhere around here, the researcher promises never to do this again. In the next survey, people get a list of predefined options and a small “Other” field. The method is still good. Dealing with its results is just too much trouble.
Give people room to answer
That cost interested me. When processing a free-text response takes too long, we change more than the analysis. We change what we are willing to ask. Instead of an opportunity to describe their problem, people get our assumptions about it.
That led me to build Grounded for coding open-ended responses. It is a Telegram bot: give it a set of responses, and it proposes categories, codes the material, and explains its decisions. If a category is barely used, it offers to rerun the analysis without it. A separate model checks the coding.
Categories need revisiting. A response might fit several at once, or none at all. Explanations let you see why the bot put it there. Rerunning the analysis means you do not have to cling to a bad first codebook just because you have already spent two days on it.
An AI bot written by AI
I built the tool with AI: AI wrote the bot’s code. I started with a familiar research problem, rather than a new model in search of something to do.
For the public version, I chose Telegram and tried payments in Stars. I also made the limitation clear from the start: this was an experiment, not ready-made corporate infrastructure. The bot let me try a payment model and different ways to improve the coding.
What counts as a result?
In the 2024 announcement, I reported a processing time: a thousand responses in five minutes. That describes the speed of that version, not its accuracy. Having another model check the output does not turn it into ground truth, either.
The researcher is still responsible for what the categories mean, which important distinctions have disappeared, and whether the result answers the question. An automatic rerun makes it easier to revisit categories that did not work the first time.
I wanted to make open-ended questions practical where the analysis had made them too much trouble. “Other — please specify” deserves a better life.