Learn about one tool that can help you run more effective user interviews.
Generative user interviews are a foray into the unknown. As opposed to evaluative interviews, where you are assessing a reaction to an existing product, generative interviews are meant to help you figure out what to build in the first place. But every style of research has its limits — a random poll might be too, well, random, and a questionnaire is too rigid to navigate the complex lives and motivations of your users.
In this piece, I will discuss the use of a topic map as a great way to improve your interviews by remaining accountable to your assumptions and keeping the conversation as natural as possible.
Why topic maps?
User interviews might be the most versatile qualitative user research tool at your team’s disposal. They are often the cheapest way to effectively validate some of your most fundamental assumptions. Frequently running user interviews to de-risk your product will help ensure long term success by surfacing real problems experienced by your users.
During generative user interviews, your main goals are to gain empathy for your target users, validate or invalidate actionable problems, and gain new useful insights. This type of interview is ideal when you build a new product and you have little confidence in the hypothetical user behaviors, needs, and goals. Ideally, you want to start an interview with a clear set of topics to discuss, but you should also be open to discover new topics you have not anticipated. Keeping track of all these topics and potential topics is what a topic map is for.
At Pivotal, we believe that having an unfiltered, open-ended conversations with your interviewees yields higher quality insights. Asking questions based on relevant topics that start with words such as “How”, “What”, or “Why” will prompt the user to explain her context without giving hints one answer is better than another. This reduces your own bias in the outcome of the conversation. Running the interview one-on-one in a comfortable setting can help the person open-up and reveal why he or she behaves in a certain way.
Most researchers use a questionnaire as a support tool to guide the interview process. But there are problems with relying on a series of questions in a fixed order. If you try to stick with the pre-defined order of the questions independently from the answers of the interviewee, you might inadvertently lead them through a particular structure that might color the final result. Also: the conversation will feel stilted, unnatural, and won’t create a welcoming environment to pursue a fresh insight.
Some of our teams like to use a topic map instead. It is a type of mind map with 5–7 larger topics and 4–6 subtopics. It usually looks something like this:
Why topic maps are better suited than questionnaires for generative user interviews
1) It’s more convenient than a list of questions.
You should limit your topic map to a single piece of paper. This will guarantee you will not have to switch between pages to find the next topic to talk about. Additionally, it helps you maintain your focus on the interviewee by minimizing the cognitive load of handling sheets of paper and visually parsing text to find the right questions to ask.
2) It helps you remain accountable to your assumptions.
A topic map exemplifies the famous quote from Alfred Korzybski “The map is not the territory.” In his theory of general semantics, the Polish-American scholar argues that humans can never fully grasp reality because of sensory and language limitations. Lean startups de-risk limitations in business by introducing the idea of continuous validation. Your topic map is an imperfect representation of your initial assumptions and you are about to confront it with the user’s reality.
Imagine we have made the assumption that people studying Japanese as a foreign language use iPhone apps to learn Japanese and need a more automatic way to be reminded of words they care about. We draw a topic map that includes this topic and associated sub-topics:
Let’s now imagine we have established in our interview that the user is indeed studying Japanese. We can ask the following questions to determine if they use one or more mobile apps to study:
- “What kind of material or tool do you use to study?”
- “I use the textbook from my teacher and an english-japanese dictionary”
- “Oh interesting, what is the name or title of your dictionary?”
- “Mmm.. I’m not sure but the brand is Berlitz.”
- “How did you acquire it?”
- “I bought it from a senior student at my school.”
You can see in the questions in bold that the interviewer is secretly trying to determine if the user uses a dictionary app. At the end of the last answer, the interviewer can be rather confident the dictionary is made of paper instead of bits because it is unlikely that you have bought an app from a senior student.
The topic map helps you know if you have earned the right to discuss the topics listed in it. In that case, the interviewer can only ask about the textbook and the dictionary. He can try to dig deeper or ask more general question but he cannot lead the interviewee to these particular topics by asking questions such as “Do you use a mobile app to study Japanese?”
3) It helps the conversation flow more naturally.
A topic map only includes keywords. The idea is to give you a visual reminder of what topic can be touched upon based on your assumptions. You do not have to stick with a question’s particular wording and can use the context of the conversation to form the next investigation. You can reuse the particular words that your interviewee just used in place of those on the map.
Once you have crossed off a sub-topic from a particular bubble, you do not have to stick with the same subject to drive the conversation. It is possible the interviewee will talk about something interesting from another bubble in the same sentence. The topic map can help you keep track of where the conversation has been, and you can always come back to any crossed-off topic at a later time in the interview.
At Pivotal we have observed that interviewees who talk a lot tend to have a scattered distribution of crosses along the map in the first ten minutes. These types of conversationalists will require more circling-back to clarify topics. While they usually make for better interviews, they are also harder to keep track of if they jump from one topic to the other. On the other hand, interviewees that have more trouble opening up will be easier to question about a single topic but might be harder to move between bubbles.
A good way to transition from one topic to another is to ask the interviewee to talk about a recent story. The amount of details often give the opportunity to branch off to different topics through follow-up questions. The topic map will help the interviewee open up more and answer less rigidly. A structured questionnaire carries the risk the interviewer might unintentionally break the flow of the story to get back to the script.
4) It helps ease into more complex topics.
The design of the topic map — a high level topic surrounded by more granular information — helps the conversation move from easy to more difficult questions. Asking about complex details right away might be hard for the interviewee to handle because he or she might not use the same language as you, be unfamiliar with the context of your assumption, or simply not familiar with the interview format.
For example, if you are trying to understand how a technical worker operates a certain type of software to find defects in a metal slab, you might first ask for a walk-through of what a typical work day looks like and see if something related to metal defects is mentioned.
How to help beginners use a topic map for the first time
Topic maps can sometimes feel intimidating, especially if it is one of your first user interviews. To ease the burden on beginners, we sometimes add a few more words to our sub-topics and phrase them as questions. While a bit heavier, it helps you avoid blanking on a particular question.
To reinforce the need for open-ended questions, you can rephrase the questions to be more vague so that you will be less tempted to lead the user. We also write the only keywords you are allowed to start questions with: “Who?”, “When?”, “Why?”, “Where?”, “What?” and “How?” in one of the corner of the sheet.
How to build a topic map
1) Review your assumptions.
At the start of an interview round, your business goals might be of varying clarity. This ambiguity is dangerous for user research because there is too much room to interpret the results differently from one person to another. To minimize that effect, you want to state precisely what your assumptions are and prioritize which is the most important. Several techniques let you capture these assumptions. Our teams like to synthesise these into various artifacts like User Personas and User Scenarios. Before starting your interviews, hopefully you will have produced a decent amount of assumptions that express what you think the market looks like:
- Who the user is
- What are their observable behaviors
- What problems are they trying to solve
- What are their underlying needs and goals
2) Ideate the questions with Dump & Sort.
Next, we recommend using a quick brainstorming technique called Dump and Sort where a cross-functional team and its stakeholders come up with the questions they want to ask users based on the assumptions previously reviewed. Use sticky notes to capture the questions individually and silently in a short amount of time (five minutes). Once the time is up, ask each participant to self-edit for the questions they feel are most valuable within a minute. The participant should then paste their questions on a wall, which are then sorted collectively.
3) Group into topics with an affinity map.
Once all your stickies are on the wall, aggregate them into topic groups. For example, if you are working on a diet tracking app you might have a group for “demographics” information, another one for “eating habits”, and another one for “doctor visits”. By the end of the activity, you should have six main topics to discuss. If you have more than six topics, an arbitrary way to prioritize is to keep the six groups with the most stickies in them. Once you have your six groups, you can remove duplicates and review them as a team.
4) Draw the Topic Map and prepare your opener.
Go through each group and try to extract 4–6 subtopics from your questions. If this conversation is too hard, select a cross-functional pair to help the team stack-rank the group. Keep only the main questions and simplify them to one or two keywords.
The last thing you need is a strong opening question in a storytelling format. In order to do that, you will need to use your value proposition (for example: “With product X, persona Y can do Z”) and phrase it as a general contextual question like: “Tell me about the last time you studied a foreign language.” The answer to that question should enable you to cross off your first subtopics as it will give you a recent first-hand experience to chat about. By that point, you should have everything you need to build your topic maps!
This article was written by Hadrien Raffalli (Product Manager) from Pivotal Tokyo. Illustrations are from Erika Ito (characters) and Canyon Boak (topic maps). Special thanks to Doug Blumeyer for the time he spent teaching the author basic and advanced English grammar.
Change is the only constant, so individuals, institutions, and businesses must be Built to Adapt. At Pivotal, we believe change should be expected, embraced, and incorporated continuously through development and innovation, because good software is never finished.
How to Use Topic Maps to Run Generative User Interviews was originally published in Built to Adapt on Medium, where people are continuing the conversation by highlighting and responding to this story.
