Be sure to read all three articles in this series:
Part 1 Part 2 Part 3
Interviews can be a tremendously powerful tool for game developers. Interviews allow direct insight into what players are thinking, and how their minds are working. This unlocks inspiration for ‘finding the fun’, and allows us to understand exactly what players have or haven’t understood about our games. They allow us to react to emerging themes from our playtests, and chase down the detail required to improve our games.
However they can also be intimidating to run. Interviews require speaking to real players, and hearing often very negative feedback. That takes bravery.
In this article we’re going to look at how to prepare for your first player interviews on PlaytestCloud. We’ll explore how to be confident that you’re deploying interviews at the right time, ensure you’ll learn useful things to improve your game, and remove the risk of catastrophic failures during your interviews.
Interviews are valuable throughout game development
Interviews are just one of many tools available for understanding player behavior, alongside techniques such as observation, surveys or analytics.
Interviews are particularly strong when we need to understand why things occurred.
This can be very useful throughout development.
Early in game production, it’s recommended to spend some time interviewing your target players to understand their past gaming behavior, what they know and how they make decisions. Building a picture of your target players will include understanding ‘what games they play’, ‘how they decide what to buy’, ‘what prior knowledge about the genre they have’, ‘where they play games’, and more…
Arming the whole team with a unified vision of your target player will help de-risk your game development decisions. When making calls on which features to prioritize, what areas to cut, or what might need tutorialising, there is less risk of making the wrong decision when you’ve actually spent time understanding your players. This leads to less thrown-away work, less re-work, and shipping a better game, faster.
It will also make playtesting easier later in development, because the information you learn in these early interviews will inform how you decide which players to invite to playtests later.
Later on in development, interviews are helpful for diagnosing problems we’ve spotted with other methods. We can observe in testing that players are missing crucial information, or quitting within the first 30 minutes, but we don’t know why. We need to ask them.
Asking people live is much more valuable than asking them survey questions. Survey responses are usually shallow, and don’t allow us to ask follow up questions to probe deeper into understanding the root of the problem. That makes it hard to understand the true cause of problems, and very difficult to fix!
Interviews can tell us a lot about ‘why’ things happened. But they are not strong for understanding what happened. Memory is fallible, and so players will be unable to accurately tell us where they got lost, how many times they failed, or what they got stuck on in the tutorial. Overcome this weakness by combining interviews with observational studies - the two make a very powerful combo.
Check whether interviews are right for you
When preparing for your study, start by listing out your objectives - ‘what do we want to learn from this study’. Some examples might include:
- “Will people buy my game?”
- “Where do people get stuck”
- “What’s the most important thing for us to prioritize development on?”
- “Do players successfully learn everything they need from the tutorial?”
Then assess each of your objectives. Ask yourself whether they require knowing ‘what players do’, or ‘why they do it’.
If you are seeing that you have ‘why’ questions, then interviews are the right method for you. Most objectives are best served by a combination of ‘why’ and ‘what’ methods together to draw a complete picture.
Translate your objectives into interview questions
Unfortunately, you can’t just ask players directly your research objectives as questions. Players’ behavior and opinions are important raw data, but a player can’t tell you what to prioritize your development on or whether they understood something fully.
We need to translate these objectives into the actual questions we can ask players. Later on we’ll need to take that raw data, and translate it back into an answer for our research objective.
Return to your list of research objectives. Then for each, think about ‘what do I need to understand about players to answer this?’
For example, if a research objective is “did players understand how leveling up worked in this game”, You might want to start with a broad opening question:
- Was it possible to make characters stronger?
And then once you’ve established if they know leveling up was possible, get them to walk through the process, and look for bits they’ve misunderstood:
- What did you have to do to make characters stronger?
- How did it work?
- What was difficult or confusing about how leveling up worked?
These discussions, and the follow-up questions to their answers, will start to give a lot of raw data about what the player understands about leveling up, and help us infer whether they understood it or not.
Some reliable questions that I always find are good openers for valuable discussions are…
- Overall, what did you think of what you saw today?
- What stood out as the best bit?
- What stood out as the worst bit?
- If the people making it could change one thing, what should they change?
- Anything else you wanted to say that you haven’t had the chance to?
These should be supplemented with other, bespoke questions to explore the topics in your research objectives (e.g. specific mechanics or features).
Remember to ask follow up questions to respond to their answers!
Avoid introducing bias in your questions
When writing questions, be careful about the language you use. Introducing concepts (like leveling up), changes players' understanding and they will learn answers artificially from your questions. Try to be as broad as possible with your opening questions, and then introduce focus through your follow-up questions to their answer.
The most dangerous topic is to ask about future behavior, because players just don’t know, and can’t give a reliable answer. Be careful asking directly “will you buy this game”. They may intend to buy it, but cannot possibly anticipate whether they will or not. Questions about future behavior won’t get reliable answers, and will create misleading conclusions. Focusing instead on past behavior, and infer the future.
Prepare your script
It’s impossible to remember all of the questions you need to ask, and so coming prepared with a script helps make sure the session is a success.
This script should include all of the introductory information required to build rapport (covered in the next article in this series).
In addition to your actual interview questions, it’s valuable to include a recap of your research objectives, so that you can remember why you’re asking that question in the session. This will help you realize when you need to ask clarifying questions, or when you have enough.
An example of how to display this in a script is:
Then repeat a block like this for each of your objectives.
Having a script prepared with questions will reduce the risk of forgetting to cover topics, so it’s strongly recommended. But it’s also important to not rely too heavily on the script - be ready to react to what players tell you, and ask clarifying questions. We’ll cover more on this in the next article.
Control the fear with a pilot test
Before speaking to your first real player, it’s valuable to practice in a lower-risk environment. This can be done with a pilot: a practice session, carried out with a friend or colleague.
They don’t need to be a real ‘valid’ player, but by asking them to role-play and pretend to be the player, you can practice asking your questions. This allows you to spot interview questions that are hard to understand, or don’t make sense - and they can be altered before you meet your first real player.
Regardless of the method used, pilots are very valuable for catching problems early!
Proper preparation makes interviews easy
By now you should have a script for your questions, and be confident that they will produce reliable data to answer your research objectives - you are all set to book your first interviews on PlaytestCloud.
In future articles we’ll be covering how to handle the session itself to create a safe environment where players can give you honest feedback, and then how to take this raw interview data and draw reliable conclusions from it.
