Running great player interviews - Making the most of your time with players

Be sure to read all three articles in this series:
Part 1 Part 2 Part 3

Running interviews as part of your playtesting can offer a direct view into what is happening in players’ brains. This allows us to diagnose exactly what they are thinking, and understand the behavior we see in games - helping us make good game design decisions.

In the previous article in this series, we looked at how to prepare before your first player interview. In this article, we’ll look at how to handle the interview itself, and share tips on how to create a safe environment for players to give feedback, how to take notes, and how to avoid missing important details in your interviews.

Create a safe space for feedback

Although you might feel pretty worried about talking to players, it’s nothing compared to how scary the experience can be for players. Talking to a real-life game developer can be intimidating for players. That will make it hard for them to tell us bad news.

When running player interviews, we have to create a safe environment where players are comfortable telling us their real genuine thoughts. And that starts right from the beginning of the session.

Before anything else, start your interview with some small talk - ask them how their day has been so far, where in the world are they, what did they have for breakfast - anything to get them talking. The goal is to start to put them at ease, and create some rapport with the player.

You’ll then have some details of the interview to explain to them. This starts with introducing yourself, and explaining your relationship with the game or topic of the interview, so they know who they are speaking too.

It’s then ethical to remind them they are being recorded and check they are still comfortable. We should also be reminding them that they can stop whenever they want. Then give an overview of what you’ll talk about in the interview. This disclosure will again help put worried players at ease.=

If you didn’t make the game yourself, you should tell players - it can make them more comfortable sharing negative feedback. Regardless of your relationship to the game, remind players that you won’t be upset if you hear negative feedback (it’s actually the most valuable part!), and that we’re just after their own thoughts or perspective - there are no right or wrong answers. Then give the player a chance to ask questions, before getting stuck in to your interview questions!

Putting the introduction all together in a script makes it easier to not forget the details. Here’s an example:

Welcome

Hi, my name is <<name>>, and I work for <<company>>. Today we will be talking about a new game,  but before that there are some things I need to tell you.

Consent

As you know we’ll be recording this conversation today (so I don’t have to take notes live). Is that still ok? If at any time you need to take a break, or the phone rings, or you need to leave, do say - that won’t be a problem.

The session today

So today, we’ll be talking about a game. This will involve us having a brief chat about some games you play normally and your thoughts about what you’ve seen today.

I didn’t make it

I should let you know that I personally didn’t make the things we’ll be talking about today. The reason I mention that is you should feel free to be honest about what you like and don’t like. I won’t take it personally.

We’re not testing you

There’s no right or wrong answers today, and we’re not testing how good you are at using computers or at playing games. If anything's more difficult or confusing than you’d expect, tell us and we can tell the people making it to make some changes.

Any questions

Do you have any questions about anything I’ve covered so far, or what we’ll be doing today?

Start your session with this introduction. It will not only make sure you’ve told the player everything they need to know, but can also help give some structure to the interview and get it off to a good start!

Start by understanding players

Before you dive into your questions about your game, or your specific playtest objectives, it’s often valuable to start with a broad discussion on their gaming habits, what they are playing recently, and why.

This is useful for two reasons. The first is that it helps ‘screen’ the player - check that they are the right type of player you were expecting, and that they match the criteria you’ve defined for your playtest.

The second is that those conversations will reveal how normal players think about games, and what they consider when deciding what to play, or what to purchase. Remember as game developers we are not normal players, and it can be easy to forget how real players think. Having real conversations with genuine players about their gaming habits is a useful reality check (and will help us make smarter game design decisions in the future!)

Be curious

After you’ve started with your prepared questions (as created in the previous article), it’s tempting to work down the list of questions and just ask each in turn. That is a mistake.

As well as asking questions, you need to be listening to their answers, and reacting to what they say. This is what’s called a ‘semi structured’ interview - being ready to go off the prepared script based on what comes up. This is hugely valuable for revealing what players really think, rather than forcing them down pre-defined tracks.

To do this successfully requires curiosity. When players tell you something, listen to what they say and ask follow up questions, such as a simple “why do you think that?”. Drilling down to specifics is essential, but can sometimes feel repetitive (you might end up just asking “why” or “what about the game made you think that” again and again - and that’s ok).

This can sometimes feel like you’re asking ‘stupid questions’. But that’s ok - the questions aren’t stupid or obvious, our assumptions are dangerous, and players frequently surprise us with their answers.

Being open throughout the session to abandoning your prepared script, and following the interesting threads is the right mind-set to be in!

Be aware of introducing bias

When you’re talking to players, everything you say will unconsciously change their behavior and thoughts.

Because of this, we have to be really careful not to introduce bias via the questions we ask or the words we use. Asking “What did you think about the leveling up system” reveals that there was a leveling up system - something the player might not have known.

With our questions, we therefore want to reveal as little information as possible. One technique for doing this is to use only the words that players have used themselves first, and not introducing new mechanics or terms. Starting a thread of questioning by describing the outcome of the mechanic, rather than the mechanic itself, also helps.

An example follows, where we avoid introducing names or terms the player hasn’t used first:

Question

“Is it possible to make your character more powerful in the game? How?”

Answer

“Well there is leveling up - where the character can spend XP to get stronger”

Potential probing questions, using their words

• Tell me about leveling up, how did that work?
• You said XP - what is that?
• You mentioned ‘getting stronger’ - What does getting stronger do?

What we ask, and how we ask it can change a player's responses. Try to stay as bland and neutral as possible with your questions - you can’t go far wrong with “what did you think?” or “why?”.

Don’t take full notes live

It’s impossible to take notes while actively listening to players, and thinking about sensible follow up questions. Don’t try!

With PlaytestCloud, videos are recorded automatically of the interview, so that you can go back and take proper notes later for your analysis. This means you can give participants your full attention, react to their answers, and build rapport with them.

The exception is if you need to remember a point so that you can ask more about it later - it’s handy to have a piece of paper to scribble reminders on. It’s really common for players to give many answers to a single question, for example:

Question

What did you like best about this game?

Answer

I loved the graphics, the characters, and what happened at the end of level one.

As an interviewer, you want to ask follow up questions about each of these, but you may forget the three things they listed. I’d note down ‘characters’ and ‘level one’ on a post-it, so that I can return to them after asking “What was it about the graphics that you loved?”

Make the most of interviews

Making players feel that you are genuinely listening will encourage them to share their full, and honest, opinion - generating a lot of really valuable data, and greatly increasing the usefulness of the interview.

In the next article in this series, we’ll look at how to handle the data generated by the interview to create useful and compelling conclusions.

‍