The first time I sat down for a live playtest, I was petrified. I was unsure how to build rapport, when to jump in to ask questions and the atmosphere must have been very tense for the unfortunate player stuck with me.
Overcoming my fear was worth it though - during that session we discovered critical usability issues. Fixing those meant we didn’t lose players in the first hour of gameplay.
Live playtests are one of the most valuable skills for uncovering real insight into how players think and act. By coming up with and asking new questions live, we can understand and explain player behavior in enough detail to understand the root cause of the issues we spot, and make effective fixes.
I’ve mentored many game developers on how to run their first playtests, and seen some of the challenges that people face. I’ve also seen the value that running those sessions has created for their game, and understand the potential of Live studies.
Live playtests are the fastest, most effective way of getting deep player insight on your game. In this post, I’ll share some of the top tips for making the most of those sessions, and running productive Live playtests.
If you’re feeling worried - imagine how the players are feeling? Taking part in playtests is unlikely to be an everyday occurrence for them, and they don’t know what you will ask them to do. They will be worried about getting things wrong, looking silly or disappointing you.
This will start to change their behavior, and so we need to help them overcome their fear.
The first step of that is the introduction you give to the player. Give them some context to the session, explain who you are and give them an idea of what’s about to happen.
An example introduction could include:
By setting expectations at the start of the session about what’s going to happen, and what’s expected from them, players can start to feel at ease, feel more comfortable sharing their opinions and display more natural behavior - increasing the value of the playtest.
It’s tempting to jump straight into your tasks when running a playtest with players.
If we’re interested in observing a player's behavior in a section of the game, we may ask them to “play this level and tell us what you think”. However, by starting with the task we’re missing the opportunity to gather valuable contextual information that will help us interpret what we’re seeing.
Instead, start the session with some questions about the player’s normal gaming habits, such as:
Not only will this help you interpret the behavior you see during your playtest, it’ll build your understanding of your players - what else do they play, why do they play games like yours, and how do games fit into their life. That understanding can build your understanding of your players and help make future design decisions easier.
Playing games can be intense, and questions distract players. It takes mental energy to think and give an answer, and that can be difficult in the middle of gameplay.
Part of the skill of moderation is deciding when to ask questions. Asking too early during busy sections will distract players, and cause them to fail or struggle more than they normally would - impacting their opinion. Asking too late means they will have forgotten their genuine thoughts, and will come up with post-hoc explanations that don’t necessarily match their real opinions.
Asking questions is important, to understand and interpret the behavior you’re observing. When moderating, keep an eye out for natural ‘down-times’ in gameplay, and use those to ask your questions.
By running Live studies, you are also no longer reliant on ‘think aloud’ - asking players to narrate their thoughts out loud as they play. Think aloud is a great tool for unmoderated research in order to understand what players are thinking, and gives some insight into the behavior we’re seeing. However with particularly intense games, players can get distracted and forget to talk - meaning we miss crucial detail. Live research allows us to prompt for more detail when necessary.
Whenever you ask a question, you will be revealing new information to players.
If you ask them “Why did you not upgrade your horse armor”, you are telling players that horse armor exists, that upgrading it is possible, and that it’s surprising they didn’t upgrade their horse armor already - this might not be information they had already.
Be careful not to introduce new concepts or information that players haven’t demonstrated to you already. My approach is to start with very bland questions, such as “what are you doing currently?” or “what’s happening now?” or “what’s that?”, and guide players using their own words and terms towards the topic I want to discuss.
A direct question isn’t terrible - but understand that after you ask it you will be changing the player's behavior and giving them knowledge they might not have. Use caution before deciding to deploy one!
The super-power of Live studies is it allows us to see directly inside the player's minds. In unmoderated studies, we might see players do something odd or unexpected, but can only speculate about why they did it.
Some behaviors can’t be correctly diagnosed without understanding what players are thinking. The difference between a player not seeing an objective, or a player seeing it but intentionally avoiding it will look the same behaviourally. But if we want to make game design decisions to change that behavior, we’ll need to recognise which of those it was - as they’d require radically different solutions.
Asking questions in Live studies allows us to understand why unexpected behavior has occurred. When you see something interesting, remember to ask players questions to uncover why it happened. Once again, bland questions such as “what’s happening currently”, will prompt players to talk about what they think is happening, and give essential clues that can be used to inspire creative fixes to issues we observe.
Moderating is a skill that takes practice, but is an extremely valuable part of your research toolkit. Overcoming your fear, and running your first Live playtest will start to develop your moderation muscles and unlocks a huge amount of valuable data to inspire creative decision making. Live Playtesting will bring unparalleled new possibilities to all PlaytestCloud users–and you can now register to be one of the very first to use it. Fill out the form below, and start playtesting with your players directly while helping us in shaping this new feature to the very best it can be.