We’re perpetually interested in learning more about our customers, and about how playtesting works for them. We learn a lot from talking to both current and prospective customers, and we're always happy to hear feedback of any type.
So in December of last year, we were delighted to have an opportunity to speak with Malte Volger, Market Research Analyst at InnoGames, to learn more about how playtesting works for him and the team over at InnoGames.
We discussed in-house playtesting versus remote playtesting and about who is responsible for playtesting at InnoGames, and how it fits into their development cycle. Finally, we dived into how Malte feels about using PlaytestCloud on a regular basis.
Could you tell me about your role at InnoGames?
Malte Volger (MV): Sure! My job title at InnoGames is Market Research Analyst, and I’m one of our four market researchers. Our role is to do market and competitor review and analysis, online surveys, and playtesting.
The goal with playtesting is to make sure that our games are fun and easy to understand – especially when it comes to the player’s first session or the first impression of a new game.
Of course, this is why we work with you guys a lot.
DC: Were InnoGames invested in playtesting before you joined the team?
MV: Yes. Besides testing remotely with you guys, we also have a little in-house lab, but nowadays we are moving towards doing a lot more testing externally and remotely.
So yes, we do some in-house testing. Sometimes with colleagues, sometimes with people that we bring in from outside. But the question of scalability is becoming more and more critical, especially in the early phases of development.
We want to make sure as early as possible that we're on the right track with the ideas that we have.
DC: Do you guys ever just test artwork and things like that to see what players are responding to?
MV: Yes. This is something we do with external agencies sometimes.
If we have different ideas for an art style, we might show them the same character or building but in a different style. Maybe one more realistic, one more comic-like or exaggerated. Then we ask them, “Hey, imagine a game such as this… Which art do you think would fit that better?”
When do you decide to use remote testing and when do you use other methods?
MV: We are moving more and more towards remote testing or external testing. We try to get that done as early as possible in the development process to have honest, authentic player reactions.
But there are some situations where I think internal tests make more sense for us. For example, if we need people who have a certain level of experience with our games. Let's say we have a game that is already on the market and we want to develop a new feature for it. In that case, the people we should target are in the mid-to-late game stage; then, of course, we need people who have some experience in that game so that they can put the new feature into context.
DC: When is remote testing preferable?
MV: It's preferable when the process doesn't require a lot of interference or moderation from our side.
If we can give precise tasks or if we don't need to hand out tasks at all and just want the players to play by themselves, then remote testing is better.
We often prefer remote testing because it’s good when the players can play at home in the environment they are used to and not in our lab. I mean, we always tell them to relax, we won't judge you. But it still feels like a test situation for some of them. So most of the time we believe the external or the remote feedback is a little bit more authentic. Players are then less afraid to be critical.
DC: Sounds like when the build is more stable, a bit more advanced, then you do some remote testing?
MV: Definitely. It's even becoming more and more of a requirement for games to playtest even early on in the process.
We have different stages within our development process, and at some of them, the game needs to have at least one playtest to prove that it was fun enough or understandable enough. So sometimes we might have to fix something and do another playtest so that the team can move on to the next development stage.
So that gives you an idea of the relevance and importance that playtests have reached. It used to be like, "Okay, if we have time, we can still do a playtest to make sure that we're on the right track,” but now it's becoming a hard requirement and not just a thing that we do randomly.
Do you have an example of a learning taken from playtesting that led to a significant change to a game during its development?
MV: A lot of things can change based on a playtest. When we do a first-time user experience test – like the first 15 to 20 minutes of a game – we usually see a lot of things that don't work.
For one of the games, we realized that we put in too many things into the early part of the game because we wanted to show the depth of the game. We were basically telling the user, "Hey, this is a cool feature, and that is a cool feature, and try this feature, too!”
But we saw that when they actually played the game, they hardly used these features at all because we didn't introduce them properly. We only showed them and said, "Hey, there's this, and there's that,” but we didn't tell them to actually use the feature, so they didn’t learn how it really works.
What we did in that example was that we stretched the content out a bit more over the first hour of playtime maybe, instead of only the first 20 minutes.
What’s your favorite thing about PlaytestCloud?
MV: That’s not an easy question, because luckily for you guys there's a lot of things that I like.
One really good thing that we already see in the company is that many people, also in our management, watch the videos. So I would say “shareability” is my favorite thing.
Once a playtest is over, you get the videos, and anyone and everyone can theoretically watch them. I think that's a real benefit over lab tests as well.
Most of the time in lab sessions, people need to be there at that moment to watch. But with PlaytestCloud, I can just send the link out to the game designer or product manager, and then sometimes half the team watches the playtest, and nothing else gets done. But it's really helpful.
Sometimes, people approach me and say, "Hey, I saw that issue in the video. Didn't you think that was strange?" And I'm like, "I didn't even know you were watching because I didn't even send you anything." But still, in some way, they get access to the videos. That's pretty cool.
One potential downside to this “shareability” is that it can sometimes be challenging to keep track of who’s watching the videos. We need to make sure that everyone watching is aware of the goals of the current test, and that the researcher – or whoever is in charge of the test – retains ownership of the analysis.
PlaytestCloud is also very fast, especially if you don't have any extraordinary wishes. If you have a general target group and don't want them to do anything fancy, I think the process on your website is pretty automatic. I can place an order within 10 minutes or so, and then the next morning, I have the videos. It’s definitely even faster than what we can do internally.
The experience is also great when we have a test setup that’s not explicitly listed on your website, as you can usually make it happen. All I have to do is write you guys an email and say, “Hey, we’re thinking of this or that. Do you think you can do it?” Most of the time, the answer is yes. That's another thing that is super convenient for us.
What about a tip for a new studio using PlaytestCloud?
MV: I think it depends on whether you have done playtesting in some other way before or not. I believe this is not specifically about PlaytestCloud – I think it’s more for user testing in general: It’s better to start small.
For example, I wouldn’t recommend a five-day longitudinal test with 15 people as your first project. It’s probably better to do a first-time user session for half an hour with five people to get started, to get a feeling for what it's like.
