Research goals and methods for playtesting - Part 1

(This blog is adapted from our Mobile Playtesting Playbook)

Different research methods are more useful at different stages of game development, but there are no hard and fast rules. It all depends on how you choose to schedule work and produce content as a team.

The research method you use will depend on your current research goals. Rather neatly, this explains why some methods are more common (and useful) than others at particular stages of development, and why there might be specific situations where you want to use alternative methods.

Also: did you know we have our own games user research team, ready to help with anything from designing a playtest, to analyzing your video results and surveys, to creating and presenting a full user research report? Reach out to us directly using the form below to know how we can help!

Usability testing

Usability testing is a term that comes from traditional user experience (UX) research. It refers to a suite of techniques for testing and improving products and their ease of use. Usability testing can be carried out on physical or digital products, but in UX research it is most often seen in website and software design.

Usability testing does apply in games user research, with some minor, but important, differences. For example, in traditional UX research the goal is almost always to let the user complete their task as quickly and easily as possible. In games, there is a role for intended challenge. The enjoyment and experience of the player while completing tasks is often more important than simply completing them.

What kind of research goals are common in usability testing for games? The most common use is testing game features. Features benefitting from usability testing can include:

• quality of life features.
• user interface (UI) designs.
• player menu flows.
• game mechanics.
• player onboarding or tutorials.
  

The most common formal usability testing methods are:

• UX analysis.
• Accessibility review.
• Playtesting with qualitative analysis. Remote or in-person. Moderated or unmoderated.
• Competitor analysis.
• Prototype testing.
  

The most common informal usability testing methods are:

• Community feedback.
• Opportunistic feedback (friends, family, conferences).
• Sample reviews.
  

Fun or appreciation testing

In general terms, the objective of this kind of playtesting is to look at whether players like playing your game. It can be called appreciation or enjoyment testing. It has many subtleties in terms of exactly how you measure the player experience of your game (such as going beyond player enjoyment and into appreciation of specific facets of the gameplay, narrative, and so on).

Appreciation testing differs from usability testing as a research method in these ways:

1. Usability testing can be carried out with small numbers of players, but with intensive and rich analysis of each player’s interactions with the game.
2. Appreciation testing generally requires a larger number of players, because the goal is to ensure that your results are more representative of your player base. In addition, if you are aiming to gain insights from quantitative analysis, there are requirements in terms of the number of players needed. The data can be collected from surveys and other automated methods.

The most important thing to note here is that usability and appreciation research goals are not the same. This doesn’t mean that you can’t test usability and appreciation in the same playtest or research study, but doing so can get complicated quickly.

The most common formal appreciation testing methods are:

• Playtest supported surveys.
• Diary studies.
• Multi-session or longitudinal playtests.
• Structured interviews.
• Player surveys.
• Competitor analysis.
• Concept testing.
  

Some more informal appreciation testing methods are:

• Community feedback.
• Sample reviews.
• Opportunistic feedback.
• Focus groups.

This is the second blog post in our three-part series introducing the Mobile Playtesting Playbook. You can read part 1 by using the link provided. The Mobile Playtesting Playbook is available on Amazon.com, or you can ask your PlaytestCloud representative for a complimentary copy (while supplies last).