Competitor Playtests: Your studio’s secret weapon
Want to beat competitors’ games? Playtest ‘em!
Testing in Games User Research is typically associated with playtesting one’s own game. But as the market becomes more competitive, we are now seeing many more studios testing competitors’ games.
It makes sense. If standing out is harder than ever, then understanding how your game compares is essential. And that’s exactly what competitor testing offers.
In fact, we’ve seen a 14x increase in competitor testing among our most successful clients. So let’s dig into what’s driving this trend:
- Why are top studios running more competitor tests?
- When’s the right time to use them?
- How do you run them ethically?
If you care about making smarter, data-backed decisions, there’s no better place to start than with your competitors.
The Why: Insights from a Competitive Landscape
We saw a variety of use cases, and all of which provide value in different ways.
Identifying Strengths and Weaknesses to Understand Risk
One of the most common reasons for competitor testing is understanding what players expect from games like yours.
Let’s say players consistently praise a specific UI pattern across several top titles in your genre. That’s not necessarily a sign to copy, it tells you what players are used to, and where your game might be introducing risk by doing things differently.
Knowing this helps teams make intentional design choices. Is it sensible to construct a completely novel character selection system when we see there are consistent and established genre expectations within our competitors? If this is a meaningful differentiator for us, and if the answer is yes, you can move forward knowing the unfamiliar design can be supported, tested and validated by additional research.
And when you identify pain points in competitor games, that’s often where opportunity lives. If players are frustrated with certain mechanics elsewhere, you might be able to differentiate your game in a way that’s likely to be received very positively by players.
Benchmarking
To truly understand a game's performance and predict its potential for success, robust measurement is essential. Competitor testing gives you access to performance data from successful games, helping you define meaningful internal metrics. With this context, you can set realistic goals—and raise the bar—before and during development.
The benefit to competitor testing here, is that you can establish these benchmarks based on a very wide variety of games, not just those within your own portfolio. And thanks to remote playtesting, you can test a wide range of titles across diverse player groups, without the overhead of in-house labs or recruiting.
Validating and Challenging Your Own Assumptions
I think we’ve all had instances where someone on the team is obsessed with a new feature that’s released in the latest viral sensation. Whether it’s the latest Call of Duty or a survival game that came out of no-where, people get caught up in the sensationalism of the success of these products and by association, a belief forms that every component of the experience must have played a meaningful role in the product's success story (this is called the Halo effect).
Suddenly, you have a designer or even team of designers, seeking to pull in ideas from these competitor titles. Without data guiding the way these assumptions can lead to expensive decisions being made to pivot or change work that don’t lead to materially better player experience outcomes. In a best case scenario, this work is wasteful, raising production costs and extending timelines without benefit, in a worst case scenario these changes add risk and uncertainty to your project, sometimes even hurting the player experience.
Sometimes the design assumption you want to examine might not already be represented in another project - or so you believe. Competitor testing helps cut through the hype. It gives you player data to back up (or disprove) assumptions, whether you're exploring a new feature or betting big on your game's unique selling point.
With competitor testing developers can examine these games with real player data, determining the real impact of those features within their context. This enables studios to make data-guided decisions that ensure that if expensive decisions are made, they’re grounded in evidence that reduces the risk as you move through a complex, and ever changing landscape of competitors.
The When: Pinpointing Your Moments for Competitor Playtesting
We see two main stages where competitor testing really shines:
Early: Before a single line of code is written, competitor testing can help define what your game should be. It complements ideation, highlights gaps in the market, and helps you avoid committing to risky features too early.
Think of it as part of your pre-production toolkit—right next to concept testing and audience research.
Throughout: It’s important to remember that the landscape of competition with your project isn’t static. Without pointing fingers, I think we can all identify examples of unsuccessful games that might have found success if they made it to market sooner. More importantly these represent instances where the projects failed to adapt to changing market conditions as they moved forward. The market opportunity you identified 3 years ago might not be the same today, and it's critical that studios acknowledge this as they move through their development lifecycle. Ongoing competitor testing helps you stay aware of shifting expectations. It gives you the context to decide whether and how to pivot during development—without relying on gut feel alone.
Finally we need to talk about responsibility. While competitor testing is an essential and powerful tool, with great power… and, you know the rest! This power comes with a critical responsibility to navigate the space ethically. Here’s how to ensure your competitor testing is ethical, respectful, and useful:
Focus on the Player Experience, not on imitation: A strong guiding principle is that when you’re competitor testing is to focus on the player experience. Rather than looking to extract the precise form of the game being tested, you’re looking to extract insight derived from the experience of the players playing it. This also leads to healthier differentiation, as players often chase similar experiences from games that take distinct forms.
Adhere to Confidentiality and Privacy Requirements: It’s essential that any data you’re collecting is publicly available. That means that the product is either released to the market already, or that any testing with players participating in alphas or betas, is not covered by player non-disclosure agreements.
Respect Intellectual Property boundaries: When unpacking competitor games it’s essential that you understand which components are attributes of any intellectual properties held by the studio or their partners. For example, a competitor playtest might reveal a game to have very appealing character designs, but that should not be taken as a cue to mimic those designs - instead look at this as an opportunity to zoom out and understand why those designs are valued as a component of the player experience, and you can use that insight to establish your own experiential targets.
Competitor Testing: An Essential Component of Your Development Process
As competitor testing is increasingly used by successful clients, we see it as an essential tool for studios to use as part of a successful user research strategy.
It helps you:
- Understand player expectations
- Identify risk and opportunity
- Benchmark performance
Validate decisions with data
It’s this zooming in and out at different time points that makes for an effective and highly-impactful user research strategy that affords every project the best possible chance for success.
