Survey Writing: Write Better, More Effective Survey Questions

As with any type of research, choosing the right tools and methods is extremely important to gather the information you need. Asking the right questions is vital for user research, making sure that the responses you receive will be useful in gaining insight into your games. At PlaytestCloud, we’re committed to helping you create the best surveys, but we also understand it can be a daunting task for new or inexperienced researchers. That’s why we’ve compiled a list of tips and tricks to get the most out of your surveys!

Avoid leading questions

One major pitfall of many researchers looking to use surveys in their work is to ask questions that will lead participants to answer in a way that confirms the researcher’s thoughts. These are called leading questions, which lead the participant towards answering in a certain way through how the question is written. An example of a leading question in games user research would be:

In this question, the participant may feel as though they should respond in the positive, because of the connotations around “below-average prices” and suggesting an answer that the playtester was very happy. To avoid this, be sure to check over the language of the questions and verify that the choice of words is neutral to allow the participant to answer as they see fit. In the previous example, the question could be altered to read:

This removes the leading language surrounding “below-average” and makes the pricing scheme as a neutral element.

Use a scale that makes sense for your project

Whether you are using a Likert scale^[1], a dichotomous scale^[2], or something else entirely, researchers should base their decision for the correct scale based on the needs of the research project. These decisions will affect the results and potential findings of your surveys, and can change the way that you will interpret the results. Some things to consider include:

1. How much flexibility in terms of responses should participants have? Is it necessary to have a Not Applicable option in your Likert scale, and how granular / precise do you need the responses to get the feedback you need?
2. Does a yes / no answer give enough information to make decisions based on those responses?
3. Is it important to include qualitative responses as well as quantitative responses to allow participants to freely answer and give some sentiment feedback?

On PlaytestCloud, you can set up scale questions using the Multiple Choice and Rating question types.

Consider giving an opportunity for additional free-form responses.

As mentioned previously, it may be important to allow participants to answer freely through text, allowing for the capture of their sentiments and feelings about particular questions in your survey. While PlaytestCloud does allow for a stream-of-consciousness, think aloud way of expressing the playtesters thoughts, it may also be important to give players time to reflect on what they have just played in a post-game survey that can drill down into specific questions. As well, some questions may be geared towards follow-up qualitative^[3] questioning, whether that is for sentiment analysis or to allow participants the ability to explain their reasoning for the quantitative response they have given.

Ask questions that are specific and targeted at what you need to know

Finally, make sure that the questions you are asking are not there just for the sake of asking the question; each question should be specific and target something that you would like to learn from the players. Keep in mind the goals that you have for testing with PlaytestCloud, and draft your questions to help get at the core of what you would like to learn from the players. Are you really interested in whether they liked a particular graphic, or are you more concerned with their likelihood to continue playing the game?

‍

1. A Likert scale includes a series of questions that you ask people to answer, and ideally 5-7 balanced responses people can choose from. It often comes with a neutral midpoint. ↩︎
2. A dichotomous scale is a two-point scale which presents options that are absolutely opposite each other. Often a yes/no question. ↩︎
3. Qualitative research utilizes questions that are not based on quantity (numerical) that allow researchers to determine the emotional, experiential, and cultural significance of questions being asked. These questions typically take the form of open-ended prompts that allow the participants to freely answer. Qualitative research empowers researchers to look into the meanings behind responses, allowing them to take on the role of understanding why or how instead of how many or how often. ↩︎