Co-author: Lydia Galeano, UX Researcher at Antidote
In game development, the concepts of quality assurance (QA) and playtesting are often used interchangeably, even by the most experienced devs.
Although they appear similar, both testing methods aim to achieve specific outcomes and misusing them can compromise the quality of the feedback.
In this article, we’ll explore the unique roles of QA and playtesting, explaining how they work and when to use each to create the best possible game.
Grasping the Basics of Both Testing Methods
What is Quality Assurance (QA)?
The goal of QA is to ensure that the game works correctly and meets technical standards before its public release.
A typical process involves testing the game to detect and fix bugs, glitches or other performance issues. The type of testing highly depends on the objectives and the areas of the game you want to examine.
For example, a QA team might perform regression testing, where they re-test the game after fixes to ensure that new updates haven’t introduced new problems.
Alternatively, testers can simply try to ‘break the game’ by doing exactly what players aren’t supposed to do, like climbing an invisible wall, stacking multiple items or any actions that can cause glitches.
In this case, their aim is to find anything that causes errors, write it down and share it with the dev team.
This proactive approach helps catch technical issues early, ensuring a smoother playing experience for your players by reducing frustration and early drop-offs due to buggy gameplay.
What is Playtesting?
In contrast to QA, which investigates technical issues, playtesting is a journey into the players’ world.
It involves understanding how players experience the game by determining their:
- Behavior, actions taken when interacting with the game
- Cognition, mental processes such as perception, memory and problem-solving during gameplay
- Emotions, feelings and emotional responses elicited by the game
All these dimensions help gather feedback on player engagement and satisfaction, providing insights that can shape the game’s design.
For instance, a First Time User Experience (FTUE) playtest can be used to examine various initial gameplay elements, such as:
- Effectiveness of the tutorial stage
- Engagement with the storyline or game world
- Intuitiveness of the menu’s UI and mechanics
- Clarity of game objectives and goals
or to learn what players like or dislike during their first gameplay session.
This equips developers with valuable feedback, helping them fine-tune the game based on what players think and expect.
And why is that good?
By doing so, studios can enhance long-term player retention, creating a more compelling and engaging gaming experience.
QA vs. Playtesting: 3 Key Differences
We’ve established that QA pinpoints technical issues while playtesting studies the experiential side of gameplay.
However, game devs often fail to distinguish between these two methods, which can lead to lower-quality feedback.
So, what are the common pitfalls?
Let’s explore three key differences highlighting how QA and playtesting each play a unique role in game development.
Type of Participants
QA Testing
QA is conducted by paid professionals from within the industry, using manual testing and automated tools.
This can be carried out by:
- An in-house team of testers familiar with the game’s development process
- A dedicated QA service provider to whom the testing is outsourced
Their expertise lies in technical problem-solving, ensuring the game runs smoothly and meets all required specifications.
The size of the team largely depends on the project’s size, scope and budget. For smaller indie games, a QA team might consist of 2-5 testers. For larger AAA titles, the team can include 20 or more testers.
As mentioned, the QA team is often personally invested in the project, meaning they might not fully immerse themselves in the game as a new player would.
For example, they might overlook confusing game mechanics or assume certain controls are intuitive because they have seen them develop over time.
Picture a QA tester who has worked on a game for several months. They might quickly navigate a complex menu system because they have used it countless times.
However, a new player might find the same menu confusing and frustrating, something the tester might overlook due to their extensive knowledge of the game.
Hence, it’s important not to ask a QA tester to assess the player’s overall experience, as your future players won’t have the same level of familiarity.
Playtesting
In most cases, playtesting occurs with external players rather than development team members.
Participants for this testing method are recruited through two main avenues:
- Internal player base who are already part of their community and are big fans of the game
- External player base who might not have engaged with the game yet and are not part of their existing community
While both groups offer pros and cons, the main goal is to understand how the game resonates with its target audience, the real people who will play it after its release.
This will help to uncover issues related to user experience, enjoyment and engagement.
The number of participants required for playtesting can vary greatly depending on the study’s goals and objectives.
For instance:
- A full playthrough playtest might only require five participants to play the game from start to finish.
- On the other hand, a large-scale multiplayer test could demand over hundred players to test the multiplayer mechanics and social dynamics.
Due to its research nature, playtesting sessions might not detect all the minor or hidden bugs a QA team would find.
Consider a FTUE playtest session for a new adventure game.
External playtesters, who are tasked with playing the first two hours of the game, might provide feedback on the enchanting storyline and the awkwardness of movement controls.
At the same time, they might not notice a minor graphical glitch in a specific scene or realize that a particular attack sequence causes the game to crash.
Which is not ideal.
Therefore, it’s important to recognize that both QA and playtesting aim to solve specific challenges.
Based on this understanding, you can determine which type of participants – an in-house trained team or an external player base – will be most appropriate for your test.
Testing Process
QA Testing
QA adopts a structured and systematic approach to testing.
This process often includes automated testing, where scripts are used to repetitively check for bugs and performance issues. QA testers also perform manual checks, rigorously going through the game to find glitches, crashes and other problems.
The goal is simple: to ensure the game is released without technical setbacks.
The QA team might run automated scripts to check for memory leaks and performance bottlenecks (such as CPU or GPU usage, frame rates and other metrics) throughout different game levels.
They also perform manual checks to ensure that each game level loads correctly and that there are no graphical errors.
If any errors are found, the testers record the event (for instance, with a screenshot and a timestamp) to give feedback to the dev team, which will try to replicate and solve the issue.
Playtesting
Playtesting, on the other hand, delves into how players interact and perceive the game.
The goal is to discover what they enjoy and what might be frustrating or confusing.
Notably, each study will have unique research objectives depending on the game’s development stage and the type of feedback needed.
Based on our experience, some of the playtesting objectives can include examining:
- Gameplay experience
- Story and narrative
- Game difficulty
- Usability testing
- UI and controls
You can uncover the feedback by observing, recording and analyzing playtest sessions.
But what would be the best and easiest method to do that?
Using a dedicated user research platform, you can collect and analyze playtesters’ feedback from one centralized space.
For example, in a FTUE playtest, a UX researcher might ask participants to navigate through the tutorial stage.
The process will involve recording their gameplay to analyze their actions and check if they understand what they are meant to do or where to go next.
Once the sessions are completed, post-session surveys are used to gather detailed feedback on their experience.
In this case, you can utilize a specialized platform to:
- Upload your build
- Give the project name, description and instructions
- Configure your study’s details (duration, controls, features, reward amount and others)
- Design surveys for ongoing or post-session feedback
- Invite your players, conduct the playtest and record the gameplay
- Receive automated insights from survey answers and transcripts to gauge overall player sentiment
- Export the findings to share and analyze with your team
(or, if you want to save time and avoid the logistical headaches of organizing a playtest, you can delegate the manual work to our experienced user research team💡)
This helps identify which tutorial parts are intuitive or in some cases, need some adjustment to clarify the game’s instructions or mechanics.
Results & Deliverables
When comparing the outcomes, QA and playtesting each tell a different story.
QA focuses on quantitative data, providing pragmatic insights such as performance metrics.
Playtesting, in most cases, dives into qualitative feedback to understand player experiences (though it can also collect quantitative data depending on the research objectives).
Because of this, the analysis and presentation of results differ between the two methods.
QA Testing
In QA, results are collected through a systematic process.
Testers use automated tools and manual checks to identify and document technical issues.
Some of the deliverables for this method include:
- Bug report that documents any issues or glitches found during testing. For example, if a character suddenly disappears when performing a specific action, this bug would be recorded with steps to reproduce and fix the problem.
- Compliance checklist that ensures the game meets all platform-specific standards. This includes validating that the game follows guidelines set by platforms like PlayStation, iOS or Steam. For instance, ensuring the game saves correctly on all consoles or doesn’t crash when accessing the online store.
While there are other methods and procedures, the core of QA is about testing and validating specific elements of the game to ensure they function correctly.
Playtesting
Conversely, playtesting is more than just gathering data from recordings and surveys; it’s about deciphering how players truly experience the gameplay.
This involves identifying what works well or what doesn’t and determining which elements need to be improved to enhance the overall user experience.
One effective result evaluation technique is thematic analysis. This qualitative approach identifies common themes and patterns extracted from collected data, such as survey and interview responses.
Once all the feedback is analyzed, one of the best ways to present the information is through an in-depth UX report.
This report documents the detailed findings from playtesting sessions, highlighting player satisfaction and frustration areas.
Additionally, it offers recommendations that the development team can implement to improve the game.
The Importance of QA and Playtesting in Game Development
QA and playtesting are both momentous components of game development, each tackling different but equally important aspects.
Neglecting either method can compromise the game’s quality, which can impact the success of its launch.
QA ensures the game works from a technical standpoint, while playtesting ensures it resonates with players and delivers an enjoyable experience.
As such, there is no ‘better’ method, it’s about leveraging both to release the best possible game.
Let’s say you are developing a new action-based game with a multiplayer mode.
The QA team runs automated scripts and manual tests to uncover technical issues. During one of these checks, they discover the game’s performance drops significantly when more than 100 simultaneous matchmaking rounds occur.
At the same time, a multiplayer playtest was conducted, with participants divided into different teams to battle on various maps. This study revealed that the playtesters found the round duration too long and repetitive after a few playthroughs.
(these findings are an example of what a playtest can reveal, as seen in one of our previous playtest runs on our platform)
Using this feedback to shorten the combat rounds and enhance server capacity, you can release a game that meets technical standards and provides a replayable player experience.
Summary
We hope this article clarifies the importance of understanding the difference between QA and playtesting.
Each testing method addresses specific challenges that help you meet your development needs more effectively.
Some of the key differences between QA and playtesting we discussed were:
- Type of participants
- Testing process
- Results and deliverables
We recognize the logistical and time-consuming headaches of organizing a playtest, especially with an external community.
That’s why Antidote offers support to studios and publishers in coordinating playtests.
From player recruitment to session setup to analyzing results, we handle all the heavy lifting so you can focus on perfecting your game.
If you’re looking for a research partner for your upcoming study, contact us and let’s make your playtesting process as easy as pie!
