This project aims to provide the Quality Assurance (QA) team with a robust tool to assess the quality of Abuse Moderation decisions made by agents and staff. The tool will streamline reviewing decisions and assigning scores based on a question-driven scorecard.
Creating a QA Audit Tool
Led the end-to-end design and build of an internal Quality Assurance tool at Bumble, successfully launched in June 2024, driving a 76% AHT improvement.
Overview
Objectives:
• Reduce the Average Handling Time of Quality Assurance checks on Abuse Moderation by 50%
• Implement a robust sampling method to pull in random cases
Problem #1
The QA process at the moment is very manual. QA specialist currently use Google docs and other tools alike to complete the process, which leads to a longer Average Handling Time of Quality Assurance checks.
Success metric: Reduce average handling time from 8 to 4 minutes per case (50% reduction).
Problem #2
The current QA workflow is inefficient and hinders overall process optimization.
How might we redesign the Quality Assurance workflow to improve efficiency and deliver better results through a user-centered interface and intuitive user experience?
Understanding the Requirements
The team started with a kickoff meeting to discuss the project and requirements. Following, I had multiple conversations with the project manager (PM) in order to solidify my understanding.
Some of the questions I asked the product manager:
- Can users navigate between cases they've already audited and cases they haven't started yet?
- Should users see a preview of the cases they’re going to audit before starting?
- What happens if users exit out of the queue before they finish all the cases?
The biggest challenge in this phase was translating a non-existent tool concept into a functioning solution.
Another challenge was developing design concepts without knowing the iFrame's functionalities, which were still being reviewed by engineers. To address this, I created multiple design concepts that could be adapted based on the final functionalities.
Design Concepts
This phase involved extensive brainstorming and multiple iterations, making it the most intensive part of the project.
User Flow Option: Modal
In the early stages, a modal component was one of the proposed solutions for displaying the scorecard. It was decided that a modal would interrupt the user's workflow since the main content is hidden and it would take up a lot of real estate on smaller screens.
User Flow Option: Filter Bar
In the early design stages, I chose a filter bar to display search inputs as I was attempting to display this information without taking up too much space.
However, it would be common for the user to utilize multiple search inputs at once which would require truncation and decrease readability and the user experience.
However, it would be common for the user to utilize multiple search inputs at once which would require truncation and decrease readability and the user experience.
User Flow Option: Filter Form
Due to Bumble specific back-end constraints, it was determined that results should be queried on submit as opposed to filter change.
Hand Off and Iteration
I led a comprehensive design handoff meeting with the engineers, stakeholders, and the product manager to discuss final designs, user flow, and reasoning.
This session led to many iterations like adding error states, collapsible function, and clearer button wording, resulting in a better user experience. Consistent collaboration (virtual meetings, Slack) took place between stakeholders, engineers, and operations teams to ensure the tool's success and effectiveness.
I led a comprehensive design handoff meeting with the engineers, stakeholders, and the product manager to discuss final designs, user flow, and reasoning.
This session led to many iterations like adding error states, collapsible function, and clearer button wording, resulting in a better user experience. Consistent collaboration (virtual meetings, Slack) took place between stakeholders, engineers, and operations teams to ensure the tool's success and effectiveness.
Solution
Previously, conducting quality assurance checks was a time-consuming process that lacked a central tool. To streamline this process and ensure consistent, reliable data, we've implemented a new all-in-one tool and user interface (UI). This will significantly reduce the time needed for each quality check and establish a robust sampling method for gathering accurate baseline information.
Result
The Quality Assurance (QA) Tool - Milestone 1 was launched on June 28th, 2024.
Since the launch of the tool, there has been a 72-76% decrease in Average Handling Time (AHT).