Validating a New Feature Through Usability Testing: Mandai X
Conducting remote usability tests to evaluate and refine the concept of a new digital feature called Happie Habitat, before launch.
Research Ops & UX Research
April - May 2024
Background
Mandai X was preparing to launch a new digital feature called Happie Habitat, designed to use animal content (in the form of meme generation) to boost moods of employees, and enhance performance or social connections in the workplace.
Before launch, they wanted to evaluate the feature’s usability and gather user feedback to ensure it was intuitive and well-received.
I was involved in:
Attending the initial briefing to align on research goals
Crafting a research plan
Conducting detailed screening to recruit suitable participants from our user panel
Facilitating remote usability test sessions on UXArmy
Conducting detailed analysis of the findings to craft a report for Mandai X's product team
Research Approach & Execution
Mandai X required feedback from 150 participants across Singapore, Malaysia, Thailand, and Philippines. Given the scale and geographic spread, I opted for remote unmoderated usability testing.
After crafting a detailed screener, I recruited all 150 participants, who matched the target profile Mandai X's product team was looking for.
I coordinated the testing sessions by sharing participation links and clear instructions. Each session involved participants interacting with the new feature, followed by a short survey measuring their attitudes, and reception of the concept.
The sessions were hosted on the UXArmy platform, which enabled screen and voice recordings to capture real-time reaction, user behavior and verbal feedback, providing valuable evidence for analysis.
All recordings and responses underwent quality checks, to ensure the reliability of participant feedback. Low-quality or incomplete responses that did not contribute meaningful insights were screened out to maintain the quality of the usability test findings.
Analysis and Reporting
After completing the sessions, I reviewed the video recordings and survey responses to identify patterns in user sentiment, usability issues, and the overall likelihood of adoption for Happie Habitat.
Some key insights on user sentiment and likelihood of adoption:
85% of participants were likely to use Happie Habitat if the feature was adopted by their organization.
65% of participants cited that contributing to wildlife conservation was a strong motivator for usage.
More than half of participants indicated they would recommend Happie Habitat to their colleagues.
60% of participants believed that using Happie Habitat could help relieve workplace stress.
I synthesized the findings into a structured report that included key insights, verbatim quotes and real examples, and suggested improvements to Happie Habitat that the product team could consider.
Drawing from participant feedback, one of my key recommendations was to enable users to generate GIFs instead of static images, as this could boost excitement and overall engagement. I also provided the recommendation of improving the content generator’s algorithm, as there were concerns that some outputs felt irrelevant.
Overall, the report gave Mandai X’s product team a clear understanding of how the feature was received by potential users, along with recommendations for improvement. The insights provided them with the confidence to move forward with the launch of Happie Habitat, backed by user feedback.
Key Learnings
Remote unmoderated usability testing requires clear instructions and structure. Since some participants were unfamiliar with the process, I provided detailed instructions and support for the technical setup. This experience taught me that clear instructions about tech setup are essential when running usability tests remotely, especially across multiple countries.
The positive responses from participants gave the Mandai X team confidence to launch Happie Habitat, proving that usability testing is not just about finding flaws, but also about confirming what works.
