Gamification product, Mile Play

mobile app continuous improvement, quant and qual analysis

 

United Airlines has a product called Mile Play, a personalized offer that incorporates gamification. The product team came to the UX research and design team to improve and optimize the usability of the product. I collaborated with the designer and marketing analytics team to conduct quantitative and qualitative research, providing comprehensive recommendations for improvements and roadmap strategy.

Methodologies:

  • Analytics and voice of customer data review

  • Survey (n=1,092)

  • Baseline/MVP usability testing

  • Stakeholder roadmapping workshop

My Role

This product team was not set up the same way as other teams, lacking an embedded designer, business analyst and other roles. The centralized UX team provided support for this research and subsequent design. I was the research lead, planning and conducting all research for the project.

The plan included:

  • Proposing and advising on a quantitative survey that was executed by the marketing team

  • Conducing expert and heuristic reviews of the existing platform

  • Running an in-lab, moderated baseline study of the current product

  • leading stakeholder workshops to come up with roadmap initiatives

Research Plan

The Mile Play team came to us knowing the product could be improved but only had usage data, lacking the qualitative narrative of how customers perceived and used the product. While there were clear opportunities to improve the usability of the flow, I saw a greater opportunity to first provide the team with a better understanding of how, why and when customers use their product. Unlike our core products such as booking, managing reservations, check-in, etc, this product was intended to be personalized and provide ancillary revenue. Based on analytics and voice of the customer data, it was clear the product did not feel very personalized and ancillary revenue was not as high as it could be, we just had to get to the bottom of why. While the product team hypothesized it was purely usability, the designer and I also suspected it was about overall value and communication strategy. To test these hypothesis I executed the following methodologies.

homepage.jpg

1,092 users surveyed &
22 usability sessions across 2 studies

Survey: Part one of the research included a survey with 1,092 participants that was ran by the marketing analytics team to understand current Mile Play customer’s perceptions/sentiment toward the experience, offers and incentives of the program. I proposed and oversaw this survey because there were a multitude of customer segments, game types, and levels of interactions with the product that I wanted to get representative data from.

Baseline/mvp usability study: After the survey was conducted I completed 45 minute in-lab usability sessions with the current Mile Play product and new game type MVP. The test was split between two weeks and had 2 different test flows in order to evaluate multiple game types. We also used part of the session to collect preliminary feedback on proposed enhancements. For this recruit, used customers who had previous experience with mile play in order to ensure they were not only United customers, but Mile Play users or potential users. I balanced both status levels as well as previous Mile Play interaction (game completion, partial, never completed).

Outcomes

Using both quantitative survey data and qualitative usability data I was able to provide insights around value, satisfaction, personalization and usability. The designer and I took the findings from the survey and usability sessions to propose a new UX/UI strategy. We attended and presented at a multi-day stakeholder road mapping workshop and shared our proposal to improve interaction and game completion as well as overall sentiment. The team incorporated the recommendations and plan on executing them in future projects.