The strategy for 2024 Product Roadmap was to be user-led -- shaping the year's initiatives based on user needs. This was a first for us as a maturing team.
The research themes were previously determined by a series of workshops with stakeholders. What I will expand on it primarily the planning and execution of the research, rather than the content.
We considered a few factors --
1. The nature of the research was to understand the users mental model, identify the nuances in their thinking, and behaviours. It called for a qualitative research.
2. We weren't starting from ground zero -- we already had a conception of our users through quantitative data, surveys, and past research. We wanted to see how our users interact with our product. It had to involve some form of usability testing.
We therefore landed on remote moderated testing as the research method.
Loyal users. Our organisation has a strong presence in the beauty industry and has built a loyal customer base. Both Product and Business valued our most loyal members, as they've gone through the full purchase journey. The breadth of experience was important in this study.
Markets. We targetted total of 12 users from our top 4 country markets to capture the range of cultural backgrounds.
The themes of the research (e.g. product discovery, personalisation, reviews, order fulfilment) were from the workshop we had conducted. Together, they fell along the purchase funnel -- discovery, research and evaluation, purchase, and post-order experience.
Crafting the questions was a notable challenge. We want high-yield questions that were pointed enough to keep user's focus on specific touchpoints, but also broad enough to capture the user's underlying attitudes.
Example questions-
"What inspires you to start shopping on the Sephora app?"
"Show me how you browse the product listing page and think aloud as you go along."
Wanting to keep the interview organic and comfortable for the participant, we went with a live demo from them using their own phones, to bring us through their purchase journey. We got them to thinking aloud as they went along, so we could flesh out their mental model after. Of course we had specific questions we wanted answers to, like what they thought about our Homepage layout (more attitudinal rather than behavioural), but the focus was on getting them to be as authentic to their real life transactions as possible.
We made sure to...
After each interview, the three of us - interviewer, scriber, observer - would immediately convene to gather our thoughts and impressions while still fresh. The salient points that came out of these brief reviews were surprisingly effective in speeding up the synthesis process. We could go into the synthesising exercise with clear themes in mind already.
Each interview ran 90min, going through the participant's purchase journey and thought process. With a total of 11 participants, it didn't come as a shock when we were faced with 400+ data points.
Screenshot of a small portion of the sheer amount of data we had to manage.
This was our step-by-step synthesis process, to break down the mass data into small manageable clusters:
Touchpoint: Product listing page
Broad question: Take a minute to to explore this page and share with us what some of your considerations are?
We took each sub-theme, and strung them into an outline of our users' mental model.
Outline of how our users interact physically and mentally with the product listing page
Now, it was about bringing everything together into user personas and user journeys, as a way to tell the story of our users.
Primary persona, Shafiqah
Secondary persona, Jenelle
How did this fit into 2024's Product Roadmap? We worked with Product Managers to craft How-Might-We's and organise ideation workshops for the next year's initiatives.
Parallel to this, we were working on our UX research repository framework and infrastructure. The research insights were timely in helping our shape and experiment with the respository.
In hindsight, the scale of the research was challenging and might've been untenable if not for the privilege to take a pause on our other design projects to focus on this piece. Also, because of the ambitious goal to reflect our users' purchase journey from start to finish, it naturally demanded significant resources.
To make this more sustainable, the research process for the UX team should be cycles throughout the year, each addressing one research theme based on quantitative problems.
This was an important step for the UX team - it was a chance for us to influence the product team's mindset in product development. Overall, it was a win!