UX research challenges: frontline stories and tips from 5 industry experts

The UK chapter of the UX Professionals’ Association (UXPA) put together a panel discussion in London and invited five experts to share their stories and advice.

The panelists started by introducing themselves and sharing memorable research stories.

When Sharan Gandhi, Lead UX at Aviva, joined the company a few years ago, there was a common misconception of what user experience was. People asked her the typical question: “can you do the UX on this?”. The challenges at the time were very different and included having to explain to people what UX was, as well as not analysing/applying results from testing due to lack of time.

“I quickly learned that stakeholders don’t care about the jargon we use, but it was when I started using their language that they started to care.”

The story shared by David Loughlin, Director of User Research at Goldman Sachs, happened really early in his career, when he was supporting what looked like a straightforward piece of usability testing to help UK farmers apply to European subsidies.

“One of the first challenges was that we couldn’t recruit from a pool of farmers in London, so we had to take the research lab out to a suitable location in the countryside, in the Midlands. We had a big screen and everything ready for our observers that came all the way to the countryside to watch the sessions. We thought we were really well prepared, but 30 minutes before the first session was due to start, the recruiter informed us that the participant couldn’t attend because they had been up all night helping sheep deliver babies. However, he could do it over the phone.”

David and his team weren’t aware that they were conducting the sessions during lambing season, when the farmers would be extremely busy.

“What I learned from this experience is that with research, 30% is about doing the research, 20% is analysis and 50% is all about planning and preparing to do the research.”

A memorable moment that shaped the way Cath Richardson, Research Lead at Projects by IF, conducts research and her attitude towards it was a project focusing on why people dispose or choose not to dispose of items. For this research, Cath had to do in-home interviews. When she arrived at the house of one of the participants that had a particular decor style — all rooms were black and white, with shiny elements — , she unconsciously judged the participant’s personal taste. “It wasn’t really my style, but okay…”.

“Over the course of the interview, it emerged that she had had a troubled relationship with her husband, which had ended five years before, and she wanted to start from zero and leave everything behind,” including the decor of their house, which was all in brown tones and very traditional.

“It was a very emotional story, to the point that our observer said it felt like a therapy session. We built this amazing rapport with her and what I took away was that you should avoid bringing judgement into this kind of situation.”

The one that stuck with Jess Lewes, Business Development Director at People for Research, was a user recruitment project for a charity that supported people with neurological conditions and also provided bereavement support. The charity was looking for people who had lost someone close and wanted to find out more about the support they had or hadn’t received.

For Jess, it was really difficult to talk to most of the participants; during the panel, she highlighted a call with a woman who had lost her husband very suddenly.

“I struggled not to cry, because I wasn’t ready. I had to step away after each screening call to decompress. When working with vulnerable people, it’s important to make sure they are ready to discuss a sensitive subject.”

On a more relaxed note, Jess also shared the story of a user recruitment project that People for Research did for an online retailer of adult toys. The goal was to recruit people who had recently purchased their first sex toy from this platform, which would mean they weren’t familiar with this kind of product. By accident, the recruiter working on this project ended up recruiting a dominatrix — if you’re not sure what a dominatrix is, let’s just say this is usually someone with an extensive knowledge of sex toys — who had indeed purchased her first product via this seller not long ago… but this was definitely not her first experience acquiring this kind of product.

“This showed me the importance of screening people holistically.”

David Stevens, now Director of Service Design at Idean UK, shared a story from his time at the Home Office, when he shadowed user testing with people going through digital passport renewal.

“Part of the government services is to make sure the services work for everyone and you need to test with people with different access needs. In this case, it was people with facial disfigurements and different facial structures. We were using third-party facial recognition software and, for people with facial disfigurements, we knew there was a risk that the software would not work. With some of these participants, the software actually said it couldn’t detect a face and this said to the participant ‘you’re invisible’, which is the opposite of what people with facial disfigurements feel when they are in public.

“This lack of emotional sensibility affected the sessions.”

After the introductions and memorable moments, it was time to answer questions from the audience and share pro tips.

How do you stay safe when doing research outside of a lab?

• Home visits are the only situation where you could feel like this, said David Loughlin: “when going to the field, don’t go on your own.”
• Having two researchers in each in-home sessions helps to protect both yourself and the participant.

How do you start doing international research in more than one location?

• It’s useful to have a local partner; not just a recruiter, but an agency to help you in the field and establish common understanding about the local ways of working.
• Engage with partners like Experience Labs. Do research in your home turf first and then scale up to other locations across the globe. Testing the screener in a familiar market helps you to determine if you are getting the right information from the screening process.
• Conduct lightweight agile research with freelancers in these locations. The set up is usually complicated, but you can make it work through researching the markets and building foundational understanding.
• Build relationships with translators/interpreters — don’t expect people to speak your language.
• Having a workshop at the end of the research cycle in each market to find out what was learned and share the knowledge across the company.

What to do when stakeholders act like the research findings are not surprising and devalue the research?

• The first step is to educate your stakeholders.
• Frame your research correctly to avoid it being influenced by opinions.
• Don’t be afraid to challenge the hypothesis (if the research is hypothesis-driven).
• Help the stakeholders come to the conclusions of the research by themselves by sharing snippets of the sessions. They are more likely to accept the findings if they think they have come to those conclusions by themselves.
• The panelists recommended two books that could be useful to help with this particular challenge: Nick Bowmast’s Userpalooza: A Field Researcher’s Guide, and Black Box Thinking: The Surprising Truth About Success, by Matthew Syed.

We want to keep in touch. Join our mailing list to receive our newsletter and/or our monthly round-up.