UX Cambridge research results
In the run-up to my November workshop at UX Cambridge on interviewing for design research, I ran a quick survey. It was aimed at exploring how we as designers feel about research, how much we’re doing, what we find hard.
The sample size is small - fifty respondents, recruited over a short timeframe through Twitter, so it’s far from scientific, but there is, I think, some interesting stuff in there.
User experience designers accounted for 32 of the 50 participants, with a mix of strategists, graphic designers and developers making up the rest. Half worked agency-side, with the other half either in client-side roles or freelancing (and perhaps doing a bit of both).
What I asked
I asked respondents about six types of research:
- Stakeholder interviews
- User research interviews
- Online surveys
- Usability testing
- Ethnographic research
- Focus groups
I wanted to understand: how often these research formed part of the design process (and test my own assumptions); how often the respondent personally carried out that research; and how respondents rated their own level of experience. I then asked for some feedback on what respondents found most challenging about research, what they wished they were better at and what they enjoyed.
‘How often do the these types of design research form part of your typical project process?’
Overall, the picture here feels pretty positive to me. Less FAUX (© @leisa) than I might have feared. Stakeholder interviews, user research interviews and usability testing featured ‘often’ or ‘always’ in 70 per cent of projects. Ethnographic research and focus groups were, predictably, some distance behind.
‘How regularly do you, personally, carry out the following types of design research?’
I think the responses suggest something interesting about the length of project engagements. We might be very frequently including significant user research in our project processes, but it’s not happening all that often. Most notably, usability testing and user research interviews are, for almost half of us, restricted to just once or twice a year. Stakeholder interviews are slightly more frequent, with over a third of us running these every month or so. There’s something interesting here, to me at least, in what’s perceived as being a low barrier to entry in research terms. We seem to find it easier to talk to stakeholders than users. There’s a combination of factors at play here: perceptions of cost of talking to users, sensitivities around doing so, ready (and politically expedient) access to the client. There’s a bigger discussion to be had too about the length of project engagements and the impact on how often, each year, we’re getting to put our approach into practice.
‘How would you rate your own level of experience in interviewing?’
I was fascinated by the response here. We’re interviewing consistently, if infrequently, and yet our confidence in our own experience is… not so great. Close to half of the respondents - 44 per cent - rated their experience as low. Why is this? When paired with the questions that follow, some possible answers emerge.
‘What do you find most challenging about interviewing for research?’
‘Getting enough practice to feel confident. Creating enough structure to keep interviewees on topic, but allowing room to stray onto relevant things I never would have thought to ask about.’
‘Getting clients to understand the benefits and pay for it. Organising a good quality representation of people.’
Justifying the need for research cropped up a lot - there’s still a battle being waged to get the research done. The figures suggest that particular battle isn’t going too badly, but it’s clearly still needed. Finding the right participants was seen as a problem too - some of the feedback implied that respondents were reliant on the client (whether internal or external) sourcing people, rather than difficulties using research recruitment agencies or other routes to users.
Lastly, approaching the interviews themselves remains a challenge - how to facilitate effectively, avoid introducing bias, how to best capture the interviews. Familiar territory, but regardless of our level of experience, an acknowledgement that research interviews are hard, challenging work.
‘What aspects of interviewing and research do you wish you were better at?’
‘Analysis of interviewing is pretty time consuming and difficult if patterns aren’t immediately obvious. I’ve found more collaborative analysis can really help.’
Those concerns about making the best use of the interview itself were a common thread in response to this question. Better at drawing out responses from difficult participants; better interview flow; managing the time effectively; and better note-taking. Understanding the themes and patterns of interviews mattered too - how best to share findings from interviews, and interpret and use participant’s comments.
‘What do you enjoy most about design research?’
‘People. I love people. I love people trying to use technology. I love making technology better for people.’
‘That moment where something suddenly just makes sense when it didn’t before.’
‘Meeting new people and finding out genuine new info that I can take back to people and change the way they think about how we design our solutions. For me design is about people and I love the way one thing someone says can stick with you for years and help inform how you design. Also, the power of being able to show real user views to stakeholders, especially if it doesn’t match their thinking.’
‘Talking to people, people are great.’
My favourite question, if only for the responses it generated. I mentioned this in a previous post, but it made me smile to read just how valuable respondents find interviewing. The endless fascination with people, the sense of unexpected discovery, the confidence generated from the feeling of better understanding the end-user.
I’ll be covering many of these themes in my upcoming pocket guide for Five Simple Steps, ‘Interviewing for research’, looking at recruiting for interviews, how to make the most of the interview itself, and how to use interviews as material to design with.