Interviewing for research: survey results
Taking a snapshot of how UX designers are using research, where the challenges lie, and why it matters.
In the run-up to my November workshop at UX Cambridge on interviewing for design research, I ran a quick survey aimed at exploring how we as designers feel about research, how much we’re doing, and what we find difficult.
The sample size was small - fifty respondents, recruited informally through Twitter - so it’s far from scientific, but there is, I think, some interesting stuff in there.
User experience designers accounted for 32 of the 50 participants, with a mix of strategists, graphic designers and developers making up the rest. Half worked agency-side, with the other half either in client-side roles or freelancing (and perhaps doing a bit of both).
What I asked
I asked respondents about six types of research:
- Stakeholder interviews
- User research interviews
- Online surveys
- Usability testing
- Ethnographic research
- Focus groups
I wanted to understand: how often these research formed part of the design process (and test a few assumptions of my own); how often the respondent personally carried out that research; and how respondents rated their own level of experience. I then asked for some feedback on what respondents found most challenging about research, what they wished they were better at and what they enjoyed.
‘How often do the these types of design research form part of your typical project process?’
Overall, the picture here feels pretty positive to me. Less FAUX (© @leisa) than I might have feared. Stakeholder interviews, user research interviews and usability testing featured ‘often’ or ‘always’ in 70 per cent of projects. Ethnographic research and focus groups were, predictably, some distance behind.
‘How regularly do you, personally, carry out the following types of design research?’
I think the responses suggest something interesting about the length of project engagements. We might be very frequently including significant user research in our project processes, but it’s not happening all that often. Most notably, usability testing and user research interviews are, for almost half of us, restricted to just once or twice a year. Stakeholder interviews are slightly more frequent, with over a third of us running these every month or so. There’s something interesting here, to me at least, in what’s perceived as being a barrier to research. We seem to find it easier to talk to stakeholders than users. There’s a combination of factors at play here: perceptions of cost of talking to users, sensitivities around doing so, ready (and politically expedient) access to the client. There’s a bigger discussion to be had too about the length of project engagements and the impact on how often, each year, we’re getting to put our approach into practice.
‘How would you rate your own level of experience in interviewing?’
I was fascinated by the response here. We’re interviewing consistently, if infrequently, and yet our confidence in our own experience is… well, not so great. Close to half of the respondents - 44 per cent - rated their experience at the low end of the scale. Why is this? When paired with the questions that follow, some possible answers emerge.
‘What do you find most challenging about interviewing for research?’
‘Getting enough practice to feel confident. Creating enough structure to keep interviewees on topic, but allowing room to stray onto relevant things I never would have thought to ask about.’
‘Getting clients to understand the benefits and pay for it. Organising a good quality representation of people.’
Justifying the need for research cropped up a lot - there’s still a battle being waged to get the research done. The figures suggest that particular battle isn’t going too badly, but it’s clearly still happening. Finding the right participants was often cited as an issue - the feedback implied that respondents were reliant on the client (whether internal or external) sourcing people, rather than necessarily difficulty in using research recruitment agencies or other routes to users.
Lastly, approaching the interviews themselves remains a big challenge - how to facilitate effectively, avoid introducing bias, how best to capture the interviews. We are - typically for UXers - still fretting that ‘we aren’t doing it right’. It’s familiar territory, but regardless of our level of experience, an acknowledgement that research interviews are hard, challenging work.
‘What aspects of interviewing and research do you wish you were better at?’
‘Analysis of interviewing is pretty time consuming and difficult if patterns aren’t immediately obvious. I’ve found more collaborative analysis can really help.’
Those concerns about making the best use of the interview itself were a common thread in response to this question. Respondents wanted to be better at drawing out responses from difficult participants; better at managing the flow of the interview and making good use of the time; and better at note-taking. Understanding the themes and patterns of interviews mattered too - how best to share findings from interviews, and interpret and use participant responses.
‘What do you enjoy most about design research?’
‘That moment where something suddenly just makes sense when it didn’t before.’
‘Meeting new people and finding out genuine new info that I can take back to people and change the way they think about how we design our solutions. For me design is about people and I love the way one thing someone says can stick with you for years and help inform how you design. Also, the power of being able to show real user views to stakeholders, especially if it doesn’t match their thinking.’
‘Talking to people, people are great.’
My favourite question, for the responses it generated. I mentioned this in a previous post, but it really made me smile to read just how rewarding respondents found interviewing. The endless fascination with people, the sense of unexpected discovery, the confidence generated from the feeling of better understanding the end-user.
I’ll be covering many of these themes in my upcoming pocket guide for Five Simple Steps, Interviewing for research, looking at recruiting for interviews, how to make the most of the interview itself, and how to really use interviews as material to design with.