Why You Need Dedicated User Researchers

September 6, 2016

Overview
5’ read Big data can't entirely replace qualitative research, personas are useful when done right.

I saw two pieces of news recently about User Research that made me react. They both questioned the need and quality of user research.

The first was from this article Five Design Jobs That Won't Exist In The Future.

“Design research as we know it may cease to exist—at least in terms of the types of ethnographic field work we do today. Research—-and researchers—-will likely be marginalized by new forms of automated data and insight generation, compiled via remote sensing and delivered through technologies like virtual reality.”

John Rousseau, executive director at Artefact.

Big data is certainly a promising field, and the hype is great with machine learning. But it all depends on which stage of the desin process we're talking about.

Big data will be useful in evaluating the effectiveness of a CTA, in usability or A/B testing. Regarding the early stages of design though, when you're still at the definition stage of your product, when you want to know about customers’ pain points, when you're looking to solve problems, I really doubt we can do entirely without ethnographic style studies. This has to do with the internal limitations of big data and machine learning. All the statistics you ever wanted to know (and then some!) might help you identify some problems. They're likely to do so from a business perspective though, not a user one, as the metrics collected are the ones that matter for business goals. Admittedly, we could imagine that more metrics, and those that count for the customer, are recorded too. Even so, all this data won't give you the motivations behind people's actions. They won't explain you why the patterns are what they are. And in order to solve a problem, you need to understand why there is one in the first place. Otherwise, all your attempts at designing a solution are likely to fall short.

Screenshot of a player in Second Life

Rewind and replay customer experience, Second Life style?

Source: Flickr - Creative Commons commercial use and modifications allowed - Author: TORLEY.

Or maybe what John Rousseau means could be that there are so many remote sensors that we can basically replay the customer's life at will in a virtual reality setting. Not speaking of the privacy issues this would likely have, it would be like an observation study, except it's done remotely, with less interference from the observator. Except that, unless a device to read the mind of people is invented (or people become used to think aloud for the benefit of the companies who watch them), we won't learn about the motivations and reasoning simply by watching. Interviews and real-life interactions are likely to be necessary.

That's why I think ethnographic research is and will be an essential complement to big data. Research works best when it combines results and insights from both qualitative and quantitative studies.

Another article I strongly reacted to is Personas, you don't need them, from Matthew Crist. The main argument here is that personas are biased, so not only there's no need for them, but implicitly, you shouldn't use them, and if you still do so, you're a deeply biased (sexist, racist, choose your flavor) person. Instead you should use this new JTBD framework (Jobs To Be Done). I think that task centered design or JTBD has merits, and both methodologies are not incompatible in my mind, they can even work in complementarity, so why disparage personas?

The premise of the argument is problematic: personas are biased. They shouldn't be. If they are, it means they have been done wrong. Personas should be based on research. What does that mean? Well, research is a process too, and researcher are trained to do research properly, and to do their best to eliminate or reduce bias. Matthew Crist has certainly seen a lot of biased personas, and indeed they might be common, and doing a lot of damage. The solution is not to get rid of them, it's to make unbiased personas, based on factual research.

Why so many biased personas? I think it's related to something said in the first article:

No one needs a dedicated design researcher anymore. “The role is so fundamental that every designer should know how to do it,” says West.

The problem is that you can't improvise yourself researcher, anymore that you can improvise yourself designer. Designers have usually a graphic or engineering background, but maybe no psychological, sociological or ethnographic research background. That means that they're more likely to be unaware of their own biases, they might not know or apply the best methodologies to reduce bias, they might not actively look for it and weed it out in their research data. Bias might take multiple shapes: leading questions, user sample, implicit assumptions, etc. It can be sexist or racist, but most of the times, it's far more subtle, linked to your age group, education or socio-economic background.

Screenshot of the Wikipedia page on bias

So many potential biases...

Screenshot of the Wikipedia page on bias

We all have biases. Designers are not specifically to blame. They're expected to do ever more, research, interaction design, UI, graphic design, development… I suppose budget constraints are to blame, and not enough time/budget is dedicated to research. So instead, designers are expected to do it all. No wonder they might be less effective in some areas than others, and research is usually not what they were trained, nor what they applied for. The problem is compounded by the fact that design teams are rarely diverse. A decisive advantage of a diverse group of people is that biases are less likely to be shared by all team members. So internal discussion of research results can help uncover biases and eliminate them. Even so, biases linked to the profession or corporate culture are unlikely to be spotted this way.

These are the reasons why I believe specially trained researchers are and will be essential in a design team: because, despite all the hype, big data is not the fix-it-all oracle we're led to believe, and because you need specifically trained people to achieve high standards in research.

Parting note: If you can't eliminate all biases, the least you can do for your readers is identify them. I obviously have a strong bias in favor of user research, as I am a user researcher myself. This is an opinion post in a blog though, not research results.

comments powered by Disqus