Self Data Collection Method in User Research

Observation, participative or not, is certainly one of the main methods in the toolbox of the user researcher / ethnographer. Observation is irreplaceable, because you get to know how your service or product is really used in the “wild”, not in a laboratory. And the differences are huge: users have to deal with real-life circumstances which often take priority: it might just be traffic, a phone call, or a child tantrum. In any case, the app or device suddenly becomes second or third in their attention. Studies have shown how high is the productivity cost when workers switch between tasks. They need up to 15 minutes to get back “in the flow”. The same is true when users have to divide their attention between your service and what’s going on around them.

Knowing more about the typical circumstances, the when, where and with whom your users actually use your product can help a lot in designing the right interface and interactions. It starts with some obvious features, like auto-completion or pre-filled forms as much as possible. It could also be a “finish later” button, so someone who is interrupted knows they can leave without losing everything they’ve done so far. Or an emergency exit, essential when you offer information to people who might be in danger if discovered, as was the case for Doris Women Refuge Website. Voice commands are more and more frequent, but they can become cumbersome when used in a noisy environment, or, on the contrary, in a place where you’d prefer to keep things private.

Observation is irreplaceable… but expensive

Observation is often the only way to gain this degree of insights into every day use. Even if you ask someone to explain exactly what happened, and tell them to describe the circumstances as precisely as possible, they are likely to give you a very incomplete picture. Not by any fault of their own, it’s just that human memory is highly unreliable, and totally partial.

Unfortunately, observation is not always possible: it is time-consuming and therefore costly. Plus, depending on the circumstances you want to observe, it might not be practical. An alternative or complement to observation is mobile ethnography, or self data collection technique. A few weeks ago, I did some user research in the domain of health care using a combination of three methods: observation, interview and the use of a self data collection tool Indeemo. I found this last technique really useful, when used in parallel with the other ones.

How to use mobile ethnography tools

Mobile ethnography can be adapted to a large number of study types, using current digital tools. Respondents receive one or more questions every day. They can answer using text, voice, picture or video. Video is maybe the most useful, as it’s the medium that gives you the most unintended background information. You can immediately understand how attractive this solution is for an ethnographer: it gives you direct information and data from the users’ perspective. You bypass the mediation of the researcher’s point of view.

Basic survey design principles apply here: questions need to be carefully worded and they should relate to actual situations rather hypothetical ones, recent circumstances rather than distant events.

Researchers also need to maintain their vigilance when analyzing this data. As with all self-reporting techniques, answers are biased. People want to give a good image of themselves, so they might use small white lies, or even unconsciously change their answer to fit what they think is right. Examples I have seen are tidying up an office before recording the video, or using more formal vocabulary to describe the same event when it’s on record.

This is why self data collection methods should complement usual research methods like observation and interviews, but not entirely replace them. I could pick up the discrepancies between the self-reported answers and my own observation, thanks to the time I spent there and the interview I did. Sometimes these discrepancies are minor and won’t impact the results of the study. Sometimes they might be at the center of what you’re looking for.

NB: Of course, no method is perfect. By their presence during observation, researchers also have an influence on what’s happening. First, they tend to blend in the background after a while, and users fall back into their habits. Second, precisely thanks to their presence, they can more easily spot when and how it might influence or change users’ behavior. Thus, they can take it account in their analysis. Identifying this kind of small changes when using mobile tools only is a lot more difficult.