Practical tips in running your first diary study - pt.1

What is a diary study?

A diary study is a qualitative research method that records aspects of participants’ daily lives and environments over a designated period of time. Traditionally diary studies were recorded in a physical journal, but more recently mobile technology allows us to collect richer and more varied data points - photos, videos, short or long-form text, and geolocation. Typically an exit interview is conducted with a selection of participants at the end of the study to interrogate their entries and gain deeper insight into their experiences. If possible, this happens at their own home or workplace in order to gain further insight and capture additional data points.

Why conduct a diary study?

The success of any research study is underpinned by the methodologies selected to meet the research goals. When your research goals are to gain a deep understanding of people’s behaviours, experiences and surroundings over time, interviews and lab tests will only get you so far.

This is when a diary study is a good selection. It helps to tell the story of how products and services fit into people’s daily lives, and the touchpoints and channels they choose to complete their jobs.

Where ‘true’ ethnographic field studies require researchers to be embedded in the participant’s context making first-hand observations, diary studies require participants to self-select relevant moments and share them remotely through a mobile phone app. As you can imagine, these two methods require different amounts of resources and planning and can generate different results. Here are some reasons you might want to consider a diary study:

• Remote participation allows for a broader geographic spread of locations.
• Multiple research sessions can be run concurrently.
• Costs are significantly lower than in-person field studies.

Diary studies can also be a resource-intensive method to setup and run. Here are some of our most useful learnings so you don’t have to learn them the hard way.

Evaluating tools

Having a reliable and easy-to-use tool for submitting and analysing diary entries is fundamental to a successful diary study. We recommend investigating the options available long before your study begins.

There are many digital diary study tools on the market, most of which offer a free trial or a demo. We highly recommend setting up a 1:1 call to run through the software and answer any questions you might have. Many have an example project to run through that will give you a feel for the study setup, entry monitoring and analysis features on offer. This is also your opportunity to talk about pricing if you are on a restricted budget.

We’d advise getting your hands on a trial version and running a pilot study with your internal team. Doing this helps to weed out any problems you or your participants might have. You might want to consider:

• Participant onboarding and app setup. How many hoops do your participants need to jump through to get started? Are participants able to get setup unaided?
• Diary study entry submission. What is the experience like for the participant submitting entries? Does it allow for structured tasks and open entries?
• Performance issues. What happens when battery saver mode is switched on? Does it work consistently on various devices, on both iOS and Android?
• Administrator view. How flexible is the backend of the system? Can you layer additional data onto your participant profiles?
• Analysis features. Does the software allow you to run any analysis? What is the export feature like?

A pilot study also helps to test your hypotheses and assumptions before you commit to the final study design.

The rule of thumb: the more tyre-kicking you do before your study, the less time you’ll be kicking yourself during it.

Designing the study

With your pilot study complete and the most appropriate tool selected, you’ll be in a good place to design your study.

Designing how your participants submit their entries is a careful balancing act, and will ultimately cause a tradeoff:

• Too many data points can be overwhelming. Too few can leave you frustrated.
• Small and frequent entries can provide granular detail. Large infrequent entries can expose relationships and reflective stories, but require work to separate into individual data points.
• Broad and unstructured entries can expose interesting unknowns but provide a shallow depth of insight. Narrow and structured entries may result in deep but biased results.

Use the result of your pilot study to guide your decisions.

One useful technique we’ve used is to reserve a few spot tasks to be completed during the study. These give you a little bit of flexibility during the study to ask deeper questions around themes that begin to emerge. Tasks may involve replying to an email with questions about a particular diary entry, completing an online survey or a specific task or mission.

With your tasks agreed, it’s time to write a guide to onboard your study participants. This should set the expectations of the study clearly, written in a language they understand. Your friendly neighbourhood content writer will be pleased to help here; if your neighbourhood doesn’t have one, go out and find one, they’re awesome and indispensable!

Preparing the participants

With your study guide written and given the seal of approval by your new content buddy, it’s time to brief your participants.

At this point we’re going to assume you’ve properly screened your participants. They are being incentivised for their time and are willing to take part in a follow-up interview. They’ve also received your study guide ahead of the study start date.

Incentives will depend on the length of the study, the number, frequency and type of entries you require your participants to share. A quick way to validate your costs is to work out an hourly rate based on how long you expect your participants to be submitting entries for, over the course of the study. Ask yourself, “Would I go out of my way to contribute to the study for this amount?” If the answer is “no” it’s likely you’re participants will answer the same way. If you are using a recruitment agency they will be able to advise on incentive costs.

The week before the study begins, it’s important to check-in with your participants to talk through the expectations of the research and answer their questions. Depending on your recruitment process this may be the first time you’ve been in direct contact with them, so think of this as the start of your rapport-building. A diary study often involves participants sharing personal aspects of their lives so showing you are trustworthy at this point will pay dividends during the study. With this in mind, we highly recommend doing this face-to-face if you can, or via a video call as an alternative.

Once the study has started, it’s easy to focus on participants that are less engaged, have technical problems or general questions.

While it’s important to address participants in need, it's just as important to reassure participants who are completing entries without problems.

We find it useful to write a couple of stock emails before the study begins, one to encourage less engaged participants, and another to reassure participants that are on track. Having these responses in reserve helps to free up time to do research instead of admin.

It’s inevitable that a small number of your participants will completely disengage during the study, so it’s worth recruiting a few extra participants to ensure you meet your target number of interviews. An early conversation with your client or stakeholder about this helps to avoid misunderstandings about the quality of the participants and your recruitment efforts.

Scheduling

Be mindful of when your study will be conducted and whether events around that time may cause unintended or unexpected behaviour.

For example, results from a diary study learning how families consume digital media conducted during term time would be vastly different from that during school holidays.

Depending on your sample size and time constraints, you may want to consider splitting your participants into two or more groups, and stagger the study over a number of weeks. A staggered schedule helps to ease the intensity of monitoring diary entries, enabling you to avoid key dates for all of your participants e.g. public holidays. It also creates an opportunity to learn and adapt the study over a longer period of time.

If you are running staggered start dates, it’s a good idea to group participants by location and ensure they start and end the study at the same time. This will save you the headache later when trying to schedule home-visit interviews across multiple cities.

Summary

Diary studies are an effective way to gain deep contextual insight at a fraction of the cost of a true field study. They’re also a lot of fun to run. With the right amount of preparation you can reduce the time spent on admin tasks and participant hand-holding, and refocus your efforts on the research itself.

Here are our top 4 takeaways:

1. Run a pilot study on the topic you are researching
2. Design the study based on what you have learned (see 1)
3. Prepare your participants with concise, understandable instructions
4. Select the best dates for your study to run

Good luck with your diary study and happy researching!

This article is part of a series. We’ll give tips on what to do once your diary study is in motion in part two.

This was originally posted on my own website.