Our friend Jared Spool tweeted:
Dashboards are often what customers ask for.
They are rarely what customers need.
If you’re building a dashboard, it’s likely your user research wasn’t finished.
We're a curious bunch here at Clearleft, always encouraged to 'Ask the question' so we thought this was perfect debating ground. But before that, we needed to get our definitions straight. What exactly is a dashboard?
Chris put forward his definition. A dashboard, he said, comprises three things:
- A visual display of data that
- monitors conditions and
- informs actions.
That last point is where Chris sees most dashboards falling down. They’re showing data but the data they show is whatever’s easiest to gather. Think of those displays in the reception area of a company’s lobby. They give the impression of a data-driven company because there’s a chart with numbers moving. But those numbers could be anything. Some beautifully generated digital art would do the same job.
Richard pointed out that the audience matters too. For instance, internally at Clearleft, we've created a new company dashboard (as part of growing transparency following becoming employee owned). The leadership team use this for forecasting and decision-making. But the same dashboard can be used to give everyone in the company a general sense of where we're at. He warned to ale always ask 'why' we're providing that information? And what do we expect our teams to do with it?
Andy agreed that you need to be strategic about what to include on a dashboard so that it informs some intelligent decision-making. But don’t expect a dashboard to pre-emptively suggest solutions for you. You need to apply a heavy dose of interpretation.
So if a number is going up or down, do you have enough information to judge whether that’s a long-term trend or a short-term blip?
Remember there is a fundamental difference between data and metrics. Data only becomes a metric when you’ve got something to judge that number by. Dashboards can help, but you need to be aware of our bias towards pattern-matching.
Jeremy agreed that dashboards can be a useful guide, but he warns against mistaking the map for the territory. When it comes to metrics, he said, you need to consider the McNamara fallacy. Reducing success to a measurement can backfire. You end up giving too much weight to what can be measured and not enough weight to things that are hard to measure. There’s Goodhart’s law too:
When a measure becomes a target, it ceases to be a good measure.
Done well, there's no doubt a dashboard can help you fight through the noise to get a really clear synthesis of information.
When we mentioned in our newsletter that we were discussing dashboards, our friend Brian Suda—informatician and author of Designing With Data—chimed in with his thoughts:
I think dashboards are great, it is just that most dashboards fail on their purpose.
A car’s dashboard are not vanity metrics; most corporate ones are. Most dashboards you see in any office will have either vanity metrics, or some metric which is not actually actionable.
Katie warned that in her experience designing business tools there's a risk to mitigate, that the longer they're around the more they are abused and everything begins to lose meaning. The sweet spot for her is designing for power users, customisable dashboards that users use every day. They work well for those users who need operational oversight.
Perhaps we’re all in agreement here. Most dashboards aren’t good. But that doesn’t mean that all dashboards are bad. Or, as Katie loves to say ...it depends.
