Can you count on what you measure?

My inbox is full of meaningless metrics.

• Grammarly insights proudly tells me ‘You’re building quite the voluminous vocabulary. You used more unique words than 95% of Grammarly users.’

• LinkedIn tricks me to click by letting me know ‘You’re getting noticed. You appeared in NNN searches this week.’

• Strava tells me I took a King of the Mountains (KOM) section accidentally without any intention on a hill that would be considered unimpressive by most average cyclists.

• Medium gives me the alliterative ‘stats for your stories’ on a weekly basis.

All meaningless metrics. At least meaningless to me. None of the above examples provide me with insights on anything I’m looking to understand or take action on. These numerical nuggets lack personal context – are the numbers good? They focus on easy-to-gather quantities and neglect any measure of quality.

Numbers somehow feel important. In business, decision-making figures are wielded as immutable facts. They become hard to argue against. Numbers are taken by many people as both powerful and persuasive.

What gets measured gets done. You are what you measure. Measurement eliminates argument. If you work in an environment that puts store in these oft-quoted business adages then I urge you to take a moment to challenge your calculations. Let’s review our metrics to ensure they can stand up and be counted.

Here are four questions to ask to see if what you measure adds up.

1. Why is this metric important to you?

Somebody clever once said ‘measure what you value, don’t value what you measure’.

If you start with the tools you are in danger of embarking on a fool's errand placing importance on what the tools can easily measure. Google Analytics (other tools are available*) presents you with a wealth of data but how much of it do you—or should you—care about?

Average time on page; bounce rates; sessions with search; page depth etc. Which of these are important for you to know? And for each metric, what number is a sign of success?

If you create or receive regular data reports, periodically ask what changes have you made with the figures you capture. If you don’t make decisions or take actions informed by any particular metrics then it’s adding clutter rather than clarity.

*Why not turn the beast on itself and google “alternatives to Google Analytics”. I hear positive reports things about privacy-first products such as Fathom, Simple analytics and Matamo.

2. How can we balance this metric?

In striving to change a data point there can be an unintentional yet detrimental impact on something else you care about.

One way to counter this is to always pair your data points. State your primary goal, usually a positive increase or change in a number. Then balance it with a secondary goal that focuses on maintaining a different number you measure. For example, “improve customer satisfaction with a product whilst not adding to customer support costs”, or “reduce the churn rate for subscriptions by N% without offering additional promotions”.

The secondary goal is there to prevent over-optimisation of the primary goal.

3. How could we game this metric?

How could you get false readings, however short-term the blip, in order to meet your key performance indicators (KPIs)?

This is a fun question to ask in a meeting or workshop. Ask everyone to temporarily embrace their Machiavellian side to work out how the targets you’ve been set could be achieved in creative or underhand ways.

4. What change are we looking for?

I often see dashboards and business reports that provide a snapshot in time divorced from timelines and trends. Figures, percentage points, graphs and charts that look like meaningful data but don’t tell the wider story.

In setting any metric it’s important to benchmark where you are, where you want to get to, and by when. This information prevents panic and helps track progress.

For your key indicators decide and document when to take measurements, what numbers will be a cause for concern, and what numbers will trigger a celebration.

Stating this upfront gives you a framework to understand and act on the data you subsequently get.

Please measure responsibly

Goodhart’s law, named after the British economist, states “When a measure becomes a target, it ceases to be a good measure”.

Numbers aren’t intrinsically good or bad. They’re just indicators to help you understand a situation and take a sensible course of action. They aren’t written in stone to be slavishly followed forever.

A good set of meaningful metrics should be personal to your situation. The numbers you track should be one of many inputs, both quantitative and qualitative. What you measure will benefit from regular review and should be changed if the measurements no longer help you chart a course into your desired future.

Numbers are important in business. Let’s make sure our numbers count.