When it comes to product management, there is no one golden metric. There’s not even a short list. The quantity of available data and the uniqueness of each product make it very difficult to suggest real apples-to-apples product comparisons at any scale. 

Whether you like to work from a pre-existing template or use brand new methods to report on outcomes, developing useful measurements of product success requires a heavy dose of do-it-yourself problem solving. This is nearly always true because of the situation-specific nature of products and features — there’s no one-size-fits-all way to track “usage” across all kinds of products. Something that the best product managers have in common is that they’re skilled at inventing new KPIs. At Sisense, our product team does an excellent job developing new measurements to find relevant product data and generate actionable insights. Here’s how we do it.

Understand the difference between macro-level metrics and micro-level metrics

To begin, not all KPI are created equally — it’s important to understand this when you’re arranging data to make product decisions. The click-through rate for your last email send is not on equal footing with your annual revenue. Click-through rate might be a good measure of how well one part of one campaign is performing, but revenue serves as a summary statistic to track how the company as a whole is performing. 

We like to sort data into two levels: micro and macro. The macro statistics (revenue, user count, retention/churn, conversion, etc.) are the ultimate outcomes we’re trying to achieve. They change slowly and are impacted by a wide range of variables. These are goals that the entire company should agree on and have visibility into. As a rule, macro metrics are relevant to multiple teams around the company and require efforts from all teams to achieve. 

The micros track the individual efforts of those teams toward the larger goal. Micro KPIs are more in the weeds. Micros are more granular and subject to change than the macros. However, the micros are the key to explaining movement in the macros. Knowing that revenue (a macro) is higher or lower than expected doesn’t mean anything unless you can point to a reason why. 

That’s why we have micros. Every team should have access to information about the micros that they are responsible for. For the list of macros above, hopefully you can imagine a long list of product micros from your team that would affect that metric in some way. 

The Real Cost of Embedded Analytics

Start with the macro metrics, then move to the micros

There’s a reason macro metrics have companywide visibility — they matter to every employee and every team. When you create a dashboard to track your product metrics, the macro metrics that are going to be affected by your product should be in the top row. 

The process to design a product metrics dashboard should mirror the broader process around product decisions. It should start with a conversation about what goals the company should hit then work backward to determine how the product can help get there. In the same way, the dashboard should address the questions of where you are in regards to the macros before it goes into how you got there.

Using data to find the “why”

To illustrate the two levels of metrics, I’ll share an example from a recent product launch at Sisense. Last year, we began offering support for Python and R in addition to SQL in our editor. A couple of the macros that I’m concerned with are adoption and revenue generation. So my team created a dashboard to track movement on those  metrics and dive into related micros.

At the top of that dashboard is a row with charts tracking:

  • Adoption (on a user and account level)
  • Total revenue attributable to the new language support
  • Revenue broken into business vs. upgrade over time

Underneath those top-line numbers, we also tracking a ton of micros to give us more details. Examples of those are: 

  • total charts created in R
  • total charts created in Python
  • code template usage, by use case
  • chart sources
  • top chart creators
  • user survival and percentage of users who keep engaging with the feature

All of those micros tell me something about why the big number is moving the way it is. On its own, the fact that our users access a specific template more often than others is not very valuable. But it tells me more information about which particular analyses existing users are running. I can use that information to help new users get more value out of our product and ultimately improve adoption. 

I check the macro numbers every day, but I’m not making new product decisions based on those. Those numbers have goals attached to them and I’m focused on the team’s progress. I want to see adoption and revenue go up, but more interesting to me is the reason why. Every time I check the dashboard, I look at that second level of micro charts to put together a data-driven hypothesis that explains macro movement.

Find out more tips to become a data-driven product manager in our Data-Driven Product Management guide.

Scott Castle is the VP of Strategy at Sisense. He brings over 25 years of experience in software development and product management at leading technology companies including Adobe, Electric Cloud, and FileNet. Scott is a prolific writer and speaker on all things data, appearing at events like the Gartner Enterprise Data Conference, Data Champions, and Strata Data NYC.