Monday, June 27, 2022
HomeProduct Management3 Product Metric Pitfalls You Must Keep away from

3 Product Metric Pitfalls You Must Keep away from


In my ebook, The Insights Pushed Product Supervisor, I cowl why it’s vital to trace much less to create extra focus and spend extra time on extracting true insights out of your knowledge.

The following step is to make it possible for on the nitty-gritty degree, you might be monitoring what I name “good high quality metrics.” This put up—an excerpt from chapter 7 of my ebook—will concentrate on how you can enhance the general high quality of your metrics, how you can make them extra actionable, and what pitfalls to keep away from.

Pitfall #1: Self-importance metrics

A number of years in the past I used to be engaged on a B2B SaaS product for managing workplace environments. We simply launched the product and began to run our first paid promoting campaigns, so I arrange a dashboard that tracked the full variety of sign-ups over 30 days:

Vanity metrics

The numbers gave the impression to be going up, so we had been fairly pleased in regards to the momentum.

The issue was that whereas this graph regarded good on shows, the laborious fact was that solely 4% of the brand new sign-ups became precise conversions and income, and because of this, we didn’t meet our month-to-month recurring income targets.

It’s a cumulative chart, so the worst case can be that the graph would simply plateau if we don’t purchase any new customers, however the quantity can by no means lower. It’s a traditional instance of an arrogance metric:

  • Taking a look at this graph made us really feel good.
  • This metric was particularly helpful in stakeholder shows.
  • It didn’t give us any perception into whether or not we had been truly doing nicely, or not.
  • As a result of it didn’t give us any perception, it didn’t drive us to take motion to enhance the product or options in any manner.
  • And regardless of taking a look at this metric every day, it took us two months (by the point all of the 30-day trials had completed and churned) to seek out out that there was a difficulty.

Whereas one a part of the issue was our lack of awareness on how you can measure higher metrics at the moment, the issue in organizations usually lies loads deeper: most groups or stakeholders are merely not prepared to listen to the reality from their metrics, so we search for the numbers that make us look good.

In my interview with Crystal Widjaja, CPO at Kumu and author for Reforge, she summarized fantastically how you can view knowledge as a solution to capitalize on failures and drive enhancements as a substitute:

“When folks make errors (failed experiment, failed deployment, and many others), you’ve already paid that value. We must always consider knowledge as a solution to capitalize on errors and study from them. Relatively than ‘paying the tutoring of the failure’ and firing the person, use the info insights to inform us WHY it was a failure, study from it, and leverage it for the following iteration to be 10x higher than the primary.”

– Crystal Widjaja

To be able to get extra insights out of your knowledge you actually need to cease monitoring vainness metrics, and as a substitute use knowledge to uncover the reality and drive precise enhancements. Should you look intently, it’s fascinating how usually groups are exhibiting very selective metrics to appease sure stakeholders or make the numbers sound higher than they really are. Be careful for different traditional examples of vainness metrics resembling:

  • Variety of web page views or guests
  • Variety of followers/likes
  • Time spent on website (session size)
  • Variety of downloads

Metrics like web page views and session size are nonetheless closely utilized in web site analytics, the place the main focus is to measure visitors, consciousness and preliminary engagement. They offer you some perception on what we name the highest of the funnel—the preliminary acquisition of consumers—however not whether or not prospects are literally activating and fascinating with the product, which can have a way more significant correlation along with your wider product and enterprise targets.

The right way to do it higher: to actually perceive if a metric is sweet or unhealthy, we have to put numbers into context. On the very minimal, you need to attempt to examine a quantity over totally different time intervals, resembling evaluating your sign-up numbers this month versus the earlier month.

One other efficient solution to make your metrics extra helpful is to make use of ratios as a substitute of complete numbers. Ratios are inherently comparative. For example, accountants don’t simply have a look at complete income, however usually examine the prices of manufacturing a product with the gross sales they created from it. This manner accountants can observe their revenue margin (a fantastic instance of a helpful ratio) over time to evaluate whether or not the enterprise is wholesome.

Examples of higher, extra comparable metrics:

  • % of sign-ups per acquisition channel
  • % of sign-ups who accomplished the total sign-up course of
  • % of sign-ups who carried out a key activation metric
  • % of customers utilizing the product after 4 weeks

Pitfall #2: Solely monitoring lagging metrics

An enormous drawback was the period of time it took us to seek out out whether or not we had been hitting our conversion targets (or not). The product had a 30-day free trial and our purpose was to transform them to paying prospects after the top of the trial, so whereas the primary month regarded good by way of sign-ups, we might finally solely know by the top of the second month what number of of these sign-ups transformed to paying prospects.

It is a traditional instance of a lagging metric. Lagging metrics report retrospectively on previous outcomes. For instance, your income numbers for the yr are lagging metrics like most of your different operational metrics. You solely know whether or not you probably did nicely upon getting the outcomes.

The true worth in monitoring consumer habits by means of your product analytics is that you may begin to search for earlier indicators than having to attend to your closing income numbers. In case your main metrics don’t carry out nicely, you’ve got the prospect to course-correct earlier than it’s too late. That is why I designed the Holistic Metrics One Pager in chapter 5 of my ebook to incorporate each buyer habits and operational metrics, so groups can observe a wholesome mixture of main and lagging metrics to get the total image.

One of the vital highly effective main metrics is the activation metric. A great activation metric represents the share of consumers who take a key motion of organising or beginning to use the product. Many corporations have discovered that if customers do a sure motion inside their product throughout onboarding, they have a tendency to comprehend the true worth of the product which ends up in greater engagement additional down the road. Some name this activation step reaching the “aha second” of their product.

Listed below are some easy examples of main activation metrics:

  • Social community product: a traditional instance was Fb’s first activation metric of including a minimal of seven pals in 10 days.
  • Dashboard aggregation product: the worth proposition is to bundle a number of instruments into one view, so that you may discover that customers who add a minimal of two or three instruments throughout onboarding notice the total potential of the product.
  • Utility product: your worth proposition may be to simplify or digitize a activity resembling monitoring gross sales conversations in a CRM, so you possibly can observe the variety of customers who full their first buyer entry as shortly as attainable as an activation metric.
  • Consideration product: in case your product is centered round leisure and content material you may observe customers who consumed a certain quantity of content material within the first week of signing up .

Lagging metrics will not be inherently unhealthy, by the way in which. In truth, they’re a vital a part of reporting, particularly for measuring enterprise metrics resembling your monetary outcomes. Their benefit is that they signify the ultimate outcomes, the true details.

Main metrics then again usually embody some quantity of assumptions like the belief {that a} excessive quantity of chilly calls day-after-day will increase the variety of paying customers additional down the road. As you get extra knowledge it is best to take a look at whether or not these assumptions are literally true, however even then there may be nonetheless some uncertainty on whether or not the activation metric actually induced the rise in retention, or whether or not different elements contributed to it.

This implies main metrics won’t ever be as correct as lagging metrics, however they’re essential to getting true insights out of your metrics. They permit us to study from buyer habits and determine early indicators which will change our product choices to optimize for higher enterprise outcomes additional down the road. Utilizing the Holistic Metrics One Pager template forces you into monitoring each main and lagging indicators, in addition to to consider how these affect one another.

Pitfall #3: Metrics nobody understands

After I interview product managers I usually hear that analytics data and knowledge insights get hidden away in darkish mysterious corners of workplaces, with occasion names that nobody however a few extremely specialised analysts perceive. Each month these specialists would meet with numerous product groups in an try to share and translate a few of their findings.

If we wish our product groups and stakeholders to create a shared understanding of our knowledge and focus on enhancements to the product collaboratively, we have to actively work on democratizing our knowledge, be sure our metrics are accessible to everybody and straightforward to know.

Intercom shared their learnings from doing a large occasions clean-up a number of years in the past. They’d round 350 occasions for his or her product that sort of regarded like this:

Long events name

Does this look acquainted to you?

Intercom shared that they failed a key precept of analytics: they made little or no sense to anybody however the analytics crew. They redefined and rebuilt their total naming construction of all their occasions to introduce higher readability as a key step to democratize their product analytics knowledge.

It’s additionally vital to make reviews extra accessible for numerous stakeholders and groups within the group. Sadly, I usually see groups afraid of opening up their dashboards, as it might once more uncover the true engagement or acquisition numbers that will not look nice to stakeholders. To keep away from uncomfortable conversations or pesky questions, it’s usually simpler for groups to cover behind a veneer of complexity.

The right way to do it higher:

  • Step 1: Work along with your engineering groups and analysts to simplify your product analytics occasion names: “Accomplished Onboarding” and “Added Dashboard Widget” are actions that everybody will perceive.
  • Step 2: If in case you have an analytics crew, embody them higher in your product groups. The extra context the analysts have of what your product crew is engaged on, which experiments you’re testing and which questions have to be answered, the higher they may help you to dig into the info to seek out probably the most related insights. It ought to be a collaboration reasonably than an outsourcing method.
  • Step 3: Make your analytics dashboards and reviews accessible to the broader group. Your dashboards ought to replicate your product’s key metrics (which you’ll be able to outline utilizing the Holistic Metrics One Pager from the ebook). That is vital for scale (your crew doesn’t need to get flooded with handbook reporting requests day-after-day) in addition to to actually construct a extra data-driven tradition throughout the wider group.

“When groups are requested in regards to the state of the enterprise, they’ll both go look it up or make up hypothetical guesses. It’s vital to make the previous the best, default manner for management to reply to these requests by constructing customized, easy-to-use drill-down dashboards for issues like cohorts, funnels, and consumer occasions.”

-Crystal Widjaja

Keep in mind that the job we employed our knowledge to do is to uncover the reality so we are able to take motion and enhance our product experiences. Making your metrics straightforward to know and extra accessible are key steps to incorporate knowledge insights into the day-to-day resolution making in your group. A powerful product group ought to be extra motivated than ever to unravel these issues as soon as they know the place the issues lie.

The right way to enhance your metrics utilizing the metrics guidelines

I created a easy guidelines that summarizes key traits of fine high quality, actionable metrics that may make it easier to get extra insights out of your knowledge. Use this guidelines to evaluate and enhance all of your present metrics:

  • Is your metric uncovering the reality, and never an arrogance metric?
  • Is your metric comparative and does it provide you with a transparent concept of its efficiency? (If not, attempt ratios!)
  • Is your metric one of the best main indicator to reply your query?
  • Is your metric straightforward to know so others can rally round it?
  • Is your metric linked to the broader enterprise targets and might you articulate the influence?

It takes actual observe to actually get your key metrics proper, and you will see the satan usually lies within the particulars. It’s completely regular, and actually inspired to incessantly revisit the metrics you’ve got chosen, and to refine them a number of occasions to make them extra helpful.

Be careful for the pitfalls of sharing vainness metrics, focusing an excessive amount of on lagging indicators the place you haven’t any time to course-correct, and be sure to simplify and democratize your metrics to actually degree up the info maturity in your group.


Get started with product analytics





Supply hyperlink

RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments