This article is about Usage. It's a fairly long article but reading it helps you understand the basics of how usage and calculated metrics works in Planhat.

Three types of data

For the typical SaaS B2B, Product Usage is one of the main pillars in customer success. The usage module in Planhat is easy to get started with, yet powerful and flexible.

Usage is all about data and you have three different types of data sources.
You may want to leverage only one, two or all of them.

These are

  1. User Activities
  2. Account Metrics
  3. Soft Metrics

1. User Activities

Activities are tracked per user, and typically you'll want to capture at least 5-10 main action that your users can take.

It's likely you'll want to track activities from different modules of your product which will help you
understand which parts of you product is being used to help your drive product adoption.

While user activities are tracked, and can be followed up per user, they're also automatically aggregated on account level, so you can see how each of your customers are using your product over all.

This data can be collected either by adding our (Planhat's) own Tracking Script to your product, just like you would with Google Analytics or similar.

Another alternative is to send these event straight to our API, which generally has some advantages over using a script.

Last option is to simply enable Planhat in your Segment.com account if you're a Segment customer. See here

2. Account Metrics

In most cases, just looking at user activity will be a good start but it won't take you all the way.

A number of important metrics can only be tracked on account level, since they do not directly result from a specific user taking some action in your product.

Examples could be "Megabyte of storage used", "Number of user accounts", "Total number of projects" etc.

Generally this data will be pulled from your own system and sent to Planhat, once a day or with some other frequency.

There's a very simple API to send this data so getting it over is generally a breeze.

3. Soft Metrics

This could be things stipulated in the license agreement, or a usage target agreed with the customer during onboarding.

For example, the contract with a specific customer may say they're allowed to use have 10 seats (user accounts). Or you may have set a common goal that the customer should Login 500 times during first year (a simple/silly example but goes to show the idea).

We call them soft metrics because they're generally not saved on any computer but exist as verbal agreements or in some contract, which mean data is manually added to Planhat when/if it changes which tends to be rarely.

For most CSMs, this data is either not available at all, or it's a available but in tools like Mixpanel, Heap, Tableau etc. which are great for data analysis and product development teams but too complicated and inaccessible for CSMs to use in practice.

So rather than being an alternative to these other analytics and BI tools, Planhat offers a similar functionality but simplified (though in some cases stronger) and design with the Customer Success team in mind!

Read on to learn why..

Calculated Metrics - Concept

One thing we hear often from our customers is that the Success Team need a list of customers (just as an example) "in the enterprise segment that on average have not used module X for more than Y days"

To get this information, the dev/tech team typically would run some database query and pull this list for the CS-team. This works great for most companies but then the next week you want some other list and since your developers have a lot of other things to do, these questions from the CS team become a problem.

In Planhat, we've solved this scaling challenge for Customer Success teams by building a module we call "Calculated Metrics"

Calculated metrics let you convert and combine a few raw metrics into new more complex metrics - as an example, instead of asking your tech team to send you:

  • "average number of logins per week",
  • "average number of logins per month",
  • "total number of logins past 14 days",
  • and so on..

Simply have your tech team send "number of logins per day" to Planhat, and then you can do the rest yourself in Planhat;

You can look at the average, min, max, sum, trend etc.. only based on that single raw metrics you send in.

Add a few more raw metrics and possibilities become endless.

The true power comes from combining the different data sources. This is perhaps best explained by an example:

Say one of your customers is on a subscription plan including 10 seats.

You add this level to the customer profile in Planhat (Soft Metric).

The you send in an automatic daily metric "total users" as defined in your own system, to track how many seats (user accounts) they really have.

Now you can create a Calculated metric in Planhat where you compare, for each customer, the agreed number of users with the actual!

Some of your customer may be using only 10% or 50% of the seats they bought - which would be signal you may see a downgrade later, while others may be at more than 100% which would be a clear upgrade opportunity.

Now use this new calculated metric in your Planhat health score, or perhaps set an alert to notify you whenever a customer climbs above 100% utilization!
 
And obviously, you're not limited to only comparing two metrics.

If you're into it, you could set up deeply nested formulas combining any number of other metrics to truly get a clear picture of the value your customers are getting out of your product.

Calculated Metrics - examples

The formulas to create calculated metrics in Planhat give you a lot of freedom and flexibility. We also appreciate that they can feel a bit nerdy/confusing to some users so here we'll provide you with some examples as inspiration.

In all the examples above we'll use the metrics "daily logins", of course this is just an example and the same would apply to any other metric or combination of metrics.

Note: calculated metric cannot be based on other calculated metrics, however anything you could have achieved by combining calculated metrics is also possible to achieve with a single calculated metric.

Example A: Number of daily logins.

So, let's assume you're tracking "login" as a user activity. In the Activity section of the usage module you'll then see logins day by day. If this is exactly what you want, you could simply make a calculated metric (if you need it for triggers, health etc) to track this original metric.

The formula to do it is:
{"type": "metricOverTime", "days": 1, "op": "LAST", "prop": "activities.login"}

The LAST "operator" will only consider the last data point/day with data of the original metric, so the number of days doesn't matter here, you could set it to 1, 30 or 100 and it would give the same result, assuming you have daily data.

Most likely, this is not what you want though. Since daily logins will fluctuate a lot, perhaps even drop to zero on a public holiday etc, reacting to daily changes is not suitable for most SaaS B2B.

Instead you might want to look a longer time period - for example, let's look at the weekly average, which should give you a much smoother curve.

There are two, slightly different ways to get this average number of daily logins depending on what you are after..

Say it's Monday and data for past 7 days data looks like this:

Sunday: 0 (logins)
Saturday: 0
Friday: 10
Thursday: 12
Wednesday: 8
Tuesday: 9
Monday: 10

The most common way of getting the average is taking the sum of all activities for these days, and divide it by the number of days.

Heres's what the formula looks like:

["DIVISION", {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}, {"type": "rawNumber", "value": 7}]

Given the data above you'd get (10 + 12 + 8 + 9 + 10) / 7 = 7

So on average 7 logins per day.

In some cases you may only want to include the days where you actually had some data in the average.

Typically this is not the case with User Activities but may well be for your account level metrics.

In this case you could use the "AVG" operator and the formula would be:

{"type": "metricOverTime", "days": 7, "op": "AVG", "prop": "activities.login"}

The result in this case would be (10 + 12 + 8 + 9 + 10) / 5 = 9.8.

Again this is just to show the concept of the AVG operator, in practice you will have (and will want to include) the zeros for Sat and Sun.

Example B: Number of daily logins vs Target

Now number of logins typically won't say a lot. You'll have some larger customers with hundreds of daily logins and some smaller with only a few daily logins. But that doesn't mean the smaller customer is doing bad, so you'll want to track Logins as compared to some expected level.

Let's say you have "expected number of weekly logins" from your customer dialog as a custom data point on your profiles in Planhat, then here's how you'd get the ratio:

["DIVISION", {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}, {"type": "propertyValue", "prop": "company.custom.expected number of logins"}]

Assuming the "expected weekly logins" is 100 for some customer and the actual logins past 7 days were 50, then you'd get a value of 0.5, which likely would vary from day to day.

If you'd rather think of it as a percentage, you could just multiply by 100:

["MULTIPLICATION", ["DIVISION", {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}, {"type": "propertyValue", "prop": "company.custom.expected number of logins"}], {"type": "rawNumber", "value": 100}]

Now instead of 0.5 you'd get a value of 50.

Example C: Number of daily logins - Volatile Trend

Perhaps you don't have an agreed expected level to compare with, then you may turn to historical data for the given customer so you can compare the number of daily logins with the same number exactly one week ago;

{"type": "metricOverTime", "days": 7, "op": "TREND", "prop": "activities.login"}

Most metrics will fluctuate a lot from day to day, and so will this metrics, though if you keep the number of days to an even multiple of 7, you'll at least be comparing the same day of the week.


Example D: Number of daily logins - Medium Volatile Trend

If the volatile trend if too much for you (and it typically is) then rather than comparing specific days, you may want to compare the past month with the previous, or this week with the previous.

Here's how you would do it.

1. Get the value for this week

{"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}

2) Get the value for previous week, by summing past 2 weeks and then subtracting the current week.

["SUBTRACTION", {"type": "metricOverTime", "days": 14, "op": "SUM", "prop": "activities.login"}, {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}]

3) Divide the two numbers.

["DIVISION", {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}, ["SUBTRACTION", {"type": "metricOverTime", "days": 14, "op": "SUM", "prop": "activities.login"}, {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}]]


Example E: Number of daily logins - Smooth Trend

If even example D is be too volatile for you, then you can get get an even smoother curve by comparing two overlapping averages, for example compare past week to the average over the past 10 weeks.

Here's how you would do that:

1) Get the value for this week

{"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}

2) Get the value for past 10 weeks

["DIVISION", {"type": "metricOverTime", "days": 70, "op": "SUM", "prop": "activities.login"}, {"type": "rawNumber", "value": 10}]

3) Divide the two numbers.

["DIVISION", {"type": "metricOverTime", "days": 7, "op": "SUM", "prop": "activities.login"}, ["DIVISION", {"type": "metricOverTime", "days": 70, "op": "SUM", "prop": "activities.login"}, {"type": "rawNumber", "value": 10}]]

These are just a few examples to get you inspired. there's plenty more to play around with, like Log functions in case you have exponential metrics.

We're more than happy to help you get this set up so please reach out to discuss your specific case!

Did this answer your question?