Quantification: an essential PM skill
As a PM being able to drive outcomes is largely down to quantifying, measuring, sizing, tracking impact whilst calibrating / reiterating & this is how you ought to be going about it:
Actionability
As the way of the world (or the “product world” to be precise) the phrase “actionable” seems to have crept deeper into the lives of all PMs, be it over “making things more actionable”, “deriving actionable insights”, “providing actionable inputs”, “making action plans” so as to align all internal teams and stakeholders onto an outcome that is hot for the moment.
Although it is possible that most PMs talk of being more actionable, it is true that some are subconsciously driven towards thinking that way as it seems embedded deep into their workflows and from a long time too, making it more of a mindset.
But can one really be driven towards “action” sans quantifying anything?
Also, what does it really mean to quantify something?
Would it suffice to quote the current position and where it all stands?
Consider this graph for an arbitrary example:
So, that is 115 sign-ups & 92 active users for that given month.
Generically, that (80%) could be seen as a great conversion rate when pitted against the stats of many products across the world, but it still doesn’t convey enough about the performance of the product and it may be too early to talk about retention rate / user adoption rate at this stage.
But take a look at this graph now:
Comparing the MAU and the no. of sign-ups you see how the conversion rate manages to stay on flat for the period in question exhibiting no great rise or fall.
Now, there could be quite some insight derived here.
For one, there is an evident problem alright w.r.t growth although it may be too early to narrow in on the causations as it could still point to a wide array of possibilities really, ranging from:
lack of strategic initiatives
sub-par marketing efforts
very low - zero value addition
lack of positioning
lack of a good user onboarding strategy
although the conversion rate still seems to paint a full-on rosy picture.
“Getting to actionable insight is literally down to dealing & conversing in numbers, rendering it over a certain degree of simplicity so as to facilitate some great decision making”.
Principles of Quantification
There are no second thoughts about the dependency decision making has had on data. But that being said, the adherence to collate a really diverse and vast degree of data points complicating the entire process is such a common sight across many teams and organizations. It may amaze you how some really critical and complex decision making could be achieved with simple data points.
“What may look like really complex decision making on the outset could be rendered ridiculously simple if the underlying data itself is made simple”
Although it is possible that there could be raw data across data lakes the quantification, segregation of data to pick so as to build a correlation to the goal / end result by defining the outcome and what success ought to look like does make for a great starting point, always!
This recent bit of my work here just stresses on that very fact:
→ https://mgmtinc.substack.com/p/journey-from-data-to-insight-part
Quantification & “SAM” principle
“Quantification is to simply be able to represent something mathematically or just count / quantify which could literally be down to 3 basic principles viz., simple, actionable & measurable”
Let’s build some context, explore & attempt to understand a few constrained situations better.
CASE STUDY:
ACME corp. is thinking of launching a mobile app that would help users who are seeking legal advice by patching them over with a legal expert who is willing to stand in & play the role of an advisor given the underlying scenario.
So, to be able to answer the question:
“What does the engagement rate on the App look like?”
we could get to it in a jiffy applying the SAM principle:
But in reality and especially over some tricky situations that could be easier said than done. The problem may not be with the quantitative accuracy (although that could also be a subject worthy of a debate), but to do with the availability and the collation of data itself which may get very tricky.
“You can measure only what can be measured”
So, let’s try and simulate this near impossible scenario here by extending the same case study.
Let’s say an user gets introduced to an advisor on the App.
Once the two people who have their wants clearly segregated meet on the platform they could exchange contact information and possibly meet F2F and further the entire thing over a tête-à-tête, which the App obviously won’t have any control over.
If the creators of the App want to measure the relevance and the effectiveness of the match, they may have to resort to micro-surveys (a short & highly subjective survey with about 3-5 questions) or the classic C-SAT or NPS which obviously are going to be lagging indicators alright.
Supposing the environment is feasible for data collection devoid any constraints as mentioned above, here are a set of 8 checkboxes you ought to tick-off to be able to quantify better.
Data sufficiency
✔ What are the data points that ought to be measured?
✔ Would measuring them suffice & unambiguously determine the outcome?
✔ What would the success factors look like over this exercise?
Probability of success
✔Assuming data availability, sufficiency & measuring whatever is needed,
what’s the probability that it would all lead to the desired success levels?
✔Are the indicators strong enough & supported by enough logic & rationale?
And if yes, how strong are they?
Metrics & proxies
✔ What metrics represent the success factors individually or collectively?
✔ Do those metrics successfully add up to help determine the outcome?
✔ Are proxy metrics needed given how vanity metrics don’t suffice?
Wider Application / Scope
As you step into the shoes of a product manager its only but a natural expectation to be able to quantify things and the beauty of the role lies in how one is pushed to think in that direction even if not naturally inclined.
When there’s no doubt that there are countless instances where quantification applies directly to the job role stemming from the responsibility a PM plays irrespective of hierarchy there is a need to be able to use quantification all-round over official communication, to enlist achievements over bullet points and present it across to leadership or C-suite making it more actionable and facilitating accurate decision making.
Let’s explore this crucial trait delving into some depth so as to help you PMs get better over your workflow and in doing so we’d revisit the same “case study of ACME Corp.” as relevant.
Product management & chops
Outcomes
Just how a business can’t survive for long sans high-level objectives, there can’t be a successful product sans the outcomes.
For ACME Corp. a list of successful outcomes could look like:
users & advisors enlist themselves positioned as seekers & providers
users are able to find advisors seamlessly defining their criteria as applicable
advisors get hit up with queries relevant to their niche (domain / subdomain of expertise)
OKRs
Once there is substantial clarity over the outcomes its only but natural that the leadership get down to formulating those objectives and key results that help determine their classification in the green zone helping them align to outcomes.
For ACME Corp. an OKR could look like this:
OBJECTIVE: Improve Customer Retention by 20% QoQ
KEY RESULT 1: Improve classification by adding micro-categories covering as many niches as possible
KEY RESULT 2: Increase the no. of advisors enlisted by 20% focusing on some niche marketing fronting them with benefits as relevant
KEY RESULT 3: Reduce bounce rate by 30% QoQ
KEY RESULT 4: Improve NPS by 20% QoQ
Metrics & KPIs
With the clarity over objectives and key results, the metrics ought to help identify what to measure & the KPIs ought to ensure that the numbers stay above a certain prescribed level so as to hit the growth levels so desired.
For ACME Corp. a few crucial Metrics & KPIs could be:
Research
Quantitative data
There are tons of numerical data that often forms the basis of any decision in any type of research conducted over the product workflow and it is imperative one collates all of it before taking as much as a step ahead over finalizing anything. Although exact data points vary, some of these ought to be captured:
Percentage of the market who correlate to it & can help establish pain points / the problem statement
Market segmentation
Demographic / ethnographic studies
Competitor analysis (products / features, market share, years of operation)
Detailed results obtained via primary research (market scoping)
Survey results (Likert scale, Rating, Straw-manning & Steel-manning, Max Difference Scaling, Conjoint analysis etc.)
Current acceptance levels existing & reaffirmed over usability testing (or moderated / unmoderated testing / interviews as applicable)
Contextual probes pertaining to highly detailed UI / UX elements
NOTE: When there could be no real bar representing a prescribed quantum of market research activities needed, it is about conducting just the desired ones so as to get to some state of semblance in supporting the strategy and emphasizing strategic direction.
Impact sizing
Over building products or features it is normal to expect the needle to move over a certain metric that indicates product adoption seamlessly. This is crucial and more often than not, a precursor to estimate this number, which is to depict the expected coverage of the market (often represented as a percentage over the total market size taken as a base).
To achieve this one could get to down any of these methods:
pinning on secondary research or buying research data (if that’s viable)
tabulating values obtained over previous research experiments & market related studies conducted
brainstorming over stakeholders representing market-fronted teams and taking a consensus
extrapolate or recalibrate percentages to represent current market conditions
Market sizing
The TAM (Total Addressable Market) is a crucial aspect and a straightforward one when it comes to the workflow of product which denotes the estimated no. of users the idea would appeal to in terms of fitting in as relevant and being a burning need. Determining this is obviously a precursor to taking any idea major which could be an entire product or minor which could be a small feature ahead.
To arrive at this figure there are pretty obvious steps to follow:
set aside a target market for a given product or feature in question
enlist the TUG (targeted user groups) & their segments
work on the data by applying the selection criterion as applicable
arrive at demographics and sort them according to coverage & priority
use secondary research channels to obtain these – MAX, MIN, AVG
arrive at a ball-park figure (averages / weighted averages as applicable)
recheck if estimations tally with current market conditions
use relevant extrapolation methods as needed
recalibrate
Remember:
It’s often easy to get drowned out by the humungous nature of data & sometimes it is possible that one may not have a choice over the complexity, but if you want to get to a state of stability & control over that decision making you ought to start off by working your way with quantification and the “SAM” principle could elegantly help you with just that.