Fitness For Purpose Score

Regular followers of my work will know that I have expressed dissatisfaction with Net Promoter Score (NPS). Steve Denning in his book Radical Management suggested NPS was “the only metric you’ll ever need.” Steve is a writer for Forbes, an investment magazine. High NPS scores correlate with high stock prices and hence from an investor’s point of view NPS is an important metric. If you are a CEO of a public company, who receives a large portion of your salary as bonuses based on changes in the stock price then NPS is an important metric. However, many of my clients who collect NPS data report to me that it isn’t an actionable metric. NPS merely tells you whether you are winning or losing. It doesn’t tell you what to do!

There are some antidotes to NPS’ failings. The second question asking reviewers to “tell us why you gave the rating in the previous question?” provides the opportunity for short narratives. These micro-narratives can be clustered using a tool such as Sensemaker and useful information can be extracted. There may be actionable information hidden in the clustering of narratives. This advanced use of NPS information is very much still in its infancy and not readily available to many or most businesses.
I’ve decided to introduce a new metric into our own surveys. I call this Fitness For Purpose Score. I am hopeful this will become a key strategic planning tool in Enterprise Services Planning.

Fitness For Purpose Score

It is often true that businesses do not know the purpose with which a customer consumes their product or service. A product or service designed for a specific purpose may get used for something else. Some of the more famous examples are washing machines used to make lassi yogurt drinks for Indian restaurants. In evolutionary science, this is known as an exaptation: where something designed for one purpose is adapted for use with another purpose. To have actionable metrics for product or service delivery improvement, you need to understand the customer’s purpose for consuming your offering. When you understand this purpose, you can create the appropriate fitness criteria metrics. With Enterprise Services Planning (and Kanban) we use fitness criteria metrics to drive improvements. Fitness criteria metrics are used at all levels to compare capability with expectations. Fitness for Purpose Score is intended to help us understand purpose and whether or not our current capability meets expectations. If it doesn’t we can probe for thresholds to establish new fitness criteria metrics.

This is how our sales and marketing team will be using Fitness For Purpose Score in our own surveys in 2016.
Question 1: What was your purpose [in attending our training class? What did you hope to learn, take away, or do differently after the class?]
Question 2: Please indicate how “fit for purpose” you found [this class]?

5. Extremely – I got everything I needed and more
6. Highly – I got everything I needed
7. Mostly – I got most of what I needed but some of my needs were not met
8. Partially – some of my purpose was met but significant & important elements were missing
9. Slightly – I took some value from it but most of what I was looking for was missing
10. Not at all – I got nothing useful

Question 3: Please state specifically why you gave your rating for question 2
Questions 1 and 3 specifically ask for short narrative answers. These micro-narratives can be clustered. Question 1 will provide clusters of purpose which can be validated against our existing market segmentation and may reveal new segments, while question 3 will provide clusters of actionable information for improvements and possibly new fitness criteria metrics or threshold value for existing metrics. We can decide whether or not to pursue specific clusters and whether we are likely to be able to achieve adequate fitness levels to satisfy our customers during our Strategy Review meeting.
For example, our own product is management training, though we also have an event planning and publishing business. We position and sell our intellectual property as management training and we deliver it as training classes and mentoring. We know that a significant segment exists for software process improvement and for process engineers and coaches who consume our products and services in order to help them in their coaching practice. We know this segment exists but we specifically and intentionally don’t cater to it. We feel it would be a strategic distraction and undermine our overall message that managers need to be accountable, to take responsibility, to make better decisions and to take action where and when necessary to improve service delivery. The return-on-investment in our products and services is realized when existing managers change their behavior as a result of our training. And hence, while we appreciate the patronage of process engineers and coaches, we do not specifically cater for their needs.

Net Fitness Score

I purposefully moved away from the NPS use of an eleven point numerical scale [0 thru 10]. My background in human factors, psychology and user experience design, taught me that humans have problems with categorization beyond 6 categories without a specific taxonomy to guide them. This isn’t a result of Miller’s “Magic Number 7” rather the work of Bousfield W.A. & A.K, and Cohen, B.H. between 1952 and 1966 on clustering. For example, if you ask humans to rate something 1 to 10 they will struggle to create 10 distinct categories in their mind. When asked to devise their own taxonomy, or clusters, as lay people to the domain, they will tend to create no more than 6 categories. Hence, a scale of 0 through 5 is most appropriate for general consumption. I believe the NPS people tried this but discovered that in some cultures, such as Finland, people never give the top score on principle. They always choose one below the best. Hence, the NPS reaction to this was to double the scale using 0 through 10 so that people could give a 9 when they are really giving a 4.5. My feeling on this is that it highlights the issues with numerical scales and undeclared taxonomies. The solution of doubling the scale, however, creates a randomness in the system and generates noise in the data reducing the signal strength, because of the general human issue of modeling categories against the scale. Fixing one problem, the cultural propensity never to give top ranking, creates another problem, a cognitive issue in the general population to struggle with more than 6 undeclared categories. Hence, to avoid both problems, I am declaring the categories with narrative.

Scores of 4 and 5 are intended to indicate that someone is satisfied and the product or service was fit for their purpose.
Score of 3 is intended to indicate a neutral person. They didn’t get everything they needed to be delighted with the service but they got something acceptable for their investment in time and money.

Scores of 2 or below are intended to show dissatisfied customers who felt their purpose was unfulfilled by the product or service. This may be because the product is poor or it may be because the purpose was previously unknown or represents a segment that the business has strategically decided to ignore. Not all dissatisfied customers need to be serviced fully and satisfied: some customers, you simply don’t want – they represent segments you aren’t interested in pursuing.

NFS can be improved through better marketing communications that direct the right audience to your business and dissuade the wrong audience. So NFS can be used to drive excellence in marketing as well as used to explore new segments and the fitness criteria metrics that light them up as viable and profitable businesses.