Limits of The Net Promoter Score (NPS) System
May 13, 2016
| Permalink
What is the Net Promoter Score?
The Net Promoter or Net
Promoter Score (NPS) is a management tool that can be used to gauge the loyalty
of a company’s customers. NPS was first introduced by Fred Reichheld in
his 2003 Harvard Business Review article "One Number You Need to
Grow"[1].
The NPS is based on a
single question:
"How likely are you to
recommend [product or company name]
to a friend or colleague?"
to a friend or colleague?"
Customers are asked to
rate their answers on an 11 point scale ranging from (0 = not at all likely to
recommend and 10 = extremely likely to recommend).
These scores are then
split into three categories “Detractors,” “Passives,” and “Promoters.”
- Score of 0 – 6: Detractors
- Score of 7 – 8: Passives
- Score of 9-10: Promoters
The "Net" in
Net Promoter Score comes from subtracting the percentage of detractors from the
percentage of promoters. A negative score means you have more detractors than
promoters and a positive score means there are more promoters (that is, more
positive word of mouth than negative word of mouth). An NPS can be as low as −100 (everybody is a detractor) or as high as +100
(everybody is a promoter). An NPS that is positive (i.e., higher than zero) is
felt to be good, and an NPS of +50 is considered excellent.
Criticisms of NPS
A key benefit of NPS
is that it provides a single score that is easy to understand. However, in our
view it has only limited practical application for companies who want to
monitor, understand and improve their overall customer experience as it
measures only one aspect of the overall experience - loyalty. Below you will
find some of the most common criticisms of the NPS.
1. NPS correlates with customer satisfaction but should not replace it
Customer loyalty and customer satisfaction
are related concepts and they are correlated (people usually provide similar
ratings for satisfaction and loyalty). At the same time customer experience
questions and satisfaction questions
are also correlated. However, where a measure is correlated with another
measure, that does not mean that one is a replacement for the other, which NPS
attempts to do.
It's unlikely to have customers that are
loyal but unsatisfied, but you can have satisfied customers who aren't
loyal. However, a key concept behind NPS
and loyalty measures in general is that customer satisfaction and experience
were excluded to focus only on loyalty.
In Fred Reichheld’s, the creater of NPS, book “The
Loyalty Effect” he makes the case that 60% to 80% of customers who defected or
didn't repurchase a product were in fact satisfied or very satisfied. In the
Auto Industry, for example, he cited figures which showed a 90% satisfaction
rate and yet, on average, only 40% of customers repurchased the same brand of
car. This is an important issue as satisfaction does not mean loyalty (see our previous blog posting about satisfaction Do you really need satisfied customers.
So, while satisfaction relates to loyalty,
they are not the same thing. Satisfaction measures the customer’s
experience at any given time whilst NPS measures long term attachment
(loyalty.) It is therefore best to measure both and not rely solely on one. Having
multiple measures requires minimal additional effort to record and analyse and
is significantly more valuable and actionable for you to monitor and manage –
this is our approach.
2. NPS is not actionable
NPS is also not
actionable. What if your company has a good score What does that mean? How can
you change the score? We simply don’t know the “So what?”. Similarly what if
you have a bad NPS? What do you do? Is it linked to a group of underperforming
or unprofitable gyms or a general company wide issue with the customer
experience?
If a metric is just an
“it is what it is” number, as NPS is, it has no context and no predictive power, can’t be used
alone, and doesn’t give you clues about what to do. Its usefulness must therefore
be questioned, especially if used in isolation.
At CJM Research we believe that you should measure
not only overall satisfaction and loyalty but measure Key Performance
Indicators of the customer experience too. This is then reported as a composite experience index score
made up of all the KPI scores. This approach has been reported as best practice
in other research[2] [3]
[4].
Combining the experience scores provides a
more robust and stable overall experience score that is directly linked to its
component KPI scores. This not only provides a truer overview of the customer
experience but also makes it easier to identify and act upon areas for
improvement and measure the impact of any changes or improvements to service.
For example, if you have a low overall
score for a part of your business you can directly link this to a particular KPI and
act to improve this specific aspect of service. With NPS you would know that a
part of your business has a lower score but not necessarily why. Our scoring is
designed to help management identify and act on the information to improve
service. This in turn makes our scoring easier to use for you as it is not only
a single experience score but one that is linked to the actual customer
experience and and this helps you improve overall satisfaction and loyalty for
customers in a faster and more targeted way.
Furthermore, by tracking specific aspects
of the customer experience it allows further detailed analysis that can tell
you which parts of the experience are most important in driving satisfaction
and loyalty. NPS on its own cannot do this.
3. NPS only works in the context of your wider industry
The Net Promoter score
has no real meaning without the context of competitor scores. Whilst a
score of +50 is considered excellent this actually differs significantly between
industries. For example, a utility company with an NPS of +1 may be excellent
in the context of an industry where the average NPS could be -20.
We have an example of
this where one of our other clients had an NPS of +69 which they thought was an
excellent score…until they conducted some competitor customer research and
found they were only average compared to their main competitors.
This is because you have to interpret NPS scores relative to other NPS scores in the same industry. However, most NPS centred research does not include measures of competitors leaving the results without context i.e. you do not know if your score is good, average or poor compared to your competitors.
This is because you have to interpret NPS scores relative to other NPS scores in the same industry. However, most NPS centred research does not include measures of competitors leaving the results without context i.e. you do not know if your score is good, average or poor compared to your competitors.
You also cannot
directly compare NPS across different industries or product categories because
some products simply don’t lend themselves to word-of-mouth (e.g., toilet paper
or car oil), while others can evoke passionate positive or negative views (e.g.,
retailers or restaurants).
4. The same NPS score can come from very different customer experiences
NPS is also problematic
because there are many possible ways to arrive at the same number, for
detractors, passive, and promoters respectively, you could achieve an NPS of
+30, for example, in dozens of ways.
A company with a +30
NPS could have 30% promoters, 70% passives, and 0%
detractors, while another company with a +30 NPS could 60% promoters,
10% passives, and 30% detractors. It is likely that a company with 30%
promoters and 0% percent detractors is very different than the one with a
polarized customer base with 60% promoters Vs 30% detractors — even though
their NPS is seemingly the same.
NPS also ignores the
fact that the voices and reach of promoters or detractors can be drastically
different depending on their motivation and the digital channels they have at
their disposal. There may be a small amount of very vocal detractors who go
online and write negative reviews. That would outweigh a large non-vocal group
of promoters, even if the company had a positive and high NPS.
5. 11 point scales are not as good as others
Research by Stanford
University and others[5].
found that the 11-point scale advocated by NPS has the lowest predictive
validity of the scales tested. We typically recommend a 10-point scale for
overall satisfaction and often mirror NPS scoring for recommendation (for consistency
with NPS rather than by preference). This provides a more detailed scoring
which is useful for tracking research.
For experience KPI’s
we suggest either a five or 10-point scale depending on our clients legacy
research and/or objectives. Five point scales are simpler to interpret and
easier for respondents to complete, especially on smart phones (50%+ of online
surveys are completed using phones) but smaller changes in average ratings are
more difficult to identify.
6. The scoring inflates the margin of error
By converting an 11
point scale into essentially a 2 point scale made up of detractors and
promoters (NPS ignores passives in the scoring), information is lost. Furthermore,
this new categorisation increases the margin of error around the net score
(promoters minus detractors). Unfortunately, this means that if you want to
show an improvement in Net Promoter Scores over time, it can take a much larger
sample size to calculate, otherwise the difference won't be distinguishable
from the acceptable range of error.
7. The relationship between NPS and growth remains unproven
One final, but important
criticism of NPS calls into question the strength and reliability of the link
between NPS and measures of business growth and profitability of those with
higher NPS. Research that has attempted to replicate the link between business
performance and NPS has not find statistically significant relationships between
the two[6].
Similarly, the relationship between growth and NPS is reported to be less
consistent or strong when using tracking data[7].
Even Fred Reicheld,
the inventor of NPS admits that the findings of his initial research for NPS
was flawed stating[8]:
"A number of perspicacious
readers have noted that the statistical evidence provided in my book The
Ultimate Question is imperfect. It does not provide proof of a causal
connection between NPS and growth. Nor are some of the timeframes ideal.”
We are not suggesting the NPS is a useless score but that it is not the one score you should only measure. Instead it has its place together with satisfaction and more detailed experience questions in a balanced customer experience research programme.
[2] For example: Hill,
Nigel; Roche, Greg; Allen, Rachel (2007). Customer Satisfaction: The customer
experience through the customer's eyes. London, England: Cogent Publishing. p.
7.
[3] Satisfaction as a
Predictor of Future Performance: A Replication. Jenny van Doorn , Peter S.H.
Leeflang, Marleen Tijs International Journal of Research in Marketing (Impact
Factor: 1.71). 12/2013
[4] Szwarc (2005)
“Researching Customer Satisfaction and Loyalty: How to Find Out What People
Really Think” Kogan Page.
[5] Schneider, Daniel;
Berent, Matt; Thomas, Randall; Krosnick, Jon (June 2008). "Measuring
Customer Satisfaction and Loyalty: Improving the 'Net-Promoter' Score"
[6] Lawrie, Jock, Matta, Alsono, Roberts, Ken
(2006). Value speaks louder than words. The management folly of adopting the
Net Promoter Score as the ‘one measure’
[7] Keiningham, Timothy L., Cooil, Bruce,
Aksoz, Lerzan, Andreassen, Tor Wallin, Weiner, Jay (2007). The value of
different customer satisfaction and loyalty metrics in predicting customer
retention, recommendation and share-of-wallet, in: Managing Service Quality, 17(4),
361-384.
Do you really need satisfied customers?
August 18, 2014
| Permalink
Do you really need satisfied customers?
Customer Satisfaction is a great measure of business performance and when combined with other measures such as recommendation demographics and reasons for the levels of satisfaction it can offer a real opportunity to understand your customers and improve your business.
It is well established that organisations with the higher levels of satisfaction have higher turnover, higher customer loyalty and larger market share than other organisations.
Why would you not want your customers to be satisfied?
Simply because measuring satisfaction alone is not enough. You really want to understand what proportion of your customers are highly satisfied (or totally satisfied or very satisfied) and why this is the case and then act to improve your service.
You see there is a difference between a satisfied and a very satisfied customer. To understand this difference we need to understand what satisfaction is. The Oxford English dictionary defines satisfaction as
“Fulfilment of one’s wishes, expectations, or needs, or the pleasure derived from this”
High satisfaction is about exceeding expectations
Essentially satisfaction is about having your expectations met: I wanted a widget and I got a widget = satisfaction. The difference between satisfaction and high satisfaction is that high satisfaction is about delivering service above and beyond your customers’ expectations.
High satisfaction is about the human experience with your brand
My experience of analysing satisfaction surveys suggests that this is essentially about personal interactions that create memorable and enjoyable experiences for customers. This might be as simple as a member of staff helping a customer or the experience in the store or online being fun and enjoyable, simple or memorable.
So high satisfaction is: I wanted a widget and I got a widget + memorable positive experience = high satisfaction.
Satisfied customers are simply having their expectations met but nothing more. This is a reason is why satisfied customers leave for competitors and why aiming for these “functionally satisfied” customers is not a strong enough measure, particularly in competitive markets.
This is why measuring customer satisfaction should not be simply about overall levels of satisfaction alone. That is only half the story. For example seeing 90% satisfaction does not explain the depth of that satisfaction. For example what if all 90% are just satisfied? Or are 85% highly satisfied?
The Harvard Business Review reported this in an article called “Why satisfied customers defect”:
“Except in a few rare instances, complete customer satisfaction is the key to securing customer loyalty and generating superior long-term financial performance. Most managers realise that the more competitive the market, the more important the level of customer satisfaction. What most do not realise, however, is just how important the level of customer satisfaction is in markets where competition is intense, such as hard and soft durables, business equipment, financial services, and retailing. In markets like these, there is a tremendous difference between the loyalty of merely satisfied and completely satisfied customers” http://hbr.org/1995/11/why-satisfied-customers-defect/ar/1
Measure depth of satisfaction and then understand what are the differences between these customers and less satisfied ones
I have noticed this in my own work, I was recently in a meeting where the client stated that they had questioned whether to continue their satisfaction survey because the results were not changing. I pointed out that they were only measuring overall satisfaction not the depth of satisfaction and that it was the way the results were being reported that had to change rather than the survey approach.
A good client of mine has seen only a small change in their satisfaction with an overall satisfaction level of 97% in 2014 up 5% over the past 8 years. However over the same period the proportion of very satisfied customers has risen 15%. So whilst overall satisfaction has remained fairly steady the proportion of highly satisfied customers has increased. Understanding what makes customers more satisfied has helped them concentrate on the levels of service that matter most to improving satisfaction – this is making a difference to their customer service and also to their levels of turnover.
What you should do?
So what should you do? First you should certainly look to measure customer satisfaction. Secondly you should measure depth of satisfaction and where possible reasons for the satisfaction. Third you should then understand what are the differences between customers with different levels of satisfaction. Finally you should act on these results to improve levels of customer satisfaction and the benefits to this brings such as improved loyalty, satisfaction, turnover and market share.
CJM Research can help you do this. contact us to start improving your business.
Customer Satisfaction is a great measure of business performance and when combined with other measures such as recommendation demographics and reasons for the levels of satisfaction it can offer a real opportunity to understand your customers and improve your business.
It is well established that organisations with the higher levels of satisfaction have higher turnover, higher customer loyalty and larger market share than other organisations.
Why would you not want your customers to be satisfied?
Simply because measuring satisfaction alone is not enough. You really want to understand what proportion of your customers are highly satisfied (or totally satisfied or very satisfied) and why this is the case and then act to improve your service.
You see there is a difference between a satisfied and a very satisfied customer. To understand this difference we need to understand what satisfaction is. The Oxford English dictionary defines satisfaction as
“Fulfilment of one’s wishes, expectations, or needs, or the pleasure derived from this”
High satisfaction is about exceeding expectations
Essentially satisfaction is about having your expectations met: I wanted a widget and I got a widget = satisfaction. The difference between satisfaction and high satisfaction is that high satisfaction is about delivering service above and beyond your customers’ expectations.
High satisfaction is about the human experience with your brand
My experience of analysing satisfaction surveys suggests that this is essentially about personal interactions that create memorable and enjoyable experiences for customers. This might be as simple as a member of staff helping a customer or the experience in the store or online being fun and enjoyable, simple or memorable.
So high satisfaction is: I wanted a widget and I got a widget + memorable positive experience = high satisfaction.
Satisfied customers are simply having their expectations met but nothing more. This is a reason is why satisfied customers leave for competitors and why aiming for these “functionally satisfied” customers is not a strong enough measure, particularly in competitive markets.
This is why measuring customer satisfaction should not be simply about overall levels of satisfaction alone. That is only half the story. For example seeing 90% satisfaction does not explain the depth of that satisfaction. For example what if all 90% are just satisfied? Or are 85% highly satisfied?
The Harvard Business Review reported this in an article called “Why satisfied customers defect”:
“Except in a few rare instances, complete customer satisfaction is the key to securing customer loyalty and generating superior long-term financial performance. Most managers realise that the more competitive the market, the more important the level of customer satisfaction. What most do not realise, however, is just how important the level of customer satisfaction is in markets where competition is intense, such as hard and soft durables, business equipment, financial services, and retailing. In markets like these, there is a tremendous difference between the loyalty of merely satisfied and completely satisfied customers” http://hbr.org/1995/11/why-satisfied-customers-defect/ar/1
Measure depth of satisfaction and then understand what are the differences between these customers and less satisfied ones
I have noticed this in my own work, I was recently in a meeting where the client stated that they had questioned whether to continue their satisfaction survey because the results were not changing. I pointed out that they were only measuring overall satisfaction not the depth of satisfaction and that it was the way the results were being reported that had to change rather than the survey approach.
A good client of mine has seen only a small change in their satisfaction with an overall satisfaction level of 97% in 2014 up 5% over the past 8 years. However over the same period the proportion of very satisfied customers has risen 15%. So whilst overall satisfaction has remained fairly steady the proportion of highly satisfied customers has increased. Understanding what makes customers more satisfied has helped them concentrate on the levels of service that matter most to improving satisfaction – this is making a difference to their customer service and also to their levels of turnover.
What you should do?
So what should you do? First you should certainly look to measure customer satisfaction. Secondly you should measure depth of satisfaction and where possible reasons for the satisfaction. Third you should then understand what are the differences between customers with different levels of satisfaction. Finally you should act on these results to improve levels of customer satisfaction and the benefits to this brings such as improved loyalty, satisfaction, turnover and market share.
CJM Research can help you do this. contact us to start improving your business.