Over the years, Net Promoter Score (NPS) has become the default question to measure and maximise value. But is it right? Is it true? Daniel Kinal joined us to share his thoughts.
Where did NPS come from? Back in 2003, Fred Reichheld introduced the concept to the world. He felt the current measures of loyalty were too convoluted and complicated. So he did his own study, with surprising results, even to him. What he came up with, Net Promoter Score – the one metric that was supposed to have the strongest correlation to company success.
Why should it work? The more likely you are to advocate for a brand, the more people will be willing to trial the product, therefore reducing your acquisition costs. Also, those advocates are more likely to be repeat customers and increasing their lifetime customer value. Score. Double score!
Simply irrelevant in some industries
Not predictive in a monopoly or near-monopoly conditions
Data analysed was historical, not future
Unconvincing replication studies.
Highly volatile measure
Obscures critical information
Is there a correlation? Well, yes. Is it good as a predictor for future success? Well, maybe not as much. In fact, in one study, NPS only explained 38% of future growth.
Bastardisation If you game your scores, what do you really achieve? From colour coding, nudging your scores, and filtering out negative results. What are you actually able to learn?
Is there an upside? Yes, some compelling aspects of NPS include, it is relatively cheap, fast, simple; and well accepted.
Already using NPS? Make the most of your data.
Don’t focus purely on the number.
Measure brand or full product experience rather than feature or interaction
Measure longitudinally and conduct trend rather than a point in time analysis
Keep it as scientific as you can (randomisation, third party research)
Compare your NPS to direct competitors
Remember what you are measuring (loyalty and propensity to evangelise, not product satisfaction).
Analyse the qualitative feedback
Collect actionable data too, such as customer satisfaction.
For a shorter version of Daniels’s talk please find this recording from his presentation at Web Directions. Here are the slides from the evening.
Thank you to Medibank for hosting, and all our Prod Anon volunteers for helping on the night, Nadia Gishen, Irene Toh, Marija Becker, Yau and Steve Cheah for this write-up.
Most teams have gotten really good at delivering quickly, and measuring our results. However sometimes we can get a bit addicted to shipping fast rather than right, listen to our assumptions too much, and relying on A/B testing to validate if we’ve delivered value.
Hear about the struggle and joy of a journey of continuous discovery, and some examples of validating ideas before building anything.
Our presenter: Caitlin Blackwell is Acting Head of Product – Candidate Experience at SEEK. She’s previously been in Product Manager roles across many parts of SEEK over the past 7 years, and is currently focused on driving the candidate vision for all of SEEK’s jobseeking products.
Teresa Torres defines Continuous Discovery as weekly touch points with customers, by the team building the product, where they conduct small research activities, in pursuit of a desired product outcome.
That sounds easy but it is a lot of work to adopt if not already a habit. Some of the mindsets needed to do this well are:
A collaborative mindset: Do you have the right people involved in each decision?
A continuous mindset: Are you continuously discovering opportunities and solutions?
An experimental mindset: Are you prepared to be wrong?
Teresa Torres has another great video explaining the value of continuous discovery and where it fits in with the many other methods we may be using already. Join us this month to hear more about what it takes to implement and the benefits to be gained for you and your team. RSVP now
Thanks to our wonderful hosts RMIT Online.
Launching Australia’s first University program in Product Management on the 1st of June to help fulfil the emerging skills gap in product management! It’s a Graduate Certificate – Masters level, 4 subjects, and can be completed in 6-12 months.
This month we will be talking about OKRs – aka Objectives & Key Results!
This is a follow on from our Roadmap discussion in February where some of the frustrations with roadmaps could potentially be addressed by working with OKRs.
OKR’s have been around awhile now, but are somewhat relatively new for product teams here in Oz. We have gathered a few folk who are actively using them, to share the pain and the success of getting started.
Our presenters will be:
Andrew Knibbe, Head of Product – Direct Hirer at Seek – has over 10 years of product management experience – cutting his product teeth in the early days alongside the ThoughtWorks team at Sensis followed by stints at Carsales and Flippa before moving to SEEK where he has had Head of Product roles across both the Consumer and Business side of the employment marketplace. He doesn’t miss the chevrons-on-a-page days of product roadmaps and remains excited about what OKRs can mean for product teams (and customers!).
Wayne Allan, Technical Product Manager, REA – A muso turned software engineer turned product manager, I love creating things people love! Currently solving problems at realestate.com.au
Brad Dunn is the Co-founder and Product Director at OHNO in Melbourne. Before that, he was the executive for Product & Customer Experience at Geo. For 7 years, Brad was the CEO of Nazori, a mobile product development business, where they worked with clients in 12 countries around the world including Samsung, Airbnb and Aesop.
For some pre-reading to get you across the area if you haven’t heard of them or used them yourself then try these resources:
Thanks to our sponsors Medibank for being our hosts this month. RSVP now.
We are Medibank – an integrated healthcare company providing private health insurance and health solutions to 3.7 million Australians through our Medibank and ahm brands, and complimentary health services.
We also provide a range of integrated healthcare services to our private health insurance policyholders, government, corporate and other retail customers. With over 3,000 employees, our head office is located in Melbourne, Victoria, with operations nationally throughout Australia.
By delivering on our promise, for Better Health for Better Lives, we work better as individuals, better as a team and better as a business.
We opened 2019 talking about roadmaps – a topic we had been asked in responses to our annual feedback to spend some more time on. We invited our speakers to share their different perspectives on roadmaps… and we heard come common themes to help understand how to keep a roadmap from controlling your life, and how to turn it into a fabulous communication and vision guide for inspiring your teams, plus some sage advice relevant to each organisation who took the stage that evening.
Below are some highlights from each talk plus the slides from each speaker – feel free to reach out to any of them if you would like to chat more. Plus we have added some references to other resources to read and explore at the end of this post.
David Bignall / Seek
David had much to share – ultimately not a fan but he did share some tips on how to help make them work for you rather than be a slave to them!
Roadmaps are a thing, every company has them so you will
encounter them. David used this deck at Seek over a year ago to his team and
people so proof they are a real thing, but after having the discussion has
helped wean the team/group he is on off them.
For David when sharing what he thinks a roadmap is showed a
map – because it is a journey to an unknown place.
“A document to capture and quickly convey a team’s big-picture goals, specific objectives and their imagined path to success” – Dave
“A company roadmap is a document to capture and quickly convey its big-picture plans and objectives” – prodplan.com
“The first purpose is because the management of a company wants to make sure that the teams are working on the highest value items first, relevant to the company strategy.
The second purpose is because businesses may have date-based commitments. The roadmap is where they see and track those commitments.” Marty Cagan, SVPG
They can be useful – but they can also be a big waste of
time – common issues:
Intended goals/purpose are not stated or are not clear
Often far to specific
You can’t have that much foresight 9 months away
They do not account for “time to value”. Iteration is almost always needed to realise the full value of a new product/feature – (David bravely shared an own example of a very bad roadmap!)
Put item on there and them immediately moving on to the next thing
Ignoring the process of iteration or things going wrong
Detail on roadmap can lead team to auto-pilot. They build what they put down on paper often months in advance
Team goes into auto-pilot. As if this is their job, rather than thinking of most value to be delivered for the customer
Distributed copies are out of date
Keeping stakeholders up to date can drain your time. You don’t want to feel like you work for the roadmap, and it is just sucking your time from doing real work.
“Many untested hypotheses, based on assumptions, plotted in an uncertain future, bearing no resemblance to reality” Jared Spool
Dave’s top tips for roadmaps
Show where you want to go
Choose granularity relative to the timeframe and
Avoid specificity (Show the problem or JTBD or
objectives as descriptors of intent rather than the solution)
Whitney opened with sharing a story about her experiences of
not liking roadmaps because she has never seen a roadmap, become reality. She
first got to know REA when working at a company in the US, and became a slave
to the roadmap as they committed to work they would deliver to this customer.
Then, she joined REA and was so excited about agile and thought, YES! I’ll get
away from roadmaps! But she was fooling herself – see the beautiful roadmap on
the wall at REA (pictured in slides). However, she soon found that REA was
using roadmaps and needed to due to the big size of the organisation and the
need to coordinate a lot across so many teams, groups etc.
However, in Whitney’s attempt to accept roadmaps and make peace with the need for them she started asking “Why do people ask for roadmaps?”.
Some of the things she learnt don’t work when using them:
Don’t work as a promise
Too much detail – just a list of lower level
Lose focus on what the customer needs.
REA owns a lot of companies and even just within
Realestate.com a dozen delivery teams.
“Satisfaction is a confirmation or dis-confirmation of expectations.”
Example of people waiting for train for 15 minutes but
dissatisfied, and others warned that train will come at 5:30 and apologies for
the delay, did not rate their travel experience as dissatisfying as compared to
the first group as their expectations were met/managed.
With that in mind let’s try to think what this artefact does
to satisfy our leaders.
So what are they currently doing with Whitney’s team – they
use a 90 day view – showing a commitment up to 90 days. No promises beyond that
– great for delivery teams. Not great for stakeholders.
For stakeholders they use a Discovery backlog (second 90 days) and an Opportunity backlog (all the rest – no priority) – people now satisfied that their idea is on there – somewhere. Others groups understand that stuff that comes out of Discovery will most likely make it to the committed version and the conversation is being moved to a different stage of team flow.
I encourage you to seek to understand with genuine curiosity, the needs of anyone who has a problem and thinks that a roadmap is the solution. Whitney
Keith Swann / Origin Engery
Keith brought to us a more positive upside to the roadmap
His beliefs are that they help with:
Alignment – up or down
Influence – rarely based on dollars
Leadership – how do we inspire people and rally
them to our cause
Everyone will scrutinize it to their own
beliefs, so do it carefully – target on your back
Strategic – Financial, PMOs, GMs, etc. etc.
interpret the stuff and then try to manage up and down.
Cultural – make sure it talks to your audience
Influence Record – Successful record of moving
things along. Better record = less scrutiny
A road map is a Story telling device and the aspects Keith uses are MUM, Problems, Position, Opportunity, Value. How do you tell the story, “up or down” the organisation. Think of the “Cone of influence” – below people can make lots of small decision but not big decisions. As you move up you get spun out if you aren’t managing those stakeholders.
Eisenhower: Plans are useless, but planning is
Every day the plan can change – the second your
plan is finished it is out of date.
Roadmap = Vision.
Don’t put in too much detail
Think of your audience – Working Tested Feature
Don’t muddle the Project Mindset – Delivery
Planning – Bookending with a roadmap
Don’t become vague in your horizon 2 and 3 –
don’t over promise
Make it easily editable and manageable
Post Its on the wall and photos
Over invested time of effort – working with
visual designers. 5 days work and 6/7000$ and printed in colour. Thus, you deliver
to the roadmap even though you don’t want it anymore because too much effort
went into the artefact.
Don’t clearly show values
Don’t focus on the feature – focus on the
problem or opportunity.
Roadmaps may very well be a necessary evil, especially in a big organisation when you have many teams and people to motivate, inspire and align. However, our speakers have shared some great tips to help keep you from being a slave to them as well. For some more references and reading on steering clear of them and/or leaning into making them work for you check out some of the links below:
For our 1st session of the year, we are focusing on the #1 requested topic in our annual survey – roadmaps!
Our speakers will be taking you through their perspective on roadmaps… we won’t give it all away just yet but you will hear that not every roadmap is f**ked, tips on how to unf**k your roadmap, and of course when to tell a roadmap to go f**k itself.
Our speakers will include examples from both personal experience and the way their current organisation navigates their use. Our three great speakers will be:
Product Lead at SEEK, currently focused on improving the eCommerce space across the APAC region.
Picked up most of my craft from stuffing things up at least once.
Current ‘product passion’ would be trying to describe things in verbs and not nouns. Gives an interesting perspective!
GM Product, Audience & Experience, REA Group
As REA Group’s GM – Product, Audience & Experience, Whitney Cali is responsible for creating smart and compelling experiences in realestate.com.au’s desktop & mobile apps to help change the way the world experiences property. She leads a team of product managers, designers and UX researchers to create intuitive and personalised experiences that help individuals make great property decisions.
Lead Product Manager, Origin Energy
There will be lots of time for Q&A and you can submit questions via Sli.do or vote on the questions of others. Details will be shared on the night.
Thanks to our sponsors! Without them these events would not happen!
Thank you to Gather for hosting us. Gather is a brand within United Co. and exists to connect and inspire a community of Melbourne’s bright-minded and open-hearted, exploring complex questions, expanding perspectives and building skills for a thriving future.
Join us for a panel discussion on Service Design & Product Management on June 21st. RSVP
Today’s customers demand more than a good product; they expect a great customer experience. Are we being experience led or product led? How do we come together to deliver both?
Most designers have felt the frustration of working on product led projects. Most product managers have felt the inevitable scope creep of designers. If you’ve felt the pain of the design vs product divide, you’re not alone. It’s time we sit down for a chat (& drinks) to work through…
We already have a waitlist for this event but spaces open up so put yourself on the list. Drinks and nibbles will be provided.
Kate Edwards-Davis, Product Manager, Karista – a start-up helping people who need care to find their ideal community-based care or disability support service.
She stumbled into a career in IT after studying philosophy and classical music performance. Always a strong advocate for the value of spending time listening to customers, it has helped her teams to deliver imaginative and valuable solutions in many sectors including finance, retail, manufacturing, government and education. She should probably say her greatest joy in life is her two young sons – but in all honesty they probably come second to her obsession with the music of Prince.
Daniel has been in product management for over 15 years, chiefly working in IT with a focus on B2B products. He gets excited about helping businesses become more effective in decision-making, more efficient in their processes and more engaging with their customers.
Daniel is at his happiest when waving his arms about in front of a whiteboard with a bunch of smart people, exploring problems and weighing up solutions. He’s passionate about product management as a discipline and is intrigued by how businesses, large and small, grapple with the sometimes-elusive concepts of innovation and collaboration.
A cognitive bias is a shortcut your brain takes to make decisions. Cognitive biases can impact our decision making, our memory, the way we interpret research and more.
As product managers need to understand our customers & their needs, motivations, problems (etc), it is a ‘mistake’ to let your lazy brain fall prey to the cognitive biases that make you hold onto your own preferences or beliefs, no matter what evidence is presented!
when we have an idea in our head we try to confirm it… in mtgs, in research etc. we should try to disprove our ideas as a way to not have confirmation bias… #prodanon
Our speaker was David Di Sipio, a registered Psychologist and currently working at Squiz as a UX Consultant. David creates great experiences by focusing on what makes people tick. His approach is grounded in academic research, big-data and best-practice.
First, we’ll hear from the folks who did Lightning Talks last year. Get their tips and hear about their experience in planning, practicing and getting up there!
Then, it’s your turn! Those of you interested will get 90 seconds to pitch what you’d like to talk about. No slides… just you sharing with the crowd & judges.
At the end of the evening, 2 people will be selected to present a Lightning Talk at Leading the Product 2018. Selection is based on the judges’ vote and votes from the audience.
At the Sydney pitchfest, Ivy Horinbrook made it thru the pitches & will be giving a talk at LTP. She said: When I first read about pitch night, I thought “no way”. 30 pitches, one winner… sounds a little too much like gladiatorial combat. But then I thought, it’s 90 seconds, what do I have to lose? And on the night the crowd was friendly, there was a sense of camaraderie between those pitching, and it was fun to engage the crowd. I still can’t believe I won – I’ll be taking the stage in front of 400 product managers in Sydney in October, that’s a pretty amazing opportunity.
This month we are having a cozy fireside chat with product leaders to discuss managing product managers.
They hire, they fire and they look after the product portfolio as well as the product managers. We will talk about what it takes to manage product managers, guide them to greatness and inspire them in their roles.
However, product managers are people managers too – so we will talk a little bit about what skills you need to enhance and work on to do this well. Most importantly you get to ask the questions!
Fiona has worked on many sides of the product before joining the product management league and then heading up the team. Considering PageUp People’s products are focused on the HR customer (recruitment, seccession, etc), she has talked with heaps of corporate recruiters over the years. She has also managed other teams – including sales – so she’ll much insight into the people side of this ‘manager’ part of the product title.
Layla Foord is a builder of businesses, saver of lost dogs and sometime singer. She has learned her craft over the last 23 years here in Australia and in the UK. She specialises in finding connections and building amazing teams who make awesome things. Managing Director of Touchtech Labs she is building a new business delivering web and mobile product solutions for entrepreneurs and enterprises.
As GM at Envato, Layla recently launched Envato Studio a $6m+ global freelance platform. She has developed strategies and digital products for Yellow Pages and launched Whereis Navigator one of the first Mobile GPS apps. In London she held the role of Product Development Director at Nielsen with responsibilities across Europe and led an EU funded project to discover new technologies which involved educational institutions in four countries pushing the boundaries of auto-detection image technology.
Layla is also on the board of 100 Story Building (http://www.100storybuilding.org.au/), an amazing social enterprise in Melbourne’s inner west helping to improve the literacy of young people in the area.
Since 1997, PageUp has helped employers worldwide attract, hire, develop, retain, and improve employee performance. Our Unified Talent Management platform, along with our talent management consulting services, help you optimize your multinational workforce strategy across the whole business, maximize business impact with a balance between global efficiency and local responsiveness, and continuously improve the return on your human capital investment.
What happens when your darling little product baby is ready for launch and support. Be it MVP or mature product, you’re letting it loose on the world and need to know how to handle what happens next.
Our speakers will discuss what happens in that launch and support timeframe plus we’ll have Q&A:
When is your product ready-or when are we ready to let go (ie transition from active delivery to support)
Handing over to a support team vs doing it ourselves
Looking after your customer. Or working out when / if they should just build their own team, and how.
Sticking to the plan (fixes, customer feedback, curveballs)
Sticking to the hours (work/life)
What if the product comes back for more? What if it doesn’t?
Where we failed. What we learnt.
Case studies: product support (4 very different models: what went well and what didn’t).
FYI, we’re not covering launch details, marketing, or metrics.
Suni Stolic has over 10 years experience in delivering multiple products in web and mobile, in both Melbourne and London. While her role names changed at a speed of light, she found that the work itself changed much less. Always caring deeply about the products she worked on, passionate about understanding her users, she enjoys planning in an iterative, goal focused manner. She is great at building rapport with business teams and delivery teams, and believes in collaboration, transparency and open communication.
Suni is a product manager at Cogent and will talk through a couple of recent client experiences, from tailor made support arrangements, to the ups and downs of how things actually unfold.
The Cogent team deliver remarkable software to meet business goals, budget and timeframes of their clients. They sometimes build products for large organisations (particularly in Health and Education), other times they work with start-ups, or consult and add capabilities to existing teams. Often these client engagements are long term relationships, but sometimes they come in bursts. Most of the time they provide support to these products after delivery has finished up.
Teamsquare provides beautiful and collaborative workspace for Melbourne’s leading creators – freelancers, startups, designers, builders and doers. They provide amazing amenities, rock solid technology infrastructure, a strong community, regular events and a raft of services to help their members get on with what they do best. Their more than just a workplace – their a community of like minded creators.
We have a combination of open-plan workstations and private offices. Flexible month-by-month subscriptions with no lock in. Free for anybody to try for a day via http://try.teamsquare.co/