September Wrap – Net Promoter Score (NPS): Science or Pseudoscience?

Over the years, Net Promoter Score (NPS) has become the default question to measure and maximise value. But is it right? Is it true? Daniel Kinal joined us to share his thoughts.

Where did NPS come from?
Back in 2003, Fred Reichheld introduced the concept to the world. He felt the current measures of loyalty were too convoluted and complicated. So he did his own study, with surprising results, even to him. What he came up with, Net Promoter Score – the one metric that was supposed to have the strongest correlation to company success.

Why should it work?
The more likely you are to advocate for a brand, the more people will be willing to trial the product, therefore reducing your acquisition costs. Also, those advocates are more likely to be repeat customers and increasing their lifetime customer value. Score. Double score!

Some caveats:

  • Simply irrelevant in some industries
  • Not predictive in a monopoly or near-monopoly conditions
  • Data analysed was historical, not future
  • Unconvincing replication studies.
  • Highly volatile measure
  • Obscures critical information

Is there a correlation? Well, yes. Is it good as a predictor for future success? Well, maybe not as much. In fact, in one study, NPS only explained 38% of future growth.

Bastardisation
If you game your scores, what do you really achieve? From colour coding, nudging your scores, and filtering out negative results. What are you actually able to learn?

Is there an upside?
Yes, some compelling aspects of NPS include, it is relatively cheap, fast, simple; and well accepted.

Already using NPS? Make the most of your data.

  • Don’t focus purely on the number.
  • Measure brand or full product experience rather than feature or interaction
  • Measure longitudinally and conduct trend rather than a point in time analysis
  • Keep it as scientific as you can (randomisation, third party research)
  • Compare your NPS to direct competitors
  • Remember what you are measuring (loyalty and propensity to evangelise, not product satisfaction).
  • Analyse the qualitative feedback
  • Collect actionable data too, such as customer satisfaction.

For a shorter version of Daniels’s talk please find this recording from his presentation at Web Directions. Here are the slides from the evening.

Thank you to Medibank for hosting, and all our Prod Anon volunteers for helping on the night, Nadia Gishen, Irene Toh, Marija Becker, Yau and Steve Cheah for this write-up.

https://twitter.com/product_anon/status/1177141467384385537

June event: Continuous Discovery – Worth the effort?

Most teams have gotten really good at delivering quickly, and measuring our results. However sometimes we can get a bit addicted to shipping fast rather than right, listen to our assumptions too much, and relying on A/B testing to validate if we’ve delivered value.

Hear about the struggle and joy of a journey of continuous discovery, and some examples of validating ideas before building anything.

Our presenter:
Caitlin Blackwell is Acting Head of Product – Candidate Experience at SEEK. She’s previously been in Product Manager roles across many parts of SEEK over the past 7 years, and is currently focused on driving the candidate vision for all of SEEK’s jobseeking products.

Continuous Discovery

Teresa Torres defines Continuous Discovery as weekly touch points with customers, by the team building the product, where they conduct small research activities, in pursuit of a desired product outcome.

That sounds easy but it is a lot of work to adopt if not already a habit. Some of the mindsets needed to do this well are:

  • A collaborative mindset: Do you have the right people involved in each decision?
  • A continuous mindset: Are you continuously discovering opportunities and solutions?
  • An experimental mindset: Are you prepared to be wrong?

Teresa Torres has another great video explaining the value of continuous discovery and where it fits in with the many other methods we may be using already. Join us this month to hear more about what it takes to implement and the benefits to be gained for you and your team. RSVP now

Thanks to our wonderful hosts RMIT Online.

Launching Australia’s first University program in Product Management on the 1st of June to help fulfil the emerging skills gap in product management! It’s a Graduate Certificate – Masters level, 4 subjects, and can be completed in 6-12 months.

May session – OKRs in Real Life

This month we will be talking about OKRs – aka Objectives & Key Results!

This is a follow on from our Roadmap discussion in February where some of the frustrations with roadmaps could potentially be addressed by working with OKRs.

OKR’s have been around awhile now, but are somewhat relatively new for product teams here in Oz. We have gathered a few folk who are actively using them, to share the pain and the success of getting started.

Our presenters will be:

  • Andrew Knibbe, Head of Product – Direct Hirer at Seek – has over 10 years of product management experience – cutting his product teeth in the early days alongside the ThoughtWorks team at Sensis followed by stints at Carsales and Flippa before moving to SEEK where he has had Head of Product roles across both the Consumer and Business side of the employment marketplace. He doesn’t miss the chevrons-on-a-page days of product roadmaps and remains excited about what OKRs can mean for product teams (and customers!).
  • Wayne Allan, Technical Product Manager, REA – A muso turned software engineer turned product manager, I love creating things people love! Currently solving problems at realestate.com.au
  • Brad Dunn is the Co-founder and Product Director at OHNO in Melbourne. Before that, he was the executive for Product & Customer Experience at Geo. For 7 years, Brad was the CEO of Nazori, a mobile product development business, where they worked with clients in 12 countries around the world including Samsung, Airbnb and Aesop.

For some pre-reading to get you across the area if you haven’t heard of them or used them yourself then try these resources:

Thanks to our sponsors Medibank for being our hosts this month. RSVP now.

We are Medibank – an integrated healthcare company providing private health insurance and health solutions to 3.7 million Australians through our Medibank and ahm brands, and complimentary health services.

We also provide a range of integrated healthcare services to our private health insurance policyholders, government, corporate and other retail customers. With over 3,000 employees, our head office is located in Melbourne, Victoria, with operations nationally throughout Australia.

By delivering on our promise, for Better Health for Better Lives, we work better as individuals, better as a team and better as a business.

F**k Roadmaps!!! – The good, the bad and the ugly – The wrap-up

We opened 2019 talking about roadmaps – a topic we had been asked in responses to our annual feedback to spend some more time on. We invited our speakers to share their different perspectives on roadmaps… and we heard come common themes to help understand how to keep a roadmap from controlling your life, and how to turn it into a fabulous communication and vision guide for inspiring your teams, plus some sage advice relevant to each organisation who took the stage that evening.

Below are some highlights from each talk plus the slides from each speaker – feel free to reach out to any of them if you would like to chat more. Plus we have added some references to other resources to read and explore at the end of this post.

David Bignall / Seek

David had much to share – ultimately not a fan but he did share some tips on how to help make them work for you rather than be a slave to them!

Roadmaps are a thing, every company has them so you will encounter them. David used this deck at Seek over a year ago to his team and people so proof they are a real thing, but after having the discussion has helped wean the team/group he is on off them.

For David when sharing what he thinks a roadmap is showed a map – because it is a journey to an unknown place.

“A document to capture and quickly convey a team’s big-picture goals, specific objectives and their imagined path to success” – Dave

 “A company roadmap is a document to capture and quickly convey its big-picture plans and objectives” – prodplan.com

“The first purpose is because the management of a company wants to make sure that the teams are working on the highest value items first, relevant to the company strategy.

The second purpose is because businesses may have date-based commitments. The roadmap is where they see and track those commitments.” Marty Cagan, SVPG

They can be useful – but they can also be a big waste of time – common issues:

  1. Intended goals/purpose are not stated or are not clear
  2. Often far to specific
    1. You can’t have that much foresight 9 months away
  3. They do not account for “time to value”. Iteration is almost always needed to realise the full value of a new product/feature – (David bravely shared an own example of a very bad roadmap!)
    1. Put item on there and them immediately moving on to the next thing
    2. Ignoring the process of iteration or things going wrong
  4. Detail on roadmap can lead team to auto-pilot. They build what they put down on paper often months in advance
    1. Team goes into auto-pilot. As if this is their job, rather than thinking of most value to be delivered for the customer
  5. Distributed copies are out of date
    1. Keeping stakeholders up to date can drain your time. You don’t want to feel like you work for the roadmap, and it is just sucking your time from doing real work.

“Many untested hypotheses, based on assumptions, plotted in an uncertain future, bearing no resemblance to reality” Jared Spool

Dave’s top tips for roadmaps

  1. Show where you want to go
  2. Choose granularity relative to the timeframe and audience
  3. Avoid specificity (Show the problem or JTBD or objectives as descriptors of intent rather than the solution)
  4. Keep it simple, centralised and accessible
  5. Don’t work for it, it works for you.

Whitney Cali / realestate.com.au

Whitney opened with sharing a story about her experiences of not liking roadmaps because she has never seen a roadmap, become reality. She first got to know REA when working at a company in the US, and became a slave to the roadmap as they committed to work they would deliver to this customer. Then, she joined REA and was so excited about agile and thought, YES! I’ll get away from roadmaps! But she was fooling herself – see the beautiful roadmap on the wall at REA (pictured in slides). However, she soon found that REA was using roadmaps and needed to due to the big size of the organisation and the need to coordinate a lot across so many teams, groups etc.

However, in Whitney’s attempt to accept roadmaps and make peace with the need for them she started asking “Why do people ask for roadmaps?”.

Some of the things she learnt don’t work when using them:

  1. Don’t work as a promise
  2. Too much detail – just a list of lower level features
  3. Lose focus on what the customer needs.

REA owns a lot of companies and even just within Realestate.com a dozen delivery teams.

“Satisfaction is a confirmation or dis-confirmation of expectations.”

Example of people waiting for train for 15 minutes but dissatisfied, and others warned that train will come at 5:30 and apologies for the delay, did not rate their travel experience as dissatisfying as compared to the first group as their expectations were met/managed.

With that in mind let’s try to think what this artefact does to satisfy our leaders.

So what are they currently doing with Whitney’s team – they use a 90 day view – showing a commitment up to 90 days. No promises beyond that – great for delivery teams. Not great for stakeholders.

For stakeholders they use a Discovery backlog (second 90 days) and an Opportunity backlog (all the rest – no priority) – people now satisfied that their idea is on there – somewhere. Others groups understand that stuff that comes out of Discovery will most likely make it to the committed version and the conversation is being moved to a different stage of team flow.

I encourage you to seek to understand with genuine curiosity, the needs of anyone who has a problem and thinks that a roadmap is the solution. Whitney

Keith Swann / Origin Engery

Keith brought to us a more positive upside to the roadmap discussion

His beliefs are that they help with:

  • Alignment – up or down
  • Influence – rarely based on dollars
  • Leadership – how do we inspire people and rally them to our cause

Alignment

  • Everyone will scrutinize it to their own beliefs, so do it carefully – target on your back
    • Strategic – Financial, PMOs, GMs, etc. etc. interpret the stuff and then try to manage up and down.
    • Cultural – make sure it talks to your audience
    • Influence Record – Successful record of moving things along. Better record = less scrutiny

A road map is a Story telling device and the aspects Keith uses are MUM, Problems, Position, Opportunity, Value. How do you tell the story, “up or down” the organisation. Think of the “Cone of influence” – below people can make lots of small decision but not big decisions. As you move up you get spun out if you aren’t managing those stakeholders.

  • Eisenhower: Plans are useless, but planning is indispensable
  • Every day the plan can change – the second your plan is finished it is out of date.
  • Roadmap = Vision.

Takeaways

  • Don’t put in too much detail
  • Think of your audience – Working Tested Feature – WTF
  • Don’t muddle the Project Mindset – Delivery Planning – Bookending with a roadmap
  • Don’t become vague in your horizon 2 and 3 – don’t over promise
  • Make it easily editable and manageable
    • Post Its on the wall and photos
  • Over invested time of effort – working with visual designers. 5 days work and 6/7000$ and printed in colour. Thus, you deliver to the roadmap even though you don’t want it anymore because too much effort went into the artefact.
  • Don’t clearly show values
  • Don’t focus on the feature – focus on the problem or opportunity.

Summary

Roadmaps may very well be a necessary evil, especially in a big organisation when you have many teams and people to motivate, inspire and align. However, our speakers have shared some great tips to help keep you from being a slave to them as well. For some more references and reading on steering clear of them and/or leaning into making them work for you check out some of the links below:

  • Brad Dunn has some opposing opinions on roadmaps.
  • Marty Cagan is not a fan and in these post he links to OKR’s which came up in both the speaker presentations and the questions afterwards.
  • And last but not least for a more practical tuition on taking back control of your roadmap you could check out Bruce McCarthy’s seminar on UIE

Thank you!!!

And thank you to our sponsors as always, without them these events do not happen.

gather
UnitedCo.
seek.com
Brainmates

F**k Roadmaps!!! – The good, the bad and the ugly

Welcome to 2019!!!

For our 1st session of the year, we are focusing on the #1 requested topic in our annual survey – roadmaps!

Our speakers will be taking you through their perspective on roadmaps… we won’t give it all away just yet but you will hear that not every roadmap is f**ked, tips on how to unf**k your roadmap, and of course when to tell a roadmap to go f**k itself.

Our speakers will include examples from both personal experience and the way their current organisation navigates their use. Our three great speakers will be:

David Bignell

  • Product Lead at SEEK, currently focused on improving the eCommerce space across the APAC region.
  • Picked up most of my craft from stuffing things up at least once.
  • Current ‘product passion’ would be trying to describe things in verbs and not nouns. Gives an interesting perspective!

Whitney Cali

  • GM Product, Audience & Experience, REA Group
  • As REA Group’s GM – Product, Audience & Experience, Whitney Cali is responsible for creating smart and compelling experiences in realestate.com.au’s desktop & mobile apps to help change the way the world experiences property. She leads a team of product managers, designers and UX researchers to create intuitive and personalised experiences that help individuals make great property decisions.

Keith Swann

  • Lead Product Manager, Origin Energy

There will be lots of time for Q&A and you can submit questions via Sli.do or vote on the questions of others. Details will be shared on the night.

Thanks to our sponsors! Without them these events would not happen!

Thank you to Gather for hosting us. Gather is a brand within United Co. and exists to connect and inspire a community of Melbourne’s bright-minded and open-hearted, exploring complex questions, expanding perspectives and building skills for a thriving future.

Making sure we don’t go hungry or die of thirst..

Brainmates

RSVP now


Every Product is a Service: Service Design and Product Management – June event

Join us for a panel discussion on Service Design & Product Management on June 21st. RSVP

Today’s customers demand more than a good product; they expect a great customer experience. Are we being experience led or product led? How do we come together to deliver both?

Most designers have felt the frustration of working on product led projects. Most product managers have felt the inevitable scope creep of designers. If you’ve felt the pain of the design vs product divide, you’re not alone. It’s time we sit down for a chat (& drinks) to work through…

We already have a waitlist for this event but spaces open up so put yourself on the list. Drinks and nibbles will be provided.

Our panelists:

Kate Edwards-Davis, Product Manager, Karista – a start-up helping people who need care to find their ideal community-based care or disability support service.

She stumbled into a career in IT after studying philosophy and classical music performance. Always a strong advocate for the value of spending time listening to customers, it has helped her teams to deliver imaginative and valuable solutions in many sectors including finance, retail, manufacturing, government and education. She should probably say her greatest joy in life is her two young sons – but in all honesty they probably come second to her obsession with the music of Prince.

Daniel Kinal, Senior Product Manager, MYOB

Daniel has been in product management for over 15 years, chiefly working in IT with a focus on B2B products. He gets excited about helping businesses become more effective in decision-making, more efficient in their processes and more engaging with their customers.
Daniel is at his happiest when waving his arms about in front of a whiteboard with a bunch of smart people, exploring problems and weighing up solutions. He’s passionate about product management as a discipline and is intrigued by how businesses, large and small, grapple with the sometimes-elusive concepts of innovation and collaboration.

Our wonderful Service design line-up:

Dr Stefanie Russo, Principal Designer, NAB
Julia Birks, Lead strategic designer, Isobar.

We will use sli.do for panel questions so start prepping your thoughts on what is the difference between service & product? I’ll add more controversial questions closer to the date 🙂

The event will be hosted by SuperSeed / NEXT – The innovation division of the Reece Group.

Check your bias at the door – May wrap-up

A cognitive bias is a shortcut your brain takes to make decisions. Cognitive biases can impact our decision making, our memory, the way we interpret research and more.

As product managers need to understand our customers & their needs, motivations, problems (etc), it is a ‘mistake’ to let your lazy brain fall prey to the cognitive biases that make you hold onto your own preferences or beliefs, no matter what evidence is presented!

Our speaker was David Di Sipio, a registered Psychologist and currently working at Squiz as a UX Consultant. David creates great experiences by focusing on what makes people tick. His approach is grounded in academic research, big-data and best-practice.

David ran us through some great examples & how to manage the 6 most common biases:

  1. Selection Bias – who are you talking to?
  2. Framing Effect – the words you use & way you present your questions
  3. Confirmation Bias – focusing on the information that reinforces your ideas & ignoring the ones that disprove your idea
  4. Observer Effect – subconsciously influencing someone with your body language
  5. Anchoring Bias – putting too much emphasis on the 1st bit of info we receive
  6. Clustering Illusion – seeing patterns when none exist

After walking thru these biases, David gave us time to talk thru some of the biases we’ve fallen for & there was a bit of conversation regarding which is our favourite bias!

Want more info? You can view David’s slides or read his article on Medium which also contains the links to all the great videos he showed.

Thanks again to Origin for hosting us.

Origin Energy

We’ve announced our June session – RSVP now for a collaboration between Product Anonymous and Service Design Melbourne discussing ‘every product is a service’.

Pitchfest for Lightning Talk slots at Leading the Product Melbourne

Leading the Product (LTP) are planning another round of Lightning Talks this year. Would you like to be one of those speakers & have the chance to present to 500+ product professionals???

We’re hosting a LTP Pitchfest to give you the chance to do exactly that!

First, we’ll hear from the folks who did Lightning Talks last year. Get their tips and hear about their experience in planning, practicing and getting up there!

Then, it’s your turn! Those of you interested will get 90 seconds to pitch what you’d like to talk about. No slides… just you sharing with the crowd & judges.

At the end of the evening, 2 people will be selected to present a Lightning Talk at Leading the Product 2018. Selection is based on the judges’ vote and votes from the audience.

At the Sydney pitchfest, Ivy Horinbrook made it thru the pitches & will be giving a talk at LTP. She said:
When I first read about pitch night, I thought “no way”. 30 pitches, one winner… sounds a little too much like gladiatorial combat. But then I thought, it’s 90 seconds, what do I have to lose? And on the night the crowd was friendly, there was a sense of camaraderie between those pitching, and it was fun to engage the crowd. I still can’t believe I won – I’ll be taking the stage in front of 400 product managers in Sydney in October, that’s a pretty amazing opportunity.

Our judges for the evening:

  • Sarah Mitchell – Product Manager for Leading the Product
  • Adrienne Tan – Co-Founder of Brainmates & Leading the Product
  • Nicole Brolan – Product Director at SEEK
  • Senior Product Leader from Culture Amp soon to be announced.

I WANT TO PRESENT! WHAT DO I NEED TO DO?

You should rsvp for this session and then tell us you’re interested via this form.

WHAT IS A LIGHTNING TALK?

It is a five minute presentation, with 20 slides that advance automatically. You can see the video of the three Lightning talks from Leading the Product Melbourne

Of course don’t forget to get your tickets for the Leading the Product 2018 as it is going to be a fabulous product event!

Thanks to our LTP sponsors Brainmates and Seek for making this event happen.

seek Brainmates

 

 

RSVP now and help us pick the line-up for LTP

What does a product manager “manager” look like? – June event

This month we are having a cozy fireside chat with product leaders to discuss managing product managers.

They hire, they fire and they look after the product portfolio as well as the product managers. We will talk about what it takes to manage product managers, guide them to greatness and inspire them in their roles.

However, product managers are people managers too – so we will talk a little bit about what skills you need to enhance and work on to do this well. Most importantly you get to ask the questions!

RSVP

Our speakers, mentors and advisers are:

Fiona Moreton – Head of Product Strategy at PageUp People.

Fiona has worked on many sides of the product before joining the product management league and then heading up the team. Considering PageUp People’s products are focused on the HR customer (recruitment, seccession, etc), she has talked with heaps of corporate recruiters over the years. She has also managed other teams – including sales – so she’ll much insight into the people side of this ‘manager’ part of the product title.

Layla Foord is a builder of businesses, saver of lost dogs and sometime singer. She has learned her craft over the last 23 years here in Australia and in the UK. She specialises in finding connections and building amazing teams who make awesome things. Managing Director of Touchtech Labs she is building a new business delivering web and mobile product solutions for entrepreneurs and enterprises.

As GM at Envato, Layla recently launched Envato Studio a $6m+ global freelance platform. She has developed strategies and digital products for Yellow Pages and launched Whereis Navigator one of the first Mobile GPS apps. In London she held the role of Product Development Director at Nielsen with responsibilities across Europe and led an EU funded project to discover new technologies which involved educational institutions in four countries pushing the boundaries of auto-detection image technology.

Layla is also on the board of 100 Story Building (http://www.100storybuilding.org.au/), an amazing social enterprise in Melbourne’s inner west helping to improve the literacy of young people in the area.

Since 1997, PageUp has helped employers worldwide attract, hire, develop, retain, and improve employee performance. Our Unified Talent Management platform, along with our talent management consulting services, help you optimize your multinational workforce strategy across the whole business, maximize business impact with a balance between global efficiency and local responsiveness, and continuously improve the return on your human capital investment.

RSVP

PageUp_LOGO_CMYK

When your product is newly wild in the world – May event

What happens when your darling little product baby is ready for launch and support. Be it MVP or mature product, you’re letting it loose on the world and need to know how to handle what happens next.

Our speakers will discuss what happens in that launch and support timeframe plus we’ll have Q&A:

  • When is your product ready-or when are we ready to let go (ie transition from active delivery to support)
  • Handing over to a support team vs doing it ourselves
  • Looking after your customer. Or working out when / if they should just build their own team, and how.
  • Sticking to the plan (fixes, customer feedback, curveballs)
  • Sticking to the hours (work/life)
  • What if the product comes back for more? What if it doesn’t?
  • Where we failed. What we learnt.
  • Current challenges
  • Case studies: product support (4 very different models: what went well and what didn’t).

FYI, we’re not covering launch details, marketing, or metrics.

Our speakers:

Suni Stolic has over 10 years experience in delivering multiple products in web and mobile, in both Melbourne and London. While her role names changed at a speed of light, she found that the work itself changed much less. Always caring deeply about the products she worked on, passionate about understanding her users, she enjoys planning in an iterative, goal focused manner. She is great at building rapport with business teams and delivery teams, and believes in collaboration, transparency and open communication.

Suni is a product manager at Cogent and will talk through a couple of recent client experiences, from tailor made support arrangements, to the ups and downs of how things actually unfold.

Our sponsors:

The Cogent team deliver remarkable software to meet business goals, budget and timeframes of their clients. They sometimes build products for large organisations (particularly in Health and Education), other times they work with start-ups, or consult and add capabilities to existing teams. Often these client engagements are long term relationships, but sometimes they come in bursts. Most of the time they provide support to these products after delivery has finished up.

teamsquare Thanks to Teamsquare for hosting!

Teamsquare provides beautiful and collaborative workspace for Melbourne’s leading creators – freelancers, startups, designers, builders and doers. They provide amazing amenities, rock solid technology infrastructure, a strong community, regular events and a raft of services to help their members get on with what they do best. Their more than just a workplace – their a community of like minded creators.

We have a combination of open-plan workstations and private offices. Flexible month-by-month subscriptions with no lock in. Free for anybody to try for a day via http://try.teamsquare.co/