This article continues the discussion from Product Anonymous back in June.  Full credit goes to the team and the attendees for providing key steps, insight and critical analysis.

In the last post we identified more alternatives that might address our key issue.  Step #3 in the process is to evaluate those alternatives.

You’ve got your problem identified, and you have alternatives A through Z. How are you going to evaluate them?  You probably have a gut feel already, but how can you do this more rigorously?

“The great thing about fact-based decisions is that they overrule the hierarchy.”
Jeff Bezos

Get them on the table.

List all the alternatives together – preferably on one page.  People find it easier to do comparisons if they can just flick through with their eyes.  If you have more than one page then you risk people not being able to remember details as they skip between pages.

Clarify each alternative so that everyone is sure they are talking about the same thing.  Perhaps you could just verbally describe each and highlight any differences.  Or alternatively document each alternative with a detailed description.  It is better to find any misunderstandings early.

Decisions are limited by assumptions.  Without proper attention, humans tend to make poor assumptions, if we even realise we make them at all.  Call out the assumptions to the team or in the document. Test the assumptions, get the reactions and feedback, and clarify if necessary.  Perhaps you’ll discover your assumption was wrong.  Again, it is better to find any misunderstandings early.

Finding a valid way to compare alternatives can be hard

Comparisons can be complicated, as you are rarely comparing apples with apples.  You will need to find ways to compare your alternatives.  This can be done in a few different ways, and the choice of comparison can be as complicated as the original list of alternatives.  But if you need a decision, then you first need to find a valid way of comparing your alternatives.

Note that evaluating alternatives is another opportunity to engage with your stakeholders.  While this could be formal meetings, it could also be over a coffee with some of the stakeholders to gain their insight.  It also has the potential to open up new ways of evaluating, build a better decision, create support and unearth the hidden ‘gotchas’ and opposition.

Cut back the options

More options means more attention, more short-term memory usage and more multitasking between different evaluation methods.  Attention and will power are both exhaustible resources, and too many options can be quite draining – possibly leading to analysis paralysis.

For example, what does the product strategy, corporate strategy or vision have to say about your alternatives?  If some of the alternatives don’t align with the overall goal then perhaps you should rule them out from the current list.  If these alternatives are compelling but disagree with the strategy then you may have a lot of work ahead to pivot, change the product strategy or corporate strategy.  It might be perfectly valid, but you probably have even more work ahead.

Make a first pass and cut down the number of options to something manageable – perhaps three to five of the best options.  Make sure the status quo is one of the options.  Identify the options that are being discarded, and be clear why they are being excluded.

Optimise for one thing

Ideally, work out one thing you are trying to optimise (people, costs, adoption, etc.), and stick with that.  If you try to combine different variables and comparison methods you risk going to go into analysis paralysis; when do you optimise for X, when do you optimise for Y?  Sort and list you remaining alternatives using only your chosen optimisation.

Dan Ariely and relativity

Dan Ariely has written quite a lot about behavioural economics, and especially how people don’t know what they want unless they see it in context.  Watch his TED talk to see how the addition of a third decoy option can make one option more compelling than another.  In his example it is hard to compare a holiday to Paris against a holiday to Rome.  But when you add a third option that is just worse than a holiday to Rome (Rome without coffee), then the Rome option becomes even more appealing.

The way out of this irrational behaviour is to take a more scientific approach, and try to be more objective.

Quantitative versus Qualitative?

When evaluating the alternatives some people will want quantitative evaluation; numerical data driven evaluation.  This can range from counting events, calculating Return On Investment (ROI), applying weighted averages, opportunity cost or even conjoint analysis (my favourite).

The reasoning is that numbers don’t lie, and that an objective decision can be made.  Unfortunately quantitative data usually has some human element and can be misled, either intentionally or not.   As the saying goes ‘lies, damn lies and statistics’.  The reality is you can still choose the statistic or assumption that supports your cause.  And if someone doesn’t want to believe your numbers they can always dispute whether the data source is relevant or whether the weighting is correct.

Quantitative data can prove correlation, but it can rarely prove causation.  Even qualitative data can only indicate causation.  My favourite example is the theory that lack of pirates is leading to global warming.

While quantitative data can be compelling, it won’t be the whole answer.

The decision may require a more qualitative evaluation; what is the impact on people, is it important to the company, what is the customer feedback?  My personal favourite is the ‘jobs to be done‘ analysis; what was the customer hiring the product to do?

Qualitative data will always be disputable as it isn’t ‘hard data’ but it is often necessary in small sample sizes or when people are directly involved.

The plural of anecdotes is not data
Frank Kotsonis / Roger Brinner.

The plural of anecdotes is data. 
Raymond Wolfinger

Apparently both are quotes (and self-referential proof that quotes are also not data).

Statistically, for small amounts of anecdotes, the plural of anecdotes should not be considered statistically significant data.  But just because the sample size is too small doesn’t mean the anecdotes should be completely ignored.  Given a large enough set of inputs, then the anecdotes start to become data where overall patterns can be identified.

Sensitivity analysis

You can perform additional sensitivity analysis on your alternatives to assess whether your comparison is solid.  One example is when you adjust the weighted averages by +/- 10% to check if it makes a difference on the outcome.  If the outcome changes then you are too dependent on the fine details of the weighting and the input data and will not have a stable and reliable answer.

Compare

Use a combination of techniques to narrow the list, perform some quantitative analysis and qualitative research.  The list should now be manageable for the next stage – decision time.

At the end of this evaluation you may even have a document.  For a large enough decision, team or project, this becomes a bit of a strategy document.  Circulating this document as a draft or straw man helps give people a chance to get on the same page and build consensus.  It gives people a chance to comment further if they’d like.

Now… a final question.  Is that gut feel answer suddenly now the best alternative?  You may need to question whether you are doing this objectively. You may be suffering from confirmation bias, where people favour information that confirms their own beliefs.

Hopefully now you have option A to Z cut back to a manageable size of less than 5 options.  The next step is deciding between them. 

Have you got any suggestions in ways to evaluate alternatives? Please feel free to comment below to add to the discussion.

Read part 4 on identifying alternatives or go forward to part 6 on making your decision.

Steve is a Product Development Manager at Telstra Wholesale.  The views expressed in this post are his only and do not necessarily reflect the views of Telstra.

This article continues the discussion from Product Anonymous back in June.  Full credit goes to the team and the attendees for providing key steps, insight and critical analysis.

In the last two posts we talked about defining the problem and addressing the right stakeholders.  The next step in the process is to identify some alternatives.

You’ve got your problem identified, and you immediately think of alternatives A, B and C. Is that enough?

What are the unwritten alternatives?

Remember that the status quo is an alternative, and not necessarily a bad one.  Any changes should always be judged against the current operation.  You need to justify why the new solution is better than what you have now.  After all, what you are doing now probably worked for a reason.

What about alternatives that allow you to learn quickly or even fail fast?  A clear decision to research an issue (by allocating a small amount of resources within a short time frame) might be the best way forward in a complex situation.  The research project may fail, but at the end you will be in a better place than before.

Delaying the decision is also an alternative, but be careful that you are not just being indecisive.  Product management is about getting out there, deciding on product issues and then making sure it happens – product managers are naturally biased towards action.  Sometimes it can be too early to decide; either you don’t have enough information or a short delay will have no effect.  It is important that any delay should be for a good reason (i.e. key information will become available), and not just due to indecision.  Not only is indecision bad for the project, it is bad for the team.  Your team needs to see you as understanding the problem and not just delaying everything until it is too late.

Valid alternatives can also include areas of uncertainty; either the details can be worked out later, or the uncertainty won’t have a big impact. Both Waterfall and Agile use this uncertainty, but in different ways.  Waterfall might simply define high level requirements without defining how they will be implemented, while Agile might define a solution for the current sprint but not expect it be the final solution.

Get out and ask a customer. 

If possible, get out an investigate the market place.  How have your competitors or customers solved this question in the past? For a more lateral approach, identify a related industry and look at how they solved issues. Remember that Nothing Interesting Happens in the Office (NIHITO).  This is a good time to collect some new ideas as well as the usual ideas.

“A desk is a dangerous place from which to view the world.”
– John le Carre

Sources of ideas

The obvious way to identify alternatives is brainstorming.  This can be the classic ‘get everyone in a room’ method, or it could be a simple chat with a colleague.  The usual brainstorming is to be completely open, or it can be bounded towards the alternatives required.  Both methods have merit, but I prefer everyone facing the same direction.

One way of doing this is SCAMPER which prompts people to alter different aspects of a product.  SCAMPER is a mnemonic that stands for:

(Brainstorming is such an interesting  and detailed topic that I’ll leave it to another blog post)

Your team and colleagues are naturally a good source of ideas for alternatives, and this is also a good time to share decision making with the team.  Ensure you get their ideas and input will help them feel like they are part of the process.  It is also a good coaching opportunity to involve newer or junior team members.

Larger projects will now start their process of generating buy-in.  Start early.

This is a good time to start any engagement and, if necessary, pre-meetings.  These are less-formal meetings where you socialise an issue and build consensus among your stakeholders and team.  Identifying alternatives is a good place to start involving the right people at an early stage.  The challenge is working out who are they.  You may get someone from the shop floor who may give great insight into what might go wrong, or they just become baggage by sticking to the “ways we do thing around here.”  Start with the RACI list you generated in the last section.

If it is for large decision making or large corporates, then ensure that all the alternatives are writen down.  Not only does this help formalise and share the alternatives, some extra thinking goes into writing things down.  Your brain needs to choose words that represent your thoughts and it is only when the ink hits the page do you discover whether you are using the right words.  This is just like those times you have to explain something to a bunch of people – it is only then when you discover you don’t really understand the topic.

Hopefully now we have more than options A, B, and C.  There are lots of alternatives out there.  Some of them are right in front of you, and some may need some new innovation.  After we consider the above, we should have D, E and F now. The next step will be evaluating those alternatives.

Have you got any suggestions in ways to generate alternatives? Please feel free to comment below to add to the discussion.

Go back & read part 3 on defining the people in the problem  or go to part 5 on evaluating alternatives

Steve is a Product Development Manager at Telstra Wholesale.  The views expressed in this post are his only and do not necessarily reflect the views of Telstra.

This article continues the discussion from Product Anonymous back in June.  Full credit goes to the team and the attendees for providing key steps, insight and critical analysis.

In the last post we talked about defining the problem, and the clarity that comes with it.  We never really talked about the people angle.  Until now.

Let us now consider the people element when making the decision; also known as engagement.

This is probably not a big issue for a small decision or small company – you can probably skip the rest of this post.  But once you get to a medium to large organisations there are usually a large amount of people you need to convince to get something decided and implemented. Engagement becomes a necessary task to overcome the role specialisation within larger organisations.

People often shy away from engagement.  They see it as too hard, takes too long, or is just ‘airy fairy’.  Or perhap, like John Paul Satre wrote in Huis-clos, they believe ‘Hell is other people’.  Yet careful engagement is an early investment that can pay off in the longer term.

In his book ‘Extreme Programming Explained: Embrace Change’, Kent Beck wrote “no matter what the client says the problem is, it is always a people problem.”  For large organisations and large decisions this phase is a good time to think about those people, acknowledge them as important, and work out how you can get them involved and onboard quickly and easily, so that future decision-making flows smoothly.

There is a fine line between gathering consensus and playing a political game.  But if you consider that you are doing this for the decision and not for yourself then you are on the right path.

Who is going to make the decision?

Who are the right people to be involved?  Are the right people making the decision? Product managers tend to think the world revolves around their decisions, but perhaps this is a question for someone else to answer.  Perhaps this is not really in your hands.

The classic tool for clarifying roles and responsibilities is the RACI matrix (and used by most attendees of Product Anonymous). This tool is a way to work through your stakeholders and identify where they stand.

The acronym stands for

• Responsible: Those who do the actual work to achieve the task.
• Accountable: The one ultimately answerable for the correct and thorough completion of the deliverable or task.  There must be only one accountable specified for each task or deliverable.
• Consulted: Those whose opinions are sought, typically subject matter experts.
• Informed: Those who are kept up-to-date on progress.

(Refer to http://en.wikipedia.org/wiki/RACI_matrix)

The process in theory is pretty simple: Identify each stakeholder in your project and assign them a letter from R, A, C, or I.  Then work through the list and ensure that each stakeholder is being treated as required.

But it gets complicated in practice:

• Responsible and Accountable can be easily confused, so keep a careful eye on their meanings at first. Remember there can only be one Accountable person.
• The Accountable person may be your boss, the project sponsor or you.  Depending on the magnitude of the outcome it tends to go further up the chain.  Don’t escalate everything if you can avoid it.
• Almost everyone believes they should be Consulted.  The reality is they don’t need to be, and they will slow you down and cause you extra work.  Take a calculated risk and mark as many people as Informed as possible instead of Consulted.
• Is your customer / end user in the Consulted list?
• Keep it as simple as possible, with the smallest group possible.
• If the stakeholders seem to have multiple roles in the matrix, then perhaps you have a multi-part question.  You may need to break the question into sub-questions.

The RACI matrix is a simple tool that can be quite useful to bring clarity to the roles of the stakeholders.  But remember it is not a substitute for a plan.  It is only a communication tool that helps with engagement – You still need to go out and talk to the stakeholders.

We know it is decision time, we have the question defined, an understanding of the problem, we know how to use the outcome, identified stakeholders and we even a decision maker who will get us there.  Next we’ll look at what alternatives are available…

Have you got any other tools to ensure you have the right people and a solid decision maker? Please feel free to comment below to add to the discussion.

Go back & read part 2 on defining the problem or go forward to read part 4 on identifying alternatives

Steve is a Product Development Manager at Telstra Wholesale.  The views expressed in this post are his only and do not necessarily reflect the views of Telstra.

Now we have had the overview of the six step process, let’s get into the detail

The first step in decision making is defining the problem.  What is on the table?

The simple example of defining the problem is when we are presented with two or more alternatives. For example, “Do we follow option X or option Y?”  Or perhaps you have received a specific question like “Should we commit resources to this?”

This is rare and clinical – the real world is much more fuzzy.

There are usually many decision situations in every discussion. For example:

• Have you have noticed a departure from the vision, or an assumption you disagree with?
• Is the discussion going around and around without reaching any conclusion?
• Has someone has come along and demanded “we need to do X”?
• Is the time-poor product manager now feeling a bit cranky about something and they can’t lay their finger on it?

It is now a decision situation.

But what is the decision about? What is the real question or issue?

It turns out that defining the exact problem you’re trying to solve can be difficult by itself. Often there is a deeper issue that hasn’t been drawn out. If you’re finding that the questions and answers are going in circles without any resolution, then you probably don’t understand the problem and some research is required.

A sure fire way to realise this is happening is when you are getting frustrated by lack of progress and competing priorities.

One tool to identify the true issue is the ‘5 Whys‘.

This technique is to ask why five times to get to the true nature of the problem. Traditionally this is a linear exploration of the cause; A was caused by B, which was caused by C, D and then E. But usually I found that there are many causes at each level, and you can develop a bit of a tree structure of causation – sometimes displayed in a fishbone diagram. Either way the tool is used to step back and look at how we got here, and can help identify what may be the real issue. Or at least the issue with the biggest impact.

Some interesting reading on the 5 Whys methodology:
http://www.mindtools.com/pages/article/newTMC_5W.htm
http://www.isixsigma.com/tools-templates/cause-effect/determine-root-cause-5-whys/
http://en.wikipedia.org/wiki/5_Whys

A related issue occurs when you get solutions thrown at you instead of questions: “We must do X!”  While we should be encouraging people to suggest solutions instead of just problems, we have to be careful. That solution might not be the only option, nor the best. Further probing and understanding of the actual issue and decision will help you find there are more alternatives, and some may be better than the initial one proposed.  We can use the 5 Whys again to get to the bottom of the issue, though often simple open ended questions can be enough.

Overall the 5 Whys approach is something we should always be utilising.  Question everything if you can; the strategy, what is best for the product and why some techniques just seem to work better than others.  After all, the product manager should always be (internally) questioning whether they are doing the right thing.

Another tool is to reverse engineer the question from the answer

A clear question should have clear options that lead to clear outcomes.  Therefore you should be able to examine the outcomes to see if they are indeed clear, and whether they correlate to your original question.

Let’s take a simplified example; you have to decide whether or not to close a factory to save money.  The outcome would be that you might save money but the employees would be out of work.  Is your original decision about closing the factory or is it about putting employees out of work?  Would you feel comfortable that a financial decision is the only considered point that leads to this outcome?

Do a first pass on your decision question and some possible options.  Choose each option and be very clear about what the expected outcome would be.  Review all the outcomes, and reassess whether the outcomes really match the original decision.

Note that if you don’t know how the outcome will be used, then you are not addressing the decision correctly, and will be making a decision in a vacuum. 

This reverse engineering of the question will help ensure that the decision making is actionable in the right way. It can be used through the whole process to ensure you are on track.

Have you got any other tools to ensure you have a solid decision question? Please feel free to comment below to add to the discussion.

Next we’ll talk about the role people play in the decision process in part 3 or go back to part 1 to cover the process

Steve is a Product Development Manager at Telstra Wholesale.  The views expressed in this post are his only and do not necessarily reflect the views of Telstra.