I recently had a request asking for help to design a win/loss analysis interview. As this topic has general applicability across industries, I thought I’d share some of that exchange with you, dear readers.

Question from PM: “My colleagues and I here are looking for a good but brief 15 minute interview guide for win/loss analysis.”

My response (AA): Why 15 minutes? I generally find it takes 30 minutes to get to the real answers. Also what is the ASP for your product? If you work large deals, I again question the 15 minute limit. Are you planning to do a single interview per sales opportunity? How many stakeholders do you generally have in a sale?

PM: 15 minutes is what we were asked to limit ourselves to by the Sales rep I believe.  I agree with your point that more would be better.  We can try for that.

AA: If all of my earlier assumptions are correct, then a 15 minute interview could actually be worse than no interview at all. It takes most interviewees about 10-15 minutes to warm up to the interviewer and start to open up. I spend the first 15 minutes helping the interviewee get comfortable with me by asking some feel-good questions such as their own professional background, the story leading up to the consideration, and so on. This also allows me to understand the real drivers of the decision as opposed to the final conclusions.  I don’t ask my high-value questions (about final conclusions) until I reach that tipping point of trust and openness with the interviewee. So if you start with your high-value questions right off the bat, you are going to get the defensive answers, not the open answers. Also you will not have any way to question the answers you get because you haven’t studied the underlying drivers.

So in 15 minutes, you will get the same answers that the sales rep already knows. You’d be better off interviewing the sales rep and recognizing that it’s sales’ opinion. If you interview the client for 15 minutes, it becomes the client’s answer when it’s really just the surface answer. That’s why it’s worse to do the interview – it just legitimizes the surface answers and gives them more weight. The sales rep will likely say (to you directly or to themselves), “you see, I knew all that. That’s why I don’t think you needed to do that interview, and that’s why I’m only ever going to give you 15 minutes with *my* customers.

Related posts:

1. What’s the deal with Win/Loss Analysis?
2. How NOT to do Win/Loss Analysis part 1: CRM Reporting
3. Win/Loss Analysis: What to do if you’re not allowed to call customers
4. Win/Loss Analysis MUST include wins (success stories don’t count)

by Saeed Khan

In my post – Your most important customer is your sales channel – I wrote about the challenges, particularly in multi-product companies, of ensuring your sales channels are really doing the best job they can in selling your product.

Sales team (both direct and indirect) have their sales targets and with the exception of teams dedicated to a single product, usually have few or no restrictions WRT how they must achieve their targets. i.e. they rarely have specific subtargets for individual products.

So the question arises:

How can you ensure your sales teams will focus on and sell your product, when they have choices on the products they sell and how they reach their sales targets?

To answer this question, I go back to what the VP of Sales told me in my very first job as a Product Manager, when I asked him about a problem I was having with the sales team. He looked at me and said:

Saeed, always remember that in the end, Sales people are “coin operated”.

I’ve never forgotten those words. At first though, his advice sounds somewhat denigrating. But how I’ve always interpreted it is that their behaviour is determined by how they are measured and compensated. i.e on meeting (and ideally exceeding) their sales targets. i.e. how your product fits the channel they manage.

Achieving Product/Channel fit

There was a time, when your sales teams sold your product. Yes, they needed sales training. Yes, they called in Product Management when particularly difficult customers needed soothing. Yes, there were the fair share of challenges and lost deals, but overall, the sales teams saw value in selling your product. You had product/channel fit.

But how exactly did you get that fit, back then? It was probably a combination of things such as:

• a new product that addressed a clear customer/market pain point
• clear positioning and messaging
• few directly competitive products
• corporate focus on the market segment the product addressed
• minimal price pressure from customers or competitors
• better overall economic conditions
• and/or it was the only (or main) product in your company

In short, you understood the market, understood the needs of the sales teams, the sales team were trained and not only understood the value proposition of the product, but also saw it as a means to achieve their sales targets.

And that last point is critical, because it means that your sales teams (or more likely a significant portion of them) will focus efforts on selling it.

Losing Product/Channel Fit

Achieving Product/Channel fit takes time and effort. But losing it can happen very quickly and suddenly. Your product may be delivering on it’s numbers for several quarters and everything looks good. But then you have a mediocre quarter, nothing disastrous, but maybe your product came in just below your quarterly target.

But no worry, the sales funnel looks strong, a couple of deals just slipped out a month or two. There are plenty of healthy prospects in the pipe and this quarter will make up for the small shortfall of the last. But as the quarter progresses, deals aren’t closing as fast as you’d expect, more deals are slipping out or not progressing, and before you know it, the quarter that should have made up for previous losses is a disaster.

This is not simply a bad quarter, but in all likelihood, you’ve lost Product/Channel fit!

Why? Something changed within the last few quarters that caused your sales decline. It probably started before your first weak quarter, but given previous history of good results, you didn’t notice it. And then WHAM!, that disaster quarter hit. And now as you look ahead, the next couple of quarters look very much at risk.

And just to be clear, this is not typically caused by product quality or defects. Those would be noticed very quickly based on your customer support cases. And this is not a case of a poor prospect funnel. You would have noticed that very quickly as well. Nope, from those two perspectives (product quality and lead generation), the world looks rosy.

So what are some reasons for loss of Product/Channel fit? In short, Sales teams have something better to sell and aren’t focusing on your product. e.g.

• Your company released a new version of a big product and all the corporate (and sales) mindshare is focusing on that
• There’s a mega-trend happening in the market (e.g. Cloud, Mobile, Social etc.) and your product doesn’t fit well in one of those, so it’s not considered “sexy”
• Your company acquired a hot company a few quarters back, and now that the sales teams are all ramped up on its products, their focus is there and not on your product
• While customer demand for your product is still strong, the market segment it’s in is commoditizing. The increased price pressure is a big disincentive for your sales teams
• While there is demand for your product from potential users, the value proposition is no longer considered strong enough by management or the buyers and your sales teams are focusing on other “higher value” products
• Your startup was bought by a larger company (hooray!), but now, your product is just one of a portfolio of products and once the honeymoon ended, the large sales force of your new employer left you for the next “big thing”
• Your channel partners are now getting much better margins from an upstart competitor who suddenly seems to be getting traction

There are lots of reasons for losing product/channel fit, but in most cases, you can, very quickly do the research to understand where the problems lie and propose remedies.

Regaining Product/Channel fit

Once you realize you’ve lot the fit, the step is to find out exactly what happened. And the best people to talk to are people in your sales teams. But do some analysis first. Look to see who is having success with your product. Talk to them first, and understand why. Are they just better informed about your product? Do they position it better? Are they selling it in conjunction with other products? Are there specific verticals or geographies or usage scenarios that help them succeed?

Learn their secrets first, so that you are well armed with success stories, and then talk to those who are struggling with or are not selling your product at all. Talk to them and find out why. For those struggling, is it an education issue, or is it something related to territory? Or are the simply not very good sales people?

And for those who are not selling your product, find out why? Is it not worth their time, or do they not understand it? Maybe they are newer to the company and were not part of the sales training you did for your product.

Next do the same with some channel partners if you sell with an indirect model. Their reasons for success or lack of it may be different than those of your direct sales teams.

Finally, if you don’t do it already, (and many of us don’t), conduct some Win/Loss analysis on customers from the last few quarters. It’s important to do both Win and Loss (and not just Loss) because reasons for winning are not always as cut and dried as you might expect. i.e. it’s not always because the product is fabulous and it delivers tremendous value and will help the customer be successful.

I guarantee that you’ll learn as much from the Win discussions as you will from the Loss discussions.

As you speak to these people, make sure you probe into specific areas such as pricing, positioning, value proposition, economic and budget realities etc. Look for patterns across the discussions. Pull out the little nuggets of insight that can help re-position or re-price or re-define the value proposition.

Once you’ve collected the data and identified the root cause(s), devise a plan to remedy the situation.

If, for example, a reason for decline in your product sales is due to the sales teams focusing on a new (released or acquired) major product, see if there an attach strategy you can use to ride the train of that major product and be pulled along as part of the deal. i.e. show them how they can “upsize” their deals by adding your product in.

Maintaining Product/Channel fit

Once you’ve regained Product/Channel fit, put some on going activities into place to ensure it doesn’t happen again.

Look at the activities you performed to regain the fit — the conversations you had, the analysis you did etc. Maintaining Product/Channel fit is simply taking those activities and performing them continuously, good quarter or bad.

Here are some suggestions to follow. Every quarter:

• Regularly validate and revalidate that the sales teams understand the high-value usage scenarios of your product, are positioning it properly, and are speaking to the right people within the prospects’ organizations.
• Around mid-quarter, contact a few sales reps with opportunities that will likely close that quarter and have a short conversation about the opportunities. Listen to them describe the prospects, the use cases, how they positioned the product etc. and ask them about any challenges they see in closing the business that quarter.
• If your company holds QBRs (Quarter Business Reviews), make sure you attend them. Try to get on the agenda (even 15-30 minutes is a helpful) to discuss your product, opportunities, challenges etc. That face-time with the sales teams is invaluable
• Monitor the “No Opp” ratio of opportunities of your product in your SFA system. This is the ratio of valid opportunities that eventually are  changed to No Opportunity (or equivalent) by the sales team. If the ratio is growing or is significantly larger than other products, you may have a problem
• Monitor the overall time to close for opportunities. i.e. the time from when the opportunity was verified/qualified until it closed (Win/Loss/No Opp). If you see anomalies compared to other products, investigate and find the root cause
• Keep in regular contact with Sales Consultants and listen to their input about your product. SCs will give very honest and valuable feedback about why your product is not succeeding.

There are many other activities you could do to maintain Product/Channel fit. The objective though is to have an ongoing communication process with your sales channels, identify potential problems early and enact remedies when the problems are small.

To paraphrase a famous quote — continuous vigilance is the price of product success.

Saeed

P.S. As much as things like product features, requirements, working with Engineering etc. are the center of a lot of discussion on what Product Managers do, it’s this kind of work — ensuring product alignment with sales channels  via pricing and win/loss analysis, repositioning  etc. — that  is where PMs can deliver the significant value and impact product success the most.

Tweet this: How to achieve, lose, regain and maintain Product/Channel fit. http://wp.me/pXBON-2PG #prodmgmt #sales #leadership – click here

Related posts:

1. Your most important customer is your sales channel
2. Guest Post: Managing Channel Conflict through a Derivative Strategy
3. How to LOSE customers!
4. The Origins of Product Management (part 1)

You’ve just introduced your latest product release, introduced a new product or capability or launched the product. Now What? What about a retrospective? While product management and product marketing professionals often see engineering and development using retrospectives, how can you use this method to look back at what happened, while positively influencing the future?

What is a Retrospective?
A retrospective thrives in organizations using SCRUM methods and is a meeting where the team discusses the Sprint that just concluded and determines what could change that might improve the next Sprint.

In SCRUM terminology, anything that affects how the team builds software is open for debate. This could include processes, practices, communication, collaboration, environment, artifacts and tools. I believe and have experienced that a retrospective for product management and product marketing will provide clarity to future activities. Why?

Why a Retrospective?
Retrospectives bring insight and clarity. While it originated to bring development teams together, product management and product marketing need similar insight and collaborative review. While we have many one-on-one conversations and numerous meetings with stakeholders, product management and product marketing benefit from looking back to improve what’s ahead. Using retrospectives, product professionals will have insight into:

• What’s happened and what’s currently happening?
• What current activities have made the most impact?
• What processes and best practices have worked and what needs improvement?
• How is communication and what can be better?

Product Management and Product Marketing contribute and bring clarity to:

• What’s changed or influencing our market?
• What stories can you tell us that will validate where we are going?
• What’s the competition doing and will it influence what’s planned?
• Has anything shifted or influenced our strategy?
• What’s the status of the product roadmap?
• How are customer using our products and what have they experienced?
• What’s been developed or delivered from the product marketing roadmap and how will it influence the channel?
• What’s the timing or status of the next product launch?
• How does our positioning resonate with customers and non-customers?
• What has product management learned since our last retrospective?
• What has product marketing experienced since we last met?

Implementing a Retrospective
At best, retrospectives are a great tool for infusing continuous improvement and collaboration, Depending on who you collaborate with most often, and the focus and role of your position, the following five areas should be considered when leading a retrospective.

• Goals – in advance of the meeting, let everyone know about the focus of the retrospective, what you like to achieve and ask for input, topics and invite other to bring their accomplishments and war stories.
• Set the Stage -let everyone know you’re hear to share, learn, listen and provide constructive feedback, as well as receive it. Look for ways to improve relationships.
• Gather Data – before you get together, prepare current information and data that would be relevant. Ask other to share 2-3 minutes worth of information.
• Generate Insight - with discussions flowing, start asking the why questions that will generate insight, support and consensus
• Decide What to Do – while this isn’t a planning meeting, capture the information, validation and feedback. If you have access to a coach or facilitor, ask them to lead the session, so you can really engage and not worry about meeting management.

To stage better product delivery and introduction, I believe you should try a retrospective. If you’ve tried a retrospective, please share your experience. If you haven’t experienced a product management or product marketing led retrospective, give it a try and let us all know how it goes.

If you like the post, please share it via LinkedIn and Twitter – New post and idea, “the Product Management Retrospective” http://wp.me/pXBON-2DO #prodmgmt #prodmktg #agile #leadership

Related posts:

1. Agile/Scrum Software Development and Product Management part 2
2. Gilbert and Sullivan present: The High-Tech Product Manager
3. Product Management: Beware of Flying Monkeys
4. Agile/Scrum Software Development and Product Management

When I say “positioning”, what do you think of?

Although tech companies have moved beyond marketing features and functions, many of them still talk mostly about the product. I spend my days interviewing buyers and analyzing how they make purchasing decisions, and I can tell you this: the product is only a part of the overall consideration. You will be more successful if you take your positioning beyond the product.

When your buyers are looking at products, they think very little about product features. They think mostly about their goals and problems, and how you can help them. They think about the risk of failure and the probability of success. When they check your references, they are not just asking about whether your product works as advertised. They want to talk with people who have been in their shoes and been successful because of a relationship with your company.

Buying a significant product is a little like signing up for a long-term relationship. When you are dating, you might at first be excited about a person’s looks, earning potential, or other aspects of their person. But as you get to know them and consider a deeper relationship (when you “buy the product”), you start to think about what your life will be like with that person. Then it’s less about their features and more about your life with them as a part.

Significant product purchases are similar. Your buyers think about what their lives will be like with you as a big part. Will you respect them? Do you have the expertise to help them when needed? Are you attentive? Can they be happier with you in their lives than with your competitor in their lives?

As you talk with your potential buyers and existing customers, listen for cues about the relationship. Dig below product features. When a feature is mentioned, ask what impact that feature has on their jobs, on whether they achieve their objectives, and even on their general happiness.

Your buyers are more emotional than you think. If you don’t think so, interview them as though you were asking them about a relationship with you. With your product. With your company.

- Alan

PS: if you liked this article, please share on your social nets: ”Positioning beyond the product: Think relationship” New from @AWArmstrong via @OnPM http://wp.me/pXBON-2Dm #prodmgmt #marketing #prodmktg

Related posts:

1. Surprise us: Seeking examples of SURPRISING positioning statements
2. Does sales really “own” the customer relationship?
3. Obama / McCain debate: long on features, short on positioning
4. Worth repeating: What makes a relationship resilient?

by Saeed Khan

I recently wrote a series of articles on metrics to help track product success. The base article can be found here – A Model and Metrics for Tracking Product Success. That article also has links near the bottom to additional articles that break out additional details. There were 4 areas of focus for the metrics:

• Business
• Organizational
• Go-to-Market
• Product

This article focuses on how to apply that type of metrics model for an important (but not very common) product management activity, the mid-year review.

What is a Mid-Year Product Review

I’ve worked in a number of software companies, and unfortunately, in most of them Product Management was immature or underdeveloped. By that I mean I was either the first Product Manager, or when not the first, worked in a very small team where the focus of Product Management was predominantly technical in nature. i.e. very little connection with the business, organizational or marketing sides of the company.

As described in the metrics articles referenced above, Product Management is about the business success of the product, and not just requirements and releases, so it’s important that Product Management teams have a way to remain focused on the business of the product.

In most companies I’ve worked, there was some sort of annual planning process, that was fairly holistic in nature. i.e. revenue targets were set, budgets were allocated, high level objectives for various departments defined (e.g. marketing, PR, sales etc.) and agreement found on the overall plan.

This was all good, except that, with the exception of revenue and budget (i.e. expense) calculations, there was rarely a reassessment or holistic status review of that plan, until about 9 months later, when the next year’s plan was being formulated. People did review things such as lead generation programs, website content etc., but only in a very tactical, siloed manner.

What was missing was a way to look at all the various pieces together, as they are interdependent, to assess how well or how poorly the business was, and what adjustments or changes would be needed for the 2nd half of the year.

This is where the metrics model becomes valuable, because it provides a standard and honest way of looking at the state of the product and the factors affecting it’s success. And from there, corrective actions can be taken where necessary to focus efforts for the second half of the year.

In most cases, those actions will not be new or revised product plans — it’s very difficult in most cases to make quick but high revenue impact changes to products — but will more than likely be related to the Go-To-Market or Organizational areas. In some cases, those changes could be related to pricing and licensing. This is one product area that can have significant business impact if done correctly. i.e. a realignment of how the product is priced/licensed can show results almost immediately.

How to conduct the review

The objective of the review is to provide a clear picture of the product state and plans to your management or executive team.  Thus, aside from needed members of your management team, the participants should be members of the Product Management, Engineering, Product Marketing and Sales teams who can directly represent the key departments or teams that have direct influence on the product and it’s objectives.

The review should take 1-2 hours depending on the complexity of the product and the amount of discussion needed.  It could take longer if you have a large or complex product situation, but for most products 1-2 hours should be the needed time.

There’s a lot you can do to prepare for the meeting and I won’t go into that in detail as it varies with the situation and product, but clearly analysis of sales funnels, marketing funnels, win/loss analysis, product quality, support case trends  etc. will likely be part of the analysis. The objective of the analysis is to get a clear picture of what is and what isn’t working with the product and how well is it on it’s way to achieve it’s goals for the year.

The review should follow a structure like that listed below. Obviously you need to adapt it to your situation but I’ve seen this format work in real-life.

1. Objectives of the meeting
   • i.e. review state of product and define action items for next 6 months
2. Review of overall annual objectives
   • i.e. revenue goals(by quarter), product plans (by release) plans, internal/external enablement plans , marketing activities, product marketing objectives etc.)
   • i.e. the 4 areas of metrics
3. Current status of annual plans
   • i.e. where the activities are today WRT the plan. Breakout activities completed, in progress, not started, and in particular show and explain why some activities have not not met their plan.NOTE: I have yet to see any product that meets all of the objectives set for it even 6 months in advance. This is simply a reality of the world we live in. Plans are great, but usually are not met, particularly those that span functional groups across periods of time.
   • Use the dashboard model defined in these articles as a starting point for discussions with management. This is a good level to start the discussion, giving them an overview, but make sure you have supporting details on any contentious areas to support your assessment. i.e. if you believe a certain metrics is positive or a category is doing well, be sure you can show why that is the case.
4. For areas behind plan or areas of concern in the coming 6 months, list reasons and recommended actions for correction
   • For areas behind plan, the reason could be due to any number of things — e.g. hiring plan is behind, change in focus, internal reorganization etc. It’s impossible to list all of the possibilities here, but you should identify why you believe targets weren’t met so that an open discussion can happen
   • For areas of concern in the coming 6 months, list why they are of concern. e.g. low sales funnel and upcoming summer slowdown makes it difficult to achieve Q3 target.  The goal here is to try to take pre-emptive action based on what you know today, to address issues in the coming month.
5. Document key action items coming out of the discussions
   • The bulk of the discussions will be about areas of concern and will generate action items for various team members. Document these items, but make sure they are worded in something measurable — i.e.  a “what by when” statement as opposed to something more abstract. Avoid generic words like “improve” and “increase” – Improve what by when, increase by how much and by when? i.e. it’s clear that doing them in a specified timeframe will have a demonstrable positive impact on the problems being addressed.
   • Finally track these action items in regular (weekly or biweekly team meetings — i.e. with other PMs, PMMs etc.).
   • You don’t want to meet again in 6 months and have the same discussion with your management team.

Does your company conduct these kinds of product reviews? If so, how are they run? What do you do that works? I’d love to hear from you.

Saeed

Tweet this: Why and How to Hold a Mid-Year Product Review http://wp.me/pXBON-2BD #prodmgmt #prodmktg #marketing #leadership

Related posts:

1. Go-to-Market and Organizational Metrics for Product Success
2. Top 7 posts from the past year
3. Defining Business-Oriented Metrics for Tracking Product Success
4. A Model and Metrics for Tracking Product Success

By Alan Armstrong

This week had a very smart CEO pose a great question to me:

What are we missing in terms of opportunities where we aren’t even considered? Frankly I worry (like all paranoid startup CEOs) that this may be the biggest opportunity for growth.  Do you have any hypothesis on how to dig in to that category of clients that have not heard of us?

One of the problems with Win/Loss Analysis is the focus on accounts where your company is already engaged. Can this analysis help you find untapped markets, or accounts where you are not being considered?

This is a crucial question for anyone concerned with product or corporate revenue – investors, product managers, sales VPs, marketing VPs.

I give my answer below, but first, here is a quote from Donald Rumsfeld that explains the problem more fully. Rumsfeld took a lot of heat for this quote, and perhaps it was overly obtuse or intellectual for the press. But if you listen to what he says, it’s remarkably insightful:

In marketing / sales terms, here’s what Rumsfeld is describing:

1. Known knowns: These are accounts that you know how to go after. This is your sweet spot in the market. Hopefully you have one or more of these!
2. Known unknowns: If you are paying close attention to your company’s deal flow, there will often be deals that are outside of your sweet spot. Even by accident, your customers will use your product in ways that you hadn’t imagined. You have to be paying close attention to notice these, but when you see one, you should take note of it and dig in to their business need, how they found you, and whether they need any additional capabilities. These accounts present a potential opportunity to expand your market. When you profile these accounts, pay very close attention. Once you notice them, you can use their profile to size the market, and if it appears attractive enough, create a pilot marketing/sales program to go after a short-list of these accounts.
3. Unknown unknowns: These are the potential opportunities that you and your customers haven’t thought about yet. Are these potential accounts buying your competitors’ products? Perhaps your competitors have discovered them before you, or maybe they are just struggling with a problem that you could solve, but no one has put two and two together.

Back to the CEO’s question: How do we find the Unknown unknowns, the untapped markets?

I call these markets “hidden market segments”. So how do we find these hidden segments? Here are my suggestions:

Pay attention to patterns in your deal flow

One possible difference between #2 and #3 above is that in #2, you are paying attention, and you notice the anomalies in your deal flow. It takes an intuitive person to notice such anomalies. But it should be a priority for everyone paying attention to the deals. If you need to get started, Win/Loss analysis is a great technique. Unfortunately many people don’t do a good job of win/loss analysis. (I wrote about some bad techniques earlier. Steve Johnson covers his views on Win/Loss here. ) You need to ensure that you are studying the whole buying process, speaking directly to buyers, not accepting the reasons given by sales. Too often companies note just the final reason for a win or a loss, but you need to avoid that temptation and look at the whole buying process.

What about the Unknown Unknowns?

The problem with the bucket #3, customers that you don’t know about, is a bit circular! Philosophers would call this an episemological problem … it is an unknowable thing.

While it is difficult, much of the difficulty can be solved in two ways:

1. Turn #3 into #2 by consistently profiling your wins and your losses. See above. (If you need help, contact me. I do this for a living.)
2. Look for hunches: If your deal flow is too large to profile consistently without huge sampling bias, start by interviewing people internally. Look around for hunches. Frame the question as I have done above. Start by profiling your sweet spot, and find some really smart, market/customer-centric people inside your company. Explain to them that you are looking for people outside your sweet spot. Once you find some potential anomalies, start your Win/Loss analysis with accounts similar to the anomalies. With 5 wins and 5 losses, you’ll have a good start.
3. Look at your competitors successes: Are they winning outside your sweet spot? Are they attending tradeshows or hosting events for use cases that you hadn’t considered? Can you find lost accounts in your past that represent cases like these?
4. Brainstorm and Sell: What are the potential uses for your product in markets where you’ve never (knowingly) won a deal? Can you put together a pitch deck identify 5-10 sample accounts, and go after them with a SWAT team? You need BD-type people, not sales-types for these markets because you are breaking new ground.

In the end, it all boils down to finding a new profile of customer. You need detective skills, market knowledge, buyer knowledge, and sometimes good luck, but it can be done.

- Alan

Related posts:

1. What’s the deal with Win/Loss Analysis?
2. How NOT to do Win/Loss Analysis part 1: CRM Reporting
3. Win/Loss Analysis: What to do if you’re not allowed to call customers
4. Win/Loss Analysis MUST include wins (success stories don’t count)

BY ALAN ARMSTRONG

Teams requires both a laboratory, to innovate and design, and a factory, to build and deliver. Unfortunately teams, companies, and individuals spend too much time in the lab or the factory, and often confuse the these two very different environments. In doing so, we frustrate ourselves and fail to capitalize on significant opportunities.

A tale of two teams

I recently interviewed two VPs of Operations for major US wireless carriers. They provided very different outlooks on situations that are in many ways nearly identical. Both had sprawling infrastructure as a result of decades of acquisitions, divestments, new product builds, and so on. And both had been recently appointed to get things under control.

The first VP, Edgar, expressed serious frustration and fatigue. He described a team fighting fires and always behind schedule. But what felt most overwhelming they were not moving forward fast enough to catch up. The team was also stressed out, and sick leave had increased. According to Edgar this was just making things worse – with fewer employees at their desks, they were getting more and more behind.

The second VP, Jack, gave a distinctly different perspective. Jack described the current state of measuring, learning, and observing; “We are learning about the environment and how it behaves.” Soon, he said, they would start experimenting with some improvements.

Turning the factory into a lab

Jack had turned his factory into a laboratory, where the team’s job was not just to keep the lights on, but to invent new processes, techniques, organizations, and technologies to make things run more smoothly. If he viewed that operation as a factory, it would be judged a huge failure. But viewed as a lab, it can be seen as a place full of upside, with opportunity for a lot of creative thinking, growth, and empowerment. If the boss things that way, it flows to the team as well.

The lab vs. the factory

The lab has very different kinds of thinking than the factory. In the lab, we set goals and make hypotheses, but no one pretends to be able to predict the outcome or the timelines with great certainty. When we release that need for certainty, we open ourselves up to experiment. It’s called trial and error for a reason; we expect errors to occur, and that is in fact how we learn and ultimately how we create something new.

Once the mindset is changed, the team starts to have fun again. We realize that we’re trying something without a predictable outcome, and so we just do our best, make our best attempts, debate hypotheses, design tests, but everyone realizes that we are in fact experimenting. Everyone relaxes a little.

Managing external expectations

If any leader decides to change modes from factory to laboratory, it is critical that the leader manages external expectations. If we are working in the lab but management or other stakeholders are expecting our usual factory delivery, they will be frustrated and disappointed. Missing expectations is a serious career limiter for any leader.

The leader’s job is to reframe the activity and persuade external stakeholders to change mindsets. If persuasion is not your game, you can also give them a choice: “We can use our factory and produce X, but if you really want Y, we need to use the lab, and that will be more expensive and take longer. We can’t guarantee success, but here is what we think we can do, and we believe you’ll be very pleased.”

Nail it. Then scale it.

“Nail it. Then scale it.” is my company’s tagline, but it’s more than just a marketing slogan. I see it as an approach to many tasks and situations. I coach my clients, employees, and friends (and myself!), to apply this philosophy many areas of work life, and even some areas of personal life. If you are trying to produce something at scale without understanding the basic mechanisms and tasks, you will suffer great frustration and likely fail. It takes patience to recognize areas that need to be ironed out before they can be successfully repeated over and over. You have to nail it before you can scale it. When we are in the lab, we our job is to nail it. It is a common and often disastrous mistake to try to scale it before we nail it.

Step 1: Become conscious of the difference between factory and lab

The first step is to become conscious of the difference between the factory and the lab. I encourage you to ask questions like:

• What does this project or task require: factory thinking or lab thinking? Or perhaps a combination? What combination?
• Am I creating a burden for me and my team by expecting a factory, when I really need to create space for a lab?
• Does my drive to “be creative” cause me to treat everything like a lab, even with tasks that need to become routinized and automated? If I treat the factory like a factory, can I open up more space for truly interesting lab work?

Stay tuned for the next instalment where I will talk more about the implications of lab and factory thinking for organizations.

- Alan

PS: If you liked this post, please share it on Twitter, LinkedIN, or Facebook!

TWEET THIS
The Factory vs The Lab: Leaders know which is which, mistakes are costly. http://bit.ly/iYm8h5 #leadership #prodmgmt #prodmktg via @ONPM

Related posts:

1. The Essential Pieces of Strategic Product Leaders
2. "Shop Talk" Q&A with PM Leaders features Alan Armstrong
3. ProductCamp: The Unconference TEDxLocal for Product Leaders?
4. 5 Right Brain Workshops for Product Leaders

by Saeed Khan

Last week, I discussed Go-to-Market and Organizational Metrics. This week I’m going to discuss Product metrics. And while the title of this post sounds somewhat repetitive, as seen in previous articles Product metrics are simply one category that need to be tracked as part of a holistic view of “the product”.

BTW, if you haven’t read the first article in this series, you can read it here: A Model and Metrics for Tracking Product Success.

As mentioned in that article, when thinking about metrics, most people think about the Product first, and for obvious reasons. It’s the focus of the business, and is the most visible and tangible outcome of everyone’s efforts.

And when thinking about Product metrics, the mind most likely turns to the common metrics related to bugs, escalations, etc.  All of these are useful, but not as primary measures for product success. Product success needs to be viewed from a higher perspective.

I look at the following list as key areas when tracking product success:

• Strategy
• Roadmap
• Capabilities
• Gaps
• Quality

Keeping abreast of these categories will provide a clear picture of the state of the product over time.

Strategy

Strategy can be a nebulous topic. And while defining product strategy is beyond the scope of this post, there are questions that can be asked to help identify whether the strategy is a good one.

Here are a few of them.

• Is the overall strategy to be market leader or fast follower or something else?
• Is there even a clear product strategy at all?
• Is it to sell into an existing customer base, or to acquire new customers?
• Is the product targeting a specific niche or problem space?
• Have changing market conditions (e.g. new entrants in the market, price pressures, regulatory changes, market awareness etc.) impacted the effectiveness of the strategy?
• What threats (if any) are known that may impact the strategy?

There are likely many other questions that should be asked, but the overall assessment of them will give an measure of how relevant and potentially effective the strategy is.

Roadmap

Ah yes, everyone’s favourite topic.  The roadmap ties directly into the product strategy. In short the roadmap is an articulation of that strategy.  A well defined strategy enables a clear roadmap.  And with a poorly defined strategy, the rule “garbage in, garbage out” takes effect. Regardless, it’s important to think through the roadmap, using the overall strategy as a guide, and assess how well that plan fit in with overall business goals and directions.

Capabilities

This is an honest assessment of what the product can do. Where are the strengths and weaknesses, and where the risks lie in those capabilities. Are there functional areas where competitors consistently win over your product or that customers/prospects say are missing?

Gaps

Gaps are called out explicitly because these are areas  that need specific additional investment. And the question really is how much investment, in what time frame, and what is the resultant business impact of making those changes? One word of caution when thinking about product gaps. It’s very easy to fixate on these above other product aspects, such as product strengths. How often have we all heard someone say, “I must have this feature.” or “Without this new feature, we won’t be able to compete.” In how many cases are those statements 100% true?

Monitor and manage product gaps, but view them as one area of focus that doesn’t disproportionately outweigh all others.

Quality

This is something that most product managers are familiar with, having sat in on bug triage sessions, been involved in customer case escalations, or simply hearing about quality issues first hand in customer conversation. Nothing irritates customers more than poor quality products, particularly ones they’ve paid for. And they won’t be shy about telling you about those problems.

This is one category where a deep analytic assessment is possible, and that should factor into how quality is measured. i.e. consider measures such as new customer reported bugs,  bugs addressed, severity of bugs reported, # of customers reporting bugs etc. All of these can be looked at over time to see if the problems are increasing or decreasing. Based on this assessment, additional investigation into specific problem areas can be conducted.

Putting it all together

As I did for other posts in this series, the following is an example of how these categories can be rendered for easy interpretation.

Other categories

Depending on the product, market and strategy, there may be other product categories to consider, so look at your product and objectives and decide what is important to measure and track.

Overall, the goal of these metrics should be to provide a holistic, yet easy to understand view of important product and business areas, with potential problems or shortcomings clearly visible for discussion and further analysis.

Saeed

Tweet this: @saeedwkhan Product related metrics for Product success http://wp.me/pXBON-2uj #prodmgmt #innovation

Related posts:

1. A Model and Metrics for Tracking Product Success
2. Defining Business-Oriented Metrics for Tracking Product Success
3. Go-to-Market and Organizational Metrics for Product Success
4. Product Management Metrics (part 2a)

BY ALAN ARMSTRONG

If you want to improve your win rates in the market, it is important that you study wins, and not just losses.

Many well intentioned leaders hear “Win/Loss Analysis”, and think immediately of “Loss Analysis”, or studying reasons why we lose. While this is of course valuable, mere “Loss Analysis” will leave you with an incomplete picture and limit your success.

OK, fine, you say, but we have success stories, so we know why we win.

Sorry, I disagree. “Success stories” generated by marketing do not qualify as Win Analysis because they are written to persuade new prospects to buy, not to study the existing account objectively.

What problem are we solving?

Let’s take a step back. Below I have listed several of the symptoms that my clients report to me. Granted, when you call a research firm, you are calling the doctor with a problem. But I have worked long enough to know that these problems are fairly pervasive, so you may recognize at least one or two:

1. Lost a few big deals: We have recently lost a few big deals that we thought we were going to win. We have no idea why. Can you find out?
2. Increased price pressure: We used to lead the market, but a competitor is really starting to beat us up. We are still winning, but we are under huge pressure to discounts. Our average sale price (ASP) is declining.
3. Decreased win rates: Never mind just the big deals! We are losing more deals, period. Our win rates have declined.
4. The Funnel with the FAT middle: Need I say more about this? We have enough leads, but they are coming in much faster than they close. Many deals seem like “non-decisions” rather than losses. I call these deals “stuck”. These deals collect like excess calories … in the middle. You call me because you want to know what’s going on. Close some, kill others, and move others back to marketing / lead nurturing.

Each of these situations is unique and calls for a different approach to diagnosis.

In the first situation, it’s fine to log a few interviews to find out what happened in the large deals that you lost. That work will certainly be valuable, you will learn a lot, and it may even be sufficient. But if you have one of symptoms 2 through 4 above, you need a different approach.

Define the “Deal Persona”

Before you start interviewing, I recommend you get clear on what kind of deal you are trying to understand. You can segment deals by size, by competitor, by geography, or by channel, to name just a few. Look at several representative deals, both winners and losers, and try to write a general description of the concerns and the questions you have. I call this the “Deal Persona”. Like buyer and user personas, this is an amalgam of deals of the sort that we are trying to understand.

For example, you may be concerned that you are underperforming in deals greater than $250k where competitor X shows up. That is a short but sufficient description. If you can flesh out some of the other aspects of deals like that (what symptoms is the field reporting back?), it will help your research.

Select the targets

Once you have the deal persona and a few representative examples, solicit the field for more deals that match the persona. You may be surprised in two ways: You may be overwhelmed with examples, or find a total scarcity. If you find a scarcity but you know that such deals exist, you may have hit on a cultural problem with the sales force; they may be afraid of sharing vulnerability, and this is another issue entirely.

Start with wins

Once you have selected the deals, I highly recommend you start by interviewing wins.

There are really two questions that you are trying to answer:

1. why do we lose as much as we do?
2. What can we do to win more?

Losses will tell you the answer to question #1, but your ultimate goal is to win more, and for that, you need to study wins also.
Wins will give you the recipe for a successful deal. For example:

1. They will tell you things you may not know about which factors were most important to them. Was it service? Was it product capabilities? Be curious!
2. They will tell you which competitive capabilities they liked, and how they decided to buy your product anyway.
3. After the deal they may reveal aspects of their negotiation strategy. (Getting insight into this area is a fine, fine art because you can’t really ask the questions directly.)

I think it’s best to start by completely understanding the winning recipe. Your wins are not 100% sold on your product; they probably liked things about they others, but they still chose you. Find out why, and use those reasons to inform your loss analysis.

In the end, you want to know how you can improve your results. If you know the winning formula, you can figure out why it didn’t work in the lost accounts.

Would love to hear your feedback and further questions on this topic.

- Alan

PS: If you liked this article, please retweet to your followers! New post from @OnPM, “Win/Loss Analysis MUST include wins (success stories don’t count)” http://bit.ly/iIMGDB #prodmgmt #prodmktg #sales

Related posts:

1. What’s the deal with Win/Loss Analysis?
2. Win/Loss Analysis: What to do if you’re not allowed to call customers
3. How NOT to do Win/Loss Analysis part 1: CRM Reporting
4. How can Win/Loss Analysis help you find untapped markets?

by Saeed Khan

Last week, I wrote about a Model and Metrics for Track Product Success.

Just to recap, there are 4 major areas I defined for tracking product metrics. These categories are:

1. Business
2. Go-to-Market
3. Organizational Readiness
4. Product

This week, I’m going to discuss how to define business-oriented metrics, and provide a simple way to track and report them.

What exactly is a “metric”?

The word “metric” is a bit of an ambiguous term. The dictionary definition doesn’t really help. The best general definition is:

n. a standard of measurement. A system of related measures that facilitates the quantification of some particular characteristic.

What we really want is a definition of a business metric.

n. A business metric is any type of measurement used to gauge some quantifiable component of a company’s performance.

Note, that even this definition covers a lot of ground. i.e. “any type of measurement”, and “some quantifiable component”.

3 rules for defining metrics

Metrics are only valuable if they either provide insight into an important aspect of your product, or are a leading indicator of potential problems ahead. Here are a few rules to help define useful metrics.

1. Define metrics to align with key goals and objectives for your product. e.g. obvious business goals relate to sales or customer acquisition.
2. Ensure that you can put programs in place to change those metrics if needed. i.e. there’s no point in tracking something if you can’t impact it in a meaningful way
3. The actual metrics will be a combination of numeric values, as well as empirical evaluations of important situations.

And now for an example

The following is an example of some key business metrics for an early stage product. For this example, it is an enterprise software product that has been on the market for about 1 year and is starting to build traction in it’s target customer base.

The status (traffic light model) and comments are all that is needed for someone to get a clear understanding of the business state of that product.

The next obvious question for the yellow and red metrics is “Why?”

For example, why is the Deal Pipeline low? Is it lack of lead generation activities, or has there been proper follow up on generated leads to convert them into opportunities? Is it a lead quality issue? Or something else?

Other common questions are “How” or “When”?

For example, for the Channels/Resellers metric, although they are currently waiting for more traction, how much more traction is needed or when will there be enough traction to get the resellers interested? And then what is the plan to enable them?

The objective is to provide some clarity for tracking, evaluation, analysis and action for the product on a regular basis. This is a core aspect of product MANAGEMENT.

Next time, I’ll look at Go-to-Market and Organizational Readiness metrics.

Let me know what you think.

Saeed

Tweet this: @saeedwkhan Defining Business-Oriented metrics for Tracking Product Success http://wp.me/pXBON-2qb #prodmgmt

Related posts:

1. A Model and Metrics for Tracking Product Success
2. Product Metrics for Product Success
3. Go-to-Market and Organizational Metrics for Product Success
4. Product Management Metrics (part 2a)