Product Management Metrics (part 1)


There’s a question that comes up regularly in Product Management circles. It’s also something I’ve thought about as well.

The question is:

What metrics can be used to measure the value and contribution of product management in a company?

It sounds like it should be an easy question to answer, but apparently it’s not.  It’s certainly not as easy as asking about metrics for sales, or marketing or even,to a lesser extent, engineering. Why is it difficult?

What is Product Management’s Mandate?

First, it goes to the issue of the definition of Product Management, and in particular technology product management. Alan asked a question in a recent blog post: What is Product Management’s mandate? There were a number of answers provided. Now, if someone asked “What is the mandate of the Sales organization?”, you know for sure there’d be little debate as to the answer.

So what is Product Management’s mandate. Here’s my answer:

Product Management’s mandate is to optimize the business at a product, product line or product portfolio level over the product lifecycle.

NOTE: This is very similar to the definition used by Steven Haines in his book, The Product Manager’s Desk Reference.

I’ve emphasized three words in the above definition: optimize, business and lifecycle.  Yes, Product Management does a lot of things, but at it’s very core, the focus on Product Management must be a business centric function.

Notice I didn’t use any of the following words in the definition: align, technology, maximize, minimize,  opportunity, marketing, sales, development, engineering, requirements or agile!

Think about it. People talk about Product Management as a cross-functional organization. What does that mean? It means that Product Management cannot function as yet-another-silo along side all the other silos in the company. Product Management plays across the other organizations in a company. Diagrammed, it might look something like this:pm-relationship1

This diagram doesn’t imply that the the 4 teams (Sales, Marketing etc.) report into Product Management, but that the responsibilities of Product Management span across those teams, impact and influence them. Another way to diagram the relationships could be like this:

pm-relationship2This again shows relationships between the groups. This is, of course, a Product Management centric view of the relationships. An analogous diagram could be drawn with, for example,  Senior Management at the center, and other teams radiating out. But the fact remains that Product Management must communicate and work with many other departments in the company, moreso than practically any other group except Sr. Management.

So, what about the actual metrics? I’ll get into that in Part 2. In the mean time, I’d like to hear your thoughts on the mandate of Product Management, as well as ideas on some metrics you believe are important to track.

Saeed

Related Posts:

If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.
44 Responses to Product Management Metrics (part 1)
  1. saeed says:

    Gotta love this job description. Sounds eerily familar…:-)

    http://www.resmatic.net/bcbsa/detail.php?id=20564

    The Product Manager is responsible for managing the life cycle of HealthNow’s commercial insurance product offerings to optimize the financial performance of these products.


  2. Product management is not an adjacent silo, but plays across all other organizations in a company http://bit.ly/3dXMaJ


  3. Product Management plays across the other organizations in a company. http://bit.ly/3dXMaJ


  4. [...] metrics. To catch up on the discussion thus far, you’ll want to review Saeed’s post on product management’s mandate and my post on two examples of key product management metrics. The lively conversation about [...]


  5. More comments on the @onpm blog entry, "Product Management Metrics part 1". http://bit.ly/1afps1 #prodmgmt

  6. davidwlocke says:

    On metrics, products are the means for advancing the adoption of an underlying technology. In an appropriately organized firm, the product and the organization are one and the same, so those product management metrics would be the same as those of the management of the firm.

  7. davidwlocke says:

    In keeping with Saeed’s desire to keep this about metrics, why do you think measuring everyone on the same metric would work? I really can’t answer unless you state what metric you have in mind.

    But, no, your product contributors have other things to do like contribute to other products and to their functional units. Using a single metric would just be a way for the PM to evade their responsibilities. A metric is supposed to measure some aspect of your responsibilities.

    Your metrics would be P&L and cash flow. The rest is noise. Specific team members might not make any contribution towards those metrics.


  8. David, you didn’t answer my question. Why not evaluate everyone on the product team (developers, sales, marcom, QA, support, product management) in exactly the same way?

    You’re claiming that the CEO’s act of hiring a product manager constitutes enablement? Anyone versed in leadership principles knows that merely giving someone a title and paying them does not itself constitute substantial enablement.

    Also, think about your statement, “The CEO isn’t going to come into your space and tell you how to run you[r] team.” The whole point is that a product manager doesn’t have a team. A product manager doesn’t “run” a team, either.

    As I mentioned in a previous comment, a product manager isn’t CEO of the product, as she doesn’t “run” a product team in the sense that a CEO “runs” the company.

    Finally, you’re bordering on distortion of my use of the word “control”. I wrote that you shouldn’t evaluate someone in terms of metrics she can’t control. I just want to be clear that I am not referring to controlling people. Metrics are not people.

    I do believe that leadership is a key skill of product management and an ingredient of success in achieving product management’s “mandate”. But I do not agree with Saeed that most companies empower product management to optimize the business at the product/line/portfolio level.

  9. davidwlocke says:

    You’ve missed one of my points. The CEO enables you when they hire you. The CEO isn’t going to come into your space and tell you how to run you team.

    You idealism goes off the tracks, however, if you try to change the rest of the organization into an Agile organization. That is the functional unit manager’s responsibility. Some things persist longer like their processes.

    Within the scope of your team, you should be able to do whatever you want.

    Forget control. Lead. If you had control, but didn’t lead, you would still find yourself having strategic alignment issues.

    Hitting your P&L is a matter of having regular releases, and a sales plan that delivers 80% of your install base in 60 days. If that isn’t working for you, something is wrong, and it is probably beyond your scope.

    Missing your targets shouldn’t necessarily kill your career. But, it will take a through analysis to expose the problems, so they can be corrected.

    As for a single metric, other than leadership, that doesn’t work. Balanced scorecards were a response to the use of only financial metrics.


  10. David, I embrace the notion that product managers should be leaders. I agree that a product manager must lead to perform.

    But you’re going way overboard and missing the point here. The ability to succeed as a leader depends on some level of enablement. Just as a leader empowers others, and leader must herself be enabled.

    I have a question for you that will demonstrate my point, but you’ll have to bear with me. It may take a little back and forth before the point becomes apparent.

    Why not evaluate everyone on the product team (developers, sales, marcom, QA, support, product management) in exactly the same way?

  11. davidwlocke says:

    Saeed, Sorry about getting somewhat off track.

    On metrics, back in the Air Force, our wing was measured on two metrics: plane crashes, and the fully mission capable (FMC) rate. It was easy. If a plane crashed, the wing commander lost his job–never to command again, ever–and musical chairs was played. If the FMC rate, on any given day was below acceptable to the war game simulator, the boss got talked to, maybe the controllers got talked to, maybe the maintenance squadron commanders got talked to, but the worker bees were never talked to. The FMC rate rolled up the entire maintenance organization. The plane crashes rolled up the entire wing.

    Being a product manager is pretty much the same deal. The metrics will be distant from the effort, but they will be clear. The metrics will be at a level far above the workers. The metrics roll up to one thing, is this product manager a leader?

    I’m being distinct here in terms of leadership vs. managership. No matter where a product manager goes, leadership matters.

  12. davidwlocke says:

    Roger, I recall one of those “I’m OK, Your OK, exercises we had to do back in NCO school. I was paired with the guy who got a card justifying his poor behavior. My card said fix it, the behavior. So we talked. He told his story from the card, and I gave him a solution. Solutions were not on my card. Everyone in the room was shocked. “You can do that?” “Yes, I’m the shop chief, I can do whatever I want.” Was the squadron commander going to tell me how to run my shop. Hell, no. He wouldn’t waste his time.

    This is so for product managers. They are what they decide to be. It is up to them whether they hit their metrics, sink into victimhood or burnout, shirk their responsibilities, define the job differently than their employer. And, whether they are a leader or not.

    I expect leadership from my product manger. I’ll ask for leadership stories during their hiring interview. I’ll look for leadership during their first week on the job. I’ll know after that first week whether they will still be there after their first quarter ends. I won’t waste my time training them to be leaders, because I don’t believe in that. I won’t listen to them stress how they are a great PO, instead of a great PM. I won’t listen to them talk about enterprise-wide agile. I want a leader in that leadership slot. The staff, the management, and the executive will demand leadership. And, they will demand the termination of the non-performing PM. Management and authority are other people’s problems.

    The metrics will tell me if the PM can lead. They will provide the justification I’ll need to satisfy HR and the lawyers. But, I’ll know before the metrics show up. If I liked the PM as a person, I’d hint that they should move on. And, if I didn’t hire them, I’ll have a long talk with their hiring manager.

    If the job isn’t getting done, it isn’t fault of the metrics, the organization, the system, the team, or the job description. Nope.


  13. Saeed, of course groups need objectives and performance measures. But those objectives and performance measures should reflect that which the groups control.

    Let’s compare the responsibilities of product management to that of a CEO of a company. The CEO’s mandate is to optimize the business at the company level. It seems awfully tempting to similarly describe product management’s mandate as optimizing the business at the product/line/portfolio level.

    However, the comparison actually demonstrates how little sense it makes to do so at most companies.

    The CEO is leading a multi-departmental team. Typically, the CEO has hire and fire authority over every person in the company. When sales, marketing, or development is not performing, the CEO is ultimately responsible, because she has chosen her team.

    The same is not usually true for product management. While product management interacts with the other departments, it is not ultimately responsible for their performance. Companies do not organize a subset of their business around a product/line/portfolio and have product management lead it in the way a CEO does. Product managers don’t choose the members of their multi-departmental product team. In short, a product manager is not CEO of the product.

    The best performance measures focus exactly on what can be controlled, and no more. It would be ludicrous, for example, to measure the performance of a police dispatcher in terms of the average amount of time it took for police to arrive at crime scenes, because dispatchers don’t have control over the competency and driving ability of the officers deployed to the scene.

    But that is precisely how you are proposing product management should be evaluated – in terms of a metric over which it has limited control, and for which it is dependent on other departments.

  14. Eyal Barnea says:

    Interesting post and remarks. I agree with Saeed that competency is a management problem and shouldn’t have a direct effect on product management objectives.
    After some thinking about the mandate described here, I feel that optimizing and balancing are important, but might be too tactical. I miss a more strategic essence. Maybe leading and innovating should be an explicit part of PM mandate.

  15. saeed says:

    Roger,

    What you describe is a management problem that ALL teams face and is NOT the objective of this set of posts.

    Companies have ways of dealing with incompetent people. Incompetent Product Managers (yes they do exist) can also have significant impact on other groups as well.

    But every group still needs to have objectives, and ways of measurement. My focus is on ways that Product Management can be measured.


  16. Saeed, I’m interested in seeing the next entry in this series. I don’t see how the distinction between “product management metrics” and “product manager metrics” addresses the problem I described. Both product management (as a role or department) and individual product managers can do their jobs perfectly and still not “optimize the business” if sales, marcom, etc. don’t also do their jobs perfectly.

    Steve Johnson has a very different way of defining product management’s mandate (at least for strategic product management), which I might mention in comments on the next entry in this series.

  17. Ken Allred says:

    This really is a great, thought provoking post. The comments are great too. I am looking forward to seeing other ideas on product management metrics and how they can help us ultimately be better product managers.

    After thinking about this post and the comments I came up with two possible metrics that I wrote about in my blog here:

    http://www.primary-intel.com/blog/?p=359

    Thanks again!


  18. [...] in his blog post at On Product Management, posited the question, why is it difficult to measure the value and contribution of product [...]


  19. [...] Product Management Metrics part 1 « On Product Management onproductmanagement.net/2009/09/03/pm-metrics1 – view page – cached #On Product Management RSS Feed On Product Management » Product Management Metrics part 1 Comments Feed On Product Management Just as we launch the Blog on Product Management … VP Engineering has a mandate. You need one too. — From the page [...]

  20. saeed says:

    Geoffrey,

    Thanks for the comment. I love the word “balance”. It’s one of my favourite words when describing product management.

    http://onproductmanagement.net/2008/06/25/and-the-one-word-for-product-management-is/

  21. saeed says:

    Roger, Chris,

    Thanks for the comments. My focus is “Product Management” metrics, and not “Product Manager” metrics. I’ll talk about the difference in my next post.

  22. Geoffrey Anderson says:

    @Roger – great points. I find that the real challenge is to try to combat those who would scuttle the product or project (either maliciously or through incompetency), and to prevent such occurrences. It is not very satisfying work, but it is necessary, and the fact that there are some sales people who purposely sabotage the company and our chances for success is particularly unnerving.

    This is a great post, and I have forwarded it to a colleague who as a new PM get very frustrated, and visibly agitated. What she is doing is destroying the balance of the product manager in the center as the “hub” and ruining the wheel around her.

    Product managers are usually implicit leaders. We must be that balancing act (and even if the hub and spoke diagram has a senior executive in the center) and must foster cooperation and synergies between the groups surrounding them.


  23. [...] Or­ig­in­ally post­ed her­e: Pr­od­uct­ M­a­n­a­gem­en­t­ M­et­r&#173… [...]


  24. In theory, the PM is captain of the USS Product, with a mandate of introducing and growing product that meets the needs of the market and meets the business objectives of the organization (and decommissioning product that no longer fit those criteria).

    Strategic & metrics-driven feature prioritization, increased customer satisfaction, and usable artifact generation are generic metrics for PM performance.

    But the reality is that different organizations have different needs and define the PM role accordingly.

    In my company, it makes sense for profitability to be a metric for my performance as a PM because I either control or strongly influence internal resources that can affect that number. In other organizations, where the PM doesn’t exert control, that particular metric would be both unfair and a terrific way to demotivate.


  25. If product management’s mandate is truly to optimize the business at the product/line/portfolio level, then I don’t know why anyone in their right mind would accept a position at most companies. As you pointed out, at most companies, the other departments don’t report to product management. Yet an incompetent sales person, marketing, or engineering person can sink the product no matter what the product manager does.

    I consider product management’s mandate to be more in line with what it actually is able to affect. A few years ago, I wrote about it in a blog entry on evaluating a product manager’s performance.

  26. saeed says:

    Roger,

    As much as we welcome discussion on the blog, this discussion is straying quite far away from the topic at hand.

    As I mentioned earlier, I’m focusing on Product MANAGEMENT metrics, and not that of an individual Product Manager.

    Feel free to comment but let’s keep it on topic.

    And if you want to have a discussion about aspects of control within an organization, you can certainly start that on your blog and invite others to comment.

    I’m certain you’ll generate healthy discussion.

    Saeed


  27. Saeed, you might want to explain your distinction between product management and product manager metrics.

    Common sense dictates that product management metrics measure the performance of product management, and that product manager metrics measure the performance of individual product managers.

    Certainly, viewed this way, the two are closely related, and not “straying quite far away from the topic at hand,” as individual product managers fill the role of product management and comprise the product management team.

    Furthermore, almost everything I’ve written about product managers in my comments applies equally to product management.

    I wrote, “The best performance measures focus exactly on what can be controlled, and no more.” Note that this statement is general and applies to individual product managers, the role of product management, and product management teams. It also strikes at the heart of what you claim to be the best way to measure product management performance.


  28. It doesn’t surprise me at all that a company would assign a product manager such a mandate without empowering them to achieve it.

    Regardless, note that the job description is for a product manager, yet you claimed the mandate was intended to apply to product management. You wrote:

    My focus is “Product Management” metrics, and not “Product Manager” metrics.

    Later, you even scolded me for “straying quite far away from the topic” when I made some statements about product managers.

    I still don’t understand why you consider this distinction so important. But you should at least be consistent.

We really want to hear your thoughts...