Todd Middlebrook:
Welcome. My name is Todd Middlebrook, and I'd like to welcome you to Win/Loss: What isn't it? There's a lot of important questions you can ask about win-loss and I think a fundamental one is what is win-loss? The point of what can it do for you, what can it do for your organization? But probably a better way of phrasing that or a more accurate way of phrasing it is what isn't win-loss? It can do so much for you and your org. And that's what we're going to cover today. Just to give you a little bit of background about myself, I'm Todd Middlebrook. I've been an instructor for the last six years at the Pragmatic Institute. Tell you more about that in just a moment. But today's session is really going to be based on an experience that I had some years ago when I spent 10 years of my career at Microsoft in Redmond, Washington.
I was working for what was called the server and tool business unit, that's the business unit that houses products such as Windows server. And one point in my career there, I was given the mandate to create a win-loss initiative to investigate, why were we winning? Why were we losing? Really, what features do we need to add to our product? And which ones were we losing because we don't have them. And so, as you can imagine, I was a part of this initiative. I created a group, we created a classification system to be able to classify the different outcomes that we saw in our wins and our losses. And of course we needed data and we got the sales organization to mandate that win-loss should be conducted on a certain class of deals that occurred worldwide. And it was awesome. We got a huge volume of data that came in.
And remembering our goal of finding out what features we needed to add to our product, the first surprise was 90% of the deals, whether they ended in the outcome of win or loss, in our experience, the results were for non-functional or non-feature reasons. These decisions were being driven by other criteria. And we were, of course, able to create fantastic charts and pie charts, and eventually present this data back to our business units. And what we realized is although we had this great quantitative data, when our leadership would double click on that and try to understand the facts that we were being presented, when they had questions, we had real difficulty answering those questions. What was that? Why was that important? How did they rate that versus that? And we didn't have that data, we had some pretty charts.
And what we eventually realized is... Well, a couple of things. We realized that, "Hey, we win and lose for reasons that may not directly be related to features." And we also realized that it's not just about having quantifiable data, but you need to be... It's really about not just the quantity, but also the quality of the data or information. Pretty intense experience with win-loss analysis at Microsoft and I've now spent the last six years as an instructor at the Pragmatic Institute, teaching product to organizations, the pragmatic framework. A framework that really describes all of the critical activities that an organization must do to ultimately result in a product that people want to buy. What do we need to do from a product management perspective? What do we need to do from a product marketing perspective?
So 37 activities, and we've been teaching organizations of many different sizes that create many different types of products all around the world for 25 plus years. I've only been doing it for six. But if you look at this framework, one of those critical activities that we focus on and teach is win-loss analysis. Now I'm glad to see you that you're attending the WinLossWeek, And of course there are a lot of great sessions, 30 plus sessions on this topic of win loss and the sessions will be on different topics. For example, there's different types of win-loss analysis. There's qualitative, there's quantitative, there's also... I'm sure going to be sessions if you'll get the abstracts on the different methods and the different sources that we can draw upon to do win-loss analysis, whether it's mining our CRN database, interviewing our sales teams, so many different ways of doing this and there's value in all of the different methods and the different data sources.
But what I want to focus on today, the benefit of qualitative win-loss analysis and specifically around buyer interviews. But before we talk about buyer interviews, we want to provide some advice that buyer interviews can be hard. And so that could provide some recommendations. First thing is you've got to have leadership buy-in if you're going to do buyer interview win-loss analysis. And I mean, of course, approval, you're going to need approval to get to access to sales teams and access to the CRM data. But not only do we want to get buy-in or approval, I want you to make sure that your leadership also understands, what are we trying to accomplish in win-loss analysis.
Do they understand what the end goals are? Because what I don't want you to do is have mixed-matched expectations, hurt how the end product, how the data is received and how ultimately the value of your win-loss analysis is viewed by the organization. So make sure you have that buy-in. Of course, you also need your sales team to buy in. I'm going to be looking at doing win-loss analysis through the eyes of product teams, product management, product marketing folks, and not from within the sales organization.
So if we're going to want to do win-loss analysis, we need access to those buyers and the gatekeepers are sales. And they're not usually pretty excited about having some other outside party, another team in the organization or an outside organization to get in and really start analyzing what's going on in their deals. And so if we're going to do win-loss analysis, we've got to overcome the fears and objections that you may have in your sales organization. And I would just suggest that what you need to keep in mind and what you need to communicate to your sales organization is that you're not analyzing their deals, you're not asking the individual salespeople themselves. Instead, you're trying to learn about your market and you're trying to benefit product management and product marketing, but you're not going to criticize their sales efforts or their salespeople.
And not only do you need to make that promise, you need to keep that promise, because if they're fearful, they can block access. Also, how many interviews do we need to do? Our advice is to interview a few. By doing a few, we can start to spot patterns, we can start to learn things and we can then, once we establish patterns, then later we can use other market research techniques like surveys to validate what it is we think we've learned about our buyers. But I'd also say let's be practical about this. Depending on what it is you're trying to learn, maybe a few interviews is enough. Maybe it's enough to gain you the insights that you need need that will help you improve or to make changes in your organization or in your products. So I would say, do as little as you need to, but not too little and don't do too much either.
I'd also like to recommend to do an equal mix of wins and losses, if possible. And I think in many organizations, a loss analysis gets the majority of attention, right? We want to know why we're losing, what can we do differently? But I would say that when analysis can be just as insightful as loss analysis, it can give as much learning and surprises. I would also suggest that when it comes to difficulty of getting people to agree, it's often much harder to get people to agree, to do a loss review than it is a win review. So that may end up tipping the scales and what the mix of the two is that you ultimately are able to do, or again, depending on what it is you're wanting to learn, what decision you're trying to make, maybe win reviews is enough.
Also, I would like to recommend that you pick good subjects for your buyer interviews and win-loss analysis. We have target markets that our objectives are pinned on that we're wanting to sell to. And we want to make sure if we're going to do win-loss analysis, that the interviews we conduct are real good representatives of our target market and they're not some anomaly. And not only do I want you to make sure that they're good representatives of your target market, I want to make sure the individuals that you interview were actually a party to the decision, they were actually involved in the decision. Seeing many win or loss reviews conducted from user communities, users of the product. And I'll be looking at this today, probably through a B2B lens.
In a B2B scenario often, buyers and users are very different. And if we're interviewing users, they may not have had any part in the product and their feedback... Well, they'll tell you what they like about the product, or don't like about the product that was chosen, but they may not have been a party in the actual making of the decision. Let's make sure we get the decision makers or the people that were involved. It doesn't help if you ask the right questions of the wrong people. I would also like to recommend that you do win-loss reviews as quickly as possible. I mean, practically, the longer you wait, the fuzzier people's memories are. Do they remember the debates that they were having, the back and forth? Do they remember their considerations.
If you wait too long, that can be becomes stale and out of date. Also, if you wait too long, often after a purchase is made, the product is turned over to different parties. It could be implementation teams eventually in the hands of the users of the product. If you wait too long, the decision makers may have disengaged and may no longer be available to be interviewed by you. Important question to answer is, who should be the interviewer? If we're going to do this within our organization, I would like to recommend that the interviewer not be somebody in the sales organization. And before you fret, I want you to know that this is not a criticism of salespeople. This is more of a commentary on human psychology.
If we really want to get accuracy, if we want to get transparency, even in something like win reviews, when the interviewer is somebody that is viewed by the buyer as being independent in some way, independent to the deal that was just conducted and somebody who is objective, that does not have any subjective concerns, they are often more transparent and more open and more verbose in their descriptions. And then of course, this has really highlighted this problem when it's loss reviews. And lost prospects, they're not inclined to answer a lot of questions or be verbose in their answers when basically the, the salesperson that they just rejected is now asking them, you know, what happened? Why did you choose to buy a product? It's very, very awkward.
So having somebody that is viewed as objective in the mind of this buyer, and that can be a third party, but it doesn't have to be, it just has to be somebody that they don't associate with this deal. It could be a, you know, somebody from the product team or somebody from another part of the organization. And with that separation from the sales engagement, you can expect more transparency from them.
Last thing I want to recommend is, know why are you doing win-loss in the first place? As I mentioned at the beginning, when loss is a very versatile tool, it can answer different questions. From a product management perspective, it can help us understand when do we win, when do we lose, which competitors are our biggest competitors in those common competitors and help us understand which features are viewed as superior from us or for them, which things do we need to match of the competition for sales teams. So lots of different questions that can be answered with win-loss analysis. It can help them analyze their sales process. Is it working effectively?
It can help them understand their sales team. Which salespeople have the highest win rate? Which ones have perhaps the lower win rates? Who are their top sales people? Which are the ones that maybe need to be coached up, trained up? Who can be mentors for them? Which of our sales motions are effective? Which ones maybe are not getting traction? Win-loss can also help us with our go-to market strategy. It can help us to analyze our efforts already in marketing. How are they working? The content that we're creating, is it being digested? Is it being used. Help us understand our buyers better? Who are they? How do they shop? How do they buy?
So there's a lot of different questions that can be answered for different parts of the organization using this multi-tool. But here's the thing, lots of questions can be asked, but what most people will tell you, if you're going to do buyer interview win-loss analysis, what you can expect is about 15 or 20 minutes for this interview. Everyone's busy and you don't want to be sitting there and regret not asking a question. You want to make sure that you're getting the most benefit out of it. So I'd like to challenge you to be clear about before you do win-loss analysis, answer this question, what problem are you trying to solve in the organization?
Knowing you could solve a lot of different things... Let's be very targeted. If you're wanting to open a bottle of wine, if that's the specific thing that you're wanting to accomplish, it's true that you could use a multi-tool for that. And it's got a corkscrew and you could eventually get that bottle open. However, is it the perfect tool? Well, I don't know about you, but I drink wine and I would much rather use a specialized tool because I know it will accomplish that goal. So I want you to think about what are those goals, what are those objectives, what are you trying to improve, and let that guide the line of questioning that you use in that short period of time that you have available to you. So some examples, we can use win-loss analysis to help us with go-to market improvement, sales enablement improvement, product improvement, competitive analysis. And as I said, we can use it for sales, execution improvement.
Not that it's not important, but I'm not going to focus on sales execution improvement. Not that it's not important, but I'm looking at this through the lens of product teams, product management, and product marketing. And it's my belief that the product management, product marketing teams should not be evaluating the sales organization or the salespeople, let their leadership do that. Lots of value can come out of it. But I want us to be clear about which of these improvements are we trying to impact for our organization. I would also like to suggest that in addition to knowing what it is we're trying to do, we need some way of framing the questions and framing the improvements. And I'd like to recommend using the concept of the buyer journey as a way to identify where are those opportunities for improvement and as a way to help you parse down and define the line of questioning that you want to ask.
And I'm using a very basic generic buyer experience here, but a buyer journey are, what are the different steps and stages that your buyers go through that ultimately ends with them buying a product. But it's not just a description of our selling process, but what is that comprehensive experience? And let's use that and know where we're trying to find improvements and direct our questioning to the questions that will give us those learnings. So I'm just going to walk quickly through different questions that you could ask, depending on what you're trying to learn. And a big part of trying to improve your go-to market efforts is understanding your buyers. They're the ones that are doing the shopping, the evaluating, and the buying. First understanding who the cast of characters are, let's ask them how they describe themselves.
And I'm going to use a B2B example here. What is their job title? This can help me later if I'm going to want to conduct a broader market research. Who do I need to be searching for? What job titles do I need to be searching for? I would also like to understand them better. I'd like to understand their roles and the responsibilities in their organization, where do they sit in the organization? What's their job? What's their boss tracking them on? I'd like to get some insight into how they're compensated, do they have a bonus structure? How does that work? And also I'd like to understand what are... If I understand what their objectives are in their job, I'd like to know what problems they have, what challenges they have, what is standing in the way of their success? This could give me a frame of reference that helps me understand whether the problem I solve is even in this buyers top 10 concerns.
And that might be something that I'm competing with later. How do I try to convince them, as a marketeer, that they should prioritize solving our problems more than maybe something else. And then critically, I'd like to understand what role did they play in the purchasing decision? What step or stage were they involved in? But also, what was their role? Did they make the final decision? Were they the initial shopper? Were they the ones who parsed out which of the competitive offerings should be evaluated? Were they that final decision-maker? Help me understand that. And where did they pop in? What stage or step in that buyer journey were they involved in?
So who are they? Let's know our buyers more deeply using win-loss analysis. And it can, of course, also help us understand what that buyer journey is, beginning with what did they do before they started shopping? Buyer journey begins with realizing that you have a problem in the first place. How did that happen? How did they know they had a problem? Let's understand. I mean, is there a regulation coming down that they're going to have to be compliant to? Was there a pandemic? I mean, what brought the problem to the surface? But what I'm also going to be listening for is how do they describe that problem? What was that problem? What were the implications of that problem to them? I'm also going to be listening for the language that they use, the negative consequences of the problem, because later this might help me understand what search terms they might be typing into Google. I might make sure that the content that I'm creating is easily discoverable.
Speaking of content, that language that they're using, content marketing, using language that will resonate with them. This might help me understand, do we need to invest in thought leadership activities, doing keynote addresses, publishing eBooks. This gives me an indication of where their journey starts and how they view the problems and the language that they use. I also want to understand those next steps is if they realize they had a problem, what are the different ways they considered solving it? What are the different categories of solutions? And that may not just be what vendors, maybe they thought about building something for themselves. Was that something that was a potential alternative or did they maybe didn't want to buy or build something? Maybe they considered hiring humans to solve their problem and use human resources as the solution.
What did they initially consider and then how did they narrow that down? And then within the categories, how did they figure out, of all the solutions in that category, which of them are the ones they can evaluate? There may be a very crowded landscape of solutions in your product category, do they have the time to evaluate them all? Probably not. So how did they narrow that list down? Do they spend time on Google? Do they talk to an analyst? Is there some Internet influencer that helped them parse that list down? Again, marketeers this could help us understand, do we need to invest in analyst relations? Do we need to double our social networking investment.
Continue with that. Maybe they hired a consultant to help them evaluate those options. And again, we're always looking for patterns here to understand what is common about our buyers in their buyer journey. Oh, marketeers. So did any of the marketing assets that we've invested in impact their decision to evaluate us? I mean, did they go to our website? Did they download our 30-day trial product? Did they read those eBooks? Did they attend one of our seminars? Were they the target of one of our marketing campaigns? So I want to understand what of our marketing efforts did they actually consume and also how did it influence their decision? Was it positive? Was`it neutral? Or maybe did it have a negative impact?
And then ultimately, I'd like to know how did they engage us? What was that path of contact? Did they click on the link of that email campaign and become a lead? Did they put the effort in, of going to our website and finding the 1-800 number or filling out a lead form? Did they talk to a sales person at an industry conference, when we used to do that in person? What was that point of contact? What is that typical contact when they've decided to become what we say is a lead? So lots of questions that we can ask to understand, even before we're selling to them, what can we do to improve and get more people into the buyer journey? And what improvements from marketing perspective and the marketing investment perspective?
Continuing with these. Okay. Once they engage and they're finally evaluating us, we can ask questions that, well, how do they buy? What is the process? Is there a single buyer that does the evaluation and makes the decision? Is there a pattern where there's a procurement process, a bid process? Is there a committee that gets formed ad hoc to evaluate and make a decision? Help us understand what those typical buying patterns or processes are. And what were the evaluation criteria? What were they thinking about? What questions did they have and need satisfied to make their decisions? Did they have economic criteria, ROI, lowering TCO? Were there technical criteria [inaudible], regulatory? Was it solely evaluated based on the functional capabilities of the product, or often it's some combination of these.
So what are they? And if I can do a drill down into that economic is, I'd like to understand pricing. Part of our go-to market is coming up with the right pricing strategy. And so how important is price to the decision? What role did it play? I also like to know, at what point did price become an issue? At what point in that buyer journey and ultimately, what did they think about our price? Did they like it? Did they dislike that strategy? They may have still purchased our product, but they may have said, "Look, you have a perpetual license. We would have much preferred a monthly subscription fee. We like the op ex budgetary versus cap ex."
Continuing with the going to the deep dive on economic is, let's understand, was there an economic return calculation? Did they do an ROI analysis? This might give you some guidance on how they are viewing price, and their different approaches to pricing your product, this might give you the pathway that is the right way, demonstrating an economic benefit. In fact, if they did an economic calculation, you might ask, would you share it with me? Because maybe we can use this for future buyers and to justify the price that we're quoting to them. And we want to know out of all of those competitors, which are the ones that they chose to evaluate. Might have a very crappy competitive landscape, but who are those typical competitors that they actually pit you against and compared you against?
And finally, how did they decide? What tipped the scale when they made the decision? Was it product related? Was it functional? Was it value proposition? Was it cost? Maybe something outside of your control, risk? But let's understand what ultimately tipped the scale in making that purchasing decision. It's not always features and functions.
So let's put on a, a different set of glasses and say, okay, what if we want to improve sales enablement? Different set of questions that we could ask. Well, let's start off with understanding of the sales tools, of the sales [inaudible] that you have created. Well, let's try to understand what was actually used. What was actually consumed. You might find out that you have an evaluation product, a 30-day trial that can be downloaded. You may find out that maybe they never downloaded it. They never used it. Or you might find out that that content is not good. I mean, did they download that ROI calculator? Did they download that data sheet? Did it contribute to their decision? That's an important part. It's not just what did they consume, but did it help them decide?
What new things do we need to do? What can we do to improve the content and tools that we create? I don't know. Let's have our buyers tell us. Let's ask them, "Hey, what else could we have done?" They may tell you that you need case studies that you you didn't have before. They may tell you that you need a proof of concept out of a box. So they may tell you that you need a try before you buy to mitigate risks, but you really don't know unless your buyers can tell you. And make sure that we're not wasting our time and our money. So let's ask open-ended questions to learn new things from our buyers.
Last but not least, I want to talk about what if we are using win-loss for competitive analysis. Some of these questions overlap. First one is, which vendors did you actually evaluate? And this can help us really parse out what are our typical, who are the most common, our biggest competitors that we face on an ongoing basis.
Yeah, before we start understanding the difference in features between the products, how about the pricing models of our product. That there may be two products, ours and the competition's in the same product category, but did they value them both the same? How did they? We may find out that they value the competitor product higher than ours, therefore willing to pay higher. I'd want to know relatively where we are to that competitor when it comes to value perception. I'd also like to follow up this question with why, why did you view my competitor product is more valuable or less valuable? What were the attributes? And it may be they had better case studies. It may be that they had an ROI calculator. It may be something, again, that's not necessarily related to the functionality of the product. And back to those criteria, what were their decision criteria that they actually compared you on?
Were there those technical criteria? Were they economic or were they functional? And ultimately, I would ask them the winner of this, what were their strengths over the loser? What ultimately tipped the scale for us or for them? And yes, it's true, win-loss can also help you understand the functional differences between your product and your competitors. And so we can ask, how did they assess those products? What did they view as the unique capabilities of our product versus that competitor? They probably saw some unique capabilities of their product versus us.
What did they view as our respective best capabilities, theirs and ours? What did they see as being about the same? And did they see some inadequacies in one of our products, or different inadequacies in both of our products? But I would like to follow up with not only understanding what were these differences that they saw, but follow that up with the question of, "Okay, how important were those differences to the decision that you ultimately made? We so often can get caught up in understanding these functional comparisons and forget to ask, did it actually tilt the scale in one direction or another?
You may have seen that they have higher performance than we do, but was that weighty in the decision that you ultimately made? That's where the rubber meets the road. That's where the learnings really come from. So I went through that quickly, but lots and lots of questions that can be asked. But let's put it all together, make sure you understand. I love quantitative data, but I'm suggesting that you marry the two, quantitative and qualitative data. That qualitative data can make those numbers sing. It helps us describe and understand what it means, those numbers. And since you have very little time, 15 to 20 minutes on average, make sure that you're asking the right questions.
So be very clear about what it is you're trying to learn. Yes, it's a multi-tool, it can do so many things, but take what you're wanting to learn, what you're wanting to improve and turn this generic tool into a very, very targeted tool to help you and your organization. Win-loss is really powerful, it can introduce you and help you know your buyers, it can help you understand how they shop and ultimately, it can help you understand how you can help those buyers buy your product, to buy more of your product, to buy your product more easily, more quickly in the future. So I hope you use qualitative buyer interviews as one of the aspects of win-loss analysis to benefit your organization, then turn it in the right tool for yourself. So I thank you for spending some time on this session. I wish you all the best of luck and have a great day.
