Win-Loss Analysis: Sample Size Considerations

Cameron England

Win-loss analysis is an underutilized practice that has the ability to drive powerful and meaningful outcomes for B2B sales organizations. To learn more about why and how to conduct win-loss analysis, read this.

Naturally, the research methodology used by Boeing to find out why they win and lose contracts with major airlines should look quite a bit different from how Coca-Cola would study consumer flavor preferences. B2B buying scenarios, like Boeing's, involve bigger-ticket items where the buyer is an entity, the decision is driven by a group of stakeholders, the feedback about each transaction is complex and unique, and there are a much smaller number of transactions.    

For these reasons, and others, the best data source for B2B win-loss analysis is qualitative interviews with decision-makers at won and lost accounts. As you look to carry out these interviews, a fundamental question you'll eventually have to answer is this:

"How many customers do we need to talk to? How many interviews do we need to conduct to capture constructive and reliable intel?"

In an ideal situation, a business' sample size would consist of its entire customer base (i.e., gathering data and feedback from every customer the business has won or lost a deal with). This approach would ensure a complete picture is painted, with representation of every size and shape of sales opportunity. However, conducting an interview with every prospect is neither feasible (economically) nor required (methodologically). 

So, what is the right number of interviews to conduct? The short answer is . . . it depends.

Sample Size Considerations

The complexity and uniqueness of each B2B transaction is a big reason why qualitative interviews are so important and why conducting more of them is generally better. However, there comes a point where the incremental insights from additional interviews are so minimal that they don't justify the additional cost. Researchers refer to this point as saturation. A sponge can only hold so much water. After reaching saturation, pouring additional water does not increase the amount of water retained by the sponge. In the context of qualitative research and win-loss analysis, saturation implies that the insights from additional win-loss interviews will not yield meaningful, incremental insights.

Identifying the point of saturation is an art, not a science. The point of saturation is . . . a rather difficult point to identify and of course a rather elastic notion. New data (especially if theoretically sampled) will always add something new." Click here for the full paper.

In general, a company that sells a single product to a narrow market segment will reach saturation much faster than a global firm with distinct lines of business, customer segments, geographies, etc. We've found that for a small business, 20-30 interviews per year may be a sufficient number to achieve saturation. For a mature, global firm it may take 20-30 interviews per year per segment. Segments may be based on product line, geographic region, use case, competitor set, or many other factors. Many of our larger clients will rotate their area of focus on a monthly or quarterly basis and move on once they've achieved saturation. Clozd consultants help coach our clients on the nuances of sample sizes and saturation, to ensure they get the most out of their investment in win-loss analysis.

Focusing Your Sample

Most of our clients have a limited budget to invest in win-loss analysis. That budget ultimately dictates the size of the sample. But, like we said before, even a relatively small number of interviews (20-30) can get you to saturation for a given segment of your business, driving a significant return on investment.

Start by interviewing the group of buyers that are keeping you up at night. Spend your time and money gathering intel from the buyers of the highest strategic importance. Work with other key stakeholders to determine which buyer segments are most important to understand. Then, as you achieve saturation for that segment, reevaluate your sampling strategy and change as your business changes. As you evolve your sampling strategy, you'll come to better understand the competitive drivers within specific pockets of your business.

Here are some practical examples of how certain Clozd clients have focused their sample to achieve saturation and drive significant ROI for their businesses:      

One company is looking to expand to new global regions. They sample based on geographic regions to understand how buyer requirements, sales process expectations, and competitive dynamics shift across APAC, EMEA, and the Americas. This has helped them understand how their go-to-market strategy needs to adapt across geographies.

One company operates in a cutthroat market with strong competitors at the top and bottom of the market. They sample based on client tier to understand how competitive dynamics change across enterprise, mid-market, and SMB buyers. This has helped them more clearly understand how buyer requirements shift across segments so they can appropriately package their solutions and adapt the sales process to align

One company has a core product that currently drives 80% of revenue; but, they're betting on a new product to drive the next wave of growth. They sample based on product - focusing on buyers of the new product line. Quickly learning and adapting to market needs at this critical juncture is helping them ensure product/market fit. 

One client wants to sharpen their playbook for selling against key competitors. They sample based the goal of developing sales messaging playbooks for each major competitor. The competitive emphasis has uncovered a key insight - that their core competitor is no longer their biggest threat. An emerging competitor with strong differentiation needs to be their primary concern.

Each unique style of sampling demonstrates the value of focus in achieving saturation and ensuring a meaningful return on your win-loss investment.

There may not be a universal or precise answer to the question, "How many interviews do we need to conduct?" But, it's possible to draw strong conclusions and drive meaningful ROI for your business with fewer interviews than you might expect. More interviews is almost always better, but even the largest enterprise can draw meaningful insights from a sample of 20-30 interviews if they narrow their focus.

About Clozd

Your business is unique and your questions are unique. Clozd specializes in win-loss analysis and is able to tailor and deliver custom programs to cater to the specific needs of each client.

We can execute a win-loss program that's tailored to your business and delivers the in-depth answers you’re looking for. Click here to learn how Clozd can help you increase your competitiveness and win more.

BLOG POST
|
5 min read

Win-Loss Analysis: Sample Size Considerations

Cameron England
Use Case
Ready to win more deals?
Learn how we capture in-depth buyer feedback—and how it can transform your business.
Book a demo

Win-loss analysis is an underutilized practice that has the ability to drive powerful and meaningful outcomes for B2B sales organizations. To learn more about why and how to conduct win-loss analysis, read this.

Naturally, the research methodology used by Boeing to find out why they win and lose contracts with major airlines should look quite a bit different from how Coca-Cola would study consumer flavor preferences. B2B buying scenarios, like Boeing's, involve bigger-ticket items where the buyer is an entity, the decision is driven by a group of stakeholders, the feedback about each transaction is complex and unique, and there are a much smaller number of transactions.    

For these reasons, and others, the best data source for B2B win-loss analysis is qualitative interviews with decision-makers at won and lost accounts. As you look to carry out these interviews, a fundamental question you'll eventually have to answer is this:

"How many customers do we need to talk to? How many interviews do we need to conduct to capture constructive and reliable intel?"

In an ideal situation, a business' sample size would consist of its entire customer base (i.e., gathering data and feedback from every customer the business has won or lost a deal with). This approach would ensure a complete picture is painted, with representation of every size and shape of sales opportunity. However, conducting an interview with every prospect is neither feasible (economically) nor required (methodologically). 

So, what is the right number of interviews to conduct? The short answer is . . . it depends.

Sample Size Considerations

The complexity and uniqueness of each B2B transaction is a big reason why qualitative interviews are so important and why conducting more of them is generally better. However, there comes a point where the incremental insights from additional interviews are so minimal that they don't justify the additional cost. Researchers refer to this point as saturation. A sponge can only hold so much water. After reaching saturation, pouring additional water does not increase the amount of water retained by the sponge. In the context of qualitative research and win-loss analysis, saturation implies that the insights from additional win-loss interviews will not yield meaningful, incremental insights.

Identifying the point of saturation is an art, not a science. The point of saturation is . . . a rather difficult point to identify and of course a rather elastic notion. New data (especially if theoretically sampled) will always add something new." Click here for the full paper.

In general, a company that sells a single product to a narrow market segment will reach saturation much faster than a global firm with distinct lines of business, customer segments, geographies, etc. We've found that for a small business, 20-30 interviews per year may be a sufficient number to achieve saturation. For a mature, global firm it may take 20-30 interviews per year per segment. Segments may be based on product line, geographic region, use case, competitor set, or many other factors. Many of our larger clients will rotate their area of focus on a monthly or quarterly basis and move on once they've achieved saturation. Clozd consultants help coach our clients on the nuances of sample sizes and saturation, to ensure they get the most out of their investment in win-loss analysis.

Focusing Your Sample

Most of our clients have a limited budget to invest in win-loss analysis. That budget ultimately dictates the size of the sample. But, like we said before, even a relatively small number of interviews (20-30) can get you to saturation for a given segment of your business, driving a significant return on investment.

Start by interviewing the group of buyers that are keeping you up at night. Spend your time and money gathering intel from the buyers of the highest strategic importance. Work with other key stakeholders to determine which buyer segments are most important to understand. Then, as you achieve saturation for that segment, reevaluate your sampling strategy and change as your business changes. As you evolve your sampling strategy, you'll come to better understand the competitive drivers within specific pockets of your business.

Here are some practical examples of how certain Clozd clients have focused their sample to achieve saturation and drive significant ROI for their businesses:      

One company is looking to expand to new global regions. They sample based on geographic regions to understand how buyer requirements, sales process expectations, and competitive dynamics shift across APAC, EMEA, and the Americas. This has helped them understand how their go-to-market strategy needs to adapt across geographies.

One company operates in a cutthroat market with strong competitors at the top and bottom of the market. They sample based on client tier to understand how competitive dynamics change across enterprise, mid-market, and SMB buyers. This has helped them more clearly understand how buyer requirements shift across segments so they can appropriately package their solutions and adapt the sales process to align

One company has a core product that currently drives 80% of revenue; but, they're betting on a new product to drive the next wave of growth. They sample based on product - focusing on buyers of the new product line. Quickly learning and adapting to market needs at this critical juncture is helping them ensure product/market fit. 

One client wants to sharpen their playbook for selling against key competitors. They sample based the goal of developing sales messaging playbooks for each major competitor. The competitive emphasis has uncovered a key insight - that their core competitor is no longer their biggest threat. An emerging competitor with strong differentiation needs to be their primary concern.

Each unique style of sampling demonstrates the value of focus in achieving saturation and ensuring a meaningful return on your win-loss investment.

There may not be a universal or precise answer to the question, "How many interviews do we need to conduct?" But, it's possible to draw strong conclusions and drive meaningful ROI for your business with fewer interviews than you might expect. More interviews is almost always better, but even the largest enterprise can draw meaningful insights from a sample of 20-30 interviews if they narrow their focus.

About Clozd

Your business is unique and your questions are unique. Clozd specializes in win-loss analysis and is able to tailor and deliver custom programs to cater to the specific needs of each client.

We can execute a win-loss program that's tailored to your business and delivers the in-depth answers you’re looking for. Click here to learn how Clozd can help you increase your competitiveness and win more.

Clozd gave us insights into the 'why' we were winning deals."

Ike Nwabah

  | VP of Marketing

Outstanding means of understanding why you win and lose."

Tripp R.

  |  Global Competitive Insights Manager

Depth of knowledge we could never achieve on our own."

Gary C.

  |  VP of Product Marketing