Pull Incentive Sizing Tool
This guide provides the theoretical framework and a technical step-by-step tutorial for estimating the appropriate size of pull funding mechanisms designed to accelerate innovation.
Contributing authors: William Arnesen & Claire McMahon, 2025. Georgia Bradley (Interactive Calculator), 2025. With thanks to Jano Costard, Sarah Daniels, Siddhartha Haria, Elisabeth Hofmeister, Rebecca Rolapp, Leah Rosenzweig, Hassan Sayed, Christopher Snyder, and Sebastian Quaade for feedback.
Key Takeaways
- When designing pull funding mechanisms, determining the right incentive size is critical. Larger incentives attract more firms and increase the chances of success, but too large an incentive wastes funds that could be used for other projects. This guide provides a practical framework for making this decision.
- The incentive size needed to motivate firm participation depends on:
- Development costs: Firms face expenses including R&D, testing, regulatory approval, and commercialization, all adjusted for the probability of failure at each stage.
- Innovation risk: More technologically distant innovations require larger incentives to compensate firms for the high likelihood of R&D failure.
- The time value of money: Firms discount future returns, and long development timelines for major innovations reduce the present value of potential rewards.
- The social value of the innovation sets an upper limit on willingness to pay and helps for prioritizing funding opportunities.
- While total funding commitments for pull mechanisms often appear large, they reflect that firms must bear both the risks of development failure and the opportunity costs of capital over long development timelines. Understanding these components helps funders right-size their commitments.
Table of Contents
1. Introduction
Funders using pull mechanisms face a crucial practical question: how large of an incentive should they offer? This guide provides a practical framework for answering that question, breaking down the factors that determine the incentive size needed to attract serious development efforts and how to optimize that size.
Pull mechanisms“Pull funding” is incentive-based funding that rewards firms after achieving specific results or delivering a successful product. Since funders do not pay upfront, pull mechanisms place risk of technological success on the innovators., such as prizes and advance market commitmentsAdvanced market commitments are agreements where governments or organizations commit to purchasing or subsidizing a product once it is successfully developed, guaranteeing a market for manufacturers. (AMCs), reward success rather than funding research upfront. They shift the burden of innovation risk and cost onto the innovator, who only gets paid when specific goals are achieved. Getting the size of the reward right is critical: too small, and firms won’t enter; too large, and the program wastes scarce funds.
In this guide, we outline a practical approach for using estimates of development costs, technical risks, and market dynamics to gauge how much pull funding might be needed to motivate firms to pursue socially valuable innovations. This framework is a first-step, order-of-magnitude tool—particularly useful at the outset of a project, when funders are considering whether a pull mechanism could be viable, estimating the scale of funding needed, or determining what kind of institution (e.g., a philanthropic funder or a government agency) might be suited to support the effort. This methodology is especially well-suited to innovations that proceed in discrete stages, where development risks and costs can be estimated using available information.
Our approach draws from our experience evaluating pull mechanisms across sectors and builds on academic work by Ahuja et. al 2021 and Kremer, Levin and Snyder 2022 , among others. While grounded in theory, this is not an academic paper; it’s a practical explainer of our current best practices at the Market Shaping Accelerator. While not exhaustive, the guide also includes an appendix with additional complexities and refinements we’ve found helpful in real-world contexts. We continue to evolve this work and welcome feedback from funders, designers, and researchers tackling similar challenges.
The sections that follow break the sizing challenge into manageable steps: starting with defining a conceptual framework (Section 2), then estimating innovation costs and risks (Section 3), calculating the incentive size needed for a desired chance of success (Section 4), and assessing total program costs and cost-effectiveness (Section 5). The appendix provides a full case study for methane-reducing livestock vaccines and extensions for more complex modeling scenarios.
2. Conceptual Frameworks
Pull mechanisms“Pull incentives” are incentive-based funding that rewards firms after achieving specific results or delivering a successful product. Since funders do not pay upfront, pull mechanisms place risk of technological success on the innovators. mimic market incentives by committing to reward innovators of socially valuable goods. At first approximation, when firms evaluate whether to enter a market, they consider whether expected revenues will exceed expected costs. To answer that question, firms need to assess: What is the likelihood of technological success? How much will it cost to pursue innovation? What is the expected market size and timing of future revenue? And how much competition do they expect to face?
The size and structure of the pull mechanism“Pull incentives” are incentive-based funding that rewards firms after achieving specific results or delivering a successful product. Since funders do not pay upfront, pull mechanisms place risk of technological success on the innovators. will dictate the answer to these questions and, hence, determine how many firms try their hand at innovation. Increasing the amount of effort firms put into innovation or increasing the expected number of firms that choose to participate in pull mechanisms increases the probability that at least one attempt yields success. To determine the appropriate reward size, the designer of the pull mechanism“Pull incentives” are incentive-based funding that rewards firms after achieving specific results or delivering a successful product. Since funders do not pay upfront, pull mechanisms place risk of technological success on the innovators., therefore, needs to consider the tradeoff between increasing the probability that firms succeed at innovation and the cost of the pull mechanism.
Firm Decision Making
When designing an innovation incentive, we need a systematic way to think about what motivates firms to participate. Let’s start with a simplified view.
Imagine a pharmaceutical company considering whether to develop a new vaccine. They know development will require significant upfront investment in research, clinical trials, and manufacturing capacity. They also know there’s a high chance of failure at each stage and that other companies might succeed first.
To decide whether to invest, firms weigh their expected costs against potential rewards. If successful, they’ll receive the incentive payment — but they might have to share it with other successful developers. If they fail, they bear all the costs with no return.
Our framework captures this decision-making process. We look at:
- the full cost of development, from research through commercialization
- the probability of success at each stage of innovation
- the time value of money between when costs are incurred and payments are received
- the impact of competition on expected returns.
From the funder’s perspective, the size of the pull mechanism principally depends on the number of firms they want to entice to invest in the target innovation. The following model then maps this desired number of investing firms into a necessary size.
Model Setup
To make this analysis tractable, we make some simplifying assumptions:
- All firms are rational and risk-neutral: they participate if (and only if) expected revenue exceeds expected costs.
- Firms face one choice: whether to invest in a single innovation attempt.
- All innovation attempts are identicalBy identical, we mean that each attempt has the same probability of success, not that they are all pursuing the same method or approach, independentBy independence, we mean that each attempt is uncorrelated with any other attempt, much like how whether a flipped coin comes up heads does not depend on the outcome of the previous flip. We relax this assumption later by allowing different attempts to become correlated with each other, and proceed along the same timetable.
- Successful firms share the reward equallyFor example, if only one firm succeeds, they receive the entire award. If two firms succeed, they each receive half the reward.
- Program benefits do not depend on the number of successful firms. The goal of the funder is for at least one firm to succeed.
This model has been deliberately kept simple to make it amenable to working with real-world data. We focus on innovation efforts that progress through discrete stages—like R&D, testing, and approval—because those can be sourced from past projects, expert interviews, or published literature. By structuring the model around commonly observable parameters, we aim to make this a tool that funders can actually apply in practice. More complex dynamics—such as firm heterogeneity, market concentration, or correlated risks—can meaningfully affect outcomes, but are harder to parameterize. We include extensions for these cases in the appendix, where they can be explored as needed for specific applications.
Another modeling choice has been to focus on a particularly simple funding mechanism: a single, lump-sum prize. While this helps clarify the core mechanics of incentive sizing, it may not match every real-world objective. In many cases, it might be more appropriate to allow for multiple winners, offer milestone payments, or design around a different market structure. These variations can significantly affect firm behavior and program cost, and should be considered carefully when adapting the model to a specific context. In later sections (see Appendix B: A Complete Example), we explain how to extend this logic to an advance market commitment with minimal conceptual changes.
Note: Throughout this post, we use “firm” to refer to any profit-motivated innovator – whether a large company, research institution, startup, or university spinoff.
3. Understanding Innovation Costs and Risks
Key components
First, we need to understand the costs and risks that firms incur when trying to innovate. The higher the cost of innovation, the larger the pull size needs to be in order for firms to find investing in innovation worthwhile. We account for the following elements:
i. Cost of R&D and commercialization. This category includes the fixed and marginal costs to a firm to invent, produce, and deliver the desired product. Costs associated with identifying promising leads, validating their efficacy, testing their safety, shepherding through regulatory approval, manufacturing, marketing, and distributing the final good all fall under this category.
For example, consider an AMC to incentivize the development of a livestock vaccine. We can use past data on livestock vaccine innovations to inform our estimates on the cost of development. The following comes from a 2014 paper estimating the costs to develop a vaccine for chicken in 2009. Any forward-looking estimates would need to adjust for inflation.
| Phase | Duration (Years) | Reported Cost (1000 €) |
|---|---|---|
| Research and Development | 3 | 1,949 |
| Patent | 1 | 137 |
| Testing | 2 | 208 |
| Approval | 1 | 48 |
ii. Likelihood of technological success. Innovation is inherently risky. There is no guarantee that firms that incur R&D and commercialization costs will succeed in developing an innovation and bringing it to market. The probability of success includes a firm’s risk of technological failure such that they are unable to fulfill the terms of the technical product parameters.
Since estimates of success rates vary substantially, designers should be careful that the data source for success rates and costs are consistent. Some sources may estimate a higher success rate because they exclude earlier stages. Or they may estimate the success rate of an entire research program, which includes multiple concurrent attempts, resulting in both a higher success rate but also a higher cost. These alternative approaches are not wrong, but one needs to be careful that the costs and success rates represent the same activities.
| Phase | Duration (Years) | Reported Cost (1000 €) | Unconditional Probability of Success |
|---|---|---|---|
| Research and Development | 3 | 1,949 | 20% |
| Patent | 1 | 137 | 75% |
| Test | 2 | 208 | 20% |
| Approval | 1 | 48 | 90% |
iii. Firm’s hurdle rate of return. Firms expect to make a profit and, all else equal, prefer investments that pay money today rather than in the future. Since pull mechanisms require firms to spend money in the present in the hope of future payment, firms will not participate unless adequately compensated for that delay. In general, the more a funder wishes to delay payments (e.g. the technology requires many years to develop and commercialize), the greater the premium the funder must pay to induce entry. MSA currently uses annual discount rates of 8-10%, depending on the sector involved. For long-dated AMCs, i.e. AMCs for technological innovations that may take ten years or more to pay off, including discount rates can more than double nominal costs.
| Payout Year | 0 | 5 | 10 | 15 | 20 |
|---|---|---|---|---|---|
| Future payout needed for firms to have net present value > 0 | 100 | 161 | 259 | 418 | 673 |
| Funder's needed commitment today if they can invest committed funds with a risk-free 3% annual return | 100 | 139 | 193 | 268 | 372 |
Calculating Firms’ Expected Costs
Innovation often requires multiple steps, including research and development, testing, trials, and regulatory approval. Each step incurs its own set of costs and its own risk of failure. However, firms do not incur all costs upfront. Instead, if the project fails at any stage, the firm simply stops and does not incur the costs of subsequent stages.
When sizing the pull mechanism, we consider the firm’s perspective of expected costs at the time of entry, not the total cost of a successful innovation. The expected costs faced by the firm at the time of entry are the cumulative, probability-weighted costs across all development phases, properly discounted for the time value of money. Therefore, the expected cost of an innovation attempt will be less than the total cost of successful innovation.
| Phase | Reported Cost (1000 €) | Cost (2024 1000 $) | Unconditional Probability of Success | Probability of reaching step | Expected Cost (1000 $) |
|---|---|---|---|---|---|
| R&D | 649.7 | 955 | 58.4% | 100% | 955 |
| R&D | 649.7 | 955 | 58.4% | 58.4% | 554 |
| R&D | 649.7 | 955 | 58.4% | 34.1% | 321 |
| Patent | 137 | 201 | 75% | 19.5% | 39 |
| Test | 104 | 153 | 44.7% | 14.6% | 22 |
| Test | 104 | 153 | 44.7% | 6.6% | 10 |
| Approval | 48 | 71 | 9% | 3% | 2 |
| Release | - | - | 100% | 2.7% | - |
| Phase | Cost (2024 1000 $) | Unconditional Probability of Success | Probability of reaching step | Expected Cost (1000 $) | Discounted Present Value (1000 $) |
|---|---|---|---|---|---|
| R&D | 955 | 58.4% | 100% | 955 | 955 |
| R&D | 955 | 58.4% | 58.4% | 554 | 513 |
| R&D | 955 | 58.4% | 34.1% | 321 | 275 |
| Patent | 201 | 75% | 19.5% | 39 | 31 |
| Test | 153 | 44.7% | 14.6% | 22 | 16 |
| Test | 153 | 44.7% | 6.6% | 10 | 7 |
| Approval | 71 | 9% | 3% | 2 | 1.3 |
| Release | - | 100% | 2.7% | - | 0 |
| Sum | 3,443 | - | - | 1,904 | 1,799 |
We use the expected cost of the innovation attempt to be able to compare to the present value of the expected reward from the pull incentive, which is also discounted by time and the probability of success. That is, we are estimating the expected costs and returns from the firm’s perspective at the time of entry.
Sourcing Parameter Estimates
Data on the costs and risks of innovation can be difficult to find, but there are several types of sources we have found useful for gathering this information:
- Academic literatureOne example: DiMasi, Grabowski and Hansen 2016 and industry reports provide historical cost averages
- Government and regulatory filings often contain detailed development costs
- Interviews with industry experts who can validate cost assumptions
- Similar completed projects that provide comparable reference points
When using these sources, it is important to be aware of their potential limitations and biases. Published costs often only represent successful attempts. They may also be dated and need adjustment for inflation. Additionally, costs can vary significantly by sector, region, and specific technical challenges. Triangulating between multiple sources can provide a more complete picture.
Additionally, remember that the goal is to identify the threshold case — the point at which firms are just willing to participate. For simplicity, our model assumes that firms are homogeneous, meaning they share the same cost structure and probability of success. In reality, firms will have a distribution of costs and probabilities, meaning some will find participation attractive at lower incentive levels, while others may require larger incentives. The characteristics we assign to the representative firm in our model should approximate this real-world threshold—capturing the point at which firms on the margin would choose to participate.
4. Determining the Overall Program Size
Simplest version
We now have a framework for calculating the expected costs firms face when pursuing innovation. How, then, do we use this to determine the total size of the pull incentive needed?
First, consider perhaps the simplest case: aiming to set an incentive size large enough to motivate one firm to attempt innovation. To accomplish this, the pull incentive needs to provide an expected reward greater than or equal to a firm’s expected cost of an innovation attempt. In the case of a single firm, the present value of a needed pull incentive is equal to the firm’s expected costs multiplied by the reciprocal of their probability of success.
For example, for an innovation with a firm’s expected cost of \$1 million and a 10% chance of success, a pull incentive would need to have a present value of at least \$10 million. While it may be shocking that \$10 million is more than an order of magnitude larger firm’s expected costs, remember that pull mechanisms are shifting the risks of development from the funder to the innovator. In this case, rather than the funder spending an expected \$1 million in up front funding with only a 10% chance of successful innovation, the funder commits to paying \$10 million only if the innovation is successful. The funder has a 10% chance of paying \$10 million; in the 90% chance that the innovator is unsuccessful, the funder can repurpose those funds for other projects.
In many cases, the funder may want to create a pull fund that is large enough to incentivize multiple attempts simultaneously in order to have a higher probability of at least one successful innovation. In the next section, we discuss how to choose the target number of attempts.
Target Number of Attempts
The key is to recognize that each additional firm attempting innovation increases the overall chances of success. Consider a case where any individual firm has a 10% chance of successfully developing a targeted product. If two firms make an attempt, the overall probability of at least one success increases to 19%, and with three firms, it increases to 27.1%. With multiple firms participating, the chance of at least one success improves.
At the same time, attracting additional firms becomes increasingly expensive as they have to have to share the reward if several of them succeed. Consider the example where each attempt has an expected cost of \$1 million and has a 10% chance of success. For a single firm to participate, the reward needs to be at least \$10 million (to offset the 10% chance of success). For two firms to participate, the reward needs to be higher – about \$10.5 million per firm, or \$21 million total – because each firm now faces the risk of having to split the reward if both succeed. For three firms, the required reward increases to about \$11.1 million per firm, or \$33.3 million total.
This creates a fundamental tradeoff: the more firms that participate, the more likely we are to achieve the innovation’s benefits—whether that’s lives saved by a new vaccine or emissions reduced by green technologies—but increasing this probability becomes increasingly expensive. As a result, setting the right reward size means asking: what is the social value of increasing our chances of achieving these benefits, and how do these compare to the costs? During the COVID-19 pandemic, the massive daily cost of delay meant even small increases in the probability of success had enormous value, justifying efforts to attract many firms (Ahuja et al 2021). However, in other cases, the funder may only seek to incentivize a few of the best-suited firms to participate.
The graph below shows how many firms must attempt an innovation to achieve a target probability of success. The slider can be used to vary the baseline, firm-level probability of success to show how the number of attempts needed changes.
- All firms are rational and risk-neutral: they participate if (and only if) expected revenue exceeds expected cost.
- Firms face one choice: whether to invest in a single innovation attempt.
- All innovation attempts are identicalBy identical, we mean that each attempt has the same probability of success, not that they are all pursuing the same method or approach and independentBy independence, we mean that each attempt is uncorrelated with any other attempt, much like how whether a flipped coin comes up heads does not depend on the outcome of the previous flip. We relax this assumption later by allowing different attempts to become correlated with each other and proceed along the same timetable.
- Successful firms share the reward equallyFor example, if only one firm succeeds, they receive the entire award. If two firms succeed, they each receive half the reward.
| Variable Name | Description |
|---|---|
| $\theta$ | Target probability of success |
| $n$ | Number of attempts |
| $p$ | Per-attempt probability of success |
| $X$ | Present value of pull incentive |
| $\mathbb{E}[c]$ | Present value of expected cost per innovation attempt |
The Cost of Incentivizing Multiple Attempts
The more firms that have already entered, the higher the marginal cost to induce an extra firm, because now firms face higher risk of splitting the prize. As the number of firms increases, the marginal cost to induce a marginal new attempt approaches the expected cost of each additional attempt. For instance, if the probability of success per attempt is 10% and the present value cost of each attempt is \$100,000 then a funder would need to increase the pull size by \$66,000 to move from nine to ten attempts. In contrast, moving from 39 to 40 attempts requires an additional marginal cost of \$95,000. The plot below shows the marginal cost of adding additional attempts as a multiple of the expected cost of an individual attempt.
| Variable Name | Description |
|---|---|
| $n$ | Number of attempts |
| $p$ | Per-attempt probability of success |
| $X$ | Present value of pull incentive |
| $\mathbb{E}[c]$ | Present value of expected cost per innovation attempt |
$$X \frac{n \cdot \mathbb{E}[c]}{1-(1-p)^n}$$
The marginal cost of inducing an additional attempt is then:
$$\frac{\mathrm{d}X}{\mathrm{d}n} = \mathbb{E}[c] \cdot \frac{1 + (1-p)^n \cdot (n \cdot \ln(1-p) - 1)}{(1-(1-p)^n)^2} $$
$$ lim_{n \to \infty}(\frac{\mathrm{d}X}{\mathrm{d}n}) = \mathbb{E}[c] $$
Now we can calculate the reward size needed in order to induce a given level of firm entry. The following graph shows the pull size if there was no time delay between private investment and receiving payment. In the real world, including the cost of time will increase costs substantially: as we saw in the previous section, the nominal value will be far higher if revenues are delayed substantially. For example, if an innovation attemptRemember that an “attempt” in this case does not refer to the cost of an entire research program, which can often involve dozens of research attempts. The cost also only refers to the expected cost of a new attempt at the time of initiating, and not the full cost of seeing an attempt all the way through to completion, since most attempts fail well-before incurring the costs of, e.g. late-stage safety & efficacy trials costs \$1 million and each attempt has only a 5% chance of success, then the present value of the incentive needs to be at least \$66 million to get at least an 80% chance of success. If the payments occurred ten years after the research begins (a realistic timeframe for novel drugs or vaccines) and firms have an 8% discount rate, then the nominal value of the incentive will have to be at least $142 million.
Note: The expected cost of an attempt (used when considering firm participation) can be substantially lower than the full actualized development costs. For instance, if full development costs are $20 million but have only a 5% chance of being incurred, the expected cost is \$1 million, as stated in the example above. This is one of the reasons pull mechanisms appear very large relative to expected costs.
- Better selection of high-quality firms. Firms differ in their underlying probability of technical success, but funders often cannot observe these differences. Under push funding—where funds are provided upfront to any qualified applicant—the funder may inadvertently support firms with low chances of success. Under a pull mechanism, firms are only paid if they deliver results, so only those confident in their likelihood of success choose to participate, improving selection.
- Stronger incentives for cost-effectiveness. Because pull rewards firms only upon successful outcomes, they have an incentive to minimize unnecessary expenditures and choose efficient development paths. This reduces the funder’s downside risk of overspending, waste, or misuse of funds compared to push funding, where costs are reimbursed regardless of outcome.
- Flexibility when the optimal solution is uncertain. In many cases, the funder cannot identify in advance which technology, platform, or approach is most promising. Pull mechanisms allow multiple possible solutions to compete on equal footing; the funder does not need to “pick a winner” upfront.
- Incentives to deliver products to market, not just prototypes. Push funding may support research that never translates into deployable products. Some pull incentives (such as AMCs) reward not just technical progress but actual delivery and use, helping to avoid “false positives” that look promising in development but never achieve real-world uptake.
| Variable Name | Description |
|---|---|
| $\theta$ | Target probability of success |
| $n$ | Number of attempts |
| $p$ | Per-attempt probability of success |
| $\mathbb{E}[c]$ | Present value of expected cost per innovation attempt |
5. Assessing Cost-Effectiveness
Additional Cost Considerations
Any complete analysis of an innovation incentive program will need to include other necessary costs, beyond those necessary to induce adequate firm entry. In some cases, these additional factors may lower the total necessary cost of the program. These include:
- Monitoring and verification. Pull mechanisms involve paying for success. This requires being able to monitor one’s outcome of interest (e.g. verifying sales, adoption, impact on outcome measures). In some cases, these costs will be significant and should be considered as part of the overall costs of the program in addition to the direct cost of the pull reward.
- Costs of designing and implementing the pull fund. The institution designing and implementing the pull funding program will incur personnel, administrative, and overhead costs. These are likely to be orders of magnitude smaller than the cost of incentivizing innovation.
- Third-party purchases. Innovations will differ to the extent to which the funder will need to make up the whole market. For some innovations, there will be significant private demand which can be subtracted from the overall market size to calculate the size of the pull fund needed. Purchases from other agents are not part of the cost to the funder. However, costs paid by other agents that would be unlikely to occur in the absence of the pull mechanism should be considered when appraising the benefits and costs of the intervention. For example, suppose one needs a \$1 billion market (present value) to create at least a 50% probability that a new diagnostic will be invented but there is only \$600 million in private demand. In this case, the needed pull size will only need to be \$400 million.
Cost Effectiveness
To determine the overall cost-effectiveness of a pull mechanism, we compare the total costs to the expected benefits of the target innovation.
Pull mechanisms are unique investments in that funders either receive the social return of successful innovation or, should no firm succeed at innovation, the chance to spend their funds on other opportunities at a later date. The money does not disappear if no one succeeds at innovation.
Two useful metrics for considering cost-effectiveness are the benefit-cost ratio and net present value of the project.
The benefit-cost ratio (BCR) indicates the social value generated per dollar spent. It is computed by dividing the expected benefits by the total costs. A BCR greater than 1 signals that the benefits outweigh the costs.
| Variable Name | Description |
|---|---|
| $p$ | Per-attempt probability of success |
| $B$ | Benefits if the innovation succeeds |
| $F$ | Funder's budget |
| $\gamma$ | Social ROI of the funder's marginal alternate investment opportunity |
The net present value (NPV) directly quantifies the net societal value created by the pull mechanism, accounting for the time value of money. A positive NPV indicates the intervention generates a net gain for society. To calculate overall net present value teams should multiple the probability of program success by the difference of the present value of benefits and costs
| Variable Name | Description |
|---|---|
| $p$ | Per-attempt probability of success |
| $n$ | Number of firms that enter |
Together, the BCR and NPV provide a balanced assessment of the pull mechanism’s cost-effectiveness. This allows funders to make informed decisions about whether to proceed with the investment and how to optimize the target probability of success for maximum impact.
6. Conclusion
Determining the right size for a pull mechanism involves carefully balancing multiple factors to create effective innovation incentives while ensuring the efficient use of resources. This guide has laid out a framework for systematically analyzing the key components that drive pull mechanism sizing in the context of pure pull funding programs. The core analysis can be summarized in a series of steps:
Step 1: Gather Information on Innovation Costs and Risks
- Map out all development stages (R&D, testing, regulatory approval, etc.)
- For each stage, estimate the direct costs, probability of success, and duration
Step 2: Calculate Per-Attempt Expected Cost and Success Probability
- Combine stage-level information to find:
- Expected cost per attempt (accounting for early failures)
- Overall probability of success per attempt
- Apply appropriate discount rate to account for time value of money
Step 3: Determine Program Size
- Start with the base calculation of a pull mechanism to incentivize a single attempt: expected costs divided by the probability of success per attempt
- Determine the desired overall target probability of achieving the innovation by considering:
- The social value of the innovation
- The marginal cost of incentivizing additional attempts
- Available budget constraints
Step 4: Factor in Additional Considerations
- Convert the present value into nominal terms based on the expected payment structure timeline
- Add program costs for:
- Program administration costs
- Monitoring and verification
- Subtract the following from the incentive size:
- Expected private market revenue
- Other funding sources
While our framework provides a structured approach to sizing decisions, we’ve made several simplifying assumptions that practitioners should consider carefully:
- We assumed firms are identical. In reality, firms have different capabilities, and a pull mechanism will incentivize the best-suited firms to participate.
- We assumed firms make decisions simultaneously and follow the same development timeline. In practice, firms may make innovation attempts sequentially. This could increase the probability of success of a pull mechanism over a longer timeframe.
- Our base model treats attempts as independent, though the appendix explores the correlation between attempts and multiple attempts made by an individual firm.
- Our model does not capture knowledge spillovers between attempts and firms nor the value of having multiple successful innovations rather than just one.
Additional resources about pull mechanisms are available on the Market Shaping Acceleerator’s website. We also welcome practitioners to contact us to discuss specific applications, as each situation requires careful consideration of context-specific factors that may influence optimal incentive design.
Appendix A: Enriching the Simple Model
Within and across firm attempts: pipeline approaches to innovation
The above analysis assumes a competitive market where each firm makes a single attempt. However, this assumption may be unrealistic, as in reality firms tend to make multiple attempts; why go through all the effort hiring researchers and expensive equipment only to make a single attempt? Suppose research trials have a 2% success rate and a pull designer desires a 50% probability of at least one success: the above formula suggests that the funder now needs to incentivize at least 35 attempts. The notion that 35 attempts will be satisfied by 35 different firms seems unlikely: maybe five firms conducting seven attempts each is more plausible.
As a result, we can consider how the model changes if one imposes constraints on the number of firms that are capable of respondingIn reality, there is no “hard cap” on the number of firms that exist in a market. After all, a large incentive can induce new firms to enter. We impose this limit not as a reflection of reality but rather to tractably illustrate the principle. Intuitively, this restriction requires firms to initiate multiple attempts each. The marginal cost to the funder to induce that additional attempt will be higher than under the previous one-attempt-per-firm model as a firm now faces a risk of self-competition. Intuitively, the fewer possible firms in the market, the greater the premium the funder must pay.
| Variable Name | Description |
|---|---|
| $\theta$ | Target probability of success |
| $n$ | Number of attempts |
| $p$ | Per-attempt probability of success |
| $X$ | Present value of pull incentive |
| $\mathbb{E}[c]$ | Present value of expected cost per innovation attempt |
$$ X \cdot p \cdot (1-p)^{n-1} \geq \mathbb{E}[c]$$
The number of attempts needed to achieve at least a $\theta$ probability of success does not change. As a result, we plug $n \geq \frac{\ln(1-\theta)}{\ln(1-p)}$ into the above equation, producing:$$ X \geq \frac{1-p}{p \cdot (1-\theta)} \cdot \mathbb{E}[c]$$
Since $\theta = 1-(1-p)^n > p$, it is always the case that the pull size necessary for a monopolist is larger than for a competitive market. In reality, neither a monopolist nor a perfectly fragmented market is likely. Some in-between world is far more probable. The extent of the premium rapidly declines as the number of firms increases. The following section details the math, though one is encouraged to skip to the charts and tables for the intuition. Assume $k$ firms in the market. Because of the homogeneity assumption, each firm will now undertake $\frac{n}{k}$ attempts. Let us now define $\mathbb{V}(X,k)$ as the present value of the pull size given the number of firms in the market.$$ V(X,k) \cdot p \cdot (1-p)^{\frac{n}{k} - 1} \geq \mathbb{E}[c]$$
In the above, a firm's marginal additional attempt is only valuable if the marginal attempt succeeds ($p$) and all other attempts by that firm ($(1-p)^{\frac{n}{k}-1}$) fails.$$\mathbb{V}(X, k) = \sum_{i=0}^{k-1} \frac{1}{i+1} \cdot \mathbb{P}(\text{successes} = i) \cdot X $$
In other words, the value of winning depends on the number of other firms that are successful. The term $\frac{1}{i+1}$ represents the share of $X$ received by the firm given $i$ other successful firms.$$ \mathbb{P}(\text{successes} = i) = \binom{k-1}{i} \cdot \left(p^{\frac{n}{k}}\right)^i \cdot \left((1-p)^{\frac{n}{k}}\right)^{k-1-i} $$
The above equation is simply an expansion of the binomial formula where the number of attempts per firm is $\frac{n}{k}$ and is equivalent to calculating the probability of any specific number of heads when one flips a weighted coin $k-1$ times when the probability of any given flip resolving as heads if $(1-p)^{\frac{n}{k}}$. In cases where the number of attempts is not divisible by the number of firms, we split proportionately. For example, if there are 27 desired attempts and only five firms, three firms will conduct five attempts each, while two firms will conduct six. In practice, this approach results in a higher needed pull size than the “one attempt per firm” model, but a lower pull size than the monopolist case. Intuitively, the more attempts per firm, the larger the price premium grows. That said, the result tails off so quickly after the monopolist case that this factor is rarely worth considering. For example, if the probability of success per attempt was 6% and the target probability was 50%, having only four firms results in a pull size 8.8% larger than if there were enough firms to have one attempt per firm, but 25% cheaper than if there were only a single firm. However, if the target probability was 66.67% instead, a four firm world requires a 15.8% premium over the one-firm-per-attempt, but a 59.3% discount from a monopolist situation.Correlated probabilities of success
The above analysis assumes that all attempts are independent, that is, that different innovation attempts are not related to each other and do not affect each other’s chances of succeeding. This assumption is transparently unrealistic, for two reasons. First, the innovation itself may be impossible (or at least impossible in the near future). No amount of separate attempts would allow ancient Roman scientists to construct a modern computer. This is akin to saying that there is an upper bound on the total probability that at least one attempt succeeds. Second, different attempts all made by the same firm are likely more correlated to each other than they are to attempts made by other firms.
Unfortunately, the level of correlation between attempts is unobservable, and designers should use their discretion to incorporate this factor. In general, considering correlation between attempts will raise costs, generally on the magnitude of 10-50%.
| Variable Name | Description |
|---|---|
| $\theta$ | Target probability of success |
| $n$ | Number of attempts |
| $p$ | Per-attempt probability of success |
| $X$ | Present value of pull incentive |
| $\mathbb{E}[c]$ | Present value of expected cost per innovation attempt |
| $\eta$ | Global possibility parameter |
| $\gamma$ | In-firm possibility parameter |
| $m$ | Number of attempts per firm |
$$\mathbb{P}_{firm}(success \geq 1) = \gamma \cdot (1-(1-p)^m) $$
The probability that all of a firm's attempts fail is the sum of the probability that the approach is fundamentally infeasible ($1 - \gamma$) and the probability that the approach is feasible ($\gamma$) but all of the attempts fail regardless ($(1-p)^m$). Putting together, one gets$$\mathbb{P}_{firm}(success \geq 1) = 1 - [(1-\gamma) + \gamma \cdot (1-p)^m]$$
This equation simplifies to the previous equation. Now to introduce the global possibility parameter $\eta$.$$\mathbb{P}_{all}(success \geq 1) = \eta \cdot (1 - [1-\gamma \cdot (1-(1-p)^m)]^k) \geq \theta$$
The goal is to get the probability of success to exceed the target threshold. With the new global possibility parameter, the probability that at least one firm succeeds is the probability that any given firm succeeds conditional on the innovation being possible multiplied by the possibility $\eta$ that the innovation is at all possible.$$m \geq \frac{\ln(1-\frac{1-(1-\frac{\theta}{\eta})^{\frac{1}{k}}}{\gamma})}{\ln(1-p)}$$
Algebraic rearrangement of the second equation produces the third. There is no simplified form for the necessary pull size. Instead, one can insert the value of $m$ into the equation for the finite firms, replacing the $\frac{n}{k}$ term that represnts attempts per firm with the value for $m$ found in the equation above.In practice, introducing correlation substantially increases the needed pull size, as the number of attempts needed to reach the same probability of success increases dramatically.
Subsidy passthrough
Our existing method estimating the appropriate size of an advance market commitment (AMC) assumes that the innovative firm receives the entirety of the subsidy pool, with no effect on the price of the novel good sold. For example, this model assumes that a three dollar per-unit subsidy would increase the profit per good sold by precisely three dollars.
This assumption is transparently erroneous. Absent binding supply constraints, increasing the profitability per good sold will induce firms to produce more. Assuming a downward-sloping demand curve, greater supply will in turn induce lower prices. By driving prices lower, the firm captures only part of the subsidy, while consumers capture the rest. A three dollar per-unit subsidy will not increase profit per good sold by three dollars, but instead some unknown value less than three dollars. We consider the transmission of the subsidy to the consumer the problem of subsidy passthrough.
In the existing model, doubling the number of firms cuts the profit per fim in half. A monopolist gets 100% of the total pool of money, a duopolist gets 50%, a triopolist 33.3%, etc. But, as above, this assumption is inaccurate for two reasons. First, competition induces firms to pass some of the subsidy onto the consumer, so a duopolist should get less than 50% of the profit of a monopolist. Second, competition induces firms to expand supply, by weakening the incentive for strategic withholding, which may increase total profits.
Let us start by assuming Cournot competition, where firms decide the amount of quantity to produce and then the market sets the price. This model of competition hews more closely to reality for manufactured goods where capacity (e.g. the size of the factory) must be installed ahead of time; firms cannot costlessly and instantaneously ramp up their supply. The precise number depends on the exact slope of the supply and demand curve, but in general, the Cournot model produces a roughly 55% reduction in profit when moving from one to two firms. More generally, each of the k-firms will receive $\frac{4}{(k+1)^2}$ of the profit of a monopolist, instead of $\frac{1}{k}$.
Assume a linear demand curve of the form $P(q) = a - b \cdot q$ where $P$ represents price, $a$ and $b$ are constants, and $q$ is the quantity set. The cost of production is constant at $c$
and the subsidy is $s$.Profit $\pi$ for a monopolist is set by the formula $\pi = (p - c + s) \cdot q$
We can plug in the formula for p we found above and get $\pi = (a - b \cdot q - c + s) \cdot q$
This expands out to $\pi = a \cdot q - b \cdot q^2 - c \cdot q + s \cdot q$
To maximize profit, we take the derivative of profit with respect to quantity, which is $\frac{d\pi}{dq} = a - 2 \cdot b \cdot q - c + s$, and set equal to 0.
In the end, we get $q = \frac{a - c + s}{2b}$
Plug that value of $q$ into our formula for profit and we get $\pi = \frac{(a-c+s)^2}{4 \cdot b}$
How about for a duopolist? Much of the set up is the same, except that $q = q_1 + q_2$ and the firm can only control $q_1$. So we have $\pi_1 = (a - b \cdot (q_1 + q_2) - c + 2) \cdot q_1$
Because only $q_1$ is under our control, we take the derivative of $\pi$ with respect to $q_1$ and get $\frac{d\pi}{dq_1} = a -2 \cdot b \cdot q_1 - b \cdot q_2 - c + s = 0 $
Since both firms are identical, we can now say $q_1 = q_2$. We can thus solve for $q_1 = \frac{a - c + 2}{3 \cdot b}$. Note that this results in a per-firm quantity $\frac{2}{3}$ of the monopolist quantity (and a total quantity that's $\frac{4}{3}$ as large).
Plugging into the formula for profit, we get $\pi_q = \frac{(a-c+s)^2}{9 \cdot b}$. We can thus see that the profit is $\frac{4}{9}$ as large with the duopolist as with the monopolist firm.
Extending to k-firms follows the same method. We just have $q = q_1 + q_2 + \ldots + q_k$ and $\pi_1 = (a - b \cdot (q_1 + q_2 + \ldots + q_k) - c + s) \cdot q_1$.
One can still take the derivative $\frac{d\pi}{dq_1} = a - 2 \cdot b \cdot q_1 - b \cdot q_2 - \ldots - b \cdot q_k - c + s = 0$. Since $q_1 = q_2 = \ldots = q_k$, we can see that a k-opolist receives $\frac{(a-c+s)^2}{(k+1)^2}$.
The ratio of profit for a k-opolist to a monopolist is thus just $\frac{4}{(k+1)^2}$. A duopoist receives 44% of the profit, a triopolist receives 25%, etc.
This result is not unique to linear demand functions. "Normal-looking" exponential or logarithmic demand functions that are standard in the literature all will produce very similar results. The sole exception will be "unusual" looking demand curves (e.g. perfectly elastic or inelastic demand, or step-wise functions, etc.)
Installing a 10-20% premium on top of the existing model would be an over-simplification, though not a terribly inaccurate one. A small AMC where the probability of multiple successful winners is very low will need little premium. A large AMC where one targets a high probability of success (and thus a high probability of duplication) will need a larger premium. In the chart below, the y-axis represents the increase in relative payments incorporating passthrough to the previous estimates, which assumed 0% passthrough.
Appendix B: A Full Example
The following button will walk the reader through a complete example from start to finish, using the case study of vaccines to reduce enteric methane emissions from cattle that the MSA has worked on over the last year.
Step 1 and 2 - Expected Costs and Risks
In previous examples, we used this table from Jensen 2014 to find the expected cost and risk per innovation attempt. Even though the all-in cost of seeing an attempt through from initiation to success is \$3.4 million, because the supermajority of attempts fail far before the test and approval phases, the expected cost at time of initiation is only \$1.904 million. After discounting using an 8.05% discount rate (to recognize the fact that costs borne in future years count “less” than costs incurred in the present, the “expected cost per attempt” becomes \$1,799,000 with a probability of success of 2.7%.
| Phase | Cost (2024 1000 $) | Unconditional Probability of Success | Probability of reaching step | Expected Cost (1000 $) | Discounted Present Value |
|---|---|---|---|---|---|
| R&D | 955 | 58.40% | 100% | 955 | 955 |
| R&D | 955 | 58.40% | 58.40% | 554 | 513 |
| R&D | 955 | 58.40% | 34.10% | 321 | 275 |
| Patent | 201 | 75% | 19.50% | 39 | 31 |
| Test | 153 | 44.70% | 14.60% | 22 | 16 |
| Test | 153 | 44.70% | 6.60% | 10 | 7 |
| Approval | 71 | 90% | 3.00% | 2 | 1.3 |
| Release | - | 100% | 2.70% | - | 0 |
| Overall | 3,443 | - | 2.70% | 1,904 | 1,799 |
Step 3 - Calculating Needed Pull Size
$$ X \geq \frac{\ln(1-\theta)}{\theta \cdot \ln(1-p)} \cdot \mathbb{E}[c]$$
We can now plug in these numbers to our formulas shown above, using \$1,799,000 as $\mathbb{E}[c]$ and 0.027 as $p$. The target probability of success $\theta$ is a choice of art, and the following chart shows the needed pull size depending on the choice of the target probability of success.Step 4 - Adding in Complications
$$ V(X,k) \cdot p \cdot (1-p)^{\frac{n}{k} - 1} \geq \mathbb{E}[c]$$
$$\mathbb{V}(X, k) = \sum_{i=0}^{k-1} \frac{1}{i+1} \cdot \mathbb{P}(\text{successes} = i) \cdot X $$
$$\mathbb{P}(\text{successes} = i) = \binom{k-1}{i} \cdot \left(p^{\frac{n}{k}}\right)^i \cdot \left((1-p)^{\frac{n}{k}}\right)^{k-1-i} $$
$$m \geq \frac{\ln(1-\frac{1-(1-\frac{\theta}{\eta})^{\frac{1}{k}}}{\gamma})}{\ln(1-p)}$$
If one wishes, one can now add in some of the complications from before, such as reasonable guesses about the number of participating firms and the extent to which this innovation is even “possible”. Both of these terms are highly subjective, and should be informed by interviews with industry experts. For this example, let us assume there are at most six firms in the market, with a global possibility parameter of 75% ($\eta$) and a within-firm possibility parameter of 66.67% ($\gamma$) (in other words, there exists only a three-quarters chance that this innovation is possible and, conditional on the innovation being possible, there is only a two-thirds chance that any given pathway to achieve that innovation could work). Note: As a side note, when one adds in these correlations, we do need to adjust the value of the probability of success $p$ conditional on the innovation being possible. If our data says (e.g.) that these trials succeed 2.7% of the time, then the probability that the trials succeed conditional on being possible at all (using our parameters) becomes $0.027 \div \frac{3}{4} \div \frac{2}{3} = 0.054 = 5.4\%$Converting into Nominal Terms
The above formula gives only the present value to the firm. But it does not tell us exactly how much money one needs to raise. To do so, we need to understand how long this innovation might take. More distant innovations will result in longer lags between research investment and ultimate payout, and thus require larger rewards in order to induce entry. Industry literature and expert interviews can give some bounds on this estimate. In this case, let us assume that research takes eight years from start to finish.
The structure of the pull incentive affects the next step. In the case of a prize, once the research target has been achieved, the firm can receive all of the reward. In the case of an advance market commitment (AMC), the payments are structured as per-unit subsidies. For this example, let us assume the incentive is an AMC that lasts for ten years and payments occur contemporaneously with sales.
In this case, we would need to have a rough model of rollout, which can be informed by the rollout of other products in the space. This payment structure will further increase costs, because sales growth tends to take time, which further increases the wedge between initial investment and ultimate payout.
At this step, one also needs to include marginal costs of production and add those to the needed subsidy, while subtracting out any private willingness-to-pay. In the case of our methane-reducing vaccines, both the cost of production and private willingness-to-pay might be extremely low (e.g. \$2.25/dose and \$0, respectively).
Consider the above logistic chart of potential rollout. Cognizant that sales made in later years are less valuable to the company in present-value terms than sales made earlier, we can use computers to find the value of a per-unit subsidy such that the present value of all of these future payments exceeds \$227.8 million. To do so, one takes the number of doses projected to be sold each year and discount them (using the firm’s 8.05% discount rate) into “effective number of doses in current-year terms”.
| Year | Projected Vaccinations (millions) | Discount Factor | Effective Present Vaccinations (millions) |
|---|---|---|---|
| 0-8 | 0 | 0.538 | 0 |
| 9 | 0.302 | 0.498 | 0.15 |
| 10 | 0.797 | 0.461 | 0.367 |
| 11 | 2 | 0.427 | 0.854 |
| 12 | 4.518 | 0.395 | 1.785 |
| 13 | 8.4 | 0.365 | 3.066 |
| 14 | 12.28 | 0.338 | 4.15 |
| 15 | 14.8 | 0.313 | 4.63 |
| 16 | 16 | 0.29 | 4.64 |
| 17 | 16.5 | 0.268 | 4.422 |
| 18 | 16.69 | 0.248 | 4.139 |
| Total | 92.287 | 28.203 |
Then divide the total needed present value (\$227.8 million) by the “effective present value of doses” and add in the \$2.25 to offset the cost of production to get a final answer of \$10.33/dose. We now have a total nominal price tag of \$952.4 million (multiplying the total subsidized doses by the total nominal cost). If we assume that the funder can earn a risk-free 2% interest rate on the money they put in escrow, the total amount of money they need to commit becomes \$702.1 million. Why is this value so much greater than the original \$227.8 million we found? Because the time value of money to the firm (8.05%) far exceeds the interest that the funder can get on the money they put in escrow (2%).
Appendix C: Firm Heterogeneity
What happens when firms have heterogeneous costs or probabilities of success? While we are reserving a full treatment of this situation for a future update to this page, it is worth briefly going through what additional considerations may arise in this more general environment.
First, suppose firms have the same probability of success but different costs, and that the funder cannot observe these costs. To induce a group of firms to enter, the expected reward must exceed the highest cost among the group of firms. For example, if a funder faces a market of 100 potential firms and wants 10 firms to enter, a reward should be sized so that the firms with the 10 lowest costs want to enter. This ensures that the firm with the ninth highest cost (and those with any lower costs) also enter. Thus, sizing pull incentives works similarly, but accounts for the fact that as the number of firms grows, the maximal innovation costs of this group also increases.
If firms differ in their probabilities of success, and these are unobservable to the funder, a funder’s problem is more subtle. Firms with higher probabilities of success are more likely to win a portion of the pull prize, and thus dissuade firms with lower probabilities of success from entering. A pull must then be appropriately sized to ensure that firms with lower probabilities of success are being sufficiently compensated for the expected share of the prize that more efficient firms are taking from them. On the flipside, the existence of these externalities ensures that firms with the highest probabilities of success are more likely to participate in the first place.
Finally if firms differ both in their probabilities of success and their costs of development, both effects are at play. Nevertheless, given information about the distribution of costs and probabilities of success in the population of firms, it is possible to calculate how large a pull fund must be to achieve a target probability of success. [Stay tuned for an update addressing this point soon!]