PSI · Evidence review

What outside research says for and against PSI.

PSI is a theory in the proving process. None of the sources below test PSI directly. They come from neighboring fields (technology adoption, implementation science, health IT, project forecasting, and measurement theory) and, on the surface, either support the idea, challenge it, or both. We may or may not lean on any of them. The only real test is the calibration loop: scored deals paired with what actually happened.

supports · 13mixed · 11challenges · 5

The short version

Creating value and capturing it are different questions
Strong support
Change cost and adoption drive realized value
Support (frameworks are descriptive, not formulas)
Wide uncertainty bands
Support (tails may be fatter than modeled)
Calibrating scores against outcomes
Support, once PSI maps to a probability
Log-compressing Gain
Mixed: theory says scale to the buyer, not raw dollars
Averaging 1 to 10 ratings and dividing by them
Challenged: ordinal ratings, needs outcome data
Trusting seller-entered estimates
Challenged: optimism and misrepresentation are documented
"70% of change efforts fail"
Do not cite: no valid empirical basis

What this means for the model

  1. Keep the core thesis. Separating value creation from value capture is the best-supported part of PSI.
  2. Treat the formula as a hypothesis. No source derives it. The weights, zone cut points, and log transform are calibration parameters until outcome data says otherwise.
  3. Scale Gain to the buyer next. $500K means something different to a three-provider practice than to an IDN. Gain as a share of buyer revenue, or as a multiple of the investment, is the theory-consistent next step.
  4. Discount seller inputs. Use the evidence levels, have implementation staff score Pain and Experience, and track each scorer's accuracy over time.
  5. Do not overclaim. Leave unsourced failure statistics out of sales material.

Technology acceptance and adoption

supports
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.

(core thesis). The founding TAM paper. Perceived usefulness correlated with current usage (r = .63) and predicted future usage. Ease of use looked more like an antecedent of usefulness than a direct driver of usage. Supports the idea that adoption depends on how users see the change, not just objective value. Caveat: PSI's Experience factors are operational metrics, not TAM's perception constructs.

doi.org/10.2307/249008
supports
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478.

(Experience factors), with caveats. UTAUT unified eight earlier models around four determinants: performance expectancy, effort expectancy, social influence, and facilitating conditions. PSI's training and support factors map loosely to facilitating conditions, and champion strength to social influence. PSI has no moderators and no performance or effort expectancy terms.

doi.org/10.2307/30036540
supports
Rogers, E. M. (2003). Diffusion of Innovations (5th ed.). Free Press. ISBN 9780743222099.

(core thesis). Adoption depends on relative advantage, compatibility, complexity, trialability, and observability, not on relative advantage alone. Complexity and compatibility correspond roughly to PSI's Pain dimensions.

www.simonandschuster.com/books/Diffusion-of-Innovations-5th-Edition/Everett-M-Rogers/9780743222099
supports
DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30.

Net benefits are an outcome of system, information, and service quality, flowing through use and user satisfaction. This matches PSI's split between value created and value captured through use.

doi.org/10.1080/07421222.2003.11045748

Implementation science in healthcare

mixed
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4, 50.

CFIR's five domains (intervention, outer setting, inner setting, individuals, process) support PSI's view that organizational context drives implementation. The authors say CFIR does not specify relationships between constructs, so PSI's weights and formula cannot be derived from it.

doi.org/10.1186/1748-5908-4-50
supports
Damschroder, L. J., Reardon, C. M., Opra Widerquist, M. A., & Lowery, J. (2022). The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science, 17, 75.

The update puts more weight on the people who receive the innovation. That is relevant to PSI's adoption inputs (end users, champions). Its constructs are still descriptive, not a scoring formula.

doi.org/10.1186/s13012-022-01245-0
supports
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82(4), 581-629.

Adoption in health service organizations depends on organizational and contextual factors, not only on the merits of the innovation.

doi.org/10.1111/j.0887-378X.2004.00325.x
supports
Greenhalgh, T., et al. (2017). Beyond adoption: A new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies (NASSS). Journal of Medical Internet Research, 19(11), e367.

(core thesis), with a caveat on the formula. Programs with complexity across several domains "rarely, if ever" became mainstream, which directly backs the split between creating and capturing value. NASSS stresses interacting complexity that changes over time, which sits uneasily with averaging four independent Pain ratings.

doi.org/10.2196/jmir.8775
supports
Proctor, E., et al. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65-76.

Defines implementation outcomes (acceptability, adoption, feasibility, penetration, implementation cost, sustainability) as separate from service and clinical outcomes. This gives formal backing for scoring "will they capture it" apart from "is the value real", plus vocabulary for the Experience factors (for example, penetration in place of adoption %).

pubmed.ncbi.nlm.nih.gov/20957426/

Organizational change

supports
Kotter, J. P. (1995). Leading change: Why transformation efforts fail. Harvard Business Review.

(weak evidence base). Eight common errors drawn from observing 100+ companies. Supports PSI's weight on change management, but it is a practitioner essay, not an empirical study.

hbr.org/1995/03/leading-change-why-transformation-efforts-fail-2
challenges
Hughes, M. (2011). Do 70 per cent of all organizational change initiatives really fail? Journal of Change Management, 11(4), 451-464.

(popular statistics). Found no valid, reliable evidence for the "70% fail" figure. PSI material should not cite it.

doi.org/10.1080/14697017.2011.630506
mixed
Weiner, B. J. (2009). A theory of organizational readiness for change. Implementation Science, 4, 67.

Defines readiness as shared commitment to the change plus shared belief in the ability to carry it out. Supports including readiness in PSI. But a seller's single 1 to 10 rating does not measure that shared state, which is an argument for collecting Pain and Experience from the buyer side as well.

doi.org/10.1186/1748-5908-4-67

IT project failure and cost overrun

mixed
Flyvbjerg, B., & Budzier, A. (2011). Why your IT project may be riskier than you think. Harvard Business Review, 89(9).

Across 1,471 IT projects the average cost overrun was 27%, but about one in six was a "black swan" averaging a 200% cost overrun. Supports wide uncertainty bands. Challenges PSI's triangular distributions, which are bounded and cannot produce fat tails.

hbr.org/2011/09/why-your-it-project-may-be-riskier-than-you-think
challenges
Eveleens, J. L., & Verhoef, C. (2010). The rise and fall of the Chaos report figures. IEEE Software, 27(1), 30-36.

(popular failure statistics). Argues the Standish CHAOS figures are meaningless because they average estimates with unknown biases. Also a caution for PSI, which averages ratings from scorers who may have different biases.

doi.org/10.1109/MS.2009.154
mixed
Flyvbjerg, B. (2006). From Nobel Prize to project management: Getting risks right. Project Management Journal, 37(3), 5-15.

Attributes forecast error to optimism bias and strategic misrepresentation, and recommends reference class forecasting. Supports grounding Gain and timelines in past deals. Challenges taking seller-estimated Gain at face value.

doi.org/10.1177/875697280603700302

EHR implementation and health IT value realization

mixed
Buntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011). The benefits of health information technology: A review of the recent literature shows predominantly positive results. Health Affairs, 30(3), 464-471.

92% of recent articles reached positive conclusions, but provider dissatisfaction remained a barrier. Value creation is real; capture depends on people.

doi.org/10.1377/hlthaff.2011.0178
mixed
Jones, S. S., Rudin, R. S., Perry, T., & Shekelle, P. G. (2014). Health information technology: An updated systematic review with a focus on meaningful use. Annals of Internal Medicine, 160(1), 48-54.

Most of 236 studies reported positive effects on quality, but 61% reported little about implementation context. That limits how far published evidence can calibrate PSI's organizational and adoption weights.

doi.org/10.7326/M13-1531
supports
Kellermann, A. L., & Jones, S. S. (2013). What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Affairs, 32(1), 63-68.

(core thesis). RAND's 2005 projection of more than $81 billion a year in savings had not materialized. The authors tie the gap to usability, interoperability, and care-process redesign. A direct example of business-case size not predicting realized value.

doi.org/10.1377/hlthaff.2012.0693
mixed
Fleming, N. S., Becker, E. R., Culler, S. D., Cheng, D., McCorkle, R., da Graca, B., & Ballard, D. J. (2014). The impact of electronic health records on workflow and financial measures in primary care practices. Health Services Research, 49(1 Pt 2).

Across 26 practices, productivity and net income dipped after EHR go-live, then recovered to or near earlier levels by 12 months. Supports a time-to-value factor and a post-go-live dip. The effects were smaller and shorter than expected.

doi.org/10.1111/1475-6773.12133

Sales forecasting and qualification

challenges
Lovallo, D., & Kahneman, D. (2003). Delusions of success: How optimism undermines executives' decisions. Harvard Business Review, July 2003, 56-63.

(input quality), SUPPORTS (calibration loop). Executives routinely overstate benefits and discount costs. Applies directly to seller-entered Gain (likely inflated) and Pain (likely understated). Companion: Flyvbjerg, B. (2003). Delusions of success: Comment on Dan Lovallo and Daniel Kahneman. HBR, December 2003, 121-122. https://arxiv.org/abs/1305.0741. Argues that deliberate misrepresentation also drives forecast error, which matters because sellers have an incentive to inflate PSI inputs.

hbr.org/2003/07/delusions-of-success-how-optimism-undermines-executives-decisions
supports
Toman, N., Adamson, B., & Gomez, C. (2017). The new sales imperative. Harvard Business Review, 95(2), 118-125.

(indirect). B2B buying has become complex enough to stall customers, so sellers should track buying progress. This matches PSI's attention to buyer-side friction and staleness. Practitioner research, not peer-reviewed. The widely quoted "6.8 stakeholders" figure could not be confirmed from the primary text, so do not cite it from here.

hbr.org/2017/03/the-new-sales-imperative

Scoring and measurement critiques

mixed
OECD & European Commission Joint Research Centre (2008). Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD Publishing.

(leans CHALLENGES). Normalization, weighting, and aggregation choices change results and should be stress-tested. PSI's Monte Carlo covers input uncertainty, not uncertainty about how the index itself is built.

doi.org/10.1787/9789264043466-en
challenges
Keiningham, T. L., Cooil, B., Andreassen, T. W., & Aksoy, L. (2007). A longitudinal examination of Net Promoter and firm revenue growth. Journal of Marketing, 71(3), 39-51.

(the "NPS for healthcare tech" positioning). Net Promoter did not clearly beat other metrics at predicting revenue growth. PSI should prove predictive validity before claiming single-number status.

doi.org/10.1509/jmkg.71.3.39
challenges
Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103(2684), 677-680.

(the arithmetic on Pain). 1 to 10 ratings are ordinal at best. Using their average as a ratio denominator assumes equal intervals and a true zero the ratings do not have.

doi.org/10.1126/science.103.2684.677
mixed
Norman, G. (2010). Likert scales, levels of measurement and the "laws" of statistics. Advances in Health Sciences Education, 15(5), 625-632.

(partial defense). Parametric statistics hold up well on Likert-type data, which partly defends averaging the ratings. It does not justify dividing by an average rating as if it were a ratio-scale quantity.

doi.org/10.1007/s10459-010-9222-y

Diminishing sensitivity (the log of Gain)

mixed
Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263-291.

Supports diminishing sensitivity to larger gains. But value is measured from a reference point, losses weigh more than gains, and the curve is not specified as a logarithm. Under prospect theory, Pain arguably deserves more weight than Gain.

doi.org/10.2307/1914185
mixed
Bernoulli, D. (1954). Exposition of a new theory on the measurement of risk (L. Sommer, Trans.). Econometrica, 22(1), 23-36. (Original work published 1738.)

The origin of logarithmic utility, which supports the log transform in principle. But in Bernoulli the value of a gain depends on existing wealth, which implies Gain should be scaled to the buyer's size, not taken in absolute dollars.

doi.org/10.2307/1909829

Calibration of forecasts

supports
Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1-3.

(the calibration loop), with a precondition. The standard accuracy score for probability forecasts. PSI outputs an index, so it must first be mapped to a predicted probability (of close, or of value realized) before Brier scoring applies.

doi.org/10.1175/1520-0493(1950
supports
Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., Scott, S. E., Moore, D., Atanasov, P., Swift, S. A., Murray, T., Stone, E., & Tetlock, P. E. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychological Science, 25(5), 1106-1115.

Training, teaming, and tracking improved forecast calibration. Supports training the people who enter PSI inputs and tracking their accuracy over time.

doi.org/10.1177/0956797614524255

Gaps

  • MEDDIC and MEDDPICC: no peer-reviewed evidence found. Treat them as practitioner methods.
  • CRM pipeline forecast accuracy: no verifiable peer-reviewed study found.
  • Buying-group size figures (5.4, 6.8 stakeholders): found only in secondary sources, so not cited.

Citations were checked against DOIs, PubMed/PMC records, or publisher pages. Two details still need a full-text check: the page range of Fleming et al. (2014) and the issue month of Kotter (1995). The same list, with notes, lives in the repository at docs/EVIDENCE.md.

How PSI worksTry the Lab