Expected ROI Is Necessary. It Is Not Sufficient.

A recently published paper by researchers and leaders at Compass (AI Strategy: How to Choose What AI Product to Implement) proposes evaluating AI initiatives through expected return on investment (eROI) rather than technical benchmarks alone.

That shift is important. It is also incomplete.

For years, AI conversations have been dominated by benchmark scores, model rankings, and context window comparisons. Those metrics answer an engineering question:

Can this model perform the task?

The eROI framework asks a fundamentally different question:

Should we invest capital in this AI initiative?

That is a better question. It moves the conversation from model performance to capital allocation. Instead of asking whether a model outperforms another on a curated test set, it asks whether deploying AI creates enough expected business value to justify the investment and the risk of getting it wrong.

For product leaders, that is genuine progress.

But it also exposes something the framework does not address: the assumption that choosing the right AI project is the hard problem.

In enterprise AI, selection is the easy problem. The harder problem begins after deployment.

Deployment begins value creation; it does not complete it. Enterprise AI realizes sustained value only when the organization can continuously trust that deployment.

The Core Insight of eROI — and Its Boundary

The eROI framework's most useful move is refusing to collapse a project into a single ROI estimate. Instead, it separates three components and rates them independently:

Value if Successful. What would the product be worth if it worked — including direct revenue or cost reduction, option value from reusable data assets and platform components, and strategic understanding of the business?

Likelihood of Success. How likely is it to work — distinguishing product certainty (will users adopt it?), technical certainty (can we build it?), data certainty (do we have the right inputs?), and scientific certainty (is the signal learnable to a useful accuracy at all?)?

Investment Required. What does it cost — including data science effort, data asset acquisition, and the full integration, productization, and governance overhead that most teams systematically underestimate?

The separation dissolves a common catch-22: teams cannot estimate ROI until they know whether the project will work, but cannot know whether it will work without building it. Separating Value if Successful from Likelihood of Success breaks the loop. A team can argue that a product would be valuable if it worked while separately assessing how likely that is.

This is a real contribution. The framework earns the problems it does not solve.

What the framework does not solve is everything that happens after the portfolio decision. And in enterprise AI, that is the majority of the economic problem.

AI Has Two Different Economic Lifecycles

Most organizations treat AI as if there is a single economic decision — the build decision. There are actually two, and they require completely different disciplines.

The Investment Decision (before build): Should we invest capital in this AI initiative? Expected ROI is the right tool. It surfaces scientific uncertainty, separates option value from direct product value, and structures the portfolio across archetypes: big bets, quick wins, and strategic bets.

The Operational Decision (after deployment): Can we continuously prove this system is functioning within its governance constraints, remaining reliable as data shifts, and creating business value without accumulating unhedged liability? Expected ROI has nothing to say about this.

The gap between these two decisions is where most enterprise AI risk actually lives.

A project with excellent eROI can become operationally catastrophic. A system that passes every pre-deployment check can silently drift into noncompliance. A model that performs well on production data at launch can degrade as retrieval indices re-embed, customer distributions shift, and implicit assumptions about the world stop holding.

Expected ROI calculates the upside of building an AI system. It does not protect against the downside of running one.

In traditional software, operational risk scales predictably with maintenance costs — roughly linear, reasonably foreseeable. In probabilistic AI, operational risk compounds non-linearly. A single unexplainable high-stakes decision — a loan denial, a medical recommendation, a fraud clearance — can trigger a regulatory examination that wipes out years of accumulated gross margin. The economics do not stop at deployment. They compound throughout the system's lifecycle.

What the Investment Decision Misses

The eROI framework identifies compliance, safety, and fairness requirements as line-items that increase "Integration & Productization Effort" — costs in the denominator that reduce the investment attractiveness of a project.

This is technically accurate and practically dangerous.

In high-consequence AI, governance is not a deployment cost. It is an ongoing control plane constraint. A system that passes its initial compliance review and then operates without executable governance controls accumulates Accountability Debt — the liability created by autonomous decisions that cannot later be reconstructed, explained, or justified.

Accountability Debt is the operational counterpart to Assumption Debt. Both accumulate silently. Both are invisible until an incident or regulatory examination forces them into the open. Both cost far more to resolve than to prevent.

The eROI framework captures the cost of building governance at deployment. It does not capture the cost of enforcing governance in production — because that cost is not a one-time investment. It is a continuous infrastructure requirement.

The Missing Infrastructure Layer

Many current AI investment frameworks focus primarily on project selection. They assume business value flows naturally from implementation.

Enterprise experience suggests otherwise. The missing layer — between deployment and sustained value — is continuous operational proof: the verifiable, ongoing demonstration that a deployed system is behaving within its governance constraints, producing decisions that can be forensically reconstructed, and remaining within authorized autonomy boundaries as data shifts, retrieval indices re-embed, and the world changes around the model.

Operational proof is not a monitoring dashboard. Monitoring tells you whether metrics are drifting. Operational proof tells you whether the specific decision the system made at a specific moment was within the rules that governed it at that moment — a distinction that matters when a regulator asks not "is your model performing well?" but "why did this customer receive this decision?"

The Technical Dependency Graph Problem

The reason project selection frameworks and model benchmarks fail as operational standards is that they treat AI as a standalone model.

In an enterprise environment, an AI decision is the output of a multi-layered technical dependency graph:

When a production system fails, the root cause is rarely that the foundation model suddenly lost intelligence. It is a dependency interaction failure: * A vector index re-embedded chunks under a modified schema. * A third-party model provider updated API weights without notice. * Context truncation silently dropped a governance rule from the prompt. * A human review queue timeout triggered an unmonitored fallback path.

Because liability lives in the interaction of these technical dependencies, an operational standard cannot evaluate a model in isolation. It must be a systemic forensic standard — capturing the complete execution graph at the exact moment a decision is made.

Expected ROI calculates the upside of building an AI system. Operational proof prevents the downside of running one. They are not competing disciplines. They are sequential ones.

From Portfolio Management to Fiduciary Defensibility

The deeper implication extends beyond any single organization.

The AI industry has spent three years optimizing intelligence. That turns out to be the easier problem.

Most AI frameworks today stop at project selection. They assume business value flows naturally from deployment. But the industry is not yet good at what comes next: continuously proving that deployed AI systems remain controlled, auditable, and worthy of the trust the business has extended to them.

Other industries have solved analogous problems:

Financial markets developed credit ratings and audited financial statements — not because regulators demanded a specific format, but because capital markets cannot price risk they cannot see.

Manufacturing developed ISO quality certifications — not because factories needed more paperwork, but because supply chains cannot trust components whose quality cannot be independently verified.

Cloud computing developed service-level agreements — not because customers needed contractual assurance of uptime, but because enterprises cannot build on infrastructure whose reliability has no common language.

Enterprise AI will require an equivalent layer: verifiable operational proof. Not because organizations need another monitoring metric, but because executives, regulators, and counterparties cannot price the risk of an AI system they cannot audit.

What an Industry Norm Looks Like

An industry norm for operational proof is not a rigid, static 500-page manual or a single test score. It is a risk-tiered, three-part evidentiary standard:

1. A Standardized Forensic Record: A common schema for capturing decision state (the Decision State Vector), recording model version, context provenance, policy hash, confidence score, and routing logic.
2. An Executable Policy Interface: A requirement that deterministic rules (thresholds, escalation, fallbacks) live in machine-readable Policy Registries tested in CI/CD pipelines.
3. A Risk-Tiered Verification Framework: A standard mapping that scales proof requirements to decision consequence — lightweight logging for low-impact recommendations, and full immutable ledgers with adversarial red-teaming for high-stakes financial, medical, or legal decisions.

Evolution and Adaptation

Like GAAP in accounting or building codes in civil engineering, an industry norm for AI is both dynamic and context-aware:

• It evolves over time: Just as GAAP evolved from 1930s industrial accounting to modern SaaS revenue recognition and digital assets, an AI operational norm evolves as technology shifts — from text retrieval and RAG to multi-modal agents and autonomous tool-use chains.
• It adapts to the project: Building codes scale from garden sheds to skyscrapers without changing structural principles. An operational norm scales proof requirements based on consequence, giving high-stakes systems strict fallback gates while leaving low-risk applications lightweight.

That standard does not yet exist as an industry norm. Organizations that build the infrastructure now will hold a structural advantage when it becomes one.

Four Questions. One Progression.

The economic history of enterprise AI can be organized around four questions, each of which represents a maturity stage the industry has addressed — or has not yet addressed.

Can it think? Benchmarks, model rankings, and capability evaluations answer this. The AI industry has largely solved it.

Should we build it? Expected ROI answers this. The industry is beginning to answer it well.

Can we trust it? Operational proof — continuous, verifiable demonstration that deployed systems behave within their governance constraints — answers this. The industry has barely started.

Can we insure it? As AI liability matures, financial markets and underwriters will eventually ask this fourth question. Transferable risk instruments require verifiable operational proof.

Each stage unlocks the next. Capital markets cannot price insurance on systems whose trust cannot be verified. You cannot verify trust without operational proof. You cannot invest in operational proof without first selecting the right systems to operate.

Expected ROI moves the industry from the first question to the second. That is genuine progress.

The third question is where enterprise AI strategy must go next.

One-Line Synthesis

Benchmarks optimize intelligence. Expected ROI optimizes investment. Operational proof optimizes trust. Enterprise AI requires all three — in that order.