Software's premium rests on one belief — near-zero marginal cost. AI is the first real threat.

Why does the market pay software companies multiples that traditional industries can only dream of? Strip away the jargon and it comes down to one number: gross margin. A pure-software company serves its thousandth customer for almost nothing on top of its nine-hundred-ninety-ninth — multi-tenant cloud makes the marginal cost near-zero — so a dollar of revenue at ~80% margin is worth far more than a dollar at 40%. That near-zero marginal cost is the physical foundation of the entire compounding-machine story, and for two decades it went unquestioned.

AI is the first thing to seriously question it. Every AI feature you ship has an inference cost — real compute burned on every single call — that lands in cost of revenue and doesn’t politely disappear. The mechanism has a name: COGS creep. If incremental inference cost enters COGS with no matching price increase, cost reduction, or retention gain, gross margin gets pressured. For the first time, “software is forever high-margin, therefore forever worth a high multiple” is genuinely in question — and gross margin has to move from a background assumption to a foreground valuation variable.

You can already feel the mechanism in a live comparison — carefully. Snowflake, which resells cloud compute inside its product, runs a gross margin of ~67%; the pure-software Veeva runs ~75%. That gap illustrates what happens when variable compute enters cost of revenue: software can still be high-margin, but its margin becomes more exposed to compute intensity. Read it as a mechanism reference, not a causal verdict — you cannot attribute the ~8-point difference to compute alone (product mix, services, scale, and pricing all move it too), still less use it to estimate AI’s future drag on some other company.

This is why the AI era forces a re-rating that has nothing to do with growth. For twenty years, a software company could slow down and still command a premium, because its margin was assumed permanent. If inference cost turns gross margin into a live variable — different by product, by pricing power, by how efficiently a company runs its models — then two companies growing at the same rate can deserve very different multiples, purely on the trajectory of their margins. Gross margin is quietly becoming a source of valuation dispersion, not just a level.

So the discipline is to stop treating gross margin as a given. Ask, for every AI feature: is it charged, or bundled free as defense? Does the incremental inference cost get covered by price, efficiency, or stickiness — or does it just quietly eat the margin? Free bundling isn’t automatically the worst outcome (if it defends retention or drives cross-sell it can still pay), but it must be weighed as incremental cost vs. incremental benefit. Gross margin was the assumption the whole valuation rested on. AI just turned it back into a question.

This is Chapter 16 of “Compounding in the Cloud.”