The Economics of Open-Weight Inference

Ornn Data Research | Published: September 7, 2026

Abstract: Conventional wisdom suggests that each new NVIDIA GPU architecture renders its predecessor obsolete, leading to rapid depreciation. This paper challenges that narrative by analyzing how the demand for open-weight models extends the economic viability of legacy GPU families.

📉 Challenging the Obsolescence Thesis

The prevailing industry assumption has been that ~~new hardware eliminates the earning capacity of older GPUs~~. However, the rise of open-weight models creates a different economic incentive.

While closed-model access is governed by subscription tiers and provider-controlled rate cards (representing the marginal cost of extra tokens), open-weight models offer a flexible alternative. According to the Artificial Analysis Intelligence Index, when standardized for intelligence, the most affordable qualifying open-weight model can perform a task at approximately $\frac{1}{5}$ the cost of a comparable closed-model alternative.

The Self-Hosting Advantage

By renting hardware to self-host, costs drop further to between $0.12 and $0.35 per million output tokens at maximum utilization. This shift actually flips the traditional hardware hierarchy. For instance, using the gpt-oss-120b (a sparse model with $5.1 \times 10^9$ active parameters), the A100 is more cost-effective than the H100 across spot pricing and 3-to-5-year lease terms.

🛠 Key Findings & Market Metrics

Below is a summary of the research outcomes:

Cost Efficiency: Hosted open-weight models are cheaper at various intelligence thresholds, though closed models still win at the absolute frontier of performance.
Hardware Paradox: In specific sparse-model deployments, older Ampere chips (A100) outperform Hopper (H100) on a price-per-token basis.
Value Retention: The A100's 5-year rental price retains $\approx 80\%$ of its 1-month price, significantly higher than the $44\%-60\%$ seen in Hopper or Blackwell families.
Utilization Trends: A100 occupancy climbed from $74\% \rightarrow 90\%$ between March and September 2026.
Market Activity: Listed capacity increased by $13\%$ while the spot index rose by $20\%$ .

Note: This paper observes a correlation but does not definitively prove that open-weight demand caused the A100's price/occupancy behavior.

📊 Data Analysis

Table 1: Compute-Only Costs (per Million Output Tokens)

Based on full utilization and September 1, 2026 spot rents.

GPU Family	Dense (Llama-2-70B) Full Use	Dense Base Case	Sparse (`gpt-oss-120b`) Full Use	Sparse Base Case
A100 SXM4	$\$ 0.28$	$\$ 0.75$	$\$ 0.12$	$\$ 0.29$
H100 SXM	$\$ 0.26$	$\$ 0.68$	$\$ 0.27$	$\$ 0.64$
H200	$\$ 0.32$	$\$ 0.81$	$\$ 0.35$	$\$ 0.98$
B200	$\$ 0.16$	$\$ 0.39$	$\$ 0.15$	$\$ 0.47$

The "Base Case" utilizes Server throughput or rescaled GPUStack baselines. Sparse results for A100/H100 are derived from third-party measurements, not MLPerf.

Table 2: A100 Market Dynamics (Mar–Sep 2026)

Metric	Value/Change
Occupancy (Mar 1)	$74\%$
Occupancy (Sep 1)	$90\%$
Mean Occupancy	$80\%$
Listed Capacity $\Delta$	$+13\%$
Spot Index $\Delta$	$+20\%$

Table 3: Forward Term-Price Retention

How rental prices hold up over time relative to the 1-month mark.

Family	6M / 1M	1Y / 1M	3Y / 1M	5Y / 1M	Age at 5Y End
A100 SXM4	$95.0\%$	$93.1\%$	$84.2\%$	$80.2\%$	$11.3\text{ yrs}$
H100 SXM	$95.0\%$	$86.2\%$	$68.2\%$	$59.8\%$	$9.4\text{ yrs}$
H200	$92.4\%$	$80.2\%$	$51.0\%$	$43.7\%$	$7.8\text{ yrs}$
B200	$99.1\%$	$97.9\%$	$69.1\%$	$53.8\%$	$7.5\text{ yrs}$
B300	$93.8\%$	$81.3\%$	$60.9\%$	$53.8\%$	$6.5\text{ yrs}$

⚙️ The Logic of Hardware Agnosticism

Modern compute-heavy workloads are shifting the demand curve. Because certain tasks are latency-tolerant, they can be routed to the cheapest available hardware regardless of generation.

Portability and Deployment

Unlike closed models, open weights allow for independent deployment. September 2026 snapshots from OpenRouter showed $18$ to $22$ different providers for several open models, with price ratios ranging from $1.8\times$ to $5.6\times$ .

However, substitution is not seamless. The following factors limit instant migration:

Quantization: Different precision levels affect quality.
Capacity: Not all providers have the same scale.
Reliability: Uptime varies across vendors.
Technical Constraints: Memory, software stacks, and licensing.

For example, the gpt-oss-120b model utilizes MXFP4 expert weights, requiring specific hardware support for optimal efficiency.

🖼 Visualizing the Trend

Figure 1: Illustrative representation of the slower depreciation curve for Ampere-generation GPUs compared to newer families.

💻 Implementation Note

To calculate the cost per million tokens, the research likely used a formula similar to this:

def calculate_token_cost(rental_price_per_hr, tokens_per_hr):
    # cost_per_million = (hourly_rate / tokens_per_hour) * 1,000,000
    return (rental_price_per_hr / tokens_per_hr) * 1_000_000

🏁 Conclusion

The acquisition of Hugging Face by NVIDIA on September 3, 2026, underscores the strategic importance of the open-weight ecosystem. The data suggests that older NVIDIA generations maintain a multi-year earning life, provided they can serve hardware-agnostic workloads competitively.