Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

Authors:

• Jinli Hu
• Ross M. Clarke
• Yichuan Zhang
• José Miguel Hernández-Lobato

---

📄 Overview

This research, archived under arXiv:2609.18842 (submitted September 16, 2026), explores a paradigm shift in how Large Language Models (LLMs) handle real-time information. While traditional scaling laws suggest that performance increases with parameter count and data volume—a trend leveraged by Mixture-of-Experts (MoE) architectures—there remains a critical gap in how deployed models handle live interaction.

"A conventional model cannot learn from this data, because its weights are frozen after training."

The Core Problem

Currently, when a user provides new facts or corrections during a session, the model relies on:

1. Retrieval-Augmented Generation (RAG)
2. In-Context Learning (ICL)

These methods are permanent. In reality, this information is placed in the prompt, re-processed with every single request, and then discarded the moment the session ends.

---

🚀 The Proposed Solution: The Infinite-Parameter LLM

The authors propose an architecture where the model learns from live interactions by writing them directly into its weights at runtime.

How it Works

Instead of a static weight bank, the system utilizes a compact hypernetwork. This network transforms live data into a low-rank modulation of a shared base network.

Technical Mechanism

Unlike previous weight generators that read context once and then freeze, this model maintains a Bayesian belief over the generator's latent code. This allows the model to update its understanding online.

$\text{Effective Weight} = f(\text{Base Network}, \text{Bayesian Belief}(\text{Live Data}))$

# Conceptual logic of the Infinite-Parameter weight generation
def generate_weights(live_data, base_network):
    latent_code = hypernetwork.encode(live_data)
    # Update Bayesian belief online
    updated_belief = bayesian_update(latent_code, current_session_state)
    # Apply low-rank modulation to base weights
    dynamic_weights = base_network.weights + low_rank_modulation(updated_belief)
    return dynamic_weights

System Architecture Flow

---

📊 Comparison: Conventional vs. Infinite-Parameter LLMs

---

🎯 Key Advantages & Evaluation

By shifting runtime knowledge from the prompt to the weights, the authors claim several benefits:

• Efficiency: It frees up the context window.
• Performance: It can generalize better than standard in-context use.
• Sustainability: The stored footprint remains constant, yet the "compilable" weights are effectively infinite.

Evaluation Protocol

The researchers have established a specific testing framework to validate these claims:

• Compare against In-Context Learning (ICL)
• Compare against Retrieval-based methods
• Measure generalization of live-learned facts
• Assess compute amortization

Subjects: cs.AI (Artificial Intelligence), cs.LG (Machine Learning).

Licensed under CC BY-NC-ND 4.0