I Built Non-Autoregressive Decision Models with RL a Year Ago

Laya — A Multilingual System 1 Decision Engine featuring calibrated probabilities. Research Engineering | Updated September 2026

---

The Journey from Research to Laya

The evolution of this project began with a March 2025 arXiv paper focusing on sequence conversion trajectories. This eventually culminated in Laya: an open-weight "System 1" decision engine capable of sub-35ms responses, utilizing RLCD (Reinforcement Learning for Calibrated Decisions), multilingual routing for over 100 languages, and industry-leading calibration.

Author: Nandakishor Mukkunnoth, Founder & CEO of ConvAI Innovations.

![Benchmark Board Placeholder: Accuracy, 9 workflows, 51-language sweep, T4 latency, and calibration repair]

The Frustration of "New" Breakthroughs

Currently, the AI community is buzzing about a specific architectural shift: models that are non-autoregressive, avoid text generation entirely, and instead provide near-instant probability predictions based on structured schemas.

While the hype is validating, it is also infuriating deeply frustrating. I dedicated months of sleepless nights to this exact concept:

• Published an initial arXiv paper (arXiv:2503.23303)
• Released model weights via Hugging Face (sales-conversion-model-reinf-learning)
• Opened a public dataset (saas-sales-conversations)
• Launched a PyPI package and engaged the community on r/LocalLLaMA.

By September 2025, I formalized the framework for RL-guided schema decisions in a second paper (arXiv:2510.01237). The core "intelligence" was always driven by Reinforcement Learning, rather than simple autoregressive LLMs or embedding models.

Then, in September 2026, TypeSafe AI (led by OpenAI co-inventor Diogo Almeida) released Jev. They presented the non-autoregressive decision concept as a brand-new discovery, yet they provided:

• No technical papers
• No open weights
• No open training data

Laya vs. Jev: A Technical Comparison

My original model utilized PPO (Proximal Policy Optimization) over sequence representations to predict turn-by-turn conversion probabilities ($0.0$ to $1.0$) for sales calls. Jev expanded this via parallel sampling (which they termed RLCD) to provide horizontal schema choices and confidence distributions. Jev costs \0.042$per million input tokens with a latency of$\approx 150\text{ms}$.

Rather than succumb to bitterness, I iterated on my architecture to eliminate previous limitations. The result is Laya, a fully open, horizontal System 1 decision family.

---

The Core Philosophy: System 1 vs. System 2

Modern AI pipelines suffer from a massive efficiency bottleneck: we use "System 2" generative LLMs for "System 1" reflex tasks.

The Problem: When a system needs to route a support ticket or decide if a prompt requires code execution, calling a $70\text{B}$ parameter generative model is overkill. You endure $500\text{ms}$ to $2,000\text{ms}$ of latency, pay high inference costs, and must use regex or JSON parsers to clean the output. Furthermore, LLMs often "hallucinate" confidence; if an LLM says Confidence: 0.95, it is simply predicting a token that sounds confident.

The Solution: A model that mimics the human brain's System 1: instant, reflex-like decisions with honest, calibrated probabilities.

---

The Three Decision Primitives

Laya processes typed questions over any state (JSON, email, raw text) in a single forward pass. It uses three primary primitives:

1. choice: Selects one option from a predefined dictionary.
   • Returns: Selected key, probability distribution, and calibrated confidence.
2. score: Maps the state onto an ordinal rubric (e.g., $0, 1, 2 \dots$).
   • Returns: Expected level, rank distribution, and confidence.
3. noul: A boolean query returning the calibrated probability: $\text{Result} = P(\text{true}) \in [0.0, 1.0]$ (Where $P(\text{false}) = 1 - P(\text{true})$ by definition).

Key Advantage: Because the output is strictly numerical/probabilistic, the model cannot generate text, meaning hallucinations and malformed JSON are physically impossible.

---

Architecture & Deployment

Since no single model fits every use case, we provide three specialized checkpoints hosted in a single Hugging Face hub.

Checkpoint Specifications

Optimized Loading

To avoid downloading the full $2.5\text{GB}$ bundle, the Laya SDK utilizes allow_patterns to fetch only the necessary subfolders.

import laya

# Downloads only the English model (~808 MB)
agent_en = laya.load("convaiinnovations/laya")

# Downloads ONLY the multilingual subfolder (~647 MB)
agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual")

Why Routing Matters: The Multi-S... (Content ends here)