InstinctFlash: Real-Time Robotics Model Serving

InstinctFlash is a high-performance serving framework engineered specifically for robotics models. It enables the deployment of 5B world-action models in real-time on Jetson Thor, while maintaining full compatibility for workstation environments.

🚀 Performance Overview

InstinctFlash provides a unified Runtime for eight different robotics model families, utilizing specialized acceleration kernels and supporting both Python and WebSocket-based serving.

In rigorous real-robot testing, we observed speedups of up to $\approx 33.78\times$ without any degradation in task performance.

Benchmark Results

Measured on Jetson Thor (p50 latency in ms) as of September 15, 2026.

Note: VA measurements track early continuations; comparisons are based on identical schedules.

---

🛠 Installation & Setup

The core system (Python 3.10+) handles checkpoint inspection and planning without requiring a GPU or PyTorch. However, inference requires a pinned environment specific to the model family.

Basic Installation

• Clone the repository: git clone https://github.com/General-Instinct/InstinctFlash
• Enter directory: cd InstinctFlash
• Create venv: python3 -m venv .venv-core
• Activate: source .venv-core/bin/activate
• Install: python -m pip install . (Requires uv==0.12.5)

Hardware-Specific Bootstrapping

For an RTX 4090, run the following to set up the pi05 model:

python3 scripts/bootstrap_vendor.py install pi05 --target rtx4090 \
  --python python3.12 --root ~/ifl-pi05-4090 \
  --ptxas /usr/local/cuda/bin/ptxas

source ~/ifl-pi05-4090/activate.sh

Supported Targets:

• --target rtx4090 or rtx5090
• --target jetson_thor (utilizes the Thor CUDA backend)

Model Family Identifiers: va, vla4, vla2, pi05, groot, edge, nano, or dreamzero.

• Note: Edge and Nano require Python 3.13; others use 3.12.

---

⚙️ Runtime Architecture

The serving process is designed to be deterministic. The system guesses never guesses parameters; it only uses what the checkpoint can prove.

Deployment Methods

1. Command Line Interface

instinctflash serve /path/to/your/checkpoint

2. Python API

from instinctflash import Runtime

# Load from local directory (requires instinctflash.json)
runtime = Runtime.from_pretrained("/path/to/your/checkpoint")

# Load from Hub (after installing family environment)
runtime = Runtime.from_pretrained("robbyant/lingbot-va-posttrain-robotwin")

Supported Model Hub IDs

---

🎯 Precision & Execution

The Runtime manages precision through "ceilings" to balance accuracy and speed:

• precision="native": Default setting; adheres to a BITEXACT transformation ceiling.
• tier_ceiling="numeric": Permits numerical variations.
• precision="fp8" (or --fp8 via CLI): Explicitly enables FP8 acceleration.
• tier_ceiling="behavioral": Required for DreamZero's opt-in dynamic step cache.

Local Inference API

with runtime.episode(prompt="put the bottle in the dustbin") as episode:
    while not done:
        result = episode.predict(observation)
        action = result["action"]

observation is a dictionary in the model's native format; the system handles action chunk updates automatically when the controller modifies executed actions.

Network Inference (WebSocket)

InstinctFlash uses the msgpack-over-websocket protocol, making it compatible with the pi0/openpi ecosystem.

from openpi_client.websocket_client_policy import WebsocketClientPolicy

client = WebsocketClientPolicy("my-server", 8000)
result = client.infer(observation)
action = result["action"]

Prompts are embedded in the observation. To trigger a new episode, include {"reset": True, ...} in the request.

---

🛠 Utility Flags & Validation

Serving Flags

• --serve.dry_run: Performs a preflight check (device, declaration, and plan) without starting the server.
• --serve.seed: Seeds native execution for A/B comparisons. Note: This is incompatible with FP8 serving.
• --serve.viz: Streams latency, actions, and observations to a Rerun viewer.

Validation

To ensure a checkpoint is ready for publication, use: instinctflash validate dir --validate.teacher_outcomes/.student_o