HFlow: Scaling Multimodal Data Pipelines for Physical AI

Hebbian Robotics (YC S26) is introducing HFlow, an open-source SDK designed to streamline the creation of scalable multimodal data pipelines for robotics and physical AI.

The goal of HFlow is to democratize the sophisticated data tooling and rigorous practices typically reserved for massive robotics corporations, making them available to any team, regardless of size.

The Core Challenge

Processing data is currently one of the most significant bottlenecks in the field of robotics.

A typical dataset is a complex mix of:

• High-resolution video
• Robot state and actions
• Precise timestamps
• Diverse metadata from various recording sources

Teams often struggle with manual data cleaning Quality Control (QC). Common issues include:

1. Frozen camera feeds.
2. Desynchronized streams (drift).
3. Missing critical topics.
4. Duplicate entries within the corpus.

As datasets scale, relying on fragmented, ad-hoc scripts makes it nearly impossible to audit results, track what has been processed, or reliably reproduce a specific dataset version.

---

How HFlow Works

HFlow provides the necessary orchestration, versioning, storage, and curation layers. Users can leverage built-in checks or develop their own custom transformations, labels, and enrichments.

Technical Boundary & Specifications

The Data Lifecycle

The movement of human and robot data follows a structured four-stage path:

---

Key Architectural Features

1. Developer-First Processing

Your logic remains decoupled from the framework. Transformations and quality checks are written as plain Python functions. Instead of rewriting code for a proprietary system, you use small adapters to plug existing logic into HFlow.

2. Optimized MCAP Integration

HFlow utilizes MCAP as its primary I/O boundary because it is highly efficient for synchronized time-series data.

• Compatibility: Native to ROS 2; opens directly in Foxglove or Rerun.
• Performance Tunings:
  • In-band H.264: GOP length is optimized for data read patterns.
  • Topic-Group Chunking: Camera and state streams are stored in separate chunks.
  • Efficiency Gain: A training sample requires $\text{1 read per group}$ rather than $\text{1 read per topic}$.

3. Provenance and the Parquet Catalog

Every processed episode is stamped with its origin. HFlow records metadata and quality evidence in a queryable Parquet catalog.

• Traceability: Link bad model results back to specific step versions.
• Queryability: Use DuckDB to analyze the corpus without loading massive media files.
• UI Access: Launch the catalog browser instantly: hflow catalog ui

4. Orchestration via Airflow 3

HFlow renders your pipeline as a graph via Airflow 3 DAGs. This allows you to:

• Monitor task status and logs.
• Manage retries and reruns.
• Extract inputs as numpy arrays, MP4 paths, or JPEG frames.

---

Hosting and Scalability

The open-source version is designed for easy ownership:

• Single-Tenant: Use the provided Docker Compose runtime.
• Enterprise-Ready: Deploy generated DAG bundles into your existing Airflow 3 environment.

Note: The OS version $\textit{does not}$ include RBAC, user accounts, or a multi-tenant control plane. By separating the data plane from the control plane, Hebbian Robotics ensures the engine can scale as isolated workspaces (e.g., one per customer) behind a future hosted control plane.

Example Configuration Logic

# Conceptual HFlow transformation
def check_camera_sync(episode):
    # Logic to detect drift between streams
    drift = calculate_drift(episode.video, episode.state)
    return drift < 0.01  # Return boolean for QC gate

Community and Contribution

We are building an open ecosystem for the future of physical AI.

• Star the repository to show support
• Share HFlow with your robotics network
• Contribute to open issues on GitHub