🚀 Career Opportunity: Rust & Performance Engineering at LiteLLM (YC W23)

LiteLLM is expanding its technical team to build the future of LLM orchestration. We are seeking a high-caliber Rust Engineer to optimize our core infrastructure.

"Our mission is to provide a universal interface for all Large Language Models, ensuring that developers can switch providers without changing a single line of code."

---

🛠 The Mission: From Python to Performance

While our initial growth was powered by Python, the scale of modern AI demands a shift toward systems-level efficiency. We are moving critical paths to Rust to eliminate bottlenecks and reduce overhead.

We used to rely solely on interpreted languages $\rightarrow$ Now we are building a high-performance data plane in Rust.

📊 Performance Metrics

We are optimizing for the following latency equation: $L_{total} = T_{network} + T_{parsing} + T_{routing} + T_{llm\_response}$ Our goal is to minimize $T_{parsing}$ and $T_{routing}$ to near-zero.

🏗 System Architecture

The following diagram illustrates where the Rust Performance Engineer will operate:

---

📋 Role Requirements & Expectations

We need someone who treats memory management as an art form and concurrency as a first-class citizen.

Technical Skill Matrix

Key Responsibilities

• Implement high-throughput proxy logic using axum or actix-web.
• Optimize memory allocation to handle millions of concurrent tokens.
• Build seamless bindings between Rust and Python.
• Reduce P99 latency across the gateway.

---

💻 A Glimpse into the Code

You will be working on performance-critical sections similar to this conceptual implementation:

use tokio;
use std::sync::Arc;

#[tokio::main]
async fn main() {
    let gateway_config = Arc::new(Config::load());
    
    // High-performance request handling loop
    let handle = tokio::spawn(async move {
        println!("🚀 LiteLLM Rust Core initialized...");
        // Logic for routing LLM requests goes here
    });

    handle.await.unwrap();
}

---

🎯 Your Onboarding Roadmap

If hired, your first 30 days will look like this:

• Set up the local development environment.
• Submit your first PR to the core-performance repo.
• Profile the current Python bottleneck using py-spy.
• Implement a Rust-based replacement for the most expensive middleware.

---

🖼 Company Culture

Join us in building the backbone of the AI era.

Interested? Apply now to help us scale the world's most flexible LLM proxy.