🍲 Soup: High-Efficiency LLM Fine-Tuning

PyPI Downloads Tests Contributors

Soup streamlines the complex process of LLM post-training and fine-tuning into a unified, single-command workflow. It is designed to eliminate the scenario where engineers spend ~~30-50% of their time battling infrastructure~~ instead of optimizing their models.

🚀 Getting Started

To install the full training suite:

pip install "soup-cli[train]"

(Note: Use pip install soup-cli for the lightweight CLI version)

Quick Launch:

soup init --template chat
soup train

🧠 The Breakthrough: Layer Streaming

Soup enables the fine-tuning of an 8B parameter model on a laptop GPU with only 4 GB of VRAM. This is achieved via Layer Streaming, which prevents the frozen base model from occupying VRAM by feeding decoder layers to the GPU sequentially.

Performance Benchmark

Tested on an RTX 3050 Laptop (4 GB) using Llama-3.1-8B-Instruct with NF4 quantization:

Peak VRAM: $3.32 \text{ GB}$
Throughput: $119.6 \text{ tok/s}$
Accuracy: Bit-exact compared to standard resident memory runs.

🆕 What's New in v0.72.4: Preference Alignment

Layer streaming now extends beyond Supervised Fine-Tuning (SFT) to support preference losses: DPO, ORPO, SimPO, and KTO.

Solving the DPO Memory Paradox

Direct Preference Optimization (DPO) typically requires a reference model, which would normally double the VRAM usage. Soup bypasses this by using the same streamed base model but simply switching the adapters off.

Method	VRAM Peak (RTX 3050)	Memory Overhead
SFT (Streamed)	$3.32 \text{ GB}$	Baseline
DPO (Streamed)	$0.914 \times \text{SFT Peak}$	$\approx 0 \text{ GB}$
DPO (Standard)	$\text{SFT Peak} + 730 \text{ MB}$	$+1 \text{ copy of weights}$

The Honest Trade-off: While memory is "free," time is not. Because DPO requires more passes, it reads the layer stack approximately $1.52\times$ more often per step than SFT.

Important Limitations:

KTO: Treated similarly to DPO as it also utilizes a reference.
GRPO/PPO: $\text{Excluded}$ . These require reading every layer for every generated token, making streaming amortization impossible.

⚙️ Configuration Example

You can trigger training via soup train --config soup.yaml.

# soup.yaml
training:
  stream_layers: true      # Base model stays out of VRAM; only adapters train
  quantization: 4bit       # NF4: reduces size ~4x to fit 8B models on 4GB cards
  batch_size: 4            # Higher batches help amortize weight streaming costs
  stream_source: auto      # Prioritizes RAM, falls back to NVMe disk

Warning for v0.72.0 users: Adapters trained with stream_layers: true in v0.72.0 are inert. Their tensors use .inner. keys. Verify via: python -c "from safetensors.torch import load_file; print([k for k in load_file('adapter_model.safetensors') if '.inner.' in k][:3])"

🛠️ Additional Tooling

1. `soup reward synth` (v0.71.40)

Automatically generates a deterministic reward verifier from a JSONL of reference outputs.

Supported Families: numeric, json_schema, regex, and tool_call.
Safety Gate: It refuses to emit a function that cannot distinguish references from incorrect answers.
Ensembles: Supports multiple functions (e.g., reward_fn: "accuracy,format").

soup reward synth references.jsonl -o reward.py --output-report calib.json

2. `soup ship` (v0.71.38 - v0.71.39)

A rigorous CI/CD pipeline for model weights.

Regression Gating: Uses a fixed scorer across seven suites:
- MCQ
- Arithmetic
- Tool-calling
- JSON Validity
- Safety/Refusal
Provenance: Binds evidence to the exact recipe. Stale evidence triggers exit 3.
Integration: soup ship --push owner/repo#N posts the verdict directly to a PR.

soup ship --base ./base --adapter ./my-lora --task-eval my_task.jsonl
# Exit 0: SHIP | Exit 2: DON'T SHIP | Exit 3: Bad Flags | Exit 1: Error

3. `soup draft` (v0.71.33)

Tools for speculative decoding.

measure: Reports acceptance rates and plain-vs-assisted tokens per second.
distill: Compresses a target model into a dense "draft" model, compatible with soup serve --auto-spec.