Walgit: A Stateless Git Server Powered by Object Storage

Walgit is a streamlined Git server implemented as a single binary that acts as a gateway to an object store. It enables the hosting of Git repositories without the need for ~~databases~~, ~~leader nodes~~, or ~~persistent local state~~.

🚀 Core Value Proposition

By pointing a single binary at a Google Cloud Storage (GCS) or S3-compatible bucket, you instantly deploy a robust Git infrastructure featuring:

Smart HTTP (v0/v2): Full support for fetch and push.
bundle-uri Clones: Serving clones as static files for maximum efficiency.
Git LFS: Integrated Large File Storage.
Management Tools: A browsing Web UI and a JSON API with an accompanying SDK.
Control: Per-repository push policies and webhooks.
Extreme Scalability: The ability to handle repositories that are physically larger than the machine hosting the binary.

The Golden Rule of Walgit: Every running instance is merely a disposable cache; the object bucket is the only true source of truth.

🛠️ Quick Start Guide

Follow these steps to get your server live:

Step 1: Configure the environment Create a walgit.toml file with your server and storage settings:

[server]
listen = "0.0.0.0:8080"
public_url = "https://git.example.com"
auto_create_on_push = true

[server.auth]
mode = "token"
anonymous_read = false
tokens = [{ principal = "me", token_env = "WALGIT_TOKEN_ME", write = true }]

[store]
backend = "s3"
bucket = "my-walgit"

[store.s3]
endpoint = "https://s3.us-east-1.amazonaws.com"
region = "us-east-1"

Step 2: Launch the binary Generate a secure token and start the service: WALGIT_TOKEN_ME=$(openssl rand -hex 24) walgit serve --config walgit.toml
Step 3: Push your first repo Since auto_create_on_push is enabled, simply push to a new path:

git -c http.extraHeader="Authorization: Bearer $WALGIT_TOKEN_ME" \
push https://git.example.com/acme/app.git main

🏗️ Architecture & Philosophy

Walgit is a Rust-based realization of the "Continuity" architecture described by Cursor in their Git at any scale whitepaper (available in docs/reference/cursor-git-at-any-scale.md).

The Problem: The "Packfile" Nightmare

Git's distributed nature is a liability for centralized hosting because of packfiles. Git operations perform "random walks" across gigabytes of compressed binary data.

Hosting Method	Performance Profile	Outcome
Local NVMe	High IOPS, low latency	Fast, but requires "pet" servers
NFS/Network FS	High latency, random access	$\text{Catastrophic failure}$
Walgit (WAL)	Immutable objects + CAS	Scalable, stateless, consistent

The Solution: Write-Ahead Log (WAL)

Instead of replicating full repositories, Walgit uses a Write-Ahead Log stored in the bucket.

Pushes are stored as immutable objects.
Visibility is managed via a tiny manifest updated using a Compare-and-Swap (CAS) operation.
Consensus is achieved through the bucket's CAS—eliminating the need for Raft, Paxos, or primary/secondary elections.

Advanced Optimizations for Monorepos

To support repositories that exceed local disk capacity, Walgit implements:

Remote Reading: Uses HTTP range requests to serve refs and pages without downloading full packs.
History Packs: Keeps commits and trees locally while leaving heavy blobs in the bucket.
bundle-uri: Offloads the heaviest part of the process (cloning) to a CDN or the bucket itself.

🔍 Technical Specifications

Git Protocol Support

Walgit handles the "plumbing" and the WAL, while relying on upstream Git for upload-pack, repack, and bundle.

Smart HTTP v0/v2: Supports ls-refs (with prefixes), fetch (including shallow, filter, and deepen), and receive-pack (supporting atomic updates, tags, and report-status-v2).
Hashing: Compatible with both sha1 and sha256 repositories.

The `bundle-uri` System

Clones are optimized via calendar-based slots:

Full Bundles: Weekly snapshots.
Incremental Bundles: Daily and hourly chains.
Logic: A client downloads the latest full bundle $\rightarrow$ the chain of increments $\rightarrow$ asks the server only for the final delta.

LFS & Web Interface

LFS: Implements the Batch API; objects reside in the bucket. It can also perform read-throughs from upstream LFS servers.
UI: A React-based interface providing views for:
- File trees and blobs.
- Commit history and diffs.
- A dedicated health page for the WAL.

The mathematical efficiency of this approach can be summarized as: $\text{State} = \text{Immutable Objects} + \text{Atomic Manifest Update}$