Freno: A Cooperative, Highly Available Throttler Service

Freno is a specialized service designed to manage write-pressure on backend resources. Rather than acting as a gatekeeper, it functions as a cooperative advisor that clients consult to determine if it is safe to perform write operations.

⚙️ Core Architecture & Logic

At its heart, Freno monitors the health of backend stores—currently focusing on MySQL clusters—and evaluates whether replication lag has exceeded a specific limit.

How it Works

Freno does not sit in the middle of the data stream. It is not a proxy. Instead, it observes the backend and provides a recommendation.

Key Operational Principles

• Cooperative Nature: Freno trusts that applications will identify themselves and honor the "stop" signal.
• Metric-Driven: It collects absolute truths from backend stores independently for each cluster.
• Manual Overrides: Administrators can forcibly throttle specific applications to ensure high-priority tasks have the necessary headroom.
• No Internal Scheduling: Freno does not decide who should go first; it simply executes instructions on who to throttle.

📊 Technical Specifications

The Throttling Condition

The decision to throttle is based on a simple mathematical threshold: $\text{Current Replication Lag} > \text{Pre-defined Threshold} \implies \text{Throttle Writes}$

API Interaction

The primary interaction occurs via the /check/... endpoint. While GET requests work, HEAD requests are preferred to reduce overhead.

*Note: The original text describes this as a "normal state" indicating thresholds aren't met.

🚀 Use Cases & Implementation

Bulk Operations: Freno is ideal for massive data migrations, archiving, or schema updates. Instead of running a million-row update in one go—which would spike replication lag—a client can process data in small batches (e.g., 100 rows) and check with Freno between each batch.

Additionally, Freno can be used to verify the validity of replicas by determining actual lag, helping developers avoid the common "write-then-read" inconsistency issues associated with master/slave architectures.

Client Performance

Clients can hit Freno hundreds of times per second without issue. However, the frequency of checks should align with the granularity of the lag threshold. For example:

• High Granularity: If you throttle at $1000\text{ms}$ lag, checking 200 times/sec is likely overkill.
• Naive Clients: If the client lacks internal caching, frequent polling is acceptable.

🛠️ System Administration

High Availability via Raft

To ensure the throttler isn't a single point of failure, Freno utilizes the raft consensus protocol.

• Nodes compete for leadership.
• Only the Leader is responsible for collecting metrics and serving client requests.
• Raft is used to synchronize user events across all member nodes.

Inventory Management

Freno can track MySQL servers using two methods:

1. Static: A hard-coded list of hostname[:port].
2. Dynamic: Integration via haproxy. Freno can poll a HAProxy URL and backend pool to automatically adapt to servers being added or removed.

Configuration & Resources

Configuration involves defining MySQL clusters, including:

• Authentication credentials.
• The specific query used to inspect lag.
• The lag thresholds.

Available Tools:

• freno-client: An open-source Ruby Gem for easy integration.
• Deployment: Refer to the official deployment documentation for recommended setups.

---

Fun Fact: The creators originally wanted to call the project "throttler" or "throttled", but those names were already taken. They eventually settled on Freno (the word for "brake" in several languages).