Achieving 118M Queries per Second with Neki

PlanetScale has officially launched the platform preview of Neki, their sharded Postgres solution. To mark the occasion, the engineering team decided to push the system to its absolute limits.

The Scaling Experiment

The initial objective was modest: reach 1 million queries per second (QPS). However, after hitting that milestone effortlessly with only 5 shards, the team decided to see how far the architecture could actually scale.

The final test reached a staggering 118.5 million QPS utilizing 512 shards and managing a dataset of $1.22 \text{ PiB}$.

Test Parameters

The workload was designed to be highly specific:

• Query Type: Single-shard point selects.
• Mechanism: One row fetched per query via the primary key.
• Isolation: No queries spanned multiple shards; each request was isolated to a single shard.

-- Example of the point-select workload used
SELECT * FROM users WHERE user_id = '12345';

Linear Scalability Results

The team aimed for a baseline of $200\text{k}$ QPS per shard, expanding the cluster to drive total throughput upward.

As the data shows, the system exhibited linear scalability: increasing the shard count by ten times resulted in a ten-fold increase in throughput. By the time they hit 512 shards, the system still had available headroom, allowing the load generator to push each shard to 231k QPS.

Technical Architecture & Performance

The infrastructure required to sustain this load was massive. Here is a high-level visualization of the request flow:

The Hardware Stack

• Shards: 512 total, each featuring one Postgres primary running on an r8g.16xlarge instance.
• Routers: 480 Neki routers, each deployed on its own 8xlarge instance.

Key Performance Metrics

The team successfully sustained 118,538,803 QPS for a duration of 16 minutes.

• Throughput: 118.5M+ QPS
• Data Volume: 1.22 PiB
• Network Traffic: $> 2 \text{ Tb/s}$
• Disk I/O: $15.8\text{M}$ read IOPS across the fleet

Latency and Reliability:

• Router p99: 6.06ms
• Client p99: 13.95ms
• Error Rate: 67 errors/sec, which calculates to approximately: $\frac{1}{1,800,000} \text{ queries}$

Important Context: To be fully transparent, this specific run utilized primary-only shards with no replicas. The workload was strictly read-only, and no failover events occurred during the measurement window.

Visual Evidence

The engineering hurdles and detailed technical challenges involved in reaching the 100M QPS milestone will be explored in a subsequent deep-dive article.