How We Made IPFS Content Publishing 10x Faster
Accelerating IPFS Content Publishing by 10x
By Dr. Dennis Trautwein | March 20, 2026 | ProbeLab Analytics
Introduction
Historically, distributing content across Distributed Hash Tables (DHTs) has been a sluggish process. This inefficiency is typically exacerbated by two primary factors:
- The sheer scale of the network.
- High rates of node churn (peers frequently joining and leaving).
The Amino DHT, which powers IPFS, suffers from both. After conducting extensive measurements, the ProbeLab team discovered a way to optimize this. We proposed a method that slashes content publication time by over an order of magnitude while simultaneously cutting network overhead by 40%.
This optimization, dubbed "Optimistic Provide," has officially been integrated as the default behavior in Kubo 0.39.0. Our findings were detailed in a study published at IEEE INFOCOM 2024.
TL;DR: Optimistic Provide speeds up IPFS by storing records with likely-closest peers during the DHT walk, terminating the search based on statistical probability rather than rigid requirements, and returning control to the user before every single
PUT RPCis confirmed.
The Core Logic of Optimistic Provide
To achieve sub-second upload latency, the system implements the following strategy:
- Immediate Storage: While traversing the DHT, records are stored immediately with peers that are statistically likely to be among the 20 closest network-wide.
- Heuristic Termination: The DHT walk ends as soon as the discovered set of 20 peers is likely the most accurate set available.
- Asynchronous Completion: The user regains control once a majority (but not all) of the storage requests succeed; the rest finish in the background.
- Network Estimation: The system uses a lightweight method—piggy-backing on routing table refreshes—to estimate total network size without adding extra traffic.
The Result: Upload latency has plummeted from seconds to less than 1 second. This allows developers and users to iterate and debug in near real-time.
Understanding the Traditional "Provide" Operation
To appreciate the fix, we must understand the legacy process. In a Kademlia-based DHT like Amino, storing a record requires finding the closest peers to a specific identifier (where ).
The Distance Metric
"Closeness" is not about physical geography; it is defined by the XOR distance.
The Workflow
The process traditionally follows two distinct phases:
- DHT Walk: An iterative search where the node asks known peers for candidates closer to the target CID.
- Follow-Up: Once the walk ends, the provider record is pushed to all 20 identified peers to ensure redundancy.
The Performance Bottleneck
The traditional approach is efficient rigid. The primary delay stems from the termination condition: the system insists on waiting for responses from the three closest discovered peers.
In a permissionless environment with high churn, these specific peers are often offline. This forces the system to "backtrack" and query more distant peers, even if the actual 20 closest peers have already been identified.
Figure 1: Median latency is , with some operations exceeding 120 seconds.
How Optimistic Provide Fixes It
Optimistic Provide replaces rigid waiting with statistical heuristics.
Comparison Table: Traditional vs. Optimistic
| Feature | Traditional Provide | Optimistic Provide |
|---|---|---|
| Termination | Wait for 3 closest peers | certainty |
| Storage Timing | After the walk completes | During the walk (predictive) |
| User Feedback | After all 20 peers confirm | After a subset (e.g., 15) confirm |
| Latency | seconds | second |
Technical Mechanisms
- Network Size Estimation: Nodes use a bias-corrected proximity model to estimate the global network size.
- Predictive Termination: The initiator calculates the probability that the current set of peers are the network-wide closest. Once the confidence hits , the walk stops.
- Early Return: Control is returned to the user as soon as of the 20 peers confirm storage. The remaining 5 requests are handled as
background tasks.
// Conceptual logic for Early Return
if (confirmedPeers.length >= 15) {
returnControlToUser();
continueStorageInBackground(remainingPeers);
}
Real-World Impact and Data
The deployment of Kubo v0.39.0 showed a dramatic shift in performance.
Figure 2: Average provide duration dropped from to in February 2026.
Network Health and Adoption
Despite the improvement, the network still faces challenges with peer connectivity.
Figure 3: no_public_ip_addresses remains the dominant error ().
Regarding adoption, while newer versions like v0.39.0 and v0.40.1 are growing (representing of peers), a significant portion of the network still runs older versions, such as v0.22.0.
Final Thoughts
By shifting from a deterministic to a probabilistic approach, we have removed a massive bottleneck in the IPFS ecosystem. We thank the IPShipyard team for their collaboration in bringing this to production.
