A Dozen Years of Distributed SQLite

By Philip O'Toole

In September 2014, I introduced the world to rqlite. The premise was straightforward: wrap SQLite with a Raft consensus log, replicating every write across a cluster. This ensured that the failure of a single node wouldn't result in catastrophic data loss.

To my knowledge, this was the inaugural open-source project to merge Raft with SQLite. For a long time, it was a lonely endeavor; few others attempted this architecture for several years.

Fast forward to today, and "distributed SQLite" has evolved into a full-fledged industry category. It now encompasses a dozen projects, including several venture-backed firms and two that have been acquired. Because modern narratives of this space usually begin around 2021, I feel it is necessary to document the history starting from 2014.

The Genesis of rqlite

The seed for rqlite was planted while I was working at jut.io. We needed a small relational store for a clustered system, but we couldn't decide which node should be the "owner" of that data. While we eventually took a different path professionally, I continued developing rqlite because the problem was universal and the implementation was elegant.

Technical Evolution

Initially, I utilized goraft (the same library used by etcd and InfluxDB—the latter of which I joined as a core developer). By early 2016, I migrated to Hashicorp's Raft implementation, which remains the backbone of rqlite today.

The fundamental architectural pillar established then remains unchanged:

Writes must be processed by the leader, acknowledged by a majority, and any partitioned minority must reject write attempts.

From this core logic, everything else emerged:

• The HTTP API
• Read-consistency configurations
• Snapshotting mechanisms

Pre-2014 and Contemporaries

For the sake of accuracy, I must acknowledge two other pioneers:

1. Expensify: They had been using a Paxos-like protocol to replicate SQLite internally for years, eventually releasing it as Bedrock in 2016.
2. ActorDB: An Erlang-based system that provided each "actor" with its own Raft-replicated SQLite database.

While neither saw widespread external adoption, they prove the concept was "in the air." rqlite simply turned this idea into a standalone, general-purpose, open-source tool.

The Timeline of Expansion

2017: The Arrival of dqlite

Canonical developed dqlite to provide an embedded replicated store for LXD and MicroK8s, avoiding the need for a separate etcd instance. Mark Shuttleworth and I actually spoke about potentially using rqlite, but it didn't quite fit their needs.

While dqlite started in Go, it was rewritten in C with a custom Raft library. The primary distinction is packaging:

• dqlite $\rightarrow$ An embedded library.
• rqlite $\rightarrow$ A standalone server.

2021: Litestream and the "Single Node" Philosophy

Ben Johnson (a former colleague at InfluxDB) launched Litestream. Instead of a cluster, it monitors the SQLite Write-Ahead Log (WAL) and streams it to S3.

"Most applications don't actually need a cluster; they just need a way to recover a single machine if it crashes."

This insight fueled the "SQLite is enough" movement. The distinction is simple:

• Litestream: "What if the machine dies?" (Recovery)
• rqlite: "What if the machine dies and I cannot wait for a restore?" (High Availability)

Later, Ben joined Fly.io and created LiteFS, which handles replication at the filesystem level using Consul leases. This shifted replicated SQLite from a "deployed tool" to a "platform feature." Interestingly, Litestream's approach influenced how rqlite v8.0 handles the WAL.

2022: Divergent Paths

Two new approaches emerged in 2022:

• cr-sqlite: Integrated CRDTs as an extension, allowing multiple writers to diverge and merge without a leader. This targets the $\text{AP}$ (Availability/Partition Tolerance) side of the CAP theorem.
• ChiselStrike $\rightarrow$ Turso: They forked the SQLite engine itself to implement changes the original project wouldn't accept. They built a hosted edge-replica service, raised VC funding, and were acquired by Supabase in 2026.

Simultaneously, SQLite Cloud launched, offering a hosted Raft-replicated service—essentially a cloud-wrapped version of the original rqlite design.

2022 and Beyond: The Platform Era

Cloudflare introduced D1 (serverless SQLite at the edge) and integrated SQLite into Durable Objects. This represents the final transition: replicated SQLite is no longer just "something you run," but "something you buy."

Summary of the Twelve-Year Journey

The evolution of this space can be visualized as a movement across three distinct design philosophies:

Comparison of Approaches

Final Reflections

Looking back, three things are clear:

1. The pattern of adding replication outside the SQLite engine has been independently reinvented by at least eight different teams.
2. The design space is a triangle of trade-offs: Consensus $\rightarrow$ Streaming $\rightarrow$ Merging.
3. Most "database wars" are actually just users arguing over which of these three questions they are actually trying to answer.

Evolution Checklist:

• Standalone Server (rqlite)
• Embedded Library (dqlite)
• S3 Backup (Litestream)
• Edge Replication (Turso)
• Managed Platform (D1)

-- The essence of the journey
SELECT evolution FROM sqlite_history 
WHERE year >= 2014 
ORDER BY complexity ASC;