Monzo Stand-In: Ensuring Financial Continuity During Cloud Outages

Published on: 13 February 2025 | Category: Technology

Note: This article reflects the state of the system at the time of publication. For the most current details, please visit the Monzo app or monzo.com.

For the modern consumer, banking is a non-stop utility. Customers expect to pay bills, transfer funds, and use their cards 24/7/365. Because human lives do not have "scheduled maintenance windows," Monzo believes its services shouldn't either.

While Monzo invests heavily in reducing risks during migrations and daily operations, the reality of distributed systems is that unforeseen outages can be completely eliminated unforeseen outages are impossible to eliminate entirely. To combat this, Monzo developed Monzo Stand-in, a secondary, isolated banking infrastructure designed as a final line of defense.

"We consider Monzo Stand-in to be a backup of last resort, not our primary mechanism of providing a reliable service to our customers."

---

🏗️ The Architecture of Resilience

Monzo Stand-in is not just a mirror of the main system; it is an independent ecosystem. While the Primary Platform resides on Amazon Web Services (AWS), the Stand-in Platform is hosted on Google Cloud Platform (GCP).

Platform Comparison

Core Capabilities

When Stand-in is active, Monzo ensures the most critical functions remain operational:

• Card spending
• ATM cash withdrawals
• Sending and receiving bank transfers
• Checking account balances and history
• Freezing/Unfreezing cards

Both platforms operate their own Kubernetes clusters, databases, and queuing systems. Crucially, the services are unique; a service running in the Stand-in environment does not exist in the Primary environment, and vice versa. This includes the logic for authorizing card payments.

---

🛡️ Strategic Design Decisions

Avoiding the "Consistency Trap"

It might seem intuitive to simply deploy the same code to both clouds. However, doing so would require strong data consistency. In such a model, a write would only be successful if it hit both AWS and GCP.

If one cloud went down, the system would be unable to write any data to maintain consistency, effectively $\text{Availability} \rightarrow 0$. Instead, Monzo accepts eventual consistency for the Stand-in platform, acknowledging that while the ledger requires absolute precision, a backup system can operate on slightly delayed data to maintain uptime.

Diversifying Software Risk

Hardware failure is a solved problem for major cloud providers. The real danger is software failure—a bug in the code or a flawed process. If you run the exact same software in two different data centers, the same bug will crash both.

By implementing separate card issuer processing code for the Stand-in platform, Monzo minimizes the risk of a single software defect causing a total blackout.

Economic Efficiency

Maintaining a full mirror of the Primary Platform would be prohibitively expensive, potentially doubling costs. By keeping the Stand-in system minimal, Monzo keeps the overhead low: $\text{Stand-in Cost} \approx 1\% \text{ of Primary Platform Cost}$

---

🔄 Data Synchronization & Recovery

Primary $\rightarrow$ Stand-in (The Feed)

The Stand-in platform only stores the bare essentials: balances, limited transaction history, card/account details, and a list of payees/pots. A Stand-in Data Syncer in the Primary Platform consumes events and pushes them to the GCP datastore.

Stand-in $\rightarrow$ Primary (The Reconciliation)

When a transaction occurs in Stand-in mode, it creates a StandinTransaction. To ensure the main ledger is eventually updated, the system uses Monzo Advices.

---

📱 User Experience & Operations

The App Interface

The Monzo app periodically polls for the status of the Stand-in platform. If enabled, the app automatically switches to a simplified UI, informing the user of the situation and limiting the available features to those supported by the backup system.

Engineering Control

Engineers manage the transition via a dedicated incident tool. This allows them to route payment traffic and redirect app users in real-time.

# Example Stand-in Incident Tool Output
Status: NOT_ENABLED
Users affected: 0%

Available Options:
  > route-payments
  > redirect-apps
  > prescale

Payment Routing

When a message arrives from a network like Mastercard, the Primary Platform determines if it should be handled by the Stand-in system. If so, it is routed to service.standin.mastercard in the GCP environment.

---

Contributors:

• Daniel Chatfield, Distinguished Engineer
• Andrew Lawson