In today’s business environment, the availability and stability of databases are the keys to success.@note] Stripe has succeeded in maintaining 99.999% uptime in 2023 while processing a total payment volume of $1 trillion. The secret to this remarkable achievement lies in Stripe’s database infrastructure strategy, which made downtime-free data migration possible. Today, we will explore the core of this innovative system.
DocDB: The Core of Stripe’s Stable Database Infrastructure
Stripe’s database infrastructure team operates the API’s foundational layer using DocDB, a DBaaS (DataBase as a Service). DocDB is an extension of MongoDB Community and consists of several services built within Stripe. It processes over 5 million queries per second and stores critical financial data across more than 2,000 database shards, distributed over petabytes. So, how has this system managed to maintain such high stability?
Data Movement Platform: The Key to True Zero-Downtime Migration
Stripe’s DocDB enables seamless data migration through a unique system called the ‘Data Movement Platform.’ This platform allows for the easy transfer of large amounts of data while maintaining data availability and consistency during migration. The technology operates primarily by splitting data during traffic surges and consolidating empty shards during periods of low traffic.
1. Chunk Migration Registration
The migration process begins with the Chunk Metadata Service. To move database chunks from a source shard to a target shard, the intention to move the chunk from the source shard to an arbitrary target shard is first registered. This process ensures data integrity and focuses on moving the data quickly and efficiently.
2. Bulk Data Ingestion
Next, bulk data ingestion is performed. During bulk data ingestion, the DocDB engine sorts the data to optimize write throughput. This has resulted in a more than tenfold improvement in write performance, with sorted inserts maximizing system efficiency.
3. Asynchronous Replication and Replication Lag
Once the data has been successfully transferred to the target shard, the asynchronous replication system replicates the data from the source shard to the target shard. To prevent performance degradation of the source shard during the replication process, oplog events from the source shard are used for replication while minimizing performance drops on the target shard.
4. Traffic Switch
After migration is completed, the Chunk Metadata Service updates the chunk path to the target shard. This process takes just a few seconds, with traffic switching to the new shard without any service interruption. This is one of the reasons Stripe can process millions of payments daily.
Conclusion: Stripe’s Success Factors in Zero-Downtime Migration
Stripe’s DocDB and Data Movement Platform have set a new standard for online database migration. The ability to move data without service interruption while maintaining 99.999% uptime has been a key factor in Stripe’s success in the global payments market. This system will serve as a crucial reference model for future database management and migration strategies.
Reference: Stripe, “How Stripe’s document databases supported 99.999% uptime with zero-downtime data migrations”