Master system design principles to build scalable applications that can handle millions of users and beyond without downtime.

Database scaling

There are two main approache for database scaling: vertical scaling and horizontal scaling.

Vertical scaling

Vertical scaling, also known as scaling up, is the scaling by adding more power (CPU, RAM, DISK, etc.) to an existing server. There are some powerful database servers. According to Amazon Relational Database Service (RDS), you can get a database server with 24 TB of RAM. This kind of powerful database server could store and handle a lots of data. For example, stackoverflow.com in 2013 had over 10 million monthly unique visitors, but it only had 1 master database. However, vertical scaling comes with some serious drawbacks:

You can add more CPU, RAM, etc. to your database server, but there are hardware limits. If you have a large user base, a single server is not enough.
Greater risk of single point of failures.
The overall cost of vertical scaling is high. Powerful servers are much more expensive.

Horizontal scaling

Horizontal scaling, also known as sharding, is the practice of adding more servers.

User data is allocated to a database server based on user IDs. Anytime you access data, a hash function is used to find the corresponding shard. In our example, user_id % 4 is used as the hash function. If the result equals to 0, shard 0 is used to store and fetch data. If the result equals to 1, shard 1 is used. The same logic applies to other shards.

The most critical factor in implementing a sharding strategy is selecting the right sharding key (also known as a partition key). A sharding key is made up of one or more columns that define how data is distributed across different shards. For example, in above Figure, the column “user_id” serves as the sharding key.

By using a well-chosen sharding key, database queries can be routed directly to the correct shard, enabling faster and more efficient data retrieval and modification. However, the effectiveness of sharding heavily depends on this choice — an ideal sharding key ensures that data is evenly distributed across shards, avoiding hotspots and imbalances.

While sharding is a powerful technique for achieving database scalability, it is not a silver bullet. It introduces new complexities and challenges, such as:

Increased operational overhead for managing multiple shards.
More complex query logic, especially for cross-shard operations.
Higher development and maintenance costs.
The need for robust monitoring and failure recovery strategies.

In short, sharding can unlock massive scalability, but it requires careful planning, precise implementation, and ongoing management to deliver its full benefits.

🚀 Scaling Systems to Millions of Users and Beyond

Scaling a system to handle millions of users is not a one-time task — it’s an iterative process. Each stage of growth brings new challenges, requiring fine-tuning and innovative strategies. What worked for thousands of users may break at scale, so continuous learning and optimization are essential.

As traffic grows, you may need to optimize system performance, improve fault tolerance, and decouple monolithic components into smaller, independent microservices. The techniques discussed in this chapter provide a strong foundation for scaling, but true scalability comes from adapting to evolving demands.

To summarize, here are the key strategies for scaling a system to millions of users and beyond:

✅ Keep the web tier stateless — avoid storing sessions locally.
✅ Build redundancy at every tier — eliminate single points of failure.
✅ Cache aggressively — reduce load with smart caching strategies.
✅ Leverage multiple data centers — improve resilience and latency.
✅ Use a CDN for static assets — speed up content delivery worldwide.
✅ Shard your data tier — scale databases horizontally.
✅ Split tiers into microservices — improve flexibility and maintainability.
✅ Monitor continuously & automate — detect issues early and respond quickly.