2 users edit the same Google Doc at the same time.

One edit is applied.

The 2nd one gets silently dropped.

And you can’t fix that with faster servers or more caching.

It’s a concurrency problem, and it’s exactly why “design Google Docs” shows up so often in senior system design interviews.

Not just Google Docs. Any real-time collaborative system runs into this.

Here’s an example of what actually happens when you ignore this problem:

1. User A deletes a word at position 3

2. User B adds an exclamation mark at position 11

3. Both edits land at the same time

4. Now one of them points at the wrong position in the document

5. Content’s corrupted, and the user has no clue

Most candidates answer this with “just use a database lock” and move on.

That’s the wrong answer.

The real answer means knowing 3 different approaches:

1. Optimistic locking with document versions (simple, but bad UX)

2. Operational transformation (what Google actually uses, no edits ever lost)

3. CRDTs (what Figma uses for complex relationships)

Knowing the difference between these puts you in a completely different tier as a candidate.

I broke down the full design from requirements, high-level architecture, and all 3 concurrency strategies with real examples.

1. Design Requirements

We need to handle 10M registered users and up to 500 concurrent editors per document.

We also need strict latency requirements to make it feel instant.

Edits must appear to other users within 100 milliseconds.

HTTP polling will not work for this level of latency.

We need to maintain persistent connections to stream these updates.

Before drawing boxes on a whiteboard we need to run the math.

Assume 1% of users are active at peak times.

That gives us 100,000 active users.

If an average document has 5 editors we are looking at 20,000 active documents at any given moment.

The maximum document size is 500 kilobytes.

Multiplying that out means we have a maximum of 10 gigabytes of active data.

This easily fits into the memory of a single Redis cluster.

At the absolute peak worst case scenario we might have 10 million concurrent connections.

If one server handles 50,000 connections we only need 200 servers at peak time.

We can set up autoscaling to scale these instances down when traffic drops.

2. API And Caching Architecture

When a user opens an existing document they connect to an API gateway.

The gateway handles authentication and rate limits.

It then proxies that connection to a WebSocket server.

We use WebSockets because two users need to see each others changes live.

But a single WebSocket connection is not enough.

If user A and user B connect to different servers they cannot talk to each other directly.

This is where Redis comes in.

We introduce a Redis cluster with a pub sub mechanism.

When user A makes an edit it goes to their WebSocket server.

That server publishes the change to Redis.

Redis broadcasts it to the WebSocket server holding user B.

User B then sees the edit appear on their screen instantly.

Redis also acts as our lightning fast cache.

It holds all active documents in memory so we do not have to read from the database on every single page load.

A separate document service runs in the background to persist these incoming edits to the main database.

3. Designing The Database Schema

Our database needs to store rich text with bolding and italics.

We can store this content in a tree structure using JSON format.

You have 3 primary options for this storage layer.

You could use MongoDB since it natively stores documents without schema friction.

The downside is you lose relational integrity for user permissions and collaborator lists.

You could try splitting the data by putting relationships in PostgreSQL and content in MongoDB.

This introduces a distributed transaction problem.

If the relational update succeeds but the content update fails you have to build complex rollback logic.

The safest choice is PostgreSQL with a JSONB column.

PostgreSQL natively supports JSONB so you can store the entire rich text tree securely.

It also lets you maintain strict foreign keys for users and document owners.

We never send the entire 500 kilobyte document on every keystroke.

We only send the small delta payload.

If a user types a single letter we send a tiny 50 byte packet containing the character and its exact position.

4. Resolving Conflicts And Concurrency

What happens when 2 users edit the same paragraph at the exact same millisecond?

The simplest solution is optimistic locking.

Every edit includes a document version number.

If user A and user B both send an edit for version 4 the server accepts the first one.

The document bumps to version 5.

When the second edit arrives the server sees a version mismatch and rejects it.

The client must then fetch the new version and retry the edit.

This works but creates a terrible user experience.

The second approach is Operational Transformation.

This is exactly what Google uses under the hood.

Both edits arrive at the server.

The server transforms the position index of the second edit so both actions survive.

No edits are ever rejected or lost.

The third approach is a CRDT or conflict free replicated data type.

Figma relies heavily on this pattern.

The data structure itself handles the merging without needing a central coordinator.

Every character gets a unique ID and maps directly to its neighbor instead of an absolute position index.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

What are Load Balancers?

Load balancers distribute incoming network traffic across multiple servers to ensure no single server bears too much load. By spreading the requests efficiently, they increase the capacity and reliability of applications. Here are some common strategies and algorithms used in load balancing:

1. Round Robin

Round Robin is the simplest form of load balancing, where each server in the pool gets a request in a sequential, rotating order. When the last server is reached, it loops back to the first.

This works well for servers with similar specifications and when the load is uniformly distributable.

2. Least Connections

Least Connections algorithm directs traffic to the server with the fewest active connections. This is particularly useful when there are sessions of variable lengths and demands.

It is ideal for longer tasks or when the server load is not evenly distributed.

3. Least Response Time

This responsiveness-focused algorithm chooses the server with the lowest response time and with the fewest active connections.

Effective when the goal is to provide the fastest response to requests.

4. IP Hash

IP Hash determines which server receives the request based on the hash of the client’s IP address. This ensures a client consistently connects to the same server.

Useful for session persistence in applications where it’s important that a client consistently connects to the same server.

5. Weighted Algorithms

There are also variants of the above methods that can be weighted. For example, in Weighted Round Robin or Weighted Least Connections, servers are assigned weights typically based on their capacity or performance metrics.

Weighted algorithms are effective when servers in the pool have different capabilities (e.g., CPU, RAM).

6. Geographical Algorithms

These are location-based algorithms that direct requests to the server geographically closest to the user or based on specific regional requirements.

Useful for global services where latency reduction and local regulatory compliance are priorities.

7. Consistent Hashing

Consistent hashing uses a hash function to distribute data across various nodes.

Imagine a hash space that forms a circle where the end wraps around to the beginning, often referred to as a “hash ring”. Both the nodes (servers) and the data (like keys of stored values) are hashed onto this ring.

This also ensures that the same client consistently connects to the same server.

Health Checks

An essential feature of load balancers is continuous health checking of servers to ensure traffic is only directed to servers that are online and responsive. If a server fails, the load balancer will stop sending traffic to it until it is back online.

Load Balancer Examples

Load balancers come in various forms, including hardware appliances, software solutions, and cloud-based services. Here are some examples:

Hardware Load Balancers

1. F5 BIG-IP: A widely used hardware load balancer known for its high performance and extensive feature set, offering local traffic management, global server load balancing, and application security.

2. Citrix ADC: Formerly known as NetScaler, it provides load balancing, content switching, and application acceleration.

Software Load Balancers

1. HAProxy: A popular open-source software load balancer and proxy server for TCP and HTTP-based applications. It’s known for its efficiency, reliability, and low memory footprint.

2. NGINX: Often used as a web server, NGINX also functions as a load balancer and reverse proxy for HTTP and other network protocols.

Cloud-Based Load Balancers

1. AWS Elastic Load Balancing (ELB): Offers several types of load balancers as part of the AWS cloud services, including the Application Load Balancer, Network Load Balancer, and Classic Load Balancer.

2. Microsoft Azure Load Balancer: Provides high availability and network performance to applications running in Azure. It supports inbound and outbound scenarios.

3. Google Cloud Load Balancing: A fully distributed, software-defined, managed service for all your traffic. It offers various types, including HTTP(S), TCP/SSL, and UDP load balancing.

Virtual Load Balancers

1. VMware NSX Advanced Load Balancer (Avi Networks): Offers a software-defined application delivery controller that can be deployed on-premises or in the cloud.

What happens when the load balancer goes down?

The load balancer is basically a single point of failure, and in case it goes down, all of the servers become unavailable for the clients.

To avoid or minimize the impact of a load balancer failure, several strategies are typically employed:

Redundancy

Implementing redundant load balancing by using more than one load balancer, often in pairs, is a common approach. If one fails, the other takes over, a method known as failover.

Health Checks and Monitoring

Continuous monitoring and health checks of the load balancer itself can ensure that any issues are detected early and can be addressed before causing significant disruption.

Auto-scaling and Self-Healing Systems

Some modern infrastructures are designed to automatically detect the failure of a load balancer and replace it with a new instance without manual intervention.

DNS Failover

In some configurations, DNS failover can reroute traffic away from an IP address that’s no longer accepting connections (like a failed load balancer) to a preconfigured standby IP.

A proxy server acts as an intermediary between a client requesting a resource and the server providing that resource. It can serve various purposes like caching resources for faster access, anonymizing requests, enforcing security policies, and load balancing among multiple servers.

Essentially, it receives requests from clients, forwards them to the relevant servers, and then returns the server’s response to the client.

Types of Proxy Servers

There are several types of proxy servers, each serving different purposes. Here are some of the main types:

1. Forward Proxy: This type of proxy server sits in front of clients and is used to send requests to other servers on the internet. It can provide anonymity and is often used within internal networks to control internet access.

2. Reverse Proxy: Sits in front of one or more web servers, intercepting requests from the internet. It’s used for load balancing, web acceleration, and as a security and anonymity layer.

3. Open Proxy: Allows users to connect and utilize the proxy server, often used to anonymize web browsing and bypass content restrictions.

4. Transparent Proxy: Passes along requests and responses without modifying them, but it’s visible to the client. Often used for caching and content filtering.

5. Anonymous Proxy: Identifiable as a proxy server but does not make the original IP address available. This type is used for anonymous browsing.

6. Distorting Proxy: Provides an incorrect original IP to the destination server, similar to an anonymous proxy, but with purposeful IP misinformation.

7. High Anonymity Proxy (Elite Proxy): This makes detecting the proxy use very difficult. These proxies do not send X-Forwarded-For or other identifying headers, ensuring maximum anonymity.

The most commonly used proxy servers are Forward and Reverse proxies.

Forward Proxy

A forward proxy is a middle layer between the client and the server. It sits between the client (e.g., a computer on an internal network) and the external servers (e.g., the websites on the internet).

When the client makes a request, it is first sent to the forward proxy. The proxy then evaluates the request and decides, based on its configuration and rules, whether to allow the request, modify it, or block it.

Press enter or click to view image in full size

One of the primary functions of a forward proxy is to hide the client’s IP address. When it forwards the request to the target server, it appears as if the request is coming from the proxy server itself.

This can provide a degree of anonymity for the user and is particularly useful for privacy concerns or circumventing IP-based restrictions or monitoring.

Forward Proxy Use Cases

Press enter or click to view image in full size

Instagram Proxies: These are a specific type of forward proxies used to manage multiple Instagram accounts without triggering bans or restrictions.

Marketers, social media managers, and influencers use Instagram proxies to appear as if they are located in different areas or as different users, allowing them to manage multiple accounts, automate tasks, or gather data without being flagged for suspicious activity. They are a good example of forward proxies tailored for a specific application.

Internet Use Control and Monitoring: Organizations also use forward proxies to monitor and control employee internet usage.

They can block access to non-work related sites, ensure compliance with organizational policies, and protect against web-based threats.

Caching Frequently Accessed Content: Forward proxies can also cache frequently accessed content.

This means if multiple users request the same content, the proxy can serve that content from its cache rather than fetching it from the external server each time, improving load times and reducing bandwidth.

This is especially beneficial in networks where bandwidth is costly or limited.

Anonymizing Web Access: Individuals concerned about privacy might use forward proxies to hide their IP address and other identifying information from websites they visit, making it difficult to track their web browsing activities.

Reverse Proxy

A reverse proxy is a type of proxy server that sits in front of one or more web servers, intercepting requests from clients before they reach the servers.

While a forward proxy hides the client’s identity, a reverse proxy essentially hides the server’s identity or the existence of multiple servers behind it. The client interacts only with the reverse proxy and may not know about the servers behind it.

Press enter or click to view image in full size

It distributes client requests across a number of servers, thereby balancing load and ensuring no single server becomes overwhelmed, which is critical for maintaining high availability and reliability.

It can also compress inbound and outbound data, cache files, and manage SSL encryption, thereby speeding up load time and reducing server load.

From the client’s point of view, it appears as if they are communicating directly with the intended server. The client sends a request to a specific web address, and the reverse proxy forwards that request to the appropriate server behind it.

Examples of Reverse Proxies

Press enter or click to view image in full size

Load Balancers: These distribute incoming network traffic across multiple servers, ensuring no single server gets too much load and preventing any server from becoming a bottleneck.

They can be considered a type of reverse proxy specifically designed for balancing load among servers.

CDNs (Content Delivery Networks): CDNs are a network of servers that deliver cached static content from websites to users based on the geographic location of the user.

They act as reverse proxies by retrieving content from the origin server and caching it so that it’s closer to the user for faster delivery.

Web Application Firewalls (WAFs): WAFs are positioned in front of web applications; they inspect incoming traffic to block hacking attempts and filter out unwanted traffic. WAFs also protect the application from common web exploits.

SSL Offloading/Acceleration: Some reverse proxies handle the encryption and decryption of SSL/TLS traffic, offloading that task from web servers to optimize their performance.

Load balancers are perhaps the most common use cases of Proxy Servers. If you’d like to learn more about them, check out my next article, which will be more in-depth about load balancers and the specific algorithms used in them.

1. Start Simple: The Single Server Setup

Designing a system to support millions of users is challenging. But every complex system starts with something simple.

We need to build a basic setup that supports just 1 single user. Then we will gradually expand it as we go.

Starting small allows us to understand each core component before adding more complexity.

Imagine we are setting up a system for a small user base. This means everything runs on 1 single server.

The web application, the database, the cache and other components all live here.

When users want to access your app, they use a web browser or a mobile application. They type your domain name into their browser.

The browser contacts a Domain Name System (DNS). The DNS maps the domain to your servers IP address and sends it back to the client.

With this IP address in hand, the user device sends an HTTP request to your server. The server processes this request and sends back the data.

2. Database Selection: SQL vs NoSQL

As our user base grows, a single server isn’t enough. We need to separate our web tier from our data tier.

This setup enables us to scale each server based on its specific load.

When it comes to database selection, you have two main options.

The first option is relational databases (SQL). These are structured in tables and rows.

Popular examples include PostgreSQL, MySQL and SQLite.

They support complex join operations across multiple tables. They also provide robust data consistency and integrity for transactions.

The second option is non relational databases (NoSQL). These are suited for applications that require flexibility and fast access to large volumes of unstructured data.

Examples include MongoDB, Cassandra and Redis.

NoSQL databases can handle highly dynamic data sets without the strict structure imposed by relational databases. They are optimized for low latency and massive scalability.

3. Scaling: Vertical vs Horizontal

Now you have your database setup. But your app is getting more traffic and your servers are struggling.

You need to scale. There are two primary approaches to scaling.

Vertical scaling means adding more resources to your existing server. You add more RAM, CPU or storage.

This approach is simple and works well for applications with moderate traffic.

But it comes with severe limitations. There is a hard cap on how much you can upgrade a single server.

It also lacks redundancy. If this server goes down, your entire application goes down with it.

Horizontal scaling means adding more servers to share the load. Instead of making 1 server bigger, you replicate it and run 3 servers.

This is generally more suitable for high traffic applications.

It comes with higher fault tolerance. If one server goes down, you still have two other servers available to serve your users.

4. Load Balancing Strategies

When you have 3 servers handling your traffic, you need a way to distribute client requests.

This is where a load balancer comes in.

The load balancer sits between your clients and your servers. It decides where each request should go.

It controls fault tolerance by monitoring server health. If a server goes offline, the load balancer stops sending traffic to it.

There are several algorithms load balancers use to distribute traffic.

Round Robin is the simplest form. Each server gets a request in sequential rotating order.

Least Connections directs traffic to the server with the fewest active connections. This is useful for applications with variable session lengths.

IP Hash determines which server receives the request based on the clients IP address. This ensures a client consistently connects to the exact same server.

You can implement load balancing using software like Nginx, hardware like F5 or cloud solutions like AWS Elastic Load Balancing.

5. Avoiding Single Points of Failure

A single point of failure is any component that will bring the entire system down if it stops working.

If you have multiple API servers but only 1 database, that database is a single point of failure.

If you have multiple servers but only 1 load balancer, that load balancer is a single point of failure.

Having single points of failure creates massive vulnerabilities in your architecture.

A single failure can mean major business losses because users cannot access your platform.

It restricts your ability to scale and creates serious security risks.

To protect your system, you need to add redundancy.

You can use multiple load balancers. If the primary load balancer goes down, traffic automatically redirects to the secondary one.

You also need health checks for the load balancers themselves. This ensures traffic only flows to healthy routing layers.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

Imagine a company is hosting a website on a server in a Google Cloud data center in Finland. It may take around 100ms to load for users in Europe, but it takes 3–5 seconds to load for users in Mexico. Fortunately, there are strategies to minimize this request latency for far-away users.

Press enter or click to view image in full size

These strategies are called Caching and Content Delivery Networks (CDNs), which are two important concepts in modern web development and systems design. Let’s break them down:

Caching

Caching is a technique used to improve the performance and efficiency of a system. It involves storing a copy of certain data in a temporary storage area (the cache) so that future requests for that data can be served faster.

Different Caching Strategies

Caching data can greatly improve the performance of applications. There are typically 4 common places where we can store cached data.

1. Browser Caching

Browser caching involves storing website resources on a user’s local computer. When a user revisits a site, the browser can load the site from the local cache rather than fetching everything from the server again.

Press enter or click to view image in full size

Disabling Cache in Browser: Users can disable caching by adjusting browser settings. In most browsers, developers can disable cache from their developer tools, usually found in the network settings. For instance, you can use the “Disable cache” option in Chrome’s Developer Tools > Network tab.

Storage Location: The cache is stored in a directory on the client’s hard drive, managed by the browser. Browser caches store HTML, CSS, and JS bundle files on the user’s local machine, typically in a dedicated cache directory managed by the browser.

Cache-Control Header: The Cache-Control: max-age=3600 directive tells the browser to cache the file for 3600 seconds (1 hour).

Press enter or click to view image in full size

Cache Hit and Cache Miss

Cache Hit occurs when the requested data is already in the cache. On the other hand, a Cache Miss occurs when the requested data is not in the cache and needs to be fetched from the original source.

Press enter or click to view image in full size

The Cache Ratio is the percentage of requests that are served from the cache compared to all requests. A higher ratio indicates a more effective cache.

You can check if the cache was hit or missed from Browsers’ developer tools indicating cache status in the network tab. A response header like X-Cache: Hit signifies a cache hit.

Press enter or click to view image in full size

2. Server Caching

Server caching involves storing frequently accessed data on the server, reducing the need for expensive operations like database queries.

Press enter or click to view image in full size

Location: Server-side caches are stored on the server itself or a separate cache server, either in memory (like Redis) or on disk.

Procedure: Typically, the server checks the cache for data before querying the database. If the data is in the cache, it is returned directly.

But if data is not in the cache, the server retrieves it from the database, returns it to the user, and then stores it in the cache for future requests.

Press enter or click to view image in full size

Cache Invalidation

The procedure we saw above is the Write-Around cache where data is written directly to permanent storage, bypassing the cache. This is used when write performance is less critical.

We also have a Write-Through cache. This is when data is simultaneously written to the cache and the permanent storage. It ensures data consistency but can be slower.

Press enter or click to view image in full size

Another type is a Write-Back Cache. In this case, data is first written to the cache and then to permanent storage at a later time. This improves write performance but risks data loss in case of a crash in the server or cache server.

Press enter or click to view image in full size

The choice depends on data type, access pattern, and performance requirements.

Eviction Policies: The Decision Makers of Caching

Eviction policies make the crucial decisions about which items to evict (or remove) from the cache when it’s filled to the brim.

Let’s delve into some of the most common eviction policies:

1. Least Recently Used (LRU): Imagine a line where the last one to arrive is the first one to leave. LRU operates on this principle. It tracks the usage history and discards the least recently accessed items first. It’s like a memory that prioritizes the freshest experiences.

2. First In First Out (FIFO): FIFO is the epitome of fairness — the first item to enter the cache is the first to exit. This method doesn’t consider how often or how recently data was accessed. It’s like a respectful queue where everyone waits their turn, regardless of their importance.

3. Least Frequently Used (LFU): LFU is like a popularity contest. It keeps tabs on how often an item is accessed. The items with the least hits get the boot first. This policy favors items that prove their worth by being frequently requested.

Press enter or click to view image in full size

Each of these policies has its strengths and weaknesses, and the choice depends on the specific needs and behavior of the system in question. For instance, LRU is generally good for scenarios where the most recently accessed data is likely to be needed again. On the other hand, LFU is ideal for situations where access patterns don’t change quickly over time.

Adaptive Policies

Some systems implement adaptive policies, which combine elements of these basic strategies. These more complex algorithms can adjust their behavior based on the evolving patterns of data access, ensuring optimal cache performance across a variety of scenarios.

Custom Policies

In some cases, particularly complex systems may require custom-designed eviction policies. These are tailored specifically to the unique needs of the system and can consider various factors such as the size, type, and cost of retrieving each item.

Impact of Eviction Policies

The choice of an eviction policy can significantly impact the performance of a caching system. A well-chosen policy ensures that the cache stores only the most useful data, thereby reducing latency and improving response time.

3. Database Caching

Database caching stands as a cornerstone in the realm of caching strategies, playing a pivotal role in enhancing the performance of applications that heavily rely on database interactions.

Press enter or click to view image in full size

Implementation

Database caching can be implemented in two primary ways:

1. Internal Caching: This is where the database system itself maintains a cache. It’s akin to having a quick reference guide built right into the database.

2. External Caching: In this approach, an external cache (like Redis or Memcached) works in tandem with the database. Think of it as having a dedicated assistant whose sole job is to remember frequently requested data.

How It Functions

When a query is sent forth into the system, the database cache steps in like a gatekeeper. It first checks its own memory to see if the answer to that query is already known. If the result is present in the cache (a cache hit), it’s returned directly, thus bypassing the need for the database to laboriously process the query again.

Press enter or click to view image in full size

Dealing with Cache Misses

A cache miss is akin to hitting a temporary roadblock. When the cache doesn’t have the required data, the system proceeds to execute the query against the database. After retrieving the needed information, it doesn’t just stop there — it stores this result in the cache. This way, the next time the same query knocks on the door, the cache is ready with the answer.

Press enter or click to view image in full size

Ideal Use Cases: When Database Caching Shines

Database caching is particularly beneficial for applications that are read-heavy — those where certain queries are like popular tunes played over and over. In these scenarios, caching can significantly reduce the time spent re-running the same queries, thereby improving overall performance.

Eviction Policies

Much like other caching systems, database caches aren’t infinite wells; they, too, need rules to decide what stays and what goes. They employ various eviction policies to manage their memory usage effectively. One common policy is LRU (Least Recently Used).

The Broader Impact

Implementing an effective database caching strategy can lead to remarkable improvements in application responsiveness and efficiency. It reduces the load on the database, speeds up data retrieval, and ensures a smoother user experience. However, like any powerful tool, it requires careful tuning and management. The choice of caching method, the size of the cache, and the eviction policies must all be calibrated to the specific needs of the application for optimal performance.

4. Content Delivery Networks (CDNs)

CDNs are a network of servers distributed geographically, generally used to serve static content such as JavaScript, HTML, CSS, images, and video assets. They cache the content from the original server and deliver it to users from the nearest CDN server.

Press enter or click to view image in full size

How CDNs Work

The process of Content Delivery Networks (CDNs) is as follows:

1. Initial Request: A user asks for a file — maybe an image, a video, or a web page.

2. Nearest Server Response: This request is swiftly rerouted to the closest CDN server.

3. Content Delivery: If this server already has the content cached, it’s delivered straight to the user.

4. Fetching and Forwarding: In cases where the CDN server doesn’t have the content, it retrieves it from the origin server, stores it (caches it), and then sends it to the user. This step ensures that the next time someone asks for the same content, it’s ready to go instantly.

Press enter or click to view image in full size

CDN Types: Push vs. Pull

Pull-based CDN: Here, the CDN plays a proactive role. It pulls content from the origin server upon the first user request, perfect for sites with regularly updated static content.

Press enter or click to view image in full size

Push-based CDN: In this scenario, you’re in charge. You directly upload content to the CDN. This method is ideal for large files that don’t change often but need rapid distribution when they do. It’s akin to sending packages to a courier service for delivery.

Press enter or click to view image in full size

Guiding CDN Behavior

We use request headers to control the CDN behavior.

• Cache-Control: This header is the rulebook for how long the CDN and browser caches should store content, with directives like max-age, no-cache, public, and private.

• Expires: It’s like an expiration date for content, marking when it becomes stale.

• Vary: This header adapts the served content based on specific request headers, ensuring the right version of the content is delivered.

Choosing Between CDN and Origin Server

Opt for CDN When:

• You’re distributing static assets (images, CSS, JavaScript).

• High availability and performance across various regions are crucial.

• Offloading the origin server is a priority.

Go Direct to Origin Server When:

• The content is dynamic, frequently changing, or personalized.

• Real-time processing or fresh data is needed.

• Complex server-side logic is involved that can’t be handled by a CDN.

Overview

In summary, caching and CDNs are crucial in enhancing web performance, reducing latency, and ensuring a scalable and efficient delivery of content over the internet. They are fundamental in the architecture of any high-traffic, performance-critical web application.

CDN benefits

• Reduced Latency: By serving content from locations closer to the user, CDNs significantly reduce latency.

• High Availability and Scalability: CDNs can handle high traffic loads and are resilient against hardware failures.

• Improved Security: Many CDNs offer security features like DDoS protection and traffic encryption.

Overall Caching Benefits

• Reduced Latency: Faster data retrieval since the data is fetched from a nearby cache rather than a remote server.

• Lowered Server Load: Caching reduces the number of requests to the primary data source, decreasing server load.

• Improved User Experience: Faster load times lead to a better user experience.

In this deep dive, we’ll go through the API design, starting from the basics and advancing towards the best practices that define exceptional APIs.

As a developer, you’re likely familiar with many of these concepts, but I’ll provide a detailed explanation to deepen your understanding.

API Design: An E-commerce Example

Let’s consider an API for an e-commerce platform like Shopify, which, if you’re not familiar with is a well-known e-commerce platform that allows businesses to set up online stores.

In API design, we’re concerned with defining the inputs (like product details for a new product) and outputs (like the information returned when someone queries a product) of an API.

Press enter or click to view image in full size

This means that we focus on the interface rather than the low-level implementation

API Design and CRUD:

So, the focus is mainly on defining how the CRUD operations are exposed to the user or system interacting with your e-commerce API.

CRUD Stands for Create, Read, Update, Delete. These are the basic operations of any data-driven application.

Press enter or click to view image in full size

For example, to add a new product (Create), you would make a POST request to /api/products where the product details are sent in the request body.

To retrieve products (Read), you need to fetch data with a GET request from /products.

For updating product information (Update), we use PUT or PATCH requests to /products/:id, where id is the id of a product we need to update.

Removing is similar to updating; we make a DELETE request to /products/:id where id is the product we need to remove (Delete).

Communication Protocol and Data Transport Mechanism

Another part is to decide on the Communication Protocol that will be used, like HTTP, WebSockets, etc, and the Data Transport Mechanism: JSON, XML, or Protocol Buffers.

This is the case with RESTful APIs, but we also have GraphQL or gRPC paradigms

API Paradigms

APIs come in different paradigms, each with its own set of protocols and standards.

REST (Representational State Transfer)

Pros: Stateless: Each request from a client to a server must contain all the information needed to understand and complete the request. Uses standard HTTP methods (GET, POST, PUT, DELETE). Easily consumable by different clients (browsers, mobile apps).

Cons: This can lead to over-fetching or under-fetching of data- because More endpoints may be required to access specific data.

Features: Supports pagination, filtering (limit, offset), and sorting. Uses JSON for data exchange.

GraphQL

Pros: Allows clients to request exactly what they need, avoiding over-fetching and under-fetching. Strongly typed schema-based queries.

Cons: Complex queries can impact server performance. All requests are sent as POST requests.

Features: Typically responds with HTTP 200 status code, even in case of errors, with error details in the response body.

gRPC (Google Remote Procedure Call)

Pros: Built on HTTP/2, it provides advanced features like multiplexing and server push. Uses Protocol Buffers, a language-neutral, platform-neutral, extensible way of serializing structured data. Efficient in terms of bandwidth and resources, especially suitable for microservices.

Cons: Less human-readable compared to JSON. Requires HTTP/2 support.

Features: Supports data streaming and bi-directional communication. Ideal for server-to-server communication.

Relationships in API Design

In an e-commerce setting, you might have relationships like the user to orders, orders to products, etc.

Press enter or click to view image in full size

Designing endpoints to reflect these relationships is important. For example, in this scenario GET /users/{userId}/orders should fetch orders for a specific user.

Queries, Limit, and Idempotence of GET Requests

Common queries also include limit and offset for pagination or startDate and endDate for filtering products within a certain date range. This allows users to retrieve specific sets of data without overwhelming the system or the user with too much information at once.

Press enter or click to view image in full size

A well-designed GET request is idempotent, meaning calling it multiple times doesn’t change the result.

GET requests should never mutate data. They are meant only for retrieval.

Backward Compatibility and Versioning:

When modifying endpoints, it’s important to maintain backward compatibility. This means ensuring that changes don’t break existing clients.

Versioning: Introducing versions (like /v2/products) is a common practice to handle significant changes.

Press enter or click to view image in full size

In the case of GraphQL, adding new fields (v2 fields) without removing old ones helps in evolving the API without breaking existing clients.

Rate Limitats and CORS

Another best practice is to set rate limitations. This is used to control the number of requests a user can make in a certain timeframe. This is crucial for maintaining the reliability and availability of your API. It also prevents the API from DDoS attacks.

Press enter or click to view image in full size

Common practice is to also set CORS settings Cross-Origin Resource Sharing (CORS) settings are important for web security. They control which domains can access your API, preventing unwanted cross-site interactions.

Next

API Design is just one part of a system design interview. If you’d like to learn more about the other concepts, I recommend you check out these System Design Concepts next.

Application layer protocols are sets of rules and standards that govern the communication between applications over a network. They operate at the highest layer of the Internet protocol suite, which is responsible for end-to-end communication and user interface functionalities.

In this article, we’ll cover all the essential application protocols from core communication, email-related, and real-time communication to file transfer protocols mostly used in System Design Interviews.

Understanding the Client-Server Model

To start, let’s understand the Client-Server model, which is the fundamental architecture of the Internet where a client requests resources or services, and a server provides them. This model is at the core of network programming and web app development.

But of course, servers can also communicate with other servers, and a client isn’t always the end user; it’s often just another program reaching out for data.

Press enter or click to view image in full size

These data transmission rules are set by Application protocols that dictate how data is transmitted and received over a network.

Core Communication Protocols

The most common protocol is HTTP, which stands for HyperText Transfer Protocol, built upon TCP/IP.

It’s a request-response protocol, but imagine it as a conversation with no memory. Each interaction is separate, with no recollection of the past — this means that the server doesn’t store any context between requests; instead, each request contains all the necessary information.

Press enter or click to view image in full size

Notice how the Headers include details like the URL and method, while the Body carries the substance of the request or response.

Each response also includes the status code, which provides feedback about the result of a client’s request to a server. For instance:

• 2xx Success Codes indicate that the request was successfully received and processed.

• 3xx Redirection Codes: These signify that further action needs to be taken by the user agent to fulfill the request.

• 4xx Client Error Codes: These are used when the request contains bad syntax or cannot be fulfilled.

• 5xx Server Error Codes: The server failed to fulfill a valid request.

The most common HTTP methods that are used are GET, POST, PUT, PATCH, and DELETE.

Press enter or click to view image in full size

WebSockets: The Live Wire of the Web

HTTP is a one-way connection, usually from a client to a server, but what if we need a bidirectional connection?

For real-time updates, we use WebSockets that provide a full-duplex communication channel over a single long-lived connection, allowing servers to push real-time updates to clients.

This is crucial for applications requiring constant data updates without the overhead of repeated HTTP request-response cycles.

Press enter or click to view image in full size

Like for chat applications, live sports updates, or stock market feeds, where the action never stops, and neither does the conversation.

Email Related Protocols

From email-related protocols, SMTP is the standard for email transmission over the Internet.

It is the protocol for sending email messages between servers. Most email clients use SMTP to send emails and either IMAP or POP3 to retrieve them.

• IMAP (Internet Message Access Protocol): IMAP retrieves emails from a server, allowing a client to access and manipulate messages as though they were stored locally, which is ideal for users who need to access their emails from multiple devices.

• POP3 (Post Office Protocol version 3): POP3 is used for downloading emails from a server to a local client, typically used when emails are managed from a single device.

Press enter or click to view image in full size

File Transfer and Management Protocols

FTP is a traditional protocol for transferring files over the Internet, often used in website maintenance and large data transfers.

It is used for transferring files between a client and a server, uploading files to a server, or backing up files.

We also have SSH (Secure Shell), which stands for operating network services securely over an unsecured network.

It’s commonly used for logging into a remote machine and executing commands or transferring files.

Press enter or click to view image in full size

Real-time Communication Protocols

We also have real-time communication protocols like WebRTC that enable browser-to-browser applications for voice calling, video chat, and file sharing without internal or external plugins.

This is essential for applications like video conferencing and live streaming.

Press enter or click to view image in full size

Some other popular real-time communication protocols are:

• MQTT (Message Queuing Telemetry Transport): MQTT is a lightweight messaging protocol, ideal for devices with limited processing power and in scenarios requiring low bandwidth, such as IoT devices. It is optimized for high-latency or unreliable networks.

• AMQP (Advanced Message Queuing Protocol): AMQP is a protocol for message-oriented middleware, providing robustness and security for enterprise-level message communication. For example, it’s used with tools like RabbitMQ.

What is RPC?

Let’s also talk about RPC, or Remote Procedure Call, a protocol that allows a program on one computer to execute code on a server computer. It’s a method used to invoke a procedure (a method or a function) as if it were a local call when, in reality, the procedure is executed on a remote machine.

So, it abstracts the details of the network communication, allowing the developer to interact with remote functions seamlessly as if they were local to the application.

And many application layer protocols may use RPC mechanisms to perform their operations. For example:

• In web services, HTTP requests can result in RPC calls being made on the backend to process data or perform actions on behalf of the client.

• SMTP servers might use RPC calls internally to process email messages or interact with databases.

Press enter or click to view image in full size

Summary

Of course, numerous other application layer protocols exist, but the ones covered here are among the most commonly used and essential for web development.

If you’d like to dive deeper, check Networking Essentials next.

In this guide, we’re focusing on the essential networking basics that every developer should know.

Let’s break down the Internet protocol suite layer by layer — From the Application Layer, where DNS and ports live, down to the Physical Hardware that makes up the backbone of our Internet.

Networking Basics

First, let’s talk about networking basics, meaning how computers communicate with each other.

At the heart of this communication is the IP Address, a unique identifier for each device on a network. IPv4 addresses are 32-bit, which allows for approximately 4 billion unique addresses. However, with the increasing number of devices, we’re moving to IPv6, which uses 128-bit addresses, significantly increasing the number of available unique addresses.

Press enter or click to view image in full size

When two computers communicate over a network, they send and receive packets of data, and each packet contains an IP Header, which contains essential information like the sender’s and receiver’s IP addresses, ensuring that the data reaches the correct destination.

Press enter or click to view image in full size

This process is governed by the Internet Protocol (IP), which is a set of rules that defines how data is sent and received.

Besides the IP layer, we also have the Application layer, where data specific to the application protocol is stored.

The data in these packets is formatted according to specific Application Protocol Data, like HTTP for web browsing so that the data is interpreted correctly by the receiving device.

Press enter or click to view image in full size

TCP and Protocol Layers

Once we understand the basics of IP addressing and data packets, we can dive into the Transport Layer, where TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) come into play.

TCP operates at the transport layer and ensures reliable communication. It is like a delivery guy who ensures your package arrives and checks that nothing is missing.

Each data packet also includes a TCP Header, carrying essential information like port numbers and control flags necessary for managing the connection and data flow.

Press enter or click to view image in full size

TCP is known for its reliability; it ensures the complete and correct delivery of data packets. It accomplishes this through features like sequence numbers, which keep track of the order of packets, and a process known as the 3-way handshake, which establishes a stable connection between two devices.

Press enter or click to view image in full size

Press enter or click to view image in full size

In contrast, UDP is faster but less reliable than TCP. It doesn’t establish a connection before sending data and doesn’t guarantee the delivery or order of the packets.

Press enter or click to view image in full size

But this makes UDP preferable for time-sensitive communications, like video calls or live streaming, where speed is crucial, and some loss of data is acceptable.

DNS

To tie all these concepts together, let’s talk about DNS (Domain Name System).

DNS acts like the internet’s phonebook, translating human-friendly domain names into IP addresses. When you enter a URL in your browser, the browser sends a DNS query to find the corresponding IP address, allowing it to establish a connection to the server and retrieve the webpage.

Press enter or click to view image in full size

The functioning of DNS is overseen by ICANN (Internet Corporation for Assigned Names and Numbers), which coordinates the global IP address space and domain name system.

Domain registrars like Namecheap or Godaddy are accredited by ICANN to sell domain names to the public.

DNS uses different types of records, like A Records, which maps a domain to its corresponding IPv4 address, ensuring that your request reaches the correct server, or AAAA Records, which maps a domain to its IPv6 address.

Networking Infrastructure

Finally, let’s talk about the infrastructure supporting all this communication.

Devices on a network have either public or private IP addresses. Public IP addresses are unique across the internet, while private IP addresses are unique within a local network.

An IP address can be static, permanently assigned to a device, or dynamic, changing over time. Dynamic IP addresses are commonly used for residential internet connections.

Devices connected in a Local Area Network (LAN) can communicate directly. To protect these networks, firewalls are used, monitoring and controlling incoming and outgoing network traffic based on security policies.

Press enter or click to view image in full size

Within a device, specific processes or services are identified by ports, which, when combined with an IP address, create a unique identifier for a network service. Some ports are reserved for specific protocols, like port 80 for HTTP or port 22 for SSH, and so on.

Simplified Layer Diagram

So here’s a simplified diagram that represents the different layers of the Internet protocol suite, from the application layer down to the physical hardware.

Press enter or click to view image in full size

• Application Layer: This is where application-specific protocols (like HTTP for web browsing, FTP for file transfer, and SMTP for email) operate. Data is formatted according to these protocols and sent to the Transport Layer. Ports are used to direct data to the correct application. DNS falls under the Application Layer as it is a network service that translates human-readable domain names (like www.example.com) into IP addresses that the network can understand.

• Transport Layer: In the Transport Layer we have TCP and UDP. TCP provides reliable, ordered delivery of data, while UDP provides faster but unreliable delivery. TCP uses sequence numbers and acknowledgments to ensure all data is received and in the correct order.

• Internet Layer: Here we have IP addresses and IP Header, routing packets across networks. IP addresses are used to identify devices on the network and can be public/private and static/dynamic.

• Link Layer: This layer is responsible for the physical network connections, using MAC addresses to identify devices on the local network and switches to direct traffic.

• Physical Hardware: This includes all the physical devices like computers, routers, firewalls, and the LANs they form.

Thanks for the read! If you’re interested in learning more about System Design, check out this comprehensive tutorial on System Design Interview Concepts:

In today’s article, we’re peeling back the layers and diving deep into the essentials of system design requirements.

Let’s understand the pillars of system design and what it takes to create a robust and resilient application.

What Makes a Good Design?

Before we jump into the technicalities, let’s talk about what makes a good design.

When we talk about good design in system architecture, we’re focusing on a few key principles: scalability, so our system grows with its user base; maintainability, ensuring future developers can understand and improve our system; and efficiency, making the best use of our resources.

But good design also means planning for failure and building a system that not only performs well when everything is running smoothly but also maintains its composure when things go wrong.

Understanding the Basics: Moving, Storing, and Transforming Data

At the heart of system design are three key elements: moving data, storing data, and transforming data. Let’s break these down.

• Moving Data: Data movement is the circulatory system of our application. It’s about ensuring that data can flow seamlessly from one part of the system to another. Whether it’s user requests hitting our servers or data transfers between databases, we need to optimize for speed and security.

• Storing Data: Storing data isn’t just choosing between SQL or NoSQL; it’s about understanding access patterns, indexing strategies, and backup solutions. We need to ensure that our data is not only stored securely but is also readily available when needed.

• Transforming Data: Data transformation is about taking raw data and turning it into meaningful information. Whether it’s aggregating log files for analysis or converting user input into a different format.

Understanding the CAP Theorem: The Balancing Act

Now, let’s take a moment to understand a crucial concept in system design — the CAP theorem, also known as Brewer’s theorem, named after computer scientist Eric Brewer. This theorem is a set of principles that guide us in making informed trade-offs between three key components of a distributed system: Consistency, Availability, and Partition Tolerance.

Consistency:

• Explanation: Consistency ensures that all nodes in a distributed system have the same data at the same time. If you make a change to one node, that change should be reflected across all nodes.

• Example: Think of it like updating a Google Doc. If one person makes an edit, everyone else sees that edit immediately.

Availability:

• Explanation: This means the system is always operational and responsive to requests, regardless of what might be happening behind the scenes.

• Example: Like a reliable online store, no matter when you visit, it’s always open and ready to take your order.

Partition Tolerance:

• Explanation: Partition tolerance refers to the system’s ability to continue functioning even when network partitions occur, meaning if there’s a disruption in communication between nodes, the system still works.

• Example: It’s like having a group chat where, even if one person loses connection, the rest of the group can continue chatting.

The Trade-Off:

According to the CAP theorem, a distributed system can only achieve two out of these three properties at the same time. If you prioritize consistency and partition tolerance, you might have to compromise on availability, and vice versa.

Imagine a banking system. It needs to be consistent and partition-tolerant to ensure financial accuracy, even if it means some transactions take longer to process, temporarily compromising availability.

So every design decision comes with trade-offs.

• For example, a system optimized for read operations might perform poorly on write operations. It’s about balancing performance, complexity, and consistency.

• Or in order to gain performance, we might have to sacrifice a bit of complexity.

So it’s not about finding the perfect solution; it’s about finding the best solution for our specific use case. And that means making informed decisions about where we can afford to compromise.

Availability, SLO, and SLA: Ensuring Your System is Up and Running

One important measurement of a system is its availability This is the measure of a system’s operational performance and reliability.

When we talk about availability, we’re asking: Is our system up and running when our users need it? This is often measured in terms of a percentage, aiming for that golden ‘five nines’ or 99.999% availability. But what does that mean in real-world terms?

Let’s say we’re running a critical service with 99.9% availability, which allows for around 8.76 hours of downtime per year. But if we increase that to 99.999%, we’re talking just about 5 minutes of downtime per year. That’s a massive difference, especially for services where every second counts.

We often measure it in terms of uptime and downtime. And here’s where Service Level Objectives (SLO) and Service Level Agreements (SLA) come into play.

SLOs are like setting goals for our system’s performance and availability. For example, we might set an SLO stating that our web service should respond to requests within 300 milliseconds, 99.9% of the time.

SLAs, on the other hand, are like formal contracts with our users or customers. They define the minimum level of service we’re committing to provide. So if our SLA guarantees 99.99% availability, and we drop below that, we might have to provide refunds or other compensations to our customers.

Our SLOs must align with our SLAs. If our SLOs are too lax, we might consistently meet them but still breach our SLAs, leading to unhappy customers and potential financial penalties.

Reliability, Fault Tolerance, and Redundancy: Preparing for the Worst

Building resilience into our system means expecting the unexpected. This could mean implementing redundant systems, and ensuring there’s always a backup ready to take over in case of failure, or it could mean designing our system to degrade gracefully, so even if certain features are unavailable, the core functionality remains intact.

To measure this aspect we use reliability, fault tolerance, and redundancy.

• Reliability means ensuring that your system works correctly and consistently.

• Fault tolerance is about preparing for when things go wrong. How does your system handle unexpected failures or attacks?

• And redundancy? It’s about having backups, ensuring that if one part of your system fails, there’s another ready to take its place.

Throughput and Latency: The Need for Speed

We also need to measure the performance of our system — and for that, we have Throughput and Latency.

Throughput measures how much data our system can handle over a certain period of time,

• Server throughput: Measured in requests per second (RPS), this metric provides an indication of how many client requests a server can handle in a given timeframe. A higher RPS value typically indicates better performance and the ability to handle more concurrent users.

• Database throughput: Measured in queries per second (QPS), this quantifies the number of queries a database can process in a second. Like server throughput, a higher QPS value usually signifies better performance.

• Data throughput: Often measured in bytes per second (B/s), this reflects the amount of data transferred over a network or processed by a system in a given period of time.

On the other hand latency measures how long it takes to handle a single request. It’s the time it takes for a request to get a response. Why do some requests take longer? It could be due to network issues, server load, or the complexity of the request itself.

Optimizing for one can often lead to sacrifices for the other. For example, batching operations can increase throughput but might also increase latency.

Designing vs. Coding: Why Getting Design Right is Crucial

Designing a system poorly can lead to many issues down the line, from performance bottlenecks to security vulnerabilities. And unlike code, which can be refactored with relative ease, redesigning a system can be a monumental task.

That’s why it’s crucial to invest time and resources into getting the design right from the start and laying a solid foundation that can support the weight of future features and user growth. The earlier we catch design flaws, the better.

Thanks for the read! If you’re interested in learning more about System Design, check out this comprehensive tutorial on System Design Interview Concepts:

Behind every flawless app you use, there’s a world of architecture, testing, monitoring, and security measures. Today, let’s have a look at the very high-level architecture of a production-ready app.

CI/CD Pipelines

Our first key area is Continuous Integration and Continuous Deployment — the CI/CD Pipeline.

This ensures that our code goes from the repository, through a series of tests and pipeline checks, and onto the production server without any manual intervention.

It’s configured with platforms like Jenkins or GitHub Actions for automating our deployment processes.

Interacting with Servers:

Once our app is in production, it has to handle lots of user requests. This is managed by our Load Balancers and Reverse Proxies, like Nginx.

They ensure that the user requests are evenly distributed across multiple servers, maintaining a smooth user experience even during traffic spikes.

The Backbone: Data Storage & External APIs

Our server is also going to need to store data. For that, we also have an external storage server that is not running on the same production server. Instead, it’s connected over a network.

Our servers might also be communicating with other servers as well. And we can have many such services not just one.

Monitoring, Logs, & Alerts: The Silent Protectors

To ensure everything runs smoothly, we have logging and monitoring systems, keeping a keen eye on every micro-interaction, storing logs, and analyzing data.

It’s standard practice to store logs on external services, often outside our primary production server.

For the backend, tools like PM2 can be used for logging and monitoring. On the front end, platforms like Sentry can be used to capture and report errors in real time.

Alerting Service

And when things don’t go as planned, meaning our logging system detects failing requests or anomalies?

First, it informs our alerting service. After that, Push notifications are sent to keep users informed. From generic ‘Something went wrong’ to specific ‘Payment Failed’, effective communication ensures users aren’t left in the dark, fostering trust and reliability.

Modern practice is to integrate these alerts directly into platforms we commonly use, like Slack.

Imagine a dedicated Slack channel where alerts pop up the moment an issue arises. This allows developers to jump into action almost instantly, addressing the root cause before it escalates.

Debugging in Production:

After that developers have to debug the issue.

Log Diving: First and foremost, the issue needs to be identified. Those logs we spoke about earlier? They’re our first port of call. Developers sift through them, searching for patterns or anomalies that could point to the source of the problem.

Replicate in a Safe Environment: The golden rule is to — Never debug directly in the production environment. Instead, developers recreate the issue in a ‘staging’ or ‘test’ environment. This ensures users don’t get affected by the debugging process.

Developers use tools to peer into the running application and start debugging.

Hotfixes: Once the bug is fixed, a ‘hotfix’ is rolled out. This is a quick, temporary fix designed to get things running again. It’s like a patch before a more permanent solution can be implemented.

What’s Next

If you’re interested in learning more about System Design, check out this comprehensive tutorial on System Design Interview Concepts:

1. Start Simple: The Single Server Setup

Designing a system to support millions of users is challenging. But every complex system starts with something simple.

We need to build a basic setup that supports just 1 single user. Then we will gradually expand it as we go.

Starting small allows us to understand each core component before adding more complexity.

Imagine we are setting up a system for a small user base. This means everything runs on 1 single server.

The web application, the database, the cache and other components all live here.

When users want to access your app, they use a web browser or a mobile application. They type your domain name into their browser.

The browser contacts a Domain Name System (DNS). The DNS maps the domain to your servers IP address and sends it back to the client.

With this IP address in hand, the user device sends an HTTP request to your server. The server processes this request and sends back the data.

2. Database Selection: SQL vs NoSQL

As our user base grows, a single server isn’t enough. We need to separate our web tier from our data tier.

This setup enables us to scale each server based on its specific load.

When it comes to database selection, you have two main options.

The first option is relational databases (SQL). These are structured in tables and rows.

Popular examples include PostgreSQL, MySQL and SQLite.

They support complex join operations across multiple tables. They also provide robust data consistency and integrity for transactions.

The second option is non relational databases (NoSQL). These are suited for applications that require flexibility and fast access to large volumes of unstructured data.

Examples include MongoDB, Cassandra and Redis.

NoSQL databases can handle highly dynamic data sets without the strict structure imposed by relational databases. They are optimized for low latency and massive scalability.

3. Scaling: Vertical vs Horizontal

Now you have your database setup. But your app is getting more traffic and your servers are struggling.

You need to scale. There are two primary approaches to scaling.

Vertical scaling means adding more resources to your existing server. You add more RAM, CPU or storage.

This approach is simple and works well for applications with moderate traffic.

But it comes with severe limitations. There is a hard cap on how much you can upgrade a single server.

It also lacks redundancy. If this server goes down, your entire application goes down with it.

Horizontal scaling means adding more servers to share the load. Instead of making 1 server bigger, you replicate it and run 3 servers.

This is generally more suitable for high traffic applications.

It comes with higher fault tolerance. If one server goes down, you still have two other servers available to serve your users.

4. Load Balancing Strategies

When you have 3 servers handling your traffic, you need a way to distribute client requests.

This is where a load balancer comes in.

The load balancer sits between your clients and your servers. It decides where each request should go.

It controls fault tolerance by monitoring server health. If a server goes offline, the load balancer stops sending traffic to it.

There are several algorithms load balancers use to distribute traffic.

Round Robin is the simplest form. Each server gets a request in sequential rotating order.

Least Connections directs traffic to the server with the fewest active connections. This is useful for applications with variable session lengths.

IP Hash determines which server receives the request based on the clients IP address. This ensures a client consistently connects to the exact same server.

You can implement load balancing using software like Nginx, hardware like F5 or cloud solutions like AWS Elastic Load Balancing.

5. Avoiding Single Points of Failure

A single point of failure is any component that will bring the entire system down if it stops working.

If you have multiple API servers but only 1 database, that database is a single point of failure.

If you have multiple servers but only 1 load balancer, that load balancer is a single point of failure.

Having single points of failure creates massive vulnerabilities in your architecture.

A single failure can mean major business losses because users cannot access your platform.

It restricts your ability to scale and creates serious security risks.

To protect your system, you need to add redundancy.

You can use multiple load balancers. If the primary load balancer goes down, traffic automatically redirects to the secondary one.

You also need health checks for the load balancers themselves. This ensures traffic only flows to healthy routing layers.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

1. The Core Requirements And Math

We need to handle 10M registered users and up to 500 concurrent editors per document.

We also need strict latency requirements to make it feel instant.

Edits must appear to other users within 100 milliseconds.

HTTP polling will not work for this level of latency.

We need to maintain persistent connections to stream these updates.

Before drawing boxes on a whiteboard we need to run the math.

Assume 1% of users are active at peak times.

That gives us 100,000 active users.

If an average document has 5 editors we are looking at 20,000 active documents at any given moment.

The maximum document size is 500 kilobytes.

Multiplying that out means we have a maximum of 10 gigabytes of active data.

This easily fits into the memory of a single Redis cluster.

At the absolute peak worst case scenario we might have 10 million concurrent connections.

If one server handles 50,000 connections we only need 200 servers at peak time.

We can set up autoscaling to scale these instances down when traffic drops.

2. The API And Caching Architecture

When a user opens an existing document they connect to an API gateway.

The gateway handles authentication and rate limits.

It then proxies that connection to a WebSocket server.

We use WebSockets because two users need to see each others changes live.

But a single WebSocket connection is not enough.

If user A and user B connect to different servers they cannot talk to each other directly.

This is where Redis comes in.

We introduce a Redis cluster with a pub sub mechanism.

When user A makes an edit it goes to their WebSocket server.

That server publishes the change to Redis.

Redis broadcasts it to the WebSocket server holding user B.

User B then sees the edit appear on their screen instantly.

Redis also acts as our lightning fast cache.

It holds all active documents in memory so we do not have to read from the database on every single page load.

A separate document service runs in the background to persist these incoming edits to the main database.

3. Designing The Database Schema

Our database needs to store rich text with bolding and italics.

We can store this content in a tree structure using JSON format.

You have 3 primary options for this storage layer.

You could use MongoDB since it natively stores documents without schema friction.

The downside is you lose relational integrity for user permissions and collaborator lists.

You could try splitting the data by putting relationships in PostgreSQL and content in MongoDB.

This introduces a distributed transaction problem.

If the relational update succeeds but the content update fails you have to build complex rollback logic.

The safest choice is PostgreSQL with a JSONB column.

PostgreSQL natively supports JSONB so you can store the entire rich text tree securely.

It also lets you maintain strict foreign keys for users and document owners.

We never send the entire 500 kilobyte document on every keystroke.

We only send the small delta payload.

If a user types a single letter we send a tiny 50 byte packet containing the character and its exact position.

4. Resolving Conflicts And Concurrency

What happens when 2 users edit the same paragraph at the exact same millisecond?

The simplest solution is optimistic locking.

Every edit includes a document version number.

If user A and user B both send an edit for version 4 the server accepts the first one.

The document bumps to version 5.

When the second edit arrives the server sees a version mismatch and rejects it.

The client must then fetch the new version and retry the edit.

This works but creates a terrible user experience.

The second approach is Operational Transformation.

This is exactly what Google uses under the hood.

Both edits arrive at the server.

The server transforms the position index of the second edit so both actions survive.

No edits are ever rejected or lost.

The third approach is a CRDT or conflict free replicated data type.

Figma relies heavily on this pattern.

The data structure itself handles the merging without needing a central coordinator.

Every character gets a unique ID and maps directly to its neighbor instead of an absolute position index.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

The Database and Transactionality

When dealing with money, you cannot afford eventual consistency in your core ledger. If a user makes a purchase, your database must guarantee that the funds are deducted and the order is marked as paid in a single, atomic step.

This is why NoSQL is generally avoided for core payment state. You need a strictly relational SQL database like PostgreSQL.

SQL provides ACID compliance, which guarantees that if any part of the payment transaction fails, the entire operation rolls back safely. Your database must be the absolute source of truth for the intent to pay, while Stripe remains the source of truth for the actual movement of money.

Idempotency and Retries

Network requests fail. When a payment request times out, you cannot simply guess if the charge went through on Stripe’s end.

If you guess wrong and retry a failed request blindly, you will charge the customer twice. This is where idempotency keys come in.

Before your server calls the Stripe API, you generate a unique key for that specific transaction and save it to your database. When you send the request to Stripe, you include this key in the header.

Stripe binds the charge result to this specific key. If your server crashes and you retry the exact same request, Stripe sees the matching key.

It simply returns the original successful response instead of charging the credit card again.

Handling the Refund Lifecycle

Refunds are not just a reverse payment. They are an entirely distinct state machine that requires careful asynchronous tracking.

A user requests a refund, and your database marks the transaction as pending. Your backend makes an API call to Stripe to initiate the process.

Stripe then communicates with the underlying credit card networks, and this process can take days to actually settle. During this time, your system must track the state without blocking other operations.

Webhooks from Stripe will eventually notify your system when the money actually moves, allowing you to finalize the refund in your database.

Daily Reconciliation at Scale

This is how companies like Shopify, Uber, and Airbnb handle the hardest part of payments at scale. You cannot blindly trust your own database to be perfectly synchronized with Stripe forever.

Your system will eventually drift from reality due to missed webhooks, network partitions, or phantom database failures. Reconciliation is the process of automatically comparing your database against Stripe’s source of truth.

Every night, your backend pulls a massive settlement report from Stripe. A background cron worker cross-references every single transaction ID in that report against your local SQL database.

If your database says a payment succeeded but Stripe says it failed, the system immediately flags the anomaly for human review.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

1. Stop confusing age with experience.

You have probably been told that putting in your time will eventually lead to senior titles and high pay.

This is completely false.

Age actually has nothing to do with your true value in the market.

Experience is defined as repeated exposure to new environments.

The more systems you see and the more projects you touch, the more valuable you become.

Take two developers with a decade of work history.

The first one has worked on the exact same project for 10 straight years.

The second one has shipped 10 different projects in those same 10 years.

The second engineer has seen multiple industries and implemented various architectures in production.

The first engineer is just a mid-level developer repeating their first year over and over again.

They get comfortable, attend endless standups, and just ship minor bug fixes on a legacy codebase.

If you want to break into the upper echelon of engineering, you must escape this trap.

2. Master the fundamental concepts first.

If you have years under your belt but still lack deep confidence, there is a clear reason why.

You know deep down that there are critical architectural concepts you should understand but do not.

This hidden impostor syndrome is exactly what keeps you underpaid.

To command multi-six figures, you need to step outside your current codebase.

You need to master architecture fundamentals and understand how large-scale production systems actually work.

Once you grasp these core patterns, the offer and salary negotiation becomes incredibly easy.

Companies can sense when you do not feel you deserve a high salary.

When you know the fundamentals, your confidence protects your market value.

3. Target well-funded startups instead of big tech.

I used to have a massive limiting belief about making real money in tech.

I thought you absolutely had to work for Meta or Google to earn a 350K salary and live a remote lifestyle.

The reality turned out to be completely different.

I have worked with around 15 different companies and none of them were big tech giants.

Well-funded AI startups often pay way more than traditional prestige companies.

They desperately need experienced talent to scale quickly.

Because they lack the household name recognition of a FAANG company, they compensate with massive paychecks and fully remote flexibility.

This is your unfair advantage.

You can skip the corporate bureaucracy and still secure top-tier compensation.

4. Change environments to force massive growth.

My very first role was with a terrible company that micromanaged every single task.

I had to work 10-hour days just to hit artificial deadlines.

If I had stayed comfortable and remained on that legacy codebase, I would still be a junior developer today.

Changing your environment frequently forces you to adapt and realize what a healthy engineering culture looks like.

You might worry that you will fail at a new company and get fired.

That exact fear is what forces you to grind, learn production architectures, and truly prove yourself.

Every highly paid role relies on the exact same fundamental concepts.

Once you master those core principles, you can drop into any tech stack or industry and immediately deliver value.

Taking calculated risks pays off much faster than staying comfortable.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

1. Understanding the API’s Core Purpose

An API, or Application Programming Interface, defines how different software components should interact with each other. Think of it as a clear contract specifying what requests can be made, what data should be provided, and what responses to expect.

This interaction happens between a client (like a mobile phone or web browser) and a server. The API sets the terms for this communication, acting as an abstraction layer.

It hides the server’s complex implementation details, exposing only the necessary functionality. For instance, when you save user data, you do not need to know the intricate logic happening behind the scenes on the server.

You just use the provided interface.

APIs also establish clear service boundaries between systems and components. This allows for multiple servers, perhaps one managing users and another handling posts, to communicate seamlessly regardless of their underlying technologies.

It is the fundamental piece connecting modern applications.

2. Mastering API Paradigms: REST, GraphQL, and gRPC

Choosing the right API style is a critical architectural decision. It directly impacts your system’s performance, scalability, and how other teams integrate with your work.

The three most dominant API styles you will encounter are REST, GraphQL, and gRPC.

REST (Representational State Transfer)

This is the most common style, using a resource-based approach with standard HTTP methods. REST APIs are stateless, meaning each request contains all necessary information, without relying on prior requests.

They utilize HTTP methods like GET for fetching, POST for storing, PUT/PATCH for updating, and DELETE for removing data.

• Pros: Stateless, uses standard HTTP methods, easily consumable by various clients.

• Cons: Can lead to over-fetching or under-fetching data as clients often receive more or less than they need.

• When to Use: Web and mobile applications where resource-based interactions are natural.

GraphQL

GraphQL is a query language that empowers clients to request exactly what data they need, nothing more, nothing less. It typically operates over a single endpoint for all operations.

• Pros: Minimizes round trips, prevents over-fetching/under-fetching, strongly typed schemas.

• Cons: Complex queries can impact server performance, typically all requests are POST.

• When to Use: Complex UIs where different pages require varied and nested data, optimizing data retrieval efficiency.

gRPC (Google Remote Procedure Call)

This is a high-performance RPC framework built by Google, leveraging HTTP/2 and Protocol Buffers for efficient communication. gRPC methods are defined as RPCs in proto files.

• Pros: Highly efficient in terms of bandwidth and resources, supports streaming and bidirectional communication.

• Cons: Less human-readable than JSON, requires HTTP/2 support.

• When to Use: Microservices and internal system communication where high efficiency and server-to-server speed are paramount.

Your protocol choice fundamentally shapes your API design options. HTTP protocol features directly enable RESTful capabilities, offering status codes ideal for CRUD operations.

WebSockets, conversely, enable real-time and bidirectional APIs, perfect for chat applications or video streaming. gRPC is optimized for microservices communication, making it faster than typical HTTP-based interactions.

3. Core Principles of Production-Grade APIs

Designing APIs effectively goes beyond just knowing the styles; it requires adhering to core principles that ensure consistency, simplicity, security, and performance. The best API is one that developers can use intuitively, without constant reference to documentation.

First, consistency is key. This means employing consistent naming conventions, casing, and patterns across all endpoints.

If you use userDetails (camel case) in one endpoint, avoid user_details (snake case) in another. Such inconsistencies create confusion and increase the learning curve for consumers.

Second, keep it simple. Focus on core use cases and intuitive design, minimizing unnecessary complexity.

Designs that developers can understand quickly are always preferred. Simplicity ultimately means an API that is easy to adopt and integrate.

Third, security is paramount. Implementing robust authentication and authorization is non-negotiable.

Input validation is crucial to prevent common vulnerabilities, and applying rate limiting protects your API from abuse and DDoS attacks. These are foundational requirements for any production system.

Finally, performance demands attention. Design for efficiency with appropriate caching strategies, which can significantly reduce database load and response times.

Implement pagination for large datasets to avoid overwhelming systems or users with too much information at once. Minimize payloads to reduce bandwidth usage, and whenever possible, reduce round trips by consolidating related data into fewer requests.

4. The End-to-End API Design Process

The API design process extends far beyond mere coding. It starts with a comprehensive understanding of requirements, identifying core use cases and user stories.

Defining the scope and boundaries is also crucial; for large APIs, you cannot develop all features at once, so scoping to specific, deliverable features is essential.

You must also determine performance requirements early, anticipating bottlenecks and ensuring the API is performant where it matters most. Never overlook security constraints, implementing basic features like authentication, authorization, and rate limiting, alongside any specific security needs dictated by your API’s function.

When it comes to design approaches, you might use a top-down approach, starting with high-level requirements and workflows, which is common in interviews. Alternatively, a bottom-up approach is suitable when you have existing data models and capabilities within a company.

The contract-first approach involves defining the API contract (what requests and responses look like) before implementation, similar to a top-down method.

An API’s lifecycle involves several phases: design, development and local testing, deployment and monitoring (staging and production testing), and ongoing maintenance. The maintenance phase highlights why simplicity is critical during development, ensuring future developers can easily manage the API.

Eventually, APIs also go through deprecation and retirement, as newer versions emerge or systems evolve. This continuous cycle emphasizes that API development is a holistic process, not just a coding task.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

“Open to Work” on LinkedIn gives away your leverage before even the screening stage with the recruiter.

When a company sees that banner, they already know you need them more than they need you, which shifts the entire dynamic.

You’re no longer a top candidate they’re competing for. Instead, you’re waiting for an opportunity, and that is the worst position to negotiate from.

Think about it this way.

5-star hotels don’t advertise available rooms. They’re booked months in advance, and that waitlist is exactly what makes them desirable. Meanwhile, average hotels have rooms open the night before because nobody’s competing to get in.

Top companies with real budgets think the same way. They’re not browsing the “available” pool. They’re going after engineers who are already in demand, and they’re willing to pay a premium for that.

I personally never had an open to work banner on my profile, and I still get contacted by recruiters every week about new opportunities.

The engineers landing top remote roles aren’t broadcasting availability. They’re being pulled in through referrals, content, and targeted outreach, which means they control the narrative before the call even happens.

The green banner works if you’re early in your career and need volume, but don’t expect to land the top opportunities that way.

You win by being the obvious choice to the right people.

But removing the badge alone won’t fix everything. Because it’s just the symptom. The real work is building demand before you need a job.

The Signal You’re Actually Sending

Here’s what most people don’t realize. When a recruiter lands on your profile and sees that badge, they’re not just noting that you’re available. They’re running a quick mental calculation.

How long have they had this badge? If it’s been weeks or months, a quiet question forms: why hasn’t anyone hired them yet?

This isn’t always fair or accurate. The job market is competitive and great engineers are sitting in long search cycles through no fault of their own. But perception is reality in hiring.

A recruiter who works with me told me directly: when she’s sourcing candidates, she looks at profiles without the badge first. Her reasoning was simple. Someone without the badge might be off the market within the week if she doesn’t move fast. Someone with the badge will probably still be available next month.

Urgency drives action. Scarcity creates urgency.

If your profile signals that you’ll always be available, it removes the urgency for recruiters to move quickly, or at all.

Why Scarcity Drives Hiring Decisions

This isn’t a new idea. Robert Cialdini wrote about the scarcity principle decades ago. People assign more value to things that appear limited and in demand. Hiring follows the same psychology.

A developer who’s currently employed and isn’t visibly searching reads like the 5-star hotel. A developer with the badge who’s been openly available for months reads like the motel with a permanent vacancy sign.

Both can write code. But the experience of recruiting them feels completely different.

Companies with real budgets and serious engineering problems aren’t searching for whoever is available. They’re trying to pull in the person who isn’t looking.

The Positioning Problem Most Engineers Ignore

Here’s the part most engineers miss completely.

Think about phones. When you walk into a store with no budget constraints, you’re picking iPhone or Samsung. Nobody reaches for Xiaomi or Oppo when money isn’t the issue. It’s not purely about specs. It’s about what the product signals.

Most engineers have built themselves into the Xiaomi of the talent market. Technically capable. Competitively priced. But not the one a well-funded company reaches for when they want the best.

And that’s not a skills problem. It’s a positioning problem.

Tech skills are the foundation. They’re not negotiable. But they’re the bottom of the pyramid.

The real leverage sits at the top: how you’re perceived, how you communicate your value, and whether you come across as someone companies compete to hire or someone companies settle for.

The “Open to Work” badge is one of the most visible ways engineers undercut their own positioning before a recruiter even reads their headline.

What to Do Instead

If you’re actively searching, here’s the honest reality: removing the badge alone won’t fix everything. But it’s the starting point for thinking about this differently.

Reconnect with your network. Former colleagues, managers, people you’ve shipped things with. Warm introductions convert at a far higher rate than cold applications. Most senior roles are filled before they’re ever posted.

Do targeted outreach. Reach out directly to senior engineers and hiring managers at companies you actually want to work at. A thoughtful, specific message beats a badge every time.

Create signal. Share insights, write about problems you’ve solved, post about what you’re learning. When a recruiter finds you because of something valuable you shared, the entire dynamic flips. You’re no longer a candidate. You’re someone worth finding.

The goal is to shift from someone who’s searching to someone who’s worth finding.

Cold applications to job postings are getting less effective every year. Your network and your positioning are the highest-leverage investments you can make right now.

The badge is just the symptom. The real work is building demand before you need a job.

Picking the wrong dev career path costs you years of salary and freedom.

Most developers just take the first fulltime job they find and never question if there’s a better path for them.

I’ve worked across almost every path on the spectrum.

Onsite. Remote. Contractor. Startups. Enterprises. Outstaff.

And here’s what I learned:

The path you choose doesn’t just affect your paycheck.

It affects how protected you are from layoffs, how fast your skills grow and whether you ever reach $200–300k or beyond.

Let’s break down all 6 career paths and their tradeoffs, lifestyle differences and which one fits where you actually are right now.

1. The Safest Route: Fullstack & Onsite

The career path of a software engineer exists on a wide spectrum of risk and reward.

On the far left side of this spectrum sits the most common and arguably the safest option available today.

Fulltime onsite roles at major tech companies or local firms provide a massive layer of security. Because management physically sees you at your desk writing code and attending meetings, they develop an innate trust in your output.

If your performance ever drops, they are much more likely to pull you aside for a coaching conversation rather than immediately terminating your employment. Your job is incredibly secure as long as you keep your skills reasonably sharp.

Earning a six-figure salary is highly achievable once you reach a mid to senior level, and in certain tech hubs, even junior developers can cross this threshold.

2. Fulltime Remote

Moving slightly to the right on the risk spectrum brings us to full-time remote roles. This path unlocks true geographical arbitrage.

You can live in a region with an incredibly low cost of living while earning a premium salary from a company based in the United States or Western Europe. The lifestyle upgrade is immediate and profound.

However, this freedom comes at a tangible cost to your job security. When economic conditions tighten and corporations decide to execute mass layoffs, remote employees are almost always the first group on the chopping block.

You are disconnected from the physical office politics, making you an easier line item to cut. You are essentially trading ultimate corporate safety for absolute personal freedom.

But once again, if you maintain elite engineering skills, you insulate yourself from the worst of these corporate risks.

3. The Contractor Advantage

Right in the middle of the risk and reward spectrum lies contracting. This has been my personal strategy of choice over the last few years.

Instead of hunting for direct employment, you partner with an intermediary agency that constantly sources fresh clients. You remain fully remote but you completely eliminate the exhausting process of doing your own sales and marketing.

The true superpower of contracting is the ability to stack multiple projects simultaneously. You can sign contracts with two or three different clients at the exact same time, effectively doubling or tripling your annual income in a matter of months.

These contracts typically run on fixed timelines, such as six months at a time. Having a hard deadline constantly approaching forces you to keep your technical skills incredibly sharp.

You never have the luxury of getting lazy because you know you must be interview-ready the moment your current contract expires. I once pushed this to the absolute limit by taking on three simultaneous projects.

I was grinding for up to twelve hours a day and nearly reached the point of total burnout. However, I managed to condense an entire year of standard salary into a brutal three-month sprint.

I originally planned to take nine full months off just to recover. But the incredible network you build with agency founders makes long-term unemployment nearly impossible.

Within four weeks, a founder was begging me to take a new project because they lacked qualified talent. I threw out a wildly inflated rate simply because I did not want to work, and they accepted it immediately.

The leverage you gain in this position is unmatched.

4. Freelancing

Moving to the far right side of the spectrum takes you out of pure engineering and plunges you straight into entrepreneurship. Freelancing is the first step into this high-risk zone.

You are no longer just writing code. You are suddenly responsible for lead generation, cold email outreach, client negotiations, and closing sales.

This path is realistically only suited for two extremes. It works perfectly for absolute beginners who have zero risk and desperately need immediate cash flow to survive.

It also works beautifully for elite multi-six-figure engineers who want to transition into high-ticket consulting. But for the average mid-level developer, it is a nightmare of hunting for cheap clients on crowded platforms like Upwork or Fiverr.

5. Agency or Service provider

If you want to push the boundaries even further, you can become a direct service provider. This involves registering a formal LLC, securing large client contracts, and hiring an entire team of junior developers to execute the actual coding beneath you.

You transition from a builder into a manager of builders. This is the definitive blueprint for scaling a six-figure income into a seven-figure business.

6. Building your SaaS

The absolute peak of the risk spectrum is building your own SaaS application. You engineer a highly scalable product and attempt to sell it to thousands of recurring users.

The financial upside is technically infinite, but the failure rate is brutal. You should aggressively avoid building a SaaS until your primary career path is already generating well over 200k a year in stable income.

You need massive personal cash flow to survive the inherent risks of building a product from scratch.

Want to go deeper? I cover this topic with more context, examples and real-world detail in the video below 👇

What is Authentication?

Authentication answers one simple question: Who is this user? When a login request comes in from a user or another service, this is where we confirm their identity.

If valid, we grant access to the system. If invalid, we reject it with a 401 Unauthorized response.

This is the first step before authorization, which controls what they can actually do once inside.

Token-Based Authentication

Instead of sessions, modern applications use tokens. The client sends a token with each request.

A Bearer token just means whoever has this token gets access. It’s a pattern, not a specific method.

The most common type of bearer token is a JWT (JSON Web Token). A JWT is a signed JSON object that contains the user ID, expiration time, and other claims.

JWTs are stateless. Your API doesn’t need to look up anything in a database.

It just verifies the signature locally and reads the data inside.

Access and Refresh Tokens

Modern systems use two types of tokens. Access tokens are short-lived and used for API calls.

Refresh tokens are long-lived and used to get new access tokens. When the access token expires, the application uses the refresh token behind the scenes to silently get a new one.

This keeps your system secure because access tokens expire quickly, but users don’t get logged out constantly.

OAuth2 and OpenID Connect

OAuth2 is an authorization framework, not authentication. It answers what this app can access on behalf of the user.

OpenID Connect (OIDC) adds authentication on top of OAuth2. When you log in with Google or GitHub, you’re using OIDC.

The provider sends back an ID token in JWT format. Your app reads this ID token to know exactly who the user is.

So OIDC is the authentication layer, and OAuth2 underneath it is the authorization layer.

Want to go deeper? I cover this topic with more context, examples, and real-world detail in the video below 👇

Before ranking anything, here’s the filter I use for AI tools as a developer.

Most of these tools cost between $20 and $40 a month.

If the tool saves you even one hour per month and you make more than $20/hour as a developer (which you do), it already pays for itself.

So the math is simple.

The discipline to apply it is what most developers skip, and most tools fail it immediately.

That’s because most AI developer tools are just wrappers for GPT.

After testing over 100, here are the 24 that are actually worth your time, ranked from S-tier to E-tier.

1. Daily AI Chat Tools (Web)

These are the primary interfaces for general-purpose reasoning and brainstorming.

• Claude (A Tier): Consistently outperforms others in coding benchmarks and complex challenges. It is highly effective for system diagrams and architectural feedback

• ChatGPT (B Tier): While the web version is less integrated than specialized tools, models like GPT-5.2 are excellent for brainstorming and solving niche tasks where Claude might fail

• Gemini (C Tier): Historically behind its competitors, though recent updates are closing the gap. It is currently less used in daily developer workflows

• Grok (C Tier): Specialized for coding but hasn’t yet replaced the daily habit of using ChatGPT or Claude

2. Coding Assistants and IDEs

These tools provide direct integration into the development environment.

• Cursor (S Tier): The leading AI IDE. Features like “Plan Mode” and “Agent Mode” allow for deep debugging and direct file modification

• Claude Code (S Tier): A terminal-based assistant that excels at high-complexity tasks, such as managing version upgrades and researching dependency compatibility

• MCP Servers (S Tier): Not a single tool, but a protocol that allows AI to access real-time documentation for frameworks like Next.js or AWS, ensuring suggestions follow the latest best practices

• GitHub Copilot (C/B Tier): The pioneer in the space. While great for basic autocompletion, it is currently perceived as less advanced than specialized AI IDEs like Cursor

• Windsurf (D Tier): Gained hype but often fails to match the performance of Cursor on specific coding tasks

• Kodium & Tab 9 (D Tier): While Tab 9 offers better privacy, both generally provide less utility than Copilot or Cursor

3. Productivity and Work Tools

• Loom (A Tier): Essential for async communication. It uses AI to automatically generate titles, summaries, and transcripts for video walkthroughs

• Grammarly (A Tier): A staple for documentation and communication, particularly the pro version for rephrasing and tone adjustments

• Whisper AI (B/A Tier): Useful for converting spoken prompts into text to save typing time

4. Workflow Automation

• n8n (A Tier): Highly customizable and developer-friendly, allowing for the creation of complex AI agents and workflows

• Zapier (B Tier): The most popular choice for simple event-based triggers, though it can become expensive at high volumes

• Make.com (B Tier): A cost-effective alternative to Zapier for scaling automations

5. Rapid Development (MVP Builders)

• v0 (C Tier): The strongest tool for Next.js development. Best for quickly validating market demand with a functional UI and SEO optimization

• Lovable (D Tier): Useful for generating React-based codebases but often lower in quality for complex needs

• Bolt.new (E Tier): A tool for “vibecoding” that builds React apps via prompts, though often ranked lowest in overall code quality

6. Design and Diagramming

• Eraser (S Tier): The top choice for technical diagrams. It supports “diagram as code,” making it easy to feed system architecture context into LLMs.

• Canva (A Tier): While not developer-specific, its AI features like “Magic Expand” are invaluable for content creators and minor UI tasks.

• Figma (B Tier): Essential for designer collaboration. Using Dev Mode is highly recommended to save time on CSS and layout implementation.

• Miro (C/B Tier): A standard for team brainstorming and retrospectives, though less specialized for technical architecture than Eraser.

Watch the full video breakdown here:

Companies don’t pay six figures for people who just code or follow instructions.

They pay for architectural decisions. For making systems performant.

For optimizing data storage. For making decisions that affect real customers and real software at scale.

Most devs never develop this skill because most environments don’t force you to. You get comfortable, attend standups, and just ship features on top of someone else’s foundation.

But the moment you sit in a senior interview or inherit a system that needs to scale from 100 to 1 million users, the gap becomes painfully obvious.

This is why I designed this post to start from scratch and go through all the foundational concepts you need to know to land senior/staff level roles.

From the actual mental models I used to pass system design interviews and reach senior level in year two of my career.

Start Simple: The Single Server Setup

Before you can design anything meaningful, you need to understand how systems handle traffic at a basic level.

Here’s how a single-server setup works:

User types website.com into their browser. Their device contacts a DNS provider, which resolves that domain to your server’s IP address.

With that IP in hand, the device sends an HTTP request to your server. Your server processes it and sends back either an HTML page (for browsers) or a JSON response (for mobile apps).

Simple. But this simplicity is the point.

When you understand this flow, you can reason about where things break when traffic scales. The server gets overwhelmed.

The database becomes a bottleneck. A single point of failure brings everything down.

From here, we separate the web tier from the data tier. Now your application servers and your database can scale independently based on their specific load.

And this is where database selection becomes one of the most important early decisions you’ll make.

Relational databases like PostgreSQL or MySQL use structured tables, rows, and SQL. They shine when your data has clear relationships and you need strong consistency — think financial applications or e-commerce order tracking.

They support complex join operations and ACID transactions: atomic, consistent, isolated, and durable.

Non-relational databases like MongoDB, Cassandra, or Redis take a different approach. Document stores let you nest complex data in a single record.

Wide column stores like Cassandra handle massive write volumes at scale. Key-value stores like Redis live in RAM and are blindingly fast.

Graph databases like Neo4j map relationships between entities, which is how Amazon powers its recommendation engine.

The choice isn’t about which is better. It’s about which fits your data model, your consistency requirements, and your scale targets.

Scaling: Vertical vs Horizontal

Once your single server can’t keep up with demand, you have two paths.

Vertical scaling means adding more resources to your existing server — more RAM, more CPU. It’s simple and works for moderate traffic.

But it has a hard ceiling. At some point, you can’t add any more resources to a single machine.

And if that machine goes down, everything goes down with it.

Horizontal scaling means adding more servers and distributing the load between them. Instead of one powerful server, you have three (or thirty) working together.

This gives you fault tolerance — if one server goes down, the others keep serving users. And it gives you flexibility — you can add more servers as demand grows.

But horizontal scaling introduces a new problem: how does a request know which server to go to?

That’s where the load balancer comes in. Think of it as a traffic cop sitting in front of your server pool.

Every incoming request hits the load balancer first, and the load balancer decides where to send it.

There are several strategies for making that decision: round robin, least connections, least response time, IP hashing, weighted algorithms, geographical routing, and consistent hashing.

Load balancers also run continuous health checks on your servers, automatically stopping traffic to any server that goes offline and resuming it when it recovers.

Popular options include Nginx and HAProxy as software load balancers you configure yourself, hardware options like F5, and cloud-managed solutions like AWS Elastic Load Balancing which handle scaling and monitoring automatically.

Single Points of Failure and How to Eliminate Them

Here’s a concept that will change how you think about every system you design: the single point of failure.

A single point of failure is any component that, if it fails, takes the entire system down with it.

In a typical setup where clients connect to a load balancer, the load balancer routes to API servers, and all API servers share one database — that database is a single point of failure. If it goes down, every API server loses its data connection, and every user hits an error.

This has three real consequences.

Reliability: one failure cascades into a total outage.

Scalability: systems built around single points of failure are hard to grow because every new component increases the risk of that one critical piece failing under load.

Security: a single point of failure is also a single attack surface. Flood it with traffic, and the whole system collapses.

The solution is redundancy at every critical layer. For load balancers specifically, you run multiple instances.

If one goes down, traffic automatically reroutes to the healthy instance. When the failed one recovers, load is rebalanced.

You also implement continuous health monitoring, and for mature systems, you build self-healing mechanisms that automatically replace a failed component with a new instance before users even notice.

API Design: The Contract That Holds Everything Together

Once you understand the infrastructure layer, the next critical skill is API design.

An API is a contract. It defines what requests can be made, what format they take, what responses to expect, and what methods are available.

A well-designed API hides implementation details while exposing clean functionality. You can call an endpoint to save user data without knowing anything about how the server stores it, what database it uses, or how the business logic runs.

You just use the interface.

There are three primary API styles you’ll encounter:

REST is the most common. It’s resource-based, stateless, and uses standard HTTP methods: GET to fetch, POST to create, PUT or PATCH to update, DELETE to remove.

Each request contains everything needed to process it. REST works well for web and mobile applications and supports HTTP caching through headers.

GraphQL is a query language that lets clients request exactly the data they need, nothing more. Instead of making three separate REST calls to get a user’s profile, posts, and followers, you make one GraphQL query.

It uses a single endpoint for all operations. GraphQL is the better choice for complex UIs where different views need different shapes of data.

gRPC is a high-performance framework that uses protocol buffers instead of JSON. Methods are defined in proto files and it supports streaming and bidirectional communication.

It’s the preferred choice for internal microservice communication where efficiency matters more than broad compatibility.

Knowing which API style to reach for — and why — is exactly the kind of decision seniors are expected to make without being told.

This was just an overview. The full course goes through each of these step by step 👇