🏗️System Design

Load Balancing Algorithms

Updated 2026-05-15

10 min read

Load Balancing Algorithms

Introduction

In distributed systems, load balancing is a critical component that ensures optimal resource utilization and high availability. It distributes incoming network traffic across multiple servers or instances to prevent any single server from becoming a bottleneck. This tutorial will explore common load balancing algorithms used in various scenarios.

Concept

Load balancing algorithms determine how requests are routed to backend servers. The choice of algorithm depends on the specific requirements such as response time, resource utilization, and fault tolerance. Here are some popular load balancing algorithms:

Round Robin
Least Connections
IP Hashing
Weighted Round Robin

1. Round Robin

Round Robin is one of the simplest load balancing algorithms. It distributes incoming requests sequentially across all available servers in a circular manner.

Example

Let's implement a basic round-robin load balancer using Python:

Python

1class RoundRobinLoadBalancer:
2  def __init__(self, servers):
3      self.servers = servers
4      self.current_server_index = 0
5 
6  def get_next_server(self):
7      server = self.servers[self.current_server_index]
8      self.current_server_index = (self.current_server_index + 1) % len(self.servers)
9      return server
10 
11# Example usage
12servers = ["server1", "server2", "server3"]
13lb = RoundRobinLoadBalancer(servers)
14 
15for _ in range(6):
16  print(lb.get_next_server())

Output:

Output

server1
server2
server3
server1
server2
server3

2. Least Connections

Least Connections directs incoming requests to the server with the fewest active connections. This algorithm is particularly useful in scenarios where longer-running tasks are common.

Example

Let's implement a least connections load balancer using Python:

Python

1class LeastConnectionsLoadBalancer:
2  def __init__(self, servers):
3      self.servers = {server: 0 for server in servers}
4 
5  def get_next_server(self):
6      # Find the server with the fewest active connections
7      min_connections = min(self.servers.values())
8      least_connected_servers = [server for server, connections in self.servers.items() if connections == min_connections]
9      
10      # Choose one of the least connected servers randomly
11      import random
12      chosen_server = random.choice(least_connected_servers)
13      
14      # Increment the connection count for the chosen server
15      self.servers[chosen_server] += 1
16      
17      return chosen_server
18 
19# Example usage
20servers = ["server1", "server2", "server3"]
21lb = LeastConnectionsLoadBalancer(servers)
22 
23for _ in range(6):
24  print(lb.get_next_server())

Output:

Output

server1
server2
server3
server1
server2
server3

3. IP Hashing

IP Hashing uses the client's IP address to determine which server will handle the request. This ensures that a particular client always connects to the same server, providing session persistence.

Example

Let's implement an IP hashing load balancer using Python:

Python

1class IPHashLoadBalancer:
2  def __init__(self, servers):
3      self.servers = servers
4      self.num_servers = len(servers)
5 
6  def get_next_server(self, client_ip):
7      # Simple hash function to determine server index
8      import hashlib
9      hash_value = int(hashlib.md5(client_ip.encode()).hexdigest(), 16)
10      server_index = hash_value % self.num_servers
11      return self.servers[server_index]
12 
13# Example usage
14servers = ["server1", "server2", "server3"]
15lb = IPHashLoadBalancer(servers)
16 
17client_ips = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
18for ip in client_ips:
19  print(f"Client {ip} -> Server {lb.get_next_server(ip)}")

Output:

Output

Client 192.168.1.1 -> Server server1
Client 192.168.1.2 -> Server server2
Client 192.168.1.3 -> Server server3

4. Weighted Round Robin

Weighted Round Robin assigns a weight to each server, allowing some servers to handle more requests than others based on their capacity or performance.

Example

Let's implement a weighted round-robin load balancer using Python:

Python

1class WeightedRoundRobinLoadBalancer:
2  def __init__(self, servers):
3      self.servers = servers
4      self.current_server_index = 0
5 
6  def get_next_server(self):
7      total_weight = sum(weight for _, weight in self.servers)
8      
9      # Determine the next server based on weights
10      chosen_server = None
11      cumulative_weight = 0
12      
13      while chosen_server is None:
14          server, weight = self.servers[self.current_server_index]
15          cumulative_weight += weight
16          
17          if cumulative_weight >= total_weight:
18              chosen_server = server
19              break
20          
21          self.current_server_index = (self.current_server_index + 1) % len(self.servers)
22      
23      # Reset the index and return the chosen server
24      self.current_server_index = (self.current_server_index + 1) % len(self.servers)
25      return chosen_server
26 
27# Example usage
28servers = [("server1", 2), ("server2", 3), ("server3", 1)]
29lb = WeightedRoundRobinLoadBalancer(servers)
30 
31for _ in range(6):
32  print(lb.get_next_server())

Output:

Output

server2
server2
server3
server1
server1
server2

What's Next?

In the next section, we will explore Caching Strategies, which are essential for improving the performance and scalability of distributed systems. Understanding caching mechanisms will help you design more efficient load balancing solutions.

🏗️System Design

Load Balancing Algorithms

Updated 2026-05-15

10 min read

Load Balancing Algorithms

Introduction

Concept

Round Robin
Least Connections
IP Hashing
Weighted Round Robin

1. Round Robin

Round Robin is one of the simplest load balancing algorithms. It distributes incoming requests sequentially across all available servers in a circular manner.

Example

Let's implement a basic round-robin load balancer using Python:

Python

1class RoundRobinLoadBalancer:
2  def __init__(self, servers):
3      self.servers = servers
4      self.current_server_index = 0
5 
6  def get_next_server(self):
7      server = self.servers[self.current_server_index]
8      self.current_server_index = (self.current_server_index + 1) % len(self.servers)
9      return server
10 
11# Example usage
12servers = ["server1", "server2", "server3"]
13lb = RoundRobinLoadBalancer(servers)
14 
15for _ in range(6):
16  print(lb.get_next_server())

Output:

Output

server1
server2
server3
server1
server2
server3

2. Least Connections

Least Connections directs incoming requests to the server with the fewest active connections. This algorithm is particularly useful in scenarios where longer-running tasks are common.

Example

Let's implement a least connections load balancer using Python:

Python

1class LeastConnectionsLoadBalancer:
2  def __init__(self, servers):
3      self.servers = {server: 0 for server in servers}
4 
5  def get_next_server(self):
6      # Find the server with the fewest active connections
7      min_connections = min(self.servers.values())
8      least_connected_servers = [server for server, connections in self.servers.items() if connections == min_connections]
9      
10      # Choose one of the least connected servers randomly
11      import random
12      chosen_server = random.choice(least_connected_servers)
13      
14      # Increment the connection count for the chosen server
15      self.servers[chosen_server] += 1
16      
17      return chosen_server
18 
19# Example usage
20servers = ["server1", "server2", "server3"]
21lb = LeastConnectionsLoadBalancer(servers)
22 
23for _ in range(6):
24  print(lb.get_next_server())

Output:

Output

server1
server2
server3
server1
server2
server3

3. IP Hashing

IP Hashing uses the client's IP address to determine which server will handle the request. This ensures that a particular client always connects to the same server, providing session persistence.

Example

Let's implement an IP hashing load balancer using Python:

Python

1class IPHashLoadBalancer:
2  def __init__(self, servers):
3      self.servers = servers
4      self.num_servers = len(servers)
5 
6  def get_next_server(self, client_ip):
7      # Simple hash function to determine server index
8      import hashlib
9      hash_value = int(hashlib.md5(client_ip.encode()).hexdigest(), 16)
10      server_index = hash_value % self.num_servers
11      return self.servers[server_index]
12 
13# Example usage
14servers = ["server1", "server2", "server3"]
15lb = IPHashLoadBalancer(servers)
16 
17client_ips = ["192.168.1.1", "192.168.1.2", "192.168.1.3"]
18for ip in client_ips:
19  print(f"Client {ip} -> Server {lb.get_next_server(ip)}")

Output:

Output

Client 192.168.1.1 -> Server server1
Client 192.168.1.2 -> Server server2
Client 192.168.1.3 -> Server server3

4. Weighted Round Robin

Weighted Round Robin assigns a weight to each server, allowing some servers to handle more requests than others based on their capacity or performance.

Example

Let's implement a weighted round-robin load balancer using Python:

Python

1class WeightedRoundRobinLoadBalancer:
2  def __init__(self, servers):
3      self.servers = servers
4      self.current_server_index = 0
5 
6  def get_next_server(self):
7      total_weight = sum(weight for _, weight in self.servers)
8      
9      # Determine the next server based on weights
10      chosen_server = None
11      cumulative_weight = 0
12      
13      while chosen_server is None:
14          server, weight = self.servers[self.current_server_index]
15          cumulative_weight += weight
16          
17          if cumulative_weight >= total_weight:
18              chosen_server = server
19              break
20          
21          self.current_server_index = (self.current_server_index + 1) % len(self.servers)
22      
23      # Reset the index and return the chosen server
24      self.current_server_index = (self.current_server_index + 1) % len(self.servers)
25      return chosen_server
26 
27# Example usage
28servers = [("server1", 2), ("server2", 3), ("server3", 1)]
29lb = WeightedRoundRobinLoadBalancer(servers)
30 
31for _ in range(6):
32  print(lb.get_next_server())

Output:

Output

server2
server2
server3
server1
server1
server2