A fault-tolerant web application can be built using two identical application servers in the same private network and a Load Balancer that accepts external requests and distributes them across the available backends.
In this guide, we will create two virtual machines running the same application, add them to the load balancer’s backend pool, configure health checks, and assign an external IP address to the Load Balancer. After that, we will verify that requests are actually distributed between both VMs.
Then we will forcibly stop one application server and confirm that the health check marks it as unavailable, while the Load Balancer continues to send requests to the second VM. As a result, the application remains available even if one server fails.
We will also cover where user sessions and data should be stored. Sessions must not be tied only to the memory of a specific VM; otherwise, the user will lose state when switching between backends. For this, it is better to use Redis, a database, or another shared external storage system. User files and persistent data should also be stored outside the application server’s local disk, for example, in a shared database or Object Storage.
As a result, we will have a basic fault-tolerant architecture in which the failure of one VM does not make the entire application unavailable. It is recommended to place VMs in different availability zones and run session databases in a fault-tolerant configuration. This protects not only against the failure of a specific VM (and eliminates downtime during scheduled maintenance), but also against failures at the cloud provider’s sites.
How a Fault-Tolerant Web Application Is Structured
A single application server remains a single point of failure. If the virtual machine shuts down, hangs, or the application stops responding, users will lose access to the entire service.
A basic way to improve fault tolerance is to use multiple identical backend servers and place a Load Balancer in front of them. Users connect only to the load balancer’s single public address, and the load balancer selects an available VM to handle each request. If it is a cloud load balancer, it is typically already implemented in a fault-tolerant configuration and does not become a single point of failure.
Why use multiple application servers
If an application runs on only one VM, any failure of that machine makes the service completely unavailable.
Possible causes include:
- Application crash;
- Insufficient memory;
- An operating system issue;
- Scheduled maintenance, such as a reboot or update;
- Virtual machine failure;
- A network incident.
When two identical application servers are used, a request can be handled by the second VM if the first one is temporarily unavailable.
At a high level, the architecture looks like this:
Both VMs must run the same version of the application and use the same configuration.
It is also important to separate the application itself from persistent data. If one server stores unique user data only on its local disk, having a second VM no longer provides full fault tolerance.
How a Load Balancer Works
A load balancer receives client requests and forwards them to backend servers in its pool.
For example:
From the client’s perspective, both VMs are hidden behind a single public address.
The load balancer can distribute requests in different ways. One simple option is Round Robin, where backends are selected in sequence:
In practice, distribution depends on the balancing algorithm, the status of the backends, and the active connections.
The main advantage of this approach is that the client does not need to know the address of each VM. It always connects to a single endpoint.
What happens when one VM fails
A Load Balancer alone is not enough. It must know which backends can actually accept requests.
Health checks are used for this purpose.
The load balancer regularly checks the application servers, for example at http://PRIVATE_IP/health or simply at /
If the VM responds successfully, the backend is considered healthy: Healthy
If several consecutive checks fail, the backend is moved to the following state: Unhealthy
After that, the Load Balancer stops sending user requests to it.
For example:
The user continues to access the same public address, but all new requests are routed to the second VM.
When the first server starts passing health checks successfully again, the load balancer can automatically return it to the pool.
Components created in this guide
For the practical example, we will need the following infrastructure:
We will create:
- A private network;
- A subnet;
- Two identical VMs;
- A Security Group;
- A Load Balancer;
- A listener;
- A backend pool;
- A health monitor;
- An external IP address for the load balancer.
A small test web page will run on each VM.
To show which server processed the request, the responses will differ slightly: Application Server 1 and Application Server 2.
This will allow us to test both standard load balancing and how the setup works after one virtual machine is stopped.
Preparing the Cloud Infrastructure
First, we will prepare the network, access rules, and two VMs. Both machines must be on the same private network and reachable by the Load Balancer via their internal IP addresses.
Creating a Private Network and Subnet
In the cloud console, create a separate private network, for example: ha-private-network
Create a subnet for it: ha-private-subnet
You can use the following CIDR: 192.168.60.0/24
As a result, both application VMs will receive internal addresses from the same range, for example:
Application Server 1 → 192.168.60.10
Application Server 2 → 192.168.60.11
The specific addresses may be assigned automatically.
The private network is required for internal communication between the Load Balancer and the backend. There is no need to expose each VM separately to the Internet for the application to work.
Preparing the Security Group
Create a separate Security Group, for example: ha-app-sg
For application servers, allow HTTP traffic from the load balancer: TCP 80
SSH will also be required for the initial configuration: TCP 22
If the provider allows rules to be restricted by source, it is best to allow access to port 80 only from the private network or the Load Balancer addresses.
For example:
TCP 80
Source: 192.168.60.0/24
It is also safer to restrict SSH to the administrator IP address, if possible.
Do not expose databases or internal services unless necessary:
3306
5432
6379
They are not required by the Load Balancer for a standard HTTP health check of the application.
Creating Two Identical Virtual Machines
Create two VMs with identical parameters.
For example:
ha-app-01
ha-app-02
For both VMs, select:
- The same Ubuntu image;
- The same flavor;
- The same security group;
- The same private network;
- The same SSH key pair.
The main goal is to make the backends as identical as possible.
If one VM has a different version of the application or system packages, the service behavior may depend on which backend handles a given request.
After launch, make sure that both machines are in the Active state and connected to ha-private-network.
Which ports must be open between components
For our simple HTTP application, the following configuration is sufficient:
| Direction | Port | Purpose |
| Administrator → VM | 22 | initial configuration over SSH |
| Load Balancer → VM | 80 | user requests |
| Health Check → VM | 80 | backend health check |
| Internet → Load Balancer | 80 or 443 | public access to the application |
If the application later runs only over HTTPS, TLS can be terminated directly at the Load Balancer, or HTTPS can be passed through to the backend, depending on the architecture and the capabilities of the cloud platform.
It is important that the VMs are not exposed to the entire internet solely to allow the load balancer to operate. User traffic should come through a single external endpoint.
Preparing the first application server
Now let’s configure the first VM. For testing, Nginx with a static page that clearly shows the backend identifier is sufficient.
Connecting to the first VM
SSH access is required for the initial VM configuration.
If a Floating IP has been temporarily assigned to the machine, connect as follows: ssh -i ~/.ssh/ha-key [email protected]
Here, 203.0.113.21 is used only as an example.
If access to the private VM is provided through a bastion host, a VPN, or the cloud console, use the appropriate method.
After logging in, update the package index: sudo apt update
Installing Nginx or a Test Web Application
Install Nginx: sudo apt install -y nginx
Check the status: sudo systemctl status nginx –no-pager
The service should be in the following state: active (running)
Enable automatic startup: sudo systemctl enable nginx
Check the local response: curl http://127.0.0.1
If Nginx is installed correctly, the server will return the default HTML page.
Creating a page with a server identifier
Replace the default page: sudo nano /var/www/html/index.html
Add some simple HTML:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Application Server 1</title>
</head>
<body>
<h1>Application Server 1</h1>
<p>Status: OK</p>
</body>
</html>
Now the command curl http://127.0.0.1 should return content containing Application Server 1.
You can use the same page for the health check, but it is more convenient to create a separate endpoint.
For example: echo ‘OK’ | sudo tee /var/www/html/health
Check it: curl http://127.0.0.1/health
Response: OK
Later, the Load Balancer will be able to use /health to determine backend availability.
Checking the response from the first application server
First, get the private IP address using hostname -I or ip addr
Then check Nginx from another VM or from an accessible segment of the private network: curl http://192.168.60.10
The response should include the following string: Application Server 1
The health endpoint, curl http://192.168.60.10/health, should return: OK
At this point, the first backend is ready. Next, install the same configuration on the second VM, changing only the page identifier to Application Server 2.
Preparing the Second Application Server
The second virtual machine should be as similar to the first as possible: the same operating system, the same web server, the same application structure, and the same health endpoints. The only difference will be the test page identifier, so we can see which backend handled a particular request.
Installing the same application on the second VM
Connect to the second virtual machine: ssh -i ~/.ssh/ha-key [email protected]
The IP address shown here is an example.
Update the package index: sudo apt update
Install Nginx: sudo apt install -y nginx
Enable Nginx to start automatically: sudo systemctl enable nginx
Check the status: sudo systemctl status nginx –no-pager
The service should be in the following state: active (running)
Creating a page with the second server ID
Now replace the default page: sudo nano /var/www/html/index.html
Add the following:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Application Server 2</title>
</head>
<body>
<h1>Application Server 2</h1>
<p>Status: OK</p>
</body>
</html>
Create the same health endpoint: echo ‘OK’ | sudo tee /var/www/html/health
Check it locally: curl http://127.0.0.1
The response should include: Application Server 2
Health check: curl http://127.0.0.1/health
Response: OK
Verifying identical server configurations
Before adding the VMs to the Load Balancer, make sure both backends operate identically.
On the first VM: nginx -v
On the second VM: nginx -v
Ideally, use the same versions.
Also check the health endpoint on both machines from the private network:
curl http://192.168.60.10/health
curl http://192.168.60.11/health
Both requests should return: OK
The main page, however, is intentionally different:
Application Server 1
Application Server 2
This difference is needed only to demonstrate request distribution. In a real production application, both VMs should serve the same version of the application.
Creating a Load Balancer
After preparing the two backends, you can create a Load Balancer. It will be assigned an interface on the private network and serve as the single entry point for user requests.
Interface element names may vary slightly depending on the OpenStack dashboard, but the overall flow remains the same: Load Balancer → Listener → Pool → Members.
Creating a load balancer in a private network
In the Load Balancers section, create a new load balancer.
Specify a name, for example: ha-web-lb
For the network or subnet, select: ha-private-subnet
The load balancer will receive an internal address in the same network infrastructure where the application servers are located.
After creating it, wait until it reaches a working state. In OpenStack, this may be displayed as ACTIVE or a similar status.
Do not add a public address to each application VM solely for user traffic. The external IP will later be assigned to the Load Balancer itself.
Adding an HTTP or HTTPS listener
A listener defines the protocol and port on which the Load Balancer accepts requests.
For the test setup, use HTTP:
Protocol: HTTP
Port: 80
Name: http-listener
As a result, the Load Balancer will accept connections on port 80 and forward them to the backend pool.
For a production service, HTTPS is typically used instead of HTTP. TLS can be terminated directly on the load balancer if the cloud platform supports certificate uploads.
Creating a backend pool
Now create a pool that will contain two application VMs.
For example:
Name: web-backend-pool
Protocol: HTTP
Select ROUND_ROBIN as the algorithm.
The pool is associated with the listener created earlier.
The diagram now looks like this:
After that, you can add actual backend servers to the pool.
Adding Two VMs to the Pool
Add the first VM as a member:
Address: 192.168.60.10
Protocol Port: 80
And the second one:
Address: 192.168.60.11
Protocol Port: 80
If the interface lets you select instances from a list, you can simply select the required VM and its private interface instead of entering the IP address manually.
After they are added, the structure will look like this:
The Load Balancer now knows where to forward user requests.
However, before using it as a fault-tolerant entry point, you need to configure health checks.
Configuring the request distribution algorithm
For a simple setup with two identical application servers, Round Robin is suitable.
Example:
OpenStack Load Balancer may also support other algorithms, such as Least Connections or Source IP.
Least Connections selects the backend with the fewest active connections. This option can be useful if requests vary in duration.
For the demonstration, we will leave it as: ROUND_ROBIN
This will make it easy to see responses from both VMs during sequential requests.
Configuring Health Checks
Without health checks, the load balancer may continue sending requests to a VM that is no longer serving the application.
Therefore, the next step is to create a Health Monitor for the backend pool.
Why the load balancer needs to check backend servers
The state of a virtual machine in OpenStack and the state of the application itself are not the same.
A VM may be ACTIVE, while Nginx or the application running inside it may no longer be working.
Therefore, the load balancer must perform its own application-level health checks.
For example: GET /health
If the server returns HTTP 200, the backend is considered healthy.
If the request fails several times in a row or times out, the server is temporarily excluded from request distribution.
Creating an HTTP health check
Create a Health Monitor for the web-backend-pool pool.
Type: HTTP
URL path: /health
Expected HTTP status: 200
If the interface supports configuring the HTTP method, leave it set to: GET
The load balancer will regularly send requests to:
http://192.168.60.10/health
http://192.168.60.11/health
and evaluate the health status of both members.
Configuring the interval, timeout, and retry count
For a test setup, you can use relatively short values:
Delay / Interval: 5 seconds
Timeout: 3 seconds
Max Retries: 3
This means the check runs approximately every 5 seconds, and the backend will not be considered unavailable after a single random error.
Choose the parameters so that the load balancer detects real failures quickly enough, but does not remove a server from the pool because of a single brief timeout.
For example, an overly aggressive configuration:
Interval: 1s
Retries: 1
may cause false failovers during brief network delays.
An overly slow configuration:
Interval: 60s
Retries: 5
will, conversely, keep sending requests to a backend that is already unavailable for too long.
Checking the status of both backends
After saving the Health Monitor, wait for several check cycles to complete.
Both VMs should be shown as healthy. Depending on the interface, the status may be labeled:
ONLINE
Healthy
UP
or similar.
The pool should then look something like this:
Application Server 1 192.168.60.10:80 ONLINE
Application Server 2 192.168.60.11:80 ONLINE
If one of the servers remains in the ERROR or OFFLINE state, first check whether the health endpoint is reachable from another node in the private network:
curl http://192.168.60.10/health
curl http://192.168.60.11/health
You also need to check the Security Group. The Load Balancer must be able to connect to TCP port 80 on both VMs.
After both members are consistently passing health checks, you can assign an external IP address to the Load Balancer and test the actual distribution of user requests.
Attaching an External IP Address to the Load Balancer
After configuring the backend pool and health checks, the Load Balancer can already distribute traffic within the private network. To make the application accessible from the internet, assign an external IP address to the Load Balancer.
Assigning a Floating IP
Open the Load Balancer you created: ha-web-lb
and associate a Floating IP from the external network with it.
For example: 203.0.113.30
This address is provided as an example only.
As a result, the diagram will look like this:
Users now use this public address to access the application.
The application servers do not need their own Floating IPs for ongoing operation.
Checking the load balancer’s public address
From your local computer or another external machine, run: curl http://203.0.113.30
You should receive an HTML page from one of the backends in response.
For example:
Application Server 1
Status: OK
or:
Application Server 2
Status: OK
You can also open the address in a browser: http://203.0.113.30
If the application does not open, check the following:
- Load Balancer status;
- Listener status;
- Backend pool status;
- Health checks;
- Security Group;
- Floating IP assignment;
- VM firewall;
- Accessibility of TCP port 80.
Next, proceed with the check.
Checking request distribution between two VMs
A single successful request only confirms that the Load Balancer itself is reachable.
To make sure that both VMs are actually participating in the pool, run several requests in sequence.
With the Round Robin algorithm, responses should be distributed across the two backends.
In simplified terms:
The exact order may vary due to the load balancer’s behavior, existing connections, or HTTP keep-alive settings, so what matters is not a strict sequence but seeing responses from both VMs.
Verifying Request Load Balancing
Next, let’s verify specifically that both application servers are actually handling user traffic.
How to verify that both VMs are working
To do this, we specified different identifiers in advance on the test pages for Application Server 1 and Application Server 2.
All other server parameters must be identical.
If accessing the same Floating IP returns responses from both servers, the Load Balancer is indeed distributing requests between the two members.
Repeated HTTP requests to the load balancer
Run several consecutive requests:
for i in {1..10}; do
curl -s http://203.0.113.30 | grep "Application Server"
done
Example output:
<h1>Application Server 1</h1>
<h1>Application Server 2</h1>
<h1>Application Server 1</h1>
<h1>Application Server 2</h1>
<h1>Application Server 1</h1>
<h1>Application Server 2</h1>
If your shell does not support this type of loop, you can run the requests manually: curl http://203.0.113.30. Run the command several times in a row.
You can also output only the server identifier:
for i in {1..10}; do
curl -s http://203.0.113.30 | grep -o "Application Server [12]"
done
Checking responses from Application Server 1 and Application Server 2
If the output contains both values:
Application Server 1
Application Server 2
this means that both VMs:
- are reachable over the private network;
- are in the backend pool;
- are successfully passing health checks;
- receive user requests through the Load Balancer.
If only one VM always responds, check the status of the second member.
For example, it may have one of the following statuses:
OFFLINE
ERROR
Another possible cause is an incorrectly specified private IP address, backend port, or Security Group.
Simulating the Failure of a Single Virtual Machine
The main test of a fault-tolerant design is a simulated failure of one backend.
We will stop the first VM and check whether the application continues to work through the same public IP address.
Forcibly stopping the first VM
In the control panel, open ha-app-01 and stop the virtual machine.
For this test, it is important to make the backend unavailable, not simply remove it from the pool manually. This lets us see how the Load Balancer responds to an actual failure.
After the VM stops, its state will change, for example, to SHUTOFF or a similar status (the status name may vary depending on the cloud platform interface).
At this stage, do not change the Load Balancer settings or remove the member from the backend pool.
Detecting a Failure via a Health Check
After the VM is stopped, the health monitor will continue to check: 192.168.60.10:80/health
Requests will start timing out or failing with a connection error.
Given the following settings:
Interval: 5s
Timeout: 3s
Max Retries: 3
the transition to the unavailable state will not happen immediately.
After several cycles, the first backend should be marked OFFLINE or Unhealthy.
The second server should remain ONLINE.
Excluding an unavailable backend from load balancing
After the health check marks the first member as unavailable, the Load Balancer stops sending new requests to it.
The backend can remain in the pool; it does not need to be removed manually.
The load balancer simply takes its status into account:
Server 1 → Unhealthy → does not receive new requests
Server 2 → Healthy → receives all traffic
This is one of the key advantages of health checks.
Without them, the Load Balancer could continue sending some requests to a failed server, resulting in intermittent errors for users.
Verifying application behavior after a failure
Now access the same Floating IP again: curl http://203.0.113.30
The application should continue to respond:
Application Server 2
Status: OK
Check it several times:
for i in {1..10}; do
curl -s http://203.0.113.30 | grep -o "Application Server [12]"
done
After removing the first backend, the expected result is:
Application Server 2
Application Server 2
Application Server 2
Application Server 2
Application Server 2
The external address remains unchanged: 203.0.113.30
The user continues to access the same endpoint and does not need to manually switch to another VM.
This test confirms basic application-layer fault tolerance: the failure of one backend does not make the service completely unavailable as long as the second server and the Load Balancer itself remain operational.
Returning a Server to Service After a Failure
After testing fault tolerance, you can start the stopped VM again and verify that the Load Balancer automatically returns it to service.
Starting the stopped VM
In the control panel, open ha-app-01 and start the virtual machine.
After the OS has booted, verify that Nginx is running again: sudo systemctl status nginx –no-pager
Expected state: active (running)
If the service did not start automatically, run: sudo systemctl start nginx
Check locally: curl http://127.0.0.1/health
The response should be: OK
Passing the Health Check Again
After the VM starts, the Load Balancer will begin receiving successful responses again from: http://192.168.60.10/health
The transition back to an operational state is also not instantaneous. The load balancer must receive the required number of successful checks.
After a few cycles, the status of the first backend should change from OFFLINE to ONLINE or a similar state.
The second server continues to operate without any changes.
Returning the VM to the backend pool
If the health monitor is configured correctly, you do not need to manually add the server back to the pool.
The member is still part of web-backend-pool
The Load Balancer only temporarily excludes it from traffic distribution based on the health check results.
After recovery, the diagram looks like this again:
You can verify this by sending repeated requests:
for i in {1..10}; do
curl -s http://203.0.113.30 | grep -o "Application Server [12]"
done
After the first VM is back in service, responses from both backends should appear in the output again.
Where to Store User Sessions
Two VMs and a load balancer address application server failure, but on their own they do not make the application fully fault-tolerant.
It is especially important to properly organize user session storage.
Why Sessions Should Not Be Stored Only in Application Server Memory
Suppose a user signs in through the first backend: User → Load Balancer → Server 1
Server 1 stores the user’s session only in memory.
The Load Balancer sends the next request to the second backend: User → Load Balancer → Server 2
Server 2 knows nothing about the session created on the first VM.
As a result, the user may be unexpectedly logged out or lose the application state.
The problem becomes even more apparent if Server 1 fails: all sessions stored only in its memory disappear with it.
Therefore, user state should not depend on a specific application server.
Storing Sessions in Redis or a Database
One common approach is to move sessions to a separate centralized store.
For example:
Both VMs access the same Redis instance and see the same user session state.
A database can be used instead of Redis if its performance and the application model are sufficient for this scenario.
For example:
The key rule is that the session must be available regardless of which backend receives the request.
For production systems, the external store itself must also be fault-tolerant. If all application servers depend on a single Redis instance or a single database without redundancy, that component becomes a new single point of failure.
When to use sticky sessions
Some load balancers support sticky sessions, also known as session persistence.
In this mode, requests from the same user are routed to the same backend whenever possible.
For example:
User A → Server 1
User A → Server 1
User A → Server 1
User B → Server 2
User B → Server 2
This can be useful for legacy applications that are difficult to quickly adapt to external state storage.
However, sticky sessions do not fully solve the problem.
If Server 1 fails, User A will still be routed to another backend, which may not know about the user’s local session.
Therefore, sticky sessions are better treated as an additional mechanism rather than a replacement for shared state storage.
Why stateless applications are easier to scale
The most convenient model for load balancing is a stateless application server.
This type of backend does not store user-specific state locally.
For example:
Any request can be routed to any available VM.
This simplifies:
- Horizontal scaling;
- Replacing application servers;
- Rolling updates;
- Failure recovery;
- Automatically adding new VMs;
- Load Balancer operation without binding a user to a specific backend.
If you need to increase performance, you can add a third or fourth application server without moving user data between them.
Where to store user data and files
Persistent application data should also not depend on the local disk of a single VM.
Otherwise, the fault-tolerant architecture will protect only the web layer, not the data itself.
Why a VM’s Local Disk Is Not Suitable for Shared Data
Suppose a user uploads a file through Server 1: User → Load Balancer → Server 1
The application saves it to /var/www/app/uploads/ on the local disk of the first VM.
The next request is routed to Server 2, where that file does not exist.
As a result, some requests will succeed, while others will return an error.
The situation becomes even worse if the first VM is deleted or irreversibly damaged: locally stored data may be lost completely.
Therefore, it is best to use the application server’s local file system only for:
- Application code;
- Temporary files;
- Cache data that can be rebuilt;
- Logs, if they are also collected centrally.
Next, we move on to using a shared database.
Using a Shared Database
Structured user data is typically stored in a shared database:
Both VMs connect to the same logical data source.
For example, it can store:
- User accounts;
- Orders;
- Settings;
- Comments;
- Application records;
- Access rights;
- Metadata.
For a truly fault-tolerant architecture, the database must also have its own redundancy or use a highly available managed service.
If the database runs on only one VM, a failure of that VM will bring the application down even if both application servers are healthy.
Object Storage for User Files
It is more convenient to use Object Storage for images, documents, archives, and other binary files.
Diagram:
The application stores the file in a shared bucket rather than on the VM’s local disk.
Both VMs access the same object using the same key.
For example: uploads/users/42/avatar.jpg
This approach is especially convenient for horizontal scaling: new application servers do not require synchronization of directories containing user files.
What happens to data if one VM fails
If the architecture is designed correctly, the failure of a single application server should not affect persistent data.
For example:
Server 1 — OFFLINE
Server 2 — ONLINE
Redis — ONLINE
Database — ONLINE
Object Storage — ONLINE
The Load Balancer stops sending requests to Server 1, while Server 2 continues to use the same:
- User sessions;
- Database records;
- Uploaded files.
The user may notice a brief delay while the failure is being detected, but their data should not disappear along with the application VM.
However, it is important to understand that two backend servers make only the application layer fault-tolerant. Full high availability requires separately addressing the reliability of the database, session storage, Object Storage, the Load Balancer, and other critical components.
Securing a Fault-Tolerant Architecture
High availability should not come at the cost of unnecessarily exposing infrastructure to the Internet. In a typical architecture, the Load Balancer remains the public entry point, while the application servers run inside a private network.
Why application servers should not have public IP addresses
If both VMs have their own Floating IP addresses and are directly accessible from the internet, a user could potentially bypass the Load Balancer.
For example:
In the second case, the request no longer passes through:
- Load balancing;
- Health checks;
- Consistent access rules;
- TLS termination on the Load Balancer.
In addition, each additional public endpoint increases the attack surface.
After the initial setup is complete, application servers should ideally retain only private addresses:
Application Server 1 → 192.168.60.10
Application Server 2 → 192.168.60.11
For administrative access, you can use a bastion host, a VPN, the cloud provider’s console, or another secure channel.
Restricting Access with a Security Group
A Security Group should allow only genuinely required traffic.
Backend servers typically require:
| Port | Source | Purpose |
| 80 | Load Balancer or private network | HTTP requests and health checks |
| 22 | administrative address or bastion | SSH |
| 443 | Load Balancer | if the backend accepts HTTPS |
You should not create the following rule unless it is necessary:
TCP 80
Source: 0.0.0.0/0
for each application VM.
If the cloud platform allows you to specify the Load Balancer’s own Security Group as the source, this is preferable to using a broad CIDR range.
You should also avoid exposing internal application services externally, such as:
PostgreSQL :5432
Redis :6379
Access to these services should be granted only to the private network components that genuinely require it.
Which ports must be open for the Load Balancer
On the external side, the Load Balancer receives user traffic.
For HTTP: Internet → Load Balancer :80
For HTTPS: Internet → Load Balancer :443
On the backend side, the application port must be allowed:
Load Balancer → VM 1 :80
Load Balancer → VM 2 :80
The health monitor must also be able to access the selected endpoint: GET /health
If the application uses a separate health port, it must be allowed separately.
For example: Load Balancer → VM :8080
However, this port does not necessarily need to be accessible to users from the internet.
HTTPS and TLS Termination on the Load Balancer
In a production environment, a public application is typically exposed over HTTPS.
One option is to terminate TLS directly on the Load Balancer:
The load balancer stores the certificate, establishes a secure connection with the client, and, after decrypting the traffic, forwards the HTTP request to the backend server.
This approach simplifies certificate management: certificates do not need to be installed separately on each application VM.
If security requirements call for traffic encryption within the private network as well, the following approach can be used:
In this case, certificates or an appropriate PKI configuration will also be required on the backend.
The specific approach depends on the capabilities of the Load Balancer and the project requirements.
Maintaining Two Application Servers
Multiple backends not only allow the application to survive failures, but also make it possible to perform some scheduled maintenance without taking the entire application offline.
Instead of updating two VMs at the same time, the servers can be maintained one at a time.
How to update an application without a full shutdown
Assume both backends are currently running:
Server 1 → ONLINE
Server 2 → ONLINE
First, Server 1 is updated while Server 2 continues to handle user requests.
After the first VM is checked, it is returned to the load balancing pool, and then Server 2 is updated in the same way.
The process looks like this:
1. Server 1 → maintenance
Server 2 → serving traffic
2. Server 1 → ONLINE
Server 2 → maintenance
3. Server 1 → ONLINE
Server 2 → ONLINE
This approach significantly reduces the time during which the service is completely unavailable.
At the same time, the new application version must remain compatible with shared external components, such as the database and the user session format.
Taking VMs out of load balancing one at a time
Before updating a backend, it is advisable to temporarily take it out of rotation for new requests.
Depending on the Load Balancer, you can:
- Disable the member;
- Set its administrative state to Disabled;
- Use a connection draining mechanism, if supported;
- Temporarily remove the server from the pool.
After that, make sure the second backend continues to handle traffic.
You can then update the application on the VM taken out of rotation: sudo systemctl stop myapp
perform the required actions and start the service again: sudo systemctl start myapp
For Nginx, when changing the configuration, a full stop is usually not required; the following is typically sufficient:
sudo nginx -t
sudo systemctl reload nginx
After verification, the server can be returned to the pool.
Checking Health Checks After the Update
Do not return a VM to serving user traffic just because the application process has started.
First, check the endpoint: curl http://127.0.0.1/health
Expected response: OK
Then wait until the Load Balancer reports the backend as: ONLINE
After that, check the public endpoint: curl http://203.0.113.30
and run several consecutive requests:
for i in {1..10}; do
curl -s http://203.0.113.30 | grep -o "Application Server [12]"
done
If responses are once again coming from both VMs, the first stage of the update is complete and you can proceed to the second server.
What to do if both backends become unavailable
A two-VM design can tolerate the failure of one application server, but not the simultaneous failure of both.
If the health monitor marks both members as unavailable:
Server 1 → OFFLINE
Server 2 → OFFLINE
The Load Balancer no longer has a healthy backend to process the request.
The user will start receiving an error or timeout, even though the load balancer’s public address remains reachable.
In this situation, check the following:
- The status of both VMs;
- Application health;
- Health check results;
- Security Group;
- Private network;
- Availability of dependent services;
- Recent configuration changes;
- The status of the shared database or Redis.
If both VMs stop responding at the same time, look for a component they have in common. The cause may not be two independent failures, but, for example, a database outage, a private network issue, or an unsuccessful application update.
For more critical systems, you can also use more than two backends, multiple availability zones, and redundancy for dependent services.
Conclusion
A Load Balancer and multiple application servers eliminate one of the main problems of a simple architecture: the entire application’s dependence on a single virtual machine.
In the architecture described, two identical VMs are in a private network, while external traffic comes through the Load Balancer. Health checks automatically detect an unavailable backend and exclude it from request distribution. Therefore, after one VM is forcibly stopped, the application continues to run through the second server without changing the public address.
However, high availability does not end at the application layer. User sessions should not be stored only in the memory of individual VMs, and persistent data and uploaded files should not be stored only on their local disks. Redis or a database is used for shared state, while Object Storage can be used for files.
As a result, the failure of a single application server becomes a routine situation rather than a reason for the service to stop completely. The same approach can be used to perform rolling backend updates and later scale the application by adding new servers to the Load Balancer.
FAQ
Does each application VM need a public IP address?
No. In a highly available architecture, the Load Balancer typically serves as the public entry point. Application servers can reside only on a private network and receive requests from the load balancer via internal IP addresses.
This reduces the number of publicly accessible components and prevents users from bypassing the Load Balancer directly.
What happens if one VM stops responding?
Health Monitor will detect that the backend is no longer passing the health check, and the Load Balancer will stop sending new requests to it. Traffic will be routed to the remaining healthy VM.
After the server is restored and successfully passes the health checks, the backend can automatically return to the load-balancing pool.
Why is a Load Balancer alone not enough for high availability?
A Load Balancer protects an application from failures of individual backend servers, but other components can also become single points of failure.
You should also account for the reliability of:
- Databases;
- Redis or another session store;
- Object Storage;
- Network infrastructure;
- the Load Balancer service itself.
For critical systems, these components must also have their own redundancy mechanisms.
Can user sessions be stored on application servers?
Storing a session only in the memory of a specific VM is not recommended. The user’s next request may be routed to another backend where that session does not exist.
For shared state, it is better to use Redis, a database, or another storage system accessible to all application servers; at worst, use session replication between VMs.
Sticky sessions can temporarily address part of the problem, but if the assigned VM fails, the local session may still be lost.
Can user-uploaded files be stored on a VM’s local disk?
For temporary files, yes; for persistent user data, it is not recommended.
If a file is stored only on the first VM, the second machine may not be able to access it. In addition, if the VM is corrupted or deleted, that data may be lost.
For user files, it is better to use shared storage, such as Object Storage.
Which load-balancing algorithm should you choose for two identical servers?
For a simple setup, Round Robin is suitable. It distributes requests sequentially across the available backends.
In some scenarios, Least Connections is more useful: the Load Balancer selects the server with the fewest active connections.
The choice depends on the specifics of the application and the nature of the workload.
Why is a separate health endpoint needed?
A /health check lets you determine the state of the application itself, not just whether the virtual machine is running.
A VM can remain in the ACTIVE state even if the web server or application process inside it has already terminated with an error.
The health endpoint should be lightweight and accurately reflect the application’s ability to handle requests.
Will the Load Balancer switch to the second VM immediately after a failure?
Not necessarily. Failover time depends on the health monitor settings: the interval between checks, the timeout, and the allowed number of failed attempts.
Very short intervals allow failures to be detected more quickly, but they can increase the likelihood of false positives. Values that are too high increase the amount of time an unavailable backend may remain in the pool.
Can application servers be updated without fully stopping the application?
Yes. You can temporarily remove one backend from load balancing, update and verify it, and then return it to the pool. The same steps are then performed on the second VM.
As long as at least one healthy server remains available, the Load Balancer can continue serving user requests.
What happens if both VMs fail at the same time?
The load balancer will have no available backends and will be unable to serve the application.
For higher availability requirements, you can use three or more application servers, distribute them across availability zones, and provide separate redundancy for the database and other shared services.
