Nginx Load Balancing Setup: upstream, Methods and Failover

This article is a localized edition of the Japanese original. Read the Japanese original

Nginx can act as a reverse proxy and distribute incoming requests across several backend servers. The configuration revolves around two pieces: the upstream block that defines the list of backends, and proxy_pass, which forwards requests to it.

This article covers the basic configuration, choosing a load-balancing method, what happens when a backend goes down, and how to verify the setup. The configuration examples were tested with nginx -t and curl on Nginx 1.30 (official image) as of November 2026. The same syntax works with the standard Ubuntu 24.04 package (1.24).

Example setup

This article uses the following setup as an example:

• Load balancer (Nginx): accepts external traffic on ports 80/443
• Backends: the same app running on port 8080 on 192.168.1.10, 192.168.1.11 and 192.168.1.12
• Spare backend: 192.168.1.20 (used only when all the others are down)

Every backend must return the same content. If you split WordPress or a similar app across several servers, decide first how to share uploaded files (NFS or object storage), how to share the database, and how to handle login sessions. Otherwise each request can land on a different server and pages or behavior will be inconsistent.

Basic configuration

Install Nginx if it isn't installed yet (Ubuntu/Debian example).

sudo apt update
sudo apt install nginx

Create a configuration file in /etc/nginx/conf.d/. Files in this directory are included inside the http block of nginx.conf. upstream must sit directly under the http block; it can't be written inside server or location.

sudo nano /etc/nginx/conf.d/app.conf

upstream app_backend {
    zone app_backend 64k;

    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://app_backend;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

The app_backend in proxy_pass http://app_backend; is the name given to the upstream. If you don't specify a method, requests are distributed in turn using weighted round robin (there is no directive such as roundrobin).

zone is shared memory that lets worker processes share the load-balancing state and failure records. Without it, each worker keeps its own counts, so on machines with many workers the distribution ratio and failover may not appear to follow your configuration.

proxy_set_header passes the original host name and the client's IP address to the backend. Without it, the backend's logs and app record every request as coming from the load balancer's IP address.

On Ubuntu, /etc/nginx/sites-enabled/default listens on port 80 as the default site. Disable it if you use the same server_name or don't need the default site.

sudo rm /etc/nginx/sites-enabled/default

Check the syntax, then apply. reload swaps in the new configuration without dropping connections in progress.

sudo nginx -t
sudo systemctl reload nginx

Choosing a load-balancing method

Specify the method with a single line inside upstream.

upstream app_backend {
    zone app_backend 64k;
    least_conn;

    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
}

ip_hash tends to become unbalanced where many users share one IP address (corporate networks, mobile carriers). And if another proxy such as a CDN sits in front of the load balancer, every user appears to have the same IP and all traffic lands on one server. In that case, consider using a cookie as the key, as in hash $cookie_session_name;.

Note that least_time, which balances by response time, and sticky, which pins sessions with a cookie, are only available in the commercial NGINX Plus. In the free version they cause an unknown directive error.

Per-server parameters

You can add parameters after each server line to change how it is balanced and how failures are handled.

An example that mixes servers with different capacity and adds a spare:

upstream app_backend {
    zone app_backend 64k;

    server 192.168.1.10:8080 weight=3 max_fails=3 fail_timeout=30s;
    server 192.168.1.11:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.12:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.20:8080 backup;
}

backup can't be used together with the hash, ip_hash or random methods. To take one server out for maintenance, add down and reload.

What happens when a backend goes down

The free version of Nginx has no "active health checks" (health_check) that probe backends periodically (that is an NGINX Plus feature). Instead, it counts failures on real requests, such as connection failures and timeouts, and removes a server that reaches max_fails for the duration of fail_timeout.

proxy_next_upstream decides when a failed request is retried on another server. The default is error timeout (can't connect, timed out). Add more conditions if you also want to retry when a backend returns 502 or 503.

location / {
    proxy_pass http://app_backend;
    proxy_next_upstream error timeout http_502 http_503;
    proxy_connect_timeout 5s;
    # proxy_set_header lines are the same as in the earlier example
}

A few caveats:

• Requests whose result changes when repeated, such as POST, are not retried by default once they have been sent to a backend, even if they fail. Adding non_idempotent makes Nginx retry them, but orders or posts may then be processed twice.
• Limiting retries with proxy_next_upstream_tries can keep requests from ever reaching the spare server. In our test, with two regular servers down and proxy_next_upstream_tries 2; set, requests failed with 502 after trying those two, and the backup server was never used.
• The default proxy_connect_timeout is 60 seconds. Setting it shorter speeds up failover so users aren't left waiting when a stopped server doesn't respond.

Reusing connections to backends (keepalive)

By default, Nginx opens and closes a connection to the backend for every request. Setting keepalive lets it keep connections open and reuse them, which reduces load under heavy traffic. Use it together with HTTP/1.1 and an empty Connection header.

upstream app_backend {
    zone app_backend 64k;
    server 192.168.1.10:8080;
    server 192.168.1.11:8080;
    server 192.168.1.12:8080;
    keepalive 16;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

keepalive 16 is the maximum number of idle connections each worker process keeps open. It is not a limit on simultaneous connections.

Verifying the setup

To see which backend answered, expose the destination in a response header while testing only.

location / {
    proxy_pass http://app_backend;
    add_header X-Upstream $upstream_addr always;
}

sudo nginx -t && sudo systemctl reload nginx
for i in $(seq 1 10); do
  curl -s -o /dev/null -D - http://example.com/ | grep -i x-upstream
done

Sample output with weights of 3:1:1 and one server down. The request that failed on the stopped server was retried on the next one (the line listing two servers separated by a comma).

X-Upstream: 192.168.1.10:8080
X-Upstream: 192.168.1.11:8080
X-Upstream: 192.168.1.10:8080
X-Upstream: 192.168.1.12:8080, 192.168.1.10:8080
X-Upstream: 192.168.1.10:8080

Failures are recorded in the error log. upstream server temporarily disabled means the server reached max_fails and was temporarily removed.

sudo tail -f /var/log/nginx/error.log

Remove the add_header X-Upstream line once you are done, because it exposes internal IP addresses to the outside.

Operational notes

• Client IP handling: Restore the original client IP on the backend with set_real_ip_from and real_ip_header X-Forwarded-For from ngx_http_realip_module in Nginx, or with mod_remoteip in Apache. Trust only the load balancer's IP address.
• HTTPS: The usual approach is to put the certificate on the load balancer and forward plain HTTP to the backends over the internal network. Pass X-Forwarded-Proto so the app can tell whether the request was HTTPS.
• Redundancy for the load balancer itself: With a single Nginx instance, everything stops when it stops. If needed, consider two load balancers that fail over with a virtual IP, or a cloud load balancer.

Related configuration is also covered in these articles:

• How to set up an Nginx reverse proxy
• Nginx load-balancing algorithms
• Configuring Nginx upstream keepalive

Summary

• Name an upstream, list the backends in it, and forward with proxy_pass http://name;. Write upstream directly under the http block
• Besides the default weighted round robin, choose from least_conn, ip_hash, hash and random. least_time and sticky are NGINX Plus only
• Share state between workers with zone, and define failure handling with max_fails/fail_timeout/backup
• The free version fails over based on failures of real requests; proxy_next_upstream sets the retry conditions
• Use keepalive together with proxy_http_version 1.1 and Connection ""
• After changes, check with nginx -t, then reload, and confirm behavior with the destination header and the error log

Questions about WordPress? The publisher of this site is based in Japan and replies in Japanese. Contact (Japanese)