Derek Neely

Nginx (Engine-X) is a very light-weight powerful web proxy server and load balancer. This is a quick how-to on configuring a weighted load balancer using Nginx. I’ll skip past the installation on your system. Mine is currently running on an OpenBSD firewall using Packet Filter, and is how I will be covering the configuration. Nginx is available and work on both Unix and Linux.

Configuring Nginx to start on system boot:

# vi /etc/rc.local

# start nginx
if [ -x /usr/local/sbin/nginx ]; then
   echo -n ' nginx'; /usr/local/sbin/nginx
fi

Load balance configuration (one public ip -> 3 private servers):

# vi /etc/nginx/nginx.conf

user  nobody;
worker_processes  2;
worker_rlimit_nofile 10240;

error_log  /var/log/nginx/error.log;

events {
   worker_connections  8192;
}

http {

   upstream  site {
        server 192.168.1.100:80 max_fails=2 fail_timeout=15 weight=3;
        server 192.168.1.101:80 max_fails=2 fail_timeout=15 weight=3;
        server 192.168.1.102:80 max_fails=2 fail_timeout=15 weight=1;
   }

   upstream  site_ssl {
        server 192.168.1.100:443 max_fails=2 fail_timeout=15 weight=3;
        server 192.168.1.101:443 max_fails=2 fail_timeout=15 weight=3;
        server 192.168.1.102:443 max_fails=2 fail_timeout=15 weight=1;
   }

   server {
        listen  AAA.BBB.CCC.DDD:80;

        location / {
             access_log off;
             proxy_connect_timeout 15;
             proxy_set_header Host $host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_pass http://site;
        }
   }

   server {
        listen  AAA.BBB.CCC.DDD:443;

        ssl on;
        ssl_certificate /etc/ssl/ssl.cert/site.com.crt;
        ssl_certificate_key /etc/ssl/ssl.key/site.com.key;

        server_name site.com;

        location / {
             access_log off;
             proxy_connect_timeout 15;
             proxy_set_header Host $host;
             proxy_set_header X-Real-IP $remote_addr;
             proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_set_header X_FORWARDED_PROTO https;
             proxy_pass http://site_ssl;
        }
   }
}

Should you have a chain file you need to add it to the end of the certificate file as such:

# cat /path/to/chain.file >> /etc/ssl/ssl.cert/site.com.crt

Packet Filter Firewall Rules:

# vi /etc/pf.conf

site_ext="AAA.BBB.CCC.DDD"
site_int1="192.168.1.100"
site_int2="192.168.1.101"
site_int3="192.168.1.102"

pass in quick log on $ext_if inet proto tcp from any to $site_ext port 80 keep state
pass in quick log on $ext_if inet proto tcp from any to $corp_ext port 443 keep state
pass out quick log on $int_if inet proto tcp from any to $site_int1 port 80 keep state
pass out quick log on $int_if inet proto tcp from any to $site_int1 port 443 keep state
pass out quick log on $int_if inet proto tcp from any to $site_int2 port 80 keep state
pass out quick log on $int_if inet proto tcp from any to $site_int2 port 443 keep state
pass out quick log on $int_if inet proto tcp from any to $site_int3 port 80 keep state
pass out quick log on $int_if inet proto tcp from any to $site_int3 port 443 keep state

Helpful Nginx server management commands:

Test Nginx Config file:

# nginx -t -c /etc/nginx/nginx.conf

Restart Nginx server:

# kill -HUP `cat /var/run/nginx.pid`

Stop Nginx server:

# kill -QUIT `cat /var/run/nginx.pid`

If your like me and want to keep tabs on the load balancing (as in ipvsadm for Linux ldirectord) you can add a label to the end of each of your firewall rules:

label “[some unique label for this rule]” (i.e. label “site-ext”)

Then using pftop -v label you will see:

RULE LABEL           PKTS     BYTES     STATES MAX  ACTION   DIR
....
16   site-ext        867991   630495K   11833       Pass     In  ...
17   site-ext-ssl    29427    13315230  847         Pass     In  ...
18   site-int-1      13761450 6842062   813483      Pass     Out ...
19   site-int-1-ssl  234678   12345     12344       Pass     Out ...
....

Part one of the OpenBSD Network Appliance is done. I’ve got all the hardware put together, everything is posting. RAM was seen. My biggest worry. A buddy of mine at work gave me 4 DIMMS of PC133 512MB. I was a little worried it wasn’t going to work. I thought the mo-bo used only PC100. Good news for me though! 2GB of RAM for this bad boy will be plenty!

Hardware Specs:

ASUS P3V4X Motherboard
Pentium Celeron 533MHz
2GB PC133 RAM (4 x 512MB)
6 NICs (3 x 100Mb 3COM – 3 x 100Mb Intel)
2 x 30GB Hard drives (RAID 1 intended)

Check out the gallery! Projects Home Page