Derek Neely

Here at work we’ve recently been moving off of a Linux/IPTables firewall setup to an OpenBSD/PF firewall. Well, anyone who has worked with firewalls knows the fun time you have with getting FTP to work through them. This includes client and server side. Well, this is my little ‘how-to’ on doing just that with PF and PureFTP.

I originally started with this tutorial from OpenBSD and was a good starting point but I felt it left out a few things.

First off. The tutorial above uses the ftp-proxy server on the box to allow proxying down to the ftp server. However, I didn’t exactly see any benefit of using it and didn’t seem to make anything easier. If I’m completely misunderstanding the usage of it please contact me and help me out. I DO want to do things right.

Here’s the overall layout of what is to happen:

[Client] <—> [PF Firewall] <—> [FTP Server]

Now, a few observations I’ve made while working with the firewall and maybe a bit clearer for others.

Here’s a great explanation of the overall FTP process from Tech Republic. Here’s the overall communication ports for the 2 different modes (as the PF firewall sees it – at least in my experience).

Active FTP:
-Inbound-
— Authentication/Commands —
Incoming:  21 (External interface – FW)
Outgoing: 21 (Internal interface – FW)
Incoming: 21 (FTP Server Interface)

-Outbound-
— Data Transfer —
Outgoing: > 30000 (FTP Server Interface)
Outgoing: > 30000 (External interface – FW)

Passive FTP:
-Inbound-
— Authentication/Commands —
Incoming:  21 (External interface – FW)
Outgoing: 21 (Internal interface – FW)
Incoming: 21 (FTP Server Interface)
— Data Transfer —
Incoming:  49000-51000 (External interface – FW)
Outgoing: 49000-50000 (Internal interface – FW)
Incoming: 49000-50000 (FTP Server Interface)

-Outbound-
Outgoing: 49000-50000 (FTP Server Interface)
Outgoing: 49000-50000 (External interface – FW)

Alright enough chat and on to the code. First we need to configure the ftp server. In the  below examples I’ll cover PureFTP and ProFTPd for just the primary pieces of passive port configuration (I’ll give a nice ProFTP advanced configuration in a post to come). We’ll also assume the default ‘listening’ port 21 of any standard FTP install/configuration. This is also on a Gentoo server so modify accordingly for your given distro.

PureFTPd:

MISC_OTHER=”[various options/flags] -p 49000:51000 -P [Public IP]“

ProFTPd:

PassivePorts 49000 51000
MasqueradeAddress [Public IP]

PF Firewall Rules: (Just the FTP Rules)

# Interface Definitions:
ftp_ext=”[Public IP]”
ftp_int=”[Private IP]”

# NAT Rules
nat on $ext_if from { $ftp_int } to any -> $ftp_ext

# Redirect Rules
rdr pass on $ext_if inet proto tcp from any to $ftp_ext port 21 -> $ftp_int port 21
rdr pass on $ext_if inet proto tcp from any to $ftp_ext port 49000:51000 -> $ftp_int

# Firewall Rules
pass out quick on $ext_if inet proto tcp from $ftp_ext to any port > 1024 keep state

pass out quick log on $int_if inet proto tcp from any to $ftp_int port 21 keep state tag FTP label “ftp”
pass out quick log on $int_if inet proto tcp from any to $ftp_int port 49000:51000 keep state tag FTP_PASV label “ftp-passive”

That should be all you need for the PF rules and configuration on the FTP servers to get everything working and passing through. Feel free to contact me should you have any additional fixes or have a good explanation of how the ftp-proxy works and benefits of. I’ve just gotta find the time to do some experimenting.

At my job we are moving to Nginx for the load balancing of our sites. Nginx is a very powerful load balancing/proxy server tool. It allows weighting, ssl acceleration, among other functionality while remaining light weight and easy to configure.

In preperation for a large web services launch, I began to analyze some logs and keep an eye on the system. I noticed one of the sites that we’ve already deployed was hammering our error messages in /var/log/nginx/error.log reading:

2009/06/23 12:38:22 [crit] 808#0: *724154 open() “/var/nginx/tmp/proxy_temp/4/83/0000002834″ failed (13: Permission denied) while reading upstream, client: XXX.XXX.XXX.XXX, server: xxx.host.com, request: “GET /dir/page.php”, upstream: “http://backendserverip/dir/page.php”, host: “host.com”, referrer: “http://referrer.com/apage.php”

Upon reviewing the site I noticed some (not all) of the pages were only partially loading. The issue is exactly what the log says. Permission denied = Permission issue.

Check your /etc/nginx/nginx.conf (OpenBSD) file for the user nginx processes will run as:

user  nobody;

Or, do:

# ps aux | grep “nginx: worker process” | awk ‘{print $1}’
nobody

In both cases you see that I’m running the nginx worker process as user nobody. Now we need to check our permissions on: /var/nginx/tmp/proxy_temp

# ls -l /var/nginx/tmp/ | grep proxy_temp
drwxrwx—  12 nobody  _nginx  512 Jun 23 13:10 proxy_temp

Looks good. The directory is owned by nobody and is writeable by both nobody and the group _nginx. What could the issue be? Lets move up a level and check the permissions.

# ls -l /var/nginx | grep tmp
drwx——  5 _nginx  _nginx  512 May  7 11:54 tmp

Ah ha! The parent directory is owned my _nginx:_nginx and is only writeable for that user. Our user ‘nobody’ therefore does not have the permissions to write in here. So, we can do a few things. Either make the entire directory writeable by everyone or change the ownership.

# chmod 777 /var/nginx/tmp

or

# chown nobody:_nginx /var/nginx/tmp

This should cure your permissions issues and all pages should load completely (at least mine do!)

Part one of the OpenBSD Network Appliance is done. I’ve got all the hardware put together, everything is posting. RAM was seen. My biggest worry. A buddy of mine at work gave me 4 DIMMS of PC133 512MB. I was a little worried it wasn’t going to work. I thought the mo-bo used only PC100. Good news for me though! 2GB of RAM for this bad boy will be plenty!

Hardware Specs:

ASUS P3V4X Motherboard
Pentium Celeron 533MHz
2GB PC133 RAM (4 x 512MB)
6 NICs (3 x 100Mb 3COM – 3 x 100Mb Intel)
2 x 30GB Hard drives (RAID 1 intended)

Check out the gallery! Projects Home Page