Building a Better OS X Firewall (or How I solved the NAT problem for VirtualBox)

My Macbook Pro’s hard drive died. Ugh. Since I was forced to rebuild my system I decided to do a clean install and not copy over any preferences from my backups, just the data that I needed (documents, secure shell (SSH) keys, music, etc.). Another decision I made was to forgo VMware Fusion for a more light-weight desktop hypervisor. Fusion has a nice feature set, but I have never known a program to be more of a resource hog, or to be so annoying (always grabbing my input, yanking me into another space) as VMware Fusion for OS X.

So the decision that had to be made was, “Which hypervisor do I use?” It really boiled down to two choices: Parallels Desktop for Mac or VirtualBox. As I’m a fan of open source, and VirtualBox is free compared to Parallel’s 79 United States Dollars (USD) price tag, I chose VirtualBox. VirtualBox is a wonderful hypervisor, it has active community forums and members waiting to assist you on its internet relay chat (IRC) channel. Albeit, there are a few annoying bugs such as it always wanting me to re-authenticate whenever I perform a seemingly innocuous action (clicking on the help menu for example). But, to the credit of the community, one of the VirtualBox developers mentioned that this is a known bug and will be fixed when the final version of Snow Leopard is released (I’m running the latest developer’s build of Snow Leopard). Right now it is just too much of a moving target for them to concern themselves with.

VirtualBox does have one major issue that was almost a non-starter for me — its terrible network address translation (NAT) implementation. The creators of VirtualBox are incredibly helpful and intelligent, but much like the wonderful people behind the Milwaukee Sawzall (which despite several incarnations, still throws waste back into the eye of the user) need to use their creation a bit more in the real world. To understand what I am harking on about, you need to understand how both the VMware and Parallels NAT implementations work. The following image illustrates a high-level view of how NAT is generally implemented by desktop hypervisors:

A Standard NAT Implementation

As you can see, all of the virtual machines (VMs) are connected to an independent NAT process. This implementation has some pros and cons associated with it.

Pros

• VMs are protected – VMs are protected behind the NAT daemon and while they can see the internet, the internet cannot see them. This is a perfect set up for testing new operating systems (OSs) or other software in a beta state.
• VMs are connected – Because all of the VMs belong to the same NATd network segment, all of the VMs can communicate with each other. This is perfect for testing client-to-server functionality between two VMs.

Cons

• NATd is persistent – The disadvantage to this implementation is that the NAT daemon is always running, taking up precious resources.

With the standard NAT implementation the pros definitely outweigh the cons in my opinion. I’d rather have a persistent NAT process hanging around then my VMs not being able to communicate with each other behind a firewall. Unfortunately for us, the creators of VirtualBox do not see it this way. Let’s take a look at VirtualBox’s NAT implementation:

VirtualBox NAT Implementation

As the above image depicts, with VirtualBox each VM’s process hosts its own NAT engine. This method also has its own pros and cons.

Pros

• NATd is not persistent – When there are no VMs around to need a NAT daemon, no NAT daemon is present.
• VMs are protected – VMs are protected behind the NAT daemon and while they can see the internet, the internet cannot see them. This is a perfect set up for testing new operating systems (OSs) or other software in a beta state.

Cons

• VMs are not connected – Because all of the VMs belong to different NATd network segments, none of the VMs can communicate with each other. This is prevents testing client-to-server functionality between two VMs.

There is a thread on VirtualBox’s forums discussing their unique implementation of NAT. My response is currently the last in the thread, and I simply point out that sometimes doing things differently is not always the best solution since people trying out VirtualBox are likely to be used to the way things work in VMware and Parallels.

I will say it again, the absence of VMs being able to communicate with each other, protected by a NAT, just baffles me. The VirtualBox developers do have two solutions, neither of which are things I recommend implementing:

1. Create a Virtual Router – The first solution is to create two or more VMs on a host-only network and then dual home the first VM on a bridged network and have it act as a router for the other VM(s). This is such a bad design that it makes me upset to think about it. First of all, this was the way ESX 3.x worked and people did not like it. We asked VMware for a proper, built-in NAT, and they finally gave us one with vSphere (hell, VMware Server, or GSX, always had a NAT implementation built-in). Not to mention that there is no way that you can tell me having a VM as a dedicated router uses fewer resources than running NATd.
2. Bridge Your VMs – Dumb. Dumb. Dumb. The reason we like NAT is to keep our VMs off of the public internet so we can get away with all sorts of things we would never dream of doing on a system open to the world.

I have been told that NAT really isn’t a security measure, and to that I respond with “For all intents and purposes it is.” Some people insist that MySQL is not a free and open database, but to these people, as well to the ones who contend NAT’s security benefits, I simply say that you are living in a 1% world and the other 99% of us do not have the time nor inclination to get into prickly details of what constitutes free, open, or ultimately 100% secure. NAT provides some measure of added security for the rest of us — deal with it.

Luckily for me, I understand just enough about networking to be dangerous to myself and others. When you place a VirtualBox VM into the networking mode, “Host Only,” you are actually able to give the VM access to outside world through NAT using nothing more than the tools that ship with OS X — all without placing VMs on a bridged network or creating a resource-hogging router VM. Here’s how I did it.

vboxnet*

When you place a VirtualBox into the networking mode, “Host Only,” you are actually attaching the VM to a virtual networking interface VirtualBox creates on the host. The first VirtualBox interface is called vboxnet0, the second vboxnet1, and so on. Since the VirtualBox documentation states that all traffic from a VM in “Host Only” mode passes through the virtual interface it occurred to me that I should be able to route that traffic to and from the outside world using IP tables/chains and NATd.

IPFW and NATD

IPFW is BSD’s (on which OS X is built) tool for implementing the Unix equivalent of IP tables/chains. You can block incoming and outgoing network as well as diverting said traffic. NATD is a popular *Nix daemon that is used to NAT outgoing and incoming packets for internal, private networks. OS X ships with both of these tools, it is just a matter of getting them set up correctly to talk to VirtualBox’s virtual network interfaces.

NATD

The first step to getting this to work was to launch NATD so that it would listen to traffic on my public-facing adapter. Since I am working on a Macbook Pro and my public-facing adapter is an AirPort, the adapter is en1. I start NAT like so:

sudo sysctl -w net.inet.ip.forwarding=1
sudo natd -use_sockets -same_ports -dynamic -interface en1

The first line enables OS X to forward packets from one adapter to another. Normally this is prohibited. The second line starts the NAT daemon on my airport. The flag -dynamic is important since my AirPort’s IP address could change. This flag will cause the NAT daemon to handle a changing IP address.

IPFW

Now that a NAT daemon is up and running we simply need to set up some IP tables about how to divert the appropriate traffic. IP tables are rather daunting and not for the faint of heart, so I am going to simply glaze over my implementation here. The important thing to notice in my IP chains is that I divert all incoming and outgoing traffic to the NAT socket. However, my rules are complicated further by the fact that I don’t just use static rules, I set up a dynamic state table to improve their performance. Additionally, I’m not just redirecting traffic to the VirtualBox interfaces, I’m also setting up a very secure firewall. For example, my rules allow specific outbound services, such as HTTP, IRC, Bittorrent, E-mail, etc. If the service is not on the list then you cannot talk to it once my firewall is enabled. You will be able to edit these on your own accord. The inbound rules in my firewall allows for even fewer services. Only DHCP, SSH, and pings (from certain addresses) are allowed. Again, feel free to edit these to suit your own needs. Here are my rules as they exist right now:

00010   1569    255611 allow ip from any to any via lo*
00011      0         0 allow ip from any to any via vboxnet*
00100      0         0 deny ip from 127.0.0.0/8 to any in
00110      0         0 deny ip from any to 127.0.0.0/8 in
00120      0         0 deny ip from 224.0.0.0/3 to any in
00130      0         0 deny tcp from any to 224.0.0.0/3 in
10000 348166 427961532 divert 8668 ip from any to any in via en1
10010      0         0 check-state
10020    256     38154 deny ip from any to any established in
10030      0         0 deny ip from any to any frag in
20000   1151     64456 skipto 50000 icmp from any to any out keep-state
20010 473220 413906904 skipto 50000 tcp from any to any dst-port 80,8080 out setup keep-state
20011  41930  12064801 skipto 50000 tcp from any to any dst-port 443,8443 out setup keep-state
20020      0         0 skipto 50000 tcp from any to any dst-port 53 out setup keep-state
20020   6996    741940 skipto 50000 udp from any to any dst-port 53 out keep-state
20030  32996  23308966 skipto 50000 tcp from any to any dst-port 110,993 out setup keep-state
20031   1290    806595 skipto 50000 tcp from any to any dst-port 25,465,587 out setup keep-state
20040      0         0 skipto 50000 tcp from any to any dst-port 22 out setup keep-state
20050      0         0 skipto 50000 tcp from any to any dst-port 21 out setup keep-state
20060      0         0 skipto 50000 tcp from any to any dst-port 123 out setup keep-state
20060    156     11856 skipto 50000 udp from any to any dst-port 123 out keep-state
20070      0         0 skipto 50000 tcp from any to any dst-port 113 out setup keep-state
20070      0         0 skipto 50000 udp from any to any dst-port 113 out keep-state
20080      0         0 skipto 50000 tcp from any to any dst-port 194 out setup keep-state
20080      0         0 skipto 50000 udp from any to any dst-port 194 out keep-state
20090      0         0 skipto 50000 tcp from any to any dst-port 137-139,445 out setup keep-state
20090    576     44928 skipto 50000 udp from any to any dst-port 137-139,445 out keep-state
20100      0         0 skipto 50000 tcp from any to any dst-port 1194 out setup keep-state
20100      0         0 skipto 50000 udp from any to any dst-port 1194 out keep-state
20110      0         0 skipto 50000 tcp from any to any dst-port 111,2049 out setup keep-state
20110      0         0 skipto 50000 udp from any to any dst-port 111,2049 out keep-state
20120      0         0 skipto 50000 tcp from any to any dst-port 6881-6999 out setup keep-state
30000      0         0 allow tcp from any to me dst-port 22 in setup keep-state
30010      0         0 allow udp from any to me dst-port 67,68 in keep-state
30020      0         0 allow icmp from 192.168.0.2 to any in keep-state
40000  13796   1511970 deny ip from any to any
50000 222149  24513546 divert 8668 ip from any to any out via en1
65535 663694 524799538 allow ip from any to any

That’s a lot, I know. But once they’re in place you will have an incredibly secure OS X box (you will no longer need to use the built-in, crappy OS X firewall) and a true NAT system for VirtualBox.

AKIN

Everything is not hoorific in hooville however. It turns out that NATD really doesn’t like when you sleep and wake your laptop. It just isn’t designed for it. And wouldn’t it be nice if your firewall started when you started your computer? Or how about if the NAT daemon would figure out if you disabled your network or you switched from using your built-in AirPort to the built-in Ethernet port? Yes, these things would be nice, wouldn’t they? Luckily for you I’ve got a solution in the form of akin.

akin stands for akutz’s ipfw and natd controller (10.5+ only) and is a project on SourceForge at http://sourceforge.net/projects/akin. Simply download the latest tar ball from SourceForge, deflate, and use the included install script to get your firewall and router up and going.

Details

So what exactly does akin do? A lot actually. There are several files included with akin:

• akin (src) – This is an OS X binary I wrote in C using Xcode that acts as a launchd daemon to monitor sleep and wake states and when your active network adapter changes.
• akinctl (src) – This is the heart of akin, a BASH script that can be used to start, stop, and restart IPFW and NATD. It will automatically detect if you have VirtualBox installed and only start NATD if you do. It adjusts the IPFW rules in the case that you do not. akinctl can also print out its status and the name of the active NIC using the status and nic arguments.
• ipfw.rules (src) – These are some of the IPFW rules that akin installs. Not all though — some are dynamically generated depending on whether or not you need NAT and which of you network interfaces is active.
• com.wordpress.akutz.launchd.akin.plist (src) – This is the launchd plist that will start akin on boot and daemonizes it so that it can listen for sleep and wake events as well as detect if your active network interface changes.
• install (src) – A script that installs akin in /opt/akin. This script also registers akin as a launch daemon and starts it immediately.
• uninstall (src) – A script that uninstalls akin from /opt/akin. This script also unregisters akin as a launch daemon and stops it immediately.

Logging

You can use Console.app to monitor the status of akin.

8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	active nic changed from disabled to en1
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	executing '/opt/akin/akinctl start'
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	enabled ip forwarding
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	enabled natd
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	enabled ipfw

Salve

Well, I hope that with this blog and akin I’ve accomplished two things. 1) I hope I’ve showed that it is possible to do NAT correctly with VirtualBox and 2) I hope that I’ve made it easy and secure to do so.

Update 1 (2009/08/20)

I forgot to give credit to the articles which helped me learn how to make a dynamic state table play nice with NATd. In no specific order of usefulness:

• Mac OS X Server 10.2 as a NAT router
• IPFW

Update 2 (2009/08/21)

Matt from the standalone-sysadmin.com makes a great point in the comments. One way to achieve a NATd VM that can see other VMs using VirtualBox is to dual home it with a NAT interface and a private interface. However, after thinking on it some more, this too has its own problems. One con I forgot to mention above about VirtualBox’s NAT implementation is that because their NAT engine is running as a low-privileged user account (likely your account), it has multiple failings:

• A Ping to Nowhere – The ping command operates over the ICMP protocol and thus requires root access. Because VirtualBox’s NAT engine is running as a normal user and not root, you cannot use the ping command in VMs that use VirtualBox’s NAT engine. Using the method I describe above with akin it is possible to use ping because the NAT daemon that I set up is running as root.
• Forwarding Ports – Again, because VirtualBox’s NAT engine is not running as root you cannot forward ports less than 1024 because these are considered privileged ports. Using my implementation you can forward away!