Building a Better OS X Firewall (or How I solved the NAT problem for VirtualBox)

20Aug09

My Macbook Pro’s hard drive died. Ugh. Since I was forced to rebuild my system I decided to do a clean install and not copy over any preferences from my backups, just the data that I needed (documents, secure shell (SSH) keys, music, etc.). Another decision I made was to forgo VMware Fusion for a more light-weight desktop hypervisor. Fusion has a nice feature set, but I have never known a program to be more of a resource hog, or to be so annoying (always grabbing my input, yanking me into another space) as VMware Fusion for OS X.

So the decision that had to be made was, “Which hypervisor do I use?” It really boiled down to two choices: Parallels Desktop for Mac or VirtualBox. As I’m a fan of open source, and VirtualBox is free compared to Parallel’s 79 United States Dollars (USD) price tag, I chose VirtualBox. VirtualBox is a wonderful hypervisor, it has active community forums and members waiting to assist you on its internet relay chat (IRC) channel. Albeit, there are a few annoying bugs such as it always wanting me to re-authenticate whenever I perform a seemingly innocuous action (clicking on the help menu for example). But, to the credit of the community, one of the VirtualBox developers mentioned that this is a known bug and will be fixed when the final version of Snow Leopard is released (I’m running the latest developer’s build of Snow Leopard). Right now it is just too much of a moving target for them to concern themselves with.

VirtualBox does have one major issue that was almost a non-starter for me — its terrible network address translation (NAT) implementation. The creators of VirtualBox are incredibly helpful and intelligent, but much like the wonderful people behind the Milwaukee Sawzall (which despite several incarnations, still throws waste back into the eye of the user) need to use their creation a bit more in the real world. To understand what I am harking on about, you need to understand how both the VMware and Parallels NAT implementations work. The following image illustrates a high-level view of how NAT is generally implemented by desktop hypervisors:

A Standard NAT Implementation

A Standard NAT Implementation

As you can see, all of the virtual machines (VMs) are connected to an independent NAT process. This implementation has some pros and cons associated with it.

Pros

  • VMs are protected – VMs are protected behind the NAT daemon and while they can see the internet, the internet cannot see them. This is a perfect set up for testing new operating systems (OSs) or other software in a beta state.
  • VMs are connected – Because all of the VMs belong to the same NATd network segment, all of the VMs can communicate with each other. This is perfect for testing client-to-server functionality between two VMs.

Cons

  • NATd is persistent – The disadvantage to this implementation is that the NAT daemon is always running, taking up precious resources.

With the standard NAT implementation the pros definitely outweigh the cons in my opinion. I’d rather have a persistent NAT process hanging around then my VMs not being able to communicate with each other behind a firewall. Unfortunately for us, the creators of VirtualBox do not see it this way. Let’s take a look at VirtualBox’s NAT implementation:

VirtualBox NAT Implementation

VirtualBox NAT Implementation

As the above image depicts, with VirtualBox each VM’s process hosts its own NAT engine. This method also has its own pros and cons.

Pros

  • NATd is not persistent – When there are no VMs around to need a NAT daemon, no NAT daemon is present.
  • VMs are protected – VMs are protected behind the NAT daemon and while they can see the internet, the internet cannot see them. This is a perfect set up for testing new operating systems (OSs) or other software in a beta state.

Cons

  • VMs are not connected – Because all of the VMs belong to different NATd network segments, none of the VMs can communicate with each other. This is prevents testing client-to-server functionality between two VMs.

There is a thread on VirtualBox’s forums discussing their unique implementation of NAT. My response is currently the last in the thread, and I simply point out that sometimes doing things differently is not always the best solution since people trying out VirtualBox are likely to be used to the way things work in VMware and Parallels.

I will say it again, the absence of VMs being able to communicate with each other, protected by a NAT, just baffles me. The VirtualBox developers do have two solutions, neither of which are things I recommend implementing:

  1. Create a Virtual Router – The first solution is to create two or more VMs on a host-only network and then dual home the first VM on a bridged network and have it act as a router for the other VM(s). This is such a bad design that it makes me upset to think about it. First of all, this was the way ESX 3.x worked and people did not like it. We asked VMware for a proper, built-in NAT, and they finally gave us one with vSphere (hell, VMware Server, or GSX, always had a NAT implementation built-in). Not to mention that there is no way that you can tell me having a VM as a dedicated router uses fewer resources than running NATd.
  2. Bridge Your VMs – Dumb. Dumb. Dumb. The reason we like NAT is to keep our VMs off of the public internet so we can get away with all sorts of things we would never dream of doing on a system open to the world.

I have been told that NAT really isn’t a security measure, and to that I respond with “For all intents and purposes it is.” Some people insist that MySQL is not a free and open database, but to these people, as well to the ones who contend NAT’s security benefits, I simply say that you are living in a 1% world and the other 99% of us do not have the time nor inclination to get into prickly details of what constitutes free, open, or ultimately 100% secure. NAT provides some measure of added security for the rest of us — deal with it.

Luckily for me, I understand just enough about networking to be dangerous to myself and others. When you place a VirtualBox VM into the networking mode, “Host Only,” you are actually able to give the VM access to outside world through NAT using nothing more than the tools that ship with OS X — all without placing VMs on a bridged network or creating a resource-hogging router VM. Here’s how I did it.

vboxnet*

When you place a VirtualBox into the networking mode, “Host Only,” you are actually attaching the VM to a virtual networking interface VirtualBox creates on the host. The first VirtualBox interface is called vboxnet0, the second vboxnet1, and so on. Since the VirtualBox documentation states that all traffic from a VM in “Host Only” mode passes through the virtual interface it occurred to me that I should be able to route that traffic to and from the outside world using IP tables/chains and NATd.

IPFW and NATD

IPFW is BSD’s (on which OS X is built) tool for implementing the Unix equivalent of IP tables/chains. You can block incoming and outgoing network as well as diverting said traffic. NATD is a popular *Nix daemon that is used to NAT outgoing and incoming packets for internal, private networks. OS X ships with both of these tools, it is just a matter of getting them set up correctly to talk to VirtualBox’s virtual network interfaces.

NATD

The first step to getting this to work was to launch NATD so that it would listen to traffic on my public-facing adapter. Since I am working on a Macbook Pro and my public-facing adapter is an AirPort, the adapter is en1. I start NAT like so:

sudo sysctl -w net.inet.ip.forwarding=1
sudo natd -use_sockets -same_ports -dynamic -interface en1

The first line enables OS X to forward packets from one adapter to another. Normally this is prohibited. The second line starts the NAT daemon on my airport. The flag -dynamic is important since my AirPort’s IP address could change. This flag will cause the NAT daemon to handle a changing IP address.

IPFW

Now that a NAT daemon is up and running we simply need to set up some IP tables about how to divert the appropriate traffic. IP tables are rather daunting and not for the faint of heart, so I am going to simply glaze over my implementation here. The important thing to notice in my IP chains is that I divert all incoming and outgoing traffic to the NAT socket. However, my rules are complicated further by the fact that I don’t just use static rules, I set up a dynamic state table to improve their performance. Additionally, I’m not just redirecting traffic to the VirtualBox interfaces, I’m also setting up a very secure firewall. For example, my rules allow specific outbound services, such as HTTP, IRC, Bittorrent, E-mail, etc. If the service is not on the list then you cannot talk to it once my firewall is enabled. You will be able to edit these on your own accord. The inbound rules in my firewall allows for even fewer services. Only DHCP, SSH, and pings (from certain addresses) are allowed. Again, feel free to edit these to suit your own needs. Here are my rules as they exist right now:

00010   1569    255611 allow ip from any to any via lo*
00011      0         0 allow ip from any to any via vboxnet*
00100      0         0 deny ip from 127.0.0.0/8 to any in
00110      0         0 deny ip from any to 127.0.0.0/8 in
00120      0         0 deny ip from 224.0.0.0/3 to any in
00130      0         0 deny tcp from any to 224.0.0.0/3 in
10000 348166 427961532 divert 8668 ip from any to any in via en1
10010      0         0 check-state
10020    256     38154 deny ip from any to any established in
10030      0         0 deny ip from any to any frag in
20000   1151     64456 skipto 50000 icmp from any to any out keep-state
20010 473220 413906904 skipto 50000 tcp from any to any dst-port 80,8080 out setup keep-state
20011  41930  12064801 skipto 50000 tcp from any to any dst-port 443,8443 out setup keep-state
20020      0         0 skipto 50000 tcp from any to any dst-port 53 out setup keep-state
20020   6996    741940 skipto 50000 udp from any to any dst-port 53 out keep-state
20030  32996  23308966 skipto 50000 tcp from any to any dst-port 110,993 out setup keep-state
20031   1290    806595 skipto 50000 tcp from any to any dst-port 25,465,587 out setup keep-state
20040      0         0 skipto 50000 tcp from any to any dst-port 22 out setup keep-state
20050      0         0 skipto 50000 tcp from any to any dst-port 21 out setup keep-state
20060      0         0 skipto 50000 tcp from any to any dst-port 123 out setup keep-state
20060    156     11856 skipto 50000 udp from any to any dst-port 123 out keep-state
20070      0         0 skipto 50000 tcp from any to any dst-port 113 out setup keep-state
20070      0         0 skipto 50000 udp from any to any dst-port 113 out keep-state
20080      0         0 skipto 50000 tcp from any to any dst-port 194 out setup keep-state
20080      0         0 skipto 50000 udp from any to any dst-port 194 out keep-state
20090      0         0 skipto 50000 tcp from any to any dst-port 137-139,445 out setup keep-state
20090    576     44928 skipto 50000 udp from any to any dst-port 137-139,445 out keep-state
20100      0         0 skipto 50000 tcp from any to any dst-port 1194 out setup keep-state
20100      0         0 skipto 50000 udp from any to any dst-port 1194 out keep-state
20110      0         0 skipto 50000 tcp from any to any dst-port 111,2049 out setup keep-state
20110      0         0 skipto 50000 udp from any to any dst-port 111,2049 out keep-state
20120      0         0 skipto 50000 tcp from any to any dst-port 6881-6999 out setup keep-state
30000      0         0 allow tcp from any to me dst-port 22 in setup keep-state
30010      0         0 allow udp from any to me dst-port 67,68 in keep-state
30020      0         0 allow icmp from 192.168.0.2 to any in keep-state
40000  13796   1511970 deny ip from any to any
50000 222149  24513546 divert 8668 ip from any to any out via en1
65535 663694 524799538 allow ip from any to any

That’s a lot, I know. But once they’re in place you will have an incredibly secure OS X box (you will no longer need to use the built-in, crappy OS X firewall) and a true NAT system for VirtualBox.

AKIN

Everything is not hoorific in hooville however. It turns out that NATD really doesn’t like when you sleep and wake your laptop. It just isn’t designed for it. And wouldn’t it be nice if your firewall started when you started your computer? Or how about if the NAT daemon would figure out if you disabled your network or you switched from using your built-in AirPort to the built-in Ethernet port? Yes, these things would be nice, wouldn’t they? Luckily for you I’ve got a solution in the form of akin.

akin stands for akutz’s ipfw and natd controller (10.5+ only) and is a project on SourceForge at http://sourceforge.net/projects/akin. Simply download the latest tar ball from SourceForge, deflate, and use the included install script to get your firewall and router up and going.

Details

So what exactly does akin do? A lot actually. There are several files included with akin:

  • akin (src) – This is an OS X binary I wrote in C using Xcode that acts as a launchd daemon to monitor sleep and wake states and when your active network adapter changes.
  • akinctl (src) – This is the heart of akin, a BASH script that can be used to start, stop, and restart IPFW and NATD. It will automatically detect if you have VirtualBox installed and only start NATD if you do. It adjusts the IPFW rules in the case that you do not. akinctl can also print out its status and the name of the active NIC using the status and nic arguments.
  • ipfw.rules (src) – These are some of the IPFW rules that akin installs. Not all though — some are dynamically generated depending on whether or not you need NAT and which of you network interfaces is active.
  • com.wordpress.akutz.launchd.akin.plist (src) – This is the launchd plist that will start akin on boot and daemonizes it so that it can listen for sleep and wake events as well as detect if your active network interface changes.
  • install (src) – A script that installs akin in /opt/akin. This script also registers akin as a launch daemon and starts it immediately.
  • uninstall (src) – A script that uninstalls akin from /opt/akin. This script also unregisters akin as a launch daemon and stops it immediately.

Logging

You can use Console.app to monitor the status of akin.

8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	active nic changed from disabled to en1
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	executing '/opt/akin/akinctl start'
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	enabled ip forwarding
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	enabled natd
8/20/09 7:52:00 AM	com.wordpress.akutz.launchd.akin[53]	enabled ipfw

Salve

Well, I hope that with this blog and akin I’ve accomplished two things. 1) I hope I’ve showed that it is possible to do NAT correctly with VirtualBox and 2) I hope that I’ve made it easy and secure to do so.

Update 1 (2009/08/20)

I forgot to give credit to the articles which helped me learn how to make a dynamic state table play nice with NATd. In no specific order of usefulness:

Update 2 (2009/08/21)

Matt from the standalone-sysadmin.com makes a great point in the comments. One way to achieve a NATd VM that can see other VMs using VirtualBox is to dual home it with a NAT interface and a private interface. However, after thinking on it some more, this too has its own problems. One con I forgot to mention above about VirtualBox’s NAT implementation is that because their NAT engine is running as a low-privileged user account (likely your account), it has multiple failings:

  • A Ping to Nowhere – The ping command operates over the ICMP protocol and thus requires root access. Because VirtualBox’s NAT engine is running as a normal user and not root, you cannot use the ping command in VMs that use VirtualBox’s NAT engine. Using the method I describe above with akin it is possible to use ping because the NAT daemon that I set up is running as root.
  • Forwarding Ports – Again, because VirtualBox’s NAT engine is not running as root you cannot forward ports less than 1024 because these are considered privileged ports. Using my implementation you can forward away!
About these ads


24 Responses to “Building a Better OS X Firewall (or How I solved the NAT problem for VirtualBox)”

  1. 1 theooze

    Maybe the Virtual Box team will read this and build it into the tool. I totally agree w/ your assessment. One thing I didn’t see was how to handle the proxydns requests. VMware handles that as well in the NAT daemon or some other daemon.

  2. 2 akutz

    I don’t currently handle DNS, although the solution I’m probably going to use myself is to use MacPorts to install bind9 and dhcpd and provide leases to my VMs and point to myself as a DNS server. I’m open to suggestions however. The problem is that I cannot use VirtualBox’s DHCP server as I cannot configure it to provide a DNS suggestion to the clients it’s handing out leases to.

  3. To get the hosts to communicate with each other, configure a second interface as type …uhh…*starts virtualbox*…”Internal Network” . Essentially, it creates a private LAN between the boxes.

    If you need multiple private LANs (for various network segregation testing), just name them different things in the interface.

    • 4 akutz

      Matt,

      Good idea. You are suggesting that each VM be dual homed on its built-in NAT interface and on an interface on an internal network that only the VMs can see. This is not a bad idea at all. However, it still requires a separate NAT engine per VM, and that seems overkill to me given that I’ve demonstrated how to reproduce a more standard architecture with relative ease. Still, your idea is very clean and efficient and makes use of technology built into VirtualBox.

    • 5 akutz

      Matt,

      I thought about your idea further and realized it still leaves two issues plaguing VirtualBox’s NAT implementation. I’ve updated the blog to talk about these issues. Thanks again!

  4. 6 theooze

    I still don’t like the idea of dual-homing the guests. Typically I’d use virtualization to simulate a production environment, where my systems are not dual-homed. It adds more complexity. There’s really no reason to use multi-homed anything anymore… If I was going to do anything I’d setup 1 VM to be a router and dual-home that only, run dns, dhcp, and whatever else on that guest.

    I think the proxy dns is still key, I don’t think you want to re-config bind everytime your host moves networks. Ideally whatever you run to service the guest would pickup the dns settings from the host automagically.

    Ideally we’d have a real DNS and DHCP server in Vbox/VMware that can handle static entries and more modern options (eg pxe boot options don’t seem to work well in VMware’s dhcp, and neither has a way to set static entries in their DNS, for say putting in dns for your guest systems)

    • 7 akutz

      Not to be cocky, but akin already detects when your active NIC changes and adjusts the NAT appropriately. Don’t you think I was going to do something similar for bind? : ) Essentially akin would rewrite the BIND proxy setting to point to the DNS server that the host (in this case, my laptop) points to. And a local DHCP server would hand out leases and point the clients to the host as their DNS server. All of this would facilitate full network simulation, such as PXE boot, etc.

  5. 8 Chris

    Nice article! .. I recently switch to OS X from Ubuntu Linux, and have been at a loss trying to configure a similar configuration for Virtualbox. This is bringing me not just one step closer, but quite a few steps. I’ve always used VirtualBox with host only interfaces especially for setting up development environments.

    What I’d like to mention is, that I had very good results with DNSMASQ on Linux for providing DNS and DHCP services to the VM’s. It’s very lightweight and quite easy to get going, with the added benefit of doing DNS caching too!. I’ve installed DNSMASQ from ports for OS X and it seems to work quite well (After I took a crash course in how .plist files and launchd+launchctl worked). Only I cant seem to get it to dish out DHCP addy’s to my VM’s at the moment. Suspecting that it might be something to do with natd or iptables …

    Perhaps you could add DNSMASQ to this sort of config as it is quite capable, without the complexity of configuring BIND .. Perhaps the “akin” tools can be extended to support DNSMASQ too …

    Other than that, great article!

  6. 9 akutz

    Chris,

    I’ve compiled dnsmasq from source (I could have used macports, but building from source allows me to redistribute the dnsmasq more easily with my own software) and have it almost working. The DNS works wonderfully, but the DHCP leases are not getting accepted by the clients. I am in an endless DHCPDISCOVER/DHCPOFFER loop and do not know why.

    • 10 Chris

      Hey akutz,

      That’s exactly the same issue I’m facing now as well, not sure where’s it’s hapeening though, cause the DHCP part used to work perfectly on Ubuntu Linux …. (Was even able to send special DHCP options for SIP devices … ) … I’m fiddling with it at the moment, but don’t have too much time as I’m under some project pressure …. If I come across anything useful, I’ll post back here on your blog… In the meantime, being new to OS X, what’s the best way to do network diagnostics on OS X a-la tcpdump on linux ??

      Good luck!

      • 11 akutz

        Well, you can always use tcpdump :) Or you can grab wireshark. I usually use tcpdump unless I need to send a compatible log to someone.

      • 12 Chris

        Hehe ….. doh! … didn’t even bother to look and see if tcpdump was available in OS X …

      • 13 Chris

        Hey akutz,

        Have you had any more development in regards to the dnsmasq dhcp issue… Did the Snow Leopard upgrade over the weekend on my MBP, and seem to be fighting some other issues with MacPorts and a change in the way DHCP works … :( ..

        Regards

      • 14 Bjorn

        I’m using a similar setup as described here – namely, a Mac OS X host machine and a virtual machine with a Host-only adapter. The two machines can communicate fine, and the virtualbox0 device is created fine on the host machine – but I can’t attach to it with tcpdump.

        I’ve tried this on a host machine running linux and there it works fine. So now I’m wondering whether my setup is simply incorrect or whether this might be a “VirtualBox on Mac host” related issue. Can you attach to the vboxnet0 virtual device with tcpdump on your host machine?

        $ ifconfig vboxnet0
        vboxnet0: flags=8843 mtu 1500
        inet 192.168.56.1 netmask 0xffffff00 broadcast 192.168.56.255
        ether 00:76:62:00:00:01

        $ sudo tcpdump -n -i vboxnet0
        tcpdump: BIOCSETIF: vboxnet0: Device not configured

        Thanks!

      • 15 akutz

        Bjorn,

        I imagine much like the VMware interfaces, the VirtualBox interfaces do not implement Berkley Packet Filter (BPF) which is what allows tools like tcpdump, ssldump, tshark, etc. to capture packets. In other words, you’re SOL. :(

  7. 16 akutz

    I contacted the dnsmasq author and he dismissed me as someone who didn’t really understand what was going on. Ugh. I was running SL when I built akin so I haven’t really noticed any change. Prior to Friday’s official release I had to build MacPorts from the trunk, but I noticed that they released an official SL build (0.8 I think) of MacPorts. You may want to give that a shot.

  8. 17 redben

    Is your solution to Mac os-x what bridge-utils is to linus ?
    http://www.tolaris.com/2009/03/05/using-host-networking-and-nat-with-virtualbox/

    • Sort of. Bridge utils is for creating a bridge between two separate network interfaces. My solution creates a network address translation (NAT) table that accomplishes the same thing with added security.


  1. 1 biotext.org.uk : Things to try when VirtualBox networking messes up
  2. 2 VirtualBox guest network lost after host disconnects Drija
  3. 3 VirtualBox guest network lost after host disconnects
  4. 4 VirtualBox guest network lost after host disconnects - Admins Goodies
  5. 5 Using a VirtualBox VM to operate a network device for your Mac
  6. 6 » OS X: How to Setup NAT on Lion and Mountain Lion

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

%d bloggers like this: