I recently obtained a second-hand Netgate SG-8860-1U firewall, which has now replaced my PC Engines apu2c4 board as the router in my home network. The apu2c4 is a perfectly good piece of hardware, but it only has three Ethernet ports compared to six on the Netgate machine. I was using all three ports (one connected to my modem and two bonded together and connected to my managed switch), so having some spare ports for expansion is useful – I’m already thinking about how I could rewire things to try to achieve 2 gigabits/sec with bonding between my desktop and server.
The Netgate machine comes bundled with pfSense, which is based on FreeBSD, but I’m not a particular fan of either – all my servers run Debian. I know a lot of people rave about how good the BSDs are at networking, but I’m not really convinced if you compare them to a modern Linux system. nftables is a significant improvement over iptables. Furthermore, people struggle to get the apu2c4 boards to do gigabit routing on BSDs, but Linux copes fine while using an insignificant amount of CPU power in comparison.
The specifications of the Netgate machine are:
The firmware is based on coreboot. It’s very bare bones – you can change the boot order and access the PXE console, but that’s about it. PXE only seems to work on the OPT1-4 ports, but I didn’t test this extensively. Booting from a USB stick also works.
It’s all fairly standard, so I thought it’d be quite easy to get Linux booting on it, but it did take me an entire evening trying to get the serial output to work.
On your desktop/laptop you’ll need to use a command like
picocom -b 115200 /dev/ttyUSB0 to open the serial port. I didn’t have any
problems getting output from the BIOS or GRUB, but I didn’t see any output after
that point – making it very difficult to debug.
I discovered, after much trial and error, that the internal USB to serial
adapter is actually connected to the COM2 port, so you need to pass
console=ttyS1,115200n8 to the Linux command line. I’d been trying with
instead, assuming it used COM1!
After that point everything went fairly smoothly, ignoring the slight annoyances that come with doing everything over a 115200 baud serial port instead of a high-speed SSH connection with a full-screen terminal.
Once you’ve installed Linux you need to make sure that your bootloader (e.g.
GRUB) will continue using the serial port and pass the correct
to Linux. This varies between distributions. In Debian you need to set the
following variables in
GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS1,115200n8" GRUB_TERMINAL=serial GRUB_SERIAL_COMMAND="serial --unit=1 --speed=115200 --parity=no --stop=1 --word=8"
and then run
update-grub to update the
/boot/grub/grub.cfg file. systemd
automatically spawns a getty on ttyS1, so you don’t neeed to worry about
fiddling around with
/etc/inittab any more.
Modern Linux distributions assign network interface names predictably, and I spent a little bit of time mapping the predictable names to the labels on the front panel (hopefully this will save some people’s time!):
|Network interface name||Front panel label|
With a fairly complicated nftables ruleset, including connection tracking, it’s capable of routing traffic between two subnets at around 930 megabits/sec, which isn’t too far from the theoretical maximum after accounting for Ethernet and TCP/IP overhead:
~$ iperf -c rictusempra ------------------------------------------------------------ Client connecting to 18.104.22.168, TCP port 5001 TCP window size: 85.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.1.2 port 43218 connected with 22.214.171.124 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 1.09 GBytes 934 Mbits/sec ~$