This document describes the process of setting up an IP traffic packet logging system on the x86 platform with OpenBSD. We will be using Daemonlogger by Martin Roesch.
The configuration assumed for this guide consists of two network interfaces. One will be for management and one will be connected to the network SPAN, TAP or hub on the edge device to monitor all traffic.
Daeomonlogger Test Platform:
- IBM xSeries 345 (w/ 2 Gigabit NICs)
- 2x Intel XEON 2.66Ghz CPUs / 3GB RAM
- OpenBSD 4.9 (x86)
Keep in mind that this document does not cover hardening the system. That process is outlined in other documents from myself or others. This machine needs to be well protected. It may be in a very vulnerable position, facing that filthy and scary Internet.
You will need root access to do most of these tasks.
II. Install and Setup the Operating Environment
I won't go into great detail for installing the operating system - you should have an idea of what you are doing. I suggest mounting your packet capture storage location separate, i.e. another drive (or RAID, or remote filesystem). We'll go over that shortly.
When selecting the installation sets, stick with the defaults, but remove the game set. Normally, this is where I'd say to remove the x sets as well, but some braintrust on the OpenBSD team has made it mandatory to have X11 installed to use the ports system. Is OpenBSD drifting away from "secure by default?"
During installation of the OS, say yes when asked to start sshd by default, yes when asked to start ntpd by default and no when asked if you expect to run the X Window System.
If you are using an SMP (multi-processor) system, be sure to keep the bsd.mp kernel selected. OpenBSD now has a feature where it will automatically replace the stock bsd kernel after installation (you don't have to manually move /bsd.mp over /bsd after the first boot anymore).
When the installation is finished, remove the install media (CD, floppy, whatever) and reboot!
Now we need to fetch the ports tree. FYI, this could take a while:
|# cd /usr
# cvs -qd email@example.com:/cvs get -r OPENBSD_4_9 -P ports
And now the latest OpenBSD 4.9 ports tree is on your system and ready to use.
III. Configure SoftRAID (optional)
For this guide, I we will setup a 5 drive SoftRAID (RAID0 striped set) filesystem mounted as /net-data (salt to taste). Depending on the amount of traffic you will be capturing, make this as large as possible. Later we will configure setting the overwrite/FIFO data writing schedule. I also suggest comming up with a backup scenerio.
First, since this system is using SCSI drives, our disks available for RAID are sd1, sd2, sd3, sd4 and sd5. Let's prepare them for RAIDification...
|# fdisk -iy sd1
# fdisk -iy sd2
# fdisk -iy sd3
# fdisk -iy sd4
# fdisk -iy sd5
# printf "a\n\n\n\nRAID\nw\nq\n\n" | disklabel -E sd1
# printf "a\n\n\n\nRAID\nw\nq\n\n" | disklabel -E sd2
# printf "a\n\n\n\nRAID\nw\nq\n\n" | disklabel -E sd3
# printf "a\n\n\n\nRAID\nw\nq\n\n" | disklabel -E sd4
# printf "a\n\n\n\nRAID\nw\nq\n\n" | disklabel -E sd5
Now, assemble the RAID:
|# bioctl -c 0 -l /dev/sd1a,/dev/sd2a,/dev/sd3a,/dev/sd4a,/dev/sd5a softraid0
This will create a RAID device on the next available sd dev - sd6.
Now let's clear up some bits, initialize it, create the file system and setup a mount point:
|# dd if=/dev/zero of=/dev/rsd6c bs=1m count=1
# fdisk -iy sd6
# printf "a\n\n\n\n4.2BSD\nw\nq\n\n" | disklabel -E sd6
# newfs /dev/rsd6a
# mkdir /net-data
# mount /dev/sd6a /net-data
# echo "/dev/sd6a /net-data ffs rw,softdep,nodev,nosuid 1 2" >> /etc/fstab
That was fun.
First, libdnet is a dependency. Let's do that first. I'd use ports, but the dependencies are not necessary and the port is broken anyway.
|# cd /usr/src
# wget http://libdnet.googlecode.com/files/libdnet-1.12.tgz
# tar zxvf libdnet-1.12.tgz
# rm libdnet-1.12.tgz
# cd libdnet-1.12
# ./configure && make && make install
|# cd /usr/src
# wget http://www.snort.org/users/roesch/code/daemonlogger-1.2.1.tar.gz
# tar zxvf daemonlogger-1.2.1.tar.gz
# rm daemonlogger-1.2.1.tar.gz
# cd daemonlogger-1.2.1
# ./configure && make && make install
On my system, the LAN (management) interface is em0. I would like to use em1 to passively sniff packets on the SPAN port of my Cisco switch. All we need to do is turn on the interface, no further configuration is needed. So just issue the following command to activate it (like I said, mine is em1, yours may be different):
Daemonlogger is a fun and incredibly easy tool to use. To view available options, run daemonlogger -h. One of the great features is the ability to utilize the Berkeley Packet Filter (BPF). This allows you to only capture the traffic you truly care about.
For example, you can define a list of ports you'd like to either include or exclude from capture by adding them to a file and, then calling the file with the -f flag (i.e. daemonlogger -f ports.bpf). ports.bpf could contain something like this to include ports 80, 8080 and 5190:
|port 80 or port 8080 or port 5190
Alternatively, you could capture all ports except 443 (HTTPS) with this in the ports.bpf file:
There are a few ways you can have Daemonlogger handle packet capturing (log rollover). I will review two here: time and size.
To rollover the log file every n bytes, use the -s flag. Though undocumented, you can use -s n<k/m/g/t> (kb, mb, gb, tb) - (thanks to Marty for the note about that!). For example, to start writing to a new log file each time it reaches 1GB, you would add the following to your execution statement:
To rollover the log file using time increments, use the -t flag (which counts by seconds). For example, to start writing to a new log file hourly, you would add the following to your execution statement:
There are other options to consider when using this software. The command daemonlogger -h will list what is available.
The following command will use the ports filter file I created (-f ports.bpf), listen to traffic on interface eth1 (-i em1), write the log files to the /net-data directory (-l /net-data), set it to rollover the log files every hour (-t 3600) and prefix each log file with DMZ (the subnet I'm listening to):
|# daemonlogger -f ports.bpf -i em1 -l /net-data -t 3600 -n dmz
The following command will use the ports filter file I created (-f ports.bpf), listen to traffic on interface eth1 (-i em1), write the log files to the /net-data directory (-l /net-data), set it to rollover the log files each time they reach 500MB (-s 500m) and prefix each log file with DMZ (the subnet I'm listening to):
|# daemonlogger -f ports.bpf -i em1 -l /net-data -s 500m -n dmz
VI. Helpful Links