Host-based intrusion detection with Samhain

On Guard

By Tim Schürmann

Gary Blakeley, 123RF

Big companies don't just lock the doors at night - they also employ a guard. The guard makes regular rounds of the building and sounds the alarm if something unusual happens. Of course, you wouldn't normally employ a guard to watch over your computer, but you would use a firewall to shut all the doors and hope that regular updates will close down any vulnerabilities.

Modern systems are complex, and attackers increasingly find doors that have been left ajar. Once the attackers have entered the building, they will try to hide from users and administrators, typically by installing a rootkit. This rootkit will leave traces, and Samhain, the electronic guard, will detect those traces and sound the alarm.

Sniffer

Samhain ensures the integrity of a Linux system by regularly checking files for changes. To do this, Samhain generates a unique checksum, or fingerprint, for each file it monitors. This fingerprint changes if the file is manipulated by malware. Samhain checks the checksum and other critical file attributes at regular intervals and alerts the administrator if it finds any deviations. If necessary, it will also monitor all logins and logouts, search the system for SUID programs, and monitor the kernel for any changes. Samhain can report suspicious events to various targets; besides keeping a classic logfile, it will send its logs to a central logging server or mail them to the administrator.

Samhain only monitors the local computer and works as a host-based intrusion detection system (HIDS) [1]. In contrast to network-based IDS (NIDS) [2] alternatives, Samhain ignores incoming and outgoing network traffic. The tool thus does not raise the alarm until the attacker has started to mess with the system.

This doesn't mean that Samhain is unnecessary, however; if attackers get past the ingress controls, the only way to identify them is to rely on a guard making regular patrols. A HIDS is thus the only way to discover novel, zero-day intrusions and exploits from the internal LAN - that is, from people working for your own company.

As you can imagine, criminals don't appreciate intrusion detection systems and often target them for attacks. Samhain uses various safeguards to protect itself from this. For example, it can hide its own process and will only accept commands with a matching password if configured to do so.

Installing

Samhain is distributed under the GPL and is available for download from the project homepage [3]. Although you will often find matching packages in your own distribution's repositories, it is not a good idea to use them. For one thing, these packages are typically obsolete (which is the case with, say, Ubuntu 9.04), and for another, it is difficult or impossible to check the origin and integrity of the program.

If you follow the instructions, you will find that the Samhain installation is initially quite simple. Just unpack the archive on disk after downloading it; this reveals the source code package and a PGP signature. Before doing anything else, it is a good idea to check the PGP signature:

gpg --keyserver pgp.mit.edu --recv-key 0F571F6C
gpg --verifysamhain-<version>.tar.gz.asc samhain-<version>.tar.gz

The signature is that of Samhain's author, Rainer Wichmann; the fingerprint is also available on the download page [1]. If everything matches, you can follow the normal procedure:

./configure
make
make install

If you want to start Samhain directly as a daemon that executes on booting,

make install-boot

will do the trick. Unfortunately, that is just the beginning, as the Samhain documentation gradually reveals. In most cases, you will have to rebuild Samhain with different options for the configure command, especially if you are rolling it out to multiple computers - another reason why you should go for the source code and not the prebuilt distribution packages. Table 1 lists more important parameters for configure. Before you enable them all, you should familiarize yourself with the configuration file.

Coffee Filter

Samhain monitors the files on the Linux system and raises the alarm if something has changed. However, many files change through normal use. For example, logfiles grow, temporary files appear in /tmp, and users working with their OpenOffice documents change their documents' timestamps. If Samhain were to report all of these events, the attacks would be lost in a flood of useless data.

Thus, it is a good idea to tell Samhain what files to monitor and what activity to report before launching the tool. The /etc/samhainrc configuration file is used for this. The default template looks fairly complex, but it still provides a useful starting point for your own additions.

Just like older (Windows) INI files, the configuration file includes sections that start with a name in square brackets. Each setting is a setting-value pair; Samhain ignores lines that start with a hash sign (#).

To start, define where Samhain should send its warnings. This takes you to the [Log] section with the ...Severity settings for output options. For example, LogSeverity stores messages in a logfile, MailSeverity uses email, PrintSeverity prints the information on the console, and SyslogSeverity uses the Syslog.

To enable one of these notification targets, just remove the hash sign and specify how serious the message must be to warrant this handling. For example, if you specify

LogSeverity=err

the logfile will only contain errors, critical issues, and Samhain's own demise. Table 2 lists the severity levels and their content.

If you want Samhain to mail alerts to you, you will need to find the right [Misc] section (careful: the sample configuration has multiple sections with this name) and enter the data from Listing 1.

01 [Misc]
02 # Recipient:
03 SetMailAdress = name@example.com
04 # If needed, Relay:
05 SetMailRelay = relay.example.com
06 # Always send this number of messages in a single mail:
07 SetMaiNum = 10
08 # Max. time span (in sec.) before sending messages:
09 SetMailTime = 86400
10 # Subject line of emails sent:
11 MailSubject = Subject

The logfile is later stored in /var/log/samhain_log by default. You can pass in a different target path at the configure stage with the --with-log-file=/path/to/file option, or add

SetLogfilePath = /path/to/file

to the [Misc] section

Figure 1: While compiling the source code, configure creates a base key, which it adds to any email it sends. You can use the "samhain -M /path/to/file" command to check the origin of stored mail.

Policies

The next step is to tell Samhain how to monitor the target files and directories. The following lines tell Samhain to raise the alarm if somebody tries to access /my/file.txt, /important/folder, or /var/logs/a.log for anything but reading:

[ReadOnly]
dir=/important/folder
file=/my/file.txt
file=/var/logs/a.log

Absolute pathnames are a must. [ReadOnly] tells Samhain that the only thing that is allowed to change for the listed files is the last access timestamp. However, watch out for a minor pitfall: The logfile keeps on growing; the IDS will issue multiple false positives. This is why we need to move a.log to the special [GrowingLogFiles] section. Listing 2 shows the results.

01 [ReadOnly]
02 dir=/important/folder
03 file=/my/file.txt
04
05 [GrowingLogFiles]
06 file=/var/logs/a.log

Samhain refers to [ReadOnly] and [GrowingLogFiles] as policies. Table 3 shows what other policies the IDS knows.

If you want Samhain to avoid digging below a certain level of nesting in a folder, add this number to the path:

dir=3/important/folder

To ignore subdirectories, you can set a recursion depth of -1 in the [IgnoreAll] policy as follows:

[IgnoreAll]
dir=-1/important/folder/exclude

As the top of the sample configuration file shows, some sections can occur multiple times. Samhain simply merges them when evaluating, but it is a good idea to keep the default directories; they define some useful basic rules for the most important files on a Linux system.

Samhain is really finicky about access privileges to its own critical files - this applies in particular to the configuration file. Only root and the user accounts whose ID Samhain uses have write permissions. If you need to anyone else modify the configuration, you will need to configure "trusted" users for Samhain. To do so, specify the UID with the --with-trusted=0,<uid>,<uid>... configure option. To prevent manipulation, you should keep the defaults, that is, only grant root write privileges and run the commands as root.

Figure 2: The "samhain -d /path/to/database" command lists the content of the signature database; the output format is reminiscent of "ls -l".

Figures

Once you have completed the configuration, Samhain needs to read the files you want it to monitor once, create a unique checksum, and make a note of the other file characteristics. The following command triggers the initialization process:

samhain -t init

Depending on the number of files you want Samhain to monitor, you can probably take an extended coffee break right now. The information that Samhain collects is stored in a database that resides in /var/lib/samhain/samhain_file by default. If this file exists, Samhain will append the data.

In other words, you should not use the previous command to update the database. A similar command will provide an update:

samhain -t update

Incidentally, Samhain uses the TIGER192 algorithm to calculate the checksums. The alternative options here are SHA-1 and MD5; you can select them by changing the DigestAlgo setting in the configuration file. Samhain's author suggests avoiding MD5 because of possible vulnerabilities.

Once you have set up the database, check system integrity with the command:

samhain -t check

This tells Samhain to parse the system again and, for each file, to make sure the checksum and file properties match the entries in the database. To automate this check, launch Samhain as a daemon. If you installed Samhain with the make install-boot option, you need to reboot the system or enter the following command:

samhain -D -t check

The configuration file sets the check interval for the daemon. This can be a fixed entry, as in

SetFilecheckTime = seconds

or a crontab-style entry:

FileCheckScheduleOne = */5 * * * *

Because the database with the checksums is the basis for all future comparisons, it is important for the Linux system to be clean when you create the database. A guard will not notice an unlocked door if it has been open from the beginning. Thus, it is a good idea to install Samhain before you connect the computer to the network.

Additionally, make sure that nobody accesses the database. Either store it on a write-once medium or a server. The server could collect all the logfiles from all your Samhain installations on the LAN and store the configuration files for them. The benefit of this approach for the administrator is centralized storage of all the critical files without malware on the clients being able to manipulate them.

Master ...

To set up a client-server team, you need to build the Yule program on the server first. Luckily, you can use the Samhain source code for this. You just need to specify the --enable-network=server configure parameter:

./configure --enable-network=server
make
sudo make install

Follow this up with

sudo make install-user

to correct a couple of permissions. The yule log server only collects data from your clients and does not perform any integrity checks itself. Before you launch the server, you need to modify the [Log] section of the /etc/yulerc configuration file. It is identical to the samhainrc counterpart for the most part.

... and Servants

If you want your Samhain client installations to send their logs to the server, you will need to compile your clients to do so. The --enable-network=client configuration parameter does this:

./configure --enable-network=client --with-logserver=server.example.com--with-config-file=REQ_FROM_SERVER --with-data-file=REQ_FROM_SERVER/var/lib/samhain/samhain_file
make

But don't install just yet - there is still some tidying up to do. The remaining configure parameters tell Samhain to pick up its configuration and signature database from the server, server.example.com. REQ_FROM_SERVER points to the log server; the path in --with-data-file points to the local machine. This setup is necessary because Samhain stores the initialization results in a (local) file first - this option points to the file.

Keyholder

To prevent an attacker from manipulating the transferred information and signatures, the client and server use a secure connection. To set up the connection, Samhain must authenticate against Yule; the program thus needs a password, which the following command generates on the server (Figure 3):

yule -G

Once the key has been generated, go to the client and introduce Samhain to it:

./samhain_setpwd samhain new <password>

The samhain_setpwd utility is created when you build Samhain. This replaces the password generated by Samhain with the new one. samhain_setpwd actually modifies the existing samhain binary program and stores the results in samhain.new. Then, you can replace the existing version with this file:

mv samhain.new samhain

After completing all of these steps, you can finally install the IDS on the client:

sudo make install

and then build the signature database:

samhain -t init

Next, you need to manually copy the file this creates (/var/lib/samhain/samhain_file if you have been following these examples) to /var/lib/yule and rename it to file.<computername>. For example, if Samhain is running on a computer called client.example.com, your signature database would be called file.client.example.com.

Yule needs to know the client password to be able to check it. The following command on the server sets this up:

yule -P <password>

Yule creates a fairly lengthy, cryptic string with a full set of authentication credentials (as shown in Figure 3 - the line highlighted in black):

Client=HOSTNAME@123456789123456@123456789ABC ...

Then you need to add this line to the last section in the yulerc configuration file, [Clients], and replace the HOSTNAME string with the hostname of the client in question, as in:

[Clients]
Client=client.example.com@123456789123456@123456789ABC...

All of these steps must be repeated for all the remaining clients if you want them to send their data to Yule. Finally, the yulerc [Clients] section needs an entry for each Samhain client. Then you can launch Yule as a daemon:

yule -D

Alternatively, make install-boot will also enable Yule at boot time. Yule logs its own error messages in /var/log/yule/yule_log by default. Additionally, the server generates the yule.html file in the same directory. The file, which is updated every two minutes, provides a status report on the currently connected clients.

Figure 3: Yule has generated the key 2D1993AF832288D07 then generated the entry (highlighted in black) for its configuration file.

Vienna Calling

When a Samhain client contacts Yule, it authenticates by providing its password. After this, the two entities negotiate another key via the Secure Remote Password (SRP) and use this key for all further actions. The exchange of data is then encrypted and sent via TCP.

Samhain then picks up its configuration file. Yule expects this to be stored as rc.<hostname> in /var/lib/yule; this would be rc.client.example.com if you've been following the examples here. The best idea is to use the sample configuration as a starting point. To tell Samhain to send logfiles to the server, don't forget to enable the ExportSeverity setting in the [Log] section.

After parsing the configuration file, Samhain will look for the signature database in the /var/lib/yule directory. Armed with the signature database, it will check the system in the normal way, create logfiles, and send them to Yule. Yule will always record the logfiles, no matter what filter settings are defined in /etc/yulerc. To allow Yule to filter incoming messages, you can use the following settings in the [Misc] section:

UseClientSeverity = yes
UseClientClass = yes

This configuration keeps all the critical signature files, the log results, and the client settings on the server. This is the heart of your Samhain monitoring system and, as such, you should keep it safe, preferably using a Samhain daemon.

Flood of Information

Once the dynamic duo of Samhain and Yule is up and running, the administrator can get down to the real work, which is regularly checking and evaluating the logfiles. The only help Samhain will give you at this point is the option of passing the data to external scripts and programs. If you will be using a logfile analyzer to process the logs, you might want to convert them to XML format. Configure has an --enable-xml-log parameter for this. Additionally, Samhain can act as a sensor for the Prelude IDS and write its information to a database. But to allow this to happen - you guessed it - you need to build Samhain with the --with-prelude, --with-database=mysql, or configure options set and enable the options in the configuration file. Besides MySQL, Samhain can also talk to PostgreSQL (...=postgresql) and Oracle (...=oracle).

Additionally, you need to enable output to a database if you want to use Beltane [4], the neat web interface for managing Yule. Available as a separate package, it helps you view the logs and update the signature database. On the downside, the user interface requires a totally obsolete PHP4 environment that is unacceptable for a security tool.

To facilitate logfile evaluation, especially in more complex environments, there are some more advanced filtering options. For example, you can specify that only special events end up in the logfile, such as information on modified timestamps, or even define your own policies and modify existing policies.

GnuPG signed configuration files and the signature database provide protection against attempts to manipulate the core system. Before tweaking the finer controls, it is a good idea to read the manual carefully. A minor typo is all it takes to lock Samhain out of its own signature database or miss important traces of intrusion because of overly restrictive filters.

Conclusions

Samhain is not a panacea against intruders, but it is a useful addition to an existing firewall and an upstream NIDS. The included Yule log server acts as a central collection point on the LAN to keep everything tidy. Once you have fought your way through the complex configuration process and learned how to read the logfiles, you will be happy to have an attentive guard that detects uninvited guests quickly and quietly.