A look at three programs designed to help the C programmer find the cause of segmentation fault errors.
All C programmers have seen, at least once, the horrible words “Segmentation fault—Core dumped” after a run of their latest creation. Usually, this message is due to errors in the memory management. (As all C programmers know, this language does not care about bounds or limit when accessing memory.) In this article, I plan to compare three products used to track down this kind of error:
Checker 0.9.9.1
Electric Fence 2.0.5
Mem-Test 0.10
Checker comes in the usual tgz format (gzipped tar file), with a simple installation procedure. Run the “configure” script, then make all the files. The installation went fine for me; I didn't see any problems. One note: you need gcc 2.8.1 to use the latest version of Checker.
Electric Fence is available in binary and source format and requires kernel 1.1.83 or higher.
Mem-Test is available in the tgz format, and it is very simple to build using the provided Makefile.
Electric Fence (EF) is a library—link it to your program, then run it. EF will cause a segmentation fault on the exact line of the wrong instruction (not 100 lines after), so by tracking the program with a debugger, you can get to the root of the problem. Place an inaccessible memory page after (or before, by using the correct option) each area allocated by your program, and EF will cause an immediate error when the program goes out of the bounds.
Mem-Test is another library you can simply link to your object—just remember to include the header file mem_test_user.h before. As we will see, this program is a bit different from the other two, and it detects particular errors. When the program runs, it creates a log where it stores all memory allocation/deallocation. By using a Perl script provided in the package, it will show you the memory leak present in the code. Since Electric Fence doesn't detect this particular error, it can be used in conjunction with Mem-Test.
Checker is also a library and exploits the -fcheck-memory-usage option of gcc. A different compiler is actually used to build your program: checkergcc. It is a stub which calls gcc and compiles the program with its own memory access libraries. Once the program is compiled, you can run it and checker will show you a complete report with the errors it found in your sources. Checker uses a bitmap to store any memory area the program is using. This bitmap will contain the access right of each memory area. For example, an area could be write-only (when the variable is not yet initialized), readable and writable, not accessible and so on. In this way, it will be able to detect the memory access error.
The six pieces of C code we will look at are:
postr.c: this code (Listing 1) performs a read (with printf) of an uninitialized memory area. The lack of the string terminator (\0) will force the printf to read after the malloc area.
prer.c: this piece of code (Listing 2) contains two errors. The printf is accessing a byte before the allocated area (the pointer was decremented), then the free is done with an address not returned by malloc.
postw.c: in this code (Listing 3), the strcpy is writing 12 bytes (with the \0) in a 10-byte area. Moreover, the printf is reading the uninitialized last two bytes.
prew.c: this code (Listing 4) is writing before the allocated memory. The free and the printf will cause an error as in the previous examples.
uninit.c: this code (Listing 5) makes an assignment to a NULL pointer. This is a common error for programmers new to the C language.
unfree.c: in this example (Listing 6), I missed freeing some allocated memory.
Post-Read
To test Checker, I compiled Listing 1 with this command line:
checkergcc -o postr postr.c
All the gcc command-line options can be used with Checker. The compilation went fine, and when I ran postr, I got this output:
From Checker (pid:00411): (ruh) read uninitialized byte(s) in a block. When Reading 5 byte(s) at address 0x0805ce1c, inside the heap (sbrk). 0 byte(s) into a block (start: 0x805ce1c, length: 10, mdesc: 0x0). The block was allocated from: pc=0x08054e2b in chkr_malloc at stubs-malloc.c:52 pc=0x08048812 in main at postr.c:10 pc=0x08054ee1 in this_main at stubs-main.c:14 pc=0x0804875a in *unknown* at *unknown*:0 Stack frames are: pc=0x08054ebf in chkr_stub_printf at stubs-stdio.c:54 pc=0x080489f1 in main at postr.c:17 pc=0x08054ee1 in this_main at stubs-main.c:14 pc=0x0804875a in *unknown* at *unknown*:0 exaChecker executed the program and found the problem—an uninitialized read at line 17 (the printf line). This was caused by the lack of a string terminator in the memory area. At first look, this output seems quite messy, but if you read it carefully, you will find a lot of information: which type of error it found, where the memory was allocated (line 10) and where the problem occurs (line 17).
To compile the program with Mem-Test, you must perform a slight modification to the postr source. Add header file (#include "mem_test_user.h") to wrap the various memory allocation functions and use a modified version. Compile the program with the command:
gcc -o postr postr.c -lmem_test
I added another library (mem_test) to the compilation command. When you run the postr executable, the new library will create a file named MEM_TEST_FILE in which all memory accesses and leaks will be logged. In this particular situation, Mem-Test does not find a problem because it was built to identify only memory leaks.
For Electric Fence, we need to recompile the program, including the reference library:
gcc -g -o postr postr.c -lefence
I added the -g option to include the debugging information in the executable. This is needed because EF will cause a segmentation fault exactly at the buggy line, so you will need to walk through the code to find the exact line causing the problem. This is the output of the executable:
Electric Fence 2.0.5 Copyright (C) 1987-1995 Bruce Perens. exaEF didn't find any problem in the code, so no errors were generated.
EF has four different switches, which can be enabled by setting one of these environment variables: EF_ALIGNMENT, EF_PROTECT_BELOW, EF_PROTECT_FREE or EF_ALLOW_MALLOC_0.
EF_ALIGNMENT sets the alignment for each memory allocation done by malloc (or calloc and realloc). By default, this size is set to sizeof(int), because this is usually the alignment required by the CPU. This could be a problem when you allocate a size that is not a multiple of the word size. Since the inaccessible page must be set to word-aligned address, you have a hole after the allocated memory to the inaccessible page. You can fix this by setting the environment variable to 0; in this way, you will be able to find a single-byte overrun. This will force malloc to return a non-aligned address, but this is not a problem in most cases. In some cases (when you have an odd-size allocation for an object that must be word-aligned), you will get a bus error (SIGBUS). I never saw a SIGBUS error using EF (and I used it in real-life programs); I got this information from the EF documentation.
EF will usually place the unaccessible page after each memory allocation. By setting EF_PROTECT_BELOW to 1, it will place this page before the allocation, so you can check for under-runs.
EF allows you to allocate freed memory. If you think your program is touching free memory, set EF_PROTECT_FREE to 1. EF will not reallocate any freed memory, and any access will be detected.
A malloc call with zero bytes is considered an error. If you need to use such a call, you can tell EF to ignore this error by making EF_ALLOW_MALLOC_0 non-zero.
I set EF_ALIGNMENT to 0 in order to see if the postr error would be detected, but again EF did not see it.
Checker found the problem at the correct line (printf) in Listing 2, and it pointed out the freeing of an address different from the one returned by malloc. Actually, I decremented the foo pointer and tried to free this address.
Mem-Test didn't find the problem, but this was expected.
If I link the EF library without specifying any switch, Electric Fence returns only an error regarding the freeing of a non-malloc returned value:
Electric Fence 2.0.5 Copyright (C) 1987-1995 Bruce Perens. ElectricFence Aborting: free(400b3ff3): address not from malloc(). Illegal Instruction (core dumped)
Then, I tried to setting the protect below switch:
export EF_PROTECT_BELOW=1With this variable, EF caused a segmentation fault. With gdb, I tracked down the program to the printf where the segmentation fault occurred.
For Listing 3, Checker found the bound violation at line 14 (strcpy). Moreover, it also found an uninitialized data read at line 16 (printf). Actually, the print is going to read after the allocated area.
Mem-Test gave no reports as expected.
Again, the first run (without any switch) of Electric Fence did not report any error. I then set EF_ALIGNMENT to 0, and the strcpy caused a core dump.
The error in Listing 4 was correctly detected by Checker as the incorrect free. Mem-Test gave no reports. When I didn't set a switch, Electric Fence detected only the wrong free, but with EF_PROTECT_BELOW set, it also found the pre-write.
Checker found the exact line in Listing 5 where the bad assignment was made. No reports were expected or created by Mem-Test. There was a core dump, but the log was not created. Electric Fence did not detect this error. When you run the program, you will get a core dump whether you use EF or not.
By default, Checker does not find memory leaks. The documentation shows several switches you can set to modify the checking. Different switches are set by defining the environment variable CHECKEROPTS. The more interesting options are:
-o=file: redirect the output to a file.
-d=xx: disable a type of memory access error.
-D=end: do leak detection at the end of the program.
-m=x: define the behavior for a malloc(0).
I ran export CHECKEROPTS="-D=end" and then recompiled. Now it found the memory leak of 50 bytes in Listing 6. Checker implements a garbage detector to find out this type of error. You can call it by setting this option or by calling a specific Checker function inside your program.
Mem-Test easily identifies the memory leak, with a clear report:
50 bytes of memory left allocated at 134524624 134524624 was last touched by licalloc at line 12 of unfree.c
Electric Fence returned no messages.
From these tests, it appears clear that Checker is a complete product which found all the errors without any problems. It is quite easy to use, and you don't have to set a lot of switches because, by default, it checks for a wide range of errors. It does have a little problem when you use external libraries and functions (such as the GDBM). Actually, to ensure it will check for everything, you should recompile all the external programs with it. If you call a function not compiled with it, the memory bitmap used to track the memory accesses will not be updated; this will create holes in your checks. You have two ways to do this: recompile the library with checkergcc, or create function stubs. The stubs are particular aliases for each function, which perform some checks on the parameters passed to and returned from the function. In particular, you must check for pointers to see if the memory area you will access by using them has the correct status (readable, writable, etc.).
Provided in the package are many ready-to-use stubs for the most popular functions (such as stdio and the string functions). After a look at these stubs, it should not be difficult to write your own for libraries you cannot recompile with checkergcc.
On the other hand, Electric Fence showed some hesitation in error detection but is easier to use. It is enough to link it to the program and run it (no problem with external libraries). If used in tandem with Mem-Test, it will also detect memory leaks. For best results, be careful with the switches: use the correct spelling and the correct alignment and protect (below or after the memory allocation).
