All posts by werner

Using Electric Fence to Debug Selected Allocations

Electric Fence is a Red-Zone memory allocator written by Bruce Perens. It provides a special version of malloc() and similar functions for debugging software that is suspected of overrunning or underrunning the boundaries of a malloc buffer, or touching free memory. It arranges for each malloc buffer to be followed (or preceded) in the address space by an inaccessable virtual memory page, and for free memory to be inaccessable. If software touches the inaccessable page, it will get an immediate segmentation fault. It is then trivial to uncover the offending code using a debugger. An advantage of this product over most malloc debuggers is that this one detects reading out of bounds as well as writing, and this one stops on the exact instruction that causes the error, rather than waiting until the next boundary check. A Secondary advantage is that it can be used as a replacement of the standard malloc, thereby debugging software that cannot be recompiled or tracking allocations within libraries themselves.

 

This however forms immediatelly the problem I encountered. When debugging my pet-project BpmDj, I found that the QT libraries use extravagant many memory allocations, making the use of electric fence impossible, due to its memory consumption (2 pages = 16K for every single allocation !). Nevertheless, the thing I wanted to do was to check the memory chunks that I had allocated, not those allocated by others. Below we explain how this can easily be done with electric fence.

Step 1: change all malloc’s, realloc’s and frees in your software with own versions. E.g: allocate, reallocate, deallocate. Prototypes of such functions can be found in the file common.h

#define allocate(size, type)          (type*) bpmdj_alloc(sizeof(type)*(size), __FILE__, __LINE__)
#define array(name,size,type)          type*  name = allocate(size,type)
#define reallocate(thing, size, type) (type*) bpmdj_realloc(thing,sizeof(type)*(size))
#define deallocate(thing) bpmdj_free(thing);void * bpmdj_alloc(int size, char* file, int line);
void * bpmdj_realloc(void* thing, int size);
void   bpmdj_free(void*);

Step 2: modify the library to NOT replace the standard memory operations. The only files needed from the electric fence package are efence.h page.c, efence.c and efence-print.cpp

Step 2.1: modify all references to malloc, calloc, realloc and free in efence.c to efence_malloc, efence_calloc, efence_realloc and efence_free. Make sure that prototypes for these functions are added into efence.h.


void * efence_realloc(void * oldBuffer, size_t newSize)
{
    void *    newBuffer = efence_malloc(newSize);

        efence_free(oldBuffer);

}void * efence_malloc(size_t size)
{

}

void * efence_calloc(size_t nelem, size_t elsize)
{

}

Step 2.2: modify the efence_free function to return a boolean. If the freeing was successfull true should be returned. If not because the pointer itself was not within range of the efence allocated memory thenen it should return false. This will later prove to be very usefull to write a generic deallocate function.

bool efence_free(void * address)
{

  if ( address == 0 )
    {
      unlock();
      return true;
    }

  if ( !slot )
    {
      unlock();
      return false;
    }
    // removed EF_Abort(“efence_free(%a): address not from efence_malloc().”, address);

  return true;
}

Step 2.3: the standard efence-print library makes ues of its own printing functions. These can be safely removed because the standard C library functions no longer pose a reentrance thread. Thus, remove printNumber and vprint. Replace all vprint calls with vprintf and include the necessary #include <stdlib.h>

Step 2.4: change efence-print.cpp, efence.cpp, page.cpp, efence.h by putting an #ifdef EFENCE in front and an #endif in the back of the file. This will make it possible to remove the entire EFENCE from compiling.

Step 3: write your own memory functions as declared in step 1. These are straightforward, except for the deallocate function. Here we must differentiate between efence-allocated chunks and an standard-malloced chunks. This can be done as follows.

void bpmdj_free(void* a)
{
#ifdef EFENCE
if (!efence_free(a))
#endif
free(a);
}void* bpmdj_alloc(int length, char* file, int line)
{
void * result;
assert(length>=0);
#ifdef EFENCE
result = efence_malloc(length);
#else
result = malloc(length);
#endif
if (!result)
printf(“Error: %s(%d): unable to allocate %d bytes \n”,file,line,length);
assert(result);
return result;
}

void* bpmdj_realloc(void* thing, int size)
{
void * result;
assert(size);
#ifdef EFENCE
result = efence_realloc(thing,size);
#else
result = realloc(thing,size);
#endif
assert(result);
return result;
}

This should do the trick. To use it simply put the object files efence.o, page.o, efence-print.o and common.o into your program. When you want checks pass -DEFENCE to the compiler, in the other case simply ommit it. These files can be downloaded here. This should make it possible to debug QT applications easily under linux.

Compiling Tulip for a Debian unstable System with an ATI 3D Card / Fglrx 3D driver

The standard tulip distribution that comes with debian does not work on a 3D fglrx installation. The reason why is not entirely clear (it seems that the linkage to the X window system is faulty). However, recompiling the tulip source solves the problem.

Prereqs:
– an Xserver that uses the fglrx 3D driver. This can be tested with fglrxinfo.

Commands:
apt-get install fglrx-driver-dev
apt-get install libxml2-dev libqt3-mt-dev libqt3-headers libqt4-dev libqt4-core libqt4-gui libqt4-qt3support libqt4-sql qt3-assistant qt3-apps-dev
apt-get source tulip

go into the unzipped tulip source directory and configure the tree with
./configure --with-qt-includes=/usr/include/qt3
make

now everything should run fnie. To install tulip use
make install
/usr/local/bin/tulip

Compiling the Fglrx driver for a Debian unstable system

To compile the fglrx driver for a debian unstable system running a 2.6.18 kernel, which was compiled from kernel.org (so no standard debian kernel).

Prereqs:

– kernel has an agpgart module
– kernel has dri support
– apt-get module-assistant

Commands:

apt-get install fglrx-driver
will load the 8.28.8-4 driver and install it

aticonfig --initial will write the X.org file
apt-get install fglrx-kernel-src
cd /usr/src/modules
rm -r fglrx
cd /usr/src
tar x fglrx.tar.bz2
module-assistant prepare
module-assistant a-i fglrx

then
modprobe agpgart
rmmod radeon (fglrx works only if the radeon driver was unloaded)
modprobe fglrx

Then extend the Xorg.conf file with
Section "Extensions"
Option  "Composite" "false"  #make DRI work with fglrx.
EndSection

Section “ServerFlags”
Option “AIGLX” “off”
EndSection

Then load the X server and start fglrxinfo. If it contains anything but ATI it didn’t work.

Problems:

To help debug one can use something like:
LIBGL_DEBUG=verbose fglrxinfo

before starting fglrxinfo. In a particular case it printed:
libGL: XF86DRIGetClientDriverName: 8.28.8 fglrx (screen 0)
libGL: OpenDriver: trying /usr/X11R6/lib/modules/dri//fglrx_dri.so
libGL error: dlopen /usr/X11R6/lib/modules/dri//fglrx_dri.so failed
(/usr/X11R6/lib/modules/dri//fglrx_dri.so: cannot open shared object
file: No such file or directory)
libGL error: unable to find driver: fglrx_dri.so
libGL: XF86DRIGetClientDriverName: 8.28.8 fglrx (screen 0)
libGL: OpenDriver: trying /usr/X11R6/lib/modules/dri//fglrx_dri.so
libGL error: dlopen /usr/X11R6/lib/modules/dri//fglrx_dri.so failed
(/usr/X11R6/lib/modules/dri//fglrx_dri.so: cannot open shared object
file: No such file or directory)
libGL error: unable to find driver: fglrx_dri.so
display: :0.0  screen: 0
OpenGL vendor string: Mesa project: www.mesa3d.org
OpenGL renderer string: Mesa GLX Indirect
OpenGL version string: 1.2 (1.5 Mesa 6.5.1)

which indicated that the wrong library was laoded. Therefore the following two commands resolved this issue:
tuuster:/usr/X11R6/lib/modules# ln -s /usr/lib/xorg/modules/drivers dri
rm dri
tuuster:/usr/X11R6/lib/modules# ln -s /usr/lib/dri .