Critique My Non-Intrusive Heap Debugger

Critique my non-intrusive heap debugger

Very nice, indeed. Your canaries could actually reveal some real cases of overflow/underflow (though not all of them as Matthieu pointed out).

What more. You might run into some problems with a multi-threaded application. Perhaps protect the hashtable from concurrent access?

Now that you log every allocation and deallocation, you can (if you like) provide more information about the program being tested. It might be interesting to know the total and average number of allocations at any given time? The total, max, min and average bytes allocated, and the average lifespan of allocations.

If you want to compare different threads, at least with Pthreads you can identify them with pthread_self(). This heap debugger could become a quite useful analysis tool.

Critique my heap debugger

Instead of doing intrusive note-keeping you could keep a list of all allocations made. Then you can free the memory without destroying your own data, and keep track of how many times a particular address is "deleted", and also find places where the program tries to delete a non-matching address (i.e. not in the list).

Production debugging: Is there a less intrusive way than WinDbg?

If I understand you correctly, you don't want to actually debug the program, only get some basic information. In such cases, Process Explorer may be sufficient.

Writing Garbage on free / delete

The following code does exactly what I want:

#include <malloc.h>

typedef void (*free_hook_t)(void*, const void*);

static free_hook_t system_free_hook;

static void my_free_hook (void *ptr, const void *caller)
     {
       __free_hook = system_free_hook;
       int size = malloc_usable_size(ptr);
       memset(ptr,0xDE, size);
       free (ptr);
       __free_hook = my_free_hook;
     }

static void init_free_hook()
     {
     system_free_hook = __free_hook;
      __free_hook = my_free_hook;
     }

/* Override initializing hook from the C library. */
void (*__malloc_initialize_hook) (void) = init_free_hook;

It is totally stand alone so technically can be included or not as required. The bit I was missing was the malloc_usable_size function.

Testing on Ubuntu 10.10, this also works in C++ where one is using new and delete

Fixing Segmentation faults in C++

1. Compile your application with -g, then you'll have debug symbols in the binary file.

2. Use gdb to open the gdb console.

3. Use file and pass it your application's binary file in the console.

4. Use run and pass in any arguments your application needs to start.

5. Do something to cause a Segmentation Fault.

6. Type bt in the gdb console to get a stack trace of the Segmentation Fault.

Finding Java memory leak when not all used heap is reachable from threads

I too faced the problem with memory leaks at our site,

Use yourkit java profiler which provide lots of information and with its ability you can have a wider image where all the memory is being utilized.

You can find a great tutorial Find Java Memory Leaks with the above tool.

Your question,

"What kind of Objects are there that are not reachable from an instance of java.lang.Thread other then Objects in the Finalizer Queue and unreferenced objects pending GC?"

There are four kinds of object,

1. Strong reachable, objects that can be reached directly via references from live objects
2. Weak/Soft reachable, objects that are having weak/Soft reference associated with them
3. Pending Finalization, objects that are pending for finalization and whose reference can be reached through finalizer queue
4. Unreachable these are objects that are unreachable from GC roots, but not yet collected

Besides these JVM also uses Native memory whose information you can find on IBM Heap and native memory use by the JVM and Thanks for the memory and according to YourKit the JVM Memory Structure has Non-Heap Memory whose definition according to them is

Also, the JVM has memory other than the heap, referred to as non-heap memory. It is created at the JVM startup and stores per-class structures such as runtime constant pool, field and method data, and the code for methods and constructors, as well as interned Strings.

Overriding new and Logging data about the caller

You can write

new(foo, bar) MyClass;

In this case the following function is called

void*operator new(std::size_t, Foo, Bar){
    ...
}

You can now call

new(__LINE__, __FILE__) MyClass;

and use the data with

void*operator new(std::size_t, unsigned line, const char*file){
    ...
}

Adding a macro

#define new new(__LINE__, __FILE__)

to the code being monitored will catch most invocations without needing source code changes.

It's not perfect as you could call the operator new directly for example. In that case the preprocessor will turn your code into garbage. I know of no better way though.

How to write a small memory leak detection in C++?

In this question i have proposed an ad-hoc solution to your problem: Immediate detection of heap corruption errors on Windows. How?

In general, you can replace your new and delete with this code:

DWORD PageSize = 0;

inline void SetPageSize()
{
    if ( !PageSize )
    {
        SYSTEM_INFO sysInfo;
        GetSystemInfo(&sysInfo);
        PageSize = sysInfo.dwPageSize;
    }
}

void* operator new (size_t nSize)
{
    SetPageSize();
    size_t Extra = nSize % PageSize;
    nSize = nSize + ( PageSize - Extra );
    return Ptr = VirtualAlloc( 0, nSize, MEM_COMMIT, PAGE_READWRITE);
}

void operator delete (void* pPtr)
{
    MEMORY_BASIC_INFORMATION mbi;
    VirtualQuery(pPtr, &mbi, sizeof(mbi));
    // leave pages in reserved state, but free the physical memory
    VirtualFree(pPtr, 0, MEM_DECOMMIT);
    DWORD OldProtect;
    // protect the address space, so noone can access those pages
    VirtualProtect(pPtr, mbi.RegionSize, PAGE_NOACCESS, &OldProtect);
}

to determine invalid access of memory. In the further discussion there you will find ideas for leaks detection and detection of other memory errors.

If you want to call global new and delete, you can use :: global namespace prefix:

return ::new(nSize);

---

Related Topics