Do (Statically Linked) Dlls Use a Different Heap Than the Main Program

Do (statically linked) DLLs use a different heap than the main program?

DLLs / exes will need to link to an implementation of C run time libraries.

In case of C Windows Runtime libraries, you have the option to specify, if you wish to link to the following:

1. Single-threaded C Run time library (Support for single threaded libraries have been discontinued now)
2. Multi-threaded DLL / Multi-threaded Debug DLL
3. Static Run time libraries.
4. Few More (You can check the link)

Each one of them will be referring to a different heap, so you are not allowed pass address obtained from heap of one runtime library to other.

Now, it depends on which C run time library the DLL which you are talking about has been linked to. Suppose let's say, the DLL which you are using has been linked to static C run time library and your application code (containing the main function) has linked to multi-threaded C Runtime DLL, then if you pass a pointer to memory allocated in the DLL to your main program and try to free it there or vice-versa, it can lead to undefined behaviour. So, the basic root cause are the C runtime libraries. Please choose them carefully.

Please find more info on the C run time libraries supported here & here

A quote from MSDN:

Caution Do not mix static and dynamic versions of the run-time libraries. Having more than one copy of the run-time libraries in a process can cause problems, because static data in one copy is not shared with the other copy. The linker prevents you from linking with both static and dynamic versions within one .exe file, but you can still end up with two (or more) copies of the run-time libraries. For example, a dynamic-link library linked with the static (non-DLL) versions of the run-time libraries can cause problems when used with an .exe file that was linked with the dynamic (DLL) version of the run-time libraries. (You should also avoid mixing the debug and non-debug versions of the libraries in one process.)

When exactly would a DLL use a different heap than the executable?

I may be reading more into your question than is there, but if you are wanting to know how you can allocate and free memory across DLL boundaries, then you might use something like the following:

#define DLLMemAlloc( size ) HeapAlloc( GetProcessHeap(), 0, size )
#define DLLMemFree( mem )   HeapFree( GetProcessHeap(), 0, mem )

That might be safer (a partial attempt at future-proofing). Relying on various build options to guarantee the safety of allocating and freeing across boundaries might lead to problems.

And (also not part of the question), you might re-think whether it is really necessary to be able to do this. It seems like there may be a design flaw if one DLL has to allocate something that another DLL (or executable) has to free.

Do different programs gets their memory from a common heap or from a separate heap?

1. There is no common heap in the libc sense - this would violate process protection and virtual memory rules. Each process maintains its own heap. The kernel (with the help of the MMU in the processor) maintains the virtual memory tables which map virtual addresses to real memory.

2. Static libraries are nothing more than linking code at compile time - there is no run time concept of a static library. It is one and the same as the process, and will use its heap.

Loading two dlls which are themselves statically linked with different version of a library causes a crash when these both are loaded in a executable

The program crashes because different versions of the C DLL have different type definitions which conflict with each other. If you have certain fields that don't exist in one version and exist in another, then depending on the type of the DLL, you'll get 2 different scenarios. Managed will fail to bind and will tell you that. Unmanaged will corrupt memory and you will get some nasty error elsewhere unexpectedly.

You're seeing different behavior between OSes probably because XP does its memory management differently than 7 (leading me to believe it's unmanaged DLLs), but it's tough to tell.

Memory Allocation in Static vs Dynamic Linking of C Runtime

If you want to allocate from the heap in one module, and free in another then you simply have to dynamically link the runtime. What's more, all parties must dynamically link the same runtime. Once you do that then there will be no problems.

Now, this is quite a severe constraint to impose on your plugins and you should think twice before doing it. Forcing all plugin authors to upgrade in lock-step with you is a heavy imposition. I would recommend that you consider refactoring your interface so that allocations are always paired with deallocations within a single module. This lifts the constraints I describe above and makes life easier for your plugin authors.

As for why you are still suffering from runtime errors, that's hard to tell. My guess is that not all modules are linking against the same version of the runtime.

---

In the comments you state that you control all plugins. That means the constraints I discuss above are not an imposition since it is easy for you to use the same compiler version for all modules. However, the rules for dynamic linking with cross-module heap access remain. You must use dynamic linking against the same single version of the runtime.

Sharing heap memory in a dll between two separate applications

This exactly is not possible - as the address spaces for the two processes are different (because they're virtual, having been created by the kernel), so a valid pointer in one won't work within the other. However, you can use shared memory to transport raw scalar data (strings, integers) between processes - here's how.

Memory / heap management across DLLs

As you suggested, you can use a boost::shared_ptr to handle that problem. In the constructor you can pass a custom cleanup function, which could be the deleteObject-Method of the dll that created the pointer. Example:

boost::shared_ptr< MyObject > Instance( getObject( ), deleteObject );

If you do not need a C-Interface for your dll, you can have getObject return a shared_ptr.

What do 'statically linked' and 'dynamically linked' mean?

There are (in most cases, discounting interpreted code) two stages in getting from source code (what you write) to executable code (what you run).

The first is compilation which turns source code into object modules.

The second, linking, is what combines object modules together to form an executable.

The distinction is made for, among other things, allowing third party libraries to be included in your executable without you seeing their source code (such as libraries for database access, network communications and graphical user interfaces), or for compiling code in different languages (C and assembly code for example) and then linking them all together.

When you statically link a file into an executable, the contents of that file are included at link time. In other words, the contents of the file are physically inserted into the executable that you will run.

When you link dynamically, a pointer to the file being linked in (the file name of the file, for example) is included in the executable and the contents of said file are not included at link time. It's only when you later run the executable that these dynamically linked files are brought in and they're only brought into the in-memory copy of the executable, not the one on disk.

It's basically a method of deferred linking. There's an even more deferred method (called late binding on some systems) that won't bring in the dynamically linked file until you actually try to call a function within it.

Statically-linked files are 'locked' to the executable at link time so they never change. A dynamically linked file referenced by an executable can change just by replacing the file on the disk.

This allows updates to functionality without having to re-link the code; the loader re-links every time you run it.

This is both good and bad - on one hand, it allows easier updates and bug fixes, on the other it can lead to programs ceasing to work if the updates are incompatible - this is sometimes responsible for the dreaded "DLL hell" that some people mention in that applications can be broken if you replace a dynamically linked library with one that's not compatible (developers who do this should expect to be hunted down and punished severely, by the way).

---

As an example, let's look at the case of a user compiling their main.c file for static and dynamic linking.

Phase     Static                    Dynamic
--------  ----------------------    ------------------------
          +---------+               +---------+
          | main.c  |               | main.c  |
          +---------+               +---------+
Compile........|.........................|...................
          +---------+ +---------+   +---------+ +--------+
          | main.o  | | crtlib  |   | main.o  | | crtimp |
          +---------+ +---------+   +---------+ +--------+
Link...........|..........|..............|...........|.......
               |          |              +-----------+
               |          |              |
          +---------+     |         +---------+ +--------+
          |  main   |-----+         |  main   | | crtdll |
          +---------+               +---------+ +--------+
Load/Run.......|.........................|..........|........
          +---------+               +---------+     |
          | main in |               | main in |-----+
          | memory  |               | memory  |
          +---------+               +---------+

You can see in the static case that the main program and C runtime library are linked together at link time (by the developers). Since the user typically cannot re-link the executable, they're stuck with the behaviour of the library.

In the dynamic case, the main program is linked with the C runtime import library (something which declares what's in the dynamic library but doesn't actually define it). This allows the linker to link even though the actual code is missing.

Then, at runtime, the operating system loader does a late linking of the main program with the C runtime DLL (dynamic link library or shared library or other nomenclature).

The owner of the C runtime can drop in a new DLL at any time to provide updates or bug fixes. As stated earlier, this has both advantages and disadvantages.

---

Related Topics