Any Reason to Overload Global New and Delete

Any reason to overload global new and delete?

We overload the global new and delete operators where I work for many reasons:

pooling all small allocations -- decreases overhead, decreases fragmentation, can increase performance for small-alloc-heavy apps
framing allocations with a known lifetime -- ignore all the frees until the very end of this period, then free all of them together (admittedly we do this more with local operator overloads than global)
alignment adjustment -- to cacheline boundaries, etc
alloc fill -- helping to expose usage of uninitialized variables
free fill -- helping to expose usage of previously deleted memory
delayed free -- increasing the effectiveness of free fill, occasionally increasing performance
sentinels or fenceposts -- helping to expose buffer overruns, underruns, and the occasional wild pointer
redirecting allocations -- to account for NUMA, special memory areas, or even to keep separate systems separate in memory (for e.g. embedded scripting languages or DSLs)
garbage collection or cleanup -- again useful for those embedded scripting languages
heap verification -- you can walk through the heap data structure every N allocs/frees to make sure everything looks ok
accounting, including leak tracking and usage snapshots/statistics (stacks, allocation ages, etc)

The idea of new/delete accounting is really flexible and powerful: you can, for example, record the entire callstack for the active thread whenever an alloc occurs, and aggregate statistics about that. You could ship the stack info over the network if you don't have space to keep it locally for whatever reason. The types of info you can gather here are only limited by your imagination (and performance, of course).

We use global overloads because it's convenient to hang lots of common debugging functionality there, as well as make sweeping improvements across the entire app, based on the statistics we gather from those same overloads.

We still do use custom allocators for individual types too; in many cases the speedup or capabilities you can get by providing custom allocators for e.g. a single point-of-use of an STL data structure far exceeds the general speedup you can get from the global overloads.

Take a look at some of the allocators and debugging systems that are out there for C/C++ and you'll rapidly come up with these and other ideas:

valgrind
electricfence
dmalloc
dlmalloc
Application Verifier
Insure++
BoundsChecker
...and many others... (the gamedev industry is a great place to look)

(One old but seminal book is Writing Solid Code, which discusses many of the reasons you might want to provide custom allocators in C, most of which are still very relevant.)

Obviously if you can use any of these fine tools you will want to do so rather than rolling your own.

There are situations in which it is faster, easier, less of a business/legal hassle, nothing's available for your platform yet, or just more instructive: dig in and write a global overload.

Why would one replace default new and delete operators?

One may try to replace new and delete operators for a number of reasons, namely:

To Detect Usage Errors:

There are a number of ways in which incorrect usage of new and delete may lead to the dreaded beasts of Undefined Behavior & Memory leaks.
Respective examples of each are:

Using more than one delete on newed memory & not calling delete on memory allocated using new.

An overloaded operator new can keep a list of allocated addresses and the overloaded operator delete can remove addresses from the list, then it is easy to detect such usage errors.

Similarly, a variety of programming mistakes can lead to data overruns(writing beyond the end of an allocated block) and underruns(writing prior to the beginning of an allocated block).

An Overloaded operator new can over-allocate blocks and put known byte patterns ("signatures") before and after the memory made available to clients. The overloaded operator deletes can check to see if the signatures are still intact.
Thus by checking if these signatures are not intact it is possible to determine that an overrun or under-run occurred sometime during the life of the allocated block, and operator delete can log that fact, along with the value of the offending pointer, thus helping in providing a good diagnostic information.

To Improve Efficiency(speed & memory):

The new and delete operators work reasonably well for everybody, but optimally for nobody. This behavior arises from the fact that they are designed for general purpose use only. They have to accommodate allocation patterns ranging from the dynamic allocation of a few blocks that exist for the duration of the program to constant allocation and deallocation of a large number of short-lived objects. Eventually, the operator new and operator delete that ship with compilers take a middle-of-the-road strategy.

If you have a good understanding of your program's dynamic memory usage patterns, you can often find that custom versions of operator new and operator delete outperform (faster in performance, or require less memory up to 50%)the default ones. Of course, unless you are sure of what you are doing it is not a good idea to do this(don't even try this if you don't understand the intricacies involved).

To Collect Usage Statistics:

Before thinking of replacing new and delete for improving efficiency as mentioned in #2, You should gather information about how your application/program uses dynamic allocation. You may want to collect information about:

Distribution of allocation blocks,

Distribution of lifetimes,

Order of allocations(FIFO or LIFO or random),

Understanding usage patterns changes over a period of time,maximum amount of dynamic memory used etc.

Also, sometimes you may need to collect usage information such as:

Count the number of dynamically objects of a class,

Restrict the number of objects being created using dynamic allocation etc.

All, this information can be collected by replacing the custom new and delete and adding the diagnostic collection mechanism in the overloaded new and delete.

To compensate for suboptimal memory alignment in `new`:

Many computer architectures require that data of particular types be placed in memory at particular kinds of addresses. For example, an architecture might require that pointers occur at addresses that are a multiple of four (i.e., be four-byte aligned) or that doubles must occur at addresses that are a multiple of eight (i.e., be eight-byte aligned). Failure to follow such constraints can lead to hardware exceptions at run-time. Other architectures are more forgiving, and may allow it to work though reducing the performance.The operator new that ship with some compilers don't guarantee eight-byte alignment for dynamic
allocations of doubles. In such cases, replacing the default operator new with one that guarantees eight-byte alignment could yield big increases in program performance & can be a good reason to replace new and delete operators.

To cluster related objects near one another:

If you know that particular data structures are generally used together and you'd like to minimize the frequency of page faults when working on the data, it can make sense to create a separate heap for the data structures so they are clustered together on as few pages as possible. custom Placement versions of new and delete can make it possible to achieve such clustering.

To obtain unconventional behavior:

Sometimes you want operators new and delete to do something that the compiler-provided versions don't offer.

For example: You might write a custom operator delete that overwrites deallocated memory with zeros in order to increase the security of application data.

What is the benefit of overloading 'new' and 'delete' for custom memory allocation in game engines?

I'm failing to see what the great benefit is of overloading and overriding these operators.

There's the obvious point: you can't escape it (not without work).

If you want all your memory allocation to go through one location, all it takes for someone to screw that up is this:

new Foo();

However, if you globally override operators new and delete, they have to do more work:

void * ptr = malloc(sizeof(Foo));
new(ptr) Foo;

There is also the point that the vast majority of C++ programmers are used to new and delete, which they will use by default. If you require them to use a function, that becomes something you have to teach them and they have to learn. And if they make a mistake and use the standard way, see above.

Oh, and then there's the syntax:

customNew(className, size, desc, file, line);

That's not going to work. That may allocate memory, but it will not construct a type. It is no different from malloc. So the very name is a lie; it's not "new"-ing anything. It's just allocating.

You would need to use C++11 variadic templates and in-place construction to do things correctly:

template<typename T, typename ...Args>
T *customNew<T>(const char *className, const char *desc, size_t file, size_t line, Args&&... args)
{
  void* ptr = allocate_memory(className, sizeof(T), desc, file, line);
  return new(ptr) T(std::forward<Args>(args)...);
}

Personally, I'd say that the new(...) T(...) syntax is much cleaner than the customNew<T>(...) version. It's clear which parameters go to the allocator, and which parameters go to the type itself. It's certainly more standard.

But WHERE do I override the global new and delete

Globally. As in the global namespace. You stick the overloads in a header somewhere and include it everywhere.

a library forces global overloads of new/delete on me!

If you're compiling in (via header inclusion) an overridden new/delete operator(s), then all calls in your code to new/delete will use them. There is no way to re-override it (link errors) or only partially override it, etc.

It is bad form to override the global new/delete operators, at all. It's a bad idea. If you don't realize why it's a bad idea, you're not qualified to do so. If you do realize why it's a bad idea, you're qualified to do so, but you'll generally choose not to.

Defining a global new/delete is exponentially more evil in a component you expect people to include directly into their project. It is your job as a customer to help the vendor doing this understand the seriousness of the situation, or stop being their customer.

You can define a custom allocator type (see this link for a good tutorial on how to do so, the interface needed, etc) and use that exclusively with your STL types (it's a template argument).

For shared_ptr, you need to do something a little different: it takes a deleter object as a parameter to the constructor if you don't want the default "delete p" behavior. This isn't a custom allocator; it's just a regular unary functor.

Why would I overload operator new or operator delete?

The point of overloading is to control the memory allocation. In some cases you want to use your own allocators instead of the standard ones (for example, when you want allocations to be from static pools and not directly from the heap).

Why is overriding both the global new operator and the class-specific operator not ambiguous behaviour?

Why doesn't it result in a compile error? Does the compiler treat the new operator differently? (Any citation to the standard would be appreciated)

Regarding the precedence between global new and class specific new, the reference says this:

As described in allocation function, the C++ program may provide global and class-specific replacements for these functions. If the new-expression begins with the optional :: operator, as in ::new T or ::new T[n], class-specific replacements will be ignored (the function is looked up in global scope). Otherwise, if T is a class type, lookup begins in the class scope of T.

So class specific new has priority.

Regarding the overload of +, you can either have the member overload or the global overload (usually as a friend of the class) but not both because of the ambiguity it produces.

Overloading global operator new/delete in C++

If you want to overload the global operator new and operator delete, you just need to implement it. You don't need to explicitly define it everywhere since it already is defined as part of the language.

Edit: If you want to define an operator new that takes different parameters, then you'll need to #include it everywhere. However you do that is up to you; it's mostly a matter of style.

And don't forget to implement all the variants of global operator new and delete: new, new[], delete, delete[], and the std::nothrow variants.

Is it possible to overload new operator for allocating something like 2d array in C++?

Is it possible to overload new operator for allocating something like 2d array in C++?

Yes.

but can I get some code example?

Example:

std::unique_ptr<int[][10]> arr {new int[n][10]};

with given "height" and "width"?

Only if the inner dimensions are compile time constant. Only the outer dimension may be dynamic.

It is easy to translate between a dynamic single dimensional flat array, and a multi dimensional one.

Any Reason to Overload Global New and Delete