How to Write Iso C++ Standard Conformant Custom New and Delete Operators

How should I write ISO C++ Standard conformant custom new and delete operators?

Part I

This C++ FAQ entry explained why one might want to overload new and delete operators for one's own class. This present FAQ tries to explain how one does so in a standard-conforming way.

Implementing a custom new operator

The C++ standard (§18.4.1.1) defines operator new as:

void* operator new (std::size_t size) throw (std::bad_alloc);

The C++ standard specifies the semantics that custom versions of these operators have to obey in §3.7.3 and §18.4.1

Let us summarize the requirements.

Requirement #1: It should dynamically allocate at least size bytes of memory and return a pointer to the allocated memory. Quote from the C++ standard, section 3.7.4.1.3:

The allocation function attempts to allocate the requested amount of storage. If it is successful, it shall return the address of the start of a block of storage whose length in bytes shall be at least as large as the requested size...

The standard further imposes:

...The pointer returned shall be suitably aligned so that it can be converted to a pointer of any complete object type and then used to access the object or array in the storage allocated (until the storage is explicitly deallocated by a call to a corresponding deallocation function). Even if the size of the space requested is zero, the request can fail. If the request succeeds, the value returned shall be a non-null pointer value (4.10) p0 different from any previously returned value p1, unless that value p1 was sub-sequently passed to an operator delete.

This gives us further important requirements:

Requirement #2: The memory allocation function we use (usually malloc() or some other custom allocator) should return a suitably aligned pointer to the allocated memory, which can be converted to a pointer of an complete object type and used to access the object.

Requirement #3: Our custom operator new must return a legitimate pointer even when zero bytes are requested.

One of the evident requirements that can even be inferred from new prototype is:

Requirement #4: If new cannot allocate dynamic memory of the requested size, then it should throw an exception of type std::bad_alloc.

But! There is more to that than what meets the eye: If you take a closer look at the new operator documentation (citation from standard follows further down), it states:

If set_new_handler has been used to define a new_handler function, this new_handler function is called by the standard default definition of operator new if it cannot allocate the requested storage by its own.

To understand how our custom new needs to support this requirement, we should understand:

What is the new_handler and set_new_handler?

new_handler is a typedef for a pointer to a function that takes and returns nothing, and
set_new_handler is a function that takes and returns a new_handler.

set_new_handler's parameter is a pointer to the function operator new should call if it can't allocate the requested memory. Its return value is a pointer to the previously registered handler function, or null if there was no previous handler.

An opportune moment for an code sample to make things clear:

#include <iostream>
#include <cstdlib>

// function to call if operator new can't allocate enough memory or error arises
void outOfMemHandler()
{
    std::cerr << "Unable to satisfy request for memory\n";

    std::abort();
}

int main()
{
    //set the new_handler
    std::set_new_handler(outOfMemHandler);

    //Request huge memory size, that will cause ::operator new to fail
    int *pBigDataArray = new int[100000000L];

    return 0;
}

In the above example, operator new (most likely) will be unable to allocate space for 100,000,000 integers, and the function outOfMemHandler() will be called, and the program will abort after issuing an error message.

It is important to note here that when operator new is unable to fulfill a memory request, it calls the new-handler function repeatedly until it can find enough memory or there is no more new handlers. In the above example, unless we call std::abort(), outOfMemHandler() would be called repeatedly. Therefore, the handler should either ensure that the next allocation succeeds, or register another handler, or register no handler, or not return (i.e. terminate the program). If there is no new handler and the allocation fails, the operator will throw an exception.

Continuation 1

---

If I write operators new and delete for a class, do I have to write all of their overloads?

No, you don't need to write all variations of the new and delete operators for your class.

There are multiple reasons to prefer some versions of new and delete over others. I will describe each reason separately.

Almost always prefer the delete operators that have a size parameter over those without one.

When I write delete operators for a base class that provides memory handling for other classes, I use these versions of the delete operators

void T::operator delete  ( void* ptr, std::size_t sz );
void T::operator delete[]( void* ptr, std::size_t sz );
void T::operator delete  ( void* ptr, std::size_t sz, std::align_val_t al ); // (since C++17)
void T::operator delete[]( void* ptr, std::size_t sz, std::align_val_t al ); // (since C++17)

and deliberately omit or =delete these versions.

void T::operator delete  ( void* ptr );
void T::operator delete[]( void* ptr );
void T::operator delete  ( void* ptr, std::align_val_t al ); // (since C++17)
void T::operator delete[]( void* ptr, std::align_val_t al ); // (since C++17)

The reason is that the std::size_t sz parameter tells me the size of the object or size of the array. I can't know the sizes of the derived class' objects when I write my base class, so using the size parameter helps. Some of my memory handlers segregate the objects by size (easier to pool memory when all the chunks are the same size). I can use the size parameter to quickly choose which memory pool to search, rather than searching all of them. That turns a O(n) algorithm into a O(1) action.

Some of my memory allocators use a "chain model" instead of a "block model", and the size parameter helps for deleting there too. (I call a memory allocator a "block model" if it preallocates a huge chunk and then partitions the chunk into separate blocks like an array. I call a memory handler a "chain model" if each chunk points to previous and next chunks like a linked list or chain.) So when somebody deletes a chunk from a chain of memory chunks, I want the delete operator to know the chunk being deleted is the correct size. I can put an assertion in the delete operation that asserts (size == address of next chunk - address of this chunk).

Where appropriate, prefer the new and delete operators with the alignment parameter.

Now that C++17 provides an alignment parameter for new operators, use them if you need them. If you need performance, align your objects on 4, 8, or 16 byte boundaries, do so! It makes the program a little faster.

So let's say you have an alignment-aware memory allocator. It knows that some objects are best stored on 4 byte boundaries because those objects are small and you can squeeze more into memory if you use 4 byte boundaries. It also knows some objects are best aligned on 8 byte boundaries because those objects are used often.

Your memory handler will know this if it provides the correct new operators and derived classes provide the correct values for alignments.

The 2017 C++ Standard says:

When allocating objects and arrays of objects whose alignment exceeds STDCPP_DEFAULT_NEW_ALIGNMENT, overload resolution is performed twice: first, for alignment-aware function signatures, then for alignment-unaware function signatures. This means that if a class with extended alignment has an alignment-unaware class-specific allocation function, it is the function that will be called, not the global alignment-aware allocation function. This is intentional: the class member is expected to know best how to handle that class.

This means the compiler will check for new and delete operators with the alignment parameter, and then check for operators without the alignment parameter.

If you have an alignment-aware memory handler, then always provide these new operators, even if you also want to give your client code the option of ignoring alignment.

void* T::operator new  ( std::size_t count, std::align_val_t al ); // (since C++17)
void* T::operator new[]( std::size_t count, std::align_val_t al ); // (since C++17)
void* T::operator new  ( std::size_t count,
    std::align_val_t al, user-defined-args... ); // (since C++17)
void* T::operator new[]( std::size_t count,
     std::align_val_t al, user-defined-args... ); // (since C++17)

You can force code to provide the alignment parameter if you provide the above new operators and omit or =delete these overloads.

void* T::operator new  ( std::size_t count );
void* T::operator new[]( std::size_t count );

void* T::operator new  ( std::size_t count, user-defined-args... );
void* T::operator new[]( std::size_t count, user-defined-args... );

Use class specific placement-new operators to provide hints.

Let's say you wrote a class that allocates several data members, and you want all those data members to be located on the same memory page. If the data is spread across several memory pages, the CPU will have to load different memory pages into the L1 or L2 cache just so you can access the member data for an object. If your memory handler can place all of an object's data members onto the same page, then your program will run faster because the CPU will not need to load multiple pages into cache.

These are the class specific placement new operators.

void* T::operator new  ( std::size_t count, user-defined-args... );
void* T::operator new[]( std::size_t count, user-defined-args... );
void* T::operator new  ( std::size_t count,
    std::align_val_t al, user-defined-args... ); // (since C++17)
void* T::operator new[]( std::size_t count,
     std::align_val_t al, user-defined-args... ); // (since C++17)

Overload them to look like this by providing a hint parameter.

void* T::operator new  ( std::size_t count, void* hint );
void* T::operator new[]( std::size_t count, void* hint );
void* T::operator new  ( std::size_t count, std::align_val_t al, void* hint ); // (since C++17)
void* T::operator new[]( std::size_t count, std::align_val_t al, void* hint ); // (since C++17)

The hint parameter tells the memory handler to try to place the object not at the location of that hint address, but on the same page as the hint address.

Now you can write a class that looks like this which is derived from your memory handling class.

class Foo : public MemoryHandler
{
public:
    Foo();
    ...
private:
    Blah * b_;
    Wham * f_;
};

Foo::Foo() : b_( nullptr ), f_( nullptr )
{
    // This should put data members on the same memory page as this Foo object.
    b_ = new ( this ) Blah;
    f_ = new ( this ) Wham;
}

Why would one replace default new and delete operators?

One may try to replace new and delete operators for a number of reasons, namely:

To Detect Usage Errors:

There are a number of ways in which incorrect usage of new and delete may lead to the dreaded beasts of Undefined Behavior & Memory leaks.
Respective examples of each are:

Using more than one delete on newed memory & not calling delete on memory allocated using new.

An overloaded operator new can keep a list of allocated addresses and the overloaded operator delete can remove addresses from the list, then it is easy to detect such usage errors.

Similarly, a variety of programming mistakes can lead to data overruns(writing beyond the end of an allocated block) and underruns(writing prior to the beginning of an allocated block).

An Overloaded operator new can over-allocate blocks and put known byte patterns ("signatures") before and after the memory made available to clients. The overloaded operator deletes can check to see if the signatures are still intact.
Thus by checking if these signatures are not intact it is possible to determine that an overrun or under-run occurred sometime during the life of the allocated block, and operator delete can log that fact, along with the value of the offending pointer, thus helping in providing a good diagnostic information.

---

To Improve Efficiency(speed & memory):

The new and delete operators work reasonably well for everybody, but optimally for nobody. This behavior arises from the fact that they are designed for general purpose use only. They have to accommodate allocation patterns ranging from the dynamic allocation of a few blocks that exist for the duration of the program to constant allocation and deallocation of a large number of short-lived objects. Eventually, the operator new and operator delete that ship with compilers take a middle-of-the-road strategy.

If you have a good understanding of your program's dynamic memory usage patterns, you can often find that custom versions of operator new and operator delete outperform (faster in performance, or require less memory up to 50%)the default ones. Of course, unless you are sure of what you are doing it is not a good idea to do this(don't even try this if you don't understand the intricacies involved).

---

To Collect Usage Statistics:

Before thinking of replacing new and delete for improving efficiency as mentioned in #2, You should gather information about how your application/program uses dynamic allocation. You may want to collect information about:

Distribution of allocation blocks,

Distribution of lifetimes,

Order of allocations(FIFO or LIFO or random),

Understanding usage patterns changes over a period of time,maximum amount of dynamic memory used etc.

Also, sometimes you may need to collect usage information such as:

Count the number of dynamically objects of a class,

Restrict the number of objects being created using dynamic allocation etc.

All, this information can be collected by replacing the custom new and delete and adding the diagnostic collection mechanism in the overloaded new and delete.

---

To compensate for suboptimal memory alignment in new:

Many computer architectures require that data of particular types be placed in memory at particular kinds of addresses. For example, an architecture might require that pointers occur at addresses that are a multiple of four (i.e., be four-byte aligned) or that doubles must occur at addresses that are a multiple of eight (i.e., be eight-byte aligned). Failure to follow such constraints can lead to hardware exceptions at run-time. Other architectures are more forgiving, and may allow it to work though reducing the performance.The operator new that ship with some compilers don't guarantee eight-byte alignment for dynamic
allocations of doubles. In such cases, replacing the default operator new with one that guarantees eight-byte alignment could yield big increases in program performance & can be a good reason to replace new and delete operators.

---

To cluster related objects near one another:

If you know that particular data structures are generally used together and you'd like to minimize the frequency of page faults when working on the data, it can make sense to create a separate heap for the data structures so they are clustered together on as few pages as possible. custom Placement versions of new and delete can make it possible to achieve such clustering.

---

To obtain unconventional behavior:

Sometimes you want operators new and delete to do something that the compiler-provided versions don't offer.

For example: You might write a custom operator delete that overwrites deallocated memory with zeros in order to increase the security of application data.

Using placement new in generic programming

(This answer uses N4659, the final C++17 draft.)

Why ::new is used instead of just new

::new ensures that the operator new is looked up in the global scope. In contrast, the plain new first looks up in the scope of the class if T is a class type (or array thereof), and only then falls back to the global scope. Per [expr.new]/9:

If the new-expression begins with a unary ​::​ operator, the
allocation function's name is looked up in the global scope.
Otherwise, if the allocated type is a class type T or array thereof,
the allocation function's name is looked up in the scope of T. If
this lookup fails to find the name, or if the allocated type is not a
class type, the allocation function's name is looked up in the global
scope.

For example, with

struct C {
    void* operator new(std::size_t, void* ptr) noexcept
    {
        std::cout << "Hello placement new!\n";
        return ptr;
    }
};

The plain new will cause this function to be found, thus printing unwanted message, whereas ::new will still find the global function and work properly.

The global operator new(std::size_t, void*) cannot be replaced because of [new.delete.placement]/1:

These functions are reserved; a C++ program may not define functions that displace the versions in the C++ standard library
([constraints]). The provisions of [basic.stc.dynamic] do not apply
to these reserved placement forms of operator new and operator
delete.

(See How should I write ISO C++ Standard conformant custom new and delete operators? for more about overloading operator new.)

Why an explicit cast to void* is required

Although the global operator new(std::size_t, void*) may not be replaced, new versions of ::operator new can be defined. For example, suppose that the following declaration is placed in the global scope:

void* operator new(std::size_t, int* ptr) noexcept
{
    std::cout << "Hello placement new!\n";
    return ptr;
}

Then ::new(ptr) T will use this version instead of the global version, where ptr is a int* value. The pointer is explicitly cast to void* to ensure that the void* version of operator new (which we intend to call) wins in overload resolution.

---

From comment:

But why do we want to call exactly global new for void* if some
type has special overload of new for itself? Seems like normal
overloaded operator is more suitable - why it's not?

Normally, new is used for allocation purposes. Allocation is something the user should have control over. The user can roll out more suitable versions for a normal new.

In this case, however, we don't want to allocate anything — all we want to do is create an object! The placement new is more of a "hack" — its presence is largely due to the lack of syntax available for constructing an object at a specified address. We don't want the user to be able to customize anything. The language itself, however, doesn't care about this hack, though — we have to treat it specially. Of course, if we have something like construct_at (which is coming in C++20), we will use it!

Also note that std::uninitialized_copy is intended for the simplest case where you just want to copy construct a sequence of objects in raw allocated space. The standard containers allow you to customize not only how the elements are allocated, but also how they are constructed, by means of an allocator. Therefore, they do not generally use std::uninitialized_copy for their elements — they call std::allocator_traits<Allocator>::construct. This feature is used by std::scoped_allocator_adaptor.

C++ new, delete, new[] and delete[]

The “function redefinition error” is probably because you’re using MFC.

The runtime library should not be in the business of defining such allocation and deallocation functions.

The current code’s

struct MemLeakInfo
{
        unsigned int addr;
        unsigned int line;
        unsigned int size;
        unsigned char file;
};

is ungood. An unsigned int is not guaranteed to be large enough to hold an address, even though it is in 32-bit Windows. Instead, use intptr_t.

Also, the current code’s

void* operator new(unsigned int Size, int Line, const char* File);

is ungood. That should be …

void* operator new( size_t Size, int Line, const char* File );

And you need a corresponding operator delete, like …

void* operator delete( void* p, int Line, const char* File );

in order to deallocate memory from a failed constructor call. It's only called in that very specific situation. But if you don’t have it, then you have a leak, as MFC once had for debug builds.

---

EDIT: fixed versions of the code you now provided:

file [minimal.h]:

• _MINIMAL_H is invalid, because it starts with underscore followed by uppercase letter, which is reserved. Changed to MINIMAL_H.
• to use size_t you need to include <stddef.h>.
• _DEBUG is not a standard macro. it's a microsoftism. the standard macro (look up documentation of assert) for this purpose is NDEBUG.

 

#ifndef MINIMAL_H
#define MINIMAL_H

#include <stddef.h>     // std::size_t

#ifndef NDEBUG

void* operator new( size_t Size, int Line, const char* File );
void* operator new[]( size_t Size, int Line, const char* File );

void operator delete( void* ptr, int Line, const char* File );
void operator delete[]( void* ptr, int Line, const char* File ); 

#endif 

#ifndef NDEBUG

#define DEBUG_NEW new( __LINE__, __FILE__ )

#else

#define DEBUG_NEW new

#endif 

#endif //MINIMAL_H

file [minimal.cpp]:

• to use malloc you need to include stdlib.h.
• when you #define new you’re wreaking havoc with the new keywords in the following code.
• in C you should never cast the result of malloc, and in C++ you should only cast something when there is a need for it. there is no such need here. casts only mask bugs, and that's not a good idea.
• missing return in error case. for error you need to throw std::bad_alloc. that's by the holy standard’s specification of these functions.

 

#include "Minimal.h"

//#define new DEBUG_NEW

#ifndef NDEBUG
#include <stdlib.h>     // malloc
#include <exception>    // std::bad_alloc

void* operator new( size_t const size, int, const char*)
{
    void* const ptr = malloc( size );

    if( !ptr ) { throw std::bad_alloc(); }
    return ptr;
};

void* operator new[]( size_t const size, int, const char*)
{
    void* const ptr = malloc( size );

    if( !ptr ) { throw std::bad_alloc(); }
    return ptr;
}

void operator delete(void* const ptr, int, const char*)
{
    free( ptr ); 
};

void operator delete[]( void* const ptr, int, const char* )
{
    free( ptr ); 
}; 

#endif

---

Related Topics