C++11 Memory Pool Design Pattern

What are the usual im­ple­men­ta­tion de­tails be­hind mem­ory pools?

Any kind of "pool" is really just resources you've acquired/initialized in advance so that they're already ready to go, not allocated on the fly with each client request. When clients finish using them, the resource returns to the pool instead of being destroyed.

Memory pools are basically just memory you've allocated in advance (and typically in big blocks). For example, you might allocate 4 kilobytes of memory in advance. When a client requests 64 bytes of memory, you just hand them a pointer to an unused space in that memory pool for them to read and write whatever they want. When the client is done, you can just mark that section of memory as being unused again.

As a basic example which doesn't bother with alignment, safety, or returning unused (freed) memory back to the pool:

class MemoryPool
{
public:
    MemoryPool(): ptr(mem) 
    {
    }

    void* allocate(int mem_size)
    {
        assert((ptr + mem_size) <= (mem + sizeof mem) && "Pool exhausted!");
        void* mem = ptr;
        ptr += mem_size;
        return mem;
    }

private:
    MemoryPool(const MemoryPool&);
    MemoryPool& operator=(const MemoryPool&);   
    char mem[4096];
    char* ptr;
};

...
{
    MemoryPool pool;

    // Allocate an instance of `Foo` into a chunk returned by the memory pool.
    Foo* foo = new(pool.allocate(sizeof(Foo))) Foo;
    ...
    // Invoke the dtor manually since we used placement new.
    foo->~Foo();
}

This is effectively just pooling memory from the stack. A more advanced implementation might chain blocks together and do some branching to see if a block is full to avoid running out of memory, deal with fixed-size chunks that are unions (list nodes when free, memory for the client when used), and it definitely needs to deal with alignment (easiest way is just max align the memory blocks and add padding to each chunk to align the subsequent one).

More fancy would be buddy allocators, slabs, ones applying fitting algorithms, etc. Implementing an allocator is not so different from a data structure, but you get knee deep in raw bits and bytes, have to think about things like alignment, and can't shuffle contents around (can't invalidate existing pointers to memory being used). Like data structures, there isn't really a golden standard that says, "thou shalt do this". There's a wide variety of them, each with their own strengths and weaknesses, but there are some especially popular algorithms for memory allocation.

Implementing allocators is something that I would actually recommend to many C and C++ developers just to kind of get in tune with the way that memory management works a bit better. It can make you a bit more conscious of how the memory being requested connects to data structures using them, and also opens up a whole new door of optimization opportunities without using any new data structures. It can also make data structures like linked lists which are normally not very efficient much more useful and reduce temptations to make opaque/abstract types less opaque to avoid the heap overhead. However, there can be an initial excitement which might want to make you shoehorn custom allocators for everything, only to later regret the additional burden (especially if, in your excitement, you forget about issues like thread safety and alignment). It's worth taking it easy there. As with any micro-optimization, it's generally best applied discretely, in hindsight, and with a profiler in hand.

Memory management patterns

You could use a memory pool, the boost one is pretty good:
http://www.boost.org/doc/libs/1_58_0/libs/pool/doc/html/boost_pool/pool.html

And every client could configure the maximum size of the pool.
Allocations and deallocations will be really fast,and you will drop your factory implementation

Is an Object Pool pattern of shared_ptr possible?

After exhaustive research and testing I have concluded that there is no legitimate way (as of C++11 or below) to make an object pool of reusable shared_ptr<T>'s directly. Certainly one can make a pool of T objects quite easily that serves shared_ptr<T>'s, but that results in heap allocation with every serve for the control block.

It is possible however to make an object pool of shared_ptr<T>'s indirectly (and this is the only way I have found to do it). By indirectly, I mean one must implement a custom 'memory pool' style allocator to store for reuse the memory released when shared_ptr<T> control blocks are destroyed. This allocator is then used as the third parameter of the `shared_ptr' constructor:

template< class Y, class Deleter, class Alloc > 
   std::shared_ptr( Y* ptr, Deleter d, Alloc alloc );

The shared_ptr<T> will still be constructed/allocated and deleted/de-allocated with heap memory - there is no way to stop it - but by making the memory reusable through the custom allocator, a deterministic memory footprint can be achieved.

C++ object-pool that provides items as smart-pointers that are returned to pool upon deletion

Naive implementation

The implementation uses unique_ptr with a custom deleter that returns objects to the pool. Both acquire and release are O(1). Additionally, unique_ptr with custom deleters can be implicitly converted to shared_ptr.

template <class T>
class SharedPool
{
 public:
  using ptr_type = std::unique_ptr<T, std::function<void(T*)> >;

  SharedPool() {}
  virtual ~SharedPool(){}

  void add(std::unique_ptr<T> t) {
    pool_.push(std::move(t));
  }

  ptr_type acquire() {
    assert(!pool_.empty());
    ptr_type tmp(pool_.top().release(),
                 [this](T* ptr) {
                   this->add(std::unique_ptr<T>(ptr));
                 });
    pool_.pop();
    return std::move(tmp);
  }

  bool empty() const {
    return pool_.empty();
  }

  size_t size() const {
    return pool_.size();
  }

 private:
  std::stack<std::unique_ptr<T> > pool_;
};

Example usage:

SharedPool<int> pool;
pool.add(std::unique_ptr<int>(new int(42)));
pool.add(std::unique_ptr<int>(new int(84)));
pool.add(std::unique_ptr<int>(new int(1024)));
pool.add(std::unique_ptr<int>(new int(1337)));

// Three ways to express the unique_ptr object
auto v1 = pool.acquire();
SharedPool<int>::ptr_type v2 = pool.acquire();    
std::unique_ptr<int, std::function<void(int*)> > v3 = pool.acquire();

// Implicitly converted shared_ptr with correct deleter
std::shared_ptr<int> v4 = pool.acquire();

// Note that adding an acquired object is (correctly) disallowed:
// pool.add(v1);  // compiler error

---

You might have caught a severe problem with this implementation. The following usage isn't unthinkable:

  std::unique_ptr< SharedPool<Widget> > pool( new SharedPool<Widget> );
  pool->add(std::unique_ptr<Widget>(new Widget(42)));
  pool->add(std::unique_ptr<Widget>(new Widget(84)));

  // [Widget,42] acquired(), and released from pool
  auto v1 = pool->acquire();

  // [Widget,84] is destroyed properly, together with pool
  pool.reset(nullptr);

  // [Widget,42] is not destroyed, pool no longer exists.
  v1.reset(nullptr);
  // Memory leak

---

We need a way to keep alive information necessary for the deleter to make the distinction

1. Should I return object to pool?
2. Should I delete the actual object?

One way of doing this (suggested by T.C.), is having each deleter keep a weak_ptr to shared_ptr member in SharedPool. This lets the deleter know if the pool has been destroyed.

Correct implementation:

template <class T>
class SharedPool
{
 private:
  struct External_Deleter {
    explicit External_Deleter(std::weak_ptr<SharedPool<T>* > pool)
        : pool_(pool) {}

    void operator()(T* ptr) {
      if (auto pool_ptr = pool_.lock()) {
        try {
          (*pool_ptr.get())->add(std::unique_ptr<T>{ptr});
          return;
        } catch(...) {}
      }
      std::default_delete<T>{}(ptr);
    }
   private:
    std::weak_ptr<SharedPool<T>* > pool_;
  };

 public:
  using ptr_type = std::unique_ptr<T, External_Deleter >;

  SharedPool() : this_ptr_(new SharedPool<T>*(this)) {}
  virtual ~SharedPool(){}

  void add(std::unique_ptr<T> t) {
    pool_.push(std::move(t));
  }

  ptr_type acquire() {
    assert(!pool_.empty());
    ptr_type tmp(pool_.top().release(),
                 External_Deleter{std::weak_ptr<SharedPool<T>*>{this_ptr_}});
    pool_.pop();
    return std::move(tmp);
  }

  bool empty() const {
    return pool_.empty();
  }

  size_t size() const {
    return pool_.size();
  }

 private:
  std::shared_ptr<SharedPool<T>* > this_ptr_;
  std::stack<std::unique_ptr<T> > pool_;
};

---

Related Topics