Is the Pimpl Idiom Really Used in Practice

Is the PIMPL idiom really used in practice?

So, I am wondering it this technique is really used in practice? Should I use it everywhere, or with caution?

Of course it is used. I use it in my project, in almost every class.

---

Reasons for using the PIMPL idiom:

Binary compatibility

When you're developing a library, you can add/modify fields to XImpl without breaking the binary compatibility with your client (which would mean crashes!). Since the binary layout of X class doesn't change when you add new fields to Ximpl class, it is safe to add new functionality to the library in minor versions updates.

Of course, you can also add new public/private non-virtual methods to X/XImpl without breaking the binary compatibility, but that's on par with the standard header/implementation technique.

Data hiding

If you're developing a library, especially a proprietary one, it might be desirable not to disclose what other libraries / implementation techniques were used to implement the public interface of your library. Either because of Intellectual Property issues, or because you believe that users might be tempted to take dangerous assumptions about the implementation or just break the encapsulation by using terrible casting tricks. PIMPL solves/mitigates that.

Compilation time

Compilation time is decreased, since only the source (implementation) file of X needs to be rebuilt when you add/remove fields and/or methods to the XImpl class (which maps to adding private fields/methods in the standard technique). In practice, it's a common operation.

With the standard header/implementation technique (without PIMPL), when you add a new field to X, every client that ever allocates X (either on stack, or on heap) needs to be recompiled, because it must adjust the size of the allocation. Well, every client that doesn't ever allocate X also need to be recompiled, but it's just overhead (the resulting code on the client side will be the same).

What is more, with the standard header/implementation separation XClient1.cpp needs to be recompiled even when a private method X::foo() was added to X and X.h changed, even though XClient1.cpp can't possibly call this method for encapsulation reasons! Like above, it's pure overhead and is related with how real-life C++ build systems work.

Of course, recompilation is not needed when you just modify the implementation of the methods (because you don't touch the header), but that's on par with the standard header/implementation technique.

---

Is this technique recommended to be used in embedded systems (where the performance is very important)?

That depends on how powerful your target is. However the only answer to this question is: measure and evaluate what you gain and lose. Also, take into consideration that if you're not publishing a library meant to be used in embedded systems by your clients, only the compilation time advantage applies!

The Pimpl Idiom in practice

I'd say that whether you do it per-class or on an all-or-nothing basis depends on why you go for the pimpl idiom in the first place. My reasons, when building a library, have been one of the following:

• Wanted to hide implementation in order to avoid disclosing information (yes, it was not a FOSS project :)
• Wanted to hide implementation in order to make client code less dependent. If you build a shared library (DLL), you can change your pimpl class without even recompiling the application.
• Wanted to reduce the time it takes to compile the classes using the library.
• Wanted to fix a namespace clash (or similar).

None of these reasons prompts for the all-or-nothing approach. In the first one, you only pimplize what you want to hide, whereas in the second case it's probably enough to do so for classes which you expect to change. Also for the third and fourth reason there's only benefit from hiding non-trivial members that in turn require extra headers (e.g., of a third-party library, or even STL).

In any case, my point is that I wouldn't typically find something like this too useful:

class Point {
  public:      
    Point(double x, double y);
    Point(const Point& src);
    ~Point();
    Point& operator= (const Point& rhs);

    void setX(double x);
    void setY(double y);
    double getX() const;
    double getY() const;

  private:
    class PointImpl;
    PointImpl* pimpl;
}

In this kind of a case, the tradeoff starts to hit you because the pointer needs to be dereferenced, and the methods cannot be inlined. However, if you do it only for non-trivial classes then the slight overhead can typically be tolerated without any problems.

Why should the PIMPL idiom be used?

• Because you want Purr() to be able to use private members of CatImpl. Cat::Purr() would not be allowed such an access without a friend declaration.
• Because you then don't mix responsibilities: one class implements, one class forwards.

Is pimpl idiom better than using always unique_ptr as member variables?

After a while I have a broader understanding of the problem and finally I can answer to my own question.

It turned out that what I was saying was not completely correct.

In fact in the code below only Bp class is pImpl. If we change Ap to be pImpl aswell we obtain that, if we change Bp.h we need to recompile only Ap.cpp, Bp.cpp, which is the same of the corresponding solution with unique_ptrs.

Said that, I think I can say that the solution with pImpl seems in general better than the solution with unique_ptrs (we just have to pImpl the correct classes!).

For this reason we decided to switch to pImpl idiom as default for our classes.

Using the PIMPL idiom, should the implementation always be a private member of the class?

This is, obviously, a matter of opinion.

I think using the second approach goes against the spirit of the Pimpl idiom.

1. The details of the implementation are now visible to the outside.
2. If any changes are made to the interface of Example, it will most likely affect four files instead of two.
3. If any changes are made to the way ExampleImpl is implemented, it will most likely affect three files instead of one.

Given the above points, I would recommend using the nested class approach.

Pimpl idiom usage in Qt, searching for laconic way

Introduction

I know about Q_DECLARE_PRIVATE, Q_D and other macros

You know about them, but have you actually used them and understand their purpose, and - for the most part - their inevitability? Those macros weren't added to make stuff verbose. They are there because you end up needing them.

There are no differences in Qt PIMPL implementation between Qt versions, but you are depending on Qt's implementation details when you inherit from QClassPrivate, should you do so. The PIMPL macros have nothing to do with moc. You can use them in plain C++ code that doesn't use any Qt classes at all.

Alas, there's no escaping what you want for as long as you implement the PIMPLs the usual way (which is also Qt way).

Pimpl-pointer vs this

First of all, let's observe that impl stands for this, but the language lets you skip using this-> in most cases. Thus, it's nothing too foreign.

class MyClassNoPimpl {
  int foo;
public:
  void setFoo(int s) { this->foo = s; }
};

class MyClass {
  struct MyClassPrivate;
  QScopedPointer<MyClassPrivate> const d;
public:
  void setFoo(int s);
  ...
  virtual ~MyClass();
};

void MyClass::setFoo(int s) { d->foo = s; }

Inheritance demands...

Things become generally outlandish when you have inheritance, though:

class MyDerived : public MyClass {
  class MyDerivedPrivate;
  QScopedPointer<MyDerivedPrivate> const d;
public:
  void SetBar(int s);
};

void MyDerived::setFooBar(int f, int b) {
  MyClass::d->foo = f;
  d->bar = b;
}

You'll want to re-use a single d-pointer in the base class, but it will have the wrong type in all derived classes. Thus you might think of casting it - that's even more boilerplate! Instead, you use a private function that returns a correctly-cast d-pointer. Now you need to derive both public and private classes, and you need private headers for the private classes, so that the derived classes can use them. Oh, and you need to pass the pointer to the derived pimpl to the base class - because that's the only way you can initialize the d_ptr while keeping it const, as it must be. See - Qt's PIMPL implementation is verbose because you do actually need all of it to write safe, composable, maintainable code. No way around it.

MyClass1.h

class MyClass1 {
protected:
  struct Private;
  QScopedPointer<Private> const d_ptr;
  MyClass1(Private &); // a signature that won't clash with anything else
private:
  inline Private *d() { return (Private*)d_ptr; }
  inline const Private *d() const { return (const Private*)d_ptr; }
public:
  MyClass1();
  virtual ~MyClass1();
  void setFoo(int);
};

MyClass1_p.h

struct MyClass1::Private {
  int foo;
};

MyClass1.cpp

#include "MyClass1.h"
#include "MyClass1_p.h"

MyClass1::MyClass1(Private &p) : d_ptr(&p) {}

MyClass1::MyClass1() : d_ptr(new Private) {}    

MyClass1::~MyClass1() {} // compiler-generated

void MyClass1::setFoo(int f) {
  d()->foo = f;
}

MyClass2.h

#include "MyClass1.h"

class MyClass2 : public MyClass1 {
protected:
  struct Private;
private:
  inline Private *d() { return (Private*)d_ptr; }
  inline const Private *d() { return (const Private*)d_ptr; }
public:
  MyClass2();
  ~MyClass2() override; // Override ensures that the base had a virtual destructor.
                        // The virtual keyword is not used per DRY: override implies it.
  void setFooBar(int, int);
};

MyClass2_p.h

#include "MyClass1_p.h"

struct MyClass2::Private : MyClass1::Private {
  int bar;
};

MyClass2.cpp

MyClass2::MyClass2() : MyClass1(*new Private) {}

MyClass2::~MyClass2() {}

void MyClass2::setFooBar(int f, int b) {
  d()->foo = f;
  d()->bar = b;
}

Inheritance, Qt way

Qt's PIMPL macros take care of implementing d() functions. Well, they implement d_func() and then you use the Q_D macro to obtain a local variable that is simply d. Rewriting the above:

MyClass1.h

class MyClass1Private;
class MyClass1 {
  Q_DECLARE_PRIVATE(MyClass1)
protected:
  QScopedPointer<Private> d_ptr;
  MyClass1(MyClass1Private &);
public:
  MyClass1();
  virtual ~MyClass1();
  void setFoo(int);
};

MyClass1_p.h

struct MyClass1Private {
  int foo;
};

MyClass1.cpp

#include "MyClass1.h"
#include "MyClass1_p.h"

MyClass1::MyClass1(MyClass1Private &d) : d_ptr(*d) {}

MyClass1::MyClass1() : d_ptr(new MyClass1Private) {}  

MyClass1::MyClass1() {}

void MyClass1::setFoo(int f) {
  Q_D(MyClass1);
  d->foo = f;
}

MyClass2.h

#include "MyClass1.h"

class MyClass2Private;
class MyClass2 : public MyClass1 {
  Q_DECLARE_PRIVATE(MyClass2)
public:
  MyClass2();
  ~MyClass2() override;
  void setFooBar(int, int);
};

MyClass2_p.h

#include "MyClass1_p.h"

struct MyClass2Private : MyClass1Private {
  int bar;
};

MyClass2.cpp

MyClass2() : MyClass1(*new MyClass2Private) {}

MyClass2::~MyClass2() {}

void MyClass2::setFooBar(int f, int b) {
  Q_D(MyClass2);
  d->foo = f;
  d->bar = b;
}

Factories simplify pimpl

For class hierarchies that are sealed (i.e. where the user doesn't derive), the interface can be sanitized from any private details whatsoever by the use of factories:

Interfaces

class MyClass1 {
public:
  static MyClass1 *make();
  virtual ~MyClass1() {}
  void setFoo(int);
};

class MyClass2 : public MyClass1 {
public:
  static MyClass2 *make();
  void setFooBar(int, int);
};

class MyClass3 : public MyClass2 {
public:
  static MyClass3 *make();
  void setFooBarBaz(int, int, int);
};

Implementations

template <class R, class C1, class C2, class ...Args, class ...Args2> 
R impl(C1 *c, R (C2::*m)(Args...args), Args2 &&...args) {
  return (*static_cast<C2*>(c).*m)(std::forward<Args2>(args)...);
}

struct MyClass1Impl {
  int foo;
};
struct  MyClass2Impl : MyClass1Impl {
  int bar;
};
struct MyClass3Impl : MyClass2Impl {
  int baz;
};

struct MyClass1X : MyClass1, MyClass1Impl {
   void setFoo(int f) { foo = f; }
};
struct MyClass2X : MyClass2, MyClass2Impl {
   void setFooBar(int f, int b) { foo = f; bar = b; }
};
struct MyClass3X : MyClass3, MyClass3Impl {
   void setFooBarBaz(int f, int b, int z) { foo = f; bar = b; baz = z;}
};

MyClass1 *MyClass1::make() { return new MyClass1X; }
MyClass2 *MyClass2::make() { return new MyClass2X; }
MyClass3 *MyClass3::make() { return new MyClass3X; }

void MyClass1::setFoo(int f) { impl(this, &MyClass1X::setFoo, f); }
void MyClass2::setFooBar(int f, int b) { impl(this, &MyClass2X::setFooBar, f, b); }
void MyClass3::setFooBarBaz(int f, int b, int z) { impl(this, &MyClass3X::setFooBarBaz, f, b, z); }

This is very basic sketch that should be further refined.

How does the pimpl idiom reduce dependencies?

The main advantage is that the clients of the interface aren't forced to include the headers for all your class's internal dependencies. So any changes to those headers don't cascade into a recompile of most of your project. Plus general idealism about implementation-hiding.

Also, you wouldn't necessarily put your impl class in its own header. Just make it a struct inside the single cpp and make your outer class reference its data members directly.

Edit: Example

SomeClass.h

struct SomeClassImpl;

class SomeClass {
    SomeClassImpl * pImpl;
public:
    SomeClass();
    ~SomeClass();
    int DoSomething();
};

SomeClass.cpp

#include "SomeClass.h"
#include "OtherClass.h"
#include <vector>

struct SomeClassImpl {
    int foo;
    std::vector<OtherClass> otherClassVec;   //users of SomeClass don't need to know anything about OtherClass, or include its header.
};

SomeClass::SomeClass() { pImpl = new SomeClassImpl; }
SomeClass::~SomeClass() { delete pImpl; }

int SomeClass::DoSomething() {
    pImpl->otherClassVec.push_back(0);
    return pImpl->otherClassVec.size();
}

Pimpl Idiom Memory Usage

...the reason is to reduce the compile time.

You meant to say the recompilation time I guess, as suggested in Is the pImpl idiom really used in practice? ("recompilation time is really decreased, since only the source file needs to be rebuilt, but not the header, and every file that includes it").

Doesn't pimpl require dynamic allocation of memory?

Not really, it needs pointers, but pointers can be set to point to anything, whether this is static or not. Read more in Pimpl idiom without using dynamic memory allocation.

And if it is used a lot, you end up using more memory.

Well, the overhead is due to the pointers (4 or 8 bytes). The data have to be stored somewhere in any case, and whether this "somewhere" is static or not, the memory is pretty much the same.

I said pretty much, because if the memory is dynamically allocated, the system has to do some housekeeping, which incurs in an overhead.

However, it is extremely unlikely to run out of memory because you used the Pimpl Idiom. If you do, then the problem somewhere else, and you would go out of memory without that idiom too.

---

PS: As juanchopanza stated, Memory Fragmentation("when most of your memory is allocated in a large number of non-contiguous blocks, or chunks - leaving a good percentage of your total memory unallocated, but unusable for most typical scenarios") should be taken into account too

Is the pimpl idiom used in c#?

Is the pimpl idiom used in c#?

That depends on what you mean by this idiom.

The idiom in question is essentially to separate the implementation details of a type into one class, and the public surface area into a wrapper class that simply holds onto a pointer to the implementation class.

This has two benefits.

First, in C++ it can lead to improvements in compile time because consumers of the public surface area need only to parse the header file containing the public surface area; they need not parse the header file containing the implementation details.

Second, it leads to a very clean separation of interface from implementation; the implementation can change utterly without there ever being any impact upon the consumer, because the consumer never sees the implementation details.

The first benefit is of no matter in C#. The C# compiler is fast. (In large part of course because the language was designed to be fast to compile.)

The second benefit is of considerable use in C#. In C# the idiomatic way to implement this pattern would be to make a private nested class that does the "real" work and a public class that is just a facade with the public surface area.

A technique I particularly like in C# is to make the public class the base class of the private nested class, give the base class a private constructor to prevent third party extension, and use the factory pattern to hand out instances of the private nested class.

I don't really understand why the same thing can't be done with C++.

Then I encourage you to attempt to write a C++ compiler that does so. You'll either succeed in creating a much faster C++ compiler, or you'll find out why the same thing cannot be done in C++. You benefit either way!

---

Related Topics