How to Execute a Functor or a Lambda in a Given Thread in Qt, Gcd-Style

  • Home
  • Languages
  • C++
  • How to Execute a Functor or a Lambda in a Given Thread in Qt, Gcd-Style

How to execute a functor or a lambda in a given thread in Qt, GCD-style?

It is certainly possible. Any solution will center on delivering an event that wraps the functor to a consumer object residing in the desired thread. We shall call this operation metacall posting. The particulars can be executed in several ways.

Qt 5.10 & up TL;DR

// invoke on the main thread
QMetaObject::invokeMethod(qApp, []{ ... });

// invoke on an object's thread
QMetaObject::invokeMethod(obj, []{ ... });

// invoke on a particular thread
QMetaObject::invokeMethod(QAbstractEventDispatcher::instance(thread),
                         []{ ... });

TL;DR for functors

// https://github.com/KubaO/stackoverflown/tree/master/questions/metacall-21646467

// Qt 5.10 & up - it's all done

template <typename F>
static void postToObject(F &&fun, QObject *obj = qApp) {
  QMetaObject::invokeMethod(obj, std::forward<F>(fun));
}

template <typename F>
static void postToThread(F && fun, QThread *thread = qApp->thread()) {
   auto *obj = QAbstractEventDispatcher::instance(thread);
   Q_ASSERT(obj);
   QMetaObject::invokeMethod(obj, std::forward<F>(fun));
}

// Qt 5/4 - preferred, has least allocations

namespace detail {
template <typename F>
struct FEvent : public QEvent {
   using Fun = typename std::decay<F>::type;
   Fun fun;
   FEvent(Fun && fun) : QEvent(QEvent::None), fun(std::move(fun)) {}
   FEvent(const Fun & fun) : QEvent(QEvent::None), fun(fun) {}
   ~FEvent() { fun(); }
}; }

template <typename F>
static void postToObject(F && fun, QObject * obj = qApp) {
   if (qobject_cast<QThread*>(obj))
      qWarning() << "posting a call to a thread object - consider using postToThread";
   QCoreApplication::postEvent(obj, new detail::FEvent<F>(std::forward<F>(fun)));
}

template <typename F>
static void postToThread(F && fun, QThread * thread = qApp->thread()) {
   QObject * obj = QAbstractEventDispatcher::instance(thread);
   Q_ASSERT(obj);
   QCoreApplication::postEvent(obj, new detail::FEvent<F>(std::forward<F>(fun)));
}
// Qt 5 - alternative version

template <typename F>
static void postToObject2(F && fun, QObject * obj = qApp) {
   if (qobject_cast<QThread*>(obj))
      qWarning() << "posting a call to a thread object - consider using postToThread";
   QObject src;
   QObject::connect(&src, &QObject::destroyed, obj, std::forward<F>(fun),
                    Qt::QueuedConnection);
}

template <typename F>
static void postToThread2(F && fun, QThread * thread = qApp->thread()) {
   QObject * obj = QAbstractEventDispatcher::instance(thread);
   Q_ASSERT(obj);
   QObject src;
   QObject::connect(&src, &QObject::destroyed, obj, std::forward<F>(fun),
                    Qt::QueuedConnection);
}
void test1() {
   QThread t;
   QObject o;
   o.moveToThread(&t);

   // Execute in given object's thread
   postToObject([&]{ o.setObjectName("hello"); }, &o);
   // or
   postToObject(std::bind(&QObject::setObjectName, &o, "hello"), &o);

   // Execute in given thread
   postToThread([]{ qDebug() << "hello from worker thread"; });

   // Execute in the main thread
   postToThread([]{ qDebug() << "hello from main thread"; });
}

TL;DR for methods/slots

// Qt 5/4
template <typename T, typename R>
static void postToObject(T * obj, R(T::* method)()) {
   struct Event : public QEvent {
      T * obj;
      R(T::* method)();
      Event(T * obj, R(T::*method)()):
         QEvent(QEvent::None), obj(obj), method(method) {}
      ~Event() { (obj->*method)(); }
   };
   if (qobject_cast<QThread*>(obj))
      qWarning() << "posting a call to a thread object - this may be a bug";
   QCoreApplication::postEvent(obj, new Event(obj, method));
}

void test2() {
   QThread t;
   struct MyObject : QObject { void method() {} } obj;
   obj.moveToThread(&t);

   // Execute in obj's thread
   postToObject(&obj, &MyObject::method);
}

TL;DR: What about a single shot timer?

All of the above methods work from threads that don't have an event loop. Due to QTBUG-66458, the handy appropriation of QTimer::singleShot needs an event loop in the source thread as well. Then postToObject becomes very simple, and you could possibly just use QTimer::singleShot directly, although it's an awkward name that hides the intent from those unfamiliar with this idiom. The indirection via a function named to better indicate the intent makes sense, even if you don't need the type check:

template <typename F>
static void postToObject(F && fun, QObject * obj = qApp) {
   if (qobject_cast<QThread*>(obj))
      qWarning() << "posting a call to a thread object - consider using postToThread";
   QTimer::singleShot(0, obj, std::forward<F>(fun));
}

Common Code

Let's define our problem in terms of the following common code. The simplest solutions will post the event to either the application object, iff the target thread is the main thread, or to an event dispatcher for any other given thread. Since the event dispatcher will exist only after QThread::run has been entered, we indicate the requirement for the thread to be running by returning true from needsRunningThread.

#ifndef HAS_FUNCTORCALLCONSUMER
namespace FunctorCallConsumer {
   bool needsRunningThread() { return true; }
   QObject * forThread(QThread * thread) {
      Q_ASSERT(thread);
      QObject * target = thread == qApp->thread()
            ? static_cast<QObject*>(qApp) : QAbstractEventDispatcher::instance(thread);
      Q_ASSERT_X(target, "postMetaCall", "the receiver thread must have an event loop");
      return target;
   }
}
#endif

The metacall posting functions, in their simplest form, require the functor call consumer to provide object for a given thread, and instantiate the functor call event. The implementation of the event is still ahead of us, and is the essential difference between various implementations.

The second overload takes a rvalue reference for the functor, potentially saving a copy operation on the functor. This is helpful if the continuation contains data that is expensive to copy.

#ifndef HAS_POSTMETACALL
void postMetaCall(QThread * thread, const std::function<void()> & fun) {
   auto receiver = FunctorCallConsumer::forThread(thread);
   QCoreApplication::postEvent(receiver, new FunctorCallEvent(fun, receiver));
}

void postMetaCall(QThread * thread, std::function<void()> && fun) {
   auto receiver = FunctorCallConsumer::forThread(thread);
   QCoreApplication::postEvent(receiver,
                               new FunctorCallEvent(std::move(fun), receiver));
}
#endif

For demonstration purposes, the worker thread first posts a metacall to the main thread, and then defers to QThread::run() to start an event loop to listen for possible metacalls from other threads. A mutex is used to allow the thread user to wait in a simple fashion for the thread to start, if necessitated by the consumer's implementation. Such wait is necessary for the default event consumer given above.

class Worker : public QThread {
   QMutex m_started;
   void run() {
      m_started.unlock();
      postMetaCall(qApp->thread(), []{
         qDebug() << "worker functor executes in thread" << QThread::currentThread();
      });
      QThread::run();
   }
public:
   Worker(QObject * parent = 0) : QThread(parent) { m_started.lock(); }
   ~Worker() { quit(); wait(); }
   void waitForStart() { m_started.lock(); m_started.unlock(); }
};

Finally, we start the above worker thread that posts a metacall to the main (application) thread, and the application thread posts a metacall to the worker thread. 

int main(int argc, char *argv[])
{
   QCoreApplication a(argc, argv);
   a.thread()->setObjectName("main");
   Worker worker;
   worker.setObjectName("worker");
   qDebug() << "worker thread:" << &worker;
   qDebug() << "main thread:" << QThread::currentThread();
   if (FunctorCallConsumer::needsRunningThread()) {
      worker.start();
      worker.waitForStart();
   }
   postMetaCall(&worker, []{ qDebug() << "main functor executes in thread" << QThread::currentThread(); });
   if (!FunctorCallConsumer::needsRunningThread()) worker.start();
   QMetaObject::invokeMethod(&a, "quit", Qt::QueuedConnection);
   return a.exec();
}

The output will look approximately as follows in all implementations. The functors cross the threads: the one created in the main thread is executed in the worker thread, and vice-versa.

worker thread: QThread(0x7fff5692fc20, name = "worker") 
main thread: QThread(0x7f86abc02f00, name = "main") 
main functor executes in thread QThread(0x7fff5692fc20, name = "worker") 
worker functor executes in thread QThread(0x7f86abc02f00, name = "main")

Qt 5 Solution Using a Temporary Object as The Signal Source

The simplest approach for Qt 5 is to use a temporary QObject as a signal source, and connect the functor to its destroyed(QObject*) signal. When postMetaCall returns, the signalSource gets destructed, emits its destroyed signal, and posts the metacall to the proxy object.

This is perhaps the most concise and straightforward implementation in the C++11 style. The signalSource object is used in the C++11 RAII fashion for the side effects of its destruction. The phrase "side effects" has a meaning within C++11's semantics and should not be interpreted to mean "unreliable" or "undesirable" - it's anything but. QObject's contract with us is to emit destroyed sometime during the execution of its destructor. We're more than welcome to use that fact.

#include <QtCore>
#include <functional>

namespace FunctorCallConsumer { QObject * forThread(QThread*); }

#define HAS_POSTMETACALL
void postMetaCall(QThread * thread, const std::function<void()> & fun) {
   QObject signalSource;
   QObject::connect(&signalSource, &QObject::destroyed,
                    FunctorCallConsumer::forThread(thread), [=](QObject*){ fun(); });
}
#ifdef __cpp_init_captures
void postMetaCall(QThread * thread, std::function<void()> && fun) {
   QObject signalSource;
   QObject::connect(&signalSource, &QObject::destroyed,
                    FunctorCallConsumer::forThread(thread), [fun(std::move(fun))](QObject*){ fun(); });
}
#endif
// Common Code follows here

If we only intend to post to the main thread, the code becomes almost trivial:

void postToMainThread(const std::function<void()> & fun) {
  QObject signalSource;
  QObject::connect(&signalSource, &QObject::destroyed, qApp, [=](QObject*){
    fun();
  });
}

#ifdef __cpp_init_captures
void postToMainThread(std::function<void()> && fun) {
  QObject signalSource;
  QObject::connect(&signalSource, &QObject::destroyed, qApp, [fun(std::move(fun))](QObject*){
    fun();
  });
}
#endif

Qt 4/5 Solution Using QEvent Destructor

The same approach can be applied to QEvent directly. The event's virtual destructor can call the functor. The events are deleted right after they are delivered by the consumer object's thread's event dispatcher, so they always execute in the right thread. This will not change in Qt 4/5.

#include <QtCore>
#include <functional>

class FunctorCallEvent : public QEvent {
   std::function<void()> m_fun;
   QThread * m_thread;
public:
   FunctorCallEvent(const std::function<void()> & fun, QObject * receiver) :
      QEvent(QEvent::None), m_fun(fun), m_thread(receiver->thread()) {}
   FunctorCallEvent(std::function<void()> && fun, QObject * receiver) :
      QEvent(QEvent::None), m_fun(std::move(fun)), m_thread(receiver->thread()) { qDebug() << "move semantics"; }
   ~FunctorCallEvent() {
      if (QThread::currentThread() == m_thread)
         m_fun();
      else
         qWarning() << "Dropping a functor call destined for thread" << m_thread;
   }
};
// Common Code follows here

To post to main thread only, things become even simpler:

class FunctorCallEvent : public QEvent {
   std::function<void()> m_fun;
public:
   FunctorCallEvent(const std::function<void()> & fun) :
      QEvent(QEvent::None), m_fun(fun) {}
   FunctorCallEvent(std::function<void()> && fun, QObject * receiver) :
      QEvent(QEvent::None), m_fun(std::move(fun)) {}
   ~FunctorCallEvent() {
      m_fun();
   }
};

void postToMainThread(const std::function<void()> & fun) {
   QCoreApplication::postEvent(qApp, new FunctorCallEvent(fun);
}

void postToMainThread(std::function<void()> && fun) {
   QCoreApplication::postEvent(qApp, new FunctorCallEvent(std::move(fun)));
}

Qt 5 Solution Using the Private QMetaCallEvent

The functor can be wrapped in the Qt 5 slot object payload of the QMetaCallEvent. The functor will be invoked by QObject::event, and thus can be posted to any object in the target thread. This solution uses the private implementation details of Qt 5.

#include <QtCore>
#include <private/qobject_p.h>
#include <functional>

class FunctorCallEvent : public QMetaCallEvent {
public:
   template <typename Functor>
   FunctorCallEvent(Functor && fun, QObject * receiver) :
      QMetaCallEvent(new QtPrivate::QFunctorSlotObject<Functor, 0, typename QtPrivate::List_Left<void, 0>::Value, void>
                     (std::forward<Functor>(fun)), receiver, 0, 0, 0, (void**)malloc(sizeof(void*))) {}
   // Metacalls with slot objects require an argument array for the return type, even if it's void.
};
// Common Code follows here

Qt 4/5 Solution Using a Custom Event and Consumer

We reimplement the event() method of the object, and have it call the functor. This calls for an explicit event consumer object in each thread that the functors are posted to. The object is cleaned up when its thread is finished, or, for the main thread, when the application instance is destructed. It works on both Qt 4 and Qt 5. The use of rvalue references avoids copying of the temporary functor.

#include <QtCore>
#include <functional>

class FunctorCallEvent : public QEvent {
   std::function<void()> m_fun;
public:
   FunctorCallEvent(const std::function<void()> & fun, QObject *) :
      QEvent(QEvent::None), m_fun(fun) {}
   FunctorCallEvent(std::function<void()> && fun, QObject *) :
      QEvent(QEvent::None), m_fun(std::move(fun)) { qDebug() << "move semantics"; }
   void call() { m_fun(); }
};

#define HAS_FUNCTORCALLCONSUMER
class FunctorCallConsumer : public QObject {
   typedef QMap<QThread*, FunctorCallConsumer*> Map;
   static QObject * m_appThreadObject;
   static QMutex m_threadObjectMutex;
   static Map m_threadObjects;
   bool event(QEvent * ev) {
      if (!dynamic_cast<FunctorCallEvent*>(ev)) return QObject::event(ev);
      static_cast<FunctorCallEvent*>(ev)->call();
      return true;
   }
   FunctorCallConsumer() {}
   ~FunctorCallConsumer() {
      qDebug() << "consumer done for thread" << thread();
      Q_ASSERT(thread());
      QMutexLocker lock(&m_threadObjectMutex);
      m_threadObjects.remove(thread());
   }
   static void deleteAppThreadObject() {
      delete m_appThreadObject;
      m_appThreadObject = nullptr;
   }
public:
   static bool needsRunningThread() { return false; }
   static FunctorCallConsumer * forThread(QThread * thread) {
      QMutexLocker lock(&m_threadObjectMutex);
      Map map = m_threadObjects;
      lock.unlock();
      Map::const_iterator it = map.find(thread);
      if (it != map.end()) return *it;
      FunctorCallConsumer * consumer = new FunctorCallConsumer;
      consumer->moveToThread(thread);
      if (thread != qApp->thread())
         QObject::connect(thread, SIGNAL(finished()), consumer, SLOT(deleteLater()));
      lock.relock();
      it = m_threadObjects.find(thread);
      if (it == m_threadObjects.end()) {
         if (thread == qApp->thread()) {
            Q_ASSERT(! m_appThreadObject);
            m_appThreadObject = consumer;
            qAddPostRoutine(&deleteAppThreadObject);
         }
         m_threadObjects.insert(thread, consumer);
         return consumer;
      } else {
         delete consumer;
         return *it;
      }
   }
};

QObject * FunctorCallConsumer::m_appThreadObject = nullptr;
QMutex FunctorCallConsumer::m_threadObjectMutex;
FunctorCallConsumer::Map FunctorCallConsumer::m_threadObjects;
// Common Code follows here

How to queue lambda function into Qt's event loop?

Your problem is of How to leverage Qt to make a QObject method thread-safe? Let's adapt the solutions offered there to your use case. First, let's factor out the safety check:

bool isSafe(QObject * obj) {
   Q_ASSERT(obj->thread() || qApp && qApp->thread() == QThread::currentThread());
   auto thread = obj->thread() ? obj->thread() : qApp->thread();
   return thread == QThread::currentThread();
}

The approach you suggested takes a functor, and lets the compiler deal with packing up the arguments (if any) within the functor: 

template <typename Fun> void postCall(QObject * obj, Fun && fun) {
   qDebug() << __FUNCTION__;
   struct Event : public QEvent {
      using F = typename std::decay<Fun>::type;
      F fun;
      Event(F && fun) : QEvent(QEvent::None), fun(std::move(fun)) {}
      Event(const F & fun) : QEvent(QEvent::None), fun(fun) {}
      ~Event() { fun(); }
   };
   QCoreApplication::postEvent(
            obj->thread() ? obj : qApp, new Event(std::forward<Fun>(fun)));
}

A second approach stores the copies of all the parameters explicitly within the event and doesn't use a functor:

template <typename Class, typename... Args>
struct CallEvent : public QEvent {
   // See https://stackoverflow.com/a/7858971/1329652
   // See also https://stackoverflow.com/a/15338881/1329652
   template <int ...> struct seq {};
   template <int N, int... S> struct gens { using type = typename gens<N-1, N-1, S...>::type; };
   template <int ...S>        struct gens<0, S...> { using type = seq<S...>; };
   template <int ...S>        void callFunc(seq<S...>) { (obj->*method)(std::get<S>(args)...); }
   Class * obj;
   void (Class::*method)(Args...);
   std::tuple<typename std::decay<Args>::type...> args;
   CallEvent(Class * obj, void (Class::*method)(Args...), Args&&... args) :
      QEvent(QEvent::None), obj(obj), method(method), args(std::move<Args>(args)...) {}
   ~CallEvent() { callFunc(typename gens<sizeof...(Args)>::type()); }
};

template <typename Class, typename... Args> void postCall(Class * obj, void (Class::*method)(Args...), Args&& ...args) {
   qDebug() << __FUNCTION__;
   QCoreApplication::postEvent(
            obj->thread() ? static_cast<QObject*>(obj) : qApp, new CallEvent<Class, Args...>{obj, method, std::forward<Args>(args)...});
}

It's used as follows:

struct Class : QObject {
   int num{};
   QString str;
   void method1(int val) {
      if (!isSafe(this))
         return postCall(this, [=]{ method1(val); });
      qDebug() << __FUNCTION__;
      num = val;
   }
   void method2(const QString &val) {
      if (!isSafe(this))
         return postCall(this, &Class::method2, val);
      qDebug() << __FUNCTION__;
      str = val;
   }
};

A test harness:

// https://github.com/KubaO/stackoverflown/tree/master/questions/safe-method-40382820
#include <QtCore>

// above code

class Thread : public QThread {
public:
   Thread(QObject * parent = nullptr) : QThread(parent) {}
   ~Thread() { quit(); wait(); }
};

void moveToOwnThread(QObject * obj) {
  Q_ASSERT(obj->thread() == QThread::currentThread());
  auto thread = new Thread{obj};
  thread->start();
  obj->moveToThread(thread);
}

int main(int argc, char ** argv) {
   QCoreApplication app{argc, argv};
   Class c;
   moveToOwnThread(&c);

   const auto num = 44;
   const auto str = QString::fromLatin1("Foo");
   c.method1(num);
   c.method2(str);
   postCall(&c, [&]{ c.thread()->quit(); });
   c.thread()->wait();
   Q_ASSERT(c.num == num && c.str == str);
}

Output:

postCall 
postCall 
postCall 
method1 
method2

The above compiles and works with either Qt 4 or Qt 5.

See also this question, exploring various ways of invoking functors in other thread contexts in Qt.

Qt run lambda on main thread [duplicate]

The automatism for crossing thread boundaries in signal/slot connections requires a receiver object, so that the emitting code can check if the current thread is equal or different to the thread of the receiver.

You could try the connect variant that has a reference object as the third argument or calling the slot with QMetaObject::invokeMethod() and explicitly stating Qt::QueuedConnection as the connection type.

E.g.

QMetaObject::invokeMethod(ui.tblFiles, "selectRow", Qt::QueuedConnection, Q_ARG(int, row));

Is there a GCD-Like approach of Multithreading in QT? [duplicate]

Yes, you can use Qt Concurrent framework to run a function in a different thread.
An example from docs:

extern void aFunctionWithArguments(int arg1, double arg2, const QString &string);

int integer = ...;
double floatingPoint = ...;
QString string = ...;

QFuture<void> future = QtConcurrent::run(aFunctionWithArguments, integer, floatingPoint, string);

http://qt-project.org/doc/qt-4.8/qtconcurrentrun.html

QThread & C++11 lambda: wait for finished

There are two issues:

  1. The connect from QThread::started to the lambda lacks the object context for the given thread. Because of that, the lambda will be executed in the current thread (specifically in thread->thread() and that's not same as thread!).

  2. You never quit the thread.

Object context is not the same as the thread. You need a QObject that lives in a given thread; it can't be the thread itself. A QThread is really a thread handle and isn't meant to live in its own thread (and it doesn't!). Resist the urge to move a QThread instance to its thread: you then have a sad case of a handle that lives in the thread it manages. As soon as the thread ends, the handle becomes non-functional since it moves to a null thread.

Side notes:

  1. Unless you really need to share the thread, just allocate it locally. Try to minimize the number of explicit memory management in your code.
  2. QThread::sleep is static.
  3. QEventLoop::quit is thread-safe, even if it's not documented to be so. You could quit from the lambda, saving one connect.
QThread thread;
QEventLoop loop;
QObject context;
context.moveToThread(&thread);
connect(&thread, &QThread::started, &context, [&]() {
    qDebug() << "waiting... ";
    QThread::sleep(10);
    qDebug() << "done";
    loop.quit();
});
thread.start();
loop.exec();
thread.quit();
thread.wait();
// some other code

Alas, this leads to reentering the event loop and spaghetti code: the world is asynchronous. There's nothing guaranteeing that the method where you run all this won't be reentered from the event loop. Instead, you should return control to the base event loop. You should also leverage the thread pool so as not to manage your threads manually. Creation of a thread is expensive, transient threads are an anti-pattern and a premature pessimization. Of course perhaps your shared thread was meant to be reused, but even then you're very likely to underutilize it. The global thread pool instance has global insight into the needs of your whole application and is better positioned to manage thread lifetime.

void doFirst() {
  QtConcurrent::run([this]{
    qDebug() << "waiting...";
    QThread::sleep(10);
    qDebug() << "done";
    QObject src;
    src.connect(&src, &QObject::destroyed, this, [this]{ doNext(); });
    // see https://stackoverflow.com/q/21646467/1329652 for better ways
    // of invoking doNext
  });
} 

void doNext() {
  // some other code
}

For nicer ways of executing code in a given thread/object context, see this question.

If your lambdas are I/O bound, you should use a custom, larger thread pool for them (and only for them). QtConcurrent::run can take your thread pool as the first argument since Qt 5.4.

How to know in which thread Qt executes slots?

It depends how you set up your connection.

  • If you use Qt::DirectConnection, the slot will be executed immediately in the signaling thread, bypassing any event loop.

  • If you use a Qt::QueuedConnection, it will be executed in the receiving objects event loop, in the receiving objects thread.

  • If you don't specify the connection type, it defaults to Qt::AutoConnection, which defaults to Qt::QueuedConnection if the two QObjects have different thread affinities.


Related Topics

What Is Array to Pointer Decay

How to Profile C++ Code Running on Linux

What Is the Copy-And-Swap Idiom

What Are the Differences Between a Pointer Variable and a Reference Variable

How to Use Arrays in C++

What Is Move Semantics

What Is a Smart Pointer and When Should I Use One

Where Do I Find the Current C or C++ Standard Documents

Difference Between Const Int*, Const Int * Const, and Int Const *

How to Set, Clear, and Toggle a Single Bit

When Should Static_Cast, Dynamic_Cast, Const_Cast, and Reinterpret_Cast Be Used

My Attempt At Value Initialization Is Interpreted as a Function Declaration, and Why Doesn't a A(()); Solve It

Using Arrays or Std::Vectors in C++, What's the Performance Gap

How to Get the Directory That a Program Is Running From

Using Generic Std::Function Objects With Member Functions in One Class

Operator[][] Overload

Signed/Unsigned Comparisons

When Should You Use a Class VS a Struct in C++


Leave a reply



Submit