Threading in a Pyqt Application: Use Qt Threads or Python Threads

Threading in a PyQt application: Use Qt threads or Python threads?

This was discussed not too long ago in PyQt mailing list. Quoting Giovanni Bajo's comments on the subject:

It's mostly the same. The main difference is that QThreads are better
integrated with Qt (asynchrnous signals/slots, event loop, etc.).
Also, you can't use Qt from a Python thread (you can't for instance
post event to the main thread through QApplication.postEvent): you
need a QThread for that to work.

A general rule of thumb might be to use QThreads if you're going to interact somehow with Qt, and use Python threads otherwise.

And some earlier comment on this subject from PyQt's author: "they are both wrappers around the same native thread implementations". And both implementations use GIL in the same way.

How to use Threading correctly in pyqt?

Qt, as with most UI frameworks, does not allow any kind of access to objects from outside its main thread, which means that external threads cannot create widgets, reading properties (like toPlainText()) is unreliable and writing (such as using append()) might even lead to crash.

Even assuming that that was possible, as you pointed out the worker has no clue about self.ui.textEdit and other objects, and that's pretty obvious: self.ui is an attribute created on the main window instance, the thread has no ui object. I suggest you to do some research about how classes and instances work, and how their attributes are accessible.

The only safe and correct way to do so is to use custom signals that are emitted from the thread and connected to the slots (functions) that will actually manipulate the UI.

In the following code I made some adjustments to make it working:

  • the worker thread directly subclasses from QThread;
  • only one worker thread is created, and it uses a Queue to get requests from the main thread;
  • two specialized signals are used to notify whether the request is valid or not, and directly connected to the append() function of the QTextEdits;
  • I removed the finished signal, which is unnecessary since the thread is going to be reused (and, in any case, QThread already provides such a signal);
  • changed the "old style" self.connect as it's considered obsolete and will not be supported in newer versions of Qt;
class Worker(QThread):
found = Signal(str)
notFound = Signal(str)
def __init__(self):
QThread.__init__(self)
self.queue = Queue()

def run(self):
while True:
hostname = self.queue.get()
output_text = collect_host_status(hostname)
for i in output_text:
if "not found" in i:
self.notFound.emit(i.replace(" not found", ""))
else:
self.found.emit(i)

def lookUp(self, hostname):
self.queue.put(hostname)

class MainWindow(QMainWindow):
def __init__(self):
# ...
self.ui.pushButton_2.clicked.connect(self.buttonclicked)

self.thread = Worker()
self.thread.found.connect(self.ui.textEdit_2.append)
self.thread.notFound.connect(self.ui.textEdit_3.append)
self.thread.start()

def buttonclicked(self):
if self.ui.textEdit.toPlainText():
self.thread.lookUp(self.ui.textEdit.toPlainText())

Understanding threading and PyQT

There are several issues with what you've written (hopefully I've caught them all, I'm reluctant to run code on my machine that does anything related to deleting files!)

  1. You are constructing a new QCoreApplication in the usingMoveToThread method. You should only have one QApplication per python instance. I am not sure exactly what doing this will have done, but I suspect it explains the behaviour you see when you create a modal dialog. Creating the QApplication (or QCoreApplication) starts the main Qt event loop which handles everything from processing signals/calling slots and handling window redraws. You've created a second event loop in the main thread, but not started it by called exec_(). I suspect when you create the dialog, this is calling exec_() (or creating a third event loop, which somehow fixes the issues created by the second event loop). Regardless there is no reason to manually create a second event loop. The main one will handle the thread stuff just fine. (note: you can have multiple event loops per pyqt application, for instance threads can have an event loop (see below) and dialogs sometimes have their own event loops. However, Qt handles the creation for you!)

  2. The loop in runSetups is going to block the main thread, so your GUI is going to be unresponsive. This shouldn't effect the thread, but it begs the question what the point of offloading it to a thread was if you are just going to block the GUI! Note that the Python GIL prevents multiple threads from running concurrently anyway, so if you were hoping to have multiple threads delete separate folders simultaneously, you'll want to look into multiprocessing instead to avoid the Python GIL. However, this is likely premature optimisation. There are probably disk I/O considerations and you may find that one of these approaches provides any meaningful speedup.

  3. The main issue is the following line: self.objThread.started.connect(self.obj.deleteFolder(path)). You are actually running deleteFolder(path) in the main thread, and passing the return value of that function to the connect method. This results in nothing being run in the thread, and everything happening in the main thread (and blocking the GUI). This mistake has come about because you want to pass a parameter to the deleteFolder method. But you can't because you are not emitting the started signal (Qt does it) and so you can't supply the parameter. Usually, you would get around this by wrapping up your method in lambda, or partial, but this brings other issues when working with threads (see here). Instead, you need to define a new signal in your Ui_Form object, which takes a str argument, connect that signal to your deleteFolder slot and emit the signal manually after you start the thread.

Something like this:

class Ui_Form(object):
deleteFolder = QtCore.pyqtSignal(str)

...

def usingMoveToThread(self,path):
self.objThread = QtCore.QThread()
self.obj = workerThread()
self.obj.moveToThread(self.objThread)
self.deleteFolder.connect(self.obj.deleteFolder)
self.objThread.start()
self.deleteFolder.emit(path)

You might want to check out this question for slightly more information on this approach (basically, Qt can handle connecting between signals/slots in different threads, as each thread has it's own event loop).

Simplest way for PyQT Threading

I believe the best approach is using the signal/slot mechanism. Here is an example. (Note: see the EDIT below that points out a possible weakness in my approach).

from PyQt4 import QtGui
from PyQt4 import QtCore

# Create the class 'Communicate'. The instance
# from this class shall be used later on for the
# signal/slot mechanism.

class Communicate(QtCore.QObject):
myGUI_signal = QtCore.pyqtSignal(str)

''' End class '''

# Define the function 'myThread'. This function is the so-called
# 'target function' when you create and start your new Thread.
# In other words, this is the function that will run in your new thread.
# 'myThread' expects one argument: the callback function name. That should
# be a function inside your GUI.

def myThread(callbackFunc):
# Setup the signal-slot mechanism.
mySrc = Communicate()
mySrc.myGUI_signal.connect(callbackFunc)

# Endless loop. You typically want the thread
# to run forever.
while(True):
# Do something useful here.
msgForGui = 'This is a message to send to the GUI'
mySrc.myGUI_signal.emit(msgForGui)
# So now the 'callbackFunc' is called, and is fed with 'msgForGui'
# as parameter. That is what you want. You just sent a message to
# your GUI application! - Note: I suppose here that 'callbackFunc'
# is one of the functions in your GUI.
# This procedure is thread safe.

''' End while '''

''' End myThread '''

In your GUI application code, you should create the new Thread, give it the right callback function, and make it run.

from PyQt4 import QtGui
from PyQt4 import QtCore
import sys
import os

# This is the main window from my GUI

class CustomMainWindow(QtGui.QMainWindow):

def __init__(self):
super(CustomMainWindow, self).__init__()
self.setGeometry(300, 300, 2500, 1500)
self.setWindowTitle("my first window")
# ...
self.startTheThread()

''''''

def theCallbackFunc(self, msg):
print('the thread has sent this message to the GUI:')
print(msg)
print('---------')

''''''

def startTheThread(self):
# Create the new thread. The target function is 'myThread'. The
# function we created in the beginning.
t = threading.Thread(name = 'myThread', target = myThread, args = (self.theCallbackFunc))
t.start()

''''''

''' End CustomMainWindow '''

# This is the startup code.

if __name__== '__main__':
app = QtGui.QApplication(sys.argv)
QtGui.QApplication.setStyle(QtGui.QStyleFactory.create('Plastique'))
myGUI = CustomMainWindow()
sys.exit(app.exec_())

''' End Main '''

EDIT

Mr. three_pineapples and Mr. Brendan Abel pointed out a weakness in my approach. Indeed, the approach works fine for this particular case, because you generate / emit the signal directly. When you deal with built-in Qt signals on buttons and widgets, you should take another approach (as specified in the answer of Mr. Brendan Abel).

Mr. three_pineapples adviced me to start a new topic in StackOverflow to make a comparison between the several approaches of thread-safe communication with a GUI. I will dig into the matter, and do that tomorrow :-)

PyQt5 black window while threads are working

There are two main problems in your code:

  1. you're executing the methods used for the thread, but you should use the callable object instead; you did the same mistake for the QThread, since connections to signals also expects a callable, but you're actually executing the run function from the main thread (completely blocking everything), since you're using the parentheses; those lines should be like this:
    t1 = threading.Thread(target=test_generator_obj.sender)

    t2 = threading.Thread(target=test_collectioner_obj.get_data)

    or, for the QThread:

    self.thread.started.connect(self.threaded.run)

  2. due to the python GIL, multithreading only releases control to the other threads whenever it allows to, but the while cycle you're using prevents that; adding a small sleep function ensures that control is periodically returned to the main thread;

Other problems also exist:

  • you're already using a subclass of QThread, so there's no use in using another QThread and move your subclass to it;
  • even assuming that an object is moved to another thread, the slot should not be decorated with arguments, since the started signal of a QThread doesn't have any; also consider that slot decorators are rarely required;
  • the thread quit() only stops the event loop of the thread, but if run is overridden no event loop is actually started; if you want to stop a thread with a run implementation, a running flag should be used instead; in any other situation, quitting the application is normally enough to stop everything;

Note that if you want to interact with the UI, you can only use QThread and custom signals, and basic python threading doesn't provide a suitable mechanism.

class Threaded(QThread):
result = pyqtSignal(int)
def __init__(self, parent=None, **kwargs):
super().__init__(parent, **kwargs)

def run(self):
self.keepGoing = True
while self.keepGoing:
print("test")
self.msleep(1)

def stop(self):
self.keepGoing = False

class MainWindow(QMainWindow):
def __init__(self):
# ...
self.threaded = Threaded()
self.threaded.start()
qApp.aboutToQuit.connect(self.threaded.stop)

How to create threads in Python for use with PyQt

Classes you need to love, the sooner the better!
One option for what you want may look like this:

import sys
import threading
from PyQt5 import QtWidgets, QtCore # ,uic

def thread(my_func):
""" Runs a function in a separate thread. """
def wrapper(*args, **kwargs):
my_thread = threading.Thread(target=my_func, args=args, kwargs=kwargs)
my_thread.start()
return wrapper

@thread
def processing(signal):
""" Emulates the processing of some data. """
ind = 1
for i in range (1,10001): # +++
html = ("<p style='color:blue;'> Completed: <b>%s </b> </p> <br>") % i
QtCore.QThread.msleep(5)
ind = ind if i%100!=0 else ind+1 # +++
signal.emit(html, ind) # +++

def mySignalHandler(html, val): # Called to process a signal
plainTextEdit.appendHtml(html)
progressBar.setValue(val)

class WindowSignal(QtWidgets.QWidget):
""" New signals can be defined as class attributes using the pyqtSignal() factory. """
my_signal = QtCore.pyqtSignal(str, int, name='my_signal')

if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)

window = WindowSignal()

button = QtWidgets.QPushButton("Emit your signal!", window)
button.clicked.connect(lambda: processing(window.my_signal))

# since you do not publish test.ui, I replaced it with the line below:
plainTextEdit = QtWidgets.QPlainTextEdit(window)
progressBar = QtWidgets.QProgressBar()
progressBar.setTextVisible(False)

layoutHBox = QtWidgets.QHBoxLayout()
layoutHBox.addWidget(button)
layoutHBox.addWidget(plainTextEdit)

layoutVBox = QtWidgets.QVBoxLayout()
window.setLayout(layoutVBox)
layoutVBox.addLayout(layoutHBox)
layoutVBox.addWidget(progressBar)

window.my_signal.connect(mySignalHandler, QtCore.Qt.QueuedConnection)

window.show()

sys.exit(app.exec_())

Sample Image

Example 2:

test.ui is a file containing the description of the main window automatically generated by Qt Designer.

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
<class>Form</class>
<widget class="QWidget" name="Form">
<property name="geometry">
<rect>
<x>0</x>
<y>0</y>
<width>400</width>
<height>300</height>
</rect>
</property>
<property name="windowTitle">
<string>Form</string>
</property>
<widget class="QWidget" name="">
<property name="geometry">
<rect>
<x>12</x>
<y>12</y>
<width>371</width>
<height>281</height>
</rect>
</property>
<layout class="QVBoxLayout" name="verticalLayout">
<item>
<layout class="QHBoxLayout" name="horizontalLayout">
<item>
<widget class="QPushButton" name="button">
<property name="text">
<string>Emit your signal!</string>
</property>
</widget>
</item>
<item>
<widget class="QPlainTextEdit" name="plainTextEdit"/>
</item>
</layout>
</item>
<item>
<widget class="QProgressBar" name="progressBar">
<property name="value">
<number>0</number>
</property>
<property name="textVisible">
<bool>false</bool>
</property>
<property name="invertedAppearance">
<bool>false</bool>
</property>
</widget>
</item>
</layout>
</widget>
</widget>
<resources/>
<connections/>
</ui>

main.py

import sys
import threading
from PyQt5 import QtWidgets, QtCore, uic

def thread(my_func):
""" Runs a function in a separate thread. """
def wrapper(*args, **kwargs):
my_thread = threading.Thread(target=my_func, args=args, kwargs=kwargs)
my_thread.start()
return wrapper

@thread
def processing(signal):
""" Emulates the processing of some data. """
for i in range (1,101): # (1,100000)
html = ("<p style='color:blue;'> Completed: <b>%s </b> </p> <br>") % i
QtCore.QThread.msleep(10)
signal.emit(html, i) # Send a signal in which we transfer the received data

def mySignalHandler(html, val): # Called to process a signal
window.plainTextEdit.appendHtml(html)
window.progressBar.setValue(val)

class WindowSignal(QtWidgets.QWidget):
""" New signals can be defined as class attributes using the pyqtSignal() factory. """
my_signal = QtCore.pyqtSignal(str, int, name='my_signal')

if __name__ == '__main__':
app = QtWidgets.QApplication(sys.argv)

window = WindowSignal()
uic.loadUi('test.ui', window)

window.button.clicked.connect(lambda: processing(window.my_signal))
window.my_signal.connect(mySignalHandler, QtCore.Qt.QueuedConnection)

window.show()

sys.exit(app.exec_())

Sample Image

Clean separation between application thread and Qt thread (Python - PyQt)

What I suggest is to do what most others do. Wait until there is code that needs to be run in a separate thread, and then only put that piece of code in a thread. There is no need for your code to be in a separate thread to have good code separation. The way I would do it is the following:

Have your appThread code (the code that has no knowledge of a GUI) in a base class that only has knowledge of non-GUI libraries. This makes it easy to support a command-line version of your code later as well. Put code that you need to execute asynchronously inside regular Python threads for this base class. Make sure the code you want to execute asynchronously is just a single function call to make it easier for my next point.

Then, have a child class in a separate file that inherits from both the base class you just wrote and the QMainWindow class. Any code that you need to run asynchronously can be called via the QThread class. If you made the code you want to run asynchronously available in one function call as I mentioned above, it's easy to make this step work for your QThread child class.

Why do the above?

It makes it much easier to manage state and communication. Why make yourself go insane with race conditions and thread communication when you don't have to? There's also really no performance reason to have separate threads in a GUI for app code vs GUI code since most of the time the user is not actually inputting much as far as the CPU is concerned. Only the parts that are slow should be put in threads, both to save sanity and to make code management easier. Plus, with Python, you don't gain anything from separate threads thanks to the GIL.

What's the correct pattern for threading in a python qt5 application?

Below is a simple demo showing how to start and stop a worker thread, and safely comminucate with the gui thread.

import sys
from PyQt5 import QtCore, QtWidgets

class Worker(QtCore.QThread):
dataSent = QtCore.pyqtSignal(dict)

def __init__(self, parent=None):
super(Worker, self).__init__(parent)
self._stopped = True
self._mutex = QtCore.QMutex()

def stop(self):
self._mutex.lock()
self._stopped = True
self._mutex.unlock()

def run(self):
self._stopped = False
for count in range(10):
if self._stopped:
break
self.sleep(1)
data = {
'message':'running %d [%d]' % (
count, QtCore.QThread.currentThreadId()),
'time': QtCore.QTime.currentTime(),
'items': [1, 2, 3],
}
self.dataSent.emit(data)

class Window(QtWidgets.QWidget):
def __init__(self):
super(Window, self).__init__()
self.edit = QtWidgets.QPlainTextEdit()
self.edit.setReadOnly(True)
self.button = QtWidgets.QPushButton('Start')
self.button.clicked.connect(self.handleButton)
layout = QtWidgets.QVBoxLayout(self)
layout.addWidget(self.edit)
layout.addWidget(self.button)
self._worker = Worker()
self._worker.started.connect(self.handleThreadStarted)
self._worker.finished.connect(self.handleThreadFinished)
self._worker.dataSent.connect(self.handleDataSent)

def handleThreadStarted(self):
self.edit.clear()
self.button.setText('Stop')
self.edit.appendPlainText('started')

def handleThreadFinished(self):
self.button.setText('Start')
self.edit.appendPlainText('stopped')

def handleDataSent(self, data):
self.edit.appendPlainText('message [%d]' %
QtCore.QThread.currentThreadId())
self.edit.appendPlainText(data['message'])
self.edit.appendPlainText(data['time'].toString())
self.edit.appendPlainText(repr(data['items']))

def handleButton(self):
if self._worker.isRunning():
self._worker.stop()
else:
self._worker.start()

if __name__ == '__main__':

app = QtWidgets.QApplication(sys.argv)
window = Window()
window.setGeometry(500, 100, 400, 400)
window.show()
sys.exit(app.exec_())


Related Topics



Leave a reply



Submit