Signal Handling in Multi-Threaded Python

Signal handling in multi-threaded Python

The problem is that, as explained in Execution of Python signal handlers:

A Python signal handler does not get executed inside the low-level (C) signal handler. Instead, the low-level signal handler sets a flag which tells the virtual machine to execute the corresponding Python signal handler at a later point(for example at the next bytecode instruction)

…

A long-running calculation implemented purely in C (such as regular expression matching on a large body of text) may run uninterrupted for an arbitrary amount of time, regardless of any signals received. The Python signal handlers will be called when the calculation finishes.

Your main thread is blocked on threading.Thread.join, which ultimately means it's blocked in C on a pthread_join call. Of course that's not a "long-running calculation", it's a block on a syscall… but nevertheless, until that call finishes, your signal handler can't run.

And, while on some platforms pthread_join will fail with EINTR on a signal, on others it won't. On linux, I believe it depends on whether you select BSD-style or default siginterrupt behavior, but the default is no.

So, what can you do about it?

Well, I'm pretty sure the changes to signal handling in Python 3.3 actually changed the default behavior on Linux so you won't need to do anything if you upgrade; just run under 3.3+ and your code will work as you're expecting. At least it does for me with CPython 3.4 on OS X and 3.3 on Linux. (If I'm wrong about this, I'm not sure whether it's a bug in CPython or not, so you may want to raise it on python-list rather than opening an issue…)

On the other hand, pre-3.3, the signal module definitely doesn't expose the tools you'd need to fix this problem yourself. So, if you can't upgrade to 3.3, the solution is to wait on something interruptible, like a Condition or an Event. The child thread notifies the event right before it quits, and the main thread waits on the event before it joins the child thread. This is definitely hacky. And I can't find anything that guarantees it will make a difference; it just happens to work for me in various builds of CPython 2.7 and 3.2 on OS X and 2.6 and 2.7 on Linux…

Handling Signals in Python Threads

If you set newthread.daemon = True before starting each thread, the threads will automatically be killed when the main thread exits. That's not precisely what you were asking, but from what you've described, it sounds like it could be worth knowing.

How can I catch SIGINT in threading python program?

Threads and signals don't mix. In Python this is even more so the case than outside: signals only ever get delivered to one thread (the main thread); other threads won't get the message. There's nothing you can do to interrupt threads other than the main thread. They're out of your control.

The only thing you can do here is introduce a communication channel between the main thread and whatever threads you start, using the queue module. You can then send a message to the thread and have it terminate (or do whatever else you want) when it sees the message.

Alternatively, and it's often a very good alternative, is to not use threads. What to use instead depends greatly on what you're trying to achieve, however.

Multi Threaded Signal Handling in C on Linux

Some signals can be losts when there is a pending signal with he same code. From the specification of sigaction:

If a subsequent occurrence of a pending signal is generated, it is implementation-dependent as to whether the signal is delivered or accepted more than once in circumstances other than those in which queueing is required under the Realtime Signals Extension option. The order in which multiple, simultaneously pending signals outside the range SIGRTMIN to SIGRTMAX are delivered to or accepted by a process is unspecified.

If you want to catch all the signals you have two solutions:

Use real-time signals with a value from SIGRTMIN to SIGRTMAX, instead of SIGUSR1 and SIGUSR2. Both pthread_sigqueue() and pthread_kill() will fail to send the signal if SIGQUEUE_MAX signals are pending or if the system hasn't enough resources to queue the signal.
Wait the precedent signal has been caught before to send another one.

EDIT:

1. Some explainations to answer your last comment.

You can't block-only a signal using signal(), you can ignore it (using SIG_IGN instead of a handler function) or register a handler function. With a handler function, I think we can say the signal is blocked AND caught.

I think your t.a. want you to handle one type of signal, for exemple SIGUSR1, using signal() and a handler function, and to handle SIGUSR2 with a thread using sigwaitinfo().

Using signal() you don't need to block the signals that you want to catch, and it can be done in the main thread.

Using sigwaitinfo() you need to block the signal you want to catch at least in the thread that will receive it.

You can have a look to the source code I have pasted at the end of this post.

2. More precisions.

To block a signal without placing an automatic catch/handler function, you have to use sigprocmask() in a single-threaded program, or pthread_sigmask() in a multi-threaded program. You also can use sigaction() in order to block some incomming signals during the execution of a signal handler function.

About signal catching, there are two ways to catch a signal:

A signal handler function is registered with signal() (or sigaction()) and automatically called when the signal is received, unless the signal was blocked in all threads. In order to make signal() work, you have to let at least one thread that non block the signal. You haven't to use sigwait() to handle the signal, because the program will automatically wait in parallel of its execution.
Using signal() will create a signal context when the signal is received and you will have to use async-signal-safe functions in the signal handler function. signal() register a handler function for the whole process, not only for the calling thread.
A handling thread need to catch the signals with sigwait() or sigwaitinfo(), and these threads aren't restricted to async-signal-safe functions. The signals to catch must be blocked using pthread_sigmask() at least in the thread that is the target of pthread_kill().
And must be blocked in all threads in order to catch process-wide signals for example triggered with kill() (if at least one thread doesn't block the signal, then it will have the default effect on the process).

3. Some explanations on what your program is doing.

In the main thread, the signals SIGUSR1 and SIGUSR2 are blocked, so all the threads created by the main thread after this blocking will have these signals blocked, because they inherits of the mask of the creating thread.
When you call signal() it will register the functions handler1() and handler2() as signal handling functions to be called when a thread receive the signals. But these signals are blocked for all the threads, so handler1() and handler2() won't be called as signal handler functions. So, using signal() in your program is useless.
Moreover, handler1() and handler2() are designed to be handling threads, not signal handler functions. So you shouldn't register them with signal(), you have to register non-thread functions.
You should increment the counters for sent signals only when pthread_kill() didn't failed.
When creating the handling threads, the program create 2 useless threads, because the loop is executed for i = 3 and i = 4, and you create 2 threads in this loop. So the correct code is while(i < 4), or better remove the loop.

4. I modified your program in order to catch SIGUSR1 using `signal()`:

You will see it only needs to block SIGUSR2 in handler2_thread(). No other blocking are needed in the program.
In this code, you will see the difference between a handling thread and a signal handler function, the signals received by thread1 are handled by the signal handler function handler1_func(), while the signals receveid by handler2_thread are handled in the thread itself.
The variable receivedSignal1_flag is declared volatile and of type sig_atomic_t because there is a race condition on it between the thread that check and reset it and the handler function that set it to 1. Using this way, some caught signals won't be counted. Regarding what I have read on sig_atomic_t, I'm not sure if it is possible to increment the counter receivedSignal1 directly in handler1_func() because the increment operation isn't atomic, and so can be disturbed by another signal handler. But maybe it is possible if handler_func() is the only one signal handler to read and write receivedSignal1 and having declared it volatile and sig_atomic_t. Also note that receivedSignal1_flag isn't locked with a semaphore nor a mutex, because only one thread is using it.

#include<semaphore.h>
#include<stdio.h>
#include<stdlib.h>
#include<unistd.h>
#include<pthread.h>
#include<time.h>
#include<signal.h>
#include<string.h>
#include<math.h>

/*
   Pre-definitions of functions
 */
void generator();
void handler1_func(int);
void thread1();
void handler2_thread();
void reporter();
/*
   Global Variables 
 */
int total_signal_count=0;
int sentSignal1=0;
int sentSignal2=0;

///////////////////////////////////////
//
// receivedSignal1_flag is volatile and
// sig_atomic_t because there is a race
// condition on it (used in the signal
// handler, and in the thread).
//
///////////////////////////////////////
volatile sig_atomic_t receivedSignal1_flag;

int receivedSignal1=0;
int receivedSignal2=0;

sem_t s_lock;
sem_t r_lock;

pthread_mutex_t lock;
pthread_t tid[5];
/*
   Main function
 */
int main(int argc, char ** argv)
{

    int i=0;
    int randomNum=0;
    int error;
    int pid;
    sigset_t mask_all,mask_one,prev_one;
    //Setting up signals 
    //Get Random time
    time_t now;
    time(&now);
    //semaphore is initialized to be global and val 1
    sem_init(&s_lock,0,1);
    sem_init(&r_lock,0,1);
    srand((unsigned) time(&now));
    //Loops until more threads created than 2
    while(i<3)
    {   error=pthread_create(&tid[i],NULL,(void*)generator,NULL);
        if(error!=0)
        {
            printf("failed to create thread\n");
        }
        i++;
    }//end while loop

    error=pthread_create(&tid[3],NULL,(void*)thread1,NULL);
    if(error!=0)
    {
        printf("failed to create thread\n");
    }
    error=pthread_create(&tid[4],NULL,(void*)handler2_thread,NULL);
    if(error!=0)
    {
       printf("failed to create thread \n");
    }

    //join the threads so main won't return
    i=0;
    int returnVal;
    sleep(15);
    printf("\n sigSent1==%d,sigSent2==%d,sigReceived1==%d,sigReceived2==%d\n",sentSignal1,sentSignal2,receivedSignal1,receivedSignal2);
    while(i<5)//Loops until threads are joined
    {
        //  printf("gonna join %d\n",i);
        pthread_join(tid[i],NULL);
        /*if((returnVal=pthread_join(tid[i],(void**)&returnVal))!=0)
          {
          printf("Error joining thread: %s at %d\n", strerror(returnVal),i);
          }*/
        i++;
    }//end while
    return 0;
}//end of main function
/*
   Generator threads
 */
void generator()
{
    sleep(5);
    int i=3;
    int randomNum=0;
    int val=0;
    int total_signal_c=9990;
    while(total_signal_c<10000)
    {
        usleep(1);
        //Randomly select to generate SIGUSR1 or SIGUSR2
        //Use pthread_kill(tid,SIGUSR1/SIGUSR2) to send the signal to a thread
        //  printf("total_signal_count%d\n",total_signal_c);
        //Create either a sig1 signal or sig2 signal
        randomNum=rand()%2;
        switch(randomNum)
        {
            case 0:
                /////////////////////////////////////////
                // Send SIGUSR1 to thread1
                /////////////////////////////////////////
                val=pthread_kill(tid[3],SIGUSR1);
                if(val!=0)
                {
                    printf("\nkill fail ==%d",val);
                } else {
                  sem_wait(&s_lock);
                  //semaphore
                  //mutex
                  sentSignal1++;
                  sem_post(&s_lock);
                }
                break;
            case 1:
                /////////////////////////////////////////
                // Send SIGUSR2 to handler2_thread
                /////////////////////////////////////////
                val=pthread_kill(tid[4],SIGUSR2);
                if(val!=0)
                {
                    printf("\nkill fail2");
                } else {
                  sem_wait(&s_lock);
                  sentSignal2++;
                  sem_post(&s_lock);
                  //
                  //
                }
                break;
        }

        i++;
        total_signal_c++;
        //delay for a random time, 0.01 to 0.1 second
    }
}

//////////////////////////////////////////
//
// Signal handler function for SIGUSR1:
//
//////////////////////////////////////////
void  handler1_func(int signo)
{
  // write on stdout using an async-signal-safe function:
  write(STDOUT_FILENO,"\nSignal handler function: SIGUSR1 caught\n",41);
  // set the received signal flag to 1:
  if(signo == SIGUSR1) receivedSignal1_flag = 1;
}

/////////////////////////////////////////////////////////////
//
// The thread that will receive SIGUSR1 but not handle it
// because handler1_func() will handle it automatically:
//
/////////////////////////////////////////////////////////////
void  thread1()
{
    //////////////////////////////////////////////
    //
    // register handler1_func() as signal handler
    // for the whole process, not only the thread.
    // It means that if another thread doesn't 
    // block SIGUSR1 and receive it, then
    // handler1_func() will also be called:
    //
    //////////////////////////////////////////////
    signal(SIGUSR1,handler1_func);

    while(1)
    {   
        ///////////////////////////////////////////////////
        // If a signal has been handled by handler1_func()
        // then receivedSignal1_flag = 1.
        // And so increment receivedSignal1 and print.
        ///////////////////////////////////////////////////
        if(receivedSignal1_flag == 1) {
          // reset the flag:
          receivedSignal1_flag = 0;

          sem_wait(&r_lock);
          receivedSignal1++;
          printf("\nThread1: SIGUSR1 received and handled by handler1_func()\n");
          sem_post(&r_lock);
        }

    }
    pthread_exit(NULL);
}

////////////////////////////////////////
//
// Handling thread for SIGUSR2:
//
////////////////////////////////////////
void handler2_thread()
{
    ///////////////////////////////////////////////
    //
    // Need to block SIGUSR2 in order to avoid
    // the default handler to be called.
    //
    ///////////////////////////////////////////////
    sigset_t set;
    sigemptyset(&set);
    sigaddset(&set,SIGUSR2);
    pthread_sigmask(SIG_BLOCK,&set,NULL);

    siginfo_t info;
    int val=-1;
    while(1)
    {       
        val=-1;
        val=sigwaitinfo(&set,&info);
        //if signal received modify the corresponding counter
        if(info.si_signo==SIGUSR2){
            //increment semaphore lock
            sem_wait(&r_lock);
            receivedSignal2++;
            //decrement semaphore lock
            printf("\nhandler2_thread: signal 2 received\n");
            sem_post(&r_lock);
        }       
    }
    pthread_exit(NULL);
}

Python signals for multiprocessing

The signal is handled in two separate processes rather than two distinct threads of a single process. (Hint: you import multiprocessing rather than import threading.)

The child process inherits the signal handler for SIGINT and gets its own copy of x. Now, the shell runs both your parent and child processes in a foreground process group, and sends keyboard-generated signals (like Ctrl-C => SIGINT) to the whole process group, which is why both parent and child receive the signal. Both parent and child, then, print their own value of x.

If you do switch to a threaded implementation, then only the main thread will receive signals, and your question will be moot.

Thread-Safe Signal API in Python 2.7

Only the main thread is listening to SIGINT. Make sure all threads are listening to the SIGINT value.

Signal Handling in Multi-Threaded Python