Allowing Ctrl-C to Interrupt a Python C-Extension

Allowing Ctrl-C to interrupt a python C-extension

I would redesign the C extensions so that they don't run for a long period.

So, split them into more elementary steps (each running for a short period of time, e.g. 10 to 50 milliseconds), and have these more elementary steps called by Python code.

continuation passing style might be relevant to understand, as a programming style...

Interrupting a Python C-extension with Ctrl-C using ctypes

    LIBC.main(len(args),args)
    signal.signal(signal.SIGINT, lambda s,f:os.kill(os.getpid(), signal.SIGTERM))

If you want the signal handler to be invoked while LIBC.main is running, you must install it (via signal.signal) before LIBC.main is called, not after it returns.

But, as you noticed: It still does't work. That's because a Python signal handler doesn't get executed while the C-extension is running, and since Python on its own initiative installs a SIGINT handler, by default Ctrl-C doesn't work under this condition. In order to make it interrupt the program, restore the default signal behavior:

    signal.signal(signal.SIGINT, signal.SIG_DFL)
    LIBC.main(len(args), args)

Python C extension for a function that sleeps

You don't need to do anything on the Python side, you can just treat it as a synchronous function. On the C side, you just block until the event occurs, possibly allowing interrupts. For example, take a look at the implementation of the time.sleep function:

/* LICENSE: http://docs.python.org/license.html */

/* Implement floatsleep() for various platforms.
   When interrupted (or when another error occurs), return -1 and
   set an exception; else return 0. */

static int
floatsleep(double secs)
{
/* XXX Should test for MS_WINDOWS first! */
#if defined(HAVE_SELECT) && !defined(__BEOS__) && !defined(__EMX__)
    struct timeval t;
    double frac;
    frac = fmod(secs, 1.0);
    secs = floor(secs);
    t.tv_sec = (long)secs;
    t.tv_usec = (long)(frac*1000000.0);
    Py_BEGIN_ALLOW_THREADS
    if (select(0, (fd_set *)0, (fd_set *)0, (fd_set *)0, &t) != 0) {
#ifdef EINTR
        if (errno != EINTR) {
#else
        if (1) {
#endif
            Py_BLOCK_THREADS
            PyErr_SetFromErrno(PyExc_IOError);
            return -1;
        }
    }
    Py_END_ALLOW_THREADS
#elif defined(__WATCOMC__) && !defined(__QNX__)
    ...

All it does is use the select function to sleep for the given period of time. select is used so that if any signal is received (such as SIGINT from hitting Ctrl+C at the terminal), the system call is interrupted and control returns to Python.

Hence. your implementation can just call the C wait_int function. If it supports being interrupted by signals, than great, that will allow the user to interrupt it by hitting Ctrl+C, but make sure to react appropriately such that an exception will be thrown (I'm not certain of how this works, but it looks like returning NULL from the top-level function (time_sleep in this example) will do the trick).

Likewise, for better multithreaded performance, surround the wait call with a pair of Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS macros, but this is not required, especially if you're not using multithreading at all.

How to handle Ctrl+C in python app with PyQt?

CTRL+C causes a signal to be sent to the process. Python catches the signal, and sets a global variable, something like CTRL_C_PRESSED = True. Then, whenever the Python interpreter gets to execute a new opcode, it sees the variable set and raises a KeybordInterrupt.

This means that CTRL+C works only if the Python interpreter is spinning. If the interpreter is executing an extension module written in C that executes a long-running operation, CTRL+C won't interrupt it, unless it explicitly "cooperates" with Python. Eg: time.sleep() is theoretically a blocking operation, but the implementation of that function "cooperates" with the Python interpreter to make CTRL+C work.

This is all by design: CTRL+C is meant to do a "clean abort"; this is why it gets turned into an exception by Python (so that the cleanups are executed during stack unwind), and its support by extension modules is sort of "opt-in". If you want to totally abort the process, without giving it a chance to cleanup, you can use CTRL+.

When Python calls QApplication::exec() (the C++ function), Qt doesn't know how to "cooperate" with Python for CTRL+C, and this is why it does not work. I don't think there's a good way to "make it work"; you may want to see if you can handle it through a global event filter. — Giovanni Bajo

Adding this to the main program solved the problem.

import signal

signal.signal(signal.SIGINT, signal.SIG_DFL)

I'm not sure what this has to do with the explanation.

How to raise Python exception from a C extension

You're just not in a state to execute Python code when getting the signal. Before raising an error you need to acquire the GIL:

static void siginfo_handler(int signum, siginfo_t *siginfo, void *context) {
    printf("got signal from '%d'\n", siginfo->si_pid);
    PyGILState_STATE gstate = PyGILState_Ensure();
    PyErr_SetString(PyExc_KeyboardInterrupt, "SIGINT received");
    PyGILState_Release(gstate);
}

Ctrl-C doesn't work with PyQt

CTRL+C causes a signal to be sent to
the process. Python catches the
signal, and sets a global variable,
something like CTRL_C_PRESSED = True.
Then, whenever the Python interpreter
gets to execute a new opcode, it sees
the variable set and raises a
KeybordInterrupt.

This means that CTRL+C works only if
the Python interpreter is spinning. If
the interpreter is executing an
extension module written in C that
executes a long-running operation,
CTRL+C won't interrupt it, unless it
explicitly "cooperates" with Python.
Eg: time.sleep() is theoretically a
blocking operation, but the
implementation of that function
"cooperates" with the Python
interpreter to make CTRL+C work.

This is all by design: CTRL+C is meant
to do a "clean abort"; this is why it
gets turned into an exception by
Python (so that the cleanups are
executed during stack unwind), and its
support by extension modules is sort
of "opt-in". If you want to totally
abort the process, without giving it a
chance to cleanup, you can use CTRL+.

When Python calls QApplication::exec()
(the C++ function), Qt doesn't know
how to "cooperate" with Python for
CTRL+C, and this is why it does not
work. I don't think there's a good way
to "make it work"; you may want to see
if you can handle it through a global
event filter.
— Giovanni Bajo

Adding this to the main program solved the problem.

import signal

signal.signal(signal.SIGINT, signal.SIG_DFL)

I'm not sure what this has to do with the explanation.

Cython, Python and KeyboardInterrupt ignored

You have to periodically check for pending signals, for example, on every Nth iteration of the simulation loop:

from cpython.exc cimport PyErr_CheckSignals

cdef Run(self):
    while True:
        # do some work
        PyErr_CheckSignals()

PyErr_CheckSignals will run signal handlers installed with signal module (this includes raising KeyboardInterrupt if necessary).

PyErr_CheckSignals is pretty fast, it's OK to call it often. Note that it should be called from the main thread, because Python runs signal handlers in the main thread. Calling it from worker threads has no effect.

Explanation

Since signals are delivered asynchronously at unpredictable times, it is problematic to run any meaningful code directly from the signal handler. Therefore, Python queues incoming signals. The queue is processed later as part of the interpreter loop.

If your code is fully compiled, interpreter loop is never executed and Python has no chance to check and run queued signal handlers.

---

Related Topics