Linux Kernel Interrupt Handler Mutex Protection

Linux kernel interrupt handler mutex protection?

Don't use semaphores in interrupt context, use spin_lock_irqsave instead. quoting LDD3:

If you have a spinlock that can be
taken by code that runs in (hardware
or software) interrupt context, you
must use one of the forms of spin_lock
that disables interrupts. Doing
otherwise can deadlock the system,
sooner or later. If you do not access
your lock in a hardware interrupt
handler, but you do via software
interrupts (in code that runs out of a
tasklet, for example, a topic covered
in Chapter 7), you can use
spin_lock_bh to safely avoid deadlocks
while still allowing hardware
interrupts to be serviced.

As for point 2, make your set_intr and clear_intr require the caller to lock the spinlock, otherwise you'll find your code deadlocking. Again from LDD3:

To make your locking work properly,
you have to write some functions with
the assumption that their caller has
already acquired the relevant lock(s).
Usually, only your internal, static
functions can be written in this way;
functions called from outside must
handle locking explicitly. When you
write internal functions that make
assumptions about locking, do yourself
(and anybody else who works with your
code) a favor and document those
assumptions explicitly. It can be very
hard to come back months later and
figure out whether you need to hold a
lock to call a particular function or
not.

How to protect data shared between multiple interrupt handler in Linux Kernel?

Whenever there is a critical section (CS) that can be run in interrupt context, you use a spinlock to protect it, it does not matter if the CS is shared between interrupt handler and a process or between interrupt handlers.

The reson why you do not use semaphore or mutex is obvious, because you can't sleep in interrupt context.

Why not to use mutex inside an interrupt

The original question you cite refers to code on an Atmel ATMegaAVR - a simple 8 mit microcontroller. In that context, one can assume that the mutex machanism is part of a simple RTOS.

In such a system, there is a thread context and an interrupt context. Interrupts are invoked by the hardware, while threads are scheduler by the RTOS scheduler. Now when an interrupt occurs, any thread will be immediately pre-empted; the interrupt must run to completion and can only be preempted by a higher priority interrupt (where nested interrupts are supported). All pending interrupts will run to completion before the scheduler can run.

Blocking on a mutex (or indeed any blocking kernel object) is a secheduling event. If you were to make any blocking call in an interrupt, the scheduler will never run. In prectice an RTOS would either ignore the blocking call, raise an exception, or enter a terminal error handler.

Some OS's such as SMX, Velocity or even WinCE have somewhat more complex interrupt architectures and support variety of deferred interrupt handler. Deferred interrupt handlers are run-to-completion scheduled from an interrupt but running outside of the interrupt context; the rules for blocking in such handlers may differ, but you would need to refer to the specific OS documentation. Without deferred interrupt handlers, the usual solution is to have a thread wait on a some blocking object such as a semaphore, and have the interrupt itself do little more that cause the object to unblock (such as giving a semaphore for example).

Multi-processor/core and parallel processing systems are another issue altogether, such systems are way beyond the scope of the question where the original comment was made, and beyond my experience - my comment may not apply in such a system, but there are no doubt additional complexities and considerations in any case

In Linux kernel, why a mutex cannot be acquired in bottom half?

The main motive for creating a mutex is simplicity and efficiency. As synchronization in bottom halves can be complicated, it is suggested that mutex be avoided in bottom halves. The design of bottom halves is not suitable for mutex. Eg. Mutex should be locked/unlocked in the same context - this would be hard to follow in case of bottom halves.

In theory you can decide to implement the whole interrupt handling different in which use of mutex in justified. Like the "threaded" interrupt handlers. http://lwn.net/Articles/380931/

Why isn't mutex_trylock safe for use in interrupts?

Imagine if you had a platform whose native, low-level primitive mutexes do not have a "try lock" operation. In that case, to implement a high-level mutex that does, you'd have to use a condition variable and a boolean "is locked" protected by the low-level mutex to indicate the high-level mutex was locked.

So a waitable mutex could be implemented using a low-level primitive mutex (that does not support a "trylock" operation) to implement a high-level mutex (that does). The "high-level mutex" can just be a boolean that's protected by the low-level mutex.

With that design, mutex_lock would be implemented as follows:

1. Acquire low-level mutex (this is a real lock operation on the primitive, implementation mutex).
2. If high-level mutex is held, do a condition wait for the high-level mutex.
3. Acquire high-level mutex (just locked = true;).
4. Release low-level mutex.

And mutex_unlock would be implemented as follows:

1. Acquire low-level mutex.
2. Release high-level mutex (just locked = false;)
3. Signal the condition variable.
4. Release the low-level mutex.

In that case, mutex_trylock would be implemented as follows:

1. Acquire low-level mutex.
2. Check if high-level mutex is held.
3. If so, release low-level mutex and return failure.
4. Take high-level mutex.
5. Release low-level mutex.
6. Return success.

Imagine if we're interrupted after step 2 but before step 3.

Can an interrupt handler be preempted by the same interrupt handler?

x86 disables all local interrupts (except NMI of course) before jumping to the interrupt vector. Linux normally masks the specific interrupt and re-enables the rest of the interrupts (which aren't masked), unless a specific flags is passed to the interrupt handler registration.

Note that while this means your interrupt handler will not race with itself on the same CPU, it can and will race with itself running on other CPUs in an SMP / SMT system.

Use spin_lock() vs down_interruptible() in workqueue

It depends entirely on what data you are trying to protect with this lock and from where (which context) that data can be accessed (besides the workqueue). If the lock and corresponding data can be also accessed from the atomic context (such as interrupt handlers), you should use appropriate locking.

Workqueue is about process context, thus it's allowed to go to sleep if necessary. Also going to sleep while holding spin_lock would be a fatal error. So in workqueue you can use sleep-able functions but not while holding spin_lock.

Semaphores are sleep-able locks like mutexes, so if your data and the corresponding lock will not be used from within atomic context, I see no reason to give up mutexes/semaphores.

More info:

Interrupts, Spin Locks, and Preemption

Linux kernel interrupt handler mutex protection.

---

Related Topics