The locking scheme for most device drivers should be kept straightforward. Using additional locks allows more concurrency but increases overhead. Using fewer locks is less time consuming but allows less concurrency. Generally, use one mutex per data structure, a condition variable for each event or condition the driver must wait for, and a mutex for each major set of data global to the driver. Avoid holding mutexes for long periods of time.
Use the multithreading semantics of the entry point to your advantage.
Make all entry points re-entrant and reduce the amount of shared data by changing static variable to automatic.
If your driver acquires multiple mutexes, acquire and release the mutexes in the same order in all code paths.
Hold and release locks within the same functional space.
Avoid holding driver mutexes when calling DDI interfaces which can block, for example,kmem_alloc(9F) with KM_SLEEP.
To look at lock usage, use lockstat(1M). lockstat(1M) monitors all kernel lock events, gathers frequency and timing data about the events, and displays the data.
See the Multithreaded Programming Guide for more details on multithreaded operations.
Here is a set of mutex-related panics:
panic: recursive mutex_enter. mutex %x caller %x
Mutexes are not re-entrant by the same thread. If you already own the mutex, you cannot own it again. Doing this leads to this panic.
panic: mutex_adaptive_exit: mutex not held by thread
Releasing a mutex that the current thread does not hold causes this panic.
panic: lock_set: lock held and only one CPU
This panic occurs only on a uniprocessor. It indicates that a spin mutex is held and it will spin forever, because there is no other CPU to release it. This could happen because the driver forgot to release the mutex on one code path, or blocked while holding it.
A common cause of this panic is that the device's interrupt is high-level and is calling a routine that blocks the interrupt handler while holding a spin mutex. This is obvious if the driver explicitly calls cv_wait(9F), but might not be so if it's blocking while grabbing an adaptive mutex with mutex_enter(9F).