Writing Device Drivers

Chapter 6 Interrupt Handlers

This chapter describes device driver interrupt handlers. It provides information on registering an interrupt handler and discusses the responsibilities of an interrupt handler.

Interrupt Handler Overview

An interrupt is a hardware signal from a device to the CPU. It tells the CPU that the device needs attention and that the CPU should stop performing what it is doing and respond to the device. If the CPU is available (it is not performing a task with higher priority, such as servicing a higher-priority interrupt), it suspends the current thread and eventually invokes the interrupt handler for that device. The job of the interrupt handler is to service the device and stop it from interrupting. Once the handler returns, the CPU resumes what it was doing before the interrupt occurred.

The Solaris 7 DDI/DKI provides a bus-architecture independent interface for registering and servicing interrupts. Drivers must register their interrupt handlers before they can receive and service interrupts.

Interrupt Specification

The interrupt specification is the information the system needs to link the device interrupt source with a specific device interrupt handler. The specification describes the information provided by the hardware to the system when making an interrupt request. Because an interrupt specification is bus specific, the information it contains varies from bus to bus.

Interrupt specifications typically include a bus-interrupt level. For vectored interrupts the specifications include an interrupt vector. On x86 platforms the interrupt specification defines the relative interrupt priority of the device. Because interrupt specifications are bus specific, see isa(4), eisa(4), mca(4),sbus(4), vme(4), and pci(4) for information on interrupt specifications for these buses.

Interrupt Number

When registering interrupts the driver must provide the system with an interrupt number. This interrupt number identifies the interrupt specification (with bus-specific interrupt information) for which the driver is registering a handler. Most devices have one interrupt--interrupt number equals zero. However, there are devices that have different interrupts for different events. A communications controller may have one interrupt for receive ready and one for transmit ready. The device driver normally knows how many interrupts the device has, but if the driver has to support several variations of a controller, it can call ddi_dev_nintrs(9F) to find out the number of device interrupts. For a device with n interrupts, the interrupt numbers range from 0 to n-1.

Interrupt Block Cookies

The iblock cookie is an opaque data structure that is returned from either ddi_get_iblock_cookie(9F) or ddi_add_intr(9F). These interfaces use an interrupt number to return the iblock cookie associated with a specific interrupt source.

The iblock cookie gives the system information on how to block interrupts. It is passed to mutex_init(9F) when allocating driver mutexes to be used in the interrupt routine. See mutex_init(9F) for more information.

Bus Interrupt Levels

Buses prioritize device interrupts at one of several bus-interrupt levels. These bus interrupt levels are then mapped to different processor-interrupt levels. For example, SBus devices that interrupt at SBus level 7 interrupt at SPARC level 9 on SPARCstation 2 systems.

High-Level Interrupts

A bus interrupt level that maps to a CPU interrupt priority level above the scheduler priority level is called a high-level interrupt. High-level interrupts must be handled without using system services that manipulate threads. In particular, the only kernel routines that high-level interrupt handlers are allowed to call are:

mutex_enter(9F) and mutex_exit(9F) on a mutex initialized with an interrupt block cookie associated with the high-level interrupt
ddi_trigger_softintr(9F)

A bus-interrupt level by itself does not determine whether a device interrupts at high level: a given bus-interrupt level may map to a high-level interrupt on one platform, but map to an ordinary interrupt on another platform.

The driver can choose whether to support devices that have high-level interrupts, but it always has to check--it cannot assume that its interrupts are not high level. The function ddi_intr_hilevel(9F), given an interrupt number, returns a value indicating whether the interrupt is high level. For information on checking for high-level interrupts see "Registering Interrupts".

Types of Interrupts

There are two common ways in which buses implement interrupts: vectored and polled. Both methods commonly supply a bus-interrupt priority level. However, vectored devices also supply an interrupt vector; polled devices do not.

Vectored Interrupts

Devices that use vectored interrupts are assigned an interrupt vector. This is a number that identifies a particular interrupt handler. This vector may be fixed, configurable (using jumpers or switches), or programmable. In the case of programmable devices, an interrupt device cookie is used to program the device interrupt vector. When the interrupt handler is registered, the kernel saves the vector in a table.

When the device interrupts, the system enters the interrupt acknowledge cycle, asking the interrupting device to identify itself. The device responds with its interrupt vector. The kernel then uses this vector to find the responsible interrupt handler.

The VMEbus supports vectored interrupts.

Polled Interrupts

In polled (or autovectored) devices, the only information the system has about a device interrupt is either the bus interrupt priority level (IPL, on an SBus in a SPARC machine, for example) or the interrupt request number (IRQ on an ISA bus in an x86 machine, for example).

When an interrupt handler is registered, the system adds the handler to a list of potential interrupt handlers for each IPL or IRQ. Once the interrupt occurs, the system must determine which device, of all the devices associated with a given IPL or IRQ, actually interrupted. It does this by calling all the interrupt handlers for the designated IPL or IRQ, until one handler claims the interrupt.

The SBus, ISA, EISA, MCA, and PCI buses are capable of supporting polled interrupts.

Software Interrupts

The Solaris 7 DDI/DKI supports software interrupts, also known as soft interrupts. Soft interrupts are not initiated by a hardware device; they are initiated by software. Handlers for these interrupts must also be added to and removed from the system. Soft interrupt handlers run in interrupt context and therefore can be used to do many of the tasks that belong to an interrupt handler.

Commonly, hardware interrupt handlers are supposed to perform their tasks quickly, since they may suspend other system activity while running. This is particularly true for high-level interrupt handlers, which operate at priority levels greater than that of the system scheduler. High-level interrupt handlers mask the operations of all lower-priority interrupts--including those of the system clock. Consequently, the interrupt handler must avoid involving itself in an activity (such as acquiring a mutex) that might cause it to sleep.

If the handler sleeps, then the system may hang because the clock is masked and incapable of scheduling the sleeping process. For this reason, high-level interrupt handlers normally perform a minimum amount of work at high-priority levels and delegate remaining tasks to software interrupts, which run below the priority level of the high-level interrupt handler. Because software interrupt handlers run below the priority level of the system scheduler, they can do the work that the high-level interrupt handler was incapable of doing. For more information on high-level interrupts, see "Handling High-Level Interrupts".

Note -

Drivers have the option of using a high-level mutex to protect shared data between the high-level interrupt handler and the soft interrupt handler. See "High-level Mutexes".

Software interrupt handlers must not perform as if they have work to do when they run, since (like hardware interrupt handlers) they can run because some other driver triggered a soft interrupt. For this reason, the driver must indicate to the soft interrupt handler that it should do work before triggering the soft interrupt.

Registering Interrupts

Before a device driver can receive and service interrupts, it must register them with the system by calling ddi_add_intr(9F). Registering interrupts provides the system with a way to associate an interrupt handler with an interrupt specification. This interrupt handler is called when the device might have been responsible for the interrupt. It is the handler's responsibility to determine if it should handle the interrupt and, if so, claim it.

To register a driver's interrupt handler, the driver usually performs the following steps in attach(9E).

Test for high-level interrupts.

Call ddi_intr_hilevel(9F) to find out if the interrupt specification maps to a high-level interrupt. If it does, one possibility is to post a message to that effect and return DDI_FAILURE. See Example 6-1.
Get the iblock cookie by calling ddi_get_iblock_cookie(9F).
Initialize any associated mutexes with the iblock cookie by calling mutex_init(9F).

Note -

There is a potential race condition between adding the interrupt handler and initializing mutexes. The interrupt routine is eligible to be called as soon as ddi_add_intr(9F) returns, as another device might interrupt and cause the handler to be invoked. This may result in the interrupt routine being called before any mutexes have been initialized with the returned interrupt block cookie. If the interrupt routine acquires the mutex before it has been initialized, undefined behavior may result. To ensure that this race condition does not occur, always initialize mutexes and any other data used in the interrupt handler before adding the interrupt.

Example 6-1 shows how to install an interrupt handler.

Example 6-1 attach(9E) Routine Installing an Interrupt Handler

static int
xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
	struct xxstate *xsp;
	switch (cmd) {
	case DDI_ATTACH:
	  	...
	  	if (ddi_intr_hilevel(dip, inumber) != 0){
			cmn_err(CE_CONT,
				"xx: high-level interrupts are not supported\n");
			return (DDI_FAILURE);
	  	}
	  	ddi_get_iblock_cookie(dip, inumber, &xsp->iblock_cookie);
		mutex_init(&xsp->mu, "xx mutex", MUTEX_DRIVER,
			    (void *)xsp->iblock_cookie);
	  	cv_init(&xsp->cv, "xx cv", CV_DRIVER, NULL);
	  	if (ddi_add_intr(dip, inumber, &xsp->iblock_cookie,
			      &xsp->idevice_cookie, xxintr,
					(caddr_t)xsp) != DDI_SUCCESS){
			 cmn_err(CE_WARN, "xx: cannot add interrupt handler.");
			 goto failed;
	  	}
	  	return (DDI_SUCCESS);

	case DDI_PM_RESUME:
	    	For information, see Chapter 8, Power Management	case DDI_RESUME:
			For information, see Chapter 8, Power Management	default:
	    	return (DDI_FAILURE);
	}
failed:
	remove interrupt handler if necessary, destroy mutex and condition variable
	return (DDI_FAILURE);
}

Responsibilities of an Interrupt Handler

The interrupt handler has a set of responsibilities to perform. Some are required by the framework, and some are required by the device. All interrupt handlers are required to do the following:

Determine if the device is interrupting and possibly reject the interrupt.

The interrupt handler must first examine the device and determine if it has issued the interrupt. If it has not, the handler must return DDI_INTR_UNCLAIMED. This step allows the implementation of device polling: it tells the system whether this device, among a number of devices at the given interrupt priority level, has issued the interrupt.
Inform the device that it is being serviced.

This is a device-specific operation, but it is required for the majority of devices. For example, SBus devices are required to interrupt until the driver tells them to stop. This guarantees that all SBus devices interrupting at the same priority level will be serviced. Most vectored devices, on the other hand, stop interrupting after the bus interrupt-acknowledge cycle; however, their internal state still indicates that they have interrupted but have not yet been serviced.
Perform any I/O request-related processing.

Devices interrupt for different reasons, such as transfer done or transfer error. This step may involve using data access functions to read the device's data buffer, examine the device's error register, and set the status field in a data structure accordingly.

Interrupt dispatching and processing are relatively time consuming. The following points apply to interrupt processing:
- Do only what absolutely requires interrupt context.
Do any additional processing that could save another interrupt, for example, read the next data from the device.

Return DDI_INTR_CLAIMED.

Example 6-2 shows an interrupt routine.

Example 6-2 Interrupt Routine

static u_int
xxintr(caddr_t arg)
{
	struct xxstate *xsp = (struct xxstate *)arg;
	uint8_t 		status, temp;

	/*
	 * Claim or reject the interrupt.This example assumes
	 * that the device's CSR includes this information.
	 */
	mutex_enter(&xsp->high_mu);
	/* use data access routines to read status */
	status = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
	if (!(status & INTERRUPTING)) {
	    	mutex_exit(&xsp->high_mu);
	     	return (DDI_INTR_UNCLAIMED); /* dev not interrupting */
	}
	/*
	 * Inform the device that it is being serviced, and re-enable
	 * interrupts. The example assumes that writing to the
	 * CSR accomplishes this. The driver must ensure that this
data
	 * access operation makes it to the device before the
interrupt
	 * service routine returns. For example, using the data access
	 * functions to read the CSR, if it does not result in unwanted
	 * effects, can ensure this.
	 */
	ddi_put8(xsp->data_access_handle, &xsp->regp->csr,
			CLEAR_INTERRUPT | ENABLE_INTERRUPTS);
	temp = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
	perform any I/O related and synchronization processing
	signal waiting threads (biodone(9F) or cv_signal(9F)
	mutex_exit(&xsp->mu);
	return (DDI_INTR_CLAIMED);
}

When the system detects an interrupt on a bus architecture that does not support vectored hardware, it calls the driver interrupt handler function for each device that could have issued the interrupt. The interrupt handler must determine whether the device it handles issued an interrupt.

On architectures supporting vectored interrupts, this step is unnecessary but not harmful, and it enhances portability. The syntax and semantics of the interrupt-handling routine therefore can be the same for both vectored interrupts and polling interrupts.

In the example presented here, the argument passed to xxintr() is a pointer to the state structure for the device that may have issued the interrupt. This was set up by passing a pointer to the state structure as the intr_handler_arg argument to ddi_add_intr(9F) in attach(9E).

Most of the steps performed by the interrupt routine depend on the specifics of the device itself. Consult the hardware manual for the device to determine the cause of the interrupt, detect error conditions, and access the device data registers.

Interrupt Handling Additions to the State Structure

This section adds the following fields to the state structure. See "Software State Structure" for more information.

	ddi_iblock_cookie_t					high_iblock_cookie;
 	ddi_idevice_cookie_t			high_idevice_cookie;
 	kmutex_t								high_mu;
 	int									softint_running;
 	ddi_iblock_cookie_t				low_iblock_cookie;
 	kmutex_t								low_mu;
 	ddi_softintr_t						id;

Handling High-Level Interrupts

High-level interrupts are those that interrupt at the level of the scheduler and above. This level does not allow the scheduler to run; therefore, high-level interrupt handlers cannot be preempted by the scheduler, nor can they rely on the scheduler (cannot block)--they can only use mutual exclusion locks for locking.

Because of this, the driver must use ddi_intr_hilevel(9F) to determine if it uses high-level interrupts. If ddi_intr_hilevel(9F) returns true, the driver can fail to attach, or it can use a two-level scheme to handle interrupts. Properly handling high-level interrupts is the preferred solution.

Note -

By writing the driver as if it always uses high-level interrupts, a separate case can be avoided. However, this does result in an extra (software) interrupt for each hardware interrupt.

The suggested method is to add a high-level interrupt handler, which simply triggers a lower-priority software interrupt to handle the device. The driver should allow more concurrency by using a separate mutex for protecting data from the high-level handler.

High-level Mutexes

A mutex initialized with the interrupt block cookie that represents a high-level interrupt is known as a high-level mutex. While holding a high-level mutex, the driver is subject to the same restrictions as a high-level interrupt handler. The only routines it can call are:

mutex_exit(9F) to release the high-level mutex
ddi_trigger_softintr(9F) to trigger a soft interrupt

High-Level Interrupt Handling Example

In the example presented in Example 6-3, the high-level mutex (xsp->high_mu) is used only to protect data shared between the high-level interrupt handler and the soft interrupt handler. This includes a queue that the high-level interrupt handler appends data to (and the low-level handler removes data from), and a flag that indicates the low-level handler is running. A separate low-level mutex (xsp->low_mu) protects the rest of the driver from the soft interrupt handler.

Example 6-3 attach(9E) Routine Handling High-Level Interrupts

static int
xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
		struct xxstate *xsp;
		...
		if (ddi_intr_hilevel(dip, inumber)) {
			ddi_get_iblock_cookie(dip, inumber,
				&xsp->high_iblock_cookie);
			mutex_init(&xsp->high_mu, "xx high mutex", MUTEX_DRIVER,
				(void *)xsp->high_iblock_cookie);
			if (ddi_add_intr(dip, inumber, &xsp->high_iblock_cookie,
				&xsp->high_idevice_cookie, xxhighintr, (caddr_t)xsp)
				!= DDI_SUCCESS)
				goto failed;
			ddi_get_soft_iblock_cookie(dip, DDI_SOFTINT_HI,
				&xsp->low_iblock_cookie)
			mutex_init(&xsp->low_mu, "xx low mutex", MUTEX_DRIVER,
				(void *)xsp->low_iblock_cookie);
			if (ddi_add_softintr(dip, DDI_SOFTINT_HI, &xsp->id,
				&xsp->low_iblock_cookie, NULL,
				xxlowintr, (caddr_t)xsp) != DDI_SUCCESS)
				goto failed;
		} else {
			add normal interrupt handler
		}
		cv_init(&xsp->cv, "xx condvar", CV_DRIVER, NULL);
		...
		return (DDI_SUCCESS);
	failed:
			free allocated resources, remove interrupt handlers
		return (DDI_FAILURE);
}

The high-level interrupt routine services the device, and enqueues the data. The high-level routine triggers a software interrupt if the low-level routine is not running, as Example 6-4 demonstrates.

Example 6-4 High-level Interrupt Routine

static u_int
xxhighintr(caddr_t arg)
{
 	struct xxstate				*xsp = (struct xxstate *)arg;
 	uint8_t		status, temp;
 	int		need_softint;
	
 	mutex_enter(&xsp->high_mu);
 	/* read status */
 	status = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
 	if (!(status & INTERRUPTING)) {
	    	 mutex_exit(&xsp->high_mu);
	     	 return (DDI_INTR_UNCLAIMED); /* dev not interrupting */
 	}

 	ddi_put8(xsp->data_access_handle,&xsp->regp->csr,
	    	 CLEAR_INTERRUPT | ENABLE_INTERRUPTS);
 	temp = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
	  read data from device and queue the data for the low-level interrupt handler;
 	if (xsp->softint_running)
	    	need_softint = 0;
 	else
	    	need_softint = 1;
 	mutex_exit(&xsp->high_mutex);
 	/* read-only access to xsp->id, no mutex needed */
 	if (need_softint)
	     	ddi_trigger_softintr(xsp->id);
 	return (DDI_INTR_CLAIMED);
}

The low-level interrupt routine is started by the high-level interrupt routine triggering a software interrupt. Once running, it should continue to do so until there is nothing left to process, as Example 6-5 shows.

Example 6-5 Low-level Interrupt Routine

static u_int
xxlowintr(caddr_t arg)
{
 	struct xxstate *xsp = (struct xxstate *)arg;
 	....
 	mutex_enter(&xsp->low_mu);
 	mutex_enter(&xsp->high_mu);
	   if (queue empty|| xsp->softint_running) {
	    	 mutex_exit(&xsp->high_mu);
	    	 mutex_exit(&xsp->low_mu);
	    	 return (DDI_INTR_UNCLAIMED);
 	}
 	xsp->softint_running = 1;
 	while (data on queue) {
  	    	ASSERT(mutex_owned(&xsp->high_mu);
	    	dequeue data from high-level queue;
	    	mutex_exit(&xsp->high_mu);
	    	normal interrupt processing
	    	mutex_enter(&xsp->high_mu);
 	}
 	xsp->softint_running = 0;
 	mutex_exit(&xsp->high_mu);
 	mutex_exit(&xsp->low_mu);
   return (DDI_INTR_CLAIMED);
}