Example Implementation of Interrupt Resource Management
One type of device driver that is an excellent candidate for interrupt resource management is a network device driver. The network device hardware supports multiple transmit and receive channels.
The network device generates a unique interrupt condition whenever the device receives a packet on one of its receive channels or transmits a packet on one of its transmit channels. The hardware can send a specific MSI-X interrupt for each event that can occur. A table in the hardware determines which MSI-X interrupt to generate for each event.
To optimize performance, the driver requests enough interrupts from the system to give each separate interrupt its own interrupt vector. The driver makes this request when it first calls ddi_intr_alloc(9F) in its attach(9F) routine.
The driver then evaluates the actual number of interrupts it received from ddi_intr_alloc() in actualp. It might receive all the interrupts it requested, or it might receive fewer interrupts.
A separate function inside the driver uses the total number of available interrupts to calculate which MSI-X interrupts to generate for each event. This function programs the table in the hardware accordingly.
-
If the driver receives all of its requested interrupt vectors, each entry in the hardware table has its own unique MSI-X interrupt. A one-to-one mapping exists between interrupt conditions and interrupt vectors. The hardware generates a unique MSI-X interrupt for each type of event.
-
If the driver has fewer interrupt vectors available, some MSI-X interrupt numbers must appear multiple times in the hardware table. The hardware generates the same MSI-X interrupt for more than one type of event.
The driver should have two different interrupt handler functions.
-
One interrupt handler function performs a specific task in response to an interrupt. This simple function handles interrupts that are generated by only one of the possible hardware events.
-
A second interrupt handler function is more complicated. This function handles the case where multiple interrupts are mapped to the same MSI-X interrupt vector.
In the example driver in this section, the function xx_setup_interrupts() uses the number of available interrupt vectors to program the hardware and calls the appropriate interrupt handler for each of those interrupt vectors. The xx_setup_interrupts() function is called in two places: after ddi_intr_alloc() is called in xx_attach(), and after interrupt vector allocations are adjusted in the xx_cbfunc() callback handler function.
int xx_setup_interrupts(xx_state_t *statep, int navail, xx_intrs_t *xx_intrs_p);
The xx_setup_interrupts() function is called with an array of xx_intrs_t data structures.
typedef struct {
ddi_intr_handler_t inthandler;
void *arg1;
void *arg2;
} xx_intrs_t;
This xx_setup_interrupts() functionality must exist in the driver independent of whether the Interrupt Resource Management feature is available. Drivers must be able to function with fewer interrupt vectors than the number requested during attach. If the Interrupt Resource Management feature is available, you can modify the driver to dynamically adjust to a new number of available interrupt vectors.
Other functionality that the driver must provide independent of whether the Interrupt Resource Management feature is available includes the ability to quiesce the hardware and resume the hardware. Quiesce and resume are needed for certain events related to power management and hotplugging. Quiesce and resume also are required to handle interrupt callback actions.
The quiesce function is called in xx_detach().
int xx_quiesce(xx_state_t *statep);
The resume function is called in xx_attach().
int xx_resume(xx_state_t *statep);
Make the following modifications to enhance this device driver to use the Interrupt Resource Management feature:
-
Register a callback handler. The driver must register for the actions that indicate when fewer or more interrupts are available.
-
Handle callbacks. The driver must quiesce its hardware, reprogram its interrupt handling, and resume its hardware in response to each such callback action.
/*
* attach(9F) routine.
*
* Creates soft state, registers callback handler, initializes
* hardware, and sets up interrupt handling for the driver.
*/
xx_attach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
xx_state_t *statep = NULL;
xx_intr_t *intrs = NULL;
ddi_intr_handle_t *hdls;
ddi_cb_handle_t cb_hdl;
int instance;
int type;
int types;
int nintrs;
int nactual;
int inum;
/* Get device instance */
instance = ddi_get_instance(dip);
switch (cmd) {
case DDI_ATTACH:
/* Get soft state */
if (ddi_soft_state_zalloc(state_list, instance) != 0)
return (DDI_FAILURE);
statep = ddi_get_soft_state(state_list, instance);
ddi_set_driver_private(dip, (caddr_t)statep);
statep->dip = dip;
/* Initialize hardware */
xx_initialize(statep);
/* Register callback handler */
if (ddi_cb_register(dip, DDI_CB_FLAG_INTR, xx_cbfunc,
statep, NULL, &cb_hdl) != 0) {
ddi_soft_state_free(state_list, instance);
return (DDI_FAILURE);
}
statep->cb_hdl = cb_hdl;
/* Select interrupt type */
ddi_intr_get_supported_types(dip, &types);
if (types & DDI_INTR_TYPE_MSIX) {
type = DDI_INTR_TYPE_MSIX;
} else if (types & DDI_INTR_TYPE_MSI) {
type = DDI_INTR_TYPE_MSI;
} else {
type = DDI_INTR_TYPE_FIXED;
}
statep->type = type;
/* Get number of supported interrupts */
ddi_intr_get_nintrs(dip, type, &nintrs);
/* Allocate interrupt handle array */
statep->hdls_size = nintrs * sizeof (ddi_intr_handle_t);
statep->hdls = kmem_zalloc(statep->hdls_size, KMEM_SLEEP);
/* Allocate interrupt setup array */
statep->intrs_size = nintrs * sizeof (xx_intr_t);
statep->intrs = kmem_zalloc(statep->intrs_size, KMEM_SLEEP);
/* Allocate interrupt vectors */
ddi_intr_alloc(dip, hdls, type, 0, nintrs, &nactual, 0);
statep->nactual = nactual;
/* Configure interrupt handling */
xx_setup_interrupts(statep, statep->nactual, statep->intrs);
/* Install and enable interrupt handlers */
for (inum = 0; inum < nactual; inum++) {
ddi_intr_add_handler(&hdls[inum],
intrs[inum].inthandler,
intrs[inum].arg1, intrs[inum].arg2);
ddi_intr_enable(hdls[inum]);
}
break;
case DDI_RESUME:
/* Get soft state */
statep = ddi_get_soft_state(state_list, instance);
if (statep == NULL)
return (DDI_FAILURE);
/* Resume hardware */
xx_resume(statep);
break;
}
return (DDI_SUCESS);
}
/*
* detach(9F) routine.
*
* Stops the hardware, disables interrupt handling, unregisters
* a callback handler, and destroys the soft state for the driver.
*/
xx_detach(dev_info_t *dip, ddi_detach_cmd_t cmd)
{
xx_state_t *statep = NULL;
int instance;
int inum;
/* Get device instance */
instance = ddi_get_instance(dip);
switch (cmd) {
case DDI_DETACH:
/* Get soft state */
statep = ddi_get_soft_state(state_list, instance);
if (statep == NULL)
return (DDI_FAILURE);
/* Stop device */
xx_uninitialize(statep);
/* Disable and free interrupts */
for (inum = 0; inum < statep->nactual; inum++) {
ddi_intr_disable(statep->hdls[inum]);
ddi_intr_remove_handler(statep->hdls[inum]);
ddi_intr_free(statep->hdls[inum]);
}
/* Unregister callback handler */
ddi_cb_unregister(statep->cb_hdl);
/* Free interrupt handle array */
kmem_free(statep->hdls, statep->hdls_size);
/* Free interrupt setup array */
kmem_free(statep->intrs, statep->intrs_size);
/* Free soft state */
ddi_soft_state_free(state_list, instance);
break;
case DDI_SUSPEND:
/* Get soft state */
statep = ddi_get_soft_state(state_list, instance);
if (statep == NULL)
return (DDI_FAILURE);
/* Suspend hardware */
xx_quiesce(statep);
break;
}
return (DDI_SUCCESS);
}
/*
* (*ddi_cbfunc)() routine.
*
* Adapt interrupt usage when availability changes.
*/
int
xx_cbfunc(dev_info_t *dip, ddi_cb_action_t cbaction, void *cbarg,
void *arg1, void *arg2)
{
xx_state_t *statep = (xx_state_t *)arg1;
int count;
int inum;
int nactual;
switch (cbaction) {
case DDI_CB_INTR_ADD:
case DDI_CB_INTR_REMOVE:
/* Get change in availability */
count = (int)(uintptr_t)cbarg;
/* Suspend hardware */
xx_quiesce(statep);
/* Tear down previous interrupt handling */
for (inum = 0; inum < statep->nactual; inum++) {
ddi_intr_disable(statep->hdls[inum]);
ddi_intr_remove_handler(statep->hdls[inum]);
}
/* Adjust interrupt vector allocations */
if (cbaction == DDI_CB_INTR_ADD) {
/* Allocate additional interrupt vectors */
ddi_intr_alloc(dip, statep->hdls, statep->type,
statep->nactual, count, &nactual, 0);
/* Update actual count of available interrupts */
statep->nactual += nactual;
} else {
/* Free removed interrupt vectors */
for (inum = statep->nactual - count;
inum < statep->nactual; inum++) {
ddi_intr_free(statep->hdls[inum]);
}
/* Update actual count of available interrupts */
statep->nactual -= count;
}
/* Configure interrupt handling */
xx_setup_interrupts(statep, statep->nactual, statep->intrs);
/* Install and enable interrupt handlers */
for (inum = 0; inum < statep->nactual; inum++) {
ddi_intr_add_handler(&statep->hdls[inum],
statep->intrs[inum].inthandler,
statep->intrs[inum].arg1,
statep->intrs[inum].arg2);
ddi_intr_enable(statep->hdls[inum]);
}
/* Resume hardware */
xx_resume(statep);
break;
default:
return (DDI_ENOTSUP);
}
return (DDI_SUCCESS);
}
- © 2010, Oracle Corporation and/or its affiliates