Writing Device Drivers

Hotplug-Capable Device Driver Development

This section briefly discusses how to make Solaris device drivers compliant with the new Solaris technologies of Dynamic Reconfiguration and future Solaris hotplug technologies.

D_HOTPLUG Flag

Hotplug-capable drivers should set D_HOTPLUG in their cb_ops cb_flag.

Detach Entry Point

The driver's detach entry point can be called with these commands:

It is strongly recommended to fully implement support for all detach commands.

Note the following uses are possible:

During Dynamic Reconfiguration, there are two possible detach scenarios.

  1. All the devices on the detaching system board receive a DDI_DETACH, and all other devices keep running; or

  2. All the devices on the detaching system board receive a DDI_DETACH. All other devices receive a DDI_SUSPEND followed by a DDI_RESUME.

DDI_DETACH and DDI_SUSPEND are discussed in more detail below, with emphasis on common errors in device drivers that support these commands.

DDI_DETACH Command

The DDI_DETACH operation is the inverse of DDI_ATTACH. The DDI_DETACH handling should only deallocate the data structures for the specified instance. Driver global resources must only be deallocated in _fini(9E). Since instances are assigned in arbitrary order, the driver must be able to handle out-of-sequence presentation of instances. In fact, no assumption should be made about the instance number.

The DDI_DETACH command code should perform the following actions.

  1. Check whether DDI_DETACH is safe.

    The driver can assume that the device has been closed before DDI_DETACH is issued. However, there may be outstanding callbacks that cannot be cancelled, or the device may not currently be in a state that permits it to be reliably shut down and restarted later. While timeouts or callbacks are still active, proper locking must be enforced.

    The driver should not block while waiting for callback completion or for the device to become idle.

    Devices that maintain some state after a close operation must be carefully analyzed. When a driver not currently in use is automatically unloaded (for example, because the system memory is low) and later automatically reloaded when the user opens the device, this might cause undesirable operation.

    For example, a tape driver that supports non-rewinding tape access might fail the detach operation when the tape head is not at the beginning of the tape. If the drive is powered down, the head position will be lost.

  2. Shut down the device and disable interrupts.

    A device needs enough hardware information/support to be able to shut off and restart interrupts. This may already be coded in the driver as a function of the existing detach routines.

  3. Remove any interrupts registered with the system.

  4. Cancel any outstanding timeouts and callbacks.

  5. Quiesce or remove any driver threads.

  6. Deallocate memory resources.

    The driver should be unloadable without memory leaks.

  7. Unmap any mapped device registers.

  8. Execute ddi_set_driver_private(dip, NULL).

  9. Free the softstate structure for this instance.

When there is failure during detach, the driver must decide whether to continue the detach and return success or undo the detach actions completed to that point and return failure. Undoing might be risky and it is usually preferable to continue the detach operation.

DDI_DETACH may be followed by power interruption; any further references to the device will need to be preceeded by a DDI_ATTACH.

Note the following when using timeout() routines:

The timeout routine should take the form of:

static void
 
XX_timeout(caddr_t arg)
{
    struct xx *un = (struct un *)arg;
    mutex_enter(&un->un_lock);
 
    .....
 
    XX_start_timeout(un);
 
    mutex_exit(&un->un_lock);
}
static void

XX_start_timeout(struct xx *un)

{
ASSERT(MUTEX_HELD(&un->un_lock));

    if ((un->un_tid == 0) && ((un->un_flags & XXSTOP) ==
                0)){un->un_tid = timeout(XX_to, arg, ticks);
 
    }
}
static void

XX_stop_timeout(struct xx *un)

{
int     tid;
mutex_enter(&un->un_lock);

    if ((tid = un->un_tid) != 0) {

    /* do not reschedule timeout */
 
        un->un_flags |= XXSTOP;
 
        /* do not hold across untimeout() */
 
        mutex_exit(&un->un_lock);
        (void) untimeout(tid);
        mutex_enter(&un->un_lock);
 
        un->un_flags &= ~XXSTOP;
        mutex_exit(&un->un_lock);
    } else {
        mutex_exit(&un->un_lock);
    }   
}

When deallocating memory, always verify first that the pointer is valid:

#if NONONONO

kmem_free(un->un_buf, un->un_buf_len);
#else
if (un->un_buf) {

    kmem_free(un->un_buf, un->un_buf_len);

    un->un_buf = NULL;
}   
#endif

DDI_SUSPEND Command

System power management and the Dynamic Reconfiguration framework pass the command DDI_SUSPEND to the detach(9E) driver entry point to request that the driver save the device hardware state. The driver may fail the suspend operation if outstanding operations cannot be completed soon or aborted, or if non-cancellable callbacks are outstanding, in which case the system will abort the suspend operation. Note that the driver instance may already have been power managed using DDI_PM_SUSPEND.

To process the DDI_SUSPEND command, the driver must:

  1. Set a suspended flag to block new operations.

  2. Wait until outstanding operations have completed, or abort them if they can be restarted.

  3. Block further operations from being initiated until the device is resumed (except for dump(9E) requests). Refer to sample code in Writing Device Drivers and the bst sample driver.

  4. Cancel pending callouts such as timeout callbacks, and quiesce or destroy other driver threads.

  5. Save any volatile hardware state in memory. This state includes the contents of device registers, and can also include downloaded firmware.

Chapter 8, Power Management describes in more detail some special power management considerations.

DDI_SUSPEND is always followed by DDI_RESUME. There may or may not be power interruption.

Attach Entry Point

The system calls attach(9E) to attach a device instance to the system or to resume operation after power has been suspended. The driver's attach(9E) entry point should handle these commands:

DDI_PM_RESUME Command

When power is restored to the device that was suspended with DDI_PM_SUSPEND, that device is resumed using the DDI_PM_RESUME command. The device driver should restore the hardware state, set up timeouts again if necessary, and enable interrupts again.

The DDI_PM_RESUME code should make no assumptions about the state of the hardware, which may or may not have lost power.

DDI_RESUME Command

When power is restored to the system or the system is unquiesced, each device that was suspended will be resumed using the DDI_RESUME command. The device driver should restore the hardware state, set up timeouts again if necessary, and enable interrupts again.

If the device is still suspended by DDI_PM_SUSPEND, the driver has to enter a state where it will call ddi_dev_is_needed(9F) for any new or pending requests, since an attach(9E) with DDI_PM_RESUME is still forthcoming.

The resume code should make no assumptions about the state of the hardware, which may or may not have lost power.

Special Issues With SCSI HBA Drivers

SCSI HBA drivers usually do not have a cb_ops(9S) structure, so to enable hotplugging, a minimal cb_ops(9S) structure must be created and exported in the dev_ops(9S).

 /*
      * autoconfiguration routines.
      */ 
     static struct dev_ops xx_dev_ops = {
             ....
             &xx_cb_ops,             /* devo_cb_ops */
             ....
     };
     static struct cb_ops xx_cb_ops = {
     nodev,                  /* open */
      nodev,                  /* close */
      nodev,                  /* strategy */
      nodev,                  /* print */
      nodev,                  /* dump */
      nodev,                  /* read */
      nodev,                  /* write */
      nodev,                  /* ioctl */
      nodev,                  /* devmap */
      nodev,                  /* mmap */
      nodev,                  /* segmap */
      nochpoll,               /* poll */
      nodev,                  /* cb_prop_op */
      0,                      /* streamtab  */
      D_MP | D_HOTPLUG,        /* Driver compatibility flag */
      CB_REV,                 /* cb_rev */
      nodev,                  /* async I/O read entry point */
      nodev                   /* async I/O write entry point */
     };

The DDI_DETACH and DDI_SUSPEND/RESUME requirements are very similar to those of leaf drivers described above.