This chapter contains information on creating SCSI host bus adapter (HBA) drivers and provides sample code illustrating the structure of a typical HBA driver and showing the use of the HBA driver interfaces provided by the Sun Common SCSI Architecture (SCSA). This chapter provides information on the following subjects:
As described in Chapter 14, SCSI Target Drivers, the Solaris 9 DDI/DKI divides the software interface to SCSI devices into two major parts:
Target devices and drivers
Host bus adapter devices and drivers
Target device refers to a device on a SCSI bus, such as a disk or a tape drive. Target driver refers to a software component installed as a device driver. Each target device on a SCSI bus is controlled by one instance of the target driver.
Host bus adapter device refers to HBA hardware, such as an SBus or PCI SCSI adapter card. Host bus adapter driver refers to a software component installed as a device driver, such as the esp driver on a SPARC machine or the ncrs driver on an IA machine, and the isp driver, which works on both. An instance of the HBA driver controls each of its host bus adapter devices configured in the system.
The Sun Common SCSI Architecture (SCSA) defines the interface between these target and HBA components.
Understanding SCSI target drivers is an essential prerequisite to writing effective SCSI HBA drivers. For information on SCSI target drivers, see Chapter 14, SCSI Target Drivers. Target driver developers can also benefit from reading this chapter.
The host bus adapter driver is responsible for:
Managing host bus adapter hardware
Accepting SCSI commands from the SCSI target driver
Transporting the commands to the specified SCSI target device
Performing any data transfers that the command requires
Collecting status
Handling auto-request sense (optional)
Informing the target driver of command completion (or failure)
SCSA is the Solaris 9 DDI/DKI programming interface for the transmission of SCSI commands from a target driver to a host adapter driver. By conforming to the SCSA, the target driver can pass any combination of SCSI commands and sequences to a target device without knowledge of the hardware implementation of the host adapter. SCSA conceptually separates the building of a SCSI command (by the target driver) from the transporting of the command to and data to and from the SCSI bus (by the HBA driver) for the appropriate target device. SCSA manages the connections between the target and HBA drivers through an HBA transport layer, as shown in the following figure.
The HBA transport layer is a software and hardware layer responsible for transporting a SCSI command to a SCSI target device. The HBA driver provides resource allocation, DMA management, and transport services in response to requests made by SCSI target drivers through SCSA. The host adapter driver also manages the host adapter hardware and the SCSI protocols necessary to perform the commands. When a command has been completed, the HBA driver calls the target driver's SCSI pkt command completion routine.
Figure 15–2 illustrates this flow, with emphasis placed on the transfer of information from target drivers to SCSA to HBA drivers. The following figure also shows typical transport entry points and function calls.
SCSA HBA interfaces include HBA entry points, HBA data structures, and an HBA framework.
SCSA defines a number of HBA driver entry points, listed in the following table. These entry points are called by the system when configuring a target driver instance connected to the HBA driver, or when the target driver makes a SCSA request. See SCSA HBA Entry Points for more information.
Table 15–1 SCSA HBA Entry Point Summary
Function Name |
Called as a Result of |
---|---|
tran_tgt_init(9E) |
System attaching target device instance |
tran_tgt_probe(9E) |
Target driver calling scsi_probe(9F) |
tran_tgt_free(9E) |
System detaching target device instance |
tran_start(9E) |
Target driver calling scsi_transport(9F) |
tran_reset(9E) |
Target driver calling scsi_reset(9F) |
tran_abort(9E) |
Target driver calling scsi_abort(9F) |
tran_getcap(9E) |
Target driver calling scsi_ifgetcap(9F) |
tran_setcap(9E) |
Target driver calling scsi_ifsetcap(9F) |
tran_init_pkt(9E) |
Target driver calling scsi_init_pkt(9F) |
tran_destroy_pkt(9E) |
Target driver calling scsi_destroy_pkt(9F) |
tran_dmafree(9E) |
Target driver calling scsi_dmafree(9F) |
tran_sync_pkt(9E) |
Target driver calling scsi_sync_pkt(9F) |
(9E)tran_reset_notify |
Target driver calling scsi_reset_notify(9F) |
tran_quiesce(9E) |
System quiescing bus |
tran_unquiesce(9E) |
System resuming activity on bus |
tran_bus_reset(9E) |
System resetting bus |
SCSA defines data structures to enable the exchange of information between the target and HBA drivers. These data structures include:
Each instance of an HBA driver must allocate a scsi_hba_tran(9S) structure using scsi_hba_tran_alloc(9F) in the attach(9E) entry point. scsi_hba_tran_alloc(9F) initializes the scsi_hba_tran(9S) structure before it returns. The HBA driver must initialize specific vectors in the transport structure to point to entry points within the HBA driver. Once initialized, the HBA driver exports the transport structure to SCSA by calling scsi_hba_attach_setup(9F).
Because SCSA keeps a pointer to the transport structure in the driver-private field on the devinfo node, HBA drivers must not use ddi_set_driver_private(9F). They can, however, use ddi_get_driver_private(9F) to retrieve the pointer to the transport structure.
The scsi_hba_tran(9S) structure contains the following fields:
struct scsi_hba_tran { dev_info_t *tran_hba_dip; void *tran_hba_private; /* HBA softstate */ void *tran_tgt_private; /* target-specific info */ struct scsi_device *tran_sd; int (*tran_tgt_init)(); int (*tran_tgt_probe)(); void (*tran_tgt_free)(); int (*tran_start)(); int (*tran_reset)(); int (*tran_abort)(); int (*tran_getcap)(); int (*tran_setcap)(); struct scsi_pkt *(*tran_init_pkt)(); void (*tran_destroy_pkt)(); void (*tran_dmafree)(); void (*tran_sync_pkt)(); int (*tran_reset_notify)(); int (*tran_quiesce)(); int (*tran_unquiesce)(); int (*tran_bus_reset)(); };
Code fragments presented subsequently in this chapter use these fields to describe practical HBA driver operations. See SCSA HBA Entry Points for more information.
where:
Pointer to the HBA device instance dev_info structure. The function scsi_hba_attach_setup(9F) sets this field.
Pointer to private data maintained by the HBA driver. Usually, tran_hba_private contains a pointer to the state structure of the HBA driver.
Pointer to private data maintained by the HBA driver when using cloning. By specifying SCSI_HBA_TRAN_CLONE when calling scsi_hba_attach_setup(9F), the scsi_hba_tran(9S) structure is cloned once per target, permitting the HBA to initialize this field to point to a per-target instance data structure in the tran_tgt_init(9E) entry point. If SCSI_HBA_TRAN_CLONE is not specified, tran_tgt_private is NULL and must not be referenced. See Transport Structure Cloning for more information.
Pointer to a per-target instance scsi_device(9S) structure used when cloning. If SCSI_HBA_TRAN_CLONE is passed to scsi_hba_attach_setup(9F), tran_sd is initialized to point to the per-target scsi_device structure before any HBA functions are called on behalf of that target. If SCSI_HBA_TRAN_CLONE is not specified, tran_sd is NULL and must not be referenced. See Transport Structure Cloning for more information.
Pointer to the HBA driver entry point called when initializing a target device instance. If no per-target initialization is required, the HBA can leave tran_tgt_init set to NULL.
Pointer to the HBA driver entry point called when a target driver instance calls scsi_probe(9F) to probe for the existence of a target device. If no target probing customization is required for this HBA, the HBA should set tran_tgt_probe to scsi_hba_probe(9F).
Pointer to the HBA driver entry point called when a target device instance is destroyed. If no per-target deallocation is necessary, the HBA can leave tran_tgt_free set to NULL.
Pointer to the HBA driver entry point called when a target driver calls scsi_transport(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_reset(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_abort(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_ifgetcap(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_ifsetcap(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_init_pkt(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_destroy_pkt(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_dmafree(9F).
Pointer to the HBA driver entry point called when a target driver calls scsi_sync_pkt(9F).
Pointer to the HBA driver entry point called when a target driver calls tran_reset_notify(9E).
The scsi_address(9S) structure provides transport and addressing information for each SCSI command allocated and transported by a target driver instance.
The scsi_address structure contains the following fields:
struct scsi_address { struct scsi_hba_tran *a_hba_tran; /* Transport vectors */ ushort_t a_target; /* Target identifier */ uchar_t a_lun; /* Lun on that Target */ uchar_t a_sublun; /* Sublun on that Lun */ /* Not used */ };
Pointer to the scsi_hba_tran(9S) structure, as allocated and initialized by the HBA driver. If SCSI_HBA_TRAN_CLONE was specified as the flag to scsi_hba_attach_setup(9F), a_hba_tran points to a copy of that structure.
Identifies the SCSI target on the SCSI bus.
Identifies the SCSI logical unit on the SCSI target.
The HBA framework allocates and initializes a scsi_device(9S) structure for each instance of a target device before calling an HBA driver's tran_tgt_init(9E) entry point. This structure stores information about each SCSI logical unit, including pointers to information areas that contain both generic and device-specific information. There is one scsi_device(9S) structure for each target device instance attached to the system.
If the per-target initialization is successful (in other words, if either tran_tgt_init(9E) returns success or the vector is NULL), the HBA framework will set the target driver's per-instance private data to point to the scsi_device(9S) structure, using ddi_set_driver_private(9F).
The scsi_device(9S) structure contains the following fields:
struct scsi_device { struct scsi_address sd_address; /* routing information */ dev_info_t *sd_dev; /* device dev_info node */ kmutex_t sd_mutex; /* mutex used by device */ void *sd_reserved; struct scsi_inquiry *sd_inq; struct scsi_extended_sense *sd_sense; caddr_t sd_private; /* for driver's use */ };
Data structure that is passed to the SCSI resource allocation routines.
Pointer to the target's dev_info structure.
Mutex for use by the target driver. This is initialized by the HBA framework and can be used by the target driver as a per-device mutex. This mutex should not be held across a call to scsi_transport(9F) or scsi_poll(9F). See Chapter 3, Multithreading for more information on mutexes.
Pointer for the target device's SCSI inquiry data. The scsi_probe(9F) routine allocates a buffer, fills it in, and attaches it to this field.
Pointer to a buffer to contain Request Sense data from the device. The target driver must allocate and manage this buffer itself. See the target driver's attach(9E) routine in The attach() Entry Point for more information.
Pointer field for use by the target driver. It is commonly used to store a pointer to a private target driver state structure.
To execute SCSI commands, a target driver must first allocate a scsi_pkt(9S) structure for the command, specifying its own private data area length, the command status, and the command length. The HBA driver is responsible for implementing the packet allocation in the tran_init_pkt(9E) entry point. The HBA driver is also responsible for freeing the packet in its tran_destroy_pkt(9E) entry point. See scsi_pkt(9S) in Chapter 14, SCSI Target Drivers, for more information.
The scsi_pkt(9S) structure contains these fields:
struct scsi_pkt { opaque_t pkt_ha_private; /* private data for host adapter */ struct scsi_address pkt_address; /* destination address */ opaque_t pkt_private; /* private data for target driver */ void (*pkt_comp)(struct scsi_pkt *); /* completion routine */ uint_t pkt_flags; /* flags */ int pkt_time; /* time allotted to complete command */ uchar_t *pkt_scbp; /* pointer to status block */ uchar_t *pkt_cdbp; /* pointer to command block */ ssize_t pkt_resid; /* data bytes not transferred */ uint_t pkt_state; /* state of command */ uint_t pkt_statistics; /* statistics */ uchar_t pkt_reason; /* reason completion called */ };
Pointer to per-command HBA-driver private data.
Pointer to the scsi_address(9S) structure providing address information for this command.
Pointer to per-packet target-driver private data.
Pointer to the target driver completion routine called by the HBA driver when the transport layer has completed this command.
Flags for the command.
Specifies the completion timeout in seconds for the command.
Pointer to the status completion block for the command.
Pointer to the command descriptor block (CDB) for the command.
Count of the data bytes not transferred when the command has been completed or the amount of data for which resources have not been allocated. The HBA must modify this field during transport.
State of the command. The HBA must modify this field during transport.
Provides a history of the events the command experienced while in the transport layer. The HBA must modify this field during transport.
Reason for command completion. The HBA must modify this field during transport.
An HBA driver must allocate a scsi_hba_tran(9S) structure during attach(9E) and initialize the vectors in this transport structure to point to the required HBA driver entry points. This scsi_hba_tran(9S) structure is then passed into scsi_hba_attach_setup(9F).
The scsi_hba_tran(9S) structure contains a tran_hba_private field, which can be used to refer to the HBA driver's per-instance state.
Each scsi_address(9S) structure contains a pointer to the scsi_hba_tran(9S) structure and also provides the target (a_target) and logical unit (a_lun) addresses for the particular target device. Because every HBA driver entry point is passed a pointer to the scsi_address(9S) structure, either directly or indirectly through the scsi_device(9S) structure, the HBA driver can reference its own state and can identify the target device being addressed.
The following figure illustrates the HBA data structures for transport operations.
Cloning can be useful if an HBA driver needs to maintain per-target private data in the scsi_hba_tran(9S) structure, or if it needs to maintain a more complex address than is provided in the scsi_address(9S) structure.
When cloning, the HBA driver must still allocate a scsi_hba_tran structure at attach(9E) time and initialize the tran_hba_private soft state pointer and HBA entry point vectors as before. The difference occurs when the framework begins to connect an instance of a target driver to the HBA driver. Before calling the HBA driver's tran_tgt_init(9E) entry point, the framework duplicates (clones) thescsi_hba_tran structure associated with that instance of the HBA. This means that each scsi_address(9S) structure, allocated and initialized for a particular target device instance, points to a per-target instance copy of the scsi_hba_tran structure, not to the scsi_hba_tran structure allocated by the HBA driver at attach(9E) time.
Two important pointers that an HBA driver can use when it has specified cloning are contained in the scsi_hba_tran structure. The first pointer is the tran_tgt_private field, which the driver can use to point to per-target HBA private data. This is useful, for example, if an HBA driver needs to maintain a more complex address than the a_target and a_lun fields in the scsi_address(9S) structure allow. The second pointer is the tran_sd field, which is a pointer to the scsi_device(9S) structure referring to the particular target device.
When specifying cloning, the HBA driver must allocate and initialize the per-target data and initialize the tran_tgt_private field to point to this data during its tran_tgt_init(9E) entry point. The HBA driver must free this per-target data during its tran_tgt_free(9E) entry point.
When cloning, the framework initializes the tran_sd field to point to the scsi_device(9S) structure before the HBA driver tran_tgt_init(9E) entry point is called. The driver requests cloning by passing the SCSI_HBA_TRAN_CLONE flag to scsi_hba_attach_setup(9F). Figure 15–4 illustrates the HBA data structures for cloning transport operations.
SCSA also provides a number of functions, listed in Table 15–2, for use by HBA drivers.
Table 15–2 SCSA HBA Functions
Function Name |
Called by Driver Entry Point |
---|---|
scsi_hba_init(9F) |
_init(9E) |
scsi_hba_fini(9F) |
_fini(9E) |
scsi_hba_attach_setup(9F) |
attach(9E) |
scsi_hba_detach(9F) |
detach(9E) |
scsi_hba_tran_alloc(9F) |
attach(9E) |
scsi_hba_tran_free(9F) |
detach(9E) |
scsi_hba_probe(9F) |
tran_tgt_probe(9E) |
scsi_hba_pkt_alloc(9F) |
tran_init_pkt(9E) |
scsi_hba_pkt_free(9F) |
tran_destroy_pkt(9E) |
scsi_hba_lookup_capstr(9F) |
tran_getcap(9E) and tran_setcap(9E) |
In addition to incorporating SCSA HBA entry points, structures, and functions into a driver, HBA driver developers must also concern themselves with issues surrounding driver dependency and configuration. These issues involve configuration properties, dependency declarations, state structure and per-command structure, module initialization entry points, and autoconfiguration entry points.
HBA drivers must include the following header files:
#include <sys/scsi/scsi.h> #include <sys/ddi.h> #include <sys/sunddi.h>
To inform the system that the module depends on SCSA routines (see SCSA HBA Interfaces for more information), the driver binary must be generated with the following command:
% ld -r xx.o -o xx -N "misc/scsi"
The code samples are derived from a simplified isp driver for the QLogic Intelligent SCSI Peripheral device. The isp driver supports WIDE SCSI, with up to 15 target devices and 8 logical units (LUNs) per target.
An HBA driver will usually need to define a structure to maintain state for each command submitted by a target driver. The layout of this per-command structure is entirely up to the device driver writer and needs to reflect the capabilities and features of the hardware and the software algorithms used in the driver.
The following structure is an example of a per-command structure. The remaining code fragments of this chapter use this structure to illustrate the HBA interfaces.
struct isp_cmd { struct isp_request cmd_isp_request; struct isp_response cmd_isp_response; struct scsi_pkt *cmd_pkt; struct isp_cmd *cmd_forw; uint32_t cmd_dmacount; ddi_dma_handle_t cmd_dmahandle; uint_t cmd_cookie; uint_t cmd_ncookies; uint_t cmd_cookiecnt; uint_t cmd_nwin; uint_t cmd_curwin; off_t cmd_dma_offset; uint_t cmd_dma_len; ddi_dma_cookie_t cmd_dmacookies[ISP_NDATASEGS]; u_int cmd_flags; u_short cmd_slot; u_int cmd_cdblen; u_int cmd_scblen; };
Drivers for different types of devices have different sets of entry points, depending on the operations they perform. Some operations, however, are common to all drivers, such as the as _init(9E), _info(9E), and _fini(9E) entry points for module initialization. This section describes only those entry points associated with operations performed by SCSI HBA drivers.
The following code for a SCSI HBA driver illustrates a representative dev_ops(9S) structure. The driver must initialize the devo_bus_ops field in this structure to NULL. A SCSI HBA driver can provide leaf driver interfaces for special purposes, in which case the devo_cb_ops field might point to a cb_ops(9S) structure. In this example, no leaf driver interfaces are exported, so the devo_cb_ops field is initialized to NULL.
The _init(9E) function initializes a loadable module and is called before any other routine in the loadable module.
In a SCSI HBA, the _init() function must call scsi_hba_init(9F) to inform the framework of the existence of the HBA driver before calling mod_install(9F). If scsi_hba__init() returns a nonzero value,_init() should return this value. Otherwise, _init() must return the value returned by mod_install(9F).
The driver should initialize any required global state before calling mod_install(9F).
If mod_install() fails, the _init() function must free any global resources allocated and must call scsi_hba_fini(9F) before returning.
Example 15–1 uses a global mutex to show how to allocate data that is global to all instances of a driver. The code declares global mutex and soft-state structure information. The global mutex and soft state are initialized during _init().
The _fini(9E) function is called when the system is about to try to unload the SCSI HBA driver. The _fini() function must call mod_remove(9F) to determine if the driver can be unloaded. If mod_remove() returns 0, the module can be unloaded, and the HBA driver must deallocate any global resources allocated in _init(9E) and must call scsi_hba_fini(9F).
_fini() must return the value returned by mod_remove().
The HBA driver must not free any resources or call scsi_hba_fini(9F) unless mod_remove(9F) returns 0.
Example 15–1 shows SCSI HBA module initialization.
static struct dev_ops isp_dev_ops = { DEVO_REV, /* devo_rev */ 0, /* refcnt */ isp_getinfo, /* getinfo */ nulldev, /* probe */ isp_attach, /* attach */ isp_detach, /* detach */ nodev, /* reset */ NULL, /* driver operations */ NULL, /* bus operations */ isp_power, /* power management */ }; /* * Local static data */ static kmutex_t isp_global_mutex; static void *isp_state; int _init(void) { int err; if ((err = ddi_soft_state_init(&isp_state, sizeof (struct isp), 0)) != 0) { return (err); } if ((err = scsi_hba_init(&modlinkage)) == 0) { mutex_init(&isp_global_mutex, "isp global mutex", MUTEX_DRIVER, NULL); if ((err = mod_install(&modlinkage)) != 0) { mutex_destroy(&isp_global_mutex); scsi_hba_fini(&modlinkage); ddi_soft_state_fini(&isp_state); } } return (err); } int _fini(void) { int err; if ((err = mod_remove(&modlinkage)) == 0) { mutex_destroy(&isp_global_mutex); scsi_hba_fini(&modlinkage); ddi_soft_state_fini(&isp_state); } return (err); }
Associated with each device driver is a dev_ops(9S) structure, which allows the kernel to locate the autoconfiguration entry points of the driver. A complete description of these autoconfiguration routines is given in Chapter 5, Driver Autoconfiguration. This section describes only those entry points associated with operations performed by SCSI HBA drivers. These include attach(9E) and detach(9E).
The attach(9E) entry point for a SCSI HBA driver must perform a number of tasks to configure and attach an instance of the driver for the device. For a typical driver of real devices, the following operating system and hardware concerns must be addressed:
Soft-state structure
DMA
Transport structure
Attaching an HBA driver
Register mapping
Interrupt specification
Interrupt handling
Create power manageable components
Report attachment status
The driver should allocate the per-device-instance soft state structure, being careful to clean up properly if an error occurs.
The HBA driver must describe the attributes of its DMA engine by properly initializing the ddi_dma_attr_t structure.
static ddi_dma_attr_t isp_dma_attr = { DMA_ATTR_V0, /* ddi_dma_attr version */ 0, /* low address */ 0xffffffff, /* high address */ 0x00ffffff, /* counter upper bound */ 1, /* alignment requirements */ 0x3f, /* burst sizes */ 1, /* minimum DMA access */ 0xffffffff, /* maximum DMA access */ (1<<24)-1, /* segment boundary restrictions */ 1, /* scatter/gather list length */ 512, /* device granularity */ 0 /* DMA flags */ };
The driver, if providing DMA, should also check that its hardware is installed in a DMA-capable slot:
if (ddi_slaveonly(dip) == DDI_SUCCESS) { return (DDI_FAILURE); }
The driver should further allocate and initialize a transport structure for this instance. The tran_hba_private field is set to point to this instance's soft-state structure. tran_tgt_probe can be set to NULL to achieve the default behavior, if no special probe customization is needed.
tran = scsi_hba_tran_alloc(dip, SCSI_HBA_CANSLEEP); isp->isp_tran = tran; isp->isp_dip = dip; tran->tran_hba_private = isp; tran->tran_tgt_private = NULL; tran->tran_tgt_init = isp_tran_tgt_init; tran->tran_tgt_probe = scsi_hba_probe; tran->tran_tgt_free = (void (*)())NULL; tran->tran_start = isp_scsi_start; tran->tran_abort = isp_scsi_abort; tran->tran_reset = isp_scsi_reset; tran->tran_getcap = isp_scsi_getcap; tran->tran_setcap = isp_scsi_setcap; tran->tran_init_pkt = isp_scsi_init_pkt; tran->tran_destroy_pkt = isp_scsi_destroy_pkt; tran->tran_dmafree = isp_scsi_dmafree; tran->tran_sync_pkt = isp_scsi_sync_pkt; tran->tran_reset_notify = isp_scsi_reset_notify; tran->tran_bus_quiesce = isp_tran_bus_quiesce tran->tran_bus_unquiesce = isp_tran_bus_unquiesce tran->tran_bus_reset = isp_tran_bus_reset
The driver should attach this instance of the device, and perform error cleanup if necessary.
i = scsi_hba_attach_setup(dip, &isp_dma_attr, tran, 0); if (i != DDI_SUCCESS) { do error recovery return (DDI_FAILURE); }
The driver should map in its device's registers, specifying the index of the register set, the data access characteristics of the device and the size of the register set to be mapped.
ddi_device_acc_attr_t dev_attributes; dev_attributes.devacc_attr_version = DDI_DEVICE_ATTR_V0; dev_attributes.devacc_attr_dataorder = DDI_STRICTORDER_ACC; dev_attributes.devacc_attr_endian_flags = DDI_STRUCTURE_LE_ACC; if (ddi_regs_map_setup(dip, 0, (caddr_t *)&isp->isp_reg, 0, sizeof (struct ispregs), &dev_attributes, &isp->isp_acc_handle) != DDI_SUCCESS) { do error recovery return (DDI_FAILURE); }
The driver must first obtain the iblock cookie to initialize mutexes used in the driver handler. Only after those mutexes have been initialized can the interrupt handler be added.
i = ddi_get_iblock_cookie(dip, 0, &isp->iblock_cookie}; if (i != DDI_SUCCESS) { do error recovery return (DDI_FAILURE); } mutex_init(&isp->mutex, "isp_mutex", MUTEX_DRIVER, (void *)isp->iblock_cookie); i = ddi_add_intr(dip, 0, &isp->iblock_cookie, 0, isp_intr, (caddr_t)isp); if (i != DDI_SUCCESS) { do error recovery return (DDI_FAILURE); }
The driver should determine if a high-level interrupt handler is required. If a high-level handler is required and the driver is not coded to provide one, rewrite the driver to include either a high-level interrupt or fail the attach. See Handling High-Level Interrupts for a description of high-level interrupt handling.
If the host bus adapter hardware supports power management, and the host bus adapter only needs to be powered down when all of the target adapters are power manageable and are at power level 0, then the host bus adapter driver only needs to provide a power(9E) entry point as described in Chapter 9, Power Management and create a pm-components(9P) property that describes the components that the device implements.
Nothing more is necessary, since the components will default to idle, and the power management framework's default dependency processing will ensure that the host bus adapter will be powered up whenever an target adapter is powered up and will automatically power down the host bus adapter whenever all of the target adapters are powered down (provided that automatic power management is enabled).
Finally, the driver should report that this instance of the device is attached and return success.
ddi_report_dev(dip); return (DDI_SUCCESS);
The driver should perform standard detach operations, including calling scsi_hba_detach(9F).
For an HBA driver to work with target drivers using the SCSA interface, each HBA driver must supply a number of entry points, callable through the scsi_hba_tran(9S) structure.These entry points fall into five functional groups:
Target driver instance initialization
Resource allocation and deallocation
Command transport
Capability management
Abort and reset handling
Dynamic reconfiguration
Table 15–3 lists the SCSA HBA entry points arranged by function groups.
Table 15–3 SCSA Entry Points
Function Groups |
Entry Points Within Group |
Description |
---|---|---|
Target Driver Instance Initialization |
tran_tgt_init(9E) |
Performs per-target initialization (optional) |
|
tran_tgt_probe(9E) |
Probes SCSI bus for existence of a target (optional) |
|
tran_tgt_free(9E) |
Performs per-target deallocation (optional) |
Resource Allocation |
tran_init_pkt(9E) |
Allocates SCSI packet and DMA resources |
|
tran_destroy_pkt(9E) |
Frees SCSI packet and DMA resources |
|
tran_sync_pkt(9E) |
Synchronizes memory before and after DMA |
|
tran_dmafree(9E) |
Frees DMA resources |
Command Transport |
tran_start(9E) |
Transports a SCSI command |
Capability Management |
tran_getcap(9E) |
Inquires about a capability's value |
|
tran_setcap(9E) |
Sets a capability's value |
Abort and Reset |
tran_abort(9E) |
Aborts one or all outstanding SCSI commands |
|
tran_reset(9E) |
Resets a target device or the SCSI bus |
|
tran_bus_reset(9E) |
Resets the SCSI bus |
|
tran_reset_notify(9E) |
Request to notify target of bus reset (optional) |
Dynamic Reconfiguration |
tran_quiesce(9E) |
Stops activity on the bus |
|
tran_unquiesce(9E) |
Resume activity on the bus |
The following sections explain target entry points.
The tran_tgt_init(9E) entry point enables the HBA to allocate and/or initialize any per-target resources. It also enables the HBA to qualify the device's address as valid and supportable for that particular HBA. By returning DDI_FAILURE, the instance of the target driver for that device will not be probed or attached.
This entry point is not required, and if none is supplied, the framework will attempt to probe and attach all possible instances of the appropriate target drivers.
static int isp_tran_tgt_init( dev_info_t *hba_dip, dev_info_t *tgt_dip, scsi_hba_tran_t *tran, struct scsi_device *sd) { return ((sd->sd_address.a_target < N_ISP_TARGETS_WIDE && sd->sd_address.a_lun < 8) ? DDI_SUCCESS : DDI_FAILURE); }
The tran_tgt_probe(9E) entry point enables the HBA to customize the operation of scsi_probe(9F), if necessary. This entry point is called only when the target driver calls scsi_probe(9F).
The HBA driver can retain the normal operation of scsi_probe(9F) by calling scsi_hba_probe(9F) and returning its return value.
This entry point is not required, and if not needed, the HBA driver should set the tran_tgt_probe vector in the scsi_hba_tran(9S) structure to point to scsi_hba_probe(9F).
scsi_probe(9F) allocates a scsi_inquiry(9S) structure and sets the sd_inq field of the scsi_device(9S) structure to point to the data in scsi_inquiry(9S). scsi_hba_probe(9F) handles this automatically. scsi_unprobe(9F) then frees the scsi_inquiry(9S) data.
Other than during the allocation of scsi_inquiry(9S) data, normally handled by scsi_hba_probe(9F), tran_tgt_probe(9E) must be stateless, as the same SCSI device might call it multiple times.
The allocation of the scsi_inquiry(9S) structure is handled automatically by scsi_hba_probe(9F). This is only of concern if you want custom scsi_probe(9F) handling.
static int isp_tran_tgt_probe( struct scsi_device *sd, int (*callback)()) { Perform any special probe customization needed. /* * Normal probe handling */ return (scsi_hba_probe(sd, callback)); }
The tran_tgt_free(9E) entry point enables the HBA to perform any deallocation or clean-up procedures for an instance of a target. This entry point is optional.
static void isp_tran_tgt_free( dev_info_t *hba_dip, dev_info_t *tgt_dip, scsi_hba_tran_t *hba_tran, struct scsi_device *sd) { Undo any special per-target initialization done earlier in tran_tgt_init(9F) and tran_tgt_probe(9F) }
The following sections discuss resource allocation.
The tran_init_pkt(9E) entry point is the HBA driver function that allocates and initializes, on behalf of the target driver, a scsi_pkt(9S) structure and DMA resources for a target driver request.
The tran_init_pkt(9E) entry point is called when the target driver calls the SCSA function scsi_init_pkt(9F).
Each call of the tran_init_pkt(9E) entry point is a request to perform one or more of three possible services:
Allocation and initialization of a scsi_pkt(9S) structure
Allocation of DMA resources for data transfer
Reallocation of DMA resources for the next portion of the data transfer
The tran_init_pkt(9E) entry point must allocate a scsi_pkt(9S) structure if pkt is NULL through scsi_hba_pkt_alloc(9F).
scsi_hba_pkt_alloc(9F) allocates the following:
SCSI CDB of length cmdlen
SCSI status completion area of length statuslen
Per-packet target driver private data area of length tgtlen
Per-packet HBA driver private data area of length hbalen
The scsi_pkt(9S) structure members, as well as pkt itself, must be initialized to zero except for the following members: pkt_scbp (status completion), pkt_cdbp (CDB), pkt_ha_private (HBA driver private data), pkt_private (target driver private data). These members are pointers to memory space where the values of the fields are stored, as illustrated in Figure 15–5. For more information, refer to scsi_pkt Structure.
Example 15–2 provides an example of allocation and initialization of a scsi_pkt(9S) structure.
static struct scsi_pkt * isp_scsi_init_pkt( struct scsi_address *ap, struct scsi_pkt *pkt, struct buf *bp, int cmdlen, int statuslen, int tgtlen, int flags, int (*callback)(), caddr_t arg) { struct isp_cmd *sp; struct isp *isp; struct scsi_pkt *new_pkt; ASSERT(callback == NULL_FUNC || callback == SLEEP_FUNC); isp = (struct isp *)ap->a_hba_tran->tran_hba_private; /* * First step of isp_scsi_init_pkt: pkt allocation */ if (pkt == NULL) { pkt = scsi_hba_pkt_alloc(isp->isp_dip, ap, cmdlen, statuslen, tgtlen, sizeof (struct isp_cmd), callback, arg); if (pkt == NULL) { return (NULL); } sp = (struct isp_cmd *)pkt->pkt_ha_private; /* * Initialize the new pkt */ sp->cmd_pkt = pkt; sp->cmd_flags = 0; sp->cmd_scblen = statuslen; sp->cmd_cdblen = cmdlen; sp->cmd_dmahandle = NULL; sp->cmd_ncookies = 0; sp->cmd_cookie = 0; sp->cmd_cookiecnt = 0; sp->cmd_nwin = 0; pkt->pkt_address = *ap; pkt->pkt_comp = (void (*)())NULL; pkt->pkt_flags = 0; pkt->pkt_time = 0; pkt->pkt_resid = 0; pkt->pkt_statistics = 0; pkt->pkt_reason = 0; new_pkt = pkt; } else { sp = (struct isp_cmd *)pkt->pkt_ha_private; new_pkt = NULL; } /* * Second step of isp_scsi_init_pkt: dma allocation/move */ if (bp && bp->b_bcount != 0) { if (sp->cmd_dmahandle == NULL) { if (isp_i_dma_alloc(isp, pkt, bp, flags, callback) == 0) { if (new_pkt) { scsi_hba_pkt_free(ap, new_pkt); } return ((struct scsi_pkt *)NULL); } } else { ASSERT(new_pkt == NULL); if (isp_i_dma_move(isp, pkt, bp) == 0) { return ((struct scsi_pkt *)NULL); } } } return (pkt); }
If bp is not NULL and bp->b_bcount is not zero and DMA resources have not yet been allocated for this scsi_pkt(9S), the tran_init_pkt(9E) entry point must allocate DMA resources for a data transfer. The HBA driver needs to keep track of whether DMA resources have been allocated for a particular command with a flag bit or a DMA handle in the per-packet HBA driver private data.
By setting the PKT_DMA_PARTIAL flag in the pkt, the target driver indicates it can accept breaking up the data transfer into multiple SCSI commands to accommodate the complete request. This might be necessary if the HBA hardware scatter-gather capabilities or system DMA resources are insufficient to accommodate the complete request in a single SCSI command.
If the PKT_DMA_PARTIAL flag is set, the HBA driver can set the DDI_DMA_PARTIAL flag when allocating DMA resources (using, for example, ddi_dma_buf_bind_handle(9F)) for this SCSI command. The DMA attributes used when allocating the DMA resources should accurately describe any constraints placed on the ability of the HBA hardware to perform DMA. If the system can only allocate DMA resources for part of the request, ddi_dma_buf_bind_handle(9F) will return DDI_DMA_PARTIAL_MAP.
The tran_init_pkt(9E) entry point must return the amount of DMA resources not allocated for this transfer in the field pkt_resid.
A target driver can make one request to tran_init_pkt(9E) to simultaneously allocate both a scsi_pkt(9S) structure and DMA resources for that pkt. In this case, if the HBA driver is unable to allocate DMA resources, it must free the allocated scsi_pkt(9S) before returning. The scsi_pkt(9S) must be freed by calling scsi_hba_pkt_free(9F).
The target driver might first allocate the scsi_pkt(9S) and allocate DMA resources for this pkt at a later time. In this case, if the HBA driver is unable to allocate DMA resources, it must not free pkt. The target driver in this case is responsible for freeing the pkt.
static int isp_i_dma_alloc( struct isp *isp, struct scsi_pkt *pkt, struct buf *bp, int flags, int (*callback)()) { struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private; int dma_flags; ddi_dma_attr_t tmp_dma_attr; int (*cb)(caddr_t); int i; ASSERT(callback == NULL_FUNC || callback == SLEEP_FUNC); if (bp->b_flags & B_READ) { sp->cmd_flags &= ~CFLAG_DMASEND; dma_flags = DDI_DMA_READ; } else { sp->cmd_flags |= CFLAG_DMASEND; dma_flags = DDI_DMA_WRITE; } if (flags & PKT_CONSISTENT) { sp->cmd_flags |= CFLAG_CMDIOPB; dma_flags |= DDI_DMA_CONSISTENT; } if (flags & PKT_DMA_PARTIAL) { dma_flags |= DDI_DMA_PARTIAL; } tmp_dma_attr = isp_dma_attr; tmp_dma_attr.dma_attr_burstsizes = isp->isp_burst_size; cb = (callback == NULL_FUNC) ? DDI_DMA_DONTWAIT : DDI_DMA_SLEEP; if ((i = ddi_dma_alloc_handle(isp->isp_dip, &tmp_dma_attr, cb, 0, &sp->cmd_dmahandle)) != DDI_SUCCESS) { switch (i) { case DDI_DMA_BADATTR: bioerror(bp, EFAULT); return (0); case DDI_DMA_NORESOURCES: bioerror(bp, 0); return (0); } } i = ddi_dma_buf_bind_handle(sp->cmd_dmahandle, bp, dma_flags, cb, 0, &sp->cmd_dmacookies[0], &sp->cmd_ncookies); switch (i) { case DDI_DMA_PARTIAL_MAP: if (ddi_dma_numwin(sp->cmd_dmahandle, &sp->cmd_nwin) == DDI_FAILURE) { cmn_err(CE_PANIC, "ddi_dma_numwin() failed\n"); } if (ddi_dma_getwin(sp->cmd_dmahandle, sp->cmd_curwin, &sp->cmd_dma_offset, &sp->cmd_dma_len, &sp->cmd_dmacookies[0], &sp->cmd_ncookies) == DDI_FAILURE) { cmn_err(CE_PANIC, "ddi_dma_getwin() failed\n"); } goto get_dma_cookies; case DDI_DMA_MAPPED: sp->cmd_nwin = 1; sp->cmd_dma_len = 0; sp->cmd_dma_offset = 0; get_dma_cookies: i = 0; sp->cmd_dmacount = 0; for (;;) { sp->cmd_dmacount += sp->cmd_dmacookies[i++].dmac_size; if (i == ISP_NDATASEGS || i == sp->cmd_ncookies) break; ddi_dma_nextcookie(sp->cmd_dmahandle, &sp->cmd_dmacookies[i]); } sp->cmd_cookie = i; sp->cmd_cookiecnt = i; sp->cmd_flags |= CFLAG_DMAVALID; pkt->pkt_resid = bp->b_bcount - sp->cmd_dmacount; return (1); case DDI_DMA_NORESOURCES: bioerror(bp, 0); break; case DDI_DMA_NOMAPPING: bioerror(bp, EFAULT); break; case DDI_DMA_TOOBIG: bioerror(bp, EINVAL); break; case DDI_DMA_INUSE: cmn_err(CE_PANIC, "ddi_dma_buf_bind_handle:" " DDI_DMA_INUSE impossible\n"); default: cmn_err(CE_PANIC, "ddi_dma_buf_bind_handle:" " 0x%x impossible\n", i); } ddi_dma_free_handle(&sp->cmd_dmahandle); sp->cmd_dmahandle = NULL; sp->cmd_flags &= ~CFLAG_DMAVALID; return (0); }
For a previously allocated packet with data remaining to be transferred, the tran_init_pkt(9E) entry point must reallocate DMA resources when the following conditions apply:
Partial DMA resources have already been allocated.
A non-zero pkt_resid was returned in the previous call to tran_init_pkt(9E).
bp is not NULL.
bp->b_bcount is not zero.
When reallocating DMA resources to the next portion of the transfer, tran_init_pkt(9E) must return the amount of DMA resources not allocated for this transfer in the field pkt_resid.
If an error occurs while attempting to move DMA resources, tran_init_pkt(9E) must not free the scsi_pkt(9S). The target driver in this case is responsible for freeing the pkt.
If the callback parameter is NULL_FUNC, the tran_init_pkt(9E) entry point must not sleep or call any function that might sleep. If the callback parameter is SLEEP_FUNC and resources are not immediately available, the tran_init_pkt(9E) entry point should sleep until resources are available, unless the request is impossible to satisfy.
static int isp_i_dma_move( struct isp *isp, struct scsi_pkt *pkt, struct buf *bp) { struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private; int i; ASSERT(sp->cmd_flags & CFLAG_COMPLETED); sp->cmd_flags &= ~CFLAG_COMPLETED; /* * If there are no more cookies remaining in this window, * must move to the next window first. */ if (sp->cmd_cookie == sp->cmd_ncookies) { /* * For small pkts, leave things where they are */ if (sp->cmd_curwin == sp->cmd_nwin && sp->cmd_nwin == 1) return (1); /* * At last window, cannot move */ if (++sp->cmd_curwin >= sp->cmd_nwin) return (0); if (ddi_dma_getwin(sp->cmd_dmahandle, sp->cmd_curwin, &sp->cmd_dma_offset, &sp->cmd_dma_len, &sp->cmd_dmacookies[0], &sp->cmd_ncookies) == DDI_FAILURE) return (0); sp->cmd_cookie = 0; } else { /* * Still more cookies in this window - get the next one */ ddi_dma_nextcookie(sp->cmd_dmahandle, &sp->cmd_dmacookies[0]); } /* * Get remaining cookies in this window, up to our maximum */ i = 0; for (;;) { sp->cmd_dmacount += sp->cmd_dmacookies[i++].dmac_size; sp->cmd_cookie++; if (i == ISP_NDATASEGS || sp->cmd_cookie == sp->cmd_ncookies) break; ddi_dma_nextcookie(sp->cmd_dmahandle, &sp->cmd_dmacookies[i]); } sp->cmd_cookiecnt = i; pkt->pkt_resid = bp->b_bcount - sp->cmd_dmacount; return (1); }
The tran_destroy_pkt(9E) entry point is the HBA driver function that deallocates scsi_pkt(9S) structures. The tran_destroy_pkt() entry point is called when the target driver calls scsi_destroy_pkt(9F).
Thetran_destroy_pkt() entry point must free any DMA resources allocated for the packet. Freeing the DMA resources causes an implicit DMA synchronization if any cached data remained after the completion of the transfer. The tran_destroy_pkt() entry point frees the SCSI packet itself by calling scsi_hba_pkt_free(9F).
static void isp_scsi_destroy_pkt( struct scsi_address *ap, struct scsi_pkt *pkt) { struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private; /* * Free the DMA, if any */ if (sp->cmd_flags & CFLAG_DMAVALID) { sp->cmd_flags &= ~CFLAG_DMAVALID; (void) ddi_dma_unbind_handle(sp->cmd_dmahandle); ddi_dma_free_handle(&sp->cmd_dmahandle); sp->cmd_dmahandle = NULL; } /* * Free the pkt */ scsi_hba_pkt_free(ap, pkt); }
The tran_sync_pkt(9E) entry point is the HBA driver function that synchronizes the DMA object allocated for the scsi_pkt(9S) structure before or after a DMA transfer. The tran_sync_pkt() entry point is called when the target driver calls scsi_sync_pkt(9F).
If the data transfer direction is a DMA read from device to memory, tran_sync_pkt() must synchronize the CPU's view of the data. If the data transfer direction is a DMA write from memory to device, tran_sync_pkt() must synchronize the device's view of the data.
static void isp_scsi_sync_pkt( struct scsi_address *ap, struct scsi_pkt *pkt) { struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private; if (sp->cmd_flags & CFLAG_DMAVALID) { (void)ddi_dma_sync(sp->cmd_dmahandle, sp->cmd_dma_offset, sp->cmd_dma_len, (sp->cmd_flags & CFLAG_DMASEND) ? DDI_DMA_SYNC_FORDEV : DDI_DMA_SYNC_FORCPU); } } }
The tran_dmafree(9E) entry point is the HBA driver function that deallocates DMA resources allocated for a scsi_pkt(9S) structure. The tran_dmafree() entry point is called when the target driver calls scsi_dmafree(9F).
tran_dmafree() must free only DMA resources allocated for a scsi_pkt(9S) structure, not the scsi_pkt(9S) itself. Freeing the DMA resources implicitly performs a DMA synchronization.
The scsi_pkt(9S) will be freed in a separate request to tran_destroy_pkt(9E). Because tran_destroy_pkt() must also free DMA resources, the HBA driver must keep accurate note of whether scsi_pkt() structures have DMA resources allocated.
static void isp_scsi_dmafree( struct scsi_address *ap, struct scsi_pkt *pkt) { struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private; if (sp->cmd_flags & CFLAG_DMAVALID) { sp->cmd_flags &= ~CFLAG_DMAVALID; (void)ddi_dma_unbind_handle(sp->cmd_dmahandle); ddi_dma_free_handle(&sp->cmd_dmahandle); sp->cmd_dmahandle = NULL; } }
As part of command transport, the HBA driver accepts a command from the target driver, issues the command to the device hardware, services any interrupts that occur, and manages timeouts.
The tran_start(9E) entry point for a SCSI HBA driver is called to transport a SCSI command to the addressed target. The SCSI command is described entirely within the scsi_pkt(9S) structure, which the target driver allocated through the HBA driver's tran_init_pkt(9E) entry point. If the command involves a data transfer, DMA resources must also have been allocated for the scsi_pkt(9S) structure.
The tran_start() entry point is called when a target driver calls scsi_transport(9F).
tran_start() should perform basic error checking along with whatever initialization the command requires. If the flag FLAG_NOINTR is not set in the pkt_flags field of the scsi_pkt(9S) structure,tran_start() must queue the command for execution on the hardware and return immediately. Upon completion of the command, the HBA driver should call the pkt() completion routine.
For commands with the FLAG_NOINTR bit set in the pkt_flags field of the scsi_pkt(9S) structure, tran_start(9E) should not return until the command has been completed, and the HBA driver should not call the pkt() completion routine.
Example 15–8 demonstrates how to handle the tran_start(9E) entry point. The ISP hardware provides a queue per-target device. For devices that can manage only one active outstanding command, the driver itself is typically required to manage a per-target queue and starts up a new command upon completion of the current command in a round-robin fashion.
static int isp_scsi_start( struct scsi_address *ap, struct scsi_pkt *pkt) { struct isp_cmd *sp; struct isp *isp; struct isp_request *req; u_long cur_lbolt; int xfercount; int rval = TRAN_ACCEPT; int i; sp = (struct isp_cmd *)pkt->pkt_ha_private; isp = (struct isp *)ap->a_hba_tran->tran_hba_private; sp->cmd_flags = (sp->cmd_flags & ~CFLAG_TRANFLAG) | CFLAG_IN_TRANSPORT; pkt->pkt_reason = CMD_CMPLT; /* * set up request in cmd_isp_request area so it is ready to * go once we have the request mutex */ req = &sp->cmd_isp_request; req->req_header.cq_entry_type = CQ_TYPE_REQUEST; req->req_header.cq_entry_count = 1; req->req_header.cq_flags = 0; req->req_header.cq_seqno = 0; req->req_reserved = 0; req->req_token = (opaque_t)sp; req->req_target = TGT(sp); req->req_lun_trn = LUN(sp); req->req_time = pkt->pkt_time; ISP_SET_PKT_FLAGS(pkt->pkt_flags, req->req_flags); /* * Set up dma transfers data segments. */ if (sp->cmd_flags & CFLAG_DMAVALID) { if (sp->cmd_flags & CFLAG_CMDIOPB) { (void) ddi_dma_sync(sp->cmd_dmahandle, sp->cmd_dma_offset, sp->cmd_dma_len, DDI_DMA_SYNC_FORDEV); } ASSERT(sp->cmd_cookiecnt > 0 && sp->cmd_cookiecnt <= ISP_NDATASEGS); xfercount = 0; req->req_seg_count = sp->cmd_cookiecnt; for (i = 0; i < sp->cmd_cookiecnt; i++) { req->req_dataseg[i].d_count = sp->cmd_dmacookies[i].dmac_size; req->req_dataseg[i].d_base = sp->cmd_dmacookies[i].dmac_address; xfercount += sp->cmd_dmacookies[i].dmac_size; } for (; i < ISP_NDATASEGS; i++) { req->req_dataseg[i].d_count = 0; req->req_dataseg[i].d_base = 0; } pkt->pkt_resid = xfercount; if (sp->cmd_flags & CFLAG_DMASEND) { req->req_flags |= ISP_REQ_FLAG_DATA_WRITE; } else { req->req_flags |= ISP_REQ_FLAG_DATA_READ; } } else { req->req_seg_count = 0; req->req_dataseg[0].d_count = 0; } /* * Set up cdb in the request */ req->req_cdblen = sp->cmd_cdblen; bcopy((caddr_t)pkt->pkt_cdbp, (caddr_t)req->req_cdb, sp->cmd_cdblen); /* * Start the cmd. If NO_INTR, must poll for cmd completion. */ if ((pkt->pkt_flags & FLAG_NOINTR) == 0) { mutex_enter(ISP_REQ_MUTEX(isp)); rval = isp_i_start_cmd(isp, sp); mutex_exit(ISP_REQ_MUTEX(isp)); } else { rval = isp_i_polled_cmd_start(isp, sp); } return (rval); }
The interrupt handler must check the status of the device to be sure the device is generating the interrupt in question. It must also check for any errors that have occurred and service any interrupts generated by the device.
If data was transferred, the hardware should be checked to determine how much data was actually transferred, and the pkt_resid field in the scsi_pkt(9S) structure should be set to the residual of the transfer.
For commands marked with the PKT_CONSISTENT flag when DMA resources were allocated through tran_init_pkt(9E), the HBA driver must ensure that the data transfer for the command is correctly synchronized before the target driver's command completion callback is performed.
Once a command has completed, there are two requirements:
Start a new command (if one is queued up) on the hardware as quickly as possible.
Call the command completion callback as set up in the scsi_pkt(9S) structure by the target driver to notify the target driver that the command is now complete.
Start a new command on the hardware, if possible, before calling the PKT_COMP command completion callback. The command completion handling can take considerable time, as the target driver will typically call functions such as biodone(9F) and possibly scsi_transport(9F) to begin a new command.
The interrupt handler must return DDI_INTR_CLAIMED if this interrupt is claimed by this driver; otherwise, the handler returns DDI_INTR_UNCLAIMED.
Example 15–9 shows an interrupt handler for the SCSI HBA isp driver. The caddr_t argument is the parameter set up when the interrupt handler was added in attach(9E) and is typically a pointer to the state structure allocated per instance.
static u_int isp_intr(caddr_t arg) { struct isp_cmd *sp; struct isp_cmd *head, *tail; u_short response_in; struct isp_response *resp; struct isp *isp = (struct isp *)arg; struct isp_slot *isp_slot; int n; if (ISP_INT_PENDING(isp) == 0) { return (DDI_INTR_UNCLAIMED); } do { again: /* * head list collects completed packets for callback later */ head = tail = NULL; /* * Assume no mailbox events (e.g. mailbox cmds, asynch * events, and isp dma errors) as common case. */ if (ISP_CHECK_SEMAPHORE_LOCK(isp) == 0) { mutex_enter(ISP_RESP_MUTEX(isp)); /* * Loop through completion response queue and post * completed pkts. Check response queue again * afterwards in case there are more */ isp->isp_response_in = response_in = ISP_GET_RESPONSE_IN(isp); /* * Calculate the number of requests in the queue */ n = response_in - isp->isp_response_out; if (n < 0) { n = ISP_MAX_REQUESTS - isp->isp_response_out + response_in; } while (n-- > 0) { ISP_GET_NEXT_RESPONSE_OUT(isp, resp); sp = (struct isp_cmd *)resp->resp_token; /* * copy over response packet in sp */ isp_i_get_response(isp, resp, sp); } if (head) { tail->cmd_forw = sp; tail = sp; tail->cmd_forw = NULL; } else { tail = head = sp; sp->cmd_forw = NULL; } } ISP_SET_RESPONSE_OUT(isp); ISP_CLEAR_RISC_INT(isp); mutex_exit(ISP_RESP_MUTEX(isp)); if (head) { isp_i_call_pkt_comp(isp, head); } } else { if (isp_i_handle_mbox_cmd(isp) != ISP_AEN_SUCCESS) { return (DDI_INTR_CLAIMED); } /* * if there was a reset then check the response * queue again */ goto again; } } while (ISP_INT_PENDING(isp)); return (DDI_INTR_CLAIMED); } static void isp_i_call_pkt_comp( struct isp *isp, struct isp_cmd *head) { struct isp *isp; struct isp_cmd *sp; struct scsi_pkt *pkt; struct isp_response *resp; u_char status; while (head) { sp = head; pkt = sp->cmd_pkt; head = sp->cmd_forw; ASSERT(sp->cmd_flags & CFLAG_FINISHED); resp = &sp->cmd_isp_response; pkt->pkt_scbp[0] = (u_char)resp->resp_scb; pkt->pkt_state = ISP_GET_PKT_STATE(resp->resp_state); pkt->pkt_statistics = (u_long) ISP_GET_PKT_STATS(resp->resp_status_flags); pkt->pkt_resid = (long)resp->resp_resid; /* * if data was xferred and this is a consistent pkt, * we need to do a dma sync */ if ((sp->cmd_flags & CFLAG_CMDIOPB) && (pkt->pkt_state & STATE_XFERRED_DATA)) { (void) ddi_dma_sync(sp->cmd_dmahandle, sp->cmd_dma_offset, sp->cmd_dma_len, DDI_DMA_SYNC_FORCPU); } sp->cmd_flags = (sp->cmd_flags & ~CFLAG_IN_TRANSPORT) | CFLAG_COMPLETED; /* * Call packet completion routine if FLAG_NOINTR is not set. */ if (((pkt->pkt_flags & FLAG_NOINTR) == 0) && pkt->pkt_comp) { (*pkt->pkt_comp)(pkt); } } }
The HBA driver should be prepared to time out the command if it is not complete within a specified time unless a zero timeout was specified in the scsi_pkt(9S) structure.
When a command times out, the HBA driver should mark the scsi_pkt(9S) with pkt_reason set to CMD_TIMEOUT and pkt_statistics OR'd with STAT_TIMEOUT. The HBA driver should also attempt to recover the target and/or bus and, if this recovery can be performed successfully, mark the scsi_pkt(9S) with pkt_statistics OR'd with either STAT_BUS_RESET or STAT_DEV_RESET.
Once the command has timed out and the target and bus recovery attempt has completed, the HBA driver should call the command completion callback.
If recovery was unsuccessful or not attempted, the target driver might attempt to recover from the timeout by calling scsi_reset(9F).
The ISP hardware manages command timeout directly and returns timed-out commands with the necessary status, so the isp sample driver timeout handler checks active commands for timeout state only once every 60 seconds.
The isp sample driver uses the timeout(9F) facility to arrange for the kernel to call the timeout handler every 60 seconds. The caddr_t argument is the parameter set up when the timeout is initialized at attach(9E) time. In this case, the caddr_t argument is a pointer to the state structure allocated per driver instance.
If the driver discovers timed-out commands that have not been returned as timed-out by the ISP hardware, the hardware is not functioning correctly and needs to be reset.
The following sections discuss capability management.
The tran_getcap(9E) entry point for a SCSI HBA driver is called when a target driver calls scsi_ifgetcap(9F) to determine the current value of one of a set of SCSA-defined capabilities.
The target driver can request the current setting of the capability for a particular target by setting the whom parameter to nonzero. A whom value of 0 means the request is for the current setting of the capability for the SCSI bus or for adapter hardware in general.
tran_getcap() should return -1 for undefined capabilities or the current value of the requested capability.
The HBA driver can use the function scsi_hba_lookup_capstr(9F) to compare the capability string against the canonical set of defined capabilities.
static int isp_scsi_getcap( struct scsi_address *ap, char *cap, int whom) { struct isp *isp; int rval = 0; u_char tgt = ap->a_target; /* * We don't allow getting capabilities for other targets */ if (cap == NULL || whom == 0) { return (-1); } isp = (struct isp *)ap->a_hba_tran->tran_hba_private; ISP_MUTEX_ENTER(isp); switch (scsi_hba_lookup_capstr(cap)) { case SCSI_CAP_DMA_MAX: rval = 1 << 24; /* Limit to 16MB max transfer */ break; case SCSI_CAP_MSG_OUT: rval = 1; break; case SCSI_CAP_DISCONNECT: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_DR) == 0) { break; } else if ( (isp->isp_cap[tgt] & ISP_CAP_DISCONNECT) == 0) { break; } rval = 1; break; case SCSI_CAP_SYNCHRONOUS: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_SYNC) == 0) { break; } else if ( (isp->isp_cap[tgt] & ISP_CAP_SYNC) == 0) { break; } rval = 1; break; case SCSI_CAP_WIDE_XFER: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_WIDE) == 0) { break; } else if ( (isp->isp_cap[tgt] & ISP_CAP_WIDE) == 0) { break; } rval = 1; break; case SCSI_CAP_TAGGED_QING: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_DR) == 0 || (isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_TAG) == 0) { break; } else if ( (isp->isp_cap[tgt] & ISP_CAP_TAG) == 0) { break; } rval = 1; break; case SCSI_CAP_UNTAGGED_QING: rval = 1; break; case SCSI_CAP_PARITY: if (isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_PARITY) { rval = 1; } break; case SCSI_CAP_INITIATOR_ID: rval = isp->isp_initiator_id; break; case SCSI_CAP_ARQ: if (isp->isp_cap[tgt] & ISP_CAP_AUTOSENSE) { rval = 1; } break; case SCSI_CAP_LINKED_CMDS: break; case SCSI_CAP_RESET_NOTIFICATION: rval = 1; break; case SCSI_CAP_GEOMETRY: rval = (64 << 16) | 32; break; default: rval = -1; break; } ISP_MUTEX_EXIT(isp); return (rval); }
The tran_setcap(9E) entry point for a SCSI HBA driver is called when a target driver calls scsi_ifsetcap(9F) to change the current one of a set of SCSA-defined capabilities.
The target driver might request that the new value be set for a particular target by setting the whom parameter to nonzero. A whom value of 0 means the request is to set the new value for the SCSI bus or for adapter hardware in general.
tran_setcap() should return -1 for undefined capabilities, 0 if the HBA driver cannot set the capability to the requested value, or 1 if the HBA driver is able to set the capability to the requested value.
The HBA driver can use the function scsi_hba_lookup_capstr(9F) to compare the capability string against the canonical set of defined capabilities.
static int isp_scsi_setcap( struct scsi_address *ap, char *cap, int value, int whom) { struct isp *isp; int rval = 0; u_char tgt = ap->a_target; int update_isp = 0; /* * We don't allow setting capabilities for other targets */ if (cap == NULL || whom == 0) { return (-1); } isp = (struct isp *)ap->a_hba_tran->tran_hba_private; ISP_MUTEX_ENTER(isp); switch (scsi_hba_lookup_capstr(cap)) { case SCSI_CAP_DMA_MAX: case SCSI_CAP_MSG_OUT: case SCSI_CAP_PARITY: case SCSI_CAP_UNTAGGED_QING: case SCSI_CAP_LINKED_CMDS: case SCSI_CAP_RESET_NOTIFICATION: /* * None of these are settable via * the capability interface. */ break; case SCSI_CAP_DISCONNECT: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_DR) == 0) { break; } else { if (value) { isp->isp_cap[tgt] |= ISP_CAP_DISCONNECT; } else { isp->isp_cap[tgt] &= ~ISP_CAP_DISCONNECT; } } rval = 1; break; case SCSI_CAP_SYNCHRONOUS: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_SYNC) == 0) { break; } else { if (value) { isp->isp_cap[tgt] |= ISP_CAP_SYNC; } else { isp->isp_cap[tgt] &= ~ISP_CAP_SYNC; } } rval = 1; break; case SCSI_CAP_TAGGED_QING: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_DR) == 0 || (isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_TAG) == 0) { break; } else { if (value) { isp->isp_cap[tgt] |= ISP_CAP_TAG; } else { isp->isp_cap[tgt] &= ~ISP_CAP_TAG; } } rval = 1; break; case SCSI_CAP_WIDE_XFER: if ((isp->isp_target_scsi_options[tgt] & SCSI_OPTIONS_WIDE) == 0) { break; } else { if (value) { isp->isp_cap[tgt] |= ISP_CAP_WIDE; } else { isp->isp_cap[tgt] &= ~ISP_CAP_WIDE; } } rval = 1; break; case SCSI_CAP_INITIATOR_ID: if (value < N_ISP_TARGETS_WIDE) { struct isp_mbox_cmd mbox_cmd; isp->isp_initiator_id = (u_short) value; /* * set Initiator SCSI ID */ isp_i_mbox_cmd_init(isp, &mbox_cmd, 2, 2, ISP_MBOX_CMD_SET_SCSI_ID, isp->isp_initiator_id, 0, 0, 0, 0); if (isp_i_mbox_cmd_start(isp, &mbox_cmd) == 0) { rval = 1; } } break; case SCSI_CAP_ARQ: if (value) { isp->isp_cap[tgt] |= ISP_CAP_AUTOSENSE; } else { isp->isp_cap[tgt] &= ~ISP_CAP_AUTOSENSE; } rval = 1; break; default: rval = -1; break; } ISP_MUTEX_EXIT(isp); return (rval); }
The following sections discuss the abort and reset entry points for SCSI HBA.
The tran_abort(9E) entry point for a SCSI HBA driver is called to abort one or all the commands currently in transport for a particular target. This entry point is called when a target driver calls scsi_abort(9F).
The tran_abort() entry point should attempt to abort the command denoted by the pkt parameter. If the pkt parameter is NULL, tran_abort() should attempt to abort all outstanding commands in the transport layer for the particular target or logical unit.
Each command successfully aborted must be marked with pkt_reason CMD_ABORTED and pkt_statistics OR'd with STAT_ABORTED.
The tran_reset(9E) entry point for a SCSI HBA driver is called to reset either the SCSI bus or a particular SCSI target device. This entry point is called when a target driver calls scsi_reset(9F).
The tran_reset() entry point must reset the SCSI bus if level is RESET_ALL. If level is RESET_TARGET, just the particular target or logical unit must be reset.
Active commands affected by the reset must be marked with pkt_reason CMD_RESET, and with pkt_statistics OR'd with either STAT_BUS_RESET or STAT_DEV_RESET, depending on the type of reset.
Commands in the transport layer, but not yet active on the target, must be marked with pkt_reason CMD_RESET, and pkt_statistics OR'd with STAT_ABORTED.
tran_bus_reset(9E) must reset the SCSI bus without resetting targets.
#include <sys/scsi/scsi.h> int tran_bus_reset(dev_info_t *hba_dip, int level);
Where level must be the following:
Reset the SCSI bus only, not the targets
The tran_bus_reset() vector in the scsi_hba_tran(9S) structure should be initialized during the HBA driver's attach(9E) to point to an HBA entry point to be called when a user initiates a bus reset.
Implementation is hardware specific. If it cannot reset the SCSI bus without affecting the targets, the HBA driver should fail RESET_BUS or not initialize this vector.
The tran_reset_notify(9E) entry point for a SCSI HBA driver is called to request that the HBA driver notify the target driver by callback when a SCSI bus reset occurs.
isp_scsi_reset_notify( struct scsi_address *ap, int flag, void (*callback)(caddr_t), caddr_t arg) { struct isp *isp; struct isp_reset_notify_entry *p, *beforep; int rval = DDI_FAILURE; isp = (struct isp *)ap->a_hba_tran->tran_hba_private; mutex_enter(ISP_REQ_MUTEX(isp)); /* * Try to find an existing entry for this target */ p = isp->isp_reset_notify_listf; beforep = NULL; while (p) { if (p->ap == ap) break; beforep = p; p = p->next; } if ((flag & SCSI_RESET_CANCEL) && (p != NULL)) { if (beforep == NULL) { isp->isp_reset_notify_listf = p->next; } else { beforep->next = p->next; } kmem_free((caddr_t)p, sizeof (struct isp_reset_notify_entry)); rval = DDI_SUCCESS; } else if ((flag & SCSI_RESET_NOTIFY) && (p == NULL)) { p = kmem_zalloc(sizeof (struct isp_reset_notify_entry), KM_SLEEP); p->ap = ap; p->callback = callback; p->arg = arg; p->next = isp->isp_reset_notify_listf; isp->isp_reset_notify_listf = p; rval = DDI_SUCCESS; } mutex_exit(ISP_REQ_MUTEX(isp)); return (rval); }
To support the minimal set of hot-plugging operations, drivers might need to implement support for bus quiesce, bus unquiesce, and bus reset. The scsi_hba_tran(9S) structure supports these operations. If quiesce/unquiesce/reset is not required by hardware, no driver changes are needed.
The scsi_hba_tran structure includes the following fields:
int (*tran_quiesce)(dev_info_t *hba_dip); int (*tran_unquiesce)(dev_info_t *hba_dip); int (*tran_bus_reset)(dev_info_t *hba_dip, int level);
These interfaces quiesce and unquiesce a SCSI bus.
#include <sys/scsi/scsi.h> int prefixtran_quiesce(dev_info_t *hba_dip); int prefixtran_unquiesce(dev_info_t *hba_dip);
tran_quiesce(9E) and tran_unquiesce(9E) must be implemented by an HBA driver to support dynamic reconfiguration (DR) of SCSI devices on buses that were not designed to support hot-plugging.
The tran_quiesce(9E) and tran_unquiesce(9E) vectors in the scsi_hba_tran(9S) structure should be initialized during the HBA driver's attach(9E) to point to HBA entry points so they are called when a user initiates quiesce and unquiesce operations.
tran_quiesce(9E) is called by the SCSA framework to stop all activity on a SCSI bus prior to and during the reconfiguration of devices attached to the SCSI bus. tran_unquiesce(9E) is called by the SCSA framework to resume activity on the SCSI bus after the reconfiguration operation has been completed.
HBA drivers are required to handle tran_quiesce(9E) by waiting for all outstanding commands to complete before returning success. After the HBA has quiesced the bus, it must queue any new I/O requests from target drivers until the SCSA framework calls the corresponding tran_unquiesce(9E) entry point.
HBA drivers handle calls to tran_unquiesce(9E) by starting any target driver I/O requests that were queued by the HBA during the time the bus was quiesced.
The section covers issues specific to SCSI HBA drivers.
A SCSI HBA driver is installed like a leaf driver (see Chapter 17, Compiling, Loading, Packaging, and Testing Drivers), except that the add_drv(1M) command must specify the driver class as SCSI, such as:
# add_drv -m" * 0666 root root" -i'"pci1077,1020"' -c scsi isp
When attaching an instance of an HBA device, scsi_hba_attach_setup(9F) creates a number of SCSI configuration parameter properties for that HBA instance. A particular property is created only if there is no existing property of the same name already attached to the HBA instance, permitting a default property value to be overridden in an HBA configuration file.
An HBA driver must use ddi_prop_get_int(9F) to retrieve each property. The HBA driver then modifies (or accepts the default value of) the properties to configure its specific operation.
The scsi-reset-delay property is an integer specifying the SCSI bus or device reset delay recovery time in milliseconds.
The scsi-options property is an integer specifying a number of options through individually defined bits. The bits in scsi_options are:
SCSI_OPTIONS_DR (0x008) – If not set, the HBA should not grant disconnect privileges to a target device.
SCSI_OPTIONS_LINK (0x010) – If not set, the HBA should not enable linked commands.
SCSI_OPTIONS_SYNC (0x020) – If not set, the HBA should not negotiate synchronous data transfer, and should reject any attempt to negotiate synchronous data transfer initiated by a target.
SCSI_OPTIONS_PARITY (0x040) – If not set, the HBA should run the SCSI bus without parity.
SCSI_OPTIONS_TAG (0x080) – If not set, the HBA should not operate in Command Tagged Queuing mode.
SCSI_OPTIONS_FAST (0x100) – If not set, the HBA should not operate the bus in FAST SCSI mode.
SCSI_OPTIONS_WIDE (0x200) – If not set, the HBA should not operate the bus in WIDE SCSI mode.
An HBA driver might support a per-target scsi-options feature in the following format:
target<n>-scsi-options=<hex value>
In this example, < n> is the target ID. If the per-target scsi-options property is defined for a particular target, the HBA driver uses the value of the per-target scsi-options property for that target rather than the per-HBA driver instance scsi-options property. This can provide more precise control if, for example, synchronous data transfer needs to be disabled for just one particular target device. The per-target scsi-options property can be defined in the driver.conf(4) file.
Here is an example of a per-target scsi-options property definition to disable synchronous data transfer for target device 3:
target3-scsi-options=0x2d8
Some IA SCSI target drivers (such as the cmdk disk target driver) use the following configuration properties:
disk
queue
flow_control
When using the cmdk sample driver to write an HBA driver for an IA platform, one or more of these properties (as appropriate to the HBA driver and hardware) need to be defined in the driver.conf(4) file.
These property definitions should appear only in an HBA driver's driver.conf(4) file. The HBA driver itself should not inspect or attempt to interpret these properties in any way. These properties are advisory only and serve as an adjunct to the cmdk driver. They should not be relied upon in any way. The property definitions might or might not be used in future releases.
The disk property can be used to define the type of disk supported by cmdk. For a SCSI HBA, the only possible value for the disk property is:
disk="scdk" – Disk type is a SCSI disk
The queue property defines how the disk driver sorts the queue of incoming requests during strategy(9E). There are two possible values:
queue="qsort" – One-way elevator queuing model, provided by disksort(9F)
queue="qfifo" – FIFO (first in, first out) queuing model
The flow_control property defines how commands are transported to the HBA driver. There are three possible values:
flow_control="dsngl" – Single command per HBA driver
flow_control="dmult" – Multiple commands per HBA driver—when the HBA queue is full, the driver returns TRAN_BUSY
flow_control="duplx" – The HBA can support separate read and write queues, with multiple commands per queue. FIFO ordering is used for the write queue; the queuing model used for the read queue is described by the queue property. When an HBA queue is full, the driver returns TRAN_BUSY
Here is an example of a driver.conf(4) file for use with an IA HBA PCI device designed for use with the cmdk sample driver:
# # config file for ISP 1020 SCSI HBA driver # flow_control="dsngl" queue="qsort" disk="scdk" scsi-initiator-id=7;
For a definition of tagged queuing, refer to the SCSI-2 specification. To support tagged queuing, first check the scsi_options flag SCSI_OPTIONS_TAG to see if tagged queuing is enabled globally. Next, check to see if the target is a SCSI-2 device and whether it has tagged queuing enabled. If this is all true, attempt to enable tagged queuing by using scsi_ifsetcap(9F).
If tagged queuing fails, you can attempt to set untagged queuing. In this mode, you submit as many commands as you think necessary or optimal to the host adapter driver. Then the host adapter queues the commands to the target one at a time (as opposed to tagged queuing, where the host adapter submits as many commands as it can until the target indicates that the queue is full).