Writing Device Drivers for Oracle® Solaris 11.2

Exit Print View

Updated: September 2014
 
 

Resource Allocation

The following sections discuss resource allocation.

tran_init_pkt() Entry Point

The tran_init_pkt(9E) entry point allocates and initializes a scsi_pkt(9S) structure and DMA resources for a target driver request.

The tran_init_pkt(9E) entry point is called when the target driver calls the SCSA function scsi_init_pkt(9F).

Each call of the tran_init_pkt(9E) entry point is a request to perform one or more of three possible services:

  • Allocation and initialization of a scsi_pkt(9S) structure

  • Allocation of DMA resources for data transfer

  • Reallocation of DMA resources for the next portion of the data transfer

Allocation and Initialization of a scsi_pkt(9S) Structure

The tran_init_pkt(9E) entry point must allocate a scsi_pkt(9S) structure through scsi_hba_pkt_alloc(9F) if pkt is NULL.

scsi_hba_pkt_alloc(9F) allocates space for the following items:

  • scsi_pkt(9S)

  • SCSI CDB of length cmdlen

  • Completion area for SCSI status of length statuslen

  • Per-packet target driver private data area of length tgtlen

  • Per-packet HBA driver private data area of length hbalen

The scsi_pkt(9S) structure members, including pkt, must be initialized to zero except for the following members:

  • pkt_scbp – Status completion

  • pkt_cdbp – CDB

  • pkt_ha_private – HBA driver private data

  • pkt_private – Target driver private data

These members are pointers to memory space where the values of the fields are stored, as shown in the following figure. For more information, refer to scsi_pkt Structure (HBA).

Figure 18-5  scsi_pkt(9S) Structure Pointers

image:Diagram shows the scsi_pkt structure with those members that point to values rather than being initialized to zero.

The following example shows allocation and initialization of a scsi_pkt structure.

Example 18-2  HBA Driver Initialization of a SCSI Packet Structure
static struct scsi_pkt                 *
isp_scsi_init_pkt(
    struct scsi_address    *ap,
    struct scsi_pkt        *pkt,
    struct buf             *bp,
    int                    cmdlen,
    int                    statuslen,
    int                    tgtlen,
    int                    flags,
    int                    (*callback)(),
    caddr_t                arg)
{
    struct isp_cmd         *sp;
    struct isp             *isp;
    struct scsi_pkt        *new_pkt;

    ASSERT(callback == NULL_FUNC || callback == SLEEP_FUNC);

    isp = (struct isp *)ap->a_hba_tran->tran_hba_private;
    /*
     * First step of isp_scsi_init_pkt:  pkt allocation
     */
    if (pkt == NULL) {
        pkt = scsi_hba_pkt_alloc(isp->isp_dip, ap, cmdlen,
            statuslen, tgtlen, sizeof (struct isp_cmd),
            callback, arg);
        if (pkt == NULL) {
            return (NULL);
        }

        sp = (struct isp_cmd *)pkt->pkt_ha_private;
        /*
         * Initialize the new pkt
         */
        sp->cmd_pkt         = pkt;
        sp->cmd_flags       = 0;
        sp->cmd_scblen      = statuslen;
        sp->cmd_cdblen      = cmdlen;
        sp->cmd_dmahandle   = NULL;
        sp->cmd_ncookies    = 0;
        sp->cmd_cookie      = 0; 
        sp->cmd_cookiecnt   = 0;
        sp->cmd_nwin        = 0;
        pkt->pkt_address    = *ap;
        pkt->pkt_comp       = (void (*)())NULL;
        pkt->pkt_flags      = 0;
        pkt->pkt_time       = 0;
        pkt->pkt_resid      = 0;
        pkt->pkt_statistics = 0;
        pkt->pkt_reason     = 0;
        new_pkt = pkt;
    } else {
        sp = (struct isp_cmd *)pkt->pkt_ha_private;
        new_pkt = NULL;
    }
    /*
     * Second step of isp_scsi_init_pkt:  dma allocation/move
     */
    if (bp && bp->b_bcount != 0) {
        if (sp->cmd_dmahandle == NULL) {
            if (isp_i_dma_alloc(isp, pkt, bp, flags, callback) == 0) {
                if (new_pkt) {
                    scsi_hba_pkt_free(ap, new_pkt);
                }
                return ((struct scsi_pkt *)NULL);
            }
        } else {
            ASSERT(new_pkt == NULL);
            if (isp_i_dma_move(isp, pkt, bp) == 0) {
                return ((struct scsi_pkt *)NULL);
            }
        }
    }
    return (pkt);
}

Allocation of DMA Resources

The tran_init_pkt(9E) entry point must allocate DMA resources for a data transfer if the following conditions are true:

  • bp is not null.

  • bp->b_bcount is not zero.

  • DMA resources have not yet been allocated for this scsi_pkt(9S).

The HBA driver needs to track how DMA resources are allocated for a particular command. This allocation can take place with a flag bit or a DMA handle in the per-packet HBA driver private data.

The PKT_DMA_PARTIAL flag in the pkt enables the target driver to break up a data transfer into multiple SCSI commands to accommodate the complete request. This approach is useful when the HBA hardware scatter-gather capabilities or system DMA resources cannot complete a request in a single SCSI command.

The PKT_DMA_PARTIAL flag enables the HBA driver to set the DDI_DMA_PARTIAL flag. The DDI_DMA_PARTIAL flag is useful when the DMA resources for this SCSI command are allocated. For example the ddi_dma_buf_bind_handle(9F)) command can be used to allocate DMA resources. The DMA attributes used when allocating the DMA resources should accurately describe any constraints placed on the ability of the HBA hardware to perform DMA. If the system can only allocate DMA resources for part of the request, ddi_dma_buf_bind_handle(9F) returns DDI_DMA_PARTIAL_MAP.

The tran_init_pkt(9E) entry point must return the amount of DMA resources not allocated for this transfer in the field pkt_resid.

A target driver can make one request to tran_init_pkt(9E) to simultaneously allocate both a scsi_pkt(9S) structure and DMA resources for that pkt. In this case, if the HBA driver is unable to allocate DMA resources, that driver must free the allocated scsi_pkt(9S) before returning. The scsi_pkt(9S) must be freed by calling scsi_hba_pkt_free(9F).

The target driver might first allocate the scsi_pkt(9S) and allocate DMA resources for this pkt at a later time. In this case, if the HBA driver is unable to allocate DMA resources, the driver must not free pkt. The target driver in this case is responsible for freeing the pkt.

Example 18-3  HBA Driver Allocation of DMA Resources
static int
isp_i_dma_alloc(
    struct isp         *isp,
    struct scsi_pkt    *pkt,
    struct buf         *bp,
    int                flags,
    int                (*callback)())
{
    struct isp_cmd     *sp  = (struct isp_cmd *)pkt->pkt_ha_private;
    int                dma_flags;
    ddi_dma_attr_t     tmp_dma_attr;
    int                (*cb)(caddr_t);
    int                i;

    ASSERT(callback == NULL_FUNC || callback == SLEEP_FUNC);

    if (bp->b_flags & B_READ) {
        sp->cmd_flags &= ~CFLAG_DMASEND;
        dma_flags = DDI_DMA_READ;
    } else {
        sp->cmd_flags |= CFLAG_DMASEND;
        dma_flags = DDI_DMA_WRITE;
    }
    if (flags & PKT_CONSISTENT) {
        sp->cmd_flags |= CFLAG_CMDIOPB;
        dma_flags |= DDI_DMA_CONSISTENT;
    }
    if (flags & PKT_DMA_PARTIAL) {
        dma_flags |= DDI_DMA_PARTIAL;
    }

    tmp_dma_attr = isp_dma_attr;
    tmp_dma_attr.dma_attr_burstsizes = isp->isp_burst_size;

    cb = (callback == NULL_FUNC) ? DDI_DMA_DONTWAIT : DDI_DMA_SLEEP;

    if ((i = ddi_dma_alloc_handle(isp->isp_dip, &tmp_dma_attr,
      cb, 0, &sp->cmd_dmahandle)) != DDI_SUCCESS) {
        switch (i) {
          case DDI_DMA_BADATTR:
              bioerror(bp, EFAULT);
              return (0);
          case DDI_DMA_NORESOURCES:
              bioerror(bp, 0);
              return (0);
        }
    }

    i = ddi_dma_buf_bind_handle(sp->cmd_dmahandle, bp, dma_flags,
    cb, 0, &sp->cmd_dmacookies[0], &sp->cmd_ncookies);

    switch (i) {
      case DDI_DMA_PARTIAL_MAP:
          if (ddi_dma_numwin(sp->cmd_dmahandle, &sp->cmd_nwin) == DDI_FAILURE) {
              cmn_err(CE_PANIC, "ddi_dma_numwin() failed\n");
          }

          if (ddi_dma_getwin(sp->cmd_dmahandle, sp->cmd_curwin,
            &sp->cmd_dma_offset, &sp->cmd_dma_len, &sp->cmd_dmacookies[0], 
            &sp->cmd_ncookies) == DDI_FAILURE) {
              cmn_err(CE_PANIC, "ddi_dma_getwin() failed\n");
          }
          goto get_dma_cookies;

      case DDI_DMA_MAPPED:
          sp->cmd_nwin = 1;
          sp->cmd_dma_len = 0;
          sp->cmd_dma_offset = 0;

      get_dma_cookies:
          i = 0;
          sp->cmd_dmacount = 0;
          for (;;) {
              sp->cmd_dmacount += sp->cmd_dmacookies[i++].dmac_size;
              if (i == ISP_NDATASEGS || i == sp->cmd_ncookies)
                  break;
              ddi_dma_nextcookie(sp->cmd_dmahandle,
              &sp->cmd_dmacookies[i]);
          }
          sp->cmd_cookie = i;
          sp->cmd_cookiecnt = i;
          sp->cmd_flags |= CFLAG_DMAVALID;
          pkt->pkt_resid = bp->b_bcount - sp->cmd_dmacount;
          return (1);

      case DDI_DMA_NORESOURCES:
          bioerror(bp, 0);
          break;

      case DDI_DMA_NOMAPPING:
          bioerror(bp, EFAULT);
          break;

      case DDI_DMA_TOOBIG:
          bioerror(bp, EINVAL);
          break;

      case DDI_DMA_INUSE:
          cmn_err(CE_PANIC, "ddi_dma_buf_bind_handle:"
            " DDI_DMA_INUSE impossible\n");

      default:
          cmn_err(CE_PANIC, "ddi_dma_buf_bind_handle:"
            " 0x%x impossible\n", i);
    }
    ddi_dma_free_handle(&sp->cmd_dmahandle);
    sp->cmd_dmahandle = NULL;
    sp->cmd_flags &= ~CFLAG_DMAVALID;
    return (0);
}

Reallocation of DMA Resources for Data Transfer

For a previously allocated packet with data remaining to be transferred, the tran_init_pkt(9E) entry point must reallocate DMA resources when the following conditions apply:

  • Partial DMA resources have already been allocated.

  • A non-zero pkt_resid was returned in the previous call to tran_init_pkt(9E).

  • bp is not null.

  • bp->b_bcount is not zero.

When reallocating DMA resources to the next portion of the transfer, tran_init_pkt(9E) must return the amount of DMA resources not allocated for this transfer in the field pkt_resid.

If an error occurs while attempting to move DMA resources, tran_init_pkt(9E) must not free the scsi_pkt(9S). The target driver in this case is responsible for freeing the packet.

If the callback parameter is NULL_FUNC, the tran_init_pkt(9E) entry point must not sleep or call any function that might sleep. If the callback parameter is SLEEP_FUNC and resources are not immediately available, the tran_init_pkt(9E) entry point should sleep. Unless the request is impossible to satisfy, tran_init_pkt() should sleep until resources become available.

Example 18-4  DMA Resource Reallocation for HBA Drivers
static int
isp_i_dma_move(
    struct isp         *isp,
    struct scsi_pkt    *pkt,
    struct buf         *bp)
{
    struct isp_cmd     *sp  = (struct isp_cmd *)pkt->pkt_ha_private;
    int                i;

    ASSERT(sp->cmd_flags & CFLAG_COMPLETED);
    sp->cmd_flags &= ~CFLAG_COMPLETED;
    /*
     * If there are no more cookies remaining in this window,
     * must move to the next window first.
     */
    if (sp->cmd_cookie == sp->cmd_ncookies) {
        /*
         * For small pkts, leave things where they are
         */
        if (sp->cmd_curwin == sp->cmd_nwin && sp->cmd_nwin == 1)
            return (1);
        /*
         * At last window, cannot move
         */
        if (++sp->cmd_curwin >= sp->cmd_nwin)
            return (0);
        if (ddi_dma_getwin(sp->cmd_dmahandle, sp->cmd_curwin,
          &sp->cmd_dma_offset, &sp->cmd_dma_len,
          &sp->cmd_dmacookies[0], &sp->cmd_ncookies) == DDI_FAILURE)
            return (0);
        sp->cmd_cookie = 0;
    } else {
        /*
         * Still more cookies in this window - get the next one
         */
        ddi_dma_nextcookie(sp->cmd_dmahandle, &sp->cmd_dmacookies[0]);
    }
    /*
     * Get remaining cookies in this window, up to our maximum
     */
    i = 0;
    for (;;) {
        sp->cmd_dmacount += sp->cmd_dmacookies[i++].dmac_size;
        sp->cmd_cookie++;
        if (i == ISP_NDATASEGS || sp->cmd_cookie == sp->cmd_ncookies)
            break;
        ddi_dma_nextcookie(sp->cmd_dmahandle, &sp->cmd_dmacookies[i]);
    }
    sp->cmd_cookiecnt = i;
    pkt->pkt_resid = bp->b_bcount - sp->cmd_dmacount;
    return (1);
}

tran_destroy_pkt() Entry Point

The tran_destroy_pkt(9E) entry point is the HBA driver function that deallocates scsi_pkt(9S) structures. The tran_destroy_pkt() entry point is called when the target driver calls scsi_destroy_pkt(9F).

The tran_destroy_pkt() entry point must free any DMA resources that have been allocated for the packet. An implicit DMA synchronization occurs if the DMA resources are freed and any cached data remains after the completion of the transfer. The tran_destroy_pkt() entry point frees the SCSI packet by calling scsi_hba_pkt_free(9F).

Example 18-5  HBA Driver tran_destroy_pkt(9E) Entry Point
static void
isp_scsi_destroy_pkt(
    struct scsi_address    *ap,
    struct scsi_pkt    *pkt)
{
    struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private;
    /*
     * Free the DMA, if any
     */
    if (sp->cmd_flags & CFLAG_DMAVALID) {
        sp->cmd_flags &= ~CFLAG_DMAVALID;
        (void) ddi_dma_unbind_handle(sp->cmd_dmahandle);
        ddi_dma_free_handle(&sp->cmd_dmahandle);
        sp->cmd_dmahandle = NULL;
    }
    /*
     * Free the pkt
     */
    scsi_hba_pkt_free(ap, pkt);
}

tran_sync_pkt() Entry Point

The tran_sync_pkt(9E) entry point synchronizes the DMA object allocated for the scsi_pkt(9S) structure before or after a DMA transfer. The tran_sync_pkt() entry point is called when the target driver calls scsi_sync_pkt(9F).

If the data transfer direction is a DMA read from device to memory, tran_sync_pkt() must synchronize the CPU's view of the data. If the data transfer direction is a DMA write from memory to device, tran_sync_pkt() must synchronize the device's view of the data.

Example 18-6  HBA Driver tran_sync_pkt(9E) Entry Point
static void
isp_scsi_sync_pkt(
    struct scsi_address    *ap,
    struct scsi_pkt        *pkt)
{
    struct isp_cmd *sp = (struct isp_cmd *)pkt->pkt_ha_private;

    if (sp->cmd_flags & CFLAG_DMAVALID) {
        (void)ddi_dma_sync(sp->cmd_dmahandle, sp->cmd_dma_offset,
        sp->cmd_dma_len,
        (sp->cmd_flags & CFLAG_DMASEND) ?
        DDI_DMA_SYNC_FORDEV : DDI_DMA_SYNC_FORCPU);
    }
}

tran_dmafree() Entry Point

The tran_dmafree(9E) entry point deallocates DMA resources that have been allocated for a scsi_pkt(9S) structure. The tran_dmafree() entry point is called when the target driver calls scsi_dmafree(9F).

tran_dmafree() must free only DMA resources allocated for a scsi_pkt(9S) structure, not the scsi_pkt(9S) itself. When DMA resources are freed, a DMA synchronization is implicitly performed.


Note - The scsi_pkt(9S) is freed in a separate request to tran_destroy_pkt(9E). Because tran_destroy_pkt() must also free DMA resources, the HBA driver must keep accurate note of whether scsi_pkt() structures have DMA resources allocated.
Example 18-7  HBA Driver tran_dmafree(9E) Entry Point
static void
isp_scsi_dmafree(
    struct scsi_address    *ap,
    struct scsi_pkt        *pkt)
{
    struct isp_cmd    *sp = (struct isp_cmd *)pkt->pkt_ha_private;

    if (sp->cmd_flags & CFLAG_DMAVALID) {
        sp->cmd_flags &= ~CFLAG_DMAVALID;
        (void)ddi_dma_unbind_handle(sp->cmd_dmahandle);
        ddi_dma_free_handle(&sp->cmd_dmahandle);
        sp->cmd_dmahandle = NULL;
    }
}