C H A P T E R  4

Driver API

This chapter describes the driver application programming interface (API) which consists of the Netra DPS Crypto and Hashing API and Ethernet API. Topics include:


Netra DPS Crypto and Hashing API

Netra DPS Crypto and Hashing API is an interface that allows Netra DPS developers to access the cryption and hash hardware functions supported by UltraSPARC T2 based platforms.



Note - Netra DPS Cryptography API requires the SUNWndpsc Cryptography Driver package.


Developers do not necessarily need to know the details in implementing the crypto and hash APIs when accessing these APIs.

The Netra DPS reference application, IPSec Gateway, is an example of how to use this API. The package SUNWndpsc (required export clearance) contains this API.

The user needs to include the following header files under src/libs/ndps_crypto_api/ in the application:

The SPU driver is provided in the binary format located in SUNWndpsc: lib/n2cp/lwrten2cp.o

The user needs to link the driver into the application.

Netra DPS Crypto and Hash API Functions

API functions are detailed in Netra DPS Crypto and Hash API Function Descriptions. The API functions include the following:

AES-XCBC-MAC-96 support:

Netra DPS Crypto and Hash API Function Descriptions

Crypto and Hash Context Setup Part

NDPSCreateCryptoContext

Description

NDPSCreateCryptoContext creates a context for the crypto or hash task to be submitted to the UltraSPARC T2 crypto engine; the caller supplies the cipher, or hash, and the mode which is the algorithms supported in the UltraSPARC T2 crypto engine. This function allocates the necessary resource to fulfill the crypto or hash task, such as the SPU (Stream Processing Unit) CG devices.

Function

NDPS_crypto_ctx_t NDPSCreateCryptoContext (
const NDPS_CIPHER
cipher, int mode);

Parameters

cipher - An algorithm supported in UltraSPARC T2. See ndpscrypt.h, among them are AES/DES/3DES/RC4 and MD5/SHA1/SHA256

mode - The variation for each cipher, such as ECB/CBC/CTR for AES and ECB/CBC/CFB for DES/3DES

Return Values

Returns the opaque handle NDPS_crypto_ctx_t to the available hardware CG and SPU devices.

A CG device is used for symmetric key encryption and hashing. The user needs to have NDPS_crypto_ctx_t to be able to use other API functions.



Note - To implement, use the following approach: since each core owns one SPU, any strands on the same core accesses the same SPU, therefore, the users routine to access SPU depends on the strand the user is running. For UltraSPARC T2 platforms, the CG the user is accessing = strand# % 8. If the application has only one strand per core to access the SPU on that core, then no other action is required. If the user has two functions running different strands on the same core and both access the SPU.The user then needs to either place the mutex around the crypto function by accessing the SPU, or allocate one strand whose task is to access the SPU only. It then handles the callers in round-robin fashion to avoid locking.


NDPSDestroyCryptoContext

Description

Releases the context of accessing the SPU/CG device after finishing the crypto and hash task. The released hardware resources are available for the next caller.

Function

int NDPSDestroyCryptoContext (NDPS_crypto_ctx_t ctx);

Parameters

The opaque NDPS_crypto_ctx_t handle is allocated through NDPSCreateCryptoContext.

Return Values

0 - Success

1 - Failure

Crypto API

NDPSCryptKeyLength

Description

Loads the Key length for the crypto. The key length could be 128-bit, 192-bit, or 256-bit.

Function

int NDPSCryptKeyLength (NDPS_crypto_ctx_t ctx, int key_len);

Parameters

ctx - The NDPS_crypto_ctx_t handle

key_len - The key length

Return Values

0 - Success

1 - Failure

NDPSCryptKeyLoad

Description

Loads the Key for the crypto.

Function

int NDPSCryptKeyLoad (NDPS_crypto_ctx_t ctx, NDPS_key_t *key);

Parameters

ctx - The NDPS_crypto_ctx_t handle

key - The key



Note - To avoid key copy, the caller must maintain space for its key until it calls NDPSDestroyContext()


Return Values

0 - Success

1 - Failure

NDPSCryptIVLoad

Description

Loads the IV for the crypto.

Function

int NDPSCryptIVLoad (NDPS_crypto_ctx_t ctx, NDPS_iv_t *iv);

Parameters

ctx - The NDPS_crypto_ctx_t handle

iv - The iv



Note - To avoid IV copy, the caller must maintain space for its IV until it calls NDPSDestroyContext()


Return Values

0 - Success

1 - Failure

NDPSCrypt

Description

Submits the crypto task with a single data block to the UltraSPARC T2 crypto device.

Function

int NDPSCrypt (NDPS_crypto_ctx_t ctx, int encrypt_flag,
uchar_t *
outbuf, int *outlen, uchar_t *inbuf, int inlen);

Parameters

ctx - The NDPS_crypto_ctx_t handle

encrypt_flag = 1 - For encrypt

encrypt_flag = 0 - For decrypt

inbuf - The text to be encrypted or decrypted

inlen - Number of the text in bytes

outbuf - Where the crypted or decrypted data is placed

outlen - Number of the crypted or decrypted data in bytes

Return Values

0 - Success

1 - Failure

NDPSCryptMultiple

Description

Submits the crypto task with chained multiple data blocks to the UltraSPARC T2 crypto device.

Function

int NDPSCryptMultiple (NDPS_crypto_ctx_t ctx, int encrypt_flag,
int
num_blk, uchar_t **outbuf, size_t *outlen, uchar_t **inbuf,
size_t *
inlen);

Parameters

ctx - The NDPS_crypto_ctx_t handle

encrypt_flag = 1 - For encrypt

encrypt_flag = 0 - For decrypt

num_blk - Number of data blocks to be chained

inbuf - Array of the input chained data blocks

inlen - Array of the input lengths of the chained data blocks

outbuf - Array of the chained output data blocks

outlen - Array of the lengths of the chained output data blocks

Return Values

0 - Success

1 - Failure

NDPSCryptAndHashMultiple

Description

Submits the Crypto and Hashing tasks with multiple data blocks to the UltraSPARC T2 Crypto device.

Function

int NDPSCryptAndHashMultiple(NDPS_crypto_ctx_t ctx, int encrypt_flag,
int
num_blk, char **outbuf, size_t *outlen,
char **
inbuf, size_t *inlen, NDPS_crypto_ctx_t
h_ctx, char **h_outbuf, size_t *h_outlen,
char **
h_inbuf, size_t *h_inlen)

Parameters

ctx - The handler NDPS_crypto_ctx_t for Crypto

encrypt_flag = 1 - For encrypt and hash

encrypt_flag = 0 - For unhash and decrypt

num_blk - Number of data block CryptHash pairs to be submitted in one request

outbuf - Array of the output data blocks for Crypto

outlen - Array of the lengths of the output data blocks for Crypto

inbuf - Array of the input data blocks for Crypto

inlen - Array of the lengths of the input data blocks for Crypto

h_ctx - The handler NDPS_Crypto_ctx_t for Hash

h_outbuf - Array of the output data blocks for Hash

h_outlen - Array of the lengths of the output data blocks for Hash

h_inbuf - Array of the input data blocks for Hash

h_inlen - Array of the lengths of the input data blocks for Hash

Return Values

0 - Success

1 - Failure

Hash API

NDPSHashLength

Description

Sets the Hash length.

Function

int NDPSHashLength (NDPS_crypto_ctx_t ctx, int len);

Parameters

ctx - The NDPS_crypto_ctx_t handle

len - The hash length

Return Values

0 - Success

1 - Failure

NDPSHashIVLoad

Description

Loads the Hash IV (initialization vector) load.



Note - To avoid IV copy, the caller must maintain space for its IV until it calls NDPSDestroyContext()


Function

int NDPSHashIVLoad(NDPS_crypto_ctx_t ctx, NDPS_iv_t *iv);

Parameters

ctx - The NDPS_crypto_ctx_t handle

iv - The hash IV value

Return Values

0 - Success

1 - Failure

NDPSHashIVGet

Description

Acquires the IV (initialization vector) address for the hash.

Function

int NDPSHashIVGet (NDPS_crypto_ctx_t ctx, NDPS_iv_t **iv);

Parameters

ctx - The NDPS_crypto_ctx_t handle

iv - The pointer to the IV location

Return Values

0 - Success

1 - Failure

NDPSHashDirect

Description

Produces the Hash value from the input data with its length. This Hash function does not overwrite the internal IV, but rather does a complete hash operation and stores the result in the provided outbuf.

Function

int NDPSHashDirect (NDPS_crypto_ctx_t ctx, uchar_t *outbuf,
uchar_t *
inbuf, int inlen);

Parameters

ctx - The NDPS_crypto_ctx_t handle

inbuf - The input data to be hashed

inlen - The length of data to be hashed

outbuf - The resulting hash value

Return Values

0 - Success

1 - Failure

NDPSHashDirectMultiple

Description

Submits the Hash task with chained multiple data blocks to the UltraSPARC T2 crypto device.

Function

int NDPSHashDirectMultiple (NDPS_crypto_ctx_t ctx, int num_blk,
uchar_t **
outbuf, size_t *outlen, uchar_t **inbuf,
size_t *
inlen);

Parameters

ctx - The NDPS_crypto_ctx_t handle

num_blk - Number of data blocks to be chained

outbuf - Array of the chained output data blocks

outlen - Array of the lengths of the chained output data blocks

inbuf - Array of the input chained data blocks

inlen - Array of the input lengths of the chained data blocks

Return Values

0 - Success

1 - Failure

Crypto and Hash Combined API

NDPSCryptAndHash

Description

Combines crypto and hash operations in one function call. It calls SPU in one call to get a performance boost.

Function

int NDPSCryptAndHash(NDPS_crypto_ctx_t ctx, int encrypt_flag,
char *
outbuf, int *outlen, char *inbuf, int inlen,
NDPS_crypto_ctx_t
h_ctx,
char *
h_outbuf, int h_outlen, char *h_inbuf, int h_inlen);

Parameters

ctx - The NDPS_crypto_ctx_t for crypto; handle

encrypt_flag = 1 - For encrypt

encrypt_flag = 0 - For decrypt

outbuf - Array of the chained output data blocks for crypto

outlen - Array of the lengths of the chained output data blocks for crypto

inbuf - Array of the input chained data blocks for crypto

inlen - Array of the input lengths of the chained data blocks for crypto

h_ctx - The NDPS_crypto_ctx_t handle for Hash

h_outbuf - Array of the chained output data blocks for Hash

h_outlen - Array of the lengths of the chained output data blocks for Hash

h_inbuf - Array of the input chained data blocks for Hash

h_inlen - Array of the input lengths of the chained data blocks for Hash

Return Values

0 - Success

1 - Failure

Miscellaneous APIs

The following APIs support AES-XCBC-MAC-96.

NDPSAESXCBCMAC96init

Description

Initializes AES-XCBC-MAC-96.

Function

int NDPSAESXCBCMAC96init();

Parameters

None

Return Values

0 - Success

1 - Failure

NDPSAESXCBCMAC96fini

Description

Finalizes AES-XCBC-MAC-96.

Function

int NDPSAESXCBCMAC96fini();

Parameters

None

Return Values

0 - Success

1 - Failure

NDPSAESXCBCMAC96KeyLoad

Description

Loads the initial key for AES-XCBC-MAC-96. This is supplied by the caller.

Function

int NDPSAESXCBCMAC96KeyLoad (
NDPS_crypto_ctx_t
ctx, NDPS_key_t *key);

Parameters

ctx - The NDPS_crypto_ctx_t handle for crypto

key - The key

Return Values

0 - Success

1 - Failure

NDPSAESXCBCMAC96AuthGenerate

Description

Generates the AES-XCBC-MAC-96 authentic value in 96-bit.

Function

int NDPSAESXCBCMAC96AuthGenerate(NDPS_crypto_ctx_t ctx,
uchar_t *
inbuf, int inlen, uchar_t **auth_buf, int *auth_len);

Parameters

ctx - The NDPS_crypto_ctx_t handle for crypto

inbuf - The input data for AES-XCBC-MAC-96

inlen - The input lengths for AES-XCBC-MAC-96

auth_buf - The resulting AES-XCBC-MAC-96 hash value

auth_len - Lengths, in 96-bits

Return Values

0 - Success

1 - Failure


Ethernet API

The Ethernet API is an interface between the user network application and the device drivers. A Netra DPS application developer should be aware of the device features and capabilities but does not need to have the knowledge of the detailed implementation of the device driver. TABLE 4-1 shows the relationship among Ethernet device, device driver, Ethernet API, and the user application.


TABLE 4-1 Ethernet API and User Applications

Network Application

For example: RLP, IP packet forwarding, IPSec

Ethernet API

For example: eth_open, eth_close, eth_read

Device Driver

For example: nxge

Ethernet Device

For example: 10Gb Ethernet with NIU, Ophir


Network Applications

Network Applications are any applications that requires network hardware resources. The RLP, IP packet forwarding, and IPSec reference applications are all network applications.

Ethernet Device Driver

TABLE 4-2 lists the Ethernet device supported in Netra DPS platforms.


TABLE 4-2 Ethernet Devices Supported on Netra DPS Platforms

Device Driver

Ethernet Device

nxge

Sun multithreaded 10Gb Ethernet with NIU


See Note 11 for Ethernet device driver nxge tunables.

Ethernet API Functions

The API list of functions include the following:

Description of Ethernet API Functions

eth_pbuf_alloc

Description

This function is used to allocate a message block for managing incoming packet data. The allocated entity is returned as a pointer to the buffer block structure (pbuf_t). pbuf_t is a message block struct (mblk) consists of the all necessary pointers and fields for manipulating the data buffer. See mblk_t in mblk.h header file for the details of the message block. Packet data begins at b_wptr. The size of the mblk must be the size specified as mblk_size in the eth_open() call. This API is implemented in the user application space. (See Note 4). The device driver calls this function.

Function

pbuf_t *eth_pbuf_alloc(void *hook, size_t bufsz, uint16_t pool);

Parameters

hook - User provided hook. (See in Note 1)

bufsz - User provided buffer size to be allocated. (See Note 2.)

pool - DMA channel pool (See Note 3.)

Return Values

On success, returns pointer to mblk with b_rptr and b_wptr pointing to the start of a valid data buffer. An error returns NULL.

eth_pbuf_free

Description

This function is used to free a message block allocated by eth_pbuf_alloc().This function is implemented by the user and it is called by the device driver.

Function

void eth_pbuf_free(void *hook, pbuf_t * mblkp, void *arg,
uint16_t
pool);

Parameters

hook - User provided hook. (See in Note 1.)

mblkp - Pointer to message block to be freed

arg - Not used (pass in NULL)

pool - DMA channel pool. (See Note 3.)

eth_buf_alloc

Description

This function is used to allocate a data buffer for storing incoming packet data. The allocated entity is a pointer to the allocated buffer. This function is implemented in the user application space (see Note 4). The device driver calls this function.

Function

char *eth_buf_alloc(void *hook, size_t bufsz, uint16_t pool);

Parameters

hook - User provided hook. (See Note 1.)

bufsz - User provided buffer size to be allocated. (See Note 2.)

pool - DMA channel pool (See Note 3.)

Return Values

On success, returns the pointer to a valid data buffer. An error returns NULL.

eth_buf_free

Description

This function is used to free a buffer allocated by eth_buf_alloc(). This function is implemented in the user application space. (See Note 4.) The device driver calls this function.

Function

void eth_buf_free(void *hook, char *buf, void *arg, uint16_t pool);

Parameters

hook - User provided hook. (See Note 1.)

buf - Pointer to data buffer to be freed

arg - Not used (pass in NULL)

pool - DMA channel pool (See Note 3.)

eth_open

Description

This function is used to probe a network device in the target platform and if the device is found, it initializes the network device. On a successful completion, it returns an opaque handle which needs to be used in other API calls that is targeted to a specific device. When multiple ports are opened, eth_open() must be invoked in the increasing order of the port numbers, that is, port0, then port1, and so on, during initialization.

Function

ihandle_t eth_open(uint16_t vid, uint16_t did, eth_port_t port,
int num_chans, void *
txhook, void* rxhook,
size_t
mblk_siz, uint_t mpbase);

Parameters

vid - Vendor id of network device

did - Device id of network device

port - Port number of the ethernet interface. (See Note 5.)

txhook - Application provided hook to tx fastq table. (See Note 6.)

rxhook - Application provided hook to rx fastq table (See Note 7.)

mblk_siz - Size of buffer that is returned by eth_pbuf_alloc()

mpbase - Base index into the mempool type array used in application
(See Note 8.)

Return Values

On success - Returns a valid opaque device handle. (ihandle_t)

On error - Returns INVALID_IHANDLE

eth_close

Description

This function is used to release the ethernet interface instance and all resources held by it.

Function

int eth_close(ihandle_t ihandle);

Parameters

ihandle - Opaque handle returned by eth_open().

Return Values

0 - Success

1 - Failure

eth_read

Description

This function is used to receive messages from the ethernet interface instance specified by ihandle. It can be configured to return a chain of packets. The maximum number of packets in the chain is configurable through ETH_IOC_SET_MAX_PKT_CHAIN. This function is non-blocking.

Function

pbuf_t *eth_read(ihandle_t ihandle, eth_chan_t chan_num);

Parameters

ihandle - Opaque handle returned by eth_open().

chan_num - DMA channel number (See Note 9.)

Return Values

On success - Returns mblk packet chain containing message

On error - Returns NULL

eth_write

Description

This function is used to send a message which is specified by the message block structure pointer (mblk). This function is non-blocking and can fail if the hardware transmit descriptor ring is full.

Function

int eth_write(ihandle_t ihandle, eth_chan_t chan_num,
pbuf_t *
mblkp);

Parameters

ihandle - Opaque handle returned by eth_open().

chan_num - DMA channel number (See Note 9.)

mblkp - Message block pointer. This can be a chain; the maximum size of chain supported is implementation-specific and can be discovered via ETH_IOC_GET_MAX_TX_PKT_CHAIN.

Return Values

0 - Success

1 - Failure

eth_ioc

Description

This function is a catch all configuration API that can be used to control the device driver attributes.

Function

int eth_ioc(ihandle_t ihandle, ioc_cmd_t cmd, void *arg);

Parameters

ihandle - Opaque handle returned by eth_open().

cmd - Command to execute (See Note 10.)

*arg - Argument passed to command

Return Values

0 - Success

1 - Failure

Summary

TABLE 4-3 lists a summary of the ethernet API functions.


TABLE 4-3 Ethernet API Function Summary

API

Purpose

Implemented by

Called by

eth_pbuf_alloc

Allocate message block

User application

Device driver

eth_pbuf_free

Free message block

User application

Device driver

eth_buf_alloc

Allocate buffer

User application

Device driver

eth_buf_free

Free buffer

User application

Device driver

eth_open

Find and init device

Device driver

User application

eth_close

Free up device resource

Device driver

User application

eth_read

Poll for received packet

Device driver

User application

eth_write

Send a packet

Device driver

User application

eth_ioc

Device Control

Device driver

User application


Notes

Note 1

This is a “catch all” argument for the user application. It can be used for any purpose. If not used, pass in NULL.

Note 2

The size value should be large enough to hold an ethernet packet.

Note 3

When using multiple memory pools (one for each DMA channel), pool indicates the ID of the memory pool, which is normally indexed by the DMA channel number. When a single memory pool is used, always pass in a zero.

Note 4

The user application has the best knowledge and control of how system memory is utilized. The device driver calls this function in the Packet Read routine.

Note 5

The port number of the device can be determined using the -v option during boot. For example: boot net:,my_binary_file -v

Part of the console output is similar to the following:


NIU : SUNW,niumx
netdev[0]: VendorId 0x108e DevId 0xabce
netdev[0]: Subsystem VendorId 0x0 SubsystemId 0x0
netdev[0]: RevisionId 0x0
netdev[0]: PhyType gsd
netdev[0]: Compatible SUNW,niusl
netdev[0]: cfg_addr 0x0 pio_addr 0x8100000000
netdev[0]: mac_addr 0x0:14:4f:8c:4:3e
netdev[1]: VendorId 0x108e DevId 0xabce
netdev[1]: Subsystem VendorId 0x0 SubsystemId 0x0
netdev[1]: RevisionId 0x0
netdev[1]: PhyType gsd
netdev[1]: Compatible SUNW,niusl
netdev[1]: cfg_addr 0x0 pio_addr 0x8102000000
netdev[1]: mac_addr 0x0:14:4f:8c:4:3f 

This output indicates that there are two NIU ports. The number inside the netdev[] (0 and 1) are the port numbers used in eth_open calls.

Note 6

For an application that needs to forward packets (for example, RLP, IP packet forwarding, and IPSec applications), the application needs to pass in the pointer to the transmit fastq allocated by the application.

Note 7

For an application that needs to forward packets (for example, RLP, IP packet forwarding, and IPSec applications), the application needs to pass in the pointer to the receive fastq allocated by the application.

Note 8

When the application is using multiple memory pools, this is the index to the first memory pool used by the device. For example, if the device to be opened has eight memory pools (one for each DMA channel) and the memory pool ID are identified from 0 to 7, then the base index is 0.

Note 9

This is the DMA channel number of the DMA channel receiving the packet. In Sun multithreaded 10GbE with NIU, up to 16 DMA channels can be used. The number of DMA channels to be used is specified when calling eth_open().

Note 10

In Netra DPS 2.0, the following control commands are implemented:

Refer to the reference application (for example, RLP, IP packet forwarding, and IPSec) for their usage.

Note 11

TABLE 4-4 lists the Ethernet device driver nxge tunables.


TABLE 4-4 Ethernet Device Driver nxge Tunables

Driver Tunable

Description

extern uint_t nxge_max_dmas

max_dmas is used to increase the maximum DMAs passed to the eth_open() API: the default is 8, however, it can be setup to use more than 8 channels per port. For example, it can be set up for all 16 channels on one port.

extern uint_t nxge_tx_kicks

tx_kicks specifies the number of packets the driver tx is going to store before it actually kicks the hardware to send packets out.

extern uint_t nxge_rxoff_var

rxoff_var enables spreading of ingress packets to several cache line-aligned start addresses for each DMA: 64 | 128.

extern uint_t nxge_rxoff_var_n2_2

rxoff_var_n2_2 enables the UltraSPARC T2 2.2 behavior which allows two more start offsets to be used by ingress packets: 320 | 384

extern uint_t nxge_flow_cfg

flow_cfg enables different flow/classification policies as defined above: FLOW_USE_ALL, FLOW_TCAM_LOOKUP, and so on.

extern uint_t nxge_rx_pref

rx_pref enables prefetch instructions on the rx packet data buffer so that subsequent stages could have the data ready in cache.

extern uint_t nxge_kstat_on

kstat_on flag is 1 by default, if off will not update the ipackets/opackets counters in the driver, it can be turned off by setting it to the 0 setting.

extern uint_t nxge_prmsc_all

prmsc_all flag is 1 by default; if set to 1, it turns on promiscuous mode; packets that do not match port mac-id would be received on the default DMA.

extern uint_t nxge_prmsc_mgrp

prmsc_mgrp flag is 1 by default; it enables receive of multicast packets; if set to 0, it turns off multicast packets.

extern uint_t nxge_prmsc_virt

prmsc_virt flag is 0 by default; if set to 1, it turns on virtual promiscuous mode, which is a special promiscuous mode to spread packets to all available DMAs for that port. When this flag is enabled, it takes precedence over the other two promiscuous mode flags: “prmsc_all” and “prmsc_mgrp”


Note 12

This note describes how to enable the hardware checksum offload features on the Sun multithreaded 10Gb/NIU Ethernet hardware using the nxge driver:

The following mblk fields are used:


unsigned char b_ick_flag; /* H/W checksum enable flag : TX */

unsigned char *b_ick_start; /* Pointer to start offset : TX/RX */
unsigned char *b_ick_stuff; /* Pointer to stuff offset : TX */

If (b_ick_flag and NXGE_TX_CKENB), then the hardware is programmed to compute hardware checksum. It is expected that the ick_start/stuff point to the L4 payload start/stuff offsets, respectively. Also, the udp/tcp header checksum field needs to be filled with the pseudo header checksum value. The hardware will use this field for computing the full-checksum.

On rx, if (b_ick_flag and NXGE_RX_CKERR), then the hardware detected a checksum error in the ingress packet.