Configuring the Driver Parameters

C H A P T E R 5

The ixgbe device driver controls the Sun 10GbE SFP+ PCIe ExpressModule interfaces. You can manually set the ixgbe device driver parameters to customize each device in your system.

This chapter lists the available device driver parameters and describes how you can set these parameters.

Driver Overview

Driver Parameters for Solaris

Driver Parameters for Linux Systems

Setting ixgbe Driver Parameters in Linux Systems

Driver Overview

Each ixgbe channel provides 10000BASE-T networking interfaces.

The ixgbe driver is capable of supporting 10000 Mbit/sec, full-duplex.

Driver Parameters for Solaris

TABLE 5-1 describes the functions of the driver parameters.

TABLE 5-1 Driver Parameters
Type	Keyword	Description
`Jumbo Frame`	`default_mtu=u`	The size of the default MTU (payload without the Ethernet header) Allowed values: 1500 to 16366 (default =1500)
Flow Control	`flow_control`	Ethernet flow control Allowed values (default = 3): 0 - Disable 1 - Receive only 2 - Transmit only 3 - Receive and transmit
Transmit/ Receive Queues	`tx/rx/queue: tx_queue_number`	The number of the transmit queues Allowed values: 1 to 32 (default = 1)
Transmit/ Receive Queues	tx/rx/queue: tx_ring_size	The number of the transmit descriptors per transmit queue Allowed values: 64 to 4096 (default = 512)
Transmit/ Receive Queues	tx/rx/queue: rx_queue_number	The number of the receive queues Allowed values: 1 to 64 (default = 1)
Transmit/ Receive Queues	tx/rx/queue: rx_ring_size	The number of the receive descriptors per receive queue Allowed values: 64 to 4096 (default = 512)

Setting `ixgbe` Driver Parameters in Solaris

To set driver parameters using the `ixgbe.conf` file

1. Obtain the hardware path names for the ixgbe devices in the device tree.

Check the /etc/driver_aliases file to identify the name associated with a particular device. For example:

# grep ixgbe /etc/driver_aliases
ixgbe "pciex8086,10e1"

2. Locate the path names and the associated instance numbers in the /etc/path_to_inst file.

For example:

# grep ixgbe /etc/path_to_inst
"/pci@0,0/pci10de,376@a/pci108e,f35f@0" 0 "ixgbe"
"/pci@79,0/pci10de,376@a/pci108e,f25f@0" 1 "ixgbe"
"/pci@79,0/pci10de,376@a/pci108e,f25f@0,1" 2 "ixgbe"

In the examples shown above:

The first part within the double quotes specifies the hardware node name in the device tree.

The number not enclosed in quotes is the instance number (shown in bold for emphasis).

The last part in double quotes is the driver name. To identify a PCI-E device unambiguously in the ixgbe.conf file, use the name, parent name, and the unit-address for the device. Refer to the pci(4) man page for more information about the PCI-E device specification.

The name is "pciex8086,10e1", the parent is "/pci@0,0/pci10de,376@a", and the unit address is “0”.

3. Set the parameters for the ixgbe devices in the /kernel/drv/ixgbe.conf file.

For example, to set the flow_control parameter to 3 for ixgbe0:

name = "pciex8086,10e1"
parent = "/pci@0,0/pci10de,376@a"
unit-address = "0"
flow_control = 3;

4. Save the ixgbe.conf file.

5. Reboot the system.

Improving Performance in Solaris

Based on system configuration, some system and driver variables may need to be tuned to appropriate values for better performance in Solaris. For example:

1. rx_queue_number should be less than or equal to a minimum of #CPU - 1, and the MSI-X allocation limit (that is, dde_msix_alloc_limit).

2. ddi_msix_alloc_limit / pcplusmp:apic_multi_msi_max / pcplusmp:apic_msix_max should be equal to rx_queue_number + 1, as tx and other events, such as link status change, require an additional interrupt vector.

3. ip:ip_soft_rings_cnt should be tuned based on system type.

Changes similar to the following might improve performance on both x86 and SPARC platforms.

To improve performance in the case of large numbers of connections and packets

1. Add the following lines to the /etc/system file:

set ddi_msix_alloc_limit=9 set pcplusmp:apic_multi_msi_max=9 set pcplusmp:apic_msix_max=9 set pcplusmp:apic_intr_policy=1 set ip:ip_soft_rings_cnt=4 set ip_squeue_soft_ring=1

2. Set the rx_queue number to 8 in the /kernel/drv/ixgbe.conf file:

rx_queue_number = 8;

3. Reboot the server.

Driver Parameters for Linux Systems

TABLE 5-2 lists the tunable ixgbe driver parameters for Linux operating systems and describes their function.

TABLE 5-2 Tunable `ixgbe` Driver Parameters for Linux Operating Systems
Keyword	Valid Range	Default Value	Description
FlowControl	0 to 3 (0=none, 1=Rx only, 2=Tx only, 3=Rx and Tx)	Read from the EEPROM. If EEPROM is not detected, default is 3.	This parameter controls the automatic generation (Tx) and response (Rx) to Ethernet PAUSE frames.
RxDescriptiors	64 to 512	512	This value is the number of receive descriptors allocated by the driver. Increasing this value allows the driver to buffer more incoming packets. Each descriptor is 16 bytes. A receive buffer is also allocated for each descriptor and can be either 2048, 4056, 8192, or 16384 bytes, depending on the MTU setting. When the MTU size is 1500 or less, the receive buffer size is 2048 bytes. When the MTU is greater than 1500, the receive buffer size will be either 4056, 8192, or 16384 bytes. The maximum MTU size is 16114.
RxIntDelay	0 to 65535 (0=off)	72	This value delays the generation of receive interrupts in units of 0.8192 microseconds. Receive interrupt reduction can improve CPU efficiency if properly tuned for specific network traffic. Increasing this value adds extra latency to frame reception and can end up decreasing the throughput of TCP traffic. If the system is reporting dropped receives, this value might be set too high, causing the driver to run out of available receive descriptors.
TxDescriptors	80 to 4096	256	This value is the number of transmit descriptors allocated by the driver. Increasing this value allows the driver to queue more transmits. Each descriptor is 16 bytes.
XsumRX	0 to 1	1	A value of 1 indicates that the driver should enable IP checksum offload for received packets (both UDP and TCP) to the Ethernet adapter hardware.

Setting `ixgbe` Driver Parameters in Linux Systems

To Configure Jumbo Frames

Jumbo Frames can support up to 15000 MTU. The default value is 1500 MTU.

Use the ifconfig command to increase MTUs to allow transmission of Jumbo Frames.

For example, where the IP address for eth7 is 192.1.1.200, the following command increases MTUs to the maximum:

# ifconfig eth7 192.1.1.200 mtu 15000  up