Configuring ixge Driver Parameters for SPARC Platforms
|
This chapter describes how to configure the driver parameters used by the Sun 10-Gigabit Ethernet PCI-X ixge adapter for SPARC platforms. This chapter contains the following sections:
10-Gigabit Ethernet Device Driver Parameters
The ixge device driver controls the Sun 10-Gigabit Ethernet PCI-X adapter. The ixge driver is attached to the UNIX pci name property pci8086,1048 for the Sun 10-Gigabit Ethernet PCI-X adapter (8086 is the vendor ID and 1048 is the PCI device ID).
You can manually configure the ixge device driver parameters to customize each Sun 10-Gigabit Ethernet adapter device in your system. This section provides an overview of the capabilities of the 10-Gigabit Ethernet driver used in the adapter, lists the available ixge device driver parameters, and describes how to configure these parameters.
10-Gigabit Ethernet Driver Parameter Values and Definitions
TABLE 3-1 describes the parameters and settings for the ixge device driver.
TABLE 3-1 ixge Driver Parameter, Status, and Descriptions
Parameter
|
Status
|
Description
|
instance
|
Read and write
|
Device instance
|
adv-asmpause-cap
|
Read and write
|
Flow control parameter
|
adv-pause-cap
|
Read and write
|
Flow control parameter
|
rx-intr-pkts
|
Read and write
|
Receive interrupt blanking values
|
rx-intr-time
|
Read and write
|
Receive interrupt blanking values
|
accept-jumbo
|
Read and write
|
Enable jumbo frames
|
Flow Control Parameters
The ixge device sources (transmits) and terminates (receives) pause frames conforming to the IEEE 802.3x Frame Based Link Level Flow Control Protocol. In response to received flow control frames, the ixge device can slow down its transmit rate. The ixge device is capable of sourcing flow control frames by requesting the link partner to slow down, provided that the link partner supports this feature. By default, the driver advertises both transmit and receive pause capability during autonegotiation.
TABLE 3-2 provides flow control keywords and describes their function.
TABLE 3-2 Read-Write Flow Control Keyword Descriptions
Keyword
|
Description
|
adv_asmpause_cap
|
The adapter supports asymmetric pause, which means it can pause only in one direction.
0=Off (default)
1=On
|
adv_pause_cap
|
This parameter has two meanings, each depending on the value of adv_asmpause_cap. (Default=0)
- If adv_asmpause_cap = 1 while adv_pause_cap = 1, pauses are received.
- If adv_asmpause_cap = 1 while adv_pause_cap = 0, pauses are transmitted.
- If adv_asmpause_cap = 0 while adv_pause_cap = 1, pauses are sent and received.
- If adv_asmpause_cap = 0, then adv_pause_cap determines whether Pause capability is on or off.
|
Interrupt Parameters
TABLE 3-3 describes the receive interrupt blanking values.
TABLE 3-3 RX Blanking Register for Alias Read
Field Name
|
Values
|
Description
|
rx_intr_pkts
|
0 to 511
|
Interrupt after this number of packets have arrived since the last packet was serviced. A value of zero indicates no packet blanking. (Default=3)
|
rx_intr_time
|
0 to 32768
|
Interrupt after 4.5 US ticks have elapsed since the last packet was serviced. A value of zero indicates no time blanking. (Default=0)
|
Jumbo Frames
The variable accept-jumbo enables the driver to instruct the upper layers that the driver is operating in Jumbo-frame mode.
TABLE 3-4 accept-jumbo Parameters
Parameter
|
Values
|
Description
|
accept-jumbo
|
1
|
Enables Jumbo-frame mode and sets MTU to greater than 1500.
|
0
|
Jumbo frames is not enabled. MTU is at 1500.
|
Refer to the documentation that came with your switch for exact commands to configure jumbo frames support.
To Configure Jumbo Frames Using ndd
|
1. Unplumb the interface you want to configure Jumbo Frames on.
2. Kill any snoop or netstat processes running over the interface to be configured.
3. Set the instance number:
% ndd -set /dev/ixge instance 0
|
In this example, the instance number set is 0.
4. Set the instance to accept jumbo frames:
% ndd -set /dev/ixge accept-jumbo 1
|
5. Plumb the interface up:
% ifconfig ixge0 plumb xxx.xxx.xx.xxx up
|
where xxx.xxx.xx.xxx = the IP address of the interface.
To Configure Jumbo Frames in a Linux Environment
|
1. Ensure that the ixge software is installed.
2. Plumb the Sun 10-Gigabit Ethernet PCI-X interface:
# modprobe ixge
# ifconfig eth2 xxx.xxx.xx.xxx up
# ifconfig eth2 mtu 8170
|
where xxx.xxx.xx.xxx = the IP address of the interface.
Setting ixge Driver Parameters
You can set the ixge device driver parameters in two ways:
- Using the ndd utility
- Using the ixge.conf file
If you use the ndd utility, the parameters are valid only until you reboot the system. This method is good for testing parameter settings.
To set parameters so they remain in effect after you reboot the system, create a
/platform/sun4u/kernel/drv/ixge.conf file and add parameter values to this file when you need to set a particular parameter for a device in the system.
Setting Parameters Using the ndd Utility
Use the ndd utility to configure parameters that are valid until you reboot the system. The ndd utility supports any networking driver that implements the Data Link Provider Interface (DLPI).
The following sections describe how you can use the ixge driver and the ndd utility to modify (with the -set option) or display (without the -set option) the parameters for each ixge device.
To Specify Device Instances for the ndd Utility
|
Before you use the ndd utility to get or set a parameter for a ixge device, you must specify the device instance for the utility.
1. Check the /etc/path_to_inst file to identify the instance associated with a particular device.
# grep ixge /etc/path_to_inst
"/pci@9,600000/ethernet@1" 0 "ixge"
|
In this example, the 10-Gigabit Ethernet instance is from the installed adapter. The instance number is in bold italics for clarity.
2. Use the instance number to select the device.
# ndd -set /dev/ixge instance number
|
The device remains selected until you change the selection.
Noninteractive and Interactive Modes
You can use the ndd utility in two modes:
- Noninteractive
- Interactive
In noninteractive mode, you invoke the utility to execute a specific command. Once the command is executed, you exit the utility. In interactive mode, you can use the utility to get or set more than one parameter value. Refer to the ndd(1M) man page for more information.
To Use the ndd Utility in Noninteractive Mode
|
This section describes how to modify and display parameter values.
To modify a parameter value, use the -set option.
If you invoke the ndd utility with the -set option, the utility passes value, which must be specified, down to the named /dev/ixge driver instance, and assigns the value to the parameter:
# ndd -set /dev/ixge parameter value
|
When you change any parameter, a message similar to the following appears:
ixge0: xcvr addr:0x01 - link up 10000 Mbps full duplex
|
To display the value of a parameter, specify the parameter name and omit the value.
When you omit the -set option, the utility queries the named driver instance, retrieves the value associated with the specified parameter, and prints it:
# ndd /dev/ixge parameter
|
To Use the ndd Utility in Interactive Mode
|
To modify a parameter value in interactive mode, specify ndd /dev/ixge:
# ndd /dev/ixge
name to get/set? (Enter the parameter name or ? to view all parameters)
|
After you enter the parameter name, the ndd utility prompts you for the parameter value (see TABLE 3-1 through TABLE 3-6).
To list all the parameters supported by the ixge driver, type ?.
See TABLE 3-1 through TABLE 3-6 for parameter descriptions.
# ndd -get /dev/ixge \?
? (read only)
instance (read and write)
adv_asmpause_cap (read and write)
adv_pause_cap (read and write)
rx_intr_pkts (read and write)
rx_intr_time (read and write)
accept_jumbo (read and write)
laggr_multistream (read and write)#
|
Note - By default, autonegotiation is set to off. This adapter operates only at full-duplex.
|
Setting Parameters Using the ixge.conf File
Specify the driver parameter properties for each device by creating a ixge.conf file in the /platform/sun4u/kernel/drv directory. Use a ixge.conf file when you need to set a particular parameter for a device in the system. The parameters you can set are the read and write parameters listed in 10-Gigabit Ethernet Driver Parameter Values and Definitions.
Note - Configuring the parameters by putting ndd commands in rcX.d scripts is not supported.
|
The man pages for prtconf(1M) and driver.conf(4) include additional details. The next procedure shows an example of setting parameters in a ixge.conf file.
To access any man page, type the man command plus the name of the man page. For example, to access man pages for prtconf(1M), type:
To Set Driver Parameters Using a ixge.conf File
|
1. Obtain the hardware path names for the ixge devices in the device tree.
a. Check the /etc/driver_aliases file to identify the name associated with a particular device:
# grep ixge /etc/driver_aliases
ixge "pci8086,1048.108e.7036"
|
Note - ixge "pci8086,1048.108e.7036" is used by the Sun 10-Gigabit Ethernet adapter.
|
b. Locate the path names and the associated instance numbers in the
/etc/path_to_inst file.
# grep ixge /etc/path_to_inst
"/pci@9,600000/ethernet@1" 0 "ixge"
|
In this example:
- The first part within the double quotes specifies the hardware node name in the device tree.
- The number not enclosed in quotes is the instance number (shown in bold italics).
- The last part in double quotes is the driver name.
- In the hardware node name, the last component after the last / character and before the @ character is the device name.
- The path name before the last component is the parent name.
- The number after the final @ character within quotes is the unit-address.
To identify a PCI device unambiguously in the ixge.conf file, use the name, parent name, and the unit-address for the device. Refer to the pci(4) man page for more information about the PCI device specification.
2. Set the parameters for the above devices in the
/platform/sun4u/kernel/drv/ixge.conf file.
In the following example, the accept-jumbo parameter is set to enable jumbo frames for all Sun 10-Gigabit Ethernet devices. See the driver.conf(4) man page for more information.
In the following example, the accept-jumbo parameter is set to enable jumbo frames is set for a single instance of the Sun 10-Gigabit Ethernet device.
In this example:
- parent = "/pci@8,600000"
- unit-address = "0"
name="pci8086,1048.108e.7036" parent="pci@8,600000" unit-address="0" accept-jumbo=1;
|
Note - If you omit the name=, parent=, and unit-address= definitions, the settings become global to all Sun 10-Gigabit Ethernet instances.
|
3. Save the ixge.conf file.
10-Gigabit Ethernet Driver Operating Statistics
These statistics are part of the statistics presented by the kstat command.
TABLE 3-5 describes the read-only Media Independent Interface (MII) capabilities. These parameters define the capabilities of the hardware. The Gigabit Media Independent Interface (GMII) supports all of the following capabilities.
TABLE 3-5 Read-Only ixge Device Capabilities
Parameter
|
Description (Local interface Capabilities)
|
cap_autoneg
|
0 = Not capable of autonegotiation
1 = Autonegotiation capable
|
cap_10gfdx
|
Local interface full-duplex capability
0 = Not 10000 Mbit/sec full-duplex capable
1 = 10000 Mbit/sec full-duplex capable
|
cap_10ghdx
|
Local interface half-duplex capability
0 = Not 10000 Mbit/sec half-duplex capable
1 = 10000 Mbit/sec half-duplex capable
|
cap_1000fdx
|
Local interface full-duplex capability
0 = Not 1000 Mbit/sec full-duplex capable
1 = 1000 Mbit/sec full-duplex capable
|
cap_1000hdx
|
Local interface half-duplex capability
0 = Not 1000 Mbit/sec half-duplex capable
1 = 1000 Mbit/sec half-duplex capable
|
cap_100fdx
|
Local interface full-duplex capability
0 = Not 100 Mbit/sec full-duplex capable
1 = 100 Mbit/sec full-duplex capable
|
cap_100hdx
|
Local interface half-duplex capability
0 = Not 100 Mbit/sec half-duplex capable
1 = 100 Mbit/sec half-duplex capable
|
cap_10fdx
|
Local interface full-duplex capability
0 = Not 10 Mbit/sec full-duplex capable
1 = 10 Mbit/sec full-duplex capable
|
cap_10hdx
|
Local interface half-duplex capability
0 = Not 10 Mbit/sec half-duplex capable
1 = 10 Mbit/sec half-duplex capable
|
cap_asm_pause
|
Local interface flow control capability
0 = Not asymmetric pause capable
1 = Asymmetric pause (from the local device) capable
|
cap_pause
|
Local interface flow control capability
0 = Not Symmetric pause capable
1 = Symmetric pause capable
|
Table describes the read-only ixge device capabilities.
TABLE 3-6 describes the kstat ixge:1 receive and transmit parameters:
TABLE 3-6 Receive and Transmit Parameters
Parameter
|
Description
|
lb_mode
|
Copy of the Loopback mode the device is in, if any.
|
rev_id
|
Revision ID of the 10-Gigabit Ethernet device useful for recognition of device being used in the field.
|
rx_allocb_fail
|
Number of times the driver failed to allocate OS receive buffer.
|
rx_hdr_drops
|
Number of times an entire page of packets less than 256 bytes each was dropped because the driver was unable to map a new page to replace it.
|
rx_hdr_pkts
|
Number of packets received that were less than 256 bytes.
|
rx_inits
|
Number of packets received
|
rx_jumbo_pkts
|
Number of jumbo packets received.
|
rx_len_mm
|
Receive length error count.
|
rx_msgdup_fail
|
Number of times driver failed to duplicate a packet on receipt.
|
rx_mtu_drops
|
Number of times an entire page of packets greater than 256 bytes and less than 1514 each was dropped because the driver was unable to map a new page to replace it.
|
rx_new_hdr_pgs
|
Number of pages that were filled with packets less than 256 bytes that got replaced during reception.
|
rx_new_mtu_pgs
|
Number of pages that were filled with packets greater than 256 bytes and less than 1514 that got replaced during reception.
|
rx_new_nxt_pgs
|
Number of pages that contained packets that were split across pages that got replaced during reception.
|
rx_new_pages
|
Number of pages that got replaced during reception.
|
rx_no_buf
|
Number of times the hardware cannot receive data because there is no more receive buffer space.
|
rx_nocanput
|
Number of packets dropped due to full STREAMS message queue.
|
rx_no_comp_wb
|
Number of times the hardware cannot post completion entries for received data.
|
rx_nxt_drops
|
Number of times a page with a split packet was dropped because the driver was unable to map a new one to replace it.
|
rx_ov_flow
|
Number of packets dropped due to full h/w FIFO.
|
rx_pkts_dropped
|
Number of packets dropped due to service FIFO.
|
rx_reused_pgs
|
Number of times driver able to reuse a system page.
|
rx_split_pkts
|
Number of packets that were split across two pages.
|
snaptime
|
System snapshot time in seconds.
|
trunk_mode
|
A value of 1 indicates that a link is aggregated.
|
tx_allocb_fail
|
Number of times driver failed to duplicate a packet on transmit.
|
tx_ddi_pkts
|
Number of packets that used DDI_DMA.
|
tx_dma_bind_fail
|
Number of times a page table entry was not available to enable the driver to map the kernel memory to device-accessible memory for transmission.
|
tx_dvma_pkts
|
Number of packets that used dvma.
|
tx_hdr_pkts
|
Number of packets that used bcopy (buffer copy).
|
tx_jumbo_pkts
|
Number of packets greater that Ethernet MTU of 1522.
|
tx_max_pend
|
Maximum number of transmits pending on any of the four queues.
|
tx_msgdup_fail
|
Number of times driver failed to duplicate a packet for promiscuous stream.
|
tx_no_desc
|
Number of tx packets dropped due to full descriptor ring.
|
tx_starts
|
Number of times that the driver attempted to transmit a packet.
|
xcvr_addr
|
GMII/MII physical layer device address for management interface.
|
xcvr_id
|
GMII/MII physical layer device Identification Decimal copy of MII registers 2 and 3.
|
xcvr_inits
|
Number of physical layer re-initializations incremented each time link parameters are changed using NDD.
|
xcvr_inuse
|
Type of Transceiver/SERDES which is always PCS (3).
|
To Check Link Partner Settings
|
Type the kstat command as superuser:
CODE EXAMPLE 3-1 Output from kstat Command
# kstat ixge:1
module: ixge instance: 1
name: ixge1 class: net
brdcstrcv 0
brdcstxmt 0
cap_1000fdx 0
cap_1000hdx 0
cap_100fdx 0
cap_100hdx 0
cap_100T4 0
cap_10fdx 0
cap_10gfdx 1
cap_10ghdx 0
cap_10hdx 0
cap_asmpause 0
cap_autoneg 0
cap_pause 0
crc_err 0
crtime 147.15892392
ierrors 2
ifspeed 10000000000
ipackets 0
ipackets64 0
lb_mode 0
link_asmpause 0
link_duplex 2
link_pause 0
link_speed 10000
link_T4 0
link_up 1
local_faults 1
mac_mtu 0
multircv 0
multixmt 0
norcvbuf 0
noxmtbuf 0
obytes 0
obytes64 0
oerrors 0
opackets 0
opackets64 0
pci_bad_ack_err 0
pci_bus_speed 100
pci_dmarz_err 0
pci_dmawz_err 0
pci_drto_err 0
pci_err 0
pci_parity_err 0
pci_rma_err 0
pci_rta_err 0
pcix_bus_mode 1
promisc off
qos_mode 0
rbytes 0
rbytes64 0
remote_faults 1
rev_id 28726
rx_allocb_fail 0
rx_hdr_drops 0
rx_hdr_pkts 0
rx_inits 0
rx_jumbo_pkts 0
rx_len_mm 0
rx_msgdup_fail 0
rx_mtu_drops 0
rx_mtu_pkts 0
rx_new_hdr_pgs 0
rx_new_mtu_pgs 0
rx_new_nxt_pgs 0
rx_new_pages 0
rx_no_buf 0
rx_nocanput 0
rx_no_comp_wb 0
rx_nxt_drops 0
rx_ov_flow 0
rx_pkts_dropped 0
rx_reused_pgs 0
rx_split_pkts 0
rx_tag_err 0
rx_taskq_waits 0
snaptime 7883.09014984
trunk_mode 0
tx_allocb_fail 0
tx_ddi_pkts 0
tx_dma_bind_fail 0
tx_dvma_pkts 0
tx_hdr_pkts 0
tx_inits 0
tx_jumbo_pkts 0
tx_max_pend 0
tx_msgdup_fail 0
tx_nocanput 0
tx_no_desc 0
tx_starts 0
tx_uflo 0
xcvr_addr 0
xcvr_id 0
xcvr_inits 1
xcvr_inuse 3
|
To Discover Device Capabilities
|
Use the kstat command to discover device capabilities.
# kstat ixge:1 | grep cap_
cap_1000fdx 0
cap_1000hdx 0
cap_100fdx 0
cap_100hdx 0
cap_100T4 0
cap_10fdx 0
cap_10gfdx 1
cap_10ghdx 0
cap_10hdx 0
cap_asmpause 0
cap_autoneg 0
cap_pause 0
|
To Discover Link Settings
|
Use the kstat command to discover link settings.
# kstat ixge:1 | grep link
link_asmpause 0
link_duplex 2
link_pause 0
link_speed 10000
link_T4 0
link_up 1
|
Configuration Checking
The jumbo frame configuration checking occurs at Layer 2 or Layer 3, depending on the configuration method.
Layer 2 Configuration Checking
You can view the MTU configuration of a ixge instance at any time using the kstat command. The kstat mac_mtu variable represents the complete size of the Ethernet frame, which includes the Ethernet header, maximum payload, and crc. This value is consistent with the definition of MTU presented by switch vendors.
Use the following command to get the information:.
# kstat ixge:0 | grep mac_mtu
mac_mtu 8192
|
Layer 3 Configuration Checking
You can check the Layer 3 configuration by invoking the ifconfig command on a preconfigured interface as shown in the previous examples:.
# ifconfig ixge138000
ixge138000: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu
8170 index 5
inet 192.12.38.69 netmask ffffff00 broadcast 192.12.38.255
ether 0:3:ba:c4:f7:1d
|
Sun 10-Gigabit Ethernet PCI-X Adapter Installation and User’s Guide
|
819-0938-13
|
|
Copyright © 2008 Sun Microsystems, Inc. All Rights Reserved.