This chapter discusses how to customize properties of an Ethernet network driver to fulfill specific performance requirements.
In this release, configuration of properties of the NIC driver is performed by using the dladm command. This command allows you to configure the properties dynamically without causing any network disruption on other NICs of similar types. The values that you set are stored into a dladm repository and therefore persist even after you reboot the system or unplumb the interface.
A Driver Configuration Framework (GLDv3) is implemented in this release. If used when you configure drivers, this framework provides the following benefits:
Only single command interface, dladm, is needed to configure network driver properties.
A uniform syntax is used regardless of the properties: dladm subcommand properties data-link.
Use of the dladm command applies to both public and private properties of the driver.
Using the dladm command on a specific driver does not disrupt other network connections.
NIC driver properties that are configurable by using the dladm command fall into one of two categories:
Public properties that can be applied to any driver of the given media type such as link speed, autonegotiation for Ethernet, or the MTU size that can be applied to all data-link drivers.
Private properties that are particular to a certain subset of drivers for a given media type. These properties can be specific to that subset because they are closely related either to the hardware that is associated with the driver or to the details of the driver implementation itself, such as debugging-related tunables.
Properties of a NIC driver are typically set with default values. However, certain networking scenarios might require you to change specific property settings of a NIC. These property settings can be either public or private properties. For example, a NIC might be communicating with an old switch that does not properly perform autonegotiation. Or, a switch might have been configured to support Jumbo frames. Or, driver specific properties that regulate packet transmission or packet receiving might need to be modified for the given driver. In this Solaris release, all of these settings can now be reset by a single administrative tool, dladm.
For NIC drivers that have been converted to the GLDv3 framework, properties are configured by using the dladm command. This command enables you to configure the properties dynamically without causing any network disruption on other NICs of similar types. The values that you set are stored in a dladm repository and persist even after you reboot the system or unplumb the interface. Therefore, use dladm as the preferred command to configure NICs, instead of the ndd command.
To administer NIC drivers, you use the following dladm subcommands:
dladm show-linkprop displays the properties that are associated with the data link.
dladm set-linkprop sets values for specified data-link properties.
dladm reset-linkprop restores property settings to the default values.
dladm show-ether displays Ethernet parameter settings of a data link.
For more information about these commands, see the dladm(1M) man page.
Customizing NIC properties by using the dladm command is supported only in network drivers that have been converted to the GLDv3 framework, such as bge, nge, e1000g, and nxge.
Work continues to make other drivers become supported in the GLDv3 framework. To confirm whether your specific driver supports this feature, refer to the driver's man page.
The following section provides procedures to set certain NIC driver properties. The selected properties are public and common to all NIC drivers. A separate section describes driver specific properties as well as procedures to configure selected private properties of the e1000g driver.
The following section provides procedures with examples that show how to configure public and private properties of NIC drivers by using the dladm command.
Task |
Description |
For Instructions |
---|---|---|
Modify the MTU size. |
Increases the MTU size of packet transmission to handle Jumbo frames. | |
Modify the link speed. |
Switches off higher link speed and advertises only the lower link speed to allow communications with an older system. | |
Display information about NIC properties. |
Lists NIC properties and their current configuration; lists Ethernet parameter settings. | |
Configure driver to use DMA binding. |
Sets threshold that causes the driver to switch from DMA binding or bcopy function during transmission. |
How to Set the e1000g Driver to Use Direct Memory Access Binding |
Set interrupt rates |
Manually defines rates at which interrupts are delivered by the driver instead of the rate being defined automatically. |
Enabling support for Jumbo frames in a network setup is a common task for most network scenarios. Support for Jumbo frames requires increasing the size of a data link's maximum transmission unit (MTU). The following procedure includes the use of customized names to identify data links. For an overview of customized names and their use in network configuration, see Overview of the Networking Stack.
On the system that has the link whose MTU you want to modify, assume the System Administrator role.
The System Administrator role includes the Network Management profile. To create the role and assign the role to a user, see Chapter 9, Using Role-Based Access Control (Tasks), in System Administration Guide: Security Services.
To identify the specific Ethernet device whose MTU size you need to reset, display the links in the system.
# dladm show-phys |
Perform this step especially if your network configuration uses customized names for data links. With customized names, data links are no longer necessarily identified by their hardware-based names. For example, the Ethernet device is bge0. However, the data link over the device is renamed net0. Therefore, you would need to configure the MTU size of net0. Refer to Data Link and IP Interface Configuration (Tasks) for examples of configuration tasks on data links that use customized names.
(Optional) Display the data link's current MTU size and other properties.
To display a specific property of a data link, use the following syntax:
dladm show-linkprop -p property data-link |
This command displays the settings of the property that you specify.
To display several selected properties of the data link, use the following syntax:
# dladm show-link data-link |
This command displays data-link information, including MTU size.
See Link Administration and Monitoring for additional examples of the use of the dladm show-link syntax to display data-link information.
Unplumb the interface that is configured over the data link.
# ifconfig interface unplumb |
Change the value of the link's MTU size to 9000, the value for Jumbo frames.
# dladm set-linkprop -p mtu=9000 data-link |
Plumb the IP interface over the link.
# ifconfig interface plumb IP-address up |
For additional options that you can use with the ifconfig command, see the ifconfig(1M) man page.
(Optional) Verify that the interface uses the new MTU size by using one of the command syntaxes in Step 3.
# dladm show-linkprop -p mtu data-link |
(Optional) Display the link's current Ethernet settings.
# dladm show-ether data-link |
The following example that enables support for Jumbo frames builds on the following scenario:
The system has two bge NICS: bge0 and bge1.
The device bge0 is used as a primary interface, while the device bge1 is used for test purposes.
You want to enable support for Jumbo frames on bge1, while you retain the default MTU size of the primary interface.
The network configuration uses customized names for data links. The link name of bge0 is net0. The link name of bge1 is web1.
# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 ether up 100Mb full bge0 itops1 ether up 100Mb full qfe3 web1 ether up 100Mb full bge1 # dladm show-linkprop -p mtu web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 mtu 1500 1500 -- # ifconfig web1 unplumb # dladm set-linkprop -p mtu=9000 web1 # ifconfig web1 plumb 10.10.1.2/24 up # dladm show-link web1 LINK CLASS MTU STATE OVER web1 phys 9000 up -- |
Notice that the MTU value is now 9000. In this example, the dladm command enabled you to change web1's MTU size directly. The previous method would have required you to unplumb net0 as well, which would have unnecessarily disrupted the primary interface's operations.
Most network setups consist of a combination of systems with varying speed capabilities. For example, the advertised speed between an older system and a newer system might need to be changed to a lower setting to allow communication. By default, all the speed and duplex capabilities of a NIC card are advertised. This procedure shows how to turn off the gigabit capabilities and advertise only the megabit capabilities.
On the system that has the NIC whose properties you want to modify, assume the System Administrator role.
The System Administrator role includes the Network Management profile. To create the role and assign the role to a user, see Chapter 9, Using Role-Based Access Control (Tasks), in System Administration Guide: Security Services.
(Optional) Display the current status of the property you want to modify.
# dladm show-linkprop -p property data-link |
To advertise lower speed capabilities, turn off the higher speed capabilities to prevent them from being advertised.
# dladm set-linkprop -p property=value1 data-link |
This example shows how you can prevent the link web1 from advertising gigabit capabilities.
# dladm show-linkprop -p adv_1000fdx_cap web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 adv_1000fdx_cap 1 -- 1,0 # dladm show-linkprop -p adv_1000hdx_cap web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 adv_1000hdx_cap 1 -- 1,0 |
The properties that advertise the link's gigabit capabilities are adv_1000fdx_cap and adv_1000hdx_cap. To disable these properties from being advertised, you would type the following commands:
# dladm set-linkprop -p adv_1000fdx_cap=0 web1 # dladm set-linkprop -p adv_1000hdx_cap=0 web1 |
Listing the Ethernet parameter settings would display the following output:
# dladm show-ether web1 LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE web1 current up yes 1G-f both |
You can obtain information about the NIC driver's properties by displaying either the Ethernet parameter settings or the link properties.
On the system that has the NIC whose properties you want to modify, assume the System Administrator role.
The System Administrator role includes the Network Management profile. To create the role and assign the role to a user, see Chapter 9, Using Role-Based Access Control (Tasks), in System Administration Guide: Security Services.
To obtain information about the Ethernet parameter settings, use the following command:
# dladm show-ether [-x] data-link |
where the -x option includes additional parameter information about the link. Without the -x option, only the current parameter settings are displayed.
To obtain information about all the properties of the link, use the following command:
# dladm show-linkprop data-link |
This example displays an extended list of parameter information about a specified link.
# dladm show-ether -x web1 LINK PTYPE STATE AUTO SPEED-DUPLEX PAUSE web1 current up yes 1G-f both -- capable -- yes 1G-fh,100M-fh,10M-fh both -- adv -- yes 100M-fh,10M-fh both -- peeradv -- yes 100M-f,10M-f both |
With the -x option, the command also displays the built-in capabilities of the specified link, as well as the capabilities that are currently advertised between the host and the link partner. The following information is displayed:
For the Ethernet device's current state, the link is up and functioning at 1 gigabits per second at full duplex. Its autonegotiation capability is enabled and has bidirectional flow control, in which both host and link partner can send and receive pause frames.
Regardless of the current setting, the capabilities of the Ethernet device are listed. The negotiation type can be set to automatic, the device can support speeds of 1 gigabits per second, 100 megabits per second, and 10 megabits per second, at both full and half duplex. Likewise, pause frames can be received or sent in both directions between host and link partner.
The capabilities of web1 are advertised as follows: autonegotiation, speed-duplex, and flow control of pause frames.
Similarly, web1's link or peer partner advertises the following capabilities: autonegotiation, speed-duplex, and flow control of pause frames.
This example shows how to list all the properties of a link. If you want to display only a specific property, you use the -p option with the specific property that you want to monitor.
# dladm show-linkprop web1 LINK PROPERTY VALUE DEFAULT POSSIBLE web1 speed 1000 -- -- web1 autopush -- -- -- web1 zone -- -- -- web1 duplex half -- half,full web1 state unknown up up,down web1 adv_autoneg_cap 1 1 1,0 web1 mtu 1500 1500 -- web1 flowctrl no bi no,tx,rx,bi web1 adv_1000fdx_cap 1 1 1,0 web1 en_1000fdx_cap 1 1 1,0 web1 adv_1000hdx_cap 1 1 1,0 web1 en_1000hdx_cap 1 1 1,0 web1 adv_100fdx_cap 0 0 1,0 web1 en_100fdx_cap 0 0 1,0 web1 adv_100hdx_cap 0 0 1,0 web1 en_100hdx_cap 0 0 1,0 web1 adv_10fdx_cap 0 0 1,0 web1 en_10fdx_cap 0 0 1,0 web1 adv_10hdx_cap 0 0 1,0 web1 en_10hdx_cap 0 0 1,0 |
The settings for the speed and duplex capabilities of the link are manually configured on the enabled-speed properties which are labeled en_*_cap. For example, en_1000fdx_cap is the property for the gigabit full-duplex capability, and en_100hdx_cap is the property for the 100 megabits half-duplex capability. The settings of these enabled speed properties are advertised between the host and its link partner by corresponding advertised speed properties, which are labeled adv_*_cap such as adv_1000fdx_cap and adv_100hdx_cap.
Normally, the settings of a given enabled speed property and the corresponding advertised property are identical. However, if a NIC supports some advanced features such as Power Management, those features might set limits on the bits that are actually advertised between the host and its link partner. For example, with Power Management, the values of the adv_*_cap properties might only be a subset of the values of the en_*_cap properties. For more details about the enabled and advertised speed properties, see the dladm(1M) man page.
This procedure and the next procedure show how to configure private properties. Both procedures apply to properties specific to the e1000g driver. However, the general steps can be used to configure private properties of other NIC drivers as well.
Bulk traffic, such as file transfers, normally involves negotiation of large packets across the network. In such cases, you can obtain better performance from the e1000g driver by configuring it to automatically use DMA binding, where a threshold is defined for packet fragment sizes. If a fragment size surpasses the threshold, then DMA binding is used for transmitting. If a fragment size is within the threshold, then bcopy mode is used, where the fragment data is copied to the preallocated transmit buffer.
To set the threshold, perform the following steps:
On the system that has the NIC whose properties you want to modify, assume the System Administrator role.
Set the appropriate value for the _tx_bcopy_threshold property.
# dladm set-linkprop -p _tx_bcopy_threshold=value e1000g-data-link |
For this property, the valid values for the threshold range from 60 through 2048.
As with configuring public properties, the interface must also be unplumbed before private property settings can be modified.
(Optional) Verify the new threshold value.
# dladm show-linkprop -p _tx_bcopy_threshold e1000g-data-link |
Parameters that regulate the rate at which interrupts are delivered by the e1000g driver also affect network and system performance. Typically network packets are delivered to the upper layer of the stack by generating an interrupt for every packet. In turn the interrupt rate, by default, is automatically adjusted by the GLD layer in the kernel. However, this mode might not be desirable in all network traffic conditions. For a discussion of this issue, refer to this document (http://www.stanford.edu/class/cs240/readings/mogul.pdf) that was presented at the USENIX technical conference in 1996. Thus, in certain circumstances, setting the interrupt rate manually becomes necessary to obtain better performance.
To define the interrupt rate, you set the following parameters:
_intr_throttling_rate determines the delay between interrupt assertions regardless of network traffic conditions.
_intr_adaptive determines whether automatic tuning of the interrupt throttling rate is enabled. By default, this parameter is enabled.
On the system that has the NIC whose driver properties you want to modify, assume the System Administrator role.
The System Administrator role includes the Network Management profile. To create the role and assign the role to a user, see Chapter 9, Using Role-Based Access Control (Tasks), in System Administration Guide: Security Services.
If necessary, identify the device whose driver property you want to modify.
# dladm show-phys |
Disable automatic tuning of the interrupt throttling rate.
# dladm set-linkprop -p _intr_adaptive=0 e1000g-data-link |
When automatic tuning of the interrupt throttling rate is enabled, then any value that is set for the parameter _intr_throttling_rate is ignored.
Unplumb the network interface.
Set the value for the minimum inter interrupt level.
# dladm set-linkprop -p _intr_throttling_rate=value e1000g-data-link |
The default value of the _intr_throttling_rate parameter is 550 on SPARC® based systems and 260 on x86 based systems. Setting the minimum inter-interrupt level to 0 disables the interrupt throttling logic.
Plumb the interface and configure an IP address for the interface.
(Optional) Display the threshold's new settings.
This example uses an x86 based system with an e1000g NIC. The driver is configured with a threshold setting toggle between using DMA binding or the bcopy mode for transmitting packets. The setting for the interrupt throttling rate is also modified. Further, the e1000g data link has been renamed with a customized name. Therefore, the configuration is performed on the data link by referring to the customized name, public0.
# dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE public0 ether up 100Mb full e1000g0 # dladm show-linkprop -p _tx_bcopy_threshold public0 LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _tx_bcopy_threshold 512 512 -- # dladm show-linkprop -p _intr-throttling_rate LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _intr-throttling_rate 260 260 -- # ifconfig public0 unplumb # dladm set-linkprop -p _tx_bcopy_threshold=1024 public0 # dladm set-linkprop -p _intr_adaptive=0 public0 # dladm set-linkprop -p _intr-throttling_rate=1024 public0 # ifconfig public0 plumb 10.10.1.2/24 up # dladm show-linkprop -p _tx_bocopy_threshold=1024 public0 LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _tx_bcopy_threshold 1024 512 -- # dladm show-linkprop -p _intr_adaptive public0 LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _intr-adaptive 0 1 -- # dladm show-linkprop -p _intr-throttling_rate LINK PROPERTY VALUE DEFAULT POSSIBLE public0 _intr-throttling_rate 1024 260 -- |