Using the Large Receive Offload Feature in Oracle Solaris

Language:

In Oracle Solaris, you can use the large receive offload (LRO) feature to merge successive incoming packets into a single packet before the packets are delivered to the IP layer. The incoming packets must share the same transport protocol, local or remote IP address, and port number. This set of attributes are also known as five-tuple. If most of the packets share the same five-tuple, the packet processing overhead in the IP layer and the layers above it reduces improving network throughput. Typically, the TCP links under heavy load contain packets that share the same five-tuple.

Benefits Of Using the LRO Feature

In Oracle Solaris, the merging of the packets is implemented in the MAC layer. The NIC delivers the packets to the networking stack and the MAC layer merges the successive incoming packets that share the same five-tuple information.

The LRO feature in Oracle Solaris provides the following benefits:

Significantly improves the system's receive-side TCP performance. This improvement is higher in the kernel zone environment.
Allows you to enable LRO on a physical NIC that does support the LRO feature.
Allows you to selectively enable or disable LRO for each datalink in the host that includes a physical NIC, VNIC or anet resource, and PV NIC or SR-IOV VF within the Oracle Solaris Kernel Zone.

Enabling LRO for Datalinks

You can administer the LRO feature on both physical NICs and VNICs. You can use the lro property to enable or disable the LRO feature on a per-VNIC-basis. By default, the VNIC lro property is inherited from the underlying datalink. Because the default value for the lro link property of a physical NIC is off, the lro property that is inherited by the VNICs from the physical NIC is disabled by default.

You can enable LRO on a datalink in the following ways:

When you enable the lro link property on a physical NIC, LRO is enabled for the primary MAC client of the NIC. If the lro property is set to auto for other MAC clients such as VNICs configured on the NIC, the effective value of the lro property for the MAC clients is inherited from the NIC.
When you enable the lro link property on a MAC client such as a VNIC, the lro property is enabled for only the VNIC.

Similarly, you can disable LRO on a datalink by using the lro link property.

You can enable or disable the lro link property on any network device such as a VNIC, a physical NIC, a SR-IOV VF, a link aggregation, or a para-virtualized (PV) NIC in a guest domain. You can enable or disable LRO for a network device by setting the lro property. You use the following command syntax to set the lro property for a datalink:

# dladm set-linkprop -p lro=value link

For a VNIC, you can specify the following values for the lro property:

on: Enables LRO for the VNIC MAC client.
off: Disables LRO for the VNIC MAC client.
auto: Inherits the effective value of the lower datalink and enables merging of the packets if LRO is effective on the lower datalink. This value is the default value.

For a physical NIC, you can specify the following values for the lro property:

on: Enables LRO for the primary MAC client of the NIC. The other MAC clients of the NIC inherit the state of the lro property if the value of the MAC client's lro property is auto.
off: Disables LRO for the primary MAC client of the NIC. The other MAC clients of the NIC inherit the state of the lro property if the value of the MAC client's lro property is auto.
auto: Default value of the lro property. When you set the lro property to auto, the effective value of the lro property is off.

For a PV NIC, you can specify the following values for the lro property:

on: Enables LRO for the primary MAC client of the PV NIC.
off: Disables LRO for the primary MAC client of the PV NIC.
auto: Inherits the effective value of the lower shadow VNIC and enables merging of packets on the shadow VNIC if LRO is enabled.

Note - The SR-IOV VF in a guest domain is considered to be like a physical NIC. When you set the lro property to auto, the effective value of the property is off.

Example 77 Enabling LRO for a Physical NIC

The following example shows how to enable LRO for the physical NIC net0 and check the status of the lro property.

# dladm set-linkprop -p lro=on net0
# dladm show-linkprop -p lro net0
LINK     PROPERTY        PERM VALUE        EFFECTIVE    DEFAULT   POSSIBLE
net0     lro             rw   on           on           auto      on,off,auto

Example 78 Enabling LRO for PV NICs and anet Resources

The following example shows how to enable LRO for the PV NIC net0.

# dladm set-linkprop -t -p lro=on net0

The following example shows the LRO status of a PV VNIC for a kernel zone after the value for the lro property is set to on.

# dladm show-linkprop -p lro zone1/net0
LINK            PROPERTY        PERM    VALUE   EFFECTIVE       DEFAULT POSSIBLE
zone1/net0      lro             rw      on      on              auto    on, off, auto

The following example shows the default LRO status of a PV VNIC for a kernel zone if the effective value is on for the lro property of the lower shadow VNIC.

# dladm show-linkprop -p lro zone1/net0
LINK            PROPERTY        PERM    VALUE   EFFECTIVE       DEFAULT POSSIBLE
zone1/net0      lro             rw      auto    on              auto    on, off, auto

The following example shows how to enable LRO for the anet resource z1/net0 of the native zone z1.

# dladm set-linkprop -t -p lro=on z1/net0

LRO Support for Link Aggregations

The LRO feature is supported for link aggregations. For information about link aggregations, see Chapter 2, Configuring High Availability by Using Link Aggregations in Managing Network Datalinks in Oracle Solaris 11.3.

Use the following command syntax to set the lro property for a link aggregation:

# dladm set-linkprop -p lro=value aggr

For a link aggregation, you can specify the following values for the lro property:

on: Enables LRO for the primary MAC client of the link aggregation.
off: Disables LRO for the primary MAC client of the link aggregation.
auto: Default value of the lro property. When you set the lro property to auto, the effective value of the lro property is off.

Note - The default setting on the link aggregation disables LRO on the aggregated datalinks even though LRO is enabled. To enable LRO on the aggregated datalinks, you need to explicitly set the lro property to on.

LRO Support for Zones

You can configure the lro link property for the anet resource of a kernel zone by setting the lro property by using the zonecfg command.

For the anet resource of a native zone, you can set the lro property by using the dladm command. See Example 78, Enabling LRO for PV NICs and anet Resources.

Example 79 Enabling LRO for a Kernel Zone

This example shows how to enable LRO for the anet resource of the kernel zone kzone1.

# zonecfg -z kzone1
zonecfg:kzone1> select anet id=1
zonecfg:kzone1:anet> set lro=on

To show the status of the lro property for the anet resource kzone1/net1 of the kernel zone kzone1:

# dladm show-linkprop -p lro kzone1/net1
LINK        PROPERTY        PERM VALUE        EFFECTIVE    DEFAULT   POSSIBLE
kzone1/net1 lro             rw   on           on           auto      on,off,auto