Using NIU Hybrid I/O

The virtual I/O framework implements a hybrid I/O model for improved functionality and performance. The hybrid I/O model combines direct and virtualized I/O to allow flexible deployment of I/O resources to virtual machines. It is particularly useful when direct I/O does not provide full capability for the virtual machine, or direct I/O is not persistently or consistently available to the virtual machine. This could be because of resource availability or virtual machine migration. The hybrid I/O architecture is well-suited for the Network Interface Unit (NIU) on Sun UltraSPARC T2, SPARC T3, and SPARC T4 platforms. An NIU is a network I/O interface that is integrated on chip. This architecture enables the dynamic assignment of Direct Memory Access (DMA) resources to virtual networking devices and, thereby, provides consistent performance to applications in the domain.

NIU hybrid I/O is available for Sun UltraSPARC T2, SPARC T3, and SPARC T4 platforms. This feature is enabled by an optional hybrid mode that provides for a virtual network (vnet) device where the DMA hardware resources are loaned to a vnet device in a guest domain for improved performance. In the hybrid mode, a vnet device in a guest domain can send and receive unicast traffic from an external network directly into the guest domain using the DMA hardware resources. The broadcast or multicast traffic and unicast traffic to the other guest domains in the same system continue to be sent using the virtual I/O communication mechanism.

Figure 8-11 Hybrid Virtual Networking

The hybrid mode applies only for the vnet devices that are associated with a virtual switch (vsw) configured to use an NIU network device. As the shareable DMA hardware resources are limited, up to only three vnet devices per vsw can have DMA hardware resources assigned at a given time. If more than three vnet devices have the hybrid mode enabled, the assignment is done on a first-come, first-served basis. As there are two NIU network devices in a system, there can be a total of six vnet devices on two different virtual switches with DMA hardware resources assigned.

Following are points you need to be aware of when using this feature:

• Hybrid mode option for a vnet device is treated as a suggestion only. That means the DMA resources are assigned only when they are available and the device is capable of using them.

• Logical Domains Manager CLI commands do not validate the hybrid mode option; that is, it is possible to set the hybrid mode on any vnet or any number of vnet devices.

• Guest domains and the service domain need to run Oracle Solaris 10 10/08 OS at a minimum.

• Up to a maximum of only three vnet devices per vsw can have DMA hardware resources loaned at a given time. As there are two NIU network devices, there can be a total of six vnet devices with DMA hardware resources loaned.

  ---

  Note - Set the hybrid mode only for three vnet devices per vsw so that they are guaranteed to have DMA hardware resources assigned.

  ---

• Hybrid mode is disabled by default for a vnet device. It must be explicitly enabled by using the ldm command. See How to Enable Hybrid Mode and the ldm(1M) man page.

• The hybrid mode option cannot be changed dynamically while the guest domain is active.

• The DMA hardware resources are assigned only when a vnet device is active that is created in the guest domain.

• The NIU 10-gigabit Ethernet driver (nxge) is used for the NIU card. The same driver is also used for other 10-gigabit network cards. However, the NIU hybrid I/O feature is available for NIU network devices only.

How to Configure a Virtual Switch With an NIU Network Device

1. Determine an NIU network device.

   The following example shows the output on an UltraSPARC T2 server:

   # grep nxge /etc/path_to_inst
   "/niu@80/network@0" 0 "nxge"
   "/niu@80/network@1" 1 "nxge"

   The following example shows the output on a SPARC T3-1 or SPARC T4-1 server:

   # grep nxge /etc/path_to_inst
   "/niu@480/network@0" 0 "nxge"
   "/niu@480/network@1" 1 "nxge"

2. Oracle Solaris 11 OS only: Identify the link name that corresponds to the NIU network device, such as nxge0.

   primary# dladm show-phys -L |grep nxge0
   net2              nxge0         /SYS/MB

3. Configure a virtual switch.
   • Oracle Solaris 10 OS. Use the following command:

     # ldm add-vsw net-dev=nxge0 primary-vsw0 primary

   • Oracle Solaris 11 OS. Use the following command:

     The following example uses net0 instead of nxge0.

     # ldm add-vsw net-dev=net0 primary-vsw0 primary

How to Enable Hybrid Mode

• For example, enable a hybrid mode for a vnet device while it is being created.

  # ldm add-vnet mode=hybrid vnet01 primary-vsw0 ldom01

How to Disable Hybrid Mode

• For example, disable hybrid mode for a vnet device.

  # ldm set-vnet mode= vnet01 ldom01