Paravirtual (PV) drivers are high-performance network and disk drivers that significantly reduce the overhead of the traditional implementation of I/O device emulation. These drivers provide improved network performance, disk throughput, and system efficiency because these drivers do no not emulate other devices such as physical NICs. The paravirtualized network driver ZVNET for Oracle Solaris Kernel Zones, interact with the hypervisor in the host OS through hypercall to achieve low-delay and high-throughput network performance.
Starting with Oracle Solaris 11.3, the paravirtualized IPoIB datalink is created as an anet resource in Oracle Solaris Kernel Zone and you can configure this datalink by using the zonecfg command. The anet resource creates an IPoIB VNIC when the kernel zone boots up. The IPoIB VNIC is created over a partition of the lower link InfiniBand host channel adapter (IB HCA) and the port tuple in the global zone. Each IPoIB VNIC has one-to-one match and communicates with paravirtualized IPoIB datalink in the kernel zone. Each of these VNICs have a unique MAC address and can have a unique or different partition key (pkey). For each anet resource, you can configure the mode over which the IPoIB datalinks are run. Connected mode (CM) and unreliable datagram (UD) mode are supported and you can configure these modes by using the zonecfg command. For more information, see Zone Resource Types and Their Properties in Oracle Solaris Zones Configuration Resources.
To display the configured IPoIB datalinks within the kernel zone, use the dladm command.
Example 35 Creating a Paravirtualized IPoIB DatalinkYou create a paravirtualized IPoIB datalink by creating an automatic network (anet) in the kernel zone and specifying the mandatory properties lower-link and pkey. Set the lower-link property to one of the valid IB partitions and set pkey to one of the partition keys provided by that partition. The property linkmode, which can be either cm or ud, is optional. If you do not specify a value, the value is set to cm by default.
$ zonecfg -z kzone0 zonecfg:kzone0> add anet zonecfg:kzone0:anet> set lower-link=net1 zonecfg:kzone0:anet> set pkey=0x8001 zonecfg:kzone0:anet> set linkmode=cm zonecfg:kzone0:anet> end $ zoneadm -z kzone0 bootExample 36 Displaying Physical Device Information in Kernel Zones
The following example displays the physical device and attributes of all physical datalinks in a kernel zone including the InfiniBand devices.
solariskzone0:~$ dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 1000 full zvnet0 net1 Infiniband up 32000 full zvnet1
The following example displays the physical device and all the key attributes of physical links in a kernel zone.
solariszone1:~$ dladm show-phys -o all LINK MEDIA STATE SPEED DUPLEX DEVICE VFS-AVAIL VFS-INUSE FLAGS net0 Ethernet up 1000 full zvnet0 -- -- ----- net1 Infiniband up 32000 full zvnet1 -- -- -----Example 37 Displaying MAC Addresses for the Physical Device
The following example displays the MAC addresses for the physical device in a kernel zone.
solariszone1:~$ dladm show-phys -m LINK SLOT ADDRESS INUSE CLIENT net0 primary 2:8:20:5:32:5a yes net0 net1 primary 80:0:0:4a:fe:80:... yes net1Example 38 Displaying the IPoIB VNIC in the Host
The following example displays the IPoIB VNIC in the host.
solariszone1:~$ dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE IDS kzone1/net0 net0 1000 2:8:20:5:32:5a random VID:0 kzone1/net1 net4 32000 80:0:0:4a:fe:.. fixed PKEY:0x8001
In this example, the notation PKEY in the IDS field indicates that the VNIC is an IPoIB VNIC.
The following example displays the MAC addresses of the IPoIB VNICs.
solariszone1:~$ dladm show-vnic -o macaddress MACADDRESS 2:8:20:5:32:5a 80:0:0:4a:fe:80:0:0:0:0:0:0:0:21:28:0:1:a0:e5:55Example 39 Displaying Datalinks in the Host
The following example displays the data links in the host including the IPoIB VNIC created on the kernel zone.
solariszone1:~$ dladm show-link LINK CLASS MTU STATE OVER net0 phys 1500 up -- net1 phys 1500 unknown -- kzone1/net0 vnic 1500 up net0 kzone1/net1 vnic 65520 up net1
For more information, see the dladm(8) man page.
You can create an IPoIB VNIC over a paravirtualized IPoIB datalink in an Oracle Solaris Kernel Zone. The IPoIB VNIC is assigned the primary MAC address of the IPoIB datalink instance if it is not already used by another MAC client. As a result, you can create a non-global zone inside a kernel zone by using the paravirtualized IPoIB datalink as the lower link. The IPoIB VNIC inherits the partition key stored in the paravirtualized IPoIB link, which is again inherited from the pkey of the IB partition in the global zone.
To support multiple Oracle Solaris zones in a kernel zone, multiple IPoIB anet instances need to be assigned to the kernel zone from the global zone, and the kernel zone needs to assign a different anet to each of its zones.
You can create the IPoIB VNIC over paravirtualized IPoIB datalink similar to the way in which you create IPoIB VNICs over the IB partition in the global zone. For information, see Configuring IPoIB VNICs.
You can specify the partition key while creating the IPoIB VNIC. A paravirtualized IPoIB datalink stores the pkey that is inherited from the partition created in the host. You can specify this pkey value when you create the VNIC. You can use the dladm show-phys -v command to see the pkey value of the paravirtualized datalink. However, you can also create a VNIC without specifying the pkey. In this case, the inherited pkey is assigned by default.
Example 40 Creating an IPoIB VNIC Over Paravirtualized IPoIB Datalink by Specifying the pkeyThe following example shows how to create the IPoIB VNIC by specifying the pkey over the IPoIB datalink net1.
$ dladm show-phys -v LINK IDS INUSE CLIENT net0 VIDS:23 yes -- net1 PKEYS:ffff no -- $ dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE net0 Ethernet up 1000 full zvnet0 net1 Infiniband up 32000 full zvnet1 $ dladm create-vnic -l net1 -P 0xffff vnic1 $ dladm LINK CLASS MTU STATE OVER net0 phys 1500 up -- net1 phys 65520 up -- vnic1 vnic 65520 up net1 $ dladm show-vnic vnic1 LINK OVER SPEED MACADDRESS MACADDRTYPE IDS vnic1 net1 32000 80:0:0:4a:fe:.. fixed PKEY:0xffff $ dladm show-phys -v LINK IDS INUSE CLIENT net0 VIDS:23 yes -- net1 PKEYS:ffff yes vnic1Example 41 Creating an IPoIB VNIC Over Paravirtualized IPoIB Datalink Without Specifying the pkey
The following example shows how to create IPoIB VNIC vnic1 without specifying the pkey over the IPoIB datalink net1.
$ dladm show-phys -v LINK ID INUSE CLIENT net0 VIDS:23 yes -- net1 PKEYS:ffff no --
$ dladm create-vnic -l net1 vnic1 $ dladm LINK CLASS MTU STATE OVER net0 phys 1500 up -- net1 phys 65520 up -- vnic1 vnic 65520 up net1
In this example, the pkey value inherited by the IPoIB datalink is assigned to the VNIC.
Example 42 Displaying the Partition KeyThe following example displays the partition key pkey for an IPoIB datalink.
$ dladm show-phys -v -o link,vids,pkeys LINK VIDS PKEYS net0 23 -- net1 -- ffff