Go to main content

Creating and Using Oracle® Solaris Kernel Zones

Exit Print View

Updated: August 2021
 
 

Managing Single-Root I/O NIC Virtualization on Kernel Zones

You can create and administer single root I/O (SR-IOV) NIC virtual functions (VF) on kernel zones by using the iov property of the zonecfg anet resource type. SR-IOV enables the efficient sharing of Peripheral Component Interconnect Express (PCIe) devices among virtual machines and is implemented in the system hardware to achieve I/O performance that is comparable to bare metal performance.

SR-IOV must be enabled on the datalink in the global zone in order to enable it on the anet resource in a kernel zone. For information about using SR-IOV in Oracle Solaris, see Using Single Root I/O Virtualization With VNICs in Managing Network Virtualization and Network Resources in Oracle Solaris 11.4.

The iov property is only supported on kernel zones and native (solaris) zones.

When you enable the iov property, the ability to suspend and resume the kernel zone and migrate it using warm or live migration is limited to host systems and zones running Oracle Solaris 11.4. See About Migration of Kernel Zones with SR-IOV-Enabled anet Resources for more information.

See Zone Global Properties in Oracle Solaris Zones Configuration Resources for information about how to enable and configure the iov property of the anet resource type.


Tip  -  When using some Intel network adapters that support SR-IOV, a virtual function might be the target of malicious behavior. Unexpected software-generated frames can slow traffic between the host system and the virtual switch, which might negatively affect performance. You can work around this issue by configuring all SR-IOV-enabled ports to use VLAN tagging to drop unexpected and potentially malicious frames, See Example 15, Configuring SR-IOV and VLAN Tagging on an anet Resource for an example.

How to Enable SR-IOV NIC Virtual Functions on a Kernel Zone With a Single anet Resource

  1. Become a zone administrator.

    For more information, see Using Rights Profiles to Install and Manage Zones in Creating and Using Oracle Solaris Zones.

  2. Enable iov on an anet.

    Using zonecfg, set the iov property on a selected anet resource.

    $ pfbash zonecfg -z kernel-zone
    zonecfg:kernel-zone> select anet id=id-number
    zonecfg:kernel-zone:anet> set lower-link=network-interface
    zonecfg:kernel-zone>set iov=iov-value
    zonecfg:kernel-zone:anet> set iov=auto
    zonecfg:kernel-zone:anet> end ; exit

    The following example demonstrates enabling the iov property on an anet resource belonging to the kernel zone kzone1.

    global$ pfbash zonecfg -z kzone1
    zonecfg:kzone1> select anet id=0
    zonecfg:kzone1:anet> set lower-link=net1
    zonecfg:kzone1:anet> set iov=auto
    zonecfg:kzone1:anet> end ; exit
  3. (Optional) Confirm that the iov property is set for the anet resource in the kernel zone configuration.
    $ zonecfg -z kernel-zone info anet id=id-number

    For example, on the system global and the anet resource with ID 0 of the kernel zone kzone1:

    $ zonecfg -z kzone1 info anet id=0
    anet:
            lower-link: net1
            configure-allowed-address: true
            iov: auto
            id: 0
  4. Ensure that SR-IOV is enabled on the chosen network interface.
    $ dladm show-linkprop -p iov network-interface

    For example, on the system global and the network interface net1:

    global$ dladm show-linkprop -p iov net1
    LINK     PROPERTY        PERM VALUE        EFFECTIVE    DEFAULT   POSSIBLE
    net1     iov             rw   on           on           auto      auto,on,off
  5. Boot the kernel zone.
    $ zoneadm -z kernel-zone boot

    For example, to boot the kernel zone kzone1 on the system global:

    global$ zoneadm -z kzone1 boot
  6. Verify that the VF was successfully added.
    $ zlogin kernel-zone
    kernel-zone$ dladm show-phys -i

    The output from this command varies depending on which version of Oracle Solaris is running in the global zone of the host system and in the kernel zone. The following is sample output for selected Oracle Solaris version combinations.

    • The global zone and the kernel zone are both running Oracle Solaris 11.4:

      global$ pfexec zlogin kzone
      kzone$ dladm show-phys -i
      LINK              MEDIA          ID       DEVICE      ACTIVE     STANDBY
      net0              Ethernet       anet:0   vnic1000    ixgbevf0   zvnet0
    • The global zone is running Oracle Solaris 11.4 and the kernel zone is running Oracle Solaris 11.3:

      global$ pfexec zlogin kzone
      kzone$ dladm show-phys -i
      LINK         MEDIA        STATE     SPEED    DUPLEX      DEVICE
      net0         Ethernet     down      0        unknown     ixgbevf0
    • The global zone is running Oracle Solaris 11.3 and the kernel zone is running Oracle Solaris 11.4:

      global$ pfexec zlogin kzone
      kzone$ dladm show-phys -i
      LINK              MEDIA          ID       DEVICE      ACTIVE     STANDBY
      net0              Ethernet       anet:0   vnic1000    ixgbevf0   --
Example 14  Confirming the zonecfg iov Value on an anet

The following example shows the iov value on anet 0. The value is set to auto. If set to the default value off, it would not be displayed.

global$ pfbash zonecfg -z kzone1
zonecfg:kzone1> select anet id=0
zonecfg:kzone1:anet> info
anet:
        lower-link: net1
        configure-allowed-address: true
        iov: auto
        id: 0
Example 15  Configuring SR-IOV and VLAN Tagging on an anet Resource

The following example shows how to explicitly set a VLAN ID to enable VLAN tagging on an anet resource, which allows untagged and potentially malicious frames to be dropped.

global$ pfbash zonecfg -z kzone1
zonecfg:kzone1> select anet id=0
zonecfg:kzone1:anet> set iov=auto
zonecfg:kzone1:anet> set vlan-id=11
zonecfg:kzone1:anet> end ; exit

For more information about setting VLAN IDs and VLAN tagging, see Configuring Virtual LANs in Kernel Zones.

About Migration of Kernel Zones with SR-IOV-Enabled anet Resources

For a kernel zone that is using SR-IOV, the ability to suspend and resume the kernel zone and migrate using warm or live migration is limited to host systems and zones running Oracle Solaris 11.4. If the kernel zone configuration includes the settings iov=auto or iov=on, migration fails if the source host, target host, or the kernel zone is running an older release.

If you must migrate a kernel zone that has iov enabled, and either the kernel zone or the global zone on the source host or target host is running a release that is older than Oracle Solaris 11.4, you must perform a cold migration.

Configuring Network High Availability for SR-IOV-Enabled Kernel Zones

You can achieve network high availability for SR-IOV devices within a kernel zone if the global zone is using datalink multipathing (DLMP) with SR-IOV.

To enable high availability, you must create a DLMP link aggregation of links with SR-IOV enabled in the global zone, then add an anet resource in the kernel zone that uses the DLMP aggregation as its lower link. Set the anet resource's iov property to auto and boot the zone.

For more information, see Datalink Multipathing Aggregations in Managing Network Datalinks in Oracle Solaris 11.4.

Example 16  Configure a DLMP Link Aggregation for Network High Availability in a Kernel Zone
global$ pfbash dladm set-linkprop -p iov=on net3
global$ dladm set-linkprop -p iov=on net4
global$ dladm create-aggr -l net3 -l net4 -m dlmp dlmp0
global$ zonecfg -z kz1
zonecfg:kz1> create -t SYSsolaris-kz
zonecfg:kz1> add anet
zonecfg:kz1:anet> set lower-link=dlmp0
zonecfg:kz1:anet> set iov=auto
zonecfg:kz1:anet> set id=0
zonecfg:kz1:anet> end

global$ zoneadm -z kz1 boot
global$ dladm show-aggr -C dlmp0
LINK       PORT           SPEED DUPLEX   STATE     CLIENTS
dlmp0      --             10000Mb full   up        --
           net3           10000Mb full   up        kz1/net0
           net4           10000Mb full   up        dlmp0

global$ zlogin kz1 dladm show-phys -i
LINK              MEDIA          ID       DEVICE      ACTIVE     STANDBY
net0              Ethernet       anet:0   vnic1000    ixgbevf0   zvnet0

Using Virtual Functions and Shadow VNICs With Kernel Zones

A virtual function (VF) on a kernel zone is created when an anet belonging to a kernel zone is configured with the zonecfg iov property set to on or auto. The VF is assigned by the host system to the kernel zone.

Each VF assigned to a kernel zone has an associated shadow VNIC in the host system. You can use shadow VNICs to show network statistics.

The following shows example output of the shadow VNIC kzone1/net0 on the system global:

global$ dladm show-link
LINK                CLASS     MTU    STATE    OVER
net1                phys      1500   unknown  --
net0                phys      1500   up       --
net2                phys      1500   up       --
kzone1/net0         vnic      1500   unknown  net1

global$ dlstat show-link kzone1/net0
LINK               IPKTS   RBYTES    OPKTS   OBYTES
kzone1/net0        0        0        3      126

VF can be allocated from a DLMP aggregation. You can set iov=auto on a DLMP aggregation, which causes VF to be allocated when there's an available VF resource. An example is shown in Example 16, Configure a DLMP Link Aggregation for Network High Availability in a Kernel Zone.

Setting iov=on over either DLMP or trunk aggregation is prohibited.

The zonecfg anet property bwshare enables a shadow VNIC to be set on a link only if the underlying physical link is supported. See the dladm(8) and zonecfg(8) man pages for additional information.

For additional information about VNICs and network configuration, consult Managing Network Virtualization and Network Resources in Oracle Solaris 11.4.