test

Logical Domains 1.2 Administration Guide

Chapter 7 Using Virtual Networks

This chapter describes how to use a virtual network with Logical Domains software, and covers the following topics:

Introduction to a Virtual Network

A virtual network allows domains to communicate with each other without using any external physical networks. A virtual network also can allow domains to use the same physical network interface to access a physical network and communicate with remote systems. A virtual network is created by having a virtual switch to which you can connect virtual network devices.

Virtual Switch

A virtual switch (vsw) is a component running in a service domain and managed by the virtual switch driver. A virtual switch can be connected to some guest domains to enable network communications between those domains. In addition, if the virtual switch is associated also with a physical network interface, then this allows network communications between guest domains and the physical network over the physical network interface. A virtual switch also has a network interface, vswn, which allows the service domain to communicate with the other domains connected to that virtual switch. It can be used like any regular network interface and configured with the ifconfig(1M) command.

Note –

When a virtual switch is added to a service domain, its network interface is not plumbed. So, by default, the service domain is unable to communicate with the guest domains connected to its virtual switch. To enable network communications between guest domains and the service domain, the network interface of the associated virtual switch must be plumbed and configured in the service domain. See Enabling Networking Between the Control/Service Domain and Other Domains for instructions.

Virtual Network Device

A virtual network (vnet) device is a virtual device that is defined in a domain connected to a virtual switch. A virtual network device is managed by the virtual network driver, and it is connected to a virtual network through the hypervisor using logical domain channels (LDCs).

A virtual network device can be used as a network interface with the name vnetn, which can be used like any regular network interface and configured with the ifconfig(1M) command.

Figure 7–1 Setting Up a Virtual Network

Diagram shows how to set up a virtual network as described in the text.

Following is a explanation for the example in Figure 7–1.

Basically the virtual switch behaves like a regular physical network switch and switches network packets between the different systems, such as guest domains, service domain, and physical network, to which it is connected.

Managing a Virtual Switch

This section describes adding a virtual switch to a domain, setting options for a virtual switch, and removing a virtual switch.

ProcedureAdd a Virtual Switch

  1. Use the following command syntax to add a virtual switch.


    # ldm add-vsw [default-vlan-id=vlan-id] [pvid=port-vlan-id] [vid=vlan-id1,vlan-id2,...]
[mac-addr=num] [net-dev=device] [mode=sc] [mtu=size] [id=switch-id] vswitch-name ldom

    Where:

    • default-vlan-id=vlan-id specifies the default virtual local area network (VLAN) to which a virtual switch and its associated virtual network devices belong to implicitly, in untagged mode. It serves as the default port VLAN id (pvid) of the virtual switch and virtual network devices. Without this option, the default value of this property is 1. Normally, you would not need to use this option. It is provided only as a way to change the default value of 1. See Using VLAN Tagging With Logical Domains Software for more information.

    • pvid=port-vlan-id specifies the VLAN to which the virtual switch needs to be a member, in untagged mode. See Using VLAN Tagging With Logical Domains Software for more information.

    • vid=vlan-id specifies one or more VLANs to which a virtual switch needs to be a member, in tagged mode. See Using VLAN Tagging With Logical Domains Software for more information.

    • mac-addr=num is the MAC address to be used by this switch. The number must be in standard octet notation; for example, 80:00:33:55:22:66. If you do not specify a MAC address, the switch is automatically assigned an address from the range of public MAC addresses allocated to the Logical Domains Manager. See Assigning MAC Addresses Automatically or Manually for more information.

    • net-dev=device is the path to the network device over which this switch operates.

    • mode=sc enables virtual networking support for prioritized processing of Solaris Cluster heartbeat packets in a Logical Domains environment. Applications like Solaris Cluster need to ensure that high priority heartbeat packets are not dropped by congested virtual network and switch devices. This option prioritizes Solaris Cluster heartbeat frames and ensures that they are transferred in a reliable manner.

      You must set this option when running Solaris Cluster in a Logical Domains environment and using guest domains as Solaris Cluster nodes. Do not set this option when you are not running Solaris Cluster software in guest domains, because you could impact virtual network performance.

    • mtu=size specifies the maximum transmission unit (MTU) of a virtual switch device. Valid values are in the range of 1500-16000.

    • id=switch-id is the ID of a new virtual switch device. By default, ID values are generated automatically, so set this property if you need to match an existing device name in the OS. See Virtual Device Identifier and Network Interface Name.

    • vswitch-name is the unique name of the switch that is to be exported as a service. Clients (network) can attach to this service.

    • ldom specifies the logical domain in which to add a virtual switch.

ProcedureSet Options for an Existing Virtual Switch

  1. Use the following command syntax to set options for a virtual switch that already exists.


    # ldm set-vsw [pvid=port-vlan-id] [vid=vlan-id1,vlan-id2,...] [mac-addr=num]
[net-dev=device] [mode=[sc]] [mtu=size] vswitch-name

    Where:

    • mode= (left blank) stops special processing of Solaris Cluster heartbeat packets.

    • Otherwise, the command arguments are the same as described in Add a Virtual Switch.

ProcedureRemove a Virtual Switch

  1. Use the following command syntax to remove a virtual switch.


    # ldm rm-vsw [-f] vswitch-name

    Where:

    • -f attempts to force the removal of a virtual switch. The removal might fail.

    • vswitch-name is the name of the switch that is to be removed as a service.

Managing a Virtual Network Device

This section describes adding a virtual network device to a domain, setting options for an existing virtual network device, and removing a virtual network device.

ProcedureAdd a Virtual Network Device

  1. Use the following command syntax to add a virtual network device.


    # ldm add-vnet [mac-addr=num] [mode=hybrid] [pvid=port-vlan-id] 
[vid=vlan-id1,vlan-id2,...] [mtu=size] [id=network-id] if-name vswitch-name ldom

    Where:

    • mac-addr=num is the MAC address for this network device. The number must be in standard octet notation; for example, 80:00:33:55:22:66. See Assigning MAC Addresses Automatically or Manually for more information.

    • mode=hybrid to request the system to use NIU Hybrid I/O on this vnet if possible. If it is not possible, the system reverts to virtual I/O. This hybrid mode is considered a delayed reconfiguration if set on an active vnet. See Using NIU Hybrid I/O for more information.

    • pvid=port-vlan-id specifies the VLAN to which the virtual network device needs to be a member, in untagged mode. See Using VLAN Tagging With Logical Domains Software for more information.

    • vid=vlan-id specifies one or more VLANs to which a virtual network device needs to be a member, in tagged mode. See Using VLAN Tagging With Logical Domains Software for more information.

    • mtu=size specifies the maximum transmission unit (MTU) of a virtual network device. Valid values are in the range of 1500-16000.

    • id=network-id is the ID of a new virtual network device. By default, ID values are generated automatically, so set this property if you need to match an existing device name in the OS. See Virtual Device Identifier and Network Interface Name.

    • if-name, interface name, is a unique name to the logical domain, assigned to this virtual network device instance for reference on subsequent ldm set-vnet or ldm rm-vnet commands.

    • vswitch-name is the name of an existing network service (virtual switch) to which to connect.

    • ldom specifies the logical domain to which to add the virtual network device.

ProcedureSet Options for an Existing Virtual Network Device

  1. Use the following command syntax to set options for a virtual network device that already exists.


    # ldm set-vnet [mac-addr=num] [vswitch=vswitch-name] [mode=[hybrid]] 
[pvid=port-vlan-id] [vid=vlan-id1,vlan-id2,...] [mtu=size] if-name ldom

    Where:

    • mode= (left blank) disables NIU Hybrid I/O.

    • if-name, interface name, is the unique name assigned to the virtual network device you want to set.

    • ldom specifies the logical domain from which to remove the virtual network device.

    • Otherwise, the command arguments are the same as described in Add a Virtual Network Device.

ProcedureRemove a Virtual Network Device

  1. Use the following command syntax to remove a virtual network device.


    # ldm rm-vnet [-f] if-name ldom

    Where:

    • -f attempts to force the removal of a virtual network device from a logical domain. The removal might fail.

    • if-name, interface name, is the unique name assigned to the virtual network device you want to remove.

    • ldom specifies the logical domain from which to remove the virtual network device.

Virtual Device Identifier and Network Interface Name

When you add a virtual switch or virtual network device to a domain, you can specify its device number by setting the id property.


# ldm add-vsw [id=switch-id] vswitch-name ldom
# ldm add-vnet [id=network-id] if-name vswitch-name ldom

Each virtual switch and virtual network device of a domain has a unique device number that is assigned when the domain is bound. If a virtual switch or virtual network device was added with an explicit device number (by setting the id property), the specified device number is used. Otherwise, the system automatically assigns the lowest device number available. In that case, the device number assigned depends on how virtual switch or virtual network devices were added to the system. The device number eventually assigned to a virtual switch or virtual network device is visible in the output of the ldm list-bindings command when a domain is bound.

The following example shows that the primary domain has one virtual switch, primary-vsw0. This virtual switch has a device number of 0 (switch@0).


primary# ldm list-bindings primary
...
VSW
    NAME         MAC               NET-DEV DEVICE   DEFAULT-VLAN-ID PVID VID MTU MODE
    primary-vsw0 00:14:4f:fb:54:f2 e1000g0 switch@0 1               1    5,6 1500
...

The following example shows that the ldg1 domain has two virtual network devices: vnet and vnet1. The vnet device has a device number of 0 (network@0) and the vnet1 device has a device number of 1 (network@1).


primary# ldm list-bindings ldg1
...
NETWORK
    NAME  SERVICE              DEVICE    MAC               MODE   PVID VID MTU
    vnet  primary-vsw0@primary network@0 00:14:4f:fb:e0:4b hybrid 1        1500
    ...
    vnet1 primary-vsw0@primary network@1 00:14:4f:f8:e1:ea        1        1500
...

When a domain with a virtual switch is running the Solaris OS, the virtual switch has a network interface, vswN. However, the network interface number of the virtual switch, N, is not necessarily the same as the device number of the virtual switch, n.

Similarly, when a domain with a virtual network device is running the Solaris OS, the virtual network device has a network interface, vnetN. However, the network interface number of the virtual network device, N, is not necessarily the same as the device number of the virtual network device, n.

Caution – Caution –

The Solaris OS preserves the mapping between the name of a network interface and a virtual switch or virtual network based on the device number. If a device number is not explicitly assigned to a virtual switch or virtual network device, its device number can change when the domain is unbound and is later bound again. In that case, the network interface name assigned by the OS running in the domain can also change and break the existing configuration of the system. This might happen, for example, when a virtual switch or a virtual network interface is removed from the configuration of the domain.

You cannot use the ldm list-* commands to directly determine the Solaris OS network interface name that corresponds to a virtual switch or virtual network device. However, you can obtain this information by using a combination of the output from ldm list -l command and from the entries under /devices on the Solaris OS.

ProcedureFind Solaris OS Network Interface Name

In this example procedure, guest domain ldg1 contains two virtual network devices, net-a and net-c. To find the Solaris OS network interface name in ldg1 that corresponds to net-c, do the following. This example also shows differences if you are looking for the network interface name of a virtual switch instead of a virtual network device.

  1. Use the ldm command to find the virtual network device number for net-c.


    # ldm list -l ldg1
...
NETWORK
NAME         SERVICE                     DEVICE       MAC
net-a        primary-vsw0@primary        network@0    00:14:4f:f8:91:4f
net-c        primary-vsw0@primary        network@2    00:14:4f:f8:dd:68
...

    The virtual network device number for net-c is 2 (network@2).

    To determine the network interface name of a virtual switch, find the virtual switch device number, n as switch@n.

  2. To find the corresponding network interface on ldg1, log into ldg1 and find the entry for this device number under /devices.


    # uname -n
ldg1
# find /devices/virtual-devices@100 -type c -name network@2\*
/devices/virtual-devices@100/channel-devices@200/network@2:vnet1

    The network interface name is the part of the entry after the colon; that is, vnet1.

    To determine the network interface name of a virtual switch, replace the argument to the -name option with virtual-network-switch@n\*. Then, find the network interface with the name vswN.

  3. Plumb vnet1 to see that it has the MAC address 00:14:4f:f8:dd:68 as shown in the ldm list -l output for net-c in Step 1.


    # ifconfig vnet1
vnet1: flags=1000842<BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
          inet 0.0.0.0 netmask 0
          ether 0:14:4f:f8:dd:68

Assigning MAC Addresses Automatically or Manually

You must have enough media access control (MAC) addresses to assign to the number of logical domains, virtual switches, and virtual networks you are going to use. You can have the Logical Domains Manager automatically assign MAC addresses to a logical domain, a virtual network (vnet), and a virtual switch (vsw), or you can manually assign MAC addresses from your own pool of assigned MAC addresses. The ldm subcommands that set MAC addresses are add-domain, add-vsw, set-vsw, add-vnet, and set-vnet. If you do not specify a MAC address in these subcommands, the Logical Domains Manager assigns one automatically.

The advantage to having the Logical Domains Manager assign the MAC addresses is that it utilizes the block of MAC addresses dedicated for use with logical domains. Also, the Logical Domains Manager detects and prevents MAC address collisions with other Logical Domains Manager instances on the same subnet. This frees you from having to manually manage your pool of MAC addresses.

MAC address assignment happens as soon as a logical domain is created or a network device is configured into a domain. In addition, the assignment is persistent until the device, or the logical domain itself, is removed.

Range of MAC Addresses Assigned to Logical Domains Software

Logical domains have been assigned the following block of 512K MAC addresses:

00:14:4F:F8:00:00 ~ 00:14:4F:FF:FF:FF

The lower 256K addresses are used by the Logical Domains Manager for automatic MAC address allocation, and you cannot manually request an address in this range:

00:14:4F:F8:00:00 - 00:14:4F:FB:FF:FF

You can use the upper half of this range for manual MAC address allocation:

00:14:4F:FC:00:00 - 00:14:4F:FF:FF:FF

Automatic Assignment Algorithm

When you do not specify a MAC address in creating logical domain or a network device, the Logical Domains Manager automatically allocates and assigns a MAC address to that logical domain or network device. To obtain this MAC address, the Logical Domains Manager iteratively attempts to select an address and then checks for potential collisions.

Before selecting a potential address, the Logical Domains Manager first looks to see if it has a recently freed, automatically assigned address saved in a database for this purpose (see Freed MAC Addresses). If so, the Logical Domains Manager selects its candidate address from the database.

If no recently freed addresses are available, the MAC address is randomly selected from the 256K range of addresses set aside for this purpose. The MAC address is selected randomly to lessen the chance of a duplicate MAC address being selected as a candidate.

The address selected is then checked against other Logical Domains Managers on other systems to prevent duplicate MAC addresses from actually being assigned. The algorithm employed is described in Duplicate MAC Address Detection. If the address is already assigned, the Logical Domains Manager iterates, choosing another address, and again checking for collisions. This continues until a MAC address is found that is not already allocated, or a time limit of 30 seconds has elapsed. If the time limit is reached, then the creation of the device fails, and an error message similar to the following is shown.


Automatic MAC allocation failed.  Please set the vnet MAC address manually.

Duplicate MAC Address Detection

To prevent the same MAC address from being allocated to different devices, one Logical Domains Manager checks with other Logical Domains Managers on other systems by sending a multicast message over the control domain's default network interface, including the address that the Logical Domain Manager wants to assign to the device. The Logical Domains Manger attempting to assign the MAC address waits for one second for a response back. If a different device on another LDoms-enabled system has already been assigned that MAC address, the Logical Domains Manager on that system sends back a response containing the MAC address in question. If the requesting Logical Domains Manager receives a response, it knows the chosen MAC address has already been allocated, chooses another, and iterates.

By default, these multicast messages are sent only to other managers on the same subnet; the default time-to-live (TTL) is 1. The TTL can be configured using the Service Management Facilities (SMF) property ldmd/hops.

Each Logical Domains Manager is responsible for:

If the Logical Domains Manager on a system is shut down for any reason, duplicate MAC addresses could occur while the Logical Domains Manager is down.

Automatic MAC allocation occurs at the time the logical domain or network device is created and persists until the device or the logical domain is removed.

Freed MAC Addresses

When a logical domain or a device associated with an automatic MAC address is removed, that MAC address is saved in a database of recently freed MAC addresses for possible later use on that system. These MAC addresses are saved to prevent the exhaustion of Internet Protocol (IP) addresses from a Dynamic Host Configuration Protocol (DHCP) server. When DHCP servers allocate IP addresses, they do so for a period of time (the lease time). The lease duration is often configured to be quite long, generally hours or days. If network devices are created and removed at a high rate without the Logical Domains Manager reusing automatically allocated MAC addresses, the number of MAC addresses allocated could soon overwhelm a typically configured DHCP server.

When a Logical Domains Manager is requested to automatically obtain a MAC address for a logical domain or network device, it first looks to the freed MAC address database to see if there is a previously assigned MAC address it can reuse. If there is a MAC address available from this database, the duplicate MAC address detection algorithm is run. If the MAC address had not been assigned to someone else since it was previously freed, it will be reused and removed from the database. If a collision is detected, the address is simply removed from the database. The Logical Domains Manager then either tries the next address in the database or if none is available, randomly picks a new MAC address.

Using Network Adapters With LDoms

In a logical domains environment, the virtual switch service running in a service domain can directly interact with GLDv3-compliant network adapters. Though non-GLDv3 compliant network adapters can be used in these systems, the virtual switch cannot interface with them directly. See Configuring Virtual Switch and Service Domain for NAT and Routing for information about how to use non-GLDv3 compliant network adapters.

For information about using link aggregations, see Using Link Aggregation With a Virtual Switch.

ProcedureDetermine If a Network Adapter Is GLDv3-Compliant

  1. Use the Solaris OS dladm(1M) command, where, for example, bge0 is the network device name.


    # dladm show-link bge0
bge0            type: non-vlan   mtu: 1500      device: bge0

  2. Look at type: in the output:

    • GLDv3-compliant drivers will have a type of non-vlan or vlan.

    • Non-GLDv3-compliant drivers will have a type of legacy.

Configuring Virtual Switch and Service Domain for NAT and Routing

The virtual switch (vsw) is a layer-2 switch, that also can be used as a network device in the service domain. The virtual switch can be configured to act only as a switch between the virtual network (vnet) devices in the various logical domains but with no connectivity to a network outside the box through a physical device. In this mode, plumbing the vsw as a network device and enabling IP routing in the service domain enables virtual networks to communicate outside the box using the service domain as a router. This mode of operation is very essential to provide external connectivity to the domains when the physical network adapter is not GLDv3-compliant.

The advantages of this configuration are:

Figure 7–2 Virtual Network Routing

Diagram shows virtual network routing as described in the text.

ProcedureSet Up the Virtual Switch to Provide External Connectivity to Domains

  1. Create a virtual switch with no associated physical device.

    If assigning an address, ensure that the virtual switch has an unique MAC address.


    primary# ldm add-vsw [mac-addr=xx:xx:xx:xx:xx:xx] primary-vsw0 primary

  2. Plumb the virtual switch as a network device in addition to the physical network device being used by the domain.

    See Configure the Virtual Switch as the Primary Interface for more information about plumbing the virtual switch.

  3. Configure the virtual switch device for DHCP, if needed.

    See Configure the Virtual Switch as the Primary Interface for more information about configuring the virtual switch device for DHCP.

  4. Create the /etc/dhcp.vsw file, if needed.

  5. Configure IP routing in the service domain, and set up required routing tables in all the domains.

    For information about how to do this, refer to Packet Forwarding and Routing on IPv4 Networks in System Administration Guide: IP Services.

Configuring IPMP in a Logical Domains Environment

Internet Protocol Network Multipathing (IPMP) provides fault-tolerance and load balancing across multiple network interface cards. By using IPMP, you can configure one or more interfaces into an IP multipathing group. After configuring IPMP, the system automatically monitors the interfaces in the IPMP group for failure. If an interface in the group fails or is removed for maintenance, IPMP automatically migrates, or fails over, the failed interface's IP addresses. In a Logical Domains environment, either the physical or virtual network interfaces can be configured for failover using IPMP.

Configuring Virtual Network Devices into an IPMP Group in a Logical Domain

A logical domain can be configured for fault-tolerance by configuring its virtual network devices to an IPMP group. When setting up an IPMP group with virtual network devices, in a active-standby configuration, set up the group to use probe-based detection. Link-based detection and failover currently are not supported for virtual network devices in Logical Domains 1.2 software.

The following diagram shows two virtual networks (vnet1 and vnet2) connected to separate virtual switch instances (vsw0 and vsw1) in the service domain, which, in turn, use two different physical interfaces (e1000g0 and e1000g1). In the event of a physical interface failure, the IP layer in LDom_A detects failure and loss of connectivity on the corresponding vnet through probe-based detection, and automatically fails over to the secondary vnet device.

Figure 7–3 Two Virtual Networks Connected to Separate Virtual Switch Instances

Diagram shows two virtual networks connected to separate virtual switch instances as described in the text.

Further reliability can be achieved in the logical domain by connecting each virtual network device (vnet0 and vnet1) to virtual switch instances in different service domains (as shown in the following diagram). In this case, in addition to network hardware failure, LDom_A can detect virtual network failure and trigger a failover following a service domain crash or shutdown.

Figure 7–4 Each Virtual Network Device Connected to Different Service Domains

Diagram shows how each virtual network device is connected to a different service domain as described in the text.

Refer to the Solaris 10 System Administration Guide: IP Services for more information about how to configure and use IPMP groups.

ProcedureConfigure a Host Route

If no explicit route is configured for a router in the network corresponding to the IPMP interfaces, then one or more explicit host routes to target systems need to be configured for the IPMP probe-based detection to work as expected. Otherwise, probe detection can fail to detect the network failures.

  1. Configure a host route.


    # route add -host destination-IP gateway-IP -static

    For example:


    # route add -host 192.168.102.1 192.168.102.1 -static

    Refer to Configuring Target Systems in System Administration Guide: IP Services for more information.

Configuring and Using IPMP in the Service Domain

Network failure detection and recovery can also be set up in a Logical Domains environment by configuring the physical interfaces in the service domain into a IPMP group. To do this, configure the virtual switch in the service domain as a network device, and configure the service domain itself to act as an IP router. (Refer to the Solaris 10 System Administration Guide: IP Services for information on setting up IP routing).

Once configured, the virtual switch sends all packets originating from virtual networks (and destined for an external machine), to its IP layer, instead of sending the packets directly by means of the physical device. In the event of a physical interface failure, the IP layer detects failure and automatically re-routes packets through the secondary interface.

Since the physical interfaces are directly being configured into a IPMP group, the group can be set up for either link-based or probe-based detection. The following diagram shows two network interfaces (e1000g0 and e1000g1) configured as part of an IPMP group. The virtual switch instance (vsw0) has been plumbed as a network device to send packets to its IP layer.

Figure 7–5 Two Network Interfaces Configured as Part of IPMP Group

Diagram shows how two network interfaces are configured as part of an IPMP group as described in the text.

Using VLAN Tagging With Logical Domains Software

As of the release of Solaris 10 10/08 OS and LDoms 1.1 software, 802.1Q VLAN-Tagging support is available in the Logical Domains network infrastructure.

Note –

Tagged VLANs are not supported in any of the previous releases for LDoms networking components.

The virtual switch (vsw) and virtual network (vnet) devices support switching of Ethernet packets based on the virtual local area network (VLAN) identifier (ID) and handle the necessary tagging or untagging of Ethernet frames.

You can create multiple VLAN interfaces over a vnet device in a guest domain. You can use the Solaris OS ifconfig(1M) command to create a VLAN interface over a virtual network device, the same way it is used to configure a VLAN interface over any other physical network device. The additional requirement in the LDoms environment is that you must assign the vnet to the corresponding VLANs using the Logical Domains Manager CLI commands. Refer to the ldm(1M) for complete information about the Logical Domains Manager CLI commands.

Similarly, you can configure VLAN interfaces over a virtual switch device in the service domain. VLAN IDs 2 through 4094 are valid; VLAN ID 1 is reserved as the default-vlan-id.

When you create a vnet device on a guest domain, you must assign it to the required VLANs by specifying a port VLAN ID and zero or more VLAN IDs for this vnet, using the pvid= and vid= arguments to the ldm add-vnet command. This configures the virtual switch to support multiple VLANs in the LDoms network and switch packets using both MAC address and VLAN IDs in the network.

Similarly, any VLANs to which the vsw device itself should belong, when plumbed as a network interface, must be configured in the vsw device using the pvid= and vid= arguments to the ldm add-vsw command.

You can change the VLANs to which a device belongs using ldm set-vnet or ldm set-vsw command.

Port VLAN ID (PVID)

The PVID indicates a VLAN to which the virtual network device needs to be a member, in untagged mode. In this case, the vsw device provides the necessary tagging or untagging of frames for the vnet device over the VLAN specified by its PVID. Any outbound frames from the virtual network that are untagged are tagged with its PVID by the virtual switch. Inbound frames tagged with this PVID are untagged by the virtual switch, before sending it to the vnet device. Thus, assigning a PVID to a vnet implicitly means that the corresponding virtual network port on the virtual switch is marked untagged for the VLAN specified by the PVID. You can have only one PVID for a vnet device.

The corresponding virtual network interface, when configured using the ifconfig(1M) command without a VLAN ID and using only its device instance, results in the interface being implicitly assigned to the VLAN specified by the virtual network's PVID.

For example, if you were to plumb vnet instance 0, using the following command, and if the pvid= argument for the vnet has been specified as 10, the vnet0 interface would be implicitly assigned to belong to the VLAN 10.


# ifconfig vnet0 plumb

VLAN ID (VID)

The VID indicates the VLAN to which a virtual network device or virtual switch needs to be a member, in tagged mode. The virtual network device sends and receives tagged frames over the VLANs specified by its VIDs. The virtual switch passes any frames that are tagged with the specified VID between the virtual network device and the external network.

ProcedureAssign VLANs to a Virtual Switch and Virtual Network Device

  1. Assign the virtual switch (vsw) to two VLANs, for example. Configure VLAN 21 as untagged and VLAN 20 as tagged. Assign the virtual network (vnet) to three VLANs, for example. Configure VLAN 20 as untagged and VLAN 21 and 22 as tagged.


    # ldm add-vsw net-dev=e1000g0 pvid=21 vid=20 primary-vsw0 primary
# ldm add-vnet pvid=20 vid=21,22 vnet01 primary-vsw0 ldom1

  2. Plumb the VLAN interfaces.

    This example assumes that the instance number of these devices is 0 in the domains and the VLANs are mapped to these subnets:

    VLAN 

    Subnet 

    20 

    192.168.1.0 (netmask: 255.255.255.0) 

    21 

    192.168.2.0 (netmask: 255.255.255.0) 

    22 

    192.168.3.0 (netmask: 255.255.255.0) 

    1. Plumb the VLAN interface in the service (primary) domain.


      primary# ifconfig vsw0 plumb
primary# ifconfig vsw0 192.168.2.100 netmask 0xffffff00 broadcast + up
primary# ifconfig vsw20000 plumb
primary# ifconfig vsw20000 192.168.1.100 netmask 0xffffff00 broadcast + up

    2. Plumb the VLAN interface in the guest (ldom1) domain.


      ldom1# ifconfig vnet0 plumb
ldom1# ifconfig vnet0 192.168.1.101 netmask 0xffffff00 broadcast + up
ldom1# ifconfig vnet21000 plumb
ldom1# ifconfig vnet21000 192.168.2.101 netmask 0xffffff00 broadcast + up
ldom1# ifconfig vnet22000 plumb
ldom1# ifconfig vnet22000 192.168.3.101 netmask 0xffffff00 broadcast + up

      For more information about how to configure VLAN interfaces in the Solaris OS, refer to Administering Virtual Local Area Networks in System Administration Guide: IP Services.

Using NIU Hybrid I/O

The virtual I/O framework implements a hybrid I/O model for improved functionality and performance. The hybrid I/O model combines direct and virtualized I/O to allow flexible deployment of I/O resources to virtual machines. It is particularly useful when direct I/O does not provide full capability for the virtual machine, or direct I/O is not persistently or consistently available to the virtual machine. This could be because of resource availability or virtual machine migration. The hybrid I/O architecture is well-suited to the Network Interface Unit (NIU), a network I/O interface integrated on chip, on Sun UltraSPARC T2–based platforms. This allows the dynamic assignment of Direct Memory Access (DMA) resources to virtual networking devices and, thereby, provides consistent performance to applications in the domain.

NIU hybrid I/O is available for Sun UltraSPARC T2–based platforms. This feature is enabled by an optional hybrid mode that provides for a virtual network (vnet) device where the DMA hardware resources are loaned to a vnet device in a guest domain for improved performance. In the hybrid mode, a vnet device in a guest domain can send and receive unicast traffic from an external network directly into the guest domain using the DMA hardware resources. The broadcast traffic and unicast traffic to the other guest domains in the same system continue to be sent using the virtual I/O communication mechanism.

Figure 7–6 Hybrid Virtual Networking

Diagram shows hybrid virtual networking as described in the text.

The hybrid mode applies only for the vnet devices that are associated with a virtual switch (vsw) configured to use an NIU network device. As the shareable DMA hardware resources are limited, up to only three vnet devices per vsw can have DMA hardware resources assigned at a given time. If more than three vnet devices have the hybrid mode enabled, the assignment is done on a first-come, first-served basis. As there are two NIU network devices in a system, there can be a total of six vnet devices on two different virtual switches with DMA hardware resources assigned.

Following are points you need to be aware of when using this feature:

ProcedureConfigure a Virtual Switch With an NIU Network Device

  1. For example, do the following to configure a virtual switch with a NIU network device.

    1. For example, determine an NIU network device.


      # grep nxge /etc/path_to_inst
"/niu@80/network@0" 0 "nxge"
"/niu@80/network@1" 1 "nxge"

    2. For example, configure a virtual switch.


      # ldm add-vsw net-dev=nxge0 primary-vsw0 primary

ProcedureEnable Hybrid Mode

  1. For example, enable a hybrid mode for a vnet device while it is being created.


    # ldm add-vnet mode=hybrid vnet01 primary-vsw0 ldom01

ProcedureDisable Hybrid Mode

  1. For example, disable hybrid mode for a vnet device.


    # ldm set-vnet mode= vnet01 ldom01

Using Link Aggregation With a Virtual Switch

As of the release of the Solaris 10 10/08 OS and the Logical Domains 1.1 software, the virtual switch can be configured to use a link aggregation. A link aggregation is used as the virtual switch's network device to connect to the physical network. This configuration enables the virtual switch to leverage the features provided by the IEEE 802.3ad Link Aggregation Standard. Such features include increased bandwidth, load balancing, and failover. For information about how to configure link aggregation, see the System Administration Guide: IP Services.

After you create a link aggregation, you can assign it to the virtual switch. Making this assignment is similar to assigning a physical network device to a virtual switch. Use the ldm add-vswitch or ldm set-vswitch command to set the net-dev property.

When the link aggregation is assigned to the virtual switch, traffic to and from the physical network flows through the aggregation. Any necessary load balancing or failover is handled transparently by the underlying aggregation framework. Link aggregation is completely transparent to the virtual network (vnet) devices that are on the guest domains and that are bound to a virtual switch that uses an aggregation.

Note –

You cannot group the virtual network devices (vnet and vsw) into a link aggregation.

You can plumb and use the virtual switch that is configured to use a link aggregation in the service domain. See Configure the Virtual Switch as the Primary Interface.

The following figure illustrates a virtual switch configured to use an aggregation, aggr1, over physical interfaces e1000g0 and e1000g1.

Figure 7–7 Configuring a Virtual Switch to Use a Link Aggregation

Diagram shows how to set up a virtual switch to use a link aggregation as described in the text.

Configuring Jumbo Frames

The Logical Domains virtual switch (vsw) and virtual network (vnet) devices can now support Ethernet frames with payload sizes larger than 1500 bytes. This change results in these drivers being able to increase network throughput.

ProcedureConfigure Virtual Network and Virtual Switch Devices to Use Jumbo Frames

You enable jumbo frames by specifying the maximum transmission unit (MTU) for the virtual switch device. In such cases, the virtual switch device and all virtual network devices that are bound to the virtual switch device use the specified MTU value.

In certain circumstances, you can specify an MTU value directly on a virtual network device. You might do this if the required MTU value for the virtual network device should be less than that supported by the virtual switch.

Note –

On the Solaris 10 5/09 OS, the MTU of a physical device must be configured to match the MTU of the virtual switch. For more information about configuring particular drivers, see the man page that corresponds to that driver in Section 7D of the Solaris reference manual. For example, to get information about the e1000g driver, see the e1000g(7D) man page.

On the OpenSolarisTM OS, the vsw driver automatically configures the MTU of the physical device. Thus, you need not perform additional configuration steps.

  1. Become superuser on the control domain.

  2. Determine the value of MTU that you want to use for the virtual network.

    You can specify an MTU value from 1500 to 16000 bytes. The specified MTU must match the MTU of the physical network device that is assigned to the virtual switch.

  3. Specify the MTU value of a virtual switch device or virtual network device.

    Do one of the following:

    • Enable jumbo frames on a new virtual switch device in the service domain by specifying its MTU as a value of the mtu property.


      # ldm add-vsw mtu=value vswitch-name ldom

      In addition to configuring the virtual switch, this command updates the MTU value of each virtual network device that will be bound to this virtual switch.

    • Enable jumbo frames on an existing virtual switch device in the service domain by specifying its MTU as a value of the mtu property.


      # ldm set-vsw mtu=value vswitch-name

      In addition to configuring the virtual switch, this command updates the MTU value of each virtual network device that will be bound to this virtual switch.

    In rare circumstances, you might need to use the ldm add-vnet or ldm set-vnet command to specify an MTU value for a virtual network device that differs from the MTU value of the virtual switch. For example, you might change the virtual network device's MTU value if you configure VLANs over a virtual network device and the largest VLAN MTU is less than the MTU value on the virtual switch. A vnet driver that supports jumbo frames might not be required for domains where only the default MTU value is used. However, if the domains have virtual network devices bound to a virtual switch that uses jumbo frames, ensure that the vnet driver supports jumbo frames.

    If you use the ldm set-vnet command to specify an mtu value on a virtual network device, future updates to the MTU value of the virtual switch device are not propagated to that virtual network device. To reenable the virtual network device to obtain the MTU value from the virtual switch device, run the following command:


    # ldm set-vnet mtu= vnet-name ldom

    Note that enabling jumbo frames for a virtual network device automatically enables jumbo frames for any HybridIO resource that is assigned to that virtual network device.

    On the control domain, the Logical Domains Manager updates the MTU values that are initiated by the ldm set-vsw and ldm set-vnet commands as delayed reconfiguration operations. To make MTU updates to domains other than the control domain, you must stop a domain prior to running the ldm set-vsw or ldm set-vnet command to modify the MTU value.

    Note –

    You cannot use the OpenSolaris 2009.06 dladm set-linkprop command to change the MTU value of Logical Domains virtual network devices.

Example 7–1 Configuring Jumbo Frames on Virtual Switch and Virtual Network Devices

Compatibility With Older (Jumbo-Unaware) Versions of the vnet and vsw Drivers

Drivers that support jumbo frames can interoperate with drivers that do not support jumbo frames on the same system. This interoperability is possible as long as jumbo frame support is not enabled when you create the virtual switch.

Note –

Do not set the mtu property if any guest or service domains that are associated with the virtual switch do not use Logical Domains drivers that support jumbo frames.

Jumbo frames can be enabled by changing the mtu property of a virtual switch from the default value of 1500. In this instance, older driver versions ignore the mtu setting and continue to use the default value. Note that the ldm list output will show the MTU value you specified and not the default value. Any frames larger than the default MTU are not sent to those devices and are dropped by the new drivers. This situation might result in inconsistent network behavior with those guests that still use the older drivers. This applies to both client guest domains and the service domain.

So, while jumbo frames are enabled, ensure that all virtual devices in the Logical Domains network are upgraded to use the new drivers that support jumbo frames. Also ensure that you upgrade to Logical Domains Manager 1.2 so that you can configure jumbo frames.