Skip Headers
Oracle® Exalogic Elastic Cloud Machine Owner's Guide
Release EL X2-2, X3-2, and X4-2

E18478-18
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

11 Virtual LANs (VLANs)

This chapter describes how to set up a VLAN on the Ethernet connector of a Sun Network QDR InfiniBand Gateway Switch, and it describes how to assign a virtual NIC (VNIC) on a compute node to use that VLAN.

The tasks described in this chapter are optional.

It contains the following topics:

11.1 Introduction to VLAN

The Ethernet standard has a provision to combine multiple broadcast domains, and thus IP subnets, onto a single Ethernet cable using a Virtual LAN (IEEE 802.1Q VLAN) configuration. To use VLANs, both ends of the Ethernet link must be configured to support the defined VLANs. The benefits include a logical division of workload, enforcing security isolation, and splitting traffic across several manageable broadcast domains. VLANs allow traffic separation from the 10 GbE switch to compute nodes. By design, Ethernet traffic on one VLAN cannot be seen by any host on a different VLAN. To enable communication between two VLANs, you should use an external router.

Note:

You can create more than one VLAN per Ethernet connection.

For a general introduction to VNICs, see Section 10.1, "Introduction to Virtual NICs (VNICs)".

11.2 Example Scenario

To understand the use of VLANs in an Exalogic environment, consider the following example scenario.

You want to combine Production, Test, and Development environments in the same Exalogic machine. However, you do not want these systems to communicate with each other directly. The production systems require dedicated Ethernet interfaces. You wish to share resources, such as Ethernet connectors, between Test and Development systems.

For the Production systems, you may dedicate a few of the external 10 GbE connectors on the gateway switch (for example, 0A-ETH-1 to 0A-ETH-4). Production systems will be on one VLAN using these four dedicated 10 GbE external uplinks.

For the Development systems, you may use one Ethernet connector on the gateway switch and a VLAN of their own. For example, 1A-ETH-3 associated with VLAN ID 10. In this VLAN, the resources using 1A-ETH-3 are dedicated to Development systems.

The Test systems require two Ethernet interfaces, and they can use Development's Ethernet connector 1A-ETH-3, but on a different VLAN. For example, you can create two Ethernet interfaces using 1A-ETH-3 associated with a VLAN ID 11 for use by the Test systems. In this manner, Development systems on their VLAN get their resources while sharing the Ethernet connector or uplink with Test systems. Since the two VLANs exist on the same 10 GbE Ethernet link on the gateway switch, any traffic between the two VLANs should travel through an external router if they are required to be seen by each other.

Note:

On a single Ethernet connection using the connector on the gateway switch, you can create up to 4094 VLANs.

11.3 Tagging Ethernet Connectors With a VLAN Identifier

To tag an Ethernet connector on the gateway switch with a VLAN identifier, you must run the createvlan command on the gateway switch that the VLAN will be associated with.

In this process, you are mapping the following:

Note:

Exalogic uses the default partition, and the partition key is 0xFFFF. You can associate multiple VLANs to a single Ethernet connector.

For example, you can associate VLAN identifiers 10 and 11 to the same Ethernet connector 1A-ETH-3.

To do so, run the createvlan command, as in the following example:

  1. Log in to the gateway switch interface as root, and run the following commands:

    # createvlan 1A-ETH-3 -VLAN 10 -PKEY default

    Where 1A-ETH-3 is the Ethernet connector on the gateway switch, 10 is the VLAN identifier, and default is the partition key used in Exalogic.

    # createvlan 1A-ETH-3 -VLAN 11 -PKEY default

    Where 1A-ETH-3 is the Ethernet connector, 11 is the VLAN identifier, and default is the partition key used in Exalogic.

    If you are using Oracle Solaris compute nodes, you should also enable the connector for untagged traffic, by running the following command on the gateway switch:

    # createvlan 1A-ETH-3 -VLAN -1 -PKEY default

  2. To verify, run the following command:

    # showvlan

    The following information is displayed:

    Connector/LAG    VLN    PKEY
    --------------   ---   -----
    1A-ETH-3           0    ffff
    1A-ETH-3          10    ffff
    1A-ETH-3          11    ffff
    

    Tip:

    See the Example Scenario for more information.

11.4 Oracle Linux: Creating VNICs and Associating Them with VLANs

If you plan to associate a VNIC with a VLAN, you should provide a VLAN identifier when mapping the MAC address, partition key, GUID, and Ethernet connector. In Exalogic, the default partition key (0xFFFF) is used.

To create a VNIC and associate with a VLAN, use the following example procedure:

  1. On the gateway switch CLI, as root, complete the steps 1 through 6, as described in Section 10.2, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Linux".

  2. Run the following command to create a VNIC and associate it with a VLAN (for example, with VLAN 10, as shown in Tagging Ethernet Connectors With a VLAN Identifier):

    # createvnic 1A-ETH-3 -GUID 00:21:28:56:d0:a2:c0:a0 -mac a2:c0:a0:a8:1:1 -vlan 10 -pkey default

    Where 1A-ETH-3 is the Ethernet connector, 00:21:28:56:d0:a2:c0:a0 is the GUID, a2:c0:a0:a8:1:1 is the dummy MAC address, 10 is the VLAN identifier, and default is the partition key used in Exalogic.

    This example creates a VNIC, such as eth4.

  3. To create a second VNIC using the same Ethernet connector and GUID, run the following command to tag the VNIC with a different VLAN identifier (11):

    # createvnic 1A-ETH-3 -GUID 00:21:28:56:d0:a2:c0:a0 -mac a2:c0:a0:a8:1:a -vlan 11 -pkey default

    Where 1A-ETH-3 is the Ethernet connector, 00:21:28:56:d0:a2:c0:a0 is the GUID, a2:c0:a0:a8:1:a is the dummy MAC address defined for this second interface, 11 is the VLAN identifier, and default is the partition key used in Exalogic.

    This example creates a VNIC, such as eth5.

  4. Run the following command to verify the VNICs:

    # showvnics

    The following message is displayed:

    ID  STATE    FLG  IOA_GUID       NODE    IID    MAC    VLN  PKEY  GW
    --- --------  --- ----------------------- ---------- ---- -------------
      8 UP    N 00:21:28:00:01:A0:A3:65 computenode1 EL-C 192.168.10.29 0000 a2:c0:a0:a8:1:1 10  ffff   1A-ETH-3
      9 UP    N 00:21:28:00:01:A0:A3:65 computenode1 EL-C 192.168.10.29 0001 a2:c0:a0:a8:1:a 11  ffff   1A-ETH-3
    

    Tip:

    After creating the interfaces, you can run the ifconfig command with the -a option to verify the MAC address on the compute node. For example, to verify the new interface and its MAC address, run the following command on the Oracle Linux compute node for which the VNIC was created:

    # ifconfig -a eth4

    The output of this command shows the HWADDR, which is the MAC address you defined for the VNIC in Section 10.2, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Linux".

  5. If you want your VNIC configuration to persist across reboots, you should save VNIC configuration to a file. For information about doing this on Oracle Linux, see step 13 in Section 10.2, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Linux". Be sure to create a bonded interface comprising two VNICs on Oracle Linux, for high availability purposes.

11.5 Oracle Solaris 11 Express: Creating VNICs and Associating Them with VLANs

If you wish to associate a VNIC with a VLAN, you should provide a VLAN identifier when mapping the MAC address.

To create a VNIC and associate with a VLAN, use the following example procedure:

  1. On the gateway switch CLI, as root, complete the steps 1 through 7, as described in Section 10.3, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11 Express". This procedure creates a VNIC, such as eoib0.

  2. On Oracle Solaris, VLAN-tagged VNICs may only be created from the Oracle Solaris compute node. Therefore, run the following command on the Oracle Solaris compute node to create a VNIC and associate it with a VLAN (for example, with VLAN 10, as shown in Section 11.3, "Tagging Ethernet Connectors With a VLAN Identifier"):

    # dladm create-vnic -l eoib0 -m a2:c0:a0:a8:1:2 -v 10 eoib0_v10

    This step creates a VNIC named eoib0_v10 associated with VLAN 10 that uses the same port GUID and Ethernet Connector that eoib0 uses. The value a2:c0:a0:a8:1:2 is the dummy MAC address defined for this interface.

  3. To create another VNIC using the same Ethernet connector and GUID, run the following command on the Oracle Solaris compute node to tag the VNIC with a different VLAN identifier (11):

    # dladm create-vnic -l eoib0 -m a2:c0:a0:a8:1:3 -v 11 eoib0_v11

    This step creates a VNIC named eoib0_v11 associated with VLAN 11 that uses the same port GUID and Ethernet Connector that eoib0 uses. The value a2:c0:a0:a8:1:3 is the dummy MAC address defined for this second interface.

  4. Run the following command on the gateway switch CLI, as root, to verify the VNICs:

    # showvnics

    The following message is displayed:

    ID  STATE    FLG  IOA_GUID            NODE    IID    MAC    VLN    PKEY    GW
    --- --------  --- ----------------------- ---------- ---- -------------
    0   UP       N    00:21:28:00:01:A0:A6:95  computenode1  EL-C  192.168.10.29 0000  62:C0:A0:A8:01:05  NO  ffff   1A-ETH-3
    1   UP       H    00:21:28:00:01:A0:A6:95  computenode1  EL-C  192.168.10.29 8001  62:C0:A0:A8:01:02  10  ffff   1A-ETH-3
    2   UP       H    00:21:28:00:01:A0:A6:95  computenode1  EL-C  192.168.10.29 8002  62:C0:A0:A8:01:03  11  ffff   1A-ETH-3
    

    Note that the VNIC with ID 0 corresponds to an already created network-administered VNIC (created using the steps described in Section 10.3, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11 Express"). The two new host-administered interfaces are the ones with IDs 1 and 2 (with VLANs 10 and 11, respectively).

    Tip:

    After creating the interfaces, you can run the dladm command to verify the MAC address on the compute node. For example, to verify the new interface and its MAC address, run the following command on the Oracle Solaris compute node for which the VNIC was created:

    # dladm show-vnic eoib0_v10

    The output of this command shows the MAC address, which is the MAC address you defined for the VNIC.

  5. If you want your VNIC configuration to persist across reboots on Oracle Solaris, see step 11 in Section 10.3, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11 Express". Be sure to configure the VNIC interfaces in an IPMP group on Oracle Solaris, for high availability purposes.

11.6 Oracle Solaris 11.1: Creating VNICs and Associating Them with VLANs

If you wish to associate a VNIC with a VLAN, you should provide a VLAN identifier when mapping the MAC address.

To create a VNIC and associate with a VLAN, use the following example procedure:

  1. On the gateway switch CLI, as root, complete the steps 1 through 15, as described in Section 10.4, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11.1". This procedure creates a VNIC, such as eoib0.

  2. On Oracle Solaris, VLAN-tagged VNICs may only be created from the Oracle Solaris compute node. Run step 16 in Section 10.4, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11.1".

  3. You can verify that the VNICs were created by running step 17 in Section 10.4, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11.1."

  4. Run the following command on the gateway switch CLI, as root, to verify the VNICs:

    # showvnics

    The following message is displayed:

    ID  STATE    FLG  IOA_GUID            NODE    IID    MAC    VLN    PKEY    GW
    --- --------  --- ----------------------- ---------- ---- -------------
    0   UP       N    00:21:28:00:01:A0:A6:95  computenode1  EL-C  192.168.10.29 0000  62:C0:A0:A8:01:05  NO  ffff   1A-ETH-3
    1   UP       H    00:21:28:00:01:A0:A6:95  computenode1  EL-C  192.168.10.29 8001  62:C0:A0:A8:01:02  10  ffff   1A-ETH-3
    2   UP       H    00:21:28:00:01:A0:A6:95  computenode1  EL-C  192.168.10.29 8002  62:C0:A0:A8:01:03  11  ffff   1A-ETH-3
    

    Note that the VNIC with ID 0 corresponds to an already created network-administered VNIC (created using the steps described in Section 10.3, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11 Express"). The two new host-administered interfaces are the ones with IDs 1 and 2 (with VLANs 10 and 11, respectively).

    Tip:

    After creating the interfaces, you can run the dladm command to verify the MAC address on the compute node. For example, to verify the new interface and its MAC address, run the following command on the Oracle Solaris compute node for which the VNIC was created:

    # dladm show-vnic eoib0_v10

    The output of this command shows the MAC address, which is the MAC address you defined for the VNIC.

  5. If you want to configure the VNIC interfaces in an IPMP group on Oracle Solaris, for high availability purposes, see step 18 in Section 10.3, "Setting Up Ethernet Over InfiniBand (EoIB) on Oracle Solaris 11 Express".