6 Creating Exalogic Virtual Servers (vServers)

This chapter describes how to create virtual servers using Enterprise Manager Operations Control (EMOC) and to verify that servers of the same type do not run on the same underlying hardware.

Refer to your product-specific enterprise deployment guide to determine the number and size of vServers you must create.

A virtual server (vServer) is a virtual host created in the Oracle Exalogic Elastic Cloud. A virtual server is similar to a physical server. If you are planning to deploy on Virtual Exalogic, then you need to create a specific number of virtual servers to host the various components of the deployment.

Because a virtual server can run on any physical compute node in the Exalogic machine, for maximum availability, it is recommended that vServers hosting the same component do not run on the same physical server. This ensures continuity of service, even if the underlying compute node fails. For example, both wls_proda1 and wls_proda2 should not run on compute node 1. They should be distributed across two different compute nodes.

This section contains the following topics:

6.1 Prerequisites

This topic provides the prerequisites for virtual servers.

Before starting an Exalogic deployment, ensure that the following tasks have been performed:

  1. Exalogic rack has been commissioned and one-command run.
  2. Accounts have been created in Exalogic Control.
  3. Private IPoIB network has been created for the account, enabling secure communications between the virtual servers assigned to the account as described in Creating a Private IPoIB Network.
  4. You have created and loaded a Server Template for the operating system you wish to deploy.
  5. You have created a vServer type that matches the specification of the virtual servers you want to create.
  6. A Client Access Network has been created, using a bonded Network Interface for communication between the vServers and an external load balancer.

6.2 Sizing a Virtual Server

Oracle provides a few out of the box templates for virtual servers. These templates have a base operating system of Oracle Linux 5 or Oracle Linux 6.

Virtual Servers come in a number of sizes, each size having an assigned number of virtual CPU's and memory. These templates are only examples and users can create their own with different operating system images and different CPUs and memory.

When you create a Virtual Server, you need to create a Virtual Server which is sufficient to your needs. If you are using this guide in conjunction with a product EDG such as the Identity and Access Management EDG then those guides will have guidelines on the sizes of Virtual Servers to use.

If you do not have this information, you can create a vServer with the same characteristics as a physical server. For example, if you are deploying Oracle HTTP server and it requires 1 CPU 4GB of memory and 10GB of disk space then you need to create a Virtual Server of the same size.

For additional information, refer to the product installation guides and associated support notes such as Oracle Fusion Middleware System and Requirements and Specifications document.

6.3 Obtaining a vServer Guest Template

Oracle provides a number of base templates for Oracle Exalogic and are available to download from the Oracle E-delivery.

If the default template provided is not sufficient, download another template from Oracle E-Delivery. It is recommended that Oracle Linux v6 or later is used. For example to download the OL6.5 template:
  1. Login to edelivery.oracle.com.
    Search for
    • Product Pack: Oracle Fusion Middleware

    • Platform: Linux x86-64

  2. Click the link for Oracle Exalogic Elastic Cloud Software 11g Media Pack.
  3. Click download next to Oracle Exalogic 2.0.6.1.2 Base Guest Template for Exalogic Linux 6 x86-64(64bit).

6.4 Loading the Guest Template into Exalogic Control

Server Templates contain the configuration of an individual vServer with its virtual disk. Templates can be of the format .tgz, .tar or other file types.

You can use HTTP, HTTPS, or FTP protocols to upload Server Templates from any network, including the external EoIB network, that is available to the virtual machine hosting the Enterprise Controller component of Exalogic Control. You can upload a Server Template to Exalogic Control as follows:
  1. Log in to the Exalogic Control as a Cloud User.
  2. In the navigation pane, click vDC Management.
  3. Under vDC Accounts, click the name of your account.
    The vDC Account dashboard is displayed.
  4. Click Server Templates on the top navigation bar.
    The Server Templates available in your account are listed.
  5. Under Server Templates, click the Upload Server Template icon. Alternatively, click Upload Server Template under Operate on the Actions pane.
  6. On the Identify Server Template screen, enter a name and description for the Server Template to be uploaded.
  7. You can add tags for later identification and search and click Next.
    The Specify Server Template Details screen is displayed.
  8. Enter the following information:
    • Image Sub Type : Template

    • Upload Source: Local Host

  9. Select the template file you downloaded from edelivery and click Upload.

6.5 About Distribution Groups

A distribution group prevents virtual servers assigned to it from running on the same physical nodes. By preventing different vServers of the same type running on the same physical server, you prevent the failure of the underlying physical server from taking out the complete system.

In a Fusion Middleware Exalogic implementation, the following distribution groups are required:

  • EDG_OTD: Prevents two Oracle Traffic Director Servers from running on the same physical server

  • EDG_PRODA: Prevents two instances of PRODA from running on the same physical server

  • EDG_PRODB: Prevents two instances of PRODB from running on the same physical server

  • EDG_LDAP: Prevents two LDAP instances from running on the same physical server.

This section contains the following topic:

6.5.1 Creating a Distribution Group

To create a distribution group, perform the following steps:

  1. Log in to Exalogic Control using the following URL:

    https://emochost.example.com:9443/emoc/

    Log in using your assigned administrator account. For more information on accounts, refer to the Exalogic Elastic Cloud Administrator's Guide.

  2. Expand vDC Management.
  3. Navigate to vDCs - Accounts - Cloud Admin Account.
  4. In the actions window, click Create Distribution Group.
  5. Enter a Name, for example: EDG_OTD.
  6. Click Next.
  7. Enter Number of Elements. This is a number that defines the number of EECS Servers on which the vServers can be placed.
  8. Click Next.
  9. Click Finish.

Repeat for each distribution group to be created. Table 6-1 lists distribution groups and the number of elements for each.

Table 6-1 Number of Elements for Distribution Groups

Distribution Group Number of Elements

EDG_LDAP

2

EDG_OTD

2

EDG_PRODA

2

EDG_PRODB

2

6.6 Creating vServer Volumes

When you create a vServer, by default, it creates one default volume and allocates the space to swap and the root file system. For a more efficient controlled way to do this, create separate volumes for each vServer to mount for the swap and temp space.

To create separate volumes for each vServer:

  1. Log in to Exalogic Control using the following URL:

    https://emochost.example.com:9443/emoc/

    Log in using your assigned administrator account. For more information on accounts, refer to the Oracle Elastic Cloud Administration Guide.

  2. Expand vDC Management.
  3. Navigate to vDCs, Accounts, and then Cloud Admin Account.
  4. Select Create Volume from the Actions menu.
  5. Give the volume a name, for example vServer1_tmp, and a description.
  6. Click Next.
  7. On the Volume Configuration screen, enter a size for the volume.

    Note:

    Swap space follows the Linux best practice of 2x memory upto 16GB memory.

    Do not select shared.

  8. Click Next.
  9. On the Volume Summary screen, click Finish to create the volume.
  10. Repeat the above steps for the swap volume, example vServer1_swap.

6.7 vServer Types

This topic provides information on the vServer types along with memory and swap space.

Table 6-2 lists the vServer types used in this document. These vServer types can be used as a guide. Refer to the Oracle Fusion Middleware System Requirements and Specifications for the latest hardware requirements.

Table 6-2 vServer Types

vServer Type Memory Swap Space Tmp Space

LARGE

8GB

16GB

2GB

EXTRA_LARGE

16GB

16GB

2GB

6.8 Creating a vServer

This topic provides the steps for creating a vServer.

To create a vServer, perform the following steps:
  1. Log into Exalogic Control.
  2. Expand vDC Management.
  3. Navigate to vDCs - Accounts - Cloud Admin Account.
  4. In the Actions window, click Create vServer.
  5. Enter the following:
    • Name: For example: otdhost1

    • Number of vServers:1

    Select: Support High Availability.

  6. Click Next.
  7. Choose the Server Template you want to deploy.
  8. Click Next.
  9. Choose the vServer type you wish to create, for example: LARGE
  10. If you have created volumes for swap and tmp for example, select them here.
  11. Click Next.
  12. Enter all of the virtual networks you want to assign.

    Note:

    Refer to the network diagrams for the networks you wish to assign.
    Network Diagrams Description
    Web Servers EoIB-client, IPoIB-FMW, IPoIB-Storage
    Application Servers IPoIB-FMW, IPoIB-Storage. IPoIB-Data
    LDAP Servers IPoIB-FMW, IPoIB-Storage, IPoIB-Data (*)

    LDAP servers will only require access to the IPoIB-Data network, if the directory type is OID.

  13. For each chosen network, enter the following:
    • IP Address Type - Static or Automatic

    • IP Address - Enter the IP address if you have a predetermined IP address to use.

      Note:

      IPoIB-vserver shared storage typically does not require pre-assigned IP addresses.
    • Hostname - Select the fully qualified host name you wish to assign to the IP address.

  14. Click Next.
  15. Enter the Distribution Group to use.
  16. Click Next.
  17. Click Next on vServerAccessControl screen.
  18. Click Finish.
Repeat for each vServer to be created.

6.9 Updating vServers

This topic provides information on updating the root password, post network configuration and in setting the MTU size on Infiniband interfaces.

Now that the vServers have been created, you need to perform the following tasks to make them available for use.

This section contains the following topics:

6.9.1 Updating the root Password

This topic provides information on updating the root password.

When the vServer is created, it has a default password, which is generally ovsroot. Change this to a value appropriate for your organization.

6.9.2 Updating /etc/hosts File

This topic provides information on updating the host file.

After configuration, your hosts file will look something like:

IP Address   Host_Name

For example:

192.168.32.3 host1-stor
Change the hosts file so that it contains both fully qualified and short names for each network, for example:
192.168.10.3 host1-int.example.com host1-int
192.168.32.3 host1-stor.example.com host1-stor
192.168.10.3 host1-data.example.com host1-data
10.10.10.5   host1-ext.example.com host1-ext

Note:

External Network interface names are assumed to be in DNS.

6.9.3 Post Network Configuration

Now that your vServer has been created, you must configure it as appropriate to your organization.

This typically includes the following steps:

6.9.3.1 Determine vServer Storage IP Address

When you created your vServer, you added the network IPoIB-Storage. This is the network the vServers use to communicate with the ZFS storage appliance.

In order for them to communicate properly, you must determine the appropriate IP address of the storage appliance to use.

To determine the IP address, perform the following steps:

  1. Log in to Exalogic Control as a Cloud user.
  2. From the navigation pane on the left, select vDC Management.
  3. Under vDC Accounts, expand the name of your account, and select the vServer for which you want to configure access to the storage appliance.
    The vServer dashboard is displayed.
  4. Select the Network tab, and note the IP address of the vServer for the IPoIB-Storage network. This corresponds with the -stor entry in the /etc/hosts file.
    For example: 172.17.0.100

6.9.3.2 Determine Storage Appliance IP Address

This topic provides information on determining storage appliance IP address.

  1. Log in to the storage appliance as root.

    For example, type:

    ssh root@exalogicsn01.example.com
    
  2. Show the network interfaces using the command:
    configuration net interfaces show
    
  3. The output is similar to the following:
    configuration net interfaces show
    Interfaces:
     
    INTERFACE   STATE    CLASS LINKS       ADDRS                  LABEL
    igb0        up       ip    igb0        10.244.64.60/21        igb0
    igb1        offline  ip    igb1        10.244.64.61/21        igb1
    ipmp1       up       ipmp  pffff_ibp1  192.168.10.15/24       ipmp1
                               pffff_ibp0
    ipmp2       up       ipmp  p8001_ibp0  192.168.20.9/24        IB_IF_8001
                               p8001_ibp1
    ipmp3       up       ipmp  p8002_ibp0  192.168.21.9/24        IB_IF_8002
                               p8002_ibp1
    ipmp4       up       ipmp  p8005_ibp0  172.17.0.9/16          IB_IF_8005p8005_ibp1
    p8001_ibp0  up       ip    p8001_ibp0  0.0.0.0/8              ibp0.8001
    p8001_ibp1  up       ip    p8001_ibp1  0.0.0.0/8              ibp1.8001
    p8002_ibp0  up       ip    p8002_ibp0  0.0.0.0/8              ibp0.8002
    p8002_ibp1  up       ip    p8002_ibp1  0.0.0.0/8              ibp1.8002
    p8005_ibp0  up       ip    p8005_ibp0  0.0.0.0/8              ibp0.8005
    p8005_ibp1  up       ip    p8005_ibp1  0.0.0.0/8              ibp1.8005
    pffff_ibp0  up       ip    pffff_ibp0  0.0.0.0/8              ibp0
    pffff_ibp1  up       ip    pffff_ibp1  0.0.0.0/8              ibp1
    
  4. Determine the corresponding IP address by looking for the IP address in the same range as 172.17.0.100.

    In this example, it is the one associated with interface ipmp4, for example: 172.17.0.9.

  5. Create an entry in the /etc/hosts on all vservers, for example:
    172.17.0.9  zfsinternal.example.com  zfsinternal
    

6.9.4 Set MTU size on InfiniBand Interfaces

To maintain optimum performance, you must update the MTU size of each of the InfiniBand interfaces on the vServer to 65520.

To do this, perform the following steps:
  1. Log in to the vServer as the root user.
  2. Verify the current MTU for bond2 by running the ifconfig command.

    Note:

    The steps in this procedure use bond2 as an example. This procedure should be repeated for all the InfiniBand interfaces.

    ifconfig bond2bond2     Link encap:InfiniBand  HWaddr
              80:58:08:CA:FE:80:00:00:00:00:00:00:00:00:00:00:00:00:00:00 
              inet addr:192.168.1.12  Bcast:192.168.1.255  Mask:255.255.255.0
              UP BROADCAST RUNNING MASTER MULTICASTMTU:1500Metric:1
              RX packets:9 errors:0 dropped:0 overruns:0 frame:0
              TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:0
              RX bytes:504 (504.0 b)  TX bytes:420 (420.0 b)
    
  3. Append the line MTU=64000 to the ifcfg file corresponding to the bond2 interface:
    echo MTU=64000 >> /etc/sysconfig/network-scripts/ifcfg-bond2
    
  4. Verify whether the MTU=64000 line was added to the ifcfg-bond2 file:
    cat /etc/sysconfig/network-scripts/ifcfg-bond2 | grep MTU
    MTU=64000
    
  5. Find the slave interfaces for bond2:
    cd /etc/sysconfig/network-scripts
    grep "MASTER=bond2" ifcfg-* | awk -F":" '{print $1}'
    ifcfg-ib0.8009
    ifcfg-ib1.8009
    
  6. Set the mode to connected for both the slave interfaces of the bond2 interface:
    echo connected > /sys/class/net/ib0.8009/mode
    echo connected > /sys/class/net/ib1.8009/mode
    
  7. Perform Steps 2 through 6 for the other InfiniBand interfaces.
  8. Stop and start the vServer as described in Oracle Exalogic Elastic Cloud Machine Owner’s Guide.
  9. After the vServer starts, log in again to the vServer using SSH as the root user.
  10. Run the ifconfig command for each InfiniBand interface, and verify that the output of the command displays MTU:64000, as shown in the following example for bond2 and its slave interfaces:
    ifconfig bond2| grep MTU
    UP BROADCAST RUNNING MASTER MULTICASTMTU:64000Metric:1
    
    ifconfig ib0.8009| grep MTU
    UP BROADCAST RUNNING SLAVE MULTICASTMTU:64000Metric:1
    
    ifconfig ib1.8009| grep MTU
    UP BROADCAST RUNNING SLAVE MULTICASTMTU:64000Metric:1
    

6.10 Moving Swap and TMP to Separate Volumes

If you create separate disk volumes for swap and tmp, update your vServer to use these new volumes.

The disk volumes are added to your virtual server as virtual volumes. They appear in the /dev directory as xvdb/c.

To determine the exact names, run the following command:

fdisk -l

Sample command output:

255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c520c
 
Device Boot Start End Blocks Id System
/dev/xvda1 * 1 32 256000 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/xvda2 32 1305 10223616 8e Linux LVM
 
Disk /dev/xvdb: 18.3 GB, 18253611008 bytes
255 heads, 63 sectors/track, 2219 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
 
 
Disk /dev/xvdc: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

As you can see from the output, /dev/xvda has a partition created on it so it is in use. This is the default disk.

Disks /dev/xvdb and /dev/xvdc do not have a partition and are therefore the attached disk volumes. You can determine which is which by the size of the volumes.

To use these volumes in the vServer, create a partition of type LVM. This enables the use of Linux LVM, and make resizing easier if required later. The procedure is the same if you are using the disk for swap or tmp.

This section contains the following topics:

6.10.1 Creating a LVM partition

This topic provides detailed steps to create a LVM partition.

  1. Choose a disk to work on using the following command:
    fdisk disk_name
    

    For example:

    fdisk /dev/xvdb
    
  2. When prompted for a command, type n.
  3. You are asked if you wish to create an extended or primary partition. Select p for primary.
  4. When promoted for a partition number, enter 1.
  5. You are then asked where on the disk to create the partition. Accept the default value of 1. Accept the default end value to use the entire disk.
  6. You are now asked First cylinder (1-2088, default 1), use default value of 1. Last cylinder, +cylinders or +size{K,M,G} (1-2088, default 2088): use the default value of 2088.
  7. Now that the partition has been created, give it a type. To do this, when prompted for a command, enter t.
  8. You can see the list of types available by entering the command L.
  9. When prompted for the Hex code, enter the code (from the previous list) for the Linux LVM. This is typically 8e.
  10. Save your changes using the command w.
  11. Validate that the changes are correct using the command fdisk -l
  12. Repeat the procedure for each disk volume.
Now that you have disk partitions, create logical volumes to use those disks.

6.10.2 Creating Logical Volumes

Now that you have disk partitions, create logical volumes to use those disks.

  1. Create a physical volume on the disk partition by using the command:
    pvcreate disk_partition
    

    For example:

    pvcreate /dev/xvdb1
    

    Note:

    The number 1 at the end of the disk, which denotes the partition number, is the same as the values you saw in the fdisk -l command.

    Repeat for each disk partition you created above.

  2. Verify that the physical volumes have been created correctly using the following command.
    pvdisplay
    
  3. Create a volume group, one for each virtual disk. You can create a single volume group for all disks, but this example uses one per disk.

    To create a volume group, use the following command:

    vgcreate volume_group_name disk partition
    

    For example:

    vgcreate volGroupSwap /dev/xvdb1
    

    Repeat for each volume group. For example: volGroupTemp volGroupSwap.

  4. Validate that the volume groups have been created properly using the following:
    vgdisplay
    
  5. Once you've created the volume groups, create a logical volume inside the volume group using the following command:
    lvcreate --name lvname --size 40G volume_group
    

    size is the size of space you wish to assign to the volume group. This equates to the size of the file system.

    For example:

    lvcreate --name Temp1 --size 2G volGroupTemp
    

    Repeat for each logical volume to be created.

  6. Validate that the logical volumes were created successful using the following command:
    lvdisplay
    

6.10.3 Creating a Swap File on the New Logical Volume

This topic provides instructions to create a sawp file on the new logical volume.

  1. Create a swap file using the following command
    mkswap volume_group
    

    For example:

    mkswap /dev/volGroupSwap/Swap1
    
  2. Create an entry in the /etc/fstab directory for the new swap file. The entry will look similar to the following:
    /dev/volGroupSwap/Swap1 swap swap defaults 0 0 
    

    Comment out the original swap entry.

  3. Validate that the new swap space is being used by issuing the following commands:
    • swapoff -a

    • swapon -a

    • swapon -s

    You can disable the original swap using the following command:

    swapoff 
    

    Note:

    This is not necessary as only your new swap space will be available after a reboot.

6.10.4 Moving /tmp to the New Logical Volume

If you have created a logical volume for /tmp, you can enable this by first creating a file system on it and then mounting it as a disk.

You do this by performing the following commands:
  1. Create a file system using the command:
    mkfs.ext3 volume_name
    

    For example:

    mkfs.ext3 /dev/volGroupTemp/Temp1
    
  2. Add the new file system to /etc/fstab so that it is automatically mounted.

    Create an entry similar to:

    /dev/volGroupTemp/Temp1 /tmp ext3 defaults 1 1
    
  3. Mount the file system using the following command
    mount -a
    
  4. Verify that the file system is created correctly using the following command:
    df -k