2 Configuring Exalytics Machines for Virtualization

This chapter describes the tasks to perform when configuring an Exalytics Machine for virtualization. The tasks include deploying Oracle Virtual Machine Server (Oracle VM Server) on the Exalytics Machine, configuring Oracle VM Server with Oracle Virtual Machine Manager (Oracle VM Manager), and deploying VM templates.

This chapter includes the following sections:

2.1 Before You Virtualize an Exalytics Machine

Before you virtualize an Exalytics Machine, review the following sections:

2.1.1 Documents to Review

As you prepare to virtualize an Exalytics Machine, do the following:

2.1.2 Prerequisites for Configuring Exalytics Machines for Virtualization

The following prerequisites must be met before you can virtualize an Exalytics Machine:

  • The new Exalytics Machine has been racked and mounted in a data center.

  • Oracle Integrated Lights Out Manager (ILOM), is configured to be accessible from the network using the web interface.

  • All networking information such as host name, IP address, subnet, gateway, and DNS are available.

  • Spare, unassigned IP addresses are available for the server pool and for each virtual machine deployed on the Exalytics Machine.

  • A Linux environment is available to install Oracle VM Manager.

  • A database server is available to install Oracle VM Manager schemas.

  • You have access to the internet from a computer (not necessarily the Exalytics Machine) to download required software.

    For information about hardware and software specifications, see Oracle VM Installation and Upgrade Guide for Release 3.0.3.

2.1.3 Reconfiguring Existing RAID Configurations on the Exalytics Machine

The Exalytics Machine comprises six hard disks, each having a capacity of 600GB. Two disks are configured as RAID1 and four disks are configured as RAID5. To optimize the use of the disk space, you must reconfigure the RAID1 and RAID5 configurations on the Exalytics Machine. Reconfigure RAID1 so that it consists of two virtual drives of 100GB and 457GB, and RAID5 consists of the remaining drives.

To reconfigure existing RAID configurations on the Exalytics Machine:

  1. Restart the Exalytics Machine.

  2. Press Ctrl+Y and enter RAID CLI (Command Line Interface) as the Exalytics Machine is restarting.

    Note:

    The "Copyright LSI Logic Corporation" displays during the restart sequence when it is possible to enter the RAID CLI. Observe the screen carefully as the message displays for a very short time. This avoids restarting the machine again.

  3. Enter the following MegaRAID command to delete the existing RAID configuration:

    # -CfgLdDel -LALL -a0

  4. Enter the following command to create a RAID1 partition with two virtual disks of size 100GB and 457GB:

    # -CfgLdAdd -r1[252:0, 252:1] WB Direct NoCachedBadBBU -sz102400 -sz467968 -a0

  5. Enter the following command to create a RAID5 partition with the remaining disks:

    # -CfgLdAdd -r5[252:2, 252:3, 252:4, 252:5] WB Direct NoCachedBadBBU -a0

  6. Perform an action to restart the Exalytics Machine:

    • In ILOM 3.0.14.20, select ILOM, then Remote Control, then Remote Power Control, and then Power Cycle.

    • In ILOM 3.1.2.24, select ILOM, then Host Management, and then Power Cycle.

      The Exalytics Machine restarts.

2.2 Configuring an Exalytics Machine for Virtualization

This section includes the following topics:

2.2.1 Installing and Configuring Oracle VM Server

This section includes the following topics:

2.2.1.1 Installing Oracle VM Server

Oracle VM Server is a managed virtualization environment providing a secure, server-based platform for running virtual machines. Before you can create a virtual machine you must first install Oracle VM Server on the Exalytics Machine.

To install Oracle VM Server:

  1. Navigate to the Oracle Software Delivery Cloud web site at the following link:

    http://edelivery.oracle.com

  2. Download the Oracle Exalytics Base Image 2.0.1.1.0 for Exalytics Oracle VM x86-64 to a local directory.

  3. On the Exalytics Machine, connect and log on to Oracle Integrated Lights Out Manager (ILOM).

  4. Perform an action:

    • In ILOM 3.0.14.20, select Remote Control, then Redirection, then Launch Redirection, and then Launch Remote Control.

    • In ILOM 3.1.2.24, select Remote Control, then Redirection, then Storage Redirection, and then Launch Service.

      The ILOM remote console is displayed.

  5. From the menu bar, select Devices and then CD-ROM Image to link the Oracle VM Server image as a remote virtual CD-ROM in ILOM.

  6. Perform an action to restart the machine:

    • In ILOM 3.0.14.20, select ILOM, then Remote Control, then Remote Power Control, and then Power Cycle.

    • In ILOM 3.1.2.24, select ILOM, then Host Management, and then Power Cycle.

  7. Perform an action:

    • In ILOM 3.0.14.20, select Remote Control, then Host Control, and then Next Boot Device.

    • In ILOM 3.1.2.24, select Host Management, then Host Control, and then Next Boot Device.

    The setting for the Next Boot Device is displayed.

  8. Perform the following steps to move CD-ROM to the top of the BOOT menu:

    1. In the Next Boot Device list, select CD-ROM.

    2. Click Save.

  9. Perform an action:

    • In ILOM 3.0.14.20, select ILOM, then Remote Control, then Remote Power Control, and then Power Cycle.

    • In ILOM 3.1.2.24, select ILOM, then Host Management, and then Power Cycle.

    The Exalytics Machine restarts and displays the Oracle VM Server screen.

  10. Press Enter to install Oracle VM Server.

    The Oracle VM Server is installed on the Exalytics Machine. After Oracle VM Server is installed, the Exalytics Machine restarts automatically.

  11. From the menu bar, select Devices, and then deselect CD-ROM Image.

  12. From the menu bar, select Devices, and then Save as host defaults.

  13. Perform an action:

    • In ILOM 3.0.14.20, select ILOM, then Remote Control, then Remote Power Control, and then Power Cycle.

    • In ILOM 3.1.2.24, select ILOM, then Host Management, and then Power Cycle.

      The Exalytics Machine is ready for configuration.

  14. Log on to Oracle VM Server, and enter the following command to inspect the RAID configuration:

    /opt/MegaRAID/MegaCli/MegaCli64 -LDinfo -LALL -aALL

2.2.1.2 Configuring the Network on Oracle VM Server

Before configuring the network, ensure that the Exalytics network configuration is complete.

Note:

Because you can use Exalogic scripts, you are prompted to enter details of the Exalogic machine after you restart the Exalytics Machine.

To configure the network on Oracle VM Server:

After the Exalytics Machine restarts, perform the following actions:

  1. Enter "I" to access the interactive mode.

  2. At the prompt, enter the following information:

    1. Enter 3 for Full Rack and confirm your entry.

    2. Enter 1 for Node Index.

    3. Enter an IP address for the Infiniband interface. If the Infiniband is not configured, enter a network configuration that does not conflict with existing networks. For example, enter 192.168.21.112.

    4. Enter a netmask address for the Infiniband interface. If the Infiniband is not configured, enter a network configuration that does not conflict with existing networks. For example, enter 255.255.255.0.

    5. Enter the system IP address for the ethernet bond0.

    6. Enter the bond0 netmask and gateway addresses.

    7. Enter the host name.

    8. Enter the domain.

The Exalytics Machine restarts automatically.

2.2.1.3 Pinning Dom0 CPU

To ensure performance of the Exalytics Machine, pin or bind Dom0 CPU.

To pin Dom0 CPU:

  1. Navigate to the /etc/xen directory.

  2. Edit the file xend-config.sxp to set the following:

    (dom0-cpus 10)

  3. Restart the Exalytics Machine.

2.2.2 Installing and Configuring Oracle VM Manager

Oracle VM Manager provides a graphical user interface to manage Oracle VM Servers, virtual machines, and resources. You use Oracle VM Manager to create virtual machines and server pools, import VM templates, and manage networks and storage.

This section includes the following topics:

2.2.2.1 Installing Oracle VM Manager

Oracle recommends that you install Oracle VM Manager on a separate machine than the Exalytics Machine, preferably colocated geographically within Oracle VM Server.

To install Oracle VM Manager:

  1. Download the Oracle VM Manager 3.0.3 installer (Part Number V29653-01) from the Oracle Software Delivery Cloud web site at the following link:

    http://edelivery.oracle.com/oraclevm

  2. Download the Oracle VM Manager patch 3.0.3.369 from My Oracle Support (Oracle_VM_Manager_13614645).

  3. Install Oracle VM Manager. For instructions, see Chapter 3, "Installing Oracle VM Manager" in Oracle VM Installation and Upgrade Guide for Release 3.0.3.

  4. Apply patch 3.0.3.369 to the Oracle VM Manager installation.

2.2.2.2 Discovering Oracle VM Servers

Adding Oracle VM Servers to your Oracle VM Manager environment is known as discovering Oracle VM Servers. Discovering servers is the first step to configuring the virtualization environment.

To discover Oracle VM Servers:

  1. Connect to Oracle VM Manager.

  2. In the left pane, select Hardware.

  3. From the menu bar, select Actions, and then Discover to display the Discover Servers dialog.

  4. Enter the following information for the server:

    1. IP Address.

    2. Oracle VM Agent Port. The default port number is 8899.

    3. Oracle VM Agent Password. The default password is "oracle".

  5. Click OK.

    Oracle VM Server and its details, including number of processors, IP address, processor speed, RAM, networking ports and configuration are discovered and added to the Unassigned Servers folder.

For more information, see "Chapter 6, Managing Server Pools" in Oracle VM User's Guide for Release 3.0.3.

2.2.2.3 Creating Virtual Network Interface Controllers

A Virtual Network Interface Controller (VNIC) is used by virtual machines as network cards. You create virtual network interfaces by defining a range of MAC addresses to use for each VNIC. Each MAC address corresponds with a single virtual Network Interface Controller (NIC), which is used by a virtual machine.

To create Virtual Network Interface Controllers:

  1. Log on to Oracle VM Manager.

  2. From the menu bar, select Tools, and then VNIC Manager.

    The MAC selector panel is displayed.

  3. Click Next to retrieve the next available MAC addresses.

  4. Select the number of addresses that you want to create, and click Generate.

    Note:

    Select the number of addresses that corresponds to the number of virtual machines you are deploying.

    The VNICs are created and listed in the Virtual NIC table.

  5. Click Close.

For more information, see "Chapter 7, Managing Virtual Machines" in Oracle VM User's Guide for Release 3.0.3.

2.2.2.4 Creating Server Pools and Adding a Server

A server pool consists of at least one Oracle VM Server. If you find a server pool does not have sufficient resources, such as CPU or memory, to run the virtual machines, you can expand the server pool by adding more Oracle VM Servers.

Before creating a server pool, you must have:

  • An IP address for the server pool

  • An IP address for each virtual machine

  • The password to access the Oracle VM Agent installed on Oracle VM Server

For information on creating server pools and adding servers, see "Chapter 6, Managing Server Pools" in Oracle VM User's Guide for Release 3.0.3.

Note:

When creating a server pool, you must deselect the Activate Cluster option.

2.2.2.5 Creating an Exalytics Repository

A repository stores Oracle VM resources, such as virtual machines, templates for creating virtual machines, virtual machine assemblies, and so on. You use Oracle VM Manager to create and configure Exalytics repositories, and to present the repositories to Oracle VM Servers.

Note:

Oracle recommends that you create a repository on each physical disk available. Use the smaller repository to store Oracle VM templates and cloned virtual machines and the larger repository to store virtual disks that are assigned to the virtual machines.

To create an Exalytics repository:

  1. Log on to Oracle VM Manager.

  2. In the left pane, select Home, and then Server Pools.

  3. Select the Repositories tab.

  4. Select Create to display the Create a Data Repository dialog.

  5. Perform the following actions:

    1. In the Repository Name field, enter the name for the repository.

    2. For Repository Location, select Physical Disk and click Search.

    3. From the drop-down list, select a local storage array and select the smaller of the two storage disks.

    4. Click OK.

    5. Select the server pool for the repository.

    6. Click Next.

      The new repository is available to be presented to Oracle VM Server. To enable your Oracle VM Server to use the repository, you must present the repository to Oracle VM Server.

  6. Select and move Oracle VM Server from the Servers field to the Present to Server(s) field and click Finish.

    The repository is listed in the Repository tab.

For more information, see "Chapter 4, Managing Storage" in Oracle VM User's Guide for Release 3.0.3.

2.2.2.6 Creating Virtual Machines

You use virtual templates to create virtual machines. The process of creating a virtual machine from a template is known as cloning. Before you create virtual machines, ensure that you have the following resources available:

  • A server pool

  • An Oracle VM Server

  • VM templates

The process of creating virtual machines from templates consists of the following steps:

2.2.2.6.1 Downloading VM Templates

To download VM templates:

  1. Navigate to the Oracle Software Delivery Cloud web site at the following link:

    http://edelivery.oracle.com

  2. Download the Exalytics Oracle VM Template 2.0.1.1.0 for Exalytics Oracle VM x86-64 and save it on a web server.

2.2.2.6.2 Importing VM Templates into Oracle VM Server

After downloading the templates, you import them into Oracle VM Server.

To import VM templates into Oracle VM Server:

  1. Log on to Oracle VM Server.

  2. In the left pane, select Home, and then Server Pools.

  3. Select the Templates tab.

  4. Click Import Template to display the Import Template dialog.

  5. Select the repository and Oracle VM Server and enter the URL or FTP server to the VM template that you downloaded on the web server.

  6. Click OK to import the VM template.

2.2.2.6.3 Creating Virtual Machines from VM Templates

After importing the template in Oracle VM Server, you use the template to create a virtual machine.

Note:

Oracle recommends that you use Oracle VM Manager to create virtual disks of the sizes required for your system on the larger repository. Associate these virtual disks with the virtual machines deployed on your system. These virtual disks provide enough storage for installing the Exalytics software and the various files associated with the Exalytics Machine.

To create virtual machines from a VM template:

  1. Log on to Oracle VM Server.

  2. In the left pane, select Home, and then Server Pools.

  3. Select the Templates tab.

  4. Select the template to clone, and click Clone Virtual Machine or Template to display the Clone Virtual Machine or Template dialog.

  5. Perform the following actions:

    1. For Select Clone Type, select Simple Clone.

    2. For Target Clone Type, select Virtual Machine.

    3. Enter a name and description for the virtual machine.

    4. In the Target Server Pool list, select a server pool.

  6. Click OK.

    The virtual machine is created and is ready to be configured.

For more information, see "Chapter 7, Managing Virtual Machines" in Oracle VM User's Guide for Release 3.0.3.

2.2.2.7 Maintaining Virtual Machines

This section includes the following topics:

2.2.2.7.1 Configuring and Starting a Virtual Machine

You start a virtual machine from Oracle VM Manager.

To configure and start a virtual machine:

  1. Log on to Oracle VM Manager.

  2. In the left pane, select Home.

  3. Select the server pool on which the virtual machine resides.

  4. Right-click the virtual machine and select Edit.

    The Modify Virtual Machine screen is displayed.

  5. Select the Configuration tab.

  6. Select the necessary configuration for the virtual machine, and click OK.

  7. Select the Networks tab.

  8. Select and assign a VNIC to the virtual machine, and click OK.

  9. Select the Disks tab.

  10. Select and allocate the virtual disk to the virtual machine.

  11. Right-click the virtual machine and select Start.

    The virtual machine starts.

For more information, see "Chapter 7, Managing Virtual Machines" in Oracle VM User's Guide for Release 3.0.3.

2.2.2.7.2 Configuring a Network on a Virtual Machine

After starting a virtual machine, you then configure the network on the virtual machine to enable it to access the network.

To configure a network on the virtual machine:

  1. Log on to Oracle VM Server.

  2. Enter the following command:

    xm list

    The Domain-0 and UUID number for the virtual machine are displayed.

  3. Execute the following command:

    xm console <UUID_NUMBER_FOR_VIRTUAL_MACHINE>

    The console of the virtual machine is displayed.

  4. Log in as root user with default password "ovsroot".

  5. Enter the following command:

    system-config-network

  6. Edit the eth0 device by entering the following information:

    • IP addresses

    • Netmask

    • Gateway

    • DNS settings

  7. Click Save and Quit.

  8. Run the following command to reload the network configuration:

    service network reload

  9. Edit the /etc/hosts file to add the machine name and IP address.

2.2.2.7.3 Configuring SWAP on a Virtual Machine

When the virtual machine first restarts, it may not have the swap correctly configured. If needed, configure the swap correctly.

To configure SWAP on the virtual machine:

  1. Ensure that the last line of the file /etc/fstab reads as follows:

    LABEL=SWAP-VM swap swap defaults 0 0

  2. If the first entry in the file /etc/fstab reads as LABEL=SWAP-hda3, edit the file to read as:

    LABEL=SWAP-VM

  3. To make the swap device available, run the following command:

    swapon-a

  4. To confirm that swap is configured, run the following command:

    swapon-s

2.3 Installing and Configuring Exalytics Software on Virtual Machines

The process of installing Exalytics software on virtual machines is identical to installing the software on the Exalytics Machine, except that the virtual machines cannot access the entire Exalytics memory. Keep in mind this limitation when setting kernel parameters related to shared memory or creating huge pages.

You install Oracle Business Intelligence and Oracle TimesTen on one virtual machine, and Oracle Essbase (if deployed) on the second virtual machine.

This section consists of the following topics:

2.3.1 Prerequisites to Installing and Configuring Exalytics Software on Virtual Machines

The following prerequisites must be met before installing the Exalytics software on the virtual machines:

  • You have created a virtual machine. If deploying Oracle Essbase, you have created two virtual machines.

  • You have created a virtual disk on RAID5 and associated it with the virtual machines.

2.3.2 Installing and Configuring Exalytics Software on Virtual Machines

This section consists of the following topics:

2.3.2.1 Creating and Mounting a /u01 Partition

Before you can install the Exalytics software, you create and mount a /u01 partition on the virtual disk.

To create and mount a /u01 partition:

After logging in as the root user, configure the /u01 directory on the virtual disk using commands such as the following ones:

  1. To create a partition, enter the following commands:

    parted /dev/xvdb

    GNU Parted 1.8.1

    Using /dev/xvdb

    Welcome to GNU Parted! Type 'help' to view a list of commands.

    (parted) mklabel msdos

    (parted) mkpart primary 0 1600GB

    (parted) print

    Model: Xen Virtual Block Device (xvd)

    Disk /dev/xvdb: 1600GB

    Sector size (logical/physical): 512B/512B

    Partition Table: msdos

    Number Start End Size Type File system Flags

    1 32.3kB 1600GB 1600GB primary

    (parted) quit

  2. To create logical volume, enter the following commands:

    lvm pvcreate /dev/xvdb1

    Physical volume "/dev/xvdb1" successfully created

    lvm vgcreate VolGroup01 /dev/xvdb1

    Volume group "VolGroup01" successfully created

  3. To view free extents on VolGroup01, enter the following commands:

    vgs -o +vg_free_count,vg_extent_count

    VG #PV #LV #SN Attr VSize VFree Free Ext

    VolGroup01 1 0 0 wz--n- 1.46T 1.46T 381469 381469

    lvm lvcreate --extents 381469 --name LogVol00 VolGroup01

    Logical volume "LogVol00" created

  4. To create a file system, enter the following command:

    mkfs.ext3 /dev/VolGroup01/LogVol00

  5. Add the following line to the /etc/fstab file:

    /dev/VolGroup01/LogVol00 /u01 ext3 defaults 0 0

  6. To mount the directory, enter the following commands:

    mkdir /u01

    mount /u01

  7. Verify that the partition has been created by entering the following command:

    df –lh

2.3.2.2 Modifying Oracle User and Granting Permissions

The VM template contains "oracle" as the operating system user name. Associate the user name with the "oinstall" group and create a password for it.

To modify Oracle user and grant permissions:

Perform the following actions as root:

  1. Enter the following commands to associate the user name "oracle" with the group "oinstall" and to create a password:

    groupadd oinstall

    usermod –g oinstall oracle

    passwd oracle

  2. Enter the following commands to grant permissions to the /u01 drive where the software is to be installed:

    chown oracle:oinstall /u01

    chmod 775 /u01

  3. Start the VNC Server on the virtual machine.

2.3.2.3 Installing Oracle Business Intelligence and Oracle TimesTen

To install Oracle Business Intelligence and Oracle TimesTen:

  1. Connect to a virtual machine.

  2. Download the following required software installers from the Media Pack for Oracle Exalytics under Oracle Business Intelligence software for Linux X86-64 on Oracle Software Delivery Cloud at the following location:

    http://edelivery.oracle.com/

    • Oracle BI Enterprise Edition Release 11.1.1.6.0

    • Repository Creation Utility that corresponds with the appropriate Oracle BI EE Release

    • Oracle WebLogic Server Release 10.3.6

  3. Download Oracle TimesTen 11.2.2.3 from the Media Pack for Oracle Exalytics under Oracle Business Intelligence software for Linux X86-64 on Oracle Software Delivery Cloud at the following location:

    http://edelivery.oracle.com/

    For detailed instructions on how to install Oracle Business Intelligence and Oracle TimesTen, see Chapter 3, "Installing Software on the Exalytics Machine."

2.3.2.4 Installing Oracle Essbase

If deploying Oracle Essbase, install Oracle Essbase on a separate virtual machine.

To install Oracle Essbase:

  1. Connect to a virtual machine.

  2. Navigate to the Oracle Software Delivery Cloud web site at the following link:

    http://edelivery.oracle.com

  3. For Product Pack, select Oracle Enterprise Performance System.

  4. For Platform, select Linux x86-64.

  5. Click Go.

  6. Select and download Oracle Enterprise Performance Management System Media Pack 11.1.2.2.0 for Linux x86-64

  7. Download Oracle Essbase, which includes the following:

    • System Configurator

    • Foundation Services

    • Essbase and Static Content

  8. Unzip the zip files to the MW_HOME directory.

  9. Navigate to the folder where you unzipped the files (MW_HOME), and run the following command to install Oracle Essbase:

    ./installTool.sh

  10. Navigate to MW_HOME/EPMSystem11R1/common/config/11.1.2.0.

  11. Run the following command to create a domain:

    ./configtool.sh

For more information, see Oracle Enterprise Performance Management System Installation and Configuration Guide Release 11.1.2.2 at the following location:

http://docs.oracle.com/cd/E17236_01/epm.1112/epm_install_1112200.pdf