This chapter includes the following information:
A server pool is a group of one or more virtualization hosts with the same processor architecture that have access to the same virtual and physical networks, and storage resources. Server pools provide load balancing, high availability capabilities, and sharing of some resources for all members of the pool.
You can create server pools for the following types of virtualization servers:
Oracle VM Server for SPARC
Oracle VM Server for x86
Oracle Solaris Zones
A virtualization host can refer to Oracle VM Server for SPARC, Oracle Solaris Zones, or Oracle VM Server for x86 that are managed through Oracle Enterprise Manager Ops Center. A virtual host or guest in a server pool refers to a non-global zone, logical domain or virtual machine running on the virtualization host.
Server pools are resource pools of virtualization hosts that share compatible chip architecture, which facilitates actions such as moving guests between hosts. You can also apply resource configurations and policies to them. The policies that you establish for a server pool manage many of the CPU utilization and resource balancing functions. Operations to the server pool are delegated to the individual virtualization hosts in the server pool.
To manage the guests within a server pool, you can perform warm and live guest migration and you can balance all of the guests' load among the members of the server pool. You can configure a policy to balance the load automatically, based on a schedule that you determine, or you can balance the load manually. When a virtualization host system shuts down, such as a hardware failure, you can start the guests on another host in the same pool.
Virtualization hosts in a server pool share network and storage libraries and several server pools can share the same networks and storage resources.
Server pools form a key concept for virtual datacenter management. Pooling your virtualization supported systems plays an important role and requires proper understanding of your requirements in your environment.
All the requirements, procedures, and different functions available for a server pool depend on the type of virtualization technology selected. See the appropriate sections to create a server pool for different virtualization technology.
When you create a server pool, you must plan the storage and network resources such that they are accessible for all the members of the server pool.
You must set the policies to place the guests on the virtualization hosts and choose how you want to do the load balancing of the server pool. To understand more on this, refer to Server Pool Policies.
The following table lists the tasks and the role required to complete the task. Contact your administrator if you do not have the necessary role or privilege to complete a task. See User and Role Management in the Oracle Enterprise Manager Ops Center Administration Guide for information about the different roles and the permissions they grant.
When you create a server pool, you define guest placement, and automatic load balancing polices. When you create guests, you define the guest resource consumption, including physical and virtual CPUs. You can edit also the policies of a server pool.
The placement policy is set to determine the preferred virtualization host for new guests within the server pool, automatic or manual load balancing of the server pool, and balance the server pool during any server failure or maintenance. The placement policy is defined when a server pool is created. The following are the placement policy options:
Place the guest on the virtualization host with the lowest relative load, based on the lowest CPU resource utilization. The calculation is based on a combination of the average load for the time period defined by the user. The default value is the last 10 minutes. The default threshold value for CPU utilization is 75%.
Place the guest on the virtualization host with the lowest allocated CPU and memory, that is, the total static resource allocation across all guests on the host. The resource allocation is the sum of the number of vCPUs and virtual memory specified for each guest. This policy first verifies whether the resource allocation is possible and then ranks the server with available resources. The relative weight of CPU and memory resource is 1:1 while calculating the load on the host.
Place the guest on minimum number of virtualization host and thereby consume less power.
You can modify the placement policy of a server pool. Following is an example of how the placement policy is worked upon a server pool.
Example 17-1 Example of Least Allocated Virtualization Host
A server pool has two virtualization hosts with different CPU and memory allocations. Host A has 8 available CPUs and 16 GB of memory. Host B has 4 available CPUs and 8 GB of memory.
Host A has three guests:
Guest X has one vCPU and 1024 MB of memory.
Guest Y has two vCPUs and 2048 MB of memory.
Guest Z has one vCPU and 1024 MB of memory.
Host A's total static allocation is 4 vCPUs and 4 GB of memory.
Host B has one guest:
Guest W has 3 vCPU and 4096 MB of memory.
Host B's total static allocation is 3 vCPUs and4 GB of memory.
The allocation percentage for Host A is: CPU allocation is 4 vCPUs/8 physical CPUs, or 50%. Memory allocation is 4 GB/16 GB or 25%.
The allocation percentage for Host B is: CPU allocation is 3 vCPUs/4 physical CPUs or 75%. Memory allocation is 4 GB/8 GB or 50%
Therefore, Host A is the less or least allocated of the virtualization hosts.
The policy to minimize power consumption places the guests on minimum number of virtualization hosts and powers off the idle servers. The idle virtualization hosts are powered off or set to low-power mode on explicit approval.
Scheduling the automatic balancing is applicable only for Oracle VM Server for SPARC and Oracle Solaris Zones server pool.
Use the Automatic Load Balancing Policy to schedule load balancing within a server pool. You can schedule the automatic balancing to occur weekly, daily, or hourly on a specific day and time of the week. The default is to balance the load on the servers in the server pool every Saturday at midnight according to the defined placement policy.
Note:
The day and time are in the Enterprise Controller's time zone.When you do not want to balance the server pool's load automatically, schedule a reminder to balance the server pool's load manually.
Automatic load balancing policy does not consider the virtualization hosts that are placed in maintenance mode as a target in the server pool for migrating the guests. When you place a virtualization host in maintenance mode, then all the migratable guests in it are migrated to other servers in the server pool. This action is triggered automatically.
Resource Balancing Enhancements
You can define the resource usage threshold for the servers in the server pool. The servers are considered to be loaded heavily depending on the threshold set for the resources. Depending on the placement policy selected, the CPU or total CPU and memory resource usage is calculated.
The virtualization hosts are checked whether the utilization exceeds the threshold. If so, the servers with enough resources are identified and then the guests that can be migrated from the server are identified to balance the load.
When the virtualization hosts are well within the threshold and power minimization policy is selected, the automatic balancing policy attempts to free a virtualization host from its guest so that the administrator can power off the server.
The automatic recovery option is available for all types of supported virtualization technology. You can enable or disable the automatic recovery of zones, logical domains, and the virtual machines.
This feature provides the option to recover the guests that are attached to a failed server. The recovered guests are restarted on other servers in the server pool.
The automatic recovery for virtual machines is managed by Oracle VM Manager. For logical domains and zones, the automatic recovery is managed by Oracle Enterprise Manager Ops Center.
Oracle Enterprise Manager Ops Center monitors the managed assets by checking the connection between the agent deployed on the assets and the Proxy Controller. To detect the unreachability of the assets, it takes about 5 minutes approximately. The unreachability detection steps are as follows:
A heartbeat time-out of 3 minutes allows the Proxy Controller to detect that the connection is down.
A re-connection attempt is made for every 30 seconds for four times.
When the connection cannot be re-established, the asset is seen as unreachable from Oracle Enterprise Manager Ops Center.
An automatic recovery job is initiated when the global zone or Oracle VM Server for SPARC is:
Identified as unreachable by Oracle Enterprise Manager Ops Center, and its attached zones or logical domains are also seen as unreachable.
Placed in a server pool.
Has at least a zone or logical domain that is configured to recover automatically.
If an automatic recovery failed or cannot be performed because of non availability of free resources in the server Pool, Oracle Enterprise Manager Ops Center periodically checks for every minute if there are enough free resources and retries the automatic recovery.
You can also manually recover the zones and logical domains. See Recovering Zones for more information about manual recovery.
See the sections Oracle Solaris Zones Server Pool, Oracle VM Server for SPARC Server Pool, and Oracle VM Server for x86 Server Pool for more information about automatic recovery options.
Oracle Enterprise Manager Ops Center uses software libraries to store ISO images and guest metadata. Storage libraries are also used to provide storage disks for the guest operating systems and for guests' data.
The virtualization hosts in a server pool share the libraries associated with the server pool. The type of library you can associate with the server pool depends on type of virtualization hosts in the server pool and on whether all members of the server pool can get access to the storage resource.
See Chapter 6, "Storage Libraries" and Chapter 5, "Software Libraries" for more information about setting up these libraries.
Oracle Enterprise Manager Ops Center provides groups related networking components such as fabrics and networks as network domains. All the managed and declared networks in Oracle Enterprise Manager Ops Center are available contained in the default network domain. Associating a virtualization host or a server pool with the default network domain does not require fabric connection. All the networks in the default network domain are available to be attached to the server pool.
You can also create network domains. Associating a server pool with a user-defined network domain requires connection between the physical interface of each server in the pool and each fabric in the network domain. Only networks assigned to the new network domain are available to be attached to the server pool.
Plan your networks and network domains that you must attach to the server pool. See Chapter 7, "Networks" for more information about creating network domains and managed networks.
You can pool the Oracle VM Server for SPARC resources and manage your logical domains. Oracle VM Servers for SPARC must meet the following requirements to be added to a server pool:
Must have the same chip architecture as the other virtualization hosts in the server pool to support load balancing and guest migration. Oracle Enterprise Manager Ops Center and Oracle VM Server for SPARC support SPARC T-Series chip architectures.
Overview of the type of Oracle VM Servers present. You must group your servers that have the same CPU architecture and frequency. This enables migration of logical domains within the server pool. For example, you cannot migrate a logical domain of Oracle VM Server for SPARC 2.1 version to a server lower than this version. So, plan your servers that must be pooled.
Must be running at least Oracle VM Server for SPARC 1.2 version.
Must be running on at least the Solaris 10 10/09 OS and must meet specific patch and firmware requirements.
Plan the network that must be attached to the server pool. Either place those networks in the user-defined network domain or use the default network domain. You can connect to a network multiple times. For each network connection, a virtual switch is created. For each network connection, you must define the NIC and IP address in each of the Oracle VM Servers in the server pool. Select the virtual switches when you start a logical domain. The multiple network connection allows you to create IPMP or aggregate the links in the logical domain.
The storage libraries must be associated with the server pool to store logical domain metadata, ISO images, and for virtual disk storage of logical domains. You can associate file system and block storage libraries with the server pool. Filesystem storage includes the NAS storage libraries. Block storage includes the SAN, iSCSI, and Dynamic Storage libraries. Ensure that you have at least one NAS storage library assigned to the server pool. For using migration capabilities, the server pool must be on shared storage facility.
Logical domain in the virtualization host must be created using Oracle Enterprise Manager Ops Center.
Do not have logical domains in running, shutdown, or suspended state.
The server pool policies that are applicable for Oracle VM Server for SPARC are as follows:
Placement Policy
Set the CPU utilization threshold. Place the logical domains on the Oracle VM Server for SPARC that has the lowest relative load.
Set the CPU and memory allocation threshold. Place the logical domains on the Oracle VM Server for SPARC allocated with lowest CPU and memory resources.
Set the CPU utilization threshold. Place the logical domains on the minimum number of Oracle VM Servers to minimize the power consumption.
Automatic Balancing
Set the automatic load balancing. Logical domains are migrated automatically in the server pool whenever the thresholds are exceeded. Schedule the automatic load balancing to take place at definite time interval.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some Oracle VM Server host servers are freed up from the logical domains and powered off to minimize power consumption.
Set whether administrator approval is required for migrating the logical domains.
You can enable automatic recovery for a logical domain in the following scenarios:
During logical domain creation, you can enable automatic recovery option and set the value for priority of recovery. See Creating a Logical Domain Profile for more information about setting the priority of recovery while creating a logical domain profile.
For an existing logical domain, use the options Enable Automatic Recovery and Disable Automatic Recovery to set the automatic recovery. Also, you can edit the attributes of a logical domain to set the value for priority of recovery. See Automatic Recovery of Logical Domains for more information.
In an Oracle VM Server for SPARC server pool, the option to power-off the failed server and recover the logical domains is enabled by default. When a server fails, Oracle Enterprise Manager Ops Center tries to power-off the failed server, ensure that it is stopped and then initiates the automatic recovery of the logical domains. The logical domains that are configured for automatic recovery are recovered and restarted on other servers in the server pool.
When the option to power-off the failed server is enabled and Oracle Enterprise Manager Ops Center cannot power-off the failed server, that is the Service Processor is also unreachable, the automatic recovery of the logical domains does not take place. This is to avoid any potential data corruption during the recovery process of the logical domains.
When Oracle Enterprise Manager Ops Center detects that the failed server is unreachable, the logical domains that were running in the server disappear from the Navigation pane in the UI until they are recovered to other servers in the server pool.
The metadata of disappeared logical domains is still available in the storage libraries. The logical domains that are not configured for automatic recovery or the logical domains that cannot be recovered re-appear in the UI in the following scenario:
When the failed server is restarted, the logical domains are booted and attached to the server again.
When you have followed the manual recovery procedure and deleted the failed server from Oracle Enterprise Manager Ops Center, the logical domains re-appear in the list of Shutdown Guests in the Server Pool. You can start them on other server in the server pool. See Recovering Logical Domains for the logical domains recovery.
Table 17-2 provides a quick view of different conditions that exist for recovering a logical domain.
Table 17-2 Recovery of Logical Domains
| Conditions | Result | |
|---|---|---|
|
Logical domains |
|
The logical domain is restarted and placed in another server in the server pool. If there are no server available in the server pool, then you must manually recover the logical domains. See Recovering Logical Domains for more information. |
|
Logical domains |
|
The automatic recovery of the logical domain does not take place. This is to avoid any data corruption during the recovery procedure. You must manually recover the logical domains. |
|
Logical domains |
|
Oracle Enterprise Manager Ops Center proceeds to recover the logical domains configured to automatically recover without trying to power-off the failed server from its processor. This can lead to data corruption if the failed server is just isolated from the network point of view, and has still logical domains writing to disks. |
|
Logical domains |
|
Follow the procedure in Recovering Logical Domains to recover the logical domains. Only after this the logical domains appear in the Shutdown Guests list in the UI. You can restart them on the available server in the server pool. |
Oracle Enterprise Manager Ops Center launches a wizard that collects information about the servers, storage and network resources, and placement policies to create a server pool.
To Create a Server Pool for Oracle VM Server for SPARC
Select Server Pools in the Systems Group list on the Navigation pane.
Click Create Server Pool in the Actions pane.
The Create Server Pool Wizard is displayed.
In the Identify Server Pool step, enter the following information:
Name and description of the server pool.
Enter tags for categorizing your server pool.
Select Oracle VM Server for SPARC from the Virtualization Technology.
Click Next to select the members of the server pool.
Select one or more Oracle VM Servers to add to the server pool.
The list of Oracle VM Servers displayed have the following characteristics:
Not associated with a server pool
In a healthy state
Not placed in maintenance mode
Have no logical domains in the running, shutdown or suspended state
The list displays the details of CPU frequency, type, and architecture of Oracle VM Servers. Select Oracle VM Servers that are compatible for migration of logical domains.
Click Next to associate the network domain.
Select the network domain to associate with the server pool.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects to supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to select the networks and associate with the server pool.
For default network domain, all the networks that are declared and managed in Oracle Enterprise Manager Ops Center are listed. For an user-defined network domain, only the networks assigned to it are listed.
Select the networks that you want to associate with all the servers in the server pool. You can make multiple connections to a network. Enter the total number of connections. For each network connection, a virtual switch is created. Choose the networks that you want to use for migration of logical domains.
Click Next to configure the interfaces of the servers.
For each network connection, provide the connection details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, then no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs.
You can select System Allocated for the NIC and IP address to be automatically allocated by the system.
You can select Do Not Allocate IP to allocate no IP address for the network connection.
Click Next to associate the storage libraries.
The storage libraries that are reachable from the selected members of the pool are displayed. The storage libraries are required to store logical domain metadata, ISO images, and for virtual disks of logical domain.
You can select the following types of storage libraries:
Filesystem Storage Libraries: This includes the NAS storage libraries. Associate at least one NAS storage library with the server pool to store the logical domain metadata.
Block Storage Libraries: This includes the Static Block Storage and Dynamic Block Storage. Static block storage libraries comprises the exported LUNs of storage arrays that are not managed by Oracle Enterprise Manager Ops Center. Dynamic block storage libraries comprises the exported LUNs of storage array servers that are discovered and managed in Oracle Enterprise Manager Ops Center.
Select the storage libraries from the list that you want to associate with all the virtualization hosts in the server pool.
Click Next to select the server pool policies.
Select the following policies in the server pool to manage the under utilized and overutilized servers in the pool:
Placement Policy: This policy decides the preferred virtualization host in the server pool to place the logical domains.
Auto Balancing Policy: This policy performs load balancing of the server pool automatically at set intervals.
Select one of the following placement policies:
Lowest relative load: The recent lowest memory and CPU utilization for the Oracle VM Servers in the server pool are calculated. The server with the lowest relative load is considered to place a logical domain. Provide the threshold for CPU utilization above which the server is considered to be over-utilized and logical domains are migrated to server with lowest relative load.
Lowest allocated CPU and memory resources: The total number of virtual CPU and memory resources allocated for all the logical domains in an Oracle VM Server are calculated. The server with the lowest allocated resources is considered to place a logical domain in the pool. Provide the threshold values for CPU and memory allocation. When the allocation is exceeded, the server is considered to be over-allocated and logical domains are migrated to server with lowest allocated CPU and memory resources.
Minimize power consumption: The logical domains running in a server pool are placed on the minimum number of Oracle VM Servers and the unused Oracle VM Servers are powered off. The threshold value set for the server over-utilization ensures that the servers are not overloaded.
Select the automatic load balancing policies:
Select automatic load balancing and set the interval in which the server pool must be checked for resource balancing. You can set the approval to migrate the logical domains automatically. Or send notifications about the approval.
You can also select to manually balance the resources. See Balancing Resources for more information.
For Automatic Recovery, you can specify whether an attempt to power-off the failed server must be performed before initiating the automatic recovery of the logical domains.
When Oracle Enterprise Manager Ops Center has the capability to power-off the Service Processor of a failed server and you have selected the automatic recovery option, the failed server is powered off and then the logical domain recovery is started.
If Oracle Enterprise Manager Ops Center cannot power-off the failed server, the recovery of the attached logical domains does not take place. This is to avoid any data corruption during the recovery process.
Click Next to view the summary of the server pool details.
Review the information to create a server pool for Oracle VM Server for SPARC. Click Finish to create the server pool.
You can now pool Oracle Solaris Zones resources and manage your zones. Oracle Solaris Zones must meet the following requirements to be added to a server pool:
All the assets in the pool must have compatible architecture for supporting guest migration. For a global zone to be added to the server pool, it must have the same release or must be at least Oracle Solaris 10 10/08 version. This ensures that the zones can be migrated to a global zone in the server pool.
The global zones must be in healthy state.
The global zones must not be associated with a server pool.
The global zones must not be placed in maintenance mode.
The network deployment for zones server pool vary depending on the Oracle Solaris OS version.
For a stand-alone Oracle Solaris 10 OS, the network can be attached to in either shared IP or exclusive IP mode. Whereas, in a server pool, you can attach a network only in shared IP mode for Oracle Solaris 10 OS. You cannot make multiple connections to a network.
For Oracle Solaris 11 OS, the network is always attached to in exclusive IP mode. In a server pool also, the network for Oracle Solaris 11 OS is always deployed in exclusive IP mode. You can make multiple connections to a network.
When you want to have a server pool with mixture of Oracle Solaris 10 and 11 OS, then you cannot make multiple connections to a network. So, plan your server pools with compatible Oracle Solaris OS versions.
The server pool policies that are applicable for Oracle Solaris Zones are as follows:
Placement Policy
Place the zones on the global zone that has the lowest relative load. Set the CPU utilization threshold.
Place the zones on the global zone allocated with lowest CPU and memory resources. Set the CPU and memory allocation threshold.
Place zones on minimum number of global zones to minimize the power consumption. Set the CPU utilization threshold.
Automatic Balancing
You can select the automatic load balancing so that the zones can be migrated automatically in the server pool whenever the thresholds are exceeded. You can schedule the automatic load balancing to take place at definite time interval.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some servers are freed from the zones and powered off to minimize power consumption.
You can set whether the approval is required from the administrator for migrating the zones.
Automatic Recovery
You can set the automatic recovery for the zones in the following scenarios:
During zone creation, you can enable automatic recovery of the zones. You can set the value for priority of recovery. Zone with a highest priority is migrated first.
For existing zones, use the option Enable Automatic Recovery and Disable Automatic Recovery to set the automatic recovery. You can edit the zone configuration to modify the value for priority of recovery.
In an Oracle Solaris Zones server pool, you can specify that the automatic recovery of a failed server must start first with an attempt to power off the failed server, ensure that it is stopped and then initiate the automatic recovery of the zones.
Oracle Enterprise Manager Ops Center takes you through a series of steps to collect information for creating a zones server pool.
Select Server Pool in the Systems Group list on the Navigation pane.
Click Create Server Pool in the Actions pane.
The Create Server Pool Wizard is displayed.
In the Identify Server Pool step, enter the following information:
Name and description of the server pool.
Enter tags for categorizing your server pool.
Select Oracle Solaris Zones – SPARC or x86 from the Virtualization Technology.
You cannot have a zones server pool with a mixture of SPARC or x86 architecture.
Click Next to select the members of the server pool.
Select one or more global zones to add to the server pool.
The list of global zones displayed have the following characteristics:
The same architecture – SPARC or x86
Oracle Solaris 10 10/08 OS or higher version
Not associated with a server pool
In a healthy state
Not placed in maintenance mode
The list displays the details of global zones such as CPU architecture and OS version. Select global zones that are compatible for migration of zones within the pool.
Click Next to associate the network domain.
Select the network domain to associate with the server pool.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects to supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to select the networks and associate with the server pool.
For default network domain, all the networks that are declared and managed in Oracle Enterprise Manager Ops Center are listed. For a user-defined network domain, only the networks assigned to it are listed.
Select the networks that you want to associate with all the global zones in the server pool. Depending on the Oracle Solaris OS versions selected for the server pool, you can make multiple connections to a network. See IP Stack in Server Pool for more information about IP stack mode for networks in the zones server pool.
Enter the total number of connections and click Next to configure the interfaces of the servers.
For each network connection, provide the connection details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs.
When supported by the network, you can select System Allocated for the NIC and IP address to be automatically allocated by the system.
You can select Do Not Allocate IP to allocate no IP address for the network connection.
Click Next to associate the storage libraries.
The storage libraries that are reachable from the selected members of the pool are displayed. The storage libraries are required to store zone metadata, ISO images, and for virtual disks of zones.
You can select the following type of storage libraries:
Filesystem Storage Libraries: This includes the NAS storage libraries. Associate at least one NAS storage library with the server pool to store the zone metadata.
Block Storage Libraries: This includes the Static Block Storage and Dynamic Block Storage. Static block storage libraries comprises the exported LUNs of storage arrays that are not managed by Oracle Enterprise Manager Ops Center. Dynamic block storage libraries comprises the exported LUNs of storage array servers that are discovered and managed in Oracle Enterprise Manager Ops Center.
Select the storage libraries from the list that you want to associate with all the virtualization hosts in the server pool.
Note:
When you have unmanaged storage attached to the non-global zones, then upload the script to be placed in all the global zones in the server pool. See Script to Migrate a Zone With Dependencies for more information about migrating zone with unmanaged storage.Click Next to select the server pool policies.
You must select the following policies in the server pool to manage the under utilized and overutilized servers in the pool:
Placement Policy: This policy decides the preferred virtualization host in the server pool to place the zones.
Auto Balancing Policy: This policy performs load balancing of the server pool automatically at set intervals.
Select one of the following placement policies:
Lowest relative load: The recent lowest memory and CPU utilization for the global zones in the server pool are calculated. The server with the lowest relative load is considered to place the zone. Provide the threshold for CPU utilization above which the server is considered to be over-utilized and zones are migrated to server with lowest relative load.
Lowest allocated CPU and memory resources: The total number of virtual CPU and memory resources allocated for all the zones in an Oracle VM Server are calculated. The server with the lowest allocated resources is considered to place a zone in the pool. Provide the threshold values for CPU and memory allocation. When the allocation is exceeded, the server is considered to be over-allocated and zones are migrated to server with lowest allocated CPU and memory resources.
Minimize power consumption: The zones running in the server pool are placed on the minimum number of global zone and the unused servers are powered off. The threshold value set for the server over-utilization ensures that the servers are not overloaded. Otherwise, the powered off servers can be powered on to host the zones.
Select the automatic load balancing policies:
Select automatic load balancing and set the interval in which the server pool must be checked for resource balancing. You can set the approval to migrate the zones automatically. Or send notifications for the approval.
You can also select to manually balance the resources. See Balancing Resources for more information.
For Automatic Recovery, you can specify whether an attempt to power-off the failed server must be performed before initiating the automatic recovery of the zones.
When Oracle Enterprise Manager Ops Center has the capability to power-off the Service Processor of a failed server and you have selected the automatic recovery option, the failed server is powered off and then the zone recovery is started.
Click Next to view the summary of the server pool details.
Review the information to create a server pool for Oracle Solaris Zones. Click Finish to create the server pool.
You can group one or more Oracle VM Servers in an Oracle VM Manager and create server pools. When you create a server pool for Oracle VM Server for x86 systems, you must have the following information defined:
Cluster file system
Server pool master
Virtual IP address for the server pool master
Shared access to the server pool resources is a must for providing for high availability for the virtual machines running in the Oracle VM Servers of the server pool. This is achieved by cluster file system OCFS2 which allows multiple Oracle VM Servers to access the same disk at the same time. OCFS2 ensures that the Oracle VM Servers in a server pool can access and modify resources in the shared repositories in a controlled manner.
When you create a server pool, you must specify the server pool file system for the cluster heartbeat and other cluster information. The file system can be NFS shares or LUNs of iSCSI or SAN based storage servers. Oracle VM formats the server pool file system as OCFS2 file system.
In Oracle Enterprise Manager Ops Center, the cluster is always enabled by default. The cluster configuration is pushed out to all the Oracle VM Servers in the server pool and the cluster heartbeat starts when the server pool is created. You can set a separate network for this cluster heartbeat. See Manage Networks for more information about setting up the networks and their role for an Oracle VM Server.
An Oracle VM Server is internally elected as server pool master. You cannot set the role for an Oracle VM Server. When the elected Oracle VM Server fails, the role is set for another Oracle VM Server in the server pool. The virtual IP address provided while creating the server pool is assigned to the Oracle VM Server that has been elected as server pool master.
The server pool policies that are applicable for Oracle VM Server for x86 are as follows:
Placement Policy
Places the virtual machines on the Oracle VM Server that has the lowest relative load. Set the CPU utilization threshold.
Places the virtual machines on minimum number of Oracle VM servers to minimize the power consumption. Set the CPU utilization threshold.
Automatic Balancing
You can select the automatic load balancing so that the virtual machines can be migrated automatically in the server pool whenever the thresholds are exceeded.
When you enable automatic load balancing for Oracle VM Server for x86 server pool, the server pool is checked continuously for the selected placement policy. When the threshold exceeds, the virtual machines are migrated from one Oracle VM Server to another.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some Oracle VM Server host servers are freed from the logical domains and powered off to minimize power consumption.
When the servers are overloaded and there are no other servers in the pool to host the virtual machines, then the policy starts a powered-off server using its Wake-on-LAN capability and live migrate the virtual machines to take up the load. The Wake-on-LAN capability must be enabled on the BIOS of the Oracle VM Server.
Automatic Recovery
When an Oracle VM Server fails, the virtual machines are migrated to another Oracle VM Server in the server pool. The automatic recovery for a virtual machine is set in the following scenarios:
During virtual machine creation, select the option Enable High Availability. This ensures that the virtual machines are migrated when an Oracle VM Server fails.
For an existing virtual machine, use the option Enable or Disable Automatic Recovery to set the automatic recovery.
The following procedure describes the steps in the wizard that is launched to create an Oracle VM Server for x86 server pool.
To Create an Oracle VM Server for x86 Server Pool
Select Server Pool in the Systems Group list on the Navigation pane.
Click Create Server Pool in the Actions pane.
The Create Server Pool Wizard is displayed.
In the Identify Server Pool step, enter the following information:
Name and description of the server pool.
Enter tags for categorizing your server pool.
Select Oracle VM Server for x86 from the Virtualization Technology.
Click Next to define the server pool configuration.
Define the following configuration details:
Select the Oracle VM Manager in which the Oracle VM Servers are discovered and owned.
Select the network domain.
Select a network from the list of networks available in the network domain.
Enter the virtual IP address that is assigned to the server pool master. When the server pool master changes, the IP address is assigned to the new Oracle VM Server.
Select the server pool file system to store cluster hearbeat and other cluster information. The file system can be either NFS shares on a NFS file servers or LUNs of SAN and iSCSI based storages. The NFS file servers and the storage servers are known and reachable to the selected Oracle VM Manager.
Click Next to select the members of the pool.
Select one or more Oracle VM Servers to add to the server pool.
The list of Oracle VM Servers displayed have the following characteristics:
Owned by the Oracle VM Manager. Refer to Manage Ownership of Oracle VM Servers to own an Oracle VM Server.
Not associated with a server pool
In a healthy state
Not placed in maintenance mode
Have no virtual machines in the running, shutdown or suspended state
Select Oracle VM Servers that are compatible for migration of virtual machines. To migrate virtual machines within a server pool, the Oracle VM Servers systems must be identical in model.
Click Next to associate the network domain.
Select the network domain to associate with the server pool.
When you have selected default network domain, go to Step 7.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. You cannot bond the interfaces in Oracle VM Server for x86. Bonding can be done in only individual OS of the Oracle VM Server.
Click Next to select the networks and associate with the server pool.
For default network domain, all the networks that are declared and managed in Oracle Enterprise Manager Ops Center are listed. For a user-defined network domain, only the networks assigned to it are listed.
Select the networks that you want to associate with all the servers in the server pool. You cannot make multiple connections to a network. Limit the number of connections to 1.
Click Next to configure the interfaces of the servers.
For each network connection, specify the following details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs.
You can select System Allocated for the NIC and IP address to be automatically allocated by the system.
Click Next to associate the storage libraries.
The storage libraries that are reachable from the selected members of the pool are displayed. The storage libraries are required to store virtual machine metadata, ISO images, and for virtual disks of virtual machine.
You can select the following type of storage libraries:
Filesystem Storage Libraries: This includes the Oracle VM storage repositories. At least one Oracle VM storage repository must be associated with the server pool.
Block Storage Libraries: This includes the Static Block Storage and Dynamic Block Storage. Static block storage libraries comprises the exported LUNs of storage arrays that are not managed by Oracle Enterprise Manager Ops Center. Dynamic block storage libraries comprises the exported LUNs of storage array servers that are discovered and managed in Oracle Enterprise Manager Ops Center.
Select the storage libraries from the list that you want to associate with all the virtualization hosts in the server pool.
Click Next to select the server pool policies.
You must select the following policies in the server pool to manage the under utilized and overutilized servers in the pool:
Placement Policy: This policy decides the preferred virtualization host in the server pool to place the logical domains.
Auto Balancing Policy: This policy performs load balancing of the server pool automatically at set intervals.
Select one of the following placement policies:
Lowest relative load: The recent lowest memory and CPU utilization for the Oracle VM Servers in the server pool are calculated. The server with the lowest relative load is considered to place a virtual machine. Provide the threshold for CPU utilization above which the server is considered to be over-utilized and virtual machines are migrated to server with lowest relative load.
Minimize power consumption: The virtual machines running in a server pool are placed on the minimum number of Oracle VM Servers and the unused Oracle VM Servers are powered off. The threshold value set for the server over-utilization ensures that the servers are not overloaded. Otherwise, the powered off Oracle VM Servers can be powered on to host the virtual machines.
Select the automatic load balancing policies:
Select automatic load balancing and set the interval in which the server pool must be checked for resource balancing. You can set the approval to migrate the virtual machines automatically. Or send notifications about the approval.
You can also select to manually balance the resources. See Balancing Resources for more information.
Click Next to view the summary of the server pool details.
Review the information to create a server pool for Oracle VM Server for x86. Click Finish to create the server pool.
You must monitor the requirements of the server pool so that there are resources to host the guests and run them efficiently. Also, the server pools gives the infrastructure support for the virtual datacenters. You might need to add storage and network resources to run the guests. Oracle Enterprise Manager Ops Center provides the following management functions to manage the resources in the server pool:
You can modify the parameters set during the creation of a server pool. Click Edit Attributes in the Actions pane and you can edit the following parameters of a server pool:
Name
Description
Change the placement policy
Change the auto balancing policy
To add, remove or modify the tags added to the server pool, use the option Edit Tags in the Actions pane.
For Oracle Solaris Zones and Oracle VM Server for SPARC server pool, you can edit the following additional parameters:
Enable or disable automatic recovery option.
Upload scripts to manage the file systems on unmanaged storage. See Step 10 in Oracle Solaris Zones Server Pool for more information.
You can add more virtualization hosts to a server pool when you require more CPU and memory resources to take up the load in the server pool. Depending on the virtualization type, you can add global zones, and Oracle VM Servers of x86 and SPARC architecture.
The list of global zones that are displayed to be added to the server pool has the following characteristics:
Same release as the existing global zones in the pool or at least Oracle Solaris 10 10/08 OS or higher version
Not associated with a server pool
In a healthy state and not placed in maintenance mode
When the server pool has Oracle Solaris 11 OS and multiple network connections, then the list of available global zones is limited to only Oracle Solaris 11 OS. You cannot add Oracle Solaris 10 OS as you cannot make multiple network connections to it.
Load balancing in the server pool requires the zones to be migrated within the pool. Therefore, ensure compatibility for migration within the pool when you select the members.
The list of Oracle VM Servers displayed to be added to the server pool have the following characteristics:
Compatible architecture with the other members of the server pool
Not associated with a server pool
In a healthy state and not placed in maintenance mode
Have no logical domains in the running, shutdown or suspended states
The list of Oracle VM Servers displayed to be added to the server pool have the following characteristics:
Owned by the Oracle VM Manager
Not associated with a server pool
In a healthy state and not placed in maintenance mode
Have no virtual machines in the running, shutdown or suspended states
To Add Virtualization Hosts to the Server Pool
Select the server pool and click Add Global Zones or Add Oracle VM Servers in the Actions pane.
The Add Virtualization Host to Server Pool Wizard is displayed.
Select one or more virtualization hosts to be added to the server pool.
When the server pool is associated with the default network domain, you must proceed to configure the interfaces. Otherwise, you must associate the network domain with the selected assets.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. When you do not want to bond the interfaces, go to Step 6. For Oracle VM Server for x86, you cannot bond the interfaces and go to Step 6.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to define the network connection.
For each network connection, specify the following details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs.
You can select System Allocated for the NIC and IP address to be automatically allocated by the system.
Click Next to view the summary.
Review the information provided and click Finish to add the selected assets to the server pool.
All networks in a network domain that is associated with a server pool are available to the members of the server pool. To associate a user-defined network domain with a server pool, there must be a connection between the virtualization servers and the fabrics in the network domain. The fabric connection is not required for the default network domain.
The default network domain includes all the networks in Oracle Enterprise Manager Ops Center. When you associate a server pool with the default network domain, all the networks in Oracle Enterprise Manager Ops Center are available for attaching with the server pool.
For a user-defined network domain, you must define the physical interfaces of each virtualization server in the server pool to connect to each fabric in the network domain. You can also bond the interfaces of the virtualization servers.
Note:
You cannot the bond the interfaces of Oracle VM Server for x86 servers.To Associate the Network Domain to a Server Pool
Click Associate Network Domain in the Actions pane.
The Associate Network Domain Wizard is displayed.
Select the network domain from the list.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. When you do not want to bond the interfaces, go to Step 7. For Oracle VM Server for x86, you cannot bond the interfaces and go to Step 7.
For default network domain, you are taken to the Summary step.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects to supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to view the summary.
Review the information and click Finish to associate the network domain with the server pool.
When you associate with a user-defined network domain, only the assigned networks in the network domain are available for attaching with the server pool. Therefore, ensure that you assign the required networks to the domain to attach to the server pool.
After the network domain is associated with a server pool, you can attach the networks in the network domain. Depending on the type of virtualization, the multiple connection to a network is allowed.
For Oracle VM Server for SPARC, you can make multiple connections to a network. Refer to the Chapter 16, "Oracle VM Server for x86" for more information.
For Oracle Solaris Zones, you can make multiple connections to a network when the server pool contains only Oracle Solaris 11 OS. For Oracle Solaris 10 OS, you can make only one connection. Also, when the server pool has mixture of Oracle Solaris 10 and 11 OS, then also you can make only single connection.
To Attach Networks to Server Pool
Click Attach Networks in the Action pane.
The Attach Network Wizard is displayed. All the networks assigned to the network domains are listed. For default network domain, all the networks are listed. The table also displays existing number of connections to the server pool.
Select one or more networks to attach to the server pool.
Click Next to specify the number of connections.
Enter the total number of connections required as per the virtualization type:
For Oracle VM Server for SPARC, you can enter multiple connections. For each network connection, a virtual switch is created. You must physical interface for each virtual switch. You assign these virtual switches to the logical domains.
For Oracle VM Server for x86, you can enter only one connection per network. You cannot make multiple connections.
For Oracle Solaris Zones, the Oracle Solaris OS version plays an important role. For Oracle Solaris 10 OS, the network is always deployed in shared IP mode. You cannot make multiple connections. For Oracle Solaris 11 OS, the network is always deployed in exclusive IP mode. You can make multiple connections to the network. For each network connection, a VNIC is created when you boot the zone. For a server pool of mixture of Oracle Solaris 10 and 11 OS, the network connection is limited to single connection.
Note:
The number of connections do not limit the number of guests to be connected to be network.Click Next to configure the interfaces.
When one of the virtualization hosts are connected to the network, the corresponding rows for network configuration for that hosts are not displayed. If not, specify the information for each connection:
Specify the NIC and IP address for each connection.
You can assign the same NIC to different network connection when the VLAN IDS are different for the networks.
Use System Allocated for the system to take care of the NIC and IP address allocation.
Use Do Not Allocate IP which is available for zones and Oracle VM Server for SPARC. You can connect to the network without specifying the IP address.
Click Next to view the summary of the information.
Review the information provided and click Finish to attach the networks to the server pool.
Ensure that you have provided correct information. Otherwise, the network attach job fails.
Table 17-3 lists the types of storage libraries that you can associate with the server pool.
Table 17-3 Supported Libraries
| Oracle VM Storage Repositories | NAS Libraries | SAN Fibre Channel Libraries | SAN iSCSI Libraries | |
|---|---|---|---|---|
|
No |
Yes |
Yes |
Yes |
|
|
Yes |
Yes |
Yes |
Yes |
|
|
No |
Yes |
Yes |
Yes |
When you associate libraries with the server pool, only the libraries that are reachable from the virtualization hosts in the server pool are listed. Depending on the type of library, you either provide virtual disks or LUNs to the guests' storage.
To Associate a Storage Library
Click Associate Libraries from the Actions pane.
The Associate Library window is displayed.
The storage libraries that are accessible from the virtualization hosts in the server pool are listed. Select the libraries that you want to associate with the server pool. You can select one or more storage libraries from the list.
Click Associate Libraries to associate the selected libraries with the server pool.
The guests refer to the logical domains, zones or the virtual machines that can be created in the virtualization hosts. According to the virtualization type of the server pool, you have the following options:
These options trigger the deployment plans for the corresponding guests. Refer to the following chapters to refer to the profile and plan creation for the guests:
When you place a virtualization host in a server pool, the option to migrate one or more guests is enabled. The following options become available for the virtualization hosts:
Oracle Solaris Zones – Migrating Zones
Oracle VM Server for SPARC - Migrating Logical Domains
Oracle VM Server for x86 – Migrating Virtual Machines
You can migrate multiple zones from a global zone which is in a server pool. Migrate the zones to an individual global zone or other zones server pool. When you migrate zones, the source and the target global zone must be compatible. The target global zones must have the following characteristics:
Running at least Oracle Solaris 10 8/07 OS.
Can access all the storage libraries associated with the zone.
Apart from compatibility, the target and the source global zone might have differences in the patches and packages installed on them. Choose to update the zone to match the patches and packages of the target global zone.
Note:
You cannot downgrade the patches and packages of the zone. The migration fails in such scenario.You can also force the migration of the zone without updating the patches and packages. Select the update options while migrating the zone.
Select the global zone from which you want to migrate the zones.
Click Migrate Zones in the Actions pane.
The Migrate Zones Wizard is displayed.
The list includes the zones running in the global zone. Select one or more zones from the list.
Click Next.
Select an individual global zone or server pool to which you can migrate the zones.
The table displays the list of eligible global zones and server pool to which you can migrate the zones. The target global zone in the server pool depends on the server pool placement policy.
Select an update option to continue with migration.
The source and the target global zones might not be in the same patch level. Either you can select to update the patches and packages of zone to match the target global zone or continue migration without updating the zone.
Review the details and click Finish to migrate the zones.
Migrate multiple logical domains from an Oracle VM Server to another Oracle VM Server in the same server pool. You cannot do cross server pool migration.
Only from Oracle VM Server for SPARC 2.1 version, live migration of logical domains is supported. For earlier releases, it is warm migration. The logical domains are shut down and then restarted on the target server.
Select the Oracle VM Server in a server pool.
Click Migrate Logical Domains in the Actions pane.
The list of logical domains running in the Oracle VM Server are listed. Select one or more logical domains from the list.
Select an Oracle VM Server from the list.
The table lists the Oracle VM Servers that have enough resources to host the logical domains in the same server pool. The Oracle VM Server are listed in the decreasing order of preference.
Review the information and click Finish to migrate the logical domains.
For Oracle VM Server for x86 systems, you can migrate virtual machines only within a server pool. Live migration of virtual machines is supported. The eligible Oracle VM Servers have the following characteristics:
Owned by the Oracle VM Manager.
Placed in the same server pool as the source Oracle VM Server.
Have required resources to host the virtual machines.
Identical in machine make and model of the source Oracle VM Server.
Select the Oracle VM Server in the server pool.
Click Migrate Virtual Machines in the Actions pane.
Select the virtual machines that you want to migrate.
Select the Oracle VM Server to which you want to migrate the virtual machines.
The table lists the Oracle VM Servers that are eligible to host the virtual machines.
Review the summary and click Finish to migrate the virtual machines.
When you have selected to balance server pool resource manually, use the Balance Resources action to check for the under-utilization or over-utilization of resources. Select a server pool and click Balance Resources in the Actions pane to display the current utilization of the virtualization hosts in the server pool. When balancing is required according to the placement policy, Oracle Enterprise Manager Ops Center displays a list of target server that can accept migrated guests. Click Balance Resources to start the migration job.
Use the option Delete Server Pool to release the virtualization hosts back to stand-alone state. The type of server pool imposes some limitations to delete a server pool.
When you want to delete an Oracle VM Server for SPARC server pool, shut down the logical domains running in the server pool. Only when the logical domains are detached, you can delete the server pool.
For a zones server pool, you can delete the server pool and the zones continue to be attached to the global zone. Oracle Enterprise Manager Ops Center does not provide any restrictions to shut down the zones before deleting a zones server pool.
Shut down all the virtual machines and remove all Oracle VM Servers except for the Master Oracle VM Server before deleting the server pool. The virtual machine metadata and virtual disks are stored in the storage library. The virtual machine details are not lost and you can start the virtual machine.
For instructions in performing actions or to learn more about the role of this feature, go to one of the following resources:
Oracle Enterprise Manager Ops Center Administration Guide
For end-to-end examples, see the How To library at http://docs.oracle.com/cd/E27363_01/nav/howto.htm.