This chapter includes the following information:
A server pool is a group of one or more virtualization hosts with the same processor architecture that have access to the same virtual and physical networks, and storage resources. Server pools provide load balancing, high availability capabilities, and sharing of some resources for all members of the pool.
You can create server pools for the following types of virtualization servers:
Oracle VM Server for SPARC
Oracle VM Server for x86
Oracle Solaris Zones x86
Oracle Solaris Zones SPARC
A virtualization host can refer to Oracle VM Server for SPARC, Oracle Solaris Zones, or Oracle VM Server for x86 that are managed through Oracle Enterprise Manager Ops Center. A virtual host or guest in a server pool refers to a non-global zone, logical domain or virtual machine running on the virtualization host.
Server pools are resource pools of homogeneous virtualization hosts that allow actions such as balancing load between servers and moving guests between hosts. You can also apply resource configurations and policies to them. The policies that you establish for a server pool manage many of the CPU utilization and resource balancing functions. Operations to the server pool are delegated to the individual virtualization hosts in the server pool.
To manage the guests within a server pool, you can perform warm and live guest migration and you can balance all of the guests' load among the members of the server pool. You can configure a policy to balance the load automatically, based on a schedule that you determine, or you can balance the load manually. When a virtualization host system shuts down, such as a hardware failure, you can start the guests on another host in the same pool.
Virtualization hosts in a server pool share network and storage libraries and several server pools can share the same networks and storage resources.
Server pools form a key concept for virtual datacenter management. Pooling your virtualization supported systems plays an important role and requires proper understanding of your requirements in your environment.
All the requirements, procedures, and different functions available for a server pool depend on the type of virtualization technology selected. See the appropriate sections to create a server pool for different virtualization technology.
When you create a server pool, you must plan the storage and network resources such that they are accessible for all the members of the server pool.
You must set the policies to place the guests on the virtualization hosts and choose how you want to do the load balancing of the server pool. To understand more on this, refer to Server Pool Policies.
The following table lists the tasks and the role required to complete the task. Contact your administrator if you do not have the necessary role or privilege to complete a task. See User and Role Management in the Oracle Enterprise Manager Ops Center Administration Guide for information about the different roles and the permissions they grant.
Using Oracle Enterprise Manager Ops Center, you can perform the following tasks:
Create server pools for different types of virtualization technologies.
Balance load on the resources in the server pool.
Manage storage and network resources for the server pool.
Set placement policies for automatic load balancing.
Perform automatic recovery of the guests during server failure.
Edit the attributes of an existing server pool.
Table 21-2 lists where to find different information for server pools in the UI.
Table 21-2 Location of Server Pool Information in the UI
| To See | Location |
|---|---|
|
Server pools |
Expand Assets in the Navigation pane and select Server Pools in the Resource Management Views. |
|
Server pool resources |
Expand Assets in the Navigation pane and select Server Pools in the Resource Management Views. Select the server pool and the center pane displays all the details of the server pool resources such as network, libraries, and fabrics. |
|
Server pool actions |
Expand Assets in the Navigation pane and select Server Pools in the Resource Management Views. Select the server pool and the Actions pane list the various options such as Associate Libraries, Attach Networks, Create Guests, Associate Network Domains and Edit Attributes for managing the server pool. |
You can create server pools for different types of virtualization technology. See the appropriate sections to create a server pool for different virtualization technology.
Each time Oracle Enterprise Manager Ops Center adds a new system to the pool, it checks the network connections on all the members of the pool. You will not be allowed to continue if something is wrong.
The following are some server pool capabilities and limitations:
You can create a zones server pool and use the logical domains as the global zones. However, those logical domains should not be part of an Oracle VM Server server pool. The user interface does not prevent you from creating these server pools, but the configuration is not supported and it can cause problems; particularly when automatic recovery is enabled on both server pools.
Before performing maintenance in your datacenter, it is a good idea to place the systems affected in maintenance mode to suppress the creation of alerts and incidents. When the system is part of a server pool, you can decide whether you want to migrate the guests to a different host in the pool before placing the system in maintenance mode. Maintenance mode does not stop monitoring, it only stops the alerts and incidents.
Note:
If you do not relocate the guests before putting the system in maintenance mode, consider disabling the server pool monitoring and disabling automatic recovery for each guest. Beginning with 12.2.2, you can disable automatic recovery at the server pool level instead of disabling it for each guest. See Editing Server Pool Parameters for how to disable automatic recovery for the server pool.The following information is covered in this section:
When you create a server pool, you define guest placement, and automatic load balancing polices. When you create guests, you define the guest resource consumption, including physical and virtual CPUs.
You can edit the server pool policies in the server pool's Summary tab. See Editing Server Pool Parameters for how to edit the pool attributes.
The placement policy determines the preferred virtualization host for new guests within the server pool, the type of load balancing (automatic or manual) of the server pool, and balances the server pool during any server failure or maintenance. You define the placement policy when you create the server pool; however, you can modify the policy at any time.
The following are the server pool placement policy options:
Place the guest on the virtualization host with the lowest relative load, based on the lowest CPU resource utilization. The calculation is based on a combination of the average load for the time period defined by the user. The default value is the last 10 minutes. The default threshold value for CPU utilization is 75%.
Place the guest on the virtualization host with the lowest allocated CPU and memory, that is, the total static resource allocation across all guests on the host. The resource allocation is the sum of the number of vCPUs and virtual memory specified for each guest. This policy first verifies whether the resource allocation is possible and then ranks the server with available resources. The relative weight of CPU and memory resource is 1:1 while calculating the load on the host.
Place the guest on minimum number of virtualization host and thereby consume less power.
The following is an example of how the placement policy works in a server pool.
Example 21-1 Example of Least Allocated Virtualization Host
A server pool has two virtualization hosts with different CPU and memory allocations. Host A has 8 available CPUs and 16 GB of memory. Host B has 4 available CPUs and 8 GB of memory.
Host A has three guests:
Guest X has one vCPU and 1024 MB of memory.
Guest Y has two vCPUs and 2048 MB of memory.
Guest Z has one vCPU and 1024 MB of memory.
Host A's total static allocation is 4 vCPUs and 4 GB of memory.
Host B has one guest:
Guest W has 3 vCPU and 4096 MB of memory.
Host B's total static allocation is 3 vCPUs and4 GB of memory.
The allocation percentage for Host A is: CPU allocation is 4 vCPUs/8 physical CPUs, or 50%. Memory allocation is 4 GB/16 GB or 25%.
The allocation percentage for Host B is: CPU allocation is 3 vCPUs/4 physical CPUs or 75%. Memory allocation is 4 GB/8 GB or 50%
Therefore, Host A is the least allocated of the virtualization hosts.
The policy to minimize power consumption places the guests on the minimum number of virtualization hosts and powers off the idle servers. The idle virtualization hosts are powered off or set to low-power mode on explicit approval.
For Oracle VM Server for x86, Oracle Enterprise Manager Ops Center will power off unused servers in the pool.
Beginning with Oracle Enterprise Manager Ops Center 12.2.2, you can enable power management for Oracle VM Server for SPARC server pools. When you enable power management, Oracle Enterprise Manager Ops Center will load as much work as it can onto a physical server before adding a virtual machine to a different server in the pool.
When power management is enabled, the software performs the following tasks:
Places unassigned resources into a lower power state
Identifies the members with the highest load and determines if those members have enough unallocated resources to support a new virtual machine placement request
Checks the number of logical domains for each server in the pool and sets the appropriate placement policy
Adjusts the placement policy every time you create, migrate, or delete a logical domain.
You can enable to policy and time interval to check for resource changes in Administration. The default check.delay time interval checks for resource changes every 5 minutes (300 seconds). See the Oracle Enterprise Manager Ops Center Administration Guide for how to set the power consumption policy and edit the property values.
Scheduling automatic load balancing is applicable only for Oracle VM Server for SPARC and Oracle Solaris Zones server pool.
Use the Automatic Load Balancing Policy to schedule load balancing within a server pool. You can schedule the automatic balancing to occur weekly, daily, or hourly on a specific day and time of the week. The default is to balance the load on the servers in the server pool every Saturday at midnight according to the defined placement policy.
Note:
The day and time are in the Enterprise Controller's time zone.When you do not want to balance the server pool's load automatically, schedule a reminder to balance the server pool's load manually.
Automatic load balancing policy does not consider the virtualization hosts that are placed in maintenance mode as a target in the server pool for migrating the guests. When you place a virtualization host in maintenance mode, all of the migratable guests in it are migrated to other servers in the server pool. This action is triggered automatically.
You can define the resource usage threshold for the servers in the server pool. The servers are considered to be loaded heavily depending on the threshold set for the resources. Depending on the placement policy selected, the CPU or total CPU and memory resource usage is calculated.
When the virtualization host utilization exceeds the threshold, the software identifies the servers with sufficient resources and the guests that can be migrated from the server to balance the load.
When the virtualization hosts are well within the threshold and power minimization policy is selected, the automatic balancing policy attempts to free a virtualization host from its guest so that the administrator can power off the server.
The automatic recovery feature provides the option to recover the guests that are attached to a failed server. The recovered guests are restarted on other servers in the server pool.
This feature is available for all types of supported virtualization technology. You can enable or disable the automatic recovery of zones, logical domains, and the virtual machines. The automatic recovery for virtual machines is managed by Oracle VM Manager. For logical domains and zones, the automatic recovery is managed by Oracle Enterprise Manager Ops Center.
Oracle Enterprise Manager Ops Center follows the recovery process described in this section, and is applicable for zones and logical domains.
By default, automatic recovery is enabled at the server pool level. When enabled at the server pool level, the recovery options selected for zones and logical domains determines whether automatic recovery is attempted.
Oracle Enterprise Manager Ops Center monitors the managed assets by checking the connection between the agent deployed on the assets and the Proxy Controller. You can set the interval of periodic check of the assets reachability while creating the server pool. The default value is 180 seconds. If any of the Oracle Enterprise Manager Ops Center Agent Controllers installed on the server do not respond within the fixed timeout interval, then the server is considered to be a failed server. The timeout interval is a minimum of 20 seconds and a maximum of 60 seconds.
For example, if the interval is set at 180 seconds, the server pool checks the status of the members of the pool every 180 seconds and if there is no response from the Agent Controller in 60 seconds, then the automatic recovery of the zones or the logical domains in the server are started almost after four (4) minutes after the server failure.
If the interval is set at 20 seconds, the server pool checks the status of the members of the pool every 20 seconds and if there is no response from the Agent Controller in 20 seconds, then the automatic recovery of the zones or the logical domains in the server are started almost after 40 seconds after the server failure.
An automatic recovery job is initiated when the global zone or Oracle VM Server for SPARC is:
Placed in a server pool.
Has at least a zone or logical domain that is configured to recover automatically.
The job initiated first performs the preliminary check of pinging all the known IP address of the failed server from another member in the server pool and the heartbeat in the metadata that is refreshed by the servers. The recovery job continues if the asset is seen as unreachable from Oracle Enterprise Manager Ops Center.
The recovery job fails if the assets are reachable and only the Agent Controller is down on the server. The attempt to recover the logical domains or zones are stopped as they are reachable.
If an automatic recovery failed or cannot be performed because of non availability of free resources in the server pool, Oracle Enterprise Manager Ops Center periodically checks if there are enough free resources and retries the automatic recovery.
The software check every 60 seconds until it can perform the automatic recovery. Beginning with Oracle Enterprise Manager Ops Center 12.2.2.0.0, a retry counter is available to control how many times the auto-recovery manager will attempt to perform an automatic recovery. Setting the counter to zero (0) results in an unlimited number of times that the automatic recovery is attempted.
Beginning with Oracle Enterprise Manager Ops Center 12.2.2.0.0, you can disable the automatic recovery feature for a server pool or for an individual asset. To disable the automatic recovery feature for a server pool, and all assets in the server pool, select the check box when you create the server pool. You can edit an existing pool to disable or enable the feature. When you disable automatic recover at the server pool level, the software will not check for resources or attempt to recover a guest, regardless of what option is selected at the zone or guest level.
You can enable to policy and time interval to check for resource changes in Administration. The default check.delay time interval checks for resource changes every 5 minutes (300 seconds). See the Administration guide for how to set the power consumption policy and edit the property values.
You can also manually recover the zones and logical domains. See Recovering Zones for more information about manual recovery.
See the sections Oracle Solaris Zones Server Pool, Oracle VM Server for SPARC Server Pool, and Oracle VM Server for x86 Server Pool for more information about automatic recovery options.
Oracle Enterprise Manager Ops Center uses software libraries to store ISO images and guest metadata. Storage libraries are also used to provide storage disks for the guest operating systems and for guests' data.
The virtualization hosts in a server pool share the libraries associated with the server pool. The type of library you can associate with the server pool depends on the type of virtualization hosts in the server pool and on whether all members of the server pool can access the storage resource.
See Chapter 16, "Storage Libraries for Virtualization" and Chapter 5, "Software Libraries" for more information about setting up these libraries.
Oracle Enterprise Manager Ops Center provides group related networking components, such as fabrics, and networks, such as network domains. By default, all managed and declared networks in Oracle Enterprise Manager Ops Center are placed in the default network domain. Associating a virtualization host or a server pool with the default network domain does not require a fabric connection. All of the networks in the default network domain are available to be attached to the server pool.
You can create network domains. Associating a server pool with a user-defined network domain requires connection between the physical interface of each server in the pool and each fabric in the network domain. Only networks assigned to the user-defined network domain are available to be attached to the server pool.
Note:
Before attaching one or more private ethernet networks to a server pool, verify that all members of the pool have access to the private networks. When all members do not have access, you might encounter inconsistent guest creation and network information.You can use server pools with Oracle SuperCluster systems:
When you attach public networks to a server pool, the server pool members can belong to different Oracle SuperCluster systems.
When you attach private (internal) networks to an Oracle SuperCluster server pool, the members must all belong to the same Oracle SuperCluster system.
Note:
Do not create server pools using members from more than one Oracle SuperCluster systems and attach private, or internal, networks to the systems.Before creating a server pool, plan your networks and network domains that you must attach to the server pool. See Chapter 17, "Networks for Virtualization" for more information about creating network domains and managed networks.
The following information is covered in this section:
You can pool the Oracle VM Server for SPARC resources and manage your logical domains. The Oracle VM Server for SPARC can be configured and running with I/O domains and root domains.
Oracle VM Servers for SPARC must meet the following requirements to be added to a server pool:
You are able to place Oracle VM Servers of different CPU types in the server pool; however, you might lose the ability to migrate the logical domains between virtualization hosts with different CPU architectures. When you want to live migrate the logical domains, ensure that you create server pools of compatible CPU types. Cold migration and automatic guest recovery are available by using the shutdown/detach operation of the logical domain. Plan your servers that must be pooled.
Must be running a supported version of Oracle VM Server for SPARC. The minimum supported version for server pools is Oracle VM Server for SPARC version 1.2.
Must be running a supported version of Oracle Solaris operating system. The minimum supported version for server pools is Oracle Solaris 10 10/09 OS and must meet specific patch and firmware requirements.
Plan the network that must be attached to the server pool. Either place those networks in the user-defined network domain or use the default network domain. You can connect to a network multiple times. The multiple network connection allows you to create IPMP or aggregate the links in the logical domain.
The storage libraries must be associated with the server pool to store logical domain metadata, ISO images, and for virtual disk storage of logical domains. You can associate file system and block storage libraries with the server pool. Filesystem storage includes the NAS storage libraries. Block storage includes the SAN, iSCSI, and Dynamic Storage libraries. Ensure that you have at least one NAS storage library assigned to the server pool. For using migration capabilities, the server pool must be on shared storage facility.
The following enhancements are available for supporting Oracle VM Server environments created outside Oracle Enterprise Manager Ops Center and advanced I/O domain configurations. Some of the enhancements are available from 3.0 or higher version:
You can add Oracle VM Servers to the server pool even if they have logical domains configured and running.
You can select the servers in the server pool to create I/O domains and root domains. The server pool will not be available as a target.
Discover and manage user-configured Oracle VM Server for SPARC environments. The metadata of logical domains are stored in the local library of the Control Domain. To enable logical domain migration, you must the move the metadata to a NFS storage.
Oracle VM Servers of different CPU type can be pooled together. The migration is not possible for logical domains between incompatible servers. Still, you can perform cold migration, that is shutting down the domain and starting on other servers in the server pool. Also, the guest recovery can be performed as it is often possible to start the domain on another server even though live migration is not possible.
You can configure exclusive access of I/O domain and root domain resources to other logical domains. The exclusive access prevents creation of zones in the I/O domain or root domain OS and the global zone is not available to placed in a zones server pool.
You can delete a server pool even if the logical domains are attached to the Oracle VM Servers.
Placing an Oracle VM Server in the server pool does not remove any existing network and storage resource that are already attached to it.
Deleting a server pool also does not unconfigure all the Oracle VM Servers of the attached network and storage resources. All the attached resources are retained and only the server pool is deleted.
The server pool policies that are applicable for Oracle VM Server for SPARC are as follows:
Placement Policy
Set the CPU utilization threshold. Place the logical domains on the Oracle VM Server for SPARC that has the lowest relative load.
Set the CPU and memory allocation threshold. Place the logical domains on the Oracle VM Server for SPARC allocated with lowest CPU and memory resources.
Set the CPU utilization threshold. Place the logical domains on the minimum number of Oracle VM Servers to minimize the power consumption.
Automatic Balancing
Set the automatic load balancing. Logical domains are migrated automatically in the server pool whenever the thresholds are exceeded. Schedule the automatic load balancing to take place at definite time interval.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some Oracle VM Server host servers are freed up from the logical domains and powered off to minimize power consumption.
Set whether administrator approval is required for migrating the logical domains.
You can enable automatic recovery for a logical domain in the following scenarios:
During logical domain creation, you can enable automatic recovery option and set the value for priority of recovery. See Creating a Guest Domain Profile for more information about setting the priority of recovery while creating a logical domain profile.
For an existing logical domain, use the options Enable Automatic Recovery and Disable Automatic Recovery to set the automatic recovery. Also, you can edit the attributes of a logical domain to set the value for priority of recovery. See Automatic Recovery of Logical Domains for more information.
See Automatic Recovery for the recovery process of the logical domains when a server fails in the server pool.
In an Oracle VM Server for SPARC server pool, the option to power-off the failed server and recover the logical domains is enabled by default. When a server fails, Oracle Enterprise Manager Ops Center tries to power-off the failed server, ensure that it is stopped and then initiates the automatic recovery of the logical domains. The logical domains that are configured for automatic recovery are recovered and restarted on other servers in the server pool.
When the option to power-off the failed server is enabled and Oracle Enterprise Manager Ops Center cannot power-off the failed server, that is the Service Processor is also unreachable, the automatic recovery of the logical domains does not take place. This is to avoid any potential data corruption during the recovery process of the logical domains.
When Oracle Enterprise Manager Ops Center detects that the failed server is unreachable, the logical domains that were running in the server disappear from the Navigation pane in the UI until they are recovered to other servers in the server pool.
The metadata of disappeared logical domains is still available in the storage libraries. The logical domains that are not configured for automatic recovery or the logical domains that cannot be recovered re-appear in the UI in the following scenario:
When the failed server is restarted, the logical domains are booted and attached to the server again.
When you have followed the manual recovery procedure and deleted the failed server from Oracle Enterprise Manager Ops Center, the logical domains re-appear in the list of Shutdown Guests in the Server Pool. You can start them on other server in the server pool. See Recovering Logical Domains for the logical domains recovery.
Table 21-3 provides a quick view of different conditions that exist for recovering a logical domain.
Table 21-3 Recovery of Logical Domains
| Conditions | Result | |
|---|---|---|
|
Logical domains |
|
The logical domain is restarted and placed in another server in the server pool. If there are no server available in the server pool, then you must manually recover the logical domains. See Recovering Logical Domains for more information. |
|
Logical domains |
|
The automatic recovery of the logical domain does not take place. This is to avoid any data corruption during the recovery procedure. You must manually recover the logical domains. |
|
Logical domains |
|
Oracle Enterprise Manager Ops Center proceeds to recover the logical domains configured to automatically recover without trying to power-off the failed server from its processor. This can lead to data corruption if the failed server is just isolated from the network point of view, and has still logical domains writing to disks. |
|
Logical domains |
|
Follow the procedure in Recovering Logical Domains to recover the logical domains. Only after this the logical domains appear in the Shutdown Guests list in the UI. You can restart them on the available server in the server pool. |
There are certain conditions in which the network tagging mode must be selected while creating a server pool. Review the following conditions and scenarios before deciding on the network tagging mode when you create an Oracle VM Server for SPARC server pool:
You can select networks without VLAN ID. The UI does not provide the option to select Tagged or Untagged mode.
You can select to associate and configure the networks with VLAN ID in Tagged mode.
You can select to associate and configure the networks with VLAN ID in Untagged mode.
You can select to configure the networks in mixed tagging mode in the server pool. For example, you can attach the network N1 with VLAN ID = 100 in tagged mode with the server S1 and in untagged mode for server S2. Refer to Mixed Network Tagging Mode Configurations in Server Pool for more detailed information.
You can attach networks whose VLAN ID is similar to another network already connected to the servers. For example, a server S1 is already connected to network N1 with VLAN ID = 100, then while creating the server pool with S1 as the member of the pool, you can also attach a network N2 with VLAN ID =100.
You can edit the VLAN ID of a network when you are attaching the network in Tagged mode for the first time.
When you can edit the VLAN ID of the network, you cannot enter -1 as the value for the VLAN ID.
If the selected network with a VLAN ID is already connected to the selected assets in Tagged mode, then you cannot edit the VLAN ID and make another connection.
You cannot make multiple network connections to a member of the server pool over the same network in both tagged and untagged modes. The mode can be either in tagged or untagged mode only. For example, if you attach network N1 with VLAN ID =100 for the first time to server S1 in Tagged mode, then you cannot make another connection to the same network N1 in Untagged mode. Every other connection with network N1 must always be in Tagged mode for server S1.
If the selected members of the pool are already connected to network N1 with VLAN ID =100, then you cannot select the same network with different VLAN ID to be connected for the server pool.
In an Oracle VM Server for SPARC server pool, you must group the servers that are homogenous in the network VLAN tagging mode to avoid any network outage. When you create a server pool, you can select the network tagging mode for the networks configured with VLAN ID to be attached to the Oracle VM Servers. You must group the servers in a way that all the Oracle VM Servers in the pool are either untagged or tagged mode for each network connection.
If the server pool is mixed with servers attached to networks in untagged and tagged mode, then there is a possibility that you will lose the network configuration of the logical domain OS.
When a logical domain is using an untagged VLAN, you cannot migrate the guest to a server that uses a tagged VLAN.
For example, in the following scenario, there is a server pool with two Oracle VM Servers CD1 and CD2. CD1 has a logical domain LD1. A network 192.0.2.0/24 with VID=100 is attached to CD1 in tagged mode. When you attach the network on the CD1, a VNIC is created with VID=100 and configured with an IP address to reach the network 192.0.2.0/24. The same network is attached to CD2 in untagged mode. LD1 is also connected to the network 192.0.2.0/24. When you migrate the LD1 from CD1 to CD2 or when you start the shutdown-detached LD1 on CD2, the LD1 will successfully start on the CD2. However, the OS of the LD1 is not able to reach the network 192.0.2.0.24 with VID=100 because, the VNIC is created with VID=100. To re-establish the network connection, you must change the configuration of the VNIC in the OS of the Oracle VM Server CD2.
To avoid such issues, create server pools with homogenous network tagging mode. Or to maintain mixed network configurations, check with your administrator for maintaining the network connection of the logical domain operating system.
Oracle Enterprise Manager Ops Center launches a wizard that collects information about the servers, storage and network resources, and placement policies to create a server pool.
With the support of I/O domains in Oracle VM Server for SPARC, you can assign network interfaces from I/O domains or root domains to connect to the network. The Create Server Pool wizard provides the option to select the network interfaces from other domains.
The wizard provides the option to select the SR-IOV enabled network interfaces to connect to the network.
To Create a Server Pool for Oracle VM Server for SPARC
Select Server Pools in the Systems Group list on the Navigation pane.
Click Create Server Pool in the Actions pane.
The Create Server Pool Wizard is displayed.
In the Identify Server Pool step, enter the following information:
Name and description of the server pool.
Enter tags for categorizing your server pool.
Select Oracle VM Server for SPARC from the Virtualization Technology.
Click Next to select the members of the server pool.
Select one or more Oracle VM Servers to add to the server pool.
The list of Oracle VM Servers displayed have the following characteristics:
Not associated with a server pool
In a healthy state
Not placed in maintenance mode
The list displays the details of CPU frequency, type, and architecture of Oracle VM Servers. You can add Oracle VM Servers of different CPU type. If you want to live migrate the logical domains, then select Oracle VM Servers that are compatible for migration.
Click Next for configuring the I/O domains and root domains I/O resources for exclusive access.
When Oracle VM Servers have I/O domains or root domains, you can select to use the I/O resources exclusively for logical domains only. The exclusive access does not allow to create zones on them.
Click Next to select the network domain.
Select the network domain to associate with the server pool.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects to supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to select the networks and associate with the server pool.
For default network domain, all the networks that are declared and managed in Oracle Enterprise Manager Ops Center are listed. For an user-defined network domain, only the networks assigned to it are listed. Select the following details:
Select the networks that you want to associate with all the servers in the server pool.
Select the network tagging mode as Tagged or Untagged for networks configured with VLAN ID.
You can edit the VLAN ID if you are attaching the network for the first time in Tagged mode with the selected server.
Enter the number of connections for each network. You can make multiple connections to a network.
Select the network that you want to use for migration of logical domains.
Click Next to configure the interfaces of the servers.
For each network connection, provide the connection details:
Service Domain: Select the domain that provides the network interface. The domain can be primary, I/O domain or the root domain.
SR-IOV: Select this option if you want to assign SR-IOV enabled network interface for the network connection.
Mode: For networks configured with VLAN ID, you can modify the option to select the network to be attached in Tagged or Untagged mode.
NIC: The network interfaces that are available in the selected domain are listed. If you have selected SR-IOV, the network interfaces that are SR-IOV enabled are listed.
You can assign the same NIC to different network connection when the networks have different VLAN IDs and every connection is in tagged mode.
Switch Name: For SR-IOV enabled networks, there is no virtual switch creation. Instead, select the physical functions listed in the column.
For non SR-IOV enabled networks, a virtual switch is created. Enter a name for the virtual switch for a new network connection. You can also leave it blank for Oracle Enterprise Manager Ops Center to create a name using the default naming pattern.
When any of the selected members for the server pool is already connected to the selected network, then the existing connection details is displayed. The Connected option is selected and the virtual switch name is displayed. You can either keep the existing connection or modify the network connection to make an additional connection to the same network. To modify the network connection, you can change the service domain that provides the NIC or select another NIC from the list.
Address Allocation Method: You can select the following options:
Assign by DHCP: Select this option to automatically allocate the IP address by the system.
Use Static IP: Select this option and provide the IP address for the network connection.
Do not Plumb Interface: Select this option when you do not want to plumb an interface. This option allows you to avoid creating a virtual NIC in the control domain for networks used by logical domains. This option is available for non SR-IOV Ethernet connections beginning with Oracle Enterprise Manager Ops Center 12.2.2.
Do not Allocate: Select this option when you do not want to allocate any IP address for the network connection. You can allocate the IP address later in the logical domain OS. When you select the network interfaces from domains other than primary, then the Address Allocation Method is set to this option automatically and you cannot change it.
Click Next to associate the storage libraries.
The storage libraries that are reachable from the selected members of the pool are displayed. The storage libraries are required to store logical domain metadata, ISO images, and for virtual disks of logical domain.
You can select the following types of storage libraries:
Filesystem Storage Libraries: This includes the NAS storage libraries. Associate at least one NAS storage library with the server pool to store the logical domain metadata.
Block Storage Libraries: This includes the Static Block Storage and Dynamic Block Storage. Static block storage libraries consists of the exported LUNs of storage arrays that are not managed by Oracle Enterprise Manager Ops Center. Dynamic block storage libraries consists of the exported LUNs of storage array servers that are discovered and managed in Oracle Enterprise Manager Ops Center.
Select the storage libraries from the list that you want to associate with all the virtualization hosts in the server pool.
Click Next. If there are I/O domains and root domains in the selected Oracle VM Server for SPARC, then you are directed to define the association details for the libraries in the next step. Otherwise, you are directed to select the server pool policies in Step 13.
For each selected member of the server pool, you must select at least one domain to be associated with the storage library. The domains that are already associated with the library are displayed with the Associate option selected. You can deselect to remove the association and select another domain in the server.
The action to associate the Control Domain or the I/O domains is repeated for all the selected storage libraries in the previous step.
Click Next to select the server pool policies.
Select the following policies in the server pool to manage the under utilized and overutilized servers in the pool:
Placement Policy: This policy decides the preferred virtualization host in the server pool to place the logical domains.
Auto Balancing Policy: This policy performs load balancing of the server pool automatically at set intervals.
Select one of the following placement policies:
Lowest relative load: The recent lowest memory and CPU utilization for the Oracle VM Servers in the server pool are calculated. The server with the lowest relative load is considered to place a logical domain. Provide the threshold for CPU utilization above which the server is considered to be over-utilized and logical domains are migrated to server with lowest relative load.
Lowest allocated CPU and memory resources: The total number of virtual CPU and memory resources allocated for all the logical domains in an Oracle VM Server are calculated. The server with the lowest allocated resources is considered to place a logical domain in the pool. Provide the threshold values for CPU and memory allocation. When the allocation is exceeded, the server is considered to be over-allocated and logical domains are migrated to server with lowest allocated CPU and memory resources.
Minimize power consumption: The logical domains running in a server pool are placed on the minimum number of Oracle VM Servers and the unused Oracle VM Servers are powered off. The threshold value set for the server over-utilization ensures that the servers are not overloaded.
Select the automatic load balancing policies:
Select automatic load balancing and set the interval in which the server pool must be checked for resource balancing. You can set the approval to migrate the logical domains automatically. Or send notifications about the approval.
You can also select to manually balance the resources. See Balancing Resources for more information.
For Automatic Recovery, you can specify whether an attempt to power-off the failed server must be performed before initiating the automatic recovery of the logical domains.
When Oracle Enterprise Manager Ops Center has the capability to power-off the Service Processor of a failed server and you have selected the automatic recovery option, the failed server is powered off and then the logical domain recovery is started. If Oracle Enterprise Manager Ops Center cannot power-off the failed server, the recovery of the attached logical domains does not take place. This is to avoid any data corruption during the recovery process.
Enter the time interval to check the state of the server pool members reachability. The time value is entered in seconds. The minimum interval for checking the status must be 20 seconds and the default value is 180 seconds.
Click Next to view the summary of the server pool details.
Review the information to create a server pool for Oracle VM Server for SPARC. Click Finish to create the server pool.
The following information is covered in this section:
You can now pool Oracle Solaris Zones resources and manage your zones. Oracle Solaris Zones must meet the following requirements to be added to a server pool:
All the assets in the pool must have compatible architecture for supporting guest migration. For a global zone to be added to the server pool, it must have the same release or must be at least Oracle Solaris 10 10/08 version. This ensures that the zones can be migrated to a global zone in the server pool.
The global zones must be in healthy state.
The global zones must not be associated with a server pool.
The global zones must not be placed in maintenance mode.
Note:
'Oracle Solaris 11 shared IP zones should not be part of a server pool. You cannot perform migration, connect networks or add storage resources to shared IP zones.The network deployment for zones server pool vary depending on the Oracle Solaris OS version.
For a stand-alone Oracle Solaris 10 OS, you can attach the network in either shared IP or exclusive IP mode. Whereas, in a server pool, you can attach a network only in shared IP mode for Oracle Solaris 10 OS. You cannot make multiple connections to a network.
For Oracle Solaris 11 OS, the network is always attached in exclusive IP mode. In a server pool, the network for Oracle Solaris 11 OS is always deployed in exclusive IP mode. You can make multiple connections to a network.
When you want to have a server pool with mixture of Oracle Solaris 10 and 11 OS, then you cannot make multiple connections to a network. So, plan your server pools with compatible Oracle Solaris OS versions.
There are certain conditions in which the networking configuration that must be selected while creating a server pool. Refer to the following scenarios for deciding on the networking configuration when you create an Oracle Solaris Zones server pool:
You can select networks without VLAN ID. The UI does not provide the option to select Tagged or Untagged mode.
You can select to associate and configure the networks with VLAN ID in Tagged mode.
You can select to associate and configure the networks with VLAN ID in Untagged mode.
You can select to configure the networks in mixed tagging mode in the server pool. For example, you can attach the network N1 with VLAN ID = 100 in tagged mode with the server S1 and in untagged mode for server S2. Refer to Mixed Network Tagging Mode Configuration for more detailed information.
You can attach networks whose VLAN ID is similar to another network already connected to the servers. For example, a server S1 is already connected to network N1 with VLAN ID = 100, then while creating the server pool with S1 as the member of the pool, you can attach a network N2 with VLAN ID =100.
You can edit the VLAN ID of a network when you are attaching the network in Tagged mode for the first time.
When you can edit the VLAN ID of the network, you cannot enter -1 as the value for the VLAN ID.
If the selected network with a VLAN ID is already connected to the selected assets in Tagged mode, then you cannot edit the VLAN ID and make another connection.
You cannot make multiple network connections to a member of the server pool over the same network in both tagged and untagged modes. The mode can be either in tagged or untagged mode only. For example, if you attach network N1 with VLAN ID =100 for the first time to server S1 in Tagged mode, then you cannot make another connection to the same network N1 in Untagged mode. Every other connection with network N1 must always be in Tagged mode for server S1.
If the selected members of the pool are already connected to network N1 with VLAN ID =100, then you cannot select the same network with different VLAN ID to be connected for the server pool.
When you attach network to zones server pool, you can specify the network tagging mode for the networks configured with VLAN ID. You must ensure that the server pools have global zones either in tagged or untagged mode. If the server pool has mixed network tagging modes for the global zones, there might be any network outage issues for the zones depending on the action performed upon them.
If you want to maintain mixed network tagging configuration in your server pool, check with your administrator for re-establishing the network connection of the zones when the network outage occurs.
The server pool policies that are applicable for Oracle Solaris Zones are as follows:
Placement Policy
Place the zones on the global zone that has the lowest relative load. Set the CPU utilization threshold.
Place the zones on the global zone allocated with lowest CPU and memory resources. Set the CPU and memory allocation threshold.
Place zones on minimum number of global zones to minimize the power consumption. Set the CPU utilization threshold.
Automatic Balancing
You can select the automatic load balancing so that the zones can be migrated automatically in the server pool whenever the thresholds are exceeded. You can schedule the automatic load balancing to take place at definite time interval.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some servers are freed from the zones and powered off to minimize power consumption.
You can set whether the approval is required from the administrator for migrating the zones.
Automatic Recovery
You can set the automatic recovery for the zones in the following scenarios:
During zone creation, you can enable automatic recovery of the zones. You can set the value for priority of recovery. Zone with a highest priority is migrated first.
For existing zones, use the option Enable Automatic Recovery and Disable Automatic Recovery to set the automatic recovery. You can edit the zone configuration to modify the value for priority of recovery.
In an Oracle Solaris Zones server pool, you can specify that the automatic recovery of a failed server must start first with an attempt to power off the failed server, ensure that it is stopped and then initiate the automatic recovery of the zones.
Oracle Enterprise Manager Ops Center takes you through a series of steps to collect information for creating a zones server pool.
Select Server Pools in the Systems Group list on the Navigation pane.
Click Create Server Pool in the Actions pane.
The Create Server Pool Wizard is displayed.
In the Identify Server Pool step, enter the following information:
Enter a descriptive name for the server pool.
(Optional) Enter a description for the server pool.
(Optional) Enter tags for categorizing your server pool.
Select Oracle Solaris Zones – SPARC or Oracle Solaris Zones –x86 from the Virtualization Technology menu.
You cannot have a zones server pool with a mixture of SPARC and x86 architectures.
Click Next to select the members of the server pool.
Select one or more global zones to add to the server pool.
The list of global zones displayed have the following characteristics:
The same architecture – SPARC or x86
Oracle Solaris 10 10/08 OS or higher version
Not associated with a server pool
In a healthy state
Not placed in maintenance mode
Not running in an I/O domain or root domain that is configured to provide its resources exclusively to other logical domains
The list includes the CPU architecture and OS version of the global zone. Select global zones that are compatible for migration of zones within the pool.
Click Next to associate the network domain.
Select the network domain to associate with the server pool from the available network domains in the list, then click Next.
Default network domain: If you select the default network domain, go to Step 8.
User defined network domain: If you select a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. When you do not want to bond the interfaces, go to Step 8.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects to supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to select the networks and associate with the server pool.
For a default network domain, all of the networks that are declared and managed in Oracle Enterprise Manager Ops Center are listed. For a user-defined network domain, only the networks assigned to it are listed.
Select the networks that you want to associate with all the global zones in the server pool. Depending on the Oracle Solaris OS versions selected for the server pool, you can make multiple connections to a network. See IP Stack in Server Pool for more information about IP stack mode for networks in the zones server pool.
For networks configured with VLAN ID, you can specify whether the network must be attached in Tagged or Untagged mode. If you are attaching the network in Tagged mode for the first time, then you can edit the VLAN ID of the network, provided the VLAN ID is not used by any of the networks attached to the asset.
Enter the total number of connections and click Next to configure the interfaces of the servers.
For each network connection, provide the connection details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
If required, you can modify the network tagging mode specified in the previous step.
You can assign the same NIC to different network connection when the networks have different VLAN IDs and every connection is in tagged mode.
When supported by the network, you can select System Allocated for the NIC and IP address to be automatically allocated by the system.
You can select Do Not Allocate IP to skip the option of providing the IP address for the network connection.
Click Next to associate the storage libraries.
The storage libraries that are reachable from the selected members of the pool are displayed. The storage libraries are required to store zone metadata, ISO images, and for virtual disks of zones.
You can select the following type of storage libraries:
Filesystem Storage Libraries: This includes the NAS storage libraries. Associate at least one NAS storage library with the server pool to store the zone metadata.
Block Storage Libraries: This includes the Static Block Storage and Dynamic Block Storage. Static block storage libraries consists of the exported LUNs of storage arrays that are not managed by Oracle Enterprise Manager Ops Center. Dynamic block storage libraries consists of the exported LUNs of storage array servers that are discovered and managed in Oracle Enterprise Manager Ops Center.
Select the storage libraries from the list that you want to associate with all the virtualization hosts in the server pool.
Note:
When you have unmanaged storage attached to the non-global zones, then upload the script to be placed in all the global zones in the server pool. See Script to Migrate a Zone With Dependencies for more information about migrating zone with unmanaged storage.Click Next to select the server pool policies.
You must select the following policies in the server pool to manage the under utilized and overutilized servers in the pool:
Placement Policy: This policy decides the preferred virtualization host in the server pool to place the zones.
Auto Balancing Policy: This policy performs load balancing of the server pool automatically at set intervals.
Select one of the following placement policies:
Lowest relative load: The recent lowest memory and CPU utilization for the global zones in the server pool are calculated. The server with the lowest relative load is considered to place the zone. Provide the threshold for CPU utilization above which the server is considered to be over-utilized and zones are migrated to server with lowest relative load.
Lowest allocated CPU and memory resources: The total number of virtual CPU and memory resources allocated for all the zones in an Oracle VM Server are calculated. The server with the lowest allocated resources is considered to place a zone in the pool. Provide the threshold values for CPU and memory allocation. When the allocation is exceeded, the server is considered to be over-allocated and zones are migrated to server with lowest allocated CPU and memory resources.
Minimize power consumption: The zones running in the server pool are placed on the minimum number of global zone and the unused servers are powered off. The threshold value set for the server over-utilization ensures that the servers are not overloaded. Otherwise, the powered off servers can be powered on to host the zones.
Select the automatic load balancing policies:
Select automatic load balancing and set the interval in which the server pool must be checked for resource balancing. You can set the approval to migrate the zones automatically. Or send notifications for the approval.
You can also select to manually balance the resources. See Balancing Resources for more information.
For Automatic Recovery, you can specify whether an attempt to power-off the failed server must be performed before initiating the automatic recovery of the zones.
When Oracle Enterprise Manager Ops Center has the capability to power-off the Service Processor of a failed server and you have selected the automatic recovery option, the failed server is powered off and then the zone recovery is started.
Enter the time interval to check the state of the server pool members reachability. The time value is entered in seconds. The minimum interval for checking the status must be 20 seconds and the default value is 180 seconds.
Click Next to view the summary of the server pool details.
Review the information to create a server pool for Oracle Solaris Zones. Click Finish to create the server pool.
You can group one or more Oracle VM Servers in an Oracle VM Manager and create server pools. When you create a server pool for Oracle VM Server for x86 systems, you must have the following information defined:
Cluster file system
Server pool master
Virtual IP address for the server pool master
Shared access to the server pool resources is a must for providing for high availability for the virtual machines running in the Oracle VM Servers of the server pool. This is achieved by cluster file system OCFS2 which allows multiple Oracle VM Servers to access the same disk at the same time. OCFS2 ensures that the Oracle VM Servers in a server pool can access and modify resources in the shared repositories in a controlled manner.
When you create a server pool, you must specify the server pool file system for the cluster heartbeat and other cluster information. The file system can be NFS shares or LUNs of iSCSI or SAN based storage servers. Oracle VM formats the server pool file system as OCFS2 file system.
In Oracle Enterprise Manager Ops Center, the cluster is always enabled by default. The cluster configuration is pushed out to all the Oracle VM Servers in the server pool and the cluster heartbeat starts when the server pool is created. You can set a separate network for this cluster heartbeat. See Manage Networks for more information about setting up the networks and their role for an Oracle VM Server.
An Oracle VM Server is internally elected as server pool master. You cannot set the role for an Oracle VM Server. When the elected Oracle VM Server fails, the role is set for another Oracle VM Server in the server pool. The virtual IP address provided while creating the server pool is assigned to the Oracle VM Server that has been elected as server pool master.
The server pool policies that are applicable for Oracle VM Server for x86 are as follows:
Placement Policy
Places the virtual machines on the Oracle VM Server that has the lowest relative load. Set the CPU utilization threshold.
Places the virtual machines on minimum number of Oracle VM servers to minimize the power consumption. Set the CPU utilization threshold.
Automatic Balancing
You can select the automatic load balancing so that the virtual machines can be migrated automatically in the server pool whenever the thresholds are exceeded.
When you enable automatic load balancing for Oracle VM Server for x86 server pool, the server pool is checked continuously for the selected placement policy. When the threshold exceeds, the virtual machines are migrated from one Oracle VM Server to another.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some Oracle VM Server host servers are freed from the logical domains and powered off to minimize power consumption.
When the servers are overloaded and there are no other servers in the pool to host the virtual machines, then the policy starts a powered-off server using its Wake-on-LAN capability and live migrate the virtual machines to take up the load. The Wake-on-LAN capability must be enabled on the BIOS of the Oracle VM Server.
Automatic Recovery
When an Oracle VM Server fails, the virtual machines are migrated to another Oracle VM Server in the server pool. The automatic recovery for a virtual machine is set in the following scenarios:
During virtual machine creation, select the option Enable High Availability. This ensures that the virtual machines are migrated when an Oracle VM Server fails.
For an existing virtual machine, use the option Enable or Disable Automatic Recovery to set the automatic recovery.
The following procedure describes the steps in the wizard that is launched to create an Oracle VM Server for x86 server pool.
To Create an Oracle VM Server for x86 Server Pool
Select Server Pool in the Systems Group list on the Navigation pane.
Click Create Server Pool in the Actions pane.
The Create Server Pool Wizard is displayed.
In the Identify Server Pool step, enter the following information:
Name and description of the server pool.
Enter tags for categorizing your server pool.
Select Oracle VM Server for x86 from the Virtualization Technology.
Click Next to define the server pool configuration.
Define the following configuration details:
Select the Oracle VM Manager in which the Oracle VM Servers are discovered and owned.
Select the network domain.
Select a network from the list of networks available in the network domain.
Enter the virtual IP address that is assigned to the server pool master. When the server pool master changes, the IP address is assigned to the new Oracle VM Server.
Select the server pool file system to store cluster hearbeat and other cluster information. The file system can be either NFS shares on a NFS file servers or LUNs of SAN and iSCSI based storages. The NFS file servers and the storage servers are known and reachable to the selected Oracle VM Manager.
Click Next to select the members of the pool.
Select one or more Oracle VM Servers to add to the server pool.
The list of Oracle VM Servers displayed have the following characteristics:
Owned by the Oracle VM Manager. Refer to Manage Ownership of Oracle VM Servers to own an Oracle VM Server.
Not associated with a server pool
In a healthy state
Not placed in maintenance mode
Have no virtual machines in the running, shutdown or suspended state
Select Oracle VM Servers that are compatible for migration of virtual machines. To migrate virtual machines within a server pool, the Oracle VM Servers systems must be identical in model.
Click Next to associate the network domain.
Select the network domain to associate with the server pool.
When you have selected default network domain, go to Step 7.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. You cannot bond the interfaces in Oracle VM Server for x86. Bonding can be done in only individual OS of the Oracle VM Server.
Click Next to select the networks and associate with the server pool.
For default network domain, all the networks that are declared and managed in Oracle Enterprise Manager Ops Center are listed. For a user-defined network domain, only the networks assigned to it are listed.
Select the networks that you want to associate with all the servers in the server pool. You cannot make multiple connections to a network. Limit the number of connections to 1.
Click Next to configure the interfaces of the servers.
For each network connection, specify the following details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs.
You can select System Allocated for the NIC and IP address to be automatically allocated by the system.
Click Next to associate the storage libraries.
The storage libraries that are reachable from the selected members of the pool are displayed. The storage libraries are required to store virtual machine metadata, ISO images, and for virtual disks of virtual machine.
You can select the following type of storage libraries:
Filesystem Storage Libraries: This includes the Oracle VM storage repositories. At least one Oracle VM storage repository must be associated with the server pool.
Block Storage Libraries: This includes the Static Block Storage and Dynamic Block Storage. Static block storage libraries comprises the exported LUNs of storage arrays that are not managed by Oracle Enterprise Manager Ops Center. Dynamic block storage libraries comprises the exported LUNs of storage array servers that are discovered and managed in Oracle Enterprise Manager Ops Center.
Select the storage libraries from the list that you want to associate with all the virtualization hosts in the server pool.
Click Next to select the server pool policies.
You must select the following policies in the server pool to manage the under utilized and overutilized servers in the pool:
Placement Policy: This policy decides the preferred virtualization host in the server pool to place the logical domains.
Auto Balancing Policy: This policy performs load balancing of the server pool automatically at set intervals.
Select one of the following placement policies:
Lowest relative load: The recent lowest memory and CPU utilization for the Oracle VM Servers in the server pool are calculated. The server with the lowest relative load is considered to place a virtual machine. Provide the threshold for CPU utilization above which the server is considered to be over-utilized and virtual machines are migrated to server with lowest relative load.
Minimize power consumption: The virtual machines running in a server pool are placed on the minimum number of Oracle VM Servers and the unused Oracle VM Servers are powered off. The threshold value set for the server over-utilization ensures that the servers are not overloaded. Otherwise, the powered off Oracle VM Servers can be powered on to host the virtual machines.
Select the automatic load balancing policies:
Select automatic load balancing and set the interval in which the server pool must be checked for resource balancing. You can set the approval to migrate the virtual machines automatically. Or send notifications about the approval.
You can also select to manually balance the resources. See Balancing Resources for more information.
Click Next to view the summary of the server pool details.
Review the information to create a server pool for Oracle VM Server for x86. Click Finish to create the server pool.
You must monitor the requirements of the server pool so that there are resources to host the guests and run them efficiently. You might need to add more storage and network resources to run the guests. Also, the server pools gives the infrastructure support for the virtual datacenters. Oracle Enterprise Manager Ops Center provides the following management functions to manage the resources in the server pool:
You can edit the following parameters and attributes of a server pool:
Name
Description
Placement policy, including CPU Utilization
Auto balancing policy
Enable or disable automatic recovery option.
Automatic Recovery Authorization.
Automatic Recovery Number of Retries.
For Oracle Solaris Zones server pool, you can edit the following additional parameters:
Check server reachability interval.
Upload scripts to manage the file systems on unmanaged storage. See Step 10 in Oracle Solaris Zones Server Pool for more information.
For Oracle VM Server for SPARC server pool, you can edit the following additional parameters:
Migration networks.
Enable or disable automatic recovery to power off a failed server from Service Processor, given capabilities, before automatic recovery of attached logical domains.
Check servers reachability.
Perform the following to modify the parameters of a server pool:
Expand Assets in the Navigation pane.
Select Server Pools in the Resource Management view.
Select the server pool in the Navigation pane, then click the Summary tab.
To edit the parameters, click Edit Attributes in the Actions pane. To add, remove or modify the tags added to the server pool, click Edit Tags in the Actions pane.
Click the Save icon.
You can add more virtualization hosts to a server pool when you require more CPU and memory resources to take up the load in the server pool. Depending on the virtualization type, you can add global zones, and Oracle VM Servers of x86 and SPARC architecture.
The list of global zones that are displayed to be added to the server pool has the following characteristics:
Same release as the existing global zones in the pool or at least Oracle Solaris 10 10/08 OS or higher version
Not associated with a server pool
In a healthy state and not placed in maintenance mode
When the server pool has Oracle Solaris 11 OS and multiple network connections, then the list of available global zones is limited to only Oracle Solaris 11 OS. You cannot add Oracle Solaris 10 OS as you cannot make multiple network connections to it.
Load balancing in the server pool requires the zones to be migrated within the pool. Therefore, ensure compatibility for migration within the pool when you select the members.
Refer to Network Tagging Mode Conditions for more information about the network tagging conditions while adding an asset to the server pool.
To Add Global Zones to the Server Pool
Select the server pool and click Add Global Zones in the Actions pane.
The Add Global Zones to Server Pool Wizard is displayed.
Select one or more global zones to be added to the server pool.
When the server pool is associated with the default network domain, you must proceed to configure the interfaces. Otherwise, you must associate the network domain with the selected assets.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. When you do not want to bond the interfaces for global zones, go to Step 4.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to define the network connection.
For each network connection of the global zones, specify the following details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs and every connection is in tagged mode.
For networks configured with VLAN ID, you can select the network to be attached in Tagged or Untagged mode. If you are attaching the network in Tagged mode for the first time, then you can edit the VLAN ID of the network, provided the VLAN ID is not used by any of the networks attached to the asset.
You can select System Allocated for the NIC and IP address to be automatically allocated by the system.
Click Next to view the summary.
Review the information provided and click Finish to add the selected global zones to the server pool.
The list of Oracle VM Servers displayed to be added to the server pool have the following characteristics:
Not associated with a server pool
In a healthy state and not placed in maintenance mode
Refer to Network Tagging Mode Conditions for more information about the networking configurations when you add Oracle VM Server to the server pool.
To Add Oracle VM Servers to the Server Pool
Select the server pool and click Add Oracle VM Servers in the Actions pane.
The Add Oracle VM Servers to Server Pool Wizard is displayed.
Select one or more Oracle VM Servers to be added to the server pool.
When the server pool is associated with the default network domain, you must proceed to configure the interfaces. Otherwise, you must associate the network domain with the selected assets.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. When you do not want to bond the interfaces for Oracle VM Server for SPARC server pool, go to Step 4.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding parameters.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to define the network connection.
For Oracle VM Server for SPARC, you can select the network interfaces from I/O domains and root domains to connect to the network. You can also select SR-IOV enabled network interfaces that are available from primary and root domains. Select the following information for the network connection:
Service Domain: Select the domain that provides the network interface. The domain can be primary, I/O domain, or the root domain.
SR-IOV: Select this option if you want to assign SR-IOV enabled network interfaces for the network connection. SR-IOV enabled network interfaces are available only from primary and root domain.
Mode: For networks configured with VLAN ID, you can select the network to be attached in Tagged or Untagged mode. If you are attaching the network in Tagged mode for the first time, then you can edit the VLAN ID of the network, provided the VLAN ID is not used by any of the networks attached to the asset.
Note:
It is recommended to attach the network in the same tagging mode as the other Oracle VM Servers of the server pool to avoid any network outage.NIC: The network interfaces that are available in the selected domain are listed. If you have selected SR-IOV, the network interfaces that are SR-IOV enabled are listed.
You can assign the same NIC to different network connection when the networks have different VLAN IDs and every connection is in tagged mode.
Switch Name: For SR-IOV enabled networks, there is no virtual switch creation. Instead, select the physical functions listed in the column.
For non SR-IOV enabled networks, a virtual switch is created. Enter a name for the virtual switch for a new network connection. You can also leave it blank for Oracle Enterprise Manager Ops Center to create a name using the default naming pattern.
When any of the selected members for the server pool is already connected to the selected network, then the existing connection details is displayed. The Connected option is selected and the virtual switch name is displayed. You can either keep the existing connection or modify the network connection to make an additional connection to the same network.
Address Allocation: Select Do not Allocate to skip the option of providing the IP address. Select Use Static IP to provide the IP address, or select System Allocated to have the system automatically allocate an IP address.
Do Not Plumb Interface: Select this option when you want to avoid creating a virtual NIC in the control domain for networks used by logical domains. This option is available for non SR-IOV Ethernet connections beginning with Oracle Enterprise Manager Ops Center 12.2.2.
Click Next to view the summary.
Review the information provided and click Finish to add the selected assets to the server pool.
The list of Oracle VM Servers displayed to be added to the server pool have the following characteristics:
Owned by the Oracle VM Manager
Not associated with a server pool
In a healthy state and not placed in maintenance mode
Have no virtual machines in the running, shutdown or suspended states
To Add Virtualization Hosts to the Server Pool
Select the server pool and click Add Oracle VM Servers in the Actions pane.
The Add Oracle VM Servers to Server Pool Wizard is displayed.
Select one or more virtualization hosts to be added to the server pool.
When the server pool is associated with the default network domain, you must proceed to configure the interfaces. Otherwise, you must associate the network domain with the selected assets.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. For Oracle VM Server for x86, you cannot bond the interfaces and click Next.
For each network connection of Oracle VM Server for x86 server pool, specify the following details:
Specify the NIC and IP address for each network connection. When a selected server is connected to the network, no rows are displayed for that server.
You can assign the same NIC to different network connection when the networks have different VLAN IDs.
You can select System Allocated for the NIC and IP address to be automatically allocated by the system.
Click Next to view the summary.
Review the information provided and click Finish to add the selected assets to the server pool.
Removing Virtualization Hosts from Server Pool
You can remove the virtualization hosts from the server pool. For zones and Oracle VM Server for SPARC, you can remove them from the server pool even when there are zones and logical domains running on them. All the network and storage resources are not unconfigured. Instead, the resources are retained as it is in the virtualization hosts.
To remove Oracle VM Server for x86 from a server pool, you must stop or migrate the virtual machines that are running in it.
All networks in a network domain that is associated with a server pool are available to the members of the server pool. To associate a user-defined network domain with a server pool, there must be a connection between the virtualization servers and the fabrics in the network domain. The fabric connection is not required for the default network domain.
The default network domain includes all the networks in Oracle Enterprise Manager Ops Center. When you associate a server pool with the default network domain, all the networks in Oracle Enterprise Manager Ops Center are available for attaching with the server pool.
For a user-defined network domain, you must define the physical interfaces of each virtualization server in the server pool to connect to each fabric in the network domain. You can also bond the interfaces of the virtualization servers.
Note:
You cannot the bond the interfaces of Oracle VM Server for x86 servers.To Associate the Network Domain to a Server Pool
Click Associate Network Domain in the Actions pane.
The Associate Network Domain Wizard is displayed.
Select the network domain from the list.
For a user-defined network domain, select the physical interfaces of each selected servers to connect to each fabric in the network domain. When you do not want to bond the interfaces, go to Step 7. For Oracle VM Server for x86, you cannot bond the interfaces and go to Step 7.
For default network domain, you are taken to the Summary step.
You can bond the interfaces of the servers into a single logical link. The aggregation is done according to the standard IEEE 802.3ad Link Aggregation. Select Bond Interface in the Physical Interface column. A Bond ID is provided. Select the physical interfaces for the aggregation.
Click Next to configure the bonding.
In the Configure Bonding step, specify the following parameters for Link Aggregation:
Load balancing policy.
LACP mode. If the Ethernet switch to which the physical interface connects to supports aggregation, then specify the LACP mode.
LACP timer.
MAC address policy and provide the MAC address.
Click Next to view the summary.
Review the information and click Finish to associate the network domain with the server pool.
When you associate with a user-defined network domain, only the assigned networks in the network domain are available for attaching with the server pool. Therefore, ensure that you assign the required networks to the domain to attach to the server pool.
After the network domain is associated with a server pool, you can attach the networks in the network domain. Depending on the type of virtualization, the multiple connection to a network is allowed.
For Oracle VM Server for SPARC, you can make multiple connections to a network. Refer to the Chapter 19, "Oracle VM Server for SPARC" for more information.
For Oracle Solaris Zones, you can make multiple connections to a network when the server pool contains only Oracle Solaris 11 OS. For Oracle Solaris 10 OS, you can make only one connection. Also, when the server pool has mixture of Oracle Solaris 10 and 11 OS, then also you can make only single connection.
Refer to Network Tagging Mode Conditions for more information about selecting the correct tagging modes when you attach the network to Oracle Solaris Zones and Oracle VM Server for SPARC.
To Attach Networks to Server Pool
Click Attach Networks in the Action pane.
The Attach Network Wizard is displayed. All the networks assigned to the network domains are listed. For default network domain, all the networks are listed. The table also displays existing number of connections to the server pool.
Select one or more networks to attach to the server pool.
Click Next to specify the number of connections.
Enter the total number of connections required as per the virtualization type:
For Oracle VM Server for SPARC, you can enter multiple connections. For each network connection, a virtual switch is created. Virtual switch creation is not applicable for SR-IOV enabled networks. For each network connection, you can assign the network interfaces from the I/O domains and root domains. Specify the tagging mode as Untagged or Tagged for the networks configured with VLAN ID.
For Oracle VM Server for x86, you can enter only one connection per network. You cannot make multiple connections.
For Oracle Solaris Zones, the Oracle Solaris OS version plays an important role. For Oracle Solaris 10 OS, the network is always deployed in shared IP mode. You cannot make multiple connections. For Oracle Solaris 11 OS, the network is always deployed in exclusive IP mode. You can make multiple connections to the network. For each network connection, a VNIC is created when you boot the zone. For a server pool of mixture of Oracle Solaris 10 and 11 OS, the network connection is limited to single connection.
Note:
The number of connections do not limit the number of guests to be connected to be network.Click Next to configure the interfaces.
When one of the virtualization hosts are connected to the network, the corresponding row for network configuration for that host is not displayed. If not, specify the information for each connection:
Specify the NIC and IP address for each connection.
You can assign the same NIC to different network connection when the VLAN IDS are different for the networks and every connection is in tagged mode.
Select Do Not Allocate IP to connect to the network without specifying the IP address. This option is available for zones and Oracle VM Server for SPARC systems.
Select System Allocated for the system to take care of the NIC and IP address allocation.
For Oracle VM Server for SPARC and zones, you can select the network interfaces from I/O domains and root domains to connect to the network. You can also select SR-IOV enabled network interfaces that are available from primary and root domains. Select the following information for the network connection:
Service Domain: Select the domain that provides the network interface. The domain can be primary, I/O domain or the root domain.
SR-IOV: Select this option if you want to assign SR-IOV enabled network interfaces for the network connection. SR-IOV enabled network interfaces are available only from primary and root domain.
Mode: For networks configured with VLAN ID, you can modify the option to select the network to be attached in Tagged or Untagged mode.
NIC: The network interfaces that are available in the selected domain are listed. If you have selected SR-IOV, the network interfaces that are SR-IOV enabled are listed.
You can assign the same NIC to different network connection when the networks have different VLAN IDs and every connection is in tagged mode.
Switch Name: For SR-IOV enabled networks, there is no virtual switch creation. Instead, select the physical functions listed in the column.
For non SR-IOV enabled networks, a virtual switch is created. Enter a name for the virtual switch for a new network connection. You can also leave it blank for Oracle Enterprise Manager Ops Center to create a name using the default naming pattern.
When any of the selected members for the server pool is already connected to the selected network, then the existing connection details is displayed. The Connected option is selected and the virtual switch name is displayed. You can either keep the existing connection or modify the network connection to make an additional connection to the same network.
Address Allocation: Select Do not Allocate to skip the option of providing the IP address. Select Use Static IP to provide the IP address, or select System Allocated to have the system automatically allocate an IP address.
Beginning with Oracle Enterprise Manager Ops Center release 12.2.2, select Do Not Plumb Interface when you do not want to plumb the interface.
Click Next to view the summary of the information.
Review the information provided and click Finish to attach the networks to the server pool.
Detaching Networks from Server Pool
You can detach the networks that have been attached to the server pool. Detaching the networks results in unplumbing the network connection to the host's NIC from all the virtualization hosts in the server pool.
To Detach a Network from Server Pool
Select the server pool.
Select the Network tab in the center pane.
Select the network that you want to remove.
Click the icon Unbind Network from Server Pool.
The Unbind Network from a Server Pool window is displayed.
Click Unbind to confirm the network removal.
Table 21-4 lists the types of storage libraries that you can associate with the server pool.
Table 21-4 Supported Libraries
| Oracle VM Storage Repositories | NAS Libraries | SAN Fibre Channel Libraries | SAN iSCSI Libraries | |
|---|---|---|---|---|
|
No |
Yes |
Yes |
Yes |
|
|
Yes |
Yes |
Yes |
Yes |
|
|
No |
Yes |
Yes |
Yes |
When you associate libraries with the server pool, only the libraries that are reachable from the virtualization hosts in the server pool are listed. Depending on the type of library, you either provide virtual disks or LUNs to the guests' storage.
Except for Oracle VM Server for SPARC server pool, the Associate Libraries action for zones and Oracle VM Server for x86 server pool displays the available libraries. Select the libraries and click Associate to associate the libraries with the zones or Oracle VM Server for x86 server pool.
For Oracle VM Server for SPARC server pool, the procedure to associate and disassociate the storage libraries are described as follows.
To Associate Libraries with Oracle VM Server for SPARC Server Pool
Select the Oracle VM Server for SPARC or server pool from the Server Pools list.
Click Associate Libraries in the Actions pane.
The Associate Library Wizard is displayed.
Select the libraries that you want to associate with the server pool, then click Next.
Select Associate action for the Control Domain, I/O domain, or both. You must select at least one domain per server.
When the storage library is already associated with domains, then the Associate option is selected. You can deselect to disassociate the storage library from the domain.
This action is repeated for each selected storage library. Click Next.
Review the summary of the association and click Finish to associate the library with the selected domains in the server pool.
To Disassociate Storage Libraries from Oracle VM Server for SPARC Server Pool
Select the Oracle VM Server for SPARC server pool in the Server Pools list.
Select the Libraries tab in the Center pane.
All the storage libraries that are associated with the server pool are displayed.
Select the storage library that you want to disassociate from the server pool.
The Disassociate Library window is displayed.
When the storage library is associated with the domains are displayed with the Disassociate option selected. You can deselect the Disassociate action if you do not want to remove the association.
Click Finish to disassociate the storage library from the selected domains in the Oracle VM Server for SPARC server pool.
The guests refer to the logical domains, zones or the virtual machines that can be created in the virtualization hosts. According to the virtualization type of the server pool, you have the following options:
These options trigger the deployment plans for the corresponding guests. Refer to the following chapters to refer to the profile and plan creation for the guests:
When you place a virtualization host in a server pool, the option to migrate one or more guests is enabled. The following options become available for the virtualization hosts:
Oracle Solaris Zones – Migrating Zones
Oracle VM Server for SPARC - Migrating Logical Domains
Oracle VM Server for x86 – Migrating Virtual Machines
You can migrate multiple zones from a global zone which is in a server pool. Migrate the zones to an individual global zone or other zones server pool. When you migrate zones, the source and the target global zone must be compatible. The target global zones must have the following characteristics:
Running at least Oracle Solaris 10 8/07 OS.
Can access all the storage libraries associated with the zone.
Apart from compatibility, the target and the source global zone might have differences in the patches and packages installed on them. Choose to update the zone to match the patches and packages of the target global zone.
Note:
You cannot downgrade the patches and packages of the zone. The migration fails in such scenario.You can also force the migration of the zone without updating the patches and packages. Select the update options while migrating the zone.
Select the global zone from which you want to migrate the zones.
Click Migrate Zones in the Actions pane.
The Migrate Zones Wizard is displayed.
The list includes the zones running in the global zone. Select one or more zones from the list.
Click Next.
Select an individual global zone or server pool to which you can migrate the zones.
The table displays the list of eligible global zones and server pool to which you can migrate the zones. The target global zone in the server pool depends on the server pool placement policy.
Select an update option to continue with migration.
The source and the target global zones might not be in the same patch level. Either you can select to update the patches and packages of zone to match the target global zone or continue migration without updating the zone.
Review the details and click Finish to migrate the zones.
Migrate multiple logical domains from an Oracle VM Server to another Oracle VM Server in the same server pool. You cannot do cross server pool migration.
Only from Oracle VM Server for SPARC 2.1 version, live migration of logical domains is supported. For earlier releases, it is cold migration. The logical domains are shut down and then restarted on the target server.
See Migrate Logical Domains in Chapter 19, "Oracle VM Server for SPARC" for more conditions in migrating a logical domain.
Select the Oracle VM Server in a server pool.
Click Migrate Logical Domains in the Actions pane.
The list of logical domains running in the Oracle VM Server are listed. Select one or more logical domains from the list.
Select an Oracle VM Server from the list.
The table lists the Oracle VM Servers that have enough resources to host the logical domains in the same server pool. The Oracle VM Server are listed in the decreasing order of preference.
Review the information and click Finish to migrate the logical domains.
For Oracle VM Server for x86 systems, you can migrate virtual machines only within a server pool. Live migration of virtual machines is supported. The eligible Oracle VM Servers have the following characteristics:
Owned by the Oracle VM Manager.
Placed in the same server pool as the source Oracle VM Server.
Have required resources to host the virtual machines.
Identical in machine make and model of the source Oracle VM Server.
Select the Oracle VM Server in the server pool.
Click Migrate Virtual Machines in the Actions pane.
Select the virtual machines that you want to migrate.
Select the Oracle VM Server to which you want to migrate the virtual machines.
The table lists the Oracle VM Servers that are eligible to host the virtual machines.
Review the summary and click Finish to migrate the virtual machines.
When you have selected to balance server pool resource manually, use the Balance Resources action to check for the under-utilization or over-utilization of resources. Select a server pool and click Balance Resources in the Actions pane to display the current utilization of the virtualization hosts in the server pool. When balancing is required according to the placement policy, Oracle Enterprise Manager Ops Center displays a list of target server that can accept migrated guests. Click Balance Resources to start the migration job.
When the server pool does not meet the policies and rules set, warning or critical incidents are raised accordingly. For example, when the servers in the pool are overloaded and need to migrate a guest to another server.
Incidents also provide information about the result of a server pool status on automatic load balancing.
The guests are recovered according to the policy set in the server pool for automatic recovery. The incidents are raised that describes the guests that are recovered and in which servers they are recovered in the server pool.
Incidents are also raised when there are no resources or servers available in the server pool for recovering the guests. Incidents provide more information about the scenario.
Refer to Chapter 9, "Incidents" for more information about handling and viewing incident messages.
Use the option Delete Server Pool to release the virtualization hosts back to stand-alone state. The type of server pool imposes some limitations to delete a server pool.
You can delete the Oracle VM Server for SPARC server pool even when the logical domains are attached to it and running. There is no restriction for deleting an Oracle VM Server for SPARC server pool to shut down the logical domains. All the network and storage resources are not unconfigured from the Oracle VM Servers in the server pool. The network and storage resources are retained by the servers.
For a zones server pool, you can delete the server pool and the zones continue to be attached to the global zone. Oracle Enterprise Manager Ops Center does not provide any restrictions to shut down the zones before deleting a zones server pool.
Shut down all the virtual machines and remove all Oracle VM Servers except for the Master Oracle VM Server before deleting the server pool. The virtual machine metadata and virtual disks are stored in the storage library. The virtual machine details are not lost and you can start the virtual machine.
For instructions in performing actions or to learn more about the role of this feature, go to one of the following resources:
See the following how to documentation in the Deploy How To tab in the library at http://docs.oracle.com/cd/E40871_01/nav/deployhowto.htm for end-to-end examples.
Oracle Enterprise Manager Ops Center Exploring Your Server Pools
Oracle Enterprise Manager Ops Center Creating a Server Pool for Zones
Oracle Enterprise Manager Ops Center Creating a Server Pool for Oracle VM Server for SPARC
See the Oracle Enterprise Manager Ops Center Exploring Your Server Pools how to documentation in the library at http://docs.oracle.com/cd/E40871_01/nav/operatehowto.htm for an end-to-end example of how to manage server pools.