This chapter includes the following sections:
A server pool is a group of one or more virtualization hosts with the same processor architecture that have access to the same virtual and physical networks, and storage resources. Server pools provide load balancing, high availability capabilities, and sharing of some resources for all members of the pool.
You can create server pools for the following types of virtualization servers:
Oracle VM Server for SPARC
Oracle VM Server for x86
Oracle Solaris Zones x86
Oracle Solaris Zones SPARC
A virtualization host can refer to Oracle VM Server for SPARC, Oracle Solaris Zones, or Oracle VM Server for x86 that are managed through Oracle Enterprise Manager Ops Center. A virtual host or guest in a server pool refers to a non-global zone, logical domain or virtual machine running on the virtualization host.
Server pools are resource pools of homogeneous virtualization hosts that allow actions such as balancing load between servers and moving guests between hosts. You can also apply resource configurations and policies to them. The policies that you establish for a server pool manage many of the CPU utilization and resource balancing functions. Operations to the server pool are delegated to the individual virtualization hosts in the server pool.
To manage the guests within a server pool, you can perform warm and live guest migration and you can balance all of the guests' load among the members of the server pool. You can configure a policy to balance the load automatically, based on a schedule that you determine, or you can balance the load manually. When a virtualization host system shuts down, such as a hardware failure, you can start the guests on another host in the same pool.
Virtualization hosts in a server pool share network and storage libraries and several server pools can share the same networks and storage resources.
Server pools form a key concept for virtual datacenter management. Pooling your virtualization supported systems plays an important role and requires proper understanding of your requirements in your environment.
All the requirements, procedures, and different functions available for a server pool depend on the type of virtualization technology selected. See the appropriate sections to create a server pool for different virtualization technology.
When you create a server pool, you must plan the storage and network resources such that they are accessible for all the members of the server pool.
You must set the policies to place the guests on the virtualization hosts and choose how you want to do the load balancing of the server pool. To understand more on this, refer to Server Pool Policies.
The following table lists the tasks and the role required to complete the task. Contact your administrator if you do not have the necessary role or privilege to complete a task.
See the Related Resources for Server Pools section for information about the different roles and the permissions they grant.
Table 7-1 Server Pool Tasks and Roles
Task | Role |
---|---|
Create, manage, update, and delete server pools |
Virtualization Admin |
Provision and manage virtualization host |
Virtualization Admin |
Create, manage, update, and delete guests |
Virtualization Admin |
Using Oracle Enterprise Manager Ops Center, you can perform the following tasks:
Create server pools for different types of virtualization technologies.
Balance load on the resources in the server pool.
Manage storage and network resources for the server pool.
Set placement policies for automatic load balancing.
Perform automatic recovery of the guests during server failure.
Edit the attributes of an existing server pool.
This table lists where to find different information for server pools in the UI.
Table 7-2 Location of Server Pool Information in the UI
To See | Location |
---|---|
Server pools |
Expand Assets in the Navigation pane and select Server Pools in the Resource Management Views. |
Server pool resources |
Expand Assets in the Navigation pane and select Server Pools in the Resource Management Views. Select the server pool and the center pane displays all the details of the server pool resources such as network, libraries, and fabrics. |
Server pool actions |
Expand Assets in the Navigation pane and select Server Pools in the Resource Management Views. Select the server pool and the Actions pane list the various options such as Associate Libraries, Attach Networks, Create Guests, Associate Network Domains and Edit Attributes for managing the server pool. |
You can create server pools for different types of virtualization technology. See the appropriate sections to create a server pool for different virtualization technology.
Each time Oracle Enterprise Manager Ops Center adds a new system to the pool, it checks the network connections on all the members of the pool. You will not be allowed to continue if something is wrong.
The following are some server pool capabilities and limitations:
You can create a zones server pool and use the logical domains as the global zones. However, those logical domains should not be part of an Oracle VM Server server pool. The user interface does not prevent you from creating these server pools, but the configuration is not supported and it can cause problems; particularly when automatic recovery is enabled on both server pools.
Before performing maintenance in your datacenter, it is a good idea to place the systems affected in maintenance mode to suppress the creation of alerts and incidents. When the system is part of a server pool, you can decide whether you want to migrate the guests to a different host in the pool before placing the system in maintenance mode. Maintenance mode does not stop monitoring, it only stops the alerts and incidents.
Note:
If you do not relocate the guests before putting the system in maintenance mode, consider disabling the server pool monitoring and disabling automatic recovery for each guest. Beginning with 12.2.2, you can disable automatic recovery at the server pool level instead of disabling it for each guest. See Editing Server Pool Parameters for how to disable automatic recovery for the server pool.
The following information is covered in this section:
When you create a server pool, you define guest placement, and automatic load balancing polices. When you create guests, you define the guest resource consumption, including physical and virtual CPUs.
You can edit the server pool policies in the server pool's Summary tab. See Editing Server Pool Parameters for how to edit the pool attributes.
The placement policy determines the preferred virtualization host for new guests within the server pool, the type of load balancing (automatic or manual) of the server pool, and balances the server pool during any server failure or maintenance. You define the placement policy when you create the server pool; however, you can modify the policy at any time.
The following are the server pool placement policy options:
Place the guest on the virtualization host with the lowest relative load, based on the lowest CPU resource utilization. The calculation is based on a combination of the average load for the time period defined by the user. The default value is the last 10 minutes. The default threshold value for CPU utilization is 75%.
Place the guest on the virtualization host with the lowest allocated CPU and memory, that is, the total static resource allocation across all guests on the host. The resource allocation is the sum of the number of vCPUs and virtual memory specified for each guest. This policy first verifies whether the resource allocation is possible and then ranks the server with available resources. The relative weight of CPU and memory resource is 1:1 while calculating the load on the host.
Place the guest on minimum number of virtualization host and thereby consume less power.
The following is an example of how the placement policy works in a server pool.
Example 7-1 Example of Least Allocated Virtualization Host
A server pool has two virtualization hosts with different CPU and memory allocations. Host A has 8 available CPUs and 16 GB of memory. Host B has 4 available CPUs and 8 GB of memory.
Host A has three guests:
Guest X has one vCPU and 1024 MB of memory.
Guest Y has two vCPUs and 2048 MB of memory.
Guest Z has one vCPU and 1024 MB of memory.
Host A's total static allocation is 4 vCPUs and 4 GB of memory.
Host B has one guest:
Guest W has 3 vCPU and 4096 MB of memory.
Host B's total static allocation is 3 vCPUs and4 GB of memory.
The allocation percentage for Host A is: CPU allocation is 4 vCPUs/8 physical CPUs, or 50%. Memory allocation is 4 GB/16 GB or 25%.
The allocation percentage for Host B is: CPU allocation is 3 vCPUs/4 physical CPUs or 75%. Memory allocation is 4 GB/8 GB or 50%
Therefore, Host A is the least allocated of the virtualization hosts.
The policy to minimize power consumption places the guests on the minimum number of virtualization hosts and powers off the idle servers. The idle virtualization hosts are powered off or set to low-power mode on explicit approval.
For Oracle VM Server for x86, Oracle Enterprise Manager Ops Center will power off unused servers in the pool.
Beginning with Oracle Enterprise Manager Ops Center 12.2.2, you can enable power management for Oracle VM Server for SPARC server pools. When you enable power management, Oracle Enterprise Manager Ops Center will load as much work as it can onto a physical server before adding a virtual machine to a different server in the pool.
When power management is enabled, the software performs the following tasks:
Places unassigned resources into a lower power state
Identifies the members with the highest load and determines if those members have enough unallocated resources to support a new virtual machine placement request
Checks the number of logical domains for each server in the pool and sets the appropriate placement policy
Adjusts the placement policy every time you create, migrate, or delete a logical domain.
You can enable the policy and time interval to check for resource changes in Administration. The default check.delay time interval checks for resource changes every 5 minutes (300 seconds). See the Related Resources for Server Pools section for how to set the power consumption policy and edit the property values.
Use the Automatic Load Balancing Policy to schedule load balancing within a server pool. You can schedule the automatic balancing to occur weekly, daily, or hourly on a specific day and time of the week. The default is to balance the load on the servers in the server pool every Saturday at midnight according to the defined placement policy.
Scheduling automatic load balancing is applicable only for Oracle VM Server for SPARC and Oracle Solaris Zones server pool.
Note:
The day and time are in the Enterprise Controller's time zone.
When you do not want to balance the server pool's load automatically, schedule a reminder to balance the server pool's load manually.
Automatic load balancing policy does not consider the virtualization hosts that are placed in maintenance mode as a target in the server pool for migrating the guests. When you place a virtualization host in maintenance mode, all of the migratable guests in it are migrated to other servers in the server pool. This action is triggered automatically.
You can define the resource usage threshold for the servers in the server pool. The servers are considered to be loaded heavily depending on the threshold set for the resources. Depending on the placement policy selected, the CPU or total CPU and memory resource usage is calculated.
When the virtualization host utilization exceeds the threshold, the software identifies the servers with sufficient resources and the guests that can be migrated from the server to balance the load.
When the virtualization hosts are well within the threshold and power minimization policy is selected, the automatic balancing policy attempts to free a virtualization host from its guest so that the administrator can power off the server.
The automatic recovery feature provides the option to recover the guests that are attached to a failed server. The recovered guests are restarted on other servers in the server pool.
This feature is available for all types of supported virtualization technology. You can enable or disable the automatic recovery of zones, logical domains, and the virtual machines. The automatic recovery for virtual machines is managed by Oracle VM Manager. For logical domains and zones, the automatic recovery is managed by Oracle Enterprise Manager Ops Center.
Oracle Enterprise Manager Ops Center follows the recovery process described in this section, and is applicable for zones and logical domains.
By default, automatic recovery is enabled at the server pool level. When enabled at the server pool level, the recovery options selected for zones and logical domains determines whether automatic recovery is attempted.
Oracle Enterprise Manager Ops Center monitors the managed assets by checking the connection between the agent deployed on the assets and the Proxy Controller. You can set the interval of periodic check of the assets reachability while creating the server pool. The default value is 180 seconds. If any of the Oracle Enterprise Manager Ops Center Agent Controllers installed on the server do not respond within the fixed timeout interval, then the server is considered to be a failed server. The timeout interval is a minimum of 20 seconds and a maximum of 60 seconds.
For example, if the interval is set at 180 seconds, the server pool checks the status of the members of the pool every 180 seconds and if there is no response from the Agent Controller in 60 seconds, then the automatic recovery of the zones or the logical domains in the server are started approximately four (4) minutes after the server failure.
If the interval is set at 20 seconds, the server pool checks the status of the members of the pool every 20 seconds and if there is no response from the Agent Controller in 20 seconds, then the automatic recovery of the zones or the logical domains in the server are started approximately 40 seconds after the server failure.
An automatic recovery job can be initiated when the global zone, kernel zone, or Oracle VM Server for SPARC is:
Placed in a server pool.
Has at least one zone or logical domain that is configured to recover automatically.
Is not in maintenance mode.
The job initiated first performs the preliminary check of pinging all the known IP address of the failed server from another member in the server pool and the heartbeat in the metadata that is refreshed by the servers. The recovery job continues if the asset is seen as unreachable from Oracle Enterprise Manager Ops Center.
The recovery job fails if the assets are reachable and only the Agent Controller is down on the server. The attempt to recover the logical domains or zones are stopped as they are reachable.
If an automatic recovery failed or cannot be performed because of non availability of free resources in the server pool, Oracle Enterprise Manager Ops Center periodically checks if there are enough free resources and retries the automatic recovery. Hosts in maintenance mode cannot be targets for automatic recovery.
The software checks every 60 seconds until it can perform the automatic recovery. A retry counter is available to control how many times the auto-recovery manager will attempt to perform an automatic recovery. Setting the counter to zero (0) results in an unlimited number of times that the automatic recovery is attempted.
You can disable the automatic recovery feature for a server pool or for an individual asset. To disable the automatic recovery feature for a server pool, and all assets in the server pool, select the check box when you create the server pool. You can edit an existing pool to disable or enable the feature. When you disable automatic recover at the server pool level, the software will not check for resources or attempt to recover a guest, regardless of what option is selected at the zone or guest level.
You can enable to policy and time interval to check for resource changes in Administration. The default check.delay time interval checks for resource changes every 5 minutes (300 seconds). See the Administration guide for how to set the power consumption policy and edit the property values.
You can also manually recover the zones and logical domains. See Recovering Zones for more information about manual recovery.
See the sections Oracle Solaris Zones Server Pool, Oracle VM Server for SPARC Server Pool, and Oracle VM Server for x86 Server Pool for more information about automatic recovery options.
You can use tags to assign affinities and disaffinities to guests. When a guest is initially placed, when it is recovered, during a live migration, or during a server pool rebalance, a guest is placed on a host with other guests sharing its affinities and lacking its disaffinities wherever possible. If guests sharing an affinity cannot be placed on the same host, an alarm is raised.
Affinities and disaffinities are useful when placing certain guests on the same hosts, or on different hosts, is counterproductive. For example, if three guests contain parts of a three-tiered application, it is beneficial to keep them together, whereas two guests that are nodes of an HA cluster should not be placed on the same host.
In a case where a guest’s affinities and disaffinities cannot both be satisfied, preference is given to satisfying the disaffinities.
Guest affinities and disaffinities are managed by tags. You can use the Edit Tags action to add and edit tags, then save the changes to apply them to the asset.
You can edit the following tags on a guest to provide affinity and disaffinity information:
affinityTags: This tag identifies this guest for other guest’s affinities and disaffinities.
affinities: This tag identifies the guest’s affinities. A guest’s affinities must include all of its affinity tags.
disaffinities: This tag identifies the guest’s disaffinities.
You can provide more than one entry for a tag using a comma-separated list. Affinity and disaffinity tags are not case-sensitive. Enclosing quotation marks are ignored.
Note:
Do not give a guest an affinity and a disaffinity for the same affinity tag.Example 7-2 Example of Affinity and Disaffinity
For example, a guest with the following tags is a member of Group A. It should be recovered onto a host with other Group A guests, and not recovered onto a host with Group B guests or Group C guests.
affinityTags: Group A
affinities: Group A
disaffinities: Group B, Group C
Oracle Enterprise Manager Ops Center uses software libraries to store ISO images and guest metadata. Storage libraries are also used to provide storage disks for the guest operating systems and for guests' data.
The virtualization hosts in a server pool share the libraries associated with the server pool. The type of library you can associate with the server pool depends on the type of virtualization hosts in the server pool and on whether all members of the server pool can access the storage resource.
See Storage Libraries for Virtualization and the Software Libraries chapter of the Oracle Enterprise Manager Ops Center Configuration Guide for more information about setting up these libraries.
Oracle Enterprise Manager Ops Center provides group related networking components, such as fabrics, and networks, such as network domains. By default, all managed and declared networks in Oracle Enterprise Manager Ops Center are placed in the default network domain. Associating a virtualization host or a server pool with the default network domain does not require a fabric connection. All of the networks in the default network domain are available to be attached to the server pool.
You can create network domains. Associating a server pool with a user-defined network domain requires connection between the physical interface of each server in the pool and each fabric in the network domain. Only networks assigned to the user-defined network domain are available to be attached to the server pool.
Note:
Before attaching one or more private ethernet networks to a server pool, verify that all members of the pool have access to the private networks. When all members do not have access, you might encounter inconsistent guest creation and network information.
You can use server pools with Oracle SuperCluster systems:
When you attach public networks to a server pool, the server pool members can belong to different Oracle SuperCluster systems.
When you attach private (internal) networks to an Oracle SuperCluster server pool, the members must all belong to the same Oracle SuperCluster system.
Note:
Do not create server pools using members from more than one Oracle SuperCluster systems and attach private, or internal, networks to the systems.
Before creating a server pool, plan your networks and network domains that you must attach to the server pool. See Networks for Virtualization for more information about creating network domains and managed networks.
The following information is covered in this section:
You can pool the Oracle VM Server for SPARC resources and manage your logical domains. The Oracle VM Server for SPARC can be configured and running with I/O domains and root domains.
Oracle VM Servers for SPARC must meet the following requirements to be added to a server pool:
You are able to place Oracle VM Servers of different CPU types in the server pool; however, you might lose the ability to migrate the logical domains between virtualization hosts with different CPU architectures. When you want to live migrate the logical domains, ensure that you create server pools of compatible CPU types. Cold migration and automatic guest recovery are available by using the shutdown/detach operation of the logical domain. Plan your servers that must be pooled.
Must be running a supported version of Oracle VM Server for SPARC. The minimum supported version for server pools is Oracle VM Server for SPARC version 1.2.
Must be running a supported version of Oracle Solaris operating system. The minimum supported version for server pools is Oracle Solaris 10 10/09 OS and must meet specific patch and firmware requirements.
Plan the network that must be attached to the server pool. Either place those networks in the user-defined network domain or use the default network domain. You can connect to a network multiple times. The multiple network connection allows you to create IPMP or aggregate the links in the logical domain.
The storage libraries must be associated with the server pool to store logical domain metadata, ISO images, and for virtual disk storage of logical domains. You can associate file system and block storage libraries with the server pool. Filesystem storage includes the NAS storage libraries. Block storage includes the SAN, iSCSI, and Dynamic Storage libraries. Ensure that you have at least one NAS storage library assigned to the server pool. For using migration capabilities, the server pool must be on shared storage facility.
The following enhancements are available for supporting Oracle VM Server environments created outside Oracle Enterprise Manager Ops Center and advanced I/O domain configurations. Some of the enhancements are available from 3.0 or higher version:
You can add Oracle VM Servers to the server pool even if they have logical domains configured and running.
You can select the servers in the server pool to create I/O domains and root domains. The server pool will not be available as a target.
Discover and manage user-configured Oracle VM Server for SPARC environments. The metadata of logical domains are stored in the local library of the Control Domain. To enable logical domain migration, you must the move the metadata to a NFS storage.
Oracle VM Servers of different CPU type can be pooled together. The migration is not possible for logical domains between incompatible servers. Still, you can perform cold migration, that is shutting down the domain and starting on other servers in the server pool. Also, the guest recovery can be performed as it is often possible to start the domain on another server even though live migration is not possible.
You can configure exclusive access of I/O domain and root domain resources to other logical domains. The exclusive access prevents creation of zones in the I/O domain or root domain OS and the global zone is not available to placed in a zones server pool.
You can delete a server pool even if the logical domains are attached to the Oracle VM Servers.
Placing an Oracle VM Server in the server pool does not remove any existing network and storage resource that are already attached to it.
Deleting a server pool also does not unconfigure all the Oracle VM Servers of the attached network and storage resources. All the attached resources are retained and only the server pool is deleted.
The server pool policies that are applicable for Oracle VM Server for SPARC are as follows:
Placement Policy
Set the CPU utilization threshold. Place the logical domains on the Oracle VM Server for SPARC that has the lowest relative load.
Set the CPU and memory allocation threshold. Place the logical domains on the Oracle VM Server for SPARC allocated with lowest CPU and memory resources.
Set the CPU utilization threshold. Place the logical domains on the minimum number of Oracle VM Servers to minimize the power consumption.
Automatic Balancing
Set the automatic load balancing. Logical domains are migrated automatically in the server pool whenever the thresholds are exceeded. Schedule the automatic load balancing to take place at definite time interval.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some Oracle VM Server host servers are freed up from the logical domains and powered off to minimize power consumption.
Set whether administrator approval is required for migrating the logical domains.
You can enable automatic recovery for a logical domain in the following scenarios:
During logical domain creation, you can enable automatic recovery option and set the value for priority of recovery. See Creating a Guest Domain Profile for more information about setting the priority of recovery while creating a logical domain profile.
For an existing logical domain, use the options Enable Automatic Recovery and Disable Automatic Recovery to set the automatic recovery. Also, you can edit the attributes of a logical domain to set the value for priority of recovery. See Automatic Recovery of Logical Domains for more information.
See Automatic Recovery for the recovery process of the logical domains when a server fails in the server pool.
In an Oracle VM Server for SPARC server pool, the option to power-off the failed server and recover the logical domains is enabled by default. When a server fails, Oracle Enterprise Manager Ops Center tries to power-off the failed server, ensure that it is stopped and then initiates the automatic recovery of the logical domains. The logical domains that are configured for automatic recovery are recovered and restarted on other servers in the server pool.
When the option to power-off the failed server is enabled and Oracle Enterprise Manager Ops Center cannot power-off the failed server, that is the Service Processor is also unreachable, the automatic recovery of the logical domains does not take place. This is to avoid any potential data corruption during the recovery process of the logical domains.
When Oracle Enterprise Manager Ops Center detects that the failed server is unreachable, the logical domains that were running in the server disappear from the Navigation pane in the UI until they are recovered to other servers in the server pool.
The metadata of disappeared logical domains is still available in the storage libraries. The logical domains that are not configured for automatic recovery or the logical domains that cannot be recovered re-appear in the UI in the following scenario:
When the failed server is restarted, the logical domains are booted and attached to the server again.
When you have followed the manual recovery procedure and deleted the failed server from Oracle Enterprise Manager Ops Center, the logical domains re-appear in the list of Shutdown Guests in the Server Pool. You can start them on other server in the server pool. See Recovering Logical Domains for the logical domains recovery.
Table 7-3 provides a quick view of different conditions that exist for recovering a logical domain.
Table 7-3 Recovery of Logical Domains
Conditions | Result | |
---|---|---|
Logical domains |
|
The logical domain is restarted and placed in another server in the server pool. If there are no server available in the server pool, then you must manually recover the logical domains. See Recovering Logical Domains for more information. |
Logical domains |
|
The automatic recovery of the logical domain does not take place. This is to avoid any data corruption during the recovery procedure. You must manually recover the logical domains. |
Logical domains |
|
Oracle Enterprise Manager Ops Center proceeds to recover the logical domains configured to automatically recover without trying to power-off the failed server from its processor. This can lead to data corruption if the failed server is just isolated from the network point of view, and has still logical domains writing to disks. |
Logical domains |
|
Follow the procedure in Recovering Logical Domains to recover the logical domains. Only after this the logical domains appear in the Shutdown Guests list in the UI. You can restart them on the available server in the server pool. |
There are certain conditions in which the network tagging mode must be selected while creating a server pool. Review the following conditions and scenarios before deciding on the network tagging mode when you create an Oracle VM Server for SPARC server pool:
You can select networks without VLAN ID. The UI does not provide the option to select Tagged or Untagged mode.
You can select to associate and configure the networks with VLAN ID in Tagged mode.
You can select to associate and configure the networks with VLAN ID in Untagged mode.
You can select to configure the networks in mixed tagging mode in the server pool. For example, you can attach the network N1 with VLAN ID = 100 in tagged mode with the server S1 and in untagged mode for server S2. Refer to Mixed Network Tagging Mode Configurations in Server Pool for more detailed information.
You can attach networks whose VLAN ID is similar to another network already connected to the servers. For example, a server S1 is already connected to network N1 with VLAN ID = 100, then while creating the server pool with S1 as the member of the pool, you can also attach a network N2 with VLAN ID =100.
You can edit the VLAN ID of a network when you are attaching the network in Tagged mode for the first time.
When you can edit the VLAN ID of the network, you cannot enter -1 as the value for the VLAN ID.
If the selected network with a VLAN ID is already connected to the selected assets in Tagged mode, then you cannot edit the VLAN ID and make another connection.
You cannot make multiple network connections to a member of the server pool over the same network in both tagged and untagged modes. The mode can be either in tagged or untagged mode only. For example, if you attach network N1 with VLAN ID =100 for the first time to server S1 in Tagged mode, then you cannot make another connection to the same network N1 in Untagged mode. Every other connection with network N1 must always be in Tagged mode for server S1.
If the selected members of the pool are already connected to network N1 with VLAN ID =100, then you cannot select the same network with different VLAN ID to be connected for the server pool.
In an Oracle VM Server for SPARC server pool, you must group the servers that are homogenous in the network VLAN tagging mode to avoid any network outage. When you create a server pool, you can select the network tagging mode for the networks configured with VLAN ID to be attached to the Oracle VM Servers. You must group the servers in a way that all the Oracle VM Servers in the pool are either untagged or tagged mode for each network connection.
If the server pool is mixed with servers attached to networks in untagged and tagged mode, then there is a possibility that you will lose the network configuration of the logical domain OS.
When a logical domain is using an untagged VLAN, you cannot migrate the guest to a server that uses a tagged VLAN.
For example, in the following scenario, there is a server pool with two Oracle VM Servers CD1 and CD2. CD1 has a logical domain LD1. A network 192.0.2.0/24 with VID=100 is attached to CD1 in tagged mode. When you attach the network on the CD1, a VNIC is created with VID=100 and configured with an IP address to reach the network 192.0.2.0/24. The same network is attached to CD2 in untagged mode. LD1 is also connected to the network 192.0.2.0/24. When you migrate the LD1 from CD1 to CD2 or when you start the shutdown-detached LD1 on CD2, the LD1 will successfully start on the CD2. However, the OS of the LD1 is not able to reach the network 192.0.2.0.24 with VID=100 because, the VNIC is created with VID=100. To re-establish the network connection, you must change the configuration of the VNIC in the OS of the Oracle VM Server CD2.
To avoid such issues, create server pools with homogenous network tagging mode. Or to maintain mixed network configurations, check with your administrator for maintaining the network connection of the logical domain operating system.
Oracle Enterprise Manager Ops Center launches a wizard that collects information about the servers, storage and network resources, and placement policies to create a server pool.
With the support of I/O domains in Oracle VM Server for SPARC, you can assign network interfaces from I/O domains or root domains to connect to the network. The Create Server Pool wizard provides the option to select the network interfaces from other domains.
The wizard provides the option to select the SR-IOV enabled network interfaces to connect to the network.
The following information is covered in this section:
You can now pool Oracle Solaris Zones resources and manage your zones. Oracle Solaris Zones must meet the following requirements to be added to a server pool:
All the assets in the pool must have compatible architecture for supporting guest migration. For a global zone to be added to the server pool, it must have the same release or must be at least Oracle Solaris 10 10/08 version. This ensures that the zones can be migrated to a global zone in the server pool.
The global zones must be in healthy state.
The global zones must not be associated with a server pool.
The global zones must not be placed in maintenance mode.
Note:
Oracle Solaris 11 shared IP zones should not be part of a server pool. You cannot perform migration, connect networks or add storage resources to shared IP zones.
The network deployment for zones server pool vary depending on the Oracle Solaris OS version.
For a stand-alone Oracle Solaris 10 OS, you can attach the network in either shared IP or exclusive IP mode. Whereas, in a server pool, you can attach a network only in shared IP mode for Oracle Solaris 10 OS. You cannot make multiple connections to a network.
For Oracle Solaris 11 OS, the network is always attached in exclusive IP mode. In a server pool, the network for Oracle Solaris 11 OS is always deployed in exclusive IP mode. You can make multiple connections to a network.
When you want to have a server pool with mixture of Oracle Solaris 10 and 11 OS, then you cannot make multiple connections to a network. So, plan your server pools with compatible Oracle Solaris OS versions.
There are certain conditions in which the networking configuration that must be selected while creating a server pool. Refer to the following scenarios for deciding on the networking configuration when you create an Oracle Solaris Zones server pool:
You can select networks without VLAN ID. The UI does not provide the option to select Tagged or Untagged mode.
You can select to associate and configure the networks with VLAN ID in Tagged mode.
You can select to associate and configure the networks with VLAN ID in Untagged mode.
You can select to configure the networks in mixed tagging mode in the server pool. For example, you can attach the network N1 with VLAN ID = 100 in tagged mode with the server S1 and in untagged mode for server S2. Refer to Mixed Network Tagging Mode Configuration for more detailed information.
You can attach networks whose VLAN ID is similar to another network already connected to the servers. For example, a server S1 is already connected to network N1 with VLAN ID = 100, then while creating the server pool with S1 as the member of the pool, you can attach a network N2 with VLAN ID =100.
You can edit the VLAN ID of a network when you are attaching the network in Tagged mode for the first time.
When you can edit the VLAN ID of the network, you cannot enter -1 as the value for the VLAN ID.
If the selected network with a VLAN ID is already connected to the selected assets in Tagged mode, then you cannot edit the VLAN ID and make another connection.
You cannot make multiple network connections to a member of the server pool over the same network in both tagged and untagged modes. The mode can be either in tagged or untagged mode only. For example, if you attach network N1 with VLAN ID =100 for the first time to server S1 in Tagged mode, then you cannot make another connection to the same network N1 in Untagged mode. Every other connection with network N1 must always be in Tagged mode for server S1.
If the selected members of the pool are already connected to network N1 with VLAN ID =100, then you cannot select the same network with different VLAN ID to be connected for the server pool.
When you attach network to zones server pool, you can specify the network tagging mode for the networks configured with VLAN ID. You must ensure that the server pools have global zones either in tagged or untagged mode. If the server pool has mixed network tagging modes for the global zones, there might be any network outage issues for the zones depending on the action performed upon them.
If you want to maintain mixed network tagging configuration in your server pool, check with your administrator for re-establishing the network connection of the zones when the network outage occurs.
The server pool policies that are applicable for Oracle Solaris Zones are as follows:
Placement Policy
Place the zones on the global zone that has the lowest relative load. Set the CPU utilization threshold.
Place the zones on the global zone allocated with lowest CPU and memory resources. Set the CPU and memory allocation threshold.
Place zones on minimum number of global zones to minimize the power consumption. Set the CPU utilization threshold.
Automatic Balancing
You can select the automatic load balancing so that the zones can be migrated automatically in the server pool whenever the thresholds are exceeded. You can schedule the automatic load balancing to take place at definite time interval.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some servers are freed from the zones and powered off to minimize power consumption.
You can set whether the approval is required from the administrator for migrating the zones.
Automatic Recovery
You can set the automatic recovery for the zones in the following scenarios:
During zone creation, you can enable automatic recovery of the zones. You can set the value for priority of recovery. Zone with a highest priority is migrated first.
For existing zones, use the option Enable Automatic Recovery and Disable Automatic Recovery to set the automatic recovery. You can edit the zone configuration to modify the value for priority of recovery.
In an Oracle Solaris Zones server pool, you can specify that the automatic recovery of a failed server must start first with an attempt to power off the failed server, ensure that it is stopped and then initiate the automatic recovery of the zones.
When a non-global zone is recovered from a failed global zone onto another global zone in the server pool, ownership of the non-global zone is transferred to the new global zone. If the failed global zone is restarted, its VC agent checks the ownership of its non-global zones, and deletes any non-global zone for which it no longer has ownership. You can change this default behavior to preserve the non-global zone on the original global zone.
You can group one or more Oracle VM Servers in an Oracle VM Manager and create server pools. When you create a server pool for Oracle VM Server for x86 systems, you must have the following information defined:
Cluster file system
Server pool master
Virtual IP address for the server pool master
Shared access to the server pool resources is a must for providing for high availability for the virtual machines running in the Oracle VM Servers of the server pool. This is achieved by cluster file system OCFS2 which allows multiple Oracle VM Servers to access the same disk at the same time. OCFS2 ensures that the Oracle VM Servers in a server pool can access and modify resources in the shared repositories in a controlled manner.
When you create a server pool, you must specify the server pool file system for the cluster heartbeat and other cluster information. The file system can be NFS shares or LUNs of iSCSI or SAN based storage servers. Oracle VM formats the server pool file system as OCFS2 file system.
In Oracle Enterprise Manager Ops Center, the cluster is always enabled by default. The cluster configuration is pushed out to all the Oracle VM Servers in the server pool and the cluster heartbeat starts when the server pool is created. You can set a separate network for this cluster heartbeat. See Manage Networks for more information about setting up the networks and their role for an Oracle VM Server.
An Oracle VM Server is internally elected as server pool master.
You cannot set the role for an Oracle VM Server. When the elected Oracle VM Server fails, the role is set for another Oracle VM Server in the server pool. The virtual IP address provided while creating the server pool is assigned to the Oracle VM Server that has been elected as server pool master.
The server pool policies that are applicable for Oracle VM Server for x86 are as follows:
Placement Policy
Places the virtual machines on the Oracle VM Server that has the lowest relative load. Set the CPU utilization threshold.
Places the virtual machines on minimum number of Oracle VM servers to minimize the power consumption. Set the CPU utilization threshold.
Automatic Balancing
You can select the automatic load balancing so that the virtual machines can be migrated automatically in the server pool whenever the thresholds are exceeded.
When you enable automatic load balancing for Oracle VM Server for x86 server pool, the server pool is checked continuously for the selected placement policy. When the threshold exceeds, the virtual machines are migrated from one Oracle VM Server to another.
When you have selected power minimization and automatic load balancing policy, and the servers in the server pool are not overloaded, some Oracle VM Server host servers are freed from the logical domains and powered off to minimize power consumption.
When the servers are overloaded and there are no other servers in the pool to host the virtual machines, then the policy starts a powered-off server using its Wake-on-LAN capability and live migrate the virtual machines to take up the load. The Wake-on-LAN capability must be enabled on the BIOS of the Oracle VM Server.
Automatic Recovery
When an Oracle VM Server fails, the virtual machines are migrated to another Oracle VM Server in the server pool. The automatic recovery for a virtual machine is set in the following scenarios:
During virtual machine creation, select the option Enable High Availability. This ensures that the virtual machines are migrated when an Oracle VM Server fails.
For an existing virtual machine, use the option Enable or Disable Automatic Recovery to set the automatic recovery.
You must monitor the requirements of the server pool so that there are resources to host the guests and run them efficiently. You might need to add more storage and network resources to run the guests. Also, the server pools gives the infrastructure support for the virtual datacenters.
Oracle Enterprise Manager Ops Center provides the following management functions to manage the resources in the server pool:
You can edit the following parameters and attributes of a server pool:
Name
Description
Placement policy, including CPU Utilization
Auto balancing policy
Enable or disable automatic recovery option.
Automatic Recovery Authorization.
Automatic Recovery Number of Retries.
For Oracle Solaris Zones server pool, you can edit the following additional parameters:
Check server reachability interval.
Upload scripts to manage the file systems on unmanaged storage. See Step 10 in Oracle Solaris Zones Server Pool for more information.
For Oracle VM Server for SPARC server pool, you can edit the following additional parameters:
Migration networks.
Enable or disable automatic recovery to power off a failed server from Service Processor, given capabilities, before automatic recovery of attached logical domains.
Check servers reachability.
Perform the following to modify the parameters of a server pool:
You can add more virtualization hosts to a server pool when you require more CPU and memory resources to take up the load in the server pool.
Depending on the virtualization type, you can add global zones, and Oracle VM Servers of x86 and SPARC architecture.
The list of global zones that are displayed to be added to the server pool has the following characteristics:
Same release as the existing global zones in the pool or at least Oracle Solaris 10 10/08 OS or higher version
Not associated with a server pool
In a healthy state and not placed in maintenance mode
When the server pool has Oracle Solaris 11 OS and multiple network connections, then the list of available global zones is limited to only Oracle Solaris 11 OS. You cannot add Oracle Solaris 10 OS as you cannot make multiple network connections to it.
Load balancing in the server pool requires the zones to be migrated within the pool. Therefore, ensure compatibility for migration within the pool when you select the members.
Refer to Network Tagging Mode Conditions for more information about the network tagging conditions while adding an asset to the server pool.
The list of Oracle VM Servers displayed to be added to the server pool have the following characteristics:
Not associated with a server pool
In a healthy state and not placed in maintenance mode
Refer to Network Tagging Mode Conditions for more information about the networking configurations when you add Oracle VM Server to the server pool.
You can add Oracle VM Servers to a server pool.
The list of Oracle VM Servers displayed to be added to the server pool have the following characteristics:
Owned by the Oracle VM Manager
Not associated with a server pool
In a healthy state and not placed in maintenance mode
Have no virtual machines in the running, shutdown or suspended states
You can remove the virtualization hosts from the server pool. For zones and Oracle VM Server for SPARC, you can remove them from the server pool even when there are zones and logical domains running on them. All the network and storage resources are not unconfigured. Instead, the resources are retained as it is in the virtualization hosts.
To remove Oracle VM Server for x86 from a server pool, you must stop or migrate the virtual machines that are running in it.
All networks in a network domain that is associated with a server pool are available to the members of the server pool. To associate a user-defined network domain with a server pool, there must be a connection between the virtualization servers and the fabrics in the network domain. The fabric connection is not required for the default network domain.
The default network domain includes all the networks in Oracle Enterprise Manager Ops Center. When you associate a server pool with the default network domain, all the networks in Oracle Enterprise Manager Ops Center are available for attaching with the server pool.
For a user-defined network domain, you must define the physical interfaces of each virtualization server in the server pool to connect to each fabric in the network domain. You can also bond the interfaces of the virtualization servers.
Note:
You cannot the bond the interfaces of Oracle VM Server for x86 servers.
When you associate with a user-defined network domain, only the assigned networks in the network domain are available for attaching with the server pool. Therefore, ensure that you assign the required networks to the domain to attach to the server pool.
After the network domain is associated with a server pool, you can attach the networks in the network domain. Depending on the type of virtualization, the multiple connection to a network is allowed.
For Oracle VM Server for SPARC, you can make multiple connections to a network. Refer to the Oracle VM Server for SPARC for more information.
For Oracle Solaris Zones, you can make multiple connections to a network when the server pool contains only Oracle Solaris 11 OS. For Oracle Solaris 10 OS, you can make only one connection. Also, when the server pool has mixture of Oracle Solaris 10 and 11 OS, then also you can make only single connection.
Refer to Network Tagging Mode Conditions for more information about selecting the correct tagging modes when you attach the network to Oracle Solaris Zones and Oracle VM Server for SPARC.
This section lists the types of storage libraries that you can associate with the server pool.
Table 7-4 Supported Libraries
Oracle VM Storage Repositories | NAS Libraries | SAN Fibre Channel Libraries | SAN iSCSI Libraries | |
---|---|---|---|---|
Oracle VM Server for SPARC |
No |
Yes |
Yes |
Yes |
Oracle VM Server for x86 |
Yes |
Yes |
Yes |
Yes |
Oracle Solaris Zones |
No |
Yes |
Yes |
Yes |
When you associate libraries with the server pool, only the libraries that are reachable from the virtualization hosts in the server pool are listed. Depending on the type of library, you either provide virtual disks or LUNs to the guests' storage.
Except for Oracle VM Server for SPARC server pool, the Associate Libraries action for zones and Oracle VM Server for x86 server pool displays the available libraries. Select the libraries and click Associate to associate the libraries with the zones or Oracle VM Server for x86 server pool.
For Oracle VM Server for SPARC server pool, the procedure to associate and disassociate the storage libraries are described as follows.
The guests refer to the logical domains, zones or the virtual machines that can be created in the virtualization hosts. According to the virtualization type of the server pool, you have the following options:
Oracle VM Server for SPARC: Create Logical Domains
Oracle Solaris Zones: Create Zones
Oracle VM Server for x86: Create Virtual Machines
These options trigger the deployment plans for the corresponding guests. Refer to the following chapters to refer to the profile and plan creation for the guests:
When you place a virtualization host in a server pool, the option to migrate one or more guests is enabled.
The following options become available for the virtualization hosts:
Oracle Solaris Zones: Migrating Zones
Oracle VM Server for SPARC: Migrating Logical Domains
Oracle VM Server for x86: Migrating Virtual Machines
You can migrate multiple zones from a global zone which is in a server pool. Migrate the zones to an individual global zone or other zones server pool.
When you migrate zones, the source and the target global zone must be compatible. The target global zones must have the following characteristics:
Running at least Oracle Solaris 10 8/07 OS.
Can access all the storage libraries associated with the zone.
Apart from compatibility, the target and the source global zone might have differences in the patches and packages installed on them. Choose to update the zone to match the patches and packages of the target global zone.
Note:
You cannot downgrade the patches and packages of the zone. The migration fails in such scenario.
You can also force the migration of the zone without updating the patches and packages. Select the update options while migrating the zone.
Migrate multiple logical domains from an Oracle VM Server to another Oracle VM Server in the same server pool. You cannot do cross server pool migration.
Only from Oracle VM Server for SPARC 2.1 version, live migration of logical domains is supported. For earlier releases, it is cold migration. The logical domains are shut down and then restarted on the target server.
See Migrate Logical Domains in Oracle VM Server for SPARC for more conditions in migrating a logical domain.
For Oracle VM Server for x86 systems, you can migrate virtual machines only within a server pool. Live migration of virtual machines is supported.
The eligible Oracle VM Servers have the following characteristics:
Owned by the Oracle VM Manager.
Placed in the same server pool as the source Oracle VM Server.
Have required resources to host the virtual machines.
Identical in machine make and model of the source Oracle VM Server.
When you have selected to balance server pool resource manually, use the Balance Resources action to check for the under-utilization or over-utilization of resources.
Select a server pool and click Balance Resources in the Actions pane to display the current utilization of the virtualization hosts in the server pool. When balancing is required according to the placement policy, Oracle Enterprise Manager Ops Center displays a list of target server that can accept migrated guests. Click Balance Resources to start the migration job.
A comprehensive view and analysis of the server pool resource usage are displayed in the Analytics tab of the selected server pool.
The Analytics tab displays the following information about a server pool:
CPU
Memory
Storage
Guests
Power Consumption
History
CPU
The CPU sub tab displays information related to CPU resources allocated to the server pool and the percentage of usage:
CPU Allocation: You can select to view the CPU allocation as number of threads or cores for an Oracle Server for SPARC server pool. Whereas, for a zones pool, the CPU allocation is represented only as threads.
CPU Usage: The CPU usage for a varied time range is available. The graph displays the CPU usage in percentage for a selected time range. It also provides the current CPU usage and average CPU usage for the selected time range.
CPU Member Details: The allocated CPU resource and the usage by each member of the server pool are displayed.
Memory
The Memory sub tab displays the memory resources allocated to the server pool and its usage by the servers in the pool.
Memory Allocation: The memory resource allocated to the server pool and the available memory for use are displayed.
Memory Usage: The memory usage in the server pool for a selected time range is displayed in a graph.
Memory Member Details: The memory resource used by each server in the server pool is displayed.
Storage
The Storage sub tab displays the allocated storage resource and the available storage resource in the storage pool.
Storage Allocation: The storage resources available in the server pool from different types of library are displayed.
Storage Usage: The storage usage for a selected time range is displayed as a graph.
Storage Library Details: The storage resource available through different storage libraries is displayed.
Guests
The Guests sub tab provides information about the number of guests running in the server pool and the details of the guest location. It also provides the option to calculate the number of guests that can be created in the server pool for a given guest parameters.
Guest Distribution: The number of guests running in the server pool and the servers on which it is distributed are displayed.
Guest Count: The number of guests running in a specified time range is displayed in a graphical format.
Guest Member Details: The number of guests running in each server of the server pool and the potential number of guests that the server can support are displayed.
Guest Template: You can specify a guest parameters such as CPU, memory and storage resource. Click Calculate to find the potential number of guests that can be created in the server pool with the defined configuration.
Power Consumption
The Power Consumption sub tab displays the power consumption details of the server pool. It displays information about the total power consumed and power consumed by each server of the server pool.
Power Consumption Distribution: The total power consumed by the servers in the server pool and the power consumption by each server are displayed.
Power Consumption History: The power consumption history for a specified time range is displayed.
Power Consumption Details: The power consumed by each server in the server pool is displayed.
History
The History sub tab displays the graphical representation of the following resource utilization in the server pool:
CPU utilization
CPU cores allocation
CPU threads allocation
Memory allocation
Storage allocation
When the server pool does not meet the policies and rules set, warning or critical incidents are raised accordingly. For example, when the servers in the pool are overloaded and need to migrate a guest to another server.
Incidents also provide information about the result of a server pool status on automatic load balancing.
The guests are recovered according to the policy set in the server pool for automatic recovery. The incidents are raised that describes the guests that are recovered and in which servers they are recovered in the server pool.
Incidents are also raised when there are no resources or servers available in the server pool for recovering the guests. Incidents provide more information about the scenario.
Refer to the Related Resources for Server Pools section for more information about handling and viewing incident messages.
Use the option Delete Server Pool to release the virtualization hosts back to stand-alone state. The type of server pool imposes some limitations to delete a server pool.
You can delete the Oracle VM Server for SPARC server pool even when the logical domains are attached to it and running. There is no restriction for deleting an Oracle VM Server for SPARC server pool to shut down the logical domains. All the network and storage resources are not unconfigured from the Oracle VM Servers in the server pool. The network and storage resources are retained by the servers.
For a zones server pool, you can delete the server pool and the zones continue to be attached to the global zone. Oracle Enterprise Manager Ops Center does not provide any restrictions to shut down the zones before deleting a zones server pool.
Shut down all the virtual machines and remove all Oracle VM Servers except for the Master Oracle VM Server before deleting the server pool. The virtual machine metadata and virtual disks are stored in the storage library. The virtual machine details are not lost and you can start the virtual machine.
For instructions in performing actions or to learn more about the role of this feature, go to one of the following resources:
See the Asset Management chapter of the Oracle Enterprise Manager Ops Center Configure Reference for information about discovering and managing assets.
See the Incidents chapter of the Oracle Enterprise Manager Ops Center Operations Reference for more information about handling and viewing incident messages.
See the Oracle Enterprise Manager Ops Center Administration Guide for information about users, roles, and the power consumption policy.
See the following how to documentation in the Deploy How To tab in the library at http://docs.oracle.com/cd/E59957_01/nav/deploy.htm
for end-to-end examples.
Oracle Enterprise Manager Ops Center Create a Server Pool for Zones
Oracle Enterprise Manager Ops Center Create a Server Pool for Oracle VM Server for SPARC
See the Oracle Enterprise Manager Ops Center Explore Your Server Pools how to documentation in the library at http://docs.oracle.com/cd/E59957_01/nav/operate.htm
for an end-to-end example of how to manage server pools.
For current discussions, see the product blog at https://blogs.oracle.com/opscenter
.