This chapter describes how to manage and monitor a virtualized Oracle Exadata Database Machine with DB clusters using Oracle Virtual Machine (OVM) for x86 and Oracle Exadata Storage Servers in Exadata plug-in 12.1.0.6 and later.
The Exadata plug-in discovers, manages, and monitors virtualized Exadata Database Machine in conjunction with the Virtualization Infrastructure plug-in. For details about this plug-in, see the "Direct Monitoring of Xen Based Systems" chapter of the Oracle® Enterprise Manager Cloud Administration Guide:
http://docs.oracle.com/cd/E24628_01/doc.121/e28814/direct_monitor_cs.htm#EMCLO531
The following sections describe how to discover a virtualized Exadata Database Machine and other supported targets:
Once you have completed the discovery of a virtualized Exadata Database Machine, continue with the configuration steps outlined in Post-Discovery Configuration and Verification.
The physical server (physical Oracle Server target), Dom0
(Virtual Platform target), and DomU
(virtual Oracle Server target) are discovered and monitored by the Virtualization Infrastructure (VI) plug-in.
The Exadata discovery can work with the VI plug-in by the physical server, Dom0
, and DomU
are discovered using the VI plug-in before Exadata Database Machine discovery. During Exadata discovery, the discovery code looks up the existing Virtual Platform target that corresponds to the Dom0
of the compute node.
The Exadata discovery flow with VI plug-in integration includes the following checks:
Check whether the Exadata Database Machine is virtualized based on the configuration metric of the host target of the discovery agent.
Check whether the VI plug-in is deployed. If not, you will be prompted to deploy it as described in the "Direct Monitoring of Xen Based Systems" chapter of the Oracle® Enterprise Manager Cloud Administration Guide:
http://docs.oracle.com/cd/E24628_01/doc.121/e28814/direct_monitor_cs.htm#EMCLO531
With virtualized Exadata, one Exadata Database Machine target will be created for each physical Database Machine instead of one DB Machine target for each DB cluster deployed through OEDA. Compute nodes, Exadata Storage Servers, InfiniBand switches, compute node ILOM, PDU, KVM, and Cisco switches targets are discovered by the Exadata plug-in.
With only the compute nodes virtualized, the physical servers, Dom0
and DomU
, are monitored by the Virtualization Infrastructure (VI) plug-in. The Exadata plug-in is integrates with the VI plug-in for discovery and target management as discussed below.
The hardware targets in virtualized Exadata are discovered in almost the same way as physical Exadata (see Discovering an Exadata Database Machine) except as noted below:
The Dom0
of the compute nodes are discovered using ibnetdiscover
. Compute node ILOM to compute node mapping and VM hierarchy are obtained from the VI plug-in.
Exadata Storage Servers are discovered using ibnetdiscover
instead of kfod
. Therefore, there is no need to specify the Database Oracle Home during discovery.
InfiniBand switches are discovered using ibnetdiscover
.
Compute node ILOM, PDU, and Cisco switch are discovered based on the databasemachine.xml
schematic file.
Cluster and Database Target Discovery
The cluster and database target discovery is similar to the physical Exadata case (see Discovering Grid Infrastructure and RAC). The only difference is that the Enterprise Manager agents needed to be deployed on the DomU
of the Database cluster before the cluster, ASM, and database targets can be discovered using the DB plug-in.
Agent Placement
The primary and backup Enterprise Manager agents monitoring the Exadata hardware targets should be deployed in two dedicated DomU
that will not be suspended or shut down, and are ideally on different physical servers to ensure high availability.
For static virtual machine configurations, the Enterprise Manager agents that are used to monitor the Database clusters can be used to monitor the Exadata hardware.
To discover a virtualized Exadata Database Machine:
From the Setup menu, select Add Target then Add Targets Manually.
On the Add Targets Manually page, select Add Using Guided Process (Figure 6-1):
In the Add Using Guided Process pop-up window, select Oracle Virtual Platform and click Add (Figure 6-2):
Note:
This option will be available only if the Virtualization Infrastructure (VI) plug-in has been deployed.If the plug-in is not deployed, then the Oracle Virtual Platform option will not appear. Deploy the VI plug-in as described in the "Direct Monitoring of Xen Based Systems" chapter of the Oracle® Enterprise Manager Cloud Administration Guide:
http://docs.oracle.com/cd/E24628_01/doc.121/e28814/direct_monitor_cs.htm#EMCLO531
On the Discover Oracle Virtual Platforms page, register Oracle Virtual Platforms with Enterprise Manager. Target credentials and/or monitoring agent can be specified for each target. If set, these override the defaults for that target.
Set the Default Monitoring Agent and Credentials (Figure 6-3). Click the magnifying glass icon to search for the monitoring agent location. For the Credential Properties, enter root
for the Username and provide the host's root password.
Note:
A non-root user withsudo
privilege can also be used. Refer to the VI plug-in documentation for the necessary setup steps.Add a Host Name or IP address. Click Add and enter the fully qualified domain name (FQDM) or IP address for the host in the pop-up window (Figure 6-4). If you have more than one, each entry must appear on a separate line.
Click Add to close the pop-up window.
Once added, the Hostname and IP address fields will update as shown in Figure 6-5:
Click Submit. A job is submitted to register the Oracle Virtual Platform, which creates the virtual targets. Approximately five (5) minutes is required to complete the job. Click Job Details to check the status, or click Close to close the window (Figure 6-6):
Once the job completes, then discover the Exadata Database Machine. From the Setup menu, select Add Target then Add Targets Manually.
On the Add Targets Manually page, select Add Using Guided Process (Figure 6-7):
For the Target Types drop-down menu, select Oracle Exadata Database Machine (Figure 6-8):
On the Oracle Exadata Database Machine Discovery page, select the Discover a new Database Machine and its hardware components as targets option and click Discover Targets.
On the Discovery Inputs page, click the magnifying glass icon to search for an agent in a DomU
that resides in the DB Machine to be discovered. Do not search for a virtual platform (Dom0
) because you cannot install an Enterprise Manager Agent in Dom0
. Just like the physical Exadata case, you are specifying a host on which the schematic file can be read. Usually, it will be the same host as the discovery agent. It will not be the Dom0
In the Schematic Files section, select a host on which the schematic file can be read, and set the host credential that can read the schematic file. Click Set Credential to set the log in and password (Figure 6-9) for the host you registered in the previous steps. Click OK.
Click Next.
On the InfiniBand Discovery page, set the nm2user
credential. Select the New option and enter nm2user for the Username field and provide a password as shown in Figure 6-10:
Click Next.
Note:
A known issue about missing CLIENTNAME and CLIENTIP information will cause an Error pop-up message. Click OK to dismiss the pop-up and to continue with the guided discovery.On the Prerequisite Check page, verify the Status is successful, then click Next.
On the Components page, review the Exadata components that were discovered, then click Next.
On the Monitoring Agents page, review the names, locations, and backup locations of the Monitoring Agents. Click Next.
On the Agent Credential page, if the credentials are the same for all agents, then you can enter a new username and password or accept the default. Click Next.
If the credentials are different for all agents, then select the Different for all agents option and enter usernames and passwords for all hosts. Click Next.
On the Monitoring Credentials page, verify or enter the credentials for the following components:
Oracle Exadata Storage Server
InfiniBand Switch
ILOM
Click Test Connection for each component to verify the connection. If the credentials are different for each component, then provide the appropriate username and password.
Click Next.
On the SNMP Subscription page, enter the SNMP Community String (default is public) value for both Oracle Exadata Storage Server and InfiniBand Switch areas. For the InfiniBand Switch, provide the root credentials (Figure 6-11). Click Next.
On the Component Properties page, enter public for both Ethernet and PDU components (Figure 6-12). Click Next.
On the Review page, review the details of the following components:
System Target
Compute Node
Oracle Exadata Storage Server
InfiniBand Switch
Ethernet Switch
Compute Node ILOM
KVM
PDU
Click Submit to promote all targets. Allow for approximately five (5) minutes to process. Once complete, all targets will now be monitored by Oracle Enterprise Manager Cloud Control.
Note:
The VI plug-in regularly checks whether a VM is added or deleted. The Oracle VM instance targets are auto-promoted once they are detected by the VI plug-in. If you want to monitor the OS, ASM, and DB in the VM, you need to push the Enterprise Manager agent to the VM.Since agents run inside DomU
nodes and remotely monitor cells, ILOMs, switches, and PDUs, ensure that Primary and Backup agents are on physically different DomU
hosts. In case of a Dom0
outage, Enterprise Manager can continue to monitor such targets using an agent that is running on an active DomU
hosted by a different Dom0
that is still running. This will achieve continuous monitoring in case of a complete Dom0
outage.
For details on how agents monitor Exadata targets remotely, refer to Chapter 4, "Post-Discovery Configuration and Verification," for details.
Provisioning involves repeatable, reliable, automated, unattended, and scheduled mass deployment of a RAC Cluster including virtual machines (VMs), Oracle Database (DB), Grid Infrastructure, and ASM on Virtualized Exadata.
With the Exadata plug-in's virtualization provisioning functionality, you can:
To create a database cluster:
From the Database Machine target menu, select Provisioning, then select Create Cluster (Figure 6-13):
The Exadata Provisioning Wizard will display.
On the Exadata Provisioning: Cluster page, provide the information for:
Cluster Definition: Create a cluster name. Click Show existing clusters to display a list of database clusters already provisioned.
Virtual Machine Location: Select the servers on which you want to create the virtual machines. Select one or more from the Available Servers pane, then click the move button to move the selected server to the Selected Servers pane (Figure 6-14):
Storage Definition: Select the Exadata Storage Servers to be used by the virtual machines. Select one or more from the Available Storage Servers pane, then click the move button to move the selected server to the Selected Storage Servers pane:
Click Next.
On the Credentials page, set the credentials for:
Compute Server: Set the credentials for the root
user. From the Set Credentials drop-down, select All to apply the settings to all servers, or select one or more servers from the list and select Selected to apply the settings to only the selected servers.
In the Set Server Credential pop-up window, select an existing named credential or select New to create a new one.
Click Test Connection to verify that the credentials are properly set. If successful, the Test Credential Status will update to show Succeeded (Figure 6-16):
Exadata Storage Server: Set the credentials for the cell administrator.
In the Set Server Credential pop-up window, select an existing named credential or select New to create a new one.
Click Test Connection to verify that the credentials are properly set. If successful, the Test Credential Status will update to show Succeeded (Figure 6-17):
Click Next to define the Virtual Machines.
On the Virtual Machines (VMs) page, provide the following information:
Note:
When available, the Exadata Provisioning Wizard will pre-populate the fields with the appropriate defaults. You can change this information to suit your environment.Virtual Machine Definition: In this region (Figure 6-18), provide the details to define the VM:
Virtual Machine Size: Select Small, Medium, or Large. Click Customize to adjust the number of CPU cores and available memory.
Root password: Create a password in the two password fields.
Verify the Prefix, DNS, and NTP fields.
Time Zone: select the appropriate time zone from the drop-down list.
Software Locations: When available, the Exadata Provisioning Wizard will pre-populate the fields (Figure 6-19); otherwise, provide the following information:
Inventory Location
Grid Infrastructure Home
Database Home Location
Agent Installation Base Directory
Agent Port (optional)
Software Language (optional)
Operating System Users and Groups: Like the Software Location region, the Exadata Provisioning Wizard will pre-populate the fields except for the password fields (Figure 6-20). Otherwise, provide the following information:
User name: including the ID, password, and home directory.
DBA group name and ID.
OINSTALL group name and ID.
Click Next to provide the Network details.
On the Network page, specify the IP address, name and domain used for the Admin, Client and Private network. Provide the details for the following information:
Gateway and Domain Details: The domain and subnet mask details should already be supplied by the Exadata Provisioning Wizard. Enter a valid IP address for the Admin and Client gateway (Figure 6-21):
Virtual Machine 1 (Figure 6-22): For the first virtual machine in the cluster, the Exadata Provisioning Wizard will auto-complete the Prefix and Start ID field. You can enter an optional Suffix. Enter a valid IP address.
Generated Virtual Machine Name and IP (Figure 6-23): After you have entered the information for the first VM, click Generate to create the information for the other VMs in the cluster. You can always enter the details manually.
Once you have entered all the necessary information, click Validate IP at the top of the page to verify the IP addresses. Click Next to continue to the enter the details for Grid Infrastructure and to create the Initial Database.
On the Grid Infrastructure and Initial Database page, enter the details for:
Grid Infrastructure (Figure 6-24): Enter the details for the Cluster (SCAN name and port) and verify the Disk Group information. The Exadata Provisioning Wizard will pre-populate the details, but you can adjust them as needed.
Initial Database: Click the check box to create the initial database. Additional information for Database Identification and Administrator Credentials will be required as shown in Figure 6-25:
Click Advanced to expand the region for additional details for Memory Parameters, Processes, and Character Sets as shown in Figure 6-26:
Click Next to set the schedule.
On the Schedule page, the Exadata Provisioning Wizard will create the Deployment Instance value. Select a schedule start and notification options:
Schedule: Select to initiate the creation immediately or later. If you select later, then you will be prompted to select a date and time.
Notification: Select the notification statuses for which you will be notified.
Click Review to review the settings and initiate the job.
On the Review page, review the selection in the summary displayed. To change any section, return to the previous page and edit the selection.
Click Submit to begin the creation job.
To scale up a database cluster:
From the Database Machine target menu, select Provisioning, then select Scale Up Cluster.
The Exadata Provisioning Wizard will display.
On the Cluster page, enter the cluster name or click the Search icon to select a cluster from the list.
Select one or more from the Available Servers pane, then click the move button to move the selected server to the Selected Servers pane.
Click Next to set the credentials.
On the Credentials page, set the credentials for:
Compute Server: Set the credentials for the root
user. From the Set Credentials drop-down, select All to apply the settings to all servers, or select one or more servers from the list and select Selected to apply the settings to only the selected servers.
In the Set Server Credential pop-up window, select an existing named credential or select New to create a new one.
Click Test Connection to verify that the credentials are properly set. If successful, the Test Credential Status will update to show Succeeded (Figure 6-27):
Virtual Machines (Figure 6-28): Set the credentials for the DomU Host and Root and for the Cluster ASM and Database.
You have the option to use preferred credentials or to override the preferred credentials.
Click Test Connection to verify that the credentials are properly set.
Click Next.
On the Virtual Machines page, click Next.
On the Network page, specify the IP address, name and domain used for the Admin, Client and Private network. Provide the details for the following information:
Gateway and Domain Details: The domain and subnet mask details should already be supplied by the Exadata Provisioning Wizard. Enter a valid IP address for the Admin and Client gateway (Figure 6-29):
Virtual Machine 1 (Figure 6-30): For the first virtual machine in the cluster, the Exadata Provisioning Wizard will auto-complete the Prefix and Start ID field. You can enter an optional Suffix. Enter a valid IP address.
Generated Virtual Machine Name and IP (Figure 6-29): After you have entered the information for the first VM, click Generate to create the information for the other VMs in the cluster. You can always enter the details manually.
Once you have entered all the necessary information, click Validate IP at the top of the page to verify the IP addresses.
Click Next to continue to schedule the scale up.
Note:
The step for Grid Infrastructure and Initial Database are skipped because they do not need to be set up again.On the Schedule page, the Exadata Provisioning Wizard will create the Deployment Instance value. Select a schedule start and notification options:
Schedule: Select to initiate the scale up immediately or later. If you select later, then you will be prompted to select a date and time.
Notification: Select the notification statuses for which you will be notified.
Click Review to review the settings and initiate the scale up.
On the Review page, review the selection in the summary displayed. To change any section, return to the previous page and edit the selection.
Click Submit to begin the scale up.
To scale down a database cluster, the Virtual Machine is removed from the cluster:
From the Database Machine target menu, select Provisioning, then select Scale Down Cluster.
The Exadata Provisioning Wizard will display.
Enter the cluster name you want to scale down or click the Search icon to select an available cluster.
Once you have selected a cluster, you will be prompted to:
Select nodes to delete.
Verify or enter new named credentials for DomU (host, root, and Exadata Server) and for the Exadata Storage Server. Click Test Credentials to verify the credentials have been selected properly.
Click Next to schedule the scale-down job.
On the Schedule page, the Exadata Provisioning Wizard will create the Deployment Instance value. Select a schedule start and notification options:
Schedule: Select to initiate the scale down immediately or later. If you select later, then you will be prompted to select a date and time.
Notification: Select the notification statuses for which you will be notified.
Click Review to review the settings and initiate the scale down.
On the Review page, review the selection in the summary displayed. To change any section, return to the previous page and edit the selection.
Click Submit to begin the scale down.
To delete an existing cluster:
From the Database Machine target menu, select Provisioning, then select Delete Cluster.
The Exadata Provisioning Wizard will display.
On the Cluster page, enter the cluster name you want to delete or click the Search icon to select an available cluster.
The page will update to show the nodes to be deleted
Verify or enter new named credentials for DomU (host, root, and Exadata Server) and for the Exadata Storage Server. Click Test Credentials to verify the credentials have been selected properly.
Click Next to schedule the delete job.
On the Schedule page, the Exadata Provisioning Wizard will create the Deployment Instance value. Select a schedule start and notification options:
Schedule: Select to initiate the scale down immediately or later. If you select later, then you will be prompted to select a date and time.
Notification: Select the notification statuses for which you will be notified.
Click Review to review the settings and initiate the delete job.
On the Review page, review the selection in the summary displayed. To change any section, return to the previous page and edit the selection.
Click Submit to begin the delete job.
Once discovered, the Exadata plug-in shows the virtual machines monitored by Enterprise Manager Cloud Control 12c, as shown in Figure 6-32:
Note:
The schematic diagram in the Database Machine home page is based on the content of thedatabasemachine.xml
file found during discovery. The virtual platforms (Dom0
) are displayed as compute nodes in the rack in the schematic diagram.
The Database Machine Software topology diagram will not display the physical Oracle Server, virtual Oracle Server targets (DomU
), and Virtual Platform target (Dom0
) targets. However, it will continue to show the host targets which are running in DomU
.
The Software tab for the Exadata Database Machine target shows all clusters, ASM, and Database targets in the whole physical Database Machine grouped by clusters as described in Figure 6-33:
The following compute resource allocation graphs are available in virtualized Exadata. These graphs are dependent on the virtual machine hierarchy and metric data from the VI plug-in:
This graph (Figure 6-34) shows the ClusterWare cluster placement on physical servers in a particular Exadata Database Machine rack. Since this is a placement graph, the widths of the data series reflect the number of clusters on the physical server that has the most number of clusters.
This graph (Figure 6-35) shows the database placement on physical servers in a particular Exadata Database Machine rack for a particular DB cluster. Since this is a placement graph, the widths of the data series reflect the number of DB on the physical server that has the most number of databases for a particular DB cluster.
This graph (Figure 6-36) shows the database CPU utilization per database per VM host for a particular DB cluster.
This graph (Figure 6-37) shows the database memory utilization per database per VM host for a particular DB cluster.