Exadata Database Machine Discovery

3 Exadata Database Machine Discovery

This chapter provides instructions for discovery of the Oracle Exadata Database Machine through Enterprise Manager Cloud Control 13c.

The following sections describe how to discover the Exadata Database Machine and other supported targets:

Discovering an Exadata Database Machine

This section describes various discovery flows that are supported in Enterprise Manager 13c. The database machine discovery using EMCLI helps to automate the discovery process where several Database Machines must be discovered offline or in background. On the other hand, use the Console for an interactive experience of the discovery.

Discover the Exadata Database Machine Target Using the Console

Fresh Discovery of Exadata Database Machine Target

This discovery flow shows the steps involved in discovering Database Machine targets. This flow is the recommended option for discovering Database Machine targets afresh.

An Exadata Database Machine can be deployed in a physical or virtual configuration. Enterprise Manager supports both of these configurations. There are additional steps required in order to discover a virtual Exadata Database Machine. These additional steps are addressed inline as necessary.

Discovering the Oracle Virtual Platform

If discovering a virtual Exadata, the Oracle Virtual Platform target must be discovered before discovering the Oracle Exadata Database Machine target. The steps below detail the discovery of the Oracle Virtual Platform target.

Note:

If you are using EMCLI to perform Exadata Database Machine discovery, the Oracle Virtual Platform targets can be discovered as part of that discovery without a separate discovery operation. See Discover the Exadata Database Machine Target Using EMCLI.

From the Enterprise Manager home page, select the Setup menu (upper right corner), Add Target, and then Add Targets Manually.
On the Add Targets Manually page, click Add Targets Using Guided Process.
Under the Guided Discovery column, select Oracle Virtual Platform and click Add.
On the Discover Oracle Virtual Platforms screen, do the following:
1. Select the agent installed on the first compute node domU (virtual machine) for the Monitoring Agent and select the second compute node domU (virtual machine) agent for the Failover Monitoring Agent.
2. For the Credential Properties, enter the user name for the root or non-root user and provide the password.
3. In the Hostname and IP Address section, click on the Add button and type in the fully qualified hostname for the dom0 servers (one per line) and click Add.
4. Click Submit.
After the discovery of the Oracle Virtual Platform target is complete, follow the steps in the next section to discover the Oracle Exadata Database Machine target.

Discovering the Oracle Exadata Database Machine

From the Enterprise Manager home page, select the Setup menu (upper right corner), Add Target, and then Add Targets Manually.
On the Add Targets Manually page, click Add Targets Using Guided Process.
From the Add Using Guided Process window, select Oracle Exadata Database Machine from the list and click Add.
On the Oracle Exadata Database Machine Discovery page, the following two options are presented. This document details the steps for discovering a new Exadata Database machine and components:
- Discover a new Database Machine and its hardware components as targets. The table will update to show the target types and the credentials required for discovery.
- Discover newly added hardware components in an existing Database Machine as targets. Select a Database Machine from the drop-down menu. The table will update to show the target types and the credentials required for discovery. The subsequent dialogs in the discovery flow will be similar to those depicted in this section but will be restricted to only undiscovered components.
Select the task and click Discover Targets. The Exadata Discovery Wizard begins.
On the Discovery Inputs page, enter the following information:
- For the Discovery Agents section:
  - Agent URL: Select an agent deployed on a physical compute node or if virtual Exadata on a domU (virtual machine) compute node. Click the search icon to select from available agents.
- For the Schematic Files section:
  - Once you have specified the Agent URL, a new row (hostname and schematic file information) is automatically added. The default schematic file, databasemachine.xml, describes the hardware components of the Exadata Database Machine.
  - Click Set Credential to set the credentials for the host.
  - Check and modify if necessary the schematic file location.
  - Select the schematic file name from drop-down menu.
  Note:
  
  You only need to customize the Schematic file location if one of the following is true:
  
  – The Schematic file is not in the default location.
  
  – The Database Machine contains a Storage Expansion Rack.
  
  – The hardware components are specified in multiple schematic files.
Click Next.
The InfiniBand Discovery page looks different depending on whether the Exadata has been identified as having an InfiniBand (IB) or RDMA over Converged Ethernet (RoCE) storage network. Screenshots and the details for each option are shown below.
If this is an IB Exadata, the following information is required,
- IB Switch Host Name: The host name for one of the IB switches for the Exadata. The IB Switch hostname is usually pre-populated.
- InfiniBand Switch ILOM host credential: The root or ilom-admin user name and password for the InfiniBand switch ILOM host.
  Click Next.
If this is Exadata X8M, no credentials are required at this point in the discovery flow. The InfiniBand Discovery page will display a message that the Database Machine to be discovered contains RoCE switches and no additional inputs are needed.

Click Next.

Note:
If this is Exadata X8M and the form displayed requests InfiniBand information, then it likely means there was an issue accessing the databasemachine.xml file specified in the previous step. Navigate back and correct this issue. Ensure the proper location and credentials are specified for the databasemachine.xml file and that the file exists.
On the Prerequisite Check page, Enterprise Manager will try to discover hardware components dynamically based on your environment. If any critical issues are encountered, then you can click Back to resolve them. Enterprise Manager will show the problem and its severity (Info, Warning, or Critical).
Any warning issues or informational messages may also be displayed. These do not interfere with the discovery process.

Click Next.
On the Components page, the components below are pre-selected. You can deselect any unwanted component. Components include, but are not limited to, the following:
- Compute Node: Select the hosts that are compute nodes in the OracleExadata Database Machine.
- Oracle Exadata Storage Server: Select the Oracle Exadata Storage Servers that are part of this Oracle Exadata Database Machine target.
- InfiniBand Switch: Displayed if this is an IB Exadata. Select the InfiniBand Switches that are part of the Oracle Exadata Database Machine. These also will be added as managed targets.
- Ethernet Switch: Select the Ethernet switches that are part of the Oracle Exadata Database Machine. If this is an IB Exadata, select the admin switch. If this is Exadata X8M, select the admin and RoCE switches. The Ethernet switches will be added as managed targets.
- Compute Node ILOM: Select the Integrated Lights Out Managers (ILOMs) of the compute nodes that are part of this Oracle Exadata Database Machine. These ILOMs will be added as managed targets.
Note:
A partitioned rack can list other components that should be deselected manually (for example, compute nodes).

Click Next.
By default, the agents are automatically assigned to the targets. To add more monitoring agents:
In the Monitoring Agents page, click Select Agents > in the pop-up window, select the required agents for each target on the Exadata Database Machine. Use the Ctrl key to select multiple agents.

Note:
A Warning will be thrown if only one agent is used as monitoring and backup agents. Click Select Agents and add more agents, if available.

Click Next.

On the Credentials page, set the credentials for all components within the Exadata Database Machine (for example, the Storage Server, PDU, InfiniBand Switch, and so forth). Click Set Credential to set the credential for the component. If the users and passwords are the same for all components of the type, choose Same for all and enter the user and password combination.

Depending on the component, you may need to provide additional information. For example, in the Exadata Storage Server Credentials window, you must enter the SNMP Credentials as well as the monitoring Credentials.
The recommended monitoring mechanism for the Oracle Exadata Storage Servers is RESTful API. An ExaCLI user is required to use the RESTful API. For details on how to create the necessary ExaCLI user, refer to Creating Users for Use with ExaCLI in Oracle Exadata Database Machine Maintenance Guide.
SNMP subscriptions are required for EM to receive alerts for hardware and software failures from Exadata components. Each component credential screen provides fields for SNMP configuration. The recommendation for SNMP subscriptions is to use SNMP v3 for all subscriptions. For IPv6 environments, only SNMPV3 credentials are supported.
The SNMP community string is like a password. It is sent along with each SNMP Get-Request, and allows or denies access to the device. Please check with your network administrator to get the appropriate community string details for your environment.
For the Compute Node Server ILOM Credentials, you can additionally specify the named credentials for monitoring using RESTful API. Click Test Connection to verify the proper working of the SSH and RESTful API credentials. The ILOM version for REST monitoring must be 5.0.1 or higher. Also, for REST monitoring, ensure to use the latest Exadata plug-in release update on the agent side.
You can create a Compute node REST credential to promote ILOM REST access point by navigating in the Enterprise Manager as follows: Click Setup in the Enterprise Manager home > Click Security > Click Named Credential > Enter the details for the REST credentials.

The table below provides sample guidance on the recommended values for the credentials.

Table 3-1 Credential Details

Component	Credential	Monitoring Mechanism	SNMP Subscription	Notes
Agent	oracle (or the user who owns the agent installation)	.	.	.
Oracle Exadata Storage Server	celladministrator	RESTful API	SNMP V3	.
Infiniband Switch	root / ilom-admin	.	SNMP V3	This component will not be present when discovering RoCE based systems
Compute Node Server ILOM	root	SSH / Enhanced monitoring using RESTful API	SNMP V3	In case of enhanced ILOM monitoring using RESTful API, both root and admin user credentials are supported.
PDU	admin	.	SNMP V3	.
Ethernet Switch	admin	.	SNMP V3	In a RoCE based system, these credentials will be used for the Management switch discovery as well as the RoCE switch discovery. If the passwords are different between the Management and RoCE switches, passwords can be specified by switch.

Click Next.

On the Review page, verify each section is correct. The following figure shows an example of an accurate review.

Click Submit.
After the Database Machine components are discovered, a Target Creation Summary page displays to show all discovered targets and components. If there are any errors, the Status column will display a red flag. Place your cursor over the flag for a pop-up window for additional information about the error.
Perform post-discovery verification before you begin using Enterprise Manager. See Post-Discovery Configuration and Verification.

Fresh Discovery of Exadata Database Machine on SuperCluster Systems

You can use the Exadata plug-in to discover and monitor an Oracle SuperCluster system.

You can monitor only the Database and Exadata components of the Oracle SuperCluster installed on LDOM and Zones. To monitor specific LDOM or virtualization configurations at the operating system level, please use Oracle Enterprise Systems Infrastructure Plug-in 13c.

The SuperCluster discovery wizard provided by “Systems Infrastructure Plug-in” is mainly responsible for discovery of the SuperCluster Hardware targets including ZFS server (For details please refer superCluster discovery document from emsi plug-in). It is recommended that the users should go through this flow first and then perform the Exadata Database Machine Discovery.

You can use the Exadata plug-in to discover and monitor an Oracle SuperCluster system. You can monitor only the Database and Exadata components of the Oracle SuperCluster installed on LDOM and Zones. To monitor specific LDOM or virtualization configurations at the operating system level, please use Oracle Enterprise Manager Ops Center.

To discover the Oracle SuperCluster as an Exadata Database Machine:

See Discovering and Promoting Oracle Solaris Zones in Oracle® Enterprise Manager Cloud Control Administrator's Guide for completing the prerequisite of pushing the agent to the control domain and zones.
Open a Service Request (SR) and upload the following files from your first LDOM:
```
/opt/oracle.SupportTools/onecommand/onecommand.params
/opt/oracle.SupportTools/onecommand/config.dat
```
Note:

Make a backup of your /opt/oracle.SupportTools/onecommand directory as you will overwrite it with the new configurations.
Oracle Support will provide you the file(s) generated based on the information provided in your configuration files. Copy all files into your /opt/oracle.SupportTools/onecommand directory.
Ensure that READ permissions are open for all files in this directory as well as in the /opt/oracle.SupportTools/em directory. The agent must be able to read these files during discovery.
Run Self Update from Enterprise Manager to download the Solaris SPARC agent software on the OMS. Apply that download to the OMS to make it available for deployment:
1. From Enterprise Manager, click Setup, then Extensibility, and finally Self Update. Click Agent Software.
2. Select Oracle Solaris on SPARC (64-bit).
3. If the status is Available, then click Download from the Actions menu.
4. Once downloaded, you must Apply it to the OMS from the same Self Update page.
For more details on Self Update in Online or Offline mode, see Updating Cloud Control in Oracle® Enterprise Manager Cloud Control Administrator's Guide.
Install an agent on each of the Database nodes of the Oracle SuperCluster:
1. From Enterprise Manager, select Setup, then Add Target, and finally Add Targets Manually.
2. Select Add Host Targets (selected by default) and click Add Host.
3. Continue with the installation wizard and ensure that you run root.sh on each node once installation is completed.
Configure the database nodes for Database Machine discovery.

This step is needed to set descriptions of each Database node by updating the Database node IP address, Host Channel Adapter ID, and management host name in the InfiniBand configurations using set_nodedesc.sh. Enterprise Manager Agents look for this information when discovering the Database Machine.

Run:
```
# ibnetdiscover | grep your_db_nodes
```
If no output is not returned, then run the following command to set the node descriptions from all database nodes:
```
# /bin/ib_set_node_desc_ssc.sh
```
Discover Exadata Database Machine from Enterprise Manager using Manual Discovery wizard. This discovery process is the same as any other Exadata Database Machine target. See Discovering an Exadata Database Machine.
Discover the HA RAC Cluster and the cluster databases and configure each target as usual.
Perform post-discovery verification before you begin using Enterprise Manager to monitor the Exadata Database Machine. See Post-Discovery Configuration and Verification.

Convert Database Machine Targets with 12c Target Types to 13c Using the Console

To convert an Exadata Database Machine discovered by Enterprise Manager 12c or by Enterprise Manager 13c with 12c target types to be monitored by Enterprise Manager 13c:

From the Database Machine menu, select Convert 12c Member Targets.
Select a Conversion Option.
- (Default) Delete 12c targets and their historical data. This option halts all monitoring of legacy targets and deletes the historical data. This options allows for a clean start for all Enterprise Manager 13c target monitoring.
- Retain 12c targets and their historical data. This option preserves the historical data for legacy targets.
Once you have selected a conversion option, click Next.
Set the Credentials. For the conversion, you must set the credentials for the Exadata Database Machine components to be converted. These are a subset of the components that comprise the Exadata. For further details on recommended settings, see the table Table 3-1 and the related details in the corresponding step in Discovering the Oracle Exadata Database Machine.

Once you have set the credentials, the Credentials page will update to show the credentials set for all components to be converted. Click Next.
Review the components to be converted in the following three sections.
- Summary: This section summarizes the number of targets to be converted and if any targets will have additional monitoring information available to you after the conversion.
- Targets to be Converted: This table shows all Enterprise Manager 12c targets to be converted to 13c target types for monitoring by Enterprise Manager 13c and provides the new target name information.
- Targets Not Impacted by Conversion: All components of the Exadata Database Machine do not need to be converted to be monitored by Enterprise Manager 13c. This list identifies those components that to not need to be converted. These components are already monitored by Enterprise Manager 13c using 13c target types.
Click Submit to begin the conversion.
A Processing pop-up window will appear which shows you the status of the conversion and a summary of the success or failure of the components to be converted.

Do not close this window until the processing is complete!

You can select the option to close the window after the processing is complete, or click Done.
Once processing is complete, a Conversion Results page provides a summary of the components converted and details of each component that succeeded or failed to be converted:

From this page, click Convert Remaining 12c Targets to repeat the conversion process for those components that failed to be converted. Click Launch New Database Machine Home Page to view the updated home page with all converted components monitored by Enterprise Manager 13c.

Discover the Exadata Database Machine Target Using EMCLI

EMCLI based discovery is customizable, enabling you to add or skip components in the event of component outage and select the monitoring agents. EMCLI Database Machine Discovery is idempotent, subsequent executions will make the best effort to discover any previously undiscovered targets (components) and will ignore any targets that have already been discovered and configured.

The discovery works for both physical and virtual Exadata Database Machines (IB/RoCE) using local or remote agent.

Ensure to complete the prerequisite tasks to discover the Exadata Database Machine target using EMCLI. For the list of prerequisites to complete, see Prerequisites for Discovery Using EMCLI.

Submit the Deployment Procedure for the Discovery

 emcli   submit_procedure -name=DBMachineDiscovery -input_file="data://<input_file_absolute_path>"

In the above command:

DBMachineDiscovery: This is the deployment procedure used for the discovery of Exadata Database Machine using EMCLI. See Deployment Procedure for EMCLI.
input_file_absolute_path: Full path of the input file that you created in the prerequisite steps. See Create the Input File.

To receive notification regarding the discovery DP status, use the -notification flag as shown below:

 emcli   submit_procedure -name=DBMachineDiscovery -input_file="data://<input_file_absolute_path>" -notification="scheduled, action required, running" ;

Deployment Procedure for EMCLI

The Deployment Procedure (DP) DBMachineDiscovery is used for the discovery of Exadata Database Machine using EMCLI. This DP is composed two DPs:

DBMachineSystemCheck: This DP takes the input file that you created while performing the prerequisite tasks and performs a set of prerequisite-checks. After the success of prerequisite-checks, this DP generates an output log file which is consumed by the next DP.
DBMachineSystemCreation: This DP consumes the output log file generated by the DP for prerequisites-check, and performs Exadata Database Machine discovery. As a result, the steps such as creation of targets, associations, and promotion of targets are completed.

View the Discovery Status

You can monitor the status of the submitted deployment procedure using the EMCLI command or through the Enterprise Manager UI.

In case of errors during discovery, resolve the error based on the deployment procedure output information. See EMCLI Discovery Issues.

Check the Deployment Procedure Status Using EMCLI

After you submitted the discovery deployment procedure, the execution ID is displayed in the command line as shown below:

emcli submit_procedure -name=”DBMachineDiscovery” -input_file=data:”/home/user_jane/dpInput/emcli_discover_dp1_vir_remote” 

Schedule not specified, defaults to immediate. 
6979503EF41F2447E05338C8F70A1796
Deployment procedure submitted successfully

Use the execution ID to query the status and display it in a specific format. For example, if the execution ID is 69818CF4995E0C88E05338C8F70AA2C3:

emcli get_instance_status -instance=69818CF4995E0C88E05338C8F70AA2C3 -xml -details -showJobOutput -tailLength=50

An example out of the above command in xml format:

</step>
<step isEnabled="true" name="promoteTargets" stepType="">
    <GUID>promoteTargets</GUID>
    <errorModeString>stop</errorModeString>
    <typeStep>Step</typeStep>
    <status></status>
    <startedOn>2018-04-10 08:56:28.0</startedOn>
    <lastUpdatedOn>2018-04-10 08:58:16.0</lastUpdatedOn>
    <completedOn>2018-04-10 08:58:16.0</completedOn>

Check the Deployment Procedure Status Using Console

The status and progress of the discovery can be viewed using the Enterprise Manager console.

From the main Enterprise menu, click Provisioning and Patching > click Procedure Activity. The following status information is displayed:

Target discovery status: Success or Failure
In case of failure, the reason for it, and additional information to correct it.

Resolve the issues displayed in deployment procedure output and re-submit the discovery through Enterprise Manager CLI or Enterprise Manager console to discover Exadata Database Machine or the remaining components of the database machine.

Convert 12c Type Database Machine Targets to 13c Using EMCLI

To convert an Exadata Database Machine discovered by Enterprise Manager 12c or by Enterprise Manager 13c with 12c target types to be monitored by Enterprise Manager 13c:

Create an input file by entering the information about the database machine target. Following is an example input file:
```
#--------------Input File credentials example-------------- 
# Config Map 
configMap.dbmTargetName=<DB Machine Target Name>
configMap.retainOldTargets=false 

# Credentials 
credMap.ibIlomCred=<IB Switch ILOM Credentials> 
credMap.ibSnmpCred=<IB Switch SNMP Credentials> 
credMap.pduHttpCred=<PDU Credentials> 
credMap.pduSnmpCred=<PDU SNMP Credentials> 
credMap.ciscoIosCred=<CISCO Switch IOS Credentials> 
credMap.ciscoSnmpCred=<CISCO Switch SNMP Credentials> 
credMap.computenodeIlomCred=<Compute Node ILOM Credentials> 
credMap.computenodeAdminCred=<Compute Node Admin Credentials> 
credMap.computenodeSnmpCred=<Compute Node SNMP Credentials> 
credMap.computeSnmpOpt=false 
credMap.agentCred=<Agent Credentials>
```
In the above example, the following parameters are used:
- configMap.dbmTargetName: The name of the Database Machine Target that you want to convert
- configMap.retainOldTargets: You can retain 12c targets and their historical data after the conversion. By default, this is set to false. This option halts all monitoring of legacy targets and deletes the historical data. Alternatively, you can set it to true to halt the monitoring of legacy targets and retain their historical data.
- credMap.ibIlomCred: IB switch ILOM credentials
- credMap.ibSnmpCred: IB switch SNMP credentials
- credMap.pduHttpCred: PDU credentials
- credMap.pduSnmpCred: PDU SNMP credentials
- credMap.ciscoIosCred: CISCO switch IOS credentials
- credMap.ciscoSnmpCred: CISCO switch SNMP credentials
- credMap.computenodeIlomCred: Compute node ILOM credentials
- credMap.computenodeAdminCred: Compute node Admin credentials. Provide this when credMap.computeSnmpOpt is set to true.
- credMap.computenodeSnmpCred: Compute node SNMP subscription credentials. Provide this when credMap.computeSnmpOpt is set to true.
- credMap.computeSnmpOpt: Compute node SNMP subscription flag. By default, this is set to false. If the flag is set to true, then after the conversion of ILOM, the respective ILOM SNMP is subscribed.
- credMap.agentCred: Agent credentials for compute node SNMP subscription.
For more information about the credentials, see Named Credentials and Commands to Create Them.
Run the following EMCLI command to start the conversion of the Database Machine target from 12c type to 13c by specifying the path of the input file:
```
emcli submit_procedure -name=Convert12cTo13cDatabaseMachine -input_file=<input_file_path> -notification="scheduled, action required, running";
```
After running the command, note the deployment procedure (DP) number. You will need this to check the status of the conversion in the console.
To check the status of the deployment procedure in the console, in the Enterprise menu, click Provisioning and Patching > click Procedure Activity. The page lists all the deployment procedures.
Click the link corresponding to the deployment procedure you started in step 2. This displays the Procedure Activity page for your deployment procedure.
Under Procedure Steps, click View > click Expand All. You will now be able to view all the steps in the procedure like Prerequisite, Conversion, and Summary. The status of each step is displayed in the same section. To view more details of each step, click on it.

In case of failure in any of the steps, a detailed diagnostic is performed and information is available for you to apply the fix. After you apply the fix, re-run the EMCLI command specified in step 2 and repeat the steps after.

In the example below, you can view the summary of the conversion which indicates that one of the targets cannot be converted:

Discovering Grid Infrastructure and RAC

The process for discovering Grid Infrastructure (Cluster) and Real Application Cluster (RAC) related targets including Oracle High Availability Service and Cluster targets as well as ASM, database, listener, and related targets on Exadata is the same as on other platforms. See Discovering and Adding Database Targets in the Oracle Enterprise Manager Cloud Control Administrator’s Guide for more information.

Switch from Using CellCLI to RESTful API

You can at any time lock down an Exadata Storage Server and switch to ExaCLI or RESTful API for monitoring/administration. It is recommended that you use RESTful API.

Ensure that the RESTful API user is created by following the steps in Create an ExaCLI or RESTful API User.

From the Exadata Storage Server menu, select Target Setup, then select Monitoring Configuration.

The Monitoring Configuration dialog box opens.
Set the value of Monitoring Mechanism to:
- 1 for ExaCLI
- 2 for RESTful API
Set Trust Self-signed Certificate to 1.

Click OK.
From the main Setup menu, click Security > click Monitoring Credentials > select Exadata Storage Server targets.

For each Exadata Storage Server, enter the RESTful API monitoring credentials.

Click Save.

For more information about ExaCLI, see Using the ExaCLI Utility in Exadata Database Machine Maintenance Guide.