3 Exadata Database Machine Discovery

This chapter provides instructions for discovery of the Oracle Exadata Database Machine through Enterprise Manager Cloud Control 12c.

The following sections describe how to discover the Exadata Database Machine and other supported targets:

1. Prerequisites for Guided Discovery

2. Discovering an Exadata Database Machine

3. Discovering the Oracle SuperCluster as an Exadata Database Machine

4. Discovering Grid Infrastructure and RAC

Note:

View a video of how to discover the components of an Oracle Exadata Database Machine:

https://apex.oracle.com/pls/apex/f?p=44785:24:114267128212806::NO:24:P24_CONTENT_ID%2CP24_PREV_PAGE:6318%2C1

3.1 Prerequisites for Guided Discovery

Before you begin the discovery process, there are several checks you should perform to ensure a smooth discovery. A discovery precheck script is available to automatically verify many of the common problem areas prior to discovery. You should perform the steps outlined in the following sections before proceeding with Discovering an Exadata Database Machine.

1. Discovery Precheck Script

2. Manual Verification

3.1.1 Secure Storage for Target Monitoring Credentials

As you begin the discovery process, cell monitoring requires log in and password credentials to be entered. These credentials are handled in the following manner:

• For the cellmonitor user: SSH equivalence is set up for between the cellmonitor user and the agent user. The root credentials being prompted for during discovery are not needed for metric collection afterward. You can optionally choose to store the cell root password as a named credential in the Enterprise Manager repository.

• For the compute node ILOM user (oemuser): This monitoring credential is encrypted and stored in the Enterprise Manager repository and in the targets.xml file on the agent.

• For the InfiniBand Switch user (nm2user): SSH equivalence is set up for between the nm2user user and the agent user. The nm2user credentials being prompted for during discovery are not needed for metric collection afterward. You can optionally choose to store the nm2user password as a named credential in the Enterprise Manager repository.

3.1.2 Discovery Precheck Script

Some Exadata discoveries in Oracle Enterprise Manager 12c may run into issues because of various configuration mismatches in software setup. A discovery precheck script has been developed to resolve the most common configuration problems. Run the script before all Database Machine discovery and examine the output thoroughly before proceeding with Discovering an Exadata Database Machine.

• Download the Discovery Precheck Script

• Run the Discovery Precheck Script

3.1.2.1 Download the Discovery Precheck Script

You can obtain the script in one of the following ways

• Download the script from Prerequisite script for Exadata Discovery in Oracle Enterprise Manager Cloud Control 12c (Doc ID 1473912.1) in My Oracle Support:

  https://support.oracle.com
  

• Access the script as part of the Oracle Enterprise Manager Cloud Control 12c Setup Automation kit (Exakit) 12.1.0.3.0. See Doc ID 1440951.1 in My Oracle Support for more information on the kit:

  https://support.oracle.com
  

• Access the script as part of Exadata plug-in 12.1.0.3.0 after the plug-in is deployed to the agent:

  <agent installation directory>/plugins/oracle.sysman.xa.discovery.plugin_12.1.0.3.0/discover/dbmPreReqCheck/exadataDiscoveryPreCheck.pl
  

3.1.2.2 Run the Discovery Precheck Script

To run the script:

$ <agent installation directory>/plugins/oracle.sysman.xa.discovery.plugin_12.1.0.3.0/discover/dbmPreReqCheck/
$ $ORACLE_HOME/perl/bin/perl ./exadataDiscoveryPreCheck.pl

As the script runs, you will be prompted for various inputs. It will execute all the built-in checks and will display all messages on standard output along with an execution log.

The discovery precheck script checks for the following information:

• Verifies that the user who is running the Enterprise Manager agent has read and execute permissions on at least one of the 11.2 (or later) Database Oracle home directory <ORACLE_HOME>/bin/kfod binaries.

• Verifies the management server and cell server is up and running on the Exadata cell. You can verify the status by running:

  $ cellcli -e 'list cell attributes cellsrvStatus,msStatus'
  

• Verifies the Database Machine schematic file requirements for Exadata Release 11.2.3.1, 11.2.3.2, and later, including:

  • That the schematic file should exist on the agent host selected for discovering the Exadata Database Machine at:

    /opt/oracle.SupportTools/onecommand/databasemachine.xml
    

  • That the schematic file has the world-readable privilege so that it is readable by the agent user.

  • That the version is 503 or later. The schematic file version is in the header of the file.

  • That the values of <ADMINNAME> and <ADMINIP> in the schematic file should be the current host names and management IP addresses of the compute nodes, Exadata cells, InfiniBand switches, KVM, PDU, and Cisco switch.

• Verifies log in credentials. These credentials are required to complete the discovery process. Make sure you have the information of the following credentials before starting the discovery process:

  • Agent host credentials: the username and password of the user running the agents on the compute nodes

  • Monitoring credentials:

    • Cell Root Password

    • ILOM username and password

    • InfiniBand Switch Nm2user Password

      Note:

      For the M6-32, the nm2user user is not activated by ACS as part of the standard M6-32 installation. Use either root or ilom-admin instead. The user exists on the systems, but needs its password set to usable for the precheck script to complete successfully.

  • InfiniBand Switch root credentials: Username and Password

• Verifies the Database Machine schematic file requirements for Exadata Release 11.2.3.2 and later:

  • Verifies that the schematic file exists on the agent host selected for discovering the Exadata Database Machine at:

    /opt/oracle.SupportTools/onecommand/catalog.xml
    

  • Verifies that the schematic file has the world-readable privilege so that it is readable by the agent user.

  • Verifies that the schematic file version is 868 or later. The schematic file version is in the header of the file.

  • In addition, the values of <ADMINNAME> and <ADMINIP> in the schematic file should be the current host names and management IP addresses of the compute nodes, Exadata cells, InfiniBand switches, KVM, PDU, and Cisco switch.

    Client network naming is only supported in Exadata plug-in Release 12.1.0.3.0 and later.

  Note:

  If the Database Machine schematic file does not exist or its version does not meet the requirement, open a Service Request with Oracle Support to get the latest version of the schematic file.

3.1.3 Manual Verification

After running the discovery precheck script, you will still need to perform the following checks manually:

• Use the 11.2 (or later) Database Oracle home directory when prompted for in Discovering an Exadata Database Machine.

• Pre-requirements for platform and Database Machine version.

• Compute node needs to be first discovered as Enterprise Manager host targets by installing the Enterprise Manager agents on these hosts.

  • The Enterprise Manager agent must be deployed to all compute nodes of the Exadata Database Machine. The host target name of the compute node must be the fully qualified host name (for example, dbm1db01.mydomain.com). Non-fully qualified hostnames (for example, dbm1db01) or IP addresses must not be used for the host target name of the compute node.

  • If you are using Exadata plug-in Release 12.1.0.1.0 or 12.1.0.2.0, the target name of the compute node host target should be the host name in the management network, not the client network.

  • If you are using Exadata plug-in Release 12.1.0.3.0 or later, the target name of the compute node host target can be the host name in the management network or the client network.

• The host names of the compute nodes and Exadata cells in each individual Database Machine should have a unique prefix. In other words, Exadata Database Machine racks that are not connected by the same InfiniBand network to form one DB Machine should not share the same prefix for the host names.

• For users of Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0 or if the installed version has been upgraded to Release 12.1.0.2.0 from 12.1.0.1.0 of the plug-in, you must manually push the Exadata plug-in to the agent.

3.2 Discovering an Exadata Database Machine

To discover a Database Machine target, follow these steps:

1. From the Enterprise Manager home page, select the Setup menu (upper right corner), Add Targets, and then Add Targets Manually. Figure 3-1 shows the Add Targets Manually page.

   Figure 3-1 Add Target Manually

   
   

2. Select the Add Non-Host Targets Using Guided Process (Also Adds Related Targets) option. From the Target Types drop-down, select Oracle Exadata Database Machine. Click Add Using Guided Discovery.

   Enterprise Manager displays the Oracle Exadata Database Machine Discovery page. From here you can add the hardware components such as Exadata Storage Servers and InfiniBand Switches in the Oracle Exadata Database Machine as managed targets. You can choose to discover a new Database Machine and its hardware components as targets or instead discover newly added hardware components in an existing Database Machine as targets.

3. On the Oracle Exadata Database Machine Discovery page, select the Discover a new Database Machine and its hardware components as targets option. Click Discover Targets.

   Enterprise Manger displays the Database Machine Discovery wizard that steps you through the process of discovering the Database Machine. The first page of the wizard, the Discovery Inputs page, appears.

4. Select an agent on one of the compute nodes that has the schematic file.

   Click the search icon . In the Search and Select pop-up, enter the name of a target you have deployed (see Install Enterprise Manager Agent Using the Push Method) and click Search. Select a Target Name from the results list and click Select.

   Note:

   If this is a partitioned Exadata Machine, be sure to pick one of the compute nodes that is part of the partition you wanted to be represented by this Database Machine target.

5. On the Discovery Inputs page, enter the following information:

   • For the Discovery Agents section:

     • Agent URL: The Agent deployed on compute node. Click the search icon to select from available URLs.

     • Database Oracle Home: The Database Oracle home directory on compute node

       Note:

       The Agent user needs execute privileges on the $ORACLE_HOME/bin/kfod binary of the Oracle home specified. (Oracle Database should be Release 11.2 or later.)

   • For the Schematic Files section:

     • Once you have specified the Agent URL, a new row (hostname and schematic file information) is automatically added. The default schematic file, databasemachine.xml, describes the hardware components of the Exadata Database Machine.

     • Click Set Credential to set the credentials for the host.

     • Check/modify the schematic file location.

     • Select the schematic file name from drop-down menu.

   Click Next.

6. On the InfiniBand Discovery page, enter the following information:

   • IB Switch Host Name: The InfiniBand switch host name (such as, exa01sw-ib2.mycompany.com)

   • Nm2user Password: The password for the InfiniBand switch

7. On the Prerequisite page, Enterprise Manager automatically performs the following checks:

   • Whether KFOD can be executed by the agent.

   • For the schematic file, if it exists, the name, location, and hostname will be displayed. For example:

     Schematic File /opt/oracle.SupportTools/onecommand/databasemachine.xml on <hostname>
     

8. On the Components page, the components below are pre-selected. You can deselect any unwanted component.

   • Compute Node: select the hosts that are compute nodes in the Oracle Exadata Database Machine.

   • Oracle Exadata Storage Server: select the Oracle Exadata Storage Servers that are part of this Oracle Exadata Database Machine target.

   • InfiniBand Switch: select the InfiniBand Switches that are part of the Oracle Exadata Database Machine. These also will be added as managed targets.

   • Ethernet Switch: select the Ethernet switches that are part of the Oracle Exadata Database Machine. The Ethernet switches will be added as managed targets.

   • Compute Node ILOM: select the Integrated Lights Out Managers (ILOM) of the compute nodes that are part of this Oracle Exadata Database Machine. Theses Integrated Lights Out Managers will be added as managed targets.

   • KVM: select the KVM switches that are part of the Oracle Exadata Database Machine. The KVM switches will be added as managed targets.

   Note:

   A partitioned rack can list other components that should be deselected manually (for example, compute nodes).

   Click Next to display the Monitoring Agents page.

9. On the Monitoring Agents page, select an agent from the drop-down selection box or click Reset to reassign. Click Next to display the Agent Credential page

   Note:

   The best practice is to use the Manually select the agents option on the Monitoring Agents page (instead of the Automatically select the agents option) and to use the Agent on compute node 1 for the Monitoring Agent and the Agent on compute node 2 for the Backup Monitoring Agent for all targets.

   This configuration is to ensure that you know which Agents to use when configuring SNMP for Exadata Storage Cell and InfiniBand switch targets (see Configure Storage Cell SNMP for Enterprise Manager Monitoring and Configure and Verify SNMP for InfiniBand Switch Targets).

10. On the Agent Credential page, specify whether the agent host users and passwords are the same for all agents. The agent users and passwords are required to set up password-less SSH between the agents and the cells monitored by the agents.

    If the users and passwords are the same, choose Same for all agents and enter the user and password combination. If they are not the same for each agent, choose Different for all agents and enter each combination for each agent.

    Click Next to display the Monitoring Credential page.

11. On the Monitoring Credential page, configure the credentials for:

    • Oracle Exadata Storage Server: select the Same for all cells option. Enter the Cell Root Password.

    • Infiniband Switch: select the Same for all Infiniband switches option. Enter the Infiniband Switch Nm2user Password.

    • ILOM: select the Same for all ILOM option. Enter the ILOM password.

    Note:

    In all cases you can choose to enter the same user/password combinations for all components or you can enter the credentials separately for each occurrence.

    Click Next to display the SNMP Subscription page.

12. On the SNMP Subscription page, automatically set up an SNMP subscription on the cell and InfiniBand Switch targets. Figure 3-2 shows an example of the SNMP subscription options. In the SNMP Community String field, enter public (it should match with the community string set on Cell and InfiniBand Switch).

    Figure 3-2 SNMP Subscription

    
    

    Click Next to display the Component Properties page.

13. On the Component Properties page, specify the target properties of the different components. For each component, enter the required information.

    Click Next to display the Review page.

14. On the Review page, verify each section is correct. Figure 3-3 shows an example of an accurate review.

    Figure 3-3 Database Machine Discovery Review Page

    
    

15. If the information is correct, click Submit.

16. On the Target Creation Summary page, click OK.

Enterprise Manager displays the Target Promotion Summary page that displays the targets that are now managed targets.

The discovery of the Oracle Exadata Database Machine is complete.

3.3 Discovering the Oracle SuperCluster as an Exadata Database Machine

You can use the Exadata plug-in to discover and monitor an Oracle SuperCluster system with the Enterprise Manager schematic file (databasemachine.xml or catalog.xml). You can monitor only the Database and Exadata components of the Oracle SuperCluster installed on LDOM and Zones. To monitor specific LDOM or virtualization configurations at the operating system level, please use Oracle Enterprise Manager Ops Center 12c.

To discover the Oracle SuperCluster as an Exadata Database Machine:

1. Open a Service Request (SR) if the databasemachine.xml file or catalog.xml file is not present.

   Important:

   Make a backup of your /opt/oracle.SupportTools/onecommand directory as you will overwrite it with the new configurations.

2. Oracle Support will provide you the file(s) generated based on the information provided in your configuration files. Copy all files into your /opt/oracle.SupportTools/onecommand directory.

3. Ensure that READ permissions are open for all files in this directory as well as in the /opt/oracle.SupportTools/em directory. The agent must be able to read these files during discovery.

4. Run Self Update from Enterprise Manager to download the Solaris SPARC agent software on the OMS. Apply that download to the OMS to make it available for deployment:

   1. From Enterprise Manager, click Setup, then Extensibility, and finally Self Update. Click Agent Software.

   2. Select Oracle Solaris on SPARC (64-bit).

   3. If the status is Available, then click Download from the Actions menu.

   4. Once downloaded, you must Apply it to the OMS from the same Self Update page.

   For more details on Self Update in Online or Offline mode, please see the "Updating Cloud Control" chapter in the Oracle® Enterprise Manager Cloud Control Administrator's Guide:

   http://docs.oracle.com/cd/E24628_01/doc.121/e24473/self_update.htm
   

5. Install an agent on each of the Database nodes of the Oracle SuperCluster:

   1. From Enterprise Manager, select Setup, then Add Target, and finally Add Targets Manually.

   2. Select Add Host Targets (selected by default) and click Add Host.

   3. Continue with the installation wizard and ensure that you run root.sh on each node once installation is completed.

   Note:

   You may run into a known issue where you have to edit the Ciphers entry in /etc/ssh/sshd_config and restart ssh (see bug 13822068 for details). See SSHD Connection Errors After Upgrading To Exadata Version 11.2.3.1.0 (Doc ID 1437596.1) in My Oracle Support:

   https://support.oracle.com
   

6. Configure the database nodes for Database Machine discovery.

   This step is needed to set descriptions of each DB node by updating the DB node IP address, Host Channel Adapter ID, and management hostname in the InfiniBand configurations using set_nodedesc.sh. Enterprise Manager Agents look for this information when discovering the Database Machine.

   Run:

   # ibnetdiscover | grep your_db_nodes
   

   If no output is not returned, then run the following command to set the node descriptions from all database nodes:

   # /bin/ib_set_node_desc_ssc.sh
   

7. Discover Exadata Database Machine from Enterprise Manager using Manual Discovery wizard. This discovery process is the same as any other Exadata DBM target. See Discovering an Exadata Database Machine.

8. Discover the HA RAC Cluster and the cluster databases and configure each target as usual.

3.4 Discovering Grid Infrastructure and RAC

The following sections provide instructions for discovering Grid Infrastructure and Oracle Real Application Clusters targets:

• Discover Grid Infrastructure (Cluster) Targets

• Discover Oracle Real Application Clusters (RAC) Targets

3.4.1 Discover Grid Infrastructure (Cluster) Targets

To discovery the Grid Infrastructure targets:

1. From the Setup menu, select Add Targets, then Add Targets Manually.

2. In the Add Targets Manually page, select Add Non-Host Targets Using Guided Process (Also Adds Related Targets) and Target Type as Oracle Cluster and High Availability Services.

3. Click Add Using Guided Discovery and follow the wizard.

4. On the Add Cluster Target: Specify Host page, select the first database node and click Continue.

5. The cluster home details are displayed. Ensure that all database nodes are shown in Selected Hosts section.

6. Click Add.

3.4.2 Discover Oracle Real Application Clusters (RAC) Targets

To discovery RAC targets:

1. From the Setup menu, select Add Targets, then Add Targets Manually.

2. In the Add Targets Manually page, select Add Non-Host Targets Using Guided Process (Also Adds Related Targets) and Target Type as Oracle Database, Listener and Automatic Storage Management.

3. Click Add Using Guided Discovery and follow the wizard.

4. In the wizard pages, configure Cluster Databases (dbsnmp password required) and Cluster ASM (asmsnmp password required).