3 Exadata Database Machine Discovery

This chapter provides instructions for discovery of the Oracle Exadata Database Machine through Enterprise Manager Cloud Control 13c.

The following sections describe how to discover the Exadata Database Machine and other supported targets:

1. Running the Discovery Precheck Script

2. Discovering an Exadata Database Machine

3. Discover Oracle SuperCluster as an Exadata Database Machine

4. Discovering Grid Infrastructure and RAC

5. Switch from Using CellCLI to ExaCLI after Storage Server Locked Down

3.1 Running the Discovery Precheck Script

Before you begin the guided discovery process, there are several checks you should perform to ensure a smooth discovery. A discovery precheck script is available to automatically verify many of the common problem areas prior to discovery.

Some Exadata discoveries in Oracle Enterprise Manager 13c may run into issues because of various configuration mismatches in software setup. This discovery precheck script helps to resolve the most common configuration problems. Run the script before all Database Machine discovery and examine the output thoroughly before proceeding with Discovering an Exadata Database Machine.

Follow the steps below to run the discovery precheck script:

1. Secure Storage for Target Monitoring Credentials.
2. Download the Discovery Precheck Script.
3. Run the Discovery Precheck Script.
4. Automatic Verification.
5. Manual Verification.

3.1.1 Secure Storage for Target Monitoring Credentials

As you begin the discovery process, cell monitoring requires log in and password credentials to be entered. These credentials are handled in the following manner:

• For the cellmonitor user: SSH equivalence is set up for between the cellmonitor user and the agent user. The root credentials being prompted for during discovery are not needed for metric collection afterwards. You can optionally choose to store the cell root password as a named credential in the Enterprise Manager repository.

• For the cell ILOM user (oemuser): This monitoring credential is encrypted and stored in the Enterprise Manager repository and in the targets.xml file on the agent.

• For the InfiniBand Switch user (nm2user): SSH equivalence is set up for between the nm2user user and the agent user. The nm2user credentials being prompted for during discovery are not needed for metric collection afterward. You can optionally choose to store the nm2user password as a named credential in the Enterprise Manager repository.

3.1.2 Download the Discovery Precheck Script

You can obtain the script in one of the following ways

• Access the script as part of Exadata plug-in 13.1.0.0.0 after the plug-in is deployed to the agent:

  <agent installation directory>/plugins/oracle.sysman.xa.discovery.plugin_12.1.0.3.0/discover/dbmPreReqCheck/exadataDiscoveryPreCheck.pl
  

  Note:

  The Exadata plug-in includes an additional script that enables enhanced monitoring through Enterprise Manager Cloud Control 13c.

• Download the script from Prerequisite script for Exadata Discovery in Oracle Enterprise Manager Cloud Control 12c (Doc ID 1473912.1) in My Oracle Support:

  https://support.oracle.com/rs?type=doc&id=1473912.1
  

• Access the script as described in Obtaining the Oracle Enterprise Manager Setup Automation kit for Exadata (Doc ID 1440951.1) in My Oracle Support for more information on the kit:

  https://support.oracle.com/rs?type=doc&id=1440951.1
  

3.1.3 Run the Discovery Precheck Script

To run the script:

$ <agent installation directory>/plugins/oracle.sysman.xa.discovery.plugin_<plugin version>/discover/dbmPreReqCheck/
$ $ORACLE_HOME/perl/bin/perl ./exadataDiscoveryPreCheck.pl

As the script runs, you will be prompted for various inputs. It will execute all the built-in checks and will display all messages on standard output along with an execution log.

3.1.4 Automatic Verification

The discovery precheck script checks for the following information:

• Verifies that the user who is running the Enterprise Manager agent has read and execute permissions on at least one of the 11.2 (or later) Database Oracle home directory <ORACLE_HOME>/bin/kfod binaries.

• Verifies the management server and cell server is up and running on the Exadata cell. You can verify the status by running:

  $ cellcli -e 'list cell attributes cellsrvStatus,msStatus'
  

• Verifies log in credentials. These credentials are required to complete the discovery process. Make sure you have the information of the following credentials before starting the discovery process:

  • Agent host credentials: the username and password of the user running the agents on the compute nodes

  • Monitoring credentials:

    • Cell Root Password or ExaCLI credentials if you plan to use ExaCLI

    • InfiniBand Switch Nm2user Password

    • ILOM username and password

  • InfiniBand Switch root credentials: Username and Password

• Verifies the Database Machine schematic file requirements for Exadata Release 11.2.3.1, 11.2.3.2, and later, including:

  • That the schematic file should exist on the agent host selected for discovering the Exadata Database Machine at:

    /opt/oracle.SupportTools/onecommand/databasemachine.xml
    

  • That the schematic file has the world-readable privilege so that it is readable by the agent user.

  • That the version is 503 or later. The schematic file version is in the header of the file.

  • That the values of <ADMINNAME> and <ADMINIP> in the schematic file should be the current host names and management IP addresses of the compute nodes, Exadata cells, InfiniBand switches, KVM, PDU, and Cisco switch.

• Verifies the Database Machine schematic file requirements for Exadata Release 11.2.3.2 and later:

  • Verifies that the schematic file exists on the agent host selected for discovering the Exadata Database Machine at:

    /opt/oracle.SupportTools/onecommand/catalog.xml
    

  • Verifies that the schematic file has the world-readable privilege so that it is readable by the agent user.

  • Verifies that the schematic file version is 868 or later. The schematic file version is in the header of the file.

  • In addition, the values of <ADMINNAME> and <ADMINIP> in the schematic file should be the current host names and management IP addresses of the compute nodes, Exadata cells, InfiniBand switches, KVM, PDU, and Cisco switch.

    Client network naming is only supported in Exadata plug-in Release 12.1.0.3.0 and later.

  Note:

  If the Database Machine schematic file does not exist or its version does not meet the requirement, open a Service Request with Oracle Support to get the latest version of the schematic file.

3.1.5 Manual Verification

After running the discovery precheck script, you will still need to perform the following checks manually:

1. Use the 11.2 (or later) Database Oracle home directory when prompted for in Discovering an Exadata Database Machine.
2. Pre-requirements for platform and Database Machine version.
3. Compute node needs to be first discovered as Enterprise Manager host targets by installing the Enterprise Manager agents on these hosts.
   • The Enterprise Manager agent must be deployed to all compute nodes of the Exadata Database Machine. The host target name of the compute node must be the fully qualified host name (for example, dbm1db01.mydomain.com). Non-fully qualified hostnames (for example, dbm1db01) or IP addresses must not be used for the host target name of the compute node.
   • If you are using Exadata plug-in Release 12.1.0.1.0 or 12.1.0.2.0, the target name of the compute node host target should be the host name in the management network, not the client network.
   • If you are using Exadata plug-in Release 12.1.0.3.0 or later, the target name of the compute node host target can be the host name in the management network or the client network.
4. The host names of the compute nodes and Exadata cells in each individual Database Machine should have a unique prefix. In other words, Exadata Database Machine racks that are not connected by the same InfiniBand network to form one DB Machine should not share the same prefix for the host names.
5. For users of Oracle Enterprise Manager Cloud Control 12c Release 12.1.0.1.0 or if the installed version has been upgraded to Release 12.1.0.2.0 from 12.1.0.1.0 of the plug-in, you must manually push the Exadata plug-in to the agent.
6. For IPV6-based Exadata Database Machine, verify the following information in the schematic files:

   Note:

   The schematic files are usually located in the /opt/oracle.SupportTools/onecommand directory on the compute nodes, and are named databasemachine.xml, catalog.xml or exdatabasemachine.xml.

   1. Check if all the ADMINNAME, ILOMNAME and CLIENTNAME entries are valid host names.
   2. Check if the ADMINIP and ILOMIP are in IPv6 format for the cells, compute nodes, and Cisco switch entries.
   3. As an example, the entries would be formatted like this:

      <ITEM ID="2">
            <TYPE>cellnode</TYPE>
            <ADMINNAME>XXXXXceladm04</ADMINNAME>
            <ADMINIP>XXXX:YYYY:XXXX:YYYY:XXXX:YYYY:XXXX:YYYY</ADMINIP>
            <ILOMNAME>XXXXXceladm04-ilom</ILOMNAME>
            <ILOMIP>XXXX:ZZZZ:XXXX:ZZZZ:XXXX:ZZZZ:XXXX:ZZZZ</ILOMIP>
            <ULOCATION>2</ULOCATION>
            <UHEIGHT>2</UHEIGHT>
      </ITEM>
      

   4. Ensure that the IPv4 address is used for InfiniBand Switch and PDU as they do not support IPV6. For more details, see IPv6 support status on Exadata Database Machine (Doc ID 2056895.1), available in My Oracle Support:
      https://support.oracle.com/rs?type=doc&id=2056895.1

3.2 Discovering an Exadata Database Machine

To discover a Database Machine target, follow these steps:

1. From the Enterprise Manager home page, select the Setup menu (upper right corner), Add Target, and then Add Targets Manually.
2. On the Add Targets Manually page (Figure 3-1), click Add Targets Using Guided Process. From Add Using Guided Process window, select Oracle Exadata Database Machine from the list and click Add (Figure 3-2).

   Figure 3-1 Add Targets Manually

   
   

   

   
   

   Figure 3-2 Select Oracle Exadata Database Machine

   
   

   

   
   
3. On the Oracle Exadata Database Machine Discovery page (Figure 3-3), select one of the following tasks:

   • Discover a new Database Machine and its hardware components as targets. With this selection, choose one of the following monitoring options:

     • 13c target type. With this target type, enhanced monitoring is available through Enterprise Manager Cloud Control 13c (additional credentials are required).

       Enhanced monitoring of the 13c target type allows the Exadata plug-in to monitor the Exadata Storage Server and KVM, and the Enterprise Manager Systems Infrastructure (EMSI) plug-in monitors of all other hardware targets with the Exadata rack. The EMSI plug-in is included with your installation of Enterprise Manager.

     • 12c target type.

     After you select the monitoring type, the table will update to show the target types and the credentials required for discovery.

   • Discover newly added hardware components in an existing Database Machine as targets. Select a Database Machine from the drop-down menu. The table will update to show the target types and the credentials required for discovery.

   Figure 3-3 Oracle Exadata Database Machine Discovery Page

   
   

   

   
   

   Click Discover Targets. The Exadata Discovery Wizard begins.

4. On the Discovery Inputs page (Figure 3-4), enter the following information:

   • For the Discovery Agents section:

     • Agent URL: The Agent deployed on compute node. Click the search icon to select from available URLs.

       Note:

       For IPv6-based DB Machines, select the agents that are deployed on hosts with dual stack (IPv4 and IPv6) support.

   • For the Schematic Files section:

     • Once you have specified the Agent URL, a new row (hostname and schematic file information) is automatically added. The default schematic file, databasemachine.xml, describes the hardware components of the Exadata Database Machine.

     • Click Set Credential to set the credentials for the host.

     • Check/modify the schematic file location.

     • Select the schematic file name from drop-down menu.

     Note:

     You only need to customize the Schematic file location if one of the following is true:

     • The Schematic file is not in the default location.

     • The Database Machine contains a Storage Expansion Rack.

     • The hardware components are specified in multiple schematic files.

     Figure 3-4 Database Machine Discovery: Discovery Inputs

     
     

     

     
     

   Click Next.

5. On the InfiniBand Discovery page (Figure 3-5), enter the following information:

   • IB Switch Host Name: The InfiniBand switch host name. The IB Switch host name is usually pre-populated.

   • InfiniBand Switch ILOM host credential: The user name (usually ilom-admin or ilom-operator) and password for the InfiniBand switch ILOM host.

   Figure 3-5 Database Machine Discovery: InfiniBand Discovery

   
   

   

   
   

   Click Next.

6. On the Prerequisite Check page (Figure 3-6), Enterprise Manager will try to discover hardware components dynamically based on your environment. If any critical issues are encountered, then you can click Back to resolve them. Enterprise Manager will the show the problem and its severity (Info, Warning, or Critical).

   Any warning issues or informational messages may also be displayed. These do not interfere with the discovery process.

   Figure 3-6 Database Machine Discovery: Prerequisite Check

   
   

   

   
   

   Click Next.

7. On the Components page (Figure 3-7), the components below are pre-selected. You can deselect any unwanted component. Components include, but are not limited to, the following:

   • Compute Node: Select the hosts that are compute nodes in the Oracle Exadata Database Machine.

   • Oracle Exadata Storage Server: Select the Oracle Exadata Storage Servers that are part of this Oracle Exadata Database Machine target.

   • InfiniBand Switch: Select the InfiniBand Switches that are part of the Oracle Exadata Database Machine. These also will be added as managed targets.

   • Ethernet Switch: Select the Ethernet switches that are part of the Oracle Exadata Database Machine. The Ethernet switches will be added as managed targets.

   • Compute Node ILOM: Select the Integrated Lights Out Managers (ILOM) of the compute nodes that are part of this Oracle Exadata Database Machine. These ILOMs will be added as managed targets.

   • KVM: Select the KVM switches that are part of the Oracle Exadata Database Machine. The KVM switches will be added as managed targets.

   Note:

   A partitioned rack can list other components that should be deselected manually (for example, compute nodes).

   Figure 3-7 Database Machine Discovery: Components

   
   

   

   
   

   Click Next.

8. On the Monitoring Agents page (Figure 3-8), select an agent for the Monitoring Agent, and then one for the Backup Monitoring Agent from the drop-down selection.

   Note:

   A Warning will be thrown if only one agent is used as monitoring and backup agents. Click Select Agents to add more agents, if available.

   Figure 3-8 Database Machine Discovery: Monitoring Agents

   
   

   

   
   

   Note:

   Reminder: For IPv6-based DB Machines, select the agents that are deployed on hosts with dual stack (IPv4 and IPv6) support.

   Click Next.

9. On the Monitoring Credential page, select either ExaCLI or CellCLI for monitoring and set the monitoring credentials appropriately. ExaCLI is recommended as it allows SSH access to the Exadata Storage Server to be locked down for higher security. Select from the following:
   • Monitor using ExaCLI where the credentials are the same for all cells. For example:
     
     
     
   • Monitor using ExaCLI where the credentials are different for all cells. For example:
     
     
     
   • Monitor using CellCLI. If you select CellCLI, then you must supply the root credentials (which are not stored) to set up SSH access from the monitoring agent to the Exadata Storage Server.
     
     
     
10. On the Agent Credential page, set the credentials for all components within the Exadata Database Machine (for example, the Storage Server, PDU, InfiniBand Switch, and so forth).

    Click Set Credential to set the credential for the component. If the users and passwords are the same, choose Same for all and enter the user and password combination.

    Select Set up ASR subscription option for receiving Oracle Auto Service Request (ASR)-oriented traps in Enterprise Manager. This option is needed if you are using Enterprise Manager to raise a Service Request (SR) instead of using ASR Manager on the cell.

    Depending on the component, you may need to provide additional information. For example, in the Exadata Storage Server Credentials window, you must enter the SNMP Credentials as well as the Root Credentials (Figure 3-9):

    Figure 3-9 Example: Set Oracle Exadata Storage Server Credentials

    
    

    

    
    

    Note:

    For IPv6 environments, only SNMPV3 credential type (SNMPV3Creds) is supported.

    The SNMP community String is like a password. It is sent along with each SNMP Get-Request, allow or deny access to device. Please check with your network administrator to get the community string details for your enterprise.

    The Agent requires SSH access to the Cisco Ethernet switch. You must specify this SSH access credentials when setting credentials for the Ethernet switch.

    Click OK and set the credentials for all other components. Once all credentials are set, the Set Credentials page will update to show the credentials set (as shown in Figure 3-10):

    Figure 3-10 Database Machine Discovery: Credentials

    
    

    

    
    

    Click Next.

11. On the Review page, verify each section is correct. Figure 3-11 shows an example of an accurate review.

    Figure 3-11 Database Machine Discovery: Review

    
    

    

    
    

    Click Submit.

12. After the Database Machine components are promoted, a Target Creation Summary page displays to show all discovered targets and components. If there are any errors, the Status column will display a red flag. Place your cursor over the flag for a pop-up window for additional information about the error (Figure 3-12):

    Figure 3-12 Database Machine Discovery: Target Creation Summary

    
    

    

    
    

The discovery of the Oracle Exadata Database Machine is complete.

3.3 Discover Oracle SuperCluster as an Exadata Database Machine

You can use the Exadata plug-in to discover and monitor an Oracle SuperCluster system. You can monitor only the Database and Exadata components of the Oracle SuperCluster installed on LDOM and Zones. To monitor specific LDOM or virtualization configurations at the operating system level, please use Oracle Enterprise Manager Ops Center 12c.

To discover the Oracle SuperCluster as an Exadata Database Machine:

1. See the Discovering and Promoting Oracle Solaris Zones in the Oracle® Enterprise Manager Cloud Control Administrator's Guide for completing the prerequisite of pushing the agent to the control domain and zones.

2. Open a Service Request (SR) and upload the following files from your first 11gR2 LDOM:

   /opt/oracle.SupportTools/onecommand/onecommand.params
   /opt/oracle.SupportTools/onecommand/config.dat
   

   Note:

   Make a backup of your /opt/oracle.SupportTools/onecommand directory as you will overwrite it with the new configurations.

3. Oracle Support will provide you the file(s) generated based on the information provided in your configuration files. Copy all files into your /opt/oracle.SupportTools/onecommand directory.

4. Ensure that READ permissions are open for all files in this directory as well as in the /opt/oracle.SupportTools/em directory. The agent must be able to read these files during discovery.

5. Run Self Update from Enterprise Manager to download the Solaris SPARC agent software on the OMS. Apply that download to the OMS to make it available for deployment:

   1. From Enterprise Manager, click Setup, then Extensibility, and finally Self Update. Click Agent Software.

   2. Select Oracle Solaris on SPARC (64-bit).

   3. If the status is Available, then click Download from the Actions menu.

   4. Once downloaded, you must Apply it to the OMS from the same Self Update page.

   For more details on Self Update in Online or Offline mode, see the Updating Cloud Control chapter in the Oracle® Enterprise Manager Cloud Control Administrator's Guide.

6. Install an agent on each of the Database nodes of the Oracle SuperCluster:

   1. From Enterprise Manager, select Setup, then Add Target, and finally Add Targets Manually.

   2. Select Add Host Targets (selected by default) and click Add Host.

   3. Continue with the installation wizard and ensure that you run root.sh on each node once installation is completed.

   Note:

   You may run into a known issue where you have to edit the Ciphers entry in /etc/ssh/sshd_config and restart ssh (see bug 13822068 for details). See SSHD Connection Errors After Upgrading To Exadata Version 11.2.3.1.0 (Doc ID 1437596.1) in My Oracle Support:

   https://support.oracle.com/rs?type=doc&id=1437596.1
   

7. Configure the database nodes for Database Machine discovery.

   This step is needed to set descriptions of each DB node by updating the DB node IP address, Host Channel Adapter ID, and management hostname in the InfiniBand configurations using set_nodedesc.sh. Enterprise Manager Agents look for this information when discovering the Database Machine.

   Run:

   # ibnetdiscover | grep your_db_nodes
   

   If no output is not returned, then run the following command to set the node descriptions from all database nodes:

   # /bin/ib_set_node_desc_ssc.sh
   

8. Discover Exadata Database Machine from Enterprise Manager using Manual Discovery wizard. This discovery process is the same as any other Exadata DBM target. See Discovering an Exadata Database Machine.

9. Discover the HA RAC Cluster and the cluster databases and configure each target as usual.

3.4 Discovering Grid Infrastructure and RAC

The following sections provide instructions for discovering Grid Infrastructure and Oracle Real Application Clusters targets:

• Discover Grid Infrastructure (Cluster) Targets

• Discover Oracle Real Application Clusters Targets

3.4.1 Discover Grid Infrastructure (Cluster) Targets

1. From the Setup menu, select Add Targets, then Add Targets Manually.
2. In the Add Targets Manually page, select Add Non-Host Targets Using Guided Process (Also Adds Related Targets) and Target Type as Oracle Cluster and High Availability Services.
3. Click Add Using Guided Discovery and follow the wizard.
4. On the Add Cluster Target: Specify Host page, select the first database node and click Continue.
5. The cluster home details are displayed. Ensure that all database nodes are shown in Selected Hosts section.
6. Click Add.

3.4.2 Discover Oracle Real Application Clusters Targets

1. From the Setup menu, select Add Targets, then Add Targets Manually.
2. In the Add Targets Manually page, select Add Non-Host Targets Using Guided Process (Also Adds Related Targets) and Target Type as Oracle Database, Listener and Automatic Storage Management.
3. Click Add Using Guided Discovery and follow the wizard.
4. In the wizard pages, configure Cluster Databases (dbsnmp password required) and Cluster ASM (asmsnmp password required).

3.5 Switch from Using CellCLI to ExaCLI after Storage Server Locked Down

You can at any time lock down an Exadata Storage Server and switch to ExaCLI for monitoring/administration (See Exadata Storage Server User Guide to know more about ExaCLI). From the Exadata Storage Server menu, select Target Setup, then select Monitoring Configuration, as shown in Figure 3-13:

Figure 3-13 Monitoring Credentials Menu Item

On the Monitoring Configuration page, set the Support ExaCLI value to 1 (the default is 0), as shown in Figure 3-14:

Figure 3-14 Monitoring Configuration: Set to Monitor Using ExaCLI