4 Post-Discovery Configuration and Verification

discoverypost-discovery configuration and verificationOnce the Oracle Exadata Database Machine has been discovered through Enterprise Manager Cloud Control 13c, you must complete post-discovery configuration on the following targets:

Note:

You must remove the SNMP notification on the cell, InfiniBand switch, ILOM, Cisco switch, PDU, and KVM manually if you remove these targets from Enterprise Manager.

Starting with Exadata plug-in Release 12.1.0.3.0, when the Exadata Database Machine target is deleted through Enterprise Manager Cloud Control, there is an option to remove the SNMP notification from the cells and InfiniBand switches.

Note:

View a video of how to monitor your Exadata environment:

https://apex.oracle.com/pls/apex/f?p=44785:24:0::NO:24:P24_CONTENT_ID,P24_PREV_PAGE:5664,1

4.1 Set Up SNMP Trap Forwarding on Compute Node

Note:

If you do not need to monitor KVM, then you can skip this section.

The traps from the KVM will be sent to UDP port 162 on the monitoring agent host. For the Enterprise Manager Agent to receive traps from the KVM, SNMP trap forwarding must be set up on the host where the Enterprise Manager Agent is running so that the snmptrapd daemon can receive traps using port 162 and forward the trap to the agent receivelet's listening port.

Ideally, you should perform the steps on all compute nodes in preparation for a scenario when monitoring is moved to a compute node that was not previously being used to monitor the KVM targets.

4.1.1 Set Up Compute Node Agent - Linux

The following commands must be run as root user:

Get the port value from EMD_URL property in the <AGENT_INST>/sysman/config/emd.properties file. This is the port at which receivelet will listen over UDP for traps (for this example, the port value is 3872).
Confirm that the receivelet is listening on this port over UDP. An entry should be seen in the below command's output:
```
# netstat -an | grep 3872 | grep udp
```
Run the following command to stop the SNMP trap daemon:
```
# service snmptrapd stop
```
Typically, the service command will be located in the /sbin directory.
Add the following entry to the /etc/snmp/snmptrapd.conf configuration file:
```
authcommunity log,execute,net <community string>
forward default udp:localhost:3872
```
Note:

If the /etc/snmp/snmptrapd.conf file does not exist, create one and add the lines as described above.
Edit the /etc/hosts.allow file to add the following line:
```
snmptrapd : ALL
```
Run the following command:
```
# chkconfig snmptrapd on
```
Run the following command to start the SNMP trap daemon:
```
# service snmptrapd start
```

4.1.2 Set Up Compute Node Agent - Solaris

The following commands must be run as root user:

Get the port value from EMD_URL property in the <AGENT_INST>/sysman/config/emd.properties file. This is the port at which receivelet will listen over UDP for traps (for this example, the port value is 3872).
Confirm that the receivelet is listening on this port over UDP. An entry should be seen in the below command's output:
```
# netstat -an | grep 3872 | grep udp
```
Run the following command to stop the SNMP trap daemon:
```
# svcadm disable snmptrapd
```
You may see the following message. If so, then you can ignore it:
```
svcadm: Pattern 'snmptrapd' doesn't match any instances
```
Typically, the svcadm command will be located in the /usr/sbin directory.
Add the following entry to the /etc/net-snmp/snmp/snmptrapd.conf configuration file:
```
authcommunity log,execute,net <community string>
forward default udp:localhost:3872
```
Note:

If the /etc/net-snmp/snmp/snmptrapd.conf file does not exist, create one and add the lines as described above.

Create a /tmp/snmptrapd.xml file with the following content:

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM
"/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<service_bundle type='manifest' name='snmptrapd'>
  <service name='application/management/snmptrapd' type='service' version='1'>
    <create_default_instance enabled='true' />
    <single_instance />
    <dependency name='network' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/milestone/network:default' />
    </dependency>
    <exec_method type='method' name='start' exec="/usr/sbin/snmptrapd -Lsd -p /var/run/snmptrapd.pid" timeout_seconds ="60"/>
    <exec_method type='method' name='stop' exec=':kill' timeout_seconds='60'/>
    <stability value='Evolving' />
    <template>
      <common_name>
        <loctext xml:lang='C'>net-snmp SNMP trap daemon</loctext>
      </common_name>
    </template>
  </service>
</service_bundle>

Validate the /tmp/snmptrapd.xml file you have just created in step 5:
```
# svccfg validate /tmp/snmptrapd.xml
```
Import the /tmp/snmptrapd.xml file:
```
# svccfg import snmptrapd.xml 
```

Make sure that the snmptrapd daemon is running:

# svcs snmptrapd

An example of the output is:

STATE          STIME    FMRI
online         16:20:00 svc:/application/management/snmptrapd:default

If needed, you can start the snmptrapd daemon with the following command:
```
# svcadm enable snmptrapd
```

4.2 Configure Storage Cell SNMP for Enterprise Manager Monitoring

Exadata Storage Cell SNMP configuration is performed using the cellcli command and can be run in batch using dcli from a compute node.

Note:

For Enterprise Manager Cloud Control 12c Bundle Pack 1 (BP1):

The SNMP trap subscription for Exadata Storage Cells is done automatically after finishing the guided discovery process.
For Exadata plug-in Release 12.1.0.2.0 and later:

During the discovery process, you can optionally provide the necessary root credentials to subscribe for SNMP traps from Exadata Storage Cells. If you have done so, then you can skip the remaining steps of this section and proceed with Configure and Verify SNMP for InfiniBand Switch Targets.
While the public string is used for the SNMP examples in the following sections, it is supported to use any valid SNMP community string.
Because of a limitation with the V1 protocol and JDMK API, if an IPv6 component sends SNMPv1 traps, then the sender's IP address is always 0.0.0.0. Therefore, if the Exadata Storage Server environment is IPv6, then the snmpv1 subscription should not be allowed.
Use an SNMP V3 subscription for IPv6-based cell targets because of the limitation in SNMP V1 protocol.

4.2.1 Using the `ALTER CELL` Command

While using the ALTER CELL command, all existing subscribers should be specified along with the new subscriber being added. Similarly, you can also modify the notificationPolicy or notificationMethod attributes.

While using the ALTER CELL command, the host= and community= attribute values should be quoted, and type= is NOT quoted.

If you are using the DCLI utility to set up SNMP alerting, then any command containing punctuation, which will be interpreted by the local shell, must be enclosed with double quotation marks. If the command includes the following characters, then outer quotation marks and escape characters are required:

$ (dollar sign)
' (quotation mark)
< (less than)
> (greater than)
( ) (parentheses)

The backslash (\) is the escape character that allows the characters to be passed to the CellCLI utility without being interpreted by the remote shell.

4.2.2 Check Current SNMP Configuration

Check the current SNMP configuration using the following cellcli commands:

To list the current subscribers for SNMP:

# cellcli -e list cell attributes snmpSubscriber

When correctly configured, this command should list the primary and backup agents for the cell target, for example:

# cellcli -e list cell attributes snmpSubscriber
((host=[hostname.mycompany.com],port=3872,community=public),
(host=[hostname2.mycompany.com],port=3872,community=public))

To list the currently configured notification methods:
```
# cellcli -e list cell attributes notificationMethod
```
Possible values are snmp, mail and snmp,mail. When correctly configured, this command should return either snmp or snmp,mail, for example:
```
# cellcli -e list cell attributes notificationMethod
snmp,mail
```
To list the currently configured notification policy for SNMP:
```
# cellcli -e list cell attributes notificationPolicy
```
Possible values are any or all of Critical, Warning, and Clear. When correctly configured, this command should return Critical, Warning, Clear, for example:
```
# cellcli -e list cell attributes notificationPolicy
Critical,Warning,Clear
```

4.2.3 Configure SNMP Values Using `cellcli`

To set the values of snmpSubscriber, notificationMethod and notificationPolicy:

Set the snmpSubscriber value:
- SNMP V1
```
# cellcli -e "alter cell snmpSubscriber=((host='[host name]',port=[port]),(host='[host name]',port=[port]))"
```
  Where [host name] and [port] values are Agents monitoring your cell targets.
- SNMP V3
  
  The process is same as above, but along with the host and port, an SNMP V3 user and type must be provided:
```
ALTER CELL snmpSubscriber=((host='host1.example.com',port=[port], snmpUser='[v3user]', type=V3))
```
  Before you run the command, make sure that a v3user is a valid SNMP user on the cell. You can check by running following command:
```
cellcli -e "list cell attributes snmpUser"
```
  If there is no valid SNMP user on the cell, then add a V3 user with the following command:
```
ALTER CELL snmpUser=((name='[v3user]', authProtocol='MD5', authPassword='[passwd]', privProtocol='DES', privPassword='[passwd]'))
```
  Note:
  
  The Enterprise Manager Agent supports only DES and MD5/SHA combination as of now. AES is not supported.
- Oracle Auto Service Request (ASR) subscription
  
  The above two steps describe the process of adding an SNMP subscription for cell traps in MS MIB format. If you want to subscribe for SNMP traps in the Oracle Auto Service Request (ASR) format as well, then along with the entries mentioned above you must add another entry with type=ASR (for V3, it is type=V3ASR).
  
  For example, for SNMP V1:
```
ALTER CELL snmpSubscriber=((host='host1.example.com',port=[port],community='[community string]'), (host='host1.example.com',port=[port],community='[community]',type=ASR))
```
  For example, for SNMP V3:
```
ALTER CELL snmpSubscriber=(((host='host1.example.com',port=[port],community='[community string]'),(host='host1.example.com',port=[port], snmpUser='[v3user]', type=V3ASR))
```
Note:

Take special care not to overwrite existing settings for snmpSubsriber. If there are existing subscribers, then append the agent subscriptions. For example, if the cellcli -e list cell attributes snmpSubscriber command returned:
```
# cellcli -e list cell attributes snmpSubscriber
 ((host=ilm-asr1.example.com,port=162,community=public,type=asr))
```
Then you must append the Agent subscriptions:
```
#cellcli -e "alter cell snmpSubscriber=((host='ilm-asr1.example.com',port=162,
community=public,type=asr),(host='[host name]',port=[port]),(host='[host name]',port=[port]))"
```
For SNMP V3, update the following in Enterprise Manager:
- Ensure that the V3 monitoring credentials are set up on the cell target in Enterprise Manager which match the v3 user on the cell:
  - Log in to Enterprise Manager.
  - From the Setup menu, select Security, then select Monitoring Credentials.
  - Select the Exadata Storage Server type and click Manage Monitoring Credentials.
  - Select the cell and set the V3 credentials.
- Ensure that the snmpengineid property is set up on the target in Enterprise Manager:
  - Log in to Enterprise Manager.
  - Go to the Exadata Storage Server home page.
  - From the Exadata Storge Server menu, select Monitoring Configuration.
  - If the snmpengineid property is not set, then set it manually in Enterprise Manager. You can obtain it from the cell with the following command
```
list cell attributes snmpengineidhex
```
    All of the above steps are automatically taken care of it SNMP V3 subscription along with ASR option are selected during Exadata Database Machine discovery.

Set the notificationMethod value:

# cellcli -e "alter cell notificationMethod='snmp,mail'"

Set the notificationPolicy value:

# cellcli -e "alter cell notificationPolicy='Critical,Warning,Clear'"

4.2.4 Configure SNMP Values Using `dcli` (optional)

The SNMP configuration commands can be run using dcli to perform the configuration in batch on all storage cells:

Set the snmpSubscriber value:

$ dcli -g cell_group -l root "cellcli -e \"alter cell snmpSubscriber=((host='[host name]',port=[port]),(host='[host name]',port=[port]))\""

Where [host name] and [port] values are Agents monitoring your cell targets.

Set the notificationMethod value:

$ dcli -g cell_group -l root "cellcli -e \"alter cell notificationMethod='snmp,mail'\""

Set the notificationPolicy value:

$ dcli -g cell_group -l root "cellcli -e \"alter cell notificationPolicy='Critical,Warning,Clear'\""

4.2.5 Verify SSH Connectivity

Open a new terminal and verify whether the SSH connectivity was successfully established:

$ ssh -l cellmonitor <cell_ipaddress> cellcli -e 'list cell detail'

If you are not prompted for any password, then you can assume that the connectivity is established.
If you are asked to confirm whether you want to continue connecting, specify Yes.

4.2.6 Remove a Subscription

To remove the subscription, use the ALTER CELL command again by excluding the host name that you want to remove from the snmpsubscriber list.

Note:

The SNMP receivelet listens on a single address and port for all monitored targets. The port is the UDP port with the same number as the TCP port used in the EMD_URL.

By default, the SNMP receivelet listens on all addresses; if the property SnmpRecvletListenNIC is set in the emd.properties file, the receivelet will attempt to resolve the value as either a name or IP address, and listen on only that address.

This parameter is independent of AgentListenOnAllNICs and EMD_URL because in some installations, the Agent may need to communicate with the OMS and with managed targets on different networks.

4.3 Configure and Verify SNMP for InfiniBand Switch Targets

The SNMP configuration for Enterprise Manager monitoring of InfiniBand Switches is done automatically as part of the Enterprise Manager guided discovery process. It is good practice, however, to verify that SNMP configuration has been successful.

Note:

For Enterprise Manager Cloud Control 12c Bundle Pack 1 (BP1):

The SNMP Trap setup for Infiniband Switches is done automatically after finishing the guided discovery process.
For Exadata plug-in Release 12.1.0.2.0 and later:

During the discovery process, you can optionally provide the necessary root credentials to set up SNMP trap for the InfiniBand Switch. If you have done so, then you can skip the remaining steps of this section and proceed with Configure the Compute Node ILOM SNMP for Enterprise Manager Monitoring.

To configure (if necessary) and verify the SNMP configuration for an InfiniBand Switch:

Log in to the InfiniBand Switch ILOM web interface using the URL https://<ib_switch_hostname> as root.

Note:

Try using Internet Explorer if the console does not display all fields/values in your browser of choice.
Click Configuration, then System Management Access, and finally SNMP.
Ensure the following values are set:
```
State=Enabled
Port=161
Protocols=v1,v2c,v3
```
If you need to make changes, make sure you click Save.
Click Configuration, then System Management Access, then SNMP, and finally the Communities link. The page scrolls to the SNMP Communities table. In the SNMP Communities table, click Add.

A new window opens.
Type the community name into the Community Name field. Select the permissions from the Permission drop-down menu. Click Save.

The community is created, and the window closes.
Click Alert Management.
If not already listed, for each Agent that monitors the InfiniBand Switch target, select an empty alert (one that has the Destination Summary 0.0.0.0, snmp v1, community 'public') and click Edit. Provide the following values:
```
Level = Minor
Type = SNMP Trap
Address = [agent compute node hostname]
Destination Port = [agent port]
SNMP Version = v1
Community Name = public
```
Click Save.
Verify the InfiniBand Switch SNMP configuration for Enterprise Manager monitoring:
```
snmpget -v 1 -c <community_string> <hostname_of_IB_switch> 1.3.6.1.4.1.42.2.70.101.1.1.9.1.1.5
```
For example:
```
$ snmpget -v 1 -c public my_IB_switch.my_company.com 1.3.6.1.4.1.42.2.70.101.1.1.9.1.1.5
SNMPv2-SMI::enterprises.42.2.70.101.1.1.9.1.1.5 = INTEGER: 1
```
Note:

If the Timeout message is displayed as a output for the above command, then it means that the InfiniBand Switch is not yet configured for SNMP.

Note:

To remove the subscription, follow steps 1 to 5 above (step 3 is not needed). In step 5, set the Agent compute node host name to 0.0.0.0 and the port to 0.

4.3.1 Set Up SNMP for InfiniBand Switch Targets Using Enterprise Manager

You can set up SNMP for InfiniBand Switch targets using the Enterprise Manager Cloud Control console:

Navigate to the IB Network target (not the individual switches) and select Administration.
Select the IB Switch target type, then one of the IB Switch targets.
Select the Setup SNMP Subscription command, then select the management agent URL that monitors the InfiniBand switch target from the Agent URL list. Click Next.
Provide credentials for the InfiniBand switch. Click Next.
Review the details you provided. If there are no further changes, then click Submit.

Perform steps 1-5 for both the Monitoring Agent and Backup Monitoring Agent of the InfiniBand switch target.

4.4 Configure the Compute Node ILOM SNMP for Enterprise Manager Monitoring

Note:

These instructions apply only for the Exadata Database Machine and are not applicable to the Oracle SuperCluster.

The compute node ILOM targets are responsible for displaying a number of disk failure alerts for their respective compute node that are received as SNMP traps. For Enterprise Manager to receive those traps, the /opt/oracle.cellos/compmon/exadata_mon_hw_asr.pl script must be run to configure SNMP subscriptions for the agents (both primary and backup agents) that have been configured to monitor the compute node ILOM targets. This step is applicable to Exadata plug-in Release 12.1.0.2.0 and later.

The exadata_mon_hw_asr.pl script is run as the root user with the -set_snmp_subscribers parameter to add SNMP subscribers. For example:

# /opt/oracle.cellos/compmon/exadata_mon_hw_asr.pl -set_snmp_subscribers "(host=hostname1.mycompany.com,port=3872,community=public,type=asr,fromip=11.222.33.444),(host=hostname2.mycompany.com,port=3872,community=public,type=asr,fromip=12.345.67.890)"
Try to add ASR destination Host - hostname1.mycompany.com IP - 11.222.33.44 Port - 3872 Community - public From IP - 22.333.44.555
Try to add ASR destination Host - hostname2.com IP - 11.111.11.111 Port - 3872 Community - public From IP - 22.333.44.555

The script needs to be run on each compute node:

The host values should be the hostnames of the agents configured to monitor the compute node ILOM target associated with the compute node.
The fromip values should be the IP address of the compute node that the compute node ILOM target is associated with.

For example, if you have an X2-2 machine with compute node targets edbm01db01 through edbm01db08 and associated compute node ILOM targets edbm01db01-c through edbm01db08-c, then you would need to run the script once on each compute node - therefore, the script would be run eight times in total.

On compute node edbm01db01, the host and port values would be the hostnames and ports of the agents monitoring compute node ILOM target edbm01db01-c and the fromip value would be the IP address of the compute node itself, edbm01db01.
On compute node edbm01db02, the host and port values would be the hostnames and ports of the agents monitoring compute node ILOM target edbm01db02-c and the 'fromip' value would be the IP address of the compute node itself, edbm01db02, ... and so on.

This is a good example of where Manual selection of Management Agents for targets is useful. If the first two compute nodes are always the Monitoring Agent and Backup Monitoring Agent, then it is easy to work out the values needed for -set_snmp_subscribers parameters, the host and port values would be the same for all compute nodes.

Note:

The exadata_mon_hw_asr.pl script, overwrites any existing SNMP subscriptions. While setting the SNMP subscribers, make sure that current subscribers are included in the new list of subscribers.

It is possible to use the exadata_mon_hw_asr.pl script to get the current set of subscribers using the -get_snmp_subscribers parameter.

For example:

# /opt/oracle.cellos/compmon/exadata_mon_hw_asr.pl -get_snmp_subscribers -type=asr

Suppose the current list is:

(host=hostname1.mycompany.com,port=162,community=public,type=asr,fromip=11.222.33.444),
(host=hostname2.mycompany.com,port=162,community=public,type=asr,fromip=11.222.33.444)

Then new subscriptions can be added using the following command:

# /opt/oracle.cellos/compmon/exadata_mon_hw_asr.pl -set_snmp_subscribers
 "(host=asrhostname1.mycompany.com,port=162,community=public,type=asr,fromip=11.222.33.444),
(host=asrhostname2.mycompany.com,port=162,community=public,type=asr,fromip=11.222.33.444),
(host=hostname1.mycompany.com,port=3872,community=public,type=asr,fromip=11.222.33.444),
(host=hostname2.mycompany.com,port=3872,community=public,type=asr,fromip=11.222.33.444)"

After adding the new subscribers, run the command exadata_mon_hw_asr.pl script with the -get_snmp_subscribers parameter to get the list of SNMP subscribers and verify the new SNMP subscriptions were added successfully. For example:

# /opt/oracle.cellos/compmon/exadata_mon_hw_asr.pl -get_snmp_subscribers -type=asr
(host=asrhostname1.mycompany.com,port=162,community=public,type=asr,fromip=10.10.10.226),
(host=asrhostname2.mycompany.com,port=162,community=public,type=asr,fromip=10.10.10.226),
(host=hostname1.mycompany.com,port=3872,community=public,type=asr,fromip=10.10.10.226),
(host=hostname2.mycompany.com,port=3872,community=public,type=asr,fromip=10.10.10.226)

4.4.1 Configure the Compute Node ILOM SNMP for Compute Nodes Running Management Server

For compute nodes running Management Server, the call to the exadata_mon_hw_asr.pl script is replaced with dbmCLI commands, which are similar to cellCLI commands on the Storage Cells.

Oracle Exadata Database Machine X5-2 (up to X6) runs the Management Server on compute nodes.

4.4.2 Configure SNMP Values Using dbmcli

Set the snmpSubscriber value:

# dbmcli -e "ALTER DBSERVER snmpSubscriber=((host=[agent1_hostname],port=[agent1_port],community=public,type=asr),(host='agent2_hostname',port=<agent's port>,type=asr))"

Where [host name] maps to the Enterprise Manager agents monitoring your compute node target.

Set the notificationMethod value:

# dbmcli -e "ALTER DBSERVER notificationMethod="mail,snmp", notificationPolicy="critical,warning,clear", smtpFrom="admin", smtpFromAddr=[sender email address], smtpPort=[port], smtpServer=[smtp server], smtpToAddr=[recipient email address], smtpUseSSL=TRUE

Verify the updates by executing:

# dbmcli -e "LIST DBSERVER attributes snmpSubscriber"

For Virtualized Exadata, these commands need to be executed in Dom0.

4.4.3 Configure SNMP for Oracle SuperCluster

Log in to the Oracle SuperCluster ILOM web interface, which is accessible via a browser using the url http://<ilom_hostname> and the root credentials.
Select Configuration, then Alert Management.
Add a row for each agent.
Select an empty slot and click Edit.
Specify the following information:
- Level: Minor
- Type: SNMP Trap
- Address: IP Address of the Agent
- Destination Port: Uncheck Autoselect, then specify Agent port manually
- SNMP Version: v1
- Community Name: {your chosen community name}
Click Save.

4.5 Verify Configuration for Oracle ILOM Server

To verify that Alerts can be successfully raised and cleared for the Oracle ILOM Server targets, perform the following steps:

Log in to the Enterprise Manager Cloud Control console as an administrator.
Click Targets and select Exadata. Select one of the Oracle ILOM Server targets using the Target Navigation pane.

The ILOM target page will be displayed, showing the current status of that target as well as any incidents that have been raised for it.
Raise an alert manually from the ILOM Server being validated. Run following command as root on the first database server in the cluster:
```
# ipmitool -I lan -H sclczdb01-c -U oemuser -P oempasswd -L OPERATOR event PS0/VINOK deassert
```
The output will be similar to:
```
Finding sensor PS0/VINOK... ok
0 | Pre-Init Time-stamp | Power Supply #0x65 | State Deasserted
```
After running the above command, wait a few minutes then refresh the ILOM target page. An incident should appear in the Incidents section.
Clear the alert raised in step 3. Run the following command as root on the first database server in the cluster:
```
# ipmitool -I lan -H sclczdb01-c -U oemuser -P oempasswd -L OPERATOR event PS0/VINOK assert
```
The output will be similar to:
```
Finding sensor PS0/VINOK... ok
0 | Pre-Init Time-stamp | Power Supply #0x65 | State Asserted
```
After running the above command, wait a few minutes then refresh the ILOM target page. The incident that was raised in step 3 should show as cleared in the Incidents section.

Note:

Do not forget to clear the alert raised in step 3, as it was raised for testing only and did not reflect a true fault condition!
Repeat for the remaining configured ILOM service processors in your Oracle Database Machine.

4.6 Set Up SNMP for Cisco Ethernet Switch Targets

The Cisco Ethernet Switch must be configured to allow the Agents that monitor it to be able to both poll the switch and to receive SNMP alerts from the switch. To allow this, perform the following steps (swapping the example switch name dm01sw-ip with the name of the Cisco Ethernet Switch target being configured):

# ssh cisco-switch-host -l <admin user>
User Access Verification Password: 
dm01sw-ip> enable
Password:
dm01sw-ip# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
dm01sw-ip(config)#

Enable access to allow the Agents monitoring Cisco Switch target to poll the switch.
In the command, [EMagentIPaddr] is the IP address of the server where the Enterprise Manager Agent is running. The SNMP community specified must match the value provided when configuring the Cisco Switch target:
```
dm01sw-ip(config)# access-list 1 permit [EMagentIPaddr]
dm01sw-ip(config)# snmp-server community <community_string> ro 1
```
Set the monitoring Agent as the location where SNMP traps are delivered. The SNMP community specified must match the value provided during Enterprise Manager Cisco Switch Management Plug-In setup:
```
dm01sw-ip(config)# snmp-server host <EMagentIPaddr> version 1 <community string> udp-port [EMagentRecvltListenPort]
```
Where [EMagentRecvltListenPort] is the EMD_URL port of the emagent or SnmpRecvletListenNIC property value if it is enabled.
Configure the Cisco Switch to send only environmental monitor SNMP traps:
```
dm01sw-ip(config)# snmp-server enable traps envmon
```

Verify settings and save the configuration:

dm01sw-ip(config)# end
dm01sw-ip# show running-config
dm01sw-ip# copy running-config startup-config

4.6.1 Verify the Cisco Ethernet Switch SNMP Configuration

Run the snmpwalk command line utility or equivalent tool to verify the Cisco Switch configuration.

Run the following commands to fetch and display the data from the Cisco switch:

$ snmpget –v 1 –c <community_string> <hostname_of_cisco_switch> 1.3.6.1.4.1.9.2.1.56.0
$ snmpget –v 2c –c <community_string> <hostname_of_cisco_switch> 1.3.6.1.4.1.9.2.1.56.0

Note:

If a timeout message is displayed as an output for the above command, then it means that the Cisco Switch is not yet configured correctly.

4.7 Set Up SNMP for Power Distribution Unit (PDU) Targets

To enable Enterprise Manager to collect metric data and raise events for the PDU target, you must configure the PDU to accept SNMP queries from the Agents that monitor the PDU target. Also, appropriate threshold values for different phase values needs to be set on the PDU.

This section assumes that this is a first time configuration of the PDU. SNMP must be enabled and the trap section completed. Granting SNMP access to a different monitoring Agent IP address is an example where only the "Trap Host Setup" section needs to be changed.

Log in to the PDU network interface through a browser at http://<pdu-name>, for example: http://edbm01-pdu1.example.com
Click Net Configuration, then log in again.
Scroll down until you reach the SNMP section of the frame.

Note:

The network interface for the PDU is a frame within a window. In order to scroll down on this page, you must see the scroll bar for the PDU frame as well as the outside scroll bar for the browser in which you accessed the PDU.
If your PDU is not SNMP-enabled, select the SNMP Enable check box, then click Submit.
Scroll to the NMS region of the frame.
Enter the following in Row 1 under NMS:
- IP: Enter the IP address of the first monitoring Agent
- Community: Enter "public"
Click Submit.

For details on configuring the PDU thresholds settings, see the "Configuring Oracle Exadata Database Machine" chapter in your Exadata Database Machine Owner's Guide. This guide is pre-loaded along with other Exadata user documentation onto your Oracle Exadata Database Machine.

4.7.1 Verify the PDU SNMP Configuration

Use the snmpwalk command line utility or equivalent tool to verify the PDU configuration.

Run the following command to fetch and display the data from PDU:

snmpget –v 1 –c <community_string> <hostname_of_pdu> 1.3.6.1.4.1.2769.1.2.3.1.1.1.0

Note:

If a timeout message is displayed as an output for the above command, then it means that the PDU is not yet configured correctly.

4.8 Set Up SNMP for KVM Targets

The KVM needs to be configured to send SMNP traps to the Agents that monitor it. See the following sections to set up SNMP for KVM targets:

Note:

The SNMP forwarder setup described in Set Up SNMP Trap Forwarding on Compute Node is required on the compute node agent that monitors the KVM target. If the SNMP forwarder is not set up, the agent will not receive traps from the KVM device.

4.8.1 Set Up SNMP in KVM Management Console

Configure the KVM to send traps to the monitoring agent host:

Log in to the KVM management console through a browser at:
```
https://<kvm-name>
```
For example:
```
https://edbm01sw-kvm.example.com
```
Click SNMP under Target Devices in the left-hand navigation bar.
Select the Enable SNMP check box and enter the appropriate community name in the Read, Write, and Trap fields (for example, enter "public").
Click Save.
Click Destinations under SNMP in the left-hand navigation bar.
Enter the IP address of the Agent monitoring the KVM target.
Click Save.

The traps from the KVM device will be sent to UDP port 162 on the monitoring agent host. Refer to Set Up SNMP Trap Forwarding on Compute Node for details to set up the trap forwarder to forward traps received on port 162 to the Enterprise Manager agent receivelet port.

4.8.2 Verify the KVM SNMP Configuration (Base SNMP Configuration) - Linux

Run the snmptrapd command line utility to verify the KVM configuration. Make sure that the steps outlined in Set Up SNMP Trap Forwarding on Compute Node has been performed.

Follow the steps below on the monitoring agent host for the KVM target:

Run the following command:

# service snmptrapd stop
# /usr/sbin/snmptrapd -d -Lsd -p /var/run/snmptrapd.pid

Reboot the KVM to generate SNMP traps. To reboot the KVM:
1. Log in to the KVM.
2. Click Overview on the left side of the screen under Unit View, Appliance.
3. Click Reboot.
4. In the confirmation window, click OK to confirm KVM reboot.

Check that snmptrapd logs messages in the /var/log/messages file from the IP address of the KVM. The following example shows the messages for the KVM's IP address (10.1.1.240):

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]:  Received 72 bytes from UDP: [10.1.1.240]:32768
.
Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0000: 30 82 00 44  02 01 00 04  06 70 75 62  6C 69 63 A4    0..D.....public.
.
Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0016: 82 00 35 06  09 2B 06 01  04 01 D1 32  12 01 40 04    ..5..+.....2..@.
.
Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0032: 0A F5 13 F0  02 01 06 02  01 02 43 03  6D 19 4C 30    ..........C.m.L0
.
Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0048: 82 00 15 30  13 06 0B 2B  06 01 04 01  D1 32 12 02    ...0...+.....2..
.
Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0064: 06 01 04 04  72 6F 6F 74                              ....root
.
Jul 30 14:51:05 dbm1db02 snmptrapd[65800]:

If step 4 is confirmed, the KVM has been configured correctly. Run the following commands:
```
# service snmptrapd stop
# service snmptrapd start
```

4.8.3 Verify the KVM SNMP Configuration (Base SNMP Configuration) - Solaris

Run the snmptrapd command line utility to verify the KVM configuration. Make sure that the steps outlined in Set Up Compute Node Agent - Solaris have been performed.

Follow the steps below on the monitoring agent host for the KVM target:

# svcadm disable snmptrapd
# /usr/sbin/snmptrapd -d -Lsd -p /var/run/snmptrapd.pid

Reboot the KVM to generate SNMP traps. To reboot the KVM:
1. Log in to the KVM.
2. Click Overview on the left side of the screen under Unit View, Appliance.
3. Click Reboot.
4. In the confirmation window, click OK to confirm KVM reboot.

Check that the snmptrapd daemon logs messages in the /var/log/messages file from the IP address of the KVM. The following example shows the messages for the KVM's IP address (10.1.1.240):

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: Received 72 bytes from UDP: [10.1.1.240]:32768

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0000: 30 82 00 44  02 01 00 04 06 70 75 62  6C 69 63 A4    0..D.....public.

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0016: 82 00 35 06  09 2B 06 01 04 01 D1 32  12 01 40 04    ..5..+.....2..@.

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0032: 0A F5 13 F0  02 01 06 02 01 02 43 03  6D 19 4C 30    ..........C.m.L0

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0048: 82 00 15 30  13 06 0B 2B 06 01 04 01  D1 32 12 02    ...0...+.....2..

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]: 0064: 06 01 04 04 72 6F 6F 74    ....root

Jul 30 14:51:05 dbm1db02 snmptrapd[65800]:

If step 3 is confirmed, the KVM has been configured correctly. Stop the snmptrapd process currently running:
```
# ps -ef | grep snmptrapd
# kill -9 <snmptrapd process ID>
```
Start the snmptrapd daemon:
```
# svcadm enable snmptrapd
```

4.8.4 Verify the KVM SNMP Configuration (SNMP Forwarding to Agent)

After performing the steps in Verify the KVM SNMP Configuration (Base SNMP Configuration) - Linux or Verify the KVM SNMP Configuration (Base SNMP Configuration) - Solaris, follow the steps below to validate that the monitoring Agent can receive the SNMP traps generated by the KVM and convert the received traps to Enterprise Manager events.

In the Enterprise Manager Cloud Control console, go to the KVM target page.
In the Targets menu, select Monitoring, then Alert History.

You should see an entry for Reboot Started Status in the Alert History table.
In the Alert History table, click Reboot Started Status. You should see the following warning alert with the message:
```
The appliance is rebooting. Command issued by user root.
```

4.9 Accessing Oracle Support Workbench for Exadata Storage Server

You can access the Oracle Support Workbench for the current Exadata Storage Server to access diagnostic data for problems and incidents related to the cell.

To access the Support Workbench for a single Exadata cell, follow these steps:

From the Target menu of the Exadata Storage Server for which you want to access the Oracle Support Workbench, select Diagnostics, then select Support Workbench as shown in Figure 4-1:

Figure 4-1 Support Workbench Menu
If you are logging in to the Support Workbench for the first time, you must first initialize the Support Workbench properties. For first-time log in, the Initialize Support Workbench Properties window will be displayed as shown in Figure 4-2:

Figure 4-2 Initialize Support Workbench Properties

Enter the Automatic Diagnostic Repository (ADR) Base Path. A typical looks like this (Figure 4-3):

Figure 4-3 ADR Base Path

The ADR Home Path is a target-specific, writable path on the agent where the generated package is to be stored. During initializing, it is not editable as it is generated automatically once the ADR base path is specified.

Click Create.
Once the Support Workbench properties have been initialized, the Support Workbench page displays showing a list of any problems for the Database Machine (Figure 4-4):

Figure 4-4 Support Workbench Page

4.10 Oracle Exadata Database Machine Dashboard Creation

You can create a Database Machine dashboard to monitor the performance and usage metrics of the Database Machine system, its sub-components, as well as all the database system components residing on the Database Machine.

4.10.1 Dashboard Creation for Exadata Database Machine Plug-in Release 12.1.0.3.0 and Later

For Exadata Database Machine plug-in Release 12.1.0.3.0 and later, create the Database Machine Dashboard:

Log in to Enterprise Manager.
From the Enterprise Manager home page, click the Enterprise menu. Select the Job submenu, then Activity.
Select Database Machine Services Dashboard from the drop-down menu next to the Create Job option.
Click Go.
Enter a name of the job (for example, CREATE_DBM_JOB_1).
Click Add and select the DBMachine target. After adding, make sure the target is selected by clicking on the check box next to it.
Click the Parameters tab.
Three options are provided through the drop-down:
- Select Create if it is a new report.
- Select Update for updating an existing report with new components.
- Select Cleanup to remove services created by the Create job executed earlier.
Finally, click Submit to perform the operation.

A message CREATE_DBM_JOB_1 submitted successfully should display.

You can monitor the job by clicking on the link corresponding to the job.

4.10.2 How to Make the Report "Public"

The generated report is accessible only by the Enterprise Manager user who creates it. To make the report public:

Select the dashboard report from the list of reports shown after following the steps mentioned above.
Click Edit.
Select the Run report using target privileges of the report owner option under Privileges section in General tab.
Click the Access tab.
Select the Allow viewing without logging in to Enterprise Manager option.
Click OK.

4.10.3 Find All Database Machine Reports

To find all Database Machine reports:

Log in to Enterprise Manager.
From the Enterprise Manager home page, click the Enterprise menu. Select Reports, then Information Publisher Reports.
Search for the report name. Dashboard report names, one for each Database Machine, are displayed in the following format:
```
[DBMachine Name]_DASHBOARD_REPORT
```
Click on the report to view the dashboard report.

4 Post-Discovery Configuration and Verification

4.1 Set Up SNMP Trap Forwarding on Compute Node

4.1.1 Set Up Compute Node Agent - Linux

4.1.2 Set Up Compute Node Agent - Solaris

4.2 Configure Storage Cell SNMP for Enterprise Manager Monitoring

4.2.1 Using the ALTER CELL Command

4.2.2 Check Current SNMP Configuration

4.2.3 Configure SNMP Values Using cellcli

4.2.4 Configure SNMP Values Using dcli (optional)

4.2.5 Verify SSH Connectivity

4.2.6 Remove a Subscription

4.3 Configure and Verify SNMP for InfiniBand Switch Targets

4.3.1 Set Up SNMP for InfiniBand Switch Targets Using Enterprise Manager

4.4 Configure the Compute Node ILOM SNMP for Enterprise Manager Monitoring

4.4.1 Configure the Compute Node ILOM SNMP for Compute Nodes Running Management Server

4.4.2 Configure SNMP Values Using dbmcli

4.4.3 Configure SNMP for Oracle SuperCluster

4.5 Verify Configuration for Oracle ILOM Server

4.6 Set Up SNMP for Cisco Ethernet Switch Targets

4.6.1 Verify the Cisco Ethernet Switch SNMP Configuration

4.7 Set Up SNMP for Power Distribution Unit (PDU) Targets

4.7.1 Verify the PDU SNMP Configuration

4.8 Set Up SNMP for KVM Targets

4.8.1 Set Up SNMP in KVM Management Console

4.8.2 Verify the KVM SNMP Configuration (Base SNMP Configuration) - Linux

4.8.3 Verify the KVM SNMP Configuration (Base SNMP Configuration) - Solaris

4.8.4 Verify the KVM SNMP Configuration (SNMP Forwarding to Agent)

4.9 Accessing Oracle Support Workbench for Exadata Storage Server

4.10 Oracle Exadata Database Machine Dashboard Creation

4.10.1 Dashboard Creation for Exadata Database Machine Plug-in Release 12.1.0.3.0 and Later

4.10.2 How to Make the Report "Public"

4.10.3 Find All Database Machine Reports

4.2.1 Using the `ALTER CELL` Command

4.2.3 Configure SNMP Values Using `cellcli`

4.2.4 Configure SNMP Values Using `dcli` (optional)