This chapter provides information to assist in troubleshooting integration issues with Microsoft SCOM. The chapter focuses on troubleshooting issues in the web service front-end and the back-end Agent.
Note:
Unless otherwise noted, these instructions apply to the SCOM 2012 connector and to the SCOM 2007 connectors. Instructions specific to the SCOM 2007 connectors are available in Microsoft SCOM 2007 Connector.
This chapter discusses the following topics:
Before you start the troubleshooting steps, you must insure that you have done the following:
Install the SCOM connector as specified in Installing the Microsoft SCOM Event Connector.
Install and start the Oracle SCOM Agent as specified in Installing and Running the Oracle SCOM Agent.
Install, start, and test the SCOM Web service as specified in Installing the Microsoft SCOM Web Service.
Create a connector instance as specified in Creating a Connector Instance.
Configure the connector instance as specified in Configuring the Connector.
Set up one or more rules to forward events to the connector instance.
If all the actions above have been completed and the connector is not working, perform the steps in Diagnosing the Problem.
createEvent
and updateEvent
operations.
To identify the cause of a startup failure, navigate to the adapters/log
directory in the SCOM Web Service install directory and open the framework.log
file in a text editor. Search for Exception to find any errors in the file. If the file does not exist, it indicates that there is a problem locating or executing the JVM. See JVM Errors for information about resolving JVM issues.
Listed below are some possible exceptions, an explanation of the root cause, and a description of the solution.
Example 7-1 java.net.BindException: Address already in use: bind
This error indicates that the web service could not start because of a port conflict. There are two possible causes for this error:
Another application is using a port that the web service is configured to use. If the web service is configured to use SSL, the port number is 8443. If it is not configured to use SSL, the port number is 8080.
There are two possible solutions to this. You can change the other application to use a different port or you can change the SCOM Web Service to use a different port. To change the SCOM Web Service to use a different port, see Changing Default Port Numbers in Customizing Microsoft SCOM.
There is an instance of the web service already running. If this is the case then there is no change required. You should only run one instance of the web service at a time.
Example 7-2 org.springframework.beans.factory.BeanInitializationException: Could not load properties; nested exception is java.io.FileNotFoundException: … framework.properties (Permission denied)
This error indicates that the web service could not start because the permissions on the framework.properties
file in the conf
directory were not set correctly.
To solve the problem, change the permissions to give the account or group under which the SCOM Web Service runs read and execute permissions.
For any other startup errors, consult Oracle Support.
JAVA_HOME
environment variable must be set to the directory where JDK 1.6 is installed in the shell where the web service is started. To properly start the web service on a UNIX platform, perform the following:
JAVA_HOME
environment variable to the JDK 1.6 install directory.adapters/bin
subdirectory in the web service install directory../service.sh start
command.adapters\bin
subdirectory in the web service install directory.iWaveAdaptersw.exe
executable.jvm.dll
file in the JDK 1.6 install directory.OracleEnterpriseManager.Alert.Creator
Management Pack has been imported into the SCOM server:
This section provides cause and solution information on troubleshooting common error messages. Find the error message in Table 7-1 that matches your error message, then refer to the corresponding section(s) indicated under Possible Cause for instructions to diagnose and correct the problem.
Table 7-1 Enterprise Manager Error Messages
Error Message | Possible Cause |
---|---|
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Bad response: 403 Forbidden from url … |
|
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Message send failed: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target |
|
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Message send failed: Connection refused |
|
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Message send failed: No route to host |
|
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Bad response: 404 Not Found from url … |
or |
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Message send failed: Connection timed out |
|
javax.xml.soap.SOAPException: javax.xml.soap.SOAPException: Message send failed: hostname |
|
javax.xml.transform.TransformerConfigurationException: Could not compile stylesheet |
|
Unable to reconnect to server after being disconnected |
|
ERROR - Could not connect to the server <hostname> because it is not operational |
|
ERROR - Could not login to the server because the account was invalid or has insufficient permissions |
or |
ERROR occurred invoking SCOM connector to insert event for null |
or |
javax.xml.ws.WebServiceException: org.apache.cxf.service.factory.ServiceConstructionException: Failed to create service |
|
Request failed because the specified management pack could not be found |
|
Successfully inserted the event but timed out waiting for the alert to be created |
The following errors are described:
Cause
The user name or password for accessing the SCOM web service is incorrect.
Solution
Verify that the port number configured for the connector is correct:
Log in to the Oracle Enterprise Manager console with an account that has Super Administrator privileges.
From the Setup menu, select Extensibility, then select Management Connectors.
On the Management Connectors page, click the name of the appropriate SCOM connector.
This invokes edit mode, enabling you to configure the connector.
Correct the SCOM Web Service Username and SCOM Web Service Password fields, then click OK.
Cause
The SSL handshake between the Oracle Enterprise Manager Connector Framework and the SCOM web service failed. This failure occurs because Oracle Enterprise Manager is not configured correctly with the SSL certificate for the SCOM web service. The SSL certificate the SCOM web service uses must be imported into the Enterprise Manager key store. The certificate is either missing from the key store or does not match the SSL certificate provided by the SCOM web service.
Solution
Import the SSL certificate from the SCOM web service into the Enterprise Manager key store. See Configuring Enterprise Manager to Use SSL for details on setting up Oracle Enterprise Manager with the SCOM SSL certificate.
Cause
The SCOM web service is down.
Solution
Perform the following steps to check the status of the web service and start it if necessary:
If the SCOM web service is installed on a Unix system:
Open a command terminal on the system where the SCOM web service is installed.
Change the working directory to the adapters/bin
directory in the SCOM web service installation directory.
Enter the following command:
./service.sh status
If the command indicates that the service is not running, enter the following command:
./service.sh start
If the SCOM web service is installed on a Windows system:
Open a command terminal on the system where the SCOM web service is installed.
Change the working directory to the adapters\log
directory in the SCOM web service installation directory.
Open the framework.log
file in a text editor.
Go to the bottom of the file and search backwards for the string iWave Adapter Framework. If the last occurrence found is iWave Adapter Framework Started, this indicates that the web service is started.
If the web service is not started, start the web service as specified in Running the Microsoft SCOM Web Service on Windows.
Cause
The IP address specified in the URL is invalid or the network is down.
Solution
Verify that the hostname/IP address configured for the connector is correct:
Log in to the Oracle Enterprise Manager console with an account that has Super Administrator privileges.
From the Setup menu, select Extensibility, then select Management Connectors.
On the Management Connectors page, click the name of the appropriate SCOM connector.
This invokes edit mode, enabling you to configure the connector.
Verify that the hostname/IP address specified in the URL for the createEvent
and updateEvent
operations are correct.
If the hostname/IP address is incorrect, provide the correct value, then click OK.
If the URLs specify a host name, make sure that the host name resolves to the correct IP address. To determine the IP address of the host name, issue the ping <hostname>
command, where <hostname>
is the actual host name. This lists the IP address that was resolved for the host name. If this is incorrect, the system administrator needs to investigate why it is incorrect.
If the hostname/IP address appears to be correct, try to ping the system where the SCOM web service is installed using the hostname/IP address. If the ping fails, the system administrator needs to investigate why there is no connectivity.
Cause
The web service received the request and rejected it because an invalid path was specified in the URL.
Solution
Perform the following steps to test the URL the connector is using:
Log in to the Oracle Enterprise Manager console with an account that has Super Administrator privileges.
From the Setup menu, select Extensibility, then select Management Connectors.
On the Management Connectors page, click the name of the appropriate SCOM connector.
This invokes edit mode, enabling you to configure the connector.
Select and copy the URL specified for the createEvent
operation.
Open an internet browser on the system where the Oracle Enterprise Manager server is installed.
In the address window, enter the URL that was copied in step 6 above. Add ?wsdl to the end of the URL. The URL should appear similar to the following example:
http://[Hostname]:8080/services/SCOM/EventService?wsdl
[Hostname]
is the actual host name or IP address where the SCOM web service is installed.
If the WSDL is loaded, this confirms that the URL is correct. If it fails to load, there is a problem with the URL. Perform the steps specified in Using the Correct URL for SCOM Web Service Operations to configure the connector to use the correct URL.
Cause
The port number specified in the URL is invalid.
Solution
Verify that the port number configured for the connector is correct:
Log in to the Oracle Enterprise Manager console with an account that has Super Administrator privileges.
From the Setup menu, select Extensibility, then select Management Connectors.
On the Management Connectors page, click the name of the appropriate SCOM connector.
This invokes edit mode, enabling you to configure the connector.
Verify that the port number specified in the URL for the createEvent
, updateEvent
, setup, initialize, and uninitialize operations are correct.
If the port number is incorrect, provide the correct value and click OK.
Cause
A firewall is blocking access to the system where the SCOM Web Service is installed.
Solution
Contact your IT department to give Enterprise Manager access to the port used by the SCOM Web Service. Perform the steps specified in Using the Correct URL for SCOM Web Service Operations to determine the URL used by the SCOM Web Service. The port number specified in the URL is the port number the IT department should open in the firewall..
Cause
The system does not recognize the host name specified in the URL.
Solution
You can use the following options to address this issue:
Coordinate with the system administrator to change the system configuration to recognize the host name.
Specify the IP address in the URL instead of the host name. To do this, perform the following steps:
Determine the IP address of the system where the SCOM web service is installed.
Log in to the Oracle Enterprise Manager console by entering a user name with a Super Administrator role, entering the appropriate password, then click Login.
From the Enterprise Manager console, click Setup, then Extensibility, and finally Management Connectors. The Management Connectors page appears, which shows the installed connectors.
Click on the Configure icon associated with the Microsoft SCOM connector. This invokes edit mode, enabling you to configure the connector.
Change the host name to the IP address in the URL specified for the createEvent
, initialize, setup, uninitialize, and updateEvent
operations.
Click OK.
Cause
The connector framework could not process the request because the XSL file was formatted incorrectly. This problem should not occur unless the connector has been customized.
Solution
Examine any changes made to the XSL template files for mistakes that could have caused the problem. If you can't find the problem manually, load the XSL in a utility that performs XML validation.
Cause
The SCOM Agent could not insert the alert into SCOM because the wrong host name is configured for SCOM or the SCOM server is down.
Solution
Perform the following steps to determine and correct the root cause of the problem:
Verify that the host name or IP address listed in the error message is correct for the SCOM server. If the host name or IP address are incorrect, perform the following steps to correct the configuration:
Open Windows Explorer on the system where the SCOM Agent is located.
Navigate to the bin
directory in the SCOM Agent installation directory.
Run the SCOMAgentConfig.exe
utility to start the SCOM Agent Configuration Tool.
Click Load to open a directory navigation window.
Navigate to the SCOM Agent installation directory and open the SCOMAgent.cfg
file.
Click the Management Groups tab, then click Edit to display the Edit Management Group window.
Correct the hostname/IP address in the Server field, then click Update.
Click Save to save the changes to the configuration file.
Click Exit to exit the utility.
Stop and restart the SCOM Agent in IIS.
Verify that the following OpsMgr services are running:
System Center Data Access
System Center Management
System Center Management Configuration
Cause
The SCOM Agent could not send the alert to the SCOM server, because the credentials configured for accessing the SCOM API are invalid.
Solution
Perform the following steps to change the credentials for accessing the SCOM API:
Open Windows Explorer on the system where the SCOM Agent is located.
Navigate to the bin directory in the SCOM Agent installation directory.
Run the SCOMAgentConfig.exe utility to start the SCOM Agent Configuration Tool.
Click Load to open a directory navigation window.
Navigate to the SCOM Agent installation directory and open the SCOMAgent.cfg file.
Click the Management Groups tab, then click Edit to display the Edit Management Group window.
Correct the credential information in the Domain, Username, and Password fields, then click Update.
Click Save to save the changes to the configuration file.
Click Exit to exit the utility.
Stop and restart the SCOM Agent in IIS.
Cause
The SCOM Agent could not send the alert to the SCOM server, because the credentials configured for accessing the SCOM API do not have sufficient permissions.
Solution
Refer to Setting Up the Agent Account. This section provides the steps required to set up the account for accessing the SCOM API.
Cause
The web service could not create an alert in SCOM because the SCOM Agent is not operational.
Solution
Open IIS Manager on the system where the SCOM Agent was installed, and start the web site for the Agent.
Cause
The web service could not connect to the SCOM Agent because the web service has an invalid configuration parameter. Either the URL for the SCOM Agent is incorrect or the credentials for accessing the SCOM Agent are invalid.
Solution
Verify that the URL for the SCOM Agent is correct. You should specify the URL that was provided at the end of the SCOM Agent installation. Note that if the host name in the URL is localhost and you are accessing it from another system, you need to replace localhost with the host name or IP address of the SCOM Agent installation machine.
If you do not know the URL, you can determine it as follows:
If the SCOM Agent was installed as a web site, the address is:
http://<IP>:<port>/Service.asmx
... where <IP>
is the IP address, and <port>
is the port number specified when installing the Agent.
If the SCOM Agent was installed as a virtual directory, the address is:
http://<IP>:<port>/<vdir>/Service.asmx
... where <IP>
is the IP address, <port>
is the port number for the web service where the agent was installed, and <vdir>
is the virtual directory name specified for the Agent.
Select a user name and password that are valid on the system where the SCOM Agent was installed.
Open a command window and change the working directory to adapters\endpoints\SCOM
in the SCOM web service installation directory.
Rerun the SCOM Web Service installer using the URL and credentials from the preceding steps. See Installing the Microsoft SCOM Web Service on UNIX or Installing the Microsoft SCOM Web Service on Windows, depending on your platform, for the procedure.
Restart the web service as instructed in Installing the Microsoft SCOM Web Service on UNIX or Installing the Microsoft SCOM Web Service on Windows.
Cause
The web service could not create an alert in SCOM because the OracleEnterpriseManager.Alert.Creator
management pack has not been imported into SCOM.
Solution
Refer to Install the Alert Creator Management Pack for Microsoft SCOM 2007 for the steps required to import the management pack into SCOM.
Cause
The web service was able to insert an event in SCOM, but an alert was not created within the timeout period. This likely indicates that an error occurred in the alert generating rule and it was unloaded by SCOM. Whenever this occurs, the System Center Operations Manager Health Service generates an error followed by a warning in the Operations Manager log. The error entry begins with the following message:
A module reported an error 0x80070057 from a callback which was running as part of rule "Create.Default.Alert" running for instance "OracleEnterpriseManager Event Source" with id ...
The warning entry begins with the following message:
Summary: 1 rule(s)/monitor(s) failed and got unloaded, 1 of them reached the failure limit that prevents automatic reload ...
Note:
This situation should not occur if the default SCOM connector configuration files are used. The only known way this can occur is if the SCOM Agent web service is directly accessed and an invalid value is passed for the Priority or Severity fields.
Solution
Restart the Windows service named "Ops Mgr Health Service" on the SCOM server.