Skip Headers
Oracle® Auto Service Request Installation and Operations Guide
Release 4.8.1 for Linux and Solaris

E18475-27
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

5 ASR General Troubleshooting

This chapter provides a variety of troubleshooting procedures for the ASR software.

The instructions provided are for Solaris. When possible, corresponding Linux instructions are provided. Please see the appropriate Linux documentation for details for general administration commands.

Note:

To enter the ASR prompt (asr>) as root, type asr on the command line. See "Install ASR" for instructions for setting the PATH environment variable.

The following troubleshooting topics are presented:

5.1 ASR Status

You can review the status of any ASR Asset from the ASR Manager or from My Oracle Support. The following ASR Status troubleshooting topics are presented:

5.1.1 View Status from the ASR Manager

The status of any ASR Asset can be obtained by running any one of the following command options from the ASR Manager system:

ASR Command Description
list_asset
Lists all assets associated with this ASR Manager.
list_asset -i <IP address>
Shows the asset associated with the IP address.
list_asset -h <hostname>
Shows the asset associated with the host name.
list_asset -hb
Displays the last heartbeat date.
list_asset -s <subnet IP address>
Lists all assets associated with subnet IP address.

Note:

The list_asset command accepts a comma-delimited list of IP addresses, subnets, or hostnames.

The results will be similar to the following example:

Surrounding text describes asr_status_example.gif.

The output of the list_asset -hb command would be similar to the following example:

SERIAL_NUMBER   HOST_NAME       LAST_HEARTBEAT_DATE       PRODUCT_NAME
-------------   -------------   -----------------------   --------------
ASRBT7777       host-123        NA                        SUNW,Sun-Fire-V210 SPARC System
ASRTEST1        asr-123         NA                        SUN FIRE X2270 x86/x64 System
ASRTST321777    abcd.name.com   2014-03-07 07:39:55.105   SPARC T5-8

The data in LAST_HEARTBEAT_DATE column can show either NA or a date/time when the ASR Manager received the last heartbeat from the asset.

A value of NA indicates that the ASR Asset never sent a heartbeat to the ASR Manager.

Note:

ASR Assets running Solaris 11 and ILOM version 3.2.2.0 will send individual heartbeats.

5.1.2 View Status from My Oracle Support

To view the status of all ASR Assets, log in to My Oracle Support (https://support.oracle.com). In the My Oracle Support Dashboard, click the ”Systems...” tab. Then select ”Settings” from the menu.

In the "Settings" pane on the left of the window, select "Assets" (located under the Administrative submenu). A complete list of all ASR Assets is displayed:

Surrounding text describes assets.gif.

5.1.3 ASR Log Files

When you are troubleshooting ASR, you can change the level of information displayed in the logs, and increase or decrease the number of logs that are saved before being overwritten. The logs are written to the sw-asr.log files. Log files are located on the ASR Manager system at /var/opt/SUNWsasm/log

Log File Description
asr-sw-autoupdate.log Status updates for the ASR Auto Update feature.
asr-http-rcvr-accepted.log.0 Messages accepted by the ASR Manager HTTP receiver
asr-http-rcvr-rejected.log.0 Messages rejected by the ASR Manager HTTP receiver
asr-http-rcvr.log.0 Messages processed by the ASR Manager HTTP receiver
sasm.log Error messages and activity regarding the Oracle Automated Service Manager (OASM)
sw-asr-accepted.log.0 Fault events accepted by the ASR Manager
sw-asr-objectpool.log.0 Troubleshooting information used by the Oracle ASR support team
sw-asr-rejected.log.0 Fault events rejected by the ASR Manager
sw-asr-servicerequest.log.0 Oracle service request numbers created by ASR
sw-asr.log.0 Error messages and activity regarding the ASR Manager
sw-asr-snmp.log.0 Activity regarding SNMP traps processing

There are four levels of logs:

  1. Fine: Displays the highest level of information. It contains fine, informational, warnings and severe messages.

  2. Info: Displays not only informational data, but also both warnings and severe messages. This is the default setting.

  3. Warning: Displays warnings and severe messages.

  4. Severe: Displays the least amount of information; severe messages only.

The default number of logs collected and saved is 5. Once that number is reached, ASR begins overwriting the oldest file. You have the option to change the number of logs collected and saved. If you are gathering as much information as possible in a short time, you might want to limit the number of logs saved to accommodate the larger files.

5.1.3.1 Set Log Level

Follow the procedure below to set logging levels:

  1. Open a terminal window and log in as root on the ASR Manager system.

  2. To view the current level of information being gathered, run:

    asr> get_loglevel
    
  3. To change the logging level, run:

    asr> set_loglevel [level]
    

    The choices for level are: Fine, Info, Warning, or Severe.

5.1.3.2 Set Log File Counts

Follow the procedure below to set log file counts:

  1. Open a terminal window and log in as root on the ASR Manager system.

  2. To view the current number of logs being saved, enter the following command:

    asr> get_logfilecount
    
  3. To change the number of logs being saved, enter the following command:

    asr> set_logfilecount [number]
    

5.1.4 Check the State of ASR Bundles

For diagnostic purposes, it may be necessary to check the state of various application bundles installed on the ASR Manager system using the following procedure.

  1. Open a terminal window and log in as root to the ASR Manager.

  2. Enter the following command:

    asr> diag
    
  3. Review the results of this command below along with the settings you should see:

    id State Bundle
    744 ACTIVE com.sun.svc.asr.sw_3.8.0
    Fragments=745, 746
    745 RESOLVED com.sun.svc.asr.sw-frag_3.8.0
    Master=744
    746 RESOLVED com.sun.svc.asr.sw-rulesdefinitions_3.8.0
    Master=744
    748 ACTIVE com.sun.svc.asr.sw.http.AsrHttpReceiver_1.0.0
    Fragments=749
    749 RESOLVED com.sun.svc.asr.sw.http-frag_1.0.0
    Master=748
    743 ACTIVE com.sun.svc.ServiceActivation_3.8.0
    
  4. The state of each bundle should be as follows:

    • com.sun.svc.asr.sw bundle should be ACTIVE

    • com.sun.svc.asr.sw-frag should be RESOLVED

    • com.sun.svc.asr.sw-rules definitions should be RESOLVED

    • com.sun.svc.ServiceActivation should be ACTIVE

    • com.sun.svc.asr.sw.http.AsrHttpReceiver bundle should be ACTIVE

    • com.sun.svc.asr.sw.http-frag should be RESOLVED

  5. If any of these states are incorrect, enter the following commands:

    asr> stop
    asr> start
    
  6. Repeat steps 1 to 3.

  7. To ensure everything is working properly, run the following commands:

    asr> test_connection
    asr> send_test
    

5.1.5 Check ASR Manager Status

For diagnostic purposes, it may be necessary to check the status of processes running on the ASR Manager system. For any failures, refer to Error Messages and Resolutions.

To verify the ASR Manager status, run the following script:

/opt/SUNWswasr/util/check_asr_status.sh

Output of a successful status check should look like this:

PASS: OASM ASR Manager bundles status is active.
PASS: OASM ASR Manager SNMP listener is running (SNMP port 162).
PASS: OASM ASR Manager HTTP receiver is running (HTTP port 8777).
PASS: OASM sw-asr database connectivity is working.
PASS: OASM Registration SSO user name is set correctly.
PASS: OASM Oracle Transport connectivity is working.
PASS: OASM Oracle Transport endpoint is set correctly.
PASS: OASM OSGI port is accessible.
PASS: OASM process is running.
PASS: OASM is running with Standard Oracle Java.

5.2 ASR Diagnostics

To assist with diagnosing issues with ASR Manager installation, configuration, and operation, ASR provides the ability to generate a diagnostic file that can be analyzed by Oracle Support as part of a Service Request, as needed.

To generate and send an ASR diagnostic file for analysis with Oracle Support:

  1. Create a Service Request in My Oracle Support.

    Note:

    If a valid SR number is not provided, then the upload to Oracle will fail.
  2. Run the following command from the ASR Manager:

    asr> send_diag -sr <SR number>
    

    Where the -sr <SR number> is the newly created Service Request number.

    For example:

    asr> send_diag -sr 3-12345678
    
    This command will collect the diagostics file from ASR Manager and upload to Oracle ASR Infrastructure. Do you want to proceed with collect the diagostics bundle? [y/n]: y
    
  3. Verify the diagnostic file has been successfully attached to the Service Request. Log in to My Oracle Support and view the Service Request you created earlier. The request should be updated with a new attachment.

  4. (Optional) Check the status of the ASR diagnostic file:

    asr show_diag_upload_status_status
    

    This command displays the ASR diagnostics file's collection status for all collection attempts, either from the ASR command line or from the ASR portal. The collection status is displayed in ascending order.

    Output will look like this:

    asr show_diag_upload_status_status
    
    Diagnostics File Upload Status
    ==========================
    File Name: /opt/SUNWswasr/util/diag/asr-diag-bundle-<client site ID>-131218104617.zip
    File Upload Time Stamp: 2013-12-18 10:48:17.605
    Asset Serial: Not Activated
    File Uploaded from Client: ASR Manager
    Client Site ID: <client site ID number>
    File Upload Status Message: User my-asr-user@mycompany.com is not entitled to upload the log files to Oracle ASR Infrastructure.
    Failure reason: PUT https://mycompany.com/upload/issue/3-12345678/asr-diag-bundle-<client site ID>-131218104617.zip returned a response status of 403 Forbidden
    File Upload Type: Log Collection via Manual Request
    File Upload Requested By: Manual Request from ASR Commandline
    File Type: ASR Manager Diagnostics Bundle
    

You can also create a ASR diagnostic file at any time. From the ASR Manager, run the following command and follow the command-line instructions:

asr> asrDiagUtil.sh

Note:

You can specify where the file is to be located. See Configure the ASR Diagnostic Utility for more information. By default, this file is stored in the following directory:
/opt/SUNWswasr/util/diag

5.2.1 Configure the ASR Diagnostic Utility

The diag-config.properties file consists a list of properties for specifying location of the configuration and log directories. It also contains "toggle switches" for enabling and disabling a particular data set to be collected:

  • com.sun.svc.asr.util.diag.home.directory – The property for specifying where the diagnostic data .zip bundle will be generated. Default is current directory where the ASR Diagnostic Utility is located.

  • com.sun.svc.asr.util.diag.zip.file.prefix – The property for configuring the diagnostic data .zip file's name.

  • com.sun.svc.asr.util.diag.zip.recursive property – The property for enabling traversing into subdirectories of any configuration or log directories.

5.2.2 ASR Diagnostic Error Messages

Error Message Resolution
ASR Manager does not have the Minimum Java version required for the 
Diagnostics file upload to Oracle ASR Infrastructure. 
Existing Java Version: 1.6.0_26, Minimum required version: 1.6.0_43
Upgrade the Java version to 1.6.0_43 or above. Then point OASM to use this latest Java version. Open the /var/opt/SUNWsasm/configuration/config.ini file and edit the java.exec= property to point valid Java path.

For example:

java.exec=/usr/java/bin/java

Save and close the file, then restart OASM to have the updates take effect:

  • For Solaris: svcadm restart sasm

  • For Linux: service sasm restart

Please enter a valid service request number.
The Service Request (SR) number format should be valid. A valid format is <single digit><-><multiple digits> (for example: 3-1234566).

Check the SR number you created and run the send_diag command again with the valid SR number.

Log collection was requested with an invalid SR Number. 
Cannot upload the logs to Oracle ASR Infrastructure.
The contact registered for the ASR Manager is not authorized to upload diagnostics files to My Oracle Support for this SR.

Log in to My Oracle Support to verify the upload permissions.

ClassCastException while uploading file to Oracle ASR Infrastructure.
A restart of OASM is required.
Restart OASM to resolve the issue.

For Solaris: svcadm restart sasm

For Linux: service sasm restart


5.3 ASR Manager Crash Recovery

In cases where an ASR Manager experiences a critical failure, you can set up a new ASR Manager and reconfigure ASR Assets to report to the new host. The following steps describe a sample scenario:

  1. An ASR Manager is set up (e.g., hostname: ASRHOST01, IP address: 10.10.10.1) and configured on the network. This ASR host is registered and activated to itself.

  2. All ASR assets are configured to report failures to the ASR Manager host (ASRHOST01), and all ASR assets are activated on the host.

  3. A critical failure occurs in the cabinet of ASRHOST01 (for example: a fire destroys the system and its data). The assets need to be attached to a different ASR Manager host (e.g., hostname: ASRHOST02).

  4. A new ASR Manager is set up (e.g., hostname: ASRHOST02, IP address: 10.10.10.2) and configured on the network. The new ASR host is registered and activated to itself.

  5. All ASR assets are now re-configured to report failures to the new ASR Manager host ASRHOST02, and the trap destination is changed to report failures to ASRHOST02.

  6. All ASR assets are now activated on ASRHOST02

Note:

In order to reduce the additional work with moving the ASR Manager to a different location (e.g., from ASRHOST1 to ASRHOST2), you can create an ASR backup on another host or on the existing host. Creating a backup is crucial when recovering from a crash (see "ASR Backup and Restore" for a details on creating an ASR backup).

5.4 ASR - No Heartbeat

Heartbeat is configured to run once every day via an internal timer thread. If there is no response after approximately 48 hours, the unit will be marked as a 'Heartbeat Failure' unit.

You can check to see if any ASR Manager or ASR Asset are in Heartbeat Failure by reviewing the ASR status in My Oracle Support.

If you feel that ASR Manager is configured correctly, then you can troubleshoot your ASR Manager hardware to resolve the problem. See MOS knowledge article 1346328.1 for the instructions to your particular hardware:

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=HOWTO&id=1346328.1

See Appendix A, "Heartbeat Failure Notification E-mail Examples" for an e-mail example you may receive should this problem occur.

5.5 ASR Assets for Solaris 11 Troubleshooting

In cases where you are having issues with configuring ASR on Solaris 11 assets using the asradm command, then review the status of the following asr-notify SMF service:

svcs asr-notify

Output should look like this:

STATE        STIME      FMRI
online       13:00:31   svc:/system/fm/asr-notify:default

Note:

If the asr-notify service status is in maintenance mode, then clear the maintenance mode:
svcadm clear asr-notify

re-register the Solaris 11 asset with ASR manager

5.6 Resolve ASR Manager Java Path Location in config.ini File

If you have an incorrect or old version of Java installed, the ASR Manager will not start. The command to start OASM will report the following message (see Start ASR and OASM for Solaris and Linux command samples):

*************************************************************************
Warning! An old Java version ( 1.5 ) was detected (tried 
'/usr/jdk/jdk1.5.0_16/bin/java').
Oracle Automated Service Manager requires a Java version of 1.6 or higher
to run correctly.

You can set 'java.exec' property in file
/var/opt/SUNWsasm/configuration/config.ini
to point to JAVA 1.7 or later

Java can be downloaded from http://www.java.com
*************************************************************************
 
  1. Check the Java version you have installed. From the ASR Manager, run:

    java -version
    

    See Java Requirements for details of the Java version requirements for ASR. ASR requires Java 7 (1.7.0_13) or later.

  2. Get the current Java path location. From the ASR Manager, run:

    cat /var/opt/SUNWsasm/configuration/config.ini | grep ^java.exec
    

    The output would look like this:

    java.exec=/usr/bin/java
    
  3. Make a backup of the config.ini file. From the ASR Manager, run:

    cp /var/opt/SUNWsasm/configuration/config.ini /var/opt/SUNWsasm/configuration/config.ini_<current-timestamp>
    
  4. Edit the java.exec property in the config.ini file to point to the value of the java.exec output from Step 2, which should be for Java 7:

    /usr/jdk/latest/bin/java
    
  5. Stop and start OASM. From the ASR Manager, run:

    • For Oracle Solaris:

      svcadm restart sasm
      
    • For Linux:

      service sasm restart
      

5.7 Service Tools Bundle (STB) Troubleshooting (Solaris 10 Only)

This section provides a variety of steps to check on the state of the Service Tools Bundle (STB) that must installed on most ASR systems. If issues arise during the installation and operation of ASR, STB may be part of the issue.

The following STB troubleshooting areas are presented:

5.7.1 Check the Service Tags

  1. Open a browser window to the system you wish to check using the following command. Be sure to include the / (slash) after agent.

    http://asr_system_hostname:6481/stv1/agent/

  2. A response similar to the following will be displayed:

    <st1:response>
    <agent>
    <agent_urn><agent urn number></agent_urn>
    <agent_version>1.1.4</agent_version>
    <registry_version>1.1.4</registry_version>
    <system_info>
    <system>SunOS</system>
    <host><your host name></host>
    <release>5.10</release>
    <architecture>sparc</architecture>
    <platform>SUNW,Sun-Fire-V215::Generic_137111-06</platform>
    <manufacturer>Sun Microsystems, Inc.</manufacturer>
    <cpu_manufacturer>Sun Microsystems, Inc.</cpu_manufacturer>
    <serial_number>0707FL2015</serial_number>
    <hostid><host ID number></hostid>
    </system_info>
    </agent>
    </st1:response>
    
  3. If you do not get a response from the Service Tags agent, consult the Service Tags man pages:

    man in.stlisten
    man stclient
    

5.7.2 Check the Service Tags Version

Follow the procedure below to check the Service Tags version:

  1. Open a terminal window and log in as root to the ASR system you wish to check.

  2. Run the following command to get the Service Tags version:

    stclient -v

ASR requires Service Tags version 1.1.4 or later.

5.7.3 Check Service Tags Probe

Follow the procedure below to determine that the Service Tag discovery probe is running:

  1. Open a terminal window and log in as root to the ASR system you wish to check.

  2. To determine that the Service Tag discovery probe is running, run the following command:

    svcs -l svc:/network/stdiscover

  3. If the probe is running correctly, the following information is displayed:

    fmri svc:/network/stdiscover:default
    name Service Tag discovery probe
    enabled true
    state online
    next_state none
    state_time Wed Sep 03 21:07:28 2008
    restarter svc:/network/inetd:default
    

5.7.4 Check Service Tags Listener

Follow the procedure below to determine that the Service Tags Listener is running:

  1. Open a terminal window and log in as root to the ASR system you wish to check.

  2. To determine if the Service Tags listener is running, run the following command:

    svcs -l svc:/network/stlisten

  3. If the listener is running correctly, the following information is displayed:

    fmri svc:/network/stlisten:default
    name Service Tag Discovery Listener
    enabled true
    state online
    next_state none
    state_time Wed Sep 03 21:07:28 2008
    restarter svc:/network/inetd:default
    xibreXR_US root@s4u-v215c-abc12
    

5.7.5 Unable to Contact Service Tags on Asset

This message indicates that the activation failed during Service Tags discovery. The issue can be either Service Tags is not installed on the ASR Asset or is installed but not running. Also the issue can be network connectivity between ASR Manager and the ASR Asset. Complete the following checks:

  1. Check if Service Tags is installed and running on an ASR Asset. Run:

    stclient -x
    

    If you cannot run this command, either Service Tags is not installed or not online.

  2. Check if the Service Tags services are installed and online using the following command:

    svcs | grep reg
    
  3. The results should be similar to the following example:

    online Aug_23 svc:/application/stosreg:default
    online Aug_23 svc:/application/sthwreg:default
    
  4. If you cannot find these services, it means Service Tags is not installed on the ASR asset.

  5. If the Service Tags services are online, check if psncollector is online. Run:

    svcs | grep psncollector
    
  6. The results should be similar to the following example:

    online Sep_09 svc:/application/psncollector:default
    
  7. Make sure that there are no TCP Wrappers installed on the ASR asset to prevent any service tags discovery issues. Run the following command from the ASR Manager system:

    wget http://[assetHostNameOrIPaddress]:6481/stv1/agent/
    
  8. If there are TCP wrappers installed on the ASR asset, edit /etc/hosts.allow on the asset by adding:

    in.stlisten:[OASM host name]
    

5.7.6 Unknown or Empty Service Tags on Asset

  1. View the ASR Asset's serial number using the following URL:

    http://[AgentipAddress]:6481/stv1/agent/
    
  2. If product name is empty or "unknown," then check if the Hardware Service Tags are installed and online. Run:

    svcs | grep sthwreg
    

    The results should look like this:

    online Aug_23 svc:/application/sthwreg:default
    
  3. If the serial number is incorrect, contact Oracle Support to resolve the problem.

5.7.7 Cannot Retrieve the OASM IP Address

This error message indicates that the ASR Asset activation failed because the Oracle Automated Service Manager (OASM) IP address could not be retrieved. The final step for activating an ASR Asset includes this command:

asr activate_asset -i [host IP address]

When activation fails, the following error message displays:

Cannot retrieve the SASM IP address, please add the SASM IP address to /etc/hosts

You must edit the /etc/hosts file to update the localhost entry. For example, as root, change an entry that looks like this:

127.0.0.1    hostname123.com hostname123 localhost.localdomain localhost

to this:

127.0.0.1    localhost.localdomain localhost

5.7.8 Services are Disabled: stdiscover or stlisten

Service tag processes (stlisten and stdiscover) must be online in order to activate assets successfully.

  1. Check to determine if the stdiscover or stlisten services are disabled. Run the following command:

    svcs stlisten stdiscover

    If the services have been disabled, the output would look like this:

    STATE       STIME         FMRI
    disabled    12:20:14      svc:/network/stdiscover:default
    disabled    12:20:14      svc:/network/stlisten:default
    
  2. To enable the stdiscover and stlisten services, run the following command:

    svcadm enable stlisten stdiscover

  3. Verify the services are online:

    svcs stlisten stdiscover

    Once the services have been enabled, the output would look like this:

    STATE       STIME         FMRI
    enabled     12:20:14      svc:/network/stdiscover:default
    enabled     12:20:14      svc:/network/stlisten:default
    

5.8 SMA Service Troubleshooting (Solaris 10 Only)

The SMA service needs to be online in order to support Solaris FMA enrichment data properly. Prior to configuring FMA, complete the following steps:

  1. To check that the state of the SMA service is online, run:

    svcs sma

  2. If SMA is online, the state should indicate online, as in the following example:

    STATE      STIME          FMRI
    online     15:40:31       svc:/application/management/sma:default
    
  3. If SMA is not online, run the following command to enable it:

    svcadm enable sma

  4. Repeat these steps to confirm SMA is online.

5.9 Error Messages and Resolutions

Error Message Resolution
WARNING: Unable to retrieve fault details. For additional information and some insights into how to correct, please see the ASR Installation and Operations Guide - located at www.oracle.com/asr. See the ASR General Troubleshooting Section.
  1. Verify that the asset has the right Solaris minimum required version and patch level as per the ASR qualified systems web page (see http://www.oracle.com/asr for more information).
  2. Review the community string properties on the asset. ASR Manager requires public as the value of the community string in order to retrieve FMA enrichment and additional fault details. (See Enable M-Series XSCF Telemetry for more details)

  3. Review the FMA trap destination configuration file, and restart sma and fmd SMF services.

WARNING: This trap is rejected because the asset is disabled Enable the ASR Asset using one of the following commands:

asr> enable_asset -i <ip>

(where ip is the IP address of the ASR asset)

or

asr> enable_asset -h <host>

(where host is the hostname of the ASR asset)

WARNING: this trap is rejected because OASM ASR Plug-in is not activated Enable the ASR Manager using one of the following commands:

asr> activate_asset -i <ip>

(where ip is the IP address of the ASR asset)

or

asr> activate_asset -h <host>

(where host is the hostname of the ASR asset)

WARNING: this trap is rejected because the asset is not found Enable the ASR Asset using one of the following commands:

asr> activate_asset -i [ip]

(where ip is the IP address of the ASR asset)

or

asr> activate_asset -h [host]

(where host is the hostname of the ASR asset)

SEVERE: Cannot attach snmp trap to snmp service! This indicates that there could be another process using port 162. Kill that process and then run:

svcadm restart sasm

Checking connection to /v1/

_register failed!

Run the asr> register command again. This time, enter 1 or the full URL: https://transport.oracle.com

See Register the ASR Manager.

Failure to Register Errors The sasm.log has more detailed information and a Java stacktrace on what failed during registration. When a failure error is encountered, additional details can be found in:
/var/opt/SUNWsasm/configuration/sasm.log
No Such Host Exception This error indicates that the host running ASR Manager cannot resolve the IP address for the Data Transport Service server. Refer to Test Connectivity from the ASR Manager to Oracle to troubleshoot and resolve the problem.
Not Authorized. The My Oracle Support account provided could not be verified by the transport server This error indicates that the communication between transport server and Oracle is down or busy. This can also indicate that the queue set-up is wrong or that the user does not have permissions to the queue.
Socket Exception: Malformed reply from SOCKS server This error indicates one of the following:
  • The socks configuration in the config.ini file is incorrect or missing. Action: This usually indicates that you need to supply a user/password for the socks settings.

  • The socks is not able to route to the transport server endpoint. Action: Add the correct http proxy information or socks settings. Refer to Configure ASR to Send HTTPS Traffic Through a Proxy Server to correct the information.

error: Failed dependencies:
       jdk >= 1.7 is needed by SUNWsasm-1.5.0-112.noarch
Action: Install OASM 1.5 using the --nodeps option:
rpm -i --nodeps SUNWsasm-1.5.0-112.rpm
This is applicable to ASR Manager on Linux when upgrading OASM 1.4.2 to OASM 1.5

# service sasm start

Starting Oracle Automated Service Manager...
Response from command took longer than expected. Closing connection to console.
Only one client can access console at a time. Closing connection to console.
Started. 
Action:
  1. Uninstall OASM 1.5 with the following command:

    rpm -e SUNWsasm
    
  2. Run the following command:

    rm -rf /var/opt/SUNWsasm/configuration/org.eclipse*
    
  3. Finally, re-install OASM 1.5.

Activation failures:
This asset cannot be activated.
Service Tags on asset abc reports:
Product Name: unknown (Invalid Product Name)
Serial Number: TEST 123 (Invalid Serial Number)
Valid serial numbers contain letters, digits, period, colons, hyphens, underscores.

See Activation Failed for Asset <asset name> Due to Data Error (Solaris 10 Only) for details to correct this issue.

FAIL: Missing Registration SSO username in OASM config.
FAIL: OASM Oracle Transport end point is incorrectly set.
FAIL: OASM Oracle Transport connectivity is not working
Refer to Register the ASR Manager.

To verify the ASR Manager status, run the following script:

/opt/SUNWswasr/util/check_asr_status.sh
FAIL: OASM OSGI port is not accessible at this time.
FAIL: Multiple OASM processes are running.
FAIL: OASM process ID is not matching the process ID saved in /var/run/sasm.pid
  1. Check the OASM processes:
    ps -ef | grep "java -cp" | grep "SUNWsasm"
    

    Output should look like this:

    root 16817 1 0 16:09:49 ? 4:24 java -cp
    /var/opt/SUNWsasm/lib/com.sun.svc.container.ManagementTier.jar:/var/opt
    
  2. If OASM is running, kill the processes with the following command:

    kill -9 [Process_ID]
    
  3. Restart OASM:

    For Solaris: svcadm restart sasm

    For Linux: service sasm restart

  4. To verify the ASR Manager status, run the following script:

    /opt/SUNWswasr/util/check_asr_status.sh
    
FAIL: OASM is not running with Standard Oracle Java.
See Resolve ASR Manager Java Path Location in config.ini File.

To verify the ASR Manager status, run the following script:

/opt/SUNWswasr/util/check_asr_status.sh
FAIL: Missing HTTP receiver enable Jetty in OASM config.
See Enable HTTP Receiver for ASR Manager Relay, Solaris 11, and VOP

To verify the ASR Manager status, run the following script:

/opt/SUNWswasr/util/check_asr_status.sh

5.9.1 "Only One Client Can Access Console at a Time" or "Can't read input from console" Error Message

If you get either of these error messages running an ASR command on the ASR Manager system, it indicates that only one command can go into the OASM admin port at a time. Each command has a max handle on the connection for 60 seconds before OASM console kills the connection. Try executing the command after 60 seconds. If you still get same message, do the following:

  1. Check if OASM is running:

    ps -ef | grep SUNWsasm

  2. Results:

    root 16817 1 0 16:09:49 ? 4:24 java -cp
    /var/opt/SUNWsasm/lib/com.sun.svc.container.ManagementTier.jar:/var/opt
    
  3. If OASM is running, kill the process using the following command:

    kill -9 [Process_ID]

  4. Restart the OASM using the following command:

    For Solaris: svcadm restart sasm

    For Linux: service sasm restart

5.9.2 "SNMP GET failed" Error Message

The ASR Manager uses the SNMP GET protocol to query ASR assets for additional fault information (as shown in Figure 5-1).

Figure 5-1 ASR Manager SNMP GET

Surrounding text describes Figure 5-1 .

This is limited to the following products and fault telemetry sources:

  • M-Series servers (for example, M3000, M4000, M5000, M9000), XSCF service processor.

  • Solaris 10 on ASR-qualified Oracle servers that require FMA for ASR.

These products send fault events to the ASR Manager using the SNMP TRAP protocol.

The ASR Manager uses the SNMP GET to retrieve additional fault information (such as, FRU part number, serial number, and slot location) from the product. This important information allows Oracle to streamline the service delivery process. For example:

snmp_get -i <IP Address>
snmp_get -h <Hostname>

The ASR Manager test_snmp_get command is used to verify SNMP GET connectivity. For example:

test_snmp_get -i <IP Address>
test_snmp_get -h <Hostname>

Failure reasons include:

  • Incorrect asset configuration.

  • Network configuration on routers and firewalls that prohibit SNMP GET traffic.

An SNMP GET error message will be returned as:

SNMP GET failed on: asset Hostname/IP

5.9.2.1 Solaris 10 FMA SNMP GET Troubleshooting

To resolve this error for ASR Assets running Solaris 10 FMA:

  1. Log in to the ASR Asset.

  2. Verify the fmd status:

    # svcs fmd
    

    Output will look like this:

    STATE STIME FMRI
    online Jan_07 svc:/system/fmd:default
    
  3. Verify the sma status:

    # svcs sma
    

    Output will look like this:

    STATE STIME FMRI
    online Jan_07 svc:/application/management/sma:default
    
  4. Enable fmd and sma:

    # svcadm enable fmd
    # svcadm enable sma
    

5.9.2.2 M-Series Servers XSCF SNMP GET Troubleshooting

To resolve this error for M-Series servers:

  1. Log in to the M-Series XSCF.

  2. Verify the following information:

    • SNMP is operational with the agent running, accepting requests on port 161.

    • The Service Processor (SP) and Fault Management (FM) Management Information Base ("MIB") is enabled.

    • The community string is set to public in all lower case.

    To verify this information, run:

    XSCF> showsnmp
    

    The output will look like this:

    Agent Status:       Enabled   <<-- Must be "Enabled"
    Agent Port:         161       <<-- Must be "161"
    System Location:    Unknown
    System Contact:     Unknown
    System Description: Unknown
    
    Trap Hosts:
    
    Hostname      Port  Type  Community String   Username   Auth Protocol
    -----------   ----  ----  ----------------   --------   -------------
    10.11.12.13   162    v1   public             n/a        n/a
    
    SNMP V1/V2c:
    
    Status:            Enabled   <<-- Must be "Enabled"
    Community String:  public    <<-- Must be "public" in lower case
    
  3. To enable SNMP:

    XSCF> setsnmp enablev1v2c public
    

    Note:

    The SNMP community string is case sensitive. For example, PUBLIC is not the same as public.

    The default community string used by ASR Manager is public.

5.10 ASR Auto Update Troubleshooting

By default, Oracle ASR will download and install the latest version of the ASR software. The following sections provide potential solutions for problems that may arise:

5.10.1 ASR Auto Update Error Codes

ASR Auto Update includes a set of error codes to help diagnose and resolve issues you may encounter. See Oracle ASR: Auto Update Error Codes (Doc ID 1568407.1) in My Oracle Support:

https://support.oracle.com

5.10.2 SELINUX Environment Variable (Linux Only)

If the SELINUX environment variable is set to "Enforcing," then the ASR Auto Update will not be able to upgrade (remove and install) the ASR software .rpm file.

To resolve this issue, change this variable to "Permissive." Run the setenforce Permissive command to enable ASR Auto Update to remove and install the .rpm file automatically.

5.11 ASR Activation Failed Troubleshooting

As part of the activation process (see Activate ASR Assets for details), Oracle ASR automatically checks to verify that the qualified ASR Asset has been properly configured and that telemetry information can be sent. If an ASR Asset fails this activation process, you will receive e-mail notification, depending on the following causes:

For a complete list of activation-related e-mail samples, including activation failures, see ASR Activation E-mail Examples.

5.11.1 Activation Denied

If you receive an "activation denied" e-mail, (as shown in the Activation Denied e-mail sample), then check to ensure that the same asset is not already activated by a different ASR Manager. If so, then you must first deactivate that asset from the previous ASR Manager or deactivate that asset in My Oracle Support before re-activating again from a different ASR Manager.

5.11.2 Activation Failed for Asset <asset name> Due to Data Error (Solaris 10 Only)

This message indicates that the message creation failed because of bad or missing data. See Activation Failed Bad Serial for a sample of this activation-failure e-mail. Most of the time, this error is the result of an incorrect or incomplete serial number or product name. To troubleshoot this message, complete the following steps:

  1. View the ASR Asset's serial number using the following URL:

    http://[AgentipAddress]:6481/stv1/agent/
    
  2. If product name is empty or "unknown," then check if the Hardware Service Tags are installed and online. Run:

    svcs | grep sthwreg
    

    The results should look like this:

    online Aug_23 svc:/application/sthwreg:default
    
  3. If the serial number is incorrect, contact Oracle Support to resolve the problem.

5.12 Troubleshooting StorageTek Virtual Storage Manager (VSM) Assets

Activate the VSM_SVA ASR Asset with the following command:

asr> activate_storage -d VSM_SVA -i <IP address>

If there are problems, common troubleshooting solutions include:

  1. If the activation failed, the output should look like this:

    Failed to configure VSM_SVA device at <IP address>. Can't proceed with activation.
    Please refer to ASR documentation for troubleshooting steps.
    

    To resolve the problem, ensure the device IP address is accessible from the ASR Manager on port 9877. Run the following command:

    telnet <device IP> 9877
    
  2. If the activation failed because the device type is unsupported, the output should look like this:

    Cannot activate device. Unsupported Device Type. Svm-sva
    Supported device Types are: VSM_SVA
    

    For example, you would see this output from the following command:

    asr> activate_storage -d Svm-sva -i <IP address>
    

    To resolve the problem, use the supported device type (-d) VSM_SVA.

  3. If the IP address is invalid, then output should look like this:

    Failed to configure VSM-SVA device at <IP address> to send alerts to ASR manager. Can't proceed with activation.
    Please check if  <IP address> belongs to a VSM_SVA asset.
    Ensure the asset <IP address> is accessible from ASR manager on port 9877.
    Please refer to ASR documentation for troubleshooting steps.
    

    To resolve the problem, verify that the setup procedures have been completely followed and implemented. Run the following command if the IP address is accessible on port 9877:

    telnet <device IP> 9877
    
  4. If the activation failed because the VSM_SVA serial number could not be determined, then the output should look like this:

    Failed to run "status id" command to obtain serial number of VSM_SVA device at <IP address>. Can't proceed with activation.
    Please refer to ASR documentation for troubleshooting steps.
    

    To resolve the problem, ensure that the VSM_SVA asset configuration is done properly. Manually run the status id command on the asset and ensure serial number is properly configured on the asset.

  5. If the activation failed because the VSM_SVA asset configuration to send the alerts to the ASR Manager has failed, then the output should look like this:

    Failed to configure VSM-SVA device at <IP address> to send alerts to ASR manager. Can't proceed with activation.Please refer to ASR documentation for troubleshooting steps to manually configure the VSM_SVA asset.
    

    To resolve the problem, you must configure the asset manually to send alerts to the ASR Manager. Run the following commands on the VSM_SVA device:

    vshell -f "rvsadd <asr manager IP> " bye
    vshell -f "rvsstem /var/opt/SUNWsasm/alerts/VSM_SVA" bye
    

5.13 Troubleshooting ILOM

The following sections provide information about troubleshooting the Integrated Lights Out Manager (ILOM):

5.13.1 Check the Service Tags on ILOM

Follow the procedure below to check the Service Tags on ILOM:

  1. Log in to the ILOM service processor CLI.

  2. To view the ILOM Service Tags properties, enter:

    show /SP/services/servicetag
    

    Output should look like this:

     /SP/services/servicetag
        Targets:
    
        Properties:
            passphrase = none
            servicetag_urn = Q9525
            state = disabled
    
        Commands:
            cd
            set
            show
    
  3. To enable Service Tags, you must enable the state property. Run:

    set /SP/services/servicetag state=enabled