Skip Headers
Oracle® Auto Service Request Installation and Operations Guide
Release 4.4 for Linux and Solaris

Part Number E18475-20
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Index
Index
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
PDF · Mobi · ePub

5 ASR General Troubleshooting

This chapter provides a variety of troubleshooting procedures for the ASR software.

The instructions provided are for Solaris. When possible, corresponding Linux instructions are provided. Please see the appropriate Linux documentation for details for general administration commands.

Note:

To enter the ASR prompt (asr>) as root, type asr on the command line. See "Install ASR" for instructions for setting the PATH environment variable.

The following troubleshooting topics are presented:

5.1 ASR Status

You can review the status of any ASR Asset from the ASR Manager or from My Oracle Support. The following ASR Status troubleshooting topics are presented:

5.1.1 View Status from the ASR Manager

The status of any ASR Asset can be obtained by running any one of the following command options from the ASR Manager system:

  • asr> list_asset - lists all assets associated with this ASR Manager

  • asr> list_asset -i IP_Address_of_ASR Asset

  • asr> list_asset -h Hostname_of_ASR Asset

  • asr> list_asset -s Subnet_IP_Address_of_ASR Asset(s)

Note:

The list_asset command accepts a comma-delimited list of IP addresses, subnets, or hostnames.

The results will be similar to the following example:

Surrounding text describes asr_status_example.gif.

5.1.2 View Status from My Oracle Support

To view the status of all ASR Assets, log in to My Oracle Support (https://support.oracle.com). In the My Oracle Support Dashboard, click the “Systems...” tab. Then select “Settings” from the menu.

In the "Settings" pane on the left of the window, select "Assets" (located under the Administrative submenu). A complete list of all ASR Assets is displayed:

Surrounding text describes assets.gif.

5.1.3 ASR Log Files

When you are troubleshooting ASR, you can change the level of information displayed in the logs, and increase or decrease the number of logs that are saved before being overwritten. The logs are written to the sw-asr.log files. Log files are located on the ASR Manager system at /var/opt/SUNWsasm/log

Log File Description
asr-sw-autoupdate.log Status updates for the ASR Auto Update feature.
asr-http-rcvr-accepted.log.0 Messages accepted by the ASR Manager HTTP receiver
asr-http-rcvr-rejected.log.0 Messages rejected by the ASR Manager HTTP receiver
asr-http-rcvr.log.0 Messages processed by the ASR Manager HTTP receiver
sasm.log Error messages and activity regarding the Oracle Automated Service Manager (OASM)
sw-asr-accepted.log.0 Fault events accepted by the ASR Manager
sw-asr-objectpool.log.0 Troubleshooting information used by the Oracle ASR support team
sw-asr-rejected.log.0 Fault events rejected by the ASR Manager
sw-asr-servicerequest.log.0 Oracle service request numbers created by ASR
sw-asr.log.0 Error messages and activity regarding the ASR Manager

There are four levels of logs:

  1. Fine: Displays the highest level of information. It contains fine, informational, warnings and severe messages.

  2. Info: Displays not only informational data, but also both warnings and severe messages. This is the default setting.

  3. Warning: Displays warnings and severe messages.

  4. Severe: Displays the least amount of information; severe messages only.

The default number of logs collected and saved is 5. Once that number is reached, ASR begins overwriting the oldest file. You have the option to change the number of logs collected and saved. If you are gathering as much information as possible in a short time, you might want to limit the number of logs saved to accommodate the larger files.

5.1.3.1 Set Log Level

Follow the procedure below to set logging levels:

  1. Open a terminal window and log in as root on the ASR Manager system.

  2. To view the current level of information being gathered, run:

    asr> get_loglevel
    
  3. To change the logging level, run:

    asr> set_loglevel [level]
    

    The choices for level are: Fine, Info, Warning, or Severe.

5.1.3.2 Set Log File Counts

Follow the procedure below to set log file counts:

  1. Open a terminal window and log in as root on the ASR Manager system.

  2. To view the current number of logs being saved, enter the following command:

    asr> get_logfilecount
    
  3. To change the number of logs being saved, enter the following command:

    asr> set_logfilecount [number]
    

5.1.4 Check the State of ASR Bundles

For diagnostic purposes, it may be necessary to check the state of various application bundles installed on the ASR Manager system using the following procedure.

  1. Open a terminal window and log in as root to the ASR Manager.

  2. Enter the following command:

    asr> diag
    
  3. Review the results of this command below along with the settings you should see:

    id State Bundle
    744 ACTIVE com.sun.svc.asr.sw_3.8.0
    Fragments=745, 746
    745 RESOLVED com.sun.svc.asr.sw-frag_3.8.0
    Master=744
    746 RESOLVED com.sun.svc.asr.sw-rulesdefinitions_3.8.0
    Master=744
    748 ACTIVE com.sun.svc.asr.sw.http.AsrHttpReceiver_1.0.0
    Fragments=749
    749 RESOLVED com.sun.svc.asr.sw.http-frag_1.0.0
    Master=748
    743 ACTIVE com.sun.svc.ServiceActivation_3.8.0
    
  4. The state of each bundle should be as follows:

    • com.sun.svc.asr.sw bundle should be ACTIVE

    • com.sun.svc.asr.sw-frag should be RESOLVED

    • com.sun.svc.asr.sw-rules definitions should be RESOLVED

    • com.sun.svc.ServiceActivation should be ACTIVE

    • com.sun.svc.asr.sw.http.AsrHttpReceiver bundle should be ACTIVE

    • com.sun.svc.asr.sw.http-frag should be RESOLVED

  5. If any of these states are incorrect, enter the following commands:

    asr> stop
    asr> start
    
  6. Repeat steps 1 to 3.

  7. To ensure everything is working properly, run the following commands:

    asr> test_connection
    asr> send_test
    

5.2 ASR Diagnostic Utility

An ASR Diagnostic Utility is included as part of the ASR software bundle to provide analysis support of any installation problems. This utility packages the data collected and stores it in a consistent, configurable location for later retrieval/delivery and analysis. The .zip file, which is created, will need to be sent to Oracle manually.

Run ./asrDiagUtil.sh and follow the on-screen instructions on where the diagnostic data file is being generated.

The following ASR Diagnostic Utility troubleshooting topics are presented:

5.2.1 Configure the ASR Diagnostic Utility

The diag-config.properties file consists a list of properties for specifying location of the configuration and log directories. It also contains "toggle switches" for enabling and disabling a particular data set to be collected:

  • com.sun.svc.asr.util.diag.home.directory – The property for specifying where the diagnostic data .zip bundle will be generated. Default is current directory where the ASR Diagnostic Utility is located.

  • com.sun.svc.asr.util.diag.zip.file.prefix – The property for configuring the diagnostic data .zip file's name.

  • com.sun.svc.asr.util.diag.zip.recursive property – The property for enabling traversing into subdirectories of any configuration or log directories.

5.3 ASR Manager Crash Recovery

In cases where an ASR Manager experiences a critical failure, you can set up a new ASR Manager and reconfigure ASR Assets to report to the new host. The following steps describe a sample scenario:

  1. An ASR Manager is set up (e.g., hostname: ASRHOST01, IP address: 10.10.10.1) and configured on the network. This ASR host is registered and activated to itself.

  2. All ASR assets are configured to report failures to the ASR Manager host (ASRHOST01), and all ASR assets are activated on the host.

  3. A critical failure occurs in the cabinet of ASRHOST01 (for example: a fire destroys the system and its data). The assets need to be attached to a different ASR Manager host (e.g., hostname: ASRHOST02).

  4. A new ASR Manager is set up (e.g., hostname: ASRHOST02, IP address: 10.10.10.2) and configured on the network. The new ASR host is registered and activated to itself.

  5. All ASR assets are now re-configured to report failures to the new ASR Manager host ASRHOST02, and the trap destination is changed to report failures to ASRHOST02.

  6. All ASR assets are now activated on ASRHOST02

Note:

In order to reduce the additional work with moving the ASR Manager to a different location (e.g., from ASRHOST1 to ASRHOST2), you can create an ASR backup on another host or on the existing host. Creating a backup is crucial when recovering from a crash (see "ASR Backup and Restore" for a details on creating an ASR backup).

5.4 ASR - No Heartbeat

The ASR Manager must be configured correctly to send the daily cron job for asr heartbeat. After 50 hours, the unit will be marked as a 'Heartbeat Failure' unit.

You can check to see if any ASR Manager or ASR Asset are in Heartbeat Failure by reviewing the ASR status in My Oracle Support.

If you feel that ASR Manager is configured correctly, then you can troubleshoot your ASR Manager hardware to resolve the problem. See MOS knowledge article 1346328.1 for the instructions to your particular hardware:

https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=HOWTO&id=1346328.1

See Appendix A, "Heartbeat Failure Notification E-mail Examples" for an e-mail example you may receive should this problem occur.

5.5 ASR Assets for Solaris 11 Troubleshooting

In cases where you are having issues with configuring ASR on Solaris 11 assets using the asradm command, then review the status of the following asr-notify SMF service:

svcs asr-notify

Output should look like this:

STATE        STIME      FMRI
online       13:00:31   svc:/system/fm/asr-notify:default

Note:

If the asr-notify service status is in maintenance mode, then clear the maintenance mode:
svcadm clear asr-notify

re-register the Solaris 11 asset with ASR manager

5.6 Resolve ASR Manager Java Path Location in config.ini File

If you have an incorrect or old version of Java installed, the ASR Manager will not start. The command to start OASM will report the following message (see Start ASR and OASM for Solaris and Linux command samples):

*************************************************************************
Warning! An old Java version ( 1.5 ) was detected (tried 
'/usr/jdk/jdk1.5.0_16/bin/java').
Oracle Automated Service Manager requires a Java version of 1.6 or higher
to run correctly.

You can set 'java.exec' property in file
/var/opt/SUNWsasm/configuration/config.ini
to point to JAVA 1.6 or later

Java can be downloaded from http://www.java.com
*************************************************************************
 
  1. Check the Java version you have installed. From the ASR Manager, run:

    java -version
    

    See Java Requirements for details of the Java version requirements for ASR. ASR requires Java 6 (1.6.0_04) or later, or Java 7 (1.7.0_13) or later.

  2. Get the current Java path location. From the ASR Manager, run:

    cat /var/opt/SUNWsasm/configuration/config.ini | grep ^java.exec
    

    The output would look like this:

    java.exec=/usr/bin/java
    
  3. Make a backup of the config.ini file. From the ASR Manager, run:

    cp /var/opt/SUNWsasm/configuration/config.ini /var/opt/SUNWsasm/configuration/config.ini_<current-timestamp>
    
  4. Edit the java.exec property in the config.ini file to point to the value of the java.exec output from Step 2, which should be for Java 6 or Java 7.

  5. Stop and start OASM. From the ASR Manager, run:

    • For Oracle Solaris:

      svcadm restart sasm
      
    • For Linux:

      /opt/SUNWsasm/bin/sasm stop-instance
      /opt/SUNWsasm/bin/sasm start-instance
      

5.7 Service Tools Bundle (STB) Troubleshooting (Solaris 10 Only)

This section provides a variety of steps to check on the state of the Service Tools Bundle (STB) that must installed on most ASR systems. If issues arise during the installation and operation of ASR, STB may be part of the issue.

The following STB troubleshooting areas are presented:

5.7.1 Check the Service Tags

  1. Open a browser window to the system you wish to check using the following command. Be sure to include the / (slash) after agent.

    http://asr_system_hostname:6481/stv1/agent/

  2. A response similar to the following will be displayed:

    <st1:response>
    <agent>
    <agent_urn><agent urn number></agent_urn>
    <agent_version>1.1.4</agent_version>
    <registry_version>1.1.4</registry_version>
    <system_info>
    <system>SunOS</system>
    <host><your host name></host>
    <release>5.10</release>
    <architecture>sparc</architecture>
    <platform>SUNW,Sun-Fire-V215::Generic_137111-06</platform>
    <manufacturer>Sun Microsystems, Inc.</manufacturer>
    <cpu_manufacturer>Sun Microsystems, Inc.</cpu_manufacturer>
    <serial_number>0707FL2015</serial_number>
    <hostid><host ID number></hostid>
    </system_info>
    </agent>
    </st1:response>
    
  3. If you do not get a response from the Service Tags agent, consult the Service Tags man pages:

    man in.stlisten
    man stclient
    

5.7.2 Check the Service Tags on ILOM

Follow the procedure below to check the Service Tags on ILOM:

  1. Log in to the ILOM service processor CLI.

  2. To view the ILOM Service Tags properties, enter:

    show /SP/services/servicetag
    

    Output should look like this:

    /SP/services/servicetag
        Targets:
    
        Properties:
            passphrase = none
            servicetag_urn = Q9525
            state = disabled
    
        Commands:
            cd
            set
            show
    
  3. To enable Service Tags, you must enable the state property. Run:

    set /SP/services/servicetag state=enabled
    

5.7.3 Check the Service Tags Version

Follow the procedure below to check the Service Tags version:

  1. Open a terminal window and log in as root to the ASR system you wish to check.

  2. Run the following command to get the Service Tags version:

    stclient -v

ASR requires Service Tags version 1.1.4 or later.

5.7.4 Check Service Tags Probe

Follow the procedure below to determine that the Service Tag discovery probe is running:

  1. Open a terminal window and log in as root to the ASR system you wish to check.

  2. To determine that the Service Tag discovery probe is running, run the following command:

    svcs -l svc:/network/stdiscover

  3. If the probe is running correctly, the following information is displayed:

    fmri svc:/network/stdiscover:default
    name Service Tag discovery probe
    enabled true
    state online
    next_state none
    state_time Wed Sep 03 21:07:28 2008
    restarter svc:/network/inetd:default
    

5.7.5 Check Service Tags Listener

Follow the procedure below to determine that the Service Tags Listener is running:

  1. Open a terminal window and log in as root to the ASR system you wish to check.

  2. To determine if the Service Tags listener is running, run the following command:

    svcs -l svc:/network/stlisten

  3. If the listener is running correctly, the following information is displayed:

    fmri svc:/network/stlisten:default
    name Service Tag Discovery Listener
    enabled true
    state online
    next_state none
    state_time Wed Sep 03 21:07:28 2008
    restarter svc:/network/inetd:default
    xibreXR_US root@s4u-v215c-abc12
    

5.7.6 Unable to Contact Service Tags on Asset

This message indicates that the activation failed during Service Tags discovery. The issue can be either Service Tags is not installed on the ASR Asset or is installed but not running. Also the issue can be network connectivity between ASR Manager and the ASR Asset. Complete the following checks:

  1. Check if Service Tags is installed and running on an ASR Asset. Run:

    stclient -x
    

    If you cannot run this command, either Service Tags is not installed or not online.

  2. Check if the Service Tags services are installed and online using the following command:

    svcs | grep reg
    
  3. The results should be similar to the following example:

    online Aug_23 svc:/application/stosreg:default
    online Aug_23 svc:/application/sthwreg:default
    
  4. If you cannot find these services, it means Service Tags is not installed on the ASR asset.

  5. If the Service Tags services are online, check if psncollector is online. Run:

    svcs | grep psncollector
    
  6. The results should be similar to the following example:

    online Sep_09 svc:/application/psncollector:default
    
  7. Make sure that there are no TCP Wrappers installed on the ASR asset to prevent any service tags discovery issues. Run the following command from the ASR Manager system:

    wget http://[assetHostNameOrIPaddress]:6481/stv1/agent/
    
  8. If there are TCP wrappers installed on the ASR asset, edit /etc/hosts.allow on the asset by adding:

    in.stlisten:[OASM host name]
    

5.7.7 Unknown or Empty Service Tags on Asset

If serial number is empty or "unknown" complete the following steps:

  1. Input the correct serial number using the SNEEP command:

    /opt/SUNWsneep/bin/sneep -s [serial number]
    
  2. Note:

    SNEEP is part of the Services Tools Bundle that is a prerequisite of ASR (for more information, see "Install Software"
  3. For versions of SNEEP older than 2.6, enter the following command:

    svcadm restart psncollector
    
  4. Note:

    If you are using SNEEP version 2.6, it is not necessary to manually restart the psncollector after inputting the serial number.
  5. You can view the serial number using the following URL:

    http://[AgentipAddress]:6481/stv1/agent/
    
  6. If product name is empty or "unknown" check if the Hardware Service Tags are installed and online. Run:

    svcs | grep sthwreg

  7. The results should be similar to the following example:

    online Aug_23 svc:/application/sthwreg:default

  8. If you cannot find this service, it means Hardware Service Tags are not installed on the ASR asset.

5.7.8 Activation Failed for Asset <asset name> Due to Data Error

This message indicates that the message creation failed because of bad or missing data. Most of the time, this error is the result of an incorrect or incomplete serial number or product name. To resolve this message, complete the following steps:

  1. Verify the serial number using the SNEEP command:

    sneep

  2. If serial number is not correct then input the correct serial number using the following SNEEP command:

    /opt/SUNWsneep/bin/setcsn -c [serial number]

  3. Note:

    SNEEP is part of the Services Tools Bundle that is a prerequisite of ASR (for more information, see "Install Software"
  4. For versions of SNEEP older than 2.6, run the following command:

    svcadm restart psncollector

  5. Note:

    If you are using SNEEP version 2.6, it is not necessary to manually restart the psncollector after inputting the serial number.
  6. You can view the serial number using the following URL:

    http://[AgentipAddress]:6481/stv1/agent/

  7. Check if the Hardware Service Tags are installed and online. Run:

    svcs | grep sthwreg

  8. The results should be similar to the following example:

    online Aug_23 svc:/application/sthwreg:default

  9. If you cannot find this service, it means Hardware Service Tags are not installed on the ASR asset.

5.7.9 Cannot Retrieve the OASM IP Address

This error message indicates that the ASR Asset activation failed because the Oracle Automated Service Manager (OASM) IP address could not be retrieved. The final step for activating an ASR Asset includes this command:

asr activate_asset -i [host IP address]

When activation fails, the following error message displays:

Cannot retrieve the SASM IP address, please add the SASM IP address to /etc/hosts

You must edit the /etc/hosts file to update the localhost entry. For example, as root, change an entry that looks like this:

127.0.0.1    hostname123.com hostname123 localhost.localdomain localhost

to this:

127.0.0.1    localhost.localdomain localhost

5.7.10 Services are Disabled: stdiscover or stlisten

Service tag processes (stlisten and stdiscover) must be online in order to activate assets successfully.

  1. Check to determine if the stdiscover or stlisten services are disabled. Run the following command:

    svcs stlisten stdiscover

    If the services have been disabled, the output would look like this:

    STATE       STIME         FMRI
    disabled    12:20:14      svc:/network/stdiscover:default
    disabled    12:20:14      svc:/network/stlisten:default
    
  2. To enable the stdiscover and stlisten services, run the following command:

    svcadm enable stlisten stdiscover

  3. Verify the services are online:

    svcs stlisten stdiscover

    Once the services have been enabled, the output would look like this:

    STATE       STIME         FMRI
    enabled     12:20:14      svc:/network/stdiscover:default
    enabled     12:20:14      svc:/network/stlisten:default
    

5.8 SMA Service Troubleshooting (Solaris 10 Only)

The SMA service needs to be online in order to support Solaris FMA enrichment data properly. Prior to configuring FMA, complete the following steps:

  1. To check that the state of the SMA service is online, run:

    svcs sma

  2. If SMA is online, the state should indicate online, as in the following example:

    STATE      STIME          FMRI
    online     15:40:31       svc:/application/management/sma:default
    
  3. If SMA is not online, run the following command to enable it:

    svcadm enable sma

  4. Repeat these steps to confirm SMA is online.

5.9 Error Messages and Resolutions

Error Message Resolution
WARNING: Unable to retrieve fault details. For additional information and some insights into how to correct, please see the ASR Installation and Operations Guide - located at www.oracle.com/asr. See the ASR General Troubleshooting Section.
  1. Verify that the asset has got the right Solaris minimum required version and patch level as per the ASR qualified systems web page (see http://www.oracle.com/asr for more information).
  2. Review the community string properties on the asset. ASR Manager requires public as the value of the community string in order to retrieve FMA enrichment and additional fault details. (See Enable M-Series XSCF Telemetry for more details)

  3. Review the FMA trap destination configuration file, and restart sma and fmd SMF services.

WARNING: This trap is rejected because the asset is disabled Enable the ASR Asset using one of the following commands:

asr> enable_asset -i <ip>

(where ip is the IP address of the ASR asset)

or

asr> enable_asset -h <host>

(where host is the hostname of the ASR asset)

WARNING: this trap is rejected because OASM ASR Plug-in is not activated Enable the ASR Manager using one of the following commands:

asr> activate_asset -i <ip>

(where ip is the IP address of the ASR asset)

or

asr> activate_asset -h <host>

(where host is the hostname of the ASR asset)

WARNING: this trap is rejected because the asset is not found Enable the ASR Asset using one of the following commands:

asr> activate_asset -i [ip]

(where ip is the IP address of the ASR asset)

or

asr> activate_asset -h [host]

(where host is the hostname of the ASR asset)

SEVERE: Cannot attach snmp trap to snmp service! This indicates that there could be another process using port 162. Kill that process and then run:

svcadm restart sasm

Failure to Register Errors The sasm.log has more detailed information and a Java stacktrace on what failed during registration. When a failure error is encountered, additional details can be found in:
/var/opt/SUNWsasm/configuration/sasm.log
No Such Host Exception This error indicates that the host running ASR Manager cannot resolve the IP address for the Data Transport Service server. Refer to Test Connectivity from the ASR Manager to Oracle to troubleshoot and resolve the problem.
Not Authorized. The My Oracle Support account provided could not be verified by the transport server This error indicates that the communication between transport server and Oracle is down or busy. This can also indicate that the queue set-up is wrong or that the user does not have permissions to the queue.
Socket Exception: Malformed reply from SOCKS server This error indicates one of the following:
  • The socks configuration in the config.ini file is incorrect or missing. Action: This usually indicates that you need to supply a user/password for the socks settings.

  • The socks is not able to route to the transport server endpoint. Action: Add the correct http proxy information or socks settings. Refer to Configure ASR to Send HTTPS Traffic Through a Proxy Server to correct the information.


"Only One Client Can Access Console at a Time" or "Can't read input from console" Error Message

If you get either of these error messages running an ASR command on the ASR Manager system, it indicates that only one command can go into the OASM admin port at a time. Each command has a max handle on the connection for 60 seconds before OASM console kills the connection. Try executing the command after 60 seconds. If you still get same message, do the following:

  1. Check if OASM is running:

    ps -ef | grep SUNWsasm

  2. Results:

    root 16817 1 0 16:09:49 ? 4:24 java -cp
    /var/opt/SUNWsasm/lib/com.sun.svc.container.ManagementTier.jar:/var/opt
    
  3. If OASM is running, kill the process using the following command:

    kill -9 [Process_ID]

  4. Restart the OASM using the following command:

    svcadm restart sasm

5.10 ASR Auto Update Troubleshooting

By default, Oracle ASR will download and install the latest version of the ASR software. The following sections provide potential solutions for problems that may arise:

5.10.1 Preparation Failed

Problem: ASR Auto Update failed with status PREPARATION_FAILED in /var/opt/SUNWsasm/log/sw-asr.log.0.

Resolution: If the preparation failed, try one of the following solutions:

  • Network connection: Check the connection between the ASR Manager to the ASR software update server. The ASR Auto Update service endpoint is:

    https://transport.oracle.com/em/upload 
    
  • Retry Auto Update: If Auto Update failed with a PREPARATION_FAILED status, it will retry again in 24 hours. Auto Update can be tried again prior to then using the autoupdate command.

  • Solaris installation: If the ASR software package is not installed properly, then the ASR Auto Update feature will not download and install updates from the ASR software update server. To verify the existing ASR package is installed properly:

    pkginfo -l SUNWswasr
    

    If this command returns an error, then remove the current package and install the latest ASR package manually. See Software Requirements for information on downloading the appropriate software packages.

  • Current package backup failed: As part of the ASR Auto Update process, a backup of the existing SWASR package will be attempted. If this process fails, then the ASR Auto Update process will not continue. To check the installed SWASR version, run:

    pkginfo -l SUNWswasr
    

    If this command returns an error or if the installed version is not current, then remove the current package and install the latest ASR package manually. See Software Requirements for information on downloading the appropriate software packages.

  • OASM is disabled or offline: To enable OASM:

    • For Solaris: svcadm enable sasm

    • For Linux: /opt/SUNWsasm/bin/sasm start-instance

5.10.2 Update for ASR Manager Available, but Auto Update is Disabled

Problem: ASR Manager will not update its software automatically if the ASR Auto Update feature is disabled. An e-mail is sent with notification if the ASR Auto Udpate feature is disabled and a new ASR Manager version is available. See Update for ASR Manager is Available, but Auto Update is Disabled for an example of the e-mail of this issue.

Resolution: If the e-mail mentioned above is received, then to resolve this issue:

  • Run Auto Update manually:

    asr> autoupdate
    

    This command will connect to the ASR software update server to initiate a download of the ASR Manager update immediately.

  • Enable the ASR Auto Update feature:

    asr> enable_autoupdate
    

    Once enabled, then ASR Auto Update will check the ASR software update server regularly and download any new updates as they appear.

5.10.3 Complete_Failed Error

Problem: While a new version of the ASR software update may have been downloaded, ASR Auto Update was not able to install it or restore the previous version. A status of COMPLETE_FAILED is returned in a /var/opt/SUNWsasm/log/sw-asr.log.0 log file:

Resolution: To resolve this issue, the ASR software package must be removed and then re-installed manually. To remove the current package:

Solaris: pkgrm SUNWswasr

Linux: rpm -e SUNWswasr

See Software Requirements for information on downloading the appropriate software packages. See Install ASR for information on installing the ASR package.

Note:

ASR Auto Update does not retry every 24 hours if it fails with a Complete_Failed error.

5.10.4 Cannot Connect to ASR Software Update Server

Problem: ASR Auto Update cannot connect to the ASR software update server. The following message will be added to the sw-asr.log file:

sw-asr.log.0 contains:
Oct 24, 2012 9:48:32 AM com.sun.svc.autoupdate.AsrAutoUpdateService autoUpdatePrep
SEVERE: ASR backend server https://transport.oracle.com/em/upload is not available for auto update.
Oct 24, 2012 9:48:32 AM com.sun.svc.autoupdate.AsrAutoUpdateService autoUpdate
SEVERE: Autoupdate preparation failed. Quiting the process.

Resolution: To resolve this issue:

  • Run the asr test_connection command to ensure the ASR Manager is registered properly with Oracle.

  • Check the sw-asr.log file to verify that ASR Auto Update failed to connect.

  • Verify that the ASR Manager has an internet connection and is able to reach the ASR software update server:

    https://transport.oracle.com/em/upload
    

5.10.5 ASR Command Line not Available After Auto Update

While ASR Auto Update is running, a lock file is created, which prevents the ASR command line from being used as long as the autoupdate process is running. The lock file is automatically deleted after the ASR Auto Update process is complete. If the lock file does not get deleted after the ASR Auto Update process is completed, then the ASR command line will be locked, and it will not be possible to run any ASR commands. If the ASR command line is locked, then running any ASR command will generate the following error message:

ASR Software is being upgraded automatically. Please try again in a few minutes.

To resolve this issue:

  1. Verify that the ASR Auto Update process /SUNWswasr_pkg_deploy.sh is still running, run:

    • Solaris:

      ps -ef | grep SUNWswasr_pkg_deploy
      
    • Linux:

      ps -ef | grep SUNWswasr_rpm_deploy
      

    Output should not list any current auto update process (SUNWswasr_pkg_deploy on Solaris, and SUNWswasr_rpm_deploy on Linux).

    WARNING:

    If the deployment script is running, then the ASR Auto Update process is still in progress. Do not delete the lock file. Wait until the Auto Update process has completed.

  2. If the ASR Auto Update process is complete, and there is no SUNWswasr_pkg_deploy (Solaris) or SUNWswasr_rpm_deploy (Linux) process, then check for the existence of the lock file at:

    /var/opt/SUNWsasm/configuration/autoupdate.lck
    
  3. If the file exists, delete it:

    rm /var/opt/SUNWsasm/configuration/autoupdate.lck
    
  4. Verify ASR connectivity. Run:

    asr> test_connection
    

5.10.6 Auto Update Fails Because of Missing Java jar Utility

If you are running JRE 1.6._04 or above (instead of JDK 1.6._04 or above), then ASR Auto Update will fail because of a missing Java jar utility. This Java jar utility that ASR Auto Update uses is not available with JRE. This issue can be corrected by using the JDK instead of the JRE.

Note:

If you have any issues with using the JDK instead of the JRE, you can manually download and upgrade to ASR Manager 4.4 from:
http://oracle.com/asr

The next version of ASR Manager (Release 4.5) will correct the Jave jar utility issue, which will allow ASR Auto Update to work with the JRE.

To check for the Java jar utility, run the following command as root on the ASR Manager server:

# jar

If the Java jar utility is not available, the following output will be shown:

jar: command not found

If you receive this error message, then run the following command to check the JRE location:

# ls -al /usr/java

Output should look like this:

lrwxrwxrwx   1 root   root   21 Dec 15 09:14 /usr/java -> /usr/j2se/jre1.6.0_xx

To resolve this missing Java jar utility issue:

  1. Update Java to JDK (from JRE):

    1. Change the Java pointing to JDK instead of JRE:

      # rm /usr/java
      
    2. Add a link with the JDK version (instead of JRE):

      # ln -s /usr/jdk/jdk1.6.0_xx /usr/java
      
    3. Test the link:

      # ls -al /usr/java
      

      Output should look like this:

      lrwxrwxrwx  1 root   root     15 Apr 16 14:52 /usr/java -> /usr/ jdk/jdk1.6.0_xx
      

      Note:

      Replace _xx in the above example with the specific JDK version running on the ASR Manager server.

      Also, ensure that you are using the same Java version in the /var/opt/SUNWsasm/configuration/config.ini file for the java.exec property.

    4. Verify the Java jar utility is available. As root on the ASR Manager, run:

      # jar
      
  2. Get the Auto Update status after fixing the Java issue. From the ASR Manager, run:

    asr> show_version
    

    You should now see a message that newer version of ASR Manager is available for download.

  3. Run Auto Update. From the ASR Manager, run:

    asr> autoupdate
    
  4. Once the Auto Update process is complete, run the following commands to verify the Auto Update and Connectivity status:

    asr> show_version
    asr> test_connection
    asr> heartbeat 
    

5.10.7 ASR Auto Update on Linux

The following ASR Auto Update troubleshooting topics specific to Linux are presented:

5.10.7.1 SELINUX Environment Variable

If the SELINUX environment variable is set to "Enforcing," then the ASR Auto Update will not be able to upgrade (remove and install) the ASR software .rpm file.

To resolve this issue, change this variable to "Permissive." Run the setenforce Permissive command to enable ASR Auto Update to remove and install the .rpm file automatically.

5.10.7.2 Missing rpm-build Package

ASR Auto Update will not work on Linux, if the Linux server is missing the rpm-build package. If the rpm-build package is missing you will see the following message during ASR rpm install:

Warning: rpm-build package is not installed on this server. ASR Manager Auto Update functionality will not work unless the rpm-build package is installed.

Auto Update functionality will be disabled until rpm-build package is installed. Please install the rpm-build package and then enable Auto Update by running \"asr enable_autoupdate\".

This error message will also appear in a log file if the Auto Update fails because of the missing rpm-build package. The log file is located at:

/var/opt/SUNWsasm/log/sw-asr-autoupdate.log

To install the rpm-build package, run either of the following commands on the Linux server:

yum install rpm-build

or:

sudo yum install rpm-build

After the rpm-build package is installed, enable ASR Auto Update. From the ASR Manager, run:

asr> enable_autoupdate

5.11 Troubleshooting StorageTek Virtual Storage Manager (VSM) Assets

Activate the VSM_SVA ASR Asset with the following command:

asr> activate_storage -d VSM_SVA -i <IP address>

If there are problems, common troubleshooting solutions include:

  1. If the activation failed, the output should look like this:

    Failed to configure VSM_SVA device at <IP address>. Can't proceed with activation.
    Please refer to ASR documentation for troubleshooting steps.
    

    To resolve the problem, ensure the device IP address is accessible from the ASR Manager on port 9877. Run the following command:

    telnet <device IP> 9877
    
  2. If the activation failed because the device type is unsupported, the output should look like this:

    Cannot activate device. Unsupported Device Type. Svm-sva
    Supported device Types are: VSM_SVA
    

    For example, you would see this output from the following command:

    asr> activate_storage -d Svm-sva -i <IP address>
    

    To resolve the problem, use the supported device type (-d) VSM_SVA.

  3. If the IP address is invalid, then output should look like this:

    Failed to configure VSM-SVA device at <IP address> to send alerts to ASR manager. Can't proceed with activation.
    Please check if  <IP address> belongs to a VSM_SVA asset.
    Ensure the asset <IP address> is accessible from ASR manager on port 9877.
    Please refer to ASR documentation for troubleshooting steps.
    

    To resolve the problem, verify that the setup procedures have been completely followed and implemented. Run the following command if the IP address is accessible on port 9877:

    telnet <device IP> 9877
    
  4. If the activation failed because the VSM_SVA serial number could not be determined, then the output should look like this:

    Failed to run "status id" command to obtain serial number of VSM_SVA device at <IP address>. Can't proceed with activation.
    Please refer to ASR documentation for troubleshooting steps.
    

    To resolve the problem, ensure that the VSM_SVA asset configuration is done properly. Manually run the status id command on the asset and ensure serial number is properly configured on the asset.

  5. If the activation failed because the VSM_SVA asset configuration to send the alerts to the ASR Manager has failed, then the output should look like this:

    Failed to configure VSM-SVA device at <IP address> to send alerts to ASR manager. Can't proceed with activation.Please refer to ASR documentation for troubleshooting steps to manually configure the VSM_SVA asset.
    

    To resolve the problem, you must configure the asset manually to send alerts to the ASR Manager. Run the following commands on the VSM_SVA device:

    vshell -f "rvsadd <asr manager IP> " bye
    vshell -f "rvsstem /var/opt/SUNWsasm/alerts/VSM_SVA" bye