F Servicing the Feature Card and Recovering ACSLS

This appendix provides procedures used to service the SL4000 feature card and recover the ACSLS application running on the feature card.

Topics include:

Note:

All hardware-related tasks described in this appendix are to be performed by Oracle Support. These tasks are combined with ACSLS software recovery tasks, which can be performed by Oracle Support or by the customer, in conjunction with Oracle Support.

Performing a Soft Boot of the Feature Card

In certain situations, you may be required to perform a soft reboot of the feature card. If possible, shut down the applications running on the card before performing a soft boot.

To perform the soft boot, log in as user root and issue the following command:

reboot -f

Monitoring the Feature Card

To monitor your feature card hardware performance, follow the directions outlined for viewing the SL4000 GUI Hardware tab. Refer to the SL40000 Library Guide for more information.

Shutting Down the Feature Card and ACSLS for Service

When it is necessary to service the SL4000 library or its components, you must shut down the card and any applications running on it. Once the card is rebooted, you can then restart the applications. Example commands follow:

  1. Log in as user acsss and enter the following command to shut down the ACSLS application.

    acsss shutdown
    
  2. Log in as user root and enter the following command to shut down the feature card.

    shutdown –h now
    
  3. If hardware service is required, physically remove the feature card, perform required service, and then re-insert the feature card in its original slot to power it up. As a reminder, once a feature card is initially inserted in a specific slot within the SL4000 Base Card Cage, you must always replace it in the same slot.

  4. Start the ACSLS software.

    acsss enable
    

Replacing a Faulty Feature Card

Note:

This procedure applies to a faulty feature card. Do not use this method to upgrade the feature card or the ACSLS application.

To replace a faulty feature card with ACSLS, perform the following steps:

  1. If the hardware is up and ACSLS is running on the feature card, you must shut down ACSLS. Log in as user acsss and enter the following command to shut down the ACSLS application:

    acsss shutdown
    
  2. If the feature card is up and running, you must shut down the feature card. Log in as user root and enter the following command to shut down the feature card and safely remove it:

    shutdown -h now
    
  3. Physically remove the faulty feature card and unseat the associated feature storage cards.

  4. Use one of the following options to initialize the replacement feature card:

    • Place the new replacement feature card in the slot that the faulty feature card was removed from, and then boot it. Then insert the feature storage cards.

    • Place the new replacement feature card into the previously unused slot and boot it. Then insert the feature storage cards and perform a soft boot. You must remove the feature storage cards from their locations above the original feature card slot and place them directly into the new feature storage card slots, directly above the new feature card slot. Then proceed with the ACSLS setup and installation process as noted below.

    Note:

    Apply new external labels to the feature card and feature storage cards. Include appropriate names, to represent changes made.

    You have now effectively initialized the feature card, just as you did in the initial installation of the feature card.

  5. Log in as user root and verify this initialization as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 3: Verify Feature Card Initialization".

  6. Connect your equipment and establish a network connection as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See:

  7. Download the ACSLS bundle as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 3: Download the ACSLS Software Bundle".

    If there was a previously installed and running version of ACSLS on the feature card, you can use an alternative approach to download the ACSLS 8.5 software bundle from one of the current feature storage cards, assuming they are running and accessible. To use this approach, perform the following steps:

    1. As user root, run the command lsblk –f to determine the Linux device name of one of the two feature storage cards. Output should appear similar to the following:

      # lsblk -f
      NAME FSTYPE LABEL UUID MOUNTPOINT
      sdX
      sdY
      sdZ
      sdZ1 ext4   Aboot f0bf63fa-3460-40dd-9a91-83022631277f /boot
      sdZ2 ext4   Au01  5f73feature cardb5-627d-4ce2-afeature card6-11a16d7310f0 /u01
      sdZ3 ext4   Aslash 60aa9dd0-e629-4f40-8e03-74b6db53d3dd /
      sdZ4
      sdZ5 swap   Aswap  65f20091-01a7-49b6-b212-985a6ebcf549 [SWAP]
      sdZ6 ext4   Avar   6036e949-5403-48db-a522-035db9f60e88 /var
      #
      

      Note:

      • sdX, sdY, and sdZ, may be sda, sdb, sdc, sdd, or some other name, as determined by Linux. sdZ refers to the SSD on the feature card; and /bkupc is also located on the SSD. sdX and sdY refer to the hard drives on the feature storage cards.

      • mdN may be md1 or md2, depending upon whether you have installed the feature card into the left side (md1) or the right side (md2).

    2. Identify the Linux device names for the two feature storage cards in the listing from lsblk. Select one of these to be temporarily mounted, from which you can access the needed ACSLS software bundle. Assume one of these is named sdX1. As user root, mount the feature storage card device to a temporary mount point and copy the previously downloaded version of ACSLS as follows:

      mkdir /tmpMount
      mount /dev/sdX1 /tmpMount
      cp /tmpMount/ACSLS_8.5.1-X.Y.Linux.zip /tmp
      umount /tmpMount
      rmdir /tmpMount
      
    3. If you were able to access a viable copy of the ACSLS software bundle, you can now find it in /tmp. If a viable copy was not available at /dev/sdX1, try this same command sequence with the other feature storage card device named /dev/sdY1. If no viable copy was found on all available feature storage cards, then you will need to use one of the methods described earlier (via browser and eDelivery or via a pre-loaded flash drive) to download the bundle.

  8. Extract the feature card scripts as described in :

    See "Step 4: Extract the ACSLS Feature Card Scripts".

  9. Ensure network, host name, DNS, and other network-related settings are established.

    The ACSLS recovery process maintains as many settings as possible. However, depending upon the nature of the fault, you may need to redo one or all of your previous configuration steps. It is recommended that you verify the settings established previously, including bond3 network connectivity and ensure that your IP address, host name, and related items are correct using the hostname command, ping command, and others as required. If you encounter issues, then review applicable steps as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Configuring the Feature Card and Preparing for ACSLS Installation".

    Note:

    If you need to change your IP address, host name, or other network-related information as part of this task for feature card replacement, you must redo these steps.
  10. Verify that the feature storage cards are initialized as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 5: Insert and Initialize All Feature Storage Cards".

  11. As user root, issue the following commands to remove remnants from the previous ACSLS installation:

    rm -rf /opt/ACSLS/ACSLS_8.5.1
    userdel acsss
    userdel acssa
    userdel acsdb
    userdel postgres
    yum remove ACSLS
    yum remove PostgreSQL
    

    You may need to kill errant processes if they prohibit successful execution of yum removals. It is important that ACSLS and PostgreSQL are successfully removed.

  12. Extract the ACSLS software bundle as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 10: Extract the ACSLS Software Bundle".

  13. Perform the steps described in "Installing, Configuring, and Running ACSLS on the Feature Card".

  14. Restore the ACSLS database from the most recent backup or an accessible backup location:

    1. As user acsss, run db_restore.sh:

      su - acsss
      cd /bin
      ./db_restore.sh latest
      or
      ./db_restore.sh/bkupb/<DB_SNAPSHOT_FILENAME>
      
      

      Substitute the specific file name of your backup for latest and
      <DB_SNAPSHOT_FILENAME>.

      Redundant copies of recent backup files reside under /bkupa, /bkupb or /bkupc. If no ACSLS database backups are available to you locally, you may want to pull the latest database backup from your remote backup server. If you do not want to use an existing database backup, and would rather create a new one, then run accss_config instead.

    2. Once you have restored the database or created a new database version, re-start ACSLS as user acsss:

      su – acsss
      

Replacing a Single Feature Storage Card

To replace a single feature storage card with ACSLS, perform the following steps:

  1. Log in as user acsss and enter the following command to shut down the ACSLS application running on the feature card:

    acsss shutdown
    
  2. Physically remove the feature storage card to be replaced.

  3. Insert the replacement feature storage card into that same location.

  4. Apply a new external label to the replacement feature storage card. Include an appropriate name that reflects the changes made.

  5. As user root, run the featureCard_acslsStorageManager.sh to establish the necessary ACSLS system. This command assumes that you have shut down the ACSLS application.

    cd /usr/local/bin
    ./featureCard_acslsStorageManager.sh
    

    As part of this process, RAID storage must be mirrored. Accordingly, you should monitor the mirroring process and ensure that it has been completed before continuing to enable ACSLS. As user root, open a separate terminal window and enter the following command to monitor the rebuilding of the mirror:

    watch cat /proc/mdstat
    

    Monitor the mdstat display to determine when rebuilding of the mirrored drive is complete. Within minutes, [UU] is displayed, indicating that the rebuilding operation has completed.After [UU] is displayed, you can proceed with the next step.

Replacing Two Feature Storage Cards

To replace two feature storage cards with ACSLS, perform the following steps:

  1. Log in as user acsss and enter the following command to shut down the ACSLS application running on the feature card:

    acsss shutdown
    
  2. Physically remove the feature storage cards to be replaced.

  3. Insert the replacement feature storage cards into the same locations.

  4. Apply a new external label to the replacement feature storage cards. Include an appropriate name that reflects the changes made.

  5. Extract the feature card scripts as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 4: Extract the ACSLS Feature Card Scripts".

  6. Ensure network, host name, DNS, and other network-related settings are established.

    The ACSLS recovery process maintains as many settings as possible. Verify bond3 network connectivity and ensure that your IP address, host name, and related items are correct using the hostname command, ping command, and others as required. If you encounter issues, then review applicable steps as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Configuring the Feature Card and Preparing for ACSLS Installation".

    Note:

    If you need to change your IP address, host name, or other network-related information as part of this task for feature card replacement, you must redo these steps.
  7. Verify that the feature storage cards are initialized as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 5: Insert and Initialize All Feature Storage Cards".

  8. As user root, issue the following commands to remove remnants from the previous ACSLS installation:

    rm -rf /opt/ACSLS/ACSLS_8.5.1
    userdel acsss
    userdel acssa
    userdel acsdb
    userdel postgres
    yum remove ACSLS
    yum remove PostgreSQL
    

    You may need to kill errant processes if they prohibit successful execution of yum removals. It is important that ACSLS and PostgreSQL are successfully removed.

  9. Extract the ACSLS software bundle as described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card":

    See "Step 10: Extract the ACSLS Software Bundle".

  10. Perform the steps described in "Installing, Configuring, and Running ACSLS on the Feature Card".

  11. Restore the ACSLS database from the most recent backup or an accessible backup location:

    1. As user acsss, run db_restore.sh:

      su - acsss
      cd /bin
      ./db_restore.sh latest
      or
      ./db_restore.sh/bkupb/<DB_SNAPSHOT_FILENAME>
      
      

      Substitute the specific file name of your backup for latest and
      <DB_SNAPSHOT_FILENAME>.

      Redundant copies of recent backup files reside under /bkupa, /bkupb or /bkupc. If no ACSLS database backups are available to you locally, you may want to pull the latest database backup from your remote backup server. If you do not want to use an existing database backup, and would rather create a new one, then run accss_config instead.

    2. Once you have restored the database or created a new database version, re-start ACSLS as user acsss:

      su – acsss
      

Upgrading ACSLS on the SL4000 Feature Card

If you have an existing SL4000 feature card running ACSLS, contact Oracle Support for guidance and required materials to perform the feature card upgrade.

  • ACSLS 8.5.1 supports SL4000 firmware version 1.0.2.75 or later.

  • ACSLS 8.5.0 supports SL4000 firmware version 1.0.1.69.30201 only. For this release, the feature card and ACSLS are installed by Oracle Support.

Note:

If you are currently running SL4000 Library firmware 1.0.1.69.30201 and ACSLS 8.5.0, Oracle highly recommends that you upgrade the SL4000 library firmware to 1.0.2.75 or later and ACSLS to release 8.5.1 in order to take advantage of valuable enhancements.

If you only need to upgrade an instance of ACSLS (but not the library firmware) running on the feature card, refer to the ACSLS upgrade procedures included in your ACSLS publications. Downgrades of ACSLS below the 8.5.1 release are not supported on the feature card.

The SL4000 library firmware must be upgraded before ACSLS can be upgraded:

  1. Ensure that you have addressed all SL4000 and ACSLS pre-installation requirements included in this document. For the SL4000, work with Oracle Support and refer to the SL4000 Modular Library System Library Guide as appropriate.

  2. Upgrade SL4000 library firmware using procedures included in the SL4000 Library Guide.

  3. Work with Oracle Support to upgrade the feature cards to the same firmware version as used for the SL4000 library firmware.

    Once you have completed the hardware tasks to upgrade the SL4000 library and feature cards, and have inserted the associated hard drives, you can then install the latest ACSLS software.

    Log in to the feature card as user root and perform the steps described in Chapter 4, "Installing ACSLS on the SL4000 Feature Card" beginning with "Initializing the Feature Card and Storage Cards".

    Follow the remaining steps to install and configure ACSLS 8.5.1 on the feature card.Once ACSLS installation is complete, you can either begin anew or restore the ACSLS database from the latest (most recent) backup as user acsss.

    To restore a previous version of the ACSLS database, load the database from a remote server onto the feature card into the /bkupc directory from a flash drive or other acceptable file transfer approach within your organization. Then run
    db_restore.sh using the backup's specific file name and directory location as shown in the following example:

    su - acssscd /bin./db_restore.sh /bkupc/<DB_SNAPSHOT_FILENAME>
    

    Once you have restored a database, re-start ACSLS as user acsss:

    su – acsssacsss enable
    

    If you do not want to use an existing database backup, then run accss_config to create a new one.