Configuring the FCAT Environment
Perform the following tasks to configure the FCAT environment:
Note:
In the steps below that require you to manually edit specific configuration files, please make a restorable copy of the original.
-
Step 3: Extract and Install the Feature Card Availability Toolkit
-
Step 6: Copy Select Feature Card Scripts to a Separate FCAT Linux Client
-
Step 7: Remove Existing Trusts Between the FCAT Client and Feature Cards
-
Step 8: Configure the FCAT Client to Recognize Both Side-A and Side-B Feature Cards
-
Step 11: Trigger Failover from One Feature Card to the Other
Step 1: Install and Configure ACSLS on Both Feature Cards
To begin, choose one of the following configuration options, based on whether you want to configure both feature cards with the same IP Address and DNS name, or with two different IP addresses and DNS names:
-
Option 1
In this option, both feature cards use the same IP address and DNS name, as shown in the following figure:
For clients attached to ACSLS through the feature card, all client addressing uses the same IP address and DNS name, regardless of which feature card you are using at a given time. bond2 is the network interface used only for internal library communications. bond3 is the customer network interface used to communicate with the feature cards.
This is the default option.
-
Option 2
In this option, each feature card uses a different IP address and DNS name, as shown in the following figure:
For clients attached to ACSLS through the feature card, client addressing uses distinct IP addresses and DNS names for each feature card. This option allows you to switch connected clients to run against a specific instance of ACSLS. bond2 is the network interface used only for internal library communications. bond3 is the customer network interface used to communicate with the feature cards.
Once you have chosen your option, install and configure ACSLS on each feature card as described in Installing ACSLS on the SL4000 Feature Card.
WARNING:
When installing ACSLS, ensure that you have configured both feature card platforms and installed instances of ACSLS identically. Otherwise, ACSLS failover, startup, and other operations may not function properly.Step 2: Sync Time Between Feature Cards and the SL4000
As noted in the ACSLS Administrator's Guide, for proper operation of ACSLS with the SL4000, the ACSLS server clock must be in sync with the SL4000 internal clock. This is important on a standalone ACSLS server running against the SL4000 as well as when using ACSLS on the SL4000 feature cards. This synchronization ensures that database backups are current and correct.
Synchronize the following:
-
The time between both feature cards and the SL4000.
-
The time between both feature cards.
If you do not already have access to an NTP (Network Time Protocol) server, establish one to keep the information replicated between the two feature cards in sync.
Step 3: Extract and Install the Feature Card Availability Toolkit
The Feature Card Availability Tool kit is bundled with the initial feature card scripts extracted during feature card installation, as described in Configuring the Feature Card and Preparing for ACSLS Installation.
The following FCAT scripts are included:
-
fcatServer_clusterizeFeatureCards.sh
-
fcatServer_featureCardStatusPayload.sh
-
fcatServer_switchActivePassiveCard.sh
-
fcatClient_featureCardStatus.sh
-
fcatClient_triggerSwitch.sh
Ensure that these scripts are available under /usr/local/bin
on both the Side-A and Side-B feature card, and reload them if necessary.
-
The Side-A feature card is installed in the Feature Card Kit 1 location.
-
The Side-B feature card is installed in the Feature Card Kit 2 location.
See Feature Card Locations for more information.
Step 4: Shut Down ACSLS on Both Feature Cards
To shut down the ACSLS application on both the Side-A and Side-B feature card:
Step 5: Cluster the Feature Cards and Establish Trust
You must cluster the feature cards to establish trust between them. This enables ACSLS to keep both feature cards current with their database updates and ensures that ACSLS can be quickly resumed.
To establish trust between both feature cards, you must establish trust from the Side-A to Side-B feature card, and from the Side-B feature card to Side-A feature card, as follows:
- Establish Trust from Side-A to Side-B feature card for user
acsss
.- On the Side-A feature card, as user
acsss
, access the directory/usr/local/bin
:cd /usr/local/bin
- Run the following command:
fcatServer_clusterizeFeatureCards.sh
The following message appears:
=============================================================Trust between two feature cards plugged in the same libraryform the backbone for ACSLS availability solution on featurecards as it enables the exchange of data and files betweenthose two feature cards.ACSLS availability solution for feature card is *not*possible without trust between the two feature cards.=============================================================Do you want to create the trust [acsss:Side-A => Side-B](yes/no)?
- At the prompt, enter
yes
to create the trust. - At the password prompt, enter the password for user
acsss
on the Side-B feature card.Trust is established and a confirmation message appears:
root: fcatServer_clusterizeFeatureCards.sh: INFO: A simplex trust [acsss : Side-A => Side-B] is created successfully. =============================================================A simplex trust is created between feature cards from Side-Ato Side-B for 'acsss'. In order to break this trust, executerunuser -l acsss -c 'fcatServer_clusterizeFeatureCards.sh -d'on <server_name> from command line.=============================================================
- As user
acsss
, exit the session:exit
- On the Side-A feature card, as user
- Establish trust from the Side-A to Side-B feature card for user
root
.- On the Side-A feature card, as user
root
, access the directory/usr/local/bin
:cd /usr/local/bin
- Run the following command:
fcatServer_clusterizeFeatureCards.sh
The following message appears:
=============================================================Trust between two feature cards plugged in the same libraryform the backbone for ACSLS availability solution on featurecards as it enables the exchange of data and files betweenthose two feature cards.ACSLS availability solution for feature card is *not*possible without trust between the two feature cards.============================================================= Do you want to create the trust [root:Side-A => Side-B](yes/no)?
- At the prompt, enter
yes
to create the trust. - At the password prompt, enter the password for user
root
on the Side-B feature card.Trust is established and a confirmation message appears:
root: fcatServer_clusterizeFeatureCards.sh: INFO: A simplex trust [root : Side-A => Side-B] is created successfully. =============================================================A simplex trust is created between feature cards from Side-Ato Side-B for 'root'. In order to break this trust, executerunuser -l root -c 'fcatServer_clusterizeFeatureCards.sh -d'on <server_name> from command line.=============================================================
- As user
root
, exit the session:exit
- On the Side-A feature card, as user
- Repeat steps 1 and 2 from the Side-B feature card to establish trust from the Side-B to Side-A feature card for both user
acsss
and userroot
.Once trust is fully established, information can flow between the two feature cards and you can use the ACSLS Feature Card Availability Toolkit tools to monitor the feature cards for any issues that may be encountered.
Step 6: Copy Select Feature Card Scripts to a Separate FCAT Linux Client
To enable the ability to monitor ACSLS and trigger a switch from one feature card to another, copy the following FCAT scripts to a dedicated Linux Client server (Client host):
-
fcatClient_featureCardStatus.sh
-
fcatClient_triggerSwitch.sh
These scripts can be copied from the /usr/local/bin
directory on either the Side-A or Side-B feature card, whichever has an enabled bond3 IP address. Although these client scripts are agnostic to the directory in which they will reside on the client host, it is recommended that you dedicate a directory for these scripts.
When you are finished, verify that the files are listed in the destination directory.
Step 7: Remove Existing Trusts Between the FCAT Client and Feature Cards
In preparation for configuring the FCAT client to recognize your feature cards, you must first ensure that any older, existing trusts are removed.
Step 8: Configure the FCAT Client to Recognize Both Side-A and Side-B Feature Cards
Complete the following steps to establish Side-A and Side-B feature card keys in the known_hosts
file. This allows net operations between the platforms.
- Proceed according to your chosen configuration option:
-
If you are using Option 1, using one IP address and DNS name for both feature cards, proceed with step 2.
-
If you are using Option 2, with distinct IP addresses and different DNS names for each feature card, skip to Step 5.
-
- Ensure that the Side-B feature card is disabled on the bond3 public network. Use the following commands if necessary:
cd /usr/local/bin featureCard_bond3.sh -d
- Ensure that the Side-A feature card is enabled on the bond3 public network. Use the following commands if necessary:
cd /usr/local/bin featureCard_bond3.sh -e
- Perform the following steps to update the
known_hosts
file on your FCAT Client, adding the feature card IP address and DNS name for the Side-A feature card.- On the FCAT Client, as user
root
, run the following command using theFC_HOSTNAME
for the Side-A feature card:ssh <FC_HOSTNAME>
The following message appears:
The authenticity of host '<FC_HOSTNAME> (FC_BOND3_IP)' can't be established. RSA key fingerprint is 85:44:72:86:3f:e1:6d:44:42:8c:6d:31:5d:b4:97:5c. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '<FC_HOSTNAME>, <IP_BOND3_IP>' (RSA) to the list of known hosts. root@<FC_HOSTNAME>'s password:
- At the prompt, enter
yes
to continue the connection.A password prompt appears.
- At the password prompt, enter the
root
user password for the feature card you are attempting to connect to.A Login message appears, indicating that you are logged into the targeted feature card.
- Exit the session to return to your FCAT Client. The key for the Side-A feature card is now in the
/root/.ssh/known_hosts
file.
- On the FCAT Client, as user
- You have now successfully established the key for the Side-A feature card. If you are using an additional (Side-B) feature card, complete the following steps:
- On the FCAT client, as user
root
, open the following file in the text editor of your choice:/root/.ssh/known_hosts
This file includes entries for various known hosts.
- Edit the file, placing a
#
sign at the beginning of the entry for the Side-A feature card to comment out this entry. - Save your changes and close the file.
- Return to step 1 of this procedure and perform steps 1-4 to establish the key for the Side-B feature card, just as you did for Side-A.
- Update the
known_hosts
file again by removing the#
sign to make the Side-A feature card active. - On the Side-A feature card, as user
root
, enable the bond3 public network:cd /usr/local/bin featureCard_bond3.sh -e
- On the Side-B feature card, as user
root
, disable the bond3 public network:cd /usr/local/bin featureCard_bond3.sh -d
- On the FCAT client, as user
- You have now successfully established the keys for both Side-A and Side-B feature cards, allowing the FCAT Client to recognize them.
Step 9: Monitor ACSLS Availability from the FCAT Client
To monitor the status for both feature cards:
The following is an example of the ACSLS Availability Status screen at initial startup:
======================================================================= ACSLS Availability Status Payload ID : 1 Request Time : Wed May 22 13:46:20 MDT 2019 Report Time : Wed May 22 13:46:22 MDT 2019 *** Feature cards need to synchronize their clocks.---------------------------------------------------------------------- Side : Side-A Node : FC_hostname_A [ <IP_BOND3_A> ] Availability Status : Active since 2019-05-22 19:33:14 Peer Status (root) : Reachable [ EXTERNAL ] Peer Status (acsss) : Reachable [ EXTERNAL ] Hardware Status : up 2:08, 6 users, load average: 0.00, 0.02, 0.05 ACSLS Uptime : 0-00:00:00 <days-hrs:min:sec> rmi-registry is offline surrogate is offline acsdb is offline acsls is offline ---------------------------------------------------------------------- ---------------------------------------------------------------------- Side : Side-B Node : FC_hostname_B [ <IP_BOND3_B> ] Availability Status : Passive Peer Status (root) : Reachable [ EXTERNAL ] Peer Status (acsss) : Reachable [ EXTERNAL ] Hardware Status : up 1:51, 3 users, load average: 0.08, 0.03, 0.05 ACSLS Uptime : 0-00:00:00 <days-hrs:min:sec> rmi-registry is offline surrogate is offline acsdb is offline acsls is offline ---------------------------------------------------------------------- ======================================================================
This display is based on Option 2, with each feature card using a distinct IP address and different DNS name.
-
Status is displayed for both feature cards, as they are both available.
-
[EXTERNAL]
on the Peer status lines indicates Option 2.[INTERNAL]
would indicate Option 1. -
ACSLS is offline as it is not yet enabled.
-
The display alerts you that the clocks must be synchronized between both feature cards.
Step 10: Enable ACSLS on the Side-A feature Card
On the Side-A feature card, enter the following command to enable ACSLS:
acsss enable
Only one instance of ACSLS on the feature card can be enabled at one time.
Step 11: Trigger Failover from One Feature Card to the Other
In the event that failover is required, you must trigger a switch between the two feature cards from the FCAT Client. The active feature card becomes passive and the passive feature card becomes active.
Note:
-
Before triggering a switch, quiesce client jobs and other operations to ensure a smooth transition to the newly active feature card.
-
While the switchover activates the other card, it does not automatically activate ACSLS on that card. You will need to do so manually after the card switchover has occurred.
To trigger a feature card switch:
Step 12: Start ACSLS (with optional Database Restore)
After completing failover from one card to the other, ACSLS must be restarted manually on the newly active card. You can choose to do a database restore before starting ACSLS, or simply start ACSLS and it will use the last automated snapshot, which is synchronized between the paired Feature Cards automatically.
To simply start ACSLS and use the latest database snapshot, on the newly active feature card, perform the following steps:
- Ensure ACSLS and the database are in a known and synchronized state by shutting down:
su - acsss acsss shutdown
- Start ACSLS for continued operation:
acsss enable
If you wish to restore another database (other than the one synchronized between the Feature Cards), perform the following steps: