This chapter provides information about the following topics.
Administering the Sun Cluster software is facilitated by three Graphical User Interfaces (GUIs):
Cluster Control Panel - Launches the Cluster Console and other system administration tools.
Cluster Console - Executes commands on multiple nodes in the cluster simultaneously to simplify cluster administration.
Sun Cluster Manager - Monitors the current status of all nodes in the cluster via a HotJava browser.
Refer to the online help for complete documentation on these GUIs. You can also use utilities to monitor Sun Cluster software.
You can use the Sun Cluster hastat(1M)utility, in addition to the /var/adm/messages files, to monitor a Sun Cluster configuration. You can also use the Sun Cluster Manager graphical user interface, which shows the status of major cluster components and subcomponents. For more information about the Sun Cluster Manager, refer to "Using Sun Cluster Manager ". Sun Cluster also provides an SNMP agent that can be used to monitor up to 32 clusters at the same time. See Appendix C, Using Sun Cluster SNMP Management Solutions.
If you are running Solstice DiskSuite, you can also use the metastat(1M), metadb(1M), metatool(1M), medstat(1M), and mdlogd(1M) utilities to monitor the status of your disksets. The SNMP-based Solstice DiskSuite log daemon, mdlogd(1M), generates a generic SNMP trap when Solstice DiskSuite logs a message to the syslog file. You can configure mdlogd(1M) to send a trap only when certain messages are logged by specifying a regular expression in the mdlogd.cf(4) configuration file. The trap is sent to the administrative host specified in the configuration file. The administrative host must be running a network management application such as Solstice SunNet Manager(TM). You can use mdlogd(1M) if you don't want to run metastat(1M) periodically or scan the syslog output looking for Solstice DiskSuite errors or warnings. See the mdlogd(1M) man page for more information.
If you are running VxVM, you can use the vxprint, vxstat, vxtrace, vxnotify, and vxva utilities. Refer to your volume management software documentation for information about these utilities.
For information about troubleshooting and repairing defective components, refer to the appropriate hardware documentation.
The hastat(1M) program displays the current state of the configuration. The program displays status information for the hosts, logical hosts, private networks, public networks, data services, local disks, and disksets, along with the most recent error messages. The hastat(1M) program extracts Sun Cluster-related error messages from the /var/adm/messages file and outputs the last few messages from each host if -m is specified. Because the recent error messages list is a filtered extract of the log messages, the context of some messages might be lost. Check the /var/adm/messages file for a complete list of the messages. The following pages show an example of output from hastat(1M):
# hastat -m 10 HIGH AVAILABILITY CONFIGURATION AND STATUS ------------------------------------------- LIST OF NODES CONFIGURED IN <ha-host1> CLUSTER phys-host1 phys-host2 CURRENT MEMBERS OF THE CLUSTER phys-host1 is a cluster member phys-host2 is a cluster member CONFIGURATION STATE OF THE CLUSTER Configuration State on phys-host1: Stable Configuration State on phys-host2: Stable UPTIME OF NODES IN THE CLUSTER uptime of phys-host1: 12:47pm up 12 day(s), 21:11, 1 user, load average: 0.21, 0.15, 0.14 uptime of phys-host2: 12:46pm up 12 day(s), 3:15, 3 users, load average: 0.40, 0.20, 0.16 |
LOGICAL HOSTS MASTERED BY THE CLUSTER MEMBERS Logical Hosts Mastered on phys-host1: ha-host-1 Loghost Hosts for which phys-host1 is Backup Node: ha-host2 Logical Hosts Mastered on phys-host2: ha-host2 Loghost Hosts for which phys-host2 is Backup Node: ha-host1 LOGICAL HOSTS IN MAINTENANCE STATE None STATUS OF PRIVATE NETS IN THE CLUSTER Status of Interconnects on phys-host1: interconnect0: selected interconnect1: up Status of private nets on phys-host1: To phys-host1 - UP To phys-host2 - UP Status of Interconnects on phys-host2: interconnect0: selected interconnect1: up Status of private nets on phys-host2: To phys-host1 - UP To phys-host2 - UP STATUS OF PUBLIC NETS IN THE CLUSTER Status of Public Network On phys-host1: bkggrp r_adp status fo_time live_adp nafo0 le0 OK NEVER le0 Status of Public Network On phys-host2: bkggrp r_adp status fo_time live_adp nafo0 le0 OK NEVER le0 |
STATUS OF SERVICES RUNNING ON LOGICAL HOSTS IN THE CLUSTER Status Of Registered Data Services q: Off p: Off nfs: On oracle: On dns: On nshttp: Off nsldap: On Status Of Data Services Running On phys-host1 Data Service HA-NFS: On Logical Host ha-host1: Ok Status Of Data Services Running On phys-host2 Data Service HA-NFS: On Logical Host ha-host2: Ok Data Service "oracle": Database Status on phys-host2: SC22FILE - running; No Status Method for Data Service "dns" RECENT ERROR MESSAGES FROM THE CLUSTER Recent Error Messages on phys-host1 ... Recent Error Messages on phys-host2 ... |
The Sun Cluster software writes messages to the /var/adm/messages file, in addition to reporting messages to the console. The following is an example of the messages reported when a disk error occurs.
... Jun 1 16:15:26 host1 unix: WARNING: /io-unit@f,e1200000/sbi@0.0/SUNW,pln@a0000000,741022/ssd@3,4(ssd49): Jun 1 16:15:26 host1 unix: Error for command `write(I))' Err Jun 1 16:15:27 host1 unix: or Level: Fatal Jun 1 16:15:27 host1 unix: Requested Block 144004, Error Block: 715559 Jun 1 16:15:27 host1 unix: Sense Key: Media Error Jun 1 16:15:27 host1 unix: Vendor `CONNER': Jun 1 16:15:27 host1 unix: ASC=0x10(ID CRC or ECC error),ASCQ=0x0,FRU=0x15 ... |
Because Solaris and Sun Cluster error messages are written to the /var/adm/messages file, the /var directory might become full. Refer to "Maintaining the /var File System" for the procedure to correct this problem.
In addition, Sun Cluster provides utilities for configuring and administering the highly available data services. The utilities are described in their associated man pages. The utilities include:
cconsole(1) - Starts the cluster console GUI.
ccp(1) - Starts the cluster control panel GUI.
ctelnet(1) - Starts a telnet session.
crlogin(1) - Starts an rlogin session.
chosts(1) - Expands a cluster name into a list of hosts belonging to the cluster.
cports(1) - Expands a host name into a host, node, port triplet. Used by cconsole(1) to identify the serial port consoles of the named hosts via the terminal server returned in the triplets.
scconf(1M) - Creates or modifies configuration information.
Each Sun Cluster administration tool includes detailed online help. To access the online help, launch one of the administration tools from the administrative workstation and select Help from the menu bar.
Alternatively, double-click on the Help icon in the Cluster Control Panel.
Help topics cover the administration tools in detail, as well as some administration tasks. Also see Chapter 4, General Sun Cluster Administration, for additional detailed instructions on performing specific tasks.
Figure 2-1 shows a sample Help window for the Cluster Control Panel. The text covers a specific topic. When you first launch the Help window from a tool, it displays the top-level home topic. Afterwards, the Help window displays the last topic you viewed. Hypertext links to other topics are displayed as underlined, colored text.
Clicking once on a hypertext link displays the text for that topic. The online help system also includes an automatic history list feature that remembers the information you previously accessed. Display this list by choosing Topic History from the View menu.
The Help window has a scrollable text area, a menu bar, and several buttons. Each of these items is described in the following sections.
With Help, you can access each pull-down menu by:
Clicking on the menu name
Pressing the mnemonic, or underlined character in a menu or entry name (only when the pull-down menu is visible)
Using an accelerator, or the sequence of keys to the right of menu items
You can customize the mnemonics and accelerators; refer to the online help for more information.
The tables in this section list each menu item, define the menu functions, and list the accelerators (keyboard combinations).
The Help window menu contains the File, View, and Help menu items. You display the menus for these items by selecting them.
The File menu contains these items:
Table 2-1 File Menu Items
Item |
Function |
Accelerator |
---|---|---|
Print Topic |
Prints the topic currently displayed in the Help window scrolled text area. |
Alt + R |
Dismiss |
Dismisses the Help window. |
Alt + D |
The View menu contains these items:
Table 2-2 View Menu Items
Item |
Function |
Accelerator |
---|---|---|
Previous Topic |
Displays the previous help topic (if any). |
Alt + P |
Next Topic |
Displays the next help topic (if any). |
Alt + N |
Home Topic |
Displays the home (top level) topic. |
Alt + O |
Topic History... |
Displays the Help Topic History dialog box which allows you to navigate easily through the help topics you have already viewed. The uppermost topic in the scrolled list is the first topic you displayed. The bottommost topic is the last topic you have viewed in the current path. The highlighted topic is the current topic. |
Alt + I |
To display the dialog box, select View -> Topic History... (Figure 2-2).
The Help menu contains these items:
Table 2-3 Help Menu Items
Item |
Function |
---|---|
Help on Help... |
Describes the Help window and explains how to use it. |
About... |
Displays the About Box, which contains information about the application, such as the version number. |
The following tables lists the help window buttons and their functions.
Table 2-4 Help Menu Items
Button |
Function |
---|---|
Home |
Displays the home page for the application. |
Dismiss |
Dismisses the Help window. |
Print Topic |
Prints the current topic on your default printer. |
Steps back through the displayed Help topics to the previous topic. Clicking on the left arrow repeatedly steps the display back through the Help windows until the first topic you viewed is redisplayed. Topics are "remembered" by the automatic Help history list. |
|
|
Steps forward through the displayed Help topics, one at a time, to the last topic in the list. |
The Cluster Control Panel (CCP) is a GUI that enables you to launch the Cluster Console, and other system administration tools. The CCP contains icons that represent these tools.
After you have installed the Sun Cluster client software on the administrative workstation, use this procedure to run an application from the CCP.
As superuser, add the Sun Cluster tools directory /opt/SUNWcluster/bin to the path on the administrative workstation.
For E10000 Platforms, you must first log into the System Service Processor (SSP) and connect by using the netcon command. Once connected, enter Shift~@ to unlock the console and gain write access. Then proceed to Step 2.
In a shell window on your workstation, bring up the CCP.
Specify the name of the cluster to be monitored:
# ccp clustername |
If the Sun Cluster tools are not installed in the default location of /opt/SUNWcluster, the environment variable $CLUSTER_HOME must be set to the alternate location.
The CCP (shown in the following figure) has a menu bar and an icon pane that displays all of the tools currently in the control panel. From the menu bar you can add, delete, or modify the tools.
From the File or Properties menu you can:
Add a new item
Delete an item
Modify an item
For detailed information about the CCP, refer to the online help.
For information about the programs represented by these tools and their usage, see "Cluster Console". For information about using the HotJava browser to monitor cluster configurations, see "Using Sun Cluster Manager ".
The CCP stores properties and other information in configuration files within a configuration directory. By default, the configuration directory is located at /opt/SUNWcluster/etc/ccp.
You must be root (superuser) to write to this default location. Only root can add, delete, or change CCP items in this configuration directory.
You can, however, create your own configuration directory and define its location using the environment variable $CCP_CONFIG_DIR. The $CCP_CONFIG_DIR variable specifies the configuration directory in which the configuration files containing item properties are stored. If the path name is not set, it defaults to the standard location, /opt/SUNWcluster/etc/ccp. To create your configuration directory, create a new directory and set the environment variable $CCP_CONFIG_DIR to the full path name of the new directory.
These files do not need to be edited manually, because they are created, modified, or deleted by ccp whenever you create, modify, or delete an item.
The Cluster Console (CC) GUI enables you to run commands on multiple nodes simultaneously, simplifying cluster administration. The Cluster Console displays one terminal window for each cluster node, plus a small Common window that you can use to control all windows simultaneously.
Different types of remote sessions enable you to connect to the console of the host, or remotely log in by using rlogin or telnet. Hosts can be specified on the command line and added or deleted from the Select Hosts dialog box after the program is running. The session type can be specified only on the command line. Once started, the session type cannot be changed.
You can issue commands to multiple hosts from the Common window, and you can issue commands to a single host from a terminal window. Terminal windows use VT100 terminal emulation.
Alternatively, you can turn off all hosts in the Hosts menu except the one you want to access, then issue commands from the Common window text field.
You can launch the Cluster Console from the CCP (see "Cluster Control Panel") or from the command line in a shell window. If an optional parameter is specified, a terminal window is created for each host in the cluster or for each host specified.
Type ctelnet to establish a telnet(1) connection from the console:
% ctelnet [clustername | hostname...] |
Launch crlogin with your user name to establish an rlogin(1) connection from the console:
% crlogin -l user name [clustername | hostname...] |
All three of the preceding commands also take the standard X/Motif command-line arguments. Once the Cluster Console has started, the Console window is displayed.
For detailed information about the Cluster Console, refer to the online help.
The Common window (shown in the following figure) is the primary window used to send input to all nodes. The Common window is always displayed when you launch the Cluster Console.
This window has a menu bar with three menus and a text field for command entry. From the Hosts menu you can use the Select dialog box to:
Add a host
Add all hosts in a cluster
Remove a host
From the Options menu, you can group or ungroup the Common Window and the terminal windows.
Two configuration files are used by the Cluster Console: clusters and serialports. These can be /etc files or NIS/NIS+ databases. The advantage to using a NIS+ environment is that you can run the Cluster Console on multiple Administrative Workstations. Refer to your NIS/NIS+ system administration documentation for complete information about NIS/NIS+.
The clusters file maps a cluster name to the list of host names that comprise the cluster. Each line in the file specifies a cluster, as in this example:
planets mercury venus earth mars wine zinfandel merlot chardonnay riesling |
The clusters file is used by all three session types of the Cluster Console (cconsole, ctelnet, and crlogin) to map cluster names to host names on the command line and in the Select Hosts dialog box. For additional information, see "Modifying the clusters File".
The serialports file maps a host name to the Terminal Concentrator and Terminal Concentrator serial port to which the host is connected. Each line in this database specifies a serial port of the host.
Sample serialports file database entries for the Sun Enterprise 10000 are:
mercury systemserviceprocessorname 23 venus systemserviceprocessorname 23 earth systemserviceprocessorname 23 mars systemserviceprocessorname 23 |
Sample serialports file database entries for all other nodes are:
mercury planets-tc 5002 venus planets-tc 5003 earth planets-tc 5004 mars planets-tc 5005 |
The serialports file is used only by the cconsole variation of this program to determine which Terminal Concentrator and port to connect to for hosts or clusters that are specified in the command line or the Select Hosts dialog box.
In the preceding example, node mercury is connected to planets-tc Port 2, while node venus is connected to planets-tc Port 3. Port 1 is reserved for the administration of the Terminal Concentrator.
For additional information, see "Modifying the serialports File".
Sun Cluster Manager (SCM) is the cluster management tool for Sun Cluster 2.2. SCM provides a single interface to many of Sun Cluster's command line monitoring features. SCM reports information about:
SCM-monitored alarms
syslog messages on each cluster node
Cluster resources, including parallel data services, logical hosts, registered HA services, cluster nodes, and volume managers
SCM consists of two parts--the SCM server software and the SCM Graphical User Interface (GUI). The SCM server must run on each node in the cluster. The SCM GUI runs as either an application or an applet. If run as an applet, SCM can be displayed through a browser compliant with the Java Development Kit (JDK(TM)), such as HotJava(TM) or Netscape. The browser can run on any machine, including the cluster nodes. For the most current information about which JDK versions are supported, see your service provider or the latest version of the Sun Cluster 2.2 Release Notes.
Use the information in the following sections to configure and run SCM.
To run SCM as an application, perform the following procedure.
On the administrative workstation, install the SCM package (SUNWscmgr) from the Sun Cluster 2.2 product CD.
Install the latest version of the SCM patch on all cluster nodes and on the administrative workstation.
For the most current information about patches and patch numbers, see the Sun Cluster 2.2 Release Notes, your service provider, or the Sun patch web site, http://sunsolve.sun.com.
Run the SCM application.
Start the application by running the following command from any cluster node, where cluster_node is a current member of the cluster.
# /opt/SUNWcluster/bin/scmgr cluster_node |
Once the SCM application is running, you can access the online help for information about menu navigation, tasks, and reference. To display the Help window from the SCM application, select Help Contents from the Help menu. Alternatively, click on the Help icon in the tool bar above the folder icon.
Refer to the scmgr(1M) man page for more information about running SCM.
To run SCM as an applet, you must perform these tasks, which are described in more detail in the procedures that follow.
On all cluster nodes, install the latest version of the SCM patch from SunSolve. For the most current information about patches and patch numbers, see the Sun Cluster 2.2 Release Notes, your service provider, or the Sun patch web site, http://sunsolve.sun.com.
Install a browser on the administrative workstation. Supported browsers for Sun Cluster 2.2 4/00 Release are HotJava (version 1.1.4 or greater) and Netscape (version 4.5 or greater).
If you installed HotJava as your browser, install the JDK on the administrative workstation.
Install and configure a web server on all cluster nodes.
Start the SCM applet by typing the appropriate URL in the browser. Make sure the host you specify in the URL is a current member of the cluster.
If you use the HotJava browser shipped with your Solaris 2.6 or 2.7 operating system, you might experience problems when using the menus. For example, after you make a menu selection, the selection might remain visible on the browser. Refer to the Sun Cluster 2.2 Release Notes for more information about SCM open issues. Solaris 8 software does not support a HotJava browser. To run SCM with Solaris 8 software, you must use another browser, such as Netscape. See "How to Run the SCM Applet in a Netscape Browser From a Cluster Node".
If you use the HotJava browser with SCM, you should have at least 40 Megabytes of free swap space. If you find that swap space gets low, restarting the HotJava browser can help.
To run SCM, you must have the correct versions of HotJava and the JDK packages (SUNWjvrt and SUNWjvjit) installed on the system containing the HotJava browser. Check your version numbers against those listed in the following table. Also see the latest version of the Sun Cluster 2.2 Release Notes for possible updates to this information:
Table 2-5 SCM Requirements: JDK and HotJava
Operating Environment |
Java Developer Kit (JDK) version |
HotJava Version |
Netscape Version |
---|---|---|---|
Solaris 2.6 |
1.1.6 or later |
1.1.4 or later |
4.5 or later |
Solaris 7 |
1.1.6 or later |
1.1.4 or later |
4.5 or later |
Solaris 8 |
1.2 or later |
not supported |
4.5 or later |
You can choose to:
Run the HotJava browser on a cluster node. If you choose this option, you will have to restart HotJava on a different node if the node running HotJava crashes.
Install a web server on each node in the cluster. If you choose this option, you will need to type an appropriate URL for another node into the HotJava browser if you see the Lost Connection dialog.
Use the following procedures as necessary to set up your preferred configuration.
Determine the current Java version by typing the following at the console prompt on a cluster server.
# java -version |
If necessary, download a later version of the JDK.
If the system displays a version of Java lower than 1.1.6, follow the instructions to download the JDK version 1.1.6 (or later) software from the following URL:
http://www.sun.com/solaris/java
From the machine running the HotJava browser, select About HotJava from the Help menu.
If the browser displays a version lower than 1.1.4, or if you do not have a HotJava browser, follow the instructions to download the HotJava software version 1.1.4 (or later) from the following URL:
http://java.sun.com/products/hotjava/index.html
Run your HotJava browser on a cluster node.
The HotJava browser is located in /usr/dt/bin.
Remotely display the HotJava browser on an X windows workstation.
In the HotJava browser, set the applet security preferences:
When you are ready to begin monitoring the cluster with SCM, type the appropriate URL. For example:
file:/opt/SUNWcluster/scmgr/index.html |
Click OK on dialog boxes that ask for permission to access certain files, ports, and so forth.
HotJava takes some time to download and run the applet. No status information will appear during this time.
Refer to the online help for complete information about menu navigation, tasks, and reference.
Run your HotJava browser on a node in the cluster.
The HotJava browser is located in /usr/dt/bin.
Set up and start a web server on all cluser nodes.
For details, see "How to Set Up a Web Server to Run With SCM".
Set the applet security preferences in your HotJava browser:
When you are ready to begin monitoring the cluster with SCM, type the appropriate URL.
http://cluster_node/link_to_scm/index.html |
Click OK on dialog boxes that ask for permission to access certain files, ports, and so forth from the remote display workstation to the cluster node on which the browser is started.
HotJava takes some time to download and run the applet. No status information will appear during this time.
Refer to the online help for complete information about menu navigation, tasks, and reference.
Install Netscape on the cluster nodes.
Install SCM and the required SCM patch on the cluster nodes.
To install SCM, use scinstall(1M). The scinstall(1M) command installs the SCM package (SUNWscmgr) as part of the server package set. To get the SCM patch, see your service representative or the SunSolve web site:
http://sunsolve.sun.com/
Add the following lines to the preferences.js file, if necessary.
The file is located in the $HOME/.netscape directory. If the preferences are not included in the file already, add the following lines:
user_pref("security.lower_java_network_security_by_trusting_proxies", true); user_pref("signed.applets.codebase_principal_support", true); |
On a cluster node, set your DISPLAY environment variable so that the Netscape browser is displayed remotely on your X Windows workstation, and then run the Netscape browser on that cluster node.
When you are ready to begin monitoring the cluster with SCM, enter the appropriate URL.
file:/opt/SUNWcluster/scmgr/index.html |
Click Grant on Java Security dialog boxes that ask for permission to access certain files, ports, and so forth from the remote display workstation.
Refer to the online help for complete information about menu navigation, tasks, and reference.
If you choose, you can install a web server on the cluster nodes to run with SCM.
If you are running the Sun Cluster HA for Netscape HTTP service and an HTTP server on SCM, you must configure the HTTP servers to listen on different ports, to prevent a port conflict between the HTTP servers.
Install a web server on all nodes in the cluster.
Follow the web server's configuration procedure to make sure that SCM's index.html file is accessible to the clients.
The client applet for SCM is in the index.html file in the /opt/SUNWcluster/scmgr directory. For example, go to your HTTP server's document_root and create a link to the /opt/SUNWcluster/scmgr directory.
Run the HotJava browser from your workstation.
Set the applet security preferences in your HotJava browser:
When you are ready to begin monitoring the cluster with SCM, type the appropriate URL.
For example, if you had created a link from the web server's document_root directory to the /opt/SUNWcluster/scmgr directory, you would type the following URL:
http://cluster_node/scmgr/index.html |
Click OK on dialog boxes that ask for permission to access certain files, ports, and so forth on the cluster node on which the browser is started.
HotJava takes some time to download and run the applet. No status information will appear during this time.
Refer to the online help for complete information about menu navigation, tasks, and reference.
SCM provides online help information about menu navigation, tasks, and reference. This help is available whether you are running SCM as an application or an applet.
To display the Help window from SCM, select Help Contents from the Help menu. Alternatively, click on the Help icon (question mark) in the tool bar above the folder.
If necessary, you can run online help in a separate browser by typing the following URL:
file:/opt/SUNWcluster/scmgr/help/locale/en/main.howtotopics.html |
For example, if you had created a link from the web server's document_root directory to the /opt/SUNWcluster/scmgr directory, you would type the following URL:
http://clusternode/scmgr/help/locale/en/main.howtotopics.html |
When you finish viewing the online help, close its HotJava browswer. Selecting online help again brings up a new browser and loads the help.