2 Node Manager Overview

The following sections provides an introduction to Node Manager:

Introduction

Server instances in a WebLogic Server production environment are often distributed across multiple domains, machines, and geographic locations. Node Manager is a WebLogic Server utility that enables you to start, shut down, and restart Administration Server and Managed Server instances from a remote location. Although Node Manager is optional, it is recommended if your WebLogic Server environment hosts applications with high availability requirements.

A Node Manager process is not associated with a specific WebLogic domain but with a machine. You can use the same Node Manager process to control server instances in any WebLogic Server domain, as long as the server instances reside on the same machine as the Node Manager process. Node Manager must run on each computer that hosts WebLogic Server instances—whether Administration Server or Managed Server—that you want to control with Node Manager.

Node Manager Versions

WebLogic Server provides two versions of Node Manager, Java-based and script-based, with similar functionality. However, each version has different configuration and security considerations.

Java-based Node Manager

Java-based Node Manager runs within a Java Virtual Machine (JVM) process. It is recommended that you run it as a Windows service on Windows platforms and as an operating system service on UNIX platforms, allowing it to restart automatically when the system is rebooted.

Oracle provides native Node Manager libraries for Windows, Solaris, HP UX, Linux on Intel, Linux on Z-Series, and AIX operating systems.

Note:

Node Manager is not supported on Open VMS, OS/390, AS400, UnixWare, or Tru64 UNIX.

This version of Node Manager determines its configuration from the nodemanager.properties file. See Reviewing nodemanager.properties.

Java-based Node Manager provides more security than the script-based version. See Configuring Java-based Node Manager Security.

Script-based Node Manager

For UNIX and Linux systems, WebLogic Server provides a script-based version of Node Manager. This script is based on UNIX shell scripts, but uses SSH for increased security. SSH uses user-id based security.

For information on configuring the script version of Node Manager, see Configuring Script Node Manager.

This version does not provide as much security as the Java-based version. However, the advantage of the script-based Node Manager is that it can remotely manage servers over a network that has been configured to use SSH. No additional server installation is required. The scripts merely have to be copied to the remote machine.

Note:

It is recommended that you run script-based Node Manager as an operating system service, which allows it to restart automatically when the system is rebooted.

Determining Which Node Manager Version to Use

Which version of Node Manager to use depends on the requirements of your WebLogic Server environment. The following considerations can help you decide which version is ideal for your environment:

  • If you are installing WebLogic Server on a Windows system, you must use the Java version of Node Manager. The scripted version of Node Manager is not supported on Windows.

  • In order to use consensus leasing, you may see faster performance when using the Java version of Node Manager.

  • The script-based Node Manager requires a much simpler security configuration than the Java version. RSH and SSH are generally easier to configure than SSL which is the security method used by the Java version of Node Manager. The script version of Node Manager also requires a smaller footprint than the Java version.

  • The Java version of Node Manager can be used in conjunction with inetd on supported UNIX systems. inetd allows Node Manager to be automatically restarted upon receiving a request on the configured port.

Accessing Node Manager

A Node Manager client can be local or remote to the Node Managers with which it communicates. You access either version of Node Manager—the Java version or the script-based (SSH) version—from the following clients: (In addition, an SSH client in the form of a shell command template is provided for use with the script-based Node Manager.)

  • Administration Server

    • Administration Console, from the Environments > Machines > Configuration > Node Manager page.

    For example, you can create JMX utilities that communicate with the Administration Server and perform operations on the ServerLifeCycleRuntimeMBean which in turn uses Node Manager internally to perform operations. For more information about JMX, see Developing Custom Management Utilities With JMX for Oracle WebLogic Server.

  • WLST commands and scripts—WLST offline serves as a Node Manager command-line interface that can run in the absence of a running Administration Server. You can use WLST commands to start, stop, and monitor a server instance without connecting to an Administration Server. Starting the Administration Server is the main purpose of the stand-alone client. However, you can also use WLST to:

    • Stop a server instance that was started by Node Manager.

    • Start a Managed Server.

    • Access the contents of a Node Manager log file.

    • Obtain server status for a server that was started with Node Manager.

    • Retrieve the contents of the server output log.

For more information on using WLST and Node Manager to control servers, see Using Node Manager to Control Servers.

What You Can Do with Node Manager

The following sections describe basic Node Manager functionality.

Start, Shut Down, and Restart an Administration Server

Using the WebLogic Scripting Tool (or SSH client for script-based Node Manager only), you connect to the Node Manager process on the machine that hosts the Administration Server and issue commands to start, shut down, or restart an Administration Server. The relationship of an Administration Server to Node Manager varies for different scenarios.

  • An Administration Server can be under Node Manager control—You can start it, monitor it, and restart it using Node Manager.

  • An Administration Server can be a Node Manager client—When you start or stop Managed Servers from the Administration Console, you are accessing Node Manager using the Administration Server.

  • An Administration Server supports the process of starting up a Managed Server with Node Manager—When you start a Managed Server with Node Manager, the Managed Server contacts the Administration Server to obtain outstanding configuration updates.

Start, Shut Down, Suspend, and Restart Managed Servers

From the WebLogic Server Scripting Tool (WLST) command line or scripts, you can issue commands to Node Manager to start, shut down, suspend, and restart Managed Server instances and clusters.

Node Manager can restart a Managed Server after failure even when the Administration Server is unavailable if Managed Server Independence (MSI) mode is enabled for that Managed Server instance. This is enabled by default.

Note:

Node Manager cannot start a Managed Server for the first time in MSI mode, because the Administration Server for the domain must be available so the Managed Server can obtain its configuration settings.

Note:

Node Manager uses the same command arguments that you supply when starting a Managed Server with a script or at the command line. For information about startup arguments, see "weblogic.Server Command-Line Reference" in Command Reference for Oracle WebLogic Server.

Restart Administration and Managed Servers

If a server instance that was started using Node Manager fails, Node Manager automatically restarts it.

Note:

Node Manager can only restart a server that was started using Node Manager.

The restart feature is configurable. Node Manager's default behavior is to:

  • Automatically restart server instances under its control that fail. You can disable this feature.

  • Restart failed server instances no more than a specific number of times. You define the number of restarts by setting the RestartMax property in the Node Manager startup.properties file.

If Node Manager fails or is explicitly shut down, upon restart, it determines the server instances that were under its control when it exited. Node Manager can restart any failed server instances as needed.

Note:

It is advisable to run Node Manager as an operating system service, so that it restarts automatically if its host machine is restarted.

Monitor Servers and View Log Data

Node Manager creates a log file for the Node Manager process and a log file of server output for each server instance it controls. You can view these log files, as well as log files for a server instance using the Administration Console or WLST commands.

How Node Manager Works in the WebLogic Server Environment

The following sections provide a "big picture" diagram of Node Manager's role in the WebLogic Server environment, as well as illustrations and descriptions of the processes Node Manager uses to communicate with servers:

Diagram of Node Manager and Servers

Figure 2-1 illustrates the relationship between Node Manager, its clients, and the server instances it controls.

Figure 2-1 Node Manager in the WebLogic Server Environment

Description of Figure 2-1 follows
Description of "Figure 2-1 Node Manager in the WebLogic Server Environment"

How Node Manager Starts an Administration Server

Figure 2-2 illustrates the process of starting an Administration Server with Node Manager.

This section assumes that you have installed the Administration Server and created its domain directory using the Configuration Wizard.

Node Manager is running on Machine A, which hosts the Administration Server. The stand-alone Node Manager client is remote.

Figure 2-2 Starting an Administration Server

Description of Figure 2-2 follows
Description of "Figure 2-2 Starting an Administration Server"

  1. An authorized user issues the WLST offline command nmConnect to connect to the Node Manager process on the machine that hosts the Administration Server. (If the Node Manager instance is the SSH version, the user can connect using the SSH client). The nmConnect command provides the Node Manager username and password that are used to authenticate the user with the Node Manager.

    Then, the user issues the nmStart command and provides the credentials for starting the Administration Server. For example:

    prps = makePropertiesObject("username=weblogic, password=welcome1")
    nmStart("AdminServer",props=prps) 
    

    Note:

    If the user has previously connected to the Node Manager, a boot.properties file exists, and the user does not have to supply username and password.

    The nmStart command identifies the domain and server instance to start.

  2. Node Manager looks up the domain directory in nodemanager.domains, and authenticates the user credentials using a local file that contains the encrypted username and password.

  3. Node Manager obtains the startup properties for the Administration Server.

  4. Node Manager creates the Administration Server process.

  5. The Administration Server obtains the domain configuration from its config directory.

How Node Manager Starts a Managed Server

Figure 2-3 illustrates the process of starting a Managed Server with Node Manager.

Node Manager is running on Machine B, which hosts Managed Server 1. The Administration Server for the domain is running on Machine A.

Figure 2-3 Starting a Managed Server

Description of Figure 2-3 follows
Description of "Figure 2-3 Starting a Managed Server"

  1. From the Administration Console, the user issues a start command for Managed Server 1.

    Note:

    A stand-alone client can also issue a start command for a Managed Server.
  2. The Administration Server issues a start command for Managed Server 1 to the Node Manager on the Machine B, providing the remote start properties configured for Managed Server 1. For information about the arguments and how to specify them, see Step 5: Configuring Remote Startup Arguments.

  3. Node Manager starts Managed Server 1.

    Node Manager starts the Managed Server using the same root directory where the Node Manager process is running. To run the Managed Server in a different directory, set the Root Directory attribute in the Server > Configuration > Server Start Administration Console page.

  4. Managed Server 1 contacts the Administration Server to check for updates to its configuration information.

  5. If there are outstanding changes to the domain configuration, Managed Server 1 updates its local cache of configuration data.

How Node Manager Restarts an Administration Server

Figure 2-4 illustrates the process of restarting an Administration Server with Node Manager.

Node Manager is running on the machine that hosts the Administration Server. The Administration Server, which was initially started with Node Manager, has exited. The Administration Server's AutoRestart attribute is set to true.

Note:

If a server instance's AutoRestart attribute is set to false, Node Manager will not restart it.

Figure 2-4 Restarting an Administration Server

Description of Figure 2-4 follows
Description of "Figure 2-4 Restarting an Administration Server"

  1. Node Manager determines from the Administration Server process exit code that it requires restart.

  2. Node Manager obtains the username and password for starting the Administration Server from the boot.properties file, and the server startup properties from the server_name/data/nodemanager/startup.properties file.

  3. Node Manager starts the Administration Server.

  4. The Administration Server reads its configuration data and starts up.

How Node Manager Restarts a Managed Server

Figure 2-5 illustrates process of restarting a Managed Server with Node Manager.

Node Manager is running on Machine B, which hosts Managed Server 1. Managed Server 1, which was initially started with Node Manager, has exited. Managed Server 1's AutoRestart attribute is set to true.

Note:

If a server instance's AutoRestart attribute is set to false, Node Manager will not restart it.

Figure 2-5 Restarting a Managed Server

Description of Figure 2-5 follows
Description of "Figure 2-5 Restarting a Managed Server"

  1. Node Manager determines from Managed Server 1's last known state that it requires restarting.

  2. Node Manager obtains the username and password for starting Managed Server 1 from the boot.properties file, and the server startup properties from the startup.properties file. These server-specific files are located in the server directory for Managed Server 1.

  3. Node Manager starts Managed Server 1.

    Note:

    Node Manager waits RestartDelaySeconds after a server instances fails before attempting to restart it.
  4. Managed Server 1 attempts to contact the Administration Server to check for updates to its configuration data. If it contacts the Administration Server and obtains updated configuration data, it updates its local cache of the config directory.

  5. If Managed Server 1 fails to contact the Administration Server, and if Managed Server Independence mode (MSI) is enabled, Managed Server 1 uses its locally cached configuration data.

    Note:

    Managed Server Independence mode is enabled by default.

How Node Manager Shuts Down a Server Instance

Figure 2-6 illustrates the communications involved in shutting down a Managed Server that is under Node Manager control. Depending on the state and availability of the Managed Server, Node Manager might need to try alternative strategies to successfully initiate the shutdown.

Node Manager is running on Machine B, which hosts Managed Server 1.

Figure 2-6 Shutting Down a Server Instance Under Node Manager Control

Description of Figure 2-6 follows
Description of "Figure 2-6 Shutting Down a Server Instance Under Node Manager Control"

  1. Through the Administration Console, an authorized user issues a shutdown command for Managed Server 1.

  2. The Administration Server issues the shutdown command directly to Managed Server 1. If it successfully contacts Managed Server 1, Managed Server 1 performs the shutdown sequence described in "Graceful Shutdown" in Managing Server Startup and Shutdown for Oracle WebLogic Server.

  3. If, in the previous step, the Administration Server failed to contact Managed Server 1, it issues a shutdown command for Managed Server 1 to Node Manager on Machine B.

  4. Node Manager issues a request to the operating system to kill Managed Server 1.

  5. The operating system ends the Managed Server 1 process.

Node Manager and System Crash Recovery

To ensure that Node Manager properly restarts servers after a system crash, you must perform the following:

  • For Java-based Node Manager, ensure that CrashRecoveryEnabled is set to true.

    The CrashRecoveryEnabled configuration property allows Node Manager to restart servers after a system crash. The property is not enabled by default.

  • For script-based Node Manager, place this line in machine start scripts or, if desired, run periodically on a given schedule:

    wlscontrol.sh -d domain_name CRASHRECOVERY
    
  • You should start the Administration Server using Node Manager.

  • All Managed Servers should be started using the Administration Server. You can accomplish this using WLST or the Administration Console.

After the system is restarted, Node Manager checks each managed domain specified in the nodemanager.domains file to determine if there are any server instances that were not cleanly shutdown. This is determined by the presence of any lock files which are created by Node Manager when a WebLogic Server process is created. This lock file contains the process identifier for WebLogic Server startup script. If the lock file exists, but the process ID is not running, Node Manager will attempt to automatically restart the server.

If the process is running, Node Manager performs an additional check to access the management servlet running in the process to verify that the process corresponding to the process ID is a WebLogic Server instance.

Note:

When Node Manager performs a check to access the management servlet, an alert may appear in the server log regarding improper credentials.

Node Manager Configuration and Log Files

In managing multiple servers, Node Manager uses multiple configuration files and outputs log files to multiple directories, as shown in Figure 2-7.

Figure 2-7 Node Manager Configuration and Logging Environment

Description of Figure 2-7 follows
Description of "Figure 2-7 Node Manager Configuration and Logging Environment"

The following sections describe Node Manager configuration and log files:

Configuration Files

Except where noted, configuration files apply to both Java-based and script-based Node Manager.

nodemanager.properties

This is the configuration file used by the Java-based version of Node Manager. See Reviewing nodemanager.properties.

This file is located in WL_HOME/common/nodemanager, where WL_HOME is the location in which you installed WebLogic Server.

nodemanager.domains

This file contains mappings between the names of domains managed by Node Manager and their corresponding directories. See Step 4: Configuring nodemanager.domains File.

This file is located in WL_HOME/common/nodemanager.

nm_data.properties

This file stores the encryption data the Node Manager uses as a symmetric encryption key. The data is stored in encrypted form.

This file is located in WL_HOME/common/nodemanager.

nm_password.properties

This file stores the Node Manager username and password. See Step 2: Specify Node Manager Username and Password.

This file is located in DOMAIN_HOME/config/nodemanager.

boot.properties

Node Manager uses this file to specify user credentials when starting a server. See General Node Manager Configuration.

This file is located in DOMAIN_HOME/servers/server_name/data/nodemanager.

startup.properties

Each Managed Server instance has its own startup.properties file with properties that control how Node Manager starts up and controls the server. Node Manager automatically creates this file by using properties passed to Node Manager when the Administration Server was last used to start the server. This allows a Node Manager client or startup scripts to restart a Managed Server using the same properties last used by the Administration Server.

For more information on startup.properties, see Step 6: Setting Server Startup Properties. These properties correspond to the server startup attributes contained in ServerStartMBean and the health monitoring attributes in ServerStartMBean.

This file is located in DOMAIN_HOME/servers/server_name/data/nodemanager.

server_name.addr

server_name.addr stores the IP address added when a server starts or is migrated. This file is generated after the server IP address is successfully brought online during migration. server_name.addr is deleted when the IP address is brought offline. The server IP address is used to validate remove requests to prevent addresses being erroneously removed while shutting down the server.

This file is located in DOMAIN_HOME/servers/server_name/data/nodemanager.

server_name.lck

server_name.lck is generated by each server and contains an internally used lock ID.

This file is located in DOMAIN_HOME/servers/server_name/data/nodemanager.

server_name.pid

server_name.pid is generated by each server and contains the process ID of the server. Node Manager checks the process ID generated by the server during crash recovery.

This file is located in DOMAIN_HOME/servers/server_name/data/nodemanager.

server_name.state

server_name.state is generated by the server and contains the server's current state. Node Manager monitors the contents of this file to determine the current state of the server.

Note:

Do not delete or alter this file. Without this file Node Manager cannot determine the current state of the server.

This file is located in DOMAIN_HOME/servers/server_name/data/nodemanager.

Log Files

Use the Node Manager and WebLogic Server log files to help troubleshoot problems in starting or stopping individual Managed Servers.

Table 2-1 Node Manager Log File Locations

Log File Location

Node Manager Log File

For Java-based Node Manager only, NodeManagerHome/nodemanager.log, where NodeManagerHome is WL_HOME/common/nodemanager

Node Manager Server Instance Log Files

DOMAIN_HOME/servers/server_name/logs/server_name.out, where DOMAIN_HOME is the location in which you installed your WebLogic Server domain, such as d:\oracle\user_projects\domains\domain-name

WebLogic Server Log Files

DOMAIN_HOME/servers/server_name/logs/server_name.log


nodemanager.log

Node Manager creates a log file located in NodeManagerHome/nodemanager.log. This log file stores data about all of the domains administered by Node Manager.

nodemanager.log is created for Java-based Node Manager only; it is not created for script-based Node Manager. This log file is generated by Node Manager and contains data for all domains that are controlled by Node Manager on a given physical machine. The file is located in WL_HOME/common/nodemanager.

Log output is appended to the current nodemanager.log. Log rotation is disabled by default, but can be enabled by setting LogCount in nodemanager.properties.

You can view the Node Manager log file by:

  • Selecting Machines > Monitoring > Node Manager Log page in the Administration Console

  • Using the WLST nmLog command

server_name.out

For each server instance that it controls, Node Manager maintains a log file that contains stdout and stderr messages generated by the server instance. If the remote start debug property is enabled as a remote start property for the server instance, or if the Node Manager debug property is enabled, Node Manager will include additional debut information in the server output log information.

Note:

You cannot limit the size of the log files Node Manager creates. Logging to stdout is disabled by default.

This file is located in domain_name/servers/server_name/logs.

Node Manager creates the server output log for a server instance in the server instance's logs directory, with the name:

server_name.out

where server_name is the name of the server instance.

You can view the Node Manager log file for a particular server instance by:

  • Selecting Diagnostics > Log Files.

  • Using the WLST nmServerLog command.

There is no limit to the number of server output logs that Node Manager can create.

WebLogic Server Log Files

A server instance under Node Manager control has its own log file, in addition to the log file created by Node Manager.

You can view the log file for a server instance by selecting Diagnostics > Log Files selecting the server log file, and clicking View.