Configuring and Managing WebLogic Server

Overview of Node Manager

The Managed Servers in a production WebLogic Server environment are often distributed across multiple machines and geographic locations.

Node Manager is a Java utility that runs as separate process from WebLogic Server and allows you to perform common operations tasks for a Managed Server, regardless of its location with respect to its Administration Server. While use of Node Manager is optional, it provides valuable benefits if your WebLogic Server environment hosts applications with high availability requirements.

If you run Node Manager on a machine that hosts Managed Servers, you can start and stop the Managed Servers remotely using the Administration Console or from the command line. Node Manager can also automatically restart a Managed Server after an unexpected failure.

The following sections provide an overview of Node Manager.

• Node Manager Environment
• Node Manager Capabilities
• Node Manager Communications for Lifecycle Operations

To configure and use Node Manager, see Configuring, Starting, and Stopping Node Manager.

 

---

Node Manager Environment

These sections describe how Node Manager fits into your WebLogic Server environment.

Run Node Manager on Each Machine that Hosts Managed Servers

To take advantage of Node Manager capabilities, you must run a Node Manager process on each machine that hosts Managed Servers. You can manage multiple Managed Servers on a single machine with one Node Manager process—in Figure 4-1, the two Managed Servers on Machine C can be controlled by a single Node Manager process.

You cannot use Node Manager to start or stop an Administration Server. In a production environment, there is no need to run Node Manager on a machine that runs an Administration Server, unless that machine also runs Managed Servers. In a development environment, you may wish to run Node Manager on a machine that hosts an Administration Server and one or more Managed Servers, because doing so allows you to start the Managed Servers using the Administration Console.

Run Node Manager as an Operating System Service

The WebLogic Server installation process installs Node Manager to run as an operating system service: a daemon on UNIX machines, or a Windows service on Windows-based machines. An operating system service starts up automatically each time the operating system boots.

A key Node Manager feature is the ability to restart Managed Servers after a failure. If the failure is a machine crash, running Node Manager as a service ensures that Node Manager starts up automatically when the machine reboots, and is available to restart Managed Servers on that machine.

Node Manager is Domain-Independent

A Node Manager process is not associated with a specific WebLogic domain. Node Manager resides outside the scope of a domain, and you can use a single Node Manager process to start Managed Servers in any WebLogic Server domain that it can access—in Figure 4-1, Managed Server 2 and Managed Server 3 could be in separate domains, and controlled by a single Node Manager process. Node Manager is associated with Managed Servers using a Machine resource, as described in Configure a Machine to Use Node Manager.

Invoking Node Manager

You can invoke Node Manager's capabilities through the WebLogic Server Administration Console or JMX utilities, either the weblogic.Admin command-line utility or those you write yourself. For more information about JMX, see Programming WebLogic Management Services with JMX.

Typically, Node Manager is called by an Administration Server on a remote machine. However, Node Manager can also be called by a Administration Server on the same machine—allowing you to start co-resident Managed Servers using the Administration Console. When Node Manager is not running on a machine, the Start command on the Server—>Control tab page for Managed Servers on that machine is not enabled.

Node Manager Uses SSL

Node Manager communicates with Managed Servers and Administration Servers using two-way SSL to verify the identity of server instances that it communicates with. You cannot use Node Manager with an unsecured communication protocol.

The connection between Node Manager and a Managed Server is an always-open SSL connection. Node Manager uses this connection to send GETSTATE messages to the Managed Server. The Managed Server uses the connection to respond to each GETSTATE message.

The connection between Node Manager and an Administration Server is a short-lived request-response SSL connection.

Native Support for Node Manager

BEA provides native Node Manager libraries for Windows, Solaris, HP UX, Linux on Intel, Linux on Z-Series, and AIX operating systems. In addition, non-native Node Manager libraries are supported on these platforms.

For other UNIX and Linux operating Systems since support for only non-native Node Manager libraries is available, you must set the weblogic.nodemanager.nativeVersionEnabled option to false at the command line when starting Node Manager to use the pure Java version. For more information, see Node Manager Properties.

Native Node Manager is not supported on Open VMS, OS/390, AS400, UnixWare, or Tru64 UNIX.

 

---

Node Manager Capabilities

Node Manager enables you to perform these tasks:

• Start and stop remote Managed Servers.
• Monitor the self-reported health of Managed Servers and automatically kill server instances whose health state is "failed".
• Automatically restart Managed Servers that have the "failed" health state, or have shut down unexpectedly due to a system crash or reboot.

The following sections describe these capabilities in more detail.

Start Managed Servers

Requests from the Administration Console (or JMX utilities such as weblogic.Admin) to start a Managed Server using Node Manager are issued to the Administration Server for the domain that contains the Managed Server. The Administration Server dispatches the start command to the Node Manager process on the machine that hosts the target Managed Server. Node Manager executes the start command and creates a Managed Server process. If the Managed Server does not respond within 60 seconds, the Node Manager sets the state of the Managed Server to UNKNOWN. Node Manager does not retry the start command. If the Managed Server successfully starts and establishes a connection with Node Manager, the state of the Managed Server is updated appropriately.

Node Manager starts a Managed Server using the startup arguments configured for the Managed Server in the Server—>Configuration—>Remote Start tab. The Managed Server is started in the startup mode configured on the advanced options portion of its Server—>Configuration—>General tab. By default the startup mode is RUNNING.

Note: Node Manager uses the same command arguments that you supply when starting a Managed Server using a script or at the command line. For information about startup arguments, see weblogic.Server Command-Line Reference in WebLogic Server Command Reference.

If you do not specify startup arguments for a Managed Server in its Remote Start tab, Node Manager uses its own properties as defaults to start the Managed Server. (See Node Manager Properties.) Although the Node Manager property values may suffice to boot a Managed Server, to ensure a consistent and reliable boot process, you should configure startup arguments for each Managed Server.

Node Manager starts a Managed Server in a dedicated process on the target machine, separate from the Node Manager and Administration Server processes, in the same directory where the Node Manager process is running. To run the Managed Server in a different directory, set the Root Directory attribute in the Server—>Configuration—>Remote Start tab.

The messages that would otherwise be output to STDOUT or STDERROR when starting a Managed Server are instead displayed in the Administration Console and written to the Node Manager log file for that server instance. For more information, see Managed Server Log Files.

Suspend or Stop Managed Servers

Requests from the Administration Console (or JMX utilities such as weblogic.Admin) to stop or suspend a Managed Server using Node Manager are issued to the Administration Server for the domain that contains the Managed Server. The Administration Server dispatches the command directly to the target Managed Server. If the Administration Server cannot reach the target Managed Server, the command is dispatched to the Node Manager process running on the target Managed Server's machine. The Node Manager forwards the request to the target Managed Server for execution. If the Managed Server fails to respond to a shutdown request from the Node Manager, the Node Manager process itself performs the shutdown.

Shut Down Failed Managed Servers

Node Manager periodically checks the self-reported heath status of Managed Servers that it has started. (For information about how a Managed Server monitors its health, see Server Self-Health Monitoring.) By default, Node Manager issues a health query to a Managed Server every 180 seconds.

Node Manager automatically kills a Managed Server that reports its health state as "failed", if the Managed Server's Auto Kill If Failed attribute is true. By default, the Auto Kill If Failed attribute is false. If you want Node Manager to restart a Managed Server that is hung, set Auto Kill If Failed to true, on the Server—>Configuration—>Health Monitoring tab for the Managed Server.

If a Managed Server does not respond to three consecutive health queries in a row, Node Manager considers the Managed Server to be "failed", and shuts it down, if Auto Kill If Failed is set.

For instructions on controlling the frequency with which Node Manager checks the health state of a Managed Server, see Configure Monitoring, Shutdown, and Restart for Managed Servers.

Restart of Crashed and Failed Managed Servers

By default, Node Manager automatically restarts Managed Servers that crash, and Managed Servers that Node Manager killed because their health state was "failed".

By default, Node Manager restarts a Managed Server no more than twice within a one-hour period. You can configure how many times Node Manager will restart a Managed Servers it controls, and the period of time over which it will do the restarts. For instructions, see Configure Monitoring, Shutdown, and Restart for Managed Servers.

Note: If you stop a Node Manager process that is currently monitoring Managed Servers, do not shut down those Managed Servers while the Node Manager process is shut down. Node Manager will be unaware of shutdowns performed on Managed Servers while it was down. When Node Manager is restarted, if a Managed Server it was previously monitoring is not running, it will automatically restart it.

Prerequisites for Automatic Restart of Managed Servers

For Node Manager to restart failed Managed Servers, the behavior must be configured appropriately, as described in Configure Monitoring, Shutdown, and Restart for Managed Servers. In addition, the following prerequisites apply:

• A Node Manager process can automatically monitor, shutdown, and restart only those Managed Servers that it started. It cannot provide these services for Managed Servers booted directly from the command line.
• If the event that caused a Managed Server to fail also causes Node Manager to fail, the Node Manager process on the machine where the Managed Server runs must be restarted, either by the operating system or manually, in order for Node Manager to restart the failed Managed Server. For this reason, BEA recommends that you run Node Manager as an operating system service.
• The Administration Server for the domain that contains the Managed Servers must run as a windows service, or on Unix systems, a daemon. For instructions, see Setting Up a WebLogic Server Instance as a Windows Service in Configuring and Managing WebLogic Server.

 

---

Node Manager Communications for Lifecycle Operations

This section describes the communications between a Node Manager client, a Node Manager process, and the Managed Servers managed by the Node Manager process.

Figure 4-1 and Table 4-1 illustrate and describe the key communications that occur during Node Manager operation.

The interactions associated with specific lifecycle events are described in:

• Node Manager Communications to Start a Managed Server
• Node Manager Communications to Shut Down a Managed Server
• Node Manager Communications to Restart a Managed Server
• Node Manager Communications to Re-establish Communications After a Failure

Figure 4-1 Node Manager Communications




 

Table 4-1 Communications Between Node Manager Process, Node Manager Client, and Managed Servers

This component...	Has these interactions with a Node Manager process...	Description
Node Manager client in the Administration Server	start shutdown kill startMonitoring stopMonitoring getState getLogs getNMLogs pingNM	Node Manager functions for the Managed Servers on a machine are invoked by a Node Manager client, either the client in the Administration Server for the domain that contains the Managed Servers, or a JMX client. Communications from the Node Manager client to the Node Manager process on a machine include: lifecycle commands commands to determine the availability of the Node Manager process and the health state of the Managed Servers under Node Manager control requests for log files
Operating System	start	As a result of receiving a command to start a Managed Server from the Node Manager client, the Node Manager process issues a command to the operating system to create the Managed Server process.
Managed Servers	startMonitoring getState changeState shutdown	After the Managed Server process is created, and the Managed Server starts up, the Managed Server tells the Node Manager process that its state is started, and to commence monitoring. If the Managed Server determines that its health state has changed to failed, it informs the Node Manager process of the state change. Note: The Node Manager client sends shutdown commands directly to the target Managed Server. If the Managed Server does not respond, the shutdown command is sent to the Node Manager process that controls the Managed Server.

Node Manager Communications to Start a Managed Server

When the Node Manager process receives a command to start a Managed Server, it issues a request to the operating system to create the Managed Server process. When the Managed Server starts up, is tells the Node Manager process that it has started up, and to start monitoring its state.

If the Managed Server fails to connect back to the Node Manager process within the period of time specified by its ScavengerDelaySeconds attribute, the Node Manager process declares the state of the Managed Server to be UNKNOWN, and the task fails.

Figure 4-2 Starting a Managed Server with Node Manager

 

 

 

 

 

 

Node Manager Communications to Shut Down a Managed Server

Figure 4-3 illustrates the Node Manager communications involved in initiating a Managed Server shutdown. Depending on the state and availability of the Managed Server, Node Manager might need to try alternative strategies to successfully initiate the shutdown.

When an end-user issues chooses a shutdown action from the Administration Console, Node Manager proceeds in this fashion:

The Node Manager client in the Administration Server issues a shutDown directly to the Managed Server. If the Node Manager client successfully contacts the Managed Server, the Managed Server performs the shutdown sequence described in Graceful Shutdown. Otherwise, Node Manager tries the strategy described below in (2).
If the Managed Server is not reachable by the Node Manager client, the client issues the shutDown command to the Node Manager process on the machine that hosts the Managed Server. If the Managed Server is reachable by the Node Manager process, it acknowledges the command and performs the shutdown sequence described in Graceful Shutdown.Otherwise, Node Manager tries the strategy described below in (3).
If the Managed Server does not acknowledge the shutDown, the Node Manager issues a request to the operating system to kill the Node Manager process.

Figure 4-3 Shutting Down a Managed Server with Node Manager




 

 

Node Manager Communications to Restart a Managed Server

Figure 4-4 illustrates Node Manager communications involved in restarting a Managed Server whose state is FAILED.

Note: Node Manager only kills a Managed Server whose AutoKillIfFailed attribute is set to true.Node Manager only restarts a Managed Server whose AutoRestart attribute is set to true.

The Node Manager issues GETSTATE requests to the Managed Server periodically, as specified by the Managed Server's HealthCheckIntervalSeconds attribute.

If the Managed Server reports its health as FAILED or if it does not respond to the GETSTATE within ScavengerDelaySeconds, the Node Manager process issues a request to the operating system to kill the Managed Server process.

After RestartDelaySeconds, the Node Manager process issues a request to the operating system to create a new Managed Server process.

When the Managed Server starts up, it contacts the Node Manager, as described in Node Manager Communications to Start a Managed Server.

If the Managed Server does not contact the Node Manager process, the Node Manager process will keep trying to create the process up to the number of attempts specified by the Managed Server's RestartMax attribute. The attempts occur periodically, as specified by the Managed Server's RestartIntervalSeconds.

Figure 4-4 Restarting a Managed Server with Node Manager

 

 

 

Node Manager Communications to Re-establish Communications After a Failure

Figure 4-4 illustrates Node Manager communications that occur when a Node Manager process starts up after it failed while monitoring a Managed Server.

After starting up, the Node Manager issues a GETSTATE requests to the Managed Server.

If the Managed Server reports status of FAILED or does not respond within ScavengerDelaySeconds, the Node Manager process proceeds as described in Node Manager Communications to Restart a Managed Server.

Figure 4-5 Node Manager Process Re-establishing Communications After Failure