Configuring Script-Based Node Manager

Overview

Script-based Node Manager is a shell script, wlscontrol.sh, located in WL_HOME/common/bin/. The wlscontrol.sh file must exist on each machine that hosts server instances that you want to control with Node Manager. This script can be customized to meet site-specific requirements.

You must have an SSH client executable on each machine where Node Manager or a Node Manager client runs. This script must also be in the path of the user ID running it. Typically, an SSH client is a standard part of a UNIX or Linux installation.

Step 1: Create User Accounts

Before running Node Manager, create a dedicated UNIX user account for performing Node Manager functions. Add this user to all machines that host the script-based Node Manager and to all machines that host a Node Manager client, including the Administration Server.

Note:

On UNIX platforms, Oracle does not recommend running Node Manager as the root user. However, to achieve Post-Bind GID, you must start Node Manager as the root user. Post-Bind GID enables a server running on your machine to bind to a UNIX group ID (GID) after it finishes all privileged startup actions.

For example:

On each host machine, as the root user, create two new operating system (OS) users: bea and ndmgr, both associated with a new group called bea.
- Use bea for installing Oracle WebLogic Server only.
- Use ndmgr to create a WebLogic domain and start the Administration Server and remote Managed Servers using Node Manager.
Both OS users should have the same OS group (bea) to ensure that the correct permissions are in place for ndmgr to run WebLogic scripts and executables.

For example:
```
> groupadd bea
> useradd -g bea -m bea
> passwd bea
> useradd -g bea -m ndmgr
> passwd ndmgr
```

Step 2: Configure Node Manager Security

The Node Manager SSH shell script relies on SSH user-based security to provide a secure trust relationship between users on different machines. You create a UNIX user account typically one per domain, for running Node Manager commands and scripts. A user logged in as this user can issue Node Manager commands without providing a user name and password.

Note:

You must also ensure that the Node Manager and Oracle WebLogic Server commands are available in the path of the UNIX user ID used to run them. Change the environment file of the user to contain the path to WL_HOME/common/bin/ or DOMAIN_HOME/bin/server_migration.

For example:

PATH=/usr/bin:/bin:ORACLE_HOME/user_projects/domains/domain_name/bin/server_migration

Configure SSH trust between the ndmgr user on each machine that will run an Oracle WebLogic Server instance and the same ndmgr user on the same machine, plus the corresponding ndmgr user on every other machine.

In other words, any ndmgr user on one machine must be able to establish an SSH session without being prompted for security credentials, with a ndmgr user of the same name on the same or a different machine. This is necessary because any Managed Server can become the cluster master for migratable servers and issue commands to start other remote Managed Servers in the cluster using SSH. For Managed Server migration to work, the ndmgr user needs only to be able to run the wlscontrol.sh script using SSH. See Configuring Security for WebLogic Server Scripts.

For example, to configure one instance of a user to trust another instance of a user for SSH version2:

From a terminal logged in as ndmgr user:
```
> ssh-keygen -t dsa
```
When prompted, accept the default locations and press Enter for passphrase so that no passphrase is specified.
Copy the ndmgr user's public key to the ndmgr user's home on the same machine and all other machines.
```
> scp .ssh/id_dsa.pub ndmgr@192.168.1.101:./
```

Establish an SSH session with the target machine as the ndmgr user and set up trust for the remote ndmgr user.

> ssh -l ndmgr 192.168.1.101 (you should be prompted for password)
> mkdir .ssh
> chmod 700 .ssh
> touch .ssh/authorized_keys
> chmod 700 .ssh/authorized_keys
> cat id_dsa.pub >> .ssh/authorized_keys
> rm id_dsa.pub
> exit

Test that you can establish an SSH session with the ndmgr user on the remote machine without requiring a password.
```
> ssh -l ndmgr 192.168.1.101
```
Repeat this process for all combinations of machines.

Alternatively, you can achieve the same result by generating a key value pair on each machine, concatenating all of the public keys into an authorized_keys file, and copying (scp) that file to all machines. Try establishing SSH sessions between all combinations of machines to ensure that the ~/.ssh/known_hosts files are correctly configured. See Generating and Distributing Key Value Pairs.

Step 3: Install Oracle WebLogic Server

Install an Oracle WebLogic Server instance in the base directory, ORACLE_HOME/wlserver, on all the machines that will run Oracle WebLogic Server, where ORACLE_HOME represents the directory you specify as the Oracle Home when you install Oracle WebLogic Server, for example, C:\Oracle\Middleware\Oracle_Home.

For example:

> java -jar wls_jrf_generic.jar

See Starting the Installation Program in Installing and Configuring Oracle WebLogic Server and Coherence.

Step 4: Create a WebLogic Domain

Create a WebLogic domain using the Configuration Wizard on the machine which hosts the Administration Server only.

Subsequently, when you start the Administration Server, it will use the configuration in the config subdirectory of this domain directory to determine the settings for the Administration Server and the domain.

It is likely that most Managed Server instances will be run remotely with respect to the Administration Server. Therefore, these Managed Servers will not have direct access to the domain configuration directory of the Administration Server. Instead, they will run from a skeleton domain directory in their respective machine's ndmgr home directory and will obtain their configuration over the network on startup from the remotely running Administration Server.

As the ndmgr user, create the WebLogic domain.

For example:

Run the Configuration Wizard:
```
> ORACLE_HOME/wlserver/common/bin/config.sh
```
Create a new WebLogic domain based on the default Oracle WebLogic Server template.
For the Administration Server, specify a fixed IP address (for example, 192.168.1.100).
In Customize Environment and Service Settings, select Yes.
In Configure Managed Servers, add two Managed Servers, MS1 and MS2.
For the Managed Servers, specify floating IP addresses (for example, 192.168.1.201 and 192.168.1.202).
In Configure Clusters, add a cluster, CLUST, and then assign MS1 and MS2 to it.
Do not specify any Machines or UNIX Machines; you will do this manually in a subsequent step.
Name the domain 'clustdomain' and save it to ORACLE_HOME/clustdomain.

Step 5: Configure nodemanager.domains File

The nodemanager.domains file specifies the domains that a Node Manager instance controls. The standalone clients do not need to specify the domain directory explicitly. The file nodemanager.domains provides additional security by restricting Node Manager client access to the domains listed in this file. The client can only execute commands for the domains listed in nodemanager.domains.

This file must contain an entry specifying the domain directory for each domain a Node Manager instance controls, in this form:

domain-name=domain-directory

When a user issues a command for a domain, Node Manager looks up the domain directory from nodemanager.domains.

For the script-based Node Manager, this file's default location is WL_HOME/common/nodemanager, where WL_HOME is the location in which you installed Oracle WebLogic Server, for example, ORACLE_HOME/wlserver.

If you are using the script-based implementation of Node Manager, you must create or copy into NodeManagerHome, a nodemanager.domains file that specifies the domains that you want a Node Manager instance to control. Alternatively, you can register WebLogic domains with the script-based Node Manager using the WLST command, nmEnroll. See Configuring nodemanager.domains File.

Step 6: Start the Administration Server

Aside from the standard method of starting the Administration Server by using the provided startWebLogic scripts, use the script-based implementation of Node Manager to monitor the Administration Server to ensure high availability. Run the wlscontrol.sh Node Manager script to start the Administration Server locally.

If using a production domain, create boot.properties for the Administration Server for successful server startup. As the ndmgr user, start the Administration Server using the wlscontrol.sh Node Manager script.

For example:

> ORACLE_HOME/wlserver/common/bin/wlscontrol.sh -d clustdomain -r
 ORACLE_HOME/clustdomain -c -f startWebLogic.sh -s AdminServer START

For verbose logging to standard out, add the -v parameter to the command.

Note:

Once the Administration Server has been successfully started, and the boot.properties file has been created, you can start the Administration Server remotely. Stop the Administration Server using the stopWebLogic.sh Node Manager script. Then, start the Administration Server remotely using SSH.

For example:

> ssh -l ndmgr -o PasswordAuthentication=no 192.168.1.100
 ORACLE_HOME/wlserver/common/bin/wlscontrol.sh -d clustdomain -r /home/ndmgr/clustdomain -c -f startWebLogic.sh -s AdminServer START

Step 7: Configure Node Manager on the Managed Servers

Each machine that will host a Managed Server will have a skeleton domain created and configured.

From a local terminal, create a new empty directory (clustdomain) in the home directory for the ndmgr user for each of the Managed Server host machines and also a back-up machine. For example:

> mkdir clustdomain
For each of the Managed Server host machines and the back-up machine, as the ndmgr user, use WLST to enroll the user's home directory as being the base directory for remotely run servers and for Node Manager.

For example:
```
> ORACLE_HOME/wlserver/common/bin/wlst.sh
> connect('weblogic','weblogic','t3://192.168.1.100:7001')
> nmEnroll('/home/ndmgr/clustdomain','/home/ndmgr')
> exit()
```
Be sure to run nmEnroll on each remote machine. This command creates a property file, /home/ndmgr/nodemanager.domains, which maps domain names to home directories, and creates the required domain configuration and security information so that Managed Servers can communicate with the Administration Server.

Note:
In previous releases, nmEnroll would download the domain encryption key (SerializedSystemIni.dat file) from the Administration Server. However, now, if it does not already exist, you must manually copy the file from the Administration Server domain directory or an existing server in the domain before running the nmEnroll command. The file location should be ${DOMAIN_HOME}/security/SerializedSystemIni.dat.

The nodemanager.domains file removes the need to specify the domain home directory (with -r) when starting wlscontrol.sh. However, since you changed the Node Manager home directory, you must specify -n /home/ndmgr. The default Node Manager home directory is ORACLE_HOME/wlserver/common/nodemanager; you might not want to use this directory as it is in the product installation directory and owned by another user.

Note:

By default, you can start a Node Manager from any directory. A warning will be issued if no nodemanager.domains file is found. You must create or copy into NodeManagerHome, a nodemanager.domains file that specifies the domains that you want a Node Manager instance to control or register WebLogic domains using the WLST command, nmEnroll.

Step 8: Test Node Manager Setup and Configuration by Starting Managed Servers

Start the Managed Servers to test the Node Manager setup and configuration.

Copy the scripts from the Administration Server's domain bin directory to the corresponding domain bin directory on each Node Manager machine (for example, /home/ndmgr/bin). For example:
```
> scp ndmgr@192.168.1.100:"~/clustdomain/bin/*" 
ndmgr@192.168.1.101:~/clustdomain/bin
```
For each Node Manager machine (including the back-up machine), edit the shell scripts in the bin directory to reflect the proper path for the local domain home, and the remote Administration Server's IP address.
For example:
1. Edit the DOMAIN_HOME and LONG_DOMAIN_HOME variables in the setDomainEnv.sh script to correctly reflect this remote domain home directory:
  - DOMAIN_HOME=/home/ndmgr/clustdomain
  - LONG_DOMAIN_HOME=/home/ndmgr/clustdomain
2. Similarly, edit the DOMAIN_HOME variable in startWebLogic.sh.
3. Edit the DOMAIN_HOME and ADMIN_URL (for example, t3://192.168.1.100:7001) variables in startManagedWebLogic.sh.
For each of the Managed Server host machines (including the back-up machine), as the ndmgr user, create a server/security subdirectory in the domain directory.
For example, for the Managed Server MS1:
```
> mkdir -p ~/clustdomain/servers/MS1/security
```
Note:
For the back-up machine, create a server directory for every migratable Managed Server (for example, both MS1 and MS2).
Create a new boot.properties file with the appropriate user name and password variables specified in each Managed Server's security directory (for example, /home/ndmgr/clustdomain/servers/MS1/security).
For example:
```
username=weblogic
password=password
```
Note:
When a Managed Server is first started using the script-based Node Manager, the values in this file will be encrypted.
For each of the Managed Server machines, as the ndmgr user, start the Managed Server locally from a terminal using the wlscontrol.sh Node Manager script.
For example, to start the Managed Server, MS1:
```
> ORACLE_HOME/wlserver/common/bin/wlscontrol.sh -d clustdomain -n /home/ndmgr -c -f startManagedWebLogic.sh -s MS1 START
```
For verbose logging to standard out, add the -v parameter to the command.

Once successfully started, stop the Managed Servers and then, as the ndmgr user, attempt to start the Managed Servers remotely using SSH.

For example to start MS1:

> ssh -l ndmgr -o PasswordAuthentication=no -p 22 192.168.1.101 ORACLE_HOME/wlserver/common/bin/wlscontrol.sh -d clustdomain -n /home/ndmgr -c -f startManagedWebLogic.sh -s MS1 START

Once successfully started, stop the Managed Servers again and then repeat the process by trying to start each Managed Server (MS1) on the back-up machine instead. Again, stop this server once it successfully starts.

Step 9: Configure UNIX Machines

Use the WebLogic Server Administration Console to configure a new UNIX Machine for each machine which hosts an Administration or Managed Server (including the back-up machine).

By using a UNIX machine, you can configure the machine to run all WebLogic Server instances under a privileged user ID (UID) or a group ID (GID). The processes are assigned (bound) to the selected UID or GID after the computer has carried out all the privileged startup actions.

For more information about configuring UID or GID, see Unix Machine Configuration in the Oracle WebLogic Server Administration Console Online Help.

Table 4-1 UNIX Machine Settings

Property	Value
O.S. Type	UNIX
Node Manager Type	`SSH`
Node Manager Listen Address	<`primary-ip-address`> (not floating IP address)
Node Manager Listen Port	`22`
Node Manager Home	`/home/ndmgr`
Node Manager Shell Command	`ssh -l ndmgr -o PasswordAuthentication=no -p %P %H ORACLE_HOME/wlserver/common/bin/wlscontrol.sh -d %D -n /home/ndmgr -c -f startManagedWebLogic.sh -s %S %C`
Node Manager Debug Enabled	`true`
Servers	<`name of the Administration or Managed Servers hosted on machine`>

Step 10: Assign Servers to Machines

After all of the UNIX Machines are created, use the WebLogic Server Administration Console to set the Machine property for each server, to ensure each server is associated with its corresponding UNIX Machine.

See Assign server instances to machines in the Oracle WebLogic Server Administration Console Online Help.

Step 11: Start Managed Servers

Use the WebLogic Server Administration Console to start the Managed Servers.

See Start Managed Servers from the Administration Console in the Oracle WebLogic Server Administration Console Online Help.

Check the server logs in the /home/ndmgr/clustdomain/servers/managed_server_name/logs directory of each Managed Server to ensure that the server has started with no errors.

Configuring Script-Based Node Manager Security

To configure the script-based Node Manager security, perform the steps to override the default SSH port, configure the security for Oracle WebLogic Server scripts, configure remote server start security, and generate and distribute key value pairs.

This section includes the following topics:

Overriding the Default SSH Port

The default SSH port used by Node Manager is 22. You can override that setting in the following ways:

Set the Port= parameter in the ~/.ssh/config file to set the default port for an individual user.
Set the Port= parameter in the /etc/ssh_config file to set the default port across the entire system.
Start the Administration Server using the following system property:
```
-Dweblogic.nodemanager.ShellCommand="ssh -o PasswordAuthentication=no -p %P
 %H
 wlscontrol.sh -d %D -r %R -s %S %C"
```
After starting the server, you can edit the SSH port in the Administration Server's configuration file.

Configuring Security for Oracle WebLogic Server Scripts

To perform server migration and other tasks, the user ID executing scripts such as wlscontrol.sh must have sufficient security permissions. This includes being able to bring an IP address online or take an IP address offline using a network interface.

Server migration is performed by the cluster master when it detects that a server has failed. It then uses SSH to launch a script on the target machine to begin the migration. The script on the target machine runs as the same user ID running the server on the cluster master.

The commands required to perform server migration are wlsifconfig and arping. Since these scripts require elevated OS privileges, it is important to note that this can prevent a potential security hole. Using sudo, you can configure SSH to allow only wlsifconfig and arping to be run using elevated privileges.

The scripts are located in the WL_HOME/common/bin/ directory or the DOMAIN_HOME/bin/server_migration directory. See Step 2: Configure Node Manager Security.

Configuring Remote Server Start Security for Script-Based Node Manager

A remote start user name and password is required to start a server instance with Node Manager. These credentials are provided differently for Administration Servers and Managed Servers.

Credentials for Managed Servers - When you invoke Node Manager to start a Managed Server, it obtains its remote start name and password from the Administration Server.
Credentials for Administration Servers - When you invoke Node Manager to start an Administration Server, the remote start user name can be provided on the command line, or obtained from the Administration Server's boot.properties file. The Configuration Wizard initializes the boot.properties file and the startup.properties file for an Administration Server when you create the domain.

Any server instance started by Node Manager encrypts and saves the credentials with which it started in a server-specific boot.properties file, for use in automatic restarts.

Generating and Distributing Key Value Pairs

The script-based Node Manager uses two types of key value pairs.

This section contains instructions for distributing key value pairs to the machines that will host a Node Manager client or server.

Shared Key Value Pair

This option distributes the same key value pair to all machines that will host a Node Manager client or server.

The simplest way to accomplish this is to set up your LAN to mount the Node Manager user home directory on each of the machines. This makes the key value pair available to the machines. Otherwise:

Generate an RSA key value pair for the user with the ssh-keygen command provided with the SSH installation.
The default location for the private and public keys are ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub, respectively.

If these keys are stored in a different location, modify the ShellCommand template, adding an option to the ssh command to specify the location of the keys.
Append the public key to the ~/.ssh/authorized_keys file on the Node Manager machine.
For example:
```
command="ORACLE_HOME/wlserver/common/nodemanager/nodemanager.sh" 1024 33 23...2323
```
In this command, you will substitute the public key that you generated, as stored in id_rsa.pub, for the string shown in the example as:

1024 33 23...2323

Note:
The prefix command=command ensures that a user that establishes a session with the machine using the public key can run only the command specified—nodemanager.sh. This ensures that the user can perform only Node Manager functions, and thus prevents unauthorized access to data, system utilities, or other resources on the machine.
Manually distribute the key value pair to each machine that will host a Node Manager server instance or client.
Execute the following command on the client machine to check that the Node Manager client can access Node Manager:
```
$ ssh montgomery wlscontrol.sh VERSION
```
This response indicates that the client accessed Node Manager successfully:
```
NodeManager version 12.1
```

Individual Key Value Pairs

On each machine that will host a Node Manager client:

Generate a separate RSA key value pair for the Node Manager user as described in step one in the previous section. See Shared Key Value Pair.
Append the public key to the machine's ~/.ssh/authorized_keys file user as described in step two in the previous section. See Shared Key Value Pair.