NMS Monitor

The NMS Monitor runs on a WebLogic 14.1.2 cluster with managed servers at each NMS site. It periodically requests the status of the NMS Service Agents, WebLogic managed servers and databases. The NMS Monitor communicates with a HAMI Ensemble to:

• Manage and read the site configuration.

• Store the current site status.

• Coordinate with other NMS Monitor instances.

The NMS Monitor hosts a web page showing the status at each site and provides configuration to allow the addition and maintenance of sites to be monitored.

Monitoring

The NMS Monitor monitors the following components using REST API calls and stores the status of each in the HAMI ensemble:

• From each NMS Services Agent, the monitor retrieves the current and previous state of the NMS Services, timestamps when those states changed, the NMS version, and the site name.

• From the cesejb deployment, the monitor retrieves the status of the deployment’s connection to NMS Services and the NMS corbagateway publisher.

• From the nms-ws deployment, the monitor retrieves the deployment’s connection to cesejb. This is the deployment used by Flex Operations and OMA users.

• From the databases, the monitor retrieves the active and staged environments, active and staged versions, and the site name.

• From the WebLogic Admin Server, the monitor retrieves the status of each managed server and the applications that are deployed under it.

The NMS Monitor uses this information when determining:

• The overall status of the NMS instances at each site.

• Which standby instances are viable candidates for failover.

Configuration

The NMS Monitor is configured via the nmsmonitor.properties configuration file located in the $DOMAIN_HOME directory of the WebLogic instance. A configuration tool, nms‑monitor‑config is provided to help in the configuration of the monitor. nms‑monitor‑config is an interactive command line tool that is run on the WebLogic server where NMS Monitor runs and requires JDK 17. JDK 17 must be the java version in the PATH. The setDomainEnv.sh script for the WebLogic domain must be sourced first before running nms-monitor-config.

. $DOMAIN_HOME/bin/setDomainEnv.sh
cd monitor-config/bin
./nms-monitor-config

Configuration Steps

Copy monitor-config to WebLogic 14 Host

Copy the directory $NMS_BASE/dist/install/wls/monitor-config from the NMS installation to one of the WebLogic hosts that will run the NMS Monitor managed server.

Copy HAMI Client Wallet to WebLogic for NMS Monitor

Copy the HAMI client wallet file, cwallet.sso, to the WebLogic hosts that will run NMS Monitor. $DOMAIN_HOME/security is the recommended location.

Generate NMS Monitor Configuration

On the WebLogic host that monitor-config was copied to, source setDomainEnv.sh for the WebLogic domain and then run monitor-config/bin/nms-monitor-config to run the NMS Monitor configuration tool. Follow the on-screen prompts to configure HAMI, create the keystore and truststore, and configure credentials. Then export the NMS Monitor certificate to a file. This file will be used for configuring NMS Agent.

• Configure HAMI

• New connect string. Enter the connect string for the HAMI Ensemble. This is a comma-separated list of hostnames and ports where the nodes of the HAMI Ensemble run.

Example:

host1.example.com:5512,host2.example.com:5512,host3.example.com:5512

• Set namespace (optional). Set the namespace NMS Monitor uses for storage in the HAMI Ensemble. This defaults to “nmsmonitor” and only needs to change if more than one NMS Monitor cluster will be using the same HAMI Ensemble.

• Set wallet location. Set the location of the HAMI client wallet. Use the full path to the cwallet.sso file.

• Keystore actions

• Set keystore location. Set the full path of the keystore file that will be used for storing the NMS Monitor certificate.

• Create keystore. Create the keystore. You will be prompted for the password to use for the keystore and the details for creating the NMS Monitor certificate.

• Export certificate. Export the NMS Monitor certificate to a file. This file will need to be imported into the truststores of the NMS Agents.

• Truststore actions

• Set truststore location. Set the full path of the truststore file that will be used for storing NMS Agent public key certificates.

• Create truststore. Create the trust store. You will be prompted for a password to use for the trust store.

• Import certificate. Used to import an NMS Agent public key certificate into the trust store. You will be prompted for the truststore password, alias for the certificate, and filename of the certificate to import.

• Credentials

• Create new credential. Creates a credential used for authentication of REST service being monitored. You will be prompted for the hostname and port of the service and the username and password for authentication. Credentials must be entered for each WebLogic Admin Server where NMS deployments run, each cesejb deployment, and each nms‑ws deployment.

• Delete credential. Delete a credential that is already in nmsmonitor.properties.

Import NMS Agent Certificates

This step is run after configuring the NMS Agent. Copy the exported certificates from each of the NMS Agent instances to the WebLogic host where NMS Monitor is configured. Run nms-monitor-config and choose Import certificate under Truststore actions.

Copy NMS Monitor Files

Copy the files generated by nms-monitor-config to each of the hosts that will run NMS Monitor managed servers. Each managed server will reference these files locally.

• Copy nmsmonitor.properties to $DOMAIN_HOME.

• Copy cwallet.sso, nmsmon_keystore.p12, and nmsmon_truststore.p12 to the same paths referenced in nmsmonitor.properties.

Configure WebLogic for NMS Monitor

Configure NMS Monitor Security Roles

NMS Monitor uses two security roles in WebLogic

• NmsMonitorRead: Role with read-only permission to NMS Monitor. Users can login and view status and configuration but cannot change configuration or perform a failover.

• NmsMonitorWrite: Role with full permission to NMS Monitor.

Existing LDAP groups will be assigned these roles so that users in these groups can log in to NMS Monitor. Perform these steps in the WebLogic Remote Console or Admin Console for the domain that will run NMS Monitor.

1. Select Security Data Tree.

2. Select Realms and then select the default security realm.

3. Select Role Mappers and then the default role mapper.

4. Select Global and then Roles.

5. Click New

6. Enter the group name (NmsMonitorWrite), and then click Create.

7. Click Add Condition.

8. In the Predicate List, choose Group

9. In Group Argument Name, enter the name of the LDAP group for users with full permission to NMS Monitor and click OK.

10. Click Save.

11. Select Roles, and then repeat steps 5-10 to create the NmsMonitorRead role using the LDAP group for read-only NMS Monitor users.

Create WebLogic Cluster

Before You Begin

• Obtain private keys and digital certificates from a reputable certificate authority such as Verisign, Inc. or Entrust.net.

• Create identity and trust keystores.

• Load the private keys and trusted CAs into the keystores.

In the WebLogic console:

1. Click on the Edit Tree

2. Under Environment, choose Clusters.

3. Click New.

4. Enter a name for the cluster and choose Create.

5. Click on the Migration tab.

6. Set Migration Basis to “consensus”.

7. Click on Candidates and then add all the machines that will run NMS Monitor managed servers as candidates for migration.

8. Select Servers.

9. Click New.

10. Enter a name for the managed server, and then click Create.

11. Set Cluster to the cluster that was created for NMS Monitor.

12. Set Machine to the machine the managed server will run on.

13. Set Listen Address to an IP address or DNS name that resolves to an IP address of the server.

14. Change Listen Port and SSL Listen Port to available ports.

15. Select the Security tab.

16. In the Keystores field, select the method Custom Identity and Java Standard Trust for storing and managing private keys/digital certificate pairs and trusted CA certificates.

17. Set Custom Identity Key Store File Name to the fully qualified path of the identity keystore.

18. Set Custom Identity Key Store Type to JKS.

19. Set Custom Identity Key Store Pass Phrase to the password for the keystore.

20. Click Save.

21. Select the SSL tab.

22. Enter Server Private Key Alias and Server Private Key Pass Phrase and click Save.

23. Repeat steps 8-22 for each additional managed server that will run NMS Monitor. There should be at least one managed server running at the primary NMS site and at least one running at the Disaster Recover (DR) site.

24. In the Edit Tree, select Environment, then select Services, and then select Data Sources.

25. Click New

26. Enter a unique Name and JNDI Name for the data source.

27. For Targets, add the cluster created above.

28. Set Data Source Type to Generic Data Source.

29. Enter the JDBC connection details for connecting to the NMS Admin schema on the primary database.

30. Click Create.

31. Repeat steps 24-30 to create a JDBC Data Source connecting to the NMS Admin schema on the standby database associated with the NMS DR site.

32. Copy $NMS_BASE/dist/install/nms-monitor.ear from the NMS installation to the host that runs the WebLogic Admin Server.

33. In the Edit Tree, select Deployments, and then select App Deployments.

34. Click New.

35. Give the deployment a name.

36. For Targets, select the cluster created above.

37. Unset the Upload flag.

38. Set Source Path to the full path to nms-monitor.ear.

39. Set On Deployment to Start Application.

40. Click Create.

41. Click the shopping cart in the upper-right, and choose Commit Changes.

Start NMS Monitor

In the WebLogic console, start the managed servers for the NMS Monitor cluster.

Configure the Site Configuration within the NMS Monitor Web Application

Using the Listen address and SSL Listen Port for one of the NMS Monitor managed servers, go to https://hostname:port/NmsMonitor/ in a web browser.

Enter the User ID and Password, and then click Validate.

Add Site

1. In the Actions menu, choose Add Site….

2. From the Site Config dialog box, enter the configuration details for a site. URLs are of the form “https://hostname:port”. The blue environment is the NMS environment where NMS_ENVIRONMENT=1 and the green environment is the NMS environment where NMS_ENVIRONMENT=2.

• Site: The name of the site. This must match the NMS_SITE_NAME environment variable on the NMS server.

• JDBC: From the drop-down, select the JNDI Name of the JDBC Data Source associated with this site.

• WebLogic Admin Server: A comma-separated list of URLs of WebLogic Admin Servers for the domains where NMS runs.

• NMS Agent (Blue): The URL of the NMS Agent for the blue environment.

• NMS Agent (Green): The URL of the NMS Agent for the green environment.

• Web Gateway (Blue): A comma-separated list of the URLs of the WebLogic servers running the cesejb deployment for the blue environment.

• Web Gateway (Green): A comma-separated list of the URLs of the WebLogic servers running the cesejb deployment for the green environment.

• Mobile Gateway (Blue): A comma-separated list of the URLs of the WebLogic servers running the nms-ws deployment for the blue environment.

• Mobile Gateway (Green): A comma-separated list of the URLs of the WebLogic servers running the nms-ws deployment for the green environment.

3. Click the OK button in the Site Config dialog.

4. Perform the Add Site again – this time adding the details for the Disaster Recovery Site.

5. Site configuration can be updated by choosing Edit Site.