Starting and Stopping Your Server Instance
Configuring the Server Instance
Configuring the Proxy Components
Configuring Security Between Clients and Servers
Configuring Security Between the Proxy and the Data Source
Configuring Servers With the Control Panel
Monitoring Sun OpenDS Standard Edition
Configuring Logs With dsconfig
Configuring Log Retention Policies
To Create a Log Retention Policy
To Modify a Log Retention Policy
Configuring Log Rotation Policies
To Create a Log Rotation Policy
Logging Access Control Information
Differences Between Logging in Sun OpenDS Standard Edition and Sun Java System Directory Server
Configuring Alerts and Account Status Notification Handlers
To View All Configured Alert Handlers
Managing Account Status Notification Handlers
To View the Configured Account Status Notification Handlers
To Enable Account Status Notification Handlers
To Create a New Account Status Notification Handler
To Delete an Account Status Notification Handler
Monitoring the Server With LDAP
Viewing Monitoring Information Using the cn=monitor Entry
To View the Available Monitoring Information
To Monitor General-Purpose Server Information
To Monitor Version Information
To Monitor the User Root Back End
To Monitor the Backup Back End
To Monitor the monitor Back End
To Monitor the Schema Back End
To Monitor the adminRoot Back End
To Monitor the ads-truststore Back End
To Monitor the LDAP Connection Handler
To Monitor LDAP Connection Handler Statistics
To Monitor Connections on the LDAP Connection Handler
To Monitor the Administration Connector
To Monitor Administration Connector Statistics
To Monitor Connections on the Administration Connector
To Monitor the LDIF Connection Handler
To Monitor JVM Stack Trace Information
To Monitor the JVM Memory Usage
To Monitor the userRoot Database Environment
To Monitor Remote LDAP Servers
To Monitor a Global Index Catalog
Monitoring Using manage-tasks Command
To View the Replication Repair Logs
General Purpose Enterprise Monitoring Solutions
Monitoring the Server With JConsole
To Configure JMX on a Server Instance
Accessing a Server Instance From JConsole
Viewing Monitoring Information With JConsole
Monitoring the Server With SNMP
Configuring SNMP in the Server
To Configure SNMP in the Server
To View the SNMP Connection Handler Properties
To Access SNMP on a Server Instance
Monitoring a Replicated Topology
Monitoring Replication Status With dsreplication
Monitoring the Directory Server With the Control Panel
To View Monitoring Information With the Control Panel
Monitoring the Proxy Server With the Control Panel
To View Proxy Configuration Information
To View Proxy Monitoring Information
Setting LDAP Data Source Monitoring Properties in the Proxy
Modifying Monitoring of Remote LDAP Servers
The easiest way to monitor replication status is by using the dsreplication status command. However, in depth replication monitoring information is available under the cn=monitor entry. You can use the ldapsearch command to track specific monitoring attributes, which provide you with a comprehensive view of the replication status. Monitoring information is consolidated by replication servers. Therefore, monitoring information can only be retrieved by searching a directory server that hosts a running replication server.
The examples in the remainder of this section assume the following simple replication topology.
These examples access the cn=monitor entry on the administration port over SSL (--useSSL) and automatically trust the certificate that is presented by the server (--trustAll).
The information under cn=monitor can be filtered to include a single replicated base DN. You can do this in two ways:
Specify the domain-name attribute as a filter, for example:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" \ "(domain-name=dc=example,dc=com)"
Include the base DN in the search base, for example:
$ ldapsearch -p 4444 --useSSL --trustAll \ -b "cn=dc_example_dc_com,cn=replication,cn=monitor" "(objectclass=*)"
Each directory server contains a list of candidate replication servers for each replicated base DN. However, a directory server is connected to only one replication server at a time.
To obtain an overview of the replication topology and its connections, run the following search on any directory server in the topology that hosts a replication server:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" "(connected-to=*)" \ "connected-to" "lost-connections" dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor lost-connections: 0 connected-to: llandudno/0:0:0:0:0:0:0:1:8989 dn: cn=Replication Domain 14142,cn=cn_schema,cn=replication,cn=monitor lost-connections: 0 connected-to: llandudno/0:0:0:0:0:0:0:1:8989 dn: cn=Connected Replica llandudno 27742,cn=Replication Server 8989 1740,cn=cn_ admin data,cn=replication,cn=monitor connected-to: Replication Server 8989 1740 dn: cn=Connected Replica llandudno 30839,cn=Replication Server 8989 1740,cn=dc_ example_dc_com,cn=replication,cn=monitor connected-to: Replication Server 8989 1740 dn: cn=Connected Replica llandudno 14142,cn=Replication Server 8989 1740,cn=cn_ schema,cn=replication,cn=monitor connected-to: Replication Server 8989 1740 dn: cn=Undirect Replica 22052,cn=Connected Replication Server noordhoek:9989 71 64,cn=Replication Server 8989 1740,cn=cn_schema,cn=replication,cn=monitor connected-to: Connected Replication Server noordhoek:9989 7164,cn=Replication Se rver 8989 1740,cn=cn_schema,cn=replication dn: cn=Undirect Replica 19984,cn=Connected Replication Server noordhoek:9989 71 64,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=moni tor connected-to: Connected Replication Server noordhoek:9989 7164,cn=Replication Se rver 8989 1740,cn=dc_example_dc_com,cn=replication dn: cn=Undirect Replica 30030,cn=Connected Replication Server noordhoek:9989 71 64,cn=Replication Server 8989 1740,cn=cn_admin data,cn=replication,cn=monitor connected-to: Connected Replication Server noordhoek:9989 7164,cn=Replication Se rver 8989 1740,cn=cn_admin data,cn=replication dn: cn=Replication Domain 27742,cn=cn_admin data,cn=replication,cn=monitor lost-connections: 0 connected-to: llandudno/0:0:0:0:0:0:0:1:8989
The connected-to attribute specifies the replication server to which each directory server is currently connected for a particular base DN. If a directory server is directly connected to the replication server, its DN includes cn=Connected Replica. A directory server that is in the topology but is connected to a different replication server has cn=Undirect Replica in its DN. Because all replication servers are permanently connected to all other replication servers, the connected-to attribute does not exist for replication servers.
The lost-connections attribute indicates the number of connection breaks between directory servers and replication servers. The value of this attribute on each directory server should be close to the number of times that replication has been stopped on that server. If the value of this attribute is much higher, there are unexpected connection losses that must be investigated.
Monitoring replication latency enables you to establish whether a specific replication server is lagging behind other servers in the topology. This provides a complete view of any replication delays and the current quality of service.
To monitor replication latency, run the following search on any server in the topology that hosts a replication server:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" \ "domain-name=dc=example,dc=com" "missing-changes" \ "approx-older-change-not-synchronized" dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor dn: cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor missing-changes: 0 dn: cn=Connected Replica llandudno 30839,cn=Replication Server 8989 1740,cn=dc_ example_dc_com,cn=replication,cn=monitor missing-changes: 0 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor missing-changes: 0 dn: cn=Undirect Replica 19984,cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor missing-changes: 0
The missing-changes attribute specifies the number of updates already pushed by the other directory servers in the topology, but not yet replayed on the specified directory server.
The approx-older-change-not-synchronized attribute specifies the approximate date of the oldest update pushed by the other directory servers in the topology, but not yet processed on the specified directory server.
Note - If the replication latency, as defined by these attributes, is high, look at the number of updates sent and received to identify the servers in the topology that are causing the latency. These attributes are described later in this document.
Monitoring data consistency enables you to establish whether each replication server in the topology is synchronized and up-to-date with the latest changes that have occurred in the topology.
To monitor the data consistency across the directory servers in the topology, run the following search on any server in the topology that hosts a replication server:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" "(generation-id=*)" \ "generation-id" dn: cn=Replication Server 8989 1740,cn=cn_admin data,cn=replication,cn=monitor generation-id: cn=admin data 94310 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=cn_admin data,cn=replication,cn=monitor generation-id: 94310 dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor generation-id: 19399981 dn: cn=Replication Domain 14142,cn=cn_schema,cn=replication,cn=monitor generation-id: 8468 dn: cn=Connected Replica llandudno 27742,cn=Replication Server 8989 1740,cn=cn_ admin data,cn=replication,cn=monitor generation-id: 94310 dn: cn=Replication Server 8989 1740,cn=cn_schema,cn=replication,cn=monitor generation-id: cn=schema 8468 dn: cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor generation-id: dc=example,dc=com 19399981 dn: cn=Connected Replica llandudno 30839,cn=Replication Server 8989 1740,cn=dc_ example_dc_com,cn=replication,cn=monitor generation-id: 19399981 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=cn_schema,cn=replication,cn=monitor generation-id: 8468 dn: cn=Connected Replica llandudno 14142,cn=Replication Server 8989 1740,cn=cn_ schema,cn=replication,cn=monitor generation-id: 8468 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor generation-id: 19399981 dn: cn=Replication Domain 27742,cn=cn_admin data,cn=replication,cn=monitor generation-id: 94310
The generation-id attribute indicates the version of the data in each replicated base DN, for each directory server. Note that the generation ID on all servers for the base DN dc=example,dc=com is 19399981. The consistency of the generation IDs means that the data on those servers is the same for that base DN.
Each directory server is also aware of the generation ID of the replication server to which it is connected. The generation ID of a replication server relates to the updates that are stored in its change log database for that base DN.
Replication is considered to be working correctly between two directory servers, for a specified base DN, when those servers and their replication server all have the same generation ID.
A secure replication topology has SSL encryption enabled between servers, for a particular base DN.
To monitor replication security, run the following search on any server in the topology that hosts a replication server:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" "(ssl-encryption=*)" \ "ssl-encryption" dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 89 89 1740,cn=cn_admin data,cn=replication,cn=monitor ssl-encryption: true dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor ssl-encryption: true dn: cn=Replication Domain 14142,cn=cn_schema,cn=replication,cn=monitor ssl-encryption: true dn: cn=Connected Replica llandudno 27742,cn=Replication Server 8989 1740,cn=cn_ admin data,cn=replication,cn=monitor ssl-encryption: true dn: cn=Connected Replica llandudno 30839,cn=Replication Server 8989 1740,cn=dc_ example_dc_com,cn=replication,cn=monitor ssl-encryption: true dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 89 89 1740,cn=cn_schema,cn=replication,cn=monitor ssl-encryption: true dn: cn=Connected Replica llandudno 14142,cn=Replication Server 8989 1740,cn=cn_ schema,cn=replication,cn=monitor ssl-encryption: true dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor ssl-encryption: true dn: cn=Replication Domain 27742,cn=cn_admin data,cn=replication,cn=monitor ssl-encryption: true
The ssl-encryption attribute specifies whether the replication protocol is encrypted between two servers for a specified base DN. This information is available for each directory server or replication server. Authentication of replication sessions is not monitored.
Monitoring the number of updates that have been sent and received by the servers in a topology provides an indication of how well replication is working.
To monitor sent and received updates, type the following command:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" \ "(&(sent-updates=*)(received-updates=*))" "sent-updates" "received-updates" dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=cn_admin data,cn=replication,cn=monitor sent-updates: 7 received-updates: 0 dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor received-updates: 28 sent-updates: 0 dn: cn=Replication Domain 14142,cn=cn_schema,cn=replication,cn=monitor received-updates: 0 sent-updates: 0 dn: cn=Connected Replica llandudno 27742,cn=Replication Server 8989 1740,cn=cn_ admin data,cn=replication,cn=monitor sent-updates: 0 received-updates: 0 dn: cn=Connected Replica llandudno 30839,cn=Replication Server 8989 1740,cn=dc_ example_dc_com,cn=replication,cn=monitor sent-updates: 28 received-updates: 0 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=cn_schema,cn=replication,cn=monitor sent-updates: 0 received-updates: 0 dn: cn=Connected Replica llandudno 14142,cn=Replication Server 8989 1740,cn=cn_ schema,cn=replication,cn=monitor sent-updates: 0 received-updates: 0 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor sent-updates: 0 received-updates: 28 dn: cn=Replication Domain 27742,cn=cn_admin data,cn=replication,cn=monitor received-updates: 0 sent-updates: 0
The sent-updates attribute indicates the number of updates that have been sent by this directory server or replication server.
The received-updates attribute indicates the number of updates that have been received by this directory server or replication server.
The values of these attributes assist in determining the flow of updates within a topology. When replication appears to be very slow, it is helpful to monitor these attributes. If the number of updates sent by one server is consistently much higher than the number of updates received by another server, it is likely that the second server is a bottleneck in the topology.
The replication protocol controls the flow of updates between two servers. This ensures that when a high number of updates is exchanged between two servers, the servers are not prevented from processing operations with a higher priority. This functionality relies on a window mechanism where the recipient server periodically provides the sending server with the number of updates that the sending server can send.
You can specify the size of the send and receive windows, by setting the max-send-window and max-rcv-window configuration attributes. For more information, see Modifying the Replication Configuration With dsconfig.
The current-send-window monitoring attribute indicates how many changes can be sent by the sending server to the recipient server at that specific time. If the value of the current-send-window attribute is often equal to 0, transmission is stopped and the recipient server is probably a bottleneck in the topology. If the value of the current-send-window attribute is often equal to the value of the max-send-window attribute, and you are experiencing high replication latency, it is likely that the sending server is a bottleneck in the topology.
To obtain the value of the current-send-window property, type the following command:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" "(current-send-window=*)" \ "current-send-window" dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=cn_admin data,cn=replication,cn=monitor current-send-window: 93 dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor current-send-window: 100 dn: cn=Replication Domain 14142,cn=cn_schema,cn=replication,cn=monitor current-send-window: 100 dn: cn=Connected Replica llandudno 27742,cn=Replication Server 8989 1740,cn=cn_ admin data,cn=replication,cn=monitor current-send-window: 100 dn: cn=Connected Replica llandudno 30839,cn=Replication Server 8989 1740,cn=dc_ example_dc_com,cn=replication,cn=monitor current-send-window: 72 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=cn_schema,cn=replication,cn=monitor current-send-window: 100 dn: cn=Connected Replica llandudno 14142,cn=Replication Server 8989 1740,cn=cn_ schema,cn=replication,cn=monitor current-send-window: 100 dn: cn=Connected Replication Server noordhoek:9989 7164,cn=Replication Server 8989 1740,cn=dc_example_dc_com,cn=replication,cn=monitor current-send-window: 100 dn: cn=Replication Domain 27742,cn=cn_admin data,cn=replication,cn=monitor current-send-window: 100
When multiple operations are performed on the same entry at the same time, replication conflicts can occur. In some cases, the replication mechanism is able to resolve these conflicts. In other cases, manual conflict resolution is required.
Three types of conflict attributes can be monitored:
unresolved-naming-conflicts. Indicates the number of naming conflicts that could not be resolved by the replication mechanism.
resolved-naming-conflicts. Indicates the number of naming conflicts that have been resolved.
resolved-modify-conflicts. Indicates the number of modify conflicts that have been resolved.
To monitor resolved and unresolved replication conflicts, run the following command:
$ ldapsearch -p 4444 --useSSL --trustAll -b "cn=monitor" \ "(&(unresolved-naming-conflicts=*)(resolved-naming-conflicts=*)\ (resolved-modify-conflicts=*))" "unresolved-naming-conflicts" \ "resolved-naming-conflicts" "resolved-modify-conflicts" dn: cn=Replication Domain 30839,cn=dc_example_dc_com,cn=replication,cn=monitor resolved-naming-conflicts: 0 unresolved-naming-conflicts: 0 resolved-modify-conflicts: 0 dn: cn=Replication Domain 14142,cn=cn_schema,cn=replication,cn=monitor resolved-naming-conflicts: 0 unresolved-naming-conflicts: 0 resolved-modify-conflicts: 0 dn: cn=Replication Domain 27742,cn=cn_admin data,cn=replication,cn=monitor resolved-naming-conflicts: 0 unresolved-naming-conflicts: 0 resolved-modify-conflicts: 0