Verification is a tool you can use to:
Perform general troubleshooting of the store.
Verification inspects all components of the store. It also checks whether all store services are available, and for those services it checks whether there are any version or metadata mismatches.
Check the status of a long-running plan
Some plans require many steps and may take some time to execute.
The administrator can verify plans to check on progress of the
plan, or as an aid to diagnose a plan that is in an ERROR state.
For example, if you can verify a Deploy Store
plan while it is running against many Storage Nodes. You can
watch the verify report at each iteration to see that more and
more nodes have created and have come online.
For more information on how to manage plans, see Plans.
Provide additional information about a plan that is in an ERROR state.
You run store verification using the verify
command in the CLI. It requires no parameters, and by default it
runs in verbose mode. For example:
> java -jar KVHOME/lib/kvstore.jar runadmin -port <registry port> \
-host <hostname>
kv-> verify
Verify: starting for mystore based upon topology sequence #1008, 1000
partitions and 3 Storage Nodes, version:11gR2.1.1.0
See <nodeHostname>:/KVRT1/mystore/log/mystore_{0..N}.log for progress
messages
Verify: == checking storage node sn1 ==
Verify: Storage Node [sn1] on <nodeHostname>:5000
Datacenter: baskin [dc1] Status: RUNNING Ver: 11gR2.1.1.0
Verify: Admin [admin1] Status: RUNNING
Verify: Rep Node [rg1-rn1] Status: RUNNING,REPLICA at
sequence number:2,025 haPort:5011
Verify: == checking storage node sn2 ==
Verify: Storage Node [sn2] on <nodeHostname>:5100
Datacenter: baskin [dc1] Status: RUNNING Ver: 11gR2.1.1.0
Verify: Rep Node [rg1-rn2] Status: RUNNING,REPLICA at sequence
number:2,025 haPort:5110
Verify: == checking storage node sn3 ==
Verify: Storage Node [sn3] on <nodeHostname>:5200
Datacenter: baskin [dc1] Status: RUNNING Ver: 11gR2.1.1.0
Verify: Rep Node [rg1-rn3] Status: RUNNING,MASTER at sequence
number:2,025 haPort:5210
Verification complete, no problems.
A silent mode is available which shows only problems.
kv-> verify -silent
Verify: starting for mystore based upon topology sequence #1008, 1000
partitions and 3 Storage Nodes, version:11gR2.1.1.0
See <nodeHostname>:/KVRT1/mystore/log/mystore_{0..N}.log for progress
messages
Verification complete, no problems.
Problems with the store are clearly reported. For example, if a Storage Node is unavailable, then in silent mode that problem is displayed in the following way:
kv-> verify -silent
Verify: starting for mystore based upon topology sequence #1008, 1000
partitions and 3 Storage Nodes, version:11gR2.1.1.0
See <nodeHostname>:/KVRT1/mystore/log/mystore_{0..N}.log for progress
messages
Verification complete, 2 problems found.
Verify: sn2: ping() failed for sn2 : Connection refused to host:
<nodeHostname>; nested exception is:
java.net.ConnectException: Connection refused
Verify: rg1-rn2: ping() failed for rg1-rn2 : Connection refused to
host: <nodeHostname>; nested exception is:
java.net.ConnectException: Connection refused
In verbose mode, the above problem is shown in the following way:
kv-> verify
Verify: starting for mystore based upon topology sequence #1008, 1000
partitions and 3 Storage Nodes, version:11gR2.1.1.0
See <nodeHostname>:/KVRT1/mystore/log/mystore_{0..N}.log for progress
messages
Verify: == checking storage node sn1 ==
Verify: Storage Node [sn1] on <nodeHostname>:5000
Datacenter: baskin [dc1] Status: RUNNING Ver: 11gR2.1.1.0
Verify: Admin [admin1] Status: RUNNING
Verify: Rep Node [rg1-rn1] Status: RUNNING,REPLICA at sequence
number:2,025 haPort:5011
Verify: == checking storage node sn2 ==
Verify: sn2: ping() failed for sn2 : Connection refused to host:
<nodeHostname>; nested exception is:
java.net.ConnectException: Connection refused
Verify: Storage Node [sn2] on <nodeHostname>:5100
Datacenter: baskin [dc1] UNREACHABLE
Verify: rg1-rn2: ping() failed for rg1-rn2 : Connection refused to
host: <nodeHostname>; nested exception is:
java.net.ConnectException: Connection refused
Verify: Rep Node [rg1-rn2] Status: UNREACHABLE
Verify: == checking storage node sn3 ==
Verify: Storage Node [sn3] on <nodeHostname>:5200
Datacenter: baskin [dc1] Status: RUNNING Ver: 11gR2.1.1.0
Verify: Rep Node [rg1-rn3] Status: RUNNING,MASTER at sequence
number:2,025 haPort:5210
Verification complete, 2 problems found.
Verify: sn2: ping() failed for sn2 : Connection refused to host:
<nodeHostname>; nested exception is:
java.net.ConnectException: Connection refused
Verify: rg1-rn2: ping() failed for rg1-rn2 : Connection refused to
host: <nodeHostname>; nested exception is:
java.net.ConnectException: Connection refused
The verify output is only displayed in the shell when the
command is finished. You can tail or grep the Oracle NoSQL Database log
file in order to get a sense of how the verify is progressing.
Look for the string Verify. For example:
grep Verify /KVRT1/mystore/log/mystore_0.log