Managing Storage Directory Sizes
We strongly recommend that you always specify storage directory sizes for each Replication Node on every Storage Node in the store. Doing so sets disk threshold levels for each replication node, even when your store has hardware with varying disk capacities. This section describes this topic, and others.
Managing Disk Thresholds
It is very important to configure each storage directory with a specific amount of available disk space. The Oracle NoSQL Database uses the configured Storage Directory sizes to enforce disk space limits. Without configuring how much disk space is available, the store opportunistically uses all available space, less 5 GB free disk space. The system maintains 5 GB of free space to allow manual recovery if the Storage Node exceeds its configured disk limit. Be sure to monitor disk usage regularly using the statistics provided, as described in Monitoring Disk Usage.
-
To store your data.
-
To save reserved files.
Note:
If a Storage Node (SN) consumes more than what is assigned as storagedirsize, including leaving 5 GB of space free, the SN automatically attempts to free up disk space by deleting reserved files (not your data files), until more than 5 GB of space is available. If the Storage Node is unable to free up enough space, it suspends write operations to the node. Read operations continue as normal. Write operations resume automatically once the node obtains sufficient free disk space.
You can limit how much disk space the store consumes on a node by node basis, by explicitly specifying a storage directory size for each storage node, as described in Specifying Storage Directory Sizes. Storage nodes can then consume all of their configured disk space as needed, leaving free the required 5 GB. However, if you do not indicate a storage directory size, the Storage Node uses disk space until it consumes the disk, except for the required 5 GB for manual recovery.
Consider a storage node with a 200 GB disk. Without configuring a storagedirsize for that disk, the store keeps consuming up to 195 GB of disk space (leaving only the 5 GB for manual recovery). If your standard policy requires a minimum 20 GB available space on each disk, you must configure the storage node with a storagedirsize of 175 GB, leaving 20 GB available, and 5 GB for store recovery.
The most common reason a node's storage directory fills up is because of reserved files. If the Storage Node exceeds its disk threshold, it continues to delete the reserved files until the threshold is no longer exceeded.
Specifying Storage Directory Sizes
Use the makebootconfig storagedirsize
parameter to specify Storage Node (SN) capacity when you initially install your store. See Configuring your data store installation and makebootconfig for details. Additionally, if your SN has the capacity to support more than one Replication Node, specify a storage directory location and storage directory size for each Replication Node.
To specify or change storage capacity after you have installed the store, use plan change-storagedir
. When you use plan change-storagedir
be sure to specify the -storagedirsize
parameter to indicate how large the new storage directory is.
Note:
If you specify the-storagedir
parameter, but not -storagedirsize
, makebootconfig
displays a warning. Always specify both parameters for control and tracking.
The value specified for the storagedirsize
parameter must be a
long, optionally followed by a unit string. Accepted unit strings are: KB, MB, GB, and TB,
corresponding to 1024, 1024^2, 1024^3, 1024^4 respectively. Acceptable strings are case
insensitive. Valid delimiters between the long value and the unit string are " ", "-", or
"_". If you specify the delimiter as " ", your value should be enclosed in double quotes,
For example "10 GB". If you have any other delimiter double quotes is not mandatory. For
example 10_GB or 10-GB.
For example:
kv-> verbose
Verbose mode is now on
kv-> show topology
store=mystore numPartitions=300 sequence=308
zn: id=zn1 name=Manhattan repFactor=3 type=PRIMARY
allowArbiters=false
sn=[sn1] zn:[id=zn1 name=Manhattan] node1:9000 capacity=1 RUNNING
[rg1-rn1] RUNNING /storage-dir/sn1 0
No performance info available
sn=[sn2] zn:[id=zn1 name=Manhattan] node2:9000 capacity=1 RUNNING
[rg1-rn2] RUNNING /storage-dir/sn2 0
single-op avg latency=0.0 ms multi-op avg latency=0.0 ms
sn=[sn3] zn:[id=zn1 name=Manhattan] node3:9000 capacity=1 RUNNING
[rg1-rn3] RUNNING /storage-dir/sn3 0
No performance info available
shard=[rg1] num partitions=300
[rg1-rn1] sn=sn1 haPort=node1:9010
[rg1-rn2] sn=sn2 haPort=node2:9010
[rg1-rn3] sn=sn3 haPort=node3:9010
partitions=1-300
kv-> plan change-storagedir -sn sn1 -storagedir /storage-dir/sn1 \
-storagedirsize "200 gb" -add -wait
Executed plan 7, waiting for completion...
Plan 7 ended successfully
kv-> plan change-storagedir -sn sn2 -storagedir /storage-dir/sn2 \
-storagedirsize "300 gb" -add -wait
Executed plan 8, waiting for completion...
Plan 8 ended successfully
kv-> plan change-storagedir -sn sn3 -storagedir /storage-dir/sn3 \
-storagedirsize "400 gb" -add -wait
Executed plan 9, waiting for completion...
Plan 9 ended successfully
kv-> show topology
store=mystore numPartitions=300 sequence=308
zn: id=zn1 name=Manhattan repFactor=3 type=PRIMARY
allowArbiters=false
sn=[sn1] zn:[id=zn1 name=Manhattan] node1:9000 capacity=1 RUNNING
[rg1-rn1] RUNNING /storage-dir/sn1 214748364800
No performance info available
sn=[sn2] zn:[id=zn1 name=Manhattan] node2:9000 capacity=1 RUNNING
[rg1-rn2] RUNNING /storage-dir/sn2 322122547200
single-op avg latency=0.0 ms multi-op avg latency=0.0 ms
sn=[sn3] zn:[id=zn1 name=Manhattan] node3:9000 capacity=1 RUNNING
[rg1-rn3] RUNNING /storage-dir/sn3 429496729600
single-op avg latency=0.0 ms multi-op avg latency=0.0 ms
shard=[rg1] num partitions=300
[rg1-rn1] sn=sn1 haPort=node1:9010
[rg1-rn2] sn=sn2 haPort=node2:9010
[rg1-rn3] sn=sn3 haPort=node3:9010
partitions=1-300
Note:
If any Storage Node stores its data in the root directory (not recommended), then instead of plan change-storagedir
, set the rootDirSize
parameter. For example:
kv-> plan change-parameters -service sn1 -params rootDirSize=200_gb
Specifying Differing Disk Capacities
By default, Oracle NoSQL Database evenly distributes data across all the Storage Nodes in your store. No check is made in advance. The store expects all of the hardware in your storee to be homogenous, and so all Storage Nodes would have the same disk capacity.
However, more likely, you are running a store in an environment where some Storage Nodes have more disk capacity than others. In this case, you must specify appropriate disk capacity for each storage node. Oracle NoSQL Database will then place more data on higher capacity Storage Nodes. Be aware that specifying greater disk capacity to a storage node can result in an increased workload. Storage Nodes with more capacity than others could then serve more read and/or write activity. Be sure to size your storage nodes accordingly to support additional workload, if any.
Monitoring Disk Usage
If a Storage Node exceeds its disk usage threshold value (storagedirsize - 5GB), then all write activity for that node is suspended until sufficient disk space is made available. The store makes disk space available by removing reserved files to satisfy the threshold requirement. No data files are removed. Read activity continues while reserved data is being removed.
To ensure that your Storage Node can continue to service write requests, monitor the availableLogSize
JMX statistic. This represents the amount of space that can be used by write operations. This value is not necessarily representative of the amount of disk space currently in use, since quite a lot of disk space can, and is, used for reserved files, which are not included in the availableLogSize
statistic.
Reserved files are data files that have already been replicated, but which are retained for replication to nodes that are out of contact with the master node. Because Oracle NoSQL Database liberally reserves files, all available storage will frequently be consumed by reserved data. However, reserved data is automatically deleted as necessary by the Storage Node to continue write operations. For this reason, monitoring the actual disk usage is not meaningful.
If availableLogSize
reaches zero, writes are suspended for the Storage Node. Earlier, as availableLogSize
approaches zero, the node has less and less space for reserved data files. The result is that the store becomes increasingly less resilient in the face of a prolonged but temporary node outage because there are increasingly fewer historical log files that the store can use to gracefully bring a node up to date once it is available again.
oracle.kv.repnode.envmetric
type. (Xref)
Statistic | Description |
---|---|
availableLogSize |
Disk space available (in bytes) for write operations. This value is calculated with consideration fo reserved data files, which are deleted automatically whenever space is required to perform write operations: In general, monitoring disk usage in the file system is not meaningful, because of the presence of reserved files that can be deleted automatically. |
activeLogSize |
Bytes used by all active data files: files required for basic operation. |
reservedLogSize |
Bytes used by all reserved data files: files that have been cleaned and can be deleted if they are not protected. |
protectedLogSize |
Bytes used by all protected data files: the subset of reserved files that are temporarily protected and cannot be deleted. |
ProtectedLogSizeMap |
A breakdown of protectedLogSize as a map of protecting entity name to protected size in bytes. |
TotalLogSize |
Total bytes used by data files on disk: activeLogSize + reservedLogSize. |
The following list from part of some JMX output, shows an example of how you will see each statistic. All of these statistic names have a Cleaning_
prefix, indicating that they may be in the log cleaning statistics group (for garbage collection):
.
.
.
"Cleaning_nRepeatIteratorReads": 0,
"Cleaning_nLNsExpired": 0,
"Cleaning_nCleanerRuns": 0,
"Cleaning_nBINDeltasDead": 0,
"Cleaning_nCleanerDisksReads": 0,
"Cleaning_protectedLogSizeMap": "",
"Cleaning_nCleanerDeletions": 0,
"Cleaning_nCleanerEntriesRead": 0,
"Cleaning_availableLogSize": 48942137344,
"Cleaning_nLNsDead": 0,
"Cleaning_nINsObsolete": 0,
"Cleaning_activeLogSize": 112716,
"Cleaning_nINsDead": 0,
"Cleaning_nINsMigrated": 0,
"Cleaning_totalLogSize": 112716,
"Cleaning_nBINDeltasCleaned": 0,
"Cleaning_nLNsObsolete": 0,
"Cleaning_nLNsCleaned": 0,
"Cleaning_nLNQueueHits": 0,
"Cleaning_reservedLogSize": 0,
"Cleaning_protectedLogSize": 0,
"Cleaning_nClusterLNsProcessed": 0,
"Node Compression_processedBins": 0,
.
.
.
You can tell if writes have been suspended for a Storage Node using the ping
command from the CLI. In the following sample output, the Shard Status shows read-only:1
. This indicates that one of the Storage Nodes is in read-only mode. The likeliest reason for that is that it has exceeded its disk threshold.
kv-> ping
Pinging components of store istore based upon topology sequence #11
3 partitions and 3 storage nodes
Time: 2018-09-28 06:57:10 UTC Version: 18.3.2
Shard Status: healthy:0 writable-degraded:0 read-only:1 offline:0
Admin Status: healthy
Zone [name=dc1 id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false]
RN Status: online:1 offline:2
Storage Node [sn1] on sn1.example.com:5000 Zone: [name=dc1
id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING
Ver: 18.3.2 2018-09-17 09:33:45 UTC Build id: a72484b8b33c
Admin [admin1] Status: RUNNING,MASTER
Rep Node [rg1-rn1] Status: RUNNING,MASTER (non-authoritative)
sequenceNumber:39,177,477 haPort:5011 available storage size:6 GB
Storage Node [sn2] on sn2.example.com:5000 Zone:
[name=dc1 id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING
Ver: 18.3.2 2018-09-17 09:33:45 UTC Build id: a72484b8b33c
Rep Node [rg1-rn2] Status: RUNNING,UNKNOWN sequenceNumber:39,176,478
haPort:5010 available storage size:NOT AVAILABLE delayMillis:? catchupTimeSecs:?
Storage Node [sn3] on sn3.example.com:5000 Zone: [name=dc1
id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING
Ver: 18.3.2 2018-09-17 09:33:45 UTC Build id: a72484b8b33c
Rep Node [rg1-rn3] Status: RUNNING,UNKNOWN sequenceNumber:39,166,804
haPort:5010 available storage size:NOT AVAILABLE delayMillis:? catchupTimeSecs:?
For information on JMX monitoring the store, see Java Management Extensions (JMX) Notifications.
Handling Disk Limit Exception
If a Storage Node exceeds its disk usage threshold value (<storagedirsize> - 5 GB
), then the store suspends all write activities on that node, until sufficient data is removed to satisfy the threshold requirement. In such a situation, there are two ways to bring the store back to read and write availability, without deleting user data.
-
Increasing
storagedirsize
on one or more Replication Nodes if there is available disk space -
Expanding the store by adding a new shard
If there is enough space left on the disk or if the complete disk size is not set as the size of storagedirsize
, you can bring back the write availability (without any additional need of the hardware) by simply increasing the storage directory size for one or more Replication Nodes.
If there is not enough space left on disk or if the complete disk size is set as the size of the storage directory, then you should follow the store expansion procedure, where you will need additional hardware to increase the number of shards by one.
Note:
If you are following the store expansion procedure, it is important to check the performance files to see if the cleaner is working well, by monitoring theminUtilization
statistics. If the minUtilization
statistics is less than 30%, it may mean that the cleaner is not keeping up. In this case it is not possible to perform store expansion.
Store expansion can only be performed if the minUtilization
statistics percentage is not less than 30%.
For example:
2018-03-07 16:07:12.499 UTC INFO [rg1-rn1] JE: Clean file 0x2b:
predicted min util is below minUtilization, current util min: 39 max: 39,
predicted util min: 39 max: 39, chose file with util min: 30 max: 30 avg: 30
2018-03-07 16:07:04.029 UTC INFO [rg1-rn2] JE: Clean file 0x27:
predicted min util is below minUtilization, current util min: 39 max: 39,
predicted util min: 39 max: 39, chose file with util min: 30 max: 30 avg: 30
2018-03-07 16:05:44.960 UTC INFO [rg1-rn3] JE: Clean file 0x27:
predicted min util is below minUtilization, current util min: 39 max: 39,
predicted util min: 39 max: 39, chose file with util min: 30 max: 30 avg: 30
Increasing Storage Directory Size
To increase the storage directory size in one or more Replication Nodes, open the CLI and execute the following commands:
-
Disable write operations on the store or on the failed shard.
plan enable-requests -request-type READONLY \ {-shard <shardId[,shardId]*> | -store}
Here,
-request-type READONLY
is the option which disables write operations on a shard. You can disable write operations on one or more shards by using the-shard
option, or on the entire store by using the—store
option.Note:
Though Replication Nodes are already in non-write availability mode whenever they hit an out of disk limit exception, it is important to disable user write operations explicitly. Disabling the user write operations ensures that the Replication Nodes are brought back up in the correct manner. -
Execute the
PING
command to analyze the state of one or more Replication Nodes.kv-> ping
Usually, when Replication Nodes hit an out of disk limit exception, Replica Replication Nodes are in the
RUNNING, UNKNOWN
state, and Master Replication Nodes are in theRUNNING, MASTER (non-authoritative)
state. -
To display the current, deployed topology, execute the
show topology —verbose
command. Make note of the current storage directory size allocated to each Replication Node.show topology -verbose [-zn] [-rn] [-an] [-sn] [-store] [-status] [-json]
-
To ensure that other Replication nodes in the store do not hit a disk limit exception while you increase the
storagedirsize
, reduce the JE free disk space on all Replication Nodes to 2 GB or 3 GB. You can use the-all-rns
option to reduce the JE free disk space on all Replication Nodes at once, or the-service -rgx-rgy
option to reduce the free disk space on a specific Replication Node.kv-> plan change-parameters [-all-rns|-service -rgx-rgy] \ -params "configProperties=je.freeDisk=XXX"
After executing this command with either option, the system will stop the Replication Nodes, update parameters, and restart Replication Nodes with the JE free disk space parameter you specify.
-
To increase the storage directory size on one or more Replication Nodes.
kv-> plan change-storagedir -wait -sn snX \ -storagedir <storagedirpath> –add -storagedirsize X_GB
Here
snX
is the Storage Node whose directory size you want to increase, and X is the new storage size in GB. -
After the
plan change-parameters
command executes successfully, verify the newstoragedirsize
value is assigned to one or more Replication Nodes in the store.show topology -verbose [-zn] [-rn] [-an] [-sn] [-store] [-status] [-json]
-
Lastly, reset the JE free disk space back to 5 GB. Also, enable write operations back on the store or a specific shard.
kv-> plan change-parameters [-all-rns|-service -rgx-rgy] \ -params "configProperties=je.freeDisk=5368709120" kv-> plan enable-requests –request-type ALL {-shard <shardId[,shardId]*> | -store}
The
–request-type ALL
option re-enables write operations on the store or on a specific shard.
Example
Let us consider a store with 1x3 topology, hitting a disk limit exception. Perform the following steps to increase the storage directory size of all Replication Nodes in the store from 16 GB to 25 GB.
-
Stop the write operations on the store level:
kv-> plan enable-requests –request-type READONLY –store;
-
Ping the store to analyze the state of one or more Replication Nodes.
kv-> ping Pinging components of store istore based upon topology sequence #11 3 partitions and 3 storage nodes Time: 2018-09-28 06:57:10 UTC Version: 18.3.2 Shard Status: healthy:0 writable-degraded:0 read-only:1 offline:0 total:1 Admin Status: healthy Zone [name=dc1 id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] RN Status: online:1 offline:2 Storage Node [sn1] on node21:port1 Zone: [name=dc1 id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 18.3.2 2018-09-17 09:33:45 UTC Build id: a72484b8b33c Admin [admin1] Status: RUNNING,MASTER Rep Node [rg1-rn1] Status: RUNNING,MASTER (non-authoritative) sequenceNumber:27,447,667 haPort:5011 available storage size:12 GB Storage Node [sn2] on node22:port1 Zone: [name=dc1 id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 18.3.2 2018-09-17 09:33:45 UTC Build id: a72484b8b33c Rep Node [rg1-rn2] Status: RUNNING,UNKNOWN sequenceNumber:27,447,667 haPort:5010 available storage size:10 GB delayMillis:? catchupTimeSecs:? Storage Node [sn3] on node23:port1 Zone: [name=dc1 id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 18.3.2 2018-09-17 09:33:45 UTC Build id: a72484b8b33c Rep Node [rg1-rn3] Status: RUNNING,UNKNOWN sequenceNumber:27,447,667 haPort:5010 available storage size:9 GB delayMillis:? catchupTimeSecs:?
The example shows that the Replication Nodes are in
RUNNING, UNKNOWN
state and Master Replication Node is inRUNNING, MASTER(non-authoritative)
state. -
View the current, deployed topology.
kv-> show topology -verbose store=istore numPartitions=3 sequence=11 zn: id=zn1 name=dc1 repFactor=3 type=PRIMARY allowArbiters=false \ masterAffinity=false sn=[sn1] zn:[id=zn1 name=dc1] node21:port1 capacity=1 RUNNING [rg1-rn1] RUNNING /scratch/kvroot 16 GB single-op avg latency=36.866146 ms multi-op avg latency=0.0 ms [rg1-rn1] RUNNING /scratch/kvroot 16 GB single-op avg latency=36.866146 ms multi-op avg latency=0.0 ms sn=[sn2] zn:[id=zn1 name=dc1] node22:port1 capacity=1 RUNNING [rg1-rn2] RUNNING /scratch/kvroot 16 GB single-op avg latency=0.0 ms multi-op avg latency=0.0 ms [rg1-rn2] RUNNING /scratch/kvroot 16 GB single-op avg latency=0.0 ms multi-op avg latency=0.0 ms sn=[sn3] zn:[id=zn1 name=dc1] node23:port1 capacity=1 RUNNING [rg1-rn3] RUNNING /scratch/kvroot 16 GB single-op avg latency=0.0 ms multi-op avg latency=0.0 ms [rg1-rn3] RUNNING /scratch/kvroot 16 GB single-op avg latency=0.0 ms multi-op avg latency=0.0 ms numShards=1 shard=[rg1] num partitions=3 [rg1-rn1] sn=sn1 haPort=node21:port2 [rg1-rn2] sn=sn2 haPort=node22:port3 [rg1-rn3] sn=sn3 haPort=node23:port3 partitions=1-3
You see that 16 GB of disk space is assigned as the storage directory size for each Replication Node.
-
Reduce the JE free disk space from 5 GB to 2 GB for all Replication Nodes in the store.
kv-> plan change-parameters -all-rns -params \ "configProperties=je.freeDisk=2147483648"; Started plan 70. Use show plan -id 70 to check status. To wait for completion, use plan wait -id 70
-
For each Replication Node, increase the storage directory size to 25 GB.
kv-> plan change-storagedir -wait -sn sn1 -storagedir /scratch/kvroot \ -add -storagedirsize 25_GB -wait Executed plan 72, waiting for completion... Plan 72 ended successfully
kv-> plan change-storagedir -wait -sn sn2 -storagedir /scratch/kvroot \ -add -storagedirsize 25_GB -wait Executed plan 73, waiting for completion... Plan 73 ended successfully
kv-> plan change-storagedir -wait -sn sn3 -storagedir /scratch/kvroot \ -add -storagedirsize 25_GB -wait Executed plan 74, waiting for completion... Plan 74 ended successfully
-
View the topology again to verify that the new value is assigned to
storagedirsize
.kv-> show topology -verbose store=istore numPartitions=3 sequence=11 zn: id=zn1 name=dc1 repFactor=3 type=PRIMARY allowArbiters=false \ masterAffinity=false sn=[sn1] zn:[id=zn1 name=dc1] node21:port1 capacity=1 RUNNING [rg1-rn1] RUNNING /scratch/kvroot 25 GB single-op avg latency=0.0 ms multi-op avg latency=0.0 ms sn=[sn2] zn:[id=zn1 name=dc1] node22:port1 capacity=1 RUNNING [rg1-rn2] RUNNING /scratch/kvroot 25 GB single-op avg latency=552.51996 ms multi-op avg latency=0.0 ms sn=[sn3] zn:[id=zn1 name=dc1] node23:port1 capacity=1 RUNNING [rg1-rn3] RUNNING /scratch/kvroot 25 GB single-op avg latency=14.317171 ms multi-op avg latency=0.0 ms numShards=1 shard=[rg1] num partitions=3 [rg1-rn1] sn=sn1 haPort=node21:port2 [rg1-rn2] sn=sn2 haPort=node22:port3 [rg1-rn3] sn=sn3 haPort=node23:port3 partitions=1-3
The example now shows that 25 GB is assigned as the storage directory size for each Replication Node.
-
Reset the JE free disk space to 5 GB and enable write operations back on the store.
kv-> plan change-parameters [-all-rns|-service -rgx-rgy] \ -params "configProperties=je.freeDisk=5368709120" kv-> plan enable-requests –request-type READONLY –store;
Adding a New Shard
Apart from increasing the storage directory size, you can also handle disk limit exceptions by adding a new shard and expanding your store.
The following example demonstrates adding three new Storage Nodes (Storage Nodes 21, 22, and 23) and deploying the new store to recover from disk limit exception:
-
Disable write operations on the store.
kv-> plan enable-requests -request-type READONLY -store;
Here,
-request-type READONLY
disables write operations on a store and allows only read operations. -
Reduce the JE free disk space to 2 GB on all nodes and increase the
je.cleaner.minUtilization
configuration parameter from 40 (the default in a KVStore) to 60.kv-> plan change-parameters -all-rns \ -params "configProperties=je.cleaner.minUtilization 60; \ je.freeDisk 2147483648";
Executing this command creates more free space for store expansion. Replication Nodes will be stopped, parameters will be updated, and the Replication Nodes will be restarted with the new parameters.
-
Create, start, and configure the new nodes for expanding the store.
-
Create the new node. Run the
makebookconfig
utility to configure each Storage Node in the store:java -Xmx256m -Xms256m -jar KVHOME/kvstore.jar makebootconfig \ -root sn1/KVROOT \ -store-security none -capacity 1 \ -port port1 -host node21 \ -harange 5010,5020 \ -storagedir /scratch/sn1/u01 –storagedirsize 20-Gb
java -Xmx256m -Xms256m -jar KVHOME/kvstore.jar makebootconfig \ -root sn2/KVROOT \ -store-security none -capacity 1 \ -port port1 -host node22 \ -harange 5010,5020 \ -storagedir /scratch/sn2/u01 –storagedirsize 20-Gb
java -Xmx256m -Xms256m -jar KVHOME/kvstore.jar makebootconfig \ -root sn3/KVROOT \ -store-security none -capacity 1 \ -port port1 -host node23 \ -harange 5010,5020 \ -storagedir /scratch/sn3/u01 –storagedirsize 20-Gb
-
Restart the Storage Node Agent (SNA) on each of the Oracle NoSQL Database nodes using the
start
utility:kv-> nohup java -Xmx256m -Xms256m -jar \ KVHOME/lib/kvstore.jar start -root KVROOT &
-
Configure the new store:
java -Xmx256m -Xms256m -jar KVHOME/lib/kvstore.jar runadmin \ -port port1 -host node21
java -Xmx256m -Xms256m -jar KVHOME/lib/kvstore.jar runadmin \ -port port1 -host node22
java -Xmx256m -Xms256m -jar KVHOME/lib/kvstore.jar runadmin \ -port port1 -host node23
-
-
Redistribute the store according to its new configuration.
kv-> java -Xmx256m -Xms256m -jar KVHOME/lib/kvstore.jar runadmin \ -port port1 -host host1
kv-> plan deploy-sn -zn zn1 -host node21 -port port1 -wait Executed plan 7, waiting for completion... Plan 7 ended successfully kv-> plan deploy-sn -zn zn1 -host node22 -port port1 -wait Executed plan 8, waiting for completion... Plan 8 ended successfully kv-> plan deploy-sn -zn zn1 -host node23 -port port1 -wait Executed plan 9, waiting for completion... Plan 9 ended successfully Plan 11 ended successfully
kv-> pool join -name ExamplePool -sn sn4 Added Storage Node(s) [sn4] to pool ExamplePool kv-> pool join -name ExamplePool -sn sn5 Added Storage Node(s) [sn5] to pool ExamplePool kv-> pool join -name ExamplePool -sn sn6 Added Storage Node(s) [sn6] to pool ExamplePool
kv-> topology clone -current -name newTopo Created newTopo
kv-> topology redistribute -name newTopo -pool ExamplePool Redistributed: newTopo
kv-> plan deploy-topology -name newTopo -wait Executed plan 11, waiting for completion...
-
Restore the Replication Nodes to its original configuration.
plan change-parameters -all-rns \ -params "configProperties=je.cleaner.minUtilization 40; \ je.freeDisk 5368709120";
-
Enable write operations back on the store.
kv-> plan enable-requests -request-type ALL -store;
Here,
—request-type ALL
enables both read and write operations on the store.