Setting Store Parameters

Changing Parameters

All of the CLI commands used for creating parameter-changing plans share a similar syntax:

plan change-parameters -service <id>...

All such commands can have multiple ParameterName=NewValue assignment arguments on the same command line. If NewValue contains spaces, then the entire assignment argument must be quoted within double quote marks. For example, to change the Admin parameter collectorPollPeriod, you would issue the command:

kv-> plan change-parameters -all-admins -params \
    "collectorPollPeriod=20 SECONDS">

If your configProperties for all Replication Nodes is set to:

"configProperties=je.cleaner.minUtilization=40;">

And you want to add new settings for configProperties, you would issue the following command:

kv-> plan change-parameters -all-rns -params \
     "configProperties=je.cleaner.minUtilization=40;\
     je.env.runVerifier=false;">

If for some reason, different Replication Nodes have different configProperties parameter values, then the change-parameters command will need to be tailored for each Replication Node.

The following commands are used to change service parameters:

plan change-parameters -service <shardId-nodeId> -params [assignments]

This command is used to change the parameters of a single Replication Node, which must be identified using the shard and node numbers. The shardId-nodeId identifier must be given as a single argument with one embedded hyphen and no spaces. The shardId identifier is represented by rgX, where X refers to the shard number.
plan change-parameters -all-rns -params [assignments]

This command is used to change the parameters of all Replication Nodes in a store. No Replication Node identifier is needed in this case.
plan change-parameters -service <storageNodeId> -params [assignments]

This command is used to change the parameters of a single Storage Node instance. The storageNodeId is a simple integer.
plan change-parameters -all-admins -params [assignments]

This command is used to change Admin parameters. Because each instance of Admin is part of the same replicated service, all instances of the Admin are changed at the same time, so no Admin identifier is needed in this command.

If an Admin parameter change requires the restarting of the Admin service, KVAdmin loses its connection to the server. Under normal circumstances, KVAdmin automatically reconnects after a brief pause, when the next command is given. At this point the plan is in the INTERRUPTED state, and must be completed manually by issuing the plan execute command.
plan change-parameters -security <id>

This command is used to change security parameters. The parameters are applied implicitly and uniformly across all SNs, RNs and Admins.

In all cases, you can choose to create a plan and execute it; or to create the plan and execute it in separate steps by using the -noexecute option of the plan command.

Setting Store Wide Policy Parameters

Most admin, Storage Node, and replication node parameters are assigned to default values when a store is deployed. It can be inconvenient to adjust them after deployment, so Oracle NoSQL Database provides a way to set the defaults that are used during deployment. These defaults are called store-wide Policy parameters.

You can set policy parameters in the CLI by using this command:

change-policy -params [name=value]

The parameters to change follow the -params flag and are separated by spaces. Parameter values with embedded spaces must be separated by spaces. Parameter values with embedded spaces must be quoted. For example: name = "value with spaces". If the optional dry-run flag is specified, the new parameters are returned without changing them.

Admin Parameters

You can set the following parameters for the Admin service:

collectorPollPeriod=<Long TimeUnit>

Sets the Monitor subsystem's delay for polling the various services for status updates. This value defaults to "20" seconds. Units are supplied as a string in the change-parameters command, for example: -params collectorPollPeriod="2 MINUTES"
loggingConfigProps=<String>

Property settings for the Logging subsystem in the Admin process. Its format is property=value;property=value.... Standard java.util.logging properties can be set by this parameter.
eventExpiryAge=<Long TimeUnit>

You can use this parameter to adjust how long the Admin stores critical event history. The default value is "30 DAYS".
configProperties=<String>

This is an omnibus string of property settings for the underlying BDB JE subsystem. Its format is property=value;property=value....
javaMiscParams=<String> [deprecated]

This parameter should ONLY be used to set flags for which there are no service parameters.

This parameter is deprecated. You are encouraged not to use it, and instead use the javaAdminParamsOverride parameter.
javaAdminParamsOverride=<String>

This is an omnibus string that is added to the command line when the Admin process is started. This parameter is intended for specifying miscellaneous JVM properties that cannot be specified using other Admin parameters. If the string is not a valid sequence of tokens for the JVM command line, the Admin process fails to start.

No default value is provided for this parameter.

Changing Admin JVM Memory Parameters

Admin processes can run out of memory. One of the most likely reasons is that the default memory setting was insufficient for the Admin services to represent all of the metadata associated with the store. Metadata includes information about tables, security information about users and roles, and information about incomplete plans. Stores with large amounts of metadata may need to increase the memory setting for Admin services if the activity logs show that Admin services are failing with OutOfMemoryError. This topic describes increasing the memory setting of the javaAdminParamsOverride.

The system continues to use the old javaMiscParams setting to specify the initial JVM memory settings for the admin. The system does not use javaAdminParamsOverride since it is reserved for use if you want to override the default settings.

To change the javaAdminParamsOverride requires a comprehensive all or nothing change. You cannot change individual parameters within the set. To change any setting, declare them all in the plan change-parameters command, described next.

First determine the basic information about all Admin services using the show admins command. You get the output as shown below.

show admins
admin1: Storage Node sn1 storageDir=/home/opc/nosql/kvroot type=PRIMARY (connected RUNNING,MASTER)
admin2: Storage Node sn2 storageDir=/home/opc/nosql/kvroot type=PRIMARY (RUNNING,REPLICA)
admin3: Storage Node sn3 storageDir=/home/opc/nosql/kvroot type=PRIMARY (RUNNING,REPLICA)

Note:

The above output is just an example. Here the replication factor RF=3 for the primary nodes. Your output will reflect your topology.

To determine the current settings of javaAdminParamsOverride and configProperties, enter the Admin CLI

show parameters
                -service admin

command as follows:

kv-> show parameters -service admin1
adminId=1
adminLogFileCount=20
adminLogFileLimit=5242880
adminMountPoint=/home/opc/nosql/kvroot
collectEnvStats=true
collectorPollPeriod=20 SECONDS
disabled=false
eventExpiryAge=30 DAYS
hideUserData=true
javaAdminParamsOverride=
loggingConfigProps=com.sleepycat.je.util.FileHandler.level=OFF
maxEvents=10000
storageNodeId=1

In this example, the javaAdminParamsOverride parameters that specify the Admin JVM memory shows the default value.

javaAdminParamsOverride=

Note:

The value may or may not have a default value. The value in your setup could also be different.

To increase Admin JVM memory when Admins are operational, use the plan change-parameters command from the Admin CLI, as follows:

kv-> plan change-parameters -wait -all-admins  -params \
javaAdminParamsOverride="-Xms2048m -Xmx2048m

Note:

It is recommended that you apply the modification to all of the admins by using the all-admins option. However, you can modify the admin parameters of every primary admin individually, but only if you have three or more primary admins. When you have only two admins, you must use the all-admins parameter; otherwise, the restart needed during this operation will cause the admins to lose the quorum.

Specifying these new values changes the Java heap size from the default values to 2 GB for both as shown below.

kv-> show parameters -service admin1
adminId=1
adminLogFileCount=20
adminLogFileLimit=5242880
adminMountPoint=/home/opc/nosql/kvroot
collectEnvStats=true
collectorPollPeriod=20 SECONDS
disabled=false
eventExpiryAge=30 DAYS
hideUserData=true
javaAdminParamsOverride=-Xms2048m -Xmx2048m
loggingConfigProps=com.sleepycat.je.util.FileHandler.level=OFF
maxEvents=10000
storageNodeId=1

Make sure that you locate the existing javaAdminParamsOverride from the Admin CLI as shown above, and update the individual entries. The javaAdminParamsOverride setting must represent all desired flags, not just new ones, so be sure to include any previously existing flag values that you want to retain.

If the Admin loses quorum, then you must use the Admin CLI repair-admin-quorum command.

Storage Node Parameters

You can set the following Storage Node parameters:

capacity=<Integer>

Sets the number of Replication Nodes that this Storage Node can host. This value informs decisions about where to place new Replication Nodes. The default value is 1. You can set the capacity level to greater than 1 if the Storage Node has sufficient disk, CPU, and memory resources to support multiple Replication Nodes.

Setting the Storage Node capacity to 0 indicates that the Storage Node can be used to host Arbiter Nodes. The pool of Storage Nodes in a zone configured to host Arbiter Nodes is used for Arbiter Node allocation. See Deploying an Arbiter Node Enabled Topology.
jvmOverheadPercent=<Integer>

Sets the percentage of Java heap size, for additional memory used by JVM overhead. Default value: 25. In standard memory allocation, 85% of the SN's memory is for Java heap and JVM overhead: 68% for Java heap (rnHeapPercent), 25% (jvmOverheadPercent) * 68 (rnHeapPercent) = 17% for JVM overhead, and 68% + 17% = 85%.
memoryMB=<Integer>

Sets the amount of memory (in megabytes) available on this Storage Node. The default value is 0, which indicates that the amount of memory is unknown. The store determines the amount of memory automatically as the total amount of RAM available on the machine.

You should not need to change this parameter. If the machine has other applications running on it, reserve some memory for those applications, and set the memoryMB parameter value with a memory allowance for application needs. Having other applications running on a Storage Node is not a recommended configuration.
mgmtClass=<String>

The name of the class that provides the Management Agent implementation. See Standardized Monitoring Interfaces. The port cannot be a privileged port number (<1024).
numCPUs=<Integer>

Sets the number of CPUs known to be available on this Storage Node. Default value: 1.
rnHeapMaxMB=<Integer>

Sets a hard limit for the maximum size of the Replication Node's Java VM heap. The default value is 0, which means the VM-specific limit is used. The default is roughly 32 GB, which represents the largest heap size that can make use of compressed object references.

Do not set this value to greater than 32 GB. Doing so can adversely impact your Replication Node's performance.

Settings larger than the maximum size that supports compressed object references will maintain the default limit unless the size is large enough that the heap can reference a larger number of objects given the increased memory requirements for uncompressed object references. Using larger heap sizes is not recommended.
rnHeapPercent=<Integer>

Sets the percentage of a Storage Node's memory reserved for heap space for all RN processes that the SN hosts. Default value: 68.
rootDirPath=<path>

The path to the Storage Node's root directory.
rootDirSize=<Long Unit_String>

Sets the storage size of the root directory. However, no run-time checks are performed to verify that the actual directory size is greater than or equal to the size you specify. Use this setting for heterogeneous installation environments where some Storage Nodes have more disk capacity than others. Then, use this parameter only for those Storage Nodes that store data in the root directory (not recommended).

The value that you specify for this parameter must be a long, followed optionally by a unit string. Accepted unit strings are: KB, MB, GB, and TB, corresponding to 1024, 1024^2, 1024^3, 1024^4, respectively. Acceptable strings are case insensitive. Valid delimiters between the long value and the unit string are " ", "-", or "_".

Note:

The rootDirSize parameter is intended for backward compatibility with older installations that were created without specifying the -storagedir parameter. We strongly recommend not storing data in your root directory. See Managing Storage Directory Sizes. However, if you do specify a -rootDirPath parameter, you must also specify -rootDirSize. If you are trying to change parameter settings (plan change-parameters), and do not specify both parameters, a warning is displayed.

Do not use the rootDir parameter if a Storage Nodes uses some other directory (such as you can specify using plan change-storagedir).

The following store-wide parameter settings apply to statistics files and performance files, as well as the service debug logs across all Storage Nodes, Replication Nodes, Admins, and Arbiters. The associated Storage Node Agent must be restarted to reflect any changes in the settings.

serviceLogFileCount=<Integer>

Sets the number of log files kept by this Storage Node, and for all the Replication Nodes it hosts. This default value is 20. Limiting the number of log files controls the amount of disk space devoted to logging history. If the value is less than 1, then it is converted to 1.
serviceLogFileLimit=<Integer>

Limits the size of each log file. After reaching this size, the logging subsystem starts a new log file. This setting applies to the Storage Node and to all Replication Nodes that it hosts. The default value is 2,000,000 bytes. The limit specifies an approximate maximum amount of bytes written to any one file. If the value is lesser than or equal to 0, then there is no limit to the size of the service log files.
serviceLogFileCompression=<Boolean>

Enables the compression of the log files to store significantly more logging output in the same amount of disk space. By default, the compression is disabled. You can enable log file compression by setting the parameter to true.

When you enable compression, the adminLogFileLimit and serviceLogFileLimit parameters are auto-adjusted to retain larger log files than the specified size. With the default value of the maximum file count, the actual size limit is approximately five times larger than the specified limit.

Note:
The size of the log files may temporarily exceed the defined limits in certain cases.
servicePortRange=<String>

Sets the range of ports used for communication among administrative services running on a Storage Node and its managed services. This parameter is optional. By default the services use anonymous ports. The format of the value string is "startPort,endPort."

The range needs to be large enough to accommodate the Storage Node, all the Replication Nodes (as defined by the capacity parameter), Admin and Arbiter services hosted on the machine, and JMX, if enabled. The number of ports required also depends on whether the system is configured for security, which is the default. For a non-secure system, the Storage Node consumes 1 port (shared with the port assigned separately for the Registry Service, if it overlaps the service port range), and each Replication Node consumes 1 port in the range. An Admin, if configured, consumes 1 port. Arbiters consume 1 port each. If JMX is enabled, that consumes 1 additional port. On a secure system, two additional ports are required for the Storage Node, and two for the Admin. As a general rule, we recommend that you specify a range significantly larger than the minimum. More available ports allows for increases in Storage Node capacity, or network problems that can render ports temporarily unavailable.

The ports that you specify in the servicePortRange should not overlap with the Admin port or with haPortRange. The service port range can include the registry port, so the registry and Storage Node share a port.

For deploying a secure Oracle NoSQL Database, use the following formula to estimate the port range size number, adding an additional port for each Storage Node, Replication Node or the Admin (if configured):
```
 3 (Storage Nodes) +
capacity (the number of Replication Nodes) +
Arbiters (the number of Arbiter Nodes) +
3 (if the Storage Node is hosting an admin) +
1 (if the Storage node is running JMX) 
```
For more information on configuring Oracle NoSQL Database securely, see Security Guide.

For a non-secure system, use the following formula to estimate the port range size number:
```
1 (Storage Node) +
capacity (the number of Replication Nodes) +
Arbiters (the number of Arbiter Nodes) +
1 (if the Storage Node is hosting an admin) +
1 (if the Storage Node is running JMX)
```
For example, if a Storage Node has capacity 1, is hosting an Admin process, and neither Arbiters nor JMX are in use, the range size must be at least 3. You can increase the range size beyond this minimum, for safety and Storage Node expansion. Then, if you expand the Storage Node, you will not need to make changes to this parameter. If capacity is 2, the range size must be greater than or equal to 4.

Replication Node Parameters

The following parameters can be set for Replication Nodes:

cacheSize=<Long>

Sets the cache size in the underlying BDB JE subsystem. The units are bytes. The size is limited by the java heap size, which in turn is limited by the amount of memory available on the machine. You should only ever change this low level parameter under explicit directions from Oracle support.
collectEnvStats=<Boolean>

If true, then the underlying BDB JE subsystem dumps statistics into the .stat file. This information is useful for tuning JE performance. Oracle Support may request these statistics to aid in tuning or to investigate a problem.
configProperties=<String>

Contains property settings for the underlying BDB JE subsystem. Its format is property=value;property=value....
enableErasure=<Boolean>

If true, then erasure is enabled for the underlying storage system. Erasure periodically wipes the obsolete data (i.e. delete data, older versions of updated records, expired data) from the storage layer by zeroing out the corresponding records. Erasure can be enabled or disabled without restarting the database. Erasure is enabled by default.

Default value is true.
erasurePeriod=<Long Timeunit>

The duration for one complete erasure pass over the entire data set (the cycle time). Erasure is throttled based on this value, to minimize its impact on performance. It is recommended that erasure period be set to less than half of the duration one expects the obsoleted data to stay around. In other words, we recommend two erasure cycles to remove the obsoleted data. For example, if we intend to remove all obsolete data in 30 days, then erasure period can be set to 14 days.

Default value is "6 DAYS".
javaMiscParams=<String> [deprecated]

This parameter should ONLY be used to set flags for which there are no service parameters.

The javaMiscParams parameter is deprecated. You are encouraged not to use it, and instead use the javaRnParamsOverride parameter.
javaRnParamsOverride=<String>

A string that is added to the command line when the Replication Node process is started. This parameters is intended for specifying miscellaneous JVM properties that cannot be specified using other RN parameters. If the string is not a valid sequence of tokens for the JVM command line, the Admin process fails to start.

No default value is provided for this parameter.

It is recommended that to specify the heap sizes for Replication Nodes you use Storage Node's memoryMB and other JVM parameters. For more information about these parameters see, Storage Node Parameters.
latencyCeiling=<Integer>

If the Replication Node's average latency exceeds this number of milliseconds, it is considered an "alertable" event. If JMX monitoring is enabled, the event also causes an appropriate notification to be sent.
loggingConfigProps=<String>

Contains property settings for the Logging subsystem. The format of this string is like that of configProperties, above. Standard java.util.logging properties can be set by this parameter.
maxTrackedLatency=<Long TimeUnit>

The highest latency that is included in the calculation of latency percentiles.
rnCachePercent=<Integer>

The portion of an RN's memory set aside for the JE environment cache.
rnStatisticsEnabled=<Boolean>

If true, then the Replication Nodes gather key distribution statistics.
rnStatisticsGatherInterval=<Long TimeUnit>

The time interval at which Replication Nodes should gather distribution statistics.
rnStatisticsTTL=<Long DaysOrHours>

Specifies the duration for which the key distribution statistics should be retained in the system tables. The duration specified must be in days or hours. By default, these statistics are retained for 60 days.
rnStatisticsIncludeStorageSize=<Boolean>

If true, then the information on storage sizes are included when gathering key distribution statistics.
throughputFloor=<Integer>

Similar to latencyCeiling, throughputFloor sets a lower bound on Replication Node throughput. Lower throughput reports are considered alertable. This value is given in operations per second.

Arbiter Node Parameters

The following parameters can be set for Arbiter Nodes:

javaAnParamsOverride=<String>

A string that is added to the command line when the Arbiter Node process is started. This parameters is intended for specifying miscellaneous JVM properties. If the string is not a valid sequence of tokens for the JVM command line, the Admin process fails to start.

No default value is provided for this parameter.

It is recommended that to specify the heap sizes for Arbiter Nodes you use Storage Node's memoryMB and other JVM parameters. For more information about these parameters see, Storage Node Parameters.

Global Parameters

The following store-wide non-security parameters can be implicitly and uniformly set across all Storage Nodes, Replication Nodes and Admins:

collectorInterval =<Long TimeUnit>

Sets the collection period for latency statistics at each component. This value defaults to 20 seconds. Values like average interval latencies and throughput are averaged over this period of time.

The following store-wide parameters can be set for the debug log files:

adminLogFileCount=<Integer>

Sets the number of log files that are kept. This value defaults to 20. It is used to control the amount of disk space devoted to logging history. If the value is less than 1, then it is converted to 1.
adminLogFileLimit=<Integer>

Limits the size of log files. After reaching this limit, the logging subsystem switches to a new log file. This value defaults to 4,000,000 bytes. The limit specifies an approximate maximum amount to write (in bytes) to any one file. If the value is lesser than or equal to 0, then there is no limit to the size of the log files.

Security Parameters

The following store-wide security parameters can be implicitly and uniformly set across all Storage Nodes, Replication Nodes and Admins:

accountErrorLockoutThresholdCount=<Integer>

Number of invalid login attempts for a user account from a particular host address over the tracking period needed to trigger an automatic account lockout for a host. The default value is 10 attempts.
accountErrorLockoutThresholdInterval=<Long TimeUnit>

Specifies the time period over which login error counts are tracked for account lockout monitoring. The default value is 10 minutes.
accountErrorLockoutTimeout=<Long TimeUnit>

Time duration for which an account will be locked out once a lockout has been triggered. The default value is 30 minutes.
loginCacheTimeout=<Long TimeUnit>

Time duration for which KVStore components cache login information locally to avoid the need to query other servers for login validation on every request. The default value is 5 minutes.
sessionExtendAllowed=<Boolean>

Indicates whether session extensions should be granted. Default value is true.
sessionTimeout=<Long TimeUnit>

Specifies the length of time for which a login session is valid, unless extended. The default value is 24 hours.

The following password security parameters can be set:

Parameter Name	Value Range and Type	Description
passwordAllowedSpecial	Sub set or full set of #$%&'()*+,-./:; <=>?@[]^_`{\|} (string)~	Lists the allowed special characters.
passwordComplexityCheck	[true\|false] (boolean)	Whether to enable the password complexity checking. The default value is true.
passwordMaxLength	1 - 2048 (integer)	The maximum length of a password. The default value is 256.
passwordMinDigit	0 - 2048 (integer)	The minimum required number of numeric digits. The default value is 2.
passwordMinLength	1 - 2048 (integer)	The Minimum length of a password. The default value is 9.
passwordMinLower	0 - 2048 (integer)	The minimum required number of lower case letters. The default value is 2.
passwordMinSpecial	0 - 2048 (integer)	The minimum required number of special characters. The default value is 2.
passwordMinUpper	0 - 2048 (integer)	The minimum required number of upper case letters. The default value is 2.
passwordNotStoreName	[true\|false] (boolean)	If true, password should not be the same as current store name, nor is it the store name spelled backwards or with the numbers 1–100 appended. The default value is true.
passwordNotUserName	[true\|false] (boolean)	If true, password should not be the same as current user name, nor is it the user name spelled backwards or with the numbers 1-100 appended. The default value is true.
passwordProhibited	list of strings separated by comma (string)	Simple list of words that are not allowed to be used as a password. The default reserved words are: oracle,password,user,nosql.
passwordRemember	0 - 256 (integer)	The maximum number of passwords to be remembered that are not allowed to be reused when setting a new password. The default value is 3.

For more information on top-level, transport, and password security parameters see the Security Guide.

Admin Restart

Changes to the following Oracle NoSQL Database parameters will result in a Admin restart by the Storage Node Agent:

Admin parameters:

adminHttpPort
adminLogFileCount
adminLogFileLimit
configProperties
javaAdminParamsOverride
javaMiscParams (deprecated)
loggingConfigProps

For example:

kv-> plan change-parameters -all-admins 
-params adminLogFileCount=10
Started plan 14. Use show plan -id 14 to check status.
        To wait for completion, use plan wait -id 14
kv-> show plan -id 14
Plan Change Admin Params (14)
Owner: null
State:                 INTERRUPTED
Attempt number:        1
Started:               2013-08-26 20:12:06 UTC
Ended:                 2013-08-26 20:12:06 UTC
Total tasks:           4
 Successful:           1
 Interrupted:          1
 Not started:          2
Tasks not started
   Task StartAdmin start admin1
   Task WaitForAdminState waits for Admin admin1 to reach RUNNING state
kv-> plan execute -id 14
Started plan 14. Use show plan -id 14 to check status.
        To wait for completion, use plan wait -id 14
kv-> show plan -id 14
Plan Change Admin Params (14)
State:                 SUCCEEDED
Attempt number:        1
Started:               2013-08-26 20:20:18 UTC
Ended:                 2013-08-26 20:20:18 UTC
Total tasks:           2
 Successful:           2

Note:

When you change a parameter that requires an Admin restart using the plan change-parameters command, the plan ends in an INTERRUPTED state. To transition it to a SUCCESSFUL state, re-issue the plan a second time using the plan execute -id <id> command.

Replication Node Restart

The Storage Node Agent must be restarted to reflect any changes in the setting of the following parameters.

Storage Node parameters:

serviceLogFileCount
serviceLogFileLimit
serviceLogFileCompression
servicePortRange

Replication Node parameters:

configProperties
javaMiscParams (deprecated)
javaRnParamsOverride
loggingConfigProps