Oracle NoSQL Database Change Log
Release 11gR220.127.116.11 Enterprise Edition
Oracle NoSQL Database 2.0 has moved to Admin version 2. This is an on-disk
format change which affects internal data stored by the Admin services. The
change is forward compatible in that Admin services deployed using 2.0 can read
data created by older releases. The change is not backwards compatible in that
Admin services which have been deployed with the new release cannot be
restarted using older NoSQL releases.
See the section
Updating an Existing Oracle NoSQL Database Deployment in the Admin
Changes in 11gR18.104.22.168 Enterprise Edition
An integration with Oracle Coherence has been provided that allows
Oracle NoSQL Database to be used as a cache for Oracle Coherence
applications, also allowing applications to directly access cached
data from Oracle NoSQL Database. This integration is a feature of the
Enterprise Edition of the product and implemented as a new, independent jar
file. It requires installation of the Oracle Coherence product as
well. The feature is described in the
Guide as well as the javadoc. [#22291].
The Enterprise Edition now has support for semantic technologies.
Specifically, the Resource Description Framework (RDF), SPARQL query
language, and a subset of the Web Ontology Language (OWL) are now
supported. These capabilities are referred to as the RDF Graph
feature of Oracle NoSQL Database. The RDF Graph feature provides a
Java-based interface to store and query semantic data in Oracle NoSQL
Database Enterprise Edition. The feature is described in the
RDF Graph manual.
The preferred approach for setting NoSQL DB memory resources is to
specify the memory_mb parameter for each SN when running the
makebootconfig utility, and to let the system calculate the ideal
Replication Node heap and cache sizes. However, it is possible to override the
standard memory configurations by explicitly setting heap and cache
sizes using the Replication Node javaMiscParams and cacheSize
parameters. In past releases, setting the explicit values worked
correctly when using the plan change-parameters command, but did not
work correctly when using the change-policy command. This has been
fixed, so that if desired, one can use the change-policy command for
the javaMiscParams and cacheSize parameters to override the default
memory allocation heuristics. [#22097]
A NoSQL DB deployment that executes on a node with no network
available, as might happen when running a NoSQL DB demo or tutorial,
would fail with this error:
java.net.InetAddress.getLocalHost() returned loopback address:<hostname> and
no suitable address associated with network interfaces.
This has been fixed. [#22252]
Prior to this release, if a write operation encountered an exception from the underlying persistent store indicating that the write completed on the shard's master but not necessarily on the desired combination of replicas within the specified time interval, then that exception would be swallowed and thus never propagated to the client. Originally, this behavior was considered desirable because not only is that exception rare (because of various preceding checks performed by the implementation), but swallowing the exception would keep the API simple by avoiding the introduction of an additional exception and/or additional communication at the API level. After further thought and discussion, the team concluded that clients should know when a write operation fails to complete because of such an exception. As a result, when such a condition occurs during a write operation, a
RequestTimeoutException will now be propagated to the client; wrapping the original exception from the underlying persistent store as the cause. For additional information, including strategies one might employ when this exception is encountered, refer to the
This has been fixed. [#21210]
A new parameter has been added which controls the display of records
in exception and error messages. When
hideUserData is set
to true, as it is by default, error messages which are printed to the
server side logs or are displayed via the show CLI commands replace
any key/values with the string "[hidden]". To see the actual record content
in errors, set the parameter to false. [#22376]
In previous releases, information about errors that occurred during
NoSQL DB component start up as a result of a
deploy-topology command would often be visible only within
the NoSQL DB logs, which made
installation troubleshooting difficult. In this
release, such start up errors can now be seen via the Admin CLI
show plan -id <id> command. [#22101]
The Storage Node Agent exposes MBeans on a non-default MBeanServer
instance. In this release, the non-default MBeanServer now exposes
the standard JVM platform MBeans as well as those relating only to
Oracle NoSQL Database.
In both SNMP and JMX interfaces, the new totalRequests metric is now available. This metric counts the number of multi-operation sequences that occurred during the sampling period.
Prior to this release, the product was compiled and built against the 1.x version of Hadoop (CDH3). Thus, when employing a previous release, if one were to run the
examples.hadoop.CountMinorKeys example against a cluster based on the 2.x version of Hadoop (CDH4), the MapReduce job initiated by that example would fail as a result of an
IncompatibleClassChangeError; which is caused by an incompatibility introduced in
org.apache.hadoop.mapreduce.JobContext between Hadoop 1.x and Hadoop 2.x. This failure occurs whether the example is compiled and built against Hadoop 1.x or Hadoop 2.x. Because the product's customer base almost exclusively uses Hadoop 2.x, this release will provide support for Hadoop 2.x instead of 1.x. Future releases may revisit support for both Hadoop version paths, but doing so will involve refactoring the codebase and its associated release artifacts, as well as substantial changes to the product's current build process.
Support of Hadoop 2.x (CDH4) has been provided. [#22157]
The java -jar kvstore.jar makebootconfig -mount flag has been changed
to -storagedir. The "plan change-mountpoints -path <storage
directory>" command is deprecated in favor of "plan
change-storagedir -storagedir <storage directory>". [#21880]
The concept of Storage Node capacity is better explained in the
The Administrator's Guide has a revamped section on how to calculate the
resources needed for operating a NoSQL DB deployment.
Changes in 11g.r22.214.171.124
This release adds the capability to remove an Admin service replica.
If you have deployed more than one Admin, you can remove one of them
using the following command:
plan remove-admin -admin <adminId>
You cannot remove the sole Admin if only one Admin instance is
For availability and durability reasons, it is highly recommended that
you maintain at least three Admin instances at all times. For that
reason, if you try to remove an Admin when the removal would result in
there being fewer than three, the command will fail unless you give
the -force flag.
If you try to remove the Admin that is currently the master,
mastership will transfer to another Admin. The plan will be
interrupted, and subsequently can be re-executed on the new master
Admin. To re-execute the interrupted plan, you would use this command:
plan execute -id <planId>
The Admin CLI verify has an added check to verify that the Replication
Nodes hosted on a single Storage Node have memory settings that fit
within the Storage Node's memory budget. This guards against mistakes
that may occur if the system administrator overrides defaults and
manually sets Replication Node heap sizes.[#21727]
The Admin CLI verify command now labels any verification issues as
violations or notes. Violations are of greater importance, and the
system administrator should determine how to adjust the system to
address the problem. Notes are warnings, and are of lesser
Several corrections were made to latency statistics. These corrections apply
to the service-side statistics in the Admin console, CLI
command, .perf files and .csv files, as well as the client-side statistics
returned by KVStore.getStats. However, corrections to the 95% and 99% values do
not apply to the client-side statistics, since these values do not appear in
the client-side API.
- The definition of latency has been corrected for the "multi"
operation requests (multiGet, multiDelete, execute, etc). These are
labeled "multi" in the
Op Type column where latency
information is displayed. The previous definition was "latency in
milliseconds per operation" while the new definition is "latency
in milliseconds per request". In other words, for a "multi"
operation request, latency now applies to the entire request rather than
to each operation. For "single" operation requests, the definition of
latency has not changed.
- To go along with the change above, a new column containing the number
of requests in the sample has been added to all latency information
TotalReq. This is also available for client-side
statistics using the new
method. For "multi" operation requests, the total number of requests is
normally smaller than the total number of operations (the
TotalOps column). For "single" operation requests, the
total number of requests and operations are equal.
- Improved the consistency of the values reported in each sample so
that, for example, the minimum latency is always less than the maximum
latency. However, note that statistics are collected without
synchronization to avoid impacting performance, and for small sample
sizes the values in a sample are not always accurate or self-consistent.
- Fixed a bug that caused the 95% and 99% values to show the maximum
latency recorded (within 1000 ms), rather than the lowest 95% or 99% as
intended. This bug only applied to the "multi" operation requests.
- Fixed a bug that caused the 95% and 99% values to sometimes
mistakenly appear as -1. These values should only appear as -1 when
there were no operations in the sample with a latency below 1000 ms.
Modified the Administration Process to allocate ports from within a port range
if one is specified by the -servicerange argument to
the makebootconfig utility. If the argument is not specified the
Administration Process will use any available port. Please see
the Admin Guide
for details regarding the configuration of ports used by Oracle NoSQL
Modified the replication node to handle the unlikely case that the locally
stored topology is missing. A missing topology results in a
java.lang.NullPointerException being thrown in the TopologyManager and will
prevent the replication node from starting. [#22015]
Replication Node memory calculations are more robust for Storage Nodes
that host multiple Replication Nodes. In previous releases, using the
plan change-params command to reduce the capacity parameter for a
Storage Node which hosts multiple Replication Nodes could result in an
over aggressive increase in RN heap, which would make the Replication
Nodes fail at start up. The problem would be fixed when a topology was
rebalanced, but until that time, the Replication Nodes were
unavailable. The default memory sizing calculation now factors in the
number of RNs resident on a Storage Node, and adjusts RN heap sizes as
Replication Nodes are relocated by the deploy-topology
Fixed a bug that could cause a NullPointerException, such as the one below,
during RN start-up. The exception would appear in the RN log and the RN would
fail to start. The conditions under which this problem occurred include
partition migration between shards along with multiple abnormal RN shutdowns.
If this bug is encountered, it can be corrected by upgrading to the current
release, and no data loss will occur.
Exception in thread "main" com.sleepycat.je.EnvironmentFailureException: (JE
5.0.XX) ... last LSN=.../... LOG_INTEGRITY: Log information is incorrect,
problem is likely persistent. Environment is invalid and must be closed.
Caused by: java.lang.NullPointerException
... 10 more
Fixed a bug that causes excess memory to be used in the storage engine cache on
an RN, which could result in poor performance as a result of cache eviction and
additional I/O. The problem occurred only when the
method was used. [#21973]
The replicas in a shard now dynamically configure the JE property
RepParams.REPLAY_MAX_OPEN_DB_HANDLES which controls the size of the cache
used to hold database handles during replication. The cache size is determined
dynamically based upon the number of partitions currently hosted by the
shard. This improved cache sizing can result in better write performance for
shards hosting large numbers of partitions. [#21967]
The names of the client and server JAR files no longer include release
version numbers. The files are now called:
This change should reduce the amount of work needed to switch to a new
release because the names of JAR files will no longer change between
releases. Note that the name of the installation directory continues to
include the release version number. [#22034]
A SEVERE level message is now logged and an admin alert is fired when the
storage engine's average log cleaner (disk reclamation) backlog increases over
time. An example of the message text is below.
121215 13:48:57:480 SEVERE [...] Average cleaner backlog has grown from 0.0 to
6.4. If the cleaner continues to be unable to make progress, the JE cache size
and/or number of cleaner threads are probably too small. If this is not
corrected, eventually all available disk space will be used.
For more information on setting the cache size appropriately to avoid such
problems, see "Determining the Per-Node Cache Size" in the Administrator's
The storage engine's log cleaner will now delete files in the latter portion of
the log, even when the application is not performing any write operations.
Previously, files were prohibited from being deleted in the portion of the log
after the last application write. When a log cleaner backlog was present (for
example, when the cache had been configured too small, relative to the data set
size and write rate), this could cause the cleaner to operate continuously
without being able to delete files or make forward progress. [#21069]
NoSQL DB 2.0.23 introduced a performance regression over R1.2.23. The
kvstore client library and Replication Node consumed a greater
percentage of system CPU time. This regression has been fixed. [#22096]
Changes in 11gR126.96.36.199
This release provides the ability to add storage nodes to the system
after it has been deployed. The system will rebalance and redistribute
the data onto the new nodes without stopping operations. See Chapter
6, of the Admin
your Store's Configuration, for more details.
oracle.kv.lob package provides operations that can
be used to read and write Large Objects (LOBs) such as audio and video
files. As a general rule, any object larger than 1 MB is a good
candidate for representation as a LOB. The LOB API permits access to
large values without having to materialize the value in its entirety
by providing streaming APIs for reading and writing these objects.
A C API has been added. The implementation uses Java JNI and requires
a Java virtual machine to run on the client. It is available as a
Added a new remove-storagenode plan. This command will
remove a storage node which is not hosting any NoSQL Database components
from the system's topology. Two examples of when this might be useful
A storage node was incorrectly configured, and cannot be deployed.
A storage node was once part of a NoSQL Database, but all components have
been migrated from it using the migrate-storagenode command, and the
storage node should be decommissioned.
Added the ability to specify additional physical configuration
information about storage nodes including:
This information is used by the system to make more intelligent
choices about resource allocation and consumption. The administration
documentation discusses how these parameters are set and used.
- Capacity - the number of RepNodes the SN may host
- Number of CPUs
- Amount of memory to use
- Specific directory paths (mount points) to use for RepNodes
- Added Avro support. The value of a kv pair can now be stored in Avro
binary format. An Avro schema is defined for each type of data stored. The
Avro schema is used to efficiently and compactly serialize the data, to
guarantee that the data conforms to the schema, and to perform automatic
evolution of the data as the schema changes over time. Bindings are supplied
that allow representing Avro data as a POJO (Plain Old Java Object), a JSON
object, or a generic Map-like data structure. For more information, see
Chapter 7 - Avro Schemas and
Chapter 8 - Avro Bindings
in the Getting Started Guide. The
oracle.kv.avro package is
described in the Javadoc. The use of the Avro format is strongly
recommended. NoSQL DB will leverage Avro in the future to provide additional
features and capabilities. [#21213]
- Added Avro support for the Hadoop
oracle.kv.hadoop.KVAvroInputFormat class returns Avro
IndexedRecords to the caller. When this class is used in
conjunction with Oracle Loader for Hadoop, it is possible to read data
directly from NoSQL Database using OLH without using an interim Map-Reduce
job to store data in HDFS. [#21157]
- Added a feature which allows Oracle Database External Tables to be
used to access Oracle NoSQL Database records. There is more
information in javadoc for the
oracle.kv.exttab package and an "cookbook" example in the
examples/externaltables directory. [#20981]
The following new methods:
have been added to allow clients to configure the socket timeouts used to make
client requests. Please review the javadoc for details.
R1 installations must ensure that the software on the storage nodes has
been upgraded as described in the upgrade documentation
accompanying this release before using the above APIs on the client.
New service parameters have been added to control the backlog
associated with sockets created by NoSQL Database. These are
controllable for the Rep Node and Storage Nodes' Monitor, Admin, and
Registry Handler interfaces. The parameters
rnMonitorSOBacklog (default 0), rnAdminSOBacklog
rnAdminSOBacklog (default 0),
snMonitorSOBacklog (default 0), and
snRegistrySOBacklog (default 1024).
Key.isPrefix with an argument containing
a smaller major or minor path than the target Key object caused an
IndexOutOfBoundsException in certain cases. This has been fixed.
KeyRange() constructor now checks that the start
Key is less than the end
Key if both are specified,
IllegalArgumentException is thrown.
KeyRange also has
fromString() methods for encoding and decoding
KeyRange instances, similar to the same methods in
Many new commands have been added to the CLI. See
Appendix A - Command Line Interface (CLI) Command Reference
of the Administrator's Guide
The Admin Console is now for monitoring only.
Administration CLI commands have been changed so that component ids
match the ids used in the topology display. Previously Datacenters,
Storage Nodes, Admin instances and Replication Nodes were identified
only by number. For example, the syntax to add Storage Node 17 to a
Storage Node pool, or to show the parameters for a given Replication Node was:
Datacenters can now be expressed as # or dc#
joinPool myStorageNodePool 17
show repnode-params 5,3
Admin instances can now be expressed as # or admin#
Storage Nodes can now be expressed as # or sn#
Replication Nodes can now be expressed as groupNum,nodeNum, or rgX-rnY
The commands shown above are still valid, but can also be expressed as:
joinPool myStorageNodePool sn17
show repnode-params rg5-rn3
The javadoc for the
Key.createKey methods has been improved to
warn that List instances passed as parameters are owned by the Key object
after calling the method. To avoid unpredictable results, they must not be
Changes in 11gR188.8.131.52
Previously, executing a change-repnode-params plan in order to
change Replication Node parameters for a node other than the one
running the Admin service would fail. This operation will now
A deploy-storage-node plan which ran into problems when attempting
to deploy a new storage node would leave the problematic SN in the
store. This would require that the user either take manual action to
remove the bad SN, or fix the problem and retry the plan. For
convenience, the deploy-storage-node plan will now clean up if it
runs into errors, and will not leave the failed SN behind. [#20530]
The command line interface's
snapshot create command
has been made significantly faster. Previously, it could take
minutes if executed on a store with a large amount of data. This
should be reduced to seconds. [#20772]
The two scripts for starting kvlite and executing control commands,
bin/kvctl, have been replaced
java -jar lib/kvstore-M.N.P.jar command. This provides portability
to all Java platforms, including Windows. The two scripts are deprecated, but
will be supported for at least one release cycle.
The translation from the old script commands to the new -jar commands is as
|Old script command||New -jar command|
java -jar lib/kvstore-M.N.P.jar kvlite args...
bin/kvctl command args...
java -jar lib/kvstore-M.N.P.jar command args...
There are a few differences to be aware of between the old and new commands.
nohup, if desired, must be explicitly specified. In
bin/kvctl script, nohup was added automatically for the
restart commands. To specify the
equivalent command, use:
nohup java -jar lib/kvstore-M.N.P.jar start args... > /dev/null <
/dev/null 2<&1 &
The logging configuration file for kvlite is now specified using
standard Java syntax. Previously, the
examples/logging.properties configuration file was added
automatically when passing
-logging to the
run-kvlite.sh script. The new equivalent is:
-jar lib/kvstore-M.N.P.jar kvlite args...
-host argument defaulted to
the local machine name (via the
`hostname` command) when
kvctl script. Now, for all control commands, no
default hostname is used and the
-host argument must be
specified explicitly. This change was made for two reasons: 1)
consistency, since the port and other arguments have no default value for
control commands, and 2) safety, since specifying an explicit hostname
guards against accidental errors.
-host argument defaulted to
localhost when running the
Now, the default is the local machine name rather than (literally)
localhost. Note that the kvlite command, unlike the control
commands, has default values for all arguments. This is because the kvlite
command is designed for ease-of-use during development on a single machine.
kvlite should not be used in production or for performance testing.
java -jar lib/kvstore-M.N.P.jar, with or
without arguments, printed the product version. Now, if no arguments are
specified, a usage message is printed. To print the version, use the
java -jar lib/kvstore-M.N.P.jar version