Changes in 21.2.19

Topics

New Features

An off-heap cache is no longer used by Oracle NoSQL Database. The off-heap cache did not provide the performance benefits we had expected, and in some cases had a negative performance impact, so it has been removed.
As part of this change, the systemPercent parameter, which was previously used to configure the size of the off-heap cache, has been made obsolete. The parameter may still be specified, but it will be ignored.

For stores with configurations where memory was allocated to the off-heap cache, that memory is now available for file system caching in the system buffer pool. In other words, data formerly stored in the off-heap cache will now be cached in the system buffer pool.

[KVSTORE-927]
Implemented unnesting of arrays and maps.
The FROM clause has been extended to allow a list of expressions after the table or tables referenced by the query. Conceptually, for each table row, the expressions are evaluated, and a cross product is formed by the current row and the results of these evaluations. The result of the whole FROM clause is a table containing the union of all these cross products.
Although any kind of expression may appear in the FROM clause, the primary purpose of this feature is to unnest arrays (or maps) that appear in table rows. The main reason to support nesting is to group by fields that are contained in the arrays or maps. As an example, consider the following table that stores data from a TV streaming service:
```
create table stream_acct(acct_id integer, value json, primary key(acct_id))
```
A sample row of this table looks like this:
```
{
  "acct_id": 1,
  "value": {
    "firstName" : "John",
    "lastName" : "Sanders",
    "country" : "USA",
    "contentStreamed": [
      {
        "showName": "Call My Agent",
        "showId": 15,
        "showType": "tvseries",
        "numSeasons" : 2,
        "seriesInfo": [
          {
            "seasonNum" : 1,
            "numEpisodes" : 2,
            "episodes": [
              { "episodeID" : 20, "lengthMin" : 45, "minWatched" : 45 },
              { "episodeID" : 30, "lengthMin" : 42, "minWatched" : 42 }
            ]
          },
          {
            "seasonNum": 2,
            "numEpisodes" : 2,
            "episodes": [
              { "episodeID" : 20, "lengthMin" : 50, "minWatched" : 50 },
              { "episodeID" : 30, "lengthMin" : 46, "minWatched" : 46 }
            ]
          }
        ]
      },
      {
        "showName": "Rita",
        "showId": 16,
        "showType": "tvseries",
        "numSeasons" : 1,
        "seriesInfo": [
          {
            "seasonNum" : 1,
            "numEpisodes" : 2,
            "episodes": [
              { "episodeID" : 20, "lengthMin" : 65, "minWatched" : 65 },
              { "episodeID" : 30, "lengthMin" : 60, "minWatched" : 60 }
]
          }
        ]
      }
    ]
  }
}
```
The following query orders the shows in USA according to their "popularity", i.e., how many users have shown an interest in them.
```
select $show.showId, count(*) as cnt
from stream_acct $s, unnest($s.value.contentStreamed[] as $show)
where $s.value.country = "USA"
group by $show.showId
order by count(*) desc
```
The above query can be optimized if the following index is created:
```
create index idx_country_showid on stream_acct(
    value.country as string,
    value.contentStreamed[].showId as integer) with unique keys per row
```
The query will use this index. The country condition will be pushed to the index, the group-by will be index-based, and the index is a covering one for this query. Notice that, for the index to be used, the index keys extracted per row must not contain any duplicates. This constraint is enforced by creating the index as "with unique keys per row".

Notice that the unnest clause in the above query (and in general) is not strictly necessary; it can be omitted without changing the semantics (and the results) of the query. However, if it is omitted, the index will not used. This is because the unnest clause puts some restrictions on the kind of expressions it wraps, and these restrictions make it possible for the query processor to find that an index on the arrays/maps that are being unnested is applicable and rewrite the query accordingly.

[KVSTORE-822]
Added seq_distinct() function. It returns the distinct values of the sequence produced by its input expression.
As an example, the following query returns, for each area code, the number of users having phone numbers in that area code. The query uses the seq_distinct() because a user may have multiple phone numbers in the same area code, but he or she must be counted only once per area code.
```
select $areacode, count(*) as cnt
from Users u, seq_distinct(u.address.phones[].areacode) as $areacode
group by $areacode
```
[KVSTORE-822]
In comparison operators, when one operand is a timestamp and the other a string, the string operand is now implicitly cast to a timestamp with the same precision as the precision of the timestamp operand. An exception is thrown if the cast fails.
[KVSTORE-1016]

Enhanced support for nested seq_transform() expressions.

seq_transform() is useful for transforming JSON documents stored in table rows. In such situations it is often the case that multiple seq_transform() expressions must be used, nested into each other. Furthermore, inner seq_transform()expressions may need to access the "current" input item of outer seq_transform() expressions. Until now this was not possible. In this release we enhance seq_transform(), so that each such expression creates an appropriately named context-item variable that can be referenced by inner seq_transform() expressions.

For example, consider the following table that records information about air travel luggage:

create table luggage (
  ticketNo string,
  passengerName string,
  bagInfo json,
  primary key(ticketNo))

A sample row for this table is shown below

{
  "ticketNo" : "1762352483606",
  "passengerName" : "Willie Hernandez",
  "bagInfo" : [
    {
      "tagNum" : "17657806243915",
      "routing" : "SFO/AMS/HER",
      "lastActionCode" : "offload",
      "lastSeenStation" : "HER",
      "lastSeenTimeGmt" : "2019-03-13T15:19:00",
      "flightLegs" : [
        {
          "flightNo" : "BM604",
          "flightDate" : "2019-03-12T20:00:00",
          "fltRouteSrc" : "SFO",
          "fltRouteDest" : "AMS",
          "estimatedArrival" : "2019-03-13T08:00:00",
          "actions" : [
            { "at":"SFO", "action":"TagScan", "time":"2019-03-12T18:14:00" },
            { "at":"SFO", "action":"onload",  "time":"2019-03-12T19:20:00" },
            { "at":"AMS", "action":"offload", "time":"2019-03-13T08:30:00" }
          ]
        },
        {
          "flightNo" : "BM667",
          "flightDate" : "2019-03-13T11:14:00",
          "fltRouteSrc" : "AMS",
          "fltRouteDest" : "HER",
          "estimatedArrival" : "2019-03-13T15:00:00",
          "actions" : [
            { "at":"AMS", "action":"TagScan", "time":"2019-03-13T10:45:00" },
            { "at":"AMS", "action":"onload",  "time":"2019-03-13T10:50:00" },
            { "at":"HER", "action":"offload", "time":"2019-03-13T15:19:00" }
          ]
        }
      ]
    },
    {
      "tagNum" : "17657806244523",
      "routing" : "SFO/AMS/HER",
      "lastActionCode" : "offload",
      "lastSeenStation" : "AMS",
      "lastSeenTimeGmt" : "2019-03-13T08:35:00",
      "flightLegs" : [
        {
          "flightNo" : "BM604",
          "flightDate" : "2019-03-12T20:00:00",
          "fltRouteSrc" : "SFO",
          "fltRouteDest" : "AMS",
          "estimatedArrival" : "2019-03-13T08:00:00",
          "actions" : [
            { "at":"SFO", "action":"TagScan", "time":"2019-03-12T18:14:00" },
            { "at":"SFO", "action":"onload",  "time":"2019-03-12T19:22:00" },
            { "at":"AMS", "action":"offload", "time":"2019-03-13T08:32:00"  }
          ]
        }
      ]
    }
  ]
}

Let's say we want to write a query that will return, for each ticketNo, a flat array containing all the actions performed on the luggage of that ticketNo. However, we want only the "at" and "action" fields of each action. Furthermore, we want the flightNo and the tagNum fields to be included with each action. Specifically, for the above sample row, the result should be the following:

{
  "actions" : [
    {"at":"SFO", "action":"TagScan", "flightNo":"BM604", "tagNum":17657806243915},
    {"at":"SFO", "action":"onload",  "flightNo":"BM604", "tagNum":17657806243915},
    {"at":"AMS", "action":"offload", "flightNo":"BM604", "tagNum":17657806243915},
    {"at":"AMS", "action":"TagScan", "flightNo":"BM667", "tagNum":17657806243915},
    {"at":"AMS", "action":"onload",  "flightNo":"BM667", "tagNum":17657806243915},
    {"at":"HER", "action":"offload", "flightNo":"BM667", "tagNum":17657806243915},
    {"at":"SFO", "action":"TagScan", "flightNo":"BM604", "tagNum":17657806244523},
    {"at":"SFO", "action":"onload",  "flightNo":"BM604", "tagNum":17657806244523},
    {"at":"AMS", "action":"offload", "flightNo":"BM604", "tagNum":17657806244523},
  ]
}

Here is the query that will return the above result:

select
  seq_transform(
      l.bagInfo[],
      seq_transform(
          $sq1.flightLegs[],
          seq_transform(
              $sq2.actions[],
              {
                "at" : $sq3.at,
                "action" : $sq3.action,
                "flightNo" : $sq2.flightNo,
                "tagNum" : $sq1.tagNum
              }
          )
      )
  ) as actions
from luggage l

In this query, $sq1 iterates over the elements of the bagInfo array, $sq2 iterates over the elements of the flightLegs array, and $sq3 iterates over the elements of the actions array.

[KVSTORE-1044]

Bug and Performance Fixes

Fixed an issue with KVStats that the NodeMetrics.getAverageLatencyNanos method always returns 10000.
[KVSTORE-996]
Disabled JVM Biased Locking for RNs. Now -XX:-UseBiasedLocking will be explicitly passed to RNs' JVM cmdline. This change should make RNs give more predictable latencies. There should not be any performance degradation due to this change.
[KVSTORE-1047]
The default partition migration concurrency has been increased. Partition migration is done during some topology changes, such as redistribute and rebalance. There are two parameters which limit how may partition migrations can take place concurrently. The default values for these parameters have been increased. Specifically:
- rnPMConcurrentSourceLimit- default changed from 1 to 2
- rnPMConcurrentTargetLimit - default changed from 2 to 4
The higher level of concurrency, in most cases, will reduce the overall time that a topology change takes to complete. The increase should not negatively impact ongoing data operations during a topology change, especially when using SSDs. If the store is running on older hardware with rotating disks, the user may want to explicitly set the parameters to the original default values.

[KVSTORE-1002]
The output of the Admin CLI commands ping and verify configuration now includes information about service start times and state change times.
[KVSTORE-876]
The use of Java RMI in the direct Java driver is now optional. The driver now only uses RMI when accessing servers running software prior to the 21.2 release, or if the driver has been configured to disable use of the async network protocol by calling KVStoreConfig.setUseAsync(false). Because the async network protocol supports concurrent calls on the same socket connection, this change should reduce socket and other resource usage in some cases, particularly for stores with large numbers of shards.
[KVSTORE-340]
The haHostname and haPortRange Storage Node parameters are now read-only. Changing these parameters already had no effect; the new behavior is just intended to make it clear that the parameters should not be changed. The only way to switch to new JE HA parameters is to migrate to a new Storage Node.
[KVSTORE-437]
There is a new flag to the kvlite command that allows users to control the storage size associated with a new kvlite store: -storagedirsizegb size_in_GB.
The default value is 10GB, which has changed from the previous 1GB default. This flag only applies for the initial creation of the store and is ignored when kvlite is started on an existing store.

In addition, the output of kvlite when started on an existing store now properly reflects the values in the existing store.

[KVSTORE-1042]
The StringDef.equals() method now considers subtypes when determining if two fields are equal. The types "STRING", "STRING AS UUID" and "STRING AS UUID GENERATED BY DEFAULT" are now not equal.
[KVSTORE-1048]
Modified the verify configuration Admin CLI command to report version differences as notes, not violations. Version differences are expected during upgrades, so they should not be considered violations.
Modified the non-JSON output format of the verify configuration and ping Admin CLI commands to add spaces after the colon separators that appear at the end of keywords in the output, to improve readability. Also removed spaces from a couple keywords, changing "available storage size" to "availableStorageSize" and "storage type" to "storageType".

Made changes to the non-JSON output format of Admin CLI commands related to login issues to make the output more consistent.

[KVSTORE-448]
Modified the Admin CLI plan deploy-topology command to require that the new topology being deployed reduce the number of outstanding topology violations, if any, rather than just not increase the number. As before, the command can still be forced to deploy a new topology by specifying the -force flag. Also fixed a problem with the way the command counted topology violations that was inaccurate in some cases.
These two changes together fix a problem where deploying a new zone without providing sufficient SNs, or providing ones with insufficiently large storage directory sizes, could result in a topology deployment appearing to succeed without actually making the changes needed to populate the new zone.

[KVSTORE-542]
Removed TLSv1 and TLSv1.1 from the default TLS protocol settings. New security configurations will be created with only TLSv1.2 by default starting with this release.
[KVSTORE-903]
Added two new Rep Node parameters 'enableErasure' and 'erasurePeriod' to control data erasure.
If 'enableErasure' is set to true, then this enables the erasure feature for the underlying BDB JE storage layer. Erasure periodically wipes the obsolete data from the storage layer, by zeroing out corresponding records. The default value is false. Note that only table and key/value data is erased. Table metadata is not subject to erasure.

When erasure is enabled, the value specified for the 'erasurePeriod' parameter specifies the duration for one complete erasure pass over data. Erasure is throttled based on this value, to minimize its impact on performance. By default, it is set to "6 DAYS".

[KVSTORE-1033]

Fixed an issue that causes some BigDataSQL queries to encounter a NullPointerException in certain cases. That issue was characterized by a stacktrace like the following:

Caused by: java.lang.NullPointerException
at java.util.Objects.requireNonNull(Objects.java:203)
at java.util.AbstractCollection.retainAll(AbstractCollection.java:405)
at oracle.kv.impl.query.runtime.RuntimeControlBlock.setPartitionSet(RuntimeControlBlock.java:258)
at oracle.kv.impl.query.runtime.ReceiveIter.open(ReceiveIter.java:1252)
at oracle.kv.impl.api.query.QueryStatementResultImpl$QueryResultIterator.<init>(QueryStatementResultImpl.java:297)
at oracle.kv.impl.api.query.QueryStatementResultImpl.<init>(QueryStatementResultImpl.java:153)
at oracle.kv.impl.api.query.QueryStatementResultImpl.<init>(QueryStatementResultImpl.java:115)
at oracle.kv.impl.api.query.PreparedStatementImpl.executeSyncPartitions(PreparedStatementImpl.java:365)
at oracle.kv.impl.api.KVStoreImpl.executeSyncPartitions(KVStoreImpl.java:3635)
at oracle.kv.hadoop.table.TableRecordReaderBase.getNextOnqlIteratorPartitions(TableRecordReaderBase.java:523)
at oracle.kv.hadoop.table.TableRecordReaderBase.getNextIterator(TableRecordReaderBase.java:500)
at oracle.kv.hadoop.table.TableRecordReaderBase.nextKeyValue(TableRecordReaderBase.java:391)
... 11 more

With this release, the issue above has been fixed and no longer occurs.

[KVSTORE-1123]

Fixed a issue that when there are multiple subscriptions for the Streams API on a shard, changes to the tables on a subscription, like adding or removing tables, would cause the same changes of tables to other subscriptions as well.
[KVSTORE-35]

Fixed a deadlock issue related to SSLDataChannel. The issue will produce stack traces similar to the following in a thread dump:

"ReplicaOutputThread" #829 daemon prio=5 os_prio=0 cpu=8695.34ms elapsed=66577.11s tid=0x000055f030c1f800 nid=0x49f2 waiting for monitor entry  [0x00007fc720171000]
   java.lang.Thread.State: BLOCKED (on object monitor)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelWriteTask.run(SSLDataChannel.java:1425)
	- waiting to lock <0x00000010046df7e0> (a java.nio.HeapByteBuffer)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelWriteTask.access$1100(SSLDataChannel.java:1413)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel.flush(SSLDataChannel.java:590)
    ......
"REPLICA rg15-rn2(2)" #68 daemon prio=5 os_prio=0 cpu=9106.65ms elapsed=68622.76s tid=0x000055f030c55800 nid=0xb24 in Object.wait()  [0x00007fc722b93000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(java.base@11.0.8/Native Method)
	- waiting on <no object reference available>
	at java.lang.Object.wait(java.base@11.0.8/Object.java:328)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelTask.transitToInProgress(SSLDataChannel.java:1335)
	- waiting to re-lock in wait() <0x00000010046e01b8> (a java.lang.Object)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelWriteTask.run(SSLDataChannel.java:1423)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelWriteTask.access$1100(SSLDataChannel.java:1413)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel.doWrap(SSLDataChannel.java:910)
	- locked <0x00000010046df7e0> (a java.nio.HeapByteBuffer)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel.handshake(SSLDataChannel.java:847)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel.unwrap(SSLDataChannel.java:771)
	at com.sleepycat.je.rep.utilint.net.SSLDataChannel.read(SSLDataChannel.java:403)
	......

[KVSTORE-1021]

Fixed a livelock issue related to SSLDataChannel. The issue is observed as an inactive channel with either SSLDataChannel doWrap or doUnwrap. Stack traces similar to the following will be seen in a thread dump:

   java.lang.Thread.State: RUNNABLE
        at java.lang.Object.hashCode(java.base@11.0.6/Native Method)
        at java.util.concurrent.ConcurrentHashMap.replaceNode(java.base@11.0.6/ConcurrentHashMap.java:1111)
        at java.util.concurrent.ConcurrentHashMap.remove(java.base@11.0.6/ConcurrentHashMap.java:1102)
        at com.sleepycat.je.utilint.LoggerUtils.logMsg(LoggerUtils.java:538)
        at com.sleepycat.je.utilint.LoggerUtils.logMsg(LoggerUtils.java:445)
        at com.sleepycat.je.rep.utilint.net.DataChannelFactoryBuilder$ChannelInstanceLogger.log(DataChannelFactoryBuilder.java:417)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelWriteTask.run(SSLDataChannel.java:1636)
        - locked <0x00000000f037cc80> (a java.lang.Object)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelWriteTask.access$1400(SSLDataChannel.java:1613)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel.doWrap(SSLDataChannel.java:990)
	......
   java.lang.Thread.State: RUNNABLE
        at java.lang.Object.notifyAll(java.base@11.0.6/Native Method)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelTask.transitToIdle(SSLDataChannel.java:1513)
        - locked <0x00000000f039e788> (a java.lang.Object)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelReadTask.run(SSLDataChannel.java:1808)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel$ChannelReadTask.access$1800(SSLDataChannel.java:1751)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel.readChannelForBufUnderflow(SSLDataChannel.java:1168)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel.doUnwrap(SSLDataChannel.java:1079)
        - locked <0x00000000f0386f20> (a java.lang.Object)
        - locked <0x00000000f0386ea8> (a java.lang.Object)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel.handshake(SSLDataChannel.java:930)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel.unwrap(SSLDataChannel.java:828)
        at com.sleepycat.je.rep.utilint.net.SSLDataChannel.read(SSLDataChannel.java:441)
    ......

[KVSTORE-1022]

Fixed a problem where RepNode or MasterRebalanceThread in StorageNode may exit unexpectedly due to uncaught InternalFaultException(s) when transient network issues occur. In this case, the logging output shows both of them exit because of SessionAccessException, which is incorrect and has also been fixed.

2021-02-12 10:16:26.575 UTC SEVERE [sn6] MasterRebalanceThread thread exiting due to exception.
oracle.kv.impl.security.SessionAccessException: java.rmi.ConnectIOException: Exception creating connection to:
...

2021-02-12 10:16:26.575 UTC SEVERE [sn6] MasterRebalanceThread thread exiting due to exception.
oracle.kv.impl.security.SessionAccessException: java.rmi.ConnectIOException: Exception creating connection to:
...

[KVSTORE-942]

Fixed a bug with the DROP NAMESPACE command where any existing tables in the namespace will also be dropped but their data remains in the store. Since the tables have been dropped their data can not be accessed or removed.
[KVSTORE-1076]
Fixed a bug that using UPDATE statement with REMOVE clause to remove an element from ARRAY(RECORD) or MAP(RECORD) fails with java.lang.IllegalArgumentException: Error: at (...) Cannot remove fields from records.
[KVSTORE-1200]

Packaging Changes

The packaging of kvclient.jar and kvstore.jar has been changed to include classes from Oracle internal libraries that are not available in public Maven repositories and were previously included as separate JAR files in the distribution. This change is intended to make it easier to create and use a Maven pom.xml file that depends on kvclient.jar since the JAR file now only has dependencies that are available publicly. The libraries included are:
- commonutil.jar
- sklogger.jar
- sdoapi.jar
- sdoutl.jar
- sdodep3prt.jar
In addition, classes from Enterprise Edition-only libraries referenced by kvstore-ee.jar are now directly included in that library, again for simplicity in handling dependencies. The libraries included are the ones associated with the Oracle Wallet:
- osdt_core.jar
- osdt_cert.jar
- osraclepki.jar
Also, the XML build files have been removed from the Community Edition release as it is not currently possible to fully build from source.

[KVSTORE-1100]