Changes in 12cR1.3.3.4

The following changes were made in Oracle NoSQL Database 12cR1.3.3.4.

New Features

  1. A number of new security features have been added.

    Users are now able to enforce table-level access checks through both the API and administrative command line interface (Admin CLI) by using the following new series of table-specific privileges:

    Privilege Description
    READ_ANY_TABLE Read from any table in kvstore
    DELETE_ANY_TABLE Delete data from any table in kvstore
    INSERT_ANY_TABLE Insert and update data to any tables in kvstore
    READ_TABLE Read from a specific table in kvstore
    DELETE_TABLE Delete data from a specific table in kvstore
    INSERT_TABLE Insert and update data to a specific table in kvstore
    CREATE_ANY_TABLE Create any table in kvstore
    DROP_ANY_TABLE Drop any table in kvstore
    EVOLVE_ANY_TABLE Evolve any table in kvstore
    CREATE_ANY_INDEX Create any index on any table in kvstore
    DROP_ANY_INDEX Drop any index on any table in kvstore
    EVOLVE_TABLE Evolve a specific table
    CREATE_INDEX Create index on a specific table
    DROP_INDEX Drop index on a specific table

    Users are now able to create new roles to group together privileges or other roles. This provides a way to grant a group of desired privileges to a user. New role management commands have been added to support create and drop roles, and to grant and revoke privileges or roles to and from other roles.

    Modifications to the data definition language (DDL) have been added to provide a declarative interface to all security operations. The language reference is in Security Guide. The language is accessible via the API as well as via the "execute" command in the Admin CLI.

    Passwords now have lifetimes and will expire when they have been in use beyond the specified lifetime. It is also possible now to explicitly expire a password when adding a new user or by altering the profile of an existing user. Users are required to renew the expired password before they can log in to the store successfully. [#23951]

  2. Admin service configuration has been enhanced to better match the use of primary and secondary zones. It is now possible for an Admin service to be a primary or a secondary Admin. This Admin type is analogous to zone type. A primary Admin can serve as a master or replica of the Admin shard, and vote in master elections. Secondary Admins can only serve as replicas and do not vote in master elections. All Admins created in earlier releases are primary Admins. With this release new Admins are created with the same type as their containing zone.

    Admin services created in a secondary zone by previous release will have the wrong type (primary). This mismatch will be reported as a violation from the verify configuration command. This condition can be remedied through the plan repair-topology command.

    In addition to the changes in Admin service type behavior, new rules have been put in place regarding Admin service deployment. In general it is recommended that Admin services follow the same rules as data nodes, specifically the number of Admin services in a zone should match the zone's replication factor. [#23985], [#24182]

  3. The Command Line Interface (Admin CLI) now supports a read only mode if the Admin shard does not have quorum or if the master Admin is unreachable. In this read-only mode any commands which require the Admin to update persistent state, such as plan creation and execution, are disabled. However, most commands which provide status or configuration information will function. In addition to a notification in the CLI informing the user that the CLI is in this mode, the show admins command will also indicate that the CLI is connected read-only.

    Additional re-connect capabilities have been added to the Admin CLI to improve robustness of the CLI in the face of Admin node failures. [#23943]

API Changes

  1. The methods oracle.kv.table.TableAPI.execute and oracle.kv.table.TableAPI.executeSync have been deprecated in favor of the new APIs, oracle.kv.KVStore.execute and oracle.kv.KVStore.executeSync. The motivation for the change is the introduction of new DDL statements which manage objects that are above the scope of a single table, such as users and roles. Likewise, the classes oracle.kv.table.ExecutionFuture and oracle.kv.table.StatementResult are deprecated in favor of oracle.kv.ExecutionFuture and oracle.kv.StatementResult. [#23937]

  2. New methods have been added to oracle.kv.table.RecordValue to provide the ability to use JSON values for complex fields in a table. In the past, JSON could be used to specify the entire row, but not portions of a row. The following 6 JSON input methods for complex types in RecordValue are new: [#24069]

    RecordValue putRecordAsJson(String fieldName, String jsonInput, boolean exact);
    RecordValue putRecordAsJson(String fieldName, InputStream jsonInput, boolean exact);
    RecordValue putArrayAsJson(String fieldName, String jsonInput, boolean exact);
    RecordValue putArrayAsJson(String fieldName, InputStream jsonInput, boolean exact);
    RecordValue putMapAsJson(String fieldName, String jsonInput, boolean exact);
    RecordValue putMapAsJson(String fieldName, InputStream jsonInput,boolean exact);
  3. New methods have been added to better support DDL use by stateless applications. oracle.kv.ExecutionFuture.toByteArray() returns a serialized version of the future that can later be passed to oracle.kv.KVStore.getFuture(byte[]) to recreate a Future instance. [#24228]

  4. There are changes to the way the results and status of DDL operations are reported in the API. Information about the execution of a DDL operation continues to be returned via oracle.kv.StatementResult. In past releases, both return values and execution status were returned via StatementResult.getInfo(), StatementResult.getInfoAsJson(). Now, results are return via a new method, StatementResult.getResult(), while status is returned via StatementResult.getInfo(), getInfoAsJson. [#24277]

Utility Changes

  1. plan deploy-admin now deploys an Admin with the same type (primary or secondary) as the zone in which it resides. Previously the Admin was always created as a primary.

  2. plan repair-topology will change an Admin's type so that the type matches the zone in which it resides.

  3. verify configuration will report the following:

    • violation for each Admin service whose type does not match the type of the zone in which it resides

    • violation if the number of Admin services in a zone is less than the zone's rep factor

    • note if the number of Admins services in a zone is greater than the zone's rep factor

  4. The put command in the Admin CLI now has a new -exact flag. When true, the input json string or file specified to the command must contain values for all columns in the table, and cannot contain extraneous fields.

  5. The Admin CLI now supports a new "show versions" command, which displays client and server version information. [#24005]

  6. The Admin CLI now provides the ability to use the semicolon character as a command terminator to enter multiple commands, or to enter commands on multiple lines. [#24119]

    For example, the following command would have had to be typed in a single line:

    kv-> execute "create table users(name string, address string, primary key (name))" 

    or

    kv-> execute "create table users \
        ->                 (name string, \
        ->                  address string, \
        ->                  primary key (name))"

    but can now be entered this way:

      kv-> execute "create table users
        ->                 (name string,
        ->                  address string,
        ->                  primary key (name))";

    In addition, multiple commands can be entered as below, using a semicolon as a terminator.

      kv-> show table -name users; get table -name users;
  7. A new command line utility has been added that disables services associated with a storage node. You can use the new utility when starting a storage node whose services had configuration changes while the node was offline, to allow the configuration to be updated so that the services can be started with the proper configuration. [#23988]

    The new utility is invoked this way:

      java jar -kvstore.jar disable-services -root ROOT_DIRECTORY [-config CONFIG_FILE]
    
  8. Improvements have been made to the output of the Admin CLI verify configuration command, and both the Admin CLI and top level versions of the ping command: [#23981]

    • Output now uses JSON format if the new -json flag is specified

    • Output now includes summaries of the status of shards and zones in the store, and information about the replication status of non-master replication nodes

    • Output for the ping commands now includes information about admin services, to match the information provided by verify configuration

  9. A new "verify" option has been added to the diagnostics command line utility. This option does a variety of health and configuration checks, such as:

    • configuration values are valid

    • inter-node clock skew is within the permissible maximum skew value

    • all nodes have supported Java versions

    • all nodes have correct network connectivity

    • all nodes have compatible NoSQL DB versions

    java -jar kvstore.jar diagnostics verify -help
    
    Usage: verify -checkLocal |
                  -checkMulti

Bug and Performance Fixes

  1. Changes have been made to transfer the RN master nodes on SN shutdown. When a SNA shuts down it checks if any of the RNs managed by the SNA is a master for its replication group. If an RN is a master, the SNA causes a master-transfer before shutting down the RN. The implementation performs the transfers one at a time so that each transfer can take into account the results of the previous one, to avoid overloading the target SNs. [#22426]

  2. Modified the replication node and admin services to permit them to start up even if other members of a service's replication group were managed by storage nodes whose hostnames cannot be resolved via DNS. This change allows the store to continue to function, and in particular to support restarting services, if failures of storage nodes cause the nodes to have unresolvable DNS names. [#23120]

  3. Modified the 'topology validate' CLI command. In addition to the 'violations' and 'notes' that are currently displayed by that command, it now also notes when the topology contains any zones that are empty; that is, the zones that contain no SNs. [#23222]

  4. Currently the 'verify configuration' CLI command performs a component-by-component comparison of the store's current state or configuration against what is reflected in the store's Admin database, and displays any inconsistencies. Modified the command to also note when any empty zones exist in the store. With respect to the current the flags taken by the 'verify configuration' command, any changes related to empty zones should apply to all arguments except the optional -sn argument; in which case, the new behavior related to identifying empty data centers should not be executed. [#23223]

  5. The CLI 'snapshot' commands currently allow one to collect a snapshot from each Admin and RN of every reachable SN in the store, or to remove one or all instances of previously collected snapshot data. These commands have been modified to take a '-zn' or '-znname' flag so that the command applies to all the SNs executing in the zone with the specified id or name. [#23224]

  6. A bug has been fixed that could cause the Admin service to hang if the Admin web console is enabled, and is displaying log file output within the logtail pane, and an administrative command that updates Admin service configuration, such as plan deploy-admin, is running. [23907]

  7. Clarified an error message that would occur when a field name was used instead of elementof() in a CHECK expression involving a map or array. [#24055]

  8. Relaxed the DDL PRIMARY KEY expression, allowing the statement to redundantly specify single primary key fields as a SHARD keys as well. In a top-level (non-child) table the primary key is equivalent to the shard key unless otherwise specified. This change makes statements like this legal even though use of "SHARD" is redundant: CREATE TABLE mytable(id INTEGER, PRIMARY KEY(SHARD(id))). [#24105]

  9. Changes have been made to the request dispatcher to favor more rapid request failover on a node failure, reducing the possibility of a RequestTimeoutException. As part of this change, the default socket open timeouts have been reduced from 5 seconds to 3 seconds to permit a redispatch of requests within the request timeout period. [#24152]