What is New in NDB Cluster 8.4

Changes in NDB 8.x Innovation Releases

ndbinfo transporter_details table.  The transporter_details table provides information about individual transporters used in an NDB cluster. It is otherwise similar to the ndbinfo transporters table, which provides such information in aggregate form.

NDB 8.4.0 provides additional columns as compared to the version introduced in NDB 8.0. These new columns, along with brief dscriptions of each, are listed here:

NDB 8.4.1 adds a type column, which displays the transport's connection type (TCP or SHM).

See Section 25.6.17.65, “The ndbinfo transporter_details Table”, for more information.

NDB Replication: Filtering of unused updates.  Previously, when SQL nodes performing binary logging used log_replica_updates=OFF, any replicated updates which were applied on a replica NDB cluster were sent on to the SQL nodes performing binary logging. These updates were not actually applied or used for any other purpose; this entailed unnecessary network traffic and consumption of resources.

In NDB 8.4 and later, updates applied on the replica SQL node are filtered out on this node, and are no longer sent onward to any other SQL nodes. Updates that do not trigger any logging are also no longer sent by the replica.

TLS for cluster node communications.  NDB Cluster 8.3 and later provides support for network communications secured by Transport Layer Security (TLS) and Internet Public Key Infrastructure (PKI) to authenticate and encrypt connections between NDB nodes, and between the NDB management server and its clients; TLS is applied to the NDB Transporter Protocol, and to the NDB Management Protocol.

This feature uses TLS mutual authentication, in which a node's own certificate file contains the chain of trust which the node uses to validate the certificates of its peers. When TLS is enabled on the cluster, data and management nodes use TLS to perform the following tasks:

Connections that use the MySQL client protocol employ MySQL user authentication, and may use TLS (including optional mutual TLS) as described elsewhere in this Manual; see Section 8.3, “Using Encrypted Connections”, for more information.

NDB implements a new tool ndb_sign_keys which can be used to create and manage CA, certificate files, and keys. You can generate a set of keys and certificates for all nodes in a cluster with a given configuration file using ndb_sign_keys --create-key.

Using ndb_sign_keys, a node certificate can be bound to a particular hostname, made to expire on a given date, and be associated with a given node type, so that clients are distinct from servers, and management servers from data nodes. (Every NDB TLS certificate can be used for MGM client connections.) Private keys are created in place, so that copying of files containing private keys is minimized. Both private keys and certificates are labeled as either active or pending; ndb_sign_keys also provides help with rotating keys to allow for pending keys to replace active keys before the active keys expire.

Testing of node TLS connections can be done from the system shell using ndb_mgm client with --test-tls, or within the ndb_mgm client using the TLS INFO command. You can obtain information about certificates used by cluster nodes by checking the ndbinfo certificates table.

To enforce a requirement for TLS, set the client option ndb-mgm-tls=strict in my.cnf on each cluster host, then set RequireTls=true in the [mgm default] section of the cluster config.ini file, and set RequireTls=true in the [ndbd default] section of the configuration file as well. Then perform a rolling restart of the cluster, restarting the management server with --reload --config-file.

Use of TLS connections is also supported in NDB Cluster API applications in NDB 8.3 and later. For information about MGM API support, see TLS Functions. The NDB API Ndb_cluster_connection class adds configure_tls() get_tls_certificate_path() methods for setting up TLS connections by clients.

For more information, see Section 25.6.15, “TLS Link Encryption for NDB Cluster”, as well as Section 25.5.28, “ndb_sign_keys — Create, Sign, and Manage TLS Keys and Certificates for NDB Cluster”.

Binary log injector memory allocation.  In previous versions of NDB Cluster, when the NDB binary log injector was engaged in handling schema changes and tracking the state of the binary log, the choice of arena for allocation of memory for these purposes was forced by changing thread local pointers, thus attempting to try and catch all allocations performed during epoch processing. At the end of the epoch, those pointers were reset, arena memory was released, and the arena structures discarded; this released the memory, but also required setting it up again for the next epoch. The thread local pointer changes also introduced the risk of memory being allocated wrongly when activating functionality in different subsystems. MySQL NDB Cluster 8.3 makes the following improvements to this functionality:

These changes are internal only but should provide a noticeable improvememnt by saving on memory release and re-allocation over successive epochs.

NDB API primary key updates.  Previously, when using any other mechanism than NdbRecord in an attempt to update a primary key value, the NDB API returned error 4202 Set value on tuple key attribute is not allowed, even setting a value identical to the existing one. In NDB 8.1 and later, checking when performing updates by other means is handed off to the data nodes, as it was already when using NdbRecord to perform the update.

This means that you can now perform primary key updates using NdbOperation::setValue(), NdbInterpretedCode::write_attr(), and other methods of NdbOperation and NdbInterpretedCode which set column values (including the NdbOperation methods incValue(), subValue(), the NdbInterpretedCode methods add_val(), sub_val(), and so on). This also applies to the NdbOperation interface's OperationOptions::OO_SETVALUE extension.

Improved warnings.  Made the following improvements in warning output:

In addition, we no longer print the message Validating excluded objects to the SQL node's error log every ndb_metadata_check_interval (default 60) seconds when log_error_verbosity is greater than or equal to 3 (INFO level), due ot the fact that such messages tended to flood the error log, making it difficult to examine, and using excess disk space, while not providing any additional benefit to the user.

Pushdown joins between queries featuring very large and possibly overlapping IN() and NOT IN() lists are now handled in a correct and safe manner.

ndbcluster plugin log messages now use SYSTEM as the log level and NDB as the subsystem for logging. This means that informational messages from the ndbcluster plugin are always printed; their verbosity can be controlled by using --ndb_extra_logging.