MySQL 8.0 Reference Manual Including MySQL NDB Cluster 8.0
This section provides information about NDB Cluster software and compatibility between different NDB Cluster 8.0 releases with regard to performing upgrades and downgrades. You should already be familiar with installing and configuring NDB Cluster prior to attempting an upgrade or downgrade. See Section 25.4, “Configuration of NDB Cluster”.
Online upgrades and downgrades between minor releases of the
NDB
storage engine are supported within NDB
8.0. In-place upgrades of the included MySQL Server (SQL node
mysqld) are also supported; with multiple SQL
nodes, it is possible to keep an SQL application online while
individual mysqld processes are restarted.
In-place downgrades of the included MySQL Server are
not supported (see
Chapter 4, Downgrading MySQL).
It may be possible in some cases to revert a recent upgrade from
one NDB 8.0 minor release version to a later one, and to restore
the needed states of any MySQL Server instances running as SQL
nodes. Against the event that this becomes desirable or
necessary, you are strongly advised to take a complete backup of
each SQL node prior to upgrading NDB Cluster. For the same
reason, you should also start the mysqld
binaries from the new version with
--ndb-schema-dist-upgrade-allowed=0
,
and not allow it to be set back to 1 until you are sure any
likelihood of reverting to an older version is past. For more
information, see Reverting an NDB Cluster 8.0 Upgrade.
For information about upgrades to NDB 8.0 from versions previous to 8.0, see Versions Supported for Upgrade to NDB 8.0.
For information about known issues and problems encountered when upgrading or downgrading NDB 8.0, see Known Issues When Upgrading or Downgrading NDB Cluster.
The following versions of NDB Cluster are supported for upgrades to GA releases of NDB Cluster 8.0 (8.0.19 and later):
NDB Cluster 7.6: NDB 7.6.4 and later
NDB Cluster 7.5: NDB 7.5.4 and later
NDB Cluster 7.4: NDB 7.4.6 and later
To upgrade from a release series previous to NDB 7.4, you must upgrade in stages, first to one of the versions just listed, and then from that version to the latest NDB 8.0 release. In such cases, upgrading to the latest NDB 7.6 release is recommended as the first step. For information about upgrades to NDB 7.6 from previous versions, see Upgrading and Downgrading NDB 7.6.
Following a recent software upgrade of an NDB Cluster to an NDB
8.0 release, it is possible to revert the NDB
software back to the earlier version, provided certain
conditions are met before the upgrade, during the time the
cluster is running the newer version, and after the NDB Cluster
software is reverted to the earlier version. Specifics depend on
local conditions; this section provides general information
about what should be done at each of the points in the upgrade
and rollback process just described.
In most cases, upgrading and downgrading the data nodes can be
done without issue, as described elsewhere; see
Section 25.6.5, “Performing a Rolling Restart of an NDB Cluster”. (Prior to
performing an upgrade or downgrade, you should perform an
NDB
backup; see
Section 25.6.8, “Online Backup of NDB Cluster”, for information about
how to do this.) Downgrading SQL nodes online is not supported,
due to the following issues:
mysqld from a version 8.0 release cannot start if it detects a file system from a later version of MySQL.
In many cases, mysqld cannot open tables that were created or modified by a later version of MySQL.
In most if not all cases, mysqld cannot read binary log files that were created or modified in a later version of MySQL.
The procedure outlined next provides the basic steps necessary
to upgrade a cluster from version X
to version Y
while allowing for a
possible future rollback to X
. (The
procedure for reverting the upgraded cluster to version
X
follows later in this section.) For
this purpose, version X
is any NDB
8.0 GA release, or any previous NDB release supported for
upgrade to NDB 8.0 (see
Versions Supported for Upgrade to NDB 8.0), and version
Y
is an NDB 8.0 release which is
later than X
.
Prior to upgrade: Take backups of NDB
X
SQL node states. This can be
accomplished as one or more of the following:
A copy of the version X
SQL
node file system in a quiescent state using one or more
system tools such as
cp,
rsync,
fwbackups, Amanda, and
so forth.
A dump of any version X
tables not stored in NDB
. You can
generate this dump using mysqldump.
A backup created using MySQL Enterprise Backup; see Section 32.1, “MySQL Enterprise Backup Overview”, for more information.
Backing up the SQL nodes is recommended prior to any
upgrade, whether or not you later intend to revert the
cluster to the previous NDB
version.
Upgrade to NDB
Y
: All NDB
Y
mysqld
binaries must be started with
--ndb-schema-dist-upgrade-allowed=0
to prevent any automatic schema upgrade. (Once any
possibility of a downgrade is past, you can safely change
the corresponding system variable
ndb_schema_dist_upgrade_allowed
back to
1, the default, in the mysql client.)
When each NDB Y
SQL node starts,
it connects to the cluster and synchronizes its
NDB
table schemas. After this, you can
restore MySQL table and state data from backup.
To assure continuity of NDB replication, it is necessary to
upgrade the cluster's SQL nodes in such a way that at
least one mysqld is acting as the
replication source at any given point in time during the
upgrade. With two SQL nodes A
and
B
, you can do so like this:
While using SQL node B
as the
replication channel, upgrade SQL node
A
from NDB version
X
to version
Y
. This results in a gap in
the binary log on A
at epoch
E1
.
After all replication appliers have consumed the binary
log from SQL node B
past
epoch E1
, switch the
replication channel to use SQL node
A
.
Upgrade SQL node B
to NDB
version Y
. This results in a
gap in the binary log on B
at
epoch E2
.
After all replication appliers have consumed the binary
log from SQL node A
past
epoch E2
, you can once again
switch the replication channel to use either SQL node as
desired.
Do not use ALTER TABLE
on any
existing NDB
tables; do not create any
new NDB
tables which cannot be safely
dropped prior to downgrading.
The following procedure shows the basic steps needed to roll
back (revert) an NDB Cluster from version
X
to version
Y
after an upgrade performed as just
described. Here, version X
is any NDB
8.0 GA release, or any previous NDB release supported for
upgrade to NDB 8.0 (see
Versions Supported for Upgrade to NDB 8.0); version
Y
is an NDB 8.0 release which is
later than X
.
Prior to rollback: Gather any
mysqld state information from the NDB
Y
SQL nodes that should be
retained. In most cases, you can do this using
mysqldump.
After backing up the state data, drop all
NDB
tables which have been created or
altered since the upgrade took place.
Backing up the SQL nodes is always recommended prior to any NDB Cluster software version change.
You must provide a file system compatible with MySQL
X
for each
mysqld (SQL node). You can use either of
the following two methods:
Create a new, compatible file system state by
reinitializing the on-disk state of the version
X
SQL node. You can do this
by removing the SQL node file system, then running
mysqld
--initialize
.
Restore a file system that is compatible from a backup taken prior to the upgrade (see Section 9.4, “Using mysqldump for Backups”).
Following NDB
downgrade: After downgrading the data nodes to
NDB X
, start the version
X
SQL nodes (instances of
mysqld). Restore or repair any other
local state information needed on each SQL node. The MySQLD
state can be aligned as necessary with some combination (0
or more) of the following actions:
Initialization commands such as
mysqld --initialize
.
Restore any desired or required state information
captured from the version X
SQL node.
Restore any desired or required state information
captured from the version Y
SQL node.
Perform cleanup such as deleting stale logs such as binary logs, or relay logs, and removing any time-dependent state which is no longer valid.
As when upgrading, it is necessary when downgrading to
maintain continuity of NDB replication to downgrade the
cluster's SQL nodes in such a way that at least one
mysqld is acting as the replication
source at any given point in time during the downgrade
process. This can be done in a manner very similar to that
described previously for upgrading the SQL nodes. With two
SQL nodes A
and
B
, you can maintain binary
logging without any gaps during the downgrade like this:
With SQL node B
acting as the
replication channel, downgrade SQL node
A
from NDB version
Y
to version
X
. This results in a gap in
the binary log on A
at epoch
F1
.
After all replication appliers have consumed the binary
log from SQL node B
past
epoch F1
, switch the
replication channel to use SQL node
A
.
Downgrade SQL node B
to NDB
version X
. This results in a
gap in the binary log on B
at
epoch F2
.
After all replication appliers have consumed the binary
log from SQL node A
past
epoch F2
, redundancy of
binary logging is restored, and you can again use either
SQL node as the replication channel as desired.
See also Section 25.7.7, “Using Two Replication Channels for NDB Cluster Replication”.
In this section, provide information about issues known to occur when upgrading or downgrading to, from, or between NDB 8.0 releases.
We recommend that you not attempt any schema changes during any NDB Cluster software upgrade or downgrade. Some of the reasons for this are listed here:
DDL statements on NDB
tables are not
possible during some phases of data node startup.
DDL statements on NDB
tables may be
rejected if any data nodes are stopped during execution;
stopping each data node binary (so it can be replaced with a
binary from the target version) is required as part of the
upgrade or downgrade process.
DDL statements on NDB
tables are not
allowed while there are data nodes in the same cluster
running different release versions of the NDB Cluster
software.
For additional information regarding the rolling restart procedure used to perform an online upgrade or downgrade of the data nodes, see Section 25.6.5, “Performing a Rolling Restart of an NDB Cluster”.
You should be aware of the issues in the following list when you perform an online upgrade between minor versions of NDB 8.0. These issues also apply when upgrading from a previous major version of NDB Cluster to any of the NDB 8.0 releases stated.
NDB 8.0.22 adds support for IPv6 addressing for management
nodes and data nodes in the config.ini
file. To begin using IPv6 addresses as part of an upgrade,
perform the following steps:
Perform an upgrade of the cluster to version 8.0.22 or a later version of the NDB Cluster software in the usual manner.
Change the addresses used in the
config.ini
file to IPv6 addresses.
Perform a system restart of the cluster.
A known issue on Linux platforms when running NDB 8.0.22 and later was that the operating system kernel was required to provide IPv6 support, even when no IPv6 addresses were in use. This issue is fixed in NDB 8.0.34 and later (Bug #33324817, Bug #33870642).
If you are using an affected version and wish to disable support for IPv6 on the system (because you do not plan to use any IPv6 addresses for NDB Cluster nodes), do so after booting the system, like this:
$>sysctl -w net.ipv6.conf.all.disable_ipv6=1
$>sysctl -w net.ipv6.conf.default.disable_ipv6=1
(Alternatively, you can add the corresponding lines to
/etc/sysctl.conf
.) In NDB Cluster
8.0.34 and later, the preceding is not necessary, and you
can simply disable IPv6 support in the Linux kernel if you
do not want or need it.
Due to changes in the internal
mysql.ndb_schema
table, if you upgrade to
an NDB 8.0 release prior to 8.0.24, then you are advised to
use --ndb-schema-dist-upgrade-allowed
= 0
to avoid unexpected outages (Bug #30876990, Bug
#31016905).
In addition, if there is any possibility that you may revert
to a previous version of NDB Cluster following an upgrade to
a newer version, you must start all
mysqld processes from the newer version
with --ndb-schema-dist-upgrade-allowed = 0
to prevent changes incompatible with the older version from
being made to the ndb_schema
table. See
Reverting an NDB Cluster 8.0 Upgrade, for
information about how to do this.
The
EncryptedFileSystem
configuration parameter, introduced in NDB 8.0.29, could in
some cases cause undo log files to be encrypted, even when
set explicitly to 0
, which could lead to
issues when using Disk Data tables and attempting to upgrade
or downgrade to NDB 8.0.29. In such cases, you can work
around the problem by performing initial restarts of the
data nodes as part of the rolling restart process.
If you are using multithreaded data nodes
(ndbmtd) and the
ThreadConfig
configuration parameter, you may need to make changes in the
value set for this in the config.ini
file when upgrading from a previous release to NDB 8.0.30 or
later. When upgrading from NDB 8.0.23 or earlier, any usage
of main
, rep
,
recv
, or ldm
threads
that was implicit in the earlier version must be explicitly
set. When upgrading from NDB 8.0.23 or later to NDB 8.0.30
or later, any usage of recv
threads must
be set explicitly in the ThreadConfig
string. In addition, to avoid using main
,
rep
, or ldm
threads in
NDB 8.0.30 or later, you must set the thread count for the
given type to 0
explicitly.
An example follows.
NDB 8.0.22 and earlier:
config.ini
file contains
ThreadConfig=ldm
.
This is interpreted by these versions of
NDB
as
ThreadConfig=main,ldm,recv,rep
.
Required in config.ini
to match
effect in NDB 8.0.30 or later:
ThreadConfig=main,ldm,recv,rep
.
NDB 8.0.23—8.0.29:
config.ini
file contains
ThreadConfig=ldm
.
This is interpreted by these versions of
NDB
as
ThreadConfig=ldm,recv
.
Required in config.ini
to match
effect in NDB 8.0.30 or later:
ThreadConfig=main={count=0},ldm,recv,rep={count=0}
.
For more information, see the description of the
ThreadConfig
configuration parameter.
Upgrades from previous major versions of NDB Cluster (7.4, 7.5, 7.6) to NDB 8.0 are supported; see Versions Supported for Upgrade to NDB 8.0, for specific versions. Such upgrades are subject to the issues listed here:
In NDB 8.0, the default value for
log_bin
is 1, a change from
earlier releases. In addition, as of NDB 8.0.16, the default
value for ndb_log_bin
changed from 1 to 0, which means that
ndb_log_bin
must be set explicitly to 1
to enable binary logging in this and later versions.
Distributed privileges shared between MySQL servers as
implemented in prior release series (see
Distributed Privileges Using Shared Grant Tables)
are not supported in NDB Cluster 8.0. When started, the
mysqld supplied with NDB 8.0 and later
checks for the existence of any grant tables which use the
NDB
storage engine; if it finds any, it
creates local copies (“shadow tables”) of these
using InnoDB
. This is true for each MySQL
server connected to NDB Cluster. After this has been
performed on all MySQL servers acting as NDB Cluster SQL
nodes, the NDB
grant tables may be safely
removed using the ndb_drop_table utility
supplied with the NDB Cluster distribution, like this:
ndb_drop_table -d mysql user db columns_priv tables_priv proxies_priv procs_priv
It is safe to retain the NDB
grant
tables, but they are not used for access control and are
effectively ignored.
For more information about the MySQL privileges system used in NDB 8.0, see Section 25.6.13, “Privilege Synchronization and NDB_STORED_USER”, as well as Section 8.2.3, “Grant Tables”.
It is necessary to restart all data nodes with
--initial
when upgrading any
release prior to NDB 7.6 to any NDB 8.0 release. This is due
to the addition of support for increased numbers of nodes in
NDB 8.0.
Issues encountered when trying to downgrade from NDB 8.0 to a previous major version can be found in the following list:
Tables created in NDB 8.0 are not backwards compatible with
NDB 7.6 and earlier releases due to a change in usage of the
extra metadata property implemented by
NDB
tables to provide full support for
the MySQL data dictionary. This means that it is necessary
to take extra steps to preserve any desired state
information from the cluster's SQL nodes prior to the
downgrade, and then to restore it afterwards.
More specifically, online downgrades of the
NDBCLUSTER
storage engine—that is,
of the data nodes—are supported, but SQL nodes cannot
be downgraded online. This is because a MySQL Server
(mysqld) of a given MySQL 8.0 or earlier
version cannot use system files from a (later) 8.0 version,
and cannot open tables that were created in the later
version. It may be possible to roll back a cluster that has
recently been upgraded from a previous NDB release; see
Reverting an NDB Cluster 8.0 Upgrade, for
information regarding when and how this can be done.
For additional information relating to these issues, see Changes in NDB table extra metadata; see also Chapter 16, MySQL Data Dictionary.
In NDB 8.0, the binary configuration file format has been
enhanced to provide support for greater numbers of nodes
than in previous versions. The new format is not accessible
to nodes running older versions of NDB
,
although newer management servers can detect older nodes and
communicate with them using the appropriate format.
While upgrades to NDB 8.0 should not be problematic in this
regard, older management servers cannot read the newer
binary configuration file format, so that some manual
intervention is required when downgrading from NDB 8.0 to a
previous major version. When performing such a downgrade, it
is necessary to remove any cached binary configuration files
prior to starting the management using the older
NDB
software version, and to have the
plaintext configuration file available for the management
server to read. Alternatively, you can start the older
management server using the
--initial
option (again, it
is necessary to have the config.ini
available). If the cluster uses multiple management servers,
one of these two things must be done for each management
server binary.
Also in connection with support for increased numbers of
nodes, and due to incompatible changes implemented in NDB
8.0 in the data node LCP Sysfile
, it is
necessary, when performing an online downgrade from NDB 8.0
to a prior major version, to restart all data nodes using
the --initial
option.
Online downgrades of clusters running more than 48 data nodes, or with data nodes using node IDs greater than 48, to earlier NDB Cluster releases from NDB 8.0 are not supported. It is necessary in such cases to reduce the number of data nodes, to change the configurations for all data nodes such that they use node IDs less than or equal to 48, or both, as required not to exceed the old maximums.
If you are downgrading from NDB 8.0 to NDB 7.5 or NDB 7.4,
you must set an explicit value for
IndexMemory
in the
cluster configuration file if none is already present. This
is because NDB 8.0 does not use this parameter (which was
removed in NDB 7.6) and sets it to 0 by default, whereas it
is required in NDB 7.5 and NDB 7.4, in both of which the
cluster refuses to start with Invalid
configuration received from Management Server...
if IndexMemory
is not set to a nonzero
value.
Setting IndexMemory
is
not required for downgrades from NDB
8.0 to NDB 7.6.