Configuring with Multiple Zones
To achieve optimal use of all available physical facilities, deploy your data store across multiple zones. Multiple zones provide fault isolation and availability for your data if a single zone fails. Each zone has a copy of your complete data store, including a copy of all the shards. With this configuration, reads are always possible, as long as your data's consistency guarantees can be met, because at least one replica is located in every zone. Writes can also occur in the event of a zone loss, as long as the database maintains quorum. See Concepts Guide.
You can specify a different replication factor to each zone. A replication factor is quantified as one of the following:
- Zone Replication Factor
- The number of copies, or replicas, maintained in a zone.
- Primary Replication Factor
- The total number of replicas in all primary zones. This replication factor controls the number of replicas that participate in elections and acknowledgments. For additional information on how to identify the Primary Replication Factor and its implications, see Replication Factor.
- Secondary Replication Factor
- The total number of replicas in all secondary zones. Secondary replicas provide additional read-only copies of the data.
- Store Replication Factor
- The total number of replicas across the entire data store.
Zones that are located near each other physically benefit by avoiding bottlenecks from throughput limitations, and by reducing latency during elections and commits.
Note:
There are two types of zones: Primary, and Secondary.
Primary zones contain nodes which can serve as masters or replicas. Zones are created as primary Zones by default. For good performance, primary zones should be connected by low latency networks so that they can participate efficiently in master elections and commit acknowledgments. Primary zones can also become Master Affinity zones.
Secondary zones contain nodes which can only serve as replicas. Secondary zones can be used to provide low latency read access to data at a distant location, or to maintain an extra copy of the data to increase redundancy or increase read capacity. Because the nodes in secondary zones do not participate in master elections or commit acknowledgments, secondary zones can be connected to other zones by higher latency networks, because additional latency will not interfere with those time critical operations.
Using high throughput and low latency networks to connect primary zones leads to better results and improved performance. You can use networks with higher latency to connect to secondary zones so long as the connections provide sufficient throughput to support replication and sufficient reliability that temporary interruptions do not interfere with network throughput.
Note:
Because any primary zone can host master nodes, you can reduce write performance by connecting primary zones through a limited throughput or a high latency network link.
The following steps walk you through the process of deploying six Storage Nodes across three primary zones. You can then verify that each shard has a replica in every zone; service can be continued in the event of a zone failure. You will configure secure data store in all the six Storage Nodes. In the first Storage Node, security will be configured and the security directory and all files contained in it will be copied from the first Storage Node to other Storage Nodes to setup security.
(node01)
:
- Execute the following
command:
java -jar $KVHOME/lib/kvstore.jar makebootconfig \ -root $KVROOT \ -port 5000 \ -host $KVHOST \ -harange 5010,5020 \ -store-security configure \ -capacity 1 \ -storagedir ${KVDATA}/disk1 \ -storagedirsize 5500-MB \
- Start the Storage Node
Agent:
java -jar $KVHOME/lib/kvstore.jar start -root $KVROOT &
- Create a zip file of all the security files
created:
cd ; zip -r $HOME/security.zip $KVROOT/security; cd -
- Copy
$HOME/security.zip
from this node (node01
) to the other five nodes.
node02
,
node03
, node04
, node05
,
node06
).
- Unzip the security files copied from the first Storage Node (
node01
).cd; unzip -o security.zip -d /; cd -;
-
Execute the following command:
java -jar $KVHOME/lib/kvstore.jar makebootconfig \ -root $KVROOT \ -port 5000 \ -host $KVHOST \ -harange 5010,5020 \ -store-security enable \ -capacity 1 \ -storagedir ${KVDATA}/disk1 \ -storagedirsize 5500-MB \
- Start the Storage Node
Agent:
java -jar $KVHOME/lib/kvstore.jar start -root $KVROOT &
node01
) deploy your data store
using the following steps:
- Start the Admin CLI. Here $KVHOST is
node01
.java -Xmx64m -Xms64m \ -jar $KVHOME/lib/kvstore.jar runadmin \ -port 5000 -host $KVHOST -security $KVROOT/security/client.security
- Name your data store and deploy three primary
zones:
configure -name MetroArea; plan deploy-zone -name "Manhattan" -rf 1 -wait; plan deploy-zone -name "JerseyCity" -rf 1 -wait; plan deploy-zone -name "Queens" -rf 1 -wait;
- Deploy the first Storage Node with administration process in the
Manhattan
zone.
Deploy a second Storage Node in Manhattan zone:plan deploy-sn -znname Manhattan -host node01 -port 5000 -wait; plan deploy-admin -sn sn1 -wait;
plan deploy-sn -znname Manhattan -host node02 -port 5000 -wait;
- Deploy the first Storage Node with administration process in the
JerseyCity
zone:.
Deploy a second Storage Node in JerseyCity zone:plan deploy-sn -znname JerseyCity -host node03 -port 5000 -wait; plan deploy-admin -sn sn3 -wait;
plan deploy-sn -znname JerseyCity -host node04 -port 5000 -wait;
- Deploy the first Storage Node with administration process in the Queens
zone:.
Deploy a second Storage Node in Queens zone:plan deploy-sn -znname Queens -host node05 -port 5000 -wait; plan deploy-admin -sn sn5 -wait;
plan deploy-sn -znname JerseyCity -host node06 -port 5000 -wait;
- Create and deploy a topology:
topology create -name Topo1 -pool AllStorageNodes -partitions 300; plan deploy-topology -name Topo1 -wait;
- Follow the instructions mentioned in Create users and configure security with remote access to create the access for the users in the multiple zones.
- Check service status with the
show topology
command:
Output:kv-> show topology
store=MetroArea numPartitions=100 sequence=117 zn: id=zn1 name=Manhattan repFactor=1 type=PRIMARY allowArbiters=false masterAffinity=false zn: id=zn2 name=JerseyCity repFactor=1 type=PRIMARY allowArbiters=false masterAffinity=false zn: id=zn3 name=Queens repFactor=1 type=PRIMARY allowArbiters=false masterAffinity=false sn=[sn1] zn:[id=zn1 name=Manhattan] node01:5000 capacity=1 RUNNING [rg1-rn1] RUNNING No performance info available sn=[sn2] zn:[id=zn1 name=Manhattan] node02:5000 capacity=1 RUNNING [rg2-rn1] RUNNING No performance info available sn=[sn3] zn:[id=zn2 name=JerseyCity] node03:5000 capacity=1 RUNNING [rg1-rn2] RUNNING No performance info available sn=[sn4] zn:[id=zn2 name=JerseyCity] node04:5000 capacity=1 RUNNING [rg2-rn2] RUNNING No performance info available sn=[sn5] zn:[id=zn3 name=Queens] node05:5000 capacity=1 RUNNING [rg1-rn3] RUNNING No performance info available sn=[sn6] zn:[id=zn3 name=Queens] node06:5000 capacity=1 RUNNING [rg2-rn3] RUNNING No performance info available numShards=2 shard=[rg1] num partitions=50 [rg1-rn1] sn=sn1 [rg1-rn2] sn=sn3 [rg1-rn3] sn=sn5 shard=[rg2] num partitions=50 [rg2-rn1] sn=sn2 [rg2-rn2] sn=sn4 [rg2-rn3] sn=sn6
- Verify that each shard has a replica in every zone:
Output:kv-> verify configuration
Verify: starting verification of store MetroArea based upon topology sequence #117 100 partitions and 6 storage nodes Time: 2024-04-05 10:41:15 UTC Version: 24.1.11 See node01: $KVROOT/MetroArea/log/MetroArea_{0..N}.log for progress messages Verify: Shard Status: healthy:2 writable-degraded:0 read-only:0 offline:0 total:2 Verify: Admin Status: healthy Verify: Zone [name=Manhattan id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] RN Status: online:2 read-only:0 offline:0 Verify: Zone [name=JerseyCity id=zn2 type=PRIMARY allowArbiters=false masterAffinity=false] RN Status: online:2 read-only:0 offline:0 maxDelayMillis:1 maxCatchupTimeSecs:0 Verify: Zone [name=Queens id=zn3 type=PRIMARY allowArbiters=false masterAffinity=false] RN Status: online:2 read-only:0 offline:0 maxDelayMillis:4 maxCatchupTimeSecs:0 Verify: == checking storage node sn1 == Verify: Storage Node [sn1] on node01:5000 Zone: [name=Manhattan id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 24.1.11 2024-04-05 10:41:15 UTC Build id: c8998e4a8aa5 Edition: Enterprise Verify: Admin [admin1] Status: RUNNING,MASTER Verify: Rep Node [rg1-rn1] Status: RUNNING,MASTER sequenceNumber:1,261 haPort:5011 available storage size:31 GB Verify: == checking storage node sn2 == Verify: Storage Node [sn2] on node02:5000 Zone: [name=Manhattan id=zn1 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 24.1.11 2024-04-05 10:41:15 UTC Build id: c8998e4a8aa5 Edition: Enterprise Verify: Rep Node [rg2-rn1] Status: RUNNING,MASTER sequenceNumber:1,236 haPort:5012 available storage size:31 GB Verify: == checking storage node sn3 == Verify: Storage Node [sn3] on node03:5000 Zone: [name=JerseyCity id=zn2 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 24.1.11 2024-04-05 10:41:15 UTC Build id: c8998e4a8aa5 Edition: Enterprise Verify: Admin [admin2] Status: RUNNING,REPLICA Verify: Rep Node [rg1-rn2] Status: RUNNING,REPLICA sequenceNumber:1,261 haPort:5011 available storage size:31 GB delayMillis:1 catchupTimeSecs:0 Verify: == checking storage node sn4 == Verify: Storage Node [sn4] on node04:5000 Zone: [name=JerseyCity id=zn2 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 24.1.11 2024-04-05 10:41:15 UTC Build id: c8998e4a8aa5 Edition: Enterprise Verify: Rep Node [rg2-rn2] Status: RUNNING,REPLICA sequenceNumber:1,236 haPort:5012 available storage size:31 GB delayMillis:0 catchupTimeSecs:0 Verify: == checking storage node sn5 == Verify: Storage Node [sn5] on node05:5000 Zone: [name=Queens id=zn3 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 24.1.11 2024-04-05 10:41:15 UTC Build id: c8998e4a8aa5 Edition: Enterprise Verify: Admin [admin3] Status: RUNNING,REPLICA Verify: Rep Node [rg1-rn3] Status: RUNNING,REPLICA sequenceNumber:1,261 haPort:5011 available storage size:31 GB delayMillis:1 catchupTimeSecs:0 Verify: == checking storage node sn6 == Verify: Storage Node [sn6] on node06:5000 Zone: [name=Queens id=zn3 type=PRIMARY allowArbiters=false masterAffinity=false] Status: RUNNING Ver: 24.1.11 2024-04-05 10:41:15 UTC Build id: c8998e4a8aa5 Edition: Enterprise Verify: Rep Node [rg2-rn3] Status: RUNNING,REPLICA sequenceNumber:1,236 haPort:5012 available storage size:31 GB delayMillis:4 catchupTimeSecs:0 Verification complete, no violations.
In the above example there are three zones (zn1 = Manhattan, zn2 = JerseyCity, zn3=Queens) with six replication nodes (two masters and four replicas) in the data store. This means that this topology is not only highly available because you have three replicas within each shard, but it is also able to recover from a single zone failure. If any zone fails, the other two zones are enough to elect the new master, so service continues without any interruption.