Configuring Apache ZooKeeper as the Membership Service

To configure each Apache ZooKeeper server to act as a membership server for your grid, you need to configure the zoo.cfg and myid configuration files on each host that hosts a membership server.

zoo.cfg configuration file: In replicated mode, each membership server has a zoo.cfg configuration file. The zoo.cfg configuration file identifies all of the membership servers involved in the membership service, where each membership server is identified by its DNS (or IP address) and port number.

All configuration parameters in the zoo.cfg on each membership server must be exactly the same, except for the client port. The client port can be different (but is not required to be different) for each membership server. The client port can be the same if each membership server runs on a different host.

Place the zoo.cfg file in the Apache ZooKeeper installation /conf directory. For example, if you unpacked the apache-zookeeper-3.8.4-bin.tar.gz file into the /grid/membership directory on each membership server, then you would place the zoo.cfg file into the following directory:
```
/grid/membership/apache-zookeeper-3.8.4-bin/conf/zoo.cfg
```
myid configuration file: Provides the number that identifies this particular membership server. Each membership server is identified by a unique number. For example, if you have 5 servers, they must be identified with unique integers of 1, 2, 3, 4 and 5.

This number corresponds to the definition of the host in the zoo.cfg file by the x in the server.x parameter. All zoo.cfg files must have a listing for all membership servers. For example, if you have 5 membership servers, they are configured as server.1, server.2, and so on in the zoo.cfg file.

The myid configuration file on each host contains a single line with the integer number of that server. For example, the 2nd membership server is identified in zoo.cfg as server.2 and in its myid configuration file is a single line with a 2.

The myid configuration file is a text file located in the Apache ZooKeeper data directory of the membership server. The location of the data directory is configured with the dataDir parameter in the zoo.cfg file. For example, if you configure the data directory to be /grid/membership/apache-zookeeper-3.8.4-bin/data, then you would place the myid text configuration file as follows:
```
/grid/membership/apache-zookeeper-3.8.4-bin/data/myid
```

Table 3-1 shows the commonly used configuration parameters for the zoo.cfg file.

Table 3-1 zoo.cfg Configuration Parameters

Parameter	Description
`tickTime`	The unit of time (in milliseconds) used for each tick for both `initLimit` and `syncLimit` parameters. For the best performance, you should set this to the recommended setting of `250` milliseconds. This parameter is required to run the membership server in replicated mode.
`initLimit`	The timeout (in ticks) for how long the membership servers have to connect to the leader. For the best performance, you should set this to the recommended setting of `40` ticks. This parameter is required to run the membership server in replicated mode.
`syncLimit`	The limit of how out of date a membership server can be from a leader. This limit (in ticks) specifies how long is allowed between sending a request and receiving an acknowledgment. For best performance, you should set this recommended setting to `12` ticks. This parameter is required to run the membership server in replicated mode.
`dataDir`	You decide on and create the data directory location to store the ZooKeeper data, snapshots and its transaction logs. When creating the directory where the transaction logs are written, it is important to your performance that the transaction logs are written to non-volatile storage. A dedicated device for your transaction logs is key to consistent good performance. Logging your transactions to a busy device adversely effects performance.
`clientPort`	The port on which to listen for client connections. The default is port `2181`.
`autopurge.snapRetainCount`	Defines the number of most recent snapshots and corresponding Apache ZooKeeper transaction logs to keep in the `dataDir` and `dataLogDir` respectively. Defaults to 3.
`autopurge.purgeInterval`	The time interval in hours for when to trigger the purge of older snapshots and corresponding Apache ZooKeeper transaction logs. Set to a positive integer (1 and above) to enable the auto purge. Defaults to 0. We recommend that you set this to 1.
`minSessionTimeout`	The minimum session timeout in milliseconds that the server will allow the client to negotiate. Defaults to 2 times the `tickTime`.
`maxSessionTimeout`	The maximum session timeout in milliseconds that the server will allow the client to negotiate. Defaults to 20 times the `tickTime`.
`server.x=[systemName]:nnnnn:nnnnn`	The configuration for each membership server is identified by the `server.`x parameter. The list of hosts defined by this parameter designate all of the membership servers used by the membership service. This list must correlate to the same list of membership servers in each `zoo.cfg` file on each membership server in the membership service. This parameter is required to run the membership server in replicated mode. The x is the identifying integer number for the membership server, which is also configured in the `myid` configuration file on the membership server. The `systemName` parameter specifies the DNS (or IP address) of the host on which the membership server is installed and will run. If no `systemName` is provided for the server, the default is `localhost`. Define two port numbers after each server name. First port number: Used by peers to connect to and communicate with other peers. This port connects followers to the leader. Second port number: Used for leader election among the membership servers. If necessary, this port is used to elect a new leader in case of failure. For a production environment, each of the membership servers should be configured on different hosts. In this case, the convention is to assign the same port numbers, such as: `server.1=system1:2888:3888 server.2=system2:2888:3888 server.3=system3:2888:3888` However, for a testing environment, you may want to place all membership servers on the same host. In this case, you need to configure all membership servers with different ports.
`4lw.commands.whitelist`	Enables the specified ZooKeeper four-letter-words commands. TimesTen Scaleout utilities like `ttGridRollout` require some of these commands to operate properly.

All membership servers that are installed should be run in replicated mode. To run your membership servers in replicated mode, you need to include the tickTime, initLimit, and syncLimit parameters and provide the host name with two port numbers for each membership server.

Note:

For more details on replicated mode, go to the Apache ZooKeer website.

Then, refer to the Getting Started > Running Replicated ZooKeeper section of the documentation.

The following example demonstrates the zoo.cfg membership server configuration file, where there are three membership servers installed on hosts whose DNS names are ms_host1, ms_host2 and ms_host3. All three membership servers are configured to run in replicated mode.

# The number of milliseconds of each tick
tickTime=250
# The number of ticks that the initial synchronization phase can take
initLimit=40
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=12
# The directory where you want the ZooKeeper data stored. 
dataDir=/grid/membership/apache-zookeeper-3.8.4-bin/data
# The port at which the clients will connect
clientPort=2181
# Every hour, keep the latest three Apache ZooKeeper snapshots and 
# transaction logs and purge the rest
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
# The minimum and maximum allowable timeouts for Apache ZooKeeper sessions. 
# Actual timeout is negotiated at connect time.
minSessionTimeout=2000
maxSessionTimeout=10000
# The membership servers
server.1=ms_host1:2888:3888
server.2=ms_host2:2888:3888
server.3=ms_host3:2888:3888
# Enabled ZooKeeper four-letter-words commands
4lw.commands.whitelist=stat, ruok, conf, isro

Note:

There is a sample file that explains some of the parameters for your zoo.cfg file in the Apache ZooKeeper installation /conf directory called zoo_sample.cfg. However, it does not have all of the recommended parameters or settings for TimesTen Scaleout. Use zoo_sample.cfg for reference only.

This example creates a myid text file on three hosts, where each is a membership server. Each myid text file contains a single-line with the server id (an integer) corresponding to one of the membership servers configured in the zoo.cfg file. The server id is the number x in the server.x= entry of the configuration file. The myid text file must be located within the data directory on each membership server. The data directory location is /grid/membership/apache-zookeeper-3.8.4-bin/data.

Create a myid text file in the /grid/membership/apache-zookeeper-3.8.4-bin/data directory on ms_host1 for its membership server. The myid text file contains the value 1.
Create a myid text file in the /grid/membership/apache-zookeeper-3.8.4-bin/data directory on ms_host2 for its membership server. The myid text file contains the value 2.
Create a myid text file in the /grid/membership/apache-zookeeper-3.8.4-bin/data directory on ms_host3 for its membership server. The myid text file contains the value 3.

When the membership server starts up, it identifies which server it is in by the integer configured in the myid file in the ZooKeeper data directory.

Note:

For full details of the configuration parameters that can exist in the Apache ZooKeeper zoo.cfg configuration file, see the Apache ZooKeer website.