Planning your Grid

Determine the Number of Hosts and Membership Servers

You need to determine how many hosts and membership servers you are going to use based on these considerations:

Membership servers: In a production environment, you need an odd number of membership servers greater than or equal to three to ensure a majority quorum in case one or more membership servers fail. You should ensure that:
- Each membership server uses independent physical resources (such as power, network nodes, and storage) from each other.
- Membership servers do not run on the same system as hosts with data instances.
Management instances: You need two management instances to ensure some measure of availability to the configuration and management capabilities of your grid. Ensure that hosts with management instances use independent physical resources (such as power, network nodes, and storage) from each other.
Data instances: You determine the number of hosts you require for data instances based on the level of K-safety and the number of replica sets. For example, if you set k to 3 and you decide to have two replica sets, you need six data instances.

Also, the level of K-safety determines how many data space groups or independent physical locations you must have for your hosts. Ensure that the hosts with data instances assigned to data space group 1 use independent physical resources than hosts with data instances that are assigned to data space group 2 and so on.

Figure 1-11 shows an example of a setup of three membership servers, one repository, two management instances, and six data instances. The example co-locates a membership server with the repository for a total of 11 hosts.

Figure 1-11 Example of a Grid

Description of "Figure 1-11 Example of a Grid"

Figure 1-12 shows how the hosts with data instances in this example are organized into three data space groups for a grid with k set to 3. The hosts of each data space group share a rack.

Figure 1-12 Example of Hosts Organized into Data Space Groups

Description of "Figure 1-12 Example of Hosts Organized into Data Space Groups"

See Table 1-1 for an example of how you might assign the hosts with data instances into data space groups based on the physical resources they share.

Table 1-1 Systems and Their Roles

Host name	Membership server	Repository	Management instance	Data instance	Data space group	Physical resources
ms_host1	Yes	Yes	-	-	-	Rack 1
ms_host2	Yes	-	-	-	-	Rack 2
ms_host3	Yes	-	-	-	-	Rack 3
host1	-	-	Yes	-	-	Rack 1
host2	-	-	Yes	-	-	Rack 2
host3	-	-	-	Yes	1	Rack 1
host4	-	-	-	Yes	2	Rack 2
host5	-	-	-	Yes	3	Rack 3
host6	-	-	-	Yes	1	Rack 1
host7	-	-	-	Yes	2	Rack 2
host8	-	-	-	Yes	3	Rack 3

Define the Network Parameters of Each Host and Membership Server

Ensure that you know the network addresses and TCP/IP ports that you expect each host and membership server to use. See Network Requirements.

See Table 1-2 for an example of the internal and external addresses of the topology described in Table 1-1.

Table 1-2 Internal and External Addresses

Host name	Internal address	External address
ms_host1	`ms-host1`	-
ms_host2	`ms-host2`	-
ms_host3	`ms-host3`	-
host1	`int-host1`	-
host2	`int-host2`	-
host3	`int-host3`	`ext-host3.example.com`
host4	`int-host4`	`ext-host4.example.com`
host5	`int-host5`	`ext-host5.example.com`
host6	`int-host6`	`ext-host6.example.com`
host7	`int-host7`	`ext-host7.example.com`
host8	`int-host8`	`ext-host8.example.com`

Note:

All systems must be part of the same internal network. It is recommended that you create an external network for applications outside of your internal network to connect to your database.

You need to consider which TCP/IP ports each instance will use, especially if your setup is behind a firewall. You must define the TCP/IP ports for the following:

Membership servers: You must define three port numbers (client, peer, and leader) for each membership server. See Table 3-1.
Management instances: There are three port numbers (daemon, server, and management) for each management instance. TimesTen Scaleout sets the default values for the daemon, server, and management ports if you do not specify them.
Data instances: There are two port numbers (daemon and server) for each data instance. TimesTen Scaleout sets the default values for the daemon and server ports if you do not specify them.

If a firewall is in place, you must open all the ports mentioned above plus the local ephemeral ports for the internal network, except the server ports assigned to each instance. The server ports assigned to each instance must be open for the external network.

See Table 1-3 for an example of the TCP/IP ports assigned to each membership server or instance. The example uses the default values for each port.

Table 1-3 TCP/IP Ports

Host name	Membership server (client/peer/leader)	Management instance (daemon/server/management)	Data instance (daemon/server)
ms_host1	2181 / 2888 / 3888	-	-
ms_host2	2181 / 2888 / 3888	-	-
ms_host3	2181 / 2888 / 3888	-	-
host1	-	6624 / 6625 / 3574	-
host2	-	6624 / 6625 / 3574	-
host3	-	-	6624 / 6625
host4	-	-	6624 / 6625
host5	-	-	6624 / 6625
host6	-	-	6624 / 6625
host7	-	-	6624 / 6625
host8	-	-	6624 / 6625

Define the Locations for the Installation Directory and Instance Home of Each Instance

You must define the locations for the installation directory and the instance home that you expect your grid to use. Defining the locations for these grid objects includes defining the name TimesTen Scaleout uses to identify them. Consider these while defining these locations:

In the case of the instance home, TimesTen Scaleout adds the instance name to the defined location. For example, if you define /grid as the location for an instance named instance1, the full path for the instance home of that instance becomes /grid/instance1.
A similar behavior applies for installation objects. Instead of adding the installation name, TimesTen Scaleout adds the release version to the defined location. For example, if you define /grid as the location of the installation, the full path for the installation becomes /grid/tt22.1.1.18.0.

TimesTen Scaleout creates the locations you define for the installation directory and instance home if they do not exist already.

See Table 1-4 for an example of the locations for the membership server installation. You must create these locations on their respective systems prior to installing the membership server.

Table 1-4 Installation Location of the Membership Servers

Host name	Installation location
ms_host1	`/grid/membership`
ms_host2	`/grid/membership`
ms_host3	`/grid/membership`

See Table 1-5 for an example of the installation directory and instance home locations for the management instances.

Table 1-5 Installation Directory and Instance Home of the Management Instances

Host name	Installation name	Installation directory	Instance name	Instance home
host1	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`
host2	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`

See Table 1-6 for an example of the installation directory and instance home locations for the data instances.

Table 1-6 Installation Directory and Instance Home of the Data Instances

Host name	Installation name	Installation directory	Instance name	Instance home
host3	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`
host4	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`
host5	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`
host6	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`
host7	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`
host8	installation1	`/grid/tt22.1.1.18.0`	instance1	`/grid/instance1`

See Table 1-7 for an example of the location for the repository.

Table 1-7 Repository

Host name	Repository location
ms_host1	`/grid/repository`

Ensure You Have All the Information You Need to Deploy a Grid

To verify that you have all the information you need before you start deploying your grid, answer the questionnaire provided in Table 1-8.

Table 1-8 Questionnaire

Question	Source of information
What will your K-safety setting be?	K-Safety
How many membership servers will you have?	Determine the Number of Hosts and Membership Servers and Setting Up the Membership Service
How many management instances will you have?	Management Instances
How many replica sets will you have?	Data Instances and Understanding Replica Sets
Where will you store your database backups?	Backups and Define the Locations for the Installation Directory and Instance Home of Each Instance
How many hosts are you going to use for your grid?	Determine the Number of Hosts and Membership Servers
Which of those hosts are going to run management instances?	Management Instances
Which of those hosts are going to run data instances?	Data Instances
What will be the data space group assignments of each host with a data instance?	Assigning Hosts to Data Space Groups
How will you organize your hosts and membership servers across independent physical resources?	Assigning Hosts to Data Space Groups
Will you use a single network or separate internal and external networks for your grid?	Internal and External Networks
What is the DNS name or IP address of each host and membership server?	Define the Network Parameters of Each Host and Membership Server
Which TCP/IP ports will you use for each instance?	Define the Network Parameters of Each Host and Membership Server
What will be the location for the installation files of each membership server?	Define the Locations for the Installation Directory and Instance Home of Each Instance
What will be the locations for the installation directory and instance home of each instance?	Define the Locations for the Installation Directory and Instance Home of Each Instance