This chapter discusses how to create and configure a grid in TimesTen Scaleout.
A grid is a set of associated instances that contain the distributed data of one or more databases. There are two types of instances in a grid:
Management instances control a grid and maintain the model, which is the central configuration of a grid. You can configure up to two management instances to provide availability for the management of the grid.
Data instances store the data of every database managed by the grid.
These sections describe the tasks to set up a grid:
Note:
While this chapter describes the tasks necessary to completely configure a grid by using the command line and thettGridAdmin
utility, it is also possible to configure a grid by using Oracle SQL Developer. See "Working with TimesTen Scaleout" in the Oracle SQL Developer Oracle TimesTen In-Memory Database Support User's Guide for more information.
Additionally, TimesTen Scaleout provides the ttGridRollout
to quickly set up a simple grid with a single database for development and testing purposes. See "ttGridRollout" in the Oracle TimesTen In-Memory Database Reference and "Deploy a grid and database" in this document for more information on the utility and an example of its use, respectively.
TimesTen Scaleout uses management instances to configure and manage a grid. A management instance stores and maintains the model, a comprehensive list of the objects that give shape to a grid.
Important:
TimesTen Scaleout stores multiple versions of the model that may describe a previous, present, or desired structure of a grid. See "Model versioning" for more information on how these versions of the model are created or managed.
Most model objects have a user-defined name. TimesTen Scaleout uses those names to define relationships between model objects. In general, each type of model object has its own namespace. See "Grid objects and object naming" in the Oracle TimesTen In-Memory Database Reference for further details.
See "Central configuration of the grid" for a complete list of the types of model objects and their descriptions.
To ensure high availability for the management of the grid, TimesTen Scaleout enables you to create a standby management instance in an active standby configuration. It is highly recommended that you configure a standby management instance, which would be available in the case of a failure of the active management instance. If you only have a single management instance and it fails, the databases remain operational, but most management operations are unavailable until the management instance is restored. The steps to set up a standby management instance are discussed later in this chapter.
The ttInstanceCreate
utility creates new instances. You create the initial management instance with the ttInstanceCreate
utility by including the -grid
option to enable the instance for TimesTen Scaleout management. Once you create a grid from this instance, all subsequent instances associated with the grid are created through the ttGridAdmin
utility. All instances in the grid share the same OS username as instance administrator.
Note:
The tasks described in this and the next several sections use a scenario of a grid with a K-safety (k
) set to 2
and that consists of eight hosts: two hosts with a TimesTen installation and a management instance, and two data space groups with three hosts each, each host with a TimesTen installation and a data instance. Figure 4-1 shows a graphical representation of this scenario.On a host with a TimesTen 18.1 installation, create a management instance in a location of your choice, for example, the/grid
directory.
Note:
See Chapter 2, "Prerequisites and Installation of TimesTen Scaleout" for information on how to install TimesTen and its prerequisites for TimesTen Scaleout.% /grid/tt18.1.4.1.0/bin/ttInstanceCreate -name instance1 -location /grid -grid Creating instance in /grid/instance1 ... INFO: Mapping files from the installation to /grid/instance1/install NOTE: The TimesTen daemon startup/shutdown scripts have not been installed. The startup script is located here : '/grid/instance1/startup/tt_instance1' Run the 'setuproot' script : /grid/instance1/bin/setuproot -install This will move the TimesTen startup script into its appropriate location. The 18.1.4.1 Release Notes are located here : '/grid/tt18.1.4.1.0/README.html'
Note:
TimesTen Scaleout sets instance1
as the default instance name of new instances when you create them with the ttGridAdmin
utility. Subsequent instances that you create on the same host require that you provide a different name for the instances. The example uses instance1
to stay in line with the default value. You may use the name of your choice.
TimesTen Scaleout creates a subdirectory with the instance name in the specified location. TimesTen Scaleout creates all instance files in this subdirectory. For example, the instance files of the instance1
management instance are allocated in the /grid/instance1
directory of the local system.
TimesTen Scaleout sets the default values for the TCP/IP port numbers of the instance daemon and server (6624
and 6625
, respectively) if you do not specify a value for the port numbers. Use the -daemonPort
or -csPort
options of the ttInstanceCreate
utility to set different values for the port numbers.
Ensure that you set the environment variables for the instance1
management instance with the ttenv
script (ttenv.csh
or ttenv.sh
) appropriate for your shell.
For a Bourne-type shell, such as sh
, bash
, zsh
, or ksh
:
$ . /grid/instance1/bin/ttenv.sh
For a csh
or tcsh
shell:
% source /grid/instance1/bin/ttenv.csh
For more information on the ttInstanceCreate
utility, see "ttInstanceCreate" in the Oracle TimesTen In-Memory Database Reference.
See "Environment variables" for more information on the environment variables.
You can manipulate the state and configuration of a grid with the ttGridAdmin
utility. All operations that require the use of the ttGridAdmin
utility must be performed by the instance administrator and from the active management instance, unless stated otherwise. Use this utility to perform all the operations related to the configuration and maintenance of a grid, which include:
Creating a new grid
Creating and removing model objects such as hosts and instances
Creating and destroying databases
Defining and modifying how the user data is distributed across the available data instances
Modifying the attributes of model objects such as the connection attributes of the databases
Querying the status of the grid and its databases
Maintaining the different versions of the model
Applying the changes made to the latest version to the model to the operational grid.
Note:
For more information on thettGridAdmin
utility, see "ttGridAdmin" in the Oracle TimesTen In-Memory Database Reference.The ttGridAdmin gridCreate
command performs the next operations:
Starts the active management instance.
Creates a grid with a user-defined name.
Creates the required number of data space groups as indicated by the value of K-safety.
Defines the client configuration of the membership service. See Chapter 3, "Setting Up the Membership Service" for details on the membership service and the required ZooKeeper client configuration file.
Adds the management instance and its associated host and installation as model objects to the latest version of the model.
Create a grid with k
set to 2
. Specify a name for the grid, the internal address or the internal and external address of the local system, and provide the ZooKeeper client configuration file.
% ttGridAdmin gridCreate grid1 -k 2 -internalAddress int-host1 -externalAddress ext-host1.example.com -membershipConfig /tmp/membership.conf Grid grid1 created
Note:
If you do not specify the -host
option of the ttGridAdmin gridCreate
command, TimesTen Scaleout sets the hostname of the local system as the name of the host in the model.
TimesTen Scaleout automatically identifies the local TimesTen installation as the installation1
installation.
TimesTen Scaleout sets the default value for the TCP/IP port number of the replication agent of the active management instance (3754
) if you do not specify a value for the port number. Use the -mgmtPort
option in the ttGridAdmin gridCreate
command to specify a different value for the port number.
To create the grid1
grid, TimesTen Scaleout starts the instance1
management instance. Then, the instance1
management instance creates the grid1
grid and its model. Finally, the instance1
management instance performs these operations in the model of the grid1
grid:
Creates a host object, host1
, in the model to represent the local system.
Creates an installation object, installation1
, in the model to represent the local TimesTen installation.
Creates an instance object, instance1
, in the model.
Associates the installation1
installation with both the host1
host and the instance1
instance.
Creates two data space groups (since k
is set to 2
).
Important:
From this point forward, the described tasks only add and modify model objects to the latest version of the model and do not make any changes on the systems associated with such model objects until the changes made to the latest version of the model are applied. See "Applying the changes made to the model" for full details.Figure 4-2 shows a graphical representation of the model after the creation of the grid1
grid.
Figure 4-2 The model after creating a grid
For more information on the ttGridAdmin gridCreate
command, see "Create a grid (gridCreate)" in the Oracle TimesTen In-Memory Database Reference.
TimesTen Scaleout enables you to create a second management instance for a grid. When two management instances exist in a grid, the configuration of the grid is replicated from the active management instance to the standby management instance using an active standby configuration. Replication between the active and standby management instances is asynchronous. See "Managing failover for the management instances" for more information on how TimesTen Scaleout uses an active standby configuration for the management instances.
TimesTen Scaleout automatically configures the second management instance as the standby. All operations that use and manipulate the configuration of the grid must be performed from the active management instance with the ttGridAdmin
utility.
It is highly recommended that every management instance that you configure in a grid is located on a different host. Those hosts should be in different failure domains (with independent power, storage, and other resources). You must manually add every host to the model by providing the communication parameters (fully qualified domain or IP address) of the system they are associated with.
The ttGridAdmin hostCreate
command defines a host object in the model. This command enables you to create an instance (management or data) and copy the attributes, such as the data space group, of an existing host by using the -like
option. In addition, you have the option to copy the associated installations and instances by using the -cascade
option along with the -like
option.
Create a standby management instance and its associated installation by duplicating the host associated with the active management instance, host1.instance1
. Ensure that you identify the fully qualified domain name or IP address of the new host.
% ttGridAdmin hostCreate -internalAddress int-host2 -externalAddress ext-host2.example.com -like host1 -cascade Host host2 created in Model Installation installation1 created in Model Instance instance1 created in Model
Note:
If you do not specify a name for the host, TimesTen Scaleout sets the OS hostname of the remote system as the name of the new host.
Any additional options you define in the ttGridAdmin hostCreate
command will overwrite the attributes inherited from the existing host in the new host. In this example, TimesTen Scaleout uses the same values for the daemon, server, and management ports (6624
, 6625
, and 3754
, respectively) as the values set for the host1.instance1
management instance.
This example uses the -like
and -cascade
options of the ttGridAdmin hostCreate
command to create the standby management instance and its associated host and installation. Alternatively, you can create them separately. See "Adding data instances" for more details.
Figure 4-3 shows a graphical representation of the model of the grid1
grid after creating the host2
host, host2.installation1
installation and host2.instance1
management instance.
Figure 4-3 The model after creating the standby management instance
Notice that the names assigned to the installation and management instance created for the host2
host are identical to the names assigned to the host1
host, a result of the cascade operation. This does not generate a conflict, since the fully qualified names are different. See "Grid objects and object naming" in the Oracle TimesTen In-Memory Database Reference for more information.
For more information on the ttGridAdmin hostCreate
command, see "Create a host (hostCreate)" in the Oracle TimesTen In-Memory Database Reference.
A database is distributed across multiple data instances that collectively provide a single database image. Data instances reside on hosts. You create each host and data instance that is to be included in the grid. Thus, you need to calculate how many hosts and data instances to create when you are designing your grid.
The number of copies of the data that you define for the value of K-Safety (k
) is a factor for how many data instances and hosts that you need to create for your grid. If you define a duplicate copy of the data by setting k
set to 2
, then you need twice as many data instances and hosts as when a single copy of the data is requested with k
set to 1
.
Note:
2
is the maximum number that you can assign as the value for k
.The number of data instances that you create depends on two factors:
The value of k
: If you set k
to 1, the number of data instances you create equals the number of elements you desire for each database. If you set k
to 2
, then you need to create twice as many data instances, one set of data instances to manage each copy of the database contained within the replica sets.
The number of replica sets across which you want the data distributed: The number of data instances you create is dictated by the number of elements in all replica sets, since each data instance manages one element of each database.
All elements that make up a single copy of the database are assigned within a data space. If you set k
to 2 for two copies of the database, then each replica set contains two elements, where each element is an exact copy of the other element in the replica set. Each data space contains one of the replica elements of each replica set.
Note:
Each data space logically contains a full copy of the data for the database. Since there are k copies of the data, there are k data spaces.Data instances are assigned to data spaces based on how hosts are assigned to data space groups.
To calculate the number of replica sets across which you want the data distributed, determine the maximum of the two values below:
Database size versus host memory size. The size of the database and the amount of memory you have on each host determines the number of replica sets you want. For example, if you have a two Terabyte database and hosts with 512 Gigabytes of memory each, then you need at least four replica sets to hold all of the data. More likely that you will need five hosts, since you cannot use all of the memory on each host for the data.
Throughput. Even if your database is small enough to fit in the memory of a single host, you need to spread your data over multiple hosts if a single host cannot handle the number of transactions per second that your applications require.
Once you decide on the number of replica sets, you can calculate the number of data instances.
For the equation to find the number of data instances required, r represents the number of replica sets (where each replica set contains 1 or 2 elements) and k represents the K-safety value which denotes the number of copies of the data and subsequently, the number of elements in each replica set. To create enough data instances, you need to create k * r data instances.
number of data instances = k * r
For example, if you set k
to 2
for two copies of the database and each copy of the database is to be distributed across three replica sets, then you need to create 6 data instances where three data instances contain replica elements for data space 1
and three data instances contain the replica elements for data space 2
.
See "Understanding data spaces" for more details on data spaces. See "Understanding replica sets" for more details on replica sets.
To calculate the number of physical or virtual systems for a production deployment of your grid involves considering:
Hosts for the membership servers. See "Using Apache ZooKeeper as the membership service" for details on the number of hosts needed for the membership servers.
Hosts for the management instances. See "Adding the standby management instance" for details on the number of hosts needed for the management instances.
Hosts for the data instances. This section describes how many hosts you need to support the number of data instances in your grid.
The number of hosts that you need depends on the how many data instances you install on each host. The following is described in "Data instances".
Each data instance normally resides on a separate host to provide maximum data availability and as a guard against data loss should one of the hosts fail. However, you might want to run multiple data instances on a single host if:
The hosts in the grid contain a large amount of computing resources.
For experimentation of a larger grid before deployment, you might want to test a larger grid configuration on a smaller number of hosts.
Thus, to decide on the number of hosts:
If you install a single data instance on each host, then the number of hosts required is the same number of data instances in the grid. For example, if you have six data instances, then you would create six hosts.
If you install more than one data instance on each host, then the number of hosts required depends on how many data instances are on each host. For example, if you have eight data instances and you want to install two data instances on each host, then you only need four hosts.
Once you create the hosts for data instances, you assign them to a data space group. See "Assigning hosts to data space groups" for details.
Data instances will not be created on hosts that are not part of a data space group. The number of data space groups depends on the value set for k
. If k
is set to 2
, then you will have two data space groups.
As described in "Assigning hosts to data space groups", adding hosts to data space groups specifies the physical location of your data. The hosts in one data space group should be physically separate from the group of hosts in another data space group to protect each full copy of the database from hardware failures.
Figure 4-4 is an example of a grid with two data space groups, each containing a single copy of the data. Six hosts contain data instances with the three replica sets that support two copies of the data in a K-safety environment where k
is set to 2
. Three hosts with data instances that contain a single copy of the data are assigned to data space group 1; the other three hosts with data instances that contain the second copy of the data are assigned to data space group 2.
You assign the hosts to data space groups so that there is an equal number of hosts in each data space group. However, you can assign the hosts to each data space group later or ask TimesTen Scaleout to recommend host assignments for you.
Note:
TimesTen Scaleout cannot create data instances on a host unless the host has been assigned to a data space group.You can assign the host to the data space group as part of the host creation with the -dataspacegroup
option of the ttGridAdmin hostCreate
command. "Adding data instances" shows examples of this option.
You can create the host and assign it to a data space group later with the ttGridAdmin hostModify -dataspacegroup
command.
If you have a complicated physical topology, you can use the ttGridAdmin dataSpaceGroupSuggest
command to receive data space group assignment recommendations from TimesTen Scaleout for multiple hosts with the ttGridAdmin dataSpaceGroupSuggest
command. See "Description of the physical topography of the grid" for more details on this method.
A data instance contains an element for every single database defined in the grid. An element stores a portion of the data of a single database. The data may be distributed among a number of elements equal to the number of data instances defined in the grid.
Perform the following tasks to create data instances in a grid:
Note:
Remember that the operations described in the following sections only modify the latest version of the model and do not become part of the operational grid until those changes are applied. See "Applying the changes made to the model" for full details.As with a host associated with a management instance, for every system you intend to use to store a portion of the data of a database, you must manually add the system as a host model object. Likewise, you must provide the communication parameters (fully qualified domain or IP address) of the system. Although, each host can have more than one data instance, it is recommended that you only configure one data instance per host.
To create a data instance, you need to associate the host with a data space group. All data space groups must be associated with the same number of data instances. If you follow the recommendation of one data instance per host, all data space groups must be associated with the same number of hosts.
As mentioned in "Adding the standby management instance", the ttGridAdmin hostCreate
command creates a host in the grid. You can associate the host with a data space group or physical group at host creation or later.
Note:
If you do not initially associate a host with a data space group, you have the option to let TimesTen Scaleout analyze the model after you have created all the hosts and suggest which hosts to associate with each data space group, based on the physical groups associated with each host, with the use of thettGridAdmin dataSpaceGroupSuggest
command. See "Describe your physical topology with physical groups" and "Assigning hosts to data space groups" for more information.Create a host for a data instance and associate it with data space group 1
. Ensure that you identify the fully qualified domain name or IP address of the host.
% ttGridAdmin hostCreate -internalAddress int-host3 -externalAddress ext-host3.example.com -dataSpaceGroup 1 Host host3 created in Model
Note:
If you do not specify a name for the host, TimesTen Scaleout sets the OS hostname of the remote system as the name of the new host.Figure 4-5 shows a graphical representation of the model of the grid1
grid after creating the host3
host.
Figure 4-5 The model after adding a host for a data instance
For more information on the ttGridAdmin hostCreate
command, see "Create a host (hostCreate)" in the Oracle TimesTen In-Memory Database Reference.
Every host must have an installation associated with it. A host can either have its own copy of the installation files or share an installation with one or more hosts through network-attached storage. For shared installations, an installation model object with the location of the shared installation files must be associated with the host.
For more information on how to share a TimesTen installation, see "Copying an installation on Linux/UNIX" in the Oracle TimesTen In-Memory Database Installation, Migration, and Upgrade Guide.
The ttGridAdmin installationCreate
command creates an installation in the grid and associates it with a host.
Create an installation in a directory of your choice in the host3
host, for example, the /grid
directory.
% ttGridAdmin installationCreate host3 -location /grid Installation installation1 on Host host3 created in Model
Note:
If you do not specify a name for the installation, TimesTen Scaleout sets installation1
as the name of the installation. Any subsequent installation associated with the same host requires that you provide a name for it.
If the management instance running the command has only one installation associated with it and the source for the installation files is not specified in the -source
option of the ttGridAdmin installationCreate
command, TimesTen Scaleout copies the installation files from the installation associated with the management instance running the command.
Figure 4-6 shows a graphical representation of the model of the grid1
grid after creating the installation1
installation in the host3
host.
Figure 4-6 The model after adding an installation for the data instance
For more information on the ttGridAdmin installationCreate
command, see "Create an installation (installationCreate)" in the Oracle TimesTen In-Memory Database Reference.
The ttGridAdmin instanceCreate
command creates an instance in the grid and associates it with a host and installation.
Create a data instance in the location of your choice in the host3
host, for example, the /grid
directory.
% ttGridAdmin instanceCreate host3 -location /grid Instance instance1 on Host host3 created in Model
Note:
If you do not specify a name for the instance, TimesTen Scaleout sets instance1
as the name of the instance. Any subsequent instances associated with the same host requires that you provide a name for it.
Because the host3
host only has the installation1
installation associated with it, the installation1
installation is associated with the instance1
data instance by default and there is no need to specify the -installation
option.
TimesTen Scaleout defines an instance as a data instance by default. Use the -type
management option of the ttGridAdmin instanceCreate
command to create a management instance.
TimesTen Scaleout creates a subdirectory with the instance name in the specified location. TimesTen Scaleout allocates all instance files in this subdirectory. For example, the instance files of the instance1
data instance are allocated in the /grid/instance1
directory of the host3
host.
TimesTen Scaleout sets the default values for the TCP/IP port numbers of the instance daemon and server (6624
and 6625
, respectively) if you do not specify a value for the port numbers. Use the -daemonPort
or -csPort
options of the ttGridAdmin instanceCreate
utility to set different values for the port numbers.
Figure 4-7 shows a graphical representation of the model of the grid1
grid after creating a data instance.
Figure 4-7 The model after adding a data instance
For more information on the ttGridAdmin instanceCreate
command, see "Create an instance (instanceCreate)" in the Oracle TimesTen In-Memory Database Reference.
As mentioned in "Adding the standby management instance", you can create an instance (management or data) by duplicating the configuration of an existing host, including its associated installations and instances with the -like
and -cascade
options of the ttGridAdmin hostCreate
command.
The -like
option identifies the host to be duplicated and associates the new host with the same physical groups and data space group. You can override the physical groups and data space group associated with the new host by providing different parameters in the -physicalGroup
and -dataSpaceGroup
options, respectively.
The -cascade
option duplicates the configuration of the installations and data instances associated with the specified host.
Create five data instances based on the same attributes defined for the host3
host and its associated installation and data instance. Also, associate three hosts with data space group 2
, instead of data space group 1
. Ensure that you identify the fully qualified domain name of the new hosts.
% ttGridAdmin hostCreate -internalAddress int-host4 -externalAddress ext-host4.example.com -like host3 -cascade -dataSpaceGroup 2 Host host4 created in Model Installation installation1 created in Model Instance instance1 created in Model % ttGridAdmin hostCreate -internalAddress int-host5 -externalAddress ext-host5.example.com -like host3 -cascade Host host5 created in Model Installation installation1 created in Model Instance instance1 created in Model % ttGridAdmin hostCreate -internalAddress int-host6 -externalAddress ext-host6.example.com -like host4 -cascade Host host6 created in Model Installation installation1 created in Model Instance instance1 created in Model % ttGridAdmin hostCreate -internalAddress int-host7 -externalAddress ext-host7.example.com -like host3 -cascade Host host7 created in Model Installation installation1 created in Model Instance instance1 created in Model % ttGridAdmin hostCreate -internalAddress int-host8 -externalAddress ext-host8.example.com -like host4 -cascade Host host8 created in Model Installation installation1 created in Model Instance instance1 created in Model
Note:
If you do not specify a name for the host, TimesTen Scaleout sets the OS hostname of the remote system as the name of the new host.
Any additional option you define in the ttGridAdmin hostCreate
command will overwrite the attributes inherited from the existing host in the new host, as shown with the addition of the -dataSpaceGroup 2
parameter in the command that creates the host6
host.
Notice that the ttGridAdmin hostCreate
commands that create the host6
and host8
hosts use the host4
host as reference in the -like
option.
Figure 4-8 shows a graphical representation of the model of the grid1
grid after duplicating the host3
host as the host4
, host5
, host6
, host7
, and host8
hosts and their associated installations and instances.
Figure 4-8 The model after duplicating an existing host
For more information on the ttGridAdmin hostCreate
command, see "Create a host (hostCreate)" in the Oracle TimesTen In-Memory Database Reference.
The latest version of the model describes the desired structure of a grid, not its current structure. Any changes made to the latest version of the model are not immediately reflected in the operational configuration of a grid. Changes made to the latest version of the model need to be explicitly applied to the grid.
Management instances store multiple versions of the model. Only one version of the model can be active in the grid at any given time.
TimesTen Scaleout classifies model versions as follows:
Current version: The current version of the model describes the operational configuration of the grid. This version, and all previous versions, is read-only.
Latest version: The latest version of the model can be modified and has yet to be applied to the grid. This version is read/write.
When you create a grid, the version 1
model is initially populated with the configuration of the first host, installation, and management instance, and the version 1
model is recognized as the latest version of the model. Any subsequent changes that you make to the model are added to the latest version of the model (version 1
). When you implement these changes with the ttGridAmin modelApply
command, a new latest version of the model (version 2
) is created for future changes and the previous latest version of the model (version 1
) becomes the current version of the model.
Every time you run the ttGridAdmin modelApply
command, TimesTen Scaleout:
Makes the latest version of the model (version
n
) read-only.
Creates a writable copy (version
n+1
) of the latest version of the model.
Attempts to apply the changes previously made to the version
n
model to the operational grid.
Identifies the version
n
model as the current version of the model.
Identifies the version
n+1
model as the latest version of the model.
The ttGridAdmin
utility enables the user to perform several operations regarding the model, like:
Applying the changes made to the latest version of the model
Comparing two versions of the model
Exporting a version of the model into a flat file in JSON format
Importing a flat file in JSON format as the latest version of the model
Listing all the available versions of the model
Previous versions of the model are automatically stored. With the ttGridAdmin gridModify
command, you can specify the retention period for old versions of the model either in terms of days, in terms of the number of stored versions, or both. TimesTen Scaleout by default retains the last 10 versions for a period of 30 days.
For more information on model operations or the ttGridAdmin gridModify
utility, see "Model operations" or "Modify grid settings (gridModify)", respectively, in the Oracle TimesTen In-Memory Database Reference.
The ttGridAdmin modelApply
command attempts to apply the changes made to the latest version of the model into the operational grid. If, for example, you add a new data instance to the latest version of the model, running this command performs all of the necessary operations to create and initialize the instance in the specified host. Some of the operations that the ttGridAdmin modelApply
command performs include these:
Identify and delete any object removed from the latest version of the model.
Create new installations.
Create new instances, data and management.
Overwrite the configuration files of all instances. The new versions of these files include any new entries found in the latest version of the model.
Verify the SSH connectivity between hosts.
Note:
If you recently added new instances to the model or have yet to set the required passwordless SSH access to the hosts managing instances, either manually set the required passwordless SSH access for the instance administrator or use thettGridAdmin gridSshConfig
command before applying the latest version of the model. See "Setting passwordless SSH" for more information.Start new instances.
Apply all the changes made to the latest version of the model of the grid1
grid.
% ttGridAdmin modelApply Creating new model version............................................OK Exporting current model (version 1)...................................OK Identifying any changed management instances..........................OK Identifying any deleted objects.......................................OK Verifying installations...............................................OK Verifying instances...................................................OK Creating new instances................................................OK Updating grid state...................................................OK Configuring instance authentication...................................OK Pushing new configuration files to each instance......................OK Making model version 1 current, version 2 writable....................OK Checking ssh connectivity of new instances............................OK Starting new management instance......................................OK Configuring standby management instance...............................OK Starting new data instances...........................................OK ttGridAdmin modelApply complete
Given all the tasks you performed in the previous sections, the ttGridAdmin modelApply
command performs the following operations:
Creates a copy of the installation files on every configured host:
Creates the instance home directory and files for the standby management instance and data instances on their associated hosts:
Makes the latest version of the model read-only and a creates a new writable model.
Verifies SSH connectivity to every configured host.
Starts the daemons of the standby management instance and data instances.
Configures the active and standby management instances.
For more information on the ttGridAdmin modelApply
command, see "Apply the latest version of the model (modelApply)" in the Oracle TimesTen In-Memory Database Reference.
Optionally, you can configure data instances to automatically start or shut down every time their systems boot or shut down, respectively. Each instance needs to be configured independently and only after its creation has been applied to the current version of the model.
To accomplish this, the root
user must run the setuproot
script with the -install
option. You can find this script in the timesten_home
/bin
directory of every instance of the grid. For example:
From the host of the host3
.instance1
data instance:
% /grid/instance1/bin/setuproot -install Would you like to install the TimesTen daemon startup scripts into /etc/init.d? [ yes ]
For more information on how to use the setuproot
script, see "Start an instance automatically at system startup" in the Oracle TimesTen In-Memory Database Installation, Migration, and Upgrade Guide.
As described in "Assigning hosts to data space groups", TimesTen Scaleout requires you to assign the hosts that will contain a data instance into a data space group.
Note:
The hosts in one data space group should not share physical resources with the hosts in another data space group to protect each full copy of the database from hardware failures.The example provided in "Adding data instances", demonstrates how to assign hosts to a data space group during host creation with the ttGridAdmin hostCreate -dataSpaceGroup
option:
% ttGridAdmin hostCreate -internalAddress int-host3 -externalAddress ext-host3.example.com -dataSpaceGroup 1 Host host3 created in Model
However, if your physical topography is complicated so that it is difficult to decide on the best assignments of hosts to data space groups, you can ask TimesTen Scaleout to recommend assignments of hosts to data space groups using the ttGridAdmin dataSpaceGroupSuggest
command. In order to accomplish this, you need to inform TimesTen Scaleout of the physical topology of where your hosts are co-located or hosts that use the same resources. You can inform TimesTen Scaleout of all physical hardware for each host with the physical group.
Important:
If you are going to use physical groups to have TimesTen Scaleout recommend assignments, you should not assign hosts to data space groups upon host creation. Hosts can only be assigned to data space groups once.Your physical topology may include buildings, transformers, air conditioners, racks, fans, top of rack switches, hypervisors, storage filers and power sources. For example, the following production environment includes:
2 buildings, each with their own transformer.
5 air conditioners in rooms within the 2 buildings.
9 racks of servers in these rooms, each with several hosts.
18 power surge protectors in which the hosts are plugged.
Instead of determining what would be the best data space group assignment for the hosts that share these resources, you can assign hosts to physical groups to identify where the hosts are located and what physical resources they share. This informs TimesTen Scaleout of similar potential points of failure. Thus, when TimesTen Scaleout recommends assignment of hosts to data space groups using the ttGridAdmin dataSpaceGroupSuggest
command, it physically separates the hosts that contain the separate copies of the data.
Note:
See "Assigning hosts to data space groups" for more details on data space groups. See "Get recommendations for data space group assignments (dataSpaceGroupSuggest)" in the Oracle TimesTen In-Memory Database Reference for more details on thettGridAdmin dataSpaceGroupSuggest
command.Create and manage the physical topography of your grid by:
Assigning hosts to physical groups to denote the physical topology is optional and only necessary when you want TimesTen Scaleout to assign hosts to data space groups.
The example in this section assigns the hosts to physical groups using the same host configuration of the example used in this chapter. In production, you would not use physical groups for this small example as you can easily identify which hosts should be in which data space group.
Figure 4-9 Hosts attached to physical resources
Let us assume that:
Hosts 3, 4, and 5 share an air conditioner.
Hosts 3, 4 and 5 are on the same rack.
Hosts 3 and 4 share a power source.
Host 5 has its on power source.
Hosts 6, 7 and 8 share an air conditioner.
Hosts 6, 7 and 8 are on the same rack.
Hosts 6 and 7 share a power source.
Host 8 has its own power source.
To identify which hosts are connected to which physical resources:
Define a physical group, which is simply a container object that groups hosts together to show that there is a shared physical resource. This enables TimesTen Scaleout to separate different copies of the data to hosts that do not share physical aspects.
Assign hosts to physical groups either when creating the host with the ttGridAdmin hostCreate
command or when modifying the host with the ttGridAdmin hostModify
command.
You can assign a host to one or more physical groups, as appropriate. If they are assigned to a similar rack and share a power source, you can assign the host to both of these physical groups.
See "Create a host (hostCreate)" and "Modify a host (hostModify)" in the Oracle TimesTen In-Memory Database Reference for more details on these ttGridAdmin
commands.
Example 4-1 Example of hosts that are physically located on two separate racks
This example shows how you would create physical groups to model the shared resources and assign hosts to these physical groups.
Define the physical group: Use the ttGridAdmin physicalCreate
command to create:
rack1
and rack2
to model the physical racks in the lab.
aircond1
and aircond2
to model the air conditioners.
power1
, power2
, power3
and power4
to model the power surge protectors.
Assign hosts to the physical group: Use the ttGridAdmin hostModify
to assign the hosts to each shared physical resource using the -physicalgroup
option:
host3
, host4
and host5
are on the rack1
.
host3
, host4
and host5
share aircond1
.
host3
and host4
share power1
.
host5
has its own power2
.
host6
, host7
and host8
are on rack2
.
host6
, host7
and host8
share an aircond2
.
host6
and host7
share power3
.
host8
has its own power4
.
Note:
If a host was already associated with a physical group before you execute the-physicalgroup
option of the ttGridAdmin hostModify
command, these physical groups are removed and the only physical groups associated with the host are those that are specified on the current command line.
However, you can add physical groups in addition to those physical groups that are already assigned to the existing host by using the -addPhysicalGroup
option on the ttGridAdmin hostModify
command.
% ttGridAdmin physicalCreate rack1 PhysicalGroup rack1 created. % ttGridAdmin physicalCreate rack2 PhysicalGroup rack2 created. % ttGridAdmin physicalCreate aircond1 PhysicalGroup aircond1 created. % ttGridAdmin physicalCreate aircond2 PhysicalGroup aircond2 created. % ttGridAdmin physicalCreate power1 PhysicalGroup power1 created. % ttGridAdmin physicalCreate power2 PhysicalGroup power2 created. % ttGridAdmin physicalCreate power3 PhysicalGroup power3 created. % ttGridAdmin physicalCreate power4 PhysicalGroup power4 created. % ttGridAdmin hostModify host3 -physicalgroup rack1 aircond1 power1 Host host3 modified in Model % ttGridAdmin hostModify host4 -physicalgroup rack1 aircond1 power1 Host host4 modified in Model % ttGridAdmin hostModify host5 -physicalgroup rack1 aircond1 power2 Host host5 modified in Model % ttGridAdmin hostModify host6 -physicalgroup rack2 aircond2 power3 Host host6 modified in Model % ttGridAdmin hostModify host7 -physicalgroup rack2 aircond2 power3 Host host7 modified in Model % ttGridAdmin hostModify host8 -physicalgroup rack2 aircond2 power4 Host host8 modified in Model
Note:
You can use thettGridAdmin modelExport
command to see which hosts are assigned to a physical group.If you want to remove a physical description of how the hosts are organized physically, use the -nophysicalgroup
option with the ttGridAdmin hostModify
command. The host3
host was originally associated with the physical groups rack1
, aircond1
and power1
. By executing the following command, host3
is not assigned to any physical groups.
% ttGridAdmin hostModify host3 -nophysicalgroup Host host3 modified in Model
You can delete a physical group only if no hosts are assigned to it. The following deletes the rack1
physical group after removing the host1
and host2
hosts assigned to it:
% ttGridAdmin hostModify host3 -nophysicalgroup Host host3 modified in Model % ttGridAdmin hostModify host4 -nophysicalgroup Host host4 modified in Model % ttGridAdmin hostModify host5 -nophysicalgroup Host host5 modified in Model % ttGridAdmin physicalDelete rack1 PhysicalGroup rack1 deleted.
You can use the ttGridAdmin dataSpaceGroupSuggest
command to propose an assignment of hosts to different data space groups based on how the hosts are currently assigned to physical groups. You can either accept the proposed assignments or specify your own assignments.
The ttGridAdmin dataSpaceGroupSuggest
command makes recommendations on how to assign hosts so that the hosts that contain one copy of the data do not share resources with the hosts that contain the copy of the data. Once the ttGridAdmin
command assigns the hosts to their data space groups, the assignment cannot be changed.
See "Get recommendations for data space group assignments (dataSpaceGroupSuggest)" in the Oracle TimesTen In-Memory Database Reference for more details on the ttGridAdmin dataSpaceGroupSuggest
command.
Example 4-2 Requesting TimesTen Scaleout assign hosts to data spaces
This example shows ttGridAdmin dataSpaceGroupSuggest
command, which writes its recommendations into the recommendations.sh
file. You can execute this file as it contains the ttGridAdmin hostModify
commands necessary for assigning hosts to the recommended data space groups.
% ttGridAdmin dataSpaceGroupSuggest /tmp/recommendations.sh % more /tmp/recommendations.sh #!/bin/sh # Recommendations generated by ttGridAdmin -dataSpaceGroupSuggest TIMESTEN_HOME=/grid/instance1 export TIMESTEN_HOME . $TIMESTEN_HOME/bin/ttenv.sh > /dev/null 2>/dev/null # Number of possibilities evaluated: 126 # # Number of usable possibilities found: 10 # (A 'usable' possibility is one that is compatible with pre-existing # assignments of Hosts to DataSpaceGroups) # # Number of 'ideal' possibilities found: 1 # (An 'ideal' possibility is one where no PhysicalGroups span multiple # DataSpaceGroups) # # Possibilities evaluated (best 10 displayed): # ... # # This script, if executed, would implement the only 'ideal' configuration found. # Even though this recommendation was 'ideal', you should carefully evaluate it # prior to running this script. # Host host1 is already in DataSpaceGroup 1 ttGridAdmin hostModify host3 -dataSpaceGroup 2 ttGridAdmin hostModify host4 -dataSpaceGroup 2 ttGridAdmin hostModify host5 -dataSpaceGroup 2 ttGridAdmin hostModify host6 -dataSpaceGroup 1 ttGridAdmin hostModify host7 -dataSpaceGroup 1 ttGridAdmin hostModify host8 -dataSpaceGroup 1
If you decide to accept these recommendations, execute the provided shell script, which in our example is recommendations.sh
. Once executed, all hosts are assigned to the designated data space groups.
% sh /tmp/recommendations.sh Host host3 modified in Model Host host4 modified in Model Host host5 modified in Model Host host6 modified in Model Host host7 modified in Model Host host8 modified in Model