4 Setting Up a Grid

This chapter discusses how to create and configure a grid in TimesTen Scaleout.

Configure your grid

A grid is a set of associated instances that contain the distributed data of one or more databases. There are two types of instances in a grid:

  • Management instances control a grid and maintain the model, which is the central configuration of a grid. You can configure up to two management instances to provide availability for the management of the grid.

  • Data instances store the data of every database managed by the grid.

These sections describe the tasks to set up a grid:

Note:

While this chapter describes the tasks necessary to completely configure a grid by using the command line and the ttGridAdmin utility, it is also possible to configure a grid by using Oracle SQL Developer. See "Working with TimesTen Scaleout" in the Oracle SQL Developer Oracle TimesTen In-Memory Database Support User's Guide for more information.

Additionally, TimesTen Scaleout provides the ttGridRollout to quickly set up a simple grid with a single database for development and testing purposes. See "ttGridRollout" in the Oracle TimesTen In-Memory Database Reference and "Deploy a grid and database" in this document for more information on the utility and an example of its use, respectively.

Creating the initial management instance

TimesTen Scaleout uses management instances to configure and manage a grid. A management instance stores and maintains the model, a comprehensive list of the objects that give shape to a grid.

Important:

  • TimesTen Scaleout stores multiple versions of the model that may describe a previous, present, or desired structure of a grid. See "Model versioning" for more information on how these versions of the model are created or managed.

  • Most model objects have a user-defined name. TimesTen Scaleout uses those names to define relationships between model objects. In general, each type of model object has its own namespace. See "Grid objects and object naming" in the Oracle TimesTen In-Memory Database Reference for further details.

  • See "Central configuration of the grid" for a complete list of the types of model objects and their descriptions.

To ensure high availability for the management of the grid, TimesTen Scaleout enables you to create a standby management instance in an active standby configuration. It is highly recommended that you configure a standby management instance, which would be available in the case of a failure of the active management instance. If you only have a single management instance and it fails, the databases remain operational, but most management operations are unavailable until the management instance is restored. The steps to set up a standby management instance are discussed later in this chapter.

The ttInstanceCreate utility creates new instances. You create the initial management instance with the ttInstanceCreate utility by including the -grid option to enable the instance for TimesTen Scaleout management. Once you create a grid from this instance, all subsequent instances associated with the grid are created through the ttGridAdmin utility. All instances in the grid share the same OS username as instance administrator.

Note:

The tasks described in this and the next several sections use a scenario of a grid with a K-safety (k) set to 2 and that consists of eight hosts: two hosts with a TimesTen installation and a management instance, and two data space groups with three hosts each, each host with a TimesTen installation and a data instance. Figure 4-1 shows a graphical representation of this scenario.

On a host with a TimesTen 18.1 installation, create a management instance in a location of your choice, for example, the/grid directory.

Note:

See Chapter 2, "Prerequisites and Installation of TimesTen Scaleout" for information on how to install TimesTen and its prerequisites for TimesTen Scaleout.
% /grid/tt18.1.4.1.0/bin/ttInstanceCreate -name instance1 -location
 /grid -grid
Creating instance in /grid/instance1 ...
INFO: Mapping files from the installation to /grid/instance1/install

NOTE: The TimesTen daemon startup/shutdown scripts have not been installed.

The startup script is located here :
        '/grid/instance1/startup/tt_instance1'
 
Run the 'setuproot' script :
        /grid/instance1/bin/setuproot -install
This will move the TimesTen startup script into its appropriate location.
 
The 18.1.4.1 Release Notes are located here :
  '/grid/tt18.1.4.1.0/README.html'

Note:

  • TimesTen Scaleout sets instance1 as the default instance name of new instances when you create them with the ttGridAdmin utility. Subsequent instances that you create on the same host require that you provide a different name for the instances. The example uses instance1 to stay in line with the default value. You may use the name of your choice.

  • TimesTen Scaleout creates a subdirectory with the instance name in the specified location. TimesTen Scaleout creates all instance files in this subdirectory. For example, the instance files of the instance1 management instance are allocated in the /grid/instance1 directory of the local system.

  • TimesTen Scaleout sets the default values for the TCP/IP port numbers of the instance daemon and server (6624 and 6625, respectively) if you do not specify a value for the port numbers. Use the -daemonPort or -csPort options of the ttInstanceCreate utility to set different values for the port numbers.

Ensure that you set the environment variables for the instance1 management instance with the ttenv script (ttenv.csh or ttenv.sh) appropriate for your shell.

For a Bourne-type shell, such as sh, bash, zsh, or ksh:

$ . /grid/instance1/bin/ttenv.sh

For a csh or tcsh shell:

% source /grid/instance1/bin/ttenv.csh

For more information on the ttInstanceCreate utility, see "ttInstanceCreate" in the Oracle TimesTen In-Memory Database Reference.

See "Environment variables" for more information on the environment variables.

Creating a grid

You can manipulate the state and configuration of a grid with the ttGridAdmin utility. All operations that require the use of the ttGridAdmin utility must be performed by the instance administrator and from the active management instance, unless stated otherwise. Use this utility to perform all the operations related to the configuration and maintenance of a grid, which include:

  • Creating a new grid

  • Creating and removing model objects such as hosts and instances

  • Creating and destroying databases

  • Defining and modifying how the user data is distributed across the available data instances

  • Modifying the attributes of model objects such as the connection attributes of the databases

  • Querying the status of the grid and its databases

  • Maintaining the different versions of the model

  • Applying the changes made to the latest version to the model to the operational grid.

Note:

For more information on the ttGridAdmin utility, see "ttGridAdmin" in the Oracle TimesTen In-Memory Database Reference.

The ttGridAdmin gridCreate command performs the next operations:

  • Starts the active management instance.

  • Creates a grid with a user-defined name.

  • Creates the required number of data space groups as indicated by the value of K-safety.

  • Defines the client configuration of the membership service. See Chapter 3, "Setting Up the Membership Service" for details on the membership service and the required ZooKeeper client configuration file.

  • Adds the management instance and its associated host and installation as model objects to the latest version of the model.

Create a grid with k set to 2. Specify a name for the grid, the internal address or the internal and external address of the local system, and provide the ZooKeeper client configuration file.

% ttGridAdmin gridCreate grid1 -k 2 -internalAddress int-host1
 -externalAddress ext-host1.example.com -membershipConfig
 /tmp/membership.conf
Grid grid1 created

Note:

  • If you do not specify the -host option of the ttGridAdmin gridCreate command, TimesTen Scaleout sets the hostname of the local system as the name of the host in the model.

  • TimesTen Scaleout automatically identifies the local TimesTen installation as the installation1 installation.

  • TimesTen Scaleout sets the default value for the TCP/IP port number of the replication agent of the active management instance (3754) if you do not specify a value for the port number. Use the -mgmtPort option in the ttGridAdmin gridCreate command to specify a different value for the port number.

To create the grid1 grid, TimesTen Scaleout starts the instance1 management instance. Then, the instance1 management instance creates the grid1 grid and its model. Finally, the instance1 management instance performs these operations in the model of the grid1 grid:

  • Creates a host object, host1, in the model to represent the local system.

  • Creates an installation object, installation1, in the model to represent the local TimesTen installation.

  • Creates an instance object, instance1, in the model.

  • Associates the installation1 installation with both the host1 host and the instance1 instance.

  • Creates two data space groups (since k is set to 2).

Important:

From this point forward, the described tasks only add and modify model objects to the latest version of the model and do not make any changes on the systems associated with such model objects until the changes made to the latest version of the model are applied. See "Applying the changes made to the model" for full details.

Figure 4-2 shows a graphical representation of the model after the creation of the grid1 grid.

Figure 4-2 The model after creating a grid

Description of Figure 4-2 follows
Description of ''Figure 4-2 The model after creating a grid''

For more information on the ttGridAdmin gridCreate command, see "Create a grid (gridCreate)" in the Oracle TimesTen In-Memory Database Reference.

Adding the standby management instance

TimesTen Scaleout enables you to create a second management instance for a grid. When two management instances exist in a grid, the configuration of the grid is replicated from the active management instance to the standby management instance using an active standby configuration. Replication between the active and standby management instances is asynchronous. See "Managing failover for the management instances" for more information on how TimesTen Scaleout uses an active standby configuration for the management instances.

TimesTen Scaleout automatically configures the second management instance as the standby. All operations that use and manipulate the configuration of the grid must be performed from the active management instance with the ttGridAdmin utility.

It is highly recommended that every management instance that you configure in a grid is located on a different host. Those hosts should be in different failure domains (with independent power, storage, and other resources). You must manually add every host to the model by providing the communication parameters (fully qualified domain or IP address) of the system they are associated with.

The ttGridAdmin hostCreate command defines a host object in the model. This command enables you to create an instance (management or data) and copy the attributes, such as the data space group, of an existing host by using the -like option. In addition, you have the option to copy the associated installations and instances by using the -cascade option along with the -like option.

Create a standby management instance and its associated installation by duplicating the host associated with the active management instance, host1.instance1. Ensure that you identify the fully qualified domain name or IP address of the new host.

% ttGridAdmin hostCreate -internalAddress int-host2 -externalAddress
 ext-host2.example.com -like host1 -cascade
Host host2 created in Model
Installation installation1 created in Model
Instance instance1 created in Model

Note:

  • If you do not specify a name for the host, TimesTen Scaleout sets the OS hostname of the remote system as the name of the new host.

  • Any additional options you define in the ttGridAdmin hostCreate command will overwrite the attributes inherited from the existing host in the new host. In this example, TimesTen Scaleout uses the same values for the daemon, server, and management ports (6624, 6625, and 3754, respectively) as the values set for the host1.instance1 management instance.

  • This example uses the -like and -cascade options of the ttGridAdmin hostCreate command to create the standby management instance and its associated host and installation. Alternatively, you can create them separately. See "Adding data instances" for more details.

Figure 4-3 shows a graphical representation of the model of the grid1 grid after creating the host2 host, host2.installation1 installation and host2.instance1 management instance.

Figure 4-3 The model after creating the standby management instance

Description of Figure 4-3 follows
Description of ''Figure 4-3 The model after creating the standby management instance''

Notice that the names assigned to the installation and management instance created for the host2 host are identical to the names assigned to the host1 host, a result of the cascade operation. This does not generate a conflict, since the fully qualified names are different. See "Grid objects and object naming" in the Oracle TimesTen In-Memory Database Reference for more information.

For more information on the ttGridAdmin hostCreate command, see "Create a host (hostCreate)" in the Oracle TimesTen In-Memory Database Reference.

Calculating the number of hosts and data instances for the grid

A database is distributed across multiple data instances that collectively provide a single database image. Data instances reside on hosts. You create each host and data instance that is to be included in the grid. Thus, you need to calculate how many hosts and data instances to create when you are designing your grid.

The number of copies of the data that you define for the value of K-Safety (k) is a factor for how many data instances and hosts that you need to create for your grid. If you define a duplicate copy of the data by setting k set to 2, then you need twice as many data instances and hosts as when a single copy of the data is requested with k set to 1.

Note:

2 is the maximum number that you can assign as the value for k.

Calculate the number of data instances to create

The number of data instances that you create depends on two factors:

  • The value of k: If you set k to 1, the number of data instances you create equals the number of elements you desire for each database. If you set k to 2, then you need to create twice as many data instances, one set of data instances to manage each copy of the database contained within the replica sets.

  • The number of replica sets across which you want the data distributed: The number of data instances you create is dictated by the number of elements in all replica sets, since each data instance manages one element of each database.

    All elements that make up a single copy of the database are assigned within a data space. If you set k to 2 for two copies of the database, then each replica set contains two elements, where each element is an exact copy of the other element in the replica set. Each data space contains one of the replica elements of each replica set.

    Note:

    Each data space logically contains a full copy of the data for the database. Since there are k copies of the data, there are k data spaces.

    Data instances are assigned to data spaces based on how hosts are assigned to data space groups.

    To calculate the number of replica sets across which you want the data distributed, determine the maximum of the two values below:

    • Database size versus host memory size. The size of the database and the amount of memory you have on each host determines the number of replica sets you want. For example, if you have a two Terabyte database and hosts with 512 Gigabytes of memory each, then you need at least four replica sets to hold all of the data. More likely that you will need five hosts, since you cannot use all of the memory on each host for the data.

    • Throughput. Even if your database is small enough to fit in the memory of a single host, you need to spread your data over multiple hosts if a single host cannot handle the number of transactions per second that your applications require.

Once you decide on the number of replica sets, you can calculate the number of data instances.

For the equation to find the number of data instances required, r represents the number of replica sets (where each replica set contains 1 or 2 elements) and k represents the K-safety value which denotes the number of copies of the data and subsequently, the number of elements in each replica set. To create enough data instances, you need to create k * r data instances.

number of data instances = k * r

For example, if you set k to 2 for two copies of the database and each copy of the database is to be distributed across three replica sets, then you need to create 6 data instances where three data instances contain replica elements for data space 1 and three data instances contain the replica elements for data space 2.

See "Understanding data spaces" for more details on data spaces. See "Understanding replica sets" for more details on replica sets.

Calculate the number of hosts you need to support your data instances

To calculate the number of physical or virtual systems for a production deployment of your grid involves considering:

The number of hosts that you need depends on the how many data instances you install on each host. The following is described in "Data instances".

Each data instance normally resides on a separate host to provide maximum data availability and as a guard against data loss should one of the hosts fail. However, you might want to run multiple data instances on a single host if:

  • The hosts in the grid contain a large amount of computing resources.

  • For experimentation of a larger grid before deployment, you might want to test a larger grid configuration on a smaller number of hosts.

Thus, to decide on the number of hosts:

  • If you install a single data instance on each host, then the number of hosts required is the same number of data instances in the grid. For example, if you have six data instances, then you would create six hosts.

  • If you install more than one data instance on each host, then the number of hosts required depends on how many data instances are on each host. For example, if you have eight data instances and you want to install two data instances on each host, then you only need four hosts.

Once you create the hosts for data instances, you assign them to a data space group. See "Assigning hosts to data space groups" for details.

Assigning hosts to data space groups

Data instances will not be created on hosts that are not part of a data space group. The number of data space groups depends on the value set for k. If k is set to 2, then you will have two data space groups.

As described in "Assigning hosts to data space groups", adding hosts to data space groups specifies the physical location of your data. The hosts in one data space group should be physically separate from the group of hosts in another data space group to protect each full copy of the database from hardware failures.

Figure 4-4 is an example of a grid with two data space groups, each containing a single copy of the data. Six hosts contain data instances with the three replica sets that support two copies of the data in a K-safety environment where k is set to 2. Three hosts with data instances that contain a single copy of the data are assigned to data space group 1; the other three hosts with data instances that contain the second copy of the data are assigned to data space group 2.

Figure 4-4 DataSpaceGroup example

Description of Figure 4-4 follows
Description of ''Figure 4-4 DataSpaceGroup example''

You assign the hosts to data space groups so that there is an equal number of hosts in each data space group. However, you can assign the hosts to each data space group later or ask TimesTen Scaleout to recommend host assignments for you.

Note:

TimesTen Scaleout cannot create data instances on a host unless the host has been assigned to a data space group.
  • You can assign the host to the data space group as part of the host creation with the -dataspacegroup option of the ttGridAdmin hostCreate command. "Adding data instances" shows examples of this option.

  • You can create the host and assign it to a data space group later with the ttGridAdmin hostModify -dataspacegroup command.

  • If you have a complicated physical topology, you can use the ttGridAdmin dataSpaceGroupSuggest command to receive data space group assignment recommendations from TimesTen Scaleout for multiple hosts with the ttGridAdmin dataSpaceGroupSuggest command. See "Description of the physical topography of the grid" for more details on this method.

Adding data instances

A data instance contains an element for every single database defined in the grid. An element stores a portion of the data of a single database. The data may be distributed among a number of elements equal to the number of data instances defined in the grid.

Perform the following tasks to create data instances in a grid:

Note:

Remember that the operations described in the following sections only modify the latest version of the model and do not become part of the operational grid until those changes are applied. See "Applying the changes made to the model" for full details.

Create a host for a data instance

As with a host associated with a management instance, for every system you intend to use to store a portion of the data of a database, you must manually add the system as a host model object. Likewise, you must provide the communication parameters (fully qualified domain or IP address) of the system. Although, each host can have more than one data instance, it is recommended that you only configure one data instance per host.

To create a data instance, you need to associate the host with a data space group. All data space groups must be associated with the same number of data instances. If you follow the recommendation of one data instance per host, all data space groups must be associated with the same number of hosts.

As mentioned in "Adding the standby management instance", the ttGridAdmin hostCreate command creates a host in the grid. You can associate the host with a data space group or physical group at host creation or later.

Note:

If you do not initially associate a host with a data space group, you have the option to let TimesTen Scaleout analyze the model after you have created all the hosts and suggest which hosts to associate with each data space group, based on the physical groups associated with each host, with the use of the ttGridAdmin dataSpaceGroupSuggest command. See "Describe your physical topology with physical groups" and "Assigning hosts to data space groups" for more information.

Create a host for a data instance and associate it with data space group 1. Ensure that you identify the fully qualified domain name or IP address of the host.

% ttGridAdmin hostCreate -internalAddress int-host3 -externalAddress
 ext-host3.example.com -dataSpaceGroup 1
Host host3 created in Model

Note:

If you do not specify a name for the host, TimesTen Scaleout sets the OS hostname of the remote system as the name of the new host.

Figure 4-5 shows a graphical representation of the model of the grid1 grid after creating the host3 host.

Figure 4-5 The model after adding a host for a data instance

Description of Figure 4-5 follows
Description of ''Figure 4-5 The model after adding a host for a data instance''

For more information on the ttGridAdmin hostCreate command, see "Create a host (hostCreate)" in the Oracle TimesTen In-Memory Database Reference.

Create the installation for the data instance

Every host must have an installation associated with it. A host can either have its own copy of the installation files or share an installation with one or more hosts through network-attached storage. For shared installations, an installation model object with the location of the shared installation files must be associated with the host.

For more information on how to share a TimesTen installation, see "Copying an installation on Linux/UNIX" in the Oracle TimesTen In-Memory Database Installation, Migration, and Upgrade Guide.

The ttGridAdmin installationCreate command creates an installation in the grid and associates it with a host.

Create an installation in a directory of your choice in the host3 host, for example, the /grid directory.

% ttGridAdmin installationCreate host3 -location /grid
Installation installation1 on Host host3 created in Model

Note:

  • If you do not specify a name for the installation, TimesTen Scaleout sets installation1 as the name of the installation. Any subsequent installation associated with the same host requires that you provide a name for it.

  • If the management instance running the command has only one installation associated with it and the source for the installation files is not specified in the -source option of the ttGridAdmin installationCreate command, TimesTen Scaleout copies the installation files from the installation associated with the management instance running the command.

Figure 4-6 shows a graphical representation of the model of the grid1 grid after creating the installation1 installation in the host3 host.

Figure 4-6 The model after adding an installation for the data instance

Description of Figure 4-6 follows
Description of ''Figure 4-6 The model after adding an installation for the data instance''

For more information on the ttGridAdmin installationCreate command, see "Create an installation (installationCreate)" in the Oracle TimesTen In-Memory Database Reference.

Create the data instance

The ttGridAdmin instanceCreate command creates an instance in the grid and associates it with a host and installation.

Create a data instance in the location of your choice in the host3 host, for example, the /grid directory.

% ttGridAdmin instanceCreate host3 -location /grid
Instance instance1 on Host host3 created in Model

Note:

  • If you do not specify a name for the instance, TimesTen Scaleout sets instance1 as the name of the instance. Any subsequent instances associated with the same host requires that you provide a name for it.

  • Because the host3 host only has the installation1 installation associated with it, the installation1 installation is associated with the instance1 data instance by default and there is no need to specify the -installation option.

  • TimesTen Scaleout defines an instance as a data instance by default. Use the -type management option of the ttGridAdmin instanceCreate command to create a management instance.

  • TimesTen Scaleout creates a subdirectory with the instance name in the specified location. TimesTen Scaleout allocates all instance files in this subdirectory. For example, the instance files of the instance1 data instance are allocated in the /grid/instance1 directory of the host3 host.

  • TimesTen Scaleout sets the default values for the TCP/IP port numbers of the instance daemon and server (6624 and 6625, respectively) if you do not specify a value for the port numbers. Use the -daemonPort or -csPort options of the ttGridAdmin instanceCreate utility to set different values for the port numbers.

Figure 4-7 shows a graphical representation of the model of the grid1 grid after creating a data instance.

Figure 4-7 The model after adding a data instance

Description of Figure 4-7 follows
Description of ''Figure 4-7 The model after adding a data instance''

For more information on the ttGridAdmin instanceCreate command, see "Create an instance (instanceCreate)" in the Oracle TimesTen In-Memory Database Reference.

Create data instances by duplicating the configuration of an existing host

As mentioned in "Adding the standby management instance", you can create an instance (management or data) by duplicating the configuration of an existing host, including its associated installations and instances with the -like and -cascade options of the ttGridAdmin hostCreate command.

  • The -like option identifies the host to be duplicated and associates the new host with the same physical groups and data space group. You can override the physical groups and data space group associated with the new host by providing different parameters in the -physicalGroup and -dataSpaceGroup options, respectively.

  • The -cascade option duplicates the configuration of the installations and data instances associated with the specified host.

Create five data instances based on the same attributes defined for the host3 host and its associated installation and data instance. Also, associate three hosts with data space group 2, instead of data space group 1. Ensure that you identify the fully qualified domain name of the new hosts.

% ttGridAdmin hostCreate -internalAddress int-host4 -externalAddress
 ext-host4.example.com -like host3 -cascade -dataSpaceGroup 2
Host host4 created in Model
Installation installation1 created in Model
Instance instance1 created in Model

% ttGridAdmin hostCreate -internalAddress int-host5 -externalAddress
 ext-host5.example.com -like host3 -cascade
Host host5 created in Model
Installation installation1 created in Model
Instance instance1 created in Model

% ttGridAdmin hostCreate -internalAddress int-host6 -externalAddress
 ext-host6.example.com -like host4 -cascade 
Host host6 created in Model
Installation installation1 created in Model
Instance instance1 created in Model

% ttGridAdmin hostCreate -internalAddress int-host7 -externalAddress
 ext-host7.example.com -like host3 -cascade
Host host7 created in Model
Installation installation1 created in Model
Instance instance1 created in Model

% ttGridAdmin hostCreate -internalAddress int-host8 -externalAddress
 ext-host8.example.com -like host4 -cascade
Host host8 created in Model
Installation installation1 created in Model
Instance instance1 created in Model

Note:

  • If you do not specify a name for the host, TimesTen Scaleout sets the OS hostname of the remote system as the name of the new host.

  • Any additional option you define in the ttGridAdmin hostCreate command will overwrite the attributes inherited from the existing host in the new host, as shown with the addition of the -dataSpaceGroup 2 parameter in the command that creates the host6 host.

  • Notice that the ttGridAdmin hostCreate commands that create the host6 and host8 hosts use the host4 host as reference in the -like option.

Figure 4-8 shows a graphical representation of the model of the grid1 grid after duplicating the host3 host as the host4, host5, host6, host7, and host8 hosts and their associated installations and instances.

Figure 4-8 The model after duplicating an existing host

Description of Figure 4-8 follows
Description of ''Figure 4-8 The model after duplicating an existing host''

For more information on the ttGridAdmin hostCreate command, see "Create a host (hostCreate)" in the Oracle TimesTen In-Memory Database Reference.

Applying the changes made to the model

The latest version of the model describes the desired structure of a grid, not its current structure. Any changes made to the latest version of the model are not immediately reflected in the operational configuration of a grid. Changes made to the latest version of the model need to be explicitly applied to the grid.

Model versioning

Management instances store multiple versions of the model. Only one version of the model can be active in the grid at any given time.

TimesTen Scaleout classifies model versions as follows:

  • Current version: The current version of the model describes the operational configuration of the grid. This version, and all previous versions, is read-only.

  • Latest version: The latest version of the model can be modified and has yet to be applied to the grid. This version is read/write.

When you create a grid, the version 1 model is initially populated with the configuration of the first host, installation, and management instance, and the version 1 model is recognized as the latest version of the model. Any subsequent changes that you make to the model are added to the latest version of the model (version 1). When you implement these changes with the ttGridAmin modelApply command, a new latest version of the model (version 2) is created for future changes and the previous latest version of the model (version 1) becomes the current version of the model.

Every time you run the ttGridAdmin modelApply command, TimesTen Scaleout:

  1. Makes the latest version of the model (version n) read-only.

  2. Creates a writable copy (version n+1) of the latest version of the model.

  3. Attempts to apply the changes previously made to the version n model to the operational grid.

  4. Identifies the version n model as the current version of the model.

  5. Identifies the version n+1 model as the latest version of the model.

The ttGridAdmin utility enables the user to perform several operations regarding the model, like:

  • Applying the changes made to the latest version of the model

  • Comparing two versions of the model

  • Exporting a version of the model into a flat file in JSON format

  • Importing a flat file in JSON format as the latest version of the model

  • Listing all the available versions of the model

Previous versions of the model are automatically stored. With the ttGridAdmin gridModify command, you can specify the retention period for old versions of the model either in terms of days, in terms of the number of stored versions, or both. TimesTen Scaleout by default retains the last 10 versions for a period of 30 days.

For more information on model operations or the ttGridAdmin gridModify utility, see "Model operations" or "Modify grid settings (gridModify)", respectively, in the Oracle TimesTen In-Memory Database Reference.

Apply the latest version of the model

The ttGridAdmin modelApply command attempts to apply the changes made to the latest version of the model into the operational grid. If, for example, you add a new data instance to the latest version of the model, running this command performs all of the necessary operations to create and initialize the instance in the specified host. Some of the operations that the ttGridAdmin modelApply command performs include these:

  • Identify and delete any object removed from the latest version of the model.

  • Create new installations.

  • Create new instances, data and management.

  • Overwrite the configuration files of all instances. The new versions of these files include any new entries found in the latest version of the model.

  • Verify the SSH connectivity between hosts.

    Note:

    If you recently added new instances to the model or have yet to set the required passwordless SSH access to the hosts managing instances, either manually set the required passwordless SSH access for the instance administrator or use the ttGridAdmin gridSshConfig command before applying the latest version of the model. See "Setting passwordless SSH" for more information.
  • Start new instances.

Apply all the changes made to the latest version of the model of the grid1 grid.

% ttGridAdmin modelApply
Creating new model version............................................OK
Exporting current model (version 1)...................................OK
Identifying any changed management instances..........................OK
Identifying any deleted objects.......................................OK
Verifying installations...............................................OK
Verifying instances...................................................OK
Creating new instances................................................OK
Updating grid state...................................................OK
Configuring instance authentication...................................OK
Pushing new configuration files to each instance......................OK
Making model version 1 current, version 2 writable....................OK
Checking ssh connectivity of new instances............................OK
Starting new management instance......................................OK
Configuring standby management instance...............................OK
Starting new data instances...........................................OK
ttGridAdmin modelApply complete

Given all the tasks you performed in the previous sections, the ttGridAdmin modelApply command performs the following operations:

  1. Creates a copy of the installation files on every configured host:

  2. Creates the instance home directory and files for the standby management instance and data instances on their associated hosts:

  3. Makes the latest version of the model read-only and a creates a new writable model.

  4. Verifies SSH connectivity to every configured host.

  5. Starts the daemons of the standby management instance and data instances.

  6. Configures the active and standby management instances.

For more information on the ttGridAdmin modelApply command, see "Apply the latest version of the model (modelApply)" in the Oracle TimesTen In-Memory Database Reference.

Setting instances to automatically start at system startup

Optionally, you can configure data instances to automatically start or shut down every time their systems boot or shut down, respectively. Each instance needs to be configured independently and only after its creation has been applied to the current version of the model.

To accomplish this, the root user must run the setuproot script with the -install option. You can find this script in the timesten_home/bin directory of every instance of the grid. For example:

From the host of the host3.instance1 data instance:

% /grid/instance1/bin/setuproot -install
Would you like to install the TimesTen daemon startup scripts into /etc/init.d?
 [ yes ]

For more information on how to use the setuproot script, see "Start an instance automatically at system startup" in the Oracle TimesTen In-Memory Database Installation, Migration, and Upgrade Guide.

Description of the physical topography of the grid

As described in "Assigning hosts to data space groups", TimesTen Scaleout requires you to assign the hosts that will contain a data instance into a data space group.

Note:

The hosts in one data space group should not share physical resources with the hosts in another data space group to protect each full copy of the database from hardware failures.

The example provided in "Adding data instances", demonstrates how to assign hosts to a data space group during host creation with the ttGridAdmin hostCreate -dataSpaceGroup option:

% ttGridAdmin hostCreate -internalAddress int-host3 -externalAddress
 ext-host3.example.com -dataSpaceGroup 1
Host host3 created in Model

However, if your physical topography is complicated so that it is difficult to decide on the best assignments of hosts to data space groups, you can ask TimesTen Scaleout to recommend assignments of hosts to data space groups using the ttGridAdmin dataSpaceGroupSuggest command. In order to accomplish this, you need to inform TimesTen Scaleout of the physical topology of where your hosts are co-located or hosts that use the same resources. You can inform TimesTen Scaleout of all physical hardware for each host with the physical group.

Important:

If you are going to use physical groups to have TimesTen Scaleout recommend assignments, you should not assign hosts to data space groups upon host creation. Hosts can only be assigned to data space groups once.

Your physical topology may include buildings, transformers, air conditioners, racks, fans, top of rack switches, hypervisors, storage filers and power sources. For example, the following production environment includes:

  • 2 buildings, each with their own transformer.

  • 5 air conditioners in rooms within the 2 buildings.

  • 9 racks of servers in these rooms, each with several hosts.

  • 18 power surge protectors in which the hosts are plugged.

Description of physgroup.png follows
Description of the illustration ''physgroup.png''

Instead of determining what would be the best data space group assignment for the hosts that share these resources, you can assign hosts to physical groups to identify where the hosts are located and what physical resources they share. This informs TimesTen Scaleout of similar potential points of failure. Thus, when TimesTen Scaleout recommends assignment of hosts to data space groups using the ttGridAdmin dataSpaceGroupSuggest command, it physically separates the hosts that contain the separate copies of the data.

Note:

See "Assigning hosts to data space groups" for more details on data space groups. See "Get recommendations for data space group assignments (dataSpaceGroupSuggest)" in the Oracle TimesTen In-Memory Database Reference for more details on the ttGridAdmin dataSpaceGroupSuggest command.

Create and manage the physical topography of your grid by:

Assigning hosts to physical groups

Assigning hosts to physical groups to denote the physical topology is optional and only necessary when you want TimesTen Scaleout to assign hosts to data space groups.

The example in this section assigns the hosts to physical groups using the same host configuration of the example used in this chapter. In production, you would not use physical groups for this small example as you can easily identify which hosts should be in which data space group.

Figure 4-9 Hosts attached to physical resources

Description of Figure 4-9 follows
Description of ''Figure 4-9 Hosts attached to physical resources''

Let us assume that:

  • Hosts 3, 4, and 5 share an air conditioner.

  • Hosts 3, 4 and 5 are on the same rack.

  • Hosts 3 and 4 share a power source.

  • Host 5 has its on power source.

  • Hosts 6, 7 and 8 share an air conditioner.

  • Hosts 6, 7 and 8 are on the same rack.

  • Hosts 6 and 7 share a power source.

  • Host 8 has its own power source.

To identify which hosts are connected to which physical resources:

  1. Define a physical group, which is simply a container object that groups hosts together to show that there is a shared physical resource. This enables TimesTen Scaleout to separate different copies of the data to hosts that do not share physical aspects.

  2. Assign hosts to physical groups either when creating the host with the ttGridAdmin hostCreate command or when modifying the host with the ttGridAdmin hostModify command.

    You can assign a host to one or more physical groups, as appropriate. If they are assigned to a similar rack and share a power source, you can assign the host to both of these physical groups.

See "Create a host (hostCreate)" and "Modify a host (hostModify)" in the Oracle TimesTen In-Memory Database Reference for more details on these ttGridAdmin commands.

Example 4-1 Example of hosts that are physically located on two separate racks

This example shows how you would create physical groups to model the shared resources and assign hosts to these physical groups.

  1. Define the physical group: Use the ttGridAdmin physicalCreate command to create:

    • rack1 and rack2 to model the physical racks in the lab.

    • aircond1 and aircond2 to model the air conditioners.

    • power1, power2, power3 and power4 to model the power surge protectors.

  2. Assign hosts to the physical group: Use the ttGridAdmin hostModify to assign the hosts to each shared physical resource using the -physicalgroup option:

    • host3, host4 and host5 are on the rack1.

    • host3, host4 and host5 share aircond1.

    • host3 and host4 share power1.

    • host5 has its own power2.

    • host6, host7 and host8 are on rack2.

    • host6, host7 and host8 share an aircond2.

    • host6 and host7 share power3.

    • host8 has its own power4.

      Note:

      If a host was already associated with a physical group before you execute the -physicalgroup option of the ttGridAdmin hostModify command, these physical groups are removed and the only physical groups associated with the host are those that are specified on the current command line.

      However, you can add physical groups in addition to those physical groups that are already assigned to the existing host by using the -addPhysicalGroup option on the ttGridAdmin hostModify command.

% ttGridAdmin physicalCreate rack1
 PhysicalGroup rack1 created.
% ttGridAdmin physicalCreate rack2
 PhysicalGroup rack2 created.
% ttGridAdmin physicalCreate aircond1
 PhysicalGroup aircond1 created.
% ttGridAdmin physicalCreate aircond2
 PhysicalGroup aircond2 created.
% ttGridAdmin physicalCreate power1
 PhysicalGroup power1 created.
% ttGridAdmin physicalCreate power2
 PhysicalGroup power2 created.
% ttGridAdmin physicalCreate power3
 PhysicalGroup power3 created.
% ttGridAdmin physicalCreate power4
 PhysicalGroup power4 created.
% ttGridAdmin hostModify host3 -physicalgroup rack1 aircond1 power1
 Host host3 modified in Model
% ttGridAdmin hostModify host4 -physicalgroup rack1 aircond1 power1 
 Host host4 modified in Model
% ttGridAdmin hostModify host5 -physicalgroup rack1 aircond1 power2 
 Host host5 modified in Model
% ttGridAdmin hostModify host6 -physicalgroup rack2 aircond2 power3
 Host host6 modified in Model
% ttGridAdmin hostModify host7 -physicalgroup rack2 aircond2 power3
 Host host7 modified in Model
% ttGridAdmin hostModify host8 -physicalgroup rack2 aircond2 power4 
 Host host8 modified in Model

Note:

You can use the ttGridAdmin modelExport command to see which hosts are assigned to a physical group.

Removing the physical layout of the hosts

If you want to remove a physical description of how the hosts are organized physically, use the -nophysicalgroup option with the ttGridAdmin hostModify command. The host3 host was originally associated with the physical groups rack1, aircond1 and power1. By executing the following command, host3 is not assigned to any physical groups.

% ttGridAdmin hostModify host3 -nophysicalgroup
 Host host3 modified in Model

Deleting physical groups

You can delete a physical group only if no hosts are assigned to it. The following deletes the rack1 physical group after removing the host1 and host2 hosts assigned to it:

% ttGridAdmin hostModify host3 -nophysicalgroup
 Host host3 modified in Model
% ttGridAdmin hostModify host4 -nophysicalgroup
 Host host4 modified in Model
% ttGridAdmin hostModify host5 -nophysicalgroup
 Host host5 modified in Model
% ttGridAdmin physicalDelete rack1
 PhysicalGroup rack1 deleted.

Propose data space group assignments

You can use the ttGridAdmin dataSpaceGroupSuggest command to propose an assignment of hosts to different data space groups based on how the hosts are currently assigned to physical groups. You can either accept the proposed assignments or specify your own assignments.

The ttGridAdmin dataSpaceGroupSuggest command makes recommendations on how to assign hosts so that the hosts that contain one copy of the data do not share resources with the hosts that contain the copy of the data. Once the ttGridAdmin command assigns the hosts to their data space groups, the assignment cannot be changed.

See "Get recommendations for data space group assignments (dataSpaceGroupSuggest)" in the Oracle TimesTen In-Memory Database Reference for more details on the ttGridAdmin dataSpaceGroupSuggest command.

Example 4-2 Requesting TimesTen Scaleout assign hosts to data spaces

This example shows ttGridAdmin dataSpaceGroupSuggest command, which writes its recommendations into the recommendations.sh file. You can execute this file as it contains the ttGridAdmin hostModify commands necessary for assigning hosts to the recommended data space groups.

% ttGridAdmin dataSpaceGroupSuggest /tmp/recommendations.sh
% more /tmp/recommendations.sh
#!/bin/sh
# Recommendations generated by ttGridAdmin -dataSpaceGroupSuggest
 
TIMESTEN_HOME=/grid/instance1
export TIMESTEN_HOME
. $TIMESTEN_HOME/bin/ttenv.sh > /dev/null 2>/dev/null
 
# Number of possibilities evaluated:     126
#
# Number of usable possibilities found:  10
# (A 'usable' possibility is one that is compatible with pre-existing
# assignments of Hosts to DataSpaceGroups)
#
# Number of 'ideal' possibilities found: 1
# (An 'ideal' possibility is one where no PhysicalGroups span multiple
# DataSpaceGroups)
#
# Possibilities evaluated (best 10 displayed):
# ...
#
# This script, if executed, would implement the only 'ideal' configuration found.
# Even though this recommendation was 'ideal', you should carefully evaluate it
# prior to running this script.
# Host host1 is already in DataSpaceGroup 1
ttGridAdmin hostModify host3 -dataSpaceGroup 2
ttGridAdmin hostModify host4 -dataSpaceGroup 2
ttGridAdmin hostModify host5 -dataSpaceGroup 2
ttGridAdmin hostModify host6 -dataSpaceGroup 1
ttGridAdmin hostModify host7 -dataSpaceGroup 1
ttGridAdmin hostModify host8 -dataSpaceGroup 1

If you decide to accept these recommendations, execute the provided shell script, which in our example is recommendations.sh. Once executed, all hosts are assigned to the designated data space groups.

% sh /tmp/recommendations.sh
Host host3 modified in Model
Host host4 modified in Model
Host host5 modified in Model
Host host6 modified in Model
Host host7 modified in Model
Host host8 modified in Model