Prerequisites and Installation of TimesTen Scaleout

General prerequisites

TimesTen Scaleout is only supported on the Linux platform. For the supported Linux platform versions, see the "Platforms and configurations" section in the documentation library. For the most recent information about your particular TimesTen release, see the Oracle TimesTen In-Memory Database Release Notes (README.html) in your installation directory.

Perform these steps on all hosts that will run the management and data instances and membership servers for the grid:

Install the same operating system version and release on each host.
Configure all hosts in the same internal network.

When you set up your network, you must create a single internal network for all the grid components to communicate with each other. While clients may use the same internal network to connect to instances, you may wish to create an external network for client connections.
Install and configure NTP (Network Time Protocol). Clocks must be synced.
Ensure all instances in the grid can communicate with all other instances in the grid over the internal network on any port.
To avoid problems before and after installation, confirm your file system has sufficient space. (See "Storage provisioning for TimesTen" in Oracle TimesTen In-Memory Database Operations Guide for more information.)

Operating system prerequisites

These sections discuss the operating system prerequisites:

General Linux prerequisites

On some Oracle Linux 8.x and RedHat 8.x systems, you must install the ncurses-compat-libs package (sudo yum install ncurses-compat-libs). Otherwise, cursor-based command recall and editing does not work in ttIsql.

On SUSE Linux Enterprise Server, you need to install libncurses5. To do this, run:

zypper -n install libncurses

It is recommended that you enable stack traces for TimesTen. On Linux systems, use pstack or gdb to get a stack trace.

Understanding the TimesTen users group and the operating system user

These sections describe and show how to create both the TimesTen users group and the operating system user (which will serve as the instance administrator):

The TimesTen users group

TimesTen restricts access to the installation and the instances created from that installation to members of a single operating system group. This group, called the TimesTen users group, owns the installation and the instances created from the installation. Create this group (for example, timesten) and add the desired operating system users prior to installation. Once you create the TimesTen users group, you cannot change the name of the group or the group ID. See Create the TimesTen users group and the OS user for more information.

Note that:

The instance administrator's primary group must be the TimesTen users group.
Users who wish to access databases through TimesTen utilities or direct mode applications must be members of the TimesTen users group. This group can be the user's primary or secondary group.
Users connecting to a database through a client connection do not have to be members of the TimesTen users group.

The operating system user

The instance administrator for all instances in your grid is the operating system user who creates the active management instance. This user then becomes the instance administrator of all other instances in TimesTen Scaleout, including the second management instance and all data instances.

Note that:

The instance administrator cannot be the root user.
The instance administrator configures the grid, creates and manages the databases in the grid, starts and stops the databases in the grid, performs all management activities, and performs backup and restore operations.
You cannot change the instance administrator after that administrator creates the active management instance.
The instance administrator is a member of the TimesTen users group. See The TimesTen users group for more information.
The instance administrator's user name and UID, and the group name and the group id (GID) of the TimesTen users group must be the same on all hosts in the grid, including the hosts on which the management and data instances exist, as well as any of the SCP repository hosts.
The installation and the instances must have the same owner (the instance administrator).

Create the TimesTen users group and the OS user

In this example, instanceadmin is the name of the operating system user and timesten is the name of the TimesTen users group.

Create the TimesTen users group. Name the group timesten with group ID 10000.
```
% sudo groupadd -g 10000 timesten
```
Create the instanceadmin user with UID 55000 and assign this user to the timesten primary group. Then, create a password for the instanceadmin user.
```
% sudo useradd -u 55000 -g timesten instanceadmin
% sudo passwd instanceadmin
```

Network requirements

For most production environments, TimesTen Scaleout requires a single private internal network and at least one external network. This section describes the requirements for those networks.

Internal network

Instances in a grid communicate with each other over a single internal network using the TCP protocol. TimesTen Scaleout uses this network to perform all SQL, backup, and management operations required by the grid and its databases. In addition, instances communicate with membership servers through this network. Membership servers use this network to communicate among themselves.

Ensure that your internal network has these characteristics:

High bandwidth. The faster the network the better, in terms of throughput (gigabits per second). For production environments, ensure at minimum a 10 Gigabit Ethernet network or equivalent.
Low latency. To reduce network latency (time to transmit a message from one host to another) to a minimum, the hosts and membership servers attached to your internal network should either:
- Span a single data center within a small number of racks.
- Span multiple data centers within a small geographic region (city or suburb) connected by a metropolitan area network (MAN). Only recommended with a 10 GbE network or better.
- Not span multiple data regions (states or provinces) connected by a wide area network (WAN).
IPv4 or IPv6 addresses.
No network address translation (NAT).
No TCP packet filtering.

For an on-premises environment, ensure your internal network meets these requirements:

If your internal network consists of a single network segment, all hosts are connected to a single Ethernet switch or equivalent.
If your internal network consists of multiple network segments, those segments are connected through bridges instead of IP routers.
If your internal network uses a MAN, ensure that the MAN can provide the required bandwidth and latency for your workload.

Syntax for internal addresses

When you define a host for your grid, you must specify a single value for the internal address of that host. Optionally, you specify a value for that the external address of that host. The value you specify for the internal address of a host can be either an IPv4 address, an IPv6 address or a name that resolves into one or more IPv4 or IPv6 addresses. For example:

A dot-decimal IPv4 address such as 192.168.1.1
A colon-hexadecimal IPv6 address such as 2606:fe80::f816:3eff:fe15:44b3
A name specified in the /etc/hosts file such as host1
A name defined in a private Domain Name Server (DNS) such as int-host1.example.com

If you use a name to define the internal address of a host:

If the name resolves to multiple IP addresses, those addresses must be on the same network segment.
Every host in the grid must be able to resolve a name to the same addresses. For example, if you use the hosts file to define a name, then the hosts file on each host in the grid must contain identical entries for that name.

External network

A grid may optionally use one or more public external networks. These networks enable applications running on machines that are not part of the grid to create client/server connections to databases in the grid. You cannot perform any grid or database management operations through an external network.

While the performance of an external network is important, it is less important than the performance of the internal network. If the internal network performs poorly or unreliably, the grid and its databases may perform poorly or unreliably for all users. Conversely, if an external network performs poorly or unreliably, it may only affect the applications connected to the databases in the grid through that network. As a result, there are fewer requirements for an external network than for the internal network.

Your external networks should have these characteristics:

Bandwidth based on the requirements of your client/server applications.
Latency based on the requirements of your client/server applications.
IPv4 or IPv6 addresses.
TCP connectivity to the server port of each data instance.
Any combination of network technologies (VPN, routers, LAN, WAN, etcetera).

If your grid uses a single external network, then the value you specify for the external address of a host can be in any of the forms described in the Syntax for internal addresses section. If your grid uses multiple external networks, then you must use a name to define the external address of a host. The name must resolve to at least one IP address for each external network you use.

Installing TimesTen Scaleout

When you unpack the TimesTen distribution on a host, you create an installation (that is read only). Do not add, alter, or remove files or directories within the installation, unless you are deleting the installation.

The installation may be a full installation or a client-only installation. A client-only installation supports the client use of TimesTen:

Type	Description
Full installation	Use the TimesTen full distribution for this type of installation (for example, `timesten221110.server.linux8664.zip`).
Client-only installation	You can connect and access databases in TimesTen Classic through a client. Use the full installation (for example, `timesten221110.server.linux8664.zip`) to unpack the distribution and then specify `ttInstanceCreate -clientonly`. See Database connections for details on the types of clients and how they connect to a database.

The operating system user that you designated as the instance administrator creates the installation by:

Downloading the TimesTen distribution on the host that will contain the active management instance. The distribution is a ZIP file where the ZIP file name indicates the platform, release number, and the type of distribution. For example, timesten221110.server.linux8664.zip.
Unpacking the ZIP file to create a TimesTen installation. The installation includes the binaries and the support files from which you can create a grid (and all of its components), membership servers, and clients

Only the first installation is created manually by the instance administrator on the host containing the active management instance. Additional installations used by additional instances are created by TimesTen Scaleout utilities. See Setting Up a Grid for information on when to create additional installations for additional instances.

After you download the distribution, follow these steps:

Log in as the instance administrator to the host that will contain the initial management instance. In this example, instanceadmin is the name of the instance administrator. You can verify the instance administrator with the Linux id command.
Create the desired directory for the installation such as /grid/installation1.
```
% mkdir -p /grid/installation1
```
Extract and unpack the distribution file into the directory. This example unpacks the installation using the unzip command:
```
% unzip /timesten221110.server.linux8664.zip -d /grid/installation1
[...UNZIP OUTPUT...]
```

The top level directory of the installation is the TimesTen release. For example, the directory created under /grid/installation1 is:

dr-xr-x--- 19 instanceadmin timesten 4096 Mar  2 22:07 tt22.1.1.1.0

Verifying the installation

These sections provide details on how to verify your installation:

Run the ttInstallationCheck utility

The ttInstallationCheck utility, located in the installation_dir/tt22.1.1.1.0/bin directory, verifies the success or failure of the installation. This utility generates an error if the checksum value for the installation differs from the original checksum value. Checksum values are different if there are any of these changes to the installation directory or files:

Contents of a file
Name of a file
Addition of a file to a directory
Removal of file from a directory
Changes to the permissions of a file or directory

In this example, the installation is verified:

% ttInstallationCheck
This installation has been verified.

In this example, permissions on a file were changed, and ttInstallationCheck generates an error:

% ttInstallationCheck 
Cannot validate the installation in /grid/installation1/tt22.1.1.1.0.

See "ttInstallationCheck" in the Oracle TimesTen In-Memory Database Reference for detailed information on the ttInstallationCheck utility.

Review the installation directory and subdirectories

A TimesTen full installation includes these subdirectories located under the top-level installation_dir/tt22.1.1.1.0 directory.

3rdparty: Includes resources for:
- Apache ZooKeeper
- Java Message Service (JMS)
bin: TimesTen utilities and executables
grid: Files and resources for TimesTen Scaleout
include: TimesTen include files, among them timesten.h (for TimesTen ODBC features) and tt_errCode.h (for information about TimesTen error codes)
lib: TimesTen libraries
plsql: Files and resources for TimesTen PL/SQL
ttoracle_home: Oracle Database Instant Client files and resources, for OCI, Pro*C/C++, and ODP.NET

Note:

A client-only installation does not include the 3rdparty or the grid directories.

Setting passwordless SSH

The instance administrator must be able to use SSH to log without a password to all hosts within a grid for the management instances and ttGridAdmin utility to be able set up and manage the grid and all its members.

Specifically, all hosts with management instances need passwordless SSH access for the instance administrator to all hosts with instances and repositories. Also, hosts with data instances need passwordless SSH access for the instance administrator to all hosts with repositories.

The ttGridAdmin gridSshConfig command is able to set for the current user the required passwordless SSH access. Ensure that you run the command with the user you intend for instance administrator.

Before setting up a grid, you can run the ttGridAdmin gridSshConfig command while providing the addresses or DNS names that you will later use to host management instances, data instance, and repositories. When prompted, enter the OS password of the user executing the command. The user and password must already be set on all systems and be identical. See Understanding the TimesTen users group and the operating system user for more information on the instance administrator.

% grid/installation1/tt22.1.1.1.0/bin/ttGridAdmin gridSshConfig
 -mgmtAddress int-host1 int-host2
 -dataAddress int-host3 int-host4 int-host5 int-host6 int-host7 int-host8
Enter password:
Setup ssh configuration on local system...............................OK
Setup ssh configuration on int-host1..................................OK
Setup ssh configuration on int-host2..................................OK
Setup ssh configuration on int-host3..................................OK
Setup ssh configuration on int-host4..................................OK
Setup ssh configuration on int-host5..................................OK
Setup ssh configuration on int-host6..................................OK
Setup ssh configuration on int-host7..................................OK
Setup ssh configuration on int-host8..................................OK
Setup passwordless ssh from local system to int-host1.................OK
Setup passwordless ssh from local system to int-host2.................OK
Setup passwordless ssh from local system to int-host3.................OK
Setup passwordless ssh from local system to int-host4.................OK
Setup passwordless ssh from local system to int-host5.................OK
Setup passwordless ssh from local system to int-host6.................OK
Setup passwordless ssh from local system to int-host7.................OK
Setup passwordless ssh from local system to int-host8.................OK
Setup passwordless ssh from int-host1 to int-host1....................OK
Setup passwordless ssh from int-host1 to int-host2....................OK
Setup passwordless ssh from int-host1 to int-host3....................OK
Setup passwordless ssh from int-host1 to int-host4....................OK
Setup passwordless ssh from int-host1 to int-host5....................OK
Setup passwordless ssh from int-host1 to int-host6....................OK
Setup passwordless ssh from int-host1 to int-host7....................OK
Setup passwordless ssh from int-host1 to int-host8....................OK
Setup passwordless ssh from int-host2 to int-host1....................OK
Setup passwordless ssh from int-host2 to int-host2....................OK
Setup passwordless ssh from int-host2 to int-host3....................OK
Setup passwordless ssh from int-host2 to int-host4....................OK
Setup passwordless ssh from int-host2 to int-host5....................OK
Setup passwordless ssh from int-host2 to int-host6....................OK
Setup passwordless ssh from int-host2 to int-host7....................OK
Setup passwordless ssh from int-host2 to int-host8....................OK
 
Passwordless ssh working between hosts:
 
From\To   int-host1 int-host2 int-host3 int-host4 int-host5 ... int-host8
--------- --------- --------- --------- --------- --------- ... ---------
*us*      Yes       Yes       Yes       Yes       Yes       ... Yes
int-host1 Yes       Yes       Yes       Yes       Yes       ... Yes
int-host2 Yes       Yes       Yes       Yes       Yes       ... Yes
int-host3 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host4 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host5 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host6 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host7 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host8 N/A       N/A       N/A       N/A       N/A       ... N/A

For a grid where the latest version of the model has yet to be applied and new hosts and instances were added to the model, run the ttGridAdmin gridSshConfig command on the active management instance. The ttGridAdmin utility then will query the latest version of the model and set up the appropriate SSH connectivity amongst the hosts described in the model.

% ttGridAdmin gridSshConfig
Enter password:
Setup ssh configuration on local system...............................OK
Setup ssh configuration on int-host1..................................OK
Setup ssh configuration on int-host2..................................OK
Setup ssh configuration on int-host3..................................OK
Setup ssh configuration on int-host4..................................OK
Setup ssh configuration on int-host5..................................OK
Setup ssh configuration on int-host6..................................OK
Setup ssh configuration on int-host7..................................OK
Setup ssh configuration on int-host8..................................OK
Setup passwordless ssh from local system to int-host1.................OK
Setup passwordless ssh from local system to int-host2.................OK
Setup passwordless ssh from local system to int-host3.................OK
Setup passwordless ssh from local system to int-host4.................OK
Setup passwordless ssh from local system to int-host5.................OK
Setup passwordless ssh from local system to int-host6.................OK
Setup passwordless ssh from local system to int-host7.................OK
Setup passwordless ssh from local system to int-host8.................OK
Setup passwordless ssh from int-host1 to int-host1....................OK
Setup passwordless ssh from int-host1 to int-host2....................OK
Setup passwordless ssh from int-host1 to int-host3....................OK
Setup passwordless ssh from int-host1 to int-host4....................OK
Setup passwordless ssh from int-host1 to int-host5....................OK
Setup passwordless ssh from int-host1 to int-host6....................OK
Setup passwordless ssh from int-host1 to int-host7....................OK
Setup passwordless ssh from int-host1 to int-host8....................OK
Setup passwordless ssh from int-host2 to int-host1....................OK
Setup passwordless ssh from int-host2 to int-host2....................OK
Setup passwordless ssh from int-host2 to int-host3....................OK
Setup passwordless ssh from int-host2 to int-host4....................OK
Setup passwordless ssh from int-host2 to int-host5....................OK
Setup passwordless ssh from int-host2 to int-host6....................OK
Setup passwordless ssh from int-host2 to int-host7....................OK
Setup passwordless ssh from int-host2 to int-host8....................OK
 
Passwordless ssh working between hosts:
 
From\To   int-host1 int-host2 int-host3 int-host4 int-host5 ... int-host8
--------- --------- --------- --------- --------- --------- ... ---------
*us*      Yes       Yes       Yes       Yes       Yes       ... Yes
int-host1 Yes       Yes       Yes       Yes       Yes       ... Yes
int-host2 Yes       Yes       Yes       Yes       Yes       ... Yes
int-host3 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host4 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host5 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host6 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host7 N/A       N/A       N/A       N/A       N/A       ... N/A
int-host8 N/A       N/A       N/A       N/A       N/A       ... N/A

For more information on the ttGridAdmin gridSshConfig command, see "Configure SSH (gridSshConfig)" in the Oracle TimesTen In-Memory Database Reference.