Configuring the Network

Network Bandwidth Requirements

The network bandwidth required for TimesTen Classic replication depends on the bulk and frequency of the data being replicated.

This discussion explores the types of transactions that characterize the high and low ends of the data range and the network bandwidth required to replicate the data between TimesTen databases.

Table 4-1 provides guidelines for calculating the size of replicated records.

Table 4-1 Replicated Record Sizes

Record Type	Size
Begin transaction	48 bytes
Update	116 bytes + 18 bytes per column updated + size of old column values + size of new column values + size of the primary key or unique key
Delete	104 bytes + size of the primary key or unique key
Insert	104 bytes + size of the primary key or unique key + size of inserted row

Transactions are sent between replicated databases in batches. A batch is created whenever there is no more data in the transaction log buffer in the master database, or when the current batch is roughly 256 KB. See Copying Updates Between Databases.

Replication in a WAN Environment

TimesTen Classic replication uses the TCP/IP protocol, which is not optimized for a WAN environment. You can improve replication performance over a WAN by installing a third-party "TCP stack" product.

If replacing the TCP stack is not a feasible solution, you can reduce the amount of network traffic that the TCP/IP protocol has to deal with by setting the COMPRESS TRAFFIC attribute in the CREATE ACTIVE STANDBY PAIR or CREATE REPLICATION statement. See Compressing Replicated Traffic.

See AIX Prerequisites or Linux Prerequisites in the Oracle TimesTen In-Memory Database Installation, Migration, and Upgrade Guide for information about changing TCP/IP kernel parameters for better performance.

Configuring Network Interfaces With the ROUTE Clause

In a replication scheme, you need to identify the name of the host on which your database resides. The operating system translates this host name to one or more IP addresses.

When specifying the host for a database in a replication element, you should always use the name returned by the hostname command, as replication uses the same host name to verify that the current host is involved in the replication scheme. Replication schemes may not be created that do not include the current host.

While you must specify the host name returned by the operating system's hostname command when you specify the database name, you can configure replication to send or receive traffic over a different interface (other than the default) using the ROUTE clause.

If a host contains multiple network interfaces (with different IP addresses), you should specify which interfaces are to be used by replication using the ROUTE clause, unless you want replication to use the default interface. You must specify a priority for each interface. Replication tries to first connect using the address with the highest priority, and if a connection cannot be established, it tries the remaining addresses in order of priority until a connection is established. If a connection to a host fails while using one IP address, replication attempts to re-connect (or fall back) to another IP address, if more than one address has been specified in the ROUTE clause.

The syntax of the ROUTE clause is:

ROUTE MASTER FullDatabaseName SUBSCRIBER FullDatabaseName
  {{MASTERIP MasterHost | SUBSCRIBERIP SubscriberHost}
    PRIORITY Priority} [...]

Note:

Addresses for the ROUTE clause may be specified as either host names or IP addresses. However, if your host has more than one IP address configured for a given host name, you should only configure the ROUTE clause using the IP addresses, in order to ensure that replication uses only the IP addresses that you intend.

When using the ROUTE clause in an active standby pair, each master database is a subscriber of the other master database and each read-only subscriber is a subscriber of both master databases. This means that the CREATE ACTIVE STANDBY PAIR statement should include ROUTE clauses in multiples of two to specify a route in both directions.
When using the ROUTE clause in a classic replication scheme that defines dual masters, each master database is a subscriber of the other master database. This means that the CREATE REPLICATION statement should include ROUTE clauses in multiples of two to specify a route in both directions.

The following example shows how to configure multiple network interfaces for an active standby pair.

If host1 host is configured with a second interface accessible by the host1fast host name, and host2 is configured with a second interface at IP address 192.168.1.100, you may specify that the secondary interfaces are used with the replication scheme.

CREATE ACTIVE STANDBY PAIR dsn1, dsn2
 ROUTE MASTER dsn1 ON "host1" SUBSCRIBER dsn2 ON "host2"
    MASTERIP "host1fast" PRIORITY 1
    SUBSCRIBERIP "192.168.1.100" PRIORITY 1
 ROUTE MASTER dsn2 ON "host2" SUBSCRIBER dsn1 ON "host1"
    MASTERIP "192.168.1.100" PRIORITY 1
    SUBSCRIBERIP "host1fast" PRIORITY 1;

The following example shows how to configure multiple network interfaces for a classic replication scheme.

If host1 host is configured with a second interface accessible by the host1fast host name, and host2 is configured with a second interface at IP address 192.168.1.100, you may specify that the secondary interfaces are used with the replication scheme.

CREATE REPLICATION repscheme
 ELEMENT e1 TABLE ttuser.tab
    MASTER dsn1 ON host1
    SUBSCRIBER dsn2 ON host2
 ELEMENT e2 TABLE ttuser.tab
    MASTER dsn2 ON host2
    SUBSCRIBER dsn1 ON host1
 ROUTE MASTER dsn1 ON host1 SUBSCRIBER dsn2 ON host2
    MASTERIP host1fast PRIORITY 1
    SUBSCRIBERIP "192.168.1.100" PRIORITY 1
 ROUTE MASTER dsn2 ON host2 SUBSCRIBER dsn1 ON host1
    MASTERIP "192.168.1.100" PRIORITY 1
    SUBSCRIBERIP host1fast PRIORITY 1;

Alternately, on a replication host with more than one interface, you may want to configure replication to use one or more interfaces as backups, in case the primary interface fails or the connection from it to the receiving host is broken. You can use the ROUTE clause to specify two or more interfaces for each master or subscriber that are used by replication in order of priority.

If replication on the master host is unable to bind to the MASTERIP with the highest priority, it tries to connect using subsequent MASTERIP addresses in order of priority immediately. However, if the connection to the subscriber fails for any other reason, replication tries to connect using each of the SUBSCRIBERIP addresses in order of priority before it tries the MASTERIP address with the next highest priority.

The following example shows how to configure network priority on an active standby pair.

If the host1 host is configured with two network interfaces at IP addresses 192.168.1.100 and 192.168.1.101, and the host2 host is configured with two interfaces at IP addresses 192.168.1.200 and 192.168.1.201, you may specify that replication use IP addresses 192.168.1.100 and 192.168.200 to transmit and receive traffic first, and to try IP addresses 192.168.1.101 or 192.168.1.201 if the first connection fails.

CREATE ACTIVE STANDBY PAIR dsn1, dsn2
 ROUTE MASTER dsn1 ON "host1" SUBSCRIBER dsn2 ON "host2"
   MASTERIP "192.168.1.100" PRIORITY 1
   MASTERIP "192.168.1.101" PRIORITY 2
   SUBSCRIBERIP "192.168.1.200" PRIORITY 1
   SUBSCRIBERIP "192.168.1.201" PRIORITY 2;

The following example shows how to configure network priority for a classic replication scheme.

If the host1 host is configured with two network interfaces at IP addresses 192.168.1.100 and 192.168.1.101, and the host2 host is configured with two interfaces at IP addresses 192.168.1.200 and 192.168.1.201, you may specify that replication use IP addresses 192.168.1.100 and 192.168.200 to transmit and receive traffic first, and to try IP addresses 192.168.1.101 or 192.168.1.201 if the first connection fails.

CREATE REPLICATION repscheme
 ELEMENT e TABLE ttuser.tab
   MASTER dsn1 ON host1
   SUBSCRIBER dsn2 ON host2
 ROUTE MASTER dsn1 ON host1 SUBSCRIBER dsn2 ON host2
   MASTERIP "192.168.1.100" PRIORITY 1
   MASTERIP "192.168.1.101" PRIORITY 2
   SUBSCRIBERIP "192.168.1.200" PRIORITY 1
   SUBSCRIBERIP "192.168.1.201" PRIORITY 2;

Configuring Network Interfaces When Not Using the ROUTE Clause

You can configure replication so that it uses the correct host names and IP addresses for each host when not using the ROUTE clause.

Identifying Database Hosts on UNIX or Linux Without Using the ROUTE Clause

When possible, you should use the ROUTE clause of a replication scheme to identify database hosts and the network interfaces to use for replication.

However, if you have a replication scheme configuration that does not use the ROUTE clause, you can configure the operating system and DNS files for a replication host with multiple network interfaces.

If a host contains multiple network interfaces (with different IP addresses) and replication is not configured with a ROUTE clause, TimesTen Classic replication tries to connect to the IP addresses in the same order as returned by the gethostbyname call. It tries to connect using the first address; if a connection cannot be established, it tries the remaining addresses in order until a connection is established. TimesTen Classic replication uses this same sequence each time it establishes a new connection to a host. If a connection to a host fails on one IP address, TimesTen Classic replication attempts to re-connect (or fall back) to another IP address for the host in the same manner described above.

There are two basic ways you can configure a host to use multiple IP addresses on UNIX or Linux platforms: DNS or the /etc/hosts file.

Note:

If you have multiple network interface cards (NICs), be sure that "multi on" is specified in the /etc/host.conf file. Otherwise, gethostbyname cannot return multiple addresses.

For example, if your machine has two NICs, use the following syntax for your /etc/hosts file:

127.0.0.1  localhost
IP_address_for_NIC_1  official_hostname optional_alias
IP_address_for_NIC_2  official_hostname optional_alias

The host name official_hostname is the name returned by the hostname command.

When editing the /etc/hosts file, keep in mind that:

You must log in as root to change the /etc/hosts file.
There should only be one line per IP address.
There can be multiple alias names on each line.
When there are multiple IP addresses for the same host name, they must be on consecutive lines.
The host name can be up to 30 characters long.

For example, the following entry in the /etc/hosts file on a UNIX or Linux platform describes a server named Host1 with two IP addresses:

127.0.0.1        localhost
10.10.98.102     Host1
192.168.1.102    Host1

To specify the same configuration for DNS, your entry in the domain zone file would look like:

Host1     IN     A     10.10.98.102
          IN     A     192.168.1.102

In either case, you only need to specify Host1 as the host name in your replication scheme and replication uses the first available IP address when establishing a connection.

In an environment in which multiple IP addresses are used, you can also assign multiple host names to a single IP address in order to restrict a replication connection to a specific IP address. For example, you might have an entry in your /etc/hosts file that looks like:

127.0.0.1        localhost
10.10.98.102     Host1
192.168.1.102    Host1 RepHost1

or a DNS zone file that looks like:

Host1     IN     A     10.10.98.102
          IN     A     192.168.1.102
RepHost1  IN     A     192.168.1.102

If you want to restrict replication connections to IP address 192.168.1.102 for this host, you can specify RepHost1 as the host name in your replication scheme. Another option is to simply specify the IP address as the host name in either the CREATE ACTIVE STANDBY PAIR or CREATE REPLICATION statements used to configure your replication scheme.

Note:

You must be consistent in identifying a database host in a replication scheme. Do not identify a host using its IP address for one database and then use its host name for the same or another database.

User-Specified Addresses for TimesTen Daemons and Subdaemons

By default, the TimesTen main daemon, all subdaemons, and all agents use any available address to listen on a socket for requests.

You can modify the timesten.conf file to specify an address for communication among the agents and daemons by including a listen_addr option.

See Managing TimesTen Daemon Attributes in Oracle TimesTen In-Memory Database Operations Guide.

Suppose that your machine has two NICs whose addresses are 10.10.10.100 and 10.10.11.200. The loopback address is 127.0.0.1. Then keep in mind the following as it applies to the replication agent:

If you do not set the listen_addr option in the timesten.conf file, then any process can talk to the daemons and agents.
If you set listen_addr to 10.10.10.100, then any process on the local host or the 10.10.10 net can talk to daemons and agents on 10.10.10.100. No processes on the 10.10.11 net can talk to the daemons and agents on 10.10.10.100.
If you set listen_addr to 127.0.0.1, then only processes on the local host can talk to the daemons and agents. No processes on other hosts can talk the daemons and agents.

Identifying the Local Host of a Replicated Database

Ordinarily, TimesTen Classic replication is able to identify the hosts involved in a replication configuration using normal operating system host name resolution methods.

However, in some rare instances, if the host has an unusual host name configuration, TimesTen is unable to determine that the local host matches the host name as specified in the replication scheme. When this occurs, you receive error 8191, "This store is not involved in a replication scheme," when attempting to start replication using ttRepStart or ttAdmin -repStart.

The ttHostNameSet built-in procedure may be used in this instance to explicitly indicate to TimesTen that the current database is in fact the database specified in the replication scheme. See ttHostNameSet in Oracle TimesTen In-Memory Database Reference.