Set or Modify the Distribution Scheme of a Database (dbDistribute)

The dbDistribute command can add, remove, evict, and replace elements of a database in the distribution map of the database, then distribute or redistribute data among elements. You must always use -apply to apply changes and redistribute data. You can do this either in the same command or in a separate command.

ttGridAdmin dbDistribute name 
         [-list]
         [-add all | hostname[.instancename]]
         [-remove hostname[.instancename] [-replaceWith hostname[.instancename]]]
         [-evict hostname[.instancename] [-replaceWith hostname[.instancename]]]
         [-apply|-reset|-resync]

Wait until the elements are loaded on all instances on which you will perform operations before using dbDistribute. You can use the dbStatus command to confirm this.

See Define the Distribution Map of the Database in Oracle TimesTen In-Memory Database Scaleout User's Guide for additional information.

Options

The dbDistribute command has the options:

Option	Description
`name`	Name of the database for data distribution changes.
-`add all \|` `hostname[.instancename]`	Adds elements to the distribution map, either all currently existing elements in the grid or elements on the specified instances. (If an element was not created because an instance is down, there would be no attempt to add it with `-add all`. It is not currently existing.) Specify one instance per usage, but you can use `-add` more than once on a command line. When the additions are applied, data will be distributed evenly across the grid. Notes: If you use `-add all`, you must use `-apply` in the same command. Until you issue `-apply`, the element is marked to be added but is not actually added yet. Also see Notes below for this and other options taking `hostname[.instancename]`.
`-remove` `hostname[.instancename]`	Use this option in any circumstance where you want to remove, and optionally replace, an element, such as to replace an older host system with a newer one. Also see Notes below. Specify one instance per usage, but you can use `-remove` more than once on a command line. It is typical to use `-replaceWith` to replace the element. The `-remove` option without `-replaceWith` results in redistribution of data. If you have a grid with `k=2` and you remove one element of a replica set, you must either replace it or also remove the other element of the replica set. Note: Until you issue `-apply`, the element is marked for removal but is not actually removed yet. Also see Redistributing Data in a Database in Oracle TimesTen In-Memory Database Scaleout User's Guide.
`-evict` `hostname[.instancename]`	Use this option if all elements of a replica set (one element if `k=1`, two elements if `k=2`) have unrecoverable failures and you cannot repair them. Important: Using the `-evict` option inevitably results in data loss. Use this only as a last resort. Specify one instance per usage, but you can use `-evict` more than once on a command line. If you use `-evict`, you must evict all elements in the replica set. You can use `-replaceWith` to replace the element. Notes: Do not issue or apply -evict together with -add or -remove. Until you issue `-apply`, the element is marked for eviction but is not actually evicted yet. Eviction results in the element being forcibly unloaded. Also see Notes below for additional considerations. For additional information, see Redistributing Data in a Database in Oracle TimesTen In-Memory Database Scaleout User's Guide.
`-replaceWith` `hostname[.instancename]`	Optionally use this with `-evict` or `-remove` to have the specified replacement contain the same data. The element on the replacing instance must not have previously been added to the distribution. The `-replaceWith` option must immediately follow the corresponding `-remove` or `-evict` option on the command line.
`-list`	Displays the current and pending distribution map of the database ("Holds Data" and "Will Hold Data", respectively).
`-apply`	Applies the new distribution to the database. You can use this option by itself to apply settings from previous commands, or in the same command line with the settings.
`-reset`	Discards all distribution settings that have not yet been applied. This option cannot be combined with any other option. Note: You cannot use `-reset` while distribution (`-apply`) is in progress. You can try `-resync` instead, as appropriate.
`-resync`	Attempts to resynchronize metadata in the user database with metadata in the active management instance in case the state of a `dbDistribute -apply` command is unknown. For example, the user database and management instance may not have matching states due to some failure or loss of communication. In some cases, the management instance may not know about the success or failure of a `dbDistribute` operation on the data instances and is left in an intermediate state. This option cannot be used with any other `dbDistribute` options. See Recovering from a Data Distribution Error in Oracle TimesTen In-Memory Database Scaleout User's Guide for related information. Note: The `-resync` option results in metadata in the management instance being read to see if there is a `dbDistribute` operation that is in progress but was neither committed nor rolled back. Resynchronizing may involve committing or rolling back the metadata changes of the `dbDistribute` operation (which are intended to be recorded in the management instance).

Examples

This example adds all elements in the grid to the distribution map then distributes data among the elements:

% ttGridAdmin dbDistribute database1 -add all -apply
Distribution map updated

You can then use the -list option to show the distribution map of elements in the grid (elements able to hold data):

% ttGridAdmin dbDistribute database11 -list
Distribution Map version: 1
RS Host        Instance  Holds Data Will Hold Data Removed Evicted
-- ----------- --------- ---------- -------------- ------- -------
 1  mysys3host griddata1          Y              Y       N       N
 1  mysys4host griddata2          Y              Y       N       N
 2  mysys5host griddata3          Y              Y       N       N
 2  mysys6host griddata4          Y              Y       N       N

Now remove both elements in replica set 1, then look at the -list output again, which indicates the two elements removed from the grid and therefore unable to hold data:

% ttGridAdmin dbDistribute database1 -remove mysys3host.griddata1
Element mysys3host.griddata1 is removed
Distribution map change enqueued
% ttGridAdmin dbDistribute database1 -remove mysys4host.griddata2
Element mysys4host.griddata2 is removed
Distribution map change enqueued
% ttGridAdmin dbDistribute database1 -apply
Distribution map updated

% ttGridAdmin dbDistribute database1 -list
Distribution Map version: 3
RS   Host        Instance  Holds Data Will Hold Data Removed Evicted
---- ----------- --------- ---------- -------------- ------- -------
NULL  mysys3host griddata1          N              N       Y       N
NULL  mysys4host griddata2          N              N       Y       N
   1  mysys5host griddata3          Y              Y       N       N
   1  mysys6host griddata4          Y              Y       N       N

The following is a new example that evicts two elements (from the same replica set) then looks at the -list output, which shows the two elements evicted from the grid and therefore unable to hold data.

% ttGridAdmin dbDistribute database1 -evict mysys3host.griddata1 -evict
mysys4host.griddata2 -apply
Distribution map updated
 
% ttGridAdmin dbDistribute database1 -list
Distribution Map version: 2
RS   Host       Instance  Holds Data Will Hold Data Removed Evicted
---- ---------- --------- ---------- -------------- ------- -------
NULL mysys3host griddata1          N              N       N       Y
NULL mysys4host griddata2          N              N       N       Y
   1 mysys5host griddata3          Y              Y       N       N
   1 mysys6host griddata4          Y              Y       N       N

This example shows where the -resync option successfully completed a data distribution operation:

% ttGridAdmin dbDistribute database1 -apply
...

(Process fails or is interrupted.)

% ttGridAdmin dbDistribute database1 -resync
Distributiom map updated

And this example shows where the -resync option rolled back a data distribution operation:

% ttGridAdmin dbDistribute database1 -apply
...

(Process fails or is interrupted.)

% ttGridAdmin dbDistribute database1 -resync
Distributiom map Rolled Back

Notes

You can use -list and -resync while distribution is in progress. Other operations will fail if distribution is in progress.
To specify an element, express its instance as hostname[.instancename]. The host name is required. The instance name is required only if there are multiple instances on the host. (See Grid Objects and Object Naming.)
If you need to confirm which elements are in each replica set, use the dbStatus command with the -replicaSet option.
Once an element has been removed or evicted from the distribution, the only possibility is to eliminate it with dbDestroy -instance. It is advisable to do that as soon as possible to reclaim the disk space that it used. If you want to be able to use the instance again later, you must recreate the element with dbCreate -instance, then add it to the distribution.
If k=2 and one element of a replica set has an irrecoverable failure, use -remove and -replaceWith to make the replica set fully operational again. Do not use -evict when an active replica is available.
If all elements of any replica set are down, you cannot perform global operations. If you cannot recover any element of the replica set, evicting the elements of the replica set will allow you to perform global operations again, but there will be permanent loss of data.
It is valid to use -add instead of -replaceWith to replace the elements of an evicted replica set, but in either case data on the evicted replica set is lost. Also note that -add results in redistribution of data while -replaceWith (used with either -evict or -remove) does not. See Recovering When the Replica Set Has a Permanently Failed Element in Oracle TimesTen In-Memory Database Scaleout User's Guide for additional information.
See Recovering from Failure in Oracle TimesTen In-Memory Database Scaleout User's Guide for additional information and considerations regarding failure modes.