Ingest Over InfiniBand is supported only with compute servers running release March 2015 or later with database patch 12.1.0.2 DBBP4 applied.
This chapter contains the following sections:
Oracle recommends that the Recovery Appliance receives backups from all Oracle Exadata Database Machine racks over an Ethernet network. This configuration, shown in "Example of Network Connections for Recovery Appliance", supports thousands of protected databases equally. Nonetheless, you can configure local Oracle Exadata Database Machine racks to back up to Recovery Appliance over the InfiniBand network. This configuration provides the local Exadata racks with priority access to Recovery Appliance. Redo and SQL*Net control information from the protected databases are still sent over the 10 Gb Ethernet network.
Note:
Do not attempt to use the InfiniBand network for redo transport as this configuration is not supported.To configure Ingest Over InfiniBand:
When configuring Ingest Over InfiniBand, you can connect Recovery Appliance to one or more local Oracle Exadata Database Machine racks using one of these methods:
If the Recovery Appliance rack has a spine switch, then use the multirack cabling tables for the appropriate number of racks. See Multirack Cabling Tables.
If the Recovery Appliance rack has no spine switch, then you can cable it in a way that is similar to cabling together multiple Oracle Exadata Database Machine quarter racks.
Figure 9-1 shows the cable connections between a Recovery Appliance rack and an Oracle Exadata Database Machine quarter rack. The leaf switches within each rack retain their existing seven connections. The leaf switches interconnect the racks with two links, each using the ports reserved for external connectivity.
Figure 9-1 Recovery Appliance Connected to an Exadata Quarter Rack
Figure 9-2 shows the cable connections from Recovery Appliance to an Oracle Exadata Database Machine half rack or full rack. The leaf switches within each rack retain their existing seven connections. The leaf switches interconnect the racks with two links, each using the ports reserved for external connectivity.
Figure 9-2 Recovery Appliance Connected to an Exadata Half or Full Rack
Figure 9-3 shows the cable connections from Recovery Appliance to two or more Exadata racks. All Oracle Exadata Database Machine racks must be half or full racks, and must be interconnected using a fat-tree topology.
Each Recovery Appliance leaf switch has two links to the spine switches in the Exadata racks. When connecting five or more racks, use one link to the spine switches.
Figure 9-3 Recovery Appliance Connected to Multiple Exadata Racks
After cabling the Recovery Appliance rack with one or more Oracle Exadata Database Machine racks, you must add the virtual IP addresses (VIPs) to the Recovery Appliance InfiniBand network.
To add the VIPs:
Recovery Appliance recognizes a preferred network and a default network for receiving backups into the appliance. The deployment procedure configures the 10 Gb Ethernet network as the default network. The InfiniBand network becomes the preferred network. The protected databases use the preferred network if they can contact it; otherwise, they use the default network.
The following procedure updates a table in Oracle Database with the VIPs created in "Adding VIPs to the Recovery Appliance InfiniBand Network".
To update Recovery Appliance with the Virtual IPs:
Recovery Appliance processes database backup and restore requests using dispatchers, which prevent an excessive number of dedicated connections to the appliance database.
The init.ora
file defines the dispatchers for the default ingest network as shown in this example:
*.dispatchers='' ra1.dispatchers='(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=raingest01-vip.example.com))(SDU=65536))(SERVICE=ZDLRAXDB)(DISPATCHERS=4)' ra2.dispatchers='(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=raingest02-vip.example.com))(SDU=65536))(SERVICE=ZDLRAXDB)(DISPATCHERS=4)'
You can use the SQL ALTER SYSTEM
command to add the dispatchers for the InfiniBand network to the init.ora
parameter settings.
To add the dispatchers for InfiniBand:
While still connected to Oracle Database as RASYS
, configure the dispatchers for the InfiniBand VIPs as shown in these examples:
SQL> ALTER SYSTEM SET dispatchers= '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP) (HOST=raadm01-priv-vip.example.com))(SDU=65536)) (SERVICE=ZDLRAXDB)(DISPATCHERS=4)' SCOPE=BOTH SID='ra1'; SQL> ALTER SYSTEM SET dispatchers= '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP) (HOST=raadm02-priv-vip.example.com))(SDU=65536)) (SERVICE=ZDLRAXDB)(DISPATCHERS=4)' SCOPE=BOTH SID='ra2';