Create an SDP Listener on the InfiniBand Network

Oracle RAC 11g Release 2 supports client connections across multiple networks, and it provides load balancing and failover of client connections within the network they are connecting. To add a listener for the Oracle Exalogic Elastic Cloud Software connections coming in on the Infiniband network, first add a network resource for the Infiniband network with Virtual IP addresses.

1. Edit /etc/hosts on each Database Domain in the cluster to add the virtual IP addresses you will use for the InfiniBand network. Make sure that these IP addresses are not used. The following is an example:

   # Added for Listener over IB

   192.168.10.21 ssc01db01-ibvip.mycompany.com ssc01db01-ibvip

   192.168.10.22 ssc01db02-ibvip.mycompany.com ssc01db02-ibvip

2. On one of the Database Domains, as the root user, create a network resource for the InfiniBand network, as in the following example:

   # /u01/app/grid/product/11.2.0.2/bin/srvctl add network -k 2 -S 192.168.10.0/255.255.255.0/bondib0

3. Validate that the network was added correctly, by running one of the following commands:

   # /u01/app/grid/product/11.2.0.2/bin/crsctl stat res -t | grep net

   ora.net1.network

   ora.net2.network -- Output indicating new Network resource

   or

   # /u01/app/grid/product/11.2.0.2/bin/srvctl config network -k 2

   Network exists: 2/192.168.10.0/255.255.255.0/bondib0, type static -- Output indicating Network resource on the 192.168.10.0 subnet

4. Add the Virtual IP addresses on the network created in Step 2, for each node in the cluster.

   srvctl add vip -n ssc01db01 -A ssc01db01-ibvip/255.255.255.0/bondib0 -k 2

   srvctl add vip -n ssc01db02 -A ssc01db02-ibvip/255.255.255.0/bondib0 -k 2

5. As the "oracle" user (who owns the Grid Infrastructure Home), add a listener which will listen on the VIP addresses created in Step 3.

   srvctl add listener -l LISTENER_IB -k 2 -p TCP:1522,/SDP:1522

6. For each database that will accept connections from the middle tier, modify the listener_networks init parameter to allow load balancing and failover across multiple networks (Ethernet and InfiniBand). You can either enter the full tnsnames syntax in the initialization parameter or create entries in tnsnames.ora in $ORACLE_HOME/network/admin directory. The TNSNAMES.ORA entries must exist in the GRID_HOME. The following example first updates tnsnames.ora. Complete this step on each Database Domain in the cluster with the correct IP addresses for that Database Domain. LISTENER_IBREMOTE should list all other Database Domains that are in the cluster. DBM_IB should list all Database Domains in the cluster.

   ---

   Note -  The TNSNAMES entry is only read by the database instance on startup, if you modify the entry that is referred to by any init.ora parameter (LISTENER_NETWORKS), you must restart the instance or issue an ALTER SYSTEM SET LISTENER_NETWORKS command for the modifications to take affect by the instance.

   ---

   (DESCRIPTION =
   DBM =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc01-scan)(PORT = 1521))
   (CONNECT_DATA =
   (SERVER = DEDICATED)
    
   (SERVICE_NAME = dbm)
   ))
    
   DBM_IB =
   (DESCRIPTION =
   (LOAD_BALANCE=on)
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc01db01-ibvip)(PORT = 1522))
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc01db02-ibvip)(PORT = 1522))
   (CONNECT_DATA =
   (SERVER = DEDICATED)
   (SERVICE_NAME = dbm)
   ))
    
   LISTENER_IBREMOTE =
   (DESCRIPTION =
   (ADDRESS_LIST =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc01db02-ibvip.mycompany.com)(PORT = 1522))
   ))
    
   LISTENER_IBLOCAL =
   (DESCRIPTION =
   (ADDRESS_LIST =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc01db01-ibvip.mycompany.com)(PORT = 1522))
   (ADDRESS = (PROTOCOL = SDP)(HOST = ssc01db01-ibvip.mycompany.com)(PORT = 1522))
   ))
    
   LISTENER_IPLOCAL =
   (DESCRIPTION =
   (ADDRESS_LIST =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc0101-vip.mycompany.com)(PORT = 1521))
   ))
    
   LISTENER_IPREMOTE =
   (DESCRIPTION =
   (ADDRESS_LIST =
   (ADDRESS = (PROTOCOL = TCP)(HOST = ssc01-scan.mycompany.com)(PORT = 1521))
   ))
   

7. Modify the listener_networks init parameter. Connect to the database instance as sysdba.

   SQLPLUS> alter system set listener_networks='((NAME=network2) (LOCAL_LISTENER=LISTENER_IBLOCAL)(REMOTE_LISTENER=LISTENER_IBREMOTE))', '((NAME=network1)(LOCAL_LISTENER=LISTENER_IPLOCAL)(REMOTE_LISTENER=LISTENER_IPREMOTE))' scope=both;

8. Stop and start LISTENER_IB for the modification in Step 7.

   srvctl stop listener -l LISTENER_IB

   srvctl start listener -l LISTENER_IB