6.2.3 Configuring Sun Datacenter InfiniBand Switch 36 Switch

The following procedure describes how to configure the Sun Datacenter InfiniBand Switch 36 switches.

1. Log in to the first database server as the root user. The first database server is the lowest database server in the rack, which is rack position U16.

   Note:

   If you do not have the password for the root user, then contact Oracle Support Services.

2. Use SSH to log in to the Sun Datacenter InfiniBand Switch 36 switch as the ilom-admin user. Find the default IP address for the Sun Datacenter InfiniBand Switch 36 switch for your system in "Default IP Addresses". The following is an example of the command:

   ssh ilom-admin@192.168.1.201
   

3. Use spsh to open the Integrated Lights Out Manager (ILOM) command line interface.

4. Configure the network interface using the following commands:

   set /SP/network pendingipdiscovery=static pendingipaddress=pending_ip \
   pendingipgateway=pending_gw pendingipnetmask=pending_nm
   
   set /SP/network commitpending=true 
   

   In the preceding commands, pending_ip, pending_gw, and pending_nm are IP addresses defined by the network administrator.

5. Use the exit command to exit the interface.

6. Edit the /etc/hosts file to set the IP address and host name using an editor such as vi. The following is an example of the updated file:

   #Do not remove the following link, or various programs
   #that require network functionality will fail.
   127.0.0.1     localhost.localdomain localhost
   10.7.7.32     dm01sw-ib1-ib2.example.com trnasw-ib2
   

   It is required that the second line has both fully-qualified and non-fully-qualified names.

7. Use the ILOM interface to configure the IP address, host name, NTP servers, and DNS servers.

8. Restart the switch.

9. Examine the firmware version using the following command:

   # version
   

   Refer to My Oracle Support Note 888828.1 for the current firmware version.

10. Check the health of the switch using the following command:

    # showunhealthy
    
    OK - No unhealthy sensors
    

11. Run the environment test using the following command:

    # env_test
    
    NM2 Environment test started:
    Starting Voltage test:
    Voltage ECB OK
    Measured 3.3V Main = 3.28 V
    Measured 3.3V Standby = 3.42 V
    Measured 12V =12.06 V
    Measured 5V =5.03 V
    Measured VBAT =3.06 V
    Measured 2.5V =2.53 V
    Measured 1.8V =1.79 V
    Measured I4 1.2V =1.22 V
    Voltage test returned OK
    Starting PSU test:
    PSU 0 present
    PSU 1 present
    PSU test returned OK
    Starting Temperature test:
    Back temperature 30.50
    Front temperature 33.88
    ComEx temperature 34.12
    I4 temperature 56,
    maxtemperature 57
    Temperature test returned OK
    Starting FAN test:
    Fan 0 not present
    Fan 1 running at rpm 12946
    Fan 2 running at rpm 12684
    Fan 3 running at rpm 12558
    Fan 4 not present
    FAN test returned OK
    Starting Connector test:
    Connector test returned OK
    Starting I4 test:
    I4 OK
    All I4s OK
    I4 test returned OK
    NM2 Environment test PASSED
    

12. Enable the InfiniBand Subnet Manager using the following command:

    # enablesm
    

    Note:

    If you get an error indicating the InfiniBand Subnet Manager is already running, then restart it as follows:

    # disablesm
    
    # enablesm

13. Verify the IP address is correct using the following command:

    # ifconfig eth0
    
    eth0    Link encap:Ethernet HWaddr 00:E0:4B:2A:07:2B
            inet addr:172.16.10.32 Bcast:172.16.10.255
    Mask:255.255.255.0
            inet6 addr:fe80::2e0:3a00:fe2a:61e/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
            RX packets:11927 errors:0 dropped:0 overruns:0 frame:0
            TX packets:89 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:720262 (703.3 KiB)TX bytes:11402 (11.1 KiB)
    

14. Verify the host name is correct using the following command:

    # hostname
    
    dm01sw-ib2.example.com
    

15. Set the Subnet Manager Master as described in "Setting the Subnet Manager Master on Oracle Exadata Database Machine Full Rack and Oracle Exadata Database Machine Half Rack". This step is needed for Oracle Exadata Database Machine Half Rack and Oracle Exadata Database Machine Full Rack.

16. Log out from the InfiniBand switch using the following command:

    # exit
    

Note:

If the network settings did not appear after restart, then power cycle the switch by removing both power cords for one minute.