Refer to the Configuring SDP Protocol Support for Infiniband Network Communication to the Database Server section in the Oracle Database Net Services Administrator's Guide, located at:
http://download.oracle.com/docs/cd/B28359_01/network.111/b28316/performance.htm#i1008413
Ensure that you set the protocol to SDP.