Configuring the Additional Network Card for a Non-Oracle VM Environment

2.7.2 Configuring the Additional Network Card for a Non-Oracle VM Environment

You can configure the additional network card on an Oracle Exadata X6-2 or later database server for a non-Oracle VM environment.

This procedure assumes that you have already installed the network card in the Oracle Exadata database server but have not yet completed the configuration with Oracle Exadata Deployment Assistant (OEDA).

WARNING:

If you have already installed Oracle Grid Infrastructure on Oracle Exadata, then refer to the Oracle Clusterware documentation. Use caution when changing the network interfaces for the cluster.

Ensure you have the following information for the new network card.
You will need to input this information when you run ipconf.pl.
- IP address
- Netmask
- Gateway

Run the ipconf.pl script to configure the card.

The following example shows a sample ipconf.pl session. The output shows three network cards:

A quad port 10Gb card, on eth0 to eth3
A dual port 10Gb card, on eth4 and eth5, with only one port cabled
A dual port 10Gb card, on eth6 and eth7, with only one port cabled. This is the new network card.

For sample output for Oracle Exadata X7-2, see Viewing the Network Interfaces.

# cd /opt/oracle.cellos/
# ./ipconf.pl

Logging started to /var/log/cellos/ipconf.log
Interface ib0   is                      Linked.    hca: mlx4_0
Interface ib1   is                      Linked.    hca: mlx4_0
Interface eth0  is                      Linked.    driver/mac: 
ixgbe/00:10:e0:8b:22:e8 (slave of vmeth0)
Interface eth1  is                      Linked.    driver/mac: 
ixgbe/00:10:e0:8b:22:e9 (slave of bondeth0)
Interface eth2  is                      Linked.    driver/mac: 
ixgbe/00:10:e0:8b:22:e9 (slave of bondeth0)
Interface eth3  is                      Linked.    driver/mac: 
ixgbe/00:10:e0:8b:22:eb
Interface eth4  is                      Linked.    driver/mac: 
ixgbe/90:e2:ba:ac:1d:e4
Interface eth5  is .................... Unlinked.  driver/mac: 
ixgbe/90:e2:ba:ac:1d:e5
Interface eth6  is ...                  Linked.    driver/mac: 
ixgbe/90:e2:ba:78:d0:10
Interface eth7  is .................... Unlinked.  driver/mac: 
ixgbe/90:e2:ba:78:d0:11

bondeth0 eth1,eth2 UP      vmbondeth0 10.128.1.169  255.255.240.0
10.128.0.1  SCAN       test08client02.example.com
bondeth1 None      UNCONF 
bondeth2 None      UNCONF 
bondeth3 None      UNCONF 
Select interface name to configure or press Enter to continue: eth6
Selected interface. eth6
IP address or up or none: 10.129.19.34
Netmask: 255.255.248.0
Gateway (IP address or none) or none: 10.129.16.0

Select network type for interface from the list below
1: Management
2: SCAN
3: Other
Network type: 3

Fully qualified hostname or none: test08adm02-bkup.example.com
Continue configuring or re-configuring interfaces? (y/n) [y]: n
...
Do you want to configure basic ILOM settings (y/n) [y]: n
[Info]: Custom changes have been detected in /etc/sysconfig/network-script
s/ifcfg-eth6
[Info]: Original file /etc/sysconfig/network-scripts/ifcfg-eth6 will be 
saved in /opt/oracle.cellos/conf/network-scripts/backup_by_Exadata_ipconf
[Info]: Original file /etc/ssh/sshd_config will be saved in /etc/ssh/sshd_
config.backupbyExadata
[Info]: Generate /etc/ssh/sshd_config with ListenAddress(es) 10.128.18.106, 
10.129.19.34, 10.128.1.169, 192.168.18.44, 192.168.18.45
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
[Info]: Save /etc/sysctl.conf in /etc/sysctl.conf.backupbyExadata
[Info]: Adjust settings for IB interfaces in /etc/sysctl.conf
Re-login using new IP address 10.128.18.106 if you were disconnected after 
following commands
ip addr show vmbondeth0
ip addr show bondeth0
ip addr show vmeth0
ip addr show eth0
ifup eth6
sleep 1
ifup vmeth6
sleep 1
ip addr show vmeth6
ip addr show eth6
sleep 4
service sshd condrestart

If you need to set up the network card with VLAN, perform these steps:

Add the VLAN ID to the /opt/oracle.cellos/cell.conf file.

Locate the Ethernet interface in the file. For example:

<Interfaces>
  <Gateway>10.129.16.0</Gateway>
  <Hostname>test08adm02-bkup.example.com</Hostname>
  <IP_address>10.129.19.34</IP_address>
  <IP_enabled>yes</IP_enabled>
  <IP_ssh_listen>enabled</IP_ssh_listen>
  <Inet_protocol>IPv4</Inet_protocol>
  <Name>eth6</Name>
  <Net_type>Other</Net_type>
  <Netmask>255.255.248.0</Netmask>
  <State>1</State>
  <Status>UP</Status>
  <Vlan_id>0</Vlan_id>
</Interfaces>

Add the VLAN ID to the <Vlan_id> element. The following example shows the interface configured with VLAN ID of 2122.

<Interfaces>
  <Gateway>10.129.16.0</Gateway>
  <Hostname>test08adm02-bkup.example.com</Hostname>
  <IP_address>10.129.19.34</IP_address>
  <IP_enabled>yes</IP_enabled>
  <IP_ssh_listen>enabled</IP_ssh_listen>
  <Inet_protocol>IPv4</Inet_protocol>
  <Name>eth6</Name>
  <Net_type>Other</Net_type>
  <Netmask>255.255.248.0</Netmask>
  <State>1</State>
  <Status>UP</Status>
  <Vlan_id>2122</Vlan_id>
</Interfaces>

Run the following command to configure the network interface using the modified cell.conf file:
```
# /opt/oracle.cellos/ipconf.pl -init -force
```

Validate the interface has the VLAN configured by checking that the /etc/sysconfig/network-scripts directory contains files with the VLAN ID in the filename. For example, if the VLAN ID is 2122, you should see the following files:

# ls -ltr /etc/sysconfig/network-scripts/*2122*

-rw-r----- 1 root root 250 Sep  7 14:39 /etc/sysconfig/network-scripts/ifcfg-eth6.2122
-rw-r----- 1 root root  85 Sep  7 14:39 /etc/sysconfig/network-scripts/route-eth6.2122
-rw-r----- 1 root root  56 Sep  7 14:39 /etc/sysconfig/network-scripts/rule-eth6.2122

Reboot the database server for the changes to take effect.
```
# shutdown -r now
```
Check that the network is working by pinging the gateway. For example:
```
# ping 10.129.16.0
```

Parent topic: Adding and Configuring an Extra Network Card on Oracle Exadata