6 Configuring Exalogic Networking for a Virtual Environment
This chapter describes virtual Exalogic networking.
The contents in this chapter are specific to the Exalogic virtual environment and should be read in conjunction with the Chapter 4, "Networking Overview."
This chapter contains the following sections:
6.2 Explanation of Network Interfaces Map
In a Virtual Exalogic deployment, A hardware load balancer is used to distribute requests to two vServers in the Exalogic Rack which host Oracle Traffic Director. The Oracle Traffic director instances then direct traffic to other vServers which host the Identity and Access Management components.
The diagram above shows how these vServers are networked together and to the external corporate network where the load balancer sits.
This section contains the following topics:
6.2.1 Load Balancer
An external load balancer sits outside of the exalogic machine rack. Its purpose is to receive requests on the public ethernet network and distribute those requests to the Oracle Traffic Director nodes inside the machine rack using the front end EoIB network or to the external Oracle HTTP Servers.
6.2.2 Network Interface Bonding
In Virtual Exalogic Deployments, these are assigned when the vServer is created. Therefore, the exact bonding is determined by the order the networks are attached to the vServer.
For the purposes of this document, assume the network interfaces are as follows:
6.2.3 Oracle Traffic Director
Oracle Traffic Director (OTD) serves two functions: load balancing, and HTTP server.
As a load balancer, Oracle Traffic Director is configured in a way that it can direct requests to the Oracle Unified Directory servers using the internal IPoIB network using TCP and to direct internal call back requests from Oracle Traffic Director to SOA servers using the internal IPoIB network using HTTP.
Unless External Oracle HTTP Servers are used, then Oracle Traffic Director also functions as an HTTP Server. Oracle Traffic Director listens on the front end EoIB network for HTTP requests originating from the external load balancers. If these requests require access to the WebLogic managed servers on the compute nodes, then it directs these requests accordingly using the internal IPoIB network. *HTTP* requests on the front-end EoIB network.
6.2.4 External Oracle HTTP Servers
Optionally, you can use Oracle HTTP Servers, which sit on servers outside of the Exalogic machine rack. These servers receive requests from the Load Balancer and distribute those requests to application compute nodes inside the machine rack using EoIB.
6.2.5 Virtual Servers
In an Exalogic Virtual deployment, virtual servers are used instead of physical compute nodes to host services. These virtual hosts look and feel like physical servers but they are actually virtual environments managed by Exalogic Control.
These are referred to as vServers.
The following sections describe the networking configuration of each of the vServers.
6.2.5.1 Virtual Server 1 (vServer1)
vServer1 (WEBHOST1) hosts Oracle Traffic director which acts as both an internal load balancer and a web server.
-
It is configured to use the EoIB client access network. It uses this network to communicate with the external load balancer.
-
It is configured to use the IPoIB network for internal communications.
-
Oracle Traffic Director enables an IP address using a failover group to route requests to the Oracle Unified Directory servers using the IPoIB network.
-
Oracle Traffic Director acts as a failover node in the event that the IP address used for internal callbacks fails.
-
Oracle Traffic Directory is used to route application requests to the Weblogic managed servers making up the Application Tier.
6.2.5.2 Virtual Server 2 (vServer2)
vServer2 (WEBHOST2) serves two purposes. It hosts Oracle Traffic director which acts as both an internal load balancer and a web server. It also hosts the Oracle Identity and Access Management Applications which comprise Access Manager, Oracle Identity Manager and Oracle Unified Directory.
-
It is configured to use the EoIB client access network. It uses this network to communicate with the external load balancer.
-
It is configured to use the IPoIB network for internal communications.
-
Oracle Traffic Director enables an IP address using a failover group to route internal callback requests to SOA servers using the IPoIB network.
-
Oracle Traffic Director acts as a failover node in the event that the IP address used for Oracle Unified Directory fails.
-
Oracle Traffic Directory is used to route application requests to the Weblogic managed servers making up the Application Tier.
6.2.5.3 Virtual Server 3 (vServer3)
vServer3 (OAMHOST1) hosts the Oracle Identity and Access Management Applications which support the IAMAccessDomain and comprise Access Manager and Oracle Unified Directory.
-
It can be configured to use the EoIB client access network. It uses this network to communicate with external database servers or for external Oracle HTTP servers to communicate with the Weblogic Managed Servers.
-
It is configured to use the IPoIB network for internal communications.
-
Weblogic Managed Servers receive requests from Oracle Traffic Director on the internal IPoIB network.
-
Node Manager, which is used to start and stop the WebLogic managed servers, is configured to accept requests on the internal IPoIB interface.
-
This node hosts a virtual (floating) IP address which is configured on the client access network. This virtual IP addresses is used by the administration server. Although not necessary to use the client access network. The benefit of doing so is that it is possible to monitor the administration server outside of the Exalogic machine.
-
Oracle Unified Directory listens for requests on the internal IPoIB network.
6.2.5.4 Virtual Server 4 (vServer4)
vServer4 (OAMHOST2) hosts the Oracle Identity and Access Management Applications which support the IAMAccessDomain and comprise Access Manager and Oracle Unified Directory.
-
It can be configured to use the EoIB client access network. It uses this network to communicate with external database servers or for external Oracle HTTP servers to communicate with the Weblogic Managed Servers.
-
It is configured to use the IPoIB network for internal communications.
-
Weblogic Managed Servers receive requests from Oracle Traffic Director on the internal IPoIB network.
-
Node Manager, which is used to start and stop the WebLogic managed servers is configured to accept requests on the internal IPoIB interface.
-
Oracle Unified Directory listens for requests on the internal IPoIB network.
6.2.6 Virtual Server 5 (vServer5)
vServer5 (OIMHOST1) hosts the Oracle Identity and Access Management Applications which comprise the IAMGovernanceDomain, namely Oracle Identity Manager.
-
It can be configured to use the EoIB client access network. It uses this network to communicate with external database servers or for external Oracle HTTP servers to communicate with the Weblogic Managed Servers.
-
It is configured to use the IPoIB network for internal communications.
-
Weblogic Managed Servers receive requests from Oracle Traffic Director on the internal IPoIB network.
-
Node Manager, which is used to start and stop the WebLogic managed servers is configured to accept requests on the internal IPoIB interface.
-
This node hosts a virtual (floating) IP address which is configured on the client access network. This virtual IP addresses is used by the administration server. Although not necessary to use the client access network. The benefit of doing so is that it is possible to monitor the administration server outside of the Exalogic machine.
-
Two virtual (floating) IP addresses are attached to the IPoIB interface, which are used by the Oracle Identity Manager and SOA managed servers to facilitate server migration.
6.2.6.1 Virtual Server 6 (vServer6)
vServer6 (OIMHOST2) hosts the Oracle Identity and Access Management Applications which comprise the IAMGovernanceDomain, namely Oracle Identity Manager.
-
It can be configured to use the EoIB client access network. It uses this network to communicate with external database servers or for external Oracle HTTP servers to communicate with the Weblogic Managed Servers.
-
It is configured to use the IPoIB network for internal communications.
-
Weblogic Managed Servers receive requests from Oracle Traffic Director on the internal IPoIB network.
-
Node Manager, which is used to start and stop the WebLogic managed servers is configured to accept requests on the internal IPoIB interface.
-
Two virtual (floating) IP addresses are attached to the IPoIB interface, which are used by the Oracle Identity Manager and SOA managed servers to facilitate server migration.
6.3 Host Name and Networking Overview
Networking is a complicated but critical part of any Exalogic deployment. This guide utilizes the IPoIB network for internal communications and the EoIB network for external communications.
Table 6-2, "Exalogic Virtual IP Addresses Worksheet" is a summary of the required networking setup in the Exalogic machine rack. The following sections describe in detail how to set up this networking.
A column has been added to the table to allow you to add your own values for easier cross referencing.
Appropriate host name resolution is critical to topology designs that can sustain network changes, system relocation and disaster recovery scenarios. It is important that the required DNS (either /etc/hosts or central DNS server) definitions are in place and that WebLogic Servers use host names and virtual host names instead of using IP addresses and virtual IP addresses directly. Additionally, the Exalogic enterprise deployment requires a set of virtual server names for routing requests to the proper server or service within the topology through the external load balancer and the Oracle Traffic Director servers. See Section 4.3, "Virtual Server Names Used by the Topology."
These virtual server names must be resolvable in the corporate network. IPoIB addresses must be resolved only inside the rack's name resolution system. If multiple racks are going to be connected, to elude possible IP conflict, it is good practice to place these also in a central DNS server. Network administrators at the corporate level should enable this. Alternatively host names may be resolved through appropriate /etc/hosts file propagated through the different nodes. Table 6-2, "Exalogic Virtual IP Addresses Worksheet" provides an example of names for the different floating IP addresses used by servers in the SOA system.
Table 6-2 Exalogic Virtual IP Addresses Worksheet
| Hostname Example for This Guide | Interface | IP Address/Subnet | Customer Value | Type | Host | Bound By | Details |
|---|---|---|---|---|---|---|---|
|
WEBHOST1 |
bond0 |
192.168.10.1/255.255.224.0 |
IPoIB/ Fixed |
vServer1/WEBHOST1 |
NA |
Access to vServer11/WEBHOST1 via the internal IPoIB network. |
|
|
WEBHOST2 |
bond0 |
192.168.10.2/255.255.224.0 |
IPoIB/ Fixed |
vServer2/WEBHOST2 |
NA |
Access to vServer2/WEBHOST2 via the internal IPoIB network. |
|
|
OAMHOST1 |
bond0 |
192.168.10.3/255.255.224.0 |
IPoIB/Fixed |
vServer3/OAMHOST1 |
NA |
Access to vServer3/OAMHOST1 via the internal IPoIB network |
|
|
OAMHOST2 |
bond0 |
192.168.10.4/255.255.224.0 |
IPoIB/Fixed |
vServer4/OAMHOST2 |
NA |
Access to vServer4/OAMHOST2 via the internal IPoIB network |
|
|
OIMHOST1 |
bond0 |
192.168.10.5/255.255.224.0 |
IPoIB/Fixed |
vServer5/OIMHOST1 |
NA |
Access to vServer5/OIMHOST1 via the internal IPoIB network |
|
|
OIMHOST2 |
bond0 |
192.168.10.6/255.255.224.0 |
IPoIB/Fixed |
vServer6/OIMHOST2 |
NA |
Access to vServer6/OIMHOST2 via the internal IPoIB network |
|
|
OTDADMINVHN |
bond1:1 |
10.10.30.1/255.255.224.0 |
EoIB /Floating |
vServer1/WEBHOST1 |
OTD Administration Server |
A floating IP address for the Administration Server is recommended, if you want to manually migrate the OTD Administration Server from WEBHOST1 to WEBHOST2. |
|
|
IADADMINVHN |
bond1:2 |
10.10.30.2/255.255.224.0 |
EoIB /Floating |
vServer3/OAMHOST1 |
IAMAccessDomain Administration Server |
A floating IP address for the IAMAccessDomain Administration Server is recommended, if you want to manually migrate the Administration Server from OAMHOST1 to OAMHOST2. |
|
|
IGDADMINVHN |
bond1:3 |
10.10.30.3/255.255.224.0 |
EoIB /Floating |
vServer5/OIMHOST1 |
IAMGovernanceDomain Administration Server |
A floating IP address for the IAMGovernanceDomain Administration Server is recommended, if you want to manually migrate the Administration Server from OIMHOST2 to OIMHOST1 |
|
|
WEBHOST1VHN1 |
OTD |
10.10.50.1/255.255.224.0 |
EoIB /Floating |
vServer1/WEBHOST1 |
OTD - WEBHOST1 |
A floating IP Address managed by OTD. This is the IP Address to which load balancers will connect. This is optional |
|
|
WEBHOST2VHN1 |
OTD |
10.10.50.2/255.255.224.0 |
EoIB /Floating |
vServer2/WEBHOST2 |
OTD - WEBHOST2 |
A floating IP Address managed by OTD. This is the IP Address to which load balancers will connect. |
|
|
OIMHOST1VHN |
bond0:1 |
192.168.30.1/255.255.240.0 |
IPoIB/ Floating |
vServer5/OIMHOST1 |
WLS_OIM1 Default Channel |
Initially enabled in OIMHOST1 and can be failed over by server migration to OIMHOST2 |
|
|
OIMHOST2VHN |
bond0:1 |
192.168.30.2/255.255.240.0 |
IPoIB/ Floating |
vServer6/OIMHOST2 |
WLS_OIM2 Default Channel |
Initially enabled in OIMHOST2 and can be failed over by server migration to OIMHOST1 |
|
|
SOAHOST1VHN |
bond0:2 |
192.168.30.3/255.255.240.0 |
IPoIB/ Floating |
vServer5/OIMHOST1 |
WLS_SOA1 default channel |
Initially enabled in OIMHOST1 and can be failed over by server migration to OIMHOST2 |
|
|
SOAHOST2VHN |
bond0:2 |
192.168.30.4/255.255.240.0 |
IPoIB/ Floating |
vServer6/OIMHOST2 |
WLS_SOA2 default channel |
Initially enabled in OIMHOST2 and can be failed over by server migration to OIMHOST1. |
|
|
WEBHOST1-EXT |
bond1 |
10.10.10.1/255.255.240.0 |
EoIB/Fixed |
vServer1/WEBHOST1 |
NA |
A fixed IP allowing the vServer to be accessed by an External Load balancer |
|
|
WEBHOST2-EXT |
bond1 |
10.10.10.2/255.255.240.0 |
EoIB/Fixed |
vServer2/WEBHOST2 |
NA |
A fixed IP allowing the vServer to be accessed by an External Load balancer |
|
|
WEBHOST1-STOR |
bond3 |
192.168.32.1/255.255.240.0 |
IPoIB/Fixed |
vServer1/WEBHOST1 |
NA |
A fixed IP address allowing the vServer to connect to the ZFS Storage appliance using the internal network. |
|
|
WEBHOST2-STOR |
bond3 |
192.168.32.2/255.255.240.0 |
IPoIB/Fixed |
vServer2/WEBHOST2 |
NA |
A fixed IP address allowing the vServer to connect to the ZFS Storage appliance using the internal network. |
|
|
OAMHOST1-STOR |
bond3 |
192.168.32.3/255.255.240.0 |
IPoIB/Fixed |
vServer3/OAMHOST1 |
NA |
A fixed IP address allowing the vServer to connect to the ZFS Storage appliance using the internal network. |
|
|
OAMHOST2-STOR |
bond3 |
192.168.32.4/255.255.240.0 |
IPoIB/Fixed |
vServer4/OAMHOST2 |
NA |
A fixed IP address allowing the vServer to connect to the ZFS Storage appliance using the internal network. |
|
|
OIMHOST1-STOR |
bond3 |
192.168.32.5/255.255.240.0 |
IPoIB/Fixed |
vServer5/OIMHOST1 |
NA |
A fixed IP address allowing the vServer to connect to the ZFS Storage appliance using the internal network. |
|
|
OIMHOST2-STOR |
bond3 |
192.168.32.6/255.255.240.0 |
IPoIB/Fixed |
vServer6/OIMHOST2 |
NA |
A fixed IP address allowing the vServer to connect to the ZFS Storage appliance using the internal network. |
|
|
OAMHOST1-DATA |
bond4 |
192.168.10.3/255.255.240.0 |
IPoIB/Fixed |
vServer3/OAMHOST1 |
NA |
A fixed IP address allowing the vServer to connect to the Exadata appliance using the default internal network. |
|
|
OAMHOST2-DATA |
bond3 |
192.168.10.4/255.255.240.0 |
IPoIB/Fixed |
vServer4/OAMHOST2 |
NA |
A fixed IP address allowing the vServer to connect to the Exadata appliance using the default internal network. |
|
|
OIMHOST1-DATA |
bond4 |
192.168.10.5/255.255.240.0 |
IPoIB/Fixed |
vServer5/OIMHOST1 |
NA |
A fixed IP address allowing the vServer to connect to the Exadata appliance using the default internal network. |
|
|
OIMHOST2-DATA |
bond3 |
192.168.10.6/255.255.240.0 |
IPoIB/Fixed |
vServer6/OIMHOST2 |
NA |
A fixed IP address allowing the vServer to connect to the Exadata appliance using the default internal network. |
|
|
IDMINTERNAL |
OTD |
192.168.50.1/255.255.224.0 |
IPoIB/ Floating |
vServer1/WEBHOST1 |
NA |
Oracle Traffic Director failover group for SOA |
|
|
IDSTORE |
OTD |
192.168.50.2/255.255.224.0 |
IPoIB/ Floating |
vServer2/WEBHOST2 |
NA |
Oracle Traffic Director failover group for Oracle Unified Directory |
6.3.1 Additional Requirements for External OHS
If External OHS Servers are being used then the additional host names in the following table apply to an Exalogic Virtual configuration.
Table 6-3 Exalogic Virtual OHS Configuration Worksheet
| Hostname Example for This Guide | Interface | IP Address/Subnet | Customer Value | Type | Host | Bound By | Details |
|---|---|---|---|---|---|---|---|
|
OHSHOST1 |
eth0 |
201.19.23.10/255.255.255.0 |
ETH0/Fixed |
External OHSHOST1 |
Oracle HTTP Server |
Fixed IP that Oracle HTTP Server Listens on |
|
|
OHSHOST2 |
eth0 |
201.19.23.11/255.255.255.0 |
ETH0/Fixed |
External OHSHOST2 |
Oracle HTTP Server |
Fixed IP that Oracle HTTP Server Listens on |
|
|
OIMHOST1VHN-EXT |
bond1:2 |
10.10.10.7/255.255.224.0 |
EoIB/Floating |
ComputeNode1/IAMHOST1/vServer5 |
WLS_OIM1 Default External Channel |
Initially enabled on vServer5, can be failed over by server migration to vServer6 |
|
|
OIMHOST2VHN-EXT |
bond1:2 |
10.10.10.8/255.255.224.0 |
EoIB/Floating |
ComputeNode2/IAMHOST2/vServer6 |
WLS_OIM2 Default External Channel |
Initially enabled on vServer6, can be failed over by server migration to vServer5 |
6.4 Preparing the Network on Virtual Exalogic
This section contains the following topics:
6.4.1 Public EoIB Client Access Network
If you have not already done so, you must create a public client access network, which allows virtual servers to connect to the main organizational ethernet network. For details on how to do this refer to the Oracle Exalogic Elastic Cloud Administrator's Guide.
For the purposes of this Enterprise Deployment guide, it is assumed that this network has already been created and is called: EoIB-client
6.4.2 Creating a Private IPoIB Network
You need to create a private network to allow each of the vServers in the deployment to communicate with each other. This network will only be available to assigned vServers and ensures that network communication between IAM vServers is isolated from other network traffic.
To Create a Private IPoIB network for exclusive communication between the vServers in the deployment, perform the following steps:
-
Log in to Exalogic Control at the URL listed in Section 20.2, "About Identity and Access Management Console URLs."
-
Expand vDC Management.
-
Then Navigate to vDCs - Accounts - Cloud User Account
-
In the Actions window, click Create Private vNet.
-
Enter a Name For example:
IPoIB_IAM. -
Click Next.
-
Select the Number of IP Addresses to reserve on the network.
-
Click Next.
-
Click Finish.
6.4.3 Reserving Virtual IP Addresses
In a virtual Exalogic deployment, if you wish to use IP addresses from the default pool, these IP addresses must be reserved on the Private IPoIB network created in the previous section. Reserving the IP addresses ensures that they are not automatically assigned elsewhere.
To reserve IP addresses perform the following steps:
-
Log in to Exalogic Control at the URL listed in Section 20.2, "About Identity and Access Management Console URLs."
-
Expand vDC Management.
-
Then Navigate to vDCs - Accounts - Cloud User Account
-
Click the Networks tab. The Network Dashboard is displayed.
-
Select the network IPoIB_IAM listed under Private vNets
-
Click Allocate VIP Addresses. The Allocate VIP from vNet window is displayed.
-
Choose the number of virtual IP addresses you wish to reserve, for example 6, and click Allocate VIP. A window shows what virtual IP addresses have been reserved. Make a note of these.
Note:
This is only for virtual IP addresses on the internal IPoIB network, you will need to allocate virtual IP Addresses on the client access network for communication with Oracle Traffic Director and the Administration Servers.6.5 Routing for Multi-Homed Hosts
Now that you have added new interfaces to each Host, having only one default gateway might not be sufficient. You might want to have one interface for an Internet connection and another for a corporate WAN, for example.
In the example below, the different interfaces are shown, along with example IP addresses and gateway requirements:
| Interface | IP Address | Gateway Requirements |
|---|---|---|
| eth0 | 201.19.23.128 / 24 | Gateway IP 201.19.23.1 |
| bond0 | 192.168.10.1 / 24 | No Gateway requirements |
| bond1 | 10.10.10.101/ 24 | Gateway IP 10.10.10.1 |
As you can see, eth0 and bond1 must have their own respective default gateways.
Bond0, however, does not have any default gateway requirements. It is simply confined to their actual Layer 3 subnet.
Note:
These steps are shown here for completeness. However, the actions for updating the networking files must be performed once the vServers are created.To get around this, create rules and tables for routing lookups, as follows.
-
Check the existing table IDs by issuing this command:
ip rule list
-
Choose a unique id that has NOT already been used. In this example, we'll use 224 and 225.
-
For eth0, create the following two files:
The file
/etc/sysconfig/network-scripts/rule-eth0, which contains:from 201.19.23.128/32 table 224 to 201.19.23.128 table 224
The file
/etc/sysconfig/network-scripts/route-eth0, which contains:201.19.23.0/24 dev eth0 table 224 default via 201.19.23.1 dev eth0 table 224
-
For bond1 create the following two files:
The file /etc/sysconfig/network-scripts/rule-bond1, which contains:
from 10.10.10.10/32 table 225 to 10.10.10.10 table 225
The file /etc/sysconfig/network-scripts/route-bond1, which contains:
10.10.10.0/24 dev bond1 table 225 default via 10.10.10.1 dev bond1 table 225
-
Restart the network to make the configuration effective.
# service network restart
The hosts are now accessible from both routers.
6.6 Enable Virtual IP Addresses
Having completed the network chapter you need to assign virtual IP addresses to the various network interfaces and hosts as described in link to Section 9.8, "Enabling Virtual IP Addresses."
6.7 Verifying Network Connectivity
After having defined the Network, ensure that all of the network names are resolvable from each of the compute Nodes/vServers.
You do this by performing the following command on each compute node/vServer
ping -I interface hostname
for example
ping -I bond1 IADADMINVHN
Perform this test for each entry in Table 5-1, "Exalogic Physical IP Addresses Worksheet" and Table 6-2, "Exalogic Virtual IP Addresses Worksheet", depending on your deployment type.
ping -I bond1 IADADMINVHN ping -I bond0 SOAHOST1VHN ping -I bond0 SOAHOST2VHN ping -I bond0 OIMHOST1VHN ping -I bond0 OIMHOST2VHN ping -I bond0 OIMHOST1 ping -I bond0 OIMHOST2 ping -I bond0 OAMHOST1 ping -I bond0 OAMHOST2 ping -I bond0 WEBHOST1 ping -I bond0 WEBHOST2 ping -I bond1 IAMHOST1EXT ping -I bond1 IAMHOST2ext ping -I bond1 WEBHOST1ext ping -I bond1 WEBHOST2ext ping -I bond1 OTDADMINVHN ping -I bond1 DBHOST1 ping -I bond1 DBHOST2 ping -I bond1 IAMDBSCAN