Depending on the number of available network ports on your Oracle VM Servers, and whether or not you use VLANs, you can create additional networks and assign network functions to them. The exception would be the Management function, which is already assigned, and cannot be removed from the management network(s) created when the Oracle VM Servers were discovered. For example, if your Oracle VM Servers have two NICs, you might create a second network with the Virtual Machine channel. Equally, networks can share functions, so you can add the Storage function to your Management network if your storage is connected to the same network as defined by the Management network.
Since it is possible that a single network can be used for multiple functions, the term used for a network function is channel. Therefore, you may want to separate different network functions into different channels. Some of these channels may share the same logical network, but ideally each channel should be assigned its own logical network.
After your management networks are in place, you can plan for the creation of other types of network. Note that once a port is selected for a particular network, it cannot be selected again when creating additional networks. You can use a combination of network bonding and VLAN Interfaces to create all the networks needed for your environment using your existing ports. Network bonding is covered in Section 5.4, “How is Network Bonding Used in Oracle VM?”; VLAN Interfaces are covered in Section 5.5, “How are VLANs Used in Oracle VM?”.
If you have more than two ports on your Oracle VM Servers, or if you are using VLANs, you can create additional networks for Storage channels. These networks might be used to connect your Oracle VM Servers to either iSCSI or NFS-based storage. Generally, all Oracle VM Servers that belong to the same pool access the same storage. For each network created, you select a port, bond or VLAN interface on each Oracle VM Server to participate in this network.
You can also create a separate network for the Live Migrate channel. After the initial server discovery, the Live Migrate channel is assigned to the Management network. Oracle VM encrypts migration traffic using SSL, to protect sensitive data from exploitation and to eliminate the requirement for a dedicated network. Nonetheless, if you have sufficient network resources on your Oracle VM Servers within a server pool, you can choose to create a separate network for live migration of virtual machines.
Similarly, the Cluster Heartbeat network channel is assigned to the Management network upon discovering the first Oracle VM Server. The heartbeat communication does not generate a lot of traffic on the network, and therefore does not have much impact on the Management network. It is however susceptible to latency. For this reason, you can choose to create a separate network for the cluster heartbeat function.
Though you can create several networks for the heartbeat and live migration functions, each Oracle VM Server can only participate in one heartbeat and live migration network.
Network configuration is independent of your server pool configuration, but both entities must be taken into account when designing your overall networking infrastructure. Oracle VM Manager communicates with all Oracle VM Servers in the environment, using the management port, independent of how Oracle VM Servers are grouped to form server pools. Some network configuration in your environment might be dependent on the storage available to specific server pools. Virtual machines deployed from separate server pools might use the same external network. For this reason, it is best to plan your network design based on current network and storage setup as well as anticipated growth.
For more information on creating a network, refer to Create New Network in the Oracle VM Manager User's Guide.
The Management channel is used to manage the physical Oracle VM Servers in a server pool, for example, to update the Oracle VM Agent on the different Oracle VM Servers. This network function is assigned to at least one network by default.
In Oracle VM the management network interface and the public interface (i.e. default route) are expected to be the same on each Oracle VM Server. Other types of network usage are allowed on the same interface, for example through the use of VLANs and/or network bridges.
The first step in configuring your Oracle VM environment is to discover your Oracle VM Servers. This step assumes that the Oracle VM Manager host and all of the Oracle VM Servers can communicate over the same network, though the Oracle VM Servers and Oracle VM Manager can reside in different subnets. When you discover the first Oracle VM Server, the management network is created automatically and takes its name from the subnet to which the Oracle VM Server is connected. Each additional Oracle VM Server discovered is either on a previously known subnet, or a new subnet. If the Oracle VM Server is on a new subnet then a new management network is constructed; if the Oracle VM Server is on a known subnet then it is simply added to that subnet. Each server in your Oracle VM environment can only have one interface designated for management, belonging to a single management network object in the Oracle VM Manager's database.
Although the Oracle VM Manager and its discovered and owned Oracle VM Servers may be on different subnets as long as they can reach each other, Network Address Translation (NAT) is not supported in this configuration. NAT would lead to a discrepancy between the actual management IP of the Oracle VM Server and the IP provided during discovery.
A network port on every Oracle VM Server is designated as the management interface during the installation of the Oracle VM Server and is configured as a bonded interface. Ports can be added to this bond or removed from it; however, you should not remove the initial port that you used to create the management interface from the bond. Once a management network is created, it can only be deleted if no servers have ports in the management network anymore.
The Cluster Heartbeat channel is used to verify if the Oracle VM Servers in a clustered server pool are up and running. The heartbeat function has a network component, where a TCP/IP communication channel is created with each Oracle VM Server. Each Oracle VM Server sends regular keep-alive packets and these packets are used to determine if each Oracle VM Server is alive.
It is recommended to separate the Cluster Heartbeat function from networks with high load, such as Storage and Live Migrate networks. If bandwidth drops too low, heartbeating connectivity might be interrupted, which could lead to rebooting of virtual machines and Oracle VM Servers.
Oracle VM uses OCFS2 as its underlying clustering file system to manage its storage repositories and provide access to shared storage.
A cluster heartbeat is an essential component in any OCFS2 cluster. It is charged with accurately designating nodes (in this case, nodes are Oracle VM Servers) as dead or alive. There are two types of heartbeats used in OCFS2:
The disk heartbeat where all the Oracle VM Servers in the cluster write a time stamp to the server pool file system device. See Section 3.8, “How is Storage Used for Server Pool Clustering?” for more information on this part of the clustering technology.
The network heartbeat which is where the Oracle VM Servers communicate through the network to signal to each other that every cluster member is alive.
The quorum is the group of Oracle VM Servers in a cluster that is allowed to operate on the shared storage. When there is a failure in the cluster, Oracle VM Servers may be split into groups that can communicate within their groups and with the shared storage, but not between groups. In this case, OCFS2 determines which group is allowed to continue and initiates fencing of the other group(s). Fencing is the act of forcefully removing an Oracle VM Server from a cluster. An Oracle VM Server with OCFS2 mounted will fence itself when it realizes that it does not have quorum in a degraded cluster. It does this so that other Oracle VM Servers are not stuck trying to access the cluster's resources. When an Oracle VM Server is fenced, it is rebooted and rejoins the cluster. If an Oracle VM Server is fenced, the virtual machines running on the fenced Oracle VM Server are migrated and restarted on other Oracle VM Servers if the virtual machines are HA enabled (virtual machines that are not HA enabled are not migrated).
The cluster heartbeat is sensitive to network interruptions and therefore the Cluster Heartbeat network should be given special attention and be treated separately to make sure that:
It is not sharing the same links with high traffic networks or networks that may experience sudden traffic spikes like the Storage or Live Migrate networks.
It has redundancy using a bond which ensures continued operation if one network path fails. See Section 5.4, “How is Network Bonding Used in Oracle VM?” for more information on configuring bonding.
The Live Migrate channel is used to migrate virtual machines from one Oracle VM Server to another in a server pool, without changing the status of the virtual machine.
While live migration should not be occurring frequently, during a live migration network traffic may spike. This could cause interruption to other services, particularly the cluster heartbeat functionality used for server pool clustering. As a result, configuring a separate network for this purpose can improve the performance and availability of other services within the environment.
The Storage channel lets you associate specific networks with storage use. Oracle VM Manager does not enforce you to use the Storage channel for storage traffic. This channel is primarily a logical association for your information. However, when there is an attempt to remove a server port from a network that is associated with the Storage channel, Oracle VM Manager prevents the removal if there are virtual machines running on the server. This occurs to avoid errors because the virtual machine might be accessing storage on the associated network.
The Virtual Machine channel is used for the network traffic between the different virtual machines in a server pool. The Virtual Machine channel can either be a standard inter-server network (routable through standard switches), or a server local network, an intra-server network without a route to an external physical network and dedicated to the selected Oracle VM Server. The implications of using a server local network for your Virtual Machine channel are discussed more in Section 5.7.4, “Logical Networks on a Single Oracle VM Server (local network)”.
Note that it is possible, and very likely, to have multiple networks with the Virtual Machine channel in one Oracle VM Manager.
Virtual Machine networks necessarily make use of network bridging to allow virtual machines running on different Oracle VM Servers to be able to communicate. This network bridging is discussed in more detail in Section 5.6.5.1, “Network Bridges”.
When creating a network with the virtual machine channel, a bridge is created automatically on the port, bond, or VLAN interface added to the network for each Oracle VM Server participating in this network. All network packets generated by the virtual machines are sent to the bridge configured for the virtual machines' network. The bridge acts as a Layer 2 switch, and directs packets to other virtual machines running on the Oracle VM Server, or to the port or bond, if the packets' destination is outside of the Oracle VM Server.
Though each virtual machine deployed within a network is usually assigned an IP address, either static or assigned using DHCP, there is no need to configure an IP address for the bridge on the Oracle VM Servers. When configuring your Virtual Machine network, if you specify an IP address for the port, bond, or VLAN interface you selected for this network, it is assigned to the bridge. You can choose not to assign an IP address to the selected port, bond, or VLAN interface. In this case, the bridge does not acquire an address but still functions as a Layer 2 switch.
In Figure 5.8, “Network bridge”, two network ports are specified for the network with the virtual machine channel. Therefore, these ports should be configured as a bonded interface. Since this network is configured with the virtual machine channel, a bridge is automatically created on each Oracle VM Server in the network. Neither the bridge nor the ports in the virtual machine network, have IP addresses assigned to them, though you may assign IP addresses if you wish during network creation.
Bridges are only created for networks with the virtual machine channel.