3.1 Oracle VM Network Model

3.1.1 No Network Connection
3.1.2 Isolated Local Network
3.1.3 Trusted Internal Network
3.1.4 Untrusted Internal Network
3.1.5 Internet Facing Services

Use network configuration and access restrictions to expose to the outside world only what is needed.

From a network security point of view, remote host connectivity and access control are generally defined and scaled for these deployment categories:

These categories are ordered from low to high exposure to network security risks, and they are also described below in more detail in that order.

3.1.1 No Network Connection

Some highly confidential applications cannot tolerate the possibility of any remote connection or exploit. Machines that require this level of security are usually kept in access restricted rooms and often built without network cards. This type of configuration is a recommended best practice for credential originating systems such as a root certificate authority where the compromise of the root keys would put all certificates and applications that were signed by that authority at risk.

Although some of Oracle VM's features such as High Availability and Master Failover are not available without a network, peer Oracle VM Servers and shared storage, the product can be configured for single-node service where VMs can provide these secure applications using a local text console, host networking and/or a shared disk to access them.

If host based network security and traffic restriction is needed between virtual machines on the same host, the ebtables application can provide Ethernet frame filtering across the Linux based bridges.

Guidelines for the no network connection model:

  • Restrict physical access to the machines.

  • Restrict login access to trusted administrators.

  • Inspect removable media before connection and after disconnection, or ban them entirely from the secure area.

  • Securely wipe or destroy replaced hard drives. Make sure that replacement hard drives are are inspected or even wiped prior to installation.

  • Implement ebtables rules if host-based network traffic control is needed between guests.

3.1.2 Isolated Local Network

An isolated local network consists of servers that are connected in an environment which has no connection to any other network. In this model, there is zero network connectivity to a larger internal network or the Internet. Since there is no potential for remote exploits from a large number of unknown sources, this environment provides well defined physical, network and security characteristics.

By definition, access to this configuration is limited to personnel with access to the trusted admin hosts (including Oracle Enterprise Manager or Oracle VM Manager) on the closed, local network. Threats consist of an accidental "convenience" connection being made to other networks or a trusted admin installing an unsigned package or application that may introduce a malware agent.

Guidelines for the isolated local network model:

  • Set all default passwords for uniqueness and complexity. For using the High Availability and Virtual IP features where the master role may be moved to any node, ensure that the Oracle VM Agent password on each one is complex, but identical across the server pool.

  • Limit physical Oracle VM Server(dom0) and Oracle VM Manager access to essential personnel.

  • Avoid installing untrusted 3rd party software.

  • If clustering with NFS shared storage, or exporting OCFS2 repositories for backup, make sure the storage network is also restricted to the cluster subnet, especially since the documented export with root_no_squash represents a potential exposure.

3.1.3 Trusted Internal Network

A Trusted Internal Network is a well maintained, trusted and probably small internal network where network traffic is unrestricted to and from the cluster subnet. While this configuration provides advantages of access to a company's infrastructure, services and storage, it also introduces a few threats. Most of these are not malicious, and normal internal safeguards such as an Intrusion Detection Systems, anti-virus software and active network monitoring typically address rogue employees or bot infestations that may attempt to disrupt the network. Human error via network and new admin mistakes are the most common causes of cluster service outages.

Guidelines for the trusted internal network model:

  • Implement all the guidelines for isolated local networks.

  • Disable ssh root logins. Admins should log in as themselves (using their global uid) with user privileges. Set up sudo to allow specific admin commands per user or root, if needed. See Section 3.2, “Administrator Privileges in Oracle VM” for details.

  • If shared storage is mounted from outside the server pool subnet, restrict the export to the Oracle VM Server dom0 and VM IP addresses. Do not make NFS shares and LUNs on Fibre Channel or SAN storage globally accessible.

  • Establish appropriate network firewalls as discussed below.

A default Oracle Linux install has the iptables firewall enabled. However, a best practice for network filtering is to use an additional external firewall device. This permits separation of control (network administrator control of network access, use of globally administered and audited rules), and relieves the Oracle VM dom0 of the potentially high CPU load that can be created by implementing packet filter rules. In order to use Oracle VM Manager on a system with iptables enabled you can either open the ports used by Oracle VM Manager, or open all ports by disabling iptables. The latter is not advised in the absence of an external firewall devices because it removes network protection.

The web browser connection to Oracle VM Manager requires ports 7001, 7002 and 15901. Oracle VM Serversconnect to Oracle VM Manager using ports 7001, 7002, and optionally 54321 and 54322 for remote API access.

Oracle VM Manager, in turn, connects to the Oracle VM Servers through port 8899 for Oracle VM Agent communication, and port 5900 and up for secure VNC tunneling to virtual machines (one port per VM). Be sure to open the necessary ports on the different firewalls that may be installed between different parts of your network. Additional information and a diagram can also be found in Section 2.1, “Oracle VM Pre-Installation Tasks”.

When using iptables, ensure that the service is running on each node and that at least the default policy and ruleset are present in the file /etc/sysconfig/iptables:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j DROP
-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j DROP
-A INPUT -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p 50 -j ACCEPT
-A RH-Firewall-1-INPUT -p 51 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 21 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 53 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5950 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8002 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8003 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8899 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 7777 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT

3.1.4 Untrusted Internal Network

An untrusted internal network has the characteristics of being loosely maintained, unmonitored, readily compromised from the outside or from ongoing and uncorrected malware infested workstations on the inside.

In this model, the dom0 admin control and VMs have been partitioned into separate VLANs where traffic flow is controlled by a 3 zone router/firewall. The firewall policy is to allow the admin network to make outbound connections to anywhere, but blocks inbound connections from the untrusted internal network. Firewall policies to the VM network would depend on the application but should only expose service ports to the internal network that are needed. A signature driven Intrustion Detection System is an option on the VM network to monitor for traffic patterns that indicate an attack is underway.

This model views any traffic from the internal network as potentially hostile. Additional hardening of this network can be done by implementing hardware or iptables based firewalls and policies on the admin and dom0 hosts that block inbound traffic. The dom0 firewall rules can also be enhanced to reject peer dom0 traffic on all ports except OCFS2 (7777), Oracle VM Agent (8899), and the Xen administration ports (8002 and 8003).

Guidelines for the untrusted internal network model:

  • Implement all the guidelines for trusted internal networks.

  • Disable ssh root logins. Admins should log in as themselves (using their global uid) with user privileges. Set up sudo to allow specific admin commands per user or root, if needed. See Section 3.2, “Administrator Privileges in Oracle VM” for details.

  • Add failed connection logging to the existing iptables firewall just prior to the last REJECT line:

    -A RH-Firewall-1-INPUT -m limit --limit 15/minute -j LOG --log-prefix "FW Drop:"
    -A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
  • Set up a remote syslog server to track all user logins and firewall connection failures.

  • Disable the VNC connections on each Oracle VM Server dom0. Port-forward VNC connections via ssh and vncviewer rather than Oracle VM Manager:

    #-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5950 -j ACCEPT
    ssh -L 5900:vmserverhost:5900 vmserverhost
  • Disable port 8888 and IPP ports on Oracle VM Manager:

    #-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
    #-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 631 -j ACCEPT
    #-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8888 -j ACCEPT
    Note

    This will leave secure ports 22, 443 and 4433 for admin command line access and Oracle VM Manager connections.

  • Define a trusted admin network or host and limit Oracle VM Manager and Oracle VM Server ssh connections to that network or host. To implement this, comment out the existing ssh rule in the default /etc/sysconfig/iptables configuration file. Replace it with the information applicable to your trusted network or single admin host; for example the 192.168.0.0/24 network or the host with IP address 192.168.0.67:

    #-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
    -A RH-Firewall-1-INPUT -p tcp -s 192.168.0.0/24 --dport 22 -j ACCEPT
    -- or -- 
    -A RH-Firewall-1-INPUT -p tcp -s 192.168.0.67 --dport 22 -j ACCEPT
    Note

    Connections that fail will fall through and be logged by the logging rule provided in the third bullet. All the rules in /etc/sysconfig/iptables can also be changed to restrict access to selected networks or hosts.

3.1.5 Internet Facing Services

The most challenging model for securing a network connected host is placing VM based services directly on the Internet. In this case, great care must be taken to insure that the environment won't be compromised through a wide variety of penetration techniques and exploits from a large, diverse and aggressive set of threats.

Even on an untrusted internal network without intrusion detection, internal users are known quantities and typically aware that they are being monitored, so they generally do not tend to be abusive or even overly curious. In contrast, the Internet provides virtual anonymity for malicious individuals, criminal gangs and hostile, well equipped governments. With an anonymous cloak, probing and attacking high profile network segments with automated tools that run 24/7 is a standard procedure. On perimeter firewalls, aggressive penetration attempts on the ssh, mysql, oracle, netbios and smtp service ports can typically be measured in "attacks per second".

There are a variety of topologies to provide zone isolation that can help to repel attacks and mitigate the effects of a successful intrusion event. The following describes some basic approaches.

Guidelines for network models involving Internet facing services:

  • Implement all the guidelines for untrusted internal networks.

  • All Internet facing hosts and VMs reside on an isolated network stub called a DMZ (for demilitarized zone). Connections into and out of the network are managed and monitored by a stateful firewall that enforces well defined rules and policies. If possible, an individual DMZ VLAN per VM is optimal for guest VM isolation.

  • Never place Oracle VM Manager or any other mission critical administration tool on the Internet. Use a VPN to access these hosts on secure internal admin networks. This in particular applies to networked storage: these hosts should not be on the Internet-facing network.

  • Place each dom0 (Oracle VM Server) that is hosting Internet-exposed VMs in an administrative DMZ. This zone cannot initiate connections to the internal network or the Internet, but is reachable from an administrator VLAN. The dom0 itself should not the reachable from the Internet or the internal network.

  • Use selinux on Internet-exposed Linux VM guests whenever possible.