Oracle® VM Security Guide for Release 3.4

3.1 Oracle VM Network Model

Use network configuration and access restrictions to expose to the outside world only what is needed.

From a network security point of view, remote host connectivity and access control are generally defined and scaled for these deployment categories:

No network connection.
Isolated local network.
Trusted internal network.
Untrusted internal network.
Internet-facing service.

These categories are ordered from low to high exposure to network security risks, and they are also described below in more detail in that order.

3.1.1 No Network Connection

Some highly confidential applications cannot tolerate the possibility of any remote connection or exploit. Machines that require this level of security are usually kept in access restricted rooms and often built without network cards. This type of configuration is a recommended best practice for credential originating systems such as a root certificate authority where the compromise of the root keys would put all certificates and applications that were signed by that authority at risk.

Although some of Oracle VM's features such as High Availability and Master Failover are not available without a network, peer Oracle VM Servers and shared storage, the product can be configured for single-node service where VMs can provide these secure applications using a local text console, host networking and/or a shared disk to access them.

If host based network security and traffic restriction is needed between virtual machines on the same host, the ebtables application can provide Ethernet frame filtering across the Linux based bridges.

Guidelines for the no network connection model:

Restrict physical access to the machines.
Restrict login access to trusted administrators.
Inspect removable media before connection and after disconnection, or ban them entirely from the secure area.
Securely wipe or destroy replaced hard drives. Make sure that replacement hard drives are inspected or even wiped prior to installation.
Implement ebtables rules if host-based network traffic control is needed between guests.

3.1.2 Isolated Local Network

An isolated local network consists of servers that are connected in an environment which has no connection to any other network. In this model, there is zero network connectivity to a larger internal network or the Internet. Since there is no potential for remote exploits from a large number of unknown sources, this environment provides well defined physical, network and security characteristics.

By definition, access to this configuration is limited to personnel with access to the trusted admin hosts (including Oracle Enterprise Manager or Oracle VM Manager) on the closed, local network. Threats consist of an accidental "convenience" connection being made to other networks or a trusted admin installing an unsigned package or application that may introduce a malware agent.

Guidelines for the isolated local network model:

Set all default passwords for uniqueness and complexity. For using the High Availability and Virtual IP features where the master role may be moved to any node, ensure that the Oracle VM Agent password on each one is complex, but identical across the server pool.
Limit physical Oracle VM Server(dom0) and Oracle VM Manager access to essential personnel.
Avoid installing untrusted 3rd party software.
If clustering with NFS shared storage, or exporting OCFS2 repositories for backup, make sure the storage network is also restricted to the cluster subnet, especially since the documented export with no_root_squash represents a potential exposure.

3.1.3 Trusted Internal Network

A Trusted Internal Network is a well maintained, trusted and probably small internal network where network traffic is unrestricted to and from the cluster subnet. While this configuration provides advantages of access to a company's infrastructure, services and storage, it also introduces a few threats. Most of these are not malicious, and normal internal safeguards such as an Intrusion Detection Systems, anti-virus software and active network monitoring typically address rogue employees or bot infestations that may attempt to disrupt the network. Human error via network and new admin mistakes are the most common causes of cluster service outages.

Guidelines for the trusted internal network model:

Implement all the guidelines for isolated local networks.
Disable ssh root logins. Admins should log in as themselves (using their global uid) with user privileges. Set up sudo to allow specific admin commands per user or root, if needed. See Section 3.2, “Administrator Privileges in Oracle VM” for details.
If shared storage is mounted from outside the server pool subnet, restrict the export to the Oracle VM Server dom0 and VM IP addresses. Do not make NFS shares and LUNs on Fibre Channel or SAN storage globally accessible.
Establish appropriate network firewalls as discussed below.

A default Oracle Linux install has the iptables or firewalld firewall enabled. However, a best practice for network filtering is to use an additional external firewall device. This permits separation of control (network administrator control of network access, use of globally administered and audited rules), and relieves the Oracle VM dom0 of the potentially high CPU load that can be created by implementing packet filter rules. In order to use Oracle VM Manager on a system with iptables or firewalld enabled you can either open the ports used by Oracle VM Manager, or open all ports by disabling iptables or firewalld. The latter is not advised in the absence of an external firewall devices because it removes network protection.

Be sure to open the necessary ports on the different firewalls that may be installed between different parts of your network. Additional information including a listing of required ports and a diagram showing how components connect to each other can also be found in Section 2.1, “Oracle VM Pre-Installation Tasks”.

Ensure that your external firewall device filters or blocks ICMP Timestamp requests from arbitrary hosts. It is possible to send ICMP Timestamp (Type 13) requests to obtain system information or perform various attacks.

If there is no external firewall device, for example, when using iptables, ensure that the iptables service is running on each node and that at least the default policy and ruleset are present in the file /etc/sysconfig/iptables:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1:148]
-A INPUT -p tcp -m state --state NEW -m tcp --dport 10000 -j ACCEPT 
-A INPUT -p udp -m state --state NEW -m udp --dport 123 -j ACCEPT 
-A INPUT -p tcp -m state --state NEW -m tcp --dport 7002 -j ACCEPT 
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 
-A INPUT -p icmp -j ACCEPT 
-A INPUT -i lo -j ACCEPT 
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT 
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited
-A INPUT -p icmp --icmp-type timestamp-request -j DROP
-A INPUT -p icmp --icmp-type timestamp-reply -j DROP
COMMIT

3.1.4 Untrusted Internal Network

An untrusted internal network has the characteristics of being loosely maintained, unmonitored, readily compromised from the outside or from ongoing and uncorrected malware infested workstations on the inside.

In this model, the dom0 admin control and VMs have been partitioned into separate VLANs where traffic flow is controlled by a 3 zone router/firewall. The firewall policy is to allow the admin network to make outbound connections to anywhere, but blocks inbound connections from the untrusted internal network. Firewall policies to the VM network would depend on the application but should only expose service ports to the internal network that are needed. A signature driven Intrusion Detection System is an option on the VM network to monitor for traffic patterns that indicate an attack is underway.

This model views any traffic from the internal network as potentially hostile. Additional hardening of this network can be done by implementing hardware-based or software-based firewalls and policies on the admin and dom0 hosts that block inbound traffic. The dom0 firewall rules can also be enhanced to reject peer dom0 traffic on all ports except OCFS2 (7777), Oracle VM Agent (8899), and the Xen administration ports (8002 and 8003).

Guidelines for the untrusted internal network model:

Implement all the guidelines for trusted internal networks.
Disable ssh root logins. Admins should log in as themselves (using their global uid) with user privileges. Set up sudo to allow specific admin commands per user or root, if needed. See Section 3.2, “Administrator Privileges in Oracle VM” for details.

If using iptables, add failed connection logging to the existing iptables firewall just prior to the last REJECT line:

-A RH-Firewall-1-INPUT -m limit --limit 15/minute -j LOG --log-prefix "FW Drop:"
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited

Set up a remote syslog server to track all user logins and firewall connection failures.
Disable the VNC connections on each Oracle VM Server dom0. Port-forward VNC connections via ssh and vncviewer rather than Oracle VM Manager:
```
#-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 5900:5950 -j ACCEPT
ssh -L 5900:vmserverhost:5900 vmserverhost
```

Disable port 8888 and IPP ports on Oracle VM Manager:

#-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
#-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 631 -j ACCEPT
#-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8888 -j ACCEPT

Note

This will leave secure ports 22, 7002 and 10000 for admin command line access and Oracle VM Manager connections.

Define a trusted admin network or host and limit Oracle VM Manager and Oracle VM Server ssh connections to that network or host. To implement this in iptables, comment out the existing ssh rule in the default /etc/sysconfig/iptables configuration file. Replace it with the information applicable to your trusted network or single admin host; for example the 192.168.0.0/24 network or the host with IP address 192.168.0.67:
```
#-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -s 192.168.0.0/24 --dport 22 -j ACCEPT
-- or -- 
-A RH-Firewall-1-INPUT -p tcp -s 192.168.0.67 --dport 22 -j ACCEPT
```
Note

Connections that fail will fall through and be logged by the logging rule provided in the third bullet. When using iptables, all the rules in /etc/sysconfig/iptables can also be changed to restrict access to selected networks or hosts.

3.1.5 Internet Facing Services

The most challenging model for securing a network connected host is placing VM based services directly on the Internet. In this case, great care must be taken to insure that the environment won't be compromised through a wide variety of penetration techniques and exploits from a large, diverse and aggressive set of threats.

Even on an untrusted internal network without intrusion detection, internal users are known quantities and typically aware that they are being monitored, so they generally do not tend to be abusive or even overly curious. In contrast, the Internet provides virtual anonymity for malicious individuals, criminal gangs and hostile, well equipped governments. With an anonymous cloak, probing and attacking high profile network segments with automated tools that run 24/7 is a standard procedure. On perimeter firewalls, aggressive penetration attempts on the ssh, mysql, oracle, netbios and smtp service ports can typically be measured in "attacks per second".

There are a variety of topologies to provide zone isolation that can help to repel attacks and mitigate the effects of a successful intrusion event. The following describes some basic approaches.

Guidelines for network models involving Internet facing services:

Implement all the guidelines for untrusted internal networks.
All Internet facing hosts and VMs reside on an isolated network stub called a DMZ (for demilitarized zone). Connections into and out of the network are managed and monitored by a stateful firewall that enforces well defined rules and policies. If possible, an individual DMZ VLAN per VM is optimal for guest VM isolation.
Never place Oracle VM Manager or any other mission critical administration tool on the Internet. Use a VPN to access these hosts on secure internal admin networks. This in particular applies to networked storage: these hosts should not be on the Internet-facing network.
Place each dom0 (Oracle VM Server) that is hosting Internet-exposed VMs in an administrative DMZ. This zone cannot initiate connections to the internal network or the Internet, but is reachable from an administrator VLAN. The dom0 itself should not the reachable from the Internet or the internal network.
Use selinux on Internet-exposed Linux VM guests whenever possible.

3.2 Administrator Privileges in Oracle VM

Each administrator should have an individual user account in order to access Oracle VM Servers and Oracle VM Manager.

Note

You can create unique user accounts for Oracle VM Manager with the Administrator Tool (ovm_admin). For more information, see Oracle VM Manager Administrator Tool (ovm_admin) in the Oracle VM Administrator's Guide.

The Oracle VM dom0 is a highly privileged environment. For maximum security, administrative access to it must be strictly controlled, limited to authorized individuals, and logged. It must be stressed that the recommended method for most Oracle VM management is through the graphical user interfaces provided by Oracle VM Manager, the Oracle VM Manager Command Line Interface, or Oracle Enterprise Manager Ops Center, rather than logging onto individual servers, except in specific situations where command line access to Oracle VM Servers is needed or recommended by Oracle Support Services.

That said, customers may choose to deploy "normal" Linux administrative controls and remain supported. A specific valid example includes modifying /etc/ssh/sshd_config and /etc/sudoers to prevent SSH root login and requiring administrators to login as themselves and use sudo to gain privileged access. Another valid example is modifying /etc/login.defs to control password length and expiration policies. Adding device drivers or rpms would be examples of changes that could harm supportability and proper function, and should only be done in consultation with Oracle VM Support. Customers are encouraged to carefully review their access and security controls for suitability in the Oracle VM environment.

End users of virtual machines in the Oracle VM environment should not be granted administrative rights. They should rather access their virtual machines directly via SSH, RDP or VNC. For large scale environments, instead apply role based access control via Oracle Enterprise Manager, as explained in Section 3.4, “User Access to Virtual Machines”.

3.3 Storage Configuration

The storage providers in an Oracle VM environment must also be configured in a way that exposes them only to the servers and virtual machines that make use of them. Access to the management functionality of each storage provider must be restricted to the administrators in charge of storage configuration.

First of all, connect the storage servers to a private network that is accessible from the Oracle VM Manager and Oracle VM Servers. The storage providers need not be reachable from outside the Oracle VM environment. This network can be the management network or, preferably, a separate storage network. Locking down the storage servers to the individual IP addresses of the Oracle VM servers (including the Manager) in the storage subnet, is the most restrictive and most secure way to provide access to storage. As a minimum, expose the storage only to the storage subnet.

In Oracle VM we distinguish between file servers and SANs. For both categories the recommendations above apply, because most non-local storage is both managed and provisioned over the network, meaning based on IP addresses. The exception is directly attached storage, such as Fibre Channel or InfiniBand: to prevent unauthorized access you must make sure that only the Host Bus Adapters (HBAs) of the required servers are physically connected to the Fibre Channel or InfiniBand switch. NFS-based file servers and iSCSI-based SAN servers can be restricted to a subset of IP addresses via configuration.

The management of the Oracle VM storage servers may be different depending on the Oracle VM Storage Connect plug-in used for interaction with the storage provider. If you are using generic NFS or iSCSI providers with the corresponding generic Oracle VM Storage Connect plug-in, then configuration occurs almost entirely on the storage host. If you are using a custom third-party Oracle VM Storage Connect plug-in, then you can perform a much larger portion of the storage configuration from within Oracle VM Manager. Regardless of whether you use generic or non-generic iSCSI storage, make sure that your targets, initiators and access groups are configured in the most restrictive way possible.

For information about how to configure access to your storage providers, consult the following topics in the Oracle VM documentation:

Overall storage management: refer to Storage Tab in the Oracle VM Manager User's Guide.
Storage access configuration through access groups: refer to Access Groups Perspective in the Oracle VM Manager User's Guide.
Access group and storage initiator management on Oracle VM Servers: refer to Storage Initiators Perspective in the Oracle VM Manager User's Guide.

For a technical overview of Oracle VM Storage Connect, see the Oracle VM Storage Connect technical paper at:

http://www.oracle.com/us/technologies/virtualization/ovm3-storage-connect-459309.pdf.

For specific details about the configuration of your storage hardware, consult the hardware manufacturer's documentation.

3.4 User Access to Virtual Machines

By itself, the Oracle VM Manager GUI is an administrator tool. The administrator accounts have full access to all the functionality and all resources managed through Oracle VM Manager. Therefore it is highly recommended that only a few accounts be handed out to the people who are responsible for configuration and day-to-day management of the environment. Administrators must also have access to the guest operating systems of the virtual machines, and for that they use the VM console from within the Oracle VM Manager GUI. However, not every user with virtual machine access needs to be an Oracle VM administrator. This would violate the security principle of least privilege. (See Section 1.2.3, “Follow the Principle of Least Privilege”.) Depending on your particular deployment of Oracle VM, you may want to grant virtual machine access in a different way.

We distinguish between three methods of virtual machine access control:

Oracle VM Manager console.
Direct remote connectivity.
Role-based access control with Oracle Enterprise Manager.

VM Console Access

An Oracle VM administrator can always access the guest operating system of a virtual machine via the console in Oracle VM Manager. This is the standard method to connect to a virtual machine hosted in an Oracle VM environment. If your virtual machines are servers hosting applications and services, for example, then it is likely that they are configured and maintained by system administrators. In this type of setup, end users interact with the service or application running on the server, but never log on to the virtual machine itself.

For this model it makes sense that only one or a handful of system administrators can access the virtual machine via the Oracle VM Manager console. From a security standpoint, a small number of administrator accounts is very manageable, while the Oracle VM resources remain hidden and protected from all other users.

Direct VM Access

If certain users need administrative access to virtual machines, but are not administrators of the Oracle VM environment, we recommend that you do not create additional administrator accounts for Oracle VM Manager. Instead, an Oracle VM administrator should set up the virtual machine and configure remote connectivity so that the virtual machine user can establish a connection without having to go through Oracle VM Manager. To establish direct VM access, follow the same principles and procedures as with a physical server:

Install and configure the appropriate operating system on the VM. Install any mandatory additional software as well.
Create the necessary user accounts and set the required privileges.
Connect the VM to a network that is accessible to the VM user(s), but make sure that the management network and other networks essential for your Oracle VM environment remain protected. Assign a static IP address to the VM to facilitate remote connectivity. Never use a public IP address for administrative access; instead, use a private IP address in the internal network and force users to set up a VPN connection to the internal network first.
Configure remote connectivity on the VM. For a Windows server, RDP can be used; for a Unix server you can use SSH for command line access, and VNC in case a graphical desktop environment is used.
Provide login credentials, VM IP address (or DNS name) and remote access port number to the users who require remote access.

Caution

Always apply the principle of least privilege: strictly enable only the functionality that users require.

Role Based Access

Large-scale deployments of Oracle VM have different requirements when it comes to user management and access control. A combination of the two access methods described above becomes unmanageable as the number of virtual machines increases, and different categories of users need different levels of access to groups of virtualized resources. If your environment is that large and complex, you need facilities such as role-based access control and directory service integration. Oracle VM Manager cannot provide this functionality, but if you need it, you can integrate your Oracle VM environment with Oracle Enterprise Manager.

The integration with Oracle Enterprise Manager adds a number of significant management features to Oracle VM, such as:

Role-based access control: user groups and permission profiles.
LDAP/directory service integration.
resource assignment and ownership management.
Separation, isolation and protection of resource groups (VMs, networks, storage, etc.).
Profiles and deployment plans to create multiple VMs at once and to provision operating systems and software applications.

For a quick overview of role based access control with Oracle Enterprise Manager, see this post on Oracle's Virtualization Blog: https://blogs.oracle.com/virtualization/entry/crash_course_role_based_access.

For detailed information about integrating Oracle VM with Oracle Enterprise Manager, see the Oracle Enterprise Manager documentation at: http://www.oracle.com/technetwork/oem/grid-control/documentation/oem-091904.html

3.5 Virtual Machine Security Considerations

In many ways, virtual servers have security requirements identical to those of physical servers. The same applies to the applications and services they host. Virtualization provides security benefits: each virtual machine has a private security context, potentially with separate authentication and authorization rules, and with separate process, name and file system spaces. Deploying applications onto separate virtual machines provides better security control compared to running multiple applications on the same host operating system: penetrating one virtual machine's OS doesn't necessarily compromise workload and data residing in other virtual machines. Nonetheless, some practices should be kept in mind to prevent virtualization from introducing security vulnerabilities.

One aspect is physical security. Virtual infrastructure is not as 'visible' as physical infrastructure: there is no sticky label on a virtual machine to indicate its purpose and security classification. If a data center identifies servers with extremely high security requirements, and physically isolates them in a locked room or cage to prevent tampering or theft of data, then the physical machines hosting their virtualized workloads should be isolated in a similar way. Even without secured areas, many institutions keep workloads of different security classes on different servers. Those same isolation rules apply for virtual machines. Care should be taken to ensure that the protected virtual machines are not migrated to a server in a less secure location. In the context of Oracle VM, this implies maintaining separate server pools, each with their own group of servers.

These rules of isolation should also be applied to networking: there are no color coded network cables to help staff identify and isolate different routes, segments and types network traffic to and from virtual machines or between them. There are no visual indicators that help ensure that application, management, and backup traffic are kept separate. Rather than plug network cables into different physical interfaces and switches, the Oracle VM administrator must ensure that the virtual network interfaces are connected to separate virtual networks. Specifically, use VLANs to isolate virtual machines from one another, and assign virtual networks for virtual machine traffic to different physical interfaces from those used for management, storage or backup. These can all be controlled from the Oracle VM Manager user interface. Ensure that secure live migration is selected to guarantee that virtual machine memory data is not sent across the wire unencrypted.

Additional care must be given to virtual machine disk images. In most cases the virtual disks are made available over the network for migration and failover purposes. In many cases they are files, which could easily be copied and stolen if the security of network storage is compromised. Therefore it is essential to lock down the NAS or SAN environments and prevent unauthorized access. An intruder with root access to a workstation on the storage network could mount storage assets and copy or alter their contents. Use a separate network for transmission between the storage servers and the Oracle VM hosts to ensure its traffic is not made public and subject to being snooped. Make sure that unauthorized individuals are not permitted to log into the Oracle VM Servers, as that would give them access to the guests' virtual disk images, and potentially much more.

All of these steps require controlling access to the Oracle VM Manager and Oracle VM Server domain 0 instances. Network access to these hosts should be on a private network, and the user accounts able to log into any of the servers in the Oracle VM environment should be rigorously controlled, and limited to the smallest possible number of individuals.

Chapter 3 Oracle VM Security Features

3.1 Oracle VM Network Model

3.1.1 No Network Connection

3.1.2 Isolated Local Network

3.1.3 Trusted Internal Network

3.1.4 Untrusted Internal Network

3.1.5 Internet Facing Services

3.2 Administrator Privileges in Oracle VM

3.3 Storage Configuration

3.4 User Access to Virtual Machines

3.5 Virtual Machine Security Considerations