1.1 Oracle VM Overview

Oracle VM is a platform that provides a fully equipped environment with all the latest benefits of virtualization technology. Oracle VM enables you to deploy operating systems and application software within a supported virtualization environment.

1.1.1 What is the Oracle VM Architecture?

Oracle VM is a platform that provides a fully equipped environment with all the latest benefits of virtualization technology. Oracle VM enables you to deploy operating systems and application software within a supported virtualization environment. Oracle VM insulates users and administrators from the underlying virtualization technology and allows daily operations to be conducted using goal-oriented GUI interfaces. The components of Oracle VM are shown in Figure 1.1, “Oracle VM Architecture”.

Figure 1.1 Oracle VM Architecture


  • Client Applications: Various user interfaces to Oracle VM Manager are provided, either via the graphical user interface (GUI) accessible using a web-browser; the command line interface (CLI) accessible using an SSH client; or external applications, such as Oracle Enterprise Manager, custom built applications or scripts that use the Web Services API (WS-API). All communications with Oracle VM Manager are secured using either a key or certificate based technology.

  • Oracle VM Manager: Used to manage Oracle VM Servers, virtual machines, and resources. It is comprised of a number of subcomponents, including a web browser-based user interface; and a command line interface (CLI) allowing you to manage your infrastructure directly from the command line either via external scripts or by running manual command sequences. Each of these interfaces run as separate applications and interface with the Oracle VM Manager core application using the Web Services API.

    Oracle VM Manager is usually hosted on a standalone computer but it can also be run as a virtual machine in a carefully designed environment. However, support for running Oracle VM Manager on a virtual machine is limited and caveats apply. For more information, see Running Oracle VM Manager as a Virtual Machine in the Oracle VM Installation and Upgrade Guide.

    The Oracle VM Manager core application is an Oracle WebLogic application that runs on Oracle Linux. The user interface uses the Application Development Framework (ADF) application, providing a common look and feel, in line with other Oracle web-based applications. While the Oracle VM Manager core application and the Oracle VM Manager Web Interface are both WebLogic applications, they are separate applications, even though they share the same process space.

    While the Oracle VM Manager Web Interface and Oracle VM Manager Command Line Interface both use the Web Services API to interface with the Oracle VM Manager core application, the Oracle VM Manager Web Interface can query the Oracle VM Manager database directly for read-only operations. This design decision allows the Oracle VM Manager Web Interface to provide a wider range of filtering options and improves performance for some operations.

    Oracle VM Manager communicates with each Oracle VM Server via the Oracle VM Agent, using XML-RPC over HTTPS on port 8899. Actions on servers that are initiated within Oracle VM Manager are triggered using this method. The Oracle VM Agent on each Oracle VM Server is equally able to send notifications, statistics and event information back to Oracle VM Manager. Actions within Oracle VM Manager triggered by Oracle VM Agent are achieved using the Web Services API exposed by Oracle VM Manager and are secured using HTTPS.

    While Oracle VM Manager is a critical component for configuration actions within the Oracle VM infrastructure, the virtualized environment can continue to function properly even if Oracle VM Manager experiences downtime. This includes the ability to maintain high availability and to perform live migration of virtual machines running on an x86 platform. Note that live migration of virtual machines running on a SPARC platform, does require that the Oracle VM Manager is running to succeed, since the only supported method of performing a live migration on this platform is either through the Oracle VM Manager Web Interface or the Oracle VM Manager Command Line Interface.

  • Oracle VM Manager Database: Used by the Oracle VM Manager core application to store and track configuration, status changes and events. Oracle VM Manager uses a MySQL Enterprise database that is bundled in the installer and which runs on the same host where Oracle VM Manager is installed. The database is configured for the exclusive use of Oracle VM Manager and must not be used by any other applications. The database is automatically backed up on a regular schedule, and facilities are provided to perform manual backups as well.

  • Oracle VM Server: A managed virtualization environment providing a lightweight, secure, server platform which runs virtual machines, also known as domains. At least one Oracle VM Server is required, but several are needed to take advantage of clustering.

    Oracle VM Server is installed on a bare metal computer, and contains the Oracle VM Agent to manage communication with Oracle VM Manager. dom0 is an abbreviation for domain zero, the management or control domain with privileged access to the hardware and device drivers. DomU is an unprivileged domain with no direct access to the hardware or device drivers. A user-domain (domU) is started and managed on an Oracle VM Server by dom0.

    On x86-based systems, Oracle VM Server is based upon an updated version of the underlying Xen hypervisor technology, and includes Oracle VM Agent. It also includes a Linux kernel with support for a broad array of devices and file systems. The Linux kernel is run as dom0 to manage one or more domU virtual machines, each of which could be Linux, Oracle Solaris, or Microsoft Windows™.

    In contrast, Oracle VM Server for SPARC takes advantage of the hypervisor that is already included within the SPARC firmware, alongside the Oracle VM Agent for SPARC. The default Oracle Solaris operating system is usually promoted to act as the primary domain, which is equivalent to dom0 on x86 systems. Once the primary domain is in place, it can be used to create and manage further domains running different versions of the Oracle Solaris operating system.

    Groups of Oracle VM Servers are usually clustered together to create server pools. This allows Oracle VM Manager to handle load balancing and failover for high-availability environments. Virtual machines run within a server pool and can be easily moved between the different servers that make up a server pool. Server pools also provide logical separation of servers and virtual machines. Server pools are required entities within the Oracle VM infrastructure, even if they consist of only one server.

    Each Oracle VM Server maintains its own Berkeley Database, used to store local configuration and runtime information. This allows the Oracle VM Server to continue to function normally, even if Oracle VM Manager becomes unavailable for a period. Where Oracle VM Servers are clustered together, a separate cluster database, stored in the server pool file system, is shared between the servers. This allows the server pool to continue to provide clustering features, such as High Availability, even if Oracle VM Manager is unavailable.

  • External Shared Storage: Provides storage for a variety of purposes and is required to enable high-availability options afforded through clustering. Storage discovery and management is achieved using the Oracle VM Manager, which then interacts with Oracle VM Servers via the storage connect framework to then interact with storage components. This process is discussed in more detail in How do Different Storage Types Connect?. Oracle VM provides support for a variety of external storage types including NFS, iSCSI and Fibre Channel.

1.1.2 Security Aspects of Oracle VM

The Oracle VM security architecture, by design, eliminates many security threats. The guidelines for secure deployment of virtualized solutions based on Oracle VM are largely based on network security. As these guidelines are generally applicable, they should always be reviewed for applicability in the context of each implementation and the security requirements and policies of the broader environment in which Oracle VM is deployed.

The following list describes the main aspects of the Oracle VM security architecture:

  • Both Oracle VM Server and Oracle VM Manager provide an Oracle Linux environment that includes an iptables firewall with a default ruleset and policies.

  • Oracle VM Server is a minimalist OS implementation derived from  Oracle Linux and uses the Unbreakable Enterprise Kernel (UEK) Release 4 for enhanced performance and scale. By design, it has few moving parts and a minimum of network exposed services to reduce administrative effort, overhead, and attack surface.

  • Oracle VM Manager 3.3.1 environments, and above, may only use the bundled MySQL database which is installed locally on the same host where Oracle VM Manager is installed. Access is restricted to localhost connections and is not remotely accessible. Furthermore, backup processes are automated to assist with recovery in the case of failure. The MySQL database may not be used for any other application outside of Oracle VM Manager.

  • Default installations of Oracle VM Server or Oracle VM Manager do not provide physical security. They can be booted (using runlevel 1 or a rescue cd) and compromised by anyone with access to the physical console. Suitable physical security should be provided to prevent this type of exposure.

  • TLS is used extensively to secure and authenticate communications between Oracle VM Manager and Oracle VM Servers; to secure and authenticate access to the Oracle VM Manager Web Services API; to secure all HTTPS communications and within the network component of a VM migration.

  • The Oracle VM Servers' administrative connection to Oracle VM Manager uses HTTPS by default.

  • Openssh along with public/private key authentication are fully supported on Oracle VM Server.

  • 802.1q VLANS are fully supported for segregating VM and dom0 network traffic.

All components of the Oracle VM installation communicate with each other in a secure way. The following table shows, in detail, how each individual line of communication is set up securely:

Communication

Description

Browser to Oracle VM Manager GUI

When you log on to Oracle VM Manager, we strongly recommend that you use HTTPS and connect to TCP port 7002, since the user interface expects you to authenticate using a username and password, which must be protected. SSL encrypted communication is available as of version 3.1.1, and regular HTTP connectivity at TCP/7001 is disabled by default. However, it may be enabled via Oracle WebLogic Server for testing and demo purposes.

In Oracle VM 3.3 and above, the SSL certificate that is used for communication encryption is generated by default within Oracle VM Manager and is signed by an internal CA (Certificate Authority) certificate within Oracle VM Manager. Tools are available to replace this certificate with one signed by a trusted third-party CA, if required, or alternately to obtain the internal CA certificate to add it to your own trusted CA certificates within your web-browser or application keystore. This CA certificate can be used to validate the SSL certificate presented when you connect to the HTTPS port for Oracle VM Manager. More information on the tools provided to manage these certificates is provided in Oracle VM Administrator's Guide.

As of Oracle VM Release 3.4.5, TLS version 1.2 (TLSv1.2) is the default within Oracle VM Manager.

Oracle VM Manager GUI to Oracle VM Core

The Oracle VM Manager application uses the underlying Web-Services API to communicate with Oracle VM Core, running on the same server. The Web-Services API is exposed on the same HTTPS port as the Oracle VM Manager application, since it is served out of the same process space within Oracle WebLogic Server. All communication is secured using the same SSL certificate that is used to encrypt communications between the Oracle VM Manager GUI and a web-browser. Authentication of the Oracle VM Manager application to Oracle VM Core is achieved using the username and password of the user that authenticated against the user interface front-end.

When the Oracle VM Manager application is started, it makes a connection to the Oracle VM Core Web-Services API to populate the GUI model and to periodically poll for events to keep the GUI model up-to-date. Authentication of the Oracle VM Manager application to Oracle VM Core for this purpose is achieved using an SSL certificate. This certificate is generated during the installation of Oracle VM Manager. The certificate is signed and registered by the internal CA, which is used by Oracle VM Core to validate and authenticate connections from the Oracle VM Manager application. The public certificate for this certificate-key pair is stored in a Java truststore available to the Oracle VM Core. The private key for this certificate is stored within a secure Java keystore within the Oracle VM Manager application.

Web Services to Oracle VM Core

Oracle VM Core offers a web services API (WSAPI) that provides a REST endpoint exposed via HTTPS available on TCP port 7002. Communications are encrypted using the same SSL certificate that is used to encrypt communications between the Oracle VM Manager GUI and a web-browser. It is possible to register and sign a certificate to perform certificate based authentication with Oracle VM Core using the keytool management application described in Setting Up SSL in the Oracle VM Administrator's Guide, or programmatically using the methods provided by the WSAPI itself, as discussed in Certificate Management for Certificate-based Authentication Using REST in the Oracle VM Web Services API Developer's Guide. When creating a certificate to use for authentication purposes, it is highly advisable that the certificate key is generated with an adequate passphrase, and that the certificate and key are stored either in a passhprase protected keystore, or that they are stored in a protected area on the file system with access permissions limited to the user that intends to use them.

Client to CLI

The Oracle VM Manager Command Line Interface (CLI) is officially supported as of version 3.2.1. The client connects to the CLI, which runs on the Oracle VM Manager host, using SSH over port TCP/10000. A public key can be set up in the SSH server in order to allow CLI users to log on automatically without having to enter credentials each time. If this approach is used, it is recommended that a passphrase is still set for the private key and that an SSH Agent is used to handle the authentication of the key for repeated requests. The private key must be stored in a secure location on the file system with access permissions limited to the user that intends to use it.

As of version 3.4.1, the CLI also supports the option to obscure sensitive information on screen as it is entered into the CLI. This facility is described in more detail in Masking Sensitive Data in the Oracle VM Manager Command Line Interface User's Guide. Users must ensure that this facility is used when entering sensitive data into the CLI, such as when sending passwords using the VM messaging channel.

CLI to Oracle VM Core

The Oracle VM Manager Command Line Interface (CLI) communicates with Oracle VM Core via the Web-Services API on TCP/7002 using HTTPS. Communications are encrypted using the same SSL certificate that is used to encrypt communications between the Oracle VM Manager GUI and a web-browser. Authentication of the CLI to Oracle VM Core is achieved using the username and password of the user that authenticated against the user interface front-end.

Oracle VM Agent to Oracle VM Core

The Oracle VM Agents running on the Oracle VM Servers use SSL/TLS encryption to communicate with Oracle VM Core via TCP/7002 (HTTPS) through the WSAPI. Authentication is achieved using an SSL certificate registered for the Oracle VM Agent when ownership is taken. The API access is limited to the type of communications that the Agent has with Oracle VM Core, such as the notification of events and the provision of statistical information.

Oracle VM Core to Oracle VM Agent

Oracle VM Core, in turn, uses TCP/8899 to communicate with the Oracle VM Agents in the environment. The protocol is also HTTPS. Oracle VM Core initially authenticates itself to the Oracle VM Agent using a username and password combination during the process of taking ownership of the Oracle VM Server. Once the Oracle VM Core has been authenticated by the Oracle VM Agent, an additional SSL certificate is exchanged so that the Oracle VM Agent can perform future authentication of the Oracle VM Core using an SSL certificate-key pair, and vice versa.

VNC and Serial Console Access

Oracle VM Manager opens a secure SSL-encrypted connection to the VNC server that is created by the Xen hypervisor for each remote virtual machine running on an Oracle VM Server. These are accessed on the Oracle VM Server on TCP ports 6900 and up. Connections from client web-browsers take advantage of the noVNC console that is provided within the Oracle VM Manager web user interface. This connection uses the same TCP 7002 port as used to access the web user interface, and this is secured using the same SSL certificate.

For serial console connections, Oracle VM Manager opens a secure SSL-encrypted connection to the serial terminal exported for each remote virtual machine running on an Oracle VM Server. These are accessed on the Oracle VM Server on TCP ports 10000 and up. Connections from client web-browsers take advantage of the jsTerm terminal emulator that is provided within the Oracle VM Manager web user interface. This connection uses the same TCP 7002 port as used to access the web user interface, and this is secured using the same SSL certificate.

Live Migration

Traffic related to live migration of virtual machines uses separate ports: TCP/8002 for non-encrypted and TCP/8003 for SSL-encrypted (TCPS) live migration. Secure live migration is a setting the user needs to switch on in the server pool properties as required. Based on this setting, Oracle VM Manager initiates SSL or non-SSL migration of the running virtual machine. For optimized security and performance, consider further network segregation by creating a separate network for live migration.

Oracle VM Agent Certificate

At installation, the Oracle VM Agent generates an SSL key and matching certificate. The properties are:

  • key algorithm: RSA.

  • private key size: 2048 bits.

  • certificate data management: according to X.509 standard.

  • location of the SSL key and certificate: /etc/ovs-agent/cert.

By default, VNC traffic, virtual machine migration traffic and Oracle VM Agent communications are all secured using the same SSL key and certificate. The administrator can regenerate the key/certificate combination via the Oracle VM Server command line by means of this command: ovs-agent-keygen. It is technically possible to use separate SSL keys and certificates for Oracle VM Agent communications and for secure virtual machine migration. Note that changing the SSL key and certificate combination on a server, requires that the server is released from ownership by Oracle VM Manager and that you will need to take ownership of the server again after the operation is completed.

During the process where an Oracle VM Manager instance takes ownership of the Oracle VM Server, the public certificate used by the Oracle VM Agent is exchanged with Oracle VM Manager and it is signed and registered with the Oracle VM Manager instance using an internal CA certificate. A new key is generated and provided to the Oracle VM Agent. This key and the signed certificate is used to authenticate and secure subsequent communications with Oracle VM Manager.

Other traffic

In an Oracle VM environment, the Oracle VM Manager host is frequently used as the reference to provide time synchronization. In this case, an inbound connection from all Oracle VM Servers to UDP port 123 is required for NTP traffic.

Oracle VM Servers in a clustered server pool use an OCFS2 pool file system and require a heartbeat network function to determine the status of each cluster member. The port used for this specific type of traffic is TCP/7777.

Some external applications may continue to use the legacy API, which is available on TCP/54322 and is secured using SSL/TLS. This API is deprecated, and applications currently making use of it must be upgraded in the future. If you are not using any other applications outside of Oracle VM itself, to perform management within your environment, you should ensure that access to this port is disabled in your firewall rules.

Note

As of Oracle VM Release 3.4.5, all SSL/TLS encrypted communications use the TLSv1.2 protocol for messages that are sent and accepted. SSLv3 and TLSv1 are disabled by default within Oracle VM Manager for security reasons.

1.1.3 Security Considerations for Oracle VM

In this section, we explore some important security considerations that you must be aware of when using Oracle VM.

Limitation of Access To The Oracle VM Manager Host

It cannot be stressed enough that the Oracle VM Manager Core is a very powerful component within an Oracle VM deployment, providing control over many servers and virtual machines. With this in mind, access to this host should be severely restricted. User accounts should be limited to administrators who require access to manage a deployment. Furthermore, firewall rules should be hardened to limit access to networks where administrators may be located.

Users with access to utilities or tools that communicate with the Oracle VM Manager Core in any way, should be aware that the credentials that they use to authenticate, whether by username and password or by SSL certificate, are highly sensitive and that a compromise of this nature does not only threaten Oracle VM Manager itself, but also makes every server and virtual machine within the deployment vulnerable to attack.

Appropriate security practice with regard to identity and credential management should always be observed.

See Section 1.2, “General Oracle VM Security Principles” for more guidelines on system protection.

Encrypted Communication

Many communications with Oracle VM Manager over HTTPS are encrypted using an SSL certificate to protect the communication of sensitive information. By default, this certificate is signed by an internal CA certificate that is generated for the Oracle VM Manager instance during installation. Since most client applications, including user web browsers, will not have this CA certificate installed it is not possible for many client applications to validate the SSL certificate presented when accessing an Oracle VM Manager service over HTTPS. Some client applications may fail entirely if an SSL certificate cannot be validated, while other applications may simply issue a warning and allow users to proceed at their own risk. To minimize the likelihood of a man-in-the-middle attack, it is important that validation can actually take place. Therefore, it is appropriate to take one of the following courses of action:

  • Install the CA certificate for the Oracle VM Manager instance into the trusted CA certificate store for all client applications that will have access to Oracle VM Manager.

  • Change the SSL certificate used for HTTPS communications by Oracle VM Manager to use a certificate that is signed by a trusted third-party CA, for which all of your client applications already have the CA certificate installed.

These operations are discussed in detail in Setting Up SSL in the Oracle VM Administrator's Guide.

During initial Oracle VM Server discovery and key exchange, passwords are sent over the wire from the Oracle VM Manager to the Oracle VM Agent for authentication purposes. This communication is performed over SSL so should be secure. However, no certificate validation is performed against the Oracle VM Agent certificate at this point since the Oracle VM Agent certificate is unknown. Therefore, it is possible that this channel might be subject to a man-in-the-middle attack wherein a malicious entity impersonates the server so that the manager sends the password to this entity in an attempt to authenticate. Where possible the Oracle VM Manager system should be run within the same local area network as the Oracle VM Servers within your Oracle VM environment, and this network should have appropriate security controls to mitigate the risk that a malicious attacker is able to perform a man-in-the-middle attack. Additionally, it is worthwhile ensuring that the passwords for the Oracle VM Agent on each Oracle VM Server is unique. This means that in the unlikely event that a single server is compromised via a man-in-the-middle attack, access to all servers within the deployment is not immediately gained and the damage can be limited.

Certificate Based Authentication

It is a common misconception that use of certificate-based authentication automatically makes a system more secure than the use of passwords, but this is not necessarily true. Certificate-based authentication is susceptible to many of the same attacks as password authentication. In this section we will discuss some of the ways in which this system could be attacked so that the security level can be well understood. All protocols that are used conform to current best practices documented in the OSCS (Oracle Secure Coding Standard).

In general, certificate authentication is less susceptible to dictionary attacks as well as brute-force attacks over the secure communication channel. However, in order for an SSL certificate to be signed and registered by Oracle VM Manager, password-based authentication is still used within Oracle VM Manager. This is particularly relevant to the Oracle VM Agent, which must use password-based authentication to support initial discovery and take ownership requests, as well as to allow discovery by secondary (non-owning) managers. Equally, it cannot be assumed that every user is issued with an SSL certificate-key pair that can be configured for use within a web browser to authenticate requests to use the Oracle VM Manager Web Interface. As a result, the Oracle VM Manager Web Interface only supports password-based authentication. Therefore, there are still channels within Oracle VM that are open to dictionary and brute-force attacks. These risks could be minimized by restricting access to systems using a strict firewall policy based on the requirements for different systems that need access to one another. It is important to understand that password exchanges are always encrypted using SSL/TLS, and are never exposed in clear text. Furthermore, stored passwords are always encrypted on the systems that hold them.

Where certificate-based authentication is used, if the private key used on either end of the communication is compromised, the attacker could use that private key and corresponding certificate to log into the other entity. On Oracle VM Manager, keys are encrypted and stored within a JKS keystore file which is protected by both a key password and a keystore password. These passwords are long, randomly generated passwords which are stored in the Oracle CSF (Credential Store Framework). The length and random nature of these passwords should preclude dictionary or brute-force attacks, but if the CSF is compromised then the keys could be retrieved. This mechanism conforms to the current security guidelines outlined by the OSCS.

In the case of Oracle VM Agent, the keys and certificates are stored in a non-encrypted PEM format. These files are readable only by the root user, but if the root user account is compromised, a user could then use this information to send bogus notification information to the manager. Likewise, a malicious user with root access could replace the Oracle VM Manager certificate information on the Oracle VM Agent with their own certificate to allow themselves to be authenticated for Oracle VM Agent commands. However, if they have already compromised the root account on the server this is probably a moot point since they already have unrestricted access to the system.

In the case where you choose to use certificate-based authentication within custom-built applications that make use of the WSAPI, you must ensure that appropriate actions are taken to protect keys from misuse. This includes setting a reasonable passphrase for your keys and ensuring that the keys are preferably stored in an encrypted format. A malicious user with access to a key that is registered for authentication against Oracle VM Manager has full access to the entire Oracle VM environment including all virtual machines, all storage, all servers and Oracle VM Manager itself.

When certificates are registered for authentication with Oracle VM Manager, the authorized certificate that allows authentication to the manager to take place is stored in the Oracle VM Manager database. This certificate only contains public key information, so it cannot be used in and of itself to log into Oracle VM Manager. However, a user who can modify the Oracle VM Manager database could replace this with a certificate of their own. For this attack vector to work, the certificate would need to be signed by a trusted CA (such as the Oracle VM Manager CA). Therefore, the risk can be mitigated by proper administration of the WebLogic trust store. To access the Oracle VM Manager CA private key to create such a certificate, the CSF would have to be compromised as well.