5 Oracle VM Server for SPARC

This chapter includes the following sections:

5.1 Introduction to Oracle VM Server for SPARC

Oracle VM Server for SPARC technology enables server virtualization on SPARC platforms. You can create and manage multiple virtual machine instances simultaneously on a single SPARC machine. Each virtual machine, or guest, can run a different operating system.

Oracle VM Server for SPARC technology is virtualization of SPARC servers. This technology is part of a suite of methodologies for consolidation and resource management for SPARC Chip Multi Threading (CMT) systems. Using this technology, you can allocate the various resources of the system such as memory, CPU threads, and devices, into logical groupings and create multiple discrete systems. These discrete systems have their own operating system, resources, and identity within a single system. By careful architecture, an Oracle VM Server for SPARC environment can help you achieve greater resource usage, better scaling, and increased security and isolation.

When Oracle VM Server for SPARC software is installed, a domain called the control domain is created. From this control domain, you create virtual machines called logical domains that each run an independent OS. A logical domain is a virtual machine with resources, such as CPU threads, memory, I/O devices, and its own operating system. The control domain manages the logical domains. Each logical domain can be created, destroyed, reconfigured, and rebooted independently of other logical domains.

Logical domains that are created manually for an Oracle VM Server for SPARC provisioned through the UI are also automatically discovered and managed on the UI.

You can also provision the OS on the manually created logical domains using the OS provisioning profile and plans in Oracle Enterprise Manager Ops Center.

Note:

When the Enterprise Controller and Proxy Controller are installed on an Oracle Solaris 11 operating system, you can perform more tasks than with other operating systems. For example, the Enterprise Controller must be installed on Oracle Solaris 11 to provision Oracle Solaris 11, Oracle Solaris 11 zones, and newer versions of Oracle VM Server for SPARC. In some cases, the Proxy Controller must also be installed on Oracle Solaris 11. See the Related Resources for Oracle VM Server for SPARC section for more information on operating system requirements for Oracle Solaris 11 actions.

5.2 Domain Types and Representation on the UI

You can create different types of logical domains. Depending on how the physical resources are assigned to the logical domain, distinguishes a logical domain from another domain. The different types are:

  • Control Domain: The control domain is the first domain created when you install the Oracle VM Server for SPARC software. This is also called the primary domain and denoted as primary wherever applicable in the Oracle Enterprise Manager Ops Center UI. Only from the control domain, you can create logical domains.

  • I/O Domain: An I/O domain has direct access to a physical I/O device, such as the PCIe Controller. An I/O domain either uses the physical I/O devices to host its own applications or shares the physical I/O device with other domains in the form of virtual devices.

  • Root Domain: A root domain is also an I/O domain which has PCIe root complex assigned to it.

  • Guest Domain: A guest domain is a non-I/O domain that uses virtual devices, such as virtual disks and virtual network interfaces, provided by one or more service domains.

  • HA Guest Domain: A guest domain with redundant network and storage resources.

  • Service Domain: A service domain provides virtual device services to other domains. This means that the domain has physical I/O devices. In the Oracle Enterprise Manager Ops Center UI, the service domain list includes the primary domain, I/O domains, and root domains that can provide virtual device services. It is generally recommended to not run any applications in service domains.

When you create logical domains using Oracle Enterprise Manager Ops Center, ensure to select the appropriate domain subtype. The selection of the domain subtype defines the step for selection of PCIe Endpoints for the I/O domain and PCIe buses for the root domain.

In Oracle VM Server for SPARC, an I/O domain can be created either by assigning PCIe Endpoints or SR-IOV Virtual Functions.

In Oracle Enterprise Manager Ops Center, an I/O domain is defined as Physical I/O domain, and created by assigning PCIe Endpoints only.

The subtype for the I/O domains is defined as Physical IO Domain in the UI. The I/O domain in this documentation refers to the Physical I/O Domain hereafter.

In the Oracle Enterprise Manager Ops Center UI, for I/O domains, the center pane displays the I/O Resources tab. The I/O Resources tab displays the details of the PCIe Endpoints assigned to the domain.

For root domains, the I/O Resources tab displays the PCIe buses assigned to the domain and the corresponding Endpoint devices in the bus.

Refer to View I/O Resources section for more detailed information.

5.3 Roles for Oracle VM Server for SPARC

The following table lists the tasks and the role required to complete the task. Contact your administrator if you do not have the necessary role or privilege to complete a task.

See the Related Resources for Oracle VM Server for SPARC section for information about the different roles and the permissions they grant.

Table 5-1 Oracle VM Server for SPARC Tasks and Roles

Task Role

Provision and manage virtualization host

Virtualization admin

Create, manage, update, and delete guests

Virtualization admin

Create, manage, and delete I/O domains and root domains

Virtualization admin

Discover and manage virtualization hosts

Asset admin

Create and manage profiles and plans

Profile and plan admin

Create and manage IPMP groups

Network admin

Create and manage Link Aggregation

Network admin

Set monitor threshold

Asset admin

Create credentials

Security admin

5.4 Actions for Oracle VM Server for SPARC

Using Oracle Enterprise Manager Ops Center, you can perform the following tasks:

  • Provision Oracle VM Server for SPARC.

  • Manage Oracle VM Server, including, rebooting, shutting down, changing name servers and NFS4 domains.

  • Monitor the performance of Oracle VM Server.

  • Create and provision logical domains, that includes I/O domains, root domains and guest domains.

  • Manage logical domains, including editing, migrating, starting, rebooting, and shutting down domains.

  • Monitor the performance of logical domains.

  • Discover and manage existing Oracle VM Server for SPARC systems and logical domains.

  • Create server pools to maximize capacity and for automatic load balancing.

  • Migrate guest domains.

  • Manage automatic recovery of guest domains in a server pool.

Oracle Enterprise Manager Ops Center uses special Agent Controllers, called virtualization controller agents, or VC Agents, to manage Oracle VM Server systems and Oracle Solaris Zones. See Virtualization Agent Controllers for more information about VC Agents.

5.5 Location of Oracle VM Server for SPARC Information in the User Interface

Oracle VM Server for SPARC assets are visible from the All Assets section in the user interface (UI). The server hardware, control domain, the control domain OS, the logical domains and its OS are displayed as shown in this figure:

Figure 5-1 Oracle VM Server for SPARC System in the UI

Description of Figure 5-1 follows
Description of "Figure 5-1 Oracle VM Server for SPARC System in the UI"

This table lists where to find different information for Oracle VM Server for SPARC in the UI.

Table 5-2 Location of Oracle VM Server for SPARC Information in the UI

To See Location

Control Domain

Expand Assets in the Navigation pane. The Oracle VM Server for SPARC asset is represented by the SPARC icon which is the control domain.

Logical Domains

Expand Assets in the Navigation pane. Select the control domain and all the logical domains created in that server are listed.

Virtual Services

Expand Assets in the Navigation pane and select the control domain. The center pane displays the Virtual Services tab that displays the Virtual Disks, Virtual Network Switches, and Virtual Console Concentrators details.

I/O Resources

Expand Assets in the Navigation pane and select the control domain. The center pane displays the I/O Resources tab that displays the Buses/Endpoint Devices and SR-IOV Services.

Other options

Expand Assets in the Navigation pane and select the control domain. The Actions pane lists the different options available for managing Oracle VM Server for SPARC. The options are Associate Libraries, Attach Network, Reboot and Edit Attributes.

5.6 Discovering Existing Oracle VM Server for SPARC Environments

The following information is covered in this section:

You can use Oracle Enterprise Manager Ops Center to manage Oracle VM Server for SPARC systems created outside of Oracle Enterprise Manager Ops Center. There is no difference between the logical domains created using Oracle Enterprise Manager Ops Center and the domains created outside of Oracle Enterprise Manager Ops Center except for some limitations that are described in Limitations of Discovering an Existing Oracle VM Server for SPARC System with Logical Domains.

You can manage the following types of existing Oracle VM Server for SPARC servers:

  • Oracle Solaris 11 running Oracle VM Server for SPARC server. The control domain can have non-global zones.

  • Oracle Solaris 10 running Oracle VM Server for SPARC server. The control domain can have non-global zones.

To manage an existing Oracle VM Server for SPARC environment, use the Oracle Enterprise Manager Ops Center discovery feature to display the domains (control, root, I/O, and guest domains) in the user interface. The information about the I/O resources, and SR-IOV configurations appear in the UI.

Note:

If you manually configured an Oracle VM Server for SPARC environment that you managed with an earlier version of Oracle Enterprise Manager Ops Center, the Zones VC Agent is installed. See Virtualization Agent Controllers for information about VC Agents and Agent Management Options for how to change the Agent Controller to the Oracle VM Server VC Agent.

5.6.1 Supported Logical Domain Configurations

You can discover and agent manage an existing fully configured Oracle VM Server for SPARC system that has logical domains and associated network and storage resources that are configured on the control domain.

The logical domain configuration might be one of the following types with I/O resources or virtual resources:

  • A root domain with one or more PCIe root complexes

  • A physical I/O domain with one or more PCIe Endpoint devices

  • A guest domain with virtual resources

5.6.2 Limitations of Discovering an Existing Oracle VM Server for SPARC System with Logical Domains

When you discover and deploy an agent on an existing Oracle VM Server for SPARC system that has associated logical domains, the system and logical domains appear in the Assets section of the UI. Some actions are disabled when the agent is not able to manage a resource.

5.6.2.1 Disabled Migration Action

The following are some reasons why the migration management action is disabled:

  • Migration is disabled for all discovered guest domains because the metadata of the domain is in the default local library. To resolve, move the metadata to a shared library for all guest domains that you want to migrate. See Moving Metadata to Another Library for how to move the metadata.

  • Migration is disabled for all guest domains that use storage on local devices or local files. To resolve, use shared FC or iSCSI LUNs for the guest domain's storage. Or enable shared action for the local file system storage

Note:

You cannot migrate a guest domain until you move all of its virtual disks to shared storage and move the domain's metadata to a shared library. See Migration Requirements for more details.

5.6.2.2 Monitoring Information

The monitoring information might be incomplete due to missing managed resources. The following are examples of situations of missing resources:

  • The networking view of logical domains shows only the networks that you discover with the control domain Virtualization Agent Controller.

  • The networking view of a logical domain is incomplete when the domain is provisioned without an Agent Controller running on it.

  • For a control domain with no guests and VLAN tagged networks, the Networks tab displays VLAN IDs but not a network name. The network name field is populated when a guest operating system is discovered or created using an IP address over the VLAN tagged network, or when you manually attach the network to the control domain with the Do Not Allocate option for the same VLAN ID.

  • The storage view of a logical domain might be incomplete if its backing storage is not part of a library that is managed by Oracle Enterprise Manager Ops Center.

5.6.2.3 Agentless Management

Agentless management does not deploy the Agent Controller on the control domain, which restricts the management capabilities to the following:

  • Perform basic state change command such as shutdown, start, reboot, and destroy.

  • Edit the number of vCPUs, the number of Crypto units, and the memory size.

  • Provision the Oracle Solaris operating system.

  • Access the guest's serial console.

  • View the network configuration and storage configuration for each guest.

With an agentlessly managed domain, you are unable to change the network and storage configurations, and therefore, unable to migrate the guests to another managed Oracle VM Server.

5.6.3 Discover and Manage Existing Oracle VM Server for SPARC Servers

You can discover and manage an Oracle VM Server for SPARC system that you manually provisioned. The underlying logical domains are automatically discovered and managed in Oracle Enterprise Manager Ops Center. In the Assets section, the discovered logical domains are displayed in order and grouped under the control domain.

You can manage the versions of Oracle VM Server for SPARC listed in the Oracle Enterprise Manager Ops Center Certified Systems Matrix.

Note:

To discover the server, the Logical Domains Manager must be running in the Oracle VM Server for SPARC system.

The following tasks are covered:

See Get Started With Virtualization for information about agent controllers.

5.6.3.1 To Agent Manage an Existing Oracle VM Server for SPARC System

  1. (Optional) Create SSH or SNMPv3 credentials. If you have not created the credentials, you can create them before creating an OS Discovery profile, or you can create them as part of the OS Discovery profile.

  2. Create an OS Discovery profile.

    1. Expand Plan Management in the Navigation pane.

    2. Click Discovery in Profiles and Policies.

    3. Click Create Profile in the Actions pane.

    4. Enter a name and description for the discovery profile. Expand Operating Systems and select Solaris, Linux OS. Click Next.

    5. (Optional) Click Next to skip tags.

    6. (Optional) Click Next to skip adding IP Ranges. You are prompted to provide IP addresses and host names during discovery.

    7. Click Select, select the SSH or SNMPv3 credentials that you previously created, then click OK. To create new credentials, click New and create the credentials.

      Select the Enable Oracle VM for SPARC management check box to deploy the Agent Controller during discovery. Click Next.

      Figure 5-2 Discovery Credentials for Oracle VM for SPARC Management


      Description of Figure 5-2 follows
      Description of "Figure 5-2 Discovery Credentials for Oracle VM for SPARC Management"
    8. Review the Summary page, then click Finish.

  3. Add the assets using the OS discovery profile.

    1. Expand Assets in the Navigation pane, then click Add Assets in the Actions pane.

    2. Select Add and manage various types of assets via discovery probes, then click Next.

    3. Select the Discovery profile you created in the previous step. You are prompted to complete the host names or IP addresses and network.

    4. Enter a comma-separated list of host names or IP addresses. Select a managed network with which the host is associated, or select Automatic to route the job to the most appropriate Proxy Controller.

      The IP address of a target must resolve to only one known network for automatic routing to succeed.

    5. (Optional) The Discovery credentials and management option are completed based on the profile. You can edit these fields.

    6. Click Add Now.

The control domain appears in the Asset tree under its server hardware. The control domain operating system appears under its control domain. The non-global zones, if any, appears under the control domain operating system with a limited set of capabilities.

When an Agent Controller is installed on the control domain, all the networks that are connected to the control domain and configured with an IP address are discovered in Oracle Enterprise Manager Ops Center.

Also, when the Agent Controller is installed on the logical domains of the control domain, the extra networks that are connected to the logical domains and configured with an IP address are discovered in Oracle Enterprise Manager Ops Center.

5.6.3.2 To Agentlessly Manage an Existing Oracle VM Server for SPARC System

Agentlessly managed discovery locates the logical domains through the proxy management point.

  1. (Optional) Create SSH credentials. If you have not created SSH credentials, you can create them before creating an OS Discovery profile, or you can create them as part of the OS Discovery profile.

  2. Create an agentless OS Discovery profile.

    1. Expand Plan Management in the Navigation pane.

    2. Click Discovery in Profiles and Policies.

    3. Click Create Profile in the Actions pane.

    4. Enter a name and description for the discovery profile. Expand Operating Systems and select Solaris, Linux OS. Click Next.

    5. (Optional) Click Next to skip tags.

    6. (Optional) Click Next to skip adding IP Ranges. You are prompted to provide IP addresses and host names during discovery.

    7. Select Manage without Agent Controller. Click Replace and choose the SSH management credentials. Click Next.

      Figure 5-3 Discovery Credentials for Oracle VM for SPARC Management

      Description of Figure 5-3 follows
      Description of "Figure 5-3 Discovery Credentials for Oracle VM for SPARC Management"
    8. Review the Summary page, then click Finish.

  3. Use the OS Discovery profile to add the Oracle VM Server for SPARC:

    1. Expand Assets in the Navigation pane, then click Add Assets in the Actions pane.

    2. Select Add and manage various types of assets via discovery probes, then click Next.

    3. Select the Discovery profile you created in the previous step. You are prompted to complete the host names or IP addresses and network.

    4. Enter a comma-separated list of host names or IP addresses. Select a managed network with which the host is associated, or select Automatic to route the job to the most appropriate Proxy Controller.

      The IP address of a target must resolve to only one known network for automatic routing to succeed.

    5. (Optional) The Discovery credentials and management option are completed based on the profile. You can edit these fields.

    6. Click Add Now.

The control domain appears in the Asset tree under its server hardware. The control domain operating system appears under the control domain. The logical domains appear under the control domain operating system.

Because it is agentlessly-managed, the control domain has a limited level of monitoring and management actions.

Note:

To change the server to agent managed, use Switch Management Access. To monitor and manage the Oracle VM Server for SPARC server, control domain, and operating system, select the LDom Virtualization Controller agent. To monitor and manage the global zone, select the Zone Virtualization Controller Agent.

5.7 Provisioning Oracle VM Server for SPARC

The following information is covered in this section:

5.7.1 Prerequisites for Provisioning Oracle VM Server for SPARC

The specific hardware, OS, and firmware requirements must be met for provisioning Oracle VM Server for SPARC.

5.7.1.1 Operating System

Configure your Oracle Solaris 11 Software Update Library in the Enterprise Controller. The library is a local copy of the parent repository of Oracle Solaris 11 Image Packaging System (IPS). When you create OS provisioning profile, the supported Oracle VM Server for SPARC version, Oracle Solaris 11 OS and the Support Repository Update (SRU) are populated and listed.

When you want to use Oracle Solaris 10 OS, import or upload the required update versions of Oracle Solaris 10 OS in the Initial EC Library or in a software library created using a local file system on the Enterprise Controller or on a shared file system on a NFS server. Refer to the Related Resources for Oracle VM Server for SPARC for the required update versions of Oracle Solaris 10 OS.

Upload the required Oracle Solaris OS image to the software library in Oracle Enterprise Manager Ops Center. See the Related Resources for Oracle VM Server for SPARC section for more information about uploading or importing OS images.

5.7.1.2 Hardware and Firmware

Refer to the Related Resources for Oracle VM Server for SPARC section for the supported platforms to install different versions of Oracle VM Server for SPARC.

Ensure that you install the correct system firmware. You can update the firmware using Oracle Enterprise Manager Ops Center. See the Related Resources for Oracle VM Server for SPARC section for information about downloading and updating the required firmware version.

When you use Oracle Enterprise Manager Ops Center to provision the Oracle VM Server for SPARC software, it installs the Oracle Solaris OS, the control domain, and an agent on the target system. The provisioning action removes existing virtualization software, including any previous logical domains installed on the service processor.

To install Oracle VM Server for SPARC software, apply a profile that specifies the values for resources such as CPU threads, Crypto units, and memory as described in Profiles for Provisioning and Configuring Oracle VM Server for SPARC.

When you have existing Oracle VM Server for SPARC environment that you want to discover and manage in Oracle Enterprise Manager, refer to the section Discovering Existing Oracle VM Server for SPARC Environments for more information.

5.7.2 Recommended Minimum Configuration

The recommended minimum configurations for the control domain are described in the following sections:

5.7.2.1 CPU Resource Allocation

You can allocate CPU threads or cores to the control domain.

The number of system CPUs determines the number of control domain CPU threads:

  • For less than 16 system CPUs, set the control domain CPU Threads to 2.

  • For between 16 and 64 system CPUs, set the control domain CPU Threads to 4.

  • For more than 64 system CPUs, set the control domain CPU Threads to 8.

You can select to allocate CPU resources either as CPU Threads or Whole-core. Whole-core is the default value in Oracle Enterprise Manager Ops Center. When you allocate as Whole-core, all the CPU Threads in the core are allocated to the control domain. For example, when you allocate two cores in UltraSPARC T2 servers, the control domain is allocated with all the 16 CPU Threads in the core. You can also set the maximum cores constraint when you select Whole-core allocation type. The maximum number of cores constraint specifies the number of cores that must be assigned to the domain.

5.7.2.2 Crypto Units

Crypto units are the resources on the supported platforms that provide high-performance, dedicated cryptographic engines. These can be used for tasks such as encrypting and decrypting network traffic between a Secure Socket Layer (SSL) web server and an application server.

Each CPU core has one Crypto unit and four or eight CPU threads. Because the Crypto unit is part of a core, the Crypto unit is bound only to domains that contain at least one thread from the parent core. Crypto units cannot be split as CPU threads are split. For example, you have assigned the Crypto unit for the first CPU core to the control domain. When a new logical domain is assigned a thread from the first CPU core and the Crypto unit for that core is already assigned, the control domain cannot assign that Crypto unit to the new logical domain. Allocation of Crypto units might not succeed, especially when a core is split between domains. An Oracle VM Server might allocate fewer Crypto units or none at all.

You must assign at least one Crypto unit to the control domain because the Crypto unit enables domain migration.

The use of Crypto Units is not mandatory although it might speed the logical domain migration. Allocation of Crypto Units might not be available in all the hardware supported. See the corresponding hardware data sheet and documentation for more information before planning for Crypto Units.

Example 5-1 Example of Crypto Unit Assignments

In UltraSPARC T1 based servers, one core is four CPU threads. Therefore, assign one Crypto unit and four CPU threads to the control domain. These values are set in the OS profile for Oracle VM Server for SPARC.

In UltraSPARC T2 and T2 Plus based servers, one core is eight CPU threads. Therefore, assign one Crypto unit and eight CPU threads to the control domain.

5.7.2.3 RAM

The amount of RAM for the control domain depends on the size of the system RAM and the load of the system.

  • For system RAM less than 8 GB, set the control domain's RAM to 1 GB.

  • For system RAM between 8 GB to 16 GB, set the control domain's RAM to 2 GB.

  • For system RAM greater than 64 GB, set the control domain's RAM to 8 GB.

In Oracle Enterprise Manager Ops Center the default value is 4 GB as a recommended starting point for logical domains, and the minimum value is 1GB.

Note:

See the Related Resources for Oracle VM Server for SPARC section for more information about allocating memory size.

5.7.2.4 NVRAMRC Value

Automatic booting on a SPARC system uses the default boot device that is defined in the non-volatile RAM (NVRAM).

User-defined commands that are executed during start-up are stored in the NVRAMRC file in the NVRAM. When you run an OS provisioning job on a SPARC machine, Oracle Enterprise Manager Ops Center resets the configuration to the factory default configuration and removes the information that is stored in the NVRAMRC file. The control domain OS configuration profile gives you the option to preserve the information in the NVRAMRC file before resetting the server to the factory defaults, and then restore the information after the reset.

5.7.3 Profiles for Provisioning and Configuring Oracle VM Server for SPARC

Oracle Enterprise Manager Ops Center provides two profiles for defining the parameters required for provisioning Oracle VM Server for SPARC:

  • OS Provisioning Profile: The profile collects information required for provisioning the OS.

  • OS Configuration Profile: The profile collects information about control domain parameters and network configuration.

Create an OS provisioning plan that encapsulates the OS provisioning and configuration profile for installing Oracle VM Server for SPARC software, the OS, and to set the parameters of the control domain.

The OS provisioning profile collects the following details:

  • OS provisioning parameters such as OS image, and Oracle VM Server for SPARC version.

  • OS setup details such as time zone, language, root and console serial port and baud rate, root password, and options to enable manual net boot and save NVRAMRC values.

  • File system layout.

  • Naming services that includes, DNS, NIS, and LDAP.

The OS configuration profile collects the following information:

  • Control domain configuration such as CPU Threads or Whole-cores, memory, and Crypto Units.

  • Options such as enabling SR-IOV, and detaching the unused buses.

  • OS management option to install the Agent Controller.

  • Enabling multiplexed I/O to connect to block storage libraries.

  • Networking options to use IPMP or Link Aggregation.

5.7.4 Creating an OS Provisioning Profile for Oracle VM Server for SPARC

You can create an OS Provisioning profile to provision Oracle VM Server for SPARC.

  1. Select the Plan Management section and expand Profiles and Policies.

  2. Select OS Provisioning and click Create Profile in the Actions pane.

  3. Specify the profile details in the Create Profile - OS Provisioning wizard:

    1. Enter the name and description of the profile.

    2. Select Oracle VM Server for SPARC as the Subtype.

    Click Next to select the OS image and distribution.

  4. Select the following parameters in the Specify OSP Parameters step:

    • Oracle VM Server for SPARC Version

    • Software Group

    Refer to Operating System for selecting Oracle Solaris 10 OS. You have option to include custom scripts when you use Oracle Solaris 10. For Oracle Solaris 11 OS, the SRU and the supported Oracle VM Server for SPARC version are listed.

    Click Next to specify the OS setup.

  5. Specify the following OS setup parameters:

    • Select a language from the list.

    • Specify the time zone.

    • Specify a terminal type.

    • To monitor the installation using a serial connection, select the console serial port device and the baud rate.

    • Enter the NFS4 domain name for the target system to use. A dynamic NFSv4 domain name enables the name to be derived at run time, based on the naming service configuration. You can also enter a static domain name.

    • Enter the root password for the root user on systems provisioned using this profile. Re-enter the password for confirmation.

    • Select the Manual Net Boot option to enable manual control of network boot operations for the target system. You must select this option for a target system that does not have a service processor because Oracle Enterprise Manager Ops Center cannot control the network boot process remotely on these systems.

    • Select the option Save NVRAMRC values to preserve your user-defined commands that were executed during start-up.

    Click Next.

  6. If it is Oracle Solaris 11 OS, then you are directed to Step 7 to create a user account to log in.

    Otherwise, to Step 8 for Oracle Solaris 10 OS.

  7. For Oracle Solaris 11 OS, root login to the system is not enabled. Create a user account through which you can SSH to the OS after provisioning. Enter the user name and password for the user account.

    Click Next to specify whether you want to install the OS on an iSCSI disk.

  8. Select the option Use iSCSI Disk to use iSCSI disks for provisioning Oracle VM Server for SPARC. The storage server is identified in the network and select the volume group that must be used to create a new iSCSI disk. A new iSCSI disk is created in the storage server for each target when necessary.

    A Dynamic Block Storage Library is automatically created for each storage server.

    If you want to enter the details of the storage server, select Manually Specify iSCSI Disk to enter the storage server IP address and the LU Number while deploying the plan.

    Click Next to specify the file system layout.

  9. Specify the disk partitions and file systems that you want to create on the target system. The root (/) and a swap file system are defined by default. You can modify the size of the file system. Click the Add icon to define a new partition. For each partition that you define, provide the following information:

    • File System Type: Select a file system type: ufs, unnamed, zfs, or swap.

    • Mount Point: Enter a directory to use as a mount point for partitions.

    • Device: Enter the rootdisk keyword and a slice value to describe a partition on the target system's boot disk, for example, rootdisk.s0, or enter the logical device name, for example, c1t0d0s0, of the partition that you want to create.

    • Size: Enter the size that you want to assign to the partition, expressed in MB. Do not enter any value for the size when you want to allocate the remaining unused disk space to a file system.

      Click Next to specify the name service.

  10. Specify the name service, domain name and the corresponding name server. Select one of the following name services:

    • DNS: Enter the domain name of the DNS server and enter the IP address of the DNS server in the Name Server field. You can enter up to three IP addresses as the value for the Name Server. Provide the additional domains to search for name service information in the Domain Name Search List. You can specify up to six domain names to search. The maximum length of each search entry is 250 characters.

    • NIS or NIS+: Enter the domain name of the NIS or NIS+ server. When you know the NIS server details, select the option Specify an NIS Server and enter the NIS server host name and the IP address.

    • LDAP: Enter the domain name of the LDAP server. Specify the name of the LDAP Profile you want to use to configure the system. Enter the IP address of the LDAP Profile Server. You can also provide the Proxy Bind Distinguished Name and Password.

    • None: Select None when there is no naming service configured.

    Click Next to review the properties of the profile.

  11. Review the summary of your selections. Click Finish to create the profile.

5.7.5 Creating an OS Configuration Profile for Oracle VM Server for SPARC

  1. Select the Plan Management section and expand Profiles and Policies.

  2. Select OS Configuration and click Create Profile in the Actions pane.

  3. Specify the profile details:

    1. Enter the name and description of the profile.

    2. Select Oracle VM Server for SPARC as the Subtype.

    Click Next to specify the setup for configuring control domain.

  4. Specify the resources that you want to assign to the control domain. The remaining resources are available for the logical domains.

    • Oracle VM Server Version: Select the Oracle VM Server for SPARC version to be configured.

    • CPU Model: Select how you want to allocate the CPU resources, Virtual CPU or Whole-Core. The default value is Whole-Core.

    • CPU Threads: This field is displayed when you select the CPU Model as Virtual CPU. Specify the number of CPU threads that you want to assign to the control domain.

    • CPU Cores: This field is displayed when you select the CPU Model as Whole-Core. Specify the number of cores to be allocated to the control domain.

    • Max CPU Cores: This field is displayed when you select the CPU Model as Whole-core. Specify the maximum number of cores that must be assigned to the domain.

    • Memory: Specify the amount of memory that you want to assign to the control domain. The default value is 4 GB and the minimum value is 1 GB.

    • Requested Crypto Units: Specify the number of Crypto units that you want to assign to the control domain.

    • Virtual Console Port Range: Specify the minimum port and maximum port of the virtual console of the control domain. The default port range for virtual console is 5000 to 6000.

    • Virtual Disk Server: A virtual disk server (vds), primary-vds0, is added to the control domain by default. The virtual disk services allow you to export virtual disks to logical domains. You can modify the default name of the virtual disk server.

    • Enable Single Root I/O Virtualization (SR-IOV): The supported PCIe buses are SR-IOV enabled and maximum number of virtual functions (VFs) are created on them. The type of network card decides the number of VFs. The option to enable SR-IOV is available from Oracle VM Server for SPARC 2.2 version.

    • Detach unused buses: The PCIe buses that are not in use by the control domain are released. All the unused PCIe buses are detached so that it is available for creating root domains.

    Note:

    For Oracle VM Server for SPARC 1.2 version, you can select Enable JASS to install SUNWJass package to harden the system.

    Click Next to specify the OS management for installing Agent Controller.

  5. The option to install the Agent Controller is by default and cannot be changed.

    Select Enable Multiplexed I/O option to enable multiple path for SAN library connectivity to the control domain. The action enables LUNs to be accessed through multiple host controller interfaces from a single instance of the I/O device.

    Click Next to specify the networking details for provisioning.

  6. Select one of the network options for the target system:

    Select an option and click Next to continue.

  7. Select the following details for network configuration:

    • Controller: The network interfaces built in with the server hardware are displayed as default in the Controller column. For bare-metal OS provisioning, only the in-built network interfaces of the server are displayed.

    • Interface: Select the network interface which you want to configure on the OS. The network interfaces are represented as net_x. x ranges from 0 to 31. The number of interfaces for each controller available is 32. For bare-metal ILOM servers, the device map are not available. For such instances, you can specify the MAC address instead of interface while deploying the provisioning plan.

    • Address Allocation Method: The address allocation for the network is to use static IP. You have to provide the static IP address while deploying the OS provisioning plan.

    • Virtual Switch: Select Auto to create virtual switch automatically for the network connection. Or select None if you do not want to create virtual switch. You can also enter a name for the virtual switch. Enter a switch name directly in the virtual switch list.

      Click the Add icon to add more networks and define the interfaces.

    Click Next to go to the Summary step.

  8. Review the Summary of your selections. Click Finish to create the profile.

5.7.6 Creating a Link Aggregation in Oracle VM Server for SPARC

While creating the profile for installing Oracle VM Server for SPARC, follow the procedure to create link aggregation:

  1. Select Use Link Aggregation in the Specify Networks step of the profile.
  2. Specify and configure the IEE 802.3ad Link Aggregation details:
    • Link Aggregation Name: Select the name of the Link Aggregation. The names are set to aggr<x>.

    • Load Balancing Policy: Define the policy for outgoing traffic.

    • Aggregation Mode and Switches: When the aggregation topology connects through a switch, determine whether the switch supports the Link Aggregation Control Protocol (LACP). When the switch supports LACP, you must configure LACP for the switch and the aggregation. Define one of the modes in which LACP must operate.

    • LACP Timer: Indicates the LACP timer value, either short or long.

    • MAC Address Policy: Define whether the MAC address of the NICs are fixed.

    • Virtual Switch: Select Auto to create virtual switch automatically for the network connection. Or select None if you do not want to create virtual switch. You can also enter a name for the virtual switch. Enter a switch name directly in the virtual switch list.

    • Number of Interfaces: By default, two interfaces are defined in the aggregation. You can modify the number of interfaces for the aggregation.

    You can add multiple link aggregation and define the configuration parameters for each aggregation.

    Click Next to specify the link aggregation interfaces.

  3. Select the interfaces for each link aggregation defined in the previous step. Choose the Controller from which the interfaces are used in the link aggregation.

    You must define the number of interfaces in the previous step. You cannot add or delete the required interfaces in this step.

    Click Next to continue.

5.7.7 Creating IPMP Group in Oracle VM Server for SPARC

While creating the profile for installing Oracle VM Server for SPARC, follow the procedure to create IPMP groups.

  1. Select Use IPMP in the Specify Networks step of the profile.
  2. Specify and configure the IPMP groups:
    • IPMP Group Name: The name of the IPMP group is automatically displayed in the format of ipmp<x>. You can also modify the default name of the IPMP group.

    • Failure Detection: The detection can be Link-Based or Link-Based and Probe-Based.

    • Number of Interfaces: By default, two interfaces are defined in the group. You can modify the number of interfaces for the IPMP group.

    You can add multiple IPMP groups and select the failure detection for each group.

    Click Next to specify the IPMP interfaces.

  3. Select the Controller from which the interfaces are used in the IPMP group. Select the interfaces for each IPMP group and choose whether they are Failover or Standby interface. You can add multiple interfaces to each IPMP group.

    Select whether to assign IP address to the specified NIC during configuration. The data and test addresses are assigned during profile execution.

    Select Auto to create virtual switch automatically for the network connection. Or select None if you do not want to create virtual switch. You can also enter a name for the virtual switch. Enter a switch name directly in the virtual switch list.

    Click Next to continue the profile.

5.7.8 Deployment Plans for Oracle VM Server for SPARC

The Provision OS deployment plan provides the steps to provision and configure Oracle VM Server for SPARC. The Provision OS plan consists of the following profiles:

  • OS Provisioning Profile

  • OS Configuration Profile

Create a Provision OS deployment plan with the two profiles defined in the steps of the plan and apply the plan on the target server.

See the Related Resources for Oracle VM Server for SPARC section for more information about creating and managing the profiles and plans.

5.7.8.1 Applying a Deployment Plan for Oracle VM Server for SPARC

When you apply a deployment plan to provision Oracle VM Server for SPARC, you must have the following information to complete the installation:

  • In the server hardware, obtain the network interface that is physically connected to the network that is managed by Oracle Enterprise Manager Ops Center.

  • Both tagged and untagged networks are listed for network configuration. When OpenBoot PROM (OBP) is used, only untagged networks can be used for OS provisioning as you cannot boot from a tagged network.

  • The IP address for the boot interface.

  • If you want to identify the network interface using the MAC address, you can select to enter the MAC address instead of the boot interface.

  • The details of network connection that you want to use to configure the OS after booting. For multiple network configuration, the first interface is always overwritten by the boot interface that is defined during the OS provisioning step. For single network configuration, it is same as the boot interface network.

  • Select to place the Oracle VM Server for SPARC in a server pool or not.

  1. Select the Provision OS plan in the Deployment Plans list.
  2. Select Apply Deployment Plan in the Actions pane.
  3. Select one or more assets and add to the target list.
  4. Select the plan to be applied with minimal interaction. If required to change the profile parameters, then select Allow me to override any profile values.
  5. Select not to review the steps that are not included in the plan and click Next.
  6. The wizard collects information for provisioning Oracle VM Server for SPARC. Click Next.
  7. In the Boot Interface Resource Assignments step, provide the following information:
    • Network: The network for the boot interface.

    • Controller: Select the controller that provides the network interface for OS provisioning. It is always default for the Oracle VM Server for SPARC provisioning.

    • Interface: Select the interface from the list. The network interface that is physically connected to the selected network.

    • IP Address: Enter the IP address for the boot interface.

    • (Optional) Primary Hostname: Enter the host name for Oracle VM Server for SPARC.

    If you want to identify the network interface by its MAC address, then select the option Identify Network Interface by MAC Address and enter the MAC address instead of selecting the Controller and the Interface.

    Click Next to view the OS provisioning summary.

  8. Review the parameter of OS provisioning and click Next.
  9. The following steps in the wizard collects information about OS configuration.

    Click Next.

  10. Specify the network resources that were defined in the profile. Select the network and for each network, select the network interface and enter the IP address.

    The first network interface listed is the boot interface. For multiple network configuration, the first network interface is always overwritten by the boot interface network. You can select which is the primary network interface after the provisioning of the OS.

    For single network, the boot interface network will be the defined as the primary network during OS configuration.

    Click Next.

  11. Select whether you want to add the Oracle VM Server for SPARC to a server pool. You also have option to create a new server pool with default server pool settings. For creating a new server pool, you must provide a NAS storage library for the server pool.

    Click Next.

  12. Review the summary of the OS configuration parameters and click Next to schedule the job.
  13. Schedule the provisioning job to run immediately, or at a later time.

    Click Apply to apply the deployment plan on the selected targets.

5.7.9 Overview of Oracle VM Server for SPARC Installation

You can install the software on bare-metal or on systems already configured with logical domains. When you provision Oracle VM Server through Oracle Enterprise Manager Ops Center, any previous configuration is removed and the service processor is reset to its factory defaults.

When you provision Oracle VM Server for SPARC, the instance of the Oracle Solaris OS becomes the control domain which is the first domain to be created. The virtualization host on which this instance of Oracle Solaris OS runs is called the Oracle VM Server Host or simply, Oracle VM Server. The Logical Domains Manager runs in the control domain and provides the functions to create and manage logical domains. You can have only one control domain per server. You cannot change the name or destroy the control domain.

This section describes how to install the Oracle VM Server for SPARC software on a service processor:

  1. The Oracle VM Server for SPARC software depends on particular Oracle Solaris OS versions, required software patches, and particular versions of system firmware. See Selection of Oracle VM Server Version.

  2. Discover the target system on which you want to install Oracle VM Server for SPARC software.

  3. Refer to the Related Resources for Oracle VM Server for SPARC section for information about configuring DHCP services on Proxy Controllers, if required.

  4. Create an OS provisioning and OS Configuration profiles for provisioning Oracle VM Server for SPARC. See Profiles for Provisioning and Configuring Oracle VM Server for SPARC.

  5. Create a deployment plan with the profiles created for provisioning and configuring Oracle VM Server for SPARC.

  6. (Optional) Adjust the amount of time allowed for the provisioning job. The default time is three hours or 180 minutes. You can edit the default_timeout value for OS provisioning in the Enterprise Controller Configuration in the Administration section, as shown in Figure 5-4.

    Figure 5-4 Default Timeout Settings

    Description of Figure 5-4 follows
    Description of "Figure 5-4 Default Timeout Settings"

The Oracle VM Server for SPARC provisioning job performs the following major tasks:

  • Downloads the appropriate OS image.

  • Initiates a net boot action on the service processor.

  • Installs the Oracle VM Server Host.

  • Configures Oracle VM Server Host according to the values set in the profile such as memory, CPU threads, Crypto units, and virtual console port range.

  • When selected, installs the SUNWJass package to harden the system. For example, you can install the SUNWJass package for version 1.2.

  • Enables the Fibre Channel ports on the storage system.

  • Installs and configures the Agent Controller. Do not install the Agent Controller manually on the Oracle VM Server host.

  • Detaches unused buses and enables SR-IOV feature on PCIe buses as defined in the profile.

5.7.9.1 Selection of Oracle VM Server Version

Verify that the target system has the correct hardware and firmware configuration to support the installation of selected version of Oracle VM Server for SPARC.

When the supported hardware and firmware versions are not there, the provisioning job might fail. You cannot force set the version of the software to be installed. For the complete list of supported hardware, firmware version, and the supported Oracle VM Server for SPARC version, refer to the Related Resources for Oracle VM Server for SPARC section for more information.

5.7.9.2 Important Notes for Installation

Some important notes for installing Oracle VM Server for SPARC in Oracle Enterprise Manager Ops Center:

  • The OS in the control domain must have the default locale set to C. The control domain provisioning is supported on Oracle Solaris SPARC Proxy Controllers, but not on Linux Proxy Controllers.

  • To provision the Oracle VM Server for SPARC software, the proper system model name must be populated on the service processor. The model name is not available on the service processor of the Sun Blade T6320 hardware.

  • Oracle Enterprise Manager Ops Center does not support use of the LDoms configuration manager in LDoms 1.2.

5.7.9.3 Manual Net Boot Initiation

A target system requires a manual net boot operation when Oracle Enterprise Manager Ops Center cannot perform a remote network boot process. Profiles that provision this type of system contain the Manual Net Boot option enabled. You must initiate the net boot operation when the Oracle VM Server provisioning job completes on the UI.

  1. Log in to the service processor of the target system.
  2. When the target system is running, enter halt.
  3. Enter the following command
    boot net:dhcp - install
    

5.7.9.4 Additional Configurations

A virtual console concentrator named primary-vcc0 is created during the installation of Oracle VM Server for SPARC and is not required to be included or defined in the profiles.

The virtual disk server is created by the default name primary-vds0. You can modify the name of the virtual disk server in the OS configuration profile.

5.8 Manage Oracle VM Server for SPARC

The following actions and information are available for managing the Oracle VM Server:

After you have provisioned or discovered the Oracle VM Server for SPARC, you can dynamically manage the Oracle VM Server control domain resource configurations such as CPU Threads, Crypto Units, and memory. You can associate storage libraries and attach networks with Oracle VM Server for SPARC, create logical domains, and monitor the performance of logical domains.

You can also create server pools for Oracle VM Server for SPARC for an efficient management of your resources. For creating server pools, see Server Pools.

Note:

If the Enterprise Controller is installed in a logical domain, do not take any action that would power down or restart the corresponding control domain.

5.8.1 Virtual Services in Control Domain

In Oracle Enterprise Manager, the control domain is also a service domain and the following virtual device services are created:

  • Virtual disk server: A virtual disk server (vds), primary-vds0, is added to the control domain by default. The virtual disk services allow you to export virtual disks to logical domains. The virtual storage infrastructure enables logical domains to access the storage disks that are not directly assigned to them. The virtual disk service processes the requests from the logical domains and submits them to the back end storage.

  • Virtual network switches: The virtual network switch enables networking between virtual network devices in logical domains. When you connect a network to control domain, a virtual switch is created for each network connection. You can also define the switch name. You use virtual switch to connect a logical domain to the external network for communication. A virtual network (vnet) device is defined in the logical domain when connected to a virtual switch. There is no virtual switch creation for SR-IOV enabled networks and you do not require virtual switch for connecting to SR-IOV enabled network. You can also edit the virtual switch attributes such as Oracle Solaris Cluster Mode, Physical Link State, Inter-vnet Link, Maximum Bandwidth, and Private VLAN. Note that Private VLAN settings can only be edited for a vnet if the primary VLAN ID is in untagged mode.

  • Virtual console concentrator: This functions as a concentrator for all logical domains console and for use by the virtual network terminal server daemon. The console I/O from all other domains is redirected to the control domain that is running the virtual console concentrator (vcc).

5.8.1.1 Editing Virtual Switch Attributes

You can edit the virtual switch attributes such as Oracle Solaris Cluster Mode, Physical Link State, and Inter-vnet Link.

Inter-Vnet Links

The inter-vnet LDC channels are configured so that virtual network devices can communicate directly to achieve high guest-to-guest communications performance. However, as the number of virtual network devices in a virtual switch device increases, the number of required LDC channels for inter-vnet communications increases exponentially. You can choose to enable or disable inter-vnet LDC channel allocation for all virtual network devices attached to a given virtual switch device. By disabling this allocation, you can reduce the consumption of LDC channels, which are limited in number. You can switch off or on the inter-vnet link. From Oracle VM Server for SPARC 3.4 version, an additional option Auto is available to manage the inter-vnet links.

To edit the virtual switch attributes:
  1. Select the Oracle VM Server or the control domain in the Assets section.
  2. Select the Virtual Services tab in the center pane.
  3. Select Virtual Network Switches in the sub tab.
  4. Select the virtual switch that you want to edit and click the Edit icon.
    The Edit Logical Domain Virtual Switch Attributes window appears.
  5. You can modify the following properties:
    • Physical Link State

    • Solaris Cluster Mode

    • Inter-vnet Link

  6. Click Finish to save the changes.

5.8.2 View I/O Resources

The I/O Resources tab in the Center pane displays the PCIe root complexes, PCIe Endpoint devices, Network Interface Units (NIU), and Single Root I/O virtualization (SR-IOV) functions.

5.8.2.1 Buses and End Point Devices

The PCIe root complexes and the NIUs that are available in the servers are listed here.

The number of PCIe root complexes and NIUs depend on the type of server hardware. Initially, when you provision Oracle VM Server for SPARC, all the PCIe root complexes and NIUs are assigned to the control domain. When you use the option Detach Unused Buses option in the OS configuration profile, the PCIe buses are released from the control domain.

You allocate PCIe buses to create root domains. You allocate PCIe Endpoint devices to create Physical I/O domains.

To assign PCIe Endpoint devices to Physical I/O domains, you must first release the devices from the corresponding PCIe bus that is allocated to the control domain or root domain. You must manually release the PCIe Endpoint devices using the CLI.

The PCIe Endpoint devices displays whether the PCIe card slot is occupied or empty. If occupied, the domain to which it is allocated is displayed. The devices to which the PCIe bus is attached to are also grouped and displayed.

5.8.2.2 SR-IOV Services

Single Root I/O Virtualization is a PCI-SIG standard specification that enables efficient use of PCIe devices. In this, a single PCIe card owned by a PCIe root complex is made to physically appear in multiple domains simultaneously. A single I/O resource also know as physical function is shared by many virtual machines. A physical function is a PCIe device that is SR-IOV enabled with appropriate hardware and OS support can appear as multiple, separate physical devices, each with its own configuration space.

A virtual function is a lightweight PCIe function that shares one or more physical resources with the physical function and with virtual functions that are associated with that physical function. The number of virtual functions that are supported on a physical function depends on the hardware.

The physical functions and their corresponding virtual functions are listed in the SR-IOV services tab. The number of virtual functions supported for the physical function and the logical domains to which the virtual functions are assigned are displayed.

Currently, the SR-IOV feature is enabled only for the SR-IOV cards that are installed on the control domain. SR-IOV feature are not enabled when you assign the SR-IOV card to a physical I/O domain.

With the release of Oracle VM Server for SPARC 3.1 version, the SR-IOV feature is enabled when the SR-IOV cards are assigned to a root domain.

5.8.3 Modify Configuration and Tags

You can modify the control domain configuration that includes CPU Model, CPU Threads or cores, and memory. You can select to switch between the CPU Models. When you select Whole-core option, you can edit the number of whole-cores and the maximum number of cores to be allocated to the Oracle VM Server.

You can also modify the name and description of the control domain.

Use the Edit Attributes option to modify the configuration. The Oracle VM Server must have Oracle Solaris 10 10/09 OS or higher version to edit the attributes.

When you modify the memory size, Oracle VM Server is rebooted unless it is running Oracle VM Server for SPARC 2.0 or higher version. You can choose to cancel the reboot option after editing the changes.

If you have any physical bindings constraint for the memory, CPU cores, or both, then you cannot edit CPU and memory configuration of the control domain in the UI. You can edit the CPU Model even if you have any physical bindings constraint, but the change of CPU Model will undo the physical binding as well.

Oracle Enterprise Manager Ops Center also does not provide options to explicitly assign physical resources such as CPU and memory. Whereas, you can use the CLI to assign physical resources to the logical domains. These are displayed in the Summary tab of the control domain as Physical Bindings.

Use Edit Tags option to modify the existing tags and add new tags.

5.8.3.1 Delayed Reconfiguration Mode

Some resources of a domain can be configured dynamically on a running logical domain, while others must be configured on a stopped domain. If a resource cannot be dynamically configured, the logical domain can be set in delayed reconfiguration state to postpone the configuration activities until after rebooting the domain.

Delayed reconfiguration is restricted to the control domain or a PCIe root domain (if supported). For all other logical domains, you must stop the domain to modify the configuration unless the resource can be dynamically reconfigured.

You cannot initiate a delayed reconfiguration using Oracle Enterprise Manager Ops Center UI, but Oracle Enterprise Manager Ops Center detects when a logical domain is in delayed reconfiguration state. When a domain is in delayed reconfiguration state, a message in the center pane is visible to all logical domains attached to the affected control domain. When a delayed reconfiguration is in progress, trying to perform actions such as start, shutdown, reboot, or change attributes in any domain attached to the affected control domain will fail.

You can either reboot the domain in delayed reconfiguration to apply changes and exit the delayed configuration mode, or use the Cancel Delayed Reconfiguration action to cancel the delayed reconfiguration in the logical domain.

5.8.4 Reboot Oracle VM Server

You can reboot the control domain regardless of the state of the logical domains in it. During the reboot, the logical domains might suspend temporarily but keep their current states.

5.8.5 Performance Management

You can view the following information about Oracle VM Server performance:

  • The Summary tab displays the Oracle VM Server for SPARC configuration information, and its health status.

  • The Dashboard tab displays the general information about the selected Oracle VM Server for SPARC. A graphical representation of the group members and its relationship are displayed. The unassigned incidents and the list of recent incidents are also displayed.

  • The Analytics tab displays the CPU, memory and network utilization by the logical domains in the Oracle VM Server. The consumption is represented in the form of charts.

  • The Incidents tab lists the incidents that are reported for the Oracle VM Server. The tab provides the list of unresolved incidents and a graphical representation of the problem composition.

  • The Monitoring tab, as shown in Figure 5-5, shows the monitoring values and boundaries that are set for Oracle VM Server activity. You can review the rules, monitoring attributes, specific time period for monitoring, and the level of severity for monitoring the activity of Oracle VM Server. For more information, see the Related Resources for Oracle VM Server for SPARC section.

    Figure 5-5 Oracle VM Server for SPARC Monitoring Options

    Description of Figure 5-5 follows
    Description of "Figure 5-5 Oracle VM Server for SPARC Monitoring Options"
  • The Configuration tab displays information such as remote logging, routing, NFS domain name, and name service information for the Oracle VM Server.

  • The Charts tab displays the CPU, memory and network utilization of the Oracle VM Server. The data are collected in five-minute intervals and displayed graphically. The utilization data are provided for different time intervals.

5.9 Managing Storage Resources in Oracle VM Server for SPARC

The following information is covered in this section:

Storage libraries are required for storing logical domain metadata, ISO images and to provide virtual disks for logical domain storage. You must first associate the storage library to the Oracle VM Server and then it is made available for all the logical domains under it.

You can associate the following libraries for Oracle VM Server:

  • Filesystem storage: This includes the NAS storage libraries.

  • Block storage: This includes the Static and Dynamic Block Storage libraries.

    • Static Block Storage: The Static Storage library includes the LUNs of Fibre Channel (FC) and iSCSI disks of storage servers. Oracle Enterprise Manager Ops Center manages the storage device on which the LUNs are already created. The existing LUNs are assigned to the library. You cannot create, modify or delete the LUNs in Static Storage Library.

    • Dynamic Block Storage: The Dynamic Storage Library is automatically created when the storage server is discovered and managed in Oracle Enterprise Manager Ops Center. It includes the LUNs of iSCSI disks of storage servers. You can create LUNs as you attach the library to an Oracle VM Server. You can create, modify and delete the LUNs as required and it is defined as Dynamic Storage Library.

When you are associating the libraries with the control domain, I/O domain and root domains, the storage resources are applicable at the domain level. You can associate the libraries with the control domain, I/O domain, or root domain. You must select at least one domain per server to be associated with the library. You must not attempt to associate the storage libraries from the OS of the domain.

The storage resource that is to be used for a guest domain might not be exclusively provided by the control domain but also by the I/O domains and the root domains configured on the Oracle VM Server. The same storage resource can be accessed through multiple I/O domains which results in redundancy for the access to the storage resource.

When you associate the storage library with Oracle VM Server that are configured with I/O domains and root domains, you can select the domain to which you want to associate the storage library.

Select the option Associate Library to associate the libraries with the selected Oracle VM Server.

5.9.1 Associating Storage Library with the Domains

An Oracle VM Server can be configured with I/O domains, root domains, and guest domains. When you associate storage library with Oracle VM Server, you can select to associate the library with the preferred domains.

Associate Libraries wizard provides with options to select the appropriate domains to which you want to associate the storage library.

  1. Select the Oracle VM Server or the control domain in the Assets section.
  2. Click Associate Libraries in the Actions pane.

    The Associate Library Wizard is displayed.

  3. Select the libraries that you want to associate with the domains.

    Click Next.

  4. Select the Associate action for the control domain, I/O domain, or both. You must select at least one domain per server.

    When the storage library is already associated with any of the domains, the Associate action is selected. You can deselect the option to disassociate the storage library from the domain.

    This action is repeated for each selected storage library. Click Next.

  5. Review the summary of the association and click Finish to associate the library with the selected domains.

The domains for which the association was removed are disassociated from the storage library.

5.9.2 Disassociating Storage Libraries

  1. Select the Oracle VM Server or the control domain in the Assets section.
  2. Select the Libraries tab in the Center pane.

    All the storage libraries that are associated with the system are displayed.

  3. Select the storage library that you want to disassociate from the domain, then click the Disassociate Library icon.

    The Disassociate Library window is displayed.

  4. The domains that are associated with the storage library are displayed with the Disassociate option selected. You can deselect the Disassociate action if you do not want to remove the association.
  5. Click Finish to disassociate the storage library from the selected domains.

5.10 Managing Network Resources in Oracle VM Server for SPARC

The following information is covered in this section:

Attach networks to Oracle VM Server to provide networking facilities for the logical domains.

With the support for advanced I/O domain configurations in Oracle Enterprise Manager Ops Center UI, the network configuration for Oracle VM Server is also enhanced to support the access to the available network interfaces from the I/O domains and root domains.

You can attach networks to the Oracle VM Server that involves the physical interfaces or etherstub device belonging to the control domain, root domain or the I/O domains. When you attach networks to the Oracle VM Server, you can select the domain and the physical network interfaces or the etherstub devices that belong to the domain.

etherstub is an Oracle Solaris 11 network virtualization feature. etherstub is a pseudo network device which provides functionality similar to physical network devices but only for private communications with its clients. This pseudo device can be used as a network back-end device for a virtual switch that provides the private communications between virtual networks.

When you discover and manage existing Oracle VM Server for SPARC environments, you must have the Agent Controller installed on the control domain to discover all the networks that are attached and configured with an IP address in the control domain.

Also, when the Agent Controller is installed on the logical domains of the control domain, the extra networks that are connected to the logical domains and configured with an IP address are discovered in Oracle Enterprise Manager Ops Center.

5.10.1 Virtual Switches

Virtual switches are created or re-used when you attach networks to Oracle VM Server.

This is not applicable when you select SR-IOV enabled network interface. See Attaching SR-IOV Enabled Networks to Oracle VM Server for more information about attaching SR-IOV enabled networks.

You can make multiple connections to a network in the Oracle VM Server. For each network connection, a virtual switch is required. If there is an already existing virtual switch for the physical interface or etherstub device, you can re-use the virtual switch. If there is no virtual switch for the physical interface or etherstub device, you can either provide a user-friendly name for the virtual switch or a virtual switch is automatically created with a default naming pattern. For example, the network 1.1.1.0/24, the virtual switches take the name as 1.1.1.0_24, 1.1.1.0_24_1, and 1.1.1.0_24_2. The virtual switches have a unique name. When you create and start a logical domain, you define the virtual switch that connects the logical domain to the network. Each virtual switch is connected to a NIC.

5.10.2 Service Domains

When you connect to the network interfaces from physical I/O domains and root domains, the virtual switch is created in the control domain. You cannot define the IP address allocation for the network connection. Instead, you can later define the IP address in the OS of the logical domain as required. You can define the IP address only when the network interfaces are used from the control domain.

5.10.3 Tagged and Untagged Mode

You can attach the network to Oracle VM Server for SPARC in tagged or untagged mode. This option is available for networks configured with VLAN IDs.

You can set the tagging mode while attaching the network to Oracle VM Server. When you make multiple connections to the same network, the connection must be either in tagged or untagged mode. You cannot mix the tagging mode for the multiple connection networks.

Refer to the section Mixed Network Tagging Mode Configurations in Server Pool for the limitations of having Oracle VM Servers in different tagging mode in a server pool.

5.10.4 IPMP and Link Aggregation

You can create IPMP groups and aggregate links in the control domain. Navigate to the Oracle Solaris OS of the control domain and create IPMP groups or Link Aggregation.

Refer to Networks for Virtualization for more information about creating IPMP groups and Link Aggregations in an Oracle Solaris OS.

5.10.5 Network Tagging Mode Conditions

There are certain scenarios in which the network configuration must be applied to avoid any networking issues. The scenarios that are explained here are for Oracle VM Server for SPARC in stand-alone mode:

  • You can select networks without VLAN ID. The UI does not provide the option to select Tagged or Untagged mode.

  • You can select to associate and configure the networks with VLAN ID in Tagged mode.

  • You can select to associate and configure the networks with VLAN ID in Untagged mode.

  • You can select to configure the networks in mixed tagging mode in the server pool. For example, you can attach the network N1 with VLAN ID = 100 in tagged mode with the server S1 and in untagged mode for server S2. Refer to Mixed Network Tagging Mode Configurations in Server Pool for more detailed information.

  • You can attach networks whose VLAN ID is similar to another network already connected to the servers. For example, a server S1 is already connected to network N1 with VLAN ID = 100, then while creating the server pool with S1 as the member of the pool, you can also attach another network N2 with VLAN ID =100.

  • You can edit the VLAN ID of a network when you are attaching the network in Tagged mode for the first time.

  • When you can edit the VLAN ID of the network, you cannot enter -1 as the value for the VLAN ID.

  • If the selected network with a VLAN ID is already connected to the selected assets in Tagged mode, then you cannot edit the VLAN ID and make another connection.

  • You cannot make multiple network connections to the Oracle VM Server over the same network in both tagged and untagged modes. The mode can be either in tagged or untagged mode only. For example, if you attach network N1 with VLAN ID =100 for the first time to server S1 in Tagged mode, then you cannot make another connection to the same network N1 in Untagged mode. Every other connection with network N1 must always be in Tagged mode for server S1.

  • If the selected members of the pool are already connected to network N1 with VLAN ID =100, then you cannot select the same network with different VLAN ID to be connected for the server pool.

5.10.6 Attaching Networks to Oracle VM Server for SPARC

The procedure to attach networks is applicable for Oracle VM Server for SPARC in stand-alone mode. The procedure varies if the Oracle VM Server is placed in a server pool.

Refer to Server Pools for more information.

  1. Select the Oracle VM Server in the Assets section.
  2. Click Attach Networks in the Actions pane.

    The list of available networks in Oracle Enterprise Manager Ops Center are displayed. The list also displays the existing number of connections with the server.

  3. Select one or more networks from the list. You can make multiple connections to a network.

    Click Next.

  4. Specify the Mode for networks configured with VLAN ID and the number of connections for each selected networks. Increase the total number of connections.

    If required, the VLAN ID of the network can be modified based on the condition that the network is being attached to the server for the first time in tagged mode. The VLAN ID can be already in use by the network attached to the server.

    Note:

    The number of connections does not limit the number of logical domains that can be connected to this network.

    Click Next.

  5. Select the domain in the Service Domain list from which the network interfaces are selected. The service domain can be the control domain, I/O domains, or root domains. Root domains and I/O domains OS must be managed by Oracle Enterprise Manager Ops Center.

    For control domain, it is listed as primary in the Service Domain list. Whereas for I/O domains and root domains, the names of the domains are listed.

    Note:

    When you select an network interface from an I/O domain or root domain, the virtual switch is created in the control domain.

  6. Specify the NIC and IP address for each network connection.

    For each network connection, you must provide a NIC. The virtual switch requires a physical interface to allow communication between the logical domains and external network. Follow these rules while assigning the NIC and IP address:

    • Specify the network interface or NIC. Specify the same NIC to different network when the network has different VLAN ID and every connection is in tagged mode. Otherwise, you cannot assign the same NIC to different networks.

    • If required, you can modify the network tagging mode specified in the previous step.

    • If there is a virtual switch available for the network, the virtual switch is displayed. Otherwise, enter a name for the virtual switch or leave it blank for the software to automatically create a name for the virtual switch using the default naming pattern.

    • Select an IP address allocation method according to your requirements:

      • When supported by the network, select Assign by DHCP to enable the system to automatically allocate the IP address.

      • Select Use Static IP to provide an IP address for the network connection.

      • Select Do Not Allocate IP for the IP address when you do not want to assign IP address to the selected network interface.

      • Select Do not Plumb Interface for the IP address when you do not want to assign IP address and plumb the selected network interface. This option is available for non SR-IOV enabled networks.

      Note:

      For network interfaces from I/O domains or root domains, the Address Allocation Method is automatically set to Do Not Allocate IP.

    Click Next.

  7. Review the summary and click Finish to attach the selected network to the Oracle VM Server.

5.10.7 Attaching SR-IOV Enabled Networks to Oracle VM Server

When you connect to network using SR-IOV enabled network interface, there is no virtual switch creation. An SR-IOV enabled network interface means that there are virtual functions created on the physical function of PCIe Endpoint device and you can assign the virtual functions to the logical domains. When you select SR-IOV option while attaching networks to the Oracle VM Server, only the interfaces on which the virtual functions are created are available for network configuration.

When you assign SR-IOV enabled networks to logical domains, you cannot migrate the domains. SR-IOV enabled networks are available only from control domain and root domain.

SR-IOV enabled networks on root domain are available only in the following conditions:

  • Oracle Solaris 11.1.4.5.0 or higher version is necessary for dynamic attach of networks.

  • Oracle VM Server for SPARC 3.1 version to support SR-IOV networks on root domain.

  • See the Related Resources for Oracle VM Server for SPARC section for hardware and firmware requirements for SR-IOV feature.

  1. Select the Oracle VM Server in the Assets section.
  2. Click Attach Networks in the Actions pane.

    The list of available networks in Oracle Enterprise Manager Ops Center are displayed. The list also displays the existing number of connections with the server.

  3. Select one or more networks from the list. You can make multiple connections to a network.

    Click Next.

  4. Specify the number of connections for the selected networks. Increase the total number of connections.

    Note:

    The number of connections does not limit the number of logical domains that can be connected to this network.

  5. Select the domain in the Domain list from which the network interfaces are selected. If the network interface is from the control domain, then it displays as primary. For network interfaces from the root domain is listed with the name of the domain.
  6. Select the option SR-IOV and the NIC list is populated with the NICs that are SR-IOV enabled in the selected domain. The NICs that are defined as a physical function are listed.
  7. Skip the option to define the IP address for the network interface. You can define the IP address in the logical domain OS later.

    Click Next.

  8. Review the summary and click Finish to attach the selected network to the Oracle VM Server.

5.10.8 Network Options

To manage the attached networks, use the icons available in the Network tab of the Oracle VM Server for SPARC, as shown in Figure 5-6.

  • Refresh DHCP connectivity: Reconnects to the DHCP server.

  • Unbind networks: Disconnects the selected network from Oracle VM Server.

  • Modify physical connectivity: Changes the connectivity attributes of the selected network, as shown in Figure 5-7.

    Figure 5-7 Modify Physical connectivity

    Description of Figure 5-7 follows
    Description of "Figure 5-7 Modify Physical connectivity"

5.10.9 Unbinding Networks from Oracle VM Server for SPARC

You can remove the network connection from Oracle VM Server for SPARC. The removal of a network connection does not remove the virtual switch. You must select the appropriate virtual switch or the physical function to remove the network connection.

  1. Select the Oracle VM Server or control domain in the Assets section.
  2. Select the Networks tab in the center pane.

    The list of networks that are connected to Oracle VM Server are displayed.

  3. Select the network that you want to unbind from Oracle VM Server. You can unbind only one network at a time.
  4. Click the Unbind Network icon.

    The Unbind Network Wizard is displayed.

  5. In the Remove Network Connection step, the number of New Connections Count is automatically reduced by one and displayed. If you want to remove more than one network connection, enter the number of New Connections Count after the removal of the required network connections.

    Click Next.

  6. Select the virtual switch or the physical function that connects the network to the physical interface.

    Ensure to select the correct virtual switch on the interface and click Next.

  7. Review the summary of the network connection to be removed and click Unbind to remove the network connection.

Note:

You cannot detach a network from Oracle VM Server if the network is connected to a virtual switch that is already in use.

5.10.10 Maximum Transmission Unit (MTU) Size

The default size of the network's MTU is 1500 bytes. A network can be created with MTU size that varies between 576 and 9216. Oracle Enterprise Manager Ops Center configures the IP configuration of Oracle VM Server for SPARC with the MTU value set at the network level.

When a network's MTU is modified and the network is attached a control domain and its logical domains, you must reboot the control domain for changes to take effect. You must also shut down and start the running logical domains attached to the network.

For a logical domain, the network to be attached must have minimum MTU size as 1500 bytes. The network interface cards driver configuration must be updated to support MTU size greater than or equal to 9216 in the following scenario:

  • You have VLAN tagged networks created over the virtual switches with Link Aggregation as the uplink.

  • You are using Oracle Solaris 11 OS.

Otherwise, the tagged networks become unreachable, and Oracle VM Server and logical domains cannot connect to these networks.

5.11 About Logical Domains

A logical domain is a virtual machine that has its own operating system and identity within a single SPARC server. Each logical domain can be created, destroyed, reconfigured, and rebooted independently, without requiring the server to be powered off. You can run a variety of applications in different logical domains to keep them independent for performance and security purposes.

Using Oracle Enterprise Manager Ops Center, you can create logical domains and provision Oracle Solaris OS on them. Using profiles and deployment plans, you can create more than one logical domain simultaneously and then save the configuration for future use.

You can use one of the following methods to create logical domains:

  • Create Logical Domain Profile and Plan

    1. Create a profile which defines the configuration of the logical domain.

    2. Using the profile, create a deployment plan.

    3. Apply the plan on an Oracle VM Server to create logical domains.

    4. The logical domains do not have the OS installed. You must select each logical domain and apply an OS provisioning plan to install the OS. See Provisioning OS on Logical Domains for provisioning OS on the logical domains.

  • Configure and Install Logical Domains

    This is a complex plan which contains deployment plans that create logical domains and install Oracle Solaris OS on them. You must have the required profiles and deployment plans available to create the complex plan. The outline to create a plan is as follows:

    1. Create a logical domain profile.

    2. Create an OS provisioning profile to install the OS.

    3. Create a deployment plan to install the OS and other updates if any.

    4. Create the Configure and Install Logical Domains plan.

You can now manage logical domains that were created manually using the native CLI in Oracle Enterprise Manager Ops Center. See Discovering Existing Oracle VM Server for SPARC Environments for more information.

5.11.1 Selection of Domains Types

Oracle Enterprise Manager Ops Center provides options to select the type of logical domain in the logical domain profile creation.

You can select the following subtypes:

  • Physical I/O domain

  • Root domain

  • Guest domain

  • HA Guest domain

Select the appropriate subtype in the profile for creating logical domains.

5.12 Create Logical Domains

You can create logical domains with the following roles in Oracle Enterprise Manager Ops Center:

5.12.1 Plan Your Domain Configuration

A logical domain profile captures the requirements and configuration of a logical domain, that includes the CPU Threads or whole-cores, memory, storage, and network details. You must provision Oracle Solaris OS on each domain separately. As an alternative, you can also use the complex plan Configuring and Installing Logical Domains that includes the OS provisioning profile.

Ensure that you have the following information before you create a profile:

  • System Requirements: For creating I/O domains and root domains, the hardware and firmware requirements must be met. See the Related Resources for Oracle VM Server for SPARC section for more information about the hardware, firmware and software requirements for different versions of Oracle VM Server for SPARC and hardware.

  • CPU Resource Allocation: Virtual or Whole-Core

    Each SPARC processor has multiple cores and each core has multiple CPU threads (virtual CPUs). The CPU threads are grouped into cores. For example, there are 4 Threads per core in the Oracle Sun Fire and SPARC Enterprise T1000 CMT processors.

    To a logical domain, you can either select to allocate the CPU resource as virtual CPU or as Whole-core, the default value is Whole-core. When you select to allocate as whole core, the specified number of cores and all their CPU threads are allocated to the logical domain.

  • Virtual CPU Allocation

    The physical CPUs of the Oracle VM Server for SPARC are shared among the CPU threads of all the logical domains. Each logical domain requires:

    • At least one GB of memory, the default value is 4 GB.

    • At least one CPU thread.

      Each CPU thread can be allocated independently to logical domain. Some hardware resources are provided on a per-core basis and therefore, shared between the threads in the core. When the threads in a core are allocated to two or more separate logical domains, it can lead to suboptimal performance of those threads. To get the best performance, it is best to avoid allocating the threads of a core to many logical domains. The best approach would be create large logical domains first, the logical domains with complete cores, and then the smaller logical domains.

  • Whole-Core CPU Allocation

    The CPU cores are allocated to a logical domain rather than virtual CPUs. You can also limit the maximum number of CPU cores that must be allocated to a logical domain. By default all logical domains are created for maximum throughput by tuning CPU cores to use a maximum number of CPU Threads.

  • CPU Architecture

    The CPU architecture is of importance when you want to migrate the guest domains between systems that have different CPU processor type. The following are the supported types for CPU architecture:

    • native: This is the default value which enables the logical domains to be migrated only between systems that have the same CPU architecture type.

    • generic: Use this type when you want to enable the logical domain migration to take place independent of the CPU type of the systems. Though this might result in reduced performance, it provides an increased flexibility to migrate the domains between systems that have different CPU types.

  • Crypto Units

    Crypto units are assigned based on CPU thread assignments. You can request the number of Crypto units to be assigned to the logical domain. However, the number of Crypto units assigned might be different than the amount requested because we can only allocate a Crypto unit for every given number of CPU thread allocation, depending on the server hardware. After the creation of logical domain, view the job notification to see the actual number of Crypto Units assigned to the logical domain. You can also edit the logical domain configuration, storage, and network resource configuration later.

  • PCIe Buses

    You must release the PCIe buses from the control domain. Only, then you can assign the PCIe buses to the root domain. When you configure and deploy Oracle VM Server for SPARC, you can select the option Detach Unused Buses from the control domain. Refer to Creating an OS Configuration Profile for Oracle VM Server for SPARC for more information. Otherwise, you must manually detach the buses to be used to create root domains.

  • Storage resources

    The libraries for virtual disk can be local, local device, NAS, SAN, or Dynamic Storage libraries. The default value for storage size of a logical domain is 20 GB. If Oracle Solaris 11.2 is later installed, the OS provisioning job might fail if the storage size is less than 20GB.

    Oracle Enterprise Manager Ops Center provides multipathing configuration for logical domains. For each virtual disks that have more than one path to access the back-end storage, a multipathing group is created.

    Oracle Enterprise Manager Ops Center provides storage redundant access for logical domains. You can enable virtual disk multipathing to access the back-end storage of the domain by more than one path. Provide the name of the multipathing group and select the alternate path to access the virtual disk.

    The virtual disk of a logical domain remains accessible even if one of the service domains go down. For each virtual disks of the logical domain that has more than one alternate path, a new multipathing group is created. You must always specify the virtual disk of another service domain for the alternate path to access the back-end storage. You can select to enter the name of the multipathing group or a group is created automatically in the format of logical domain name_mpGroup_devID. The devID is the disk index. For example, if the name of the logical domain is ldom, then the multipathing group name is in the format of ldom_mpGroup_1.

    When you select more than one alternate path to the virtual disk, you must select which is the active path during the failure of the main storage path.

    When the virtual disk has only one alternate path to access the back-end storage, multipathing group is not created automatically unless you enter the name of the group.

  • Network resources and number of connections for a network

    You can connect a network multiple times to the logical domain. When you create a logical domain, the number of connections for a network translate to the number of virtual network devices created for the logical domain. A vnet is created for each connection of the guest to the network. Vnets are re-used when the underlying virtual switch is used for networks with different VLAN IDs.

    When you start a logical domain, you must define the virtual switches or the virtual functions through which you connect the logical domain to the external network. If the root domain or I/O domain has connection to the physical network interfaces, you can use the direct physical network connection.

  • Automatic Recovery

    Select the automatic recovery option to recover the guest domains when the server hardware fails. This option is helpful when the Oracle VM Server is placed in a server pool and the guest domains can be recovered on the other Oracle VM Servers in the pool. If selected, enter the value for Priority of Recovery. The value must be between 0 and 100. A guest domain with a higher value is recovered first.

    The guest domains are created with virtual disk multipathing configuration by providing the alternate path to access the back-end storage. During server hardware failure in a server pool, Oracle Enterprise Manager Ops Center tries to recover the guest domains on server that provides redundant access to the storage. You can select the option Authorize Recovery without Redundant I/O to authorize the recovery of the logical domains on other servers without redundant storage access in the server pool. The configuration of the guest domain is changed to lose the redundant storage access. If you do not select this option and there are no servers with redundant storage access, the automatic recovery is not performed.

5.12.2 About Root Domains

Root domain is the domain which has the entire PCIe bus also known as root complex assigned to it.

The entire PCIe bus consists of the PCIe bus and all its switches and devices. When you assign a PCIe bus to the root domain, all the devices on that bus are owned by the domain. You cannot directly assign the PCIe Endpoint devices on that bus to any other domains. You must release the PCIe Endpoint devices so that it can be allocated to other I/O domains.

The root domain is non-migratable domain and the domain metadata is stored in the local file system.

The root domain can provide virtual I/O services to guest domains. When you create a root domain, you can define the name of the virtual disk storage server that provide virtual disk storage to guest domains. The names are checked for uniqueness during creation.

5.12.3 Creating a Root Domain Profile

  1. Select the Plan Management section in the Navigation pane.
  2. Expand Profiles and Policies and select Logical Domain from the list.
  3. Click Create Profile from the Actions pane.

    The Create Logical Domain Profile Wizard is displayed.

  4. Provide a name and description to identify the profile and select Root Domain as the role for the logical domain.

    The option Create a deployment plan for this profile is already selected and creates a plan using this profile. If required, you can deselect this option.

    Click Next to specify the domain identity.

  5. Provide a name and the starting number for the logical domain.

    Using this profile, you can create more than one logical domain. To identify each new logical domain, provide a prefix start name and starting number. For example, for a Start Name defined as TestDomain and a starting number of 10, three new logical domains have the name TestDomain10, TestDomain11, and TestDomain12.

    To avoid number in the suffix, you can add the number in between the name using the wild card ‘%?d', where ‘?' is the number of digits displayed.

    Click Next to configure the CPU and memory allocation for the domain.

  6. Select one of the following options for CPU resource allocation:
    • Virtual CPU: You allocate CPU Threads to the root domain. Enter the number of CPU Threads that must be allocated to the root domain.

    • Whole-core: You allocate whole cores to the root domain. Provide the following data for whole-core allocation:

      • CPU Cores: Enter the number of CPU cores that you want to allocate to the root domain.

      • Max CPU Cores: The maximum number of cores that you can assign to an active domain.

  7. (Optional) Enter the number of Crypto Units to be assigned to the root domain.

    The number of allocated Crypto Units might be different depending on the CPU Threads allocated and the server hardware. Check the job details for viewing the actual number of Crypto Units assigned to the logical domain.

    Note:

    The Crypto Units are not applicable from SPARC T4 server onwards and the values are ignored.

  8. Enter the size of the memory to be allocated to the logical domain. The root domain requires at least four GB of memory per I/O device.

    Click Next to specify the I/O buses.

  9. Enter the number of PCIe buses to be assigned to this domain. The number depends on the number of PCIe root complexes available on the hardware platform.

    Click Next to specify the library for virtual storage disks of the root domain.

  10. The root domains are non-migratable as it is an I/O domain that has direct access to physical I/O devices. Therefore, the root domain metadata is stored in the local filesystem.
  11. (Optional) Select one or more libraries that form the logical domain virtual disks. The libraries for virtual disk can be local, local device, NAS, SAN, or Dynamic Storage libraries.

    Note:

    It is recommended not to allocate virtual devices to root domain.

    Any physical storage or network devices attached to the root domain are made available and presented during OS provisioning.

    Click Next to specify the networks for the domains.

  12. (Optional) Select the network from the list of networks identified by Oracle Enterprise Manager Ops Center. You can associate a root domain with more than one network.

    You can use this step to attach networks using vnets. Skip this step if you want to connect to the networks available through the physical I/O devices. The physical connectivity is presented during OS provisioning.

  13. (Optional) Enter the number of connections for each network.

    You can connect to a network multiple times to the logical domain. When you create a logical domain, the number of connections for a network translate to the number of virtual network devices (vnet) created for the logical domain. Oracle Enterprise Manager Ops Center re-uses the vnets and reduces the necessity to create vnets for each network connection.

    When you start a logical domain, you must define the virtual switches or the virtual function of SR-IOV enabled network interface through which you connect the logical domain to the external network.

    Click Next to view the summary of the details selected for creating a logical domain.

  14. Review the information and click Finish to save the profile.

    When you want to create a deployment plan with this profile in the first step, then a corresponding logical domain plan is also created.

5.12.4 Applying Deployment Plan for Creating a Root Domain

You can create root domains on Oracle VM Servers that are stand-alone or placed in a server pool. When you want to select Oracle VM Servers placed in a server pool, remove the filter option in the Select Target Assets window and then select the Oracle VM Server which can be placed in a server pool.

When the root domain has access to storage devices, then it can provide storage services to other guest domains. For providing storage services to other domains, a virtual disk server (vds) must be configured on the root domain. vds is required to export virtual disk devices to other domains. During root domain creation, a single vds is created by default with the default name <domain name>-vds0. You can change the name of the vds or retain the default name. You can also configure as many vds on the root domain. When you migrate guest domains that are utilizing the storage services from the root domain, the vds must be available to the target server.

  1. Select the deployment plan created for root domain creation.
  2. Click Apply Deployment Plan in the Actions pane.

    The Select Target Assets window is displayed.

  3. The Oracle VM Server for SPARC servers are listed. To list the Oracle VM Servers placed in server pools, deselect the filter option.

    Select the targets and click Add Targets to List.

  4. Select Apply with minimum interaction to apply the plan with the selected profile values. Otherwise, select Allow me to override any profile values to change the profile values.

    Click Next to specify the resource assignments for root domain.

  5. If required, modify the name of the logical domains starting name and the number appended to it.

    Click Next to configure the CPU Threads and memory.

  6. Edit the CPU model, and memory as required.

    Click Next to specify the I/O buses to the domain.

  7. Select the PCIe buses that have been released from the control domain.

    Click Next to specify the storage resource assignments and virtual disk server name.

  8. (Optional) A default virtual disk server name is automatically set in the Name of the Virtual Disk Server to be created in the format of <domain name>-vds0. You can select to modify the name. The vds is to provide storage services to other domains.
  9. (Optional) If required, you can add virtual disks to the root domain.

    The root domain is not migratable and therefore, the root domain metadata is automatically stored in the local library of the control domain.

    When the storage resources from the profile are not available for the selected target, it is flagged in red color. Modify the storage resources accordingly. Provide the name of the multipathing group and select the virtual disk server of the service domains and the active path. When a group name is not provided, a multipathing group is automatically created with the default naming procedure.

    Click Next to specify the network resource assignments.

  10. (Optional) If you want to attach networks using vnets, use this step to provide the network connection. Select the networks and define the network resource assignment.

    When the network resource assignments from the profile are not available for the selected target, then it is flagged in red color. Modify the network resources accordingly in the profile or apply the plan in override any profile values mode.

    Click Next to schedule the job.

  11. Schedule the job to run now or at a later time. Click Next.
  12. Review the summary and click Apply to execute the deployment plan on the selected targets.

When you provision OS on the root domain, virtual functions are automatically created on the SR-IOV enabled PCIe buses that are allocated to the root domain.

5.12.5 Creating a Physical I/O Domain Profile

An I/O domain has direct access to and ownership of physical I/O devices. In Oracle Enterprise Manager Ops Center, you can create I/O domains by assigning PCIe Endpoint devices.

In the Logical Domain profile, select Physical I/O Domain subtype. In this guide, the I/O domains refer to the domains assigned with PCIe Endpoint devices only.

A domain that is assigned with a virtual function from SR-IOV enabled network interfaces is also defined as an I/O domain in Oracle VM Server for SPARC.

In Oracle Enterprise Manager Ops Center, to create domains assigned with virtual functions, you must use the Guest Domain subtype in the Logical Domain profile.

The number of I/O domains that you can create is limited by the number of PCIe Endpoint devices.

Note:

You cannot migrate I/O domains as they are assigned with PCIe Endpoint devices.

This section describes about creating I/O domains that are assigned with PCIe Endpoint devices.

  1. Select the Plan Management section in the Navigation pane.
  2. Expand Profiles and Polices and select Logical Domain from the list.
  3. Click Create Profile from the Actions pane.

    The Create Logical Domain Profile Wizard is displayed.

  4. Provide a name and description to identify the profile. The option Create a deployment plan is already selected for this profile and creates a plan using this profile. If required, you can deselect this option.
  5. Select Physical I/O Domain from the Subtype.

    Click Next to specify the domain identity.

  6. Provide a name and the starting number for the logical domain.

    Using this profile, you can create more than one logical domain. To identify each new logical domain, provide a prefix start name and starting number. For example, for a Start Name defined as TestDomain and a starting number of 10, three new logical domains have the name TestDomain10, TestDomain11, and TestDomain12.

    To avoid number in the suffix, you can add the number in between the name using the wild card ‘%?d', where ‘?' is the number of digit displayed.

    Click Next to configure the CPU and memory allocation for the I/O domain.

  7. Select one of the following options for CPU resource allocation:
    • Virtual CPU: You define the CPU resource allocation in CPU Threads to the I/O domain. Enter the number of CPU Threads that must be allocated to the I/O domain.

    • Whole-core: You allocate whole cores to the I/O domain. Provide the following data for whole-core allocation:

      • CPU Cores: Enter the number of CPU cores that you want to allocate to the I/O domain.

      • Max CPU Cores: The maximum number of cores that can be assigned to an active domain.

  8. Provide the values for Crypto units and memory allocation:
    • Memory: Enter the size of the memory to be allocated to the I/O domain.The size of the memory must be set according to the system size and load. The default is 4 GB and the minimum value is 1 GB.

    • Requested Crypto Units: Though you can enter the number of Crypto Units to be assigned to the logical domain, the number might be different depending on the CPU threads allocated and the server hardware. Check the job details for viewing the actual number of Crypto Units assigned to the logical domain.

    Click Next to specify the PCIe Endpoint devices for the I/O domain.

  9. Select the type of PCIe Endpoint device and the number of devices. Select the type of device to filter only those I/O resources attached to it during resource assignment:
    • Ethernet Device

    • InfiniBand Device

    • Fibre Channel Device

    • SCSI Device

    Select Any type when you do not want to filter the I/O resources attached to the PCIe Endpoint.

    Click Next to specify the library for the storage disks for the logical domain.

  10. (Optional) Select one or more libraries that form the I/O domain storage disks.

    The libraries for virtual disk can be local, local device, NAS, SAN, or Dynamic Storage libraries. You can skip this step when you want to provide the storage disks from the directly attached storage resources from the device.

    Click Next to specify the networks for the domains.

  11. (Optional) Select at least one network from the list of networks identified by Oracle Enterprise Manager Ops Center. You can associate a logical domain with more than one network.

    Enter the number of connections for each network. You can connect to a network multiple times to the logical domain. When you create a logical domain, the number of connections for a network translate to the number of virtual network devices created for the logical domain. When you start a logical domain, you must define the virtual switches or the virtual functions of SR-IOV enabled networks through which you connect the logical domain to the external network.

    Skip this step when you want to use the networks available from the physical devices attached to the I/O domain. You can define the physical connectivity during OS provisioning.

    Click Next to view the summary of the details selected for creating an I/O domain.

  12. Review the information and click Finish to save the profile.

    When you want to create a deployment plan with this profile in the first step, then a corresponding logical domain plan is also created.

5.12.6 Applying Deployment Plan for Creating an I/O Domain

When the I/O domain has access to storage devices, then it can provide storage services to other guest domains. For providing storage services to other domains, a virtual disk server (vds) must be configured on the I/O domain. vds is required to export virtual disk devices to other domains.

During I/O domain creation, a single vds is created by default with the default name <domain name>-vds0. You can change the name of the vds or retain the default name. You can also configure as many vds on the I/O domain. When you migrate guest domains that are utilizing the storage services from the I/O domain, the vds must be available to the target server.

  1. Select the deployment plan created for I/O domain creation.
  2. Click Apply Deployment Plan in the Actions pane.

    The Select Target Assets window is displayed.

  3. The Oracle VM Server for SPARC servers that are available and eligible are listed. Select the targets and click Add Targets to List.
  4. Select Apply with minimum interaction to apply the plan with the selected profile values. Otherwise, select Allow me to override any profile values to change the profile values.

    Click Next to specify the resource assignments for root domain.

  5. If required, modify the name of the logical domains starting name and the number appended to it.

    Click Next to configure the CPU Threads and memory.

  6. Edit the CPU model, and memory as required.

    Click Next to specify the PCIe Endpoint devices to be assigned to the domain.

  7. The selected PCIe Endpoint Type and the corresponding PCIe Endpoint devices that have been released from the control domain or the root domain are listed.

    Select the PCIe Endpoint devices that you want to assign to the I/O domain.

    Click Next to specify the storage resource assignments and virtual disk server name.

  8. (Optional) A default virtual disk server name is set in the format of <domain name>-vds0 for the Name of the Virtual Disk Server to be created. You can select to modify the name. The vds is for providing virtual disk services to other domains.

    You can also edit the assignment of your storage resources.

  9. (Optional) If required, you can add virtual disks to the I/O domain.

    The I/O domains are not migratable as they are attached to physical I/O devices and therefore, the domain metadata is automatically stored in the local library of the control domain.

    When the storage resources from the profile are not available for the selected target, it is flagged in red color. Modify the storage resources accordingly. Provide the name of the multipathing group and select the virtual disk server of the service domains and the active path. When you do not provide the group name and there is more than one alternate path to the virtual disk, a multipathing group is automatically created with the default naming procedure.

    Click Next to specify the network connection settings.

  10. (Optional) All network connection require a virtual switch or virtual function to connect the network to the logical domain. Select the following options for each network connection:
    • SR-IOV: Select this option if you want to connect to the networks using the virtual function.

    • Mode: This option is applicable for networks that are configured with a VLAN ID. Select Untagged or Tagged mode from the list.

    Click Next to specify the network resource assignments.

  11. In the network resource assignment, specify the domain that provides the network interface, and the virtual switch or the virtual function for the network connection.

    Note:

    You cannot edit the SR-IOV option in this step. Click Back to edit the SR-IOV option.

    Click Next to schedule the job.

  12. Schedule the job to run now or at a later time. Click Next.
  13. Review the summary and click Apply to execute the deployment plan on the selected targets.

5.12.7 Creating a Guest Domain Profile

  1. Select the Plan Management section in the Navigation pane.
  2. Expand Profiles and Policies and select Logical Domain from the list.
  3. Click Create Profile from the Actions pane.

    The Create Logical Domain Profile Wizard is displayed.

  4. Provide a name and description to identify the profile. The option Create a deployment plan is already selected and creates a plan using this profile. If required, you can deselect this option.
  5. Select Guest Domain from the Subtype.

    Click Next to specify the domain identity.

  6. Provide a name and the starting number for the logical domain.

    Using this profile, you can create more than one logical domain. To identify each new logical domain, provide a prefix start name and starting number. For example, for a Start Name defined as TestDomain and a starting number of 10, three new logical domains have the name TestDomain10, TestDomain11, and TestDomain12.

    To avoid number in the suffix, you can add the number in between the name using the wild card ‘%?d', where ‘?' is the number of digit displayed.

    Click Next to configure the CPU and memory allocation for the domain.

  7. Select one of the following options for CPU resource allocation:
    • Virtual CPU: You allocate CPU Threads to the logical domain. Enter the number of CPU Threads that must be allocated to the logical domain.

    • Whole-core: You allocate whole cores to the logical domain. Provide the following data for whole-core allocation:

      • CPU Cores: Enter the number of CPU cores that you want to allocate to the logical domain.

      • Max CPU Cores: The maximum number of cores that can be assigned to an active domain.

  8. Select the CPU architecture, Crypto units, memory allocation, and automatic recovery options:
    • CPU architecture: Select the architecture as native or generic according to your requirement to migrate the logical domains.

    • Memory: Enter the size of the memory allocated to the logical domain. The size of the memory must be set according to the system size and load. The default is 4 GB and the minimum value is one GB.

    • Requested Crypto Units: Though you can enter the number of Crypto Units to be assigned to the logical domain, the number might be different depending on the CPU threads allocated and the server hardware. Check the job details for viewing the actual number of Crypto Units assigned to the logical domain.

    • Automatic Recovery: Select whether you want the logical domain to be recovered automatically when the server hardware fails. If selected, enter the value for Priority of Recovery. The value must be between 0 and 100. A logical domain with a higher value is recovered first.

    Click Next to specify the library for storing the domain metadata and storage disks for the logical domain.

  9. Select a library to store the logical domain metadata.

    Select local storage library or NAS library to store the domain metadata.

  10. Select one or more libraries that form the logical domain storage disks.

    The libraries for virtual disk can be local storage, local device, NAS, SAN, or Dynamic Storage libraries. Click Next to specify the networks for the domains.

  11. Select at least one network from the list of networks identified by Oracle Enterprise Manager Ops Center. You can associate a logical domain with more than one network.
  12. Enter the number of connections for each network.

    You can connect to a network multiple times to the logical domain. When you create a logical domain, the number of connections for a network translate to the number of virtual network devices created for the logical domain.

    When you start a logical domain, you must define the virtual switches or the virtual functions through which you connect the logical domain to the external network.

  13. Click Next to view the summary of the details selected for creating a logical domain.
  14. Review the information and click Finish to save the profile.

    When you want to create a deployment plan with this profile in the first step, then a corresponding logical domain plan is also created.

5.12.8 Applying Deployment Plan for Creating a Guest Domain

Apply the created logical domain deployment plan on the target server and create the logical domains.

  1. Expand Deployment Plans in the Plan Management section.

    and select the created logical domain deployment plan.

  2. Select Create Logical Domain in the deployment plans.
  3. Click Apply Deployment Plan in the Actions pane.

    The Select Target Assets window is displayed.

  4. Select the target server from the list.

    Select one or more targets to apply the plan.

  5. Click Add to Target List option to add the selected targets on which you want to apply the plan.
  6. Select whether you want to run the plan with minimal interaction or you want to override the profile values.
  7. Click Next to specify the resource assignments in the profile.
  8. If required, modify the name of the logical domains starting name and the number appended to it.

    Click Next to configure the CPU Threads and memory.

  9. Edit the CPU Threads, memory, and automatic recovery as required.

    Click Next to specify the storage resource assignments.

  10. When the storage resources from the profile are not available for the selected target, it is flagged in red color. Modify the storage resources accordingly. Provide the name of the multipathing group and select the virtual disk server of the service domains and the active path. When you do not provide the group name and the virtual disk has more than one alternate path, a multipathing group is automatically created with the default naming procedure.

    Click Next to specify the network connection settings.

  11. You can assign a virtual function or a vnet to the guest domain for the network connection.

    Select the following options for each network connection:

    • SR-IOV: Select this option when you want to assign a virtual function to connect to the network.

    • VLAN ID / P-KEY: Specify the VLAN ID or P-KEY for the network.

    • Mode: If the network is assigned a VLAN ID, you can select the network to be connected in tagged or untagged mode.

    Click Next to specify the network resource assignments.

  12. You can select the domain which provides the network interface for the network connection. Select the following details for each network connection:
    • SR-IOV: Select this option when you want to assign a virtual function to connect to the network.

    • Service domain: Select the service domain that provides the network interface for the network connection. The domain can be primary, I/O domains or the root domains.

    • Map Connection: Select the virtual switch through which you want to connect the guest domain to the network. For SR-IOV enabled network connection, select the physical function through which you want to connect to the network.

    • VLAN ID / P-KEY: Specify the VLAN ID or P-KEY for the network.

    • Mode: If the network is assigned a VLAN ID, you can select the network to be connected in tagged or untagged mode.

    • Link Mode: Select the link mode for the network connection. In Automatic link mode, the data link is automatically generated by Ops Center. In Manual link mode, you specify the data link resource information.

    Click Next .

  13. If you selected manual link mode for at least one network, the data link resource assignments are displayed. Select the following details for each network connection:
    • Lower Link Name: Specify the lower link name for the network connection.

    • Link Name: Specify the VNet name for the network connection.

    • Nb Alt Mac Addrs: Specify a number of alternate MAC addresses for the network connection.

    • Max Bandwidth (Mbps): Specify the maximum bandwidth for the network connection in megabits per second.

    • Private VLAN ID: If the network connection is in untagged mode, you can specify a private VLAN ID for the network connection.

    • Private VLAN Type: If you provided a private VLAN ID, select a private VLAN type, isolated or community.

    Click Next .

  14. Schedule the job to run now or at a later time. Click Next.
  15. Review the summary and click Apply to execute the deployment plan on the selected targets.

5.12.9 Creating HA Guest Domain Profile

The guest domain has the following high available features defined in the profile:

  • Two network connections for each networks

  • Redundant storage access to the virtual disks

  • Automatic recovery of the guests during server failure in a server pool

The HA guest domain profile requires that you provide at least two connections for each selected network.

  1. Select the Plan Management section in the Navigation pane.
  2. Expand Profiles and Policies and select Logical Domain from the list.
  3. Click Create Profile from the Actions pane.

    The Create Logical Domain Profile Wizard is displayed.

  4. Provide a name and description to identify the profile. The option Create a deployment plan for this profile automatically creates a plan using this profile. If required, you can deselect this option.
  5. Select Guest Domain from the Subtype.

    Click Next to specify the domain identity.

  6. Provide a name and the starting number for the logical domain.

    Using this profile, you can create more than one logical domain. To identify each new logical domain, provide a prefix start name and starting number. For example, for a Start Name defined as TestDomain and a starting number of 10, three new logical domains have the name TestDomain10, TestDomain11, and TestDomain12.

    To avoid number in the suffix, you can add the number in between the name using the wild card ‘%?d', where ‘?' is the number of digit displayed.

    Click Next to configure the CPU and memory allocation for the domain.

  7. Select one of the following options for CPU resource allocation:
    • Virtual CPU: You allocate CPU Threads to the logical domain. Enter the number of CPU Threads that must be allocated to the logical domain.

    • Whole-core: You allocate whole cores to the logical domain. Provide the following data for whole-core allocation:

      • CPU Cores: Enter the number of CPU cores that you want to allocate to the logical domain.

      • Max CPU Cores: The maximum number of cores that can be assigned to an active domain.

  8. Select the CPU architecture, Crypto units, memory allocation, and recovery options:
    • CPU architecture: Select the architecture as native or generic according to your requirement to migrate the logical domains.

    • Memory: Enter the size of the memory allocated to the logical domain. The size of the memory must be set according to the system size and load. The default is 4 GB and the minimum value is one GB.

    • Requested Crypto Units: Though you can enter the number of Crypto Units to be assigned to the logical domain, the number might be different depending on the CPU threads allocated and the server hardware. Check the job details for viewing the actual number of Crypto Units assigned to the logical domain.

    • Automatic Recovery: Select whether you want the logical domain to be recovered automatically when the server hardware fails. When you select this option, provide the following details: If selected, enter the value for Priority of Recovery. The value must be between 0 and 100. A logical domain with a higher value is recovered first.

      • Priority of Recovery: The value for priority of recovery must be between 0 and 100. A guest domain with a higher value is recovered first.

      • Authorize recovery without redundant I/O: Select this option when you want to recover the guest domain in a server pool even when there is no redundant storage access in the other servers in the pool.

    Click Next to specify the library for storing the domain metadata and storage disks for the logical domain.

  9. Select a library to store the logical domain metadata.

    Select only the local storage or NAS library to store the domain metadata.

  10. Select one or more libraries that form the logical domain storage disks.

    The libraries for virtual disk can be local storage, local device, NAS, SAN, or Dynamic Storage libraries. Click Next to specify the networks for the domains.

  11. Select at least one network from the list of networks identified by Oracle Enterprise Manager Ops Center. You can associate a logical domain with more than one network.
  12. Enter the number of connections for each network. You must enter at least two connections for each network.

    You can connect to a network multiple times to the logical domain. When you create a logical domain, the number of connections for a network translate to the number of virtual network devices created for the logical domain.

    When you start a logical domain, you must define the virtual switches or the virtual functions through which you connect the logical domain to the external network.

  13. Click Next to view the summary of the details selected for creating a logical domain.
  14. Review the information and click Finish to save the profile.

    When you want to create a deployment plan with this profile in the first step, then a corresponding logical domain plan is also created.

5.12.10 Applying Deployment Plan for Creating a HA Guest Domain

Apply the created logical domain deployment plan on the target server and create the logical domains.

  1. Expand Deployment Plans in the Plan Management section and select the created logical domain deployment plan.
  2. Click Apply Deployment Plan in the Actions pane.

    The Select Target Assets window is displayed.

  3. Select the target server from the list.

    Select one or more targets to apply the plan.

  4. Click Add to Target List option to add the selected targets on which you want to apply the plan.
  5. Select whether you want to run the plan with minimal interaction or you want to override the profile values.
  6. Click Next to specify the resource assignments in the profile.
  7. If required, modify the name of the logical domains starting name and the number appended to it.

    Click Next to configure the CPU Threads and memory.

  8. Edit the CPU Threads, memory, and automatic recovery as required.

    Click Next to specify the storage resource assignments.

  9. When the storage resources from the profile are not available for the selected target, it is flagged in red color. Modify the storage resources accordingly. Provide the name of the multipathing group and select the virtual disk server of the service domains and the active path. When you do not provide the group name and the virtual disk has more than one alternate paths, a multipathing group is automatically created with the default naming procedure.

    Click Next to specify the network connection settings.

  10. You can assign a virtual function or a vnet to the guest domain for the network connection.

    Select the following options for each network connection:

    • SR-IOV: Select this option when you want to assign a virtual function to connect to the network.

    • Mode: If the network is assigned a VLAN ID, you can select the network to be connected in tagged or untagged mode.

    Click Next to specify the network resource assignments.

  11. You can select the domain which provides the network interface for the network connection. Select the following details for each network connection:
    • Service domain: Select the service domain that provides the network interface for the network connection. The domain can be primary, I/O domains or the root domains.

    • Map Connection: Select the virtual switch through which you want to connect the guest domain to the network. For SR-IOV enabled network connection, select the physical function through which you want to connect to the network.

    Click Next to schedule the job.

  12. Schedule the job to run now or at a later time. Click Next.
  13. Review the summary and click Apply to execute the deployment plan on the selected targets.

5.12.11 Selection of CPU Architecture

Newer platforms have a new class for the CPU architecture, named migration-class1 and sparc64-class1. The class1 architecture enables you to migrate the guest domains across systems while maintaining the full capabilities of the domains.

When you have a guest domain with the CPU architecture as generic in Oracle Enterprise Manager Ops Center and you start the guest in the following servers:

  • Oracle SPARC T3 and T2 Servers: Oracle Enterprise Manager Ops Center sets the CPU architecture to generic.

  • Oracle SPARC T4, T5, M5, and M6 servers: Oracle Enterprise Manager Ops Center sets the CPU architecture to generic (migration-class1). migration-class1 is a cross-CPU migration family for SPARC platforms starting with the SPARC T4 servers.

  • Fujitsu M10 Systems: Oracle Enterprise Manager Ops Center sets the CPU architecture to generic (sparc64-class1). sparc64-class1 is a cross-CPU migration family for SPARC 64 platforms. sparc64-class1 value has more instructions than generic value. This value is compatible only with Fujitsu M10 systems.

5.12.12 Creating a Deployment Plan for Installing Logical Domain

When you create a logical domain profile, you can select to create a deployment plan with that profile. You can also create logical domain plans. You can either create a simple plan to install only the logical domains or a complex plan to create the logical domains and provision OS on it.

The following procedure describes how to create a simple plan to install the logical domains only. You require to have created a logical domain profile to select in the plan.

  1. Expand Deployment Plans in the Plan Management section and select Create Logical Domains plan.
  2. Click Create Plan from Template in the Actions pane.

    The Create a Deployment Plan window is displayed.

  3. Enter a name and description for the plan.
  4. Select the failure policy.
  5. Select the logical domain profile in the Create Logical Domain step.

    This is a simple single step plan to select the logical domain profile. The logical domain profiles that are available are listed in the list of Associated Profile/Deployment Plan.

  6. Enter the number of logical domains to be created.
  7. Click Save to create the deployment plan.

The deployment plan is created with the associated profile. Now, apply the plan on a suitable target to create the logical domains.

5.13 Logical Domains Created Using CLI

You can also create logical domains using the native CLI. The new logical domain created is automatically discovered and displayed under the corresponding Oracle VM Server for SPARC in the UI. The discovered logical domain metadata is stored in the local library of the control domain and the virtual disk storage is defined as Opaque.

The logical domains cannot be migrated as such and it requires the following actions to enable the migrate option:

  • Move the metadata of logical domain to shared storage like NFS storage. See Moving Metadata to Another Library for complete procedure.

  • Enable sharing of the virtual disk storage of the logical domain.

5.14 Provisioning OS on Logical Domains

The following information is covered in this section:

The Oracle OS version that is provisioned on the logical domains is independent of the OS version on the Oracle VM Server for SPARC control domain. You can provision Oracle Solaris 10 or Oracle Solaris 11 OS on the logical domains irrespective of the Oracle Solaris OS version of the control domain.

After you create a logical domain, you must provision an OS on the domain.

  • Ensure that you have imported the appropriate ISO image into the library. If it is Oracle Solaris 10, then it must be at least Oracle Solaris 10 8/07.

  • Create or identify an OS provisioning profile for SPARC. See the Related Resources for Oracle VM Server for SPARC section for more information.

  • Create or identify a deployment plan with the OS provisioning and configuration profile and apply the deployment plan.

Note:

When you want to monitor the OS provisioning on the logical domains, you must first enable the logical domain console. See Connect to Logical Domain Console for enabling the console.

Oracle Enterprise Manager Ops Center provides OS provisioning and OS configuration profiles that capture the required parameters for provisioning OS. The profiles provide Logical Domain subtype that collects the parameters required for logical domain OS provisioning. You cannot use the Oracle Solaris OS provisioning profiles created for bare-metal provisioning.

The procedures to create the OS provisioning and configuration profiles especially for logical domains are described in this section.

5.14.1 Plan Your Network and Storage Resources

When you provision OS on the root domains and I/O domains, you can plan to provide the physical resources to the domains that are available from the PCIe Endpoint devices assigned to them. The physical network or the storage resources are presented while deploying the OS provisioning plan on the domains.

The storage resource can be either used by the domain or provide virtual disk server services to other guest domains. Whether the I/O domains are attached to a physical storage device or not, a virtual disk server service is created for the virtual disks added to it. You can modify the name of the Virtual Disk Server name during the deployment of OS provisioning plan.

For network resources, the OS provisioning deployment displays the network controllers that provides the Ethernet connection and the available network ports in the controller. The PCIe Endpoint devices that are provided with Ethernet devices and the network ports that are connected must be known before deploying the OS provisioning plan on the I/O domain or root domain. Check with your administrator for the network connection details in the server hardware.

5.14.2 Profiles and Deployment Plans for OS Provisioning

Oracle Enterprise Manager Ops Center provides the following profiles and deployment plans for OS provisioning on logical domains:

  • Profiles

    • OS Provisioning

    • OS Configuration

  • Deployment Plans

    • Provision OS

    • Install Server

    • Configure and Install Logical Domains

The Provision OS deployment plan is a simple deployment plan that consists of two steps to provision and configure OS. You use this plan to do OS provisioning on the already created logical domains.

Install Server deployment plan is a multi-step deployment plan that includes step to update the OS, and install any software apart from OS provisioning. You use this plan on the already created logical domains.

Configure and Install Logical Domains is a complex plan that includes the step to create logical domains and provision OS on them.

Depending on the requirement in your environment, choose the appropriate method to create logical domains and provision OS on them. Using the complex plan Configure and Install Logical Domains might show some differences in the boot interface assignment and network resource assignment steps. For example, when you apply a provisioning plan on already created root domain, the OS provisioning deployment plan filters the network controllers that provides the network device and the network ports that are available from that device. Whereas, when you use a complex plan such as Configure and Install Logical Domains, the network resources are not filtered for the network devices. Instead, all the PCIe Endpoint devices assigned to the domain are listed. You must enter the correct network port for the OS provisioning job.

Oracle Enterprise Manager Ops Center also provides the option to identify the network interface by providing the MAC address.

5.14.3 Creating an OS Provisioning Profile

The OS provisioning profile collects the following details:

  • OS image and version.

  • OS setup parameters such as time zone, language, console, terminal type, and root password for the OS.

  • File system layout.

  • Name services.

  • User account for Oracle Solaris 11

  1. Select the Plan Management section in the Navigation pane.
  2. Expand Profiles and Policies and select OS Provisioning profile.
  3. Click Create Profile in the Actions pane.

    The Create Profile - OS Provisioning Wizard is displayed.

  4. Provide the following details for the profile identification:
    • Provide a name and description for the profile identification.

    • Select Logical Domain as the Subtype.

    Click Next to specify the provisioning parameters.

  5. Select the OSP parameters:
    • Select the Oracle Solaris OS image from the list of available images. For Oracle Solaris 11 OS, select the SRU from the list.

    • Select the Software Group from the list.

    Click Next to specify the OS Setup.

  6. Specify the OS setup parameters:
    • Enter the time zone, language, terminal type, console serial port, and console baud rate.

    • Enter the root password.

    • The NFS4 domain is set to dynamic in this example. If a naming service is configured in your environment, enter the NFS4 domain value.

    Click Next to provide a user account for Oracle Solaris 11. Otherwise, skip to Step 8.

  7. Root login is not enabled in Oracle Solaris 11 OS. Create a user account to SSH to the OS after provisioning. Provide a user name and password for the account.

    Click Next to specify whether you want to use iSCSI disks for OS provisioning.

  8. Select the option Use iSCSI Disks if you want to use iSCSI for OS provisioning.

    You must provide the storage server IP address and the volume group during deployment.

    Click Next to specify the file system.

  9. The root (/) and a swap file system are defined by default. Click the Add icon to add more file systems. You can also change the file system type to UFS, or ZFS.

    Click Next to specify the name service.

  10. If you have a naming service in place, select the appropriate one and provide the setup details. Otherwise, select None and proceed further.
    • DNS: Enter the domain name of the DNS server and enter the IP address of the DNS server in the Name Server field. You can enter up to three IP addresses as the value for the Name Server. Provide the additional domains to search for name service information in the Domain Name Search List. You can specify up to six domain names to search. The maximum length of each search entry is 250 characters.

    • NIS or NIS+: Enter the domain name of the NIS or NIS+ server. When you know the NIS server details, choose the option Specify an NIS Server and enter the NIS server host name and the IP address.

    • LDAP: Enter the domain name of the LDAP server. Specify the name of the LDAP Profile you want to use to configure the system. Enter the IP address of the LDAP Profile Server. You can also provide the Proxy Bind Distinguished Name and Password.

    Click Next to view the summary of the parameters selected for the profile.

  11. Review the parameters selected for the profile and click Finish to create the OS provisioning profile.

5.14.4 Creating an OS Configuration Profile

The OS configuration profile collects the information related to networking details for the OS. You can specify the number of interfaces that you want to specify for the OS.

Depending on the number of interfaces that you want to configure for the OS, the network connections are constructed for the OS during the deployment. The

  1. Select the Plan Management section and expand Profiles and Policies.
  2. Select OS Configuration and click Create Profile in the Actions pane.
  3. Enter the following details to identify the profile:
    • Name and description of the profile.

      Note:

      Ensure that the name of the profile is not same as the OS provisioning profile.

    • Select Logical Domain as the Subtype and Virtual Machine as the Target Type.

    Click Next to set the OS Management properties

  4. Select to manage the OS automatically and deploy the Agent Controller to manage the asset. Select the option Enable Multiplexed I/O so that you can associate block storage libraries such as SAN and iSCSI libraries for storage with the OS. If required, deselect the option Enable Single Root I/O Virtualization (SR-IOV) to disable SR-IOV on root domains.

    Click Next to specify the networking details.

  5. Select one of the network options for the target system:

    Click Next to specify the number of network interfaces that must be used on the OS.

  6. Enter the number of network interfaces that must be used on the OS. The details of the interfaces are collected while deploying the plan.

    Click Next to view the summary of the parameters selected for OS configuration.

  7. Review the parameters and click Finish to create the OS configuration profile.

5.14.5 Creating Link Aggregation in Logical Domains

While creating the profile for OS configuration, follow the procedure to create link aggregation:

  1. Select Use Link Aggregation in the Specify Networks step of the profile.
  2. Specify and configure the IEE 802.3ad Link Aggregation details:
    • Link Aggregation Name: Select the name of the Link Aggregation. The names are set to aggr<x>.

    • Load Balancing Policy: Define the policy for outgoing traffic.

    • Aggregation Mode and Switches: When the aggregation topology connects through a switch, determine whether the switch supports the Link Aggregation Control Protocol (LACP). When the switch supports LACP, you must configure LACP for the switch and the aggregation. Define one of the modes in which LACP must operate.

    • LACP Timer: Indicates the LACP timer value, either short or long.

    • MAC Address Policy: The MAC address policy is always defined to Auto.

    • Number of Interfaces: By default, two interfaces are defined in the aggregation. You can modify the number of interfaces for the aggregation.

    You can add multiple link aggregation and define the configuration parameters for each aggregation.

    You can define the interfaces during deployment. Click Next to continue the profile.

5.14.6 Creating IPMP Groups in Logical Domains

While creating the profile for OS configuration, follow the procedure to create IPMP groups.

  1. Select Use IPMP in the Specify Networks step of the profile.
  2. Specify and configure the IPMP groups:
    • IPMP Name: The name of the IPMP group is automatically displayed in the format of ipmp<x>. You can also modify the default name of the IPMP group.

    • Failure Detection: The detection can be Link-Based or Link-Based and Probe-Based.

    • Number of Interfaces: By default, two interfaces are defined in the group. You can modify the number of interfaces for the IPMP group.

    You can add multiple IPMP groups and select the failure detection for each group.

    Click Next to specify the IPMP interfaces.

  3. The interfaces are defined during the deployment of the profile. Specify the following information for each interface:
    • Select whether the interface is Failover or Standby Interface.

    • Select to Assign IP address or not during deployment to the specified NICs. The data and test addresses are defined during deployment.

    Click Next to continue the profile.

5.14.7 Apply the OS Deployment Plan for Logical Domains

When you apply the plan for provisioning the OS on logical domains, you must provide resource information in the following steps:

  • Boot Interface Resource Assignments

  • Storage Resource Assignments

  • Specify Network Resource Assignments

When assigning a boot interface, if OpenBoot PROM (OBP) is used, only untagged networks are listed for selection. Only untagged networks can be used for OS provisioning as you cannot boot from a tagged network.

When assigning a boot interface, if you are applying a combined Logical Domain creation and OS Provisioning plan, select the first network connected to the LDom guest during the LDom creation wizard to minimize the job's runtime. Selecting another network causes the OS provisioning job to take additional time.

Other than these steps, the deployment plan in minimal interaction mode takes you through the confirmation of the OS provisioning and configuration profile information.

For logical domains that use vnets for network resource, the network resource step is displayed as shown in Figure 5-8.

Figure 5-8 Boot Interface Step for Virtual Controller

Description of Figure 5-8 follows
Description of "Figure 5-8 Boot Interface Step for Virtual Controller"

The Controller is displayed as virtual and the interface is the network interface that provides the virtual switch for the network connection of the logical domain.

For the other vnet network connections that must be configured with the OS are displayed as shown in the Figure 5-9.

Figure 5-9 Network Resource Assignments

Description of Figure 5-9 follows
Description of "Figure 5-9 Network Resource Assignments"

For every network connection that must be configured with the OS, requires a virtual switch.

5.14.7.1 Root Domains and I/O Domains

The physical network resources for root domains and I/O domains are displayed as shown in the Figure 5-10

Figure 5-10 Boot Interface Resource for Root Domain

Description of Figure 5-10 follows
Description of "Figure 5-10 Boot Interface Resource for Root Domain"

The network devices are filtered in the Controller list and displayed for selection. The Interface lists the network interfaces or the ports that are available in the selected network device.

If you know the MAC address of the network interface that must be used for the booting, you can use the option Identify Network Interface by MAC Address.

Figure 5-11 Boot Interface by Using MAC Address

Description of Figure 5-11 follows
Description of "Figure 5-11 Boot Interface by Using MAC Address"

While configuring network on the OS, the first interface is always overwritten by the boot interface resource assignment. You can select the network resource that will be the primary network resource for the OS.

When you apply the combined plan of configuring and installing guest domain on a server pool, then in the OS deployment steps, the Interface column does not display the service domain, and the virtual switch. Only the network interface is listed as net_x. You must wait for the job to complete to see in which Oracle VM Server for SPARC system, the guest domain is created. The service domain and the virtual switch are auto assigned. See for the actual assignment after the job is completed.

For detailed information about creating plans and applying the deployment plan for OS provisioning, see the Related Resources for Oracle VM Server for SPARC section.

5.14.7.2 Creation of Virtual Functions

When you provision OS on the root domain, virtual functions are automatically created on SR-IOV enabled PCIe buses that are allocated to the domain. When you want to delete the root domain, it is required to delete all the virtual functions before proceeding to delete the root domain.

5.15 Manage Logical Domains

Oracle Enterprise Manager Ops Center provides options from the UI to manage the logical domain state, add resources, migrate and recover them. You can interchangeably use the CLI or the UI to manage the logical domains. The UI reflects the current state of the logical domains.

The logical domains that are discovered and managed in the UI does not have any difference in the management options in the UI except for the logical domain metadata.

You can manage logical domains with Oracle Enterprise Manager Ops Center or from the control domain. See Discovering Existing Oracle VM Server for SPARC Environments for information about managing from the control domain.

Note:

If the Enterprise Controller is installed in a logical domain, do not take any action that would power down or restart the corresponding control domain.

The following operations are available for managing the logical domains that are created using Oracle Enterprise Manager Ops Center or that are discovered and managed with an LDom VC Agent in Oracle Enterprise Manager Ops Center:

5.15.1 View I/O Resources

The logical domain that has I/O resources assigned to it displays the PCIe buses and the corresponding endpoints in the I/O Resources tab in the center pane.

The I/O resources tab also displays the network interfaces that are SR-IOV enabled. The physical functions and their corresponding virtual functions are listed in the SR-IOV Services sub tab. The guest domains to which the virtual functions are assigned are displayed.

5.15.2 View Virtual Resources

You can view all the resources of a logical domain, including the virtual services bindings and the SR-IOV Virtual Functions of a logical domain, in the Virtual Resources tab in the center pane.

The Virtual Resources tab contains several subtabs. The Virtual Devices subtab displays all virtual devices attached to the logical domain, including the virtual console, virtual network, virtual disks, and virtual HBAs. The SR-IOV Devices subtab displays all virtual functions.

5.15.3 View Virtual Services

Depending on the type of PCIe bus or the PCIe Endpoint assigned to the logical domain, it can provide services to other domains.

If the PCIe Endpoint is attached to storage device, then the logical domain can provide virtual disk services to other domains. The type of device attached to the PCIe Endpoint or the bus, the I/O domains provide virtual services to other domains. These details are listed in the Virtual Services tab in the center pane of the selected logical domain.

  • Virtual disk server: The virtual disk services allow you to export virtual disks to other logical domains. The virtual storage infrastructure enables other logical domains to access the storage disks that are not directly assigned to them. The virtual disk service processes the requests from the logical domains and submits them to the back end storage.

  • Virtual network switches: The virtual network switch enables networking between virtual network devices in logical domains. The virtual switches that are created for the network interfaces in the I/O domain are listed. You can use the virtual switches to start or create logical domains. There is no virtual switch creation for SR-IOV enabled networks. See View I/O Resources for the SR-IOV enabled network interfaces and virtual functions.

  • Virtual console concentrator: The virtual console concentrator that has been created in the logical domain is listed. You can use this console to redirect the console I/O from other domains.

  • Virtual SANs: The virtual SANs on the guest are displayed. Virtual SANs are available prior to Oracle VM for SPARC 3.3.

5.15.4 Edit Logical Domain Configuration

Use the option Edit Attributes in the Actions pane of a selected logical domain. You can edit the name, description, CPU Model, Crypto Units, memory, and automatic recovery priority value.

You can modify the CPU model of a logical domain between Virtual CPU and Whole-Core.

You can modify the CPU architecture between generic and native. To modify the CPU architecture, the Oracle VM Server for SPARC must be at least running 2.2 version.

When you edit the CPU model, CPU architecture, or memory, the guest must be rebooted unless the Control Domain is running Logical Domains Manager 2.0 or later and the guest OS is Oracle Solaris 10 9/10 or later.

You cannot edit the CPU or memory settings for a guest if an OS is not installed and running on the guest.

If you have any physical bindings constraint for the memory, CPU cores, or both, then you cannot edit CPU and memory configuration of the logical domain in the UI. Oracle Enterprise Manager Ops Center also does not provide options to explicitly assign physical resources such as CPU and memory. Whereas, you can use the CLI to assign physical resources to the logical domains. These are displayed in the Summary tab of the logical domains as Physical Bindings.

Also, you cannot switch the CPU Model for logical domains that are assigned with physical CPU and memory resources.

Use the option Edit Tags to modify the tag values or to add new tags.

5.15.5 Shut Down a Logical Domain

You can either shutdown or shutdown and detach the domain from its running state. Shutting down a logical domain means that the domain is still associated with the control domain, and connected to its network and storage resources. The domain is said to be in the state of shutdown. When you start a shutdown domain, you are not required to define the server and the resources.

When you shutdown and detach the domain from its running state, the domain is detached from the control domain, and disconnected from its network and storage resources. To start the domain, you have to define and select the server, network and storage resources. The option to detach the domain is not available in the following conditions of the domain:

  • The domain is root or I/O domain.

  • The domain is managed agentlessly.

  • The domain metadata is on unmanaged storage. The scenario can occur when you discover and manage logical domains that are created using the CLI and storage of the logical domain is not moved to a managed storage.

When you shut down and detach a logical domain, the guest constraints are saved in an xml file in the /var/opt/sun/xvm/guests/ directory. You can use this file to manually reconstruct the guest if necessary. The xml file is named according to the guest name. For example, if you shut down and detach a guest named guest_1, the guest constraints are saved in the /var/opt/sun/xvm/guests/guest_1.xml file.

A logical domain might be attached to virtual I/O devices on multiple I/O domains. You can still shut down the logical domain unless it does not provide any network, storage and console services to other domains.

Table 5-3 lists the logical domain configuration and shows the state, either shutdown or shutdown and detached.

Table 5-3 Shutdown or Shutdown and Detached State

Logical Domain Shutdown Shutdown and Detached

Created using Oracle Enterprise Manager Ops Center and Agent Managed

Yes

Yes

Discovered and LDom VC Agent Managed

Yes

Yes

Metadata on unmanaged storage

Yes

No

Agentlessly Managed

Yes

No

I/O Domain

Yes

No

Root Domain

Yes

No

5.15.6 Starting a Logical Domain

You can start a logical domain from a shutdown or shutdown and detached state.

The logical domain in shutdown state is not detached from the server, and network resources and therefore, starting the logical domain immediately starts the logical domain in the Oracle VM Server. The Start action does not initiate the Start Logical Domain Wizard.

When the logical domain is in shutdown and detached state, it is disconnected from its network resources. To start a logical domain that is in a shutdown and detached state, you must define the network resources.

For logical domains placed in a server pool, in shutdown and detached state, the logical domain is detached from its control domain and disconnected from its network and storage resources. Starting this logical domain requires you to select the Oracle VM Server in which you want to run the logical domain, storage and network resources.

You must select the virtual switch or the virtual functions of SR-IOV enabled networks that connects to the logical domain MAC address. A virtual switch can be connected to only one MAC address. You can select not to connect to network for a virtual switch.

When a shutdown and detached guest is started, then Oracle Enterprise Manager Ops Center selects the appropriate and best option for the guest domain with generic CPU architecture.

When you have a guest with the CPU architecture as generic in Oracle Enterprise Manager Ops Center and you start the guest in the following servers:

  • Oracle SPARC T3 and T2 Servers: Oracle Enterprise Manager Ops Center sets the CPU architecture to generic.

  • Oracle SPARC T4, T5, M5, and M6 servers: Oracle Enterprise Manager Ops Center sets the CPU architecture to generic (migration-class1). migration-class1 is a cross-CPU migration family for SPARC platforms starting with the SPARC T4 servers.

  • Fujitsu M10 Systems: Oracle Enterprise Manager Ops Center sets the CPU architecture to generic (sparc64-class1). sparc64-class1 is a cross-CPU migration family for SPARC 64 platforms. sparc64-class1 value has more instructions than generic value. This value is compatible only with Fujitsu M10 systems.

  1. Select the logical domain in shutdown and detached state.
  2. Click Start in the Actions pane.

    The Start Logical Domain Wizard is displayed.

  3. Select the server pool in which you want to start the logical domain.

    When you shutdown a logical domain, the logical domain is disassociated from the control domain and it can be started on a server.

    When you have networks attached to a logical domain and is not available in the selected server pool, then that network is not available when the logical domain starts.

  4. Select the Oracle VM Server for SPARC in which you want to start the logical domain. You can select the same Oracle VM Server in which the logical domain is running already. Otherwise, select an Oracle VM Server for SPARC from the list.

    The current load for all the Oracle VM Server for SPARC are displayed. Use this information to place your logical domain.

    Click Next to specify the network interfaces.

  5. Select the virtual switch or the virtual function that must be associated with the logical domain MAC address.

    The number of virtual switches displayed depends on the number of network connections for the server pool. For each network connection, you have a virtual switch created. You can associate a virtual switch with only one MAC address.

  6. Select Do Not Connect when you do not want the virtual switch to connect to MAC address.

    Click Next to schedule the job.

  7. Schedule the job to run now or at a later time.
  8. Review the properties and click Start to run the logical domain.

5.15.7 Add Storage to Logical Domain

You can add virtual disks to a logical domain. The associated libraries to the Oracle VM Server for SPARC are available to be added as additional storage to logical domains.

Use the option Add Storage in the Actions pane to add virtual disks to the logical domain. The logical domain can be in running state for adding storage.

You can add virtual disks from the following type of storage libraries:

  • File system storage: These are NAS libraries. Specify a virtual disk name and size of the disk.

  • Static block storage: Select LUNs from SAN or iSCSI storage servers. The LUNs sizes are fixed.

  • Dynamic block storage: Add LUNs from SAN or iSCSI storage servers discovered and managed in Oracle Enterprise Manager Ops Center. You can also create LUNs by selecting a volume group and specifying the size of the LUN.

Starting in the 12.2.2.0.0 release, you can add multiple SAN LUNs to a logical domain from different libraries using the Add Multiple SAN LUNs option of the Add Storage wizard. See the Related Resources for Oracle VM Server for SPARC section for more details about using this option.

5.15.7.1 Virtual Disk Multipathing

Virtual disk multipathing enables you to configure a virtual disk on a logical domain to access its back-end storage by more than one path. The paths lead through different service domains that provide access to the same back-end storage. The virtual disk of a logical domain remains accessible even if one of the service domains go down.

For each virtual disks of the logical domain that have more than one alternate path to the backend storage, a new multipathing group is created automatically. You must always specify the virtual disk of another service domain for the alternate path to access the back-end storage. You can select to enter the name of the multipathing group or a group is created automatically in the format of logical domain name_mpGroup_devID. The devID is the disk index. For example, if the name of the logical domain is ldom, then the multipathing group name is in the format of ldom_mpGroup_1.

When you have more than one alternate path for the virtual disks, you can select the active path which serves during a failure in one of the service domains.

Unless you enter the name of the multipathing group, the group is not created when there is only one alternate path to access the back-end storage.

5.15.8 Moving Metadata to Another Library

Metadata for a logical domain is saved in either a local library or a shared library. When you install Oracle VM Server for SPARC on a system without using Oracle Enterprise Manager Ops Center, the metadata is saved in a default local library. When you install Oracle VM Server for SPARC with Oracle Enterprise Manager Ops Center, you define a shared storage location for metadata.

To migrate a logical domain to another system, both systems must use the same virtual disks for shared storage and the same shared library for the logical domain metadata.

The following conditions must be met before you can move the logical domain metadata to another library:

  • The control domain must be managed with an LDom VC Agent.

  • The Logical Domain must be known to Oracle Enterprise Manager Ops Center.

    • The Logical Domain and its associated resources are created with Oracle Enterprise Manager Ops Center

    • A manually created logical domain is agent managed with Oracle Enterprise Manager Ops Center

5.15.8.1 To Move Metadata to Another Library From the Control Domain View

You can move a guest’s metadata to another library from the control domain view.

  1. Expand Libraries, expand Storage Libraries, then select the source library in the Navigation pane.
  2. Click the Usage tab. Select the guest in the Logical Domains table, then click the Move Metadata icon.

    Figure 5-12 Move Metadata Icon

    Description of Figure 5-12 follows
    Description of "Figure 5-12 Move Metadata Icon"
  3. Select the new library from the list of available libraries to store the logical domain metadata. Optionally, add a description and add tags. Click Next.
  4. Review the Summary, then click Submit.

When the job completes, the logical domain's metadata is located on the new library.

5.15.8.2 To Move Metadata to Another Library From the Logical Domain View

You can move a logical domain’s metadata to another library.

  1. Expand Assets, then select the shutdown guest in the Assets tree.
  2. Click Move Metadata in the Actions pane.
  3. Select the new library from the list of available libraries to store the logical domain metadata. Optionally, add a description and add tags. Click Next.
  4. Review the Summary, then click Submit.

When the job completes, the logical domain's metadata is located on the new library.

5.15.9 Enabling Shared Access for Opaque Storage Disks

When you create logical domains using the native CLI, it is automatically discovered and displayed under the corresponding Oracle VM Server for SPARC in the UI.

The metadata of the logical domain is stored in the local library of the control domain. If the logical domain storage disks are on storage servers that are not managed by Oracle Enterprise Manager Ops Center, then they are tagged as Opaque and displayed in the UI. Oracle Enterprise Manager Ops Center does not identify whether the storage is local or shared. The migration action is disabled for the logical domain that has the metadata on local library and the storage disks tagged as opaque. Use Move Metadata option to move the logical domain metadata to a shared storage.

Mark the logical domain storage disks as shared to indicate that the storage is available to other managed Oracle VM Servers using the same backend name.

The storage that is marked as shared must be available for all the members in a server pool.

  1. Select the logical domain that has been created using CLI.
  2. Select the Storage tab in the center pane.
  3. Select the disks tagged as Opaque in the Disk Type.
  4. Click the Enable Sharing icon.

The selected disks are marked as sharing in the UI.

5.15.10 Connect to Logical Domain Console

You can attach to the logical domain console within the Oracle Enterprise Manager Ops Center UI. You enable the console connection and then you connect to the console.

Note:

When the logical domain is not in view in the Assets tree, the console is logged out automatically but the connection exists until the connection time expires. You must log in again.

You can view detailed information about the console. Select the Virtual Resources tab for the guest, then select the Virtual Devices tab. The Virtual Console information is displayed, including the port, console service name, and service domain. The service domain indicates the domain providing the virtual console services. The operating system of a domain must be discovered to provide virtual console services to another domain.

5.15.11 View the Console Logs

You can view the console logs for a logical domain through the Oracle Enterprise Manager Ops Center UI. Select the Console tab for the guest, then select the Console Logs tab. You can export these logs as a zip file.

By default, the maximum log size is 400 KB.

The virtual console for a domain can be provided by another domain. The domain providing the virtual console must be agent managed and must be running Oracle Solaris 11 or later.

A system must meet these prerequisites for the console logs to be visible through the UI:

  • The control domain must be managed.

  • The service domain providing the virtual console concentrator service must be running Oracle Solaris 11.0 or later. The guest domain being logged can run older versions of Oracle Solaris.

  • The system runs Oracle VM Server for SPARC 3.0 or later.

5.15.12 Delete a Logical Domain

When you delete a logical domain, it is disconnected from the associated networks and is disassociated from the Oracle VM Server. All of the associated resources are released and the domain configuration is removed from the library. All references to the logical domain, including its disk image and snapshots are removed from the system. However, the logical domain profile and plan remain unaffected.

When you delete a logical domain manually, that is, using the native CLI, the UI reflects the removal of the logical domain.

You can delete one or more logical domains simultaneously by selecting them in the Control Domain summary.

You can use the Delete action to delete root domains that have virtual functions on SR-IOV enabled PCIe devices if other domains are not using a virtual function from that root domain.

If you want to delete guest domains assigned with virtual functions, you must first remove the virtual functions from the domain and then delete the guest domains.

5.15.13 Cancel Delayed Reconfiguration

The action Cancel Delayed Reconfiguration is restricted to the control domain or a PCIe root domain, if supported. This action is disabled for all other logical domains.

See Delayed Reconfiguration Mode, for more information about the delayed reconfiguration.

5.16 Managing Logical Domain Networks

The following information is covered in this section:

You can connect or disconnect a network from a logical domain in running state. You can connect to a network multiple times. Use the Connect Network and Disconnect Network icons, shown in Figure 5-6, in the Networks tab.

You can also connect to SR-IOV enabled networks and thereby assign virtual functions to the logical domain. The SR-IOV enabled networks are available only on control domain as the SR-IOV feature is enabled only on the SR-IOV cards on the primary domain. From Oracle VM Server for SPARC 3.1 version, SR-IOV enabled networks are also available from root domains.

Each connection requires a virtual switch or the virtual function of SR-IOV enabled networks to connect the logical domain to the network.

Virtual network device (vnet) is a virtual device that is defined in the logical domain when it is connected to a virtual switch. You can create several vnets over a virtual switch. Creating too many vnets also decrease the performance. Therefore, the creation of vnets are reduced and Oracle Enterprise Manager Ops Center ensures to re-use when possible. For example, when there is an existing vnet that has the connectivity to the required network and there is no VLAN ID incompatibility then the vnet is re-used.

You can re-use a virtual switch to make multiple connections to a network from the logical domain or use the same virtual switch to connect to a network for different logical domains.

For SR-IOV enabled networks, you can use a virtual function to connect to a network multiple times for a logical domain or connect to a network for multiple logical domains.

Migration is disabled for domains that has one or more virtual functions assigned to it. For virtual switches, the virtual switch name must be identical in the source and target machine to migrate the guest domain.

Example 5-2 Re-using Vnets

Consider a virtual switch vsw0 connected to networks N1 and N2. Network N1 is a regular LAN without a VLAN ID and Network N2 is a VLAN ID with VID = 11. When you connect a guest G1 to network N1 through vsw0, a virtual network device vnet0 is created. The vnet0 is created with PVID = 1 and VID = <null>.

When you connect the guest G1 to network N2 through vsw0, then the vnet, vnet0, is re-used by setting the VID = 11 which is the VLAN ID of the network N2.

5.16.1 VLAN Tagging Support

Oracle VM Server for SPARC software supports 802.1Q VLAN-Tagging in the network infrastructure. The option to define the VLAN tagging mode is available from the Oracle Enterprise Manager Ops Center UI when you connect networks configured with a VLAN ID to the guest domains.

Configure the VLAN interfaces over the physical network devices in Oracle VM Server for SPARC. When you connect these networks to the guest domains, define the tagging mode as tagged or untagged.

In untagged mode, the Port VLAN ID (PVID) is set to the VLAN ID of the network. The outbound frames from the virtual network that are untagged are tagged with this PVID by the virtual switch. You can have only one PVID for a vnet or a virtual function (VF). If the PVID option is selected for a virtual function, you cannot use the VID option; if the VID option is selected, you cannot use the PVID option.

In tagged mode, the network's VLAN ID is added to the VID of the vnet or virtual function. The vnet sends and receives tagged frames over the VLANs specified by its VLAN IDs (VIDs).

You must have at least one untagged network connection to be used for provisioning OS on the logical domain. If the network to be used for OS provisioning is already VLAN tagged, then select the untagged mode while connecting the network. The network is assigned the PVID with the VID of the network.

When the tagging mode for a network configured with VLAN ID is set to tagged at the Oracle VM Server, then you have the option to select tagged or untagged option when you connect the logical domain to the network. If the tagging mode is set to untagged, then you cannot the change the tagging mode and it is set to untagged.

5.16.2 Alternate MAC Addresses

When you connect network to a logical domains, alternate MAC addresses are created automatically for the vnets or the virtual functions assigned to the logical domain. These alternate MAC addresses can be used to create zones in the logical domains.

The number of MAC addresses to create is set to virtualization/ldom.auto.alt.mac.addrs property value by default. This value is 20 but can be changed. If required, you can change the number of MAC addresses to create through the Administration section of the UI.

Select the Configuration tab, then select the Virtualization subsystem. The ldom.auto.alt.mac.addrs property specifies the number of MAC addresses to create:

ldom.auto.alt.mac.addrs.count=20

If you do not want any MAC address to be created, you can change the value to:

ldom.auto.alt.mac.addrs.count=0

The property is set globally for each Oracle VM Server for SPARC managed in Oracle Enterprise Manager Ops Center. Restart the Enterprise Controller after you change the property. Only after the restart, the number of alternate MAC address change is applied when you connect a network to the logical domain. The number is not altered for any existing connection.

The alternate MAC address are not created in the following scenarios or conditions:

  • When you connect to a network dynamically, that is when the logical domain is in running state.

  • The network connection is not done using Oracle Enterprise Manager Ops Center.

    Note:

    You need alternate MAC addresses to create zones on logical domains. Refer to Oracle VM Server for SPARC documentation at http://docs.oracle.com/cd/E38405_01/html/E38406/assignmacaddressesautomaticallyormanually.html#scrolltoc to create alternate MAC address manually.

  • When you use InfiniBand virtual functions.

  • The Oracle VM Server for SPARC is lower than 3.0.0.2 version.

5.16.3 Connecting to Network

You can connect or disconnect a network from a logical domain in running state.

Figure 5-13 Logical Domain Network Options

Description of Figure 5-13 follows
Description of "Figure 5-13 Logical Domain Network Options"
  1. Click the Connect Network icon to display the Connect Guests to Network window. Select the network.

  2. Select the networks from the list. For each selected network, choose the following details:

    • VLAN ID / P-KEY: Specify the VLAN ID or P-KEY for the network.

    • Mode: If the network is assigned a VLAN ID, you can select the network to be connected in tagged or untagged mode.

    • Service domain: Select the service domain that provides the network interface for the network connection. The domain can be primary, I/O domains or the root domains.

    • SR-IOV: Select this option when you want to assign a virtual function to connect to the network.

    • Map Connection: Select the virtual switch through which you want to connect the guest domain to the network. For SR-IOV enabled network connection, select the physical function through which you want to connect to the network.

      An existing vnet is re-used to connect to the same virtual switch or virtual function rather than creating a new vnet for each connection.

    • Link Mode: Select the link mode for the network connection. In Manual link mode, you specify the data link resource information. In Automatic link mode, the data link is automatically generated and configured using the following rules:

      • If an existing compatible vnet already is found, it will be used to connect the network. Vnets are re-used. Whereas, it is not applicable for SR-IOV enabled networks. A vnet supports one untagged network and one or more tagged network connections.

      • The number of alternate MAC addresses is set according to the virtualization/ldom.auto.alt.mac.addrs property value. This value is ten by default but can be edited.

      Note:

      • When you assign an Ethernet virtual function to connect to the guest, only the number of alternate MAC addresses can be specified.

      • When you assign an InfiniBand virtual function to connect to the guest, the Link Mode is automatically set to Automatic and cannot be changed to Manual mode.

    Click Next to continue.

  3. If you selected manual link mode for at least one network, the data link resource assignments are displayed. Select the following details for each network connection:

    • Lower Link Name: Specify the lower link name for the network connection.

    • Link Name: Specify the VNet name for the network connection.

    • Nb Alt Mac Addrs: Specify a number of alternate MAC addresses for the network connection.

    • Max Bandwidth (Mbps): Specify the maximum bandwidth for the network connection in megabits per second.

    • Private VLAN ID: If the network connection is in untagged mode, you can specify a private VLAN ID for the network connection.

    • Private VLAN Type: If you provided a private VLAN ID, select a private VLAN type, isolated or community.

    Note:

    When you assign an Ethernet virtual function to connect to the guest, only the number of alternate MAC addresses can be specified.

    Click Next to continue.

  4. View the summary, then click Finish to connect the networks to the logical domains.

When you connect networks to a guest domain that is in shutdown/detached state, then the service domain and Map Connection options are not available in the Connect Networks window.

You cannot connect networks to a guest domain that does not have an installed operating system.

You can create IPMP groups or aggregate the NICs allocated to the logical domain. See Networks for Virtualization for more information about creating IPMP groups and link aggregation.

When you disconnect networks from logical domains, the vnets are destroyed. If the vnets are re-used with another network connection, then it cannot be removed. To remove the vnet completely, you must remove the network connections in the following order:

  • You must detach the network in the order of highest to lowest for netx. For example, a network connection with net5 as the highest must be removed first.

  • The network connection for a network with a VLAN ID must be the last network connection to be removed.

To disconnect a network:

  1. Select the networks that you want to disconnect from the logical domain in the Network tab of the selected logical domain.
  2. Click Disconnect Network icon.

    Unbind a Network window is displayed.

  3. Click Disconnect From Network to confirm the delete action.

5.16.4 Port Connectivity

The Port Connectivity sub tab in the Network tab provides the details of the logical domain network connection.

The port connectivity tab displays the following information:

  • Port Type

  • Media Type

  • VLAN ID and Port VLAN ID

  • Alternate MAC Address

  • MAC Address

  • Port Name

Figure 5-14 Port Connectivity

Description of Figure 5-14 follows
Description of "Figure 5-14 Port Connectivity"

The Port Name displays the name of the vnet created. When the name of the vnet is of the format vnicxxxxx, it indicates that the vnet is created by Oracle Enterprise Manager Ops Center. Whereas, all other formats indicate that the network was not connected to the logical domain using Oracle Enterprise Manager Ops Center.

Also, when the network is connected to logical domain using Oracle Enterprise Manager Ops Center, then the alternate MAC address shows the list of created address provided the properties is set to create MAC address.

The Port Type indicates whether the type of underlying network connection for the logical domain. The type can be Physical Interface, SR-IOV Virtual Function, or Ldom vnet. The Media Type indicates whether the network media is Ethernet or InfiniBand.

5.17 Migrate Logical Domains

The following information is covered in this section:

Migrating a logical domain means moving the logical domain from one server (source) to another server (target). When a domain is migrated from the source to the target server in its running state without any impact to its availability is called live migration.

When the logical domain is stopped and then migrated to the target server, it is called as cold migration. You can migrate the logical domains when the requirements are met for successful migration.

Apart from these, Oracle Enterprise Manager Ops Center also migrate domains to other servers in the following scenarios:

  • Balancing the load of the servers in a server pool

  • Recovery of logical domains when the server on which it is running fails

Live migration, logical domain recovery and load balancing are available and supported only in the context of a server pool.

Note:

From Oracle VM Server for SPARC 2.1 and higher versions, you can migrate the logical domains only when the source and target machines are running at least Oracle VM Server for SPARC 2.1 version.

Note:

A logical domain with a kernel zone installed on it cannot be migrated.

Oracle Enterprise Manager Ops Center UI provides the Migrate option to migrate the logical domains. The Migrate Logical Domain Wizard provides a list of available compatible target machines to migrate the logical domain. Select a target and migrate the logical domain.

5.17.1 Setting User Accounts for Migration

Temporary user accounts are created dynamically when a logical domain migration is initiated. The user account is deleted after the migration. If your datacenter environment does not support dynamic creation of user accounts, then you can set the logical domain migration to use an existing user account instead of creating a temporary one.

You must have the Security Admin role to create the credentials. If not, contact the Ops Center Administrator for creating a user account for logical domain migration.

To create a user account, follow the procedure:

  1. Select the Administration section in the Navigation pane.

  2. Select Credentials in the Administration section.

  3. Click Create Credentials in the Actions pane.

    The Create Credentials window is displayed.

  4. Select the protocol as OVM For SPARC Migration and enter the following details:

    • Name of the credential

    • User name for the account

    • Password for the user account

  5. Click Create to create the user account.

Set the following system property in the configuration management of the Enterprise Controller:

  1. Select the Administration section in the Navigation pane.
  2. Select the Configuration tab in the center pane.
  3. Select Virtualization in the Subsystem.

    The property Virtualization.Ldoms.Migration.Username is displayed.

  4. Enter the user name of OVM For SPARC Migration user account created previously to be used for all logical domain migration.
  5. Click Save.

The migration job proceeds to use the user account as defined in the system property, that user account must exist on the source and target systems. If the system property is set default to null, then the migration job dynamically creates a temporary user account. Ensure to set this property before initiating any logical domain migration, if required.

5.17.2 Migration Requirements

When you migrate logical domains, there are certain requirements and restrictions that must be noted for successful migration. Otherwise, there can be loss in the configuration of the logical domains or unsuccessful migration job.

The requirements and restrictions that affect the migration of the logical domains are:

  • Shared Storage: The source and the target servers must have access to common storage resource. The logical domain metadata must be stored on NFS storage that must be associated with the server pool or the stand-alone Oracle VM Servers. The virtual disks must be shared. If the virtual disks are on shared storage that are not managed by Oracle Enterprise Manager Ops Center (other than FC or iSCSI LUNs, or NFS shares), enable shared access for the storage disks. Otherwise, use shared FC or iSCSI LUNs, NFS shares for domain's storage.

  • CPU Architecture: The CPU architecture is of importance when you want to migrate the logical domains between systems that have different CPU processor type.

    The following are the supported types for CPU architecture:

    • native: The logical domains that are created with this CPU type enable them to be migrated only between systems that have the same CPU architecture type.

    • generic: The logical domains that are created with this CPU type enable them to migrate to systems independent of the CPU type. Though this might result in reduced performance, it provides an increased flexibility to migrate the domains between systems that have different CPU types.

      From Oracle VM Server for SPARC 3.1 and higher versions, the value of generic varies according to the platform. For example, if generic is set for a guest CPU architecture on an Oracle SPARC T2 server, then the CPU architecture is set to generic. Whereas for servers starting from Oracle SPARC T4, when the CPU architecture is set to generic in the UI, Oracle Enterprise Manager Ops Center sets the CPU architecture to migration-class1. In the UI, the CPU architecture is displayed as generic (migration-class1). For Oracle M5 and M6 servers also, generic (migration-class1) is displayed. For Fujitsu M10 systems, the guests that are set to generic CPU architecture, are automatically set to sparc64-class1 CPU architecture. In the UI, the CPU architecture is displayed as generic (sparc64-class1).

      Depending on the server platform onto which the guest is started after shutdown-detach, the best option for the CPU architecture is selected when the guest has generic as the CPU architecture.

    You can edit the CPU architecture of the logical domain as required to facilitate the cross CPU live migration. You must shut down and detach the guest to edit the CPU architecture.

  • Logical Domains Created Using CLI: For logical domains that are not created using the Oracle Enterprise Manager Ops Center UI, you must move the metadata of the discovered logical domains to a shared NFS storage. The virtual disks of the discovered logical domains are marked as opaque and it must be shared to enable migration.

  • Whole-core: When you migrate the logical domains from latest versions of Oracle VM Server for SPARC to earlier versions results in losing the whole core configuration. For example, the logical domains configured with whole cores on Oracle VM Server for SPARC 3.0 version loses it whole-core configuration when migrated to Oracle VM Server for SPARC 2.2 version. Depending on the original number of cores allocated, it is translated to the number of CPU Threads and allocated to the logical domain.

  • Power Management Policy: The power management policy for Oracle Integrated Lights Out Manager (ILOM) firmware can be set to Disabled, Elastic or Performance. For Oracle VM Server for SPARC 2.2 and earlier versions, the migration of logical domains are not supported when the source and target machines are set with elastic power management policy. You must change to Performance policy to migrate the logical domains.

    From Oracle VM Server for SPARC 3.0 version onwards, the Elastic policy does not affect the migration of the logical domains. The power management policy Performance and Elastic are fully compatible with the whole-core constraint.

  • Physical I/O devices: The logical domains that are attached to physical I/O devices, namely the I/O domains and root domains, cannot be migrated.

  • Virtual Switch: When you migrate logical domains, the virtual switch name must be identical in the source and the target Oracle VM Server for SPARC.

  • Virtual Functions: You cannot migrate logical domains that are connected to virtual functions.

  • Virtual I/O Devices: The logical domains can be attached to virtual I/O devices from multiple I/O domains. You can still migrate the logical domains provided all the virtual I/O services are available on the target machine.

  • Virtual Disk Server: The target and the source Oracle VM Server must have the same names for the virtual disk server to migrate the guest domain. When the guest domains uses I/O resources from I/O domains or root domains, then the target Oracle VM Server must have domains that use the same name for the virtual disk server as the source Oracle VM Server.

  • Virtual HBAs: The logical domains with virtual HBAs attached can be migrated only if the source and destination servers are running Oracle VM Server for SPARC 3.4 or higher version.

  • MAU: For Oracle VM Server for SPARC 1.2 version, you can migrate with only one MAU. For Oracle VM Server for SPARC 1.3 and later versions, you can migrate with any number of MAUs.

  • Tagged and Untagged Networks: When you migrate logical domains between servers that have different tagging mode, then the network configuration of the logical domain OS can be lost. You can avoid this issue by migrating domains between servers that have similar network configuration mode. When a guest domain is connected to a network configured with a VLAN ID, migration is not allowed between servers with the same VLAN network connected using different tagging mode.

5.17.3 Migrating a Logical Domain

  1. Select the logical domain to be migrated from the Assets section.
  2. Click Migrate in the Actions pane.

    The Migrate Logical Domain Wizard is displayed.

  3. The eligible Oracle VM Servers appear in decreasing order of preference. Select an Oracle VM Server from the list.

    Click Next.

  4. Review the information and click Finish to migrate the logical domain.

This migration is initiated from the logical domain and it results in migration of single logical domain. When the Oracle VM Server is placed in a server pool and there are more than one logical domain in the host, you can use Migrate Logical Domains to migrate more than one logical domain from the host.

5.17.4 Migrating Multiple Logical Domains

You can migrate multiple logical domains to a new Oracle VM Server in the same server pool.

  1. Select an Oracle VM Server that is placed in a server pool.
  2. Click Migrate Logical Domains in the Actions pane.

    The Migrate Logical Domains Wizard is displayed.

  3. The list of logical domains running in the Oracle VM Server are listed. Select one or more logical domains from the list.

    Click Next.

  4. The eligible Oracle VM Servers in the same server pool that have the required resources to host all the logical domains are listed in the decreasing order of preference. Select an Oracle VM Server from the list.

    Click Next.

  5. Review the summary and click Finish to migrate logical domains.

5.18 Automatic Recovery of Logical Domains

You can enable automatic recovery of logical domains. This ensures that when the Oracle VM Server for SPARC placed in a server pool fails, the logical domains are recovered and restarted on another server in the server pool.

The following information is covered in this section:

You can set the priority of recovery value for the logical domain that decides the order of recovery of the logical domains in the Oracle VM Server for SPARC. The priority of recovery value can be set between 0 to 100. For example, if the priority of recovery value set for logical domain A is 12 and logical domain B is 15, then the logical domain B is recovered first.

Use the Enable Automatic Recovery and Disable Automatic Recovery actions, shown in Figure 5-15, to set the automatic recovery for a logical domain. If you have disabled the automatic recovery of a logical domain, then it is not migrated to other servers in the pool when the underlying virtualization host fails. Instead, it is listed under Shutdown Guests in the Server Pool. You can restart them on other servers as required.

Figure 5-15 Enable and Disable Automatic Recovery

Description of Figure 5-15 follows
Description of "Figure 5-15 Enable and Disable Automatic Recovery"

When you edit the attributes of a logical domain, you can edit the priority of recovery value.

If there are no resources available to recover the logical domains in the server pool, Oracle Enterprise Manager Ops Center checks periodically for every minute for free resources to retry the automatic recovery mechanism.

You can also follow the procedure described in Recovering Logical Domains to recover the attached logical domains.

When a logical domain is configured for automatic recovery, the auto boot value for the logical domain is set to false. The auto boot value is controlled by Oracle Enterprise Manager Ops Center. This is because, when the failed server is repaired and restarted, the logical domains are not started automatically. Oracle Enterprise Manager Ops Center checks whether the logical domains are recovered and running on other servers. If the logical domains are not recovered, then the logical domains are started. If the logical domains are recovered on other servers, then Oracle Enterprise Manager Ops Center cleans up those logical domains on the server.

When you perform shut down and start operations using the UI for a logical domain that is configured to automatically recover, the operating system is also booted automatically by Oracle Enterprise Manager Ops Center.

When you perform the shut down and start operations using the CLI for a logical domain that is configured to automatically recover, you must boot the OS separately. This is not automatically done by Oracle Enterprise Manager Ops Center.

See Server Pools and Automatic Recovery of Oracle VM Server for SPARC in a server pool for more information.

Note:

If the Enterprise Controller is installed in a logical domain, do not enable automatic recovery for that logical domain.

5.18.1 Virtual Disk Multipathing and Automatic Recovery Process

When you have I/O domains and root domains, it is possible to have several paths to access the LUNs allocated for virtual disk storage. The redundant storage access requires the automatic recovery options of the logical domains to be set or modified accordingly to recover the logical domains with or without the redundant storage.

For example, if an Oracle VM Server for SPARC has two HBA cards that have access to the Fiber Channel LUNs. One HBA card is assigned to the control domain or the primary domain. Other HBA card is made available to an I/O domain by assigning the PCIe bus to it. Now, the LUN from the storage array is accessible from the primary domain and the I/O domain. When you create a guest domain, select that the virtual disk is created from the virtual disk server of the primary domain and the alternate path to the storage is the virtual disk server on the I/O domain. A multipathing group is created for the virtual disk of the guest domain with both the paths to the storage. Also, while creating the guest domain select the option to automatic recovery and authorize recovery without Redundant I/O.

If the PCIe card on the primary domain fails, then the guest domain is still up and running as it has access to its virtual disk through the alternate path on the I/O domain.

The Oracle VM Server for SPARC system is placed in a server pool with other servers in the pool that do not have an I/O domain or root domain. The primary domain is the only access to the LUN.

If the server fails, then Oracle Enterprise Manager Ops Center tries to recover the guest domain on other servers in the server pool only when the option Authorize Recovery without Redundant I/O is selected. Otherwise, the recovery is not done as there is no target server that provides redundant access to the storage.

5.18.2 Recovering Logical Domains

The following procedure describes the actions that must be performed to recover the attached logical domains when an Oracle VM Server for SPARC server in a server pool has failed.

  1. Isolate the failed server.

    Log in to the ALOM or ILOM of the physical server and shut down the server.

  2. Power-off the failed server.
  3. In the Oracle Enterprise Manager Ops Center UI, select the server pool in which the failed server is the member. Check whether the server is flagged as unavailable. This status is updated within 5 minutes approximately.
  4. Select the control domain of the server in the UI and click Delete Asset in the Actions pane.
  5. Select the server pool in which the failed server was a member and expand the Shutdown Guests list.
  6. The attached logical domains of the failed server are listed. Select the logical domains and click Start to start the logical domains in the desired server in the server pool.
  7. To bring back the failed server to the server pool, the server must be repaired and provisioned again with Oracle VM Server for SPARC using Oracle Enterprise Manager Ops Center.

    To avoid provisioning the server again with Oracle VM Server for SPARC, follow the steps as defined in the section Re-introducing the Failed Server.

5.18.3 Re-introducing the Failed Server

You can bring in the repaired server back into Oracle Enterprise Manager Ops Center without the need of provisioning Oracle VM Server for SPARC again.

You must do the following when you want to re-introduce the repaired Oracle VM Server back into Oracle Enterprise Manager Ops Center:

  1. Start the server and login to the console of the server and check whether the logical domains that have been recovered on other servers are still present on the server without the OS being booted. The logical domain OS not being booted prevents data corruption.
  2. During the startup of the Oracle Enterprise Manager Ops Center Agent Controller on the server, the logical domain that were recovered on other servers without their OS being booted are removed.
  3. Wait until the agent starts up on the control domain. When the agent is starting up, the following message is displayed on executing the command /usr/bin/svcs -xv:

    svc:/application/management/common-agent-container-1:scn-agent is starting

    This message stops when the agent is started.

  4. Discover the control domain from the Oracle Enterprise Manager Ops Center UI using the discovery procedure as explained in the section Discovering Existing Oracle VM Server for SPARC Environments.

5.19 Layered Virtualization

Oracle Enterprise Manager Ops Center supports non-global zones within the logical domains.

The consolidated view of the control domain, and its service domains helps Oracle Enterprise Manager Ops Center to understand the network and storage resources that can be used to create zones within a logical domain OS. When you provision OS on logical domains, it will automatically install the Zone Virtualization Controller Agent. The Zone Virtualization Controller Agent enables you to create non-global zones on the logical domain OS.

Creating zones on a logical domain requires alternate MAC addresses which can be allocated to the zones. See the Alternate MAC Addresses section for information about the limitations of alternate MAC ID creation.

The virtual disks storage that are free and available for use in the logical domains are available for zone storage. The storage that are in use by other zones, and those exported by the logical domains and used as a virtual disk to other domains, are not available for zone creation.

When you need to migrate the zones on the logical domains placed in a server pool, then the zone must not be using the logical domains virtual disk for storage. To enable migration of zones created on logical domains, the following conditions must be satisfied:

  • The zone must be directly attached to NFS, FC LUNs or iSCSI LUNs.

  • For zones on I/O domains or root domains and to use FC LUNs, the domains must have the FC card to enable creation and migration of zones.

Refer to the Oracle Solaris Zones, for more information about creating zones.

5.20 Server Pools

You can create server pools of your Oracle VM Server for SPARC systems in Oracle Enterprise Manager Ops Center. The Oracle VM Server for SPARC can have logical domains running in it.

When you want to create logical domains for the Oracle VM Servers placed in the server pool, the designation of the domains and the virtual switches or virtual functions to provide network connection are auto assigned. You cannot specify the network connection resource assignments.

The detailed description and procedure for creating server pools are described in Server Pools.

5.21 Related Resources for Oracle VM Server for SPARC

For instructions in performing actions or to learn more about the role of this feature, go to one of the following resources.