32 Dynamic Resource Provisioning with Oracle VM Server for SPARC

Oracle Enterprise Manager Cloud Control integrates with Enterprise Manager Ops Center to create and manage virtual datacenters (vDC). Within the vDC, a vServer appears to users as a complete system, which they can optimize on demand as a DBaaS or MWaaS private cloud.

32.1 Integration for a PaaS Solution

In Oracle Enterprise Manager Ops Center, the IaaS cloud platform service is a virtual datacenter (vDC), a virtual environment supported by pooled servers, storage, and network resources. A cloud administrator sets up the vDC to run Oracle VM Servers for SPARC (logical domains). Enterprise Manager Cloud Control users are creating vServers in the logical domains automatically, as needed. Each vServer draws its requirements from the resources of the vDC.

This section explains how to create the vDC and to connect it to Enterprise Manager Cloud Control.

32.1.1 Prerequisites

  • Enterprise Manager 12c Cloud Control Release 5 with Virtual Service Adaptor configured to connect to Ops Center's Enterprise Controller. See Getting Started with Dynamic Resource Provisioning.

  • Enterprise Manager Ops Center Release 3 with managed storage and network assets. See Oracle Enterprise Manager Ops Center Installation for Oracle Solaris Operating System and Configure Reference guides for information about installing the product and discovering assets.

  • Oracle Enterprise Manager Ops Center Client Kit, which includes modifications to enable communication between the two software products. This kit is available from My Oracle Support to https://support.oracle.com/.

The minimum requirement for Ops Center is the co-located configuration: one system hosts the Enterprise Controller, one Proxy Controller, and an embedded Oracle database.

  • Requirements for Enterprise Controller:

    • A minimum of T4-1 Sun Server

    • Same or higher version of Oracle Solaris 11 as the version to be provisioned on vServers running Oracle VM Server for SPARC. Supported versions are Oracle Solaris 11, 11.1, and 11.2 but, for optimal performance use the latest SRU of Oracle Solaris 11.2.

  • Requirements for Proxy Controller:

    • Co-located with Enterprise Controller or running in a system with the same minimum requirements for server, network, and operating system as for the Enterprise Controller.

  • Requirements for Oracle VM Server for SPARC control domain:

    • Oracle VM Server for SPARC 3.1 or later

    • Oracle Solaris 11, 11.1, and 11.2. The version must be the same or later than the version to run in the vServer. To provide different versions of Oracle Solaris 11, create a control domain for each operating system.

    • For optimal performance between storage elements and control domains, equip each server that hosts a control domain with a Fibre Channel HBA of at least 4 Gb and a 10 Gb Ethernet card. A 1 Gb Ethernet card is the minimum.

    • The right size of networks and virtual assets is not uniform. This document makes recommendations but your purposes and site requirements affect them. For a discussion of sizing, see the Oracle blog for some good practices in tuning resources for virtual assets at https://blogs.oracle.com/jsavit/entry/best_practices_top_ten_tuning

  • Requirements for Oracle VMS Server for SPARC logical domain:

    • Oracle VM Server for SPARC 3.1 or later

    • The version of Oracle Solaris that is available in the Ops Center OS image repository.

  • Requirements for storage resources:

    • Dedicated LUNs exposed only to appropriate WWNs. The LUNs must be fully-provisioned, not thin-provisioned. You can use Fibre Channel LUNs or iSCSI LUNs or both.

    • For Fibre Channel LUNs, create at least one SAN static storage library

    • For iSCSI LUNs, a SAN dynamic storage library is created when you discover the storage server; however, you cannot use this library for the vDC. Create at least one static SAN storage library. Use a 10 Gb Ethernet network connection and tune the number of iSCSI connections. For 8 threads/connection: iscsiadm modify initiator-node -c 8

    • For optimal performance between storage elements and control domains, equip each server that hosts a control domain with a Fibre Channel HBA of at least 4 Gb and a 10 Gb Ethernet card. A 1 Gb Ethernet card is the minimum.

  • Requirements for networks:

    • A fabric backed by an isolated switch to provide separate interfaces to each control domain. an isolated switch is an isolated Ethernet switch or a set of non-routable VLANs spanning multiple network switches. The switch provides the VLAN IDs that provision virtual networks.

      • For host-managed or fully-managed fabric, you must reserve a range of VLAN IDs to be mapped to the control domains. When a private network is needed, one is created on demand, using one of the VLAN IDs, and deleted when no longer needed.

      • For an unmanaged fabric, you must create the private networks manually, using the Create Private Networks action and add them to the network domain. When a private network is needed, an available network is assigned.

    • A dedicated network domain. Do not use the Default network domain.

      • At least one public network to provision OS and to access vServers. Public networks supply a block of IP addresses for provisioning the vServers. The number of IP addresses equals the number of vServers that can be supported. Consider the size of the network that can accommodate all your current and future requirements. Class C (for example, 10.0.0.1/24) networks enable you to create 256 vServers, Class 16C networks (for example, 10.0.0.2/20) enable you to create 4096 vServers.

      • Private networks for Clusterware and internal host connectivity.

    • A set of dedicated VLANs IDs, configured on one or more network interfaces.

  • Requirement for software libraries:

    • Place the libraries on a storage server with high transfer rates.

    • To provision an OS version, the Oracle Solaris 11 Software Update Library must contain the packages for that version. This repository is created during the product installation. In addition, you must download the Oracle Solaris 11 SRU library.

    • Limit the Oracle Solaris 11 Software Library to versions of releases and SRUs that are in active use.

    • Add packages from client kit to the Oracle Solaris 11 Software Library.

32.1.2 Overview of the Configuration of PaaS Solution

To set up the PaaS solution, you perform tasks on each component of the solution:

32.1.2.1 Configure Enterprise Manager Cloud Control

The Enterprise Manager's Virtual Service Adapter is the interface between Enterprise Manager and Enterprise Manager Ops Center. To configure the integration, the main component of Ops Center, the Enterprise Controller, is added as a staging server, as described in Adding the Stage Server.

When the configuration of Oracle Enterprise Manager Ops Center is complete, use the following information to configure Enterprise Manager Cloud Control:

  • IP address of the Enterprise Controller

  • username and password of the cloud user account

32.1.2.2 Obtain the Client Kit

Go to https://support.oracle.com/ to obtain the Ops Center Client Kit, p21190506_121050_Generic.zip. The kit is also available in the OC DVD bundle.

The kit contains packages that provision logical domains, modified to enable PaaS operations, and a script:

  • ORCLsysman-iaas-cli.pkg

  • ORCLsysman-paas-cli.pkg

  • post-script-paas-12.3.0.<buildnumber>.p5p

Extract the files in the compressed file to a location of your choice. These packages and script will be uploaded to Ops Center's repository for Oracle Solaris 11 and used in an OS Update Profile.

32.1.2.3 Configure Enterprise Manager Ops Center

The tasks in this section take 2-3 hours to complete. Some of them can be done simultaneously, that is, you do not have to wait for one job to complete before starting another one.

  1. Add Client Kit Contents to Software Library for Oracle Solaris 11

  2. Preparing Network Resources for the vDC: fabric, network domain, and public network

  3. Preparing Storage Resources: a NAS storage library and a SAN storage library

  4. Create an OS Update Profile

  5. Deploy Control Domains

  6. Create the Server Pool

  7. Create the vDC

  8. Create the vDC account for PaaS

  9. Create a new vServer Type

  10. Create a vServer Template

  11. Create a Private vNet

  12. Validate the Integration and PaaS

32.2 Configure Resources for vDC

A vDC relies on pools of networks, storage, and control domains. It provisions its servers from images in a current OS repository according to profiles in the repository.

32.2.1 Add Client Kit Contents to Software Library for Oracle Solaris 11

The package that enables the PaaS solution is post-script-paas-12.3.0.1407.p5p. This file is in the compressed file you downloaded from My Oracle Support and resides in the directory where you extracted the contents of the compressed file. This procedure imports the file into Op Center's Oracle Solaris 11 repository so that it can enable communication between the Enterprise Manager Cloud Control staging server and the Ops Center resources.

Use the Add Content action to add ISO image files to the Oracle Solaris 11 Software Update Library manually.

  1. Locate the image in the new repository.
  2. Expand Libraries in the Navigation pane.
  3. Click Software Libraries.
  4. Click Oracle Solaris 11 Software Update Library.
  5. Click Add Content in the Action pane.
  6. In the Repository to add content from field, enter the location of the file using the syntax: file:///<fullpath>/post-script-paas-12.3.0.<buildnumber>.p5p

    If credentials are required, specify the system's credentials, SystemDefinedSupportCredential.

When the job is completed, the file is listed in the Contents tab of the library.

32.2.2 Preparing Network Resources for the vDC

Fabrics are the physical infrastructure, consisting of network interface cards and network switches. Network domains are the logical infrastructure, combining network resources from at least one fabric. The networks used by virtual assets are created from the VLAN IDs managed by a network domain. The physical fabrics support these virtual networks, through the network domain, in a manner that depends on their type. To supply network resources to a vDC, you create a network domain from an appropriate fabric.

You must have the role of Network Admin to create network domains.

32.2.2.1 Identify a Fabric

To see the available fabrics, expand Networks in the Assets pane. Select Fabrics. For each fabric, the Dashboard in the center pane includes a Management Capability field. You have these options for providing physical network resources to a new network domain:

  • Use an existing host-managed fabric.

  • Convert an unmanaged fabric to host-managed and then use it.

  • Create a new host-managed fabric.

  • Use an unmanaged fabric.

To convert an unmanaged fabric to a host-managed fabric, use the Assign VLAN ID Range action. Specify a range that matches VLANs on the isolated switch that supports the fabric. If the fabric is not supported by an isolated switch, you can use an arbitrary VLAN ID range representing configured, non-routed VLANS, for example, 1001-1200.

To create a new host-managed fabric:

  1. In the Navigation pane, navigate to Networks, then select Fabrics from the drop-down list.
  2. In the Actions pane, click Define Ethernet Fabric.
  3. In the wizard, enter a name in the Fabric Name field. Enter a description. Click Next
  4. Enter the range of VLAN IDs. If the fabric is supported by an isolated switch, specify a range that matches the switch's VLANs. If the fabric is not supported by an isolated switch, use an arbitrary VLAN ID range, for example, 1001-1200.
  5. Select the networks to be associated with the fabric, then click Next.
  6. Review the Summary, then click Finish. The new fabric is host-managed

To use an unmanaged fabric, assign a set of its public networks as private networks or convert the fabric to an host-managed fabric.

32.2.2.2 Create a Network Domain

After Ops Center manages the physical fabric, its network resources are in the system-defined Default Network Domain, but you cannot use the Default network domain for a virtual datacenter. Instead, create a network domain specifically to support a server pool that contains a vDC.

A network domain is a container for networks and handles the relationship between the physical fabrics and the server pool. The fabrics provide network resources such as links and IP subnets. Within the network domain, networks that have been discovered or specified are available for assignment. These are called public networks. Their network resources have been defined. Another type of network is dynamic, that is, the network is created when it is needed, using an IP address space available to the network domain. Because they exist only for a specific purpose and only within a specific network domain, they are called private networks.

To use virtual datacenters, the network domain must provide private networks. To use existing public networks for a virtual datacenter, add each network to the network domain, making it a static private network, because existing public networks already have an IP subnet addresses and VLAN IDs.

Note:

Do not attempt to use the Default Network Domain to support a virtual datacenter. Although you are not prevented from specifying the Default Network Domain, its networks will not be available for selection.

Although the network domain is providing the networks to the server pools, this is only a logical association using VLAN IDs. To use the network resources, each physical server that supports the control domains in the server pool must have a physical connection to a fabric in the network domain.

  1. Expand Networks in the Navigation pane.
  2. The Standard views shows Networks. The Default Network Domain is selected.
  3. Click Create Network Domain in the Actions pane.
  4. In the Identify Network Domain pane, enter the name and description of the network domain. You can also add tags.
  5. In the Add Fabrics step, choose at least one fabric.
  6. Accept the default value for the Network Creation Limit. This value is the maximum number of dynamic private networks that can be in use simultaneously and is set by the number of VLAN IDs in the range of VLAN IDs you set.

    A private network uses an IP address from the pool of available IP addresses in the network domain. To exclude IP addresses that your organization is using for other purposes, specify them in the Reserved IP Address Ranges table. These addresses will not be available to dynamic private networks.


    Description of GUID-C066CDA5-5033-4400-BC08-6A5CD76F19F9-default.png follows
    Description of the illustration GUID-C066CDA5-5033-4400-BC08-6A5CD76F19F9-default.png

  7. In the Associate Networks step, you add existing networks to the new network domain. Add the network you want to use as the public network. This network will be used to provision vServers.
  8. Review and click Finish.

    When the job is completed, the new network domain is listed in the Navigation pane.

32.2.2.3 Configure Public Network

Select the public network and then select Edit Managed IP Ranges. Specify a range or a series of IP addresses, as provided by your network administrator. These addresses will be used later to support logical domains.

32.2.3 Preparing Storage Resources

When Ops Center discovers a storage device, it creates a dynamic storage library, but these libraries cannot be used to support a vDC. Instead, create static storage libraries:

  • Create a NAS storage library to store the metadata of the control domains. This storage is exclusive for the vDC. Do not use this library for any other purpose.

  • Create at least one SAN library to store the operational data.

The storage libraries are block storage, which use Logical Unit Numbers (LUNs). The LUNs are backed by Fibre Channel disks or iSCSI disks in the form of LUNs.

SAN LUNs are associated with Fibre Channel or iSCSI target groups. Fibre Channel targets use a dedicated optical network and iSCSI targets use the IP network, but in both protocols, the targets in a target group expose one or more LUNs as a storage resource for the target's initiators.

For both protocols, acts as the initiator. To allow the target group and to identify each other as eligible initiators and targets, each one's Fibre Channel World Wide Number (WWN) or iSCSI IQN must be registered with the other one. recognizes the targets because the WWNs or IQNs of the storage server are recorded when the storage server is discovered. Any LUNs that have been assigned to that WWN or IQN are eligible to be used in a storage library.

32.2.4 Create a NAS Storage Library

These are the tasks for setting up a NAS storage library:

  1. Identify the NAS Share

  2. Create a NAS Storage Library

32.2.4.1 Identify the NAS Share

The NFS share is a file system on a NAS storage device. You can view the shares on managed storage servers from the user interface.

  1. Expand Assets in the Navigation pane and click
  2. Click All Assets to expand the section.
  3. Click on the NAS storage server in the Storage section.
  4. In the center pane, click the Storage Shares tab.
  5. View the shares and choose one with capacity to support a server pool. You will use this share in the following procedure.

32.2.4.2 Create a NAS Storage Library

  1. Expand Libraries in the Navigation pane. The new library will be created in the Filesystem Storage section.

  2. Click New NAS Software Library in the Actions pane.

  3. Enter a name for the library and a description. For example, identify how the new library will be used.

  4. Do not associate a server pool with the new storage library. The server pool will be created later. Click Next.

  5. By default, the wizard displays the option for using an exported share of a storage device. Because this procedure has set up an NFS share, click the Other option.

    Description of GUID-EB70B950-B168-4A0F-A7B9-415F0F5CD0C9-default.png follows
    Description of the illustration GUID-EB70B950-B168-4A0F-A7B9-415F0F5CD0C9-default.png

  6. Enter the URL or IP address for the NFS server.

    Description of GUID-1BAEE2B0-A7DC-4A67-8389-A1D7E622D78E-default.png follows
    Description of the illustration GUID-1BAEE2B0-A7DC-4A67-8389-A1D7E622D78E-default.png

  7. (Optional) You can specify the version of NFS that this storage uses. By default, uses the operating system's default NFS version. To specify a different version, allow the version to be changed.

    1. Click the Override NFS Version option.

    2. Click the drop-down list of NFS versions. Select a version.

    Description of GUID-42341810-3631-4183-A9EE-22C6E4718C62-default.png follows
    Description of the illustration GUID-42341810-3631-4183-A9EE-22C6E4718C62-default.png

  8. Click the Next button to review a summary of the storage library.

  9. Click the Finish button to submit the job.

When the job is completed, you can see the new storage library in the Libraries section of the Navigation pane.

32.2.5 Create SAN Storage Libraries using Fibre Channel LUNs

. This section describes how to set up a static block storage library, backed by Fibre Channel LUNs. The number of LUNs in the storage library determines the number of virtual hosts that the library can support.

32.2.5.1 What You Will Need

You will need the following to set up and use a block storage library:

  • LUNs backed by a managed SAN server using the Fibre Channel protocol

  • The role of Storage Admin

To create LUNs on the storage server, you must have the user account and password to log into the storage server and create the LUNs as targets.

32.2.5.2 Configure the Storage Server

If you need to create LUNs and make them available to , see the storage server's documentation for instructions in how to perform the following:

  • Configure the initiator and the targets. The initiator () must be able to recognize the targets (LUNs) and the targets must be able to recognize the initiator. recognizes the targets because the WWNs of the storage server are recorded when the storage server is discovered. Any LUNs that have been assigned to that WWN are eligible to be used in a storage library. On the storage server, you must specify 's WWN as an initiator and assign LUNs to that initiator.

  • Enable multipathing on the Fibre Channel ports. Multipathing is enabled by default on Oracle Solaris x86-based systems, but is disabled by default on Oracle Solaris SPARC-based systems. Use the stmsboot -e command to enable multipathing.

  • Create new LUNs. It can take several hours for a new LUN to be displayed in 's user interface.

    To verify iSCSI LUNs are available to the virtualization host, use the following command:

    # iscsiadm list target -S

    To verify Fibre Channel LUNs are available to the virtualization host, use the following command for Oracle Solaris 11:

    # fcadm lu

32.2.5.3 Create a Static Block Storage SAN Library

To specify the LUNs in a storage library, you specify the server pool that will use the storage library and then select LUNs from among the LUNs available to the server pool.

  1. Expand Libraries in the Navigation pane.
  2. Click New SAN Storage Library in the Actions pane.
  3. Enter the name and description of the library. Click Next.
  4. You have the option to enable multipathed storage for guests of Oracle VM Servers. When you click the LUNs sharing to LDOMs option, each logical domain can get access to its storage through any of the domains in the server pool. If you do not enable multi path storage, the logical domain gets access to storage through its primary domain. Click Next.
  5. In the Identify LUNs step, select at least one LUN from the list of available LUNs. Because this is a static block storage library, the number of LUNs you select determines the number of virtual hosts that the library can support. For example, if you select eight LUNs, eight virtual hosts can use the storage library. You can select one LUN and then add more LUNs later. Click Next.
  6. Review the details of the storage library in the Summary pane.
  7. Click Finish to submit the job.

When the job is completed, the new storage library is displayed in the Libraries section of the Navigation pane in the Static Block Storage section. The center pane shows the library's Summary tab.

32.2.6 Create an OS Update Profile

The Oracle Solaris 11 OS Update profile installs packages as part of an OS Provisioning profile. This example shows how to create an OS Update profile that installs the integration package. You then add this profile to an OS Provisioning profile.

  1. Expand Plan Management in the Navigation pane, then select Update Profiles under Profiles and Policies.
  2. Click New Profile in the actions pane.
  3. Enter a name and description for the profile. In this case, the profile is PaaS base update. Then select Solaris 11.
  4. Keep the default filters and then search for each one of the following packages and files and click the Install icon.

    SUNWhea

    SUNWsprot

    developer/assembler

    developer/java/jdk-6

    developer/java/jdk-7

    post-script-paas

  5. Click Create OS Update Profile. The profile appears in the list of Update profiles.

32.2.7 Deploy Control Domains

32.2.7.1 Create an Provisioning Profile for Oracle VM Server for SPARC

This profile provisions the Oracle Solaris 11 OS and includes the OS Update profile you created in Create an OS Update Profile.

  1. Select Plan Management section and expand Profiles and Policies in the Navigation pane.
  2. Select OS Provisioning profile and click Create Profile in the Actions pane.
  3. Enter the following details in the Create OS Profile - OS Provisioning wizard:

    • Name and description of the profile. In this case, the name of the profile is control_domain_osp.

    • Select Oracle VM Server for SPARC as the Subtype and V class as the target type


    Description of GUID-D821FDAE-CAEF-4BA8-8451-558FCC8CDDE9-default.png follows
    Description of the illustration GUID-D821FDAE-CAEF-4BA8-8451-558FCC8CDDE9-default.png

    Click Next.

  4. Select the following parameters:

    • For the Oracle VM Server for SPARC version, select Solaris 11.2 SRU 11.5.0 (LDom 3.2.0.1)

    • For the Systems Software roup, select solaris-small-server.

    • For the Solaris 11 Update profile, select the OS Update Profile that you created: PaaS base update


    Description of GUID-4BF21026-FA91-4F92-9780-9D2E0A7E1E37-default.png follows
    Description of the illustration GUID-4BF21026-FA91-4F92-9780-9D2E0A7E1E37-default.png

    Click Next.

  5. Keep the default values for the OS setup parameters or edit the language, time zone, and NFS4 Domain values for your environment.
  6. Enter the root password and confirm the password.
  7. Clear the Manual Net Boot option and the Save NVRAMRC option.

    Click Next.

  8. Specify the user account for Oracle Solaris 11 OS. Root login is not enabled in Oracle Solaris 11 OS. Create a user account to SSH to the OS after provisioning. Provide a user name and password for the account.
  9. Do not use iSCSI disks for provisioning Oracle VM Server for SPARC. Clear this option and click Next.
  10. The root (/) and a swap file system are defined by default but you must change the swap size, according to recommendations for sizing and performance. In general, each logical domain that this control domain supports will need 1 GB.

    Use a minimum of 8192 MB for the swap size.


    Description of GUID-586D1033-813A-46B6-95EF-3816F5771F99-default.png follows
    Description of the illustration GUID-586D1033-813A-46B6-95EF-3816F5771F99-default.png

    Click Next.

  11. Select DNS as the naming service in place, and provide the details, including the server names.

    Click Next.

  12. Review the parameters selected for the profile and click Finish to create the OS provisioning profile.

The profile appears in the center pane and in the Profiles and Policies section of Plan Management.

32.2.7.2 Create an OS Configuration Profile for Oracle VM Server for SPARC

The OS Configuration profile lets you define the network interface that you want to use on the operating system. The first interface you define is the boot interface and also the primary interface. You can add more than one network interface. When you apply the plan to a target, you can select which network interface is the primary interface and define the IP address.

  • Control domain configuration such as CPU Threads or Whole-cores, memory, Crypto Units, Virtual Console Port Range and Virtual Disk Server name.

  • Options such as enabling MPxIO and SR-IOV, detaching the unused buses, and saving NVRAMRC values.

  • OS management option to install the Agent Controller.

  • Networking options to use IPMP or Link Aggregation.

The recommended minimum configurations for the control domain are described in the following sections.

32.2.7.2.1 CPU Resource Allocation

The number of system CPUs determines the number of control domain CPU threads:

  • For less than 16 system CPUs, set the control domain CPU Threads to 2.

  • For between 16 and 64 system CPUs, set the control domain CPU Threads to 4.

  • For more than 64 system CPUs, set the control domain CPU Threads to 8.

You can select to allocate CPU resources either as CPU Threads or Whole-core. Whole-core is the default value in Oracle Enterprise Manager Ops Center. When you allocate as Whole-core, all the CPU Threads in the core are allocated to the control domain. For example, when you allocate two cores in UltraSPARC T2 servers, the control domain is allocated with all the 16 CPU Threads in the core. You can also set the maximum cores constraint when you select Whole-core allocation type. The maximum number of cores constraint specifies the number of cores that must be assigned to the domain.

32.2.7.2.2 Crypto Units

Crypto units are the resources on the supported platforms that provide high-performance, dedicated cryptographic engines. These can be used for tasks such as encrypting and decrypting network traffic between a Secure Socket Layer (SSL) web server and an application server.

Each CPU core has one Crypto unit and four or eight CPU threads. Because the Crypto unit is part of a core, the Crypto unit is bound only to domains that contain at least one thread from the parent core. Crypto units cannot be split as CPU threads are split. For example, you have assigned the Crypto unit for the first CPU core to the control domain. When a new logical domain is assigned a thread from the first CPU core and the Crypto unit for that core is already assigned, the control domain cannot assign that Crypto unit to the new logical domain. Allocation of Crypto units might not succeed, especially when a core is split between domains. An Oracle VM Server might allocate fewer Crypto units or none at all.

You must assign at least one Crypto unit to the control domain because the Crypto unit enables domain migration.

The use of Crypto Units is not mandatory although it might speed the logical domain migration. Allocation of Crypto Units might not be available in all the hardware.

32.2.7.2.3 RAM

The amount of RAM for the control domain depends on the size of the system RAM and the load of the system.

  • For system RAM less than 8 GB, set the control domain's RAM to 1 GB.

  • For system RAM between 8 GB to 16 GB, set the control domain's RAM to 2 GB.

  • For system RAM greater than 64 GB, set the control domain's RAM to 8 GB.

In Oracle Enterprise Manager Ops Center the default value is 4 GB as a recommended starting point for logical domains, and the minimum value is 1GB.

32.2.7.2.4 NVRAMRC Value

Automatic booting on a SPARC system uses the default boot device that is defined in the non-volatile RAM (NVRAM). User-defined commands that are executed during start-up are stored in the NVRAMRC file in the NVRAM. When you run an OS provisioning job on a SPARC machine, Oracle Enterprise Manager Ops Center resets the configuration to the factory default configuration and removes the information that is stored in the NVRAMRC file. The control domain OS configuration profile gives you the option to preserve the information in the NVRAMRC file before resetting the server to the factory defaults, and then restore the information after the reset.

To create an OS Configuration profile:

  1. Select the Plan Management section and expand Profiles and Policies.
  2. Select OS Configuration and click Create Profile in the Actions pane.
  3. Enter a name and description in the Create Profile - OS Configuration wizard. In this case, the new profile has the name install_cdoms_osc
  4. Select Oracle VM Server for SPARC as the Subtype.
  5. Click Next.
  6. Select the Oracle VM Server for SPARC version to be installed. The version must match the version selected in the OS Provisioning profile.
  7. Specify the resources that you want to assign to the control domain, according to the recommended minimum configuration. The remaining resources are then available for the logical domains.

    • CPU Model: Select Whole-core to allocate the CPU resource in cores.

    • CPU Cores: Enter the number of CPU core to be allocated to the control domain.

    • Max CPU Cores: Enter the number of CPU cores that must be assigned to control domain.

    • Memory: Enter the amount of memory required for control domain.

    • Do not provide any values for Requested Crypto Units.

    • Virtual Console Port Range: Enter the range between 5000 to 6000.

    • Keep the default name of the virtual disk server.

    Clear the following options:

    • Enable Single Root I/O Virtualization (SR-IOV)

    • Detach Unused Buses


    Description of GUID-07BC4877-EE17-4543-8884-55680F8D9E42-default.png follows
    Description of the illustration GUID-07BC4877-EE17-4543-8884-55680F8D9E42-default.png

    Click Next.

  8. Select the option Enable Multiplexed I/O so that you can associate block storage libraries such as SAN and iSCSI for storage with the control domain.
  9. Select None as the networking option for Oracle VM Server for SPARC.
  10. Select the network interface to use for OS provisioning. Select the Controller that hosts the network interfaces and the corresponding network interface. The Address Allocation is Use Static IP by default and cannot be modified.

    Select Auto to create a virtual switch automatically for the network connection to the control domain. The virtual switch is created in the default format. For example, the network 192.0.2.0/24, the virtual switch is created as 192.0.2.0_24.


    Description of GUID-198D73BB-33EF-4397-86F8-2205D2BBD76A-default.png follows
    Description of the illustration GUID-198D73BB-33EF-4397-86F8-2205D2BBD76A-default.png

    Click Next.

  11. Review the parameters and click Finish to create the OS configuration profile.

The job is initiated and the new OS configuration profile appears under Profiles and Policies in the Plan Management section. You can select the OS Configuration profile to view its details in the center pane.

32.2.7.3 Apply the Deployment Plan

Apply a deployment plan to provision Oracle VM Server for SPARC.

When you apply a deployment plan to provision Oracle VM Server for SPARC, you must have the following information to complete the installation:

  • In the server hardware, obtain the network interface that is physically connected to the network that is managed by Oracle Enterprise Manager Ops Center.

  • Both tagged and untagged networks are listed for network configuration. When OpenBoot PROM (OBP) is used, only untagged networks can be used for OS provisioning as you cannot boot from a tagged network.

  • The IP address for the boot interface.

  • If you want to identify the network interface using the MAC address, you can select to enter the MAC address instead of the boot interface.

  • The details of network connection that you want to use to configure the OS after booting. For multiple network configuration, the first interface is always overwritten by the boot interface that is defined during the OS provisioning step. For single network configuration, it is same as the boot interface network.

  • Do not select the option to place the Oracle VM Server for SPARC in a server pool.

  1. Select the Provision OS plan in the Deployment Plans list.
  2. Select Apply Deployment Plan in the Actions pane.
  3. Select one or more assets and add to the target list.
  4. Select the plan to be applied with minimal interaction. If required to change the profile parameters, then select Allow me to override any profile values.
  5. Select not to review the steps that are not included in the plan and click Next.
  6. The wizard collects information for provisioning Oracle VM Server for SPARC. Click Next.
  7. In the Boot Interface Resource Assignments step, provide the following information:

    • Network: The network for the boot interface.

    • Controller: Select the controller that provides the network interface for OS provisioning. It is always default for the Oracle VM Server for SPARC provisioning.

    • Interface: Select the net0 interface from the list. This network interface is physically connected to the selected network.

    • IP Address: Enter the IP address for the boot interface.

    • (Optional) Primary Hostname: Enter the host name for Oracle VM Server for SPARC.

    If you want to identify the network interface by its MAC address, then select the option Identify Network Interface by MAC Address and enter the MAC address instead of selecting the Controller and the Interface.

    Click Next to view the OS provisioning summary.

  8. Review the parameter of OS provisioning and click Next.
  9. The following steps in the wizard collects information about OS configuration.

    Click Next.

  10. Specify the network resources that were defined in the profile. Select the network and for each network, select the network interface and enter the IP address.

    The first network interface listed is the boot interface. For multiple network configuration, the first network interface is always overwritten by the boot interface network. You can select which is the primary network interface after the provisioning of the OS.

    For single network, the boot interface network will be the defined as the primary network during OS configuration.

    Click Next.

  11. Do not select the option to add the Oracle VM Server for SPARC to a server pool.

    Click Next.

  12. Review the summary of the OS configuration parameters and click Next to schedule the job.
  13. Schedule the provisioning job to run immediately.

    Click Apply to apply the deployment plan on the selected targets.

32.2.8 Create the Server Pool

A server pool is a group of control domains that has access to the same virtual and physical networks and storage resources. Server pools provide load balancing, high availability capabilities, and sharing of the resources for all the members of the pool.

This procedure creates a server pool containing the control domains you created, both storage libraries you created, and the network domain you created.

Before you begin, you must know the network interface that is the physical connection to the system that supports the control domain.

  1. Select Server Pools in the Resource Management View of the Navigation pane.
  2. Click Create Server Pool in the Actions pane.
  3. Enter a name and description for the server pool.
  4. Select Oracle VM Server - SPARC from the Virtualization Technology list.
  5. Click Next.
  6. Select the Oracle VM Servers to be added to the server pool.
  7. Select the new network domain to be associated with the server pool. When you select a the network domain, the Specify Physical Interface table is populated with the fabrics in the network domain.
  8. Specify how each fabric connects to each Oracle VM Server.

    Note:

    In the Physical Interface column, for each Oracle VM Server, select the name of the NIC that the system that hosts the Oracle VM Server uses to connect to the fabric. The NIC that is displayed initially is not a default; you must specify the NIC to use.

    Click Next.

  9. Click the Add Network icon to associate the networks in the network domain with the servers in the server pool. Enter the number of connections to the network as 1.
  10. Specify the network configuration settings for each Oracle VM Server. Both Oracle VM Servers are already connected to the selected network. The UI displays the existing connection details to the network.

    Select interfaces for each fabric you selected in network domain:

    • For the fabric that supports the public network (the routable network), select any existing interfaces to the control domain. If no interface has been connected to the control domain, select the interface that is connected to the Ethernet switch.

    • For the host-managed fabric, select the interface that is connected to Ethernet switch. Do not use bonded interfaces.

    Note:

    In the NIC column, for each Oracle VM Server, select the name of the NIC that the system that hosts the Oracle VM Server uses to connect to the fabric. Specify the same NIC that you chose in Step 8.


    Description of GUID-377FC587-B057-4C8D-B4DB-06ABDFBC8309-default.png follows
    Description of the illustration GUID-377FC587-B057-4C8D-B4DB-06ABDFBC8309-default.png

    Click Next.

  11. Storage libraries that are reachable from the selected members of the pool are displayed. Select the NAS and the SAN storage library to be associated with the server pool. Do not select a dynamic storage library.
  12. Select the placement and auto balancing policy:

    • Place the guest on Oracle VM Server with lowest relative load.

    • Do not auto balance the server pool.

    • Clear the option for Automatic Recovery.


    Description of GUID-927632D0-8E3F-4499-9612-35A30EDC3354-default.png follows
    Description of the illustration GUID-927632D0-8E3F-4499-9612-35A30EDC3354-default.png

    Click Next.

  13. Review the server pool information, then click Finish to create the server pool.

The server pool is created and listed in the Server Pools view of the Assets section.

32.3 Set Up and Configure a Virtual Data Center

32.3.1 Create the vDC

The vDC inherits network resources and storage resources from the server pool. The vDC enables cloud users to create virtual machines and run applications. The cloud user provides networking to its virtual machines from the public networks that are allocated to each account or by creating private networks. The cloud user can use the public networks, or create a private vNet for use in the account. When the cloud users create a private vNet, either a dynamic private network is created or the static private network is available for use in that account.

  1. Select vDC Management in the Navigation pane.
  2. Click Create Virtual Datacenter in the Actions pane.
  3. The first step Create Virtual Datacenter Wizard is an introduction to the vDC and the prerequisites for creating a vDC. Click Next.
  4. In Specify Virtual Datacenter Details:

    • Provide a name and description for the vDC.

    • Enter tags to distinguish this vDC from other vDCs.

    • Select Password Required to enforce using credentials when creating vServers in the vDC. When this option is enabled, cloud users must specify a root password, with or without an SSH key. In addition, cloud users must specify either an SSH key or remote user credentials when creating vServers that use Oracle Solaris 11 OS.

    Click Next.

  5. Select the new server pool from the list.
  6. Select one or more boot networks from the public network. You must select at least one network.

    Note:

    Boot networks are required for vServer OS deployment. You must also add at least one boot network to each account in the vDC for creating vServers in the account. You can also assign a boot network to an account and use it as a public network for that account.

    Click Next.

  7. The vCPU sizing displays this information:

    • Total number of vCPUs: The total number of vCPU in the vDC for the updated vCPU to physical CPU Threads ratio.

    • Avg memory per vCPU: The average memory per vCPU in GB. The total memory available for the vDC by the total number of vCPUs. When you want to use the CPU and memory resources to the full extent, the vServers must use this amount of resources.

    Accept the default values and click Next.

  8. Select Static Block Storage Libraries as the storage type for the root disk.
  9. Select the SAN storage library you created.
  10. Select Static Block Storage Libraries as the storage type for the volume and template.
  11. Select the SAN storage library you created.

    Click Next.

  12. Confirm the vDC configuration and click Finish to create the vDC. The vServers will run in this vDC, according to the vServer type.

32.3.2 Create the vDC account for PaaS

In creating the account, you allocate resources from the vDC to the account. The resource allocation for all the accounts in a vDC can be more than the actual resources in a vDC. This oversubscription of the resources must be planned. You must update the resources when the requirement increases.

  1. Expand vDC Management on the Navigation pane.
  2. Expand vDCs, select the new vDC, then click Accounts.
  3. On the Actions pane, click Create Account.
  4. Enter the name for the account: ocadmin.

    Click Next.

  5. Specify the quotas of vCPU, Memory, and Storage for the account.

    For this PaaS account, specify the maximum capacity of each resource.


    Description of GUID-3FE552D4-4076-46FB-ACB7-9B21357D8CA6-default.png follows
    Description of the illustration GUID-3FE552D4-4076-46FB-ACB7-9B21357D8CA6-default.png

  6. In the Number of private vNets field, enter the number of private vNets for this account, which is the same as the number of VLAN IDs assigned. Click Next.
  7. Select the cloud user that you want to assign to the account. Use the right arrow key to add this cloud user to the list of Assigned Cloud Users. You can use the other arrow buttons to adjust the list. Do not assign the root user.
  8. View the Summary and click Finish to create the account. When the job is completed, a new account is created.

32.3.3 Preparing the vServers

A vServer is an entity that provides the outward interface of a stand-alone operating system that consumes CPU, storage, and memory resources. A vServer has its own identity, local storage, interfaces, and configuration that exist for the full lifetime of the vServer. You determine the creation of new vServers according to the account quota limits and applications requirements.

When creating a vServer, the following account resources are required:

  • A vServer type: vServer Types are profiles that define the computing resources such as virtual CPU, memory, and storage size. You select the best suitable vServer type from those available for the account. vServer types are visible to cloud users during the vServer creation process.

  • A server template: Server templates designate the OS provisioning and OS configuration for the vServer. You can either select a server template from those that exist for the account or create a new server template.

  • One or more virtual networks: For the vServer network connectivity you must choose one or more vNets from the available vNets or create new ones before creating a vServer. vServers are only assigned to virtual networks at vServer creation time.

32.3.3.1 Create a new vServer Type

When you create a vServer type, the VM hosting details display the following information in the wizard based on the resources defined:

  • The number of virtualization servers in the vDC that have sufficient physical resources to host a vServer with the selected resources.

  • An estimation of number of vServers that can be hosted with the total number of physical resources of the vDC.

  • A warning when the current value of the memory size exceeds the selected storage size.

  1. Select the vDC and click Create vServer Type in the Actions pane.
  2. Provide a name and description to identify the type. For example:

    • single instance DBAAS

    • RAC instance DBAAS

    • RAC big instance DBAAS

  3. Add tags for identification and classification of the vServer type. Click Next.
  4. Specify values for vCPU, memory and storage resources. Click Next. The minimum amount of RAM for Oracle Solaris 11.2 is 2 GB. For example, Table 32-1 shows the recommended values for the DBaaS instances deploying Oracle 12c Database.

    Table 32-1

    Type RAM root vCPU

    single instance DBAAS

    4 GB

    45 GB

    16

    RAC instance DBAAS

    5 GB

    45 GB

    16

    RAC big instance DBAAS

    12 GB

    45 GB

    16
  5. Review the information provided and click Finish to create the vServer type.

32.3.3.2 Create a vServer Template

Create a vServer template from the new OS profiles you created: provisioning, which includes the update profile, and configuration.

  1. Select the account in the vDC Management section.
  2. Click Create Server Template in the Actions pane.
  3. In the wizard, skip the introduction and click Next.
  4. Enter the name and description for the server template.

    Click Next.

  5. Select the new OS provisioning profile, then click Next.
  6. Select the new OS configuration profile from the list, then click Next.
  7. Review the information and click Finish to create the server template.

The new template is created and available to create vServers.

32.3.3.3 Create a Private vNet

Private vNet is a private virtual network set up exclusively for an account. The vServers associated with this vNet have private virtual IP address for internal communication. In this PaaS solution, only RAC instances use private networks.

  1. Expand vDC Management on the Navigation pane.
  2. Expand vDCs, select the new vDC, then click Accounts.
  3. Select the account you created.
  4. On the Actions pane, click Create Private vNet.
  5. Enter the name and description for the private virtual network. Click Next.
  6. Use the slide bar to set the value of the number of elements. The values entered are rounded up to the next value of 13, 29, 61, 125, 253, 509, 1021, 2045, 4093, and 8189. Do not use the values of 1 or 5. You cannot change the size of a network after it is created so specify the size needed to accommodate your current and future requirements:

    • The number of elements is also the maximum number of vServers that can be part of this vNet. Class C networks (for example, 10.0.0.1/24) enable you to create 256 vServers and Class 16C networks (for example, 10.0.0.2/20) enable you to create 4096 vServers.

    • Whether the vNets will be shared or dedicated. If multiple RAC instances will share one or more networks, set the network size to 509 and more. If each RAC instance will have a dedicated network, limit the network size to 29 or 13.


    Description of GUID-1A387EE8-6E5C-4A59-BD5D-D567440CB63F-default.png follows
    Description of the illustration GUID-1A387EE8-6E5C-4A59-BD5D-D567440CB63F-default.png

  7. Click Next.
  8. Review the Summary, then click Finish to create the private virtual network.

    The new private virtual network is listed in the Networks and vDC sections of the Navigation pane.


    Description of GUID-49E3E549-C66A-4941-AF10-28778BBFDA55-default.png follows
    Description of the illustration GUID-49E3E549-C66A-4941-AF10-28778BBFDA55-default.png

  9. Connect the IP address from this new private virtual network to the vServer, using Connect Private vNet icon in the Private vNets table.

    Note:

    If you have multiple vNets, connect each vNet one at a time. Wait for each job to complete before making the next connection.

32.4 Validate the Integration and PaaS

Test the connections from Enterprise Manager Cloud Control and Enterprise Manager Ops Center to the cloud user. In this test, create two vServers with the same storage and network resources. If successful, one vServer can send information to and receive information from the other vServer.

32.4.1 Create Volumes for vServer

Use the following procedure to create volumes from shared disks. In normal operations, Enterprise Manager creates the volumes.

  1. Expand vDC Management in the Navigation pane.
  2. Select the account from the vDC Accounts list.
  3. Click Create Volume in the Actions pane.
  4. Enter a name for the volume.

    Click Next.

  5. Check the Shared option.
  6. Enter the size of the volume.

    Click Next.

  7. Review the volume information and click Finish to create the volume.
  8. Delete the volume when the job has completed.

32.4.2 Create Test vServers

Create two vServers with the same private network.

  1. Expand vDC Management in the Navigation pane.
  2. Select the account from the vDC Accounts list.
  3. Click Create vServer in the Actions pane.

    The Create vServer Wizard is displayed.

  4. Enter the following information in the vServer Details step:

    • Name and description for the vServer.

    • Tags for better identification and classification of the vServer.

    • Number of vServers to create.

    • The High Availability Support option has no effect in the PaaS integration.

    Click Next.

  5. Select a server template from the list. Click Next.
  6. Select a vServer type from the list. Click Next.
  7. Select one or more volumes from the Available Volumes list. Use the arrow keys to move the selected volumes to the Attached Volumes list.

    For this test, attach the same volume to both vServers.

    Click Next.

  8. Select one or more vNets from the list. Click Next.
  9. Select the Static method for assigning the IP address.
  10. Assign the IP address from the new private virtual network to the vServer. Click Next.

    You can also perform this step after you create the vServer by clicking the Connect vNet icon on the account's Networks tab in the Private vNet table.

    Do not select a distribution group. Click Next.

  11. Specify a root password. Click Next.
  12. Confirm the vServer information provided in the Summary and click Finish to launch the job to create the vServer.

    After the job completes, the vServer is created and listed in the Navigation pane. By default, the DNS and other naming information is taken from the selected vNet or server template and added in the /etc/resolv.conf file of the vServer.

32.4.3 Verify Shared Storage and Connectivity

To verify connectivity, create two vServers. From the console, log into one vServer and use the ping command with the IP address of the other vServer. This succeeds on public network or private network.

To verify shared storage:

  • For a Fibre Channel SAN storage:fcadm lu

  • For iSCSI SAN storage: iscsiadm list target -S

The Client Kit also contains the new_vDC.ksh script that can be useful for diagnosing problems. Enterprise Manager uses the file when it starts the integration, using the file path /tmp/INPUT/bootstrap.xml but you can also perform this manually.

Note:

After you finish testing, delete the bootstrap.xml file and the oc-pass.txt files because they contain security information such as passwords and URLs.

  1. Create a file named bootstrap.xml in the following form and using your site's information. 
    <bootStrap version="1.0" name="boot_strap"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="bootstrap.xsd">
<!-- Ops-center details required to establish connection before running the ops center commands -->
<opsCenterDetails>
           <opsCenterURL>https://<yourhostname>.com</opsCenterURL>
                   <!-- OpsCenter credentials -->
                   <opsCenterUser>ocadmin</opsCenterUser>
                   <opsCenterHome>/opt/oracle/iaas/cli/</opsCenterHome>
           <!-- Stage location on Ops Center Host -->
           <opsCenterStageLoc>/tmp/</opsCenterStageLoc>
</opsCenterDetails>
 
</bootStrap>
  2. Issue the following commands: 
    -bash-4.1$ echo password > /tmp/INPUT/oc-pass.txt
-bash-4.1$ /opt/oracle/paas/tools/new_vDC.ksh
  3. Review the output: 
    Certificate added to truststore /export/home/ocadmin/.oracle_iaas/truststore
Vserver types
4159 EXTRA_LARGE Logical Domain extra-large instance type for vDC PaaS_vDC 17179869184 28          4   
4158 LARGE       Logical Domain large instance type for vDC PaaS_vDC       8589934592  24          2   
4157 SMALL       Logical Domain small instance type for vDC PaaS_vDC       4294967296  20          1   
4161 paas_RAC                                                              12884901888 45          16  
4160 paas_SI                                                               4294967296  45          16  
Vnets
VNET-321a93e4-2d11-4642-b3ec-3ebde6e86b04 big_private_net  big_private_net OK     172.16.0.0/20  private        
VNET-badc34e5-021a-44c7-a7ad-d11915e04ec3 small_rac_net1   small_rac_net1  OK     192.168.0.0/28 private        
VNET-47f06e55-7c72-4116-919f-bb940c978b8a 10.163.96.0/22.1                 OK     1x.16x.x6.0/22 public_external
Vserver templates
TMPL-116826d2-6a96-4217-ab16-0d4a71157028 paas_default             OK     12884901888 false            false

32.4.4 Diagnosing Problems

If a virtual datacenter is unable to create vnets, edit the network domain to increase the limit of private networks it can provide simultaneously.

If control domains are already running and you change the fabric that supports the server pool, you will need to configure the Fibre Channel controller:

  1. fcadm hba-port
  2. Identify active ports.
  3. fcadm remote-port -p <each_portID_from_output>
  4. # cfgadm -c configure <controller_to_be_configured>