11 Enabling Hybrid Cloud Management

With Oracle Hybrid Cloud, you can use the Enterprise Manager Cloud Control console to administer both your on-premises and Oracle Cloud deployments. Oracle Hybrid Cloud lets on-premises Enterprise Manager administrators monitor and manage cloud services using the same Oracle Enterprise Manager tools they use to monitor, provision, and maintain Oracle Databases, Engineered Systems, Oracle Applications, Oracle Middleware, and a variety of third-party systems.

This chapter consists of the following sections:

What is Oracle Hybrid Cloud?

Your IT infrastructure may consist of a mix of on-premises and cloud-based targets. For example, you may have instances of Oracle Database Cloud Services and Java Cloud Services to manage along with various on-premises software. No matter where your IT assets reside, Enterprise Manager allows you to manage this Hybrid Cloud environment through a single pane of glass.

Configuring Enterprise Manager to manage a Hybrid Cloud environment involves deploying Management Agents throughout your Hybrid Cloud environment to allow your cloud services to communicate with Enterprise Manager. By deploying Agents on the Oracle Cloud virtual hosts running your Oracle Cloud services, you are able to manage these services just as you would any other monitored target from the Enterprise Manager Cloud Control console.

You can monitor four service types with Oracle Hybrid Cloud:

  • Database Cloud

  • Java Cloud

  • Compute Cloud

  • Cloud Machine

Communication between your cloud services and the OMS is secure from external interference. As shown in the following graphic, the on-premises OMS communicates via HTTPS, SQL*Net and JMX over SSH (if VPN is not available) with Agents installed on the VMs running your cloud services.

High-level Hybrid Cloud architectural diagram.

How do I set up Oracle Hybrid Cloud Monitoring?

Setting up your Oracle Hybrid Cloud environment is straightforward. The following table provides a quick how-to reference.

Step What you need to do:

1. Make sure the on-premises host running an Agent can communicate with the Oracle Cloud VMs.

This step is IMPORTANT! All subsequent setup tasks will fail without an open communication channel between the on-premises Agent and the Oracle Cloud.

Ensure there is network connectivity between the OMS and Oracle Cloud. SSH must work between the host that the Gateway is installed on and VMs in the Oracle Cloud.

For more information about Oracle Hybrid Cloud prerequisites, see Hybrid Cloud Management Prerequisites and Basic Setup, Prerequisites for Configuring a Management Agent as a Gatewayand Prerequisites for Installing Agents on Oracle Cloud VMs.

Make sure you have supported cloud services to monitor. Oracle Hybrid Cloud supports the following services:

2. Configure an Enterprise Manager Agent as a Gateway.

See Configuring a Management Agent as a Gateway

Important: Ensure environment requirements have been met before deploying a Management Agent as a Gateway. For more information, see Prerequisites for Configuring a Management Agent as a Gateway

3. (Optional) Set up an external proxy. Configure an external proxy between the Oracle Cloud and the Gateway to enhance security. See Configuring an External Proxy to Enable Gateways to Communicate with the Oracle Cloud.

4. Deploy Agents to the VMs running your Oracle Cloud services.

See Installing an Agent on an Oracle Cloud VM.

Important: Ensure resource and network requirements have been met before installing the Agent. For more information, see Prerequisites for Installing Agents on Oracle Cloud VMs.

Setting Up Hybrid Cloud Management in Three Steps

Now that you understand the Hybrid Cloud management setup flow shown in What is Oracle Hybrid Cloud?, you now know that setting up Hybrid Cloud management is fairly straightforward. Setting up your on-premises Enterprise Manager system to manage and monitor an Oracle Cloud can be done in as little as three steps:

Step 1 Make sure an on-premises Agent can communicate with the Oracle Cloud via SSH.

From the on-premises side: Make sure that an on-premises host running an Agent (13c or above) can connect via SSH with the Oracle Cloud VM you want to monitor.

From the Oracle Cloud side:

  • Make sure the default port is set to 1748, or that one port in the range 1830 to 1848 is free on the Oracle Cloud VM.

  • Make sure the user installing the Enterprise Manager Agent on the Oracle Cloud VM has SUDO privileges in order to run the root.sh script.

Step 2: Configure an on-premises Agent to serve as a Gateway.

Use the EM CLI register_hybridgateway_agent verb to designate an Agent as a Gateway.

emcli register_hybridgateway_agent -hybridgateway_agent_list=“<On-premises target name for the Agent chosen in Step 1>”
     -named_credential="named_credential" 
     -named_credential_owner="named_credential_owner"
     -cloud_hostname="cloud_hostname"

Step 3: Deploy Agents on Oracle Cloud VMs to communicate with the on-premises Gateway.

Before starting the Agent deployment process, make sure you have the following information:
  • IP Address of the Oracle Cloud VM.

  • SSH public keys mapped as Enterprise Manager Named Credentials.

    You can create a Named Credential either through the Enterprise Manager console (Setup –> Security –> Named Credentials) or by using the EM CLI create_named_credential verb.

  • Details about the Gateway you configured in Step 2.

You can deploy the Agent to the Cloud VM using Agent Push functionality from the Enterprise Manager console, or by using the EM CLI submit_add_host verb shown below:
emcli submit_add_host
   -host_names=<IP addresses of Oracle Cloud VM>
   -installation_base_directory=<Path for installing the Agent on the Oracle Cloud VM>
   -credential_name=<Enterprise Manager Credential for the SSH Key>
   -configure_hybrid_cloud_agent -hybrid_could_gateway_agent=<Target Name of the Gateway Agent>
   -hybrid_cloud_gateway_proxy_port=<Port on the Gateway host used for outbound SSH communication>

Hybrid Cloud Management Prerequisites and Basic Setup

Setting up Hybrid Cloud management consists of the following steps:

  1. Ensure that your on-premises OMS is version 13c, and that at least one 13c Management Agent exists in your on-premises environment.

    If your on-premises OMS is an earlier version, ensure that you upgrade the OMS to version 13c. For information on how to do so, see the Oracle Enterprise Manager Cloud Control Upgrade Guide.

    To ensure that at least one 13c Management Agent exists in your on-premises environment, either deploy a new 13c Management Agent, or upgrade an existing Management Agent of an earlier version to version 13c.

    For information on how to deploy a new 13c Management Agent, see Oracle Enterprise Manager Cloud Control Basic Installation Guide. For information on how to upgrade an existing Management Agent of an earlier version to version 13c, see the Oracle Enterprise Manager Cloud Control Upgrade Guide.

    Note:

    Oracle strongly recommends that you first upgrade the earlier version of the Management Agent to 13c and then configure that Agent to serve as a Gateway. This way, your entire stack will be at 13c. However, if you do not want to upgrade the earlier version, you can continue to use it and configure it to act as a Gateway. However, only the earlier version 12c Release 5 (12.1.0.5) is supported in this case. All pre-12.1.0.5 Agents must be upgraded to either 12c Release 5 (12.1.0.5) or 13c.

  2. Configure one or more 13c Management Agents within your on-premises environment to serve as a Gateway. A Gateway provides an SSH-based communication channel between the Oracle Cloud virtual hosts and the on-premises OMS. For more information on configuring an external proxy to enable Hybrid Cloud Gateways, see Configuring an External Proxy to Enable Hybrid Cloud Gateway Agents to Communicate with Oracle Cloud

    To ensure high availability, Oracle recommends that you configure multiple 13c Management Agents to act as Gateways.

  3. Ensure that the on-premises OMS can communicate with the Oracle Cloud targets via the Gateway.

    If the Gateway is unable to communicate with the Oracle Cloud targets directly, configure an external proxy for the communication. For information on how to do so, see Configuring an External Proxy to Enable Gateways to Communicate with the Oracle Cloud.

    To communicate with Oracle Cloud targets, the on-premises OMS uses the My Oracle Support (MOS) proxy by default. You can also configure an Agent Proxy instead of the default proxy. Ensure that the proxy configured in your enterprise supports SSH tunneling, or configure a new MOS proxy that supports SSH tunneling.

  4. Deploy Management Agents to the Oracle Cloud virtual hosts using the Add Host Targets Wizard or EM CLI. and configure them in Hybrid mode. Management Agents configured in Hybrid mode enable Enterprise Manager to manage the Oracle Cloud targets. As part of the Hybrid Cloud Agent deployment process, you will associate each with the Gateway that it will use to communicate with the on-premises OMS.

Prerequisites for Configuring a Management Agent as a Gateway

Before configuring a Management Agent to act as a Gateway, ensure the following prerequisites are met:

  • Ensure there is network connectivity between the OMS and Oracle Cloud. SSH must work between the host that the Gateway is installed on and VMs in the Oracle Cloud.

  • Ensure that the CPU, RAM, and hard disk space requirements are met.

    The CPU, RAM, and hard disk space requirements for a Hybrid Cloud Gateway are described in Oracle Enterprise Manager Cloud Control Basic Installation Guide.

    Note that the hardware requirements for the Hybrid Cloud Gateway and regular Management Agents are the same.

  • (Recommendation) You should install a new Agent on a dedicated host to serve as the Hybrid Cloud Gateway. This ensures high Gateway performance.

    Note:

    Oracle recommends that you do not designate the central Agent as a Hybrid Cloud Gateway. In an enterprise with a large number of targets, the designated central Agent may compete with the OMS for resources.

In general, any prerequisites required for deploying Management Agents also apply to Gateways. For more information, see Oracle Enterprise Manager Cloud Control Basic Installation Guide.

Configuring a Management Agent as a Gateway

To configure an existing 13c Management Agent as a Gateway, follow these three steps:

Note:

You can use an existing Management Agent of an earlier version and configure that to act as a Gateway. However, Oracle strongly recommends that you first upgrade that Management Agent of the earlier version to 13c and then configure that to act as a Gateway. This way, your entire stack will be at 13c.

  1. As SYSMAN user, log in to EM CLI. You can log in from the default EM CLI installation that is available in the OMS home, or from the EM CLI installation that is set up on any other host.
    $<emcli_install_location>/bin/emcli login -username=sysman
    

    EM CLI is set up by default on the on-premises OMS host (the EM CLI install location is the OMS home). Hence, if you choose to run EM CLI from the on-premises OMS host, no additional steps are required. This is the recommended option.

    For example, if you are logging in from the EM CLI installation that is available in the OMS home, then run the following command:

    /em13c/oraclehome/bin/emcli login -username=sysman

    If you choose to run EM CLI from a custom location on a host that is not running the on-premises OMS, you must first set up EM CLI on the required host..

  2. Designate the selected Management Agent to act as a Hybrid Cloud Gateway. To do so, run the register_hybridgateway_agent EM CLI verb from the OMS home or from any other host where EM CLI is set up. The verb can be executed from a command line or from a script.

    The verb takes a list of Management Agents and marks each Agent as a hybridgateway.

    Command Line Mode

        $<emcli_install_location>/bin/emcli register_hybridgateway_agent -hybridgateway_agent_list="<list_of_hybrid_cloud_gateway_agents>"
             [-named_credential="named_credential"]
             [-named_credential_owner="named_credential_owner"]
             [-cloud_hostname="cloud_hostname"]
             [-ignore_central_agent_check]
             [-ignore_network_check]
             [-ssh_port="ssh_port"]
             [-timeout="timeout"]

    Script/Interactive Mode

        $<emcli_install_location>/bin/emcli register_hybridgateway_agent(hybridgateway_agent_list="<list_of_hybrid_cloud_gateway_agents>"
             [,named_credential="named_credential"]
             [,named_credential_owner="named_credential_owner"]
             [,cloud_hostname="cloud_hostname"]
             [,ignore_central_agent_check=True/False]
             [,ignore_network_check=True/False]
             [,ssh_port="ssh_port"]
             [,timeout="timeout"]
             )

    For more information about the register_hybridgateway_agent verb, see the Enterprise Manager Command Line Interface Guide.

    Options:
    • hybridgateway_agent_list

      List of Management Agents that need to be registered as Gateways. You can specify more than one Management Agent (host name and port combination). Ensure that you specify the fully qualified name for the Management Agents, and separate the Management Agent names using a space.

      Multiple Gateways are only needed for failover and load balancing and are not mandatory. Multiple Gateways can be added at initial Hybrid Cloud setup or can be added at a later point in time. See Configuring Cloud-based Agents for High Availabilityfor more information.

    • named_credential

      Named credential used to make SSH connection to the cloud host. This is used for the network check.        

      *Optional only if '-ignore_network_check' is present..

    • named_credential_owner

      Owner of named credential.

      *Optional only if '-ignore_network_check' is present.

    • cloud_hostname

      Cloud hostname where you want to install hybrid agent.

      *Optional only if '-ignore_network_check' is present.

    • ignore_central_agent_check

      Flag used to skip the central Agent check for the specified list of Agents. We recommend not registering the Agent on the OMS host as a Gateway. However, you can use this flag to ignore that check.

    • ignore_network_check

      Flag used to skip the network check for the specified list of Agents. Use this flag only if you are sure that the network connection works from Gateway to the cloud host.

    • ssh_port

      Specifies the SSH port used to check network. 22 is used as the default.

    • timeout

      Specifies the amount of time (in seconds) the network check process will wait for a connection. 5 seconds is the default.

    Example 1: Basic Command Usage

    Standard Mode
    emcli register_hybridgateway_agent -hybridgateway_agent_list="agent1:port agent2:port..." 
         -named_credential="named_credential" 
         -named_credential_owner="named_credential_owner"
             -cloud_hostname="cloud_hostname"
    Interactive or Script Mode
    register_hybridgateway_agent(hybridgateway_agent_list="agent1:port agent2:port...",
                 named_credential="named_credential",
                 named_credential_owner="named_credential_owner",
                 cloud_hostname="cloud_hostname"           )
    
    Example 2: If the '-ignore_network_check' flag is present, the parameters '-named_credential', '-named_credential_owner' and '-cloud_hostname' are not required.
    Standard Mode
    emcli register_hybridgateway_agent -hybridgateway_agent_list="agent1:port agent2:port..." -ignore_network_check -ignore_central_agent_check
    Interactive or Script Mode
    register_hybridgateway_agent(hybridgateway_agent_list="agent1:port agent2:port...",
                 ignore_network_check=True,
                 ignore_central_agent_check=True
               )

    Example 3: . If the '-ignore_central_agent_check' flag is present, but the '-ignore_network_check' flag is missing, the parameters '-named_credential', '-named_credential_owner' and '-cloud_hostname' are required.

    Standard Mode
    emcli register_hybridgateway_agent -hybridgateway_agent_list="agent1:port agent2:port..." -named_credential="named_credential" -named_credential_owner="named_credential_owner" -cloud_hostname="cloud_hostname" -ignore_central_agent_check
    Interactive or Script Mode  
    register_hybridgateway_agent(hybridgateway_agent_list="agent1:port agent2:port...",
                 named_credential="named_credential",
                 named_credential_owner="named_credential_owner",
                 cloud_hostname="cloud_hostname",
                 ignore_central_agent_check=True
                )
  3. Verify that the Management Agent has been configured as a Gateway. You can do this only while installing an Agent on an Oracle Cloud VM as described in Installing an Agent on an Oracle Cloud VM Using the Add Host Targets Wizard.

Prerequisites for Installing Agents on Oracle Cloud VMs

Before deploying Agents on your Oracle Cloud VMs, ensure that the following prerequisites have been met:

  • Ensure that the CPU, RAM, and hard disk space requirements are met.

    The CPU, RAM, and hard disk space requirements for a Hybrid Cloud Agent are described in Oracle Enterprise Manager Cloud Control Basic Installation Guide.

  • Ensure that you configure at least one Management Agent to act as a Gateway. A Gateway provides a communication channel between the Oracle Cloud VMs and the on-premises OMS.

    For information on how to configure a version 13c Management Agent to act as a Gateway, see Configuring a Management Agent as a Gateway.

  • Ensure that port 22 is open on the destination Oracle Cloud virtual host (the virtual host on which you want to install an Agent), and the SSH daemon process must be running on it. To verify whether the SSH Daemon process is running on the destination virtual host, run the following command from the virtual host:

    ps –ef | grep sshd
    

    Note:

    If the SSH daemon is configured and running other than on the default port 22, then make sure the SSH port number is updated in the $MW_HOME/oui/prov/resources/Paths.properties file. For example, if the SSH daemon is running on port 23, then update the parameter SSH_PORT in the Paths.properties file and proceed with deployment.
  • Ensure that port 1748, or at least one port in the range 1830 - 1848 is free on every destination Oracle Cloud virtual host.

    By default, Cloud Control uses port 1748 as the Gateway Proxy port. If port 1748 is not free, the application uses a free port in the range 1830 - 1848.

  • Ensure that the user installing the Agent on the Cloud VM er has the root privileges to run the root.sh script. If the user installing this Agent does not have the root privileges, ensure that you run the root.sh script manually on all the destination virtual hosts, after the deployment operation. Make sure to have a write permission on the directory.

  • Meet the prerequisites required for deploying on-premises Management Agents, as described in Oracle Enterprise Manager Cloud Control Basic Installation Guide.

  • Ensure that the Cloud-based Agent is deployed only on an Oracle Linux x86-64 operating system. It is supported only on Oracle Linux x86-64 operating system.

  • To install an Agent on a Cloud-based virtual host, it is recommended to install it on the local file system of the virtual host. Optionally, you can create a mount using an external storage device and install the Agent on it. Otherwise, you will lose all the data that is stored in the boot volume every time you stop, start, or restart the virtual host.

  • Ensure that you do not modify the domain name in the Virtual Machine (VM) network or host configuration settings. The Agent must be used only for Oracle Cloud-hosted VMs, so if you change the VM domain name to reflect a non-Oracle Cloud-hosted VM, then the Agent deployment will fail.

    To verify this, log in to the VM and run the hostname -d command, and ensure that the output contains oracle.com, oraclecloud.com, or oraclecloud.internal. If you see any other domain name, remove it from the list.

    Also run the following commands, and ensure that the output contains either oracle.com or oraclecloud.internal. If you see any other domain name, remove it from the list.

    cat /etc/sysconfig/network

    cat /etc/resolv.conf

    cat /etc/hosts

Installing an Agent on an Oracle Cloud VM

This section covers the following methods to install an Agent on an Oracle Cloud VM:

Note:

Since an Agent connects to the on-premises OMS through an SSH bridge, manual deployment such as Silent Agent Installation is not supported for Cloud-based Agents. You can only deploy Agents using the Add Host Targets Wizard, or EM CLI.

You can deploy a Cloud-based Agent only on an Oracle Linux x86-64 operating system. An Agent configured as a Gateway is supported on all operating systems.

Installing an Agent on an Oracle Cloud VM Using EM CLI

Follow these steps to install a Cloud-based Agent using EM CLI:

  1. Log in to EM CLI from the /bin directory present within the EM CLI install location:
    $<emcli_install_location>/bin/emcli login -username=<user_name>
    

    Once you run this command, EM CLI will prompt you for a password. Enter the password for the user name you specified.

    EM CLI is set up by default on the on-premises OMS host (the EM CLI install location is the OMS home). Hence, if you choose to run EM CLI from the on-premises OMS host, no additional steps are required. This is the recommended option.

    If you choose to run EM CLI from a custom location on a host that is not running the on-premises OMS, you must first set up EM CLI on the required host. For information on how to do so, see Oracle Enterprise Manager Command Line Interface Guide.

  2. Run the list_add_host_platforms verb to obtain a list of the platforms for which the Hybrid Cloud Agent software is available in Self Update:
    $<emcli_install_location>/bin/emcli list_add_host_platforms 
                          [-all] 
                          [-noheader] 
                          [-script | -format]
    

    Note that the parameters mentioned in [ ] are optional.

    For example, $<emcli_install_location>/bin/emcli list_add_host_platforms -all

    If the Management Agent software for a particular platform is not available, download and apply it using Self Update. For information on how to download and apply the Management Agent software for a platform, see Enterprise Manager Cloud Control Basic Installation Guide.

    To view more information on the syntax and the usage of the list_add_host_platforms verb, run the following command:

    $<emcli_install_location>/bin/emcli help list_add_host_platforms
    
  3. If you want to deploy Agents on the selected Oracle Cloud virtual hosts in a rolling manner, such that the deployment proceeds continuously from one deployment phase to another, ignoring the failed hosts in each deployment phase, specify the following in the $OMS_HOME/sysman/prov/agentpush/agentpush.properties file:

    oracle.sysman.prov.agentpush.continueIgnoringFailedHost=true

  4. Run the submit_add_host verb, specifying the -configure_hybrid_cloud_agent, -hybrid_cloud_gateway_agent, and -hybrid_cloud_gateway_proxy_port options to submit the Add Host session and install the Cloud-based Agents:
    $<emcli_install_location>/bin/emcli submit_add_host 
                          -host_names=<list_of_hosts>
                          -platform=<platform_ID>
                          -installation_base_directory=<install_directory_of_agent>
                          -credential_name=<named_credential_for_agent_install>
                          -configure_hybrid_cloud_agent
                          -hybrid_cloud_gateway_agent=<hybrid_cloud_gateway_agent_name>
                          [-hybrid_cloud_gateway_proxy_port=<hybrid_cloud_gateway_proxy_port>]
                          [-credential_owner=<named_credential_owner>]
                          [-instance_directory=<agent_instance_directory>] 
                          [-port=<agent_port>] 
                          [-session_name=<add_host_session_name>] 
                          [-deployment_type=<type_of_agent_deployment>] 
                          [-privilege_delegation_setting=<privilege_delegation>] 
                          [-additional_parameters=<additional_params_for_install>]
                          [-source_agent=<source_agent_for_cloned_agent_install>]
                          [-master_agent=<master_agent_for_shared_agent_install>] 
                          [-properties_file=<properties_file_having_inputs>] 
                          [-preinstallation_script=<pre_install_script>] 
                          [-preinstallation_script_on_oms] 
                          [-preinstallation_script_run_as_root] 
                          [-postinstallation_script=<post_install_script>] 
                          [-postinstallation_script_on_oms] 
                          [-postinstallation_script_run_as_root] 
                          [-wait_for_completion]
    

    Note that the parameters mentioned in [ ] are optional.

    For example, $<emcli_install_location>/bin/emcli submit_add_host -host_names=oc1.example.com -platform=226 -installation_base_directory=/opt/agent -credential_name=oracle -configure_hybrid_cloud_agent -hybrid_cloud_gateway_agent=abc.example.com -hybrid_cloud_gateway_proxy_port=1748

    This example installs an Agent on the Oracle Cloud virtual host oc1.example.com having the platform ID 226, in the directory /opt/agent, using the named credential oracle. The deployed Agent will use abc.example.com as the Gateway, and use port 1748 to communicate with the Gateway Proxy.

    To view more information on the syntax and the usage of the submit_add_host verb, run the following command:

    $<emcli_install_location>/bin/emcli help submit_add_host
Installing an Agent on an Oracle Cloud VM Using the Add Host Targets Wizard

Follow these steps to install an Agent on an Oracle Cloud VM using the Add Host Targets Wizard:

  1. In Cloud Control, from the Setup menu, select Add Target, then click Add Targets Manually. On the Add Targets Manually page, select Add Host Targets, then click Add Host.

  2. On the Host and Platform page, do the following:

    1. Accept the default name assigned for this session or enter a unique name of your choice. The custom name you enter can be any intuitive name, and need not necessarily be in the same format as the default name. For example, add_host_hybrid_cloud_operation_1

    2. Click Add to enter the fully qualified host name (preferred) or IP address and select the platform of the Oracle Cloud virtual host on which you want to install the Agent. The IP address for the virtual host running each of your Oracle Cloud services would have been provided to you by Oracle.

      Note:

      • Cloud-based Agent deployment is supported for the Linux x86-64 platform only.

      • You must enter only one IP address per row. Entering multiple addresses separated by a comma is not supported.

      Alternatively, you can click Load from File to add the IP addresses that are stored in a file.

      Specify the platform as Linux x86-64 for all the virtual hosts. To do so, you can specify the platform as Linux x86-64 for the first virtual host, then from the Platform list, you can select Same for All Hosts.

    3. Click Next.

  3. On the Installation Details page, do the following:

    1. In the Deployment Type section, select Fresh Agent Install.

    2. From the table, select the first row that indicates the virtual hosts grouped by their common platform name.

    3. In the Installation Details section, provide the installation details common to the virtual hosts selected in Step 3 (b). For Installation Base Directory, enter the absolute path to the base directory on the Oracle Cloud virtual host where you want the software binaries, security files, and inventory files of the Hybrid Cloud Agent to be copied.

      For example, /u01/app/Oracle/.

      If the path you enter does not exist, the application creates a directory at the specified path, and copies the Agent software binaries, security files, and inventory files there.

    4. For Instance Directory, accept the default instance directory location or enter the absolute path to a directory of your choice where all Agent-related configuration files can be stored.

      For example, /u01/app/Oracle/agent_inst.

      If you are entering a custom location, then ensure that the directory has write permissions. Oracle recommends that you maintain the instance directory inside the installation base directory.

      If the path you enter does not exist, the application creates a directory at the specified path, and stores all the Agent-related configuration files there.

    5. For Named Credential, select the named credential that you want to use to set up SSH connectivity between the on-premises OMS and the destination Oracle Cloud virtual hosts, and to install a Agent on each of the Oracle Cloud virtual hosts. Beginning with Enterprise Manager 13c Release 2, you can create SSH key named credentials directly from the wizard so there's no need to pre-create the credentials.

      Ensure that you only specify a named credential that uses SSH public key authentication. Password based authentication is not supported. Also, note that deploying Cloud-based Agents using a locked user account (by switching to the locked user account using a privilege delegation provider) is not supported.

      For information on how to create a named credential that uses SSH public key authentication, see Prerequisites for Installing Agents on Oracle Cloud VMs.

    6. For Privileged Delegation Setting, use the default value. Privilege delegation providers and locked accounts are not supported for Agent deployment.

      If the Agent install user has root privileges, then root.sh is run automatically on the destination virtual hosts post deployment. Else, you must manually run root.sh on every destination virtual host post deployment.

    7. For Port, accept the default port (3872) that is assigned for the Agent to communicate, or enter a port of your choice.

      The custom port you enter must not be busy. If you are not sure, you can leave this field blank. Cloud Control automatically assigns the first available free port within the range of 1830 - 1849.

    8. If you want to run certain scripts before or after deploying the Agents, in the Optional Details section, enter the absolute path to the locations where the scripts that you want to run are available. Note that only shell scripts are supported, and only one pre-installation or one post-installation script can be specified.

      If you want to run the script as root, then select Run as Root. If the script is on the host where the on-premises OMS is running and is not on the virtual host where you want to install the Agent, then select Script on OMS. In this case, the script will be copied from the on-premises OMS host to the destination virtual hosts, and then run on the destination virtual hosts.

    9. If you want to specify certain additional parameters for the deployment, in the Optional Details section, for Additional Parameters, enter a white space-separated list of the additional parameters.

      For example, provide the following path:

      INVENTORY_LOCATION=/u01/app/oracle/oraInventory

      However, note that this parameter is supported only on UNIX platforms, and not on Microsoft Windows platforms.

    10. Select Configure Hybrid Cloud Agent to specify the details for the Gateway that the Cloud-based Agent must communicate with.

      For Hybrid Cloud Gateway, specify the Management Agent within your enterprise that you want to use as a Gateway for the Cloud-based Agent to communicate with. Click the magnifying glass icon, and select a Hybrid Cloud Gateway from the displayed list (only those Gateways that are up and running are displayed).

      Note that for this field, you can only select a Management Agent that has already been designated as a Gateway. For information on how to designate a particular Management Agent as a Gateway, see Configuring a Management Agent as a Gateway.

      For Hybrid Cloud Gateway Proxy Port, specify the port for communication between the Cloud-based Agent and the Gateway Proxy. If you do not specify a value, port 1748 is used, and if port 1748 is not free, then a free port between 1830 and 1848 is used.

    11. Click Next.

  4. On the Review page, review the details you have provided for the installation and if you are satisfied with the details, then click Deploy Agent to install the Agent.

    If you want to modify the details, then click Back repeatedly to reach the page where you want to make the changes.

    When you click Deploy Agent and submit the deployment session, you are automatically taken to the Agent Deployment Details page that enables you to monitor the progress of the deployment session. To understand the tasks you can perform on this page, click Help.

  5. To verify that the Agent was deployed on Oracle Cloud, from the Setup menu, select Manage Cloud Control, then select Agents. Search for, then click the name of the Cloud-based Agent to access its home page. Beside the Agent target name, Running in Oracle Cloud, and a cloud icon must be displayed.

Note:

The following features are not supported, or are partially supported for Cloud-based Agents:

  • Buddy Agent

  • Management Agent to Management Agent communication

  • Distributed Software Library

  • Target Relocation

  • Support for third party Management Agent certificates

  • Support Workbench

Advanced Topics

Discovering and Monitoring Oracle Cloud Targets

Once the Hybrid Cloud is deployed in the on-premises environment and the Agent is deployed in the Oracle Cloud environment, the Oracle Cloud virtual hosts become manageable targets in Enterprise Manager Cloud Control. To discover and monitor the targets running on these manageable virtual hosts, you should follow the instructions outlined in Oracle Enterprise Manager Cloud Control Administrator's Guide. The procedure to discover and promote the targets running on an Oracle Cloud virtual host is the same as the procedure to discover and promote targets running on any normal host in the on-premises environment.

However, for discovering Fusion Middleware domains running on Oracle Cloud virtual hosts, such as WebLogic JCS domains, you should use the public IP address and port 9001 (representing the custom t3 channel that is configured by default on these Admin Servers).

To find out more about cloning in Hybrid Cloud, see the chapter on cloning solutions in the Enterprise Manager Lifecycle Management Administrator’s Guide.

Patching Cloud-based Agents and Gateways

You can patch Agents installed on Oracle Cloud VMs and Gateways using patch plans. Patch plans are consolidated plans that include one or more patches to be rolled out as a group. The patching procedure remains the same for normal Management Agents, Agents installed on Oracle Cloud VMs, and Gateways.

Caution:

The database instance created on Oracle Cloud before the first week of June 2015 is typically based on the database patchset update released in January 2015 (Jan DB PSU). If you want to patch such a database instance with the database patchset update released in April 2015 (Apr DB PSU), then as a prerequisite, before you apply the patchset update, create the following file and add the absolute path to the directory where the Cloud-based Agent is available.

/var/opt/oracle/patch/files_to_save.ora

If you do not follow the aforementioned instruction, you will notice that the Cloud-based Agent in /u01/app/oracle is automatically moved to /u01/app.ORG/oracle as part of the database patching process. You will then have to manually copy the directory back to its original location. To circumvent this issue and avoid any manual effort from your end, Oracle recommends that you follow the aforementioned instruction to create a file as described and add the Cloud-based Agent location to it.

To patch Agents on the Oracle Cloud virtual hosts, follow these steps:

  1. If the patch you are applying accesses the sbin directory of the agent home, then first follow the instructions outlined in the ReadMe file of the patch.
  2. For scalability and performance, use Gold Image based patching to patch Hybrid Agents. For more information on upgrading agents using Gold Image, see the Oracle Enterprise Manager Cloud Control Upgrade Guide.
  3. Patch the Agents by following the instructions outlined in Patch Management from the Oracle Enterprise Manager Cloud Control Lifecycle Management Guide. The patching procedure remains the same for normal Management Agents and Hybrid Cloud Agents.

Configuring an External Proxy to Enable Gateways to Communicate with the Oracle Cloud

For security, you can optionally configure external proxies between the Cloud-based Agents and the Gateway. However, only proxies that support tunneling (for example, SOCK4, SOCK5, HTTP) are supported.

To configure an external proxy between a Cloud-based Agent and a Gateway, follow these steps:

  1. Set up a proxy server. HTTP, SOCKS4, and SOCKS5 proxy servers are supported. Ensure that the proxy server supports tunneling.
  2. From the Setup menu, select Manage Cloud Control, then select Agents.
  3. Search for and click the name of the Gateway for which you want to configure an external proxy. You should select an Agent from the list for which the 'register' command has been executed.
  4. From the Agent menu, select Properties.
  5. From the Show menu, select Basic Properties. For externalProxyPort, specify the communication port that must be used to connect to Oracle Cloud.

    Click Apply.

  6. From the Show menu, select Advanced Properties. Expand the Runtime Settings section. For externalProxyHost, specify the host name of the proxy. For externalProxyType, select whether the proxy uses HTTP, SOCKS4, or SOCKS5 for communication.

    If the proxy server that you set up requires user name and password authentication, specify values for externalProxyUsername and externalProxyPassword.

  7. Click Apply.
  8. Verify the external proxy without authentication. To do so, run the following command:

    ssh -l <user> -i <path_to_private_key> -o "ProxyCommand /usr/bin/nc -X connect -x <proxy host>:<proxy port> %h %p" <oracle_cloud_host> "<test command>"

Performing Additional Hybrid Cloud Management Tasks

This section describes the additional Hybrid Cloud Management tasks that you can perform. It consists of the following:

Configuring Cloud-based Agents for High Availability

When you deploy an Agent on an Oracle Cloud VM, you associate it with a single Gateway by default. Throughout the lifecycle of the Cloud-based Agent, the Agent is dependent on the Gateway to forward the collected monitoring data to the on-premises Cloud Control OMS. Hence, if the Gateway is down or is not reachable, the Cloud-basedAgent monitoring data will not reach the on-premises OMS. Thus, Oracle recommends that you enable every Cloud-based Agent to use multiple Gateways to decrease the probability of a loss in monitoring data.

While deploying an Agent to the Oracle Cloud, the first Gateway that you select is designated as the primary Hybrid Cloud Gateway. If you enable the deployed Agent to use additional Gateways, then the additional Gateways are designated as secondary Hybrid Cloud Gateways. This way, if the primary Hybrid Cloud Gateway for a Cloud-based Agent is down or is unreachable, then one of the secondary Hybrid Cloud Gateways takes over. If the secondary Hybrid Cloud Gateway that took over also goes down or becomes unreachable at some point of time, then the next available secondary Hybrid Cloud Gateway takes over.

Figure 11-1 depicts the communication from the Hybrid Cloud Agents to the on-premises OMS through multiple Hybrid Cloud Gateways.

Figure 11-1 Communication from the Hybrid Cloud Agents to the On-Premise OMS Using Multiple Hybrid Cloud Gateways for High Availability


Communication from the Cloud-based Agents to the On-Premise OMS Using Multiple Hybrid Cloud Gateways for High Availability

To configure a Cloud-based Agent for high availability, you must associate one or more secondary Hybrid Cloud Gateways with the Cloud-based Agents. To do so, follow these steps:

  1. Log in to EM CLI from the /bin directory present within the EM CLI install location:
    $<emcli_install_location>/bin/emcli login -username=<user_name>
    

    Once you run this command, EM CLI will prompt you for a password. Enter the password for the user name you specified.

    EM CLI is set up by default on the on-premises OMS host (the EM CLI install location is the OMS home). Hence, if you choose to run EM CLI from the on-premises OMS host, no additional steps are required. This is the recommended option.

    If you choose to run EM CLI from a custom location on a host that is not running the on-premises OMS, you must first set up EM CLI on the required host. For information on how to do so, see Oracle Enterprise Manager Command Line Interface Guide.

  2. Associate secondary Hybrid Cloud Gateway(s) with a Hybrid Cloud Agent.
    $<emcli_install_location>/bin/emcli add_hybridgateway_for_hybrid_agent 
    -hybrid_agent_name="<hybrid_cloud_agent>:<port>" -hybridgateway_agent_list="<secondary1_hybrid_cloud_gateway_agent>:<port> <secondary2_hybrid_cloud_gateway_agent>:<port> <secondaryN_hybrid_cloud_gateway_agent>:<port>"
    

    For example, emcli add_hybridgateway_for_hybrid_agent -hybrid_agent_name="abc.example.com:1831" —hybridgateway_agent_list="secondary1.example.com:1831 secondary2.example.com:1831"

    Note:

    In the —hybridgateway_agent_list, you can specify more than one Gateway. Ensure that you specify the fully qualified name for each Gateway, and separate the Gateway names using a space.
Disabling Gateways

To disable the gateway functionality of a Gateway, that is, to ensure that a Gateway functions like a regular Management Agent again and does not forward communication from the Cloud-based Agents to the on-premises OMS, follow these steps:

  1. Log in to EM CLI from the /bin directory present within the EM CLI install location:
    $<emcli_install_location>/bin/emcli login -username=<user_name>
    

    Once you run this command, EM CLI will prompt you for a password. Enter the password for the user name you specified.

    EM CLI is set up by default on the on-premises OMS host (the EM CLI install location is the OMS home). Hence, if you choose to run EM CLI from the on-premises OMS host, no additional steps are required. This is the recommended option.

    If you choose to run EM CLI from a custom location on a host that is not running the on-premises OMS, you must first set up EM CLI on the required host. For information on how to do so, see Oracle Enterprise Manager Command Line Interface Guide.

  2. Disable the Gateway functionality of a set of Gateways.
    $<emcli_install_location>/bin/emcli deregister_hybridgateway_agent -hybridgateway_agent_list="<hybrid_cloud_gateway_agent1>:<port> <hybrid_cloud_gateway_agent2>:<port> <hybrid_cloud_gateway_agentN>:<port>"
    

    For example, emcli deregister_hybridgateway_agent -hybridgateway_agent_list="abc.example.com:3873 def.example.com:3873"

    Note that for -hybridgateway_agent_list, you can specify more than one Hybrid Cloud Gateway. Ensure that you specify the fully qualified name for each Gateway, and separate the Gateway names using a space.

Disassociating Gateways from a Cloud-based Agent

To disassociate Gateways from a Cloud-based Agent, such that the specified Agent does not communicate with the Gateway and the on-premises OMS anymore, follow these steps:

  1. Log in to EM CLI from the /bin directory present within the EM CLI install location:
    $<emcli_install_location>/bin/emcli login -username=<user_name>
    

    Once you run this command, EM CLI will prompt you for a password. Enter the password for the user name you specified.

    EM CLI is set up by default on the on-premises OMS host (the EM CLI install location is the OMS home). Hence, if you choose to run EM CLI from the on-premises OMS host, no additional steps are required. This is the recommended option.

    If you choose to run EM CLI from a custom location on a host that is not running the on-premises OMS, you must first set up EM CLI on the required host. For information on how to do so, see Oracle Enterprise Manager Command Line Interface Guide.

  2. Disassociate Gateways from a Cloud-based Agent.
    $<emcli_install_location>/bin/emcli delete_hybridgateway_for_hybrid_agent 
    -hybrid_agent_name="<hybrid_cloud_agent>:<port>" -hybridgateway_agent_list="<hybrid_cloud_gateway1_agent_to_disassociate>:<port> <hybrid_cloud_gateway2_agent_to_disassociate>:<port> <hybrid_cloud_gatewayN_agent_to_disassociate>:<port>"
    

    For example, emcli delete_hybridgateway_for_hybrid_agent -hybrid_agent_name="abc.example.com:1831" -hybridgateway_agent_list="gateway1.example.com:1831 gateway2.example.com:1831"

Decommissioning Cloud-based Agents

To decommission an Agent installed on an Oracle Cloud VM, follow these steps:

  1. Stop the Agent running on the Oracle Cloud VM.
  2. On the Agent Home page of the Agent, from the Agent menu, select Target Setup, then select Agent Decommission.

Troubleshooting Cloud-based Management Agents

This section provides tips to issues that you might encounter when installing or working with Management Agents installed on Oracle VMs.

Table 11-1describes the error messages that you might encounter, along with its causes and suggestions.

Table 11-1 Troubleshooting Cloud-based Management Agents

Warnings/Error Messages Cause or Possible Causes Solution

The host names specified include IP addresses or short names. It is advised to provide Fully Qualified Host Names, such as myhost.myco.com, that are persistent over the life of the targets. It is recommended for ease of maintenance and overall security. However, you can choose to ignore this warning and proceed by clicking Next.

IP address is used in place of fully qualified name.

Click Continue all hosts.

The requiretty flag is set in the sudoers file on the remote host, and as a result the user will not be able to run sudo over ssh.

Agent push failure.

Either set the oracle.sysman.prov.agentpush.enablePty property to true in the /scratch/aime/mw_41005/oms/sysman/prov/agentpush/agentpush.properties file, which is present on the OMS host, or disable the requiretty flag in the sudoers file. You can also ignore this warning and continue in which case the root.sh, any preinstallation or postinstallation scripts specified with run as root enabled will not be run and you have to run them manually after installation.

The other option is click Continue all hosts.

Execution of command/scratch/passagt6/ADATMP_2015-04-06_04-10-01-AM/prereq_stage/core/12.1.0.4.0/oui/bin/runInstaller -prereqchecker -silent -ignoreSysPrereqs -waitForCompletion -prereqlogloc /scratch/passagt6/ADATMP_2015-04-06_04-10-01-AM/prereqlogs -entryPoint oracle.sysman.top.agent_Complete PREREQ_CONFIG_LOCATION=/scratch/passagt6/ADATMP_2015-04-06_04-10-01-AM/prereq_stage/core/12.1.0.4.0/prereqs -J-DFORWARDER_PROXY_PORT=-1 -J-DAGENT_PORT=-1 -J-DALLOW_IPADDRESS=true -J-DAGENT_BASE_DIR=/scratch/passagt6 -J-DSTAGE_LOCATION=/scratch/passagt6/ADATMP_2015-04-06_04-10-01-AM/prereq_stage on host 129.152.134.156 Failed.

Agent push failure.

  1. Check the .bashrc or.cshrc file in the installation user home directory.

  2. Comment on the following two lines

    • export TMP=$TMPDIR

    • export TEMP=$TMPDIR

    OR

    Provide Read/Write/Execute permission to the temd directory.

Execution of command /u01/app/oracle/agent/ADATMP_2016-04-25_05-56-23-AM/agentDeploy.sh AGENT_BASE_DIR=/u01/app/oracle/agent -softwareOnly AGENT_MODE=PAAS on host 129.191.1.207 Failed

Agent push failure.

Include ignorePrereqs to additional parameters during the agent deployment.

When VM is created on Oracle Cloud and user is deploying agent to Oracle Cloud VM. See this error - Port not free [ see this error for range of ports which are actually free ].

Agent push failure.

Check security rules. Enable compute instance security rule to accept connections on the desired port.

Check port connectivity using nc utility to confirm if host:port is accessible from the OMS host.

Frequently Asked Questions About Hybrid Cloud Management

This section provides answers to the following frequently asked questions about Hybrid Cloud Management.

Can I deploy more than one Agent on the same Oracle Cloud virtual host?

Yes, you can. However, make sure you first decommission the Cloud-based Agent that is already present on the Oracle Cloud virtual host, and then deploy another one.

To decommission the Agent that is already present on the Oracle Cloud virtual host, follow these steps:

  1. On the Agent Home page of the Hybrid Cloud Agent, from the Agent menu, select Target Setup, then select Agent Decommission.

  2. Deploy a new Agent as described in Installing an Agent on an Oracle Cloud VM.

Can I deinstall or deconfigure a Gateway without deinstalling an associated Cloud-based Agent?

No, you can't. You must first decommission the Agent that is present on the Oracle Cloud virtual host. When you decommission the Agent, the Gateway with which it is associated is automatically removed.

If you have a single Gateway, and if you want to deinstall it, then follow these steps:

  1. Stop the Agent running on the Oracle Cloud VM.

  2. On the Agent Home page of this Cloud-based Agent, from the Agent menu, select Target Setup, then select Agent Decommission.

If you have multiple Gateways, and if you want to deinstall the primary Hybrid Cloud Gateway, then follow these steps:

  1. Shut down the primary Hybrid Cloud Gateway. This will automatically redirect the communication from the Cloud-based Agent to the secondary Hybrid Cloud Gateway.

  2. Deinstall the primary Hybrid Cloud Gateway.

    Note:

    No need to decommission the Agent that is associated with the primary Hybrid Cloud Gateway. You only have to shut down the primary Hybrid Cloud Gateway as described in Step (1).

    After Step (2), the secondary Hybrid Cloud Gateway will act as the primary Hybrid Cloud Gateway.

    When you bring back the Hybrid Cloud Gateway that you deinstalled in Step (2), it will come back only as a secondary Hybrid Cloud Gateway.

How do I relocate the Gateway to another host without deinstalling anything else?

You can't relocate the Gateway from one host to another host because the relocate logic is only for targets monitored by the Gateway and not for the Gateway.

How can I redistribute my connections once I have added the Gateways? Does it need reconfiguration?

Yes, you can redistribute the connections once you have added additional Gateways. However, there is no automated way to do this. You must manually redistribute the connections.

For example, if you have one Gateway and multiple Cloud-based Agents associated with it, and if you now deploy another Gateway, then you can redistribute the connections between the two gateways.

To do so, follow these steps:

  1. Remove the primary Gateway from serving the Cloud-based Agent. To do so, run the following command. This command causes the OMS to switch the primary gateway to the secondary gateway.

    emcli delete_hybridgateway_for_hybrid_agent -hybrid_agent_name="<hybrid_agent_name>:<port>" -hybridgateway_agent_list="<primary_gateway_agent>:<port>"

  2. Add back the old primary gateway to the Cloud-based Agent. To do so, run the following command. This command restores the old primary gateway as a secondary gateway to the Cloud-based Agent.

    emcli add_hybridgateway_for_hybrid_agent -hybrid_agent_name="<hybrid_agent_name>:<port>" -hybridgateway_agent_list="<old_primary_gateway_agent>:<port>"

After an Oracle PaaS instance is decommissioned, what happens to the Cloud-based Agent and the related targets?

After an Oracle PaaS instance is decommissioned from Oracle Cloud, the associated Agent will be in a unreachable state. To clean up the Agent from the Enterprise Manager Cloud Control Console, follow these steps:

  1. In the Enterprise Manager Cloud Control Console, from the Setup menu, select Manage Cloud Control, then select Agents.

  2. Click the name of the Cloud-based Agent you want to clean up from the console.

  3. On the Agent Home page, from the Agent menu, select Target Setup, then click Agent Decommission.

  4. Select the targets you want to remove, and click Submit.

If I change my SSH keys on Oracle Cloud, what should I do in Enterprise Manager?

Update the monitoring credentials with the new SSH keys so that all Cloud-based Agents can automatically honor them for new deployments. Once the new keys are saved, the SSH tunnelling uses the new keys to communicate with the Cloud-based Agents.

To update the monitoring credentials, follow these steps:

  1. In the Enterprise Manager Cloud Control Console, from the Setup menu, select Security, then select Monitoring Credentials.

  2. On the Monitoring Credentials page, in the table click Hybrid Cloud Connection.

  3. On the Hybrid Cloud Connection Monitoring Credentials page, select the target name where you want to update the new SSH keys, and click Set Credentials.

  4. In the Enter monitoring credentials dialog, enter the new SSH private key and the SSH public key, and click Save.

What are the guidelines for sizing the number of Gateways? What is the indication that my gateway Agent is overloaded?

Currently, there are no statistics available. You can continue to use utilities such as EM Diag Kit to assess the load on the Hybrid Cloud Gateway.

Once the first Gateway is up after being patched, will it monitor the Cloud-based Agents?

No. The only time there is a switch of a primary Gateway is when the primary Gateway goes down.

To list the Gateways for a given Cloud-based Agent, run the following query:

SELECT emd_url FROM MGMT_TARGETS

WHERE target_name LIKE '%PAAS_AGENT_NAME%' AND

target_type='oracle_hybridcloud_connection'

What are the user restrictions on Cloud-based Agents and the targets on Oracle Cloud?

No restrictions as such for users. The Cloud-based Agent install user can be different from the Oracle Cloud target install user, but both users must belong to the same primary operating system group. Otherwise, the discovery might fail.

For example, the Cloud-based Agent install user can be oci, and the Oracle Cloud target install user can be oracle.However, both these users must belong to the oinstall operating system group.

In addition, the user must have sudo access. Otherwise, the root.sh script will have to be run as a manual step during agent deployment.

On what operating system can I deploy a Cloud-based Agent and a Gateway?

You can deploy a Gateway on any operating system, but you must deploy a Cloud-based Agent only on an Oracle Linux x86-64 operating system.

List of Unsupported Features

Table 11-2 lists the features that Hybrid Cloud Management does not currently support.

Table 11-2 Features Not Supported by Hybrid Cloud Management

Targets Features Not Supported

Database

Automatic Workload Repository Warehouse

Collection from Oracle Cloud databases.

SQL Performance Analyzer

  • Remote trials to Database Cloud Service instances.

  • Copy of workload artifacts (capture files/STS) to Oracle Cloud using deployment procedures. Workaround is to manually copy.

  • Active Data Guard support for Database Cloud Service instances (needs a database link).

Database Replay

Disabled for database PaaS targets.

Reorganized Objects

Reorganized objects.

Change Management

Data Synchronization.

Database Cloning

Data Guard

Management of standby databases on Oracle Cloud.

Oracle Exadata Cloud

  • Oracle Exadata hardware and hypervisor monitoring, configuration settings.

  • Patching and upgrade.

  • Backup and restore.

  • Provisioning database services in Oracle Cloud.

Enterprise Manager

  • Agent:

    - Manual deployment.

    - Buddy Agents.

    Sudo and Run As Different User

  • Target Relocation.

  • Software Library on Oracle Cloud.

  • Third-party certificates.

  • Support workbench of Oracle Cloud targets.