Running Commands on an Instance

You can remotely configure, manage, and troubleshoot compute instances by running scripts within the instance using the run command feature.

For example, the run command feature can help you automate tasks such as configuring secondary virtual network interface cards (VNICs), joining instances to an identity provider, troubleshooting SSH connectivity, or responding to cross-region disaster recovery scenarios.

You can run commands on an instance even when the instance does not have SSH access or open inbound ports.

The run command feature uses the Compute Instance Run Command plugin that is managed by the Oracle Cloud Agent software.

Caution

Do not use the run command feature to provide or retrieve passwords, secrets, or other confidential information in plain text. To securely provide and retrieve confidential information, use an Object Storage location to store the script file and response. Use Oracle Cloud Infrastructure Vault to manage keys and secret credentials.

Supported Images

The run command feature is supported on compute instances that use the following platform images:

  • Oracle Autonomous Linux
  • Oracle Linux
  • CentOS
  • Windows Server

Custom images that are based on a supported platform image also support the run command feature.

Supported Regions

The run command feature is supported in all regions in the Oracle Cloud Infrastructure commercial realm.

Limitations and Considerations

  • On Linux instances, the script runs in a Bash shell by default. To run the script with a different program, use #!/<path_to_program> as the first line of the script.
  • On Windows instances, the script runs in a batch shell by default. To run the script with PowerShell, use #ps1 as the first line of the script.

    See an example PowerShell script

    The following example uses PowerShell to query the instance metadata service and print the instance OCID:

    #ps1
    $instance = Invoke-RestMethod -Headers @{'Authorization' = 'Bearer Oracle'} -Uri http://169.254.169.254/opc/v2/instance/
    Write-Host ('Instance OCID is ' + $($instance.id))
  • The maximum size for a script file that you upload directly to an instance in plain text is 4 KB. To provide a larger file, save the file in an Object Storage location.
  • The output of a script when returned as plain text is limited to the last 1 KB. To save a larger response, save the output to an Object Storage location.
  • When you use an Object Storage location to save the script file or response, the instance must have outbound connectivity such as a Network Access Translation (NAT) gateway, service gateway, or internet gateway. The instance must also allow egress traffic on port 443 for the Oracle Cloud Agent software, Object Storage, and IAM.
  • Two scripts can run at a time by default. To change the default, update the run command configuration file:

    cat /etc/oracle-cloud-agent/plugins/runcommand/config.yml

    Set the following parameters:

    logDir: /var/log/oracle-cloud-agent/plugins/runcommand
    commandExecutionMaxWorkers: <number-of-parallel-scripts>
  • A maximum of five scripts can be in flight at a time. A script is considered to be in flight if it has been received by the Compute Instance Run Command plugin, but not yet deleted from the queue.
  • To perform long-running tasks, you should use the run command feature to schedule a cron job on the instance. Command orchestration is not supported.
  • Each script runs once. If you want a script to run multiple times, use cron to configure a schedule for the script.
  • Scripts that prompt for information are not supported. However, you can use the instance metadata service (IMDS) to programatically retrieve information about the instance that the script runs on.
  • The exit codes that are returned are standard Linux error codes. An exit code of 0 indicates success.
  • If you apply an optional timeout for a script, the default is 1 hour. The maximum is 24 hours.
  • The maximum time that a script can run is 1 day.
  • To monitor the resources that scripts consume, such as CPU utilization, use metrics.
  • Canceling a script is a best-effort attempt. Commands can't be canceled after they have finished running or if they have expired.
  • Script files and responses that are saved in plain text are retained for one month. Script files and responses that are saved in an Object Storage location are retained until you delete them.
  • Do not run a script that causes the Oracle Cloud Agent software or the Compute Instance Run Command plugin to stop.

Running Commands with Administrator Privileges

If a command requires sudo permissions, you must grant sudo permissions to the Compute Instance Run Command plugin to be able to run the command. The plugin runs as the ocarun user.

You can use cloud-init to configure permissions at instance launch, or connect to an instance after it has launched and configure permissions manually. Do the following:

  1. On the instance, create a sudoers configuration file for the Compute Instance Run Command plugin:

    vi ./101-oracle-cloud-agent-run-command
  2. Allow the ocarun user to run all commands as sudo by adding the following line to the configuration file:

    ocarun ALL=(ALL) NOPASSWD:ALL

    You can optionally list specific commands. See the Linux man page for sudoers for more information.

  3. Validate that the syntax in the configuration file is correct:

    visudo -cf ./101-oracle-cloud-agent-run-command

    If the syntax is correct, the follow message is returned:

    ./101-oracle-cloud-agent-run-command: parsed OK
  4. Add the configuration file to /etc/sudoers.d:

    sudo cp ./101-oracle-cloud-agent-run-command /etc/sudoers.d/

Required IAM Policy

To use Oracle Cloud Infrastructure, you must be granted security access in a policy  by an administrator. This access is required whether you're using the Console or the REST API with an SDK, CLI, or other tool. If you get a message that you don’t have permission or are unauthorized, verify with your administrator what type of access you have and which compartment  you should work in.

For administrators: To write policy for the run command feature, do the following:

  1. Create a group that includes the users who you want to allow to issue commands, cancel commands, and view the command output for the instances in a compartment. Then, write the following policy to grant access for the group:

    Allow group RunCommandUsers to manage instance-agent-command-family in compartment ABC
  2. Create a dynamic group that includes the instances that you want to allow commands to run on. For example, a rule inside the dynamic group can state:

    any { instance.id = 'ocid1.instance.oc1.phx.<unique_ID_1>', 'ocid1.instance.oc1.phx.<unique_ID_2>' }
  3. Write the following policy to grant access for the dynamic group:
    Note

    If you create an instance and then add it to a dynamic group, it takes up to 30 minutes for the instance to start to poll for commands. If you create the dynamic group first and then create the instance, the instance starts to poll for commands as soon as the instance is created.
    Allow dynamic-group RunCommandDynamicGroup to use instance-agent-command-execution-family in compartment ABC where request.instance.id=target.instance.id
  4. To allow the dynamic group to access the script file from an Object Storage bucket and save the response to an Object Storage bucket, write the following policies:

    Allow dynamic-group RunCommandDynamicGroup to read objects in compartment ABC where all {target.bucket.name = '<bucket_with_script_file>'}
    Allow dynamic-group RunCommandDynamicGroup to manage objects in compartment ABC where all {target.bucket.name = '<bucket_for_command_output>'}

If you're new to policies, see Getting Started with Policies and Common Policies. For reference material about writing policies for instances, cloud networks, or other Core Services API resources, see Details for the Core Services.

Prerequisites

  • The Compute Instance Run Command plugin must be enabled on the instance, and plugins must be running. For more information about how to enable and run plugins, see Managing Plugins with Oracle Cloud Agent.
  • For platform images that were released before October 2020, the Oracle Cloud Agent software must be updated to a version that supports the Compute Instance Run Command plugin (version 1.5.1 or later).
  • You have prepared the script that you want to run. We recommend that you test the command in a non-production environment before deploying it on instances that run production workflows.
  • To provide the script file from an Object Storage location, upload the file to an Object Storage bucket in the same region as the target instance. Note the bucket and file name, or the Object Storage URL for the file. To use the same command across tenancies, create a pre-authenticated request that points to the file.
  • To save the command output to an Object Storage location, create a bucket to save it in, in the same region as the target instance. Note the bucket name or the Object Storage URL for the bucket. You can optionally save the command output using a pre-authenticated request that points to an Object Storage location.

Using the Console

To create a command to run on an instance
  1. Open the navigation menu. Under Core Infrastructure, go to Compute and click Instances.
  2. Click the instance that you're interested in.
  3. Under Resources, click Run Command.
  4. Click Create Command.
  5. Enter a name for the command. Avoid entering confidential information.
  6. In the Timeout in seconds box, enter the amount of time to give the Compute Instance Run Command plugin to run the command on the instance before timing out. The timer starts when the plugin starts the command. For no timeout, enter 0.
  7. In the Add script section, upload the script that you want the Compute Instance Run Command plugin to run on the instance. Select one of the following options:

    • Paste script: Paste the command in the box.
    • Select a file: Upload the script as a text (.txt) file. Either browse to the file that you want to upload, or drag and drop the file into the box.
    • Import from an Object Storage bucket: Select the bucket that contains the script file. In the Object name box, enter the file name.
    • Import from an Object Storage URL: Enter the Object Storage URL for the script file.
  8. In the Output type section, select the location to save the output of the command:

    • Output as text: The output is saved as plain text. You can review the output on the Instance Details page.
    • Output to an Object Storage bucket: The output is saved to an Object Storage bucket. Select a bucket. In the Object name box, enter a name for the output file. Avoid entering confidential information.
    • Output to an Object Storage URL: The output is saved to an Object Storage URL. Enter the URL.
  9. Click Create Command.
To view the output of a command

If the command output was saved to an Object Storage location, either download the response object from the bucket where it was saved or navigate to the Object Storage pre-authenticated request URL.

If the command output was saved as a plain text file, do the following:

  1. Open the navigation menu. Under Core Infrastructure, go to Compute and click Instances.
  2. Click the instance that you're interested in.
  3. Under Resources, click Run Command.
  4. Find the command in the list, click the Actions icon (three dots), and then click View Command Details.
To cancel a command
  1. Open the navigation menu. Under Core Infrastructure, go to Compute and click Instances.
  2. Click the instance that you're interested in.
  3. Under Resources, click Run Command.
  4. Find the command in the list, click the Actions icon (three dots), and then click Cancel Command. Confirm when prompted.

Troubleshooting the Compute Instance Run Command Plugin

To troubleshoot the Compute Instance Run Command plugin, you can view the logs that the plugin generates. Connect to the instance and then use the following:

tail -f /var/log/oracle-cloud-agent/plugins/runcommand/runcommand.log

For easier visibility into the plugin's operations without having to connect to the instance, you can create custom logs using the Oracle Cloud Infrastructure Logging service.

Log Errors

This section describes how to resolve errors that appear in the log file.

Failure to Poll

If the Compute Instance Run Command plugin is failing to poll for commands, you might see the following error in the log file:

poll command err: circuitbreaker:[pollCommand] is open, last err:Service error:NotAuthorizedOrNotFound. Authorization failed or requested resource not found. http status code: 404.

This error can occur when the dynamic group policy for the run command feature is not enabled or if the instance was recently added to the dynamic group. Instances don't belong to tenancy administrator groups by default, so you need to explicitly set dynamic group permissions for the instance. For instructions explaining how to enable the dynamic group policy, see Required IAM Policy.

When you create an instance and then add it to a dynamic group, it takes up to 30 minutes for the instance to start to poll for commands. If you create the dynamic group first and then create the instance, the instance starts to poll for commands as soon as the instance is created.

To test the dynamic group policy as soon as you add the instance to a dynamic group, restart the service manually using one of the following commands:

Oracle Linux 6.x

sudo initctl restart oracle-cloud-agent

Oracle Linux 7.x and Oracle Linux 8.x

sudo systemctl restart oracle-cloud-agent

Windows Server

net restart ocarun