Automating Instance Initialization Using opc-init

20 Automating Instance Initialization Using opc-init

Topics

About opc-init

When you create an instance in Compute Classic, you get a virtual machine running the operating system specified by the image that you had selected while creating the instance. Before you start using the instance, you may want to customize it based on your business needs. For example, you may want to create users, install additional packages, add SSH keys, run certain scripts, and so on. Instead of doing all of this initial configuration manually every time an instance starts, you can use the opc-init package to set up these steps to be performed automatically when an instance starts.

The opc-init package contains scripts provided by Oracle that allow you to perform specified instance initialization tasks automatically every time an instance is created. You can specify instance initialization tasks in the form of user data when you create an instance. The opc-init scripts query the metadata service on the instance for this user data. The specified user data is then used by the opc-init scripts to perform the required prebootstrapping tasks. In addition, opc-init adds the SSH public keys specified during instance creation to the authorized_keys file of the default opc user.

The opc-init scripts are included by default in Oracle-provided Linux and Windows machine images. If you want to use opc-init with private Oracle Linux images, you can install the opc-init scripts while creating the image. See Using opc-init in a Private Machine Image.

Note:

Solaris machine images don’t include the opc-init scripts. So you can’t use opc-init to automate instance initialization of Solaris instances.

If you specify user data while creating an instance, the opc-init package retrieves this data from the metadata service and uses it to do the following:

On a Linux Instance	On a Windows Instance
Set the specified http and https proxy for the opc-init process	Set the specified administrator password
Run the specified prebootstrap scripts	Add the specified users
Execute the specified operations for yum elements, including installing yum repositories, and installing or upgrading packages	Run the specified prebootstrap scripts
Process chef elements and install, configure, and launch chef	Enable Windows remote management
-	Enable Remote Desktop Protocol (RDP) access

For user data attributes and templates, see User Data Attributes.

Prerequisites for Using opc-init

You can use the opc-init package to automate instance initialization. To effectively use this tool, you must:

Be familiar with chef-solo
Be familiar with orchestrations
Know JSON
Be aware of the Ruby syntax
Have root access to the instance you want to configure
Know the public IP address of the instance
Have the required permissions and licences for installing Chef, Ruby, and the associated Ruby gems

In addition, if you want to write your own scripts, you must be familiar with Python.

Defining Instance Initialization Attributes

You can automate instance initialization by providing scripts that install applications or perform other prebootstrap tasks when you create an instance. These scripts are specified as user data when you create an instance. If you specify user data while creating an instance, the opc-init package retrieves this data and uses it to perform the specified prebootstrap tasks.

You can specify user data in the following ways:

If you create an instance using the web console, use the Custom Attributes field to specify user data. The text you enter in this field must be in JSON format.
If you create an instance using an orchestration or a launch plan, use the attributes parameter of the instance object type to enter user data.
If you use the API to add a custom machine image to Compute Classic, then you can use the attributes parameter of the POST /machineimage/name method to enter user data.

This user data is then added to all instances created using this machine image. See REST API for Oracle Cloud Infrastructure Compute Classic.
If you use the API to create an image list entry, then you can use the attributes parameter of the POST /imagelist/name/entry/ method to enter user data.

This user data is then added to all instances created using this machine image. See REST API for Oracle Cloud Infrastructure Compute Classic.

Note:

If you want identical user data to be available to a set of instances, use a machine image or image list entry to add user data. For example, you might require a particular prebootstrap script to be executed or specific applications to be installed on all instances that use a particular image. By adding the user data in the machine image or the image list entry, you ensure that each time you use that image, your instance is created with the specified user data.
If each instance should have unique user data, use an orchestration or the web console to provide specific user data for each instance. This is useful if, for example, you want to specify a unique user name and password, or inject a unique SSH public key into each instance.

If you specify identical attributes in a machine image, an image list entry, and while creating an instance, then the values specified in the image list entry override the values specified in the machine image, and the values specified while creating the instance override the values specified in the image list entry and in the machine image. Attributes with unique keys are appended. For example, consider that in the machine image attributes, you specify the following key-value pairs:

{”key1”: “value1”}
{”key2”: “value2”}

In the image list entry attributes, you specify the following key-value pairs:

{”key1”: “value1–a”}
{”key3”: “value3”}

And in the attributes entered while creating an instance using the web console or an orchestration, you specify the following key-value pairs:

{”key1”: “value1–b”}
{”key4”: “value4”}

Then, when your instance is created, key1 will contain the value specified while creating the instance, while the other attributes specified in the machine image, image list entry, and while creating the instance will get appended. When you view user data on the instance, you’ll see the following attributes:

{”key1”: “value1–b”}
{”key2”: “value2”}
{”key3”: “value3”}
{”key4”: “value4”}

Although you can use custom attributes to enter any custom data that you require, if you want the opc-init package to use this information, you must use the userdata attribute of the attributes parameter. You can use userdata to specify the following instance initialization instructions:

The http and https proxy.
Prebootstrap scripts. You can either provide the script inline, or point to a URL where the script is available. This URL must be accessible from the instance.
Chef attributes. You can provide instructions for the instance to be configured either using chef solo, or as a chef client.
Yum repositories. If you specify a yum repository, that repository is used to download and install the specified packages.
A list of packages to be installed on the instance.

The information that you enter as userdata is stored on the instance at the location: http://192.0.0.192/latest/user-data. You can view the user data on your instance at this location using curl. For example:

curl http://192.0.0.192/latest/user-data

For information about the specific nested attributes that you can use in the userdata attribute, see User Data Attributes. For more details about retrieving user data, see Retrieving User-Defined Instance Attributes.

User Data Attributes

You can automate instance initialization by providing scripts or other instructions to perform prebootstrap tasks or install applications when you create an instance. These instance initialization instructions are provided as user-defined data using the userdata attribute when you create an instance.

User Data Attributes Used on Oracle Linux Instances

The following attributes of the userdata attribute are used by the opc-init package to perform instance initialization tasks in Oracle Linux instances.

The sequence for executing attributes on an Oracle Linux instance is as follows:

http-proxy
https-proxy
prebootstrap
yum-repos
packages
package_upgrades
chef

http and https Proxy Attributes

The following sample JSON shows the http-proxy and https-proxy attribute of the userdata attribute.

"instances": [
  {
    <Specify other instance attributes. 
    "attributes": {
      "userdata": {
        "http-proxy":  "your-http-proxy"
        "https-proxy": "your-https-proxy"
         }
        <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

Prebootstrap Attributes

The following sample JSON shows the prebootstrap attribute of the userdata attribute, with the script specified in a URL.

"instances": [
  {
    <Specify other instance attributes. 
    "attributes": {
      "userdata": {
        "pre-bootstrap": {
          "scriptURL": "http://location_of_script",
          "failonerror": true
         }
        <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

The following sample JSON shows the prebootstrap attribute of the userdata attribute, with the script specified inline.

"instances": [
  {
    <Specify other instance attributes. 
    "attributes": {
      "userdata": {
        "pre-bootstrap": {
          "failonerror": true,
          "script": [
            "line1_ofscript",
            "line2_ofscript",
            ...,
            "lineN_ofscript"
          ]
        }
        <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

The following example shows a script to install the Chef client using the prebootstrap attribute and the chef attribute. For information about Chef attributes, see Chef Solo Attributes and Chef Client Attributes.

{
	"attributes": {
		"userdata": {
			"pre-bootstrap": {
				"script": [
					"curl -o chef-client.rpm https://download_location/chef-12.3.0-1.el6.x86_64.rpm",
					"rpm -Uvh chef-client.rpm"
				]
			},
			"chef": {
				"mode": "solo",
				"version": "12.3.0",
				"run_list": ["recipe[learn_chef_httpd]"],
				"cookbooks_url": ["https://download_location/cookbooks.zip"],
				"initial_attributes": {
					"wlss": {
						"key1": "a b c",
						"key2": "/u01/app/oracle",
						"key3": ""
					}
				}
			}
		}
	}
}

A description of the prebootstrap attributes is provided in the following table. Nested attributes are indented in the Attributes column to indicate their hierarchy.

Attribute	Required or Optional	Description
`pre-bootstrap`	Optional	This attribute allows you to specify a script that must be run prior to any other instance initialization that is performed by the opc-init package. You can either enter the script here, or point to a URL. This attribute contains the following nested attributes:
script	Optional	Enter the lines of the prebootstrap script, formatted as a JSON array with each line of the script represented as one element of the array. The metadata service presents this array to the instance as text, with each array element separated by a line break. Don’t enter a script if you provide a script URL. You can specify either `script` or `scriptURL`.
scriptURL	Optional	Enter the script location. This location must be accessible to the instance. Don’t provide a script URL if you provide a script. You can specify either `script` or `scriptURL`.
failonerror	Optional	Specifies whether the prebootstrap process should stop if the script encounters an error. To stop bootstrapping, specify `failonerror` as `true`. The default is `false`. If set to `false`, the bootstrapping operations continue and any errors encountered by the script are logged in the `/var/log/opc-compute/opc-init.log` file.

Yum Repository Attributes

The following sample JSON shows the yum_repos attribute of the userdata attribute.

"instances": [
  {
    <Specify other instance attributes. 
    "attributes": {
      "userdata": {
      "yum_repos": {
            "repo1": {
                "baseurl": "http://location_of_yum_repo",
                "enabled": "0",
                "failovermethod": "priority",
                "gpgcheck": "true",
                "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL",
                "name": "Extra Packages for Enterprise Linux 5 - Testing",
                "proxy": "http://proxy_server:80"
                       }
                    }
       <Specify packages and package_upgrades here.>
       <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

A description of the yum_repos attributes is provided in the following table.

Attribute	Required or Optional	Description
`yum_repos`	Optional	This attribute allows you to specify the desired .repo file and the name of the yum repository. This attribute contains the following nested attributes:
Name of the yum repository	Required	This attribute has no name. You must specify the name of the yum repository to be added. This is used as the repository filename in the format filename`.repo`.
baseurl	Required	The URL of the yum repository.
Other optional attributes	Optional	You can add all other repository file configuration options as nested attributes under the repository name.

Packages and Package Upgrade Attributes

The following sample JSON shows the packages and package_upgrades attributes of the userdata attribute. These attributes are used to specify packages for yum install and yum upgrade.

"instances": [
  {
    <Specify other instance attributes. 
    "attributes": {
      "userdata": {
       <Specify yum attributes here.>
       "packages": ["git-core",["sysstat", "v1"]],
       "package_upgrades": true
       <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

A description of the packages and package_upgrades attributes is provided in the following table.

Attribute	Required or Optional	Description
`packages`	Optional	A JSON array of the packages you want yum to install from the repositories. Each list entry consists of a single package. If you want to specify a package version, then the list entry is represented as a two-element array of the format `[name, version]`.
`package_upgrades`	Optional	A boolean value indicating whether you want to run `yum update` on the instance.

Chef Solo Attributes

To install Chef client on an instance, either the image used for creating the instance must include the Chef client installation package, or you must install the Chef client in either of the following ways:

By using the yum_repos and packages attributes.
By specifying a script using the prebootstrap attribute. See Prebootstrap Attributes.

The following sample JSON shows the chef attribute of the userdata attribute, for a Chef solo configuration.

"instances": [
  {
    <Specify other instance attributes. 
    "attributes": {
      "userdata": {
        "chef": {
             "run_list": ["recipe[apache2]"],
             "cookbooks_url": ["http://location_of_cookbooks/cookbooks.zip"],
             "version": "11.4.2",
             "mode": "solo",
             "initial_attributes": {
                     "apache": {
                        "prefork": {
                            "maxclients": 100,
                            "keepalive": "off"
                                     }
                                }
                             }
                     }
        <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

A description of the chef attributes for a Chef solo configuration is provided in the following table. Nested attributes are indented in the Attributes column to indicate their hierarchy.

Attribute	Required or Optional	Description
`chef`	Required	This attribute allows you to specify data used by Chef for a Chef solo configuration. This attribute contains the following nested attributes:
cookbooks_url	Required	A JSON array of publicly accessible URLs containing the cookbook archives that you want to use, in zip, tar, or tgz format.
run_list	Required	A JSON array of recipes that Chef runs to configure your instance.
initial_attributes	Optional	Data in JSON format (elements, arrays, or values) that is translated into node-level attributes to be consumed by Chef recipes.
version	Required	The Chef version to install. The default is 11.4.2.
mode	Required	The mode must be set to `solo`.
ruby_version	Optional	The Ruby version to install when using gem files. The default is 1.8.
install_type	Optional	The install type. You can specify gems, packages, or omnibus. The default is gems.
force_install	Optional	This attribute allows you to specify whether Chef should be installed even if it has been installed earlier. The default is false.
omnibus_url	Optional	The location to download the omnibus Chef installer if the `install_type` attribute is specified as `omnibus`. The default location is `https://www.opscode.com/chef/install.sh`.
databag_file	Optional	A comma-separated list of publicly accessible anonymous HTTP URLs pointing to Chef databag files.

Chef Client Attributes

The following sample JSON shows the chef attribute of the userdata attribute, for a Chef client configuration. You can use these attributes to configure your instance using a Chef server, if you’ve set up a Chef server in your account.

"instances": 
[
  {
    <Specify other instance attributes. 
    "attributes": {
       "userdata": {
        "chef": {
           "run_list": ["recipe[oui]"],
           "chef_server_url": "https://server IP",
           "mode": "client",
           "chef_node_name": "testnode1",
           "chef_validator_location": "download (http:// or file:/) location with filename" ,
           "version": "11.4.2",
           "initial_attributes": {
                 "oui": {
                    "response_file_url": "responsefile url/filename.rsp",
                    "installer_url": "installer url/filename.zip",
                    "ignoreSysPrereqs": true
                         }
                     }
                 }
        <Specify other userdata attributes here, if required.>
      }
      <Specify other attributes here, if required.>
    }
  }
]

A description of the chef attributes for a Chef client configuration is provided in the following table. Nested attributes are indented in the Attribute column to indicate their hierarchy.

Attribute	Required or Optional	Description
`chef`	Required	This attribute allows you to specify data used by Chef for a Chef client configuration. This attribute contains the following nested attributes:
run_list	Optional/Required	A JSON array of recipes that Chef runs to configure your instance. If the node is predefined on your Chef server with appropriate roles, this attribute is optional. Otherwise, it is required.
initial_attributes	Optional	Data in JSON format (elements, arrays, or values) that is translated into node-level attributes to be consumed by Chef recipes.
version	Optional	The Chef version to install. The default is 11.4.2.
mode	Required	The mode must be set to `client`.
chef_server_url	Required	The URL where the Chef client can access the Chef server.
chef_node_name	Required	A unique name used by the Chef client to register with the Chef sever.
chef_validator_location	Required	The URL from where `validation.pem` is downloaded to `/etc/chef`.
validation_client_name	Optional	The name used by the validator to communicate with the Chef server. Default is `chef-validator`.
ruby_version	Optional	The Ruby version to install when using gem files. The default is 1.8.
install_type	Optional	The install type. You can specify gems, packages, or omnibus. The default is gems.
force_install	Optional	This attribute allows you to specify whether Chef should be installed even if it has been installed earlier. The default is false.
omnibus_url	Optional	The location to download the omnibus Chef installer if the `install_type` attribute is specified as `omnibus`. The default location is `https://www.opscode.com/chef/install.sh`.
databag_file	Optional	A comma-separated list of publicly accessible anonymous HTTP URLs pointing to Chef databag files.

OPC Agent Attributes

You can use the opc_guest_agent_enabled attribute of the userdata attribute to disable the OPC agent. If you create an instances using an Oracle-provided Oracle Linux image with the release version 16.4.6 or later, the OPC agent is installed and enabled by default. Disabling the OPC agent prevents memory utilization metrics from being collected on your instance.

When you create an instance with this attribute set to false, the OPC agent is disabled whenever the instance is created or re-created. You won’t be able to manually start the agent later on. If you want to reserve the ability to stop or restart the agent at any time, you can disable or enable the OPC agent after creating an instance by logging in to the instance and stopping or starting the agent. See Enabling and Disabling the OPC Agent.

The following sample JSON shows the opc_guest_agent_enabled attribute of the userdata attribute.

"instances": 
[
  {
    <Specify other instance attributes. 
    "attributes": {
          "userdata": {
              "opc_guest_agent_enabled":  "false"
                      }
        <Specify other userdata attributes here, if required.>
                  }
      <Specify other attributes here, if required.>
   }
 ]

User Data Attributes Used on Windows Instances

The following attributes of the userdata attribute are used by the opc-init package to perform instance initialization tasks in Windows instances.

Note:

Chef, yum, and proxy attributes aren’t supported on Windows instances.

The sequence for executing attributes on a Windows instance is as follows:

administrator_password
users
pre_bootstrap
winrm
enable_rdp

Enabling RDP Access and Specifying the Password for the Administrator User

To enable the Administrator user to connect to the instance by using a remote desktop protocol (RDP) connection, you must specify the password for the user and also enable RDP access. You can do this by specifying the administrator_password and enable_rdp attributes in the userdata section of your orchestration, as shown in the following example:

"instances": 
[
   {
    <Specify other instance attributes. 
    "attributes": {
          "userdata": {
             "administrator_password": "somePassword",
             "enable_rdp": true
                      }
      <Specify other attributes here, if required.>
                  }
   }
]

Creating Users

You can also specify a list of users that must be created automatically after the Windows instance is launched, by specifying the required users and their passwords in the users attribute, as shown in the following example:

"instances": 
[
    {
      <Specify other instance attributes. 
    "attributes": {
          "userdata": {
             "administrator_password": "somePassword",
             "enable_rdp": true,
             "users":[
                           {
                    "name": "john",
                    "password": "somePassword"
                           }
                           {
                    "name": "amelia",
                    "password": "somePassword"
                           }
                          ]
                      }
      <Specify other attributes here, if required.>
                  }
  }
]

Specifying a Script Using Prebootstrap Attributes

The following sample JSON shows the prebootstrap attribute of the userdata attribute, with the script specified in a URL.

"instances": 
[
  {
    <Specify other instance attributes. 
    "attributes": {
          "userdata": {
              "pre-bootstrap": 
                           {
                         "scriptURL": "http://location_of_script",
                         "failonerror": true
                           }
        <Specify other userdata attributes here, if required.>
                       }
      <Specify other attributes here, if required.>
                  }
  }
]

The following sample JSON shows the prebootstrap attribute of the userdata attribute, with the script specified inline.

"instances": 
[
  {
    <Specify other instance attributes. 
    "attributes": {
          "userdata": {
            "pre-bootstrap": 
                         {
                      "failonerror": true,
                      "script": [
                           "line1_ofscript",
                           "line2_ofscript",
                              ...,
                           "lineN_ofscript"
                                        ]
                         }
        <Specify other userdata attributes here, if required.>
                      }
      <Specify other attributes here, if required.>
                  }
  }
]

A description of the prebootstrap attributes is provided in the following table. Nested attributes are indented in the Attributes column to indicate their hierarchy.

Attribute	Required or Optional	Description
`pre-bootstrap`	Optional	This attribute allows you to specify a script that must be run prior to any other instance initialization that is performed by the opc-init package. You can either enter the script here, or point to a URL. This attribute contains the following nested attributes:
script	Optional	Enter the lines of the prebootstrap script, formatted as a JSON array with each line of the script represented as one element of the array. The metadata service presents this array to the instance as text, with each array element separated by a line break. Don’t enter a script if you provide a script URL. You can specify either `script` or `scriptURL`.
scriptURL	Optional	Enter the script location. This location must be accessible to the instance. Don’t provide a script URL if you provide a script. You can specify either `script` or `scriptURL`.
failonerror	Optional	Specifies whether the prebootstrap process should stop if the script encounters an error. To stop bootstrapping, specify `failonerror` as `true`. The default is `false`. If set to `false`, the bootstrapping operations continue and any errors encountered by the script are logged in the `/var/log/opc-compute/opc-init.log` file.

Enabling Windows Remote Management (WinRM)

You can allow users from other hosts to access the Windows instance by specifying those trusted hosts, as shown in the following example:

"instances": 
[
  {
    <Specify other instance attributes. 
    "attributes": {
         "userdata": {
             "winrm": {
             "trustedhosts": "app1.prod.example.com,*.dev.example.com,203.0.113.25"
                              }
                      }
      <Specify other attributes here, if required.>
                  }
  }
]

If you want to allow all hosts, specify “winrm”: {} or “winrm”: {”trusted hosts”: “*”}. This is also the default setting when winrm isn’t specified.

Using opc-init in a Private Machine Image

If you want to use opc-init in private Oracle Linux machine images, install opc-init while creating the image. You can install and use opc-init on Oracle Linux 6.7 images.

Here’s an overview of the process for using opc-init in a private machine image:

On the Oracle Linux 6.7 VM that you use to create the image, do the following before you create the final image file:
- Download the opc-init package from http://www.oracle.com/technetwork/topics/cloud/downloads/opc-init-3096035.html.
- Extract the files in the package and use yum to install opc-init.
- Add the opc user. The opc-init script copies the SSH keys provided during instance creation to the /home/opc/authorized_keys file of the opc user, enabling SSH access to instances as the opc user.
- For instance initialization that you want to perform on all instances that use this machine image, add the required scripts and user data attributes in the machine image. See User Data Attributes.
  
  Note:
  
  For instance-specific initialization — that is, initialization tasks that must be performed only on a specific instance, not on all instances created using this machine image — you can provide the user data attributes later, while creating the instance, not in the machine image.
- Add the line /usr/bin/opc-linux-init to the /etc/rc.local file.
For detailed instructions on building an Oracle Linux machine image, see the tutorial Building a Custom Oracle Linux Machine Image with the LAMP Stack.
After you’ve created your machine image file, upload it to Oracle Cloud Infrastructure Object Storage Classic. See Uploading Image Files to Oracle Cloud Infrastructure Object Storage Classic.
Register your machine image in Compute Classic. See Registering a Machine Image in Compute Classic.
Use your machine image to create instances. See Creating an Instance Using a Private Image.
While creating instances, provide instance-specific configuration using user data attributes that are understood by opc-init. See User Data Attributes.
If you added the opc user while creating the machine image, you can log in to your instance as the opc user using SSH. See Accessing an Oracle Linux Instance Using SSH.
See the opc-init log to view error messages, if any. The opc-init log is found at: /var/log/opc-init/opc-init.log