2 Setting Up Provisioning Environment

Follow the sections in this chapter sequentially to set up and configure the infrastructure required for provisioning.

2.1 Overview of Provisioning Environment

The deployment environment in the data center needs to be setup in a certain manner in order to support the provisioning application. Besides the Oracle Management Server (OMS) which hosts Enterprise Manager and Provisioning Application, the following need to be setup and configured before using the provisioning application.

Boot Server

One of the key requirements of application is the ability of the hardware server to boot up over the network (rather than from its hard disk). A boot server must be set up so that it is able to service the requests from the designated hardware servers in order for them to boot over the network. Boot server must be an Enterprise Manager target and should be able to receive the BOOTP and TFTP (Trivial File Transfer Protocol) requests over the network from the hardware server. Refer to Setting Up Boot Server for setting up a boot server with DHCP/TFTP combination. Also refer to section Configuring Boot Server. It is also recommended that the users read about DHCP, PXE, and Redhat Kickstart technology before going through the boot server setup. Refer to Appendix B for a detailed discussion on PXE.

Stage Server

During provisioning of an image on hardware servers, the required binaries and files are first transferred to a stage server. This is known as Staging phase and is responsible for preparing images to be installed over the network, and exposing installable or executable software elements over the network to the target hardware server being provisioned.

The Provisioning application requires at least one stage server on which all the activities related to staging can be performed. From the networking perspective, you are advised to keep the stage server as close to the target machines as possible. It will help in bringing down the installation time drastically, by reducing the time taken to transfer image data from the stage server to the hardware servers. If you have multiple hardware server groups residing at physically different locations, it would be better to have one stage server for each of these locations. Stage server should again be an Enterprise Manager target. Enterprise Manager agent on the stage server should be the same version as the OMS. Refer to section Setting Up Stage Server for setting up a stage server. Also refer to section Configuring a Stage Server.

Reference Host

A Reference Host (also called a gold machine) is the machine that the Provisioning application uses as a reference to create the Linux operating system component. The Provisioning application picks up the list of RPMs (along with their versions) installed on the reference host, and fetches those RPMs from a RPM repository to create an Linux OS component that represents the operating system installed on the reference host. The reference host must be an Enterprise Manager target. Enterprise Manager agent on the reference host should be the same version as the OMS.

RPM Repository

The Provisioning application picks up the RPMs for the Linux OS from the RPM (Redhat Package Manager) repository. At least one Redhat Yum Repository needs to be setup for use by the Provisioning application. Refer to section Setting Up RPM Repository for setting up a RPM repository. Also refer to section Configuring RPM Repository.

2.2 Creating Super Administrator for Enterprise Manager

Only a user who is a Super Administrator for Enterprise Manager can configure various elements like stage server, boot server etc. for use with the Provisioning Application. Not only that, it is only these users who can actually create Assignments for actually provisioning target machines with any image.

Follow the steps below to create a user who is a super administrator.

Log into the Enterprise Manager and click on the Setup link on top right hand corner as shown in the picture below.
On the Setup page, click on the Administrators click on the left hand side column as shown in Figure 2-1.
On the Administrators page, click Create as shown in Figure 2-1.

Figure 2-1 Administrators Setup Page
On the Create step that comes up fill up the necessary details as shown and select the Super Administrator check box as shown in Figure 2-2. Click Next.

Figure 2-2 Create Administrator: Properties Page
On the Review page that comes up, click Finish to complete the user creation.

2.3 Setting Up Provisioning Environment

Tip:

Recommended:

- 2 GB RAM for boot server, stage server, and RPM repository server.

- Boot server and stage server should be on the same physical machine.

- If boot server and stage serve reside on different machines, then the boot install directory (/tftpboot/linux-install/pxelinux.cfg) should be mounted on the stage server.

If you have the required boot server, stage server, and rpm repository already created, then move on to the section for Configuring Provisioning Environment in Enterprise Manager.

2.3.1 Setting Up Boot Server

Complete the following steps to setup a machine as the boot server:

Note:

It is recommended that you use 2 GB RAM.

Install DHCP and TFTP Servers if not already installed.

The two servers could be running either on the same machine, or on different machines. Oracle recommends running the TFTP server on the same host machine as the DHCP server. In case the two servers are installed and configured on different machines, the machine running the TFTP server will be referred to as the boot server.
Configure TFTP Server
- Get the pxelinux boot loader (pxelinux.0) from syslinux distribution, and copy it to the directory that is configured for your TFTP server (/tftpboot in the given examples).
- Create the pxelinux configuration directory (/tftpboot/linux-install/pxelinux.cfg). It is recommended that the Boot and Stage server should be co-located on the same physical machine. But in case this is not true, then the /tftpboot/linux-install/pxelinux.cfg directory should be exposed to the Stage Server via NFS.
Configure DHCP Server

Edit the dhcpd.conf (/etc/dhcpd.conf) file. A sample dhcpd.conf file for PXE setup is shown below:
```
allow booting;
allow bootp;

option domain-name  <domain_name>;
option domain-name-servers dns_servers;
option routers <default_router>;

subnet <subnet-number> netmask <netmask> {          
       [ parameters ]
       [ declarations ]          
  }
# Group the PXE bootable hosts together

group {

# PXE-specific configuration directives...

 next-server  <TFTP_server_IP_address>;

 filename "pxelinux.0";

host <hostname> {
hardware ethernet <MAC address>;
fixed-address <IP address>;
        }
  }
```
The next-server option in the DHCP configuration file specifies the hostname or IP Address of the machine hosting the TFTP server. Oracle recommends running the TFTP Server on the same host machine as the DHCP Server. Therefore, this address should be the IP Address or hostname for the local machine.

The filename option specifies the boot loader location on the TFTP server. The location of the file is relative to the main TFTP directory.

Any standard DHCP configuration file is supported.The sample file format above shows one entry (line 12-15) for each target host. The DHCP service must be restarted every time you modify the configuration file.
Enable the tftp service. Edit the /etc/xinetd.d/tftp file to change the disable flag as no (default=no).

Restart the following services

service dhcpd restart
service xinetd restart
service portmap restart

Install Oracle Management Agent.

Note:

Refer to the Installing a Management Agent section in the Enterprise Manager Grid Control Basic Installation and Configuration Guide to install a 10.2.0.4 or higher version of Management agent on the boot server.

2.3.2 Setting Up Stage Server

Stage server must meet the following requirements:

Large Storage

The files related to components and directives of an image are first copied to the stage server in preparation for the network installation, and are kept there for future use. The stage server thus acts as a huge cache of files, which requires a large storage.

The stage server can also host the staging storage on Network Attached Storage(NAS). Multiple stage servers can use the same NAS.
High Memory

The stage directives associated with the components and images are directives that are executed during staging phase of a component or Image. They contain commands to unpack and layout the files in order to facilitate the network installation. Depending on the size of the components and images, these commands place high memory requirements on part of the stage server.
Sufficient Bandwidth

Staging process could be very time consuming if the network between the Stage server and software library (on Oracle Management Service or OMS Server) does not have sufficient bandwidth to enable fast transfer of files. Similarly, the link between the stage server and hardware servers should have high bandwidth to make the installation process faster.
NFS or HTTP Support

During the installation, hardware servers mount the stage directory so that all the files required for installation appear as local files. In such a scenario, the stage server functions as the NFS server and the hardware servers as its clients. If the stage server uses NAS for staging storage, the NAS server should have the NFS support.

If the stage server cannot have NFS support, it must be accessible by HTTP.

2.3.3 Configuring a Stage Server

You must follow the instructions listed below, to set up a Linux machine as the stage server:

Create a top-level directory
- Create a top-level directory on the stage server where all the files will be stored. In the following steps, STAGE_TOP_LEVEL_DIRECTORY refers to the absolute path of this top-level directory. For provisioning 32-bit and 64-bit targets, separate STAGE_TOP_LEVEL_DIRECTORY is required.
- Depending on the type of targets, copy the required Agent RPM to the STAGE_TOP_LEVEL directory. Agent RPMs zip file can be downloaded from the following location:
```
http://www.oracle.com/technology/software/products/oem/htdocs/provisioning_agent.html
```
- Unzip the agent RPMs in the STAGE_TOP_LEVEL directory.
- Copy the agent RPMs from emagent directory to the STAGE_TOP_LEVEL directory.
Configure NFS services.

Perform the following steps on the stage server.

Note:

If the stage server uses NAS for staging storage, the following steps need to be performed on the NAS server as well

Ensure the NFS service is running. One can check this by running service nfs status.

Modify the "/etc/exports" file to have the following entry:
```
{Directory path} {host_name_or_ip_prefix}* (ro,sync)
```
For example, /STAGE_TOP_LEVEL_DIRECTORY 10.152.* (ro,sync), if the hardware servers to be provisioned have the IP prefix 10.152.

Or, /STAGE_TOP_LEVEL_DIRECTORY provision-host* (ro,sync), if the hardware servers to be provisioned have names starting with provision-host.
After the modification is made, run the service nfs restart command to make the changes visible to nfs daemons.
Install Enterprise Manager Agent.

Refer to the following section in the Enterprise Manager Grid Control Basic Installation and Configuration Guide to install a 10.2.0.3 or higher version of Management agent on the Stage Server.

Note:
Ensure that the Enterprise Manager agent has "write" access to the staging storage.
Executing the "createRepositoryMirror.pl" script

This is an optional step, which should be performed once the RPM repository has been set up.

It is recommended to execute the script "createRepositoryMirror.pl" present in <AGENT_ORACLE_HOME>/sysman/admin/scripts/provisioning/ directory by using the following command:
```
% perl createRepositoryMirror.pl
```
From Grid Control Console, set the privileged preferred credentials for the stage server. For information about setting preferred credentials in Enterprise Manager, see Setting Up Preferred Credentials in Enterprise Manager.

Oracle recommends that the stage server must have very limited access due to the criticality and sensitivity of the data it hosts. The super administrator can enforce this by creating one account on the stage server, and setting it as the preferred credential, to be used by all the provisioning users in Enterprise Manager. This preferred credential should also be a valid ORACLE_HOME credential (belonging to ORACLE_HOME owner's group).

2.3.4 Setting Up RPM Repository

Note:

It is recommended that you use RAM of 2 GB.

RPM Repository is used as the source of Linux and application packages that need to be installed on the newly provisioned bare metal box. For example, an RPM Repository may be created to contain all the 32-bit Linux rpms and another repository may be created to contain Linux x86-64 bit rpms. Two separate Linux images can then be created each based on one of the repositories.

RHEL RPM repository to be used should have the following Red Hat Install tree structure:

Surrounding text describes rpm_repos_str.gif.

There are multiple ways to create a RPM repository. If Red Hat Enterprise Linux CDs are available, do the following:

Copy all the contents of the first CD to a directory say RPM_REPOS.
Copy all rpms from other CDs to <RPM_REPOS>/Redhat/RPMS.Change directory to the RPMS directory:
```
cd <RPM_REPOS>/Redhat/RPMS
```
Add custom RPMs to the repository. This step is only required for pre-RedHat 5 versions (See Metalink Support Note 579715.1).
1. If there are custom RPMs installed on the reference host that need to be provisioned on the bare metal machine, make sure to copy them to the following repository location:
```
<RPM_REPOS>/Redhat/RPMS
```
2. Install anaconda-runtime RPM on the machine hosting the RPM repository. This might require other dependent packages to be installed.
3. Run the following commands:
```
cd /usr/lib/anaconda-runtime
./genhdlist --productpath=RedHat --withnumbers --hdlist <RPM_REPOS>/RedHat/base/hdlist <RPM_REPOS>
```
Run yum-arch :

This should create a headers directory. Make sure this directory contains a header.info file.

If yum is not installed then download it from the Linux Vendor's website.
Create a symbolic link in /var/www/html to <RPM_REPOS> directory.

The repository should now be available through http if an apache server is running.

Note:
In case the Apache server that comes with Enterprise Manger Grid Control 10g is used, enable the Apache directory index page using the "Options Indexes" directive in the Apache configuration (httpd.conf) file.

Oracle Enterprise Linux (OEL) RPM repository should have the Install tree structure shown below:

Surrounding text describes rpm_oel_str.gif.

One can set up Oracle Enterprise Linux (OEL) Repository by using the OEL installation media as follows:

Download Oracle Enterprise Linux from http://edelivery.oracle.com/linux.
Copy all the contents of the first CD to a directory say RPM_REPOS.
Copy all rpms from other CDs to <RPM_REPOS>/Enterprise/RPMS.Change directory to the RPMS directory:
```
cd <RPM_REPOS>/Enterprise /RPMS
```
Add custom RPMs to the repository. This step is only required for pre-OEL 5 versions (See Metalink Support Note 579715.1).
1. If there are custom RPMs installed on the reference host that need to be provisioned on the bare metal machine, make sure to copy them to the following repository location:
```
<RPM_REPOS>/Enterprise/RPMS
```
2. Install anaconda-runtime RPM on the machine hosting the RPM repository. This might require other dependent packages to be installed.
3. Run the following commands:
```
cd /usr/lib/anaconda-runtime
./genhdlist --productpath=Enterprise --withnumbers --hdlist <RPM_REPOS>/Enterprise/base/hdlist <RPM_REPOS>
```
Run yum-arch :

This should create a headers directory. Make sure this directory contains a header.info file.
Create a symbolic link in /var/www/html to <RPM_REPOS> directory.

The repository should now be available through http if an apache server is running.

Note:

In case the Apache server that comes with Enterprise Manger Grid Control 10g is used, enable the Apache directory index page using the "Options Indexes" directive in the Apache configuration (httpd.conf) file.

2.3.5 Setting Up Software Library

Software Library should be located in a directory accessible by all OMSes. If there is one OMS the directory can be local. For multiple OMS environments, the directory can be on a Network File Server or a Netapp filer that is accessible from all the OMSes. One has to ensure that there is enough space available on the shared storage to store files that hold the binary data for one's components.

Software components that are generated as part of the default or single-server image creation during the bare metal provisioning process are stored in the Software Library. They are accessible under the Components tab in the Provisioning Application user interface.

Ensure that the shared storage is accessible through NFS mount points to all OMS servers in the environment.

2.3.6 Checklist for Boot Server, Stage Server, RPM Repository, and Reference Host

Ensure that the following criteria are met before provisioning:

Table 2-1 Checklist for Boot Server, Stage Server, RPM Repository, and Reference Host

Resource Name	Checklist
Boot Server	DHCP is up and running. TFTP is up and running. Boot Server is co located on the same machine as Stage Server. If not then Network Install Directory (/tftpboot/linux-install/pxelinux.cfg) is exposed to the Staging server for mounting. Boot Server is present in the same subnet where the target machines to be provisioned are present or will be added. Enterprise Manager agent is installed. Agent version same as the OMS version. Boot server machine is visible as a managed target in Enterprise Manager. A brand new PXE box actually detects the boot server and starts to boot it (even if no image is installed yet)
Stage Server	Stage server is as close as possible to the target servers. Large storage, High Memory and Sufficient Memory. If NAS server is used for storage then it should have NFS support. If storage is located on NetApp NFS server as staging directory, the pathname of the staging directory, as mounted on the staging server, is the same as the pathname of the actual storage directory on the NetApp server. Enterprise Manager agent is installed. Agent version should be same as the OMS version. Boot server machine is visible as a managed target in Enterprise Manager. The required agent rpm is staged for installing agents on targets. Preferred Credentials are set. Stage server is reachable from the box to be populated (or the same subnet)
RPM Repository	Install tree structure is as indicated in Configure RPM repository section. RPM repository is available via HTTP. Provide the exact URL and test the RPM repository access over HTTP
Reference Host	Agent version is same as OMS version. Agent is installed on local disk and not on NFS mounted directory. Preferred Credentials are set.
Software Library	Shared storage used for the software library is accessible through NFS mount points to all OMS servers.

2.4 Configuring Provisioning Environment in Enterprise Manager

This section provides information about configuring the provisioning environment in Enterprise Manager.

2.4.1 Setting Up Preferred Credentials in Enterprise Manager

Preferred credentials simplify access to managed targets by storing target login credentials in the Management Repository. With preferred credentials set, users can access an Enterprise Manager target that recognizes those credentials without being prompted to log into the target. Preferred credentials are set on a per user basis, thus ensuring the security of the managed enterprise environment.

Enterprise Manager supports two types of preferred credentials:

Normal Credentials: Are used by Enterprise Manager functions that need operating system access, but do not require administrator privileges.
Privileged Credentials: Are used by functions that need administrator privileges. Credentials for users that have sudo access on the target machine can be used as privileged credentials.

The Provisioning application requires preferred credentials to be setup for machines, which are part of the application.The preferred credentials need to be set for the following machines:

Referenced Installation Host: Privileged credentials are needed to execute the command to get all the available RPMs form this machine. The credentials should also be valid ORACLE_HOME credentials (belonging to ORACLE_HOME owner's group)
Stage Server: You must set the privileged preferred credentials for the stage server. Oracle recommends the stage server to have very limited access due to the criticality and sensitivity of the data it hosts. The super administrator can enforce this by creating one account on the stage server, and setting it as the preferred credential, to be used by all the provisioning users in Enterprise Manager. This preferred credential should also be a valid ORACLE_HOME credential (belonging to ORACLE_HOME owner's group).
Provisioning Targets: In case you are planning to provision existing target machines, ensure the privileged credentials are setup. These credentials are required to clear the boot-sector and reboot the machine.

To manage Enterprise Manager preferred credentials:

Click Preferences at the top of any Enterprise Manager Grid Control page.
Click Preferred Credentials in the vertical navigation bar.
Enterprise Manager displays the Preferred Credentials page. From this page, you can manage the preferred credentials for the supported targets.

Caution:

You must not use root as the preferred credential.

2.4.2 Invoking the Bare Metal Provisioning Application

To invoke the Bare Metal Provisioning Application, do the following:

Log in to Oracle Enterprise Manager.
The Provisioning application can be accessed by going to the Deployments Tab and then to Provisioning sub tab, as shown below.

Figure 2-3 Provisioning Application

The graphical user interface of the provisioning application shows various tabs for Components, Directives, and Images etc. A user can access all or some tabs shown above depending upon the privileges assigned to him. For example, in Figure 2-3, the Administration and Assignments tabs are disabled for the user. Refer to Creating Super Administrator for Enterprise Manager for creating users that can access the Administration tab.

In this section, we will assume that the user has super user privileges and can thus access the administration tab. This tab contains different sections for configuring different elements in the environments as shown in the next two figures.

Figure 2-4 Provisioning Application: Administration Tab

2.4.3 Configuring Stage Server

In this section, it is assumed that the stage server has been created and the necessary setup has been done.

Click Add Server in the Staging Server Configuration section. The following page comes up.

Figure 2-5 Add Staging Server Page

In the above step, enter the following details:

Staging Server Hostname refers to the host name of the stage server. It is recommended that you specify the IP address of the stage server, to avoid DNS resolution problems.

Full Directory Path refers to the top-level directory on the stage server which contains the agent rpm. All the staged files will be stored in this location.

Maximum Size Limit refers to the storage space in Megabytes assigned for the staging of files.

Base URL is used to expose the top-level stage directory to hardware servers being provisioned via NFS.

NOTE: If Stage Server uses NAS then the Base URL should be of the form:

2) Provide the necessary values for all the parameters shown in the picture above and click OK to add the stage server.

2.4.4 Configuring RPM Repository

In this section it is assumed that the RPM repository has been created and the necessary setup has done.

Click Add in the RPM Repository Configuration section. The following page is displayed:

Figure 2-6 Add RPM Repository Page

Repository Name Assign a name to the RPM repository that is going to be added.

Complete URL refers to the URL pointing to the directory where the required RPMs are located. For example, http://sample.oracle.com/yum/EnterpriseLinux/EL4/oracle/i386

Provide the necessary values for all the parameters shown in the picture above and click OK to add the RPM repository.

2.4.5 Configuring Boot Server

In this section it is assumed that the Boot Server has been created and the necessary setup has done.

Click Add in the Boot Server Configuration section. The following page is displayed:

Figure 2-7 Add Boot Server Page

Boot Server Hostname refers to the host on which the boot server is setup.

Network Install Directory refers to the directory on the boot server where the pxelinux.cfg file is located. It is generally /tftpboot/linux-install directory on linux systems.

Provide the necessary values for all the parameters shown in the picture above and click OK to add the Boot Server.

2.4.6 Configuring Software Library

In this section it is assumed that the Software library has already been setup.

Click Add in the Software Library Configuration section. The following page is displayed:

Figure 2-8 Add Software Library Page

Software Library directory Location refers to the shared storage location where the deployable images will be stored. Ensure that this shared storage is accessible to all the OMS servers in the environment.

Provide the necessary values for all the parameters shown in the picture above and click OK to add the Software Library.