Oracle Enterprise Manager Best Practices for Bare Metal Provisioning 10g Release 4 (10.2.0.4.0)

10g Release 4 (10.2.0.4.0)

E13664-01

Contents

Title and Copyright Information

Preface

- Audience

- Documentation Accessibility

- Related Documents

- Conventions

- Structure

1 Introduction to Bare Metal Provisioning

- 1.1 About Provisioning

- 1.1.1 Benefits of Automation

- 1.1.2 Target Host Operating Systems

- 1.1.3 Prerequisites for Using the Bare Metal Provisioning Application

- 1.2 Terms Associated with Provisioning

- 1.3 Overview of the Bare Metal Provisioning Process

- 1.3.1 Setting Up Provisioning Environment

- 1.3.2 Provisioning Linux Using Bare Metal Provisioning Application

- 1.3.2.1 Creating Linux Default Image

- 1.3.2.2 Staging the Image on the Stage Server

- 1.3.2.3 Powering up the Bare Metal Machine on the Subnet to Begin the PXE-based OS Boot and Install Process

2 Setting Up Provisioning Environment

- 2.1 Overview of Provisioning Environment

- 2.2 Creating Super Administrator for Enterprise Manager

- 2.3 Setting Up Provisioning Environment

- 2.3.1 Setting Up Boot Server

- 2.3.2 Setting Up Stage Server

- 2.3.3 Configuring a Stage Server

- 2.3.4 Setting Up RPM Repository

- 2.3.5 Setting Up Software Library

- 2.3.6 Checklist for Boot Server, Stage Server, RPM Repository, and Reference Host

- 2.4 Configuring Provisioning Environment in Enterprise Manager

- 2.4.1 Setting Up Preferred Credentials in Enterprise Manager

- 2.4.2 Invoking the Bare Metal Provisioning Application

- 2.4.3 Configuring Stage Server

- 2.4.4 Configuring RPM Repository

- 2.4.5 Configuring Boot Server

- 2.4.6 Configuring Software Library

3 Provisioning Linux

- 3.1 Accessing the Bare Metal Provisioning Application

- 3.2 Creating a Default Image

- 3.3 Setting Up a Default Image

- 3.4 Staging the Default Image

- 3.5 Ready to Go

A Troubleshooting

B PXE Booting and Kickstart Technology

- B.1 About PXE Booting and Kickstart Technology

C Sample custom_rpm.properties File

- C.1 custom_rpm.properties File

D Frequently Asked Questions

- D.1 Frequently Asked Questions on Setting Up the Provisioning Environment

- D.2 Frequently Asked Questions on Setting Up and Provisioning Default Images

- D.3 Frequently Asked Questions on Directives and Components

- D.4 Frequently Asked Questions on Network Profiles and Images

Oracle® Enterprise Manager

Best Practices for Bare Metal Provisioning

10g Release 4 (10.2.0.4.0)

E13664-01

September 2008

Oracle Enterprise Manager Best Practices for Bare Metal Provisioning 10g Release 4 (10.2.0.4.0)

E13664-01

Copyright © 2008, Oracle. All rights reserved.

The Programs (which include both the software and documentation) contain proprietary information; they are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright, patent, and other intellectual and industrial property laws. Reverse engineering, disassembly, or decompilation of the Programs, except to the extent required to obtain interoperability with other independently created software or as specified by law, is prohibited.

The information contained in this document is subject to change without notice. If you find any problems in the documentation, please report them to us in writing. This document is not warranted to be error-free. Except as may be expressly permitted in your license agreement for these Programs, no part of these Programs may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose.

If the Programs are delivered to the United States Government or anyone licensing or using the Programs on behalf of the United States Government, the following notice is applicable:

U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the Programs, including documentation and technical data, shall be subject to the licensing restrictions set forth in the applicable Oracle license agreement, and, to the extent applicable, the additional rights set forth in FAR 52.227-19, Commercial Computer Software--Restricted Rights (June 1987). Oracle USA, Inc., 500 Oracle Parkway, Redwood City, CA 94065.

The Programs are not intended for use in any nuclear, aviation, mass transit, medical, or other inherently dangerous applications. It shall be the licensee's responsibility to take all appropriate fail-safe, backup, redundancy and other measures to ensure the safe use of such applications if the Programs are used for such purposes, and we disclaim liability for any damages caused by such use of the Programs.

Oracle, JD Edwards, PeopleSoft, and Siebel are registered trademarks of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

The Programs may provide links to Web sites and access to content, products, and services from third parties. Oracle is not responsible for the availability of, or any content provided on, third-party Web sites. You bear all risks associated with the use of such content. If you choose to purchase any products or services from a third party, the relationship is directly between you and the third party. Oracle is not responsible for: (a) the quality of third-party products or services; or (b) fulfilling any of the terms of the agreement with the third party, including delivery of products or services and warranty obligations related to purchased products or services. Oracle is not responsible for any loss or damage of any sort that you may incur from dealing with any third party.

Preface

This guide provides information about provisioning concepts and Linux provisioning using Oracle Enterprise Manager. Best Practices for Bare Metal Provisioning covers provisioning of Linux operating system on bare metal servers. Out-of-box deployment procedures in Enterprise Manager can then be used to provision Oracle Database and other software on the newly provisioned Linux servers. The following use case is covered:

- Provisioning Linux Operating System on Bare Metal Boxes

Audience

This document is mainly intended for the system administrators and DBAs who want to use the Provisioning application within Oracle Enterprise Manager 10g, to automate the manual tasks of provisioning and configuring hardware servers/software in their data center. This document also acts as a good starting point for anyone who wants to get familiar with provisioning concepts.

It is assumed that the reader is familiar with the features and architecture of Oracle Enterprise Manager 10g. Please go through Managing the Complete Oracle Environment with Oracle Enterprise Manager 10g for more information on this.

It is also assumed that the user is familiar with the Preferred Credentials feature of Oracle Enterprise Manager 10g.

NOTE: You can also use the Deployment Procedures feature to provision Oracle/Third Party Software on live servers (servers with operating system already installed on them). More information about the Deployment Procedures can be found here:

http://www.oracle.com/technology/products/oem/pdf/grid-automation-deployment-procedures.pdf

Documentation Accessibility

Our goal is to make Oracle products, services, and supporting documentation accessible, with good usability, to the disabled community. To that end, our documentation includes features that make information available to users of assistive technology. This documentation is available in HTML format, and contains markup to facilitate access by the disabled community. Accessibility standards will continue to evolve over time, and Oracle is actively engaged with other market-leading technology vendors to address technical obstacles so that our documentation can be accessible to all of our customers. For more information, visit the Oracle Accessibility Program Web site at

http://www.oracle.com/accessibility/

Accessibility of Code Examples in Documentation

Screen readers may not always correctly read the code examples in this document. The conventions for writing code require that closing braces should appear on an otherwise empty line; however, some screen readers may not always read a line of text that consists solely of a bracket or brace.

Accessibility of Links to External Web Sites in Documentation

This documentation may contain links to Web sites of other companies or organizations that Oracle does not own or control. Oracle neither evaluates nor makes any representations regarding the accessibility of these Web sites.

TTY Access to Oracle Support Services

Oracle provides dedicated Text Telephone (TTY) access to Oracle Support Services within the United States of America 24 hours a day, seven days a week. For TTY support, call 800.446.2398.

Related Documents

For more information, see the following documents in the Oracle Enterprise Manager 10g Release 2 documentation set:

- Oracle Enterprise Manager Concepts

- Oracle Enterprise Manager Grid Control Quick Installation Guide

- Oracle Enterprise Manager Grid Control Quick Installation Guide

- Oracle Enterprise Manager Grid Control Installation and Basic Configuration

- Oracle Enterprise Manager Configuration for Oracle Collaboration Suite

- Oracle Enterprise Manager Advanced Configuration

- Oracle Enterprise Manager Policy Reference Manual

- Oracle Enterprise Manager Extensibility

- Oracle Enterprise Manager Command Line Interface

- Oracle Enterprise Manager SNMP Support Reference Guide

- Oracle Enterprise Manager Licensing Information

Conventions

The following text conventions are used in this document:

Convention

Meaning

boldface

Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary.

italic

Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values.

monospace

Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter.

Structure

One should go through the following sections irrespective of the specific provisioning use case that one may have. This guide contains the following chapters and is organized as follows:

Introduction to Bare Metal Provisioning

This chapter provides an introduction to provisioning.

Setting Up Provisioning Environment

This chapter provides details on setting up and configuring various elements required by the provisioning application

Provisioning Linux

This chapter provides steps for provisioning Linux operating system on bare metal machines.

1 Introduction to Bare Metal Provisioning

This chapter provides information about the prerequisites required to use the bare metal provisioning application and provides an overview of bare metal provisioning process.

1.1 About Provisioning

Proliferation of low cost servers in our data centers has brought in a fresh set of management challenges. The well-acknowledged problems include the difficulty in managing consistency and compatibility across operating system and software deployments, server drifts and security vulnerabilities that lead to lack of compliance, difficulty in deploying software, difficulty in provisioning new servers with variety of configurations and applications, high cost of operation and difficulty in adapting to changes in workload of the environment. These lead to system administrators and DBAs spending significant amount of their time in software and server provisioning operations.

Oracle's answer to the Software and Server Management challenge is its Bare Metal Provisioning Application, which is a part of Provisioning Pack (available at http://www.oracle.com/technology/products/oem/mgmt_solutions/provisioning.html) of the Enterprise Manager Grid Control and Oracle Management Pack for Linux (available at http://www.oracle.com/technology/products/oem/omp_linux.html). Bare Metal Provisioning Application addresses the data center, server farm challenge to provision software and servers quickly, efficiently, and make them operational.

1.1.1 Benefits of Automation

Bare Metal Provisioning Application provides server lifecycle management capabilities that enable one to build manage and optimize their server infrastructure. The application provides an automated, repeatable and reliable solution that:

- Automates deployment of consistent, certified Linux operating system images along with other software on a larger number of servers.

- Leads to faster, unattended deployment of software and operating systems.

- Allows provisioning of middleware, Clusterware, Real Application cluster (RAC) etc. on top of Linux stack.

- Provides a template-based approach for provisioning variety of Linux configurations with software on servers. This also ensures compliance to standards and consistency across all deployments.

- Supports heterogeneous hardware and network configuration.

- Automatically discovers bare metal and live target servers for provisioning.

- Especially for Oracle software the application encodes best practices out-of-the-box for deployment and patching.

- Results in 10x or more reduction in manual labor that leads to substantial cost savings.

The application uses standardized PXE (Pre Boot Execution environment) booting process for provisioning both bare-metal and live servers. It provides a role based User Interface, for easily creating Gold Images and initiating automated, unattended installs.

1.1.2 Target Host Operating Systems

The Provisioning application can provision 32-bit and 64-bit variants of the following operating systems:

- Oracle Enterprise Linux 5.0

- Oracle Enterprise Linux 4.0 or higher

- RedHat Enterprise Linux (RHEL) 5.0

- RedHat Enterprise Linux (RHEL) 4.0 update 2 or higher

- RedHat Enterprise Linux (RHEL) 3.0 update 6 or higher

- SUSE Linux (SLES) 10

1.1.3 Prerequisites for Using the Bare Metal Provisioning Application

The following steps should be followed for evaluating or using the Bare Metal Provisioning application:

Install or upgrade to the latest Enterprise Manager 10g Grid Control Release 4 (10.2.0.4) and apply the one off patch.

- If you are using Enterprise Manager 10g Grid Control version 10.2.0.1 or 10.2.0.2, then first upgrade it to version 10.2.0.4. Refer to the Enterprise Manager Grid Control Release 4(10.2.0.4) notes available at http://www.oracle.com/technology/documentation/oem.html

- Patch 6129692 to the 10.2.0.4 Oracle Management Service (OMS)

- Patch 7135845 to the 10.2.0.4 Oracle Management Agents

- If you are using Enterprise Manager 10g Grid Control Release 3(10.2.0.3) and want to provision Oracle Enterprise Linux (OEL) using the provisioning application then apply the patches mentioned below. (But it is strongly recommended to upgrade the existing Enterprise Manager Grid Control deployment to the latest version 10.2.0.4 to benefit from the new features added in the latest release.) These patches can be downloaded from http://metalink.oracle.com/. To apply the patches, follow the instructions mentioned in the patch readme files.

- Patch 5878342 and 5858325 to the 10.2.0.3 Oracle Management Service (OMS)

- Patch 5912253 to the 10.2.0.3 Oracle Management Agents on the hosts that would be used as reference hosts for installing Oracle Enterprise Linux. This patch is not required if these management agents have been installed using the Agent RPM kit or the bare metal provisioning application.

Note:

For provisioning Oracle Enterprise Linux, one should have 10.2.0.3 Management Agent deployed on the reference host.

Download the agent rpm kit from the following location:

http://www.oracle.com/technology/software/products/oem/htdocs/provisioning_agent.html

1.2 Terms Associated with Provisioning

Following basic elements are associated with the provisioning application:

Components

Components represent the primary building blocks that may be combined with other components as needed, to specify the complete software configuration or image that is provisioned on target machines. Imagine components as ingredients and image as a final recipe cooked using the various ingredients. A component can represent Operating system software, Oracle software or any third party software and applications. Software components are individually maintained within the Oracle Software Library. Versions, states and maturity levels can be associated with each component.

Directives

Directives can be imagined as instructions to cook the final image (recipe) using components (ingredients). These are constructs used to associate scripts with software components and images. These scripts contain directions on how to interpret and process the contents of a particular component or an image. Directives encapsulate the script, the command line used to invoke the script, and the script configuration properties. They capture everything required for invoking the script on a machine during a provisioning operation. Directives are usually categorized based on the provisioning life cycle phases they are targeted for, or the actions they perform. Imagine Directives as set of executable instructions that run from a supported shell (for example, borne-again, Perl, Python), programming language (for example, Java), or execution framework or interpreter (such as "make" or "ant"). Directives are contained within a file that are stored in the Oracle Software Library and referenced from the software components that employ them.

Images

An image can be viewed as a set of components and may include required directives that form the required software configuration, which is deployed on the target machines. An image contains the complete software stack from operation system to application, in the form of its components. Images reference the components they logically contain by version (rather than include them directly). Images are stored in the Oracle Software Library and versions, states and maturity levels can be associated with them.

Software Library

Maintains metadata and binary content for components, images, and directives. The binary content is maintained in a shared storage location. It allows maintaining versions, maturity levels and states of components, directives and images.

Network Profile

Network profiles are used to define the network properties of the hardware servers to be provisioned. Network profiles can be can be of three types:

- Static: Use this type to specify all of the network properties manually. This type of profile can be applied to a single server only once.

- Dynamic: Use this to use DHCP server to fetch the network properties for the target machine.

- Network Configuration: Network Configurations provide a way to create a pool of IP addresses, which the provisioning application can use repeatedly until all the IP addresses in the pool are exhausted.

Assignments

The activity to select an image and provision them on hardware servers is termed as an assignment. Assignments allow specifying properties (such as image properties, file system, root password, network profiles, port, IP address, etc.) that will be used during the provisioning of hardware servers. Assignments also allow for setting of advanced parameters like NTP, NIS, NFS and kernel parameters.

1.3 Overview of the Bare Metal Provisioning Process

The provisioning process consists of following two high-level tasks:

Setting Up Provisioning Environment (Chapter 2):

- Setting up Boot/DHCP server and Stage server, setting up RPM repository and Software Library (Section 2.3)

- Configuring above entities in Enterprise Manager (Section 2.4)

Provisioning Linux using Bare Metal Provisioning Application (Chapter 3):

- Creating Linux "Default Image" by scheduling a "Create Image" job and associating the image with a particular subnet (Section 3.2)

- Staging the image on the Stage server (Section 3.4)

- Powering up the bare metal machine on the subnet to begin the PXE-based OS boot and install process (Section 3.5)

1.3.1 Setting Up Provisioning Environment

The user needs to perform a one-time activity of setting up a Boot server, Stage server, and RPM repository that are required by the provisioning application. Once this infrastructure is set up, it can be used repeatedly for provisioning Linux on several bare metal machines. See Chapter 2 for more information.

1.3.2 Provisioning Linux Using Bare Metal Provisioning Application

The following sections outline the steps to provision Linux. See Chapter 3 for more information.

Figure 1-1 User flow for Creating Deployable Images for Hardware Servers

1.3.2.1 Creating Linux Default Image

Enterprise Manager allows the user to create a Linux "Default Image" from Linux installation on a reference machine. The default image consists of an OS binary component along with directives required to stage and provision the image. Once an image is created, it can be associated with a subnet prefix or a set of MAC addresses. Doing this ensures that the image gets provisioned on bare metal machines in the specified subnet or having specified MAC address. See Section 3.2 for more information.

1.3.2.2 Staging the Image on the Stage Server

The user is required to stage the contents of the image, namely the binaries associated with the components, directives and other templates, on the stage server. This caches the image in the staging area in the subnet for improved performance and prepares the image to be provisioned. See Section 3.4 for more information.

1.3.2.3 Powering up the Bare Metal Machine on the Subnet to Begin the PXE-based OS Boot and Install Process

Once the image is staged, it is ready to be provisioned on bare metal machines that are powered on in the subnet. When a bare metal machine is connected to the subnet, the Bare Metal Provisioning Application discovers the machine and boots it over the network using the PXE protocol. It then installs the operating system and the Enterprise Manager management agent on the server. Installing the agent makes the server a managed target in the Enterprise Manager console. See Section 3.5 for more information.

Figure 1-2 explains the sequence in the PXE process:

Figure 1-2 PXE Process

See Appendix B for a description of the PXE boot process.

2 Setting Up Provisioning Environment

Follow the sections in this chapter sequentially to set up and configure the infrastructure required for provisioning.

2.1 Overview of Provisioning Environment

The deployment environment in the data center needs to be setup in a certain manner in order to support the provisioning application. Besides the Oracle Management Server (OMS) which hosts Enterprise Manager and Provisioning Application, the following need to be setup and configured before using the provisioning application.

Boot Server

One of the key requirements of application is the ability of the hardware server to boot up over the network (rather than from its hard disk). A boot server must be set up so that it is able to service the requests from the designated hardware servers in order for them to boot over the network. Boot server must be an Enterprise Manager target and should be able to receive the BOOTP and TFTP (Trivial File Transfer Protocol) requests over the network from the hardware server. Refer to Setting Up Boot Server for setting up a boot server with DHCP/TFTP combination. Also refer to section Configuring Boot Server. It is also recommended that the users read about DHCP, PXE, and Redhat Kickstart technology before going through the boot server setup. Refer to Appendix B for a detailed discussion on PXE.

Stage Server

During provisioning of an image on hardware servers, the required binaries and files are first transferred to a stage server. This is known as Staging phase and is responsible for preparing images to be installed over the network, and exposing installable or executable software elements over the network to the target hardware server being provisioned.

The Provisioning application requires at least one stage server on which all the activities related to staging can be performed. From the networking perspective, you are advised to keep the stage server as close to the target machines as possible. It will help in bringing down the installation time drastically, by reducing the time taken to transfer image data from the stage server to the hardware servers. If you have multiple hardware server groups residing at physically different locations, it would be better to have one stage server for each of these locations. Stage server should again be an Enterprise Manager target. Enterprise Manager agent on the stage server should be the same version as the OMS. Refer to section Setting Up Stage Server for setting up a stage server. Also refer to section Configuring a Stage Server.

Reference Host

A Reference Host (also called a gold machine) is the machine that the Provisioning application uses as a reference to create the Linux operating system component. The Provisioning application picks up the list of RPMs (along with their versions) installed on the reference host, and fetches those RPMs from a RPM repository to create an Linux OS component that represents the operating system installed on the reference host. The reference host must be an Enterprise Manager target. Enterprise Manager agent on the reference host should be the same version as the OMS.

RPM Repository

The Provisioning application picks up the RPMs for the Linux OS from the RPM (Redhat Package Manager) repository. At least one Redhat Yum Repository needs to be setup for use by the Provisioning application. Refer to section Setting Up RPM Repository for setting up a RPM repository. Also refer to section Configuring RPM Repository.

2.2 Creating Super Administrator for Enterprise Manager

Only a user who is a Super Administrator for Enterprise Manager can configure various elements like stage server, boot server etc. for use with the Provisioning Application. Not only that, it is only these users who can actually create Assignments for actually provisioning target machines with any image.

Follow the steps below to create a user who is a super administrator.

1. Log into the Enterprise Manager and click on the Setup link on top right hand corner as shown in the picture below.

2. On the Setup page, click on the Administrators click on the left hand side column as shown in Figure 2-1.

3. On the Administrators page, click Create as shown in Figure 2-1.

Figure 2-1 Administrators Setup Page

4. On the Create step that comes up fill up the necessary details as shown and select the Super Administrator check box as shown in Figure 2-2. Click Next.

Figure 2-2 Create Administrator: Properties Page

5. On the Review page that comes up, click Finish to complete the user creation.

2.3 Setting Up Provisioning Environment

Tip:

Recommended:
- 2 GB RAM for boot server, stage server, and RPM repository server.

- Boot server and stage server should be on the same physical machine.

- If boot server and stage serve reside on different machines, then the boot install directory (/tftpboot/linux-install/pxelinux.cfg) should be mounted on the stage server.

If you have the required boot server, stage server, and rpm repository already created, then move on to the section for Configuring Provisioning Environment in Enterprise Manager.

2.3.1 Setting Up Boot Server

Complete the following steps to setup a machine as the boot server:

Note:

It is recommended that you use 2 GB RAM.

1. Install DHCP and TFTP Servers if not already installed.
The two servers could be running either on the same machine, or on different machines. Oracle recommends running the TFTP server on the same host machine as the DHCP server. In case the two servers are installed and configured on different machines, the machine running the TFTP server will be referred to as the boot server.

2. Configure TFTP Server

- Get the pxelinux boot loader (pxelinux.0) from syslinux distribution, and copy it to the directory that is configured for your TFTP server (/tftpboot in the given examples).

- Create the pxelinux configuration directory (/tftpboot/linux-install/pxelinux.cfg). It is recommended that the Boot and Stage server should be co-located on the same physical machine. But in case this is not true, then the /tftpboot/linux-install/pxelinux.cfg directory should be exposed to the Stage Server via NFS.

3. Configure DHCP Server
Edit the dhcpd.conf (/etc/dhcpd.conf) file. A sample dhcpd.conf file for PXE setup is shown below:

allow booting;
allow bootp;

option domain-name  <domain_name>;
option domain-name-servers dns_servers;
option routers <default_router>;

subnet <subnet-number> netmask <netmask> {
[ parameters ]
[ declarations ]
}
# Group the PXE bootable hosts together

group {

# PXE-specific configuration directives...

next-server  <TFTP_server_IP_address>;

filename "pxelinux.0";

host <hostname> {
hardware ethernet <MAC address>;
fixed-address <IP address>;
}
}

The next-server option in the DHCP configuration file specifies the hostname or IP Address of the machine hosting the TFTP server. Oracle recommends running the TFTP Server on the same host machine as the DHCP Server. Therefore, this address should be the IP Address or hostname for the local machine.

The filename option specifies the boot loader location on the TFTP server. The location of the file is relative to the main TFTP directory.

Any standard DHCP configuration file is supported.The sample file format above shows one entry (line 12-15) for each target host. The DHCP service must be restarted every time you modify the configuration file.

4. Enable the tftp service. Edit the /etc/xinetd.d/tftp file to change the disable flag as no (default=no).

5. Restart the following services

service dhcpd restart
service xinetd restart
service portmap restart

6. Install Oracle Management Agent.

Note:

Refer to the Installing a Management Agent section in the Enterprise Manager Grid Control Basic Installation and Configuration Guide to install a 10.2.0.4 or higher version of Management agent on the boot server.

2.3.2 Setting Up Stage Server

Stage server must meet the following requirements:

- Large Storage
The files related to components and directives of an image are first copied to the stage server in preparation for the network installation, and are kept there for future use. The stage server thus acts as a huge cache of files, which requires a large storage.

The stage server can also host the staging storage on Network Attached Storage(NAS). Multiple stage servers can use the same NAS.

- High Memory
The stage directives associated with the components and images are directives that are executed during staging phase of a component or Image. They contain commands to unpack and layout the files in order to facilitate the network installation. Depending on the size of the components and images, these commands place high memory requirements on part of the stage server.

- Sufficient Bandwidth
Staging process could be very time consuming if the network between the Stage server and software library (on Oracle Management Service or OMS Server) does not have sufficient bandwidth to enable fast transfer of files. Similarly, the link between the stage server and hardware servers should have high bandwidth to make the installation process faster.

- NFS or HTTP Support
During the installation, hardware servers mount the stage directory so that all the files required for installation appear as local files. In such a scenario, the stage server functions as the NFS server and the hardware servers as its clients. If the stage server uses NAS for staging storage, the NAS server should have the NFS support.

If the stage server cannot have NFS support, it must be accessible by HTTP.

2.3.3 Configuring a Stage Server

You must follow the instructions listed below, to set up a Linux machine as the stage server:

1. Create a top-level directory

- Create a top-level directory on the stage server where all the files will be stored. In the following steps, STAGE_TOP_LEVEL_DIRECTORY refers to the absolute path of this top-level directory. For provisioning 32-bit and 64-bit targets, separate STAGE_TOP_LEVEL_DIRECTORY is required.

- Depending on the type of targets, copy the required Agent RPM to the STAGE_TOP_LEVEL directory. Agent RPMs zip file can be downloaded from the following location:

http://www.oracle.com/technology/software/products/oem/htdocs/provisioning_agent.html

- Unzip the agent RPMs in the STAGE_TOP_LEVEL directory.

- Copy the agent RPMs from emagent directory to the STAGE_TOP_LEVEL directory.

2. Configure NFS services.

Perform the following steps on the stage server.

Note:

If the stage server uses NAS for staging storage, the following steps need to be performed on the NAS server as well

1. Ensure the NFS service is running. One can check this by running service nfs status.
Modify the "/etc/exports" file to have the following entry:

{Directory path} {host_name_or_ip_prefix}* (ro,sync)

For example, /STAGE_TOP_LEVEL_DIRECTORY 10.152.* (ro,sync), if the hardware servers to be provisioned have the IP prefix 10.152.

Or, /STAGE_TOP_LEVEL_DIRECTORY provision-host* (ro,sync), if the hardware servers to be provisioned have names starting with provision-host.

2. After the modification is made, run the service nfs restart command to make the changes visible to nfs daemons.

3. Install Enterprise Manager Agent.
Refer to the following section in the Enterprise Manager Grid Control Basic Installation and Configuration Guide to install a 10.2.0.3 or higher version of Management agent on the Stage Server.

Note:

Ensure that the Enterprise Manager agent has "write" access to the staging storage.

4. Executing the "createRepositoryMirror.pl" script
This is an optional step, which should be performed once the RPM repository has been set up.

It is recommended to execute the script "createRepositoryMirror.pl" present in <AGENT_ORACLE_HOME>/sysman/admin/scripts/provisioning/ directory by using the following command:

% perl createRepositoryMirror.pl

5. From Grid Control Console, set the privileged preferred credentials for the stage server. For information about setting preferred credentials in Enterprise Manager, see Setting Up Preferred Credentials in Enterprise Manager.
Oracle recommends that the stage server must have very limited access due to the criticality and sensitivity of the data it hosts. The super administrator can enforce this by creating one account on the stage server, and setting it as the preferred credential, to be used by all the provisioning users in Enterprise Manager. This preferred credential should also be a valid ORACLE_HOME credential (belonging to ORACLE_HOME owner's group).

2.3.4 Setting Up RPM Repository

Note:

It is recommended that you use RAM of 2 GB.

RPM Repository is used as the source of Linux and application packages that need to be installed on the newly provisioned bare metal box. For example, an RPM Repository may be created to contain all the 32-bit Linux rpms and another repository may be created to contain Linux x86-64 bit rpms. Two separate Linux images can then be created each based on one of the repositories.

RHEL RPM repository to be used should have the following Red Hat Install tree structure:

There are multiple ways to create a RPM repository. If Red Hat Enterprise Linux CDs are available, do the following:

1. Copy all the contents of the first CD to a directory say RPM_REPOS.

2. Copy all rpms from other CDs to <RPM_REPOS>/Redhat/RPMS.Change directory to the RPMS directory:

cd <RPM_REPOS>/Redhat/RPMS

3. Add custom RPMs to the repository. This step is only required for pre-RedHat 5 versions (See Metalink Support Note 579715.1).

1. If there are custom RPMs installed on the reference host that need to be provisioned on the bare metal machine, make sure to copy them to the following repository location:

<RPM_REPOS>/Redhat/RPMS

2. Install anaconda-runtime RPM on the machine hosting the RPM repository. This might require other dependent packages to be installed.

3. Run the following commands:

cd /usr/lib/anaconda-runtime
./genhdlist --productpath=RedHat --withnumbers --hdlist <RPM_REPOS>/RedHat/base/hdlist <RPM_REPOS>

4. Run yum-arch :
This should create a headers directory. Make sure this directory contains a header.info file.

If yum is not installed then download it from the Linux Vendor's website.

5. Create a symbolic link in /var/www/html to <RPM_REPOS> directory.
The repository should now be available through http if an apache server is running.

Note:

In case the Apache server that comes with Enterprise Manger Grid Control 10g is used, enable the Apache directory index page using the "Options Indexes" directive in the Apache configuration (httpd.conf) file.

Oracle Enterprise Linux (OEL) RPM repository should have the Install tree structure shown below:

One can set up Oracle Enterprise Linux (OEL) Repository by using the OEL installation media as follows:

1. Download Oracle Enterprise Linux from http://edelivery.oracle.com/linux.

2. Copy all the contents of the first CD to a directory say RPM_REPOS.

3. Copy all rpms from other CDs to <RPM_REPOS>/Enterprise/RPMS.Change directory to the RPMS directory:

cd <RPM_REPOS>/Enterprise /RPMS

4. Add custom RPMs to the repository. This step is only required for pre-OEL 5 versions (See Metalink Support Note 579715.1).

1. If there are custom RPMs installed on the reference host that need to be provisioned on the bare metal machine, make sure to copy them to the following repository location:

<RPM_REPOS>/Enterprise/RPMS

2. Install anaconda-runtime RPM on the machine hosting the RPM repository. This might require other dependent packages to be installed.

3. Run the following commands:

cd /usr/lib/anaconda-runtime
./genhdlist --productpath=Enterprise --withnumbers --hdlist <RPM_REPOS>/Enterprise/base/hdlist <RPM_REPOS>

5. Run yum-arch :
This should create a headers directory. Make sure this directory contains a header.info file.

6. Create a symbolic link in /var/www/html to <RPM_REPOS> directory.

The repository should now be available through http if an apache server is running.

Note:

In case the Apache server that comes with Enterprise Manger Grid Control 10g is used, enable the Apache directory index page using the "Options Indexes" directive in the Apache configuration (httpd.conf) file.

2.3.5 Setting Up Software Library

Software Library should be located in a directory accessible by all OMSes. If there is one OMS the directory can be local. For multiple OMS environments, the directory can be on a Network File Server or a Netapp filer that is accessible from all the OMSes. One has to ensure that there is enough space available on the shared storage to store files that hold the binary data for one's components.

Software components that are generated as part of the default or single-server image creation during the bare metal provisioning process are stored in the Software Library. They are accessible under the Components tab in the Provisioning Application user interface.

Ensure that the shared storage is accessible through NFS mount points to all OMS servers in the environment.

2.3.6 Checklist for Boot Server, Stage Server, RPM Repository, and Reference Host

Ensure that the following criteria are met before provisioning:

Table 2-1 Checklist for Boot Server, Stage Server, RPM Repository, and Reference Host

Resource Name

Checklist

Boot Server

DHCP is up and running.

TFTP is up and running.

Boot Server is co located on the same machine as Stage Server. If not then Network Install Directory (/tftpboot/linux-install/pxelinux.cfg) is exposed to the Staging server for mounting.

Boot Server is present in the same subnet where the target machines to be provisioned are present or will be added.

Enterprise Manager agent is installed. Agent version same as the OMS version.

Boot server machine is visible as a managed target in Enterprise Manager.

A brand new PXE box actually detects the boot server and starts to boot it (even if no image is installed yet)

Stage Server

Stage server is as close as possible to the target servers.

Large storage, High Memory and Sufficient Memory.

If NAS server is used for storage then it should have NFS support.

If storage is located on NetApp NFS server as staging directory, the pathname of the staging directory, as mounted on the staging server, is the same as the pathname of the actual storage directory on the NetApp server.

Enterprise Manager agent is installed. Agent version should be same as the OMS version.

Boot server machine is visible as a managed target in Enterprise Manager.

The required agent rpm is staged for installing agents on targets.

Preferred Credentials are set.

Stage server is reachable from the box to be populated (or the same subnet)

RPM Repository

Install tree structure is as indicated in Configure RPM repository section.

RPM repository is available via HTTP.

Provide the exact URL and test the RPM repository access over HTTP

Reference Host

Agent version is same as OMS version.

Agent is installed on local disk and not on NFS mounted directory.

Preferred Credentials are set.

Software Library

Shared storage used for the software library is accessible through NFS mount points to all OMS servers.

2.4 Configuring Provisioning Environment in Enterprise Manager

This section provides information about configuring the provisioning environment in Enterprise Manager.

2.4.1 Setting Up Preferred Credentials in Enterprise Manager

Preferred credentials simplify access to managed targets by storing target login credentials in the Management Repository. With preferred credentials set, users can access an Enterprise Manager target that recognizes those credentials without being prompted to log into the target. Preferred credentials are set on a per user basis, thus ensuring the security of the managed enterprise environment.

Enterprise Manager supports two types of preferred credentials:

- Normal Credentials: Are used by Enterprise Manager functions that need operating system access, but do not require administrator privileges.

- Privileged Credentials: Are used by functions that need administrator privileges. Credentials for users that have sudo access on the target machine can be used as privileged credentials.

The Provisioning application requires preferred credentials to be setup for machines, which are part of the application.The preferred credentials need to be set for the following machines:

- Referenced Installation Host: Privileged credentials are needed to execute the command to get all the available RPMs form this machine. The credentials should also be valid ORACLE_HOME credentials (belonging to ORACLE_HOME owner's group)

- Stage Server: You must set the privileged preferred credentials for the stage server. Oracle recommends the stage server to have very limited access due to the criticality and sensitivity of the data it hosts. The super administrator can enforce this by creating one account on the stage server, and setting it as the preferred credential, to be used by all the provisioning users in Enterprise Manager. This preferred credential should also be a valid ORACLE_HOME credential (belonging to ORACLE_HOME owner's group).

- Provisioning Targets: In case you are planning to provision existing target machines, ensure the privileged credentials are setup. These credentials are required to clear the boot-sector and reboot the machine.

To manage Enterprise Manager preferred credentials:

- Click Preferences at the top of any Enterprise Manager Grid Control page.

- Click Preferred Credentials in the vertical navigation bar.

- Enterprise Manager displays the Preferred Credentials page. From this page, you can manage the preferred credentials for the supported targets.

Caution:

You must not use root as the preferred credential.

2.4.2 Invoking the Bare Metal Provisioning Application

To invoke the Bare Metal Provisioning Application, do the following:

1. Log in to Oracle Enterprise Manager.

2. The Provisioning application can be accessed by going to the Deployments Tab and then to Provisioning sub tab, as shown below.

Figure 2-3 Provisioning Application

The graphical user interface of the provisioning application shows various tabs for Components, Directives, and Images etc. A user can access all or some tabs shown above depending upon the privileges assigned to him. For example, in Figure 2-3, the Administration and Assignments tabs are disabled for the user. Refer to Creating Super Administrator for Enterprise Manager for creating users that can access the Administration tab.

In this section, we will assume that the user has super user privileges and can thus access the administration tab. This tab contains different sections for configuring different elements in the environments as shown in the next two figures.

Figure 2-4 Provisioning Application: Administration Tab

2.4.3 Configuring Stage Server

In this section, it is assumed that the stage server has been created and the necessary setup has been done.

Click Add Server in the Staging Server Configuration section. The following page comes up.

Figure 2-5 Add Staging Server Page

In the above step, enter the following details:

Staging Server Hostname refers to the host name of the stage server. It is recommended that you specify the IP address of the stage server, to avoid DNS resolution problems.

Full Directory Path refers to the top-level directory on the stage server which contains the agent rpm. All the staged files will be stored in this location.

Maximum Size Limit refers to the storage space in Megabytes assigned for the staging of files.

Base URL is used to expose the top-level stage directory to hardware servers being provisioned via NFS.

NOTE: If Stage Server uses NAS then the Base URL should be of the form:

<file>://<NAS_Hostname>/< NAS_DIRECTORY>

2) Provide the necessary values for all the parameters shown in the picture above and click OK to add the stage server.

2.4.4 Configuring RPM Repository

In this section it is assumed that the RPM repository has been created and the necessary setup has done.

Click Add in the RPM Repository Configuration section. The following page is displayed:

Figure 2-6 Add RPM Repository Page

Repository Name Assign a name to the RPM repository that is going to be added.

Complete URL refers to the URL pointing to the directory where the required RPMs are located. For example, http://sample.oracle.com/yum/EnterpriseLinux/EL4/oracle/i386

Provide the necessary values for all the parameters shown in the picture above and click OK to add the RPM repository.

2.4.5 Configuring Boot Server

In this section it is assumed that the Boot Server has been created and the necessary setup has done.

Click Add in the Boot Server Configuration section. The following page is displayed:

Figure 2-7 Add Boot Server Page

Boot Server Hostname refers to the host on which the boot server is setup.

Network Install Directory refers to the directory on the boot server where the pxelinux.cfg file is located. It is generally /tftpboot/linux-install directory on linux systems.

Provide the necessary values for all the parameters shown in the picture above and click OK to add the Boot Server.

2.4.6 Configuring Software Library

In this section it is assumed that the Software library has already been setup.

Click Add in the Software Library Configuration section. The following page is displayed:

Figure 2-8 Add Software Library Page

Software Library directory Location refers to the shared storage location where the deployable images will be stored. Ensure that this shared storage is accessible to all the OMS servers in the environment.

Provide the necessary values for all the parameters shown in the picture above and click OK to add the Software Library.

3 Provisioning Linux

This chapter provides step by step instructions on using the bare metal application in Enterprise Manager to provision Linux on bare metal servers.

3.1 Accessing the Bare Metal Provisioning Application

To access the Bare Metal Provisioning Application, do the following:

1. Log in to Oracle Enterprise Manager.

2. The Provisioning application can be accessed by going to the Deployments Tab and then to the Provisioning sub tab, as shown below.

Figure 3-1 Accessing the Provisioning Application

Tip:

Before you begin provisioning of Linux on bare metal boxes, make sure that preferred credentials are set for the Stage server and Reference Host. If not, follow instructions in Setting Up Preferred Credentials in Enterprise Manager to set up preferred credentials.

3.2 Creating a Default Image

Provisioning only Linux Operating system on target servers involves creation of a Default Image. A Default Image is a special type of image that installs operating system based on a reference Linux installation. It then deploys an Oracle Management Agent on the bare metal box so that it can subsequently be managed as an Enterprise Manager target. A default image can be created for a particular IP prefix or MAC address and provisioned on bare metal boxes as they are added to a given subnet.

The Default Image setup consists of following steps:

1. Go to the Images tab and click Create Image. The following page is displayed:

Figure 3-2 Create Image: Describe Page

In the Describe step, choose appropriate type of Default Image depending on the Operating System that needs to be provisioned on the bare metal servers. Enter a name for the image and other information that you may want to associate with it. Click Next.

2. The Configuration step comes up as shown in Figure 3-3.

Figure 3-3 Create Image: Configure Page

The Configure step depends on the type of the type of Image selected in the earlier step and is different for different types of Images. On this page, specify the following details:

Reference Installation is the Reference Host or gold host used for creating OS components for this image.

RPM repository allows you to choose an RPM repository from a list of RPM repositories configured for use with the provisioning application.

Additional RPM allows you to add RPMs for installation on the servers to be provisioned.

Exclude RPM allows you to specify the RPMs to be excluded from the image.

Time Zone and UTC Clock are used for setting the time zone of the bare-metal boxes after provisioning the operating system on them.

Root Password is the MD5 encrypted version of the root password that needs to be set for the newly provisioned Linux box. One way to encrypt is by using the /sbin/grub-md5-crypt command in Linux.

Following is an example with password 'manager1':

/sbin/grub-md5-crypt
Password: <type manager1>
Retype password: <re-type manager1>
$1$oONqW$wWSUsgzFdMMv5i2pQSqzY0

The utility will check that the same unencrypted password is encrypted twice. The result of the encryption is the string returned by the utility on the standard output.

Clicking Next brings up the Customize and Set Directives steps. These are optional for Default Image case and can be skipped. Directives are supported out-of-the-box for the Default Image case.

Final step is the Review step where all the entries made in the earlier steps can be reviewed. Clicking Finish on the review page kicks off a job to create the necessary components and directives required for the Default Image as shown in Figure 3-4.

Figure 3-4 Image Creation Confirmation

Clicking on the job link will show the various steps executed by the job.

Once the job is finished a new Image can be seen in the Images table on the Images tab. This Image is created with version 0.1 is in Ready state for use. Any subsequent changes to the Image will lead to an increment of 0.1 in revision. Select the Image and Click Activate to change the state of the Image to Active. An Image cannot be used for provisioning if it is not is Active state.

As a check, go to the components tab to check the components that have been created automatically created for the Default Image under the node "Oracle Components". The components created for the Default Image are non-editable as shown in Figure 3-5:

Figure 3-5 Components Tab

Note:

If you want to remove an existing image from the Software Library, do the following:

1. Make sure the Image is not set as a default image in the Deployments->Administration->Default Images section. If it is, remove the entry from the table.

2. Remove the assignments (if any) for the image from the Deployment->Provisioning->Assignments table.

3. Remove the image from the Software Library by removing it from Deployment->Provisioning->Images table.

4. Remove the components associated with this image from the Software Library by removing it from the Deployments->Provisioning->Components->Oracle Components section.

3.3 Setting Up a Default Image

1. Go to the Administration tab and click Add in the Default Images section on the Administration tab as shown below.

Figure 3-6 Default Images Section

2. The Add Default Image page shown in Figure 3-7 is displayed.

Figure 3-7 Add Default Image Page

Enter the following values on this page:

- IP Address Prefix or MAC Address:

- IP Address Prefix is used to specify the IP prefix of subnet where the bare-metal machines would be added.

- MAC Address is used to specify the MAC addresses of the bare metal machines to be added.

- Choose a desired Boot Server and Stage Server from the list of servers configured with the provisioning application.

- Specify the Image name to be used as the Default Image, preferred credentials for the Stage server.
Do not select anything for the Network Profile. For the bare metal machines the application automatically chooses DHCP to assign IP addresses and host names.

- Click Edit Advanced Properties to view the page shown in Figure 3-8.

Figure 3-8 Edit Advanced Properties Page

You can provide the following properties:

- In the Agent Settings section, specify the agent user, group, and Oracle home for the install, and the location of the agent RPM. If the agent RPM location is not specified, the RPMs will be picked up from the stage location.

- In the Security Settings section, you can add a script in the Security Parameters field. You can enable or disable SELinux.

- In the Kernel Settings section, you can add a script in the Kernel Parameters field.

- In the Mount Point Settings section, you can specify entries for the /etc/fstab file. You can specify mount points on the newly provisioned Linux machine. By default, mount point settings from the reference Linux machine are inherited.

- In the NTP Settings section, you can specify entries for the /etc/ntp.conf file. You can specify NTP settings for the newly provisioned Linux machine. By default, NTP settings from the reference Linux machine are inherited.

- In the NIS Settings section, specify entries for the /etc/yp.conf file. You can specify NIS settings for the newly provisioned Linux machine. By default, NIS settings from the reference Linux machine are inherited.

- In the Post section, specify any set of commands that need to be executed on the newly provisioned machine. These commands will be appended to the post section of the kickstart file

Note:

If setting up of the default image fails, delete the image, delete the component, purge the software library and retry.
Running this job may take a few minutes.

3.4 Staging the Default Image

Once the Image is created, click Stage in the Default Images section and select the Image created earlier.

Figure 3-9 Default Images

This prepares the default image ready for installation on the staging server. One will be asked to enter the OMS registration password before staging starts.

An Enterprise Manager job is submitted to do the staging of default image. The on-going status can be seen using the Enterprise Manager job console.

Note:

If staging of the image fails, correct the pre-requisite failures mentioned in the failed job and retry the staging operation. Click Edit in the Default Images section and re-submit the staging job by clicking Finish.
You can also clean up the image from the Default Images table and try again.

Running this job may take a few minutes.

3.5 Ready to Go

After completing the above steps, one is ready to go. Plug in the bare metal boxes in the subnet and the provisioning application will automatically provision them to have the default image and management agent on it.

Note:

Once Linux is provisioned on the bare metal box, out-of-box Deployment Procedures can be used to provision Database and other Oracle products on the server. See "Using Enterprise Manager For Grid Automation With Deployment Procedures" in the Enterprise Manager Advanced Configuration Guide for more information on how to use Deployment Procedures.

A Troubleshooting

This appendix suggests solutions to issues that you may encounter when provisioning Linux.

I cannot see my stage, boot server in the UI to configure them with the provisioning application?

Either Management Agents have not been installed on the Stage/Boot Server machine or its not uploading data to the OMS. Refer to Agent Deployment Best Practices for troubleshooting information and known issues.

"Cannot create under the software library, please contact your administrator" error comes up while creating default image.

This may happen because the Software Library is not configured. Refer to Configuring Software Library. If it is configured and the error still occurs, then check if the library directory is accessible from the OMS server. The software library location should be accessible from the OMS and should have enough space.

Default Image or OS component creation fails with a '404' status while copying the header.info.

This may happen because of the RPM Repository URL being incorrect or because the RPM Repository has not been configured properly. Please refer to Setting Up RPM Repository and Configuring RPM Repository.

Default Image or OS component creation fails with an error message "Following RPMs are not found in Repository".

Reference machine has additional RPMs that are not available in the RPM repository. One of the following alternatives can be selected:

- Choose a different reference machine.

- Add extra RPMs in RPM repository (Make sure the RPMs are added in the repository as per Redhat specifications. Header files should be updated with the new RPM).

- If the rpms are not required on the provisioned machine mention them in the custom_rpm.properties file. Refer to Appendix C.

Default Image or OS component creation job is suspended.

This may happen if the Preferred Credentials are not set for Reference machine. Please set the Preferred Credentials.

Default Image or OS component creation fails "Sudo error".

This may happen if

- Sudo is not installed on the reference machine

- The user mentioned for the Reference Machine is not in the sudoers list. Edit /etc/sudoers file and add the user in the list.

Default Image staging fails "cannot stage default image".

This happens if there is already a default image staged for the same IP address range as the current one. Remove the staged image by clicking on the Remove button in the Default Image section on the Administration tab.

Default Image or OS component staging/provision fails with "Sudo error".

This may happen if

- Sudo is not installed on the reference machine.

- The user mentioned for the Reference Machine is not in the sudoers list. Edit /etc/sudoers file and add the user in the list.

Default Image or OS component staging/provision job is suspended.

This may happen if the Preferred Credentials are not set for Reference machine. Please set the Preferred Credentials.

Default Image or OS component staging/provision job fails "with directory permission error".

This error happens because of insufficient user privileges on the stage server machine. STAGE_TOP_LEVEL_DIRECTORY should have the write permission for the Stage user. In case of NAS, the NAS directory should be mounted on the staging server. If the error is while writing to the '/tftpboot directory then it has to have the write permission for the Stage user.

Bare metal machine is not coming up since it cannot locate the Boot file.

Verify the dhcp settings (/etc/dhcpd.conf) & tftp settings for the target machine. Check whether the services (dhcpd, xinted, portmap) are running. Make the necessary setting changes if required or start the required services if they are down.

Even though the environment is correctly setup, bare metal box is not getting booted over network

OR

Dhcp server does not get a DHCPDISCOVER message for the MAC address of the bare metal machine.

Even if your environment is setup properly and default image is staged properly you might face this issue if there is some problem with the NIC card. Because of this boot requests do not reach the boot server. Please verify the NIC and network connections.

Provisioning Default image on the bare metal box fails with "reverse name lookup failed" error.

One needs enter IP address of the Stage/NAS server if you have issue with 'reverse lookup'. Do the following:

- Go to the Administration tab.

- Click Edit button after selecting the existing Stage server entry.

- Replace the Stage/NAS server hostname with its IP address.

Agent Installation fails after operating system has been provisioned on the bare metal box?

OR

No host name is assigned to the bare metal box after provisioning the OS?

This might happen if the "get-lease-hostnames" entry in the dhcpd.conf file is set to true. Edit the dhcpd.conf file to set get-lease-hostnames entry to false.

Bare metal machine hangs after initial boot up (tftp error/kernel error).

This may happen if the tftp service is not running. Enable the tftp service. Go to the '/etc/xinetd.d/tftp' file and change the 'disable' flag to 'no' (disable=no). Also verify the dhcp settings.

Kernel panic occurs when the Bare Metal machine boots up.

Verify the dhcp settings and tftp settings for the target machine and make the necessary changes as required. In a rare case, the intird and vmlinuz copied may be corrupted. Copying them from RPM repository again would fix the problem.

Bare metal machine hangs after loading the initial kernel image.

This may happen if the network is half duplex. Half duplex network is not directly supported but following steps below would fix the problem:

- On the Reference Machine modify 'ethtool -s eth0 duplex half' entry to 'ethtool -s eth0 duplex full' in the kickstart file.

- Once done recreate the Default Image.

Bare metal machine cannot locate the kickstart file (Redhat screen appears for manually entering the values such as 'language', 'keyboard' etc).

This happens if STAGE_TOP_LEVEL_DIRECTORY is not mountable or not accessible. Make sure the stage top level is network accessible to the target machine. Though very rare but this might also happen because of any problem in resolving the stage server hostname. Enter the IP address of the stage/NAS server instead of hostname on which they are located and try the provisioning operation again.

Bare metal machine does not go ahead with the silent installation (Redhat screen appears for manually entering the network details).

Verify the DHCP settings for the target machine. Also verify that Network profile attached with the provisioning operation is proper. Though very rare but this might also happen because of any problem in resolving the stage server hostname. Enter the IP address of the stage/NAS server instead of hostname and try the provisioning operation again.

After provisioning, the machine is not registered in EM.

This happens if the EM Agent is not placed in the STAGE_TOP_LEVEL_DIRECTORY before provisioning operation. Place the EM agent in this directory and try the operation again. It might also happen if the OMS registration password provided for securing the agents is incorrect. Go to the agent oracle home on the target machine and run the emctl secure agent command supplying the correct OMS registration password.

Component creation fails with "Cannot create under the software library, please contact your administrator".

This may happen if the Software Library is not configured with the Enterprise Manager. Refer Configuring Software Library. Create the components once the software library is configured.

"Cannot create under the software library, please contact your administrator" error comes up while creating default image.

This may happen because the Software Library is not configured. Refer to Configuring Software Library. If it is configured and the error still occurs, then check if the library directory is accessible from the OMS server. The software library location should be accessible from the OMS and should have enough space.

B PXE Booting and Kickstart Technology

This appendix explains about PXE booting and kickstart technology.

B.1 About PXE Booting and Kickstart Technology

One of the key requirements of provisioning is the hardware server's ability to boot over the network instead of a diskette or CD-ROM. There are several ways computers can boot over a network, and Preboot Execution Environment (PXE) is one of them. PXE is an open industry standard supported by a number of hardware and software vendors. PXE is part of the "Wired for Management" (WfM) specification, which is part of a bigger PC98 specification defined by Intel and Microsoft in 1998. A detailed document on PXE specification can be found at http://www.pix.net/software/pxeboot/archive/pxespec.pdf.

PXE works with Network Interface Card (NIC) of the system by making it function like a boot device. The PXE-enabled NIC of the client sends out a broadcast request to DHCP server, which returns with the IP address of the client along with the address of the TFTP server, and the location of boot files on the TFTP server. The following steps describe how it works:

1. Target Machine (either bare metal or with boot sector removed) is booted.

2. The Network Interface Card (NIC) of the machine triggers a DHCP request.

3. DHCP server intercepts the request and responds with standard information (IP, subnet mask, gateway, DNS etc.). In addition, it provides information about the location of a TFTP server and boot image (pxelinux.0).

4. When the client receives this information, it contacts the TFTP server for obtaining the boot image.

5. TFTP server sends the boot image (pxelinux.0), and the client executes it.

6. By default, the boot image searches the pxelinux.cfg directory on TFTP server for boot configuration files on the TFTP server using the following approach:
First, it searches for the boot configuration file that is named according to the MAC address represented in lower case hexadecimal digits with dash separators. For example, for the MAC Address "88:99:AA:BB:CC:DD", it searches for the file 01-88-99-aa-bb-cc-dd.

Then, it searches for the configuration file using the IP address (of the machine that is being booted) in upper case hexadecimal digits. For example, for the IP Address "192.0.2.91", it searches for the file "C000025B".

If that file is not found, it removes one hexadecimal digit from the end and tries again. However, if the search is still not successful, it finally looks for a file named "default" (in lower case).

For example, if the boot file name is /tftpboot/pxelinux.0, the Ethernet MAC address is 88:99:AA:BB:CC:DD, and the IP address 192.0.2.91, the boot image looks for file names in the following order:

/tftpboot/pxelinux.cfg/01-88-99-aa-bb-cc-dd
/tftpboot/pxelinux.cfg/C000025B
/tftpboot/pxelinux.cfg/C000025
/tftpboot/pxelinux.cfg/C00002
/tftpboot/pxelinux.cfg/C0000
/tftpboot/pxelinux.cfg/C000
/tftpboot/pxelinux.cfg/C00
/tftpboot/pxelinux.cfg/C0
/tftpboot/pxelinux.cfg/C

7. The client downloads all the files it needs (kernel and root file system), and then loads them.

8. Target Machine reboots.
The Provisioning application uses Redhat's Kickstart method to automate the installation of Redhat Linux on target machines. Using kickstart, the system administrator can create a single file containing answers to all the questions that will usually be asked during a typical Red Hat Linux installation.

The host specific boot configuration file contains the location of the kickstart file. This kickstart file would have been created earlier by the stage directive of the OS image based on the input from user.

C Sample custom_rpm.properties File

This appendix provides a sample custom_rpm.properties file.

C.1 custom_rpm.properties File

The file custom_rpm.properties is used for adding extra rpms other than the ones present on the Reference Machine or denying some rpms present on the Reference Machine while provisioning of Default Images on Bare Metal Boxes. This file can be found at:

<AGENT HOME Oracle Home on the Reference Machine >/agent10g/sysman/admin/scripts/provisioning/custom_rpms.properties

A sample custom_rpm.properties file is shown below.

A Sample custom_rpm.properties File
This appendix provides a sample custom_rpm.properties file.
custom_rpm.properties File
#
# Copyright (c) 2004, 2008, Oracle. All rights reserved.
#
# NAME
# custom_rpms.properties
#
# DESCRIPTION
# This file contains names of custom rpms that are to be added or denied during OS Component creation.
#
# The rpm names under [additional rpms] contain rpms that are included in OS Component even if
# They are not installed in the reference machine.
#
# The rpm names under [deny rpms] contain rpms that are excluded in OS Component even if
# They are installed in the reference machine.
#
# The rpm names mentioned in the list should not contain the version number.
#
#
# NOTES
# Do not remove the entries from the existing list.
#
#These rpms are added
[additional_rpms]
sudo
kernel
redhat-release
glibc
setarch
pdksh
sysstat
chkconfig
initscripts
gcc
#These rpms are denied
[deny_rpms]
emagent
comps
gpg-pubkey

D Frequently Asked Questions

This appendix outlines frequently asked questions on provisioning Linux.

D.1 Frequently Asked Questions on Setting Up the Provisioning Environment

What is PXE (Pre-boot Execution Environment)?

The Pre-boot Execution Environment (PXE, aka Pre-Execution Environment) is an environment to bootstrap computers using a network interface card independently of available data storage devices (like hard disks) or installed operating systems. Refer to Appendix B for more information.

Can my Boot server and Stage server be located on different machine?

It is recommended that the Boot server and Stage server are co-located on the same physical machine. If this is not done, then one has to ensure that the network boot directory (say /tftpboot/pxelinux0.cfg) is exposed to Stage server via NFS and can be mounted by it.

Can my boot server reside on a subnet other than the one on which the bare metal boxes will be added?

Yes. But it is a recommended best practice to have boot server in the same subnet on which the bare metal boxes will be added. If the network is subdivided into multiple virtual networks, and there is a separate DHCP/PXE boot server in each network, the Assignment must specify the boot server on the same network as the designated hardware server.

If one wants to use a boot server in a remote subnet then one of the following should be done:

-- Router should be configured to forward DHCP traffic to a DHCP server on a remote subnet. This traffic is broadcast traffic and routers do not normally forward broadcast traffic unless configured to do so. A network router can be a hardware-based router, such as those manufactured by the Cisco Corporation or software-based such as Microsoft's Routing and Remote Access Services (RRAS). In either case, you need to configure the router to relay DHCP traffic to designated DHCP servers.

-- If routers cannot be used for DHCP/BOOTP relay, set up a DHCP/BOOTP relay agent on one machine in each subnet. The DHCP/BOOTP relay agent relays DHCP and BOOTP message traffic between the DHCP-enabled clients on the local network and a remote DHCP server located on another physical network by using the IP address of the remote DHCP server.

Why is Agent rpm staged on the Stage server?

Agent rpm is used for installing the Agent on the target machine after booting over network using PXE.

Can I use the Agent rpm for installing Agent on Stage and Boot Server?

This is true only if the operating system of the Stage or Boot Server machine is RedHat Linux 4.0, 3.1 or 3.0. Refer to section Using agent rpm for Oracle Management Agent Installation on the following page for more information:

http://www.oracle.com/technology/software/products/oem/htdocs/provisioning_agent.html

Can the yum repository be accessed by any protocol other than HTTP?

Though the rpm repository can be exposed via file:// or ftp:// as well, the recommended method is to expose it via http://. The latter is faster and more secure.

How to setup a reference host if I don't have an Oracle Enterprise Linux server?

In case one does not have an Oracle Enterprise Linux server then one can switch a RedHat Linux server to Oracle Enterprise Linux. To do this, follow the steps described here:

https://linux.oracle.com/switch.html

Create an Oracle Enterprise Linux server using installation media available at:

http://edelivery.oracle.com/linux

D.2 Frequently Asked Questions on Setting Up and Provisioning Default Images

Is it possible to include extra packages other than the one on reference machine?

This can be done by using the custom_rpms.properties file. This file can be found at:

<AGENT HOME Oracle Home on the Reference Machine>/agent10g/sysman/admin/scripts/provisioning/custom_rpms.properties

Modify the [additional rpms] section of this file to mention any additional rpms not present on the Reference Machine. Only the name of the package without the version should be mentioned. Refer to Appendix C.

Is it possible to exclude any packages from reference machine while provisioning the default image on the target machines?

To prevent any rpms present on the Reference Machine from being installed on the target machines, mention the list of rpms to be excluded in the [deny_rpms] section of the custom_rpms.properties file. Refer to Appendix C.

D.3 Frequently Asked Questions on Directives and Components

Are directives associated with Components or Images?

Directives can be associated with both components and images. As explained later during the creation of components and images, directives are actually associated with one of Staging, Pre-Install, Install or Post-Installation provisioning steps of a component. In case of an Image, directives can be associated with Stage, Cleanup, Post-Install, and Diagnostics provisioning steps of an Image.

What is the significance of the Status of a directive? How can one change it?

Look at the following table to know the possible Status values and what they signify.

Table D-1 Status Values

Status

Description

Incomplete

This Status signifies that some step was not completed during the directive creation, for example uploading the actual script for the directive, or a user saved the directive while creating it and still some steps need to be performed to make complete the directive creation.

Ready

his signifies that the directive creation was successful and the directive is now ready to be used along with any component/image.

Active

A user can manually change the status of a Ready directive to Active to signify that it is ready for provisioning. Clicking Activate changes the Status to Active.

What is a Maturity Level of a directive? How can one change it?

Look at the following table to know the possible Status values and what they signify:

Table D-2 Maturity Levels

Maturity Level

Maturity Level Description

Untested

This signifies that the directive has not been tested and is the default maturity level that is assigned to the directive when it is created.

Beta

A directive can be manually promoted to Beta using the Promote button after testing the directive.

Production

A directive can be manually promoted to Production using the Promote button after a user is satisfied that the directive can be used for actual provisioning on production systems.

Can a same component be used in multiple images?

Yes. Components are reusable and a given component can be a part of multiple images at the same time.

Do I need to modify the images associated with a component, if the component is edited?

No. If the component is changed for some reason then the change is propagated to all the images with contain that component.

For creating the Linux OS component does the Reference Machine need to have a management agent running on it?

Yes. Reference Machine has to be one of the managed targets of the Enterprise Manager.

What if the working directory does not have enough space while creating a clone component?

The component creation job that is kicked off in the end will indicate that the component creations failed because of insufficient space. Create the component again by specifying a temporary directory with sufficient space and the creation will succeed.

What is the significance of the Status of a component? How can one change it?

Status of a component is similar to that of a directive. Refer to What is the significance of the Status of a directive? How can one change it?.

What is a Maturity Level of a component? How can one change it?

Maturity Level of a component is similar to that of a directive. Refer to What is a Maturity Level of a directive? How can one change it?.

What is the purpose of associating Properties with Generic Components?

Properties are used to provide flexibility for customizing the components depending on need. The property values are fed to the directives, which then customize the installation depending upon the values.

D.4 Frequently Asked Questions on Network Profiles and Images

Can I use 10.1.x IP addresses for Private Network Configuration?

No. Because of a known limitation this type of addressing cannot be done for the private IPs of the cluster nodes.

What is Reset Timeout?

After the OS installation on a target machine, management agent is installed on it, which reports to OMS and makes the machine a managed target of the Enterprise Manager. The time duration between the start of OS provisioning to agent reporting back to OMS server is known is the Reset Timeout. This is used by the provisioning application for deciding it provisioning process has timed out. If the machine or network is slow then it is advisable to set a high reset value.

Copyright © 2008, Oracle. All rights reserved.