1. Installation and Configuration Guide for Exadata Database Machine
  2. Configuring Oracle Exadata Database Machine
  3. Imaging a New System
  4. Using PXE to Image a New System
  5. Using PXE to Image a New System Using Release 12.1.2.2.0 or Later

6.12.4.1 Using PXE to Image a New System Using Release 12.1.2.2.0 or Later

You can use PXE to prepare a new Exadata system running Oracle Exadata System Software release 12.1.2.2.0 or later for deployment using a simplified method.

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA) from http://www.oracle.com/technetwork/database/exadata/oeda-download-2076737.html.
  2. Run the OEDA configuration tool to generate the configuration files.
  3. Power on all the servers.
    When the system is first started, the following conditions exist:
    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.
    2. Host names of the nodes will be of the format nodeN, where N is a number.
  4. Run OEDA to generate the preconf.csv file that you will use to image the system.
  5. Connect to the first database node.

    Use one of the following connection options:

    • Attach a serial cable between a console and the SER MGT port on the server.

      Connections through the modular jack serial port must use shielded cables to comply with radio frequency emission limits.

      Set the terminal device communication properties to these values: 9600 baud, 8 bit, no parity, 1 stop bit.

      For more details, see Connect to Oracle ILOM.

    • Attach an Ethernet cable to the NET MGT port on the server and then connect using a workstation on the network.

      For more details, see Connect to Oracle ILOM.

  6. Obtain the eth0 MAC address for each database node.
    The eth0 MAC address is the fru_macaddress field from /SYS/MB/NET0 in the ILOM (or from ip addr if logged into the console).
    1. Connect to the each database node through SSH from the console.
    2. Run ibhosts to get a list of node names and IP addresses.
  7. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node.
    This field should be empty, that is, you should see two consecutive comma characters (,,).

    For example:

    orhb42-b08-01,example.com,db,eth0,eth0,Management,00:10:E0:69:34:52,10.196.3.
106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:active-bond-ib,192.1
68.10.15,255.255.255.0,,,America/Los_Angeles
  8. Create a PXE configuration file for each node.

    The file name is hostname.xx-xx-xx-xx-xx-xx in the pxelinux.cfg directory. Below are the recommended options for PXE imaging of versions 12.1.2.1.0 and above. The line starting with append should be entered as a single line with no line breaks. 

    kernel vmlinux-iso-file-name
append initrd=img_file pxe factory stit reboot-on-success notests=diskgroup dhcp 
preconf=n.n.n.n:/directory/preconf.csv iso-uri=http://web_server/x86_64/iso/image_n
ame.iso console=ttyS0,115200n8

    On systems with Oracle Exadata System Software release 20.1.0 or later, you can enable or disable Exadata Secure RDMA Fabric Isolation by adding qinq=yes or qinq=no to the kernel options. For example: 

    kernel vmlinux-iso-file-name
append initrd=img_file pxe factory stit reboot-on-success notests=diskgroup dhcp 
preconf=n.n.n.n:/directory/preconf.csv iso-uri=http://web_server/x86_64/iso/image_n
ame.iso console=ttyS0,115200n8 qinq=no
  9. Optionally, configure key-based authentication for the root OS user.

    Starting with Oracle Exadata System Software release 20.1.0, you can configure key-based authentication for the root OS user as part of the PXE imaging process.

    If you want to use this option, then perform the following for each node:

    1. Create an RSA public and private key pair using a utility such as ssh-keygen.

      For example:

      # ssh-keygen -t rsa -b 4096
    2. Rename the public key file so that the file name conforms to the following format:
      id_rsa.short_hostname.root.pub

      In the public key file name, short_hostname is the node host name as defined in the first field of the corresponding node entry in the preconf.csv file.

    3. Copy the renamed public key file to the directory that contains the preconf.csv file.

      This is the same directory that you previously specified in the PXE configuration file preconf parameter, which defines the path to the preconf.csv file.

    If you configure key-based authentication, then password-based authentication is disabled after the imaging process. Consequently, root access to the node is only possible using the matching private key. ILOM access using the root password is also disabled.

    By default, if the public key file is not named or located correctly, then password-based authentication is enabled for the node. You can also make key-based authentication mandatory by specifying root-ssh-key=yes as a boot option in the PXE configuration file. In this case, the imaging process fails if the public key is not in the correct location.

    Regardless of any public keys, you can disable the configuration of key-based authentication by specifying root-ssh-key=no as a boot option in the PXE configuration file.

  10. Create the image files, if necessary.

    If you are using version 12.1.2.2.0 or later, it is no longer necessary to use ImageMaker to create the image files. The relevant PXE, ISO, and USB image files have already been created in the patches themselves. Simply download the patch containing the pre-built PXE image files from Oracle Software Delivery Cloud and place them into the appropriate directories on the PXE server. For all releases, see My Oracle Support note 888828.1 for the patch numbers. There are separate PXE image patches for database servers and storage servers.

    Note: For image version 12.1.2.2.0 the format of the downloadable image files has changed.

    The downloaded zip file contains four image files:

    • For 12.1.2.2.0 up to 12.2.1.1.0:
      kernel: vmlinux-version-ImageName-{DL180|DL360}
initrd: initrd-version-ImageName-{DL180|DL360}.img
image: nfsimg-version-ImageName-{DL180|DL360}.tar
image: nfsimg-version-ImageName-{DL180|DL360}.tar.md5
    • For 12.2.1.1.0 and above:
      kernel: vmlinux-iso-ImageName-{cell|compute}
initrd: initrd-iso-ImageName-{cell|compute}.img
image: vmlinux-iso-ImageName-{cell|compute}.tar
  11. Change the boot order using the ILOM to boot from PXE and then restart the server to start the imaging process.

    For example, from the Oracle Linux prompt on the host: 

    # ipmitool chassis bootdev pxe

# shutdown -r now
  12. If you are using Oracle VM, run switch_to_ovm.sh on all Oracle VM database servers before running the OEDA installation tool.

    When you run this command, the database servers will reboot.

    # /opt/oracle.SupportTools/switch_to_ovm.sh
2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LV
DbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbS
ys1
2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/
VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSy
s3 in /boot/I_am_hd_boot
2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0
 system partition
  13. On systems with InfiniBand Network Fabric only, run /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim on each database server before installing any software.

    The command reclaims disk space reserved for the deployment type not selected, which is not required on RoCE-based Exadata database servers.

    The command typically takes less than 5 minutes.

    Caution:

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh. 
    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
Model is ORACLE SERVER X5-2
Number of LSI controllers: 1
Physical disks found: 4 (252:0 252:1 252:2 252:3)
Logical drives found: 1
...
[INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
[INFO     ] Create filesystem on device /dev/sda1
[INFO     ] Tune filesystem on device /dev/sda1

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported.  For the first word, TAB
lists possible command completions.  Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
 Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
Done.
grub> quit
  14. Run ibhosts and verify that all nodes show the correct IP addresses and host names.
    There should be no nodes with elasticNode in the description.
  15. Run the OEDA tool to deploy.

    Note:

    If you configured key-based authentication for the root OS user, then you must first copy the node private keys into the OEDA WorkDir. 

    [root] # ./install.sh -cf ../machine_config_file.xml -l
1. Validate Configuration File
2. Setup Capacity-on-Demand
3. Create Virtual Machine  
4. Create Users  
5. Setup Cell Connectivity  
6. Create Cell Disks  
7. Create Grid Disks  
8. Configure Alerting  
9. Install Cluster Software  
10. Initialize Cluster Software  
11. Install Database Software  
12. Relink Database with RDS  
13. Create ASM Diskgroups  
14. Create Databases  
15. Apply Security Fixes  
16. Install Exachk  
17. Setup ASR Alerting  
18. Create Installation Summary  
19. Resecure Machine

Related Topics

Parent topic: Using PXE to Image a New System