1. Security Guide for Oracle Exadata
  2. Securely Erasing Oracle Exadata
  3. Automatic Secure Eraser through PXE Boot
  4. Automatic Secure Eraser through PXE Boot for X7 and Later Systems

5.3.1 Automatic Secure Eraser through PXE Boot for X7 and Later Systems

In this procedure, you configure Secure Eraser to run automatically when you reboot Oracle Exadata X7-2 and later nodes.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Before you begin:

  • Download and install the latest Secure Eraser package available for your system.

    To download the Secure Eraser package, examine the Supplemental Readme associated with your current Oracle Exadata System Software version. To find the Supplemental Readme associated with each Oracle Exadata System Software version, see Exadata Database Machine and Exadata Storage Server Supported Versions (My Oracle Support Doc ID 888828.1).

  • Make sure you have access to a Preboot Execution Environment (PXE) server where the nodes to be erased can boot from.

  • Make sure you have access to a NFS server that is accessible from all the nodes to be erased.

  • Make sure you have access to one of the nodes to be erased.

  1. Copy the PXE image files initrd (initrd-<version>) and kernel (vmlinux-<version>) from the Secure Eraser package to the /tftpboot directory on the PXE server.

  2. Create a file containing the names of the database servers and storage servers you want to erase.

    To generate this file, you can run the following command from one of the nodes to be erased, and verify the nodes in the files are the ones to be erased.

    # ibhosts | awk '/S [0-9\.\,]*/ || /C [0-9\.\,]*/ {print $6}' | 
sed "s/\"//g" > nodes_to_be_erased

    If you only want to erase one server, enter the name of the server into the nodes_to_be_erased file, for example Exa01celadm04.

  3. Copy the dcli utility from the Secure Eraser package and the nodes_to_be_erased file generated in step 2 to the PXE server.

  4. Create a PXE configuration template called pxe_cfg.template to contain the following lines:

    Note:

    In the following example, the following parameters must be updated to match your environment:

    • kernel (the vmlinux file)
    • initrd (the initrd*.img file)
    • logpath

    • For Oracle Exadata System Software 18c (18.1.0): 

      set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-nfs-18.1.0.0.0-170915.1 dhcp pxe boot-from=uefi 
quiet loglevel=0 secureeraser bootarea=diagnostics console=ttyS0,115200n8 
logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-nfs-18.1.0.0.0-170915.1.img 
  echo "Booting installation kernel"
}

    • For Oracle Exadata System Software release 19.1.0 or later: 

      set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-iso-19.1.2.0.0-190111 dhcp pxe boot-from=uefi 
quiet loglevel=0 secureeraser bootarea=diagnostics console=ttyS0,115200n8 
logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-iso-19.1.2.0.0-190111.img 
  echo "Booting installation kernel"
}

    • The first line (default) identifies a menu entry that should be selected by default, after the timeout value specified by the second line.

    • The third line (menuentry) represents the Linux kernel that will be used in the Secure Eraser environment.

    • The fifth line (linuxefi) indicates the kernel is on an UEFI-based system. The linuxefi statement must be on a single line in the configuration file.

      • The dhcp option specifies to use DHCP to discover the eth0 interface.

      • The pxe option suppresses search for the image on virtual CD and USB devices.

      • The boot-from=uefi option indicates the system is booting from UEFI.

      • The quiet option disables excessive kernel log messages.

      • The loglevel=0 option suppresses non-critical kernel messages.

      • The secureeraser option indicates PXE boot will automatically trigger the Secure Eraser utility to sanitize all media installed on the node, including hard drives, flash devices, persistent memory, internal USBs, and ILOM.

      • The bootarea option indicates that the boot mode is diagnostic and not imaging install or rescue.

      • The console options indicate standard output and standard error messages are printed to both ILOM web console and serial console.

      • The logpath option specifies the NFS share directory where Secure Eraser will save the certificate.

    • The seventh line (initrdefi) specifies the initrd file to load. In this case it is the initrd file copied over in step 1.

    By default, the examples shown above cause Secure Eraser to erase all components. You can use secureeraser-options to specify command-line options for Secure Eraser to change the default behavior and securely erase certain components only. For example, to erase hard drives and USBs only during the PXE boot, the template would look like this for grub2 / Secure Boot on Oracle Exadata Database Machine X7 and later systems:

    Note:

    In the following example, the following parameters must be updated to match your environment:

    • kernel (the vmlinux file)
    • initrd (the initrd*img file)
    • logpath

    • For Oracle Exadata System Software 18c (18.1.0): 

      set default 0 
set timeout=10
menuentry 'ExadataLinux' {
   echo "Loading efi/vmlinuz" 
   linuxefi efi/vmlinux-nfs-18.1.0.0.0-170915.1 stit dhcp pxe boot-from=
uefi quiet loglevel=0 secureeraser secureeraser-options="--hdd --usb" 
bootarea=diagnostics console=ttyS0,115200n8 logpath=10.133.42
.221:/export/exadata_secure_eraser_certificate_dir
   echo "Loading efi/initrd.img"
  initrdefi efi/initrd-nfs-18.1.0.0.0-170915.1.img
   echo "Booting installation kernel"
}

    • For Oracle Exadata System Software release 19.1.0 or later: 

      set default 0 
set timeout=10
menuentry 'ExadataLinux' {
   echo "Loading efi/vmlinuz" 
   linuxefi efi/vmlinux-iso-19.1.2.0.0-190111 stit dhcp pxe boot-from=
uefi quiet loglevel=0 secureeraser secureeraser-options="--hdd --usb" 
bootarea=diagnostics console=ttyS0,115200n8 logpath=10.133.42
.221:/export/exadata_secure_eraser_certificate_dir
   echo "Loading efi/initrd.img"
  initrdefi efi/initrd-iso-19.1.2.0.0-190111.img
   echo "Booting installation kernel"
}

  5. On the PXE server, use the template file to generate a PXE configuration file in the /tftpboot/pxelinux.cfg/ directory for each of the nodes to be erased.

    The PXE configuration file name is the dash-separated MAC address of the node with the prefix 01-.

    If the nodes to be erased are accessible, use the following steps to automatically generate a PXE configuration file for each node based on the template.

    1. Set up SSH equivalence with the nodes to be erased from the PXE server. The command will prompt for the root password of each node. 

      pxe_server# dcli -g nodes_to_be_erased -k -l root

    2. Create PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# dcli -g nodes_to_be_erased -l root "ip addr show eth0" | 
awk '/link\/ether/ {print "01:"$3}' | sed "s/:/-/g" | 
xargs -I {} cp pxe_cfg.template {}

    If the nodes are not accessible, use the following step to generate a PXE configuration file for each node to be erased:

    1. Manually collect the MAC address of the eth0 interface from each node and write them into a text file called mac_addresses. Write one MAC address per line. For example: 

      00:10:e0:62:c4:fa
00:10:e0:62:c2:8a
00:10:e0:62:b8:7c
00:10:e0:62:b8:3a
00:10:e0:62:c6:bc

    2. Use the following command to create a list of PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# cat mac_addresses | sed "s/:/-/g;s/^/01-/g" | 
xargs -I {} cp pxe_cfg.template {}

    In both cases, you should get a list of PXE configuration files, one for each node to be erased. For example, if the MAC addresses of the nodes in a quarter rack are 00:10:e0:62:c4:fa, 00:10:e0:62:c2:8a, 00:10:e0:62:b8:7c, 00:10:e0:62:b8:3a, and 00:10:e0:62:c6:bc, you should get the following files:

    01-00-10-e0-62-c4-fa
01-00-10-e0-62-c2-8a
01-00-10-e0-62-b8-7c
01-00-10-e0-62-b8-3a
01-00-10-e0-62-c6-bc

    The files have the same content as the configuration template.

    Check your specific PXE server requirements. Your PXE server may need slightly different names or settings.

  6. Configure the nodes to boot from PXE and reboot the nodes.

    If the nodes to be erased are accessible, run the following commands:

    pxe_server# dcli -g nodes_to_be_erased -l root "ipmitool chassis bootdev pxe”

pxe_server# dcli -g nodes_to_be_erased -l root "reboot”

    If the nodes to be erased are not remotely accessible but the ILOMs are, use the following steps

    1. Create a file called iloms_to_be_reset containing the names of ILOMs. For example: 

      db1-ilom
db2-ilom
cell1-ilom
cell2-ilom
cell3-ilom

    2. Configure the nodes to boot from PXE through ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis bootdev pxe

    3. Reboot the nodes from ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis power cycle

    If neither host nor ILOM is remotely accessible, log into ILOM using a serial console and run the following commands 

    ILOM> set /HOST/boot_device=pxe

ILOM> reset /SYS

  7. The Secure Eraser utility will be automatically called to sanitize all installed storage media, including hard drives, flash devices, persistent memory, and internal USBs, and to reset ILOM to factory default for all nodes in parallel.

    Secure Eraser creates a file called secureeraser_node_chassis_number_date_time.certificate in the specified logpath location. node_chassis_number is the ID attribute of the storage server or database server in CellCLI or DBMCLI.

    The file contains a progress report that is updated every 10 seconds. The progress report is also output to the console on each node. The following is an example of the progress report:

    
ID Type   Model                          Serial Number     Size      Status
1  Flash  Flash Accel F640 PCIe Card v2  PHLN8BQ6P4EGN-1   2.91TB    To Be Erased (0%)
2  Flash  Flash Accel F640 PCIe Card v2  PHLN8BQ6P4EGN-2   2.91TB    To Be Erased (0%)
3  Flash  Flash Accel F640 PCIe Card v2  PHLN8BL6P4EGN-2   2.91TB    To Be Erased (0%)
4  Flash  Flash Accel F640 PCIe Card v2  PHLN8AX6P4EGN-1   2.91TB    To Be Erased (0%)
5  Flash  Flash Accel F640 PCIe Card v2  PHLN8AX6P4EGN-2   2.91TB    To Be Erased (0%)
6  Flash  Flash Accel F640 PCIe Card v2  PHLN88S6P4EGN-1   2.91TB    To Be Erased (0%)
7  Flash  Flash Accel F640 PCIe Card v2  PHLN8DQ6P4EGN-1   2.91TB    To Be Erased (0%)
8  Flash  Flash Accel F640 PCIe Card v2  PHLN88S6P4EGN-2   2.91TB    To Be Erased (0%)
9  Flash  Flash Accel F640 PCIe Card v2  PHLN88G6P4EGN-1   2.91TB    To Be Erased (0%)
10 Flash  Flash Accel F640 PCIe Card v2  PHLN8BL6P4EGN-1   2.91TB    To Be Erased (0%)
11 Flash  Flash Accel F640 PCIe Card v2  PHLN88W6P4EGN-2   2.91TB    To Be Erased (0%)
12 Flash  Flash Accel F640 PCIe Card v2  PHLN88W6P4EGN-1   2.91TB    To Be Erased (0%)
13 Flash  Flash Accel F640 PCIe Card v2  PHLN89F6P4EGN-2   2.91TB    To Be Erased (0%)
14 Flash  Flash Accel F640 PCIe Card v2  PHLN8DQ6P4EGN-2   2.91TB    To Be Erased (0%)
15 Flash  Flash Accel F640 PCIe Card v2  PHLN89F6P4EGN-1   2.91TB    To Be Erased (0%)
16 Flash  Flash Accel F640 PCIe Card v2  PHLN88G6P4EGN-2   2.91TB    To Be Erased (0%)
17 M.2    INTEL SSDSCKKB24               PHYH88H240J       139.69GB  To Be Erased (0%)
18 M.2    INTEL SSDSCKKB24               PHYH84060035240J  139.69GB  To Be Erased (0%)
19 PM     NMA1XBD128GQS                  8089-a2-0000028a  126.37GB  To Be Erased (0%)
20 PM     NMA1XBD128GQS                  8089-a2-000002f4  126.37GB  To Be Erased (0%)
21 PM     NMA1XBD128GQS                  8089-a2-000009d9  126.37GB  To Be Erased (0%)
22 PM     NMA1XBD128GQS                  8089-a2-00000a27  126.37GB  To Be Erased (0%)
23 PM     NMA1XBD128GQS                  8089-a2-00000231  126.37GB  To Be Erased (0%)
24 PM     NMA1XBD128GQS                  8089-a2-0000039e  126.37GB  To Be Erased (0%)
25 PM     NMA1XBD128GQS                  8089-a2-000006be  126.37GB  To Be Erased (0%)
26 PM     NMA1XBD128GQS                  8089-a2-00000916  126.37GB  To Be Erased (0%)
27 PM     NMA1XBD128GQS                  8089-a2-00000105  126.37GB  To Be Erased (0%)
28 PM     NMA1XBD128GQS                  8089-a2-00000216  126.37GB  Being Erased (0%)
29 PM     NMA1XBD128GQS                  8089-a2-00000151  126.37GB  Being Erased (0%)
30 PM     NMA1XBD128GQS                  8089-a2-000002f5  126.37GB  To Be Erased (0%)
31 ILOM                                  1824XCA004                  To Be Reset

    As the sample progress report shows, Secure Eraser erases all storage devices in parallel. After the storage devices are securely erased, Secure Eraser will reset the ILOM to the factory default. This is to ensure that in the case that secure erasure fails on some storage device, the web console is still accessible for remote debugging, and ILOM is still accessible to control the host.

    Once secure erasure is complete, a certificate called secureeraser_node_chassis_number_date_time.certificate.pdf is generated at the NFS share location specified by the logpath option in step 4. If secure erasure is successful, the nodes will be shut down automatically. If Secure Eraser does not succeed on some components, then the node will be left in diagnostic shell for further debugging. Assuming all previous steps are successful, and you have resolved the issue, you can go back to step 6 and rerun Secure Eraser.

Related Topics

Parent topic: Automatic Secure Eraser through PXE Boot