1. Security Guide for Oracle Exadata
  2. Securely Erasing Oracle Exadata
  3. Automatic Secure Eraser through PXE Boot
  4. Automatic Secure Eraser through PXE Boot for X6 and Earlier Systems

5.3.2 Automatic Secure Eraser through PXE Boot for X6 and Earlier Systems

In this procedure, you configure Secure Eraser to run automatically when you reboot the nodes.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Before you begin:

  • Download and install the latest Secure Eraser package available for your system.

    To download the Secure Eraser package, examine the Supplemental Readme associated with your current Oracle Exadata System Software version. To find the Supplemental Readme associated with each Oracle Exadata System Software version, see Exadata Database Machine and Exadata Storage Server Supported Versions (My Oracle Support Doc ID 888828.1).

  • Make sure you have access to a Preboot Execution Environment (PXE) server where the nodes to be erased can boot from.

  • Make sure you have access to a NFS server that is accessible from all the nodes to be erased.

  • Make sure you have access to one of the nodes to be erased.

  1. Copy the PXE image files initrd (initrd-<version>) and kernel (vmlinux-<version>) from the Secure Eraser package to the /tftpboot directory on the PXE server.

  2. Create a file containing the names of the database servers and storage servers you want to erase.

    To generate this file, you can run the following command from one of the nodes to be erased, and verify the nodes in the files are the ones to be erased.

    # ibhosts | awk '/S [0-9\.\,]*/ || /C [0-9\.\,]*/ {print $6}' | 
sed "s/\"//g" > nodes_to_be_erased

    If you only want to erase one server, enter the name of the server into the nodes_to_be_erased file, for example Exa01celadm04.

  3. Copy the dcli utility from the Secure Eraser package and the nodes_to_be_erased file generated in step 2 to the PXE server.

  4. Create a PXE configuration template called pxe_cfg.template to contain the following lines:

    • For releases earlier than Oracle Exadata System Software release 19.1.0: 

      default linux
label linux
kernel vmlinux-nfs-12.2.1.1.0-161015-cell
append initrd=initrd-nfs-12.2.1.1.0-161015-cell.img dhcp pxe quiet loglevel=
0 secureeraser bootarea=diagnostics console=tty1 console=ttyS0,115200n8 logp
ath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir

    • For Oracle Exadata System Software release 19.1.0 or later: 

      default linux
label linux
kernel vmlinux-iso-19.1.2.0.0-190111-cell
append initrd=initrd-iso-19.1.2.0.0-190111-cell.img dhcp pxe quiet loglevel=
0 secureeraser bootarea=diagnostics console=tty1 console=ttyS0,115200n8 logp
ath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir

    • The first line (default) indicates that the default label to use is called linux.

    • The second line (label) defines the linux label.

    • The third line (kernel) identifies the kernel file to load. In this case it is the file copied over in step 1.

    • The fourth line (append) adds more options to the kernel command line. The append statement must be on a single line in the configuration file.

      • The initrd option specifies the initrd file to load. In this case it is the initrd file copied over in step 1.

      • The dhcp option specifies to use DHCP to discover the eth0 interface.

      • The pxe option suppresses search for the image on virtual CD and USB devices.

      • The quiet option disables excessive kernel log messages.

      • The loglevel=0 option suppresses non-critical kernel messages.

      • The secureeraser option indicates PXE boot will automatically trigger the Secure Eraser utility to sanitize all media including hard drives, flash devices, internal USBs, and ILOM on the node.

      • The bootarea option indicates that the boot mode is diagnostic and not imaging install or rescue.

      • The console options indicate standard output and standard error messages are printed to both ILOM web console and serial console.

      • The logpath option specifies the NFS share directory where Secure Eraser will save the certificate.

    By default, the examples shown above cause Secure Eraser to erase all components. You can use secureeraser-options to specify command-line options for Secure Eraser to change the default behavior and securely erase certain components only. For example, to erase hard drives and USBs only during the PXE boot, the template would look like this:

    • For releases earlier than Oracle Exadata System Software release 19.1.0: 

      default linux
label linux
kernel vmlinux-nfs-12.2.1.1.0-161015-cell
append initrd=initrd-nfs-12.2.1.1.0-161015-cell.img dhcp pxe quiet loglevel=0 
secureeraser secureeraser-options="--hdd --usb" bootarea=diagnostics 
console=tty1 console=ttyS0,115200n8 logpath=10.133.42.221:/export/exadata_
secure_eraser_certificate_dir

    • For Oracle Exadata System Software release 19.1.0 or later: 

      default linux
label linux
kernel vmlinux-iso-19.1.2.0.0-190111-cell
append initrd=initrd-iso-19.1.2.0.0-190111-cell.img dhcp pxe quiet loglevel=0 
secureeraser secureeraser-options="--hdd --usb" bootarea=diagnostics 
console=tty1 console=ttyS0,115200n8 logpath=10.133.42.221:/export/exadata_
secure_eraser_certificate_dir

  5. On the PXE server, use the template file to generate a PXE configuration file in the /tftpboot/pxelinux.cfg/ directory for each of the nodes to be erased.

    The PXE configuration file name is the dash-separated MAC address of the node with the prefix 01-.

    If the nodes to be erased are accessible, use the following steps to automatically generate a PXE configuration file for each node based on the template.

    1. Set up SSH equivalence with the nodes to be erased from the PXE server. The command will prompt for the root password of each node. 

      pxe_server# dcli -g nodes_to_be_erased -k -l root

    2. Create PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# dcli -g nodes_to_be_erased -l root "ip addr show eth0" | 
awk '/link\/ether/ {print "01:"$3}' | sed "s/:/-/g" | 
xargs -I {} cp pxe_cfg.template {}

    If the nodes are not accessible, use the following step to generate a PXE configuration file for each node to be erased:

    1. Manually collect the MAC address of the eth0 interface from each node and write them into a text file called mac_addresses. Write one MAC address per line. For example: 

      00:10:e0:62:c4:fa
00:10:e0:62:c2:8a
00:10:e0:62:b8:7c
00:10:e0:62:b8:3a
00:10:e0:62:c6:bc

    2. Use the following command to create a list of PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# cat mac_addresses | sed "s/:/-/g;s/^/01-/g" | 
xargs -I {} cp pxe_cfg.template {}

    In both cases, you should get a list of PXE configuration files, one for each node to be erased. For example, if the MAC addresses of the nodes in a quarter rack are 00:10:e0:62:c4:fa, 00:10:e0:62:c2:8a, 00:10:e0:62:b8:7c, 00:10:e0:62:b8:3a, and 00:10:e0:62:c6:bc, you should get the following files:

    01-00-10-e0-62-c4-fa
01-00-10-e0-62-c2-8a
01-00-10-e0-62-b8-7c
01-00-10-e0-62-b8-3a
01-00-10-e0-62-c6-bc

    The files have the same content as the configuration template.

    Check your specific PXE server requirements. Your PXE server may need slightly different names or settings.

  6. Configure the nodes to boot from PXE and reboot the nodes.

    If the nodes to be erased are accessible, run the following commands:

    pxe_server# dcli -g nodes_to_be_erased -l root "ipmitool chassis bootdev pxe”

pxe_server# dcli -g nodes_to_be_erased -l root "reboot”

    If the nodes to be erased are not remotely accessible but the ILOMs are, use the following steps

    1. Create a file called iloms_to_be_reset containing the names of ILOMs. For example: 

      db1-ilom
db2-ilom
cell1-ilom
cell2-ilom
cell3-ilom

    2. Configure the nodes to boot from PXE through ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis bootdev pxe

    3. Reboot the nodes from ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis power cycle

    If neither host nor ILOM is remotely accessible, log into ILOM using a serial console and run the following commands 

    ILOM> set /HOST/boot_device=pxe

ILOM> reset /SYS

  7. The Secure Eraser utility will be automatically called to sanitize all storage media including hard drives, flash devices, and internal USBs, and to reset ILOM to factory default for all nodes in parallel.

    Secure Eraser creates a file called secureeraser_node_chassis_number_date_time.certificate in the specified logpath location. node_chassis_number is the ID attribute of the storage server or database server in CellCLI or DBMCLI.

    The file contains a progress report that is updated every 10 seconds. The progress report is also output to the console on each node. The following is an example of the progress report:

    ID  Type   Model                            Serial Number     Size      Status
1   Flash  Flash Accelerator F80 PCIe Card  FL00A96H          200.00GB  Being Erased (6%)
2   Flash  Flash Accelerator F80 PCIe Card  FL00A84Y          200.00GB  Being Erased (5%)
3   Flash  Flash Accelerator F80 PCIe Card  FL00A7D4          200.00GB  Being Erased (5%)
4   Flash  Flash Accelerator F80 PCIe Card  FL00A6WG          200.00GB  Being Erased (6%)
5   Flash  Flash Accelerator F80 PCIe Card  FL008KSE          200.00GB  Being Erased (5%)
6   Flash  Flash Accelerator F80 PCIe Card  FL008KS3          200.00GB  Being Erased (5%)
7   Flash  Flash Accelerator F80 PCIe Card  FL008KL7          200.00GB  Being Erased (5%)
8   Flash  Flash Accelerator F80 PCIe Card  FL008KQR          200.00GB  Being Erased (6%)
9   Flash  Flash Accelerator F80 PCIe Card  FL00A812          200.00GB  Being Erased (5%)
10  Flash  Flash Accelerator F80 PCIe Card  FL00A79G          200.00GB  Being Erased (5%)
11  Flash  Flash Accelerator F80 PCIe Card  FL00A80C          200.00GB  Being Erased (6%)
12  Flash  Flash Accelerator F80 PCIe Card  FL00A79F          200.00GB  Being Erased (6%)
13  Flash  Flash Accelerator F80 PCIe Card  FL00A5WD          200.00GB  Being Erased (5%)
14  Flash  Flash Accelerator F80 PCIe Card  FL00A5XS          200.00GB  Being Erased (5%)
15  Flash  Flash Accelerator F80 PCIe Card  FL00A7N1          200.00GB  Being Erased (5%)
16  Flash  Flash Accelerator F80 PCIe Card  FL00A62G          200.00GB  Being Erased (5%)
17  Disk   H7240AS60SUN4.0T                 1352E5XHWX        4.00TB    Being Erased (1%)
18  Disk   H7240AS60SUN4.0T                 1352E60SYX        4.00TB    Being Erased (1%)
19  Disk   H7240AS60SUN4.0T                 1352E60U4X        4.00TB    Being Erased (1%)
20  Disk   H7240AS60SUN4.0T                 1352E5UPAX        4.00TB    Being Erased (1%)
21  Disk   H7240AS60SUN4.0T                 1352E5XK3X        4.00TB    Being Erased (1%)
22  Disk   H7240AS60SUN4.0T                 1352E62M7X        4.00TB    Being Erased (1%)
23  Disk   H7240AS60SUN4.0T                 1352E5PSPX        4.00TB    Being Erased (1%)
24  Disk   H7240AS60SUN4.0T                 1352E60TJX        4.00TB    Being Erased (1%)
25  Disk   H7240AS60SUN4.0T                 1352E5LYDX        4.00TB    Being Erased (1%)
26  Disk   H7240AS60SUN4.0T                 1352E602WX        4.00TB    Being Erased (1%)
27  Disk   H7240AS60SUN4.0T                 1352E5LY9X        4.00TB    Being Erased (1%)
28  Disk   H7240AS60SUN4.0T                 1352E5VX4X        4.00TB    Being Erased (1%)
29  USB    SSM                              1900638EA8BFB749  8.00GB    Being Erased (5%)
30  ILOM                                    1403NM50CA                        To Be Reset

    As the sample progress report shows, Secure Eraser erases all storage devices in parallel. After the storage devices are securely erased, Secure Eraser will reset the ILOM to the factory default. This is to ensure that in the case that secure erasure fails on some storage device, the web console is still accessible for remote debugging, and ILOM is still accessible to control the host.

    Once secure erasure is complete, a certificate called secureeraser_node_chassis_number_date_time.certificate.pdf is generated at the NFS share location specified by the logpath option in step 4. If secure erasure is successful, the nodes will be shut down automatically. If Secure Eraser does not succeed on some components, then the node will be left in diagnostic shell for further debugging. Assuming all previous steps are successful, and you have resolved the issue, you can go back to step 6 and rerun Secure Eraser.

Related Topics

Parent topic: Automatic Secure Eraser through PXE Boot