1. Security Guide for Oracle Exadata
  2. Securely Erasing Oracle Exadata

5 Securely Erasing Oracle Exadata

If you are repurposing or removing an Oracle Exadata from your environment, it is critical to securely erase all the information on the servers.

Starting with Oracle Exadata System Software release 19.1.0, Secure Eraser is automatically started during re-imaging if the hardware supports Secure Eraser. This significantly simplifies the re-imaging procedure while maintaining performance. Now, when re-purposing a rack, you only have to image the rack and the secure data erasure is taken care of transparently as part of the process.

5.1 Overview of Secure Eraser

Oracle Exadata System Software release 12.2.1.1.0 or later provides a secure erasure solution, called Secure Eraser, for every component within Oracle Exadata.

Oracle Exadata consists of the following components:

  • Oracle Exadata Database Servers

  • Oracle Exadata Storage Servers

  • InfiniBand Network Fabric switches or RoCE Network Fabric switches

  • Ethernet switches

  • Power distribution units

Secure Eraser is a comprehensive solution that covers all Oracle Exadatas V2 or higher, including both 2-socket and 8-socket servers. The solution securely erases all data on both database servers and storage servers, and resets the internal network switches, the Ethernet switches, and the power distribution units back to factory default.

To achieve the best possible performance, secure erasure is performed in parallel at every layer on an Oracle Exadata. All Oracle Exadata Database Servers and Oracle Exadata Storage Servers are securely erased in parallel. Within a server, all device types (such as hard drives, flash devices, persistent memory and internal USBs) are securely erased in parallel. For each device type, all devices are further securely erased in parallel. This means that the total time to securely erase an entire rack is the same regardless of whether it's a quarter, half, or full rack, and that the total time should be approximately the time it takes to erase whichever component takes the longest time.

Secure Eraser automatically detects the hardware capability of a storage device and picks the best erasure method supported by the device. Cryptographic erasure is used whenever possible to provide better security and faster speed. The cryptographic erasure method used by Secure Eraser is fully compliant with the NIST SP-800-88r1 standard.

Secure Eraser comes with flexible options. The entire process can be completely automated without any user intervention. Or, you can choose to do it interactively and choose to erase specific types of storage devices.

Secure Eraser periodically generates a progress report every 10 seconds so that you can easily monitor the progress.

When the secure erasure is completed, a certificate is generated for each server with a list of devices that have been securely erased. The following figure shows a sample certificate from Secure Eraser.

Figure 5-1 Sample Certificate from Secure Eraser

Description of Figure 5-1 follows
Description of "Figure 5-1 Sample Certificate from Secure Eraser"

Parent topic: Securely Erasing Oracle Exadata

5.2 Securely Erasing Database Servers and Storage Servers

Oracle Exadata System Software 12.2.1.1.0 or later comes with a utility called Secure Eraser which securely erases data on hard drives, flash devices, persistent memory, and internal USBs. It also resets ILOM to factory settings.

In earlier versions of Exadata, you can securely erase user data through CellCLI commands such as DROP CELL ERASE, DROP CELLDISK ERASE, or DROP GRIDDISK ERASE. These DROP commands only cover user data on hard drives and flash devices. Secure Eraser, on the other hand, sanitizes all content, not only user data but also operating system, Oracle Exadata System Software, and user configurations. In addition, it covers a wider range of hardware components including hard drives, flash devices, persistent memory, internal USBs, and ILOMs.

Caution:

The server will become unbootable after the system devices are securely erased, and ILOM will no longer be remotely accessible after being reset to factory default. ILOM will remain accessible through serial console.

The Secure Eraser utility works on both database servers and storage servers and covers all Oracle Exadatas V2 or higher.

Based on hardware capabilities, different secure erasure methods are applied. In general, Secure Eraser has two types of erasure methods: 3-pass erase and crypto erase. The 3-pass erase method overwrites all addressable locations with a character, its complement, then a random character, and finally verifies the results. The crypto erase method erases all user data present on instant secure erase (ISE) devices by deleting the encryption keys with which the user data was previously encrypted.

Refer to the table "Estimated Erasure Times for Disks by Erasure Method" in the topic DROP CELL for a summary of the secure erasure methods used and their approximate time. Note that the time for 3-pass erase varies from drives to drives based on their size and speed. It is approximately equal to the time required to overwrite the entire device three times and read it one more time. Hard drives, flash devices, persistent memory, and internal USBs are securely erased in parallel: the time required to erase one device is the same as that required for erasing multiple devices of the same kind.

Parent topic: Securely Erasing Oracle Exadata

5.3 Automatic Secure Eraser through PXE Boot

In this procedure, you configure Secure Eraser to run automatically when you reboot the nodes.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Use one of the following procedures, depending on your system:

Related Topics

Parent topic: Securely Erasing Oracle Exadata

5.3.1 Automatic Secure Eraser through PXE Boot for X7 and Later Systems

In this procedure, you configure Secure Eraser to run automatically when you reboot Oracle Exadata X7-2 and later nodes.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Before you begin:

  • Download the Secure Eraser package. Refer to the Supplemental Readme for your currently installed Oracle Exadata System Software image version to find the correct Secure Eraser patch.

  • Make sure you have access to a Preboot Execution Environment (PXE) server where the nodes to be erased can boot from.

  • Make sure you have access to a NFS server that is accessible from all the nodes to be erased.

  • Make sure you have access to one of the nodes to be erased.

  1. Copy the PXE image files initrd (initrd-<version>) and kernel (vmlinux-<version>) from the Secure Eraser package to the /tftpboot directory on the PXE server.

  2. Create a file containing the names of the database servers and storage servers you want to erase.

    To generate this file, you can run the following command from one of the nodes to be erased, and verify the nodes in the files are the ones to be erased.

    # ibhosts | awk '/S [0-9\.\,]*/ || /C [0-9\.\,]*/ {print $6}' | 
sed "s/\"//g" > nodes_to_be_erased

    If you only want to erase one server, enter the name of the server into the nodes_to_be_erased file, for example Exa01celadm04.

  3. Copy the dcli utility from the Secure Eraser package and the nodes_to_be_erased file generated in step 2 to the PXE server.

  4. Create a PXE configuration template called pxe_cfg.template to contain the following lines:

    Note:

    In the following example, the following parameters must be updated to match your environment:

    • kernel (the vmlinux file)
    • initrd (the initrd*.img file)
    • logpath

    • For Oracle Exadata System Software 18c (18.1.0): 

      set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-nfs-18.1.0.0.0-170915.1 dhcp pxe boot-from=uefi 
quiet loglevel=0 secureeraser bootarea=diagnostics console=ttyS0,115200n8 
logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-nfs-18.1.0.0.0-170915.1.img 
  echo "Booting installation kernel"
}

    • For Oracle Exadata System Software release 19.1.0 or later: 

      set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-iso-19.1.2.0.0-190111 dhcp pxe boot-from=uefi 
quiet loglevel=0 secureeraser bootarea=diagnostics console=ttyS0,115200n8 
logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-iso-19.1.2.0.0-190111.img 
  echo "Booting installation kernel"
}

    • The first line (default) identifies a menu entry that should be selected by default, after the timeout value specified by the second line.

    • The third line (menuentry) represents the Linux kernel that will be used in the Secure Eraser environment.

    • The fifth line (linuxefi) indicates the kernel is on an UEFI-based system. The linuxefi statement must be on a single line in the configuration file.

      • The dhcp option specifies to use DHCP to discover the eth0 interface.

      • The pxe option suppresses search for the image on virtual CD and USB devices.

      • The boot-from=uefi option indicates the system is booting from UEFI.

      • The quiet option disables excessive kernel log messages.

      • The loglevel=0 option suppresses non-critical kernel messages.

      • The secureeraser option indicates PXE boot will automatically trigger the Secure Eraser utility to sanitize all media installed on the node, including hard drives, flash devices, persistent memory, internal USBs, and ILOM.

      • The bootarea option indicates that the boot mode is diagnostic and not imaging install or rescue.

      • The console options indicate standard output and standard error messages are printed to both ILOM web console and serial console.

      • The logpath option specifies the NFS share directory where Secure Eraser will save the certificate.

    • The seventh line (initrdefi) specifies the initrd file to load. In this case it is the initrd file copied over in step 1.

    By default, the examples shown above cause Secure Eraser to erase all components. You can use secureeraser-options to specify command-line options for Secure Eraser to change the default behavior and securely erase certain components only. For example, to erase hard drives and USBs only during the PXE boot, the template would look like this for grub2 / Secure Boot on Oracle Exadata Database Machine X7 and later systems:

    Note:

    In the following example, the following parameters must be updated to match your environment:

    • kernel (the vmlinux file)
    • initrd (the initrd*img file)
    • logpath

    • For Oracle Exadata System Software 18c (18.1.0): 

      set default 0 
set timeout=10
menuentry 'ExadataLinux' {
   echo "Loading efi/vmlinuz" 
   linuxefi efi/vmlinux-nfs-18.1.0.0.0-170915.1 stit dhcp pxe boot-from=
uefi quiet loglevel=0 secureeraser secureeraser-options="--hdd --usb" 
bootarea=diagnostics console=ttyS0,115200n8 logpath=10.133.42
.221:/export/exadata_secure_eraser_certificate_dir
   echo "Loading efi/initrd.img"
  initrdefi efi/initrd-nfs-18.1.0.0.0-170915.1.img
   echo "Booting installation kernel"
}

    • For Oracle Exadata System Software release 19.1.0 or later: 

      set default 0 
set timeout=10
menuentry 'ExadataLinux' {
   echo "Loading efi/vmlinuz" 
   linuxefi efi/vmlinux-iso-19.1.2.0.0-190111 stit dhcp pxe boot-from=
uefi quiet loglevel=0 secureeraser secureeraser-options="--hdd --usb" 
bootarea=diagnostics console=ttyS0,115200n8 logpath=10.133.42
.221:/export/exadata_secure_eraser_certificate_dir
   echo "Loading efi/initrd.img"
  initrdefi efi/initrd-iso-19.1.2.0.0-190111.img
   echo "Booting installation kernel"
}

  5. On the PXE server, use the template file to generate a PXE configuration file in the /tftpboot/pxelinux.cfg/ directory for each of the nodes to be erased.

    The PXE configuration file name is the dash-separated MAC address of the node with the prefix 01-.

    If the nodes to be erased are accessible, use the following steps to automatically generate a PXE configuration file for each node based on the template.

    1. Set up SSH equivalence with the nodes to be erased from the PXE server. The command will prompt for the root password of each node. 

      pxe_server# dcli -g nodes_to_be_erased -k -l root

    2. Create PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# dcli -g nodes_to_be_erased -l root "ip addr show eth0" | 
awk '/link\/ether/ {print "01:"$3}' | sed "s/:/-/g" | 
xargs -I {} cp pxe_cfg.template {}

    If the nodes are not accessible, use the following step to generate a PXE configuration file for each node to be erased:

    1. Manually collect the MAC address of the eth0 interface from each node and write them into a text file called mac_addresses. Write one MAC address per line. For example: 

      00:10:e0:62:c4:fa
00:10:e0:62:c2:8a
00:10:e0:62:b8:7c
00:10:e0:62:b8:3a
00:10:e0:62:c6:bc

    2. Use the following command to create a list of PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# cat mac_addresses | sed "s/:/-/g;s/^/01-/g" | 
xargs -I {} cp pxe_cfg.template {}

    In both cases, you should get a list of PXE configuration files, one for each node to be erased. For example, if the MAC addresses of the nodes in a quarter rack are 00:10:e0:62:c4:fa, 00:10:e0:62:c2:8a, 00:10:e0:62:b8:7c, 00:10:e0:62:b8:3a, and 00:10:e0:62:c6:bc, you should get the following files:

    01-00-10-e0-62-c4-fa
01-00-10-e0-62-c2-8a
01-00-10-e0-62-b8-7c
01-00-10-e0-62-b8-3a
01-00-10-e0-62-c6-bc

    The files have the same content as the configuration template.

    Check your specific PXE server requirements. Your PXE server may need slightly different names or settings.

  6. Configure the nodes to boot from PXE and reboot the nodes.

    If the nodes to be erased are accessible, run the following commands:

    pxe_server# dcli -g nodes_to_be_erased -l root "ipmitool chassis bootdev pxe”

pxe_server# dcli -g nodes_to_be_erased -l root "reboot”

    If the nodes to be erased are not remotely accessible but the ILOMs are, use the following steps

    1. Create a file called iloms_to_be_reset containing the names of ILOMs. For example: 

      db1-ilom
db2-ilom
cell1-ilom
cell2-ilom
cell3-ilom

    2. Configure the nodes to boot from PXE through ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis bootdev pxe

    3. Reboot the nodes from ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis power cycle

    If neither host nor ILOM is remotely accessible, log into ILOM using a serial console and run the following commands 

    ILOM> set /HOST/boot_device=pxe

ILOM> reset /SYS

  7. The Secure Eraser utility will be automatically called to sanitize all installed storage media, including hard drives, flash devices, persistent memory, and internal USBs, and to reset ILOM to factory default for all nodes in parallel.

    Secure Eraser creates a file called secureeraser_node_chassis_number_date_time.certificate in the specified logpath location. node_chassis_number is the ID attribute of the storage server or database server in CellCLI or DBMCLI.

    The file contains a progress report that is updated every 10 seconds. The progress report is also output to the console on each node. The following is an example of the progress report:

    
ID Type   Model                          Serial Number     Size      Status
1  Flash  Flash Accel F640 PCIe Card v2  PHLN8BQ6P4EGN-1   2.91TB    To Be Erased (0%)
2  Flash  Flash Accel F640 PCIe Card v2  PHLN8BQ6P4EGN-2   2.91TB    To Be Erased (0%)
3  Flash  Flash Accel F640 PCIe Card v2  PHLN8BL6P4EGN-2   2.91TB    To Be Erased (0%)
4  Flash  Flash Accel F640 PCIe Card v2  PHLN8AX6P4EGN-1   2.91TB    To Be Erased (0%)
5  Flash  Flash Accel F640 PCIe Card v2  PHLN8AX6P4EGN-2   2.91TB    To Be Erased (0%)
6  Flash  Flash Accel F640 PCIe Card v2  PHLN88S6P4EGN-1   2.91TB    To Be Erased (0%)
7  Flash  Flash Accel F640 PCIe Card v2  PHLN8DQ6P4EGN-1   2.91TB    To Be Erased (0%)
8  Flash  Flash Accel F640 PCIe Card v2  PHLN88S6P4EGN-2   2.91TB    To Be Erased (0%)
9  Flash  Flash Accel F640 PCIe Card v2  PHLN88G6P4EGN-1   2.91TB    To Be Erased (0%)
10 Flash  Flash Accel F640 PCIe Card v2  PHLN8BL6P4EGN-1   2.91TB    To Be Erased (0%)
11 Flash  Flash Accel F640 PCIe Card v2  PHLN88W6P4EGN-2   2.91TB    To Be Erased (0%)
12 Flash  Flash Accel F640 PCIe Card v2  PHLN88W6P4EGN-1   2.91TB    To Be Erased (0%)
13 Flash  Flash Accel F640 PCIe Card v2  PHLN89F6P4EGN-2   2.91TB    To Be Erased (0%)
14 Flash  Flash Accel F640 PCIe Card v2  PHLN8DQ6P4EGN-2   2.91TB    To Be Erased (0%)
15 Flash  Flash Accel F640 PCIe Card v2  PHLN89F6P4EGN-1   2.91TB    To Be Erased (0%)
16 Flash  Flash Accel F640 PCIe Card v2  PHLN88G6P4EGN-2   2.91TB    To Be Erased (0%)
17 M.2    INTEL SSDSCKKB24               PHYH88H240J       139.69GB  To Be Erased (0%)
18 M.2    INTEL SSDSCKKB24               PHYH84060035240J  139.69GB  To Be Erased (0%)
19 PM     NMA1XBD128GQS                  8089-a2-0000028a  126.37GB  To Be Erased (0%)
20 PM     NMA1XBD128GQS                  8089-a2-000002f4  126.37GB  To Be Erased (0%)
21 PM     NMA1XBD128GQS                  8089-a2-000009d9  126.37GB  To Be Erased (0%)
22 PM     NMA1XBD128GQS                  8089-a2-00000a27  126.37GB  To Be Erased (0%)
23 PM     NMA1XBD128GQS                  8089-a2-00000231  126.37GB  To Be Erased (0%)
24 PM     NMA1XBD128GQS                  8089-a2-0000039e  126.37GB  To Be Erased (0%)
25 PM     NMA1XBD128GQS                  8089-a2-000006be  126.37GB  To Be Erased (0%)
26 PM     NMA1XBD128GQS                  8089-a2-00000916  126.37GB  To Be Erased (0%)
27 PM     NMA1XBD128GQS                  8089-a2-00000105  126.37GB  To Be Erased (0%)
28 PM     NMA1XBD128GQS                  8089-a2-00000216  126.37GB  Being Erased (0%)
29 PM     NMA1XBD128GQS                  8089-a2-00000151  126.37GB  Being Erased (0%)
30 PM     NMA1XBD128GQS                  8089-a2-000002f5  126.37GB  To Be Erased (0%)
31 ILOM                                  1824XCA004                  To Be Reset

    As the sample progress report shows, Secure Eraser erases all storage devices in parallel. After the storage devices are securely erased, Secure Eraser will reset the ILOM to the factory default. This is to ensure that in the case that secure erasure fails on some storage device, the web console is still accessible for remote debugging, and ILOM is still accessible to control the host.

    Once secure erasure is complete, a certificate called secureeraser_node_chassis_number_date_time.certificate.pdf is generated at the NFS share location specified by the logpath option in step 4. If secure erasure is successful, the nodes will be shut down automatically. If Secure Eraser does not succeed on some components, then the node will be left in diagnostic shell for further debugging. Assuming all previous steps are successful, and you have resolved the issue, you can go back to step 6 and rerun Secure Eraser.

Related Topics

Parent topic: Automatic Secure Eraser through PXE Boot

5.3.2 Automatic Secure Eraser through PXE Boot for X6 and Earlier Systems

In this procedure, you configure Secure Eraser to run automatically when you reboot the nodes.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Before you begin:

  • Download the Secure Eraser package. Refer to the Supplemental Readme for your currently installed Exadata image version to find the correct Secure Eraser patch.

  • Make sure you have access to a Preboot Execution Environment (PXE) server where the nodes to be erased can boot from.

  • Make sure you have access to a NFS server that is accessible from all the nodes to be erased.

  • Make sure you have access to one of the nodes to be erased.

  1. Copy the PXE image files initrd (initrd-<version>) and kernel (vmlinux-<version>) from the Secure Eraser package to the /tftpboot directory on the PXE server.

  2. Create a file containing the names of the database servers and storage servers you want to erase.

    To generate this file, you can run the following command from one of the nodes to be erased, and verify the nodes in the files are the ones to be erased.

    # ibhosts | awk '/S [0-9\.\,]*/ || /C [0-9\.\,]*/ {print $6}' | 
sed "s/\"//g" > nodes_to_be_erased

    If you only want to erase one server, enter the name of the server into the nodes_to_be_erased file, for example Exa01celadm04.

  3. Copy the dcli utility from the Secure Eraser package and the nodes_to_be_erased file generated in step 2 to the PXE server.

  4. Create a PXE configuration template called pxe_cfg.template to contain the following lines:

    • For releases earlier than Oracle Exadata System Software release 19.1.0: 

      default linux
label linux
kernel vmlinux-nfs-12.2.1.1.0-161015-cell
append initrd=initrd-nfs-12.2.1.1.0-161015-cell.img dhcp pxe quiet loglevel=
0 secureeraser bootarea=diagnostics console=tty1 console=ttyS0,115200n8 logp
ath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir

    • For Oracle Exadata System Software release 19.1.0 or later: 

      default linux
label linux
kernel vmlinux-iso-19.1.2.0.0-190111-cell
append initrd=initrd-iso-19.1.2.0.0-190111-cell.img dhcp pxe quiet loglevel=
0 secureeraser bootarea=diagnostics console=tty1 console=ttyS0,115200n8 logp
ath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir

    • The first line (default) indicates that the default label to use is called linux.

    • The second line (label) defines the linux label.

    • The third line (kernel) identifies the kernel file to load. In this case it is the file copied over in step 1.

    • The fourth line (append) adds more options to the kernel command line. The append statement must be on a single line in the configuration file.

      • The initrd option specifies the initrd file to load. In this case it is the initrd file copied over in step 1.

      • The dhcp option specifies to use DHCP to discover the eth0 interface.

      • The pxe option suppresses search for the image on virtual CD and USB devices.

      • The quiet option disables excessive kernel log messages.

      • The loglevel=0 option suppresses non-critical kernel messages.

      • The secureeraser option indicates PXE boot will automatically trigger the Secure Eraser utility to sanitize all media including hard drives, flash devices, internal USBs, and ILOM on the node.

      • The bootarea option indicates that the boot mode is diagnostic and not imaging install or rescue.

      • The console options indicate standard output and standard error messages are printed to both ILOM web console and serial console.

      • The logpath option specifies the NFS share directory where Secure Eraser will save the certificate.

    By default, the examples shown above cause Secure Eraser to erase all components. You can use secureeraser-options to specify command-line options for Secure Eraser to change the default behavior and securely erase certain components only. For example, to erase hard drives and USBs only during the PXE boot, the template would look like this:

    • For releases earlier than Oracle Exadata System Software release 19.1.0: 

      default linux
label linux
kernel vmlinux-nfs-12.2.1.1.0-161015-cell
append initrd=initrd-nfs-12.2.1.1.0-161015-cell.img dhcp pxe quiet loglevel=0 
secureeraser secureeraser-options="--hdd --usb" bootarea=diagnostics 
console=tty1 console=ttyS0,115200n8 logpath=10.133.42.221:/export/exadata_
secure_eraser_certificate_dir

    • For Oracle Exadata System Software release 19.1.0 or later: 

      default linux
label linux
kernel vmlinux-iso-19.1.2.0.0-190111-cell
append initrd=initrd-iso-19.1.2.0.0-190111-cell.img dhcp pxe quiet loglevel=0 
secureeraser secureeraser-options="--hdd --usb" bootarea=diagnostics 
console=tty1 console=ttyS0,115200n8 logpath=10.133.42.221:/export/exadata_
secure_eraser_certificate_dir

  5. On the PXE server, use the template file to generate a PXE configuration file in the /tftpboot/pxelinux.cfg/ directory for each of the nodes to be erased.

    The PXE configuration file name is the dash-separated MAC address of the node with the prefix 01-.

    If the nodes to be erased are accessible, use the following steps to automatically generate a PXE configuration file for each node based on the template.

    1. Set up SSH equivalence with the nodes to be erased from the PXE server. The command will prompt for the root password of each node. 

      pxe_server# dcli -g nodes_to_be_erased -k -l root

    2. Create PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# dcli -g nodes_to_be_erased -l root "ip addr show eth0" | 
awk '/link\/ether/ {print "01:"$3}' | sed "s/:/-/g" | 
xargs -I {} cp pxe_cfg.template {}

    If the nodes are not accessible, use the following step to generate a PXE configuration file for each node to be erased:

    1. Manually collect the MAC address of the eth0 interface from each node and write them into a text file called mac_addresses. Write one MAC address per line. For example: 

      00:10:e0:62:c4:fa
00:10:e0:62:c2:8a
00:10:e0:62:b8:7c
00:10:e0:62:b8:3a
00:10:e0:62:c6:bc

    2. Use the following command to create a list of PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# cat mac_addresses | sed "s/:/-/g;s/^/01-/g" | 
xargs -I {} cp pxe_cfg.template {}

    In both cases, you should get a list of PXE configuration files, one for each node to be erased. For example, if the MAC addresses of the nodes in a quarter rack are 00:10:e0:62:c4:fa, 00:10:e0:62:c2:8a, 00:10:e0:62:b8:7c, 00:10:e0:62:b8:3a, and 00:10:e0:62:c6:bc, you should get the following files:

    01-00-10-e0-62-c4-fa
01-00-10-e0-62-c2-8a
01-00-10-e0-62-b8-7c
01-00-10-e0-62-b8-3a
01-00-10-e0-62-c6-bc

    The files have the same content as the configuration template.

    Check your specific PXE server requirements. Your PXE server may need slightly different names or settings.

  6. Configure the nodes to boot from PXE and reboot the nodes.

    If the nodes to be erased are accessible, run the following commands:

    pxe_server# dcli -g nodes_to_be_erased -l root "ipmitool chassis bootdev pxe”

pxe_server# dcli -g nodes_to_be_erased -l root "reboot”

    If the nodes to be erased are not remotely accessible but the ILOMs are, use the following steps

    1. Create a file called iloms_to_be_reset containing the names of ILOMs. For example: 

      db1-ilom
db2-ilom
cell1-ilom
cell2-ilom
cell3-ilom

    2. Configure the nodes to boot from PXE through ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis bootdev pxe

    3. Reboot the nodes from ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H 
{} -U root chassis power cycle

    If neither host nor ILOM is remotely accessible, log into ILOM using a serial console and run the following commands 

    ILOM> set /HOST/boot_device=pxe

ILOM> reset /SYS

  7. The Secure Eraser utility will be automatically called to sanitize all storage media including hard drives, flash devices, and internal USBs, and to reset ILOM to factory default for all nodes in parallel.

    Secure Eraser creates a file called secureeraser_node_chassis_number_date_time.certificate in the specified logpath location. node_chassis_number is the ID attribute of the storage server or database server in CellCLI or DBMCLI.

    The file contains a progress report that is updated every 10 seconds. The progress report is also output to the console on each node. The following is an example of the progress report:

    ID  Type   Model                            Serial Number     Size      Status
1   Flash  Flash Accelerator F80 PCIe Card  FL00A96H          200.00GB  Being Erased (6%)
2   Flash  Flash Accelerator F80 PCIe Card  FL00A84Y          200.00GB  Being Erased (5%)
3   Flash  Flash Accelerator F80 PCIe Card  FL00A7D4          200.00GB  Being Erased (5%)
4   Flash  Flash Accelerator F80 PCIe Card  FL00A6WG          200.00GB  Being Erased (6%)
5   Flash  Flash Accelerator F80 PCIe Card  FL008KSE          200.00GB  Being Erased (5%)
6   Flash  Flash Accelerator F80 PCIe Card  FL008KS3          200.00GB  Being Erased (5%)
7   Flash  Flash Accelerator F80 PCIe Card  FL008KL7          200.00GB  Being Erased (5%)
8   Flash  Flash Accelerator F80 PCIe Card  FL008KQR          200.00GB  Being Erased (6%)
9   Flash  Flash Accelerator F80 PCIe Card  FL00A812          200.00GB  Being Erased (5%)
10  Flash  Flash Accelerator F80 PCIe Card  FL00A79G          200.00GB  Being Erased (5%)
11  Flash  Flash Accelerator F80 PCIe Card  FL00A80C          200.00GB  Being Erased (6%)
12  Flash  Flash Accelerator F80 PCIe Card  FL00A79F          200.00GB  Being Erased (6%)
13  Flash  Flash Accelerator F80 PCIe Card  FL00A5WD          200.00GB  Being Erased (5%)
14  Flash  Flash Accelerator F80 PCIe Card  FL00A5XS          200.00GB  Being Erased (5%)
15  Flash  Flash Accelerator F80 PCIe Card  FL00A7N1          200.00GB  Being Erased (5%)
16  Flash  Flash Accelerator F80 PCIe Card  FL00A62G          200.00GB  Being Erased (5%)
17  Disk   H7240AS60SUN4.0T                 1352E5XHWX        4.00TB    Being Erased (1%)
18  Disk   H7240AS60SUN4.0T                 1352E60SYX        4.00TB    Being Erased (1%)
19  Disk   H7240AS60SUN4.0T                 1352E60U4X        4.00TB    Being Erased (1%)
20  Disk   H7240AS60SUN4.0T                 1352E5UPAX        4.00TB    Being Erased (1%)
21  Disk   H7240AS60SUN4.0T                 1352E5XK3X        4.00TB    Being Erased (1%)
22  Disk   H7240AS60SUN4.0T                 1352E62M7X        4.00TB    Being Erased (1%)
23  Disk   H7240AS60SUN4.0T                 1352E5PSPX        4.00TB    Being Erased (1%)
24  Disk   H7240AS60SUN4.0T                 1352E60TJX        4.00TB    Being Erased (1%)
25  Disk   H7240AS60SUN4.0T                 1352E5LYDX        4.00TB    Being Erased (1%)
26  Disk   H7240AS60SUN4.0T                 1352E602WX        4.00TB    Being Erased (1%)
27  Disk   H7240AS60SUN4.0T                 1352E5LY9X        4.00TB    Being Erased (1%)
28  Disk   H7240AS60SUN4.0T                 1352E5VX4X        4.00TB    Being Erased (1%)
29  USB    SSM                              1900638EA8BFB749  8.00GB    Being Erased (5%)
30  ILOM                                    1403NM50CA                        To Be Reset

    As the sample progress report shows, Secure Eraser erases all storage devices in parallel. After the storage devices are securely erased, Secure Eraser will reset the ILOM to the factory default. This is to ensure that in the case that secure erasure fails on some storage device, the web console is still accessible for remote debugging, and ILOM is still accessible to control the host.

    Once secure erasure is complete, a certificate called secureeraser_node_chassis_number_date_time.certificate.pdf is generated at the NFS share location specified by the logpath option in step 4. If secure erasure is successful, the nodes will be shut down automatically. If Secure Eraser does not succeed on some components, then the node will be left in diagnostic shell for further debugging. Assuming all previous steps are successful, and you have resolved the issue, you can go back to step 6 and rerun Secure Eraser.

Related Topics

Parent topic: Automatic Secure Eraser through PXE Boot

5.4 Interactive Secure Eraser through PXE Boot

On Exadata systems prior to Oracle Exadata X7-2, you can use Preboot Execution Environment (PXE) Boot when performing a Secure Eraser.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Before you begin:

  • Download the Secure Eraser package. Refer to the Supplemental Readme for your currently installed Oracle Exadata System Software image version to find the correct Secure Eraser patch.

  • Make sure you have access to a PXE server where the nodes to be erased can boot from.

  • Make sure you have access to a NFS server that is accessible from all the nodes to be erased.

  • Make sure you have access to one of the nodes to be erased.

  1. Copy the cell PXE image files initrd (initrd-version) and kernel (vmlinux-version) from the Secure Eraser package to the /tftpboot directory on the PXE server. For Oracle Exadata Database Machine X7 and later systems, the directory is /tftpboot/efi for grub2 and Secure Boot.

  2. Create a file containing the names of the database servers and storage servers to be erased.

    To generate this file, you can run the following command from one of the nodes to be erased, and verify the nodes in the files are the ones to be erased.

    # ibhosts | awk '/S [0-9\.\,]*/ || /C [0-9\.\,]*/ {print $6}' | sed "s/\"//g" 
> nodes_to_be_erased

    If you only want to erase one server, enter the name of the server into the nodes_to_be_erased file, for example Exa01celadm04.

  3. Copy the dcli utility from the Secure Eraser package along with the file generated in step 2 to the PXE server.

  4. Create a PXE configuration template called pxe_cfg.template to contain the following lines:

    • For all models prior to Oracle Exadata Database Machine X7-2 — grub:

      • For releases earlier than Oracle Exadata System Software release 19.1.0: 

        default linux
label linux
kernel vmlinux-nfs-12.2.1.1.0-161015-cell
append initrd=initrd-nfs-12.2.1.1.0-161015-cell.img dhcp pxe quiet loglevel=
0 secureeraser bootarea=diagnostics console=tty1 console=ttyS0,115200n8 logp
ath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir

      • For Oracle Exadata System Software release 19.1.0 or later: 

        default linux
label linux
kernel vmlinux-iso-19.1.2.0.0-190111-cell
append initrd=initrd-iso-19.1.2.0.0-190111-cell.img dhcp pxe quiet loglevel=
0 secureeraser bootarea=diagnostics console=tty1 console=ttyS0,115200n8 logp
ath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir

      For a description of each component of the above task, refer to the appropriate topic in Automatic Secure Eraser through PXE Boot for your system.

    • For Oracle Exadata Database Machine X7-2 and newer models —grub2 / Secure Boot:

      Note:

      In the following example, the following parameters must be updated to match your environment:

      • kernel (the vmlinux file)

      • initrd (the initrd*.img file)

      • logpath

      • For Oracle Exadata System Software 18c (18.1.0): 

        set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-nfs-18.1.0.0.0-170915.1 stit dhcp pxe boot-from=uefi 
quiet loglevel=0 bootarea=diagnostics console=ttyS0,115200n8 
logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-nfs-18.1.0.0.0-170915.1.img 
  echo "Booting installation kernel"
}

      • For Oracle Exadata System Software release 19.1.0 or later: 

        set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-iso-19.1.2.0.0-190111 stit dhcp pxe boot-from=uefi 
quiet loglevel=0 bootarea=diagnostics console=ttyS0,115200n8 
logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-iso-19.1.2.0.0-190111.img 
  echo "Booting installation kernel"
}

    For a description of each component of the task in this step, see Automatic Secure Eraser through PXE Boot.

    This configuration file differs from the one in Automatic Secure Eraser through PXE Boot in that the secureeraser option is left out to indicate that the Secure Eraser process should not be triggered automatically. The rest of the file is the same.

  5. On the PXE server, use the template file to generate a PXE configuration file. For all systems up to Oracle Exadata Database Machine X6-2, save the file in the /tftpboot/pxelinux.cfg/ directory for each of the nodes to be erased. For Oracle Exadata Database Machine X7-2 and newer systems, save the file in the /tftpboot/efi/pxelinux.cfg/ directory for each of the nodes to be erased.

    The PXE configuration file name is the dash-separated MAC address of the node with the prefix 01-.

    If the nodes to be erased are accessible, perform the following steps to automatically generate a PXE configuration file for each node based on the template:

    1. Set up SSH equivalence with the nodes to be erased from the PXE server. The command will prompt for the root password of each node. 

      pxe_server# dcli -g nodes_to_be_erased -k -l root

    2. Create a list of PXE configuration files, one for each node to be erased based on the configuration template.

      pxe_server# dcli -g nodes_to_be_erased -l root "ip addr show eth0" | 
awk '/link\/ether/ {print "01:"$3}' | sed "s/:/-/g" | xargs -I {} 
cp pxe_cfg.template {}

    If the nodes are not accessible, perform the following steps to generate a PXE configuration file for each node.

    1. Manually collect the MAC address of the eth0 interface from each node and write them into a text file called mac_addresses. Write one MAC address per line. For example: 

      00:10:e0:62:c4:fa
00:10:e0:62:c2:8a
00:10:e0:62:b8:7c
00:10:e0:62:b8:3a
00:10:e0:62:c6:bc

    2. Use the following command to create a list of PXE configuration file, one for each node to be erased based on the configuration template.

      pxe_server# cat mac_addresses | sed "s/:/-/g;s/^/01-/g" | xargs -I {} cp 
pxe_cfg.template {}

    In both cases, you should have a list of PXE configuration files, one for each node to be erased. For example, if the MAC addresses of the nodes in a quarter rack are 00:10:e0:62:c4:fa, 00:10:e0:62:c2:8a, 00:10:e0:62:b8:7c, 00:10:e0:62:b8:3a, and 00:10:e0:62:c6:bc, then you should get the following files:

    01-00-10-e0-62-c4-fa
01-00-10-e0-62-c2-8a
01-00-10-e0-62-b8-7c
01-00-10-e0-62-b8-3a
01-00-10-e0-62-c6-bc

    The files have the same content as the configuration template.

    Check your specific PXE server requirements. Your PXE server may need slightly different names or settings.

  6. Configure the nodes to boot from PXE and reboot the nodes.

    If the nodes to be erased are accessible, run the following commands:

    pxe_server# dcli -g nodes_to_be_erased -l root "ipmitool chassis bootdev pxe”

pxe_server# dcli -g nodes_to_be_erased -l root "reboot”

    If the nodes are not accessible, then perform the following steps:

    1. Create a file called iloms_to_be_reset containing the names of ILOMs. For example: 

      db1-ilom
db2-ilom
cell1-ilom
cell2-ilom
cell3-ilom

    2. Configure the nodes to boot from PXE through ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H {} 
-U root chassis bootdev pxe

    3. Reboot the nodes from ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H {} 
-U root chassis power cycle

  7. If you get the following prompt on the remote or serial console, enter e at the prompt to enter the diagnostic shell: 

    Choose from following by typing letter in '()':
(e)nter interactive diagnostics shell. Must use credentials from Oracle
support to login (reboot or power cycle to exit the shell),
Select:e

  8. If prompted, log in to the system as the root user.

    If you require the password for the root user and do not have it, then contact Oracle Support Services. 

    localhost login: root
Password: *********
-sh-3.1#

  9. Run the Secure Eraser utility to sanitize all devices or one type of device. 

    -sh-3.1# /usr/sbin/secureeraser -–erase --all --output=REMOTE_NFS_LOCATION

    REMOTE_NFS_LOCATION is the remote NFS location in the format of IP:FILE_PATH. The Secure Eraser utility will automatically mount the remote NFS location and save the certificate there.

    For example, to erase all installed devices, including hard drives, flash devices, persistent memory, internal USBs, and ILOM, and save the certificate at this NFS location: 10.133.42.221:/export/exadata_secure_eraser_certificate_dir: 

    -sh-3.1# /usr/sbin/secureeraser --erase --all --output=10.133.42.221:/export
/exadata_secure_eraser_certificate_dir

    To erase just the hard drives:

    -sh-3.1# /usr/sbin/secureeraser --erase --hdd --output=10.133.42.221:/export
/exadata_secure_eraser_certificate_dir

    Note that it is important to point the output option to an NFS location so that the certificate can be saved properly.

    You will be prompted with a list of devices to be erased and to confirm that you want to proceed with Secure Eraser.

    A progress report, as shown in step 7 of Automatic Secure Eraser through PXE Boot for X7 and Later Systems, will be printed to the console every 10 seconds.

    In interactive mode, the server will be left on after the specified devices are securely erased. You can power off the node from the diagnostic shell.

    The web console will no longer be accessible if ILOM is reset. You can power off the server from the serial console or with the power button.

Related Topics

Parent topic: Securely Erasing Oracle Exadata

5.5 Interactive Secure Eraser through Network Boot

Starting with Oracle Exadata X7-2, you can use EFI Network Boot when using Secure Eraser.

Note:

Starting with Oracle Exadata System Software release 19.1.0, the Secure Eraser package (secureeraser_label.zip) contains ISO images instead of NFS images.

Before you begin:

  • Download the Secure Eraser package. Refer to the Supplemental Readme for your currently installed Oracle Exadata System Software image version to find the correct Secure Eraser patch.

  • Make sure you have access to a NFS server that is accessible from all the nodes to be erased.

  • Make sure you have access to one of the nodes to be erased.

  1. Copy the cell initrd file (initrd-version) and kernel file (vmlinux-version) from the Secure Eraser package to the /tftpboot/efi on the network boot server.

    The location does not have to be /tftpboot/efi/. The location is determined by the TFTP Server configuration.

  2. Create a file containing the names of the database servers and storage servers to be erased.

    To generate this file, you can run the following command from one of the nodes to be erased, and verify the nodes in the files are the ones to be erased.

    # ibhosts | awk '/S [0-9\.\,]*/ || /C [0-9\.\,]*/ {print $6}' | sed "s/\"//g" 
> nodes_to_be_erased

    If you only want to erase one server, enter the name of the server into the nodes_to_be_erased file, for example Exa01celadm04.

  3. Copy the dcli utility from the Secure Eraser package along with the file generated in step 2 to the network boot server.

  4. Create a configuration template called pxe_cfg.template to contain the following lines for grub2 and Secure Boot on Oracle Exadata Database Machine X7-2 and newer systems:

    Note:

    In the following example, the following parameters must be updated to match your environment:

    • kernel (the vmlinux file)

    • initrd (the initrd*.img file)

    • logpath

    • For releases earlier than Oracle Exadata System Software release 19.1.0: 

      set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-nfs-18.1.0.0.0-170915.1 stit dhcp pxe boot-from=uefi quiet 
 loglevel=0 bootarea=diagnostics console=ttyS0,115200n8 
 logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-nfs-18.1.0.0.0-170915.1.img 
  echo "Booting installation kernel"
}

    • For Oracle Exadata System Software release 19.1.0 or later: 

      set default 0
set timeout=10
menuentry 'ExadataLinux' {
  echo "Loading efi/vmlinuz"   
 linuxefi efi/vmlinux-nfs-19.1.2.0.0-190111 stit dhcp pxe boot-from=uefi quiet 
 loglevel=0 bootarea=diagnostics console=ttyS0,115200n8 
 logpath=10.133.42.221:/export/exadata_secure_eraser_certificate_dir   
  echo "Loading efi/initrd.img"   
  initrdefi efi/initrd-nfs-19.1.2.0.0-190111.img 
  echo "Booting installation kernel"
}

    • The first line (default) identifies a menu entry that should be selected by default, after the timeout value specified by the second line.

    • The third line (menuentry) represents the Linux kernel that will be used in the Secure Eraser environment.

    • The fifth line (linuxefi) indicates the kernel is on an UEFI-based system. The linuxefi statement must be on a single line in the configuration file.

      • The stit option specifies INSTALL imaging mode, plus BARE METAL conditions, plus ERASING ADD DATA PARTITIONS

      • The dhcp option specifies to use DHCP to discover the eth0 interface.

      • The pxe option suppresses search for the image on virtual CD and USB devices.

      • The boot-from=uefi option indicates the system is booting from UEFI.

      • The quiet option disables excessive kernel log messages.

      • The loglevel=0 option suppresses non-critical kernel messages.

      • The secureeraser option indicates the network boot will automatically trigger the Secure Eraser utility to sanitize all installed media, including hard drives, flash devices, persistent memory, internal USBs, and ILOM on the node.

      • The bootarea option indicates that the boot mode is diagnostic and not imaging install or rescue.

      • The console options indicate standard output and standard error messages are printed to both the ILOM web console and the serial console.

      • The logpath option specifies the NFS share directory where Secure Eraser will save the certificate.

    • The seventh line (initrdefi) specifies the initrd file to load. In this case it is the initrd file copied over in step 1.

  5. On the network boot server, use the template file to generate a network boot configuration file in the /tftpboot/efi/pxelinux.cfg/ directory (Oracle Exadata Database Machine X7-2 and newer) for each of the nodes to be erased.

    The network boot configuration file name is the dash-separated MAC address of the node with the prefix 01-.

    If the nodes to be erased are accessible, perform the following steps to automatically generate a network boot configuration file for each node based on the template:

    1. Set up SSH equivalence with the nodes to be erased from the network boot server. The command will prompt for the root password of each node. 

      pxe_server# dcli -g nodes_to_be_erased -k -l root

    2. Create a list of network boot configuration files, one for each node to be erased based on the configuration template.

      pxe_server# dcli -g nodes_to_be_erased -l root "ip addr show eth0" | 
awk '/link\/ether/ {print "01:"$3}' | sed "s/:/-/g" | xargs -I {} 
cp pxe_cfg.template {}

    If the nodes are not accessible, perform the following steps to generate a network boot configuration file for each node.

    1. Manually collect the MAC address of the eth0 interface from each node and write them into a text file called mac_addresses. Write one MAC address per line. For example: 

      00:10:e0:62:c4:fa
00:10:e0:62:c2:8a
00:10:e0:62:b8:7c
00:10:e0:62:b8:3a
00:10:e0:62:c6:bc

    2. Use the following command to create a list of network boot configuration file, one for each node to be erased based on the configuration template.

      pxe_server# cat mac_addresses | sed "s/:/-/g;s/^/01-/g" | xargs -I {} cp 
pxe_cfg.template {}

    In both cases, you should have a list of network boot configuration files, one for each node to be erased. For example, if the MAC addresses of the nodes in a quarter rack are 00:10:e0:62:c4:fa, 00:10:e0:62:c2:8a, 00:10:e0:62:b8:7c, 00:10:e0:62:b8:3a, and 00:10:e0:62:c6:bc, then you should get the following files:

    01-00-10-e0-62-c4-fa
01-00-10-e0-62-c2-8a
01-00-10-e0-62-b8-7c
01-00-10-e0-62-b8-3a
01-00-10-e0-62-c6-bc

    The files have the same content as the configuration template.

    Check your specific network boot server requirements. Your network boot server may need slightly different names or settings.

  6. Configure the nodes to boot from the network boot server and reboot the nodes.

    If the nodes to be erased are accessible, run the following commands:

    pxe_server# dcli -g nodes_to_be_erased -l root "ipmitool chassis bootdev pxe”

pxe_server# dcli -g nodes_to_be_erased -l root "reboot”

    If the nodes are not accessible, then perform the following steps:

    1. Create a file called iloms_to_be_reset containing the names of ILOMs. For example: 

      db1-ilom
db2-ilom
cell1-ilom
cell2-ilom
cell3-ilom

    2. Configure the nodes to boot from the network boot server through ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H {} 
-U root chassis bootdev pxe

    3. Reboot the nodes from ILOMs. The command will prompt for ILOM root password. 

      pxe_server# cat iloms_to_be_reset | xargs -I {} ipmitool -I lanplus -H {} 
-U root chassis power cycle

  7. If you get the following prompt on the remote or serial console, enter e at the prompt to enter the diagnostic shell: 

    Choose from following by typing letter in '()':
(e)nter interactive diagnostics shell. Must use credentials from Oracle
support to login (reboot or power cycle to exit the shell),
Select:e

  8. If prompted, log in to the system as the root user.

    If you require the password for the root user and do not have it, then contact Oracle Support Services. 

    localhost login: root
Password: *********
-sh-3.1#

  9. Run the Secure Eraser utility to sanitize all devices or one type of device. 

    -sh-3.1# /usr/sbin/secureeraser -–erase --all --output=REMOTE_NFS_LOCATION

    REMOTE_NFS_LOCATION is the remote NFS location in the format of IP:FILE_PATH. The Secure Eraser utility will automatically mount the remote NFS location and save the certificate there.

    For example, to erase all installed devices, including hard drives, flash devices, persistent memory, internal USBs, and ILOM, and save the certificate at this NFS location: 10.133.42.221:/export/exadata_secure_eraser_certificate_dir: 

    -sh-3.1# /usr/sbin/secureeraser --erase --all --output=10.133.42.221:/export
/exadata_secure_eraser_certificate_dir

    To erase just the hard drives:

    -sh-3.1# /usr/sbin/secureeraser --erase --hdd --output=10.133.42.221:/export
/exadata_secure_eraser_certificate_dir

    Note that it is important to point the output option to an NFS location so that the certificate can be saved properly.

    You will be prompted with a list of devices to be erased and to confirm that you want to proceed with the Secure Eraser.

    A progress report, as shown in step 7 of Automatic Secure Eraser through PXE Boot for X7 and Later Systems, will be printed to the console every 10 seconds.

    In interactive mode, the server will be left on after the specified devices are securely erased. You can power off the node from the diagnostic shell.

    The web console will no longer be accessible if ILOM is reset. You can power off the server from the serial console or with the power button.

Related Topics

Parent topic: Securely Erasing Oracle Exadata

5.6 Secure Eraser Syntax

Secure Eraser securely erases all data on both database servers and storage servers, and resets InfiniBand Network Fabric or RDMA over Converged Ethernet (RoCE) switches, Ethernet switches, and power distribution units back to the factory default.

Syntax

secureeraser options

Command-Line Options for Secure Eraser

  • --all

    Perform the action (--list or --erase) on all devices on the system. Devices include hard drives, flash devices, persistent memory, USB devices, and ILOM.

  • --devices_to_erase

    This option allows you specify individual disks to be erased by providing their serial numbers. Multiple serial numbers can be provided separated with commas. Introduced with Oracle Exadata System Software release 19.1.0.

  • --erase

    Perform a secure erase of data.

  • -–erasure_method_optional

    If a device is not eligible to be erased with the provided erasure method, the erasure method will silently fall back to the default method. Otherwise erasure will fail. This option can be used with all types of disks. Introduced with Oracle Exadata System Software release 19.1.0.

  • --flash

    Erase all flash devices.

  • --flash_erasure_method=FLASH_ERASURE_METHOD

    Force all flash devices to be erased using the specified method. The following values are supported:

    • 3pass (Only on Oracle Exadata X4-8 or earlier systems)

    • 7pass (Only on Oracle Exadata X4-8 or earlier systems)

    • crypto (Only on Oracle Exadata X5-2 or later systems, and requires Oracle Exadata System Software release 19.1.0 or later.)

  • --help, -h

    Show this help message and exit.

  • --hdd

    Erase all hard drives.

  • --hdd_erasure_method=HDD_ERASURE_METHOD

    Force all hard drives to be erased using the specified method. The following values are supported:

    • 3pass

    • 7pass

    • crypto (Only on Oracle Exadata X6-2 or later systems, and requires Oracle Exadata System Software release 19.1.0 or later.)

  • --ilom

    Reset Integrated Lights Out Manager (ILOM) to factory default.

  • --list

    List devices (hard drives, flash devices, persistent memory, USB devices, and ILOM) on the system.

  • --m2

    Erase all M.2 devices.

  • --m2_erasure_method=M2_ERASURE_METHOD

    Force all M.2 devices to be erased using the specified method. The following values are supported:

    • 3pass

    • 7pass

    • crypto (Requires Oracle Exadata System Software release 19.1.0 or later)

  • --output=CERTIFICATE_DIRECTORY, -o

    Specify a full path to the directory for the certificate output location. The default is /var/log/cellos.

  • --pmem

    Erase all persistent memory (PMEM) devices using cryptographic erasure.

  • --quiet, -q

    Quietly skip prompts.

  • --technician=TECHNICIAN_NAME, -t TECHNICIAN_NAME

    Specify the name of the technician performing the erasure. This name will be recorded in the certificate.

  • --usb

    Erase all internal USB devices.

  • --usb_erasure_method=USB_ERASURE_METHOD

    Force all internal USB devices to be erased using the specified method. The following values are supported:

    • 3pass

    • 7pass

  • --witness=WITNESS_NAME, -w WITNESS_NAME

    Specify the name of the person witnessing the erasure. This name will be recorded in the certificate.

Examples of Secure Eraser Syntax

List all devices (hard drives, flash devices, persistent memory, USB devices, and ILOM) on the system. 

secureeraser --list --all

List all hard drives.

secureeraser --list --hdd

Securely erase all devices, and enter the names of the technician and witness in the certificate.

secureeraser --erase --all --technician="jdoe" --witness="jsmith"

Reset ILOM to factory default. 

secureeraser --erase --ilom

Securely erase all hard drives.

secureeraser --erase --hdd

Securely erase all hard drives, all flash devices, and all internal USB devices. Force "crypto" method on flash devices.

secureeraser --erase --hdd --flash --usb --flash_erasure_method crypto

Parent topic: Securely Erasing Oracle Exadata

5.7 Resetting Network Switches and Power Distribution Units to Factory Default

Before you begin:

  • Download the Secure Eraser package. Refer to the Supplemental Readme for your currently installed Oracle Exadata System Software image version to find the correct Secure Eraser patch.

  • Print out the Exadata Factory Reset Certificate template in the Secure Eraser package.

The following figure shows the Factory Reset certificate:

Figure 5-2 Factory Reset Certificate

Description of Figure 5-2 follows
Description of "Figure 5-2 Factory Reset Certificate"

Use the following procedures:

Parent topic: Securely Erasing Oracle Exadata

5.7.1 Resetting a Cisco Nexus 9336C-FX2 RoCE Network Fabric Switch to Factory Default Settings

The Cisco Nexus 9336C-FX2 RoCE Network Fabric switch comes preconfigured with specific configurations for RDMA over Converged Ethernet (RoCE). If you plan to reuse this switch in Oracle Exadata you need to save this configuration to:

  • Local bootflash
  • A remote server

To reset a Cisco Nexus 9336C-FX2 RoCE Network Fabric switch:

  1. Make up backup of the current switch configuration.

    You must save the current configuration if you plan to use the switch after resetting the configuration back to the factory settings. If you are returning the switch, then you do not need to save the RoCE-specific configuration.

    Follow the steps documented in Backing Up Settings on the ROCE Switch

  2. Delete the files in all directories on the switch.

    1. List the directories available on the switch.

      switch# dir ?

    2. For each directory listed in the above output (represented as dir_name), view the directory contents. 

      switch# dir dir_name:

    3. If any files are found in a directory, then delete the files.

      switch# del dir_name:* no-prompt

  3. Use the write erase command on the switch to remove the current configuration.

    write erase will erase the RoCE-specific configurations.

    Note:

    After you enter the write erase command, you must reload the ASCII configuration twice to apply the breakout configuration.

    The write erase command erases the entire startup configuration, except for the following:

    • Boot variable definitions
    • The IPv4 and IPv6 configuration on the mgmt0 interface, including the following:
      • Address Subnet mask
      • Default Gateway/Route in the management VR

    To also remove the boot variable definitions and the IPv4/IPv6 configuration on the mgmt0 interface, use the write erase boot command.

  4. Record the serial numbers of the switches that have been reset to factory default in the Factory Reset certificate template. Sign and date the entries.

    You can identify the serial number of an RoCE Network Fabric switch by running the following command on the switch: 

    switch# show license host-id
License hostid:VDH=FOX064317SQ

    The host ID is also referred to as the device serial number. In the above example, you use all the text that appears after the equal sign (=), so the switch serial number is FOX064317SQ.

See Also:

Cisco NX-OS Licensing Guide at https://www.cisco.com/c/en/us/td/docs/switches/datacenter/sw/nx-os/licensing/guide/b_Cisco_NX-OS_Licensing_Guide.pdf

Parent topic: Resetting Network Switches and Power Distribution Units to Factory Default

5.7.2 Resetting InfiniBand Network Fabric Switches to Factory Default

To reset the InfiniBand Network Fabric switches to factory default, refer to My Oracle Support note 2180877.1.

Record the serial numbers of the switches that have been reset to factory default in the Factory Reset certificate template. Sign and date the entries.

You can identify the serial number of an InfiniBand Network Fabric switch by running the following command on the switch: 

[root@switch1 ~]# version | grep "Serial Number"

Related Topics

Parent topic: Resetting Network Switches and Power Distribution Units to Factory Default

5.7.3 Resetting the Cisco Management Network Switch to Factory Default Settings

You can reset the Cisco Management Network Switch configuration to the original default factory settings.

For Oracle Exadata X7-2 or later

To reset the Cisco Management Network Switch to factory default:

  1. Display the start up configuration.

    switch# show startup-config

  2. Display the boot configuration.

    switch# show boot

  3. Display the debug configuration.

    switch# show debug

  4. Delete the files in all directories on the switch.

    1. List the directories available on the switch.

      switch# dir ?

    2. For each directory listed in the above output (represented as dir_name), view the directory contents. 

      switch# dir dir_name:

    3. If any files are found in a directory, then delete the files.

      switch# del dir_name:* no-prompt

  5. Erase the startup-configuration file.

    switch# write erase 

Warning: This command will erase the startup-configuration.

Do you wish to proceed anyway? (y/n)  [n] y

  6. Erase the boot variable definitions.

    switch# write erase boot

Warning: This command will erase the boot variables and the ip configuration of interface mgmt 0

Do you wish to proceed anyway? (y/n)  [n] y

  7. Erase the debugging configuration.

    switch# write erase debug

  8. Reload the Cisco Nexus 93108-1G or Cisco Nexus 9348 Ethernet switch.

    switch# reload

This command will reboot the system. (y/n)  [n] y

See Also:

"Erasing a Configuration" in Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide, Release 6.x

For Oracle Exadata X6-2 or earlier

To reset the Cisco Management Network Switch to factory default, refer to "Reset Catalyst Switches Running Cisco IOS Software" in the Cisco Troubleshooting TechNotes Document ID: 24328.

Record the serial number of the Ethernet switch that has been reset to factory default in the Exadata Factory Reset certificate template. Sign and date the entry.

The serial number of an Ethernet switch can be identified by the “Processor board ID” field in the “show version” command output.

switch# show version

See Also:

"Reset Catalyst Switches Running Cisco IOS Software" at http://www.cisco.com/c/en/us/support/docs/switches/catalyst-2900-xl-series-switches/24328-156.html#reset_ios

Parent topic: Resetting Network Switches and Power Distribution Units to Factory Default

5.7.4 Resetting Power Distribution Units to Factory Default

You can reset the power distribution units (PDUs) configuration to the original default factory settings.

There are two types of power distribution units (PDUs): original PDUs and enhanced PDUs. Enhanced PDUs have SER MGT port that can be connected to a host using an RS-232 cable, whereas the original PDUs do not have SER MGT port. Typically, Oracle Exadata V2 to Oracle Exadata X3 racks have the original PDUs, and Oracle Exadata X4-2 and later have the enhanced PDUs.

You can reset both the original power distribution units and the enhanced power distribution units, as described in the Sun Rack II Power Distribution Units User’s Guide or the Oracle Rack Cabinet 1242 Power Distribution Units User's Guide (for Oracle Exadata X7-2 and later systems).

Record the serial numbers of the power distribution units that have been reset to factory default in the Exadata Factory Reset certificate template. Sign and date the entries.

The serial number can be found on the “View Module Information” topic of the appropriate Power Distribution Units User's Guide.

For enhanced PDUs, the serial number can also be retrieved through the following CLI command:

pducli -> get pdu_serial_number

See Also:

Parent topic: Resetting Network Switches and Power Distribution Units to Factory Default

5.8 Actions After Using Secure Eraser

After performing a secure erase, the system is ready for return or re-imaging.

If you plan to re-image the machine, you must perform the following steps:

  1. Connect to and configure ILOM. See "Oracle ILOM – Quick Start" in the Oracle ILOM Getting Started Guide Firmware Release 4.0.x.

    Perform the following tasks:

    1. Connect to Oracle ILOM

    2. Log In to Oracle ILOM

    3. Modify Default Network Connectivity Settings

  2. Re-image the system. Refer to Imaging a New System in Oracle Exadata Database Machine Installation and Configuration Guide.

If you are preparing to return the machine, refer to http://www.oracle.com/us/products/servers-storage/take-back-and-recycling/index.html

Parent topic: Securely Erasing Oracle Exadata