1. Installation and Configuration Guide for Exadata Database Machine
  2. Configuring Oracle Exadata Database Machine

6 Configuring Oracle Exadata Database Machine

This chapter describes how to configure the system, accounts, and software for Oracle Exadata Database Machine.

Note:

  • For ease of reading, the name "Oracle Exadata Rack" is used when information refers to both Oracle Exadata Database Machine and Oracle Exadata Storage Expansion Rack.

  • The procedures in this chapter are applicable to Oracle Exadata Storage Expansion Rack as well as Oracle Exadata Database Machine.

  • The procedures in this chapter use the information from Oracle Exadata Database Machine Deployment Assistant (OEDA). Be sure to run OEDA before starting the procedures in this chapter.

Related Topics

6.1 Verifying the Network Configuration Prior to Configuring the Rack

Use the checkip.sh script to ensure there are no IP address conflicts between the existing network and your new Oracle Exadata Rack.

The checkip.sh script performs a pre-installation check to verify that the IP addresses and host names that you specified in Oracle Exadata Deployment Assistant (OEDA) are defined in the DNS, that the NTP servers and gateways are available, and that private addresses are not pingable. Running this script before the hardware arrives help to avoid additional delays that would be caused by misconfigured network services, such as Domain Name System (DNS) and NTP.

The checkip.sh script is created in a format that matches the operating system of the client on which you ran OEDA. Because this script is run before the engineered system rack has arrived, you typically do not run this script on an engineered system server, but on a client. The client must have access to the same network where the engineered system will be deployed. The script is also available in the ZIP file generated by OEDA.

  1. On the client where OEDA was run, copy the checkip.sh script generated by OEDA and the XML file CustomerName_hostname.xml to the same directory (one directory level up) as the OEDA config.sh script.
  2. Run the checkip.sh script on the client machine or existing server.

    Use a command similar to the following, where configuration_file is the name of the configuration generated by the Oracle Exadata Deployment Assistant for the rack being installed. 

    # ./checkip.sh -cf configuration_file
    If the command is run from a Microsoft Windows machine, then the command is checkip.cmd.

    If this engineered system rack is an addition for an existing installation, then run the checkip.sh script from an existing engineered system server. This enables the script to identify in-use IP addresses in the fabric. Not identifying existing IP addresses may cause IP collisions after installation of the new engineered system rack. To create a checkip.sh that can run on an existing server, you must run OEDA on a server or client that uses the same operating system as the existing engineered system server. OEDA supports IPv6 addresses.

    The output from the script is a file that contains status messages such as GOOD or ERROR.

If there are conflicts that you are unable to resolve, then work with your assigned Oracle representative to correct the problems.

Parent topic: Configuring Oracle Exadata Database Machine

6.2 Configuring the RDMA Network Fabric Switch

You must perform an initial configuration of the RDMA Network Fabric switch.

Parent topic: Configuring Oracle Exadata Database Machine

6.2.1 Configuring the Cisco Nexus 9336C-FX2 Switch

The RoCE Network Fabric switch supplied with the engineered system rack is minimally configured during installation.

During initial system configuration, you can reset and configure the switch.

  1. Connect from the RoCE Network Fabric switch serial console to a laptop or similar device using the available RJ45 cable.
  2. Ensure the terminal session is recorded on the laptop by logging the output.
    The output can be used as a reference that the switch has been configured correctly.
  3. Power on the switch.
  4. Log in as the admin user.
    User Access Verification
dbm0sw-rocea0 login: admin
Password: ********

    Note:

    If you do not have the password for the admin user, then contact Oracle Support Services.
  5. Erase the existing configuration.
    dbm0sw-rocea0# write erase

Warning: This command will erase the startup-configuration.

Do you wish to proceed anyway? (y/n)  [n] y
  6. Restart the system so you can perform the automated setup.
    dbm0sw-rocea0# reload

This command will reboot the system. (y/n)?  [n] y

2017 Aug 31 01:09:00 dbm0sw-rocea0 %$ VDC-1 %$ %PLATFORM-2-PFM_SYSTEM_RESET: Manual system restart from Command Line Interface


CISCO SWITCH Ver7.59
Device detected on 0:1:2 after 0 msecs  
...
  7. Switch to normal setup and, when asked if you want to enforce secure password standard, enter no, then enter a new password for the admin user.
    Running S93thirdparty-script...

Populating conf files for hybrid sysmgr ...
Starting hybrid sysmgr ...
inserting /isan/lib/modules/klm_cisco_nb.o ... done

Abort Auto Provisioning and continue with normal setup ? (yes/no) [n]: yes

         ---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no) [y]: no

  Enter the password for "admin": 
  Confirm the password for "admin":
  8. When the Basic System Configuration Dialog appears, choose to enter the basic configuration dialog.
            ---- Basic System Configuration Dialog VDC: 1 ----

This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.

Please register Cisco Nexus9000 Family devices promptly with your
supplier. Failure to register may affect response times for initial
service calls. Nexus9000 devices must be registered to receive 
entitled support services.

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
to skip the remaining dialogs.

Would you like to enter the basic configuration dialog (yes/no): yes
  9. In the basic configuration, you can use the default inputs until asked to enter the switch name.

    In this example, the switch has a name of test123sw-rocea0. 

      Create another login account (yes/no) [n]: 
  Configure read-only SNMP community string (yes/no) [n]: 
  Configure read-write SNMP community string (yes/no) [n]: 
  Enter the switch name : test123sw-rocea0
  10. Respond yes when asked to configure Out-of-band management configuration, and specify appropriate network addresses when prompted.
    Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: yes
     Mgmt0 IPv4 address : 100.104.10.21
     Mgmt0 IPv4 netmask : 255.255.248.0
  Configure the default gateway? (yes/no) [y]:
     IPv4 address of the default gateway : 100.104.10.1
  11. Respond yes when asked to configure advanced IP options.
    Configure advanced IP options? (yes/no) [n]: yes
  12. Respond yes when asked to configure static route (this can be changed later).
    Configure static route? (yes/no) [n]: yes
  13. Enter the destination prefix and mask, and other values as prompted.
       Destination prefix : 10.100.100.0

   Destination prefix mask : 255.255.255.0

   Next hop IPv4 address : 10.100.100.1
  14. Configure the DNS IPv4 addresses.
    Configure the DNS IPv4 address? (yes/no) [n]: yes
   DNS IP address: 10.100.100.2
  15. Skip configuring the default domain name (this will be configured later).
    Configure the default domain name? (yes/no) [n]: no
  16. Accept the default responses until asked to configure SSH and the NTP server.
    Enable the telnet service? (yes/no) [n]: no
Enable the ssh service? (yes/no) [y]: yes
   Type of ssh key you would like to generate (dsa/rsa) [rsa]: rsa
   Number of rsa key bits <1024-2048> [1024]: 1024
 
Configure the NTP server? (yes/no) [n]: yes
     NTP server IPv4 address : 10.100.100.3
  17. Accept the default responses until asked to specify the CoPP system profile. Enter strict.
     Configure default interface layer (L3/L2) [L2]: 
 Configure default switchport interface state (shut/noshut) [noshut]: 
 Configure CoPP system profile (strict/moderate/lenient/dense) [strict]: strict
  18. After reviewing the configuration, save the configuration.
    The following configuration will be applied:
   no password strength-check
   switchname test123sw-rocea0
   ip route 100.104.8.0 255.255.248.0 100.104.10.1
   vrf context management
   ip route 0.0.0.0/0 100.104.10.1
   exit
    no feature telnet
    ssh key rsa 1024 force
    feature ssh
    ntp server 100.104.10.1
    system default switchport
    no system default switchport shutdown
    copp profile strict
   interface mgmt0
   ip address 100.104.10.21 255.255.248.0
   no shutdown

Would you like to edit the configuration? (yes/no) [n]: 

Use this configuration and save it? (yes/no) [y]: yes

[########################################] 100%
Copy complete.
  19. Enable the scp server feature on the switch.
    test123sw-rocea0# feature scp-server
  20. Save the running configuration to flash.
    test123sw-rocea0# copy running-config startup-config
[########################################] 100%
Copy complete.
  21. Apply the golden configuration on the switch.
    1. Delete the configuration file on the switch for the target configuration.

      Note:

      If you do not remove the file you are replacing, then when you attempt to overwrite the file you will get a 'permission denied' error.

      Log in to the switch, enter configuration mode, then run a command similar to the following:

      test123sw-rocea0# delete bootflash:roce_leaf_switch.cfg
Do you want to delete "/roce_leaf_switch.cfg" ? (yes/no/abort) [y] y
test123sw-rocea0#
    2. Log in to a server that has SSH access to the switch, and contains the latest RDMA Network Fabric patch ZIP file.

      To find the available RDMA Network Fabric patches, search for 'RDMA network switch' in My Oracle Support document 888828.1. Download and use the latest patch for your Oracle Exadata System Software release.

    3. Unzip the RDMA Network Fabric patch ZIP file and change directories to the location of the patchmgr utility.
    4. Locate the golden configuration files in the RDMA Network Fabric patch bundle.

      The files are located within the roce_switch_templates directory.

      The golden configuration files are as follows:

      • Single rack leaf (leaf): roce_leaf_switch.cfg
      • Multi-rack spine (mspine): roce_spine_switch_multi.cfg
      • Multi-rack leaf (mleaf): roce_leaf_switch_multi.cfg
      • Single rack leaf with Secure Fabric support (sfleaf): roce_sf_leaf_switch.cfg
      • Multi-rack leaf with Secure Fabric support (msfleaf): roce_sf_leaf_switch_multi.cfg
      • Single rack leaf configured with 23 host ports (leaf23): roce_leaf_switch_23hosts.cfg
      • Multi-rack leaf configured with 23 host ports (mleaf23): roce_leaf_switch_23hosts_multi.cfg
      • Multi-rack leaf configured with 14 inter-switch links (mleaf_u14): roce_leaf_switch_14uplinks_multi.cfg
      • Multi-rack leaf configured with 14 inter-switch links and with Secure Fabric support (msfleaf_u14): roce_sf_leaf_switch_14uplinks_multi.cfg
      • Multi-rack leaf configured with 23 host ports and 13 inter-switch links (mleaf23_u13): roce_leaf_switch_23hosts_13uplinks_multi.cfg
    5. Copy the golden configuration file to the switch.

      In the following example, 100.104.10.21 represents the IP address of the switch you are configuring. 

      # scp roce_leaf_switch.cfg admin@100.104.10.21:/
User Access Verification
Password:
roce_leaf_switch.cfg 100% 23KB 23.5KB/s 00:00
    6. Apply the golden configuration file on the switch.
      Use the run-script command while connected directly to the switch. 
      test123sw-rocea0# run-script bootflash:roce_leaf_switch.cfg | grep 'none'

      Note:

      This command may take up to 1-2 minutes on a single-rack switch and up to 3-4 minutes on a multi-rack switch.
    7. Verify the switch configuration.
      Use the patchmgr utility on the server that has SSH access to the switch, and contains the latest RDMA Network Fabric patch bundle.

      In the following command, roceswitch.lst is a file that contains the switch host name or IP address. 

      # ./patchmgr --roceswitches roceswitch.lst --verify-config
  22. Backup up the switch configuration.

    Follow the steps in Backing Up Settings on the ROCE Switch, in Oracle Exadata Database Machine Maintenance Guide.

  23. Optional: Set the clock, using the same procedure as in Setting the Clock on the Cisco 93108-1G or 9348 Ethernet Switch.

Related Topics

Parent topic: Configuring the RDMA Network Fabric Switch

6.2.2 Configuring the RoCE Network Fabric Switch Switches to Enable Exadata Secure RDMA Fabric Isolation

You can configure your RoCE Network Fabric switches to enable Exadata Secure RDMA Fabric Isolation.

Secure Fabric enables network separation between different clusters on systems with RoCE Network Fabric. This capability is conceptually similar to InfiniBand Network Fabric partitioning. Secure Fabric support is available starting with Oracle Exadata System Software release 20.1.0.

To use Secure Fabric, you must enable Secure Fabric support in the RoCE Network Fabric switch hardware before initial system deployment using Oracle Exadata Deployment Assistant (OEDA):

For each RoCE Network Fabric leaf switch, perform the basic configuration steps outlined in Configuring the Cisco Nexus 9336C-FX2 Switch.

Then, on each RoCE Network Fabric leaf switch, you must apply a specific golden configuration to enable Secure Fabric support. Use the procedure described in Applying Golden Configuration Settings on RoCE Network Fabric Switches, in Oracle Exadata Database Machine Maintenance Guide. However, when you specify the configuration type for each leaf switch, ensure that you specify the configuration type that enables Secure Fabric support.

After you complete the switch configuration, the leaf switch ports become trunk ports, which can carry network traffic with multiple VLAN IDs.

Parent topic: Configuring the RDMA Network Fabric Switch

6.2.3 Configuring Sun Datacenter InfiniBand Switch 36 Switch

The following procedure describes how to configure the Sun Datacenter InfiniBand Switch 36 switches.

  1. Log in to the first database server as the root user. The first database server is the lowest database server in the rack, which is rack position U16.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  2. Use SSH to log in to the Sun Datacenter InfiniBand Switch 36 switch as the ilom-admin user. Find the default IP address for the Sun Datacenter InfiniBand Switch 36 switch for your system in "Default IP Addresses". The following is an example of the command: 

    ssh ilom-admin@192.168.1.201

  3. Use spsh to open the Integrated Lights Out Manager (ILOM) command line interface.

  4. Configure the network interface using the following commands:

    set /SP/network pendingipdiscovery=static pendingipaddress=pending_ip \
pendingipgateway=pending_gw pendingipnetmask=pending_nm

set /SP/network commitpending=true

    In the preceding commands, pending_ip, pending_gw, and pending_nm are IP addresses defined by the network administrator.

  5. Use the exit command to exit the interface.

  6. Edit the /etc/hosts file to set the IP address and host name using an editor such as vi. The following is an example of the updated file: 

    #Do not remove the following link, or various programs
#that require network functionality will fail.
127.0.0.1     localhost.localdomain localhost
10.7.7.32     dm01sw-ib1-ib2.example.com trnasw-ib2

    It is required that the second line has both fully-qualified and non-fully-qualified names.

  7. Use the ILOM interface to configure the IP address, host name, NTP servers, and DNS servers.

  8. Restart the switch.

  9. Examine the firmware version using the following command:

    # version

    Refer to My Oracle Support Note 888828.1 for the current firmware version.

  10. Check the health of the switch using the following command:

    # showunhealthy

OK - No unhealthy sensors

  11. Run the environment test using the following command:

    # env_test

NM2 Environment test started:
Starting Voltage test:
Voltage ECB OK
Measured 3.3V Main = 3.28 V
Measured 3.3V Standby = 3.42 V
Measured 12V =12.06 V
Measured 5V =5.03 V
Measured VBAT =3.06 V
Measured 2.5V =2.53 V
Measured 1.8V =1.79 V
Measured I4 1.2V =1.22 V
Voltage test returned OK
Starting PSU test:
PSU 0 present
PSU 1 present
PSU test returned OK
Starting Temperature test:
Back temperature 30.50
Front temperature 33.88
ComEx temperature 34.12
I4 temperature 56,
maxtemperature 57
Temperature test returned OK
Starting FAN test:
Fan 0 not present
Fan 1 running at rpm 12946
Fan 2 running at rpm 12684
Fan 3 running at rpm 12558
Fan 4 not present
FAN test returned OK
Starting Connector test:
Connector test returned OK
Starting I4 test:
I4 OK
All I4s OK
I4 test returned OK
NM2 Environment test PASSED

  12. Enable the InfiniBand Subnet Manager using the following command:

    # enablesm

    Note:

    If you get an error indicating the InfiniBand Subnet Manager is already running, then restart it as follows:

    # disablesm

# enablesm

  13. Verify the IP address is correct using the following command:

    # ifconfig eth0

eth0    Link encap:Ethernet HWaddr 00:E0:4B:2A:07:2B
        inet addr:172.16.10.32 Bcast:172.16.10.255
Mask:255.255.255.0
        inet6 addr:fe80::2e0:3a00:fe2a:61e/64 Scope:Link
        UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
        RX packets:11927 errors:0 dropped:0 overruns:0 frame:0
        TX packets:89 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:720262 (703.3 KiB)TX bytes:11402 (11.1 KiB)

  14. Verify the host name is correct using the following command:

    # hostname

dm01sw-ib2.example.com

  15. Set the Subnet Manager Master as described in "Setting the Subnet Manager Master on Oracle Exadata Database Machine Full Rack and Oracle Exadata Database Machine Half Rack". This step is needed for Oracle Exadata Database Machine Half Rack and Oracle Exadata Database Machine Full Rack.

  16. Log out from the InfiniBand switch using the following command:

    # exit

Note:

If the network settings did not appear after restart, then power cycle the switch by removing both power cords for one minute.

Related Topics

Parent topic: Configuring the RDMA Network Fabric Switch

6.3 Setting the Subnet Manager Master on Oracle Exadata Database Machine Full Rack and Oracle Exadata Database Machine Half Rack

Oracle Exadata X3-2 systems and Oracle Exadata X2-2 systems have three Sun Datacenter InfiniBand Switch 36 switches. Starting with Oracle Exadata X4-2, Oracle Exadata Systems have two Sun Datacenter InfiniBand Switch 36 switches.

Note:

This procedure does not apply to Oracle Exadata X8M racks with RoCE Network Fabric.

The switch located in rack unit 1 (U1) is referred to as the spine switch. The other two switches are referred to as the leaf switches. The location of the leaf switches is as follows:

  • Oracle Exadata Two-Socket Systems (X3-2 and later): rack unit 20 (U20) and rack unit 22 (U22)

  • Oracle Exadata X2-2 racks: rack unit 20 (U20) and rack unit 24 (U24)

  • Oracle Exadata Eight-Socket Systems (X2-8 and later) Full Racks: Rack unit 21 (U21) and rack unit 23 (U23)

The spine switch is the Subnet Manager Master for the InfiniBand Network Fabric subnet. The Subnet Manager Master has priority 8, and can be verified using the following procedure:

  1. Log in to the spine switch as the root user.

  2. Run the setsmpriority list command.

    The command should show that smpriority has a value of 8. If smpriority has a different value, then do the following:

    1. Use the disablesm command to stop the Subnet Manager.

    2. Use the setsmpriority 8 command to set the priority to 8.

    3. Use the enablesm command to restart the Subnet Manager.

The leaf switches are the Standby Subnet Managers with a priority of 5. This can be verified using the preceding procedure, substituting a value of 5 in the setsmpriority command above.

Note:

Oracle Exadata Half Rack with Sun Fire X4170 Oracle Database Servers include two Sun Datacenter InfiniBand Switch 36 switches, which are set to priority 5.

To determine the Subnet Manager Master, log in as the root user on any InfiniBand Network Fabric switch, and run the getmaster command. The location of the Subnet Manager Master is displayed. The following is an example of the output from the getmaster command: 

# getmaster
20100701 11:46:38 OpenSM Master on Switch : 0x0021283a8516a0a0 ports 36 Sun DCS 36
QDR switch dm01sw-ib1.example.com enhanced port 0 lid 1 lmc 0

The preceding output shows the proper configuration. The Subnet Master Manager is running on spine switch dm01sw-ib1.example.com.

If the spine switch is not the Subnet Manager Master, then do the following procedure to set the Subnet Manager Master:

  1. Use the getmaster command to identify the current location of the Subnet Manager Master.

  2. Log in as the root user on the leaf switch that is the Subnet Manager Master.

  3. Disable Subnet Manager on the switch. The Subnet Manager Master relocates to another switch.

  4. Use the getmaster command to identify the current location of the Subnet Manager Master. If the spine switch is not Subnet Manager Master, then repeat steps 2 and 3 until the spine switch is the Subnet Manager Master.

  5. Enable Subnet Manager on the leaf switches that were disabled during this procedure.

Note:

  • If the InfiniBand Network Fabric network consists of four or more racks cabled together, then only the spine switches should run Subnet Manager. The leaf switches should have Subnet Manager disabled on them.
  • Oracle Exadata Half Racks with Sun Fire X4170 Oracle Database Servers, and Oracle Exadata Quarter Racks have two Sun Datacenter InfiniBand Switch 36 switches, and both are set to priority 5. The master is the one with the lowest GUID.

See Also:

Parent topic: Configuring Oracle Exadata Database Machine

6.4 Configuring the Management Network Switch

You must perform an initial configuration of the Management Network Switch.

Parent topic: Configuring Oracle Exadata Database Machine

6.4.1 Configuring the Cisco Catalyst 4948 Ethernet Switch

The Cisco Catalyst 4948 Ethernet switch supplied with Oracle Exadata Rack is minimally configured during installation.

The minimal configuration disables IP routing, and sets the following:

  • Host name
  • IP address setup
  • Subnet mask
  • Default gateway
  • Domain name
  • Name server
  • NTP server
  • Time
  • Time zone

Before configuring the switch, note the following:

  • The Cisco Ethernet switch should not be connected until the running configuration has been verified, and any necessary changes have been made by the network administrator.

  • The Cisco Ethernet switch should not be connected to the customer network until the IP addresses on all components have been configured in Oracle Exadata Rack. This is to prevent any duplicate IP address conflicts which are possible due to the default addresses set in the components when shipped.

Note that the Cisco 4948E-F switch supports multiple uplinks to the customer network by utilizing ports 49 - 52. This is a more complicated switch setup due to the redundant connectivity, and should be performed by the customer's network administrator.

The following procedure describes how to configure the Cisco Ethernet switch. Configuration should be done with the network administrator.

  1. Connect a serial cable from the Cisco switch console to a laptop or similar device. An Oracle supplied rollover cable is pre-installed on the Cisco serial console port. Obtain the appropriate adapter and connect it at the end of the rollover cable. An Oracle P/N 530-3100 RJ45-DB9 adapter as used on ILOM ports will also work, connected at the end of the network cable.

  2. Ensure the terminal session is recorded on the laptop by logging the output. The output can be used as a reference that the switch has been configured correctly. The default serial port speed is 9600 baud, 8 bits, no parity, 1 stop bit, and no handshake. 

    Switch con0 is now available
Press RETURN to get started.

  3. Change to the enable mode.

    Switch> enable
Password: ******
Switch#

    Note:

    If you do not have the password, then contact Oracle Support Services.

  4. Check the current version on the switch.

    Switch# show version 
Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-
IPBASEK9-M), Version 15.2(3)E2, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Tue 11-Mar-14 18:28 by prod_rel_team

ROM: 12.2(44r)SG12
zdlra1sw-ip uptime is 1 minute
System returned to ROM by reload
System image file is "bootflash:cat4500e-ipbasek9-mz.152-3.E2.bin"
Hobgoblin Revision 22, Fortooine Revision 1.40
...

Configuration register is 0x2102

Switch#

    The version of the Cisco 4948E-F switch firmware purchased and shipped by Oracle with Oracle Exadata X6 is IPBASEK9-MZ, which includes telnet and ssh support. Currently the full release version string is cat4500e-ipbasek9-mz.152-3.E2.bin.

  5. Configure the network for a single VLAN. The following example assumes you are using IPv4 addressing.

    Switch# configure terminal
Enter configuration commands,one per line.End with CNTL/Z.
Switch(config)# interface vlan 1
Switch(config-if)# ip address 10.7.7.34 255.255.255.0
Switch(config-if)# end
Switch# *Sep 15 14:12:06.309:%SYS-5-CONFIG_I:Configured from console by console
Switch# write memory
Building configuration...
Compressed configuration from 2474 bytes to 1066 bytes [OK ]

  6. If IP routing is not used on the switch, this step is required. Disable the default IP routing setting, and configure the default gateway. 

    Switch#configure terminal
Enter configuration commands,one per line.End with CNTL/Z.
Switch(config)#no ip routing
Switch(config)#ip default-gateway 10.7.7.1
Switch(config)#end 
*Sep 15 14:12:46.309:%SYS-5-CONFIG_I:Configured from console by console
Switch#write memory
Building configuration...
Compressed configuration from 2492 bytes to 1070 bytes [OK ]

  7. If IP routing is required on the switch, then leave the IP routing setting as the default, and configure the default gateway. Replace 10.7.7.1 with the IP address of the gateway for the installation: 

    Switch#configure terminal
Enter configuration commands,one per line.End with CNTL/Z.
Switch(config)#ip route 0.0.0.0 0.0.0.0 10.7.7.1
Switch(config)#end
*Sep 15 14:13:26.013:%SYS-5-CONFIG_I:Configured from console by console
Switch#write memory
Building configuration...
Compressed configuration from 2502 bytes to 1085 bytes [OK ]

  8. Set the host name of the switch.

    This example sets the name to exa1sw-ip: 

    Switch#configure terminal
Enter configuration commands,one per line.End with CNTL/Z.
Switch(config)#hostname exa1sw-ip
exa1sw-ip(config)#end
exa1sw-ip#write memory
Building configuration...
Compressed configuration from 3789 bytes to 1469 bytes [OK ]

    The system host name is used as the prompt name.

  9. Configure up to three DNS servers. Replace the domain name and IP addresses used in this example with the values for the installation:

    exa1sw-ip#configure terminal
Enter configuration commands,one per line.End with CNTL/Z.
exa1sw-ip(config)#ip domain-name example.com
exa1sw-ip(config)#ip name-server 10.7.7.3
exa1sw-ip(config)#ip name-server 198.51.100.5 
exa1sw-ip(config)#ip name-server 10.8.160.1
exa1sw-ip(config)#end 
*Sep 15 14:26:37.045:%SYS-5-CONFIG_I:Configured from console by console
exa1sw-ip#write memory
Building configuration...
Compressed configuration from 2603 bytes to 1158 bytes [OK ]

    If you do not have DNS service available, you must still set the domain-name so that you can configure the SSH keys.

  10. (Optional) Set the password.

    exa1sw-ip# configure terminal
Enter configuration commands,one per line. End with CNTL/Z.
exa1sw-ip(config)# enable password password
exa1sw-ip(config)# enable secret password 
exa1sw-ip(config)# end
exa1sw-ip# write memory 
*Sep 15 14:25:05.893:%SYS-5-CONFIG_I:Configured from console by console
Building configuration...
Compressed configuration from 2502 bytes to 1085 bytes [OK ]

  11. Verify telnet access is disabled. Telnet is not secure, and should not be enabled unless there is a compelling reason. To enable telnet, set a password. To disable it, remove the password.

    exa1sw-ip#configure terminal
Enter configuration commands,one per line. End with CNTL/Z.
exa1sw-ip(config)#line vty 0 15
exa1sw-ip(config)#login
% Login disabled on line 1, until 'password' is set
% Login disabled on line 2, until 'password' is set
 ...
% Login disabled on line 16, until 'password' is set
exa1sw-ip(config)#end

    If the login command returns output as shown above, then telnet access has been disabled. If instead you get a prompt, then telnet access is not yet disabled so should be disabled now.

    exa1sw-ip(config-line)#no password
exa1sw-ip(config-line)#end
exa1sw-ip#write memory 
Building configuration...
Compressed configuration from 3786 bytes to 1468 bytes [OK ]

  12. To configure a secure shell (SSH) on the Ethernet switch:

    exa1sw-ip# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
exa1sw-ip(config)# crypto key generate rsa
% You already have RSA keys defined named exa1sw-ip.example.com.
% Do you really want to replace them? [yes/no]: yes
Choose the size of the key modulus in the range of 360 to 2048 for
your General Purpose Keys. Choosing a key modulus greater than 512
may take a few minutes.
How many bits in the modulus [512]: 768

% Generating 768 bit RSA keys, keys will be non-exportable...[OK]
exa1sw-ip(config)# username admin password 0 welcome1
exa1sw-ip(config)# line vty 0 15
exa1sw-ip(config-line)# transport input ssh
exa1sw-ip(config-line)# exit
exa1sw-ip(config)# aaa new-model

exa1sw-ip(config)# ip ssh time-out 60
exa1sw-ip(config)# ip ssh authentication-retries 3
exa1sw-ip(config)# ip ssh version 2
exa1sw-ip(config)# end
*Sep 15 14:26:37.045: %SYS-5-CONFIG_I: Configured from console by console
exa1sw-ip# write memory
Building configuration...
Compressed configuration from 2603 bytes to 1158 bytes[OK]

  13. Set the clock and time zone. The switch keeps internal time in Coordinated Universal Time (UTC) format.

    • To use UTC, use the following command:

      no clock timezone global configuration

    • To use a time zone, use the following command:

      clock timezone zone hours-offset [minutes-offset]

      In the preceding command, zone is the time zone to display when standard time in effect, hours-offset is the hours offset from UTC, and minutes-offset is the minutes offset from UTC.

    • Daylight savings time (or summer time) is disabled by default. To set summer time hours, use the following command:

      clock summer-time zone recurring [week day monthhh:mm week day month \
hh:mm[offset]]

      In the preceding command, zone is the time zone to be displayed when summer time is in effect (EDT, for example), week is the week of the month (1 to 5 or last), day is the day of the week (Sunday, Monday, ...), month is the month (January, February, ...), hh:mm is the hours and minutes in 24-hour format, and offset is the number of minutes to add during summer time. The default offset is 60 minutes.

    • To manually set the clock to any time use the following command, where the time specified is relative to the configured time zone:

      clock set hh:mm:ss month day year

      In the preceding command, hh:mm:ss is the time in 24-hour format, day is the day by date in the month, month is the name of the month, and year is the 4-digit year.

    The ordering of commands is important when setting the local time and time zone. For example, to set the local time to US Eastern time:

    exa1sw-ip# configure terminal
Enter configuration commands,one per line. End with CNTL/Z.
exa1sw-ip(config)# clock timezone EST -5 
exa1sw-ip(config)# clock summer-time EDT recurring
exa1sw-ip(config)# end
exa1sw-ip# clock set 21:00:00 August 09 2018
exa1sw-ip# write memory
Building configuration...
Compressed configuration from 3784 bytes to 1465 bytes [OK ]
exa1sw-ip# show clock
21:00:06.643 EST Mon Aug 9 2018

  14. After setting the local time zone, you can configure up to two NTP servers. Replace the IP addresses used in this example with the values for the installation:

    exa1sw-ip# configure terminal
Enter configuration commands,one per line. End with CNTL/Z.
exa1sw-ip(config)# ntp server 10.7.7.32 prefer
exa1sw-ip(config)# ntp server 198.51.100.19
exa1sw-ip(config)# end
*Sep 15 14:51:08.665:%SYS-5-CONFIG_I:Configured from console by console
exa1sw-ip# write memory
Building configuration...
Compressed configuration from 2654 bytes to 1163 bytes [OK ]
exa1sw-ip# show ntp status
<output will vary per network>
     .
exa1sw-ip# show clock
21:00:23.175 EST Mon Aug 9 2018

    The NTP server is synchronized to local time when you connect the Cisco switch to the network and it has access to NTP.

    Symbols that precede the show clock display indicate that the time is the following:

    • * Not authoritative
    • . Authoritative, but NTP is not synchronized.
    • Authoritative (blank space).

  15. Verify the Ethernet configuration using the following command:

    exa1sw-ip# show running-config
Building configuration...
Current configuration : 3923 bytes
!
version 15.2
no service pad
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
     .
     .
     .

    Note:

    If any setting is incorrect, then repeat the appropriate step. To erase a setting, enter no in front of the same command. For example, to erase the default gateway, use the following commands: 

    exa1sw-ip#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
exa1sw-ip(config)# no ip default-gateway 10.7.7.1
exa1sw-ip(config)# end
exa1sw-ip#
*Sep 15 14:13:26.013: %SYS-5-CONFIG_I: Configured from console by console
exa1sw-ip(config)# write memory
Building configuration...
Compressed configuration from 2502 bytes to 1085 bytes[OK]

  16. Save the current configuration.

    exa1sw-ip#copy running-config startup-config
Destination filename [startup-config]?
Building configuration...
Compressed configuration from 2654 bytes to 1189 bytes[OK]

  17. Exit from the session using the following command:

    exa1sw-ip# exit

exa1sw-ip con0 is now available

Press RETURN to get started.

  18. Disconnect the cable from the Cisco console.

    The Cisco switch must not be connected to the management network at this stage. The switch will be connected later after Oracle has configured the systems with the necessary IP addresses and you have worked with the field service engineer to make any additional changes necessary for connecting to the network.

  19. To check the Cisco switch, attach a laptop computer to port 48, and ping the IP address of the internal management network to check the configuration.

Related Topics

Parent topic: Configuring the Management Network Switch

6.4.2 Configuring the Cisco Nexus 93108-1G or 9348 Ethernet Switch

The Cisco Nexus 93108-1G or 9348 Ethernet switch supplied with Oracle Exadata Rack is minimally configured during installation.

Note that the Cisco Nexus 93108-1G or 9348 switch supports multiple uplinks to the customer network by utilizing the QSFP+ ports. This is a more complicated switch setup due to the redundant connectivity, and should be performed by the customer's network administrator.

Whether you are configuring the switch for the first time, or configuring a replacement switch, use the following procedures:

Parent topic: Configuring the Management Network Switch

6.4.2.1 Performing the Initial Switch Configuration for the Cisco Nexus 93108-1G or 9348 Ethernet Switch

During the initial configuration, you reset the switch and use the Basic System Configuration Dialog to configure the switch.

Before configuring the switch, note the following:

  • The Cisco Ethernet switch should not be connected until the running configuration has been verified, and any necessary changes have been made by the network administrator.

  • The Cisco Ethernet switch should not be connected to the customer network until the IP addresses on all components have been configured in Oracle Exadata Rack. This is to prevent any duplicate IP address conflicts which are possible due to the default addresses set in the components when shipped.

Configuration should be done with the network administrator.

  1. Connect from the Cisco switch serial console to a laptop or similar device using the available RJ45 cable.
  2. Ensure the terminal session is recorded on the laptop by logging the output.
    The output can be used as a reference that the switch has been configured correctly. The default serial port speed is 9600 baud, 8 bits, no parity, 1 stop bit, and no handshake.
  3. Power on the switch.
  4. Log in as the admin user.
    User Access Verification
exadatax7-adm0 login: admin
Password: ********

    Note:

    If you do not have the password for the admin user, then contact Oracle Support Services.
  5. Erase the existing configuration.
    exadatax7-adm0# write erase

Warning: This command will erase the startup-configuration.

Do you wish to proceed anyway? (y/n)  [n] y
  6. Restart the system so you can perform the automated setup.
    exadatax7-adm0# reload

This command will reboot the system. (y/n)?  [n] y

2017 Aug 31 01:09:00 exadatax7-adm0 %$ VDC-1 %$ %PLATFORM-2-PFM_SYSTEM_RESET: Manual system restart from Command Line Interface


CISCO SWITCH Ver7.59
Device detected on 0:1:2 after 0 msecs  
...
  7. Switch to normal setup and, when asked if you want to enforce secure password standard, enter no, then enter a new password for the admin user.
    Running S93thirdparty-script...

Populating conf files for hybrid sysmgr ...
Starting hybrid sysmgr ...
inserting /isan/lib/modules/klm_cisco_nb.o ... done

Abort Auto Provisioning and continue with normal setup ? (yes/no) [n]: yes

         ---- System Admin Account Setup ----

Do you want to enforce secure password standard (yes/no) [y]: no

  Enter the password for "admin": 
  Confirm the password for "admin":
  8. When the Basic System Configuration Dialog appears, choose to enter the basic configuration dialog.
            ---- Basic System Configuration Dialog VDC: 1 ----

This setup utility will guide you through the basic configuration of
the system. Setup configures only enough connectivity for management
of the system.

Please register Cisco Nexus9000 Family devices promptly with your
supplier. Failure to register may affect response times for initial
service calls. Nexus9000 devices must be registered to receive 
entitled support services.

Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
to skip the remaining dialogs.

Would you like to enter the basic configuration dialog (yes/no): yes
  9. In the basic configuration, you can use the default inputs until asked to enter the switch name.

    In this example, the switch has a name of test123sw-adm0. 

      Create another login account (yes/no) [n]: 
  Configure read-only SNMP community string (yes/no) [n]: 
  Configure read-write SNMP community string (yes/no) [n]: 
  Enter the switch name : test123sw-adm0
  10. Respond no when asked to configure Out-of-band management configuration.
    Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: no
  11. Respond yes when asked to configure advanced IP options.
    Configure advanced IP options? (yes/no) [n]: yes
  12. Respond no when asked to configure static route (this will be configured later).
    Configure static route? (yes/no) [n]: no
  13. Enter the destination prefix and mask, and other values as prompted.
       Destination prefix : 10.100.100.0

   Destination prefix mask : 255.255.255.0

   Next hop IPv4 address : 10.100.100.1
  14. Skip configuring the DNS IPv4 addresses (this will be configured later).
    Configure the DNS IPv4 address? (yes/no) [n]: no
  15. Skip configuring the default domain name (this will be configured later).
    Configure the default domain name? (yes/no) [n]: no
  16. Accept the default responses until asked to configure SSH and the NTP server.
    Enable the telnet service? (yes/no) [n]: no
Enable the ssh service? (yes/no) [y]: yes
   Type of ssh key you would like to generate (dsa/rsa) [rsa]: rsa
   Number of rsa key bits <1024-2048> [1024]: 1024
 
Configure the ntp server? (yes/no) [n]: yes
     NTP server IPv4 address : 10.100.100.3
  17. Accept the default responses until asked to specify the CoPP system profile. Enter lenient.
     Configure default interface layer (L3/L2) [L2]: 
 Configure default switchport interface state (shut/noshut) [noshut]: 
 Configure CoPP system profile (strict/moderate/lenient/dense) [strict]: lenient
  18. After reviewing the configuration, save the configuration.
    The following configuration will be applied:
   no password strength-check
   switchname test123sw-adm0
  ...

Would you like to edit the configuration? (yes/no) [n]: 

Use this configuration and save it? (yes/no) [y]: yes

[########################################] 100%
Copy complete.
  19. Add the VLAN 1 IP address.
    test123sw-adm0(config)# feature interface-vlan
test123sw-adm0(config)# interface vlan 1
test123sw-adm0(config-if)# ip address 10.100.100.110/24
test123sw-adm0(config-if)# no shutdown
test123sw-adm0(config-if)# exit
  20. Set the spanning tree port type for ports 1-47.
    test123sw-adm0(config)# interface E1/1-47
test123sw-adm0(config-if)# spanning-tree port type edge
test123sw-adm0(config-if)# exit
  21. Set switchport on all 48 ports and set port 48 to a network port (instead of a host port).
    test123sw-adm0(config)# interface E1/1-48
test123sw-adm0(config-if)# switchport
test123sw-adm0(config-if)# exit
test123sw-adm0(config)# interface E1/48
test123sw-adm0(config-if)# spanning-tree port type network
test123sw-adm0(config-if)# ip route 0.0.0.0/0 10.100.100.1
  22. Configure the DNS information.
    test123sw-adm0(config)# ip domain-name example.com
test123sw-adm0(config)# ip name-server 10.100.100.2
test123sw-adm0(config)# exit
  23. Save the current configuration.
    test123sw-adm0# copy running-config startup-config
[########################################] 100%
Copy complete.
  24. Optional: Set the clock, as described in the next topic.
6.4.2.2 Setting the Clock on the Cisco 93108-1G or 9348 Ethernet Switch

After you have performed the initial configuration, you can adjust the time used by the switch.

  1. Log in as the admin user.
  2. View the current time.
    test123sw-adm0(config)# show clock
20:44:52.986 UTC Thu Aug 31 2017
Time source is NTP
  3. Set the timezone appropriately.
    test123sw-adm0(config)# clock timezone PST -8 0
  4. View the modified time.
    test123sw-adm0(config)# show clock
12:46:22.692 PST Thu Aug 31 2017
Time source is NTP
  5. Save the configuration.
    test123sw-adm0# copy running-config startup-config 
[########################################] 100%
Copy complete.

6.5 Configuring the Power Distribution Units

The power distribution units (PDUs) are configured with a static IP address to connect to the network for monitoring.

Ensure you have the following before connecting the PDU to the network:

  • Static IP address
  • Subnet mask
  • Default gateway
  • Ethernet cables
  • Laptop computer with either Oracle Solaris or Microsoft Windows

For systems less than full racks, the PDU Ethernet connections may use available ports in the Management Network Switch. Note that for full rack systems, you cannot connect PDUs to the Management Network Switch because all the ports on the switch are already reserved for other components.

The following procedure describes how to configure and connect the PDU to the network:

  1. Power off the PDU.

  2. Disconnect the power leads from the power source.

  3. Select an unused LAN network connection from the available network connections on the laptop as follows:

    • For Oracle Solaris:

      1. Log in as a super user.

      2. Use the dladm show-link command to find an unused Ethernet interface.

      3. Use the ifconfig -a command to determine which interface is being used.

      4. Use the following command to plumb the unused interface:

        # ifconfig interface plumb up

        In the preceding command, interface is the network interface determined in step 3.c.

      5. Using the following command to assign an IPv4 address and netmask to the interface:

        # ifconfig interface IPv4_address netmask + netmask \
  broadcast broadcast up

        In the preceding command, interface is the network interface, IPv4_address is the IP address, netmask is the netmask address, and broadcast is the broadcast address.

        Note:

        The network interfaces configured with the ifconfig command do not persist across system restarts.

    • For Microsoft Windows:

      1. Select Settings from the Start menu.

      2. Select Network Connections from the Settings menu.

      3. Select an unused local area connection from the Network Connections menu.

      4. Double-click Internet Protocol (TCP/IP) in the Local Area Connection Properties window. The Internet Protocol (TCP/IP) window appears.

      5. Select Use the following IP address.

      6. Enter the IP address in the IP address field. Do not use 192.168.0.1 because that is the default PDU metering unit address.

      7. Enter the IP address in the Subnet mask field.

      8. Leave the Default gateway field blank.

      9. Click OK.

  4. Connect the PDU and the laptop to two ports on the Management Network Switch. The equipment currently using the Management Network Switch must be disconnected temporarily.

  5. Connect the PDU power input lead to the power source. Only one PDU power input lead needs to be connected, as follows:

    • On PDUs with one power input lead, connect the one lead to the power source.

    • On PDUs with two power leads, connect the first power lead to the power source. The first power lead is labeled 0.

    • On PDUs with three power input leads, connect the middle power lead to the power source. The middle power lead is labeled 1.

  6. Use a Web browser to access the PDU metering unit by entering the factory default IP address for the unit, 192.168.0.1, in the address line of the browser. The Current Measurement page should appear.

  7. Click Network Configuration in the upper left of the page.

  8. Log in as the admin user on the PDU metering unit. Change the password after configuring the network.

  9. Confirm the DHCP Enable check box is not selected.

  10. Enter the following network settings for the PDU metering unit:

    • IP address
    • Subnet mask address
    • Default gateway

  11. Click Submit to set the network settings, and reset the PDU metering unit.

  12. Disconnect the PDU and laptop from the Management Network Switch.

  13. Reconnect the two cables that were originally connected to the Management Network Switch.

  14. Connect an Ethernet cable to the PDU metering unit RJ-45 Ethernet port and to the network.

  15. Log in to the PDU metering unit using a Web browser. Use the PDU metering unit's static IP address in the browser's address line. If the configuration was successful, then the Current Measurement page is displayed.

  16. Connect the remaining PDU power input leads to the power source.

Parent topic: Configuring Oracle Exadata Database Machine

6.6 Configuring the Threshold Settings for the Power Distribution Units

The PDU current can be monitored directly or with Oracle Enterprise Manager Grid Control. Configure the threshold settings to monitor the PDUs. The threshold settings depend on the size of Oracle Exadata Rack and type PDU. The configurable threshold values for each metering unit module and phase are Info low, Pre Warning, and Alarm.

Note:

The thresholds must be set in the Param Configuration section of the PDU metering unit. Setting threshold values using the Oracle Enterprise Manager Grid Control plug-in has no effect.

See Also:

Parent topic: Configuring Oracle Exadata Database Machine

6.6.1 PDU Thresholds for Oracle Exadata Database Machine X5-2 and Later

Starting with Oracle Exadata Database Machine X5-2, Oracle Exadata Configuration Assistant (OECA) is the only source for PDU thresholds. OECA is available on Oracle Technology Network at http://www.oracle.com/technetwork/database/exadata/oeca-download-2817713.html

If for some reason you are unable to utilize OECA, contact Oracle Support.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.2 PDU Thresholds for Oracle Exadata Database Machine X4-2

This section contains the PDU thresholds for Oracle Exadata Database Machine X4-2.

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA is available on Oracle Technology Network.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.2.1 PDU Thresholds for Oracle Exadata Database Machine X4-2 Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X4-2 Full Rack:

  • Table 6-1 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, Low-voltage PDU

  • Table 6-2 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, Low-voltage PDU

  • Table 6-3 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, High-voltage PDU

  • Table 6-4 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, High-voltage PDU

Table 6-1 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a single-phase, low-voltage PDU.

Table 6-1 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

17

22

A

Module 1, phase 3

0

19

24

B

Module 1, phase 1

0

19

24

B

Module 1, phase 2

0

17

22

B

Module 1, phase 3

0

20

24

Table 6-2 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a three-phase, low-voltage PDU.

Table 6-2 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

42

A and B

Module 1, phase 3

0

32

40

Table 6-3 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a single-phase, high-voltage PDU.

Table 6-3 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

16

20

A

Module 1, phase 3

0

18

23

B

Module 1, phase 1

0

18

23

B

Module 1, phase 2

0

16

20

B

Module 1, phase 3

0

18

23

Table 6-4 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a three-phase, high-voltage PDU.

Table 6-4 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

16

21
6.6.2.2 PDU Thresholds for Oracle Exadata Database Machine X4-2 Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine X4-2 Half Rack:

  • Table 6-5 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, Low-voltage PDU

  • Table 6-6 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, Low-voltage PDU

  • Table 6-7 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, High-voltage PDU

  • Table 6-8 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, High-voltage PDU

Table 6-5 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a single-phase, low-voltage PDU.

Table 6-5 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

10

13

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

10

13

B

Module 1, phase 3

0

20

24

Table 6-6 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a three-phase, low-voltage PDU.

Table 6-6 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

17

22

A and B

Module 1, phase 2

0

19

24

A and B

Module 1, phase 3

0

15

20

Table 6-7 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a single-phase, high-voltage PDU.

Table 6-7 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

9

12

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

9

12

B

Module 1, phase 3

0

18

23

Table 6-8 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a three-phase, high-voltage PDU.

Table 6-8 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

15

A and B

Module 1, phase 2

0

9

12

A and B

Module 1, phase 3

0

7

10
6.6.2.3 PDU Thresholds for Oracle Exadata Database Machine X4-2 Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack:

  • Table 6-9 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, Low-voltage PDU

  • Table 6-10 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, Low-voltage PDU

  • Table 6-11 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, High-voltage PDU

  • Table 6-12 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, High-voltage PDU

Table 6-9 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a single-phase, low-voltage PDU.

Table 6-9 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

10

13

Table 6-10 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a three-phase, low-voltage PDU.

Table 6-10 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

10

13

A and B

Module 1, phase 3

0

5

7

Table 6-11 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a single-phase, high-voltage PDU.

Table 6-11 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

12

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

9

12

Table 6-12 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a three-phase, high-voltage PDU.

Table 6-12 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

11

A and B

Module 1, phase 2

0

3

4

A and B

Module 1, phase 3

0

3

4

6.6.3 PDU Thresholds for Oracle Exadata Database Machine X3-2

This section contains the PDU thresholds for Oracle Exadata Database Machine X3-2.

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA is available on Oracle Technology Network.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.3.1 PDU Thresholds for Oracle Exadata Database Machine X3-2 Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X3-2 Full Rack:

  • Table 6-13 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, Low-voltage PDU

  • Table 6-14 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, Low-voltage PDU

  • Table 6-15 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, High-voltage PDU

  • Table 6-16

    Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, High-voltage PDU

Table 6-13 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a single-phase, low-voltage PDU.

Table 6-13 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

14

18

A

Module 1, phase 3

0

19

24

B

Module 1, phase 1

0

19

24

B

Module 1, phase 2

0

14

18

B

Module 1, phase 3

0

20

24

Table 6-14 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a three-phase, low-voltage PDU.

Table 6-14 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

30

38

A and B

Module 1, phase 2

0

32

40

A and B

Module 1, phase 3

0

30

38

Table 6-15 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a single-phase, high-voltage PDU.

Table 6-15 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

18

22

B

Module 1, phase 1

0

18

22

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

18

23

Table 6-16 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a three-phase, high-voltage PDU.

Table 6-16 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

17

21

A and B

Module 1, phase 2

0

17

21

A and B

Module 1, phase 3

0

15

19
6.6.3.2 PDU Thresholds for Oracle Exadata Database Machine X3-2 Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine X3-2 Half Rack:

  • Table 6-17 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, Low-voltage PDU

  • Table 6-18 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, Low-voltage PDU

  • Table 6-19 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, High-voltage PDU

  • Table 6-20 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, High-voltage PDU

Table 6-17 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a single-phase, low-voltage PDU.

Table 6-17 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

9

11

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

9

11

B

Module 1, phase 3

0

20

24

Table 6-18 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a three-phase, low-voltage PDU.

Table 6-18 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

16

21

A and B

Module 1, phase 2

0

19

24

A and B

Module 1, phase 3

0

14

18

Table 6-19 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a single-phase, high-voltage PDU.

Table 6-19 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

8

10

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

8

10

B

Module 1, phase 3

0

18

23

Table 6-20 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a three-phase, high-voltage PDU.

Table 6-20 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

15

A and B

Module 1, phase 2

0

9

11

A and B

Module 1, phase 3

0

6

8
6.6.3.3 PDU Thresholds for Oracle Exadata Database Machine X3-2 Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack:

  • Table 6-21 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, Low-voltage PDU

  • Table 6-22 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, Low-voltage PDU

  • Table 6-23 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, High-voltage PDU

  • Table 6-24 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, High-voltage PDU

Table 6-21 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a single-phase, low-voltage PDU.

Table 6-21 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

5

6

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

6

B

Module 1, phase 3

0

10

13

Table 6-22 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a three-phase, low-voltage PDU.

Table 6-22 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

10

13

A and B

Module 1, phase 3

0

4

6

Table 6-23 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a single-phase, high-voltage PDU.

Table 6-23 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

11

A

Module 1, phase 2

0

4

6

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

4

6

B

Module 1, phase 3

0

9

11

Table 6-24 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a three-phase, high-voltage PDU.

Table 6-24 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

2

3

A and B

Module 1, phase 3

0

3

4

6.6.4 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers)

This section contains the PDU thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers).

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA is available on Oracle Technology Network.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.4.1 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack:

  • Table 6-25 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, Low-voltage PDU

  • Table 6-26 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, Low-voltage PDU

  • Table 6-27 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, High-voltage PDU

  • Table 6-28 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, High-voltage PDU

Table 6-25 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, low-voltage PDU.

Table 6-25 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

22

24

A

Module 1, phase 3

0

18

23

B

Module 1, phase 1

0

18

23

B

Module 1, phase 2

0

22

24

B

Module 1, phase 3

0

18

23

Table 6-26 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, low-voltage PDU.

Table 6-26 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

43

A and B

Module 1, phase 3

0

33

42

Table 6-27 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, high-voltage PDU.

Table 6-27 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

20

21

A

Module 1, phase 3

0

16

20

B

Module 1, phase 1

0

16

20

B

Module 1, phase 2

0

20

21

B

Module 1, phase 3

0

16

20

Table 6-28 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, high-voltage PDU.

Table 6-28 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

17

21
6.6.4.2 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine Half Rack:

  • Table 6-29 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, Low-voltage PDU

  • Table 6-30 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, Low-voltage PDU

  • Table 6-31 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, High-voltage PDU

  • Table 6-32 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, High-voltage PDU

Table 6-29 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, low-voltage PDU.

Table 6-29 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

18

23

Table 6-30 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, low-voltage PDU.

Table 6-30 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

23

A and B

Module 1, phase 2

0

17

22

A and B

Module 1, phase 3

0

18

23

Table 6-31 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, high-voltage PDU.

Table 6-31 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

12

15

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

12

15

B

Module 1, phase 3

0

16

20

Table 6-32 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, high-voltage PDU.

Table 6-32 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

9

12

A and B

Module 1, phase 3

0

10

13
6.6.4.3 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine Quarter Rack:

  • Table 6-33 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, Low-voltage PDU

  • Table 6-34 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, Low-voltage PDU

  • Table 6-35 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, High-voltage PDU

  • Table 6-36 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, High-voltage PDU

Table 6-33 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, low-voltage PDU.

Table 6-33 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

6

8

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

6

8

B

Module 1, phase 3

0

10

13

Table 6-34 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, low-voltage PDU.

Table 6-34 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

12

15

A and B

Module 1, phase 2

0

11

14

A and B

Module 1, phase 3

0

5

7

Table 6-35 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, high-voltage PDU.

Table 6-35 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

12

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

9

12

Table 6-36 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, high-voltage PDU.

Table 6-36 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

3

4

A and B

Module 1, phase 3

0

4

5

6.6.5 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers)

This section contains the PDU thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers).

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA can be downloaded from Oracle Technology Network.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.5.1 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack:

  • Table 6-37 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, Low-voltage PDU

  • Table 6-38 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, Low-voltage PDU

  • Table 6-39 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, High-voltage PDU

  • Table 6-40 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, High-voltage PDU

Table 6-37 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, low-voltage PDU.

Table 6-37 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

22

24

A

Module 1, phase 2

0

22

24

A

Module 1, phase 3

0

21

24

B

Module 1, phase 1

0

21

24

B

Module 1, phase 2

0

22

24

B

Module 1, phase 3

0

22

24

Table 6-38 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, low-voltage PDU.

Table 6-38 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

36

45

A and B

Module 1, phase 2

0

39

44

A and B

Module 1, phase 3

0

38

45

Table 6-39 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, high-voltage PDU.

Table 6-39 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

21

A

Module 1, phase 2

0

20

21

A

Module 1, phase 3

0

19

21

B

Module 1, phase 1

0

19

21

B

Module 1, phase 2

0

20

21

B

Module 1, phase 3

0

20

21

Table 6-40 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, high-voltage PDU.

Table 6-40 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

20

21

A and B

Module 1, phase 2

0

21

21

A and B

Module 1, phase 3

0

18

21
6.6.5.2 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine Half Rack:

  • Table 6-41 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, Low-voltage PDU

  • Table 6-42 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, Low-voltage PDU

  • Table 6-43 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, High-voltage PDU

  • Table 6-44 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, High-voltage PDU

Table 6-41 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, low-voltage PDU.

Table 6-41 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

21

24

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

21

24

Table 6-42 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, low-voltage PDU.

Table 6-42 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

19

24

A and B

Module 1, phase 2

0

20

25

A and B

Module 1, phase 3

0

19

24

Table 6-43 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, high-voltage PDU.

Table 6-43 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

19

21

A

Module 1, phase 2

0

12

15

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

12

15

B

Module 1, phase 3

0

19

21

Table 6-44 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, high-voltage PDU.

Table 6-44 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

11

14

A and B

Module 1, phase 3

0

10

13
6.6.5.3 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine Quarter Rack:

  • Table 6-45

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, Low-voltage PDU

  • Table 6-46

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, Low-voltage PDU

  • Table 6-47

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, High-voltage PDU

  • Table 6-48

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, High-voltage PDU

Table 6-45 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, low-voltage PDU.

Table 6-45 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

12

15

A

Module 1, phase 2

0

6

8

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

6

8

B

Module 1, phase 3

0

12

15

Table 6-46 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, low-voltage PDU.

Table 6-46 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

13

17

A and B

Module 1, phase 2

0

12

15

A and B

Module 1, phase 3

0

5

7

Table 6-47 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, high-voltage PDU.

Table 6-47 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

11

14

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

11

14

Table 6-48 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, high-voltage PDU.

Table 6-48 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

2

3

A and B

Module 1, phase 3

0

3

4

6.6.6 PDU Thresholds for Oracle Exadata Database Machine X4-8 and Later

This section contains the PDU thresholds for Oracle Exadata Database Machine Eight-Socket systems for X4-8 and later.

Oracle Exadata Configuration Assistant (OECA) is the only source for PDU thresholds for Oracle Exadata Database Machine X4-8 and later systems. OECA is available on Oracle Technology Network at http://www.oracle.com/technetwork/database/exadata/oeca-download-2817713.html

If for some reason you are unable to utilize OECA, contact Oracle Support.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.7 PDU Thresholds for Oracle Exadata Database Machine X3-8 Full Rack

This section contains the PDU thresholds for Oracle Exadata Database Machine X3-8 Full Rack.

  • Table 6-49 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

  • Table 6-50 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

  • Table 6-51 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

  • Table 6-52 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

Table 6-49 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a single-phase, low-voltage 22 kVA PDU.

Table 6-49 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

25

31

A

Module 1, phase 2

0

26

33

A

Module 1, phase 3

0

24

30

B

Module 1, phase 1

0

24

30

B

Module 1, phase 2

0

26

33

B

Module 1, phase 3

0

25

31

Table 6-50 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a three-phase, low-voltage 24 kVA PDU.

Table 6-50 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

21

27

A

Module 1, phase 2

0

23

30

A

Module 1, phase 3

0

22

28

A

Module 2, phase 1

0

20

25

A

Module 2, phase 2

0

21

27

A

Module 2, phase 3

0

23

29

B

Module 1, phase 1

0

20

25

B

Module 1, phase 2

0

21

27

B

Module 1, phase 3

0

23

29

B

Module 2, phase 1

0

21

27

B

Module 2, phase 2

0

23

30

B

Module 2, phase 3

0

22

28

Table 6-51 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a single-phase, high-voltage 22 kVA PDU.

Table 6-51 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

22

28

A and B

Module 1, phase 2

0

24

30

A and B

Module 1, phase 3

0

22

28

Table 6-52 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a three-phase, high-voltage 24 kVA PDU.

Table 6-52 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

12

16

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

10

13

A

Module 2, phase 1

0

10

13

A

Module 2, phase 2

0

13

17

A

Module 2, phase 3

0

11

15

B

Module 1, phase 1

0

10

13

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

11

15

B

Module 2, phase 1

0

12

16

B

Module 2, phase 2

0

13

17

B

Module 2, phase 3

0

10

13

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.8 PDU Thresholds for Oracle Exadata Database Machine X2-8 Full Rack

This section contains the PDU thresholds for Oracle Exadata Database Machine X2-8 Full Rack.

  • Table 6-53 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

  • Table 6-54 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

  • Table 6-55 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

  • Table 6-56 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

Table 6-53 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a single-phase, low-voltage 22 kVA PDU.

Table 6-53 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

36

37

A

Module 1, phase 2

0

35

37

A

Module 1, phase 3

0

35

37

B

Module 1, phase 1

0

35

37

B

Module 1, phase 2

0

35

37

B

Module 1, phase 3

0

36

37

Table 6-54 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a three-phase, low-voltage 24 kVA PDU.

Table 6-54 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

43

A and B

Module 1, phase 3

0

33

42

Table 6-55 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a single-phase, high-voltage 22 kVA PDU.

Table 6-55 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

26

32

A and B

Module 1, phase 2

0

25

32

A and B

Module 1, phase 3

0

26

32

Table 6-56 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a three-phase, high-voltage 24 kVA PDU.

Table 6-56 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

17

21

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.9 PDU Thresholds for Oracle Exadata Storage Expansion Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

This section contains the PDU thresholds for Oracle Exadata Storage Expansion Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers.

Parent topic: Configuring the Threshold Settings for the Power Distribution Units

6.6.9.1 PDU Thresholds for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

The following tables list the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers Full Rack:

  • Table 6-57 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

  • Table 6-58 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

  • Table 6-59 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

  • Table 6-60 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

Table 6-57 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, low-voltage PDU.

Table 6-57 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

22

24

A

Module 1, phase 3

0

18

23

B

Module 1, phase 1

0

18

23

B

Module 1, phase 2

0

22

24

B

Module 1, phase 3

0

18

23

Table 6-58 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, low-voltage PDU.

Table 6-58 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

43

A and B

Module 1, phase 3

0

33

42

Table 6-59 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, high-voltage PDU.

Table 6-59 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

20

21

A

Module 1, phase 3

0

16

20

B

Module 1, phase 1

0

16

20

B

Module 1, phase 2

0

20

21

B

Module 1, phase 3

0

16

20

Table 6-60 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, high-voltage PDU.

Table 6-60 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

17

21
6.6.9.2 PDU Thresholds for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

The following tables list the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers:

  • Table 6-61 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

  • Table 6-62 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

  • Table 6-63 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

  • Table 6-64 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

Table 6-61 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, low-voltage PDU.

Table 6-61 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

18

23

Table 6-62 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, low-voltage PDU.

Table 6-62 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

23

A and B

Module 1, phase 2

0

17

22

A and B

Module 1, phase 3

0

18

23

Table 6-63 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, high-voltage PDU.

Table 6-63 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

12

15

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

12

15

B

Module 1, phase 3

0

16

20

Table 6-64 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, high-voltage PDU.

Table 6-64 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

9

12

A and B

Module 1, phase 3

0

10

13
6.6.9.3 PDU Thresholds for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

The following tables list the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers:

  • Table 6-65 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

  • Table 6-66 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

  • Table 6-67 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

  • Table 6-68 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

Table 6-65 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, low-voltage PDU.

Table 6-65 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

6

8

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

6

8

B

Module 1, phase 3

0

10

13

Table 6-66 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, low-voltage PDU.

Table 6-66 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

12

15

A and B

Module 1, phase 2

0

11

14

A and B

Module 1, phase 3

0

5

7

Table 6-67 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, high-voltage PDU.

Table 6-67 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

12

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

9

12

Table 6-68 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, high-voltage PDU.

Table 6-68 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

3

4

A and B

Module 1, phase 3

0

4

5

6.7 Checking Exadata Storage Servers

After powering up the rack, check the Oracle Exadata Storage Servers.

Perform the following checks on each Exadata Storage Server:

  1. Connect to the storage server console.

    Use one of the following connection options:

    • Attach a serial cable between a console and the SER MGT port on the server.

      Connections through the modular jack serial port must use shielded cables to comply with radio frequency emission limits.

      Set the terminal device communication properties to these values: 9600 baud, 8 bit, no parity, 1 stop bit.

      After you connect to ILOM, run the start /SP/console command to connect to the host console.

      For more details, see Connect to Oracle ILOM.

    • Attach an Ethernet cable to the NET MGT port on the server and then connect using a workstation on the network.

      After you connect to ILOM, run the start /SP/console command to connect to the host console.

      For more details, see Connect to Oracle ILOM.

    • Connect using the KVM switch, if one is available.

  2. Log in to the storage server as the root user.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  3. Verify the SysSN label for the storage server matches the product_serial_number value using the following command: 

    # ipmitool sunoem cli "show /SYS product_serial_number"
Connected.Use ^ D to exit.
->show /SYS product_serial_number
/SYS
Properties:
product_serial_number =0937XFG03B
->Session closed
Disconnected

  4. Check the hardware and firmware using the following command:

    # /opt/oracle.SupportTools/CheckHWnFWProfile

    If output does not show success, then examine the output for mismatched hardware or firmware and take steps to correct those as explained in the following steps.

  5. Verify the disks on the server are visible and online using the following command. The disks are numbered slot 0 to slot 11.

    • If you are using Oracle Exadata System Software release 19c or later, then get the enclosure ID: 

      # cd /opt/MegaRAID/storcli
# ./storcli64 /c0/eall show
Controller = 0
Status = Success
Description = None

Properties :
==========

-----------------------------------------------------------------------------------------
EID State Slots PD PS Fans TSs Alms SIM Port#                      ProdID VendorSpecific
-----------------------------------------------------------------------------------------
  8 OK       14 12  0    0   0    0   0 Port 0 - 3 & Port 4 - 7 x8 X5-2L  SPMS   v`
252 OK        8  0  0    0   0    0   1 Internal                   SGPIO        
-----------------------------------------------------------------------------------------

EID-Enclosure Device ID |PD-Physical drive count |PS-Power Supply count|
TSs-Temperature sensor count |Alms-Alarm count |SIM-SIM Count

      Then show the state of the physical devices in that enclosure:

      # cd /opt/MegaRAID/storcli
# ./storcli64 /c0/e8/sall show all | grep "8:"
8:0      16 Onln   0 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:1      17 Onln   1 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:2      20 Onln   2 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:3      15 Onln   3 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:4      12 Onln   4 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:5      18 Onln   5 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:6      11 Onln   6 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:7       9 Onln   7 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:8      14 Onln   8 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:9      21 Onln  11 7.151 TB SAS  HDD N   N  512B H7280B520SUN8.0T U
8:10     13 Onln   9 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U
8:11     10 Onln  10 7.151 TB SAS  HDD N   N  512B H7280A520SUN8.0T U

      The rows displayed are from a table that displays the following information:

      
EID:Slt DID State DG     Size Intf Med SED PI SeSz Model            Sp
-----------------------------------------------------------------------
...
EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded|Cpybck-CopyBack|CBShld-Copyback Shielded

    • If you are using Oracle Exadata System Software release 18c or earlier, then use the following command: 

      # cd /opt/MegaRAID/MegaCli
# ./MegaCli64 -Pdlist -aAll |grep "Slot \|Firmware"
Slot Number:0
Firmware state:Online, Spun Up
Slot Number:1
Firmware state:Online, Spun Up
Slot Number:2
Firmware state:Online, Spun Up
Slot Number:3
Firmware state:Online, Spun Up
Slot Number:4
Firmware state:Online, Spun Up
Slot Number:5
Firmware state:Online, Spun Up
Slot Number:6
Firmware state:Online, Spun Up
Slot Number:7
Firmware state:Online, Spun Up
Slot Number:8
Firmware state:Online, Spun Up
Slot Number:9
Firmware state:Online, Spun Up
Slot Number:10
Firmware state:Online, Spun Up
Slot Number:11
Firmware state:Online, Spun Up

  6. If applicable, verify the PMEM modules are visible. 

    # cellcli -e list physicaldisk where disktype=PMEM

  7. Verify the flash modules are visible. The number of cards and modules varies depending on the model.

    # cellcli -e list physicaldisk where disktype=flashdisk

    If any flash modules are missing, then power cycle the system, and wait 10 minutes after restarting before checking again. If this does not resolve the problem, then re-seat the module and flash card. If the problem continues, then replace the card with the on-site spare.

  8. Check the rear of the rack to ensure the flash card LEDs are green all the way across. If any LED is amber, then check again after the next step. If the LED is still amber after restart, then re-seat the card and ESM or flash module. If the problem continues, then replace the card with the on-site spare.

    Note:

    This step can be done when all systems are on. It does not have to be done on a per-server basis.

  9. Log out of the system using the following command:

    # logout

Parent topic: Configuring Oracle Exadata Database Machine

6.8 Checking Oracle Exadata Database Servers

Before installing the software, check the configuration of the Oracle Exadata Database Servers.

Perform the following checks on each Exadata database server:

  1. Connect to the database server console.

    Use one of the following connection options:

    • Attach a serial cable between a console and the SER MGT port on the server.

      Connections through the modular jack serial port must use shielded cables to comply with radio frequency emission limits.

      Set the terminal device communication properties to these values: 9600 baud, 8 bit, no parity, 1 stop bit.

      After you connect to ILOM, run the start /SP/console command to connect to the host console.

      For more details, see Connect to Oracle ILOM.

    • Attach an Ethernet cable to the NET MGT port on the server and then connect using a workstation on the network.

      After you connect to ILOM, run the start /SP/console command to connect to the host console.

      For more details, see Connect to Oracle ILOM.

    • Connect using the KVM switch, if one is available.

  2. Log in as the root user to the database server.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  3. Verify the SysSN label for the server matches the product_serial_number value using the following command: 

    # ipmitool sunoem cli "show /SYS product_serial_number"
Connected.Use ^ D to exit.
->show /SYS product_serial_number

/SYS
   Properties: 
       product_serial_number =093ABCD5001

->Session closed
Disconnected

  4. Change to the /opt/oracle.SupportTools/firstconf directory. 

    cd /opt/oracle.SupportTools/firstconf

  5. Create a group file with the list of server IP addresses to use in the dcli utility commands, if required.

    If you are configuring Oracle Exadata Rack X8M, then you must manually create a file and add the IP addresses.

    On racks that use InfiniBand Network Fabric, you can use the following command to identify the servers and copy the results into a text file. 

    # ibhosts | awk '/elastic/ {print $8}' | cut -c1-14 > group_file

    In the preceding command, group_file is name of the file to create with the server IP addresses.

  6. Verify the hardware profile is correct using the following command. The command checks CPU type and count, component firmware, and so on.

    # dcli -l root -g group_file "/opt/oracle.SupportTools/CheckHWnFWProfile  \
-c  strict" > /tmp/checkhwfw.out

# more /tmp/checkhwfw.out

    The following is an example of a check that failed:

    172.108.1.6:[WARNING ]The hardware and firmware are not supported.
See details below
[DiskControllerPCIeSlotWidth ]
Requires:
x8
Found:
x4
[WARNING ]The hardware and firmware are not supported. See details above

    If a profile check fails and the firmware needs to be refreshed, then run the following command from the system with the problem.

    # /opt/oracle.SupportTools/CheckHWnFWProfile -U /opt/oracle.cellos/iso/cellbits
Now updating the ILOM and the BIOS …

    The command must be entered on a single line. After running the command, power off the system, reset the Integrated Lights Out Manager (ILOM), and wait 10 minutes before powering on the system.

  7. Verify the disks on the server are visible and online.

    • If you are using Oracle Exadata System Software release 19c or later, then use the following command: 

      # cd /opt/MegaRAID/storcli
# ./storcli64 /call show | grep "Physical Drives" -A 25
Physical Drives = 8

PD LIST :
=======

------------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model            Sp Type
------------------------------------------------------------------------------
39:0     31 Onln   0 278.464 GB SAS  HDD N   N  512B ST930003SSUN300G U  -
39:1     35 Onln   0 278.464 GB SAS  HDD N   N  512B ST930003SSUN300G U  -
39:2     41 Onln   0 278.464 GB SAS  HDD N   N  512B H103030SCSUN300G U  -
39:3     37 Onln   0 278.464 GB SAS  HDD N   N  512B ST930003SSUN300G U  -
39:4     38 Onln   0 278.464 GB SAS  HDD N   N  512B ST930003SSUN300G U  -
39:5     44 Onln   0 278.464 GB SAS  HDD N   N  512B H109030SESUN300G U  -
39:6     33 Onln   0 278.464 GB SAS  HDD N   N  512B ST930003SSUN300G U  -
39:7     43 Onln   0 278.464 GB SAS  HDD N   N  512B ST930003SSUN300G U  -
------------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded|Cpybck-CopyBack|CBShld-Copyback Shielded

    • If you are using Oracle Exadata System Software release 18c or earlier, then use the following command: 

      # cd /opt/MegaRAID/MegaCli
# ./MegaCli64 -Pdlist -aAll |grep "Slot \|Firmware"
Slot Number:0
Firmware state:Online, Spun Up
Slot Number:1
Firmware state:Online, Spun Up
Slot Number:2
Firmware state:Online, Spun Up
Slot Number:3
Firmware state:Online, Spun Up
Slot Number:4
Firmware state:Online, Spun Up
Slot Number:5
Firmware state:Online, Spun Up
Slot Number:6
Firmware state:Online, Spun Up
Slot Number:7
Firmware state:Online, Spun Up
Slot Number:8
Firmware state:Online, Spun Up
Slot Number:9
Firmware state:Online, Spun Up
Slot Number:10
Firmware state:Online, Spun Up
Slot Number:11
Firmware state:Online, Spun Up

  8. Examine the output of the following command:

    • If you are using Oracle Exadata System Software release 19c or later, then use the following command: 

      /opt/MegaRAID/storcli/storcli64 -LdInfo -LAll -aAll

    • If you are using Oracle Exadata System Software release 18c or earlier, then you can use the following command: 

      /opt/MegaRAID/MegaCli/MegaCli64 -LdInfo -LAll -aAll

  9. (Oracle Exadata Database Machine based on Sun Fire X4170 Oracle Database Servers only) Check that the ILOM is configured for the proper fan cooling on each database server using the following command: 

    # ipmitool sunoem cli "show /SP/policy"

    • If the policy is present and enabled, then no additional steps are needed.

    • If the policy is present and disabled, then enable the policy using the following command:

      # ipmitool sunoem cli "set /SP/policy FLASH_ACCELERATOR_CARD_ \
INSTALLED=enabled"

      The command must be entered as a single line.

  10. Verify that NTP servers are configured the same for all servers using the following command. The command is run on each database server individually.

    # ntpq -p

  11. Log out of the system using the following command:

    # logout

Parent topic: Configuring Oracle Exadata Database Machine

6.9 Performing Additional Checks and Configuration

Perform these additional checks and steps to verify the configuration of your Oracle Exadata Rack.

  1. (Oracle Exadata only) Log in to the first database server as the root user.

  2. Change to the firstconf directory using the following command: 

    # cd /opt/oracle.SupportTools/firstconf

  3. Create a group file with the list of IP addresses to use in the dcli utility commands, if you want to query multiple racks.

  4. Verify the rack master serial number is set correctly using the following command:

    # dcli -l root -g group_file_name "ipmitool sunoem cli 'show   \
/SP system_identifier'" | grep "system_identifier =" > /tmp/show-rack-msn.out

# more /tmp/show-rack-msn.out

    If none of the serial numbers are set, then contact Oracle Support Services. If one serial number is not set correctly, then run the following command to correct it, where rack_type is the rack designation, such as X8-8 and the string serial_number represents the serial number: 

    # ipmitool sunoem cli 'set /SP system_identifier= \
"Exadata Database Machine rack_type serial_number"'

  5. If you are running an Oracle Exadata System Software release earlier than 18.1, then ensure disk cache policy is set to Disabled using the following command as the root user on the first database server in the cluster: 

    # dcli -g /opt/oracle.SupportTools/onecommand/all_group -l root \
/opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -i 'Disk Cache Policy'

    The following is an example of the output from the command:

    dm01db01: Disk Cache Policy   : Disabled
dm01db02: Disk Cache Policy   : Disabled
dm01db03: Disk Cache Policy   : Disabled
...
dm01cel01: Disk Cache Policy   : Disabled
dm01cel02: Disk Cache Policy   : Disabled
...

    If any server shows different output, then identify the LUN and use the following command to reset the policy to Disabled, where LUNn is the LUN number, such as L2: 

    # MegaCli64 -LDSetProp -DisDskCache -LUNn -a0

Parent topic: Configuring Oracle Exadata Database Machine

6.10 Verifying the RoCE Network Fabric Configuration

This procedure describes how to verify the RoCE Network Fabric configuration.

  1. Verify the proper oracle-rdma-release software versions are being used on the database servers. 
    [root@dbm01adm08 ~]# rpm -qa |grep oracle-rdma-release
oracle-rdma-release-0.11.0-1.el7ora.x86_64

    The oracle-rdma-release software and adapter firmware versions are automatically maintained on the Oracle Exadata storage servers.

  2. Check the adapter firmware versions on the database servers.

    Use the CheckHWnFWProfile script to check firmware versions for the RDMA Network Fabric adapters. 

    # /opt/oracle.SupportTools/CheckHWnFWProfile -action list
  3. Visually check all the RDMA Network Fabric cable connections within the rack.
    The port lights should be on, and the LEDs should be on. Do not press each connector to verify connectivity.
  4. Complete the steps described in My Oracle Support Doc ID 2587717.1

Related Topics

Parent topic: Configuring Oracle Exadata Database Machine

6.11 Verifying the InfiniBand Network Fabric Network

This procedure describes how to verify the InfiniBand Network Fabric network.

  1. Visually check all the RDMA Network Fabric cable connections within the rack. The port lights should be on, and the LEDs should be on. Do not press each connector to verify connectivity.

  2. Log in as the root user on any component in the rack.

  3. Verify the InfiniBand Network Fabric topology using the following commands: 

    # cd /opt/oracle.SupportTools/ibdiagtools
# ./verify-topology [-t rack_size]

    The following example shows the output when the network components are correct.

    [DB Machine Infiniband Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch......[SUCCESS ]
Are any external switches connected to each other................[SUCCESS ]
Are any hosts connected to spine switch..........................[SUCCESS ]
Check if all hosts have 2 CAs to different switches..............[SUCCESS ]
Leaf switch check:cardinality and even distribution..............[SUCCESS ]
Check if each rack has an valid internal ring....................[SUCCESS ]

    In the preceding command, rack_size is the size of the rack. The -t rack_size option is needed if the rack is Oracle Exadata Half Rack or Oracle Exadata Quarter Rack. Use the value halfrack or quarterrack, if needed.

    The following example shows the output when there is a bad RDMA Network Fabric switch to cable connection: 

    #./verify-topology
[DB Machine Infiniband Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch......[SUCCESS ]
Are any external switches connected to each other................[SUCCESS ]
Are any hosts connected to spine switch..........................[SUCCESS ]
Check if all hosts have 2 CAs to different switches..............[SUCCESS ]
Leaf switch check:cardinality and even distribution..............[SUCCESS ]
Check if each rack has an valid internal ring....................[ERROR ]

Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them.
They should have at least 7 links between them

    The following example shows the output when there is a bad RDMA Network Fabric cable on a database server: 

    #./verify-topology
[DB Machine Infiniband Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch......[SUCCESS ]
Are any external switches connected to each other................[SUCCESS ]
Are any hosts connected to spine switch..........................[SUCCESS ]
Check if all hosts have 2 CAs to different switches..............[ERROR ]
Node db01 has 1 endpoints.(Should be 2)
Port 2 of this node is not connected to any switch
--------fattree End Point Cabling verification failed-----
Leaf switch check:cardinality and even distribution..............[ERROR ]
Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes
It has only 3 links belonging to compute nodes                  [SUCCESS ]
Check if each rack has an valid internal ring...................[SUCCESS ]

    The following example shows the output when there is a bad connection on the switch and the system:

    #./verify-topology
[DB Machine Infiniband Cabling Topology Verification Tool ]
Is every external switch connected to every internal switch......[SUCCESS ]
Are any external switches connected to each other................[SUCCESS ]
Are any hosts connected to spine switch..........................[SUCCESS ]
Check if all hosts have 2 CAs to different switches..............[ERROR ]

Node burxdb01 has 1 endpoints.(Should be 2) 
Port 2 of this node is not connected to any switch
--------fattree End Point Cabling verifation failed-----
Leaf switch check:cardinality and even distribution..............[ERROR ]
Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes 
It has only 3 links belonging to compute nodes...................[SUCCESS ]
Check if each rack has an valid internal ring....................[ERROR ]

Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them
They should have at least 7 links between them

Parent topic: Configuring Oracle Exadata Database Machine

6.12 Imaging a New System

You have different options for preparing an Oracle Exadata system for deployment.

Using a preconf.csv file with the PXE and ISO options enables an unattended first boot configuration and greatly simplifies the installation but is not mandatory. It is possible to boot from the installation media and follow the prompts to configure the system.

Starting with Oracle Exadata System Software release 19.1.0, Secure Eraser is automatically started during re-imaging if the hardware supports Secure Eraser. This significantly simplifies the re-imaging procedure while maintaining performance. Now, when re-purposing a rack, you only have to image the rack and the secure data erasure is taken care of transparently as part of the process.

In release 12.1.2.2.0 and above, the installation media has been simplified, and it is no longer necessary to build files using the ImageMaker process.

Parent topic: Configuring Oracle Exadata Database Machine

6.12.1 Using ISO Images to Image a New System

You can download ISO images to use when upgrading your Oracle Exadata.

In versions prior to 12.1.2.2.0, ISO images are available as patches that you can download for bare metal imaging an Exadata system. This is now the default method in 12.1.2.2.0 and above. You are no longer required to build the ISO files using imagemaker. Patches are available for both database nodes and cells. You can use the ISO images to boot the server using a virtual media device on the ILOM (from the Devices menu of the ILOM console). For example, the two ISO images available for 12.1.2.1.1 are listed in the following table.

Table 6-69 ISO Images for 12.1.2.1.1

Version Type Edelivery Part / Notes ARU Bug #/ULN Channel Description
12.1.2.1.1

Cell ISO image and md5sum: 90305b9e2c7e050d7588b8a7d5c731e7 cell_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.iso

Refer to 12.1.2.1.1 Readme (Doc ID 1959143.1) as well as the readme provided with the download.

20757971

Oracle Database Machine Exadata Storage Cell (X5-2L, X4-2L, X4270M3, X4270M2, X4275) Image 12c Release 1 (12.1.2.1.1) for Linux x86_64
12.1.2.1.1

DB ISO image and md5sum: 39f12722cb338b2d4de5acba90adf8fe compute_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.iso

Refer to 12.1.2.1.1 Readme (Doc ID 1959143.1) as well as the readme provided with the download.

20757974

Oracle Database Machine Database Host (X4-8, X4800M2, X4800, X5-2, X4-2, X4170M3, X4170M2, X4170) Image 12c Release 1 (12.1.2.1.1) for Linux x86_64

To prepare a new Exadata system for deployment using ISO images, perform the following steps:

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files. You will need the configuration files when you run applyElasticConfig.sh later.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. After booting from the ISO, follow the prompts and imaging will start automatically. You will be prompted to enter IP addresses.

    You can use a preconf.csv by following the steps in Using ISO to Image a New System to mount the image on an NFS repository. You need to place the preconf.csv file in the same NFS directory.

  5. If you are using virtualization on the database servers (Oracle VM Server or Oracle Linux KVM), then run switch_to_ovm.sh on each database server. The database servers will reboot if you are using Oracle VM Server.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3 in /boot/I_am_hd_boot
2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 system partition

  6. On systems with InfiniBand Network Fabric only, run /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim on each database server before installing any software.

    The command reclaims disk space reserved for the deployment type not selected, which is not required on RoCE-based Exadata database servers.

    The command typically takes less than 5 minutes.

    Caution:

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh. 
    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
Model is ORACLE SERVER X5-2
Number of LSI controllers: 1
Physical disks found: 4 (252:0 252:1 252:2 252:3)
Logical drives found: 1
...
[INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
[INFO     ] Create filesystem on device /dev/sda1
[INFO     ] Tune filesystem on device /dev/sda1

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported.  For the first word, TAB
lists possible command completions.  Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
 Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
Done.
grub> quit

  7. If you have an Oracle Exadata Rack X8 or earlier, then run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with elasticNode in the description.

  8. Run the OEDA tool to deploy.

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
1. Validate Configuration File
2. Setup Capacity-on-Demand
3. Create Virtual Machine  
4. Create Users  
5. Setup Cell Connectivity  
6. Create Cell Disks  
7. Create Grid Disks  
8. Configure Alerting  
9. Install Cluster Software  
10. Initialize Cluster Software  
11. Install Database Software  
12. Relink Database with RDS  
13. Create ASM Diskgroups  
14. Create Databases  
15. Apply Security Fixes  
16. Install Exachk  
17. Setup ASR Alerting  
18. Create Installation Summary  
19. Resecure Machine

Related Topics

Parent topic: Imaging a New System

6.12.2 Using Elastic Configuration to Image a New System

Elastic configuration is now the standard methodology for all new deployments, and applies to Oracle Exadata X4-8 and later servers.

You can also use elastic configuration to add additional database servers or cells to an existing configuration. The process allows initial IP addresses to be assigned to database servers and cells, regardless of the exact customer configuration ordered.

Elastic configuration is only applicable to machines shipped from the factory (or to machines imaged exactly to mimic that) and only for X4-8 or later systems. You can run elastic configuration at the first time of deployment, and when adding new factory-built components to a rack. You cannot use elastic configuration to configure or reconfigure systems. It is recommended that you use elastic configuration only with machines shipped from the factory for first-time deployment.

To prepare a new Exadata system for deployment using the elastic configuration method, perform the following steps:

  1. Download the latest version of Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16.* range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. If you are using Oracle VM, then, before you run the OEDA installation tool, run switch_to_ovm.sh on each database server. The database servers will reboot. If you are using physical deployment and not virtualized deployment, then you can skip this step.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3 in /boot/I_am_hd_boot
2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 system partition

  5. On systems with InfiniBand Network Fabric only, run /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim on each database server before installing any software.

    The command reclaims disk space reserved for the deployment type not selected, which is not required on RoCE-based Exadata database servers.

    The command typically takes less than 5 minutes.

    Caution:

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh. 
    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
Model is ORACLE SERVER X5-2
Number of LSI controllers: 1
Physical disks found: 4 (252:0 252:1 252:2 252:3)
Logical drives found: 1
...
[INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
[INFO     ] Create filesystem on device /dev/sda1
[INFO     ] Tune filesystem on device /dev/sda1

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported.  For the first word, TAB
lists possible command completions.  Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
 Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
Done.
grub> quit

  6. Apply your specific configuration to the nodes by running the applyElasticConfig.sh script that is supplied with OEDA.

  7. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with elasticNode in the description.

  8. Run the OEDA tool to deploy, supplying the name of the configuration file generated by OEDA.

    For example, if your rack is named exadm01, then you would use the following command: 

    [root] # ./install.sh -cf ../exadm01.xml -l
1. Validate Configuration File
2. Setup Capacity-on-Demand
3. Create Virtual Machine  
4. Create Users  
5. Setup Cell Connectivity  
6. Create Cell Disks  
7. Create Grid Disks  
8. Configure Alerting  
9. Install Cluster Software  
10. Initialize Cluster Software  
11. Install Database Software  
12. Relink Database with RDS  
13. Create ASM Diskgroups  
14. Create Databases  
15. Apply Security Fixes  
16. Install Exachk  
17. Setup ASR Alerting  
18. Create Installation Summary  
19. Resecure Machine

Parent topic: Imaging a New System

6.12.3 Using ISO to Image a New System

You can use ISO to prepare a new Exadata system for deployment.

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. Run OEDA to generate the preconf.csv file that can be used to image the system.

  5. Connect to the first database node.

    Use one of the following connection options:

    • Attach a serial cable between a console and the SER MGT port on the server.

      Connections through the modular jack serial port must use shielded cables to comply with radio frequency emission limits.

      Set the terminal device communication properties to these values: 9600 baud, 8 bit, no parity, 1 stop bit.

      For more details, see Connect to Oracle ILOM.

    • Attach an Ethernet cable to the NET MGT port on the server and then connect using a workstation on the network.

      For more details, see Connect to Oracle ILOM.

  6. Obtain the eth0 MAC address for each database node. This is the fru_macaddress field from /SYS/MB/NET0 in the ILOM (or through ip addr if logged into the console).

    1. Connect to the other database nodes using ssh from the console.

    2. Run ibhosts to get a list of node names and IP addresses.

  7. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node. This field should be empty, that is, you should see two consecutive comma characters (,,). For example: 

    orhb42-b08-01,example.com,db,eth0,eth0,Management,00:10:E0:69:34:52,
10.196.3.106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:ac
tive-bond-ib,192.168.10.15,255.255.255.0,,,America/Los_Angeles

  8. Run ipconf to verify the generated preconf.csv file. This can be done on any Exadata server, and although not required, it is useful to ensure the preconf.csv is valid before proceeding. 

    # ipconf -verify -preconf /tmp/preconf.csv
[Info]: ipconf command line: /opt/oracle.cellos/ipconf.pl -verify -preconf /tmp
/preconf -nocodes
[Done]: Pre config verification OK

  9. Prepare and locate the files required for system imaging. The procedure varies depending on the Oracle Exadata System Software version being used.

    • If you are using version 12.1.2.2.0 or later, it is no longer necessary to use imagemaker to create the image files. The relevant PXE, ISO, and USB image files are already in the patch archive.

      Use Exadata Database Machine and Exadata Storage Server Supported Versions (My Oracle Support Doc ID 888828.1) to locate the relevant patch archive. Then, unzip the patch archive and follow the steps in the patch README.txt to copy the ISO file and optional preconf.csv file into place.

      Note:

      • Using ISO with the preconf.csv file is supported only if the ISO image is mounted on an NFS repository in ILOM and the preconf.csv file is located in the same NFS directory.

        All other methods of mounting the ISO image in ILOM (SAMBA URI and Java Remote Console) and burning ISO image to DVD will not work with the preconf.csv file. For instructions on how to mount the ISO image from a NFS central repository, see "Configure a Host Storage Device Redirection Session" in the Oracle X5 Series Servers Administration Guide

      • If a newly imaged system does not have access to DHCP, then the imaging procedure will not be able to access the preconf.csv file. In this case, the imaging procedure will attempt to locate the preconf.csv inside a dedicated ISO image located in the same directory and named preconf.iso.

        To use this method, you can create the preconf.iso file by using the following command: 

        # mkisofs -l -input-charset ascii -o preconf.iso preconf.csv

      • When a preconf.csv file is not being used, the ISO image can be made available as virtual media on the ILOM of the node being re-imaged.

      • Because of possible late echos of text to the console, be aware that the boot process will be waiting for input to complete the interactive ipconf by prompting for Nameserver:

      • Due to a bug, ipconf will prompt you twice to enter the network settings.

    • If you are using versions prior to 12.1.2.2.0, use ImageMaker to create the image files as follows:

      1. Download the imagemaker kit to prepare and create the files for the USB drive.

        This kit is available publicly on the Oracle Software Delivery Cloud (edelivery.oracle.com) site under “Oracle Database Products -> Linux x86-64”. There are separate ImageMaker files for database nodes and cells.

      2. After download, unzip the first file, called something similar to V75080-01.zip, and then unzip the subsequent file as well. Finally, extract the resulting archive. Perform the commands as the root user. 

        # unzip V75080-01.zip

# unzip cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.zip

# tar -pxvf cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.tar

        View the README_FOR_FACTORY.txt file in the resulting directory for additional details.

      3. Run makeImageMedia.sh to build the ISO file for imaging.

        For building images for 12.1.2.1.0 and above, the recommended makeImageMedia.sh options for building ISO images are:

        1. (12.1.2.1.0 up to 12.2.1.1.0) Change directory to the dl180 or dl360 directory.

        2. (12.2.1.1.0 and above) Change directory to the cell or compute directory.

        3. Make the ISO file.

          ./makeImageMedia.sh -factory -stit -reboot-on-success -nodisktests [-preconf
 preconf_file] filename.iso

        For ISO imaging, note that you must run makeImageMedia.sh from an Oracle Exadata database server if you are using the -preconf option.

        For example:

        # ./makeImageMedia.sh -factory -stit -reboot-on-success -nodisktests -preconf
 preconf.csv computenode.iso
[Info]: ipconf command line: /root/imagemaker/compute/initrd/opt/oracle.cellos
/ipconf.pl -preconf preconf.csv -verify -nocodes
[Done]: Pre config verification OK
Please wait. Calculating md5 checksums for cellbits ...
Calculating md5 checksum for exaos.tbz ...
Calculating md5 checksum for dbboot.tbz ...
Calculating md5 checksum for dbfw.tbz ...
Calculating md5 checksum for kernel.tbz ...
Calculating md5 checksum for ofed.tbz ...
Calculating md5 checksum for sunutils.tbz ...
Calculating md5 checksum for commonos.tbz ...
Calculating md5 checksum for debugos.tbz ...
Calculating md5 checksum for dbrpms.tbz ...
Calculating md5 checksum for exaosovs.tbz ...
Calculating md5 checksum for dbovsboot.tbz ...
Calculating md5 checksum for kernelovs.tbz ...
Calculating md5 checksum for ofedovs.tbz ...
Calculating md5 checksum for commonovsos.tbz ...
Calculating md5 checksum for debugosovs.tbz ...
Calculating md5 checksum for dbovsrpms.tbz ...
Calculating md5 checksum for sunovsutils.tbz ...
Calculating md5 checksum for dbvmboot.tbz ...
Calculating md5 checksum for ofedvm.tbz ...
Calculating md5 checksum for sunvmutils.tbz ...
Please wait. Making initrd ...
395864 blocks
Please wait. Calculating md5 checksums for boot ...
Size of boot image is 4 sectors -> No emulation
0.30% done, estimate finish Wed Nov 29 10:47:27 2016
0.60% done, estimate finish Wed Nov 29 10:47:27 2016
0.90% done, estimate finish Wed Nov 29 10:47:27 2016
1.21% done, estimate finish Wed Nov 29 10:47:27 2016
1.51% done, estimate finish Wed Nov 29 10:47:27 2016
1.81% done, estimate finish Wed Nov 29 10:47:27 2016
2.11% done, estimate finish Wed Nov 29 10:47:27 2016
2.41% done, estimate finish Wed Nov 29 10:47:27 2016
2.71% done, estimate finish Wed Nov 29 10:47:27 2016
3.02% done, estimate finish Wed Nov 29 10:47:27 2016
3.32% done, estimate finish Wed Nov 29 10:47:27 2016
3.62% done, estimate finish Wed Nov 29 10:47:27 2016
3.92% done, estimate finish Wed Nov 29 10:47:27 2016
4.22% done, estimate finish Wed Nov 29 10:47:27 2016
4.52% done, estimate finish Wed Nov 29 10:47:27 2016
4.83% done, estimate finish Wed Nov 29 10:47:27 2016
5.13% done, estimate finish Wed Nov 29 10:47:27 2016
5.43% done, estimate finish Wed Nov 29 10:47:27 2016
5.73% done, estimate finish Wed Nov 29 10:47:27 2016
<output truncated>
97.72% done, estimate finish Wed Nov 29 10:47:30 2016
98.02% done, estimate finish Wed Nov 29 10:47:30 2016
98.32% done, estimate finish Wed Nov 29 10:47:30 2016
98.62% done, estimate finish Wed Nov 29 10:47:30 2016
98.92% done, estimate finish Wed Nov 29 10:47:30 2016
99.22% done, estimate finish Wed Nov 29 10:47:30 2016
99.52% done, estimate finish Wed Nov 29 10:47:30 2016
99.83% done, estimate finish Wed Nov 29 10:47:30 2016
Total translation table size: 2048
Total rockridge attributes bytes: 0
Total directory bytes: 2048
Path table size(bytes): 26
Max brk space used 0
1657890 extents written (3238 MB)
Done creation of installation ISO for compute node

      4. Attach the ISO as virtual media to the ILOM of the database server to be re-imaged, and select CDROM as the next boot device.

  10. Reboot the database node or cell to start the imaging process from the ISO.

  11. If you are using Oracle VM, run switch_to_ovm.sh on each database server. The database servers will reboot.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDb
Sys3 (/dev/mapper/VGExaDb-LVDbSys3)
2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys
1
2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VG
ExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3
 in /boot/I_am_hd_boot
2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 s
ystem partition

  12. On systems with InfiniBand Network Fabric only, run /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim on each database server before installing any software.

    The command reclaims disk space reserved for the deployment type not selected, which is not required on RoCE-based Exadata database servers.

    The command typically takes less than 5 minutes.

    Caution:

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh. 
    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
Model is ORACLE SERVER X5-2
Number of LSI controllers: 1
Physical disks found: 4 (252:0 252:1 252:2 252:3)
Logical drives found: 1
...
[INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
[INFO     ] Create filesystem on device /dev/sda1
[INFO     ] Tune filesystem on device /dev/sda1

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported.  For the first word, TAB
lists possible command completions.  Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
 Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
Done.
grub> quit

  13. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with elasticNode in the description.

  14. Run the OEDA tool to deploy. 

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
1. Validate Configuration File
2. Setup Capacity-on-Demand
3. Create Virtual Machine  
4. Create Users  
5. Setup Cell Connectivity  
6. Create Cell Disks  
7. Create Grid Disks  
8. Configure Alerting  
9. Install Cluster Software  
10. Initialize Cluster Software  
11. Install Database Software  
12. Relink Database with RDS  
13. Create ASM Diskgroups  
14. Create Databases  
15. Apply Security Fixes  
16. Install Exachk  
17. Setup ASR Alerting  
18. Create Installation Summary  
19. Resecure Machine

Related Topics

Parent topic: Imaging a New System

6.12.4 Using PXE to Image a New System

You can use PXE to prepare a new Exadata system for deployment.

Parent topic: Imaging a New System

6.12.4.1 Using PXE to Image a New System Using Release 12.1.2.2.0 or Later

You can use PXE to prepare a new Exadata system running Oracle Exadata System Software release 12.1.2.2.0 or later for deployment using a simplified method.

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA) from http://www.oracle.com/technetwork/database/exadata/oeda-download-2076737.html.
  2. Run the OEDA configuration tool to generate the configuration files.
  3. Power on all the servers.
    When the system is first started, the following conditions exist:
    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.
    2. Host names of the nodes will be of the format nodeN, where N is a number.
  4. Run OEDA to generate the preconf.csv file that you will use to image the system.
  5. Connect to the first database node.

    Use one of the following connection options:

    • Attach a serial cable between a console and the SER MGT port on the server.

      Connections through the modular jack serial port must use shielded cables to comply with radio frequency emission limits.

      Set the terminal device communication properties to these values: 9600 baud, 8 bit, no parity, 1 stop bit.

      For more details, see Connect to Oracle ILOM.

    • Attach an Ethernet cable to the NET MGT port on the server and then connect using a workstation on the network.

      For more details, see Connect to Oracle ILOM.

  6. Obtain the eth0 MAC address for each database node.
    The eth0 MAC address is the fru_macaddress field from /SYS/MB/NET0 in the ILOM (or from ip addr if logged into the console).
    1. Connect to the each database node through SSH from the console.
    2. Run ibhosts to get a list of node names and IP addresses.
  7. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node.
    This field should be empty, that is, you should see two consecutive comma characters (,,).

    For example:

    orhb42-b08-01,example.com,db,eth0,eth0,Management,00:10:E0:69:34:52,10.196.3.
106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:active-bond-ib,192.1
68.10.15,255.255.255.0,,,America/Los_Angeles
  8. Create a PXE configuration file for each node.

    The file name is hostname.xx-xx-xx-xx-xx-xx in the pxelinux.cfg directory. Below are the recommended options for PXE imaging of versions 12.1.2.1.0 and above. The line starting with append should be entered as a single line with no line breaks. 

    kernel vmlinux-iso-file-name
append initrd=img_file pxe factory stit reboot-on-success notests=diskgroup dhcp 
preconf=n.n.n.n:/directory/preconf.csv iso-uri=http://web_server/x86_64/iso/image_n
ame.iso console=ttyS0,115200n8

    On systems with Oracle Exadata System Software release 20.1.0 or later, you can enable or disable Exadata Secure RDMA Fabric Isolation by adding qinq=yes or qinq=no to the kernel options. For example: 

    kernel vmlinux-iso-file-name
append initrd=img_file pxe factory stit reboot-on-success notests=diskgroup dhcp 
preconf=n.n.n.n:/directory/preconf.csv iso-uri=http://web_server/x86_64/iso/image_n
ame.iso console=ttyS0,115200n8 qinq=no
  9. Optionally, configure key-based authentication for the root OS user.

    Starting with Oracle Exadata System Software release 20.1.0, you can configure key-based authentication for the root OS user as part of the PXE imaging process.

    If you want to use this option, then perform the following for each node:

    1. Create an RSA public and private key pair using a utility such as ssh-keygen.

      For example:

      # ssh-keygen -t rsa -b 4096
    2. Rename the public key file so that the file name conforms to the following format:
      id_rsa.short_hostname.root.pub

      In the public key file name, short_hostname is the node host name as defined in the first field of the corresponding node entry in the preconf.csv file.

    3. Copy the renamed public key file to the directory that contains the preconf.csv file.

      This is the same directory that you previously specified in the PXE configuration file preconf parameter, which defines the path to the preconf.csv file.

    If you configure key-based authentication, then password-based authentication is disabled after the imaging process. Consequently, root access to the node is only possible using the matching private key. ILOM access using the root password is also disabled.

    By default, if the public key file is not named or located correctly, then password-based authentication is enabled for the node. You can also make key-based authentication mandatory by specifying root-ssh-key=yes as a boot option in the PXE configuration file. In this case, the imaging process fails if the public key is not in the correct location.

    Regardless of any public keys, you can disable the configuration of key-based authentication by specifying root-ssh-key=no as a boot option in the PXE configuration file.

  10. Create the image files, if necessary.

    If you are using version 12.1.2.2.0 or later, it is no longer necessary to use ImageMaker to create the image files. The relevant PXE, ISO, and USB image files have already been created in the patches themselves. Simply download the patch containing the pre-built PXE image files from Oracle Software Delivery Cloud and place them into the appropriate directories on the PXE server. For all releases, see My Oracle Support note 888828.1 for the patch numbers. There are separate PXE image patches for database servers and storage servers.

    Note: For image version 12.1.2.2.0 the format of the downloadable image files has changed.

    The downloaded zip file contains four image files:

    • For 12.1.2.2.0 up to 12.2.1.1.0:
      kernel: vmlinux-version-ImageName-{DL180|DL360}
initrd: initrd-version-ImageName-{DL180|DL360}.img
image: nfsimg-version-ImageName-{DL180|DL360}.tar
image: nfsimg-version-ImageName-{DL180|DL360}.tar.md5
    • For 12.2.1.1.0 and above:
      kernel: vmlinux-iso-ImageName-{cell|compute}
initrd: initrd-iso-ImageName-{cell|compute}.img
image: vmlinux-iso-ImageName-{cell|compute}.tar
  11. Change the boot order using the ILOM to boot from PXE and then restart the server to start the imaging process.

    For example, from the Oracle Linux prompt on the host: 

    # ipmitool chassis bootdev pxe

# shutdown -r now
  12. If you are using Oracle VM, run switch_to_ovm.sh on all Oracle VM database servers before running the OEDA installation tool.

    When you run this command, the database servers will reboot.

    # /opt/oracle.SupportTools/switch_to_ovm.sh
2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LV
DbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbS
ys1
2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/
VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSy
s3 in /boot/I_am_hd_boot
2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0
 system partition
  13. On systems with InfiniBand Network Fabric only, run /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim on each database server before installing any software.

    The command reclaims disk space reserved for the deployment type not selected, which is not required on RoCE-based Exadata database servers.

    The command typically takes less than 5 minutes.

    Caution:

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh. 
    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
Model is ORACLE SERVER X5-2
Number of LSI controllers: 1
Physical disks found: 4 (252:0 252:1 252:2 252:3)
Logical drives found: 1
...
[INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
[INFO     ] Create filesystem on device /dev/sda1
[INFO     ] Tune filesystem on device /dev/sda1

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported.  For the first word, TAB
lists possible command completions.  Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
 Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
Done.
grub> quit
  14. Run ibhosts and verify that all nodes show the correct IP addresses and host names.
    There should be no nodes with elasticNode in the description.
  15. Run the OEDA tool to deploy.

    Note:

    If you configured key-based authentication for the root OS user, then you must first copy the node private keys into the OEDA WorkDir. 

    [root] # ./install.sh -cf ../machine_config_file.xml -l
1. Validate Configuration File
2. Setup Capacity-on-Demand
3. Create Virtual Machine  
4. Create Users  
5. Setup Cell Connectivity  
6. Create Cell Disks  
7. Create Grid Disks  
8. Configure Alerting  
9. Install Cluster Software  
10. Initialize Cluster Software  
11. Install Database Software  
12. Relink Database with RDS  
13. Create ASM Diskgroups  
14. Create Databases  
15. Apply Security Fixes  
16. Install Exachk  
17. Setup ASR Alerting  
18. Create Installation Summary  
19. Resecure Machine
6.12.4.2 Using PXE to Image a New System for Releases Earlier than 12.1.2.2.0

You can use PXE to prepare a new Exadata system running an Oracle Exadata System Software release earlier than 12.1.2.2.0 for deployment.

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA) from http://www.oracle.com/technetwork/database/exadata/oeda-download-2076737.html.
  2. Run the OEDA configuration tool to generate the configuration files.
  3. Power on all the servers.
    When the system is first started, the following conditions exist:
    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.
    2. Host names of the nodes will be of the format nodeN, where N is a number.
  4. Run OEDA to generate the preconf.csv file that you will use to image the system.
  5. Connect to the first database node.

    Use one of the following connection options:

    • Attach a serial cable between a console and the SER MGT port on the server.

      Connections through the modular jack serial port must use shielded cables to comply with radio frequency emission limits.

      Set the terminal device communication properties to these values: 9600 baud, 8 bit, no parity, 1 stop bit.

      For more details, see Connect to Oracle ILOM.

    • Attach an Ethernet cable to the NET MGT port on the server and then connect using a workstation on the network.

      For more details, see Connect to Oracle ILOM.

  6. Obtain the eth0 MAC address for each database node.
    The eth0 MAC address is the fru_macaddress field from /SYS/MB/NET0 in the ILOM (or from ip addr if logged into the console).
    1. Connect to the each database node through SSH from the console.
    2. Run ibhosts to get a list of node names and IP addresses.
  7. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node.
    This field should be empty, that is, you should see two consecutive comma characters (,,).

    For example:

    orhb42-b08-01,example.com,db,eth0,eth0,Management,00:10:E0:69:34:52,10.196.3.
106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:active-bond-ib,192.1
68.10.15,255.255.255.0,,,America/Los_Angeles
  8. Create a PXE configuration file for each node.

    Below are the recommended options for PXE imaging of versions prior to release 12.1.2.2.0. The line starting with append should be entered as a single line with no line breaks. 

    append initrd=img_file pxe factory stit reboot-on-success notests=diskgroup dhcp 
preconf=n.n.n.n:/directory/preconf.csv sk=n.n.n.n:/directory console=ttyS0,
115200n8
  9. Download the ImageMaker kit to prepare and create the files for the PXE server.
    The kit is available publicly on the Oracle Software Delivery Cloud (edelivery) site under “Oracle Database Products -> Linux x86-64”. There are separate ImageMaker files for database nodes and cells.
  10. After downloading the kit, as the root user, unzip the first file that has a name similar to V75080-01.zip, and then unzip the subsequent ZIP file. Finally, extract the resulting archive.
    # unzip V75080-01.zip

# unzip cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.zip

# tar -pxvf cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.tar

    View the README_FOR_FACTORY.txt file in the resulting directory for additional details.

  11. Run makeImageMedia.sh to build the kernel, initrd, and image files, if necessary.
    Place the image files in the relevant directories on the PXE server. These files will be sent at boot time to the node being imaged.
    1. Change directory to the dl180 or dl360 directory and make the PXE build using:
      ./makeImageMedia.sh -pxe -pxeout ImageName

      If the dl180 or dl360 directory are not available, then change directory to the cell or compute directory.

      The makeImageMedia.sh command creates four files in the ./PXE subdirectory: 

      kernel: vmlinux-version-ImageName-{DL180|DL360}
initrd: initrd-version-ImageName-{DL180|DL360}.img
image: nfsimg-version-ImageName-{DL180|DL360}.tar
image: nfsimg-version-ImageName-{DL180|DL360}.tar.md5
  12. Change the boot order using the ILOM to boot from PXE and then restart the server to start the imaging process.

    For example, from the Oracle Linux prompt on the host:

    # ipmitool chassis bootdev pxe

# shutdown -r now
  13. If you are using Oracle VM run switch_to_ovm.sh on all Oracle VM database servers before running the OEDA installation tool.

    When you run this command, the database servers will reboot.

    # /opt/oracle.SupportTools/switch_to_ovm.sh
2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LV
DbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbS
ys1
2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/
VGExaDb-LVDbSys1
2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSy
s3 in /boot/I_am_hd_boot
2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0
 system partition
  14. On systems with InfiniBand Network Fabric only, run /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim on each database server before installing any software.

    The command reclaims disk space reserved for the deployment type not selected, which is not required on RoCE-based Exadata database servers.

    The command typically takes less than 5 minutes.

    Caution:

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh. 
    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
Model is ORACLE SERVER X5-2
Number of LSI controllers: 1
Physical disks found: 4 (252:0 252:1 252:2 252:3)
Logical drives found: 1
...
[INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
[INFO     ] Create filesystem on device /dev/sda1
[INFO     ] Tune filesystem on device /dev/sda1

GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported.  For the first word, TAB
lists possible command completions.  Anywhere else TAB lists the possible
completions of a device/filename.]
grub> root (hd0,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
 Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
 Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
Done.
grub> quit
  15. Run ibhosts and verify that all nodes show the correct IP addresses and host names.
    There should be no nodes with elasticNode in the description.
  16. Run the OEDA tool to deploy.
    [root] # ./install.sh -cf ../machine_config_file.xml -l
1. Validate Configuration File
2. Setup Capacity-on-Demand
3. Create Virtual Machine  
4. Create Users  
5. Setup Cell Connectivity  
6. Create Cell Disks  
7. Create Grid Disks  
8. Configure Alerting  
9. Install Cluster Software  
10. Initialize Cluster Software  
11. Install Database Software  
12. Relink Database with RDS  
13. Create ASM Diskgroups  
14. Create Databases  
15. Apply Security Fixes  
16. Install Exachk  
17. Setup ASR Alerting  
18. Create Installation Summary  
19. Resecure Machine

6.12.5 Updates to Imaging Parameters

Some of the command line options for imaging have changed with 12.1.2.1.0 and above.

The changes are:

  • updfrm

    Do NOT use the updfrm option in any form. It will be removed from the imaging options in the future. Allow the firmware to be updated automatically upon initial boot instead.

  • dualboot

    dualboot is no longer required with 12.1.2.1.0 as there is no support for Solaris. It has no meaning in releases 12.1.2.1.0 and above.

  • factory

    The meaning of the factory keyword has changed as of 12.1.2.1.0. The use of "factory" now enforces additional options such as "ovs=yes". This should be used for all imaging, regardless of the ultimate system configuration (Oracle VM, Physical etc.)

  • OVS=yes|no

    This option defaults to "no". However it is not recommended to use this at all because it may change in the future. In its place, use the "factory" option which will override some default options.

There are some additional command line options which may be useful:

  • To create a hot spare on dbnodes, use the enable_hot_spare option.

    • makeImageMedia.sh option: -enable_hot_spare

    • PXE option: enable_hot_spare

  • To force active bond or no active bond for InfiniBand interfaces (mainly for expansion rack cases), use the active-bond-ib option.

    • makeImageMedia.sh option: -active-bond-ib <yes|no>

    • PXE option: active-bond-ib=<yes|no>

The README_FOR_FACTORY.txt file shipped with the imagemaker software explains each option in more detail.

Related Topics

Parent topic: Imaging a New System

6.13 Performing Initial Elastic Configuration of Oracle Exadata

This topic provides background information on elastic configuration and describes how to perform the initial configuration of Oracle Exadata.

You need the configuration file generated by Oracle Exadata Deployment Assistant (OEDA).

Elastic configuration is new in Oracle Exadata System Software release 12.1.2.1.0. It applies to all rack configurations, such as the following:

  • Racks ordered with a standard number of database nodes and cells, for example, quarter or half racks
  • Rack configurations that feature additional database nodes and cells
  • Oracle Exadata Two-Socket X5 generation and later storage servers
  • Oracle Exadata Racks that use an InfiniBand Network Layer or a RoCE Network Layer.

Starting with the Oracle Exadata Two-Socket X5 generation of storage servers, the same process can also be used to add additional database servers or storage servers to an existing configuration.

The process allows initial IP addresses to be assigned to database servers and storage servers, regardless of the exact configuration you ordered. Your specific configuration can then be applied to the nodes. The procedure can be run from any database node (usually the first node,) and also from a storage server in cases where expansion involves adding only storage servers.

Every Oracle Exadata has a predefined method for the cabling of nodes to the RDMA Network Fabric switch ports. Therefore, there is a fixed mapping from each node's location in the rack to the ports of the RDMA Network Fabric switches. Assuming the rack is always populated following this map, a node's rack unit location can be identified by querying the RDMA Network Fabric to determine the RDMA Network Fabric switch port the node is connected to. Once found, that information is used to determine the rack unit location of the node. With this information, nodes can be allocated initial IP addresses based on their rack unit location, with nodes lower in the rack getting lower IP addresses.

Note:

  • Disk space for unused system partitions must be reclaimed before installing software on all database servers with InfiniBand Network Fabric. Use the following command to reclaim the disk space: /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim. This command is not required on RoCE-based Exadata database servers.

  • If a RAID rebuild/reconstruction (that is, a disk replacement or expansion) is in progress, wait until it is finished before performing the steps in this section. The reason is that the reclaimdisks.sh script is blocked from running until the RAID rebuild/reconstruction is complete.

  1. Open a console session to the first database server. The first database server is the lowest database server in the rack, which is rack position U16. In the case of an expansion involving only cells, the procedure should be run from a storage server instead of a database node.

    Note:

    The network connection and configuration for Oracle Exadata has not occurred yet.

  2. Log in as the root user on the first database server.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  3. On systems with InfiniBand Network Fabric only, verify all servers are configured with the 172.16.* IP address for the eth0 interface using the following command: 

    # ibhosts


Ca  : 0x0010e00001486fb8 ports 2 "node10 elasticNode 172.16.2.46,172.16.2.46 ETH0"
Ca  : 0x0010e00001491228 ports 2 "node9 elasticNode 172.16.2.45,172.16.2.45 ETH0"
Ca  : 0x0010e000014844f8 ports 2 "node8 elasticNode 172.16.2.44,172.16.2.44 ETH0"
Ca  : 0x0010e00001488218 ports 2 "node4 elasticNode 172.16.2.40,172.16.2.40 ETH0"
Ca  : 0x0010e000014908b8 ports 2 "node2 elasticNode 172.16.2.38,172.16.2.38 ETH0"
Ca  : 0x0010e0000148ca68 ports 2 "node1 elasticNode 172.16.2.37,172.16.2.37 ETH0"
Ca  : 0x0010e00001485fd8 ports 2 "node3 elasticNode 172.16.2.39,172.16.2.39 ETH0"

    Each server description should be hostname elasticNode. ipaddress ETH0.

  4. If using virtualization on the rack, configure each database server as follows:

    Note:

    Starting with Oracle Exadata Database Machine X8M-2, Oracle Linux KVM is the virtualization technology for systems that use RoCE Network Fabric. Otherwise, Oracle VM Server (OVM) is a Xen-based virtualization technology, which is used across Oracle Exadata Database Machine systems that use InfiniBand Network Fabric.

    This step applies to both virtualization technologies.

    # /opt/oracle.SupportTools/switch_to_ovm.sh

    The database servers will reboot during this step.

    Note:

    It is not possible or supported to switch between physical and virtualized configurations.

  5. On systems with InfiniBand Network Fabric only, reclaim the unused disk space on the database servers.

    Disk space for unused system partitions must be reclaimed before installing software on all database servers with InfiniBand Network Fabric. Use the following command to reclaim the disk space: /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim. This command is not required on RoCE-based Exadata database servers.

    reclaimdisks.sh performs partition reconfiguration and a few other necessary configuration changes that take typically less than a minute to complete.

  6. On systems with InfiniBand Network Fabric only, verify the disk configuration on the database servers. 

    # /opt/oracle.SupportTools/reclaimdisks.sh -check

    If the system is a physical (bare metal) deployment, the last line of output will be the following:

    Valid. Booted: Linux. Layout: Linux.

    If the system is an Oracle VM deployment, the last line of output will be the following:

    Valid. Booted: DOM0. Layout: DOM0.

  7. Copy the OEDA configuration files to the database server using one of the following methods. If adding only storage servers, the steps can be run from a storage server.

    • Using a USB flash drive:

      1. Copy the configuration file to a USB flash drive.

      2. Connect the USB drive to the database server or storage server.

      3. Locate the USB drive using the following command:

        for x in `ls -1 /sys/block`; do udevadm info --attribute-walk --path=/sys/block/$x | grep \
-iq 'DRIVERS=="usb-storage"'; if [ $? -eq 0 ] ; then echo /dev/${x}1; \
fi ; done

        The expected output is similar to:

        /dev/sdb1

        Note:

        If an Oracle Exadata Storage Server X6 or X5 Extreme Flash (EF) is used, the command will also return the two internal USB drives, usually /dev/sda1 and /dev/sdb1. The newly inserted USB drive will be the additional device, usually /dev/sdc1.

        The expected output is similar to:

        /dev/sda1
/dev/sdb1
/dev/sdc1

        The internal USBs can be identified by viewing mdstat: 

        # grep md4 /proc/mdstat | awk '{print $5,$6;}'

      4. Create a directory on the database server or storage server using the following command:

        # mkdir /mnt/usb

      5. Create a directory for the installation using the following command:

        # mkdir /opt/oracle.SupportTools/onecommand

        If the directory already exists, then rename the existing one to onecommand.old, and then re-create the empty directory.

        Note:

        Do not use a directory under /u01 or /EXAVMIMAGES because reclaimdidsks.sh will fail if there is content in those directories.

      6. Mount the device. Use the device name given in step 7.c. The following is an example of the command. 

        # mount -t vfat /dev/sdb1 /mnt/usb

      7. Copy OEDA and the configuration files from the USB flash drive to the onecommand directory on the database server or storage server using the following command: 

        # cp /mnt/usb/path_to_file/*.* /opt/oracle.SupportTools/onecommand

        In the preceding command, path_to_file is the directory path to the configuration file on the USB flash drive.

      8. Unmount the USB flash drive using the following command:

        # umount /mnt/usb

      9. Remove the USB flash drive from the database server or storage server.

    • Using a temporary network connection:

      This option is only needed if there were IP address conflicts, and Oracle Exadata is not already connected to the network.

      1. Connect a network cable to the Ethernet port labeled NET0 on the back of the first database server.

      2. Bring up the eth0:1 interface as follows: 

        # ifconfig eth0:1 real_ip netmask real_netmask up

        In the preceding command, real_ip is the IP address to assign to the database server, and real_netmask is the subnet mask to assign to the database server.

      3. Add the default route as follows:

        # route add -net 0.0.0.0 netmask 0.0.0.0 gw real_gateway

        In the preceding command, real_gateway is the default gateway to use for the database server.

      4. Copy the configuration files to the /opt/oracle.SupportTools/onecommand directory on the database server or storage server using a file transfer utility such as scp.

      5. Disconnect the network cable from Ethernet port NET0.

        Note:

        It is not necessary to reconfigure the network because the network is reconfigured automatically during the restart step.

  8. Download and unzip the OEDA archive to a host on the same network as the servers in the rack (172.16.*).

    The applyElasticConfig.sh script is not configured by default on the servers in the rack.

  9. Run the applyElasticConfig.sh script.

    For example, if you unzipped the OEDA ZIP file into the /opt/oracle.SupportTools/onecommand/linux-x64 directory, then use the following commands: 

    # cd /opt/oracle.SupportTools/onecommand/linux-x64

# ./applyElasticConfig.sh -cf customer_name-configFile.xml

    The applyElasticConfig.sh script performs network configuration for all database servers and storage servers. All servers restart at the end of the process.

  10. Connect the corporate network cable for the management network to the Management Network Switch.

  11. Connect the corporate network cables for the client access network to the database servers.

  12. Restart all database servers and storage servers.

  13. Log in to the first database server (or storage server, if only storage servers were added for expansion) to verify network connectivity using the following commands:

    # cd /opt/oracle.SupportTools/onecommand/linux-x64

#./checkip.sh -cf configuration_file

    In the preceding command, configuration_file is the name of the configuration file from OEDA, and includes the full path.

    Note:

    • The checkip.sh script can be run on a server outside of Oracle Exadata to verify network connectivity to Oracle Exadata. Copy the checkip.sh file to the external system, and run the command as shown in this step.

    • If the command is run from a Microsoft Windows machine, then the command is checkip.cmd.

    • Download the version of OEDA for the relevant platform.

  14. Use OEDA to install and configure the software for Oracle Exadata.

Related Topics

Parent topic: Configuring Oracle Exadata Database Machine

6.14 Adding Additional Elastic Nodes to an Existing Rack

You can add nodes to an existing rack using elastic configuration.

The elastic configuration procedure described in Performing Initial Elastic Configuration of Oracle Exadata Database Machine can also be used to add new servers to an existing rack. The new servers and existing servers must be at least X5. The procedure assigns IP addresses to the new database servers and cells, in preparation for adding them into existing setups. The addition of X5 or later model servers into previous generation racks using this methodology is not supported.

The XML configuration file generated by the Oracle Exadata Deployment Assistant (OEDA) must contain the existing nodes, plus the new nodes to be added.

If you are adding only storage servers, then you can run the applyElasticConfig.sh script from a storage server rather than a database server.

The overall process for adding a new server to an existing rack is:

  1. Use Oracle Exadata Deployment Assistant (OEDA) to add the new server to the rack configuration and generate the configuration files.

  2. Download and unzip the OEDA archive to an existing server in the rack where you are adding the new server.

  3. Install the new server into the rack.

  4. Review the OEDAconfiguration file for the rack to determine the location within the rack (the ulocation) for the new server. For example: 

    <uLocation>16</uLocation>

  5. Log in to the ILOM on the new server using the local serial management connection (SER MGT port), and review the system location in the SP.

    For example:

    -> show /sp/ system_location

 /SP
    Properties:
        system_location = ru_06

->

  6. If the location in the ILOM SP is different from the OEDA location, then modify the location in the ILOM SP to match the value from OEDAILOM.

    Note:

    Ensure that there are no duplicate locations in the OEDA rack configuration XML file.

    Set the system location in the ILOM SP with the following command:

    -> set /SP/ system_location=RU16

  7. For a database server with InfiniBand Network Fabric only, log in to the server and reclaim the unused disk space.

    Disk space for unused system partitions must be reclaimed before installing software on all database servers with InfiniBand Network Fabric. Use the following command to reclaim the disk space: /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim. This command is not required on RoCE-based Exadata database servers.

    reclaimdisks.sh performs partition reconfiguration and a few other necessary configuration changes that take typically less than a minute to complete.

  8. Log in to the server where you unzipped the OEDA configuration files and run the applyElasticConfig.sh script.

    For example, if you unzipped the OEDA ZIP file into the /opt/oracle.SupportTools/onecommand/linux-x64 directory, then use the following commands: 

    # cd /opt/oracle.SupportTools/onecommand/linux-x64

# ./applyElasticConfig.sh -cf customer_name-configFile.xml

    The applyElasticConfig.sh script performs network configuration for the new servers. The new servers are restarted at the end of the process.

Parent topic: Configuring Oracle Exadata Database Machine

6.15 Using the OEDA setuprootssh Utility

You can set up key-based authentication for the root user by using the setuprootssh.sh utility included with OEDA.

Starting with the April 2020 release of Oracle Exadata Deployment Assistant (OEDA), the setuprootssh.sh utility is located in the same directory as the install and config scripts.
  • To set up the required keys for SSH key-based authentication, run the setuprootssh.sh utility.
    ./setuprootssh.sh -cf config.xml
    For config.xml, specify the full path name to the deployment file created by OEDA.

    For each physical and virtual machine (VM) host in the deployment file, the setuprootssh.sh utility creates an SSH key pair. The keys are saved in the OEDA WorkDir using the following file naming conventions:

    • Private key: id_rsa.short_hostname.root
    • Public key: id_rsa.short_hostname.root.pub

    In the key file names, short_hostname is the corresponding server host name without any domain name qualification.

    Where possible, the utility also installs the public key on the host and disables password-based authentication for the root user.

    If the utility cannot install the public key on the host, then the key file remains ready for later use in the OEDA WorkDir. For example, this occurs to keys for VM hosts if you run the utility before you create the VMs.

  • If your system has a non-default root password, you must specify it by using the -p root_password option.
  • Use the -h option to display the usage information.

Parent topic: Configuring Oracle Exadata Database Machine

6.16 Loading the Configuration Information and Installing the Software

You can use Oracle Exadata Deployment Assistant to configure your rack, or you can do it manually.

Oracle Exadata Deployment Assistant (OEDA) loads the network settings, creates the user accounts, installs Oracle Database software, and secures the system based on the information the configuration files.

See Also:

Using Oracle Exadata Deployment Assistant for descriptions of the configuration files generated by OEDA.

Parent topic: Configuring Oracle Exadata Database Machine

6.16.1 Configuring Oracle Exadata Database Machine Using OEDA

Use Oracle Exadata Deployment Assistant (OEDA) to configure your engineered system.

The deployment phase of OEDA can be performed on one of the database servers of your new Oracle Exadata or any other host that is connected to the same network as the new engineered system rack. Do not run install.sh from a storage server. If you plan to run OEDA on a host outside of the new rack, then create the OEDA home directory and Workdir, as described below, on the host machine instead.

Note:

OEDA configures all Exadata components, including the database servers (with or without virtualization), storage servers, and the RDMA Network Fabric. For successful configuration, OEDA requires access to:

  • The client and administration network interfaces on the database servers, including the VM server host and the VM guests where applicable.
  • The administration network interfaces on the storage servers.
  • The administration network interfaces on the RDMA Network Fabric switches.

Ensure that you run the OEDA deployment phase on a host with access to all of the required network interfaces.

Note:

Running OEDA from a client machine outside of the rack means the software image files are transferred over the network, causing the process to be slower.

Note:

For this procedure, Oracle VM refers to OVM and KVM. Starting with Oracle Exadata Database Machine X8M-2, Oracle Linux KVM is the virtualization technology for systems that use RoCE Network Fabric. Otherwise, Oracle VM is a Xen-based virtualization technology, which is used across Oracle Exadata Database Machine systems that use InfiniBand Network Fabric.

You must have completed the other tasks described in Overview of Oracle Exadata Deployment Assistant before starting this task.
  1. If you are installing Oracle VMs on the database servers, then run /opt/oracle.SupportTools/switch_to_ovm.sh on each database server.
  2. Disk space for unused system partitions must be reclaimed before installing software on all database servers with InfiniBand Network Fabric. Use the following command to reclaim the disk space: /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim. This command is not required on RoCE-based Exadata database servers.
  3. Create a directory for the extracted OEDA files on the host where you will run install.sh.
    OEDA will not run if staged on the root file system (/)of any database server. Create a directory to store the latest version of OEDA and all required files on a file system that has at least 30 GB of available free space.
    • For bare-metal database servers, create a directory on the /u01 file system, for example /u01/onecommand or /u01/oeda.
    • For Oracle VM servers, create a directory on the /EXAVMIMAGES file system.
  4. Download the latest version of OEDA for the operating system of the client or server where you will run install.sh.
    Unzip the downloaded file into the OEDA home directory you created. This will create a directory like /u01/onecommand/linux-x64. This directory is referred to as the OEDA home directory.
  5. Download the required image files for the Oracle software.
    OEDA requires patches available from My Oracle Support or Oracle Software Delivery Cloud ( https://edelivery.oracle.com). Review the README file for OEDA to determine the file name or patch number, which is based on the version of Oracle software being installed. Place the files in the WorkDir sub-directory in the OEDA home directory.
    • If you are installing on a bare-metal database server, then download the patches, as specified in Section 2 of the OEDA README file. After downloading the patch ZIP file, extract the contents into the WorkDir sub-directory.

    • If you are configuring Oracle VM on the database servers, then download the gold image files as indicated in the topic "Oracle Exadata Deployment Assistant Support for Oracle Virtual Machines" in the OEDA README file. After downloading the gold image ZIP files, extract the contents into the WorkDir sub-directory.

      A gold image is a copy of a software-only, installed Oracle home. It is used to copy an image of an Oracle home to a new host on a new file system to serve as an active, usable Oracle home.

  6. Copy the required OEDA configuration files to the WorkDir sub-directory on the database server or Oracle VM.

    When you used OEDA to specify the configuration details for your new engineered system, it created several configuration files and a ZIP file that contains all the generated files. The required files are also listed at the bottom of the InstallationTemplate.html file that was generated by OEDA. You can copy the ZIP file and extract it into the WorkDir sub-directory, or copy the required files directly into the WorkDir sub-directory.

  7. Log in to the database server or Oracle VM and go to the OEDA home directory.
  8. Run the install.sh script to install and deploy the software on your new engineered system.

    In the following command syntax, configuration_file is the full path name for the main configuration file, CustomerName-rackname.xml

    # ./install.sh -cf configuration_file [-s step_number |  \
    -r step_numberA stepnumberB]

    You can use the following options with the install.sh script:

    • -cf file: Configuration file name
    • -h: Help output for the script
    • -l: List the steps of OEDA
    • -r {n-N | n }: Run steps n through N of OEDA, or run the specified step.
    • -s n: Run step n of OEDA.
    • -u {n-N | n }: Undo steps n through N, or one step.

Note:

Always use the latest OEDA patch listed in My Oracle Support note 888828.1. Refer to the README in the patch for the latest steps.

The following steps are performed by the OEDA deployment script. However, the actual steps for a specific deployment may differ depending on the deployment options chosen. For example, the "Update Nodes for Eighth Rack" step is performed only if you are deploying an Eighth Rack configuration. 

1. Validate Configuration File
2. Update Nodes for Eighth Rack
3. Create Virtual Machine
4. Create Users
5. Setup Cell Connectivity
6. Create Cell Disks
7. Create Grid Disks
8. Configure Alerting
9. Install Cluster Software
10. Initialize Cluster Software
11. Install Database Software
12. Relink Database with RDS
13. Create ASM Diskgroups
14. Create Databases
15. Apply Security Fixes
16. Install Exachk
17. Setup ASR Alerting
18. Create Installation Summary
19. Resecure Machine

Related Topics

Parent topic: Loading the Configuration Information and Installing the Software

6.16.2 Using Oracle Exadata Deployment Assistant in Conjunction with Key-Based Authentication

You can use the -sshkeys option to use SSH key-based authentication in conjunction with OEDA.

Starting with the April 2020 release of Oracle Exadata Deployment Assistant (OEDA), you can use SSH keys for root user authentication on Oracle Exadata Racks.

To use this feature, you must first set up the required SSH keys. The easiest way to do this is to use the setuprootssh.sh utility included with OEDA. For example: 

./setuprootssh.sh -cf config.xml

For more information about the setuprootssh.sh utility, see Using the OEDA setuprootssh Utility.

After you set up the required SSH keys, add the -sshkeys option to the regular command-line options when you start up OEDA. For example: 

./install.sh -cf config.xml -s 1 -sshkeys -usesu

When you specify the -sshkeys option, OEDA uses SSH key-based authentication. In conjunction with this option, the OEDA WorkDir must contain the SSH private key for each host in the deployment file (config.xml). The private keys must conform to the following file naming convention: 

id_rsa.short_hostname.root

In the key file names, short_hostname is the corresponding server host name without any domain name qualification.

As shown in the previous example, when you specify the -sshkeys option, you must also specify the -usesu option to instruct OEDA to run non-root commands from the root user using su.

When you first log in to a host following the Resecure Machine deployment step, you are prompted to reset the root password. This still occurs even when SSH key-based authentication is enabled, and password-based authentication is disabled.

Parent topic: Loading the Configuration Information and Installing the Software

6.16.3 Using Oracle Exadata Deployment Assistant on Systems with Non-Default root Passwords

You can use the -p option to prompt for a password instead of resetting the root password to the expected default value.

Starting with the November 2016 release, Oracle Exadata Deployment Assistant (OEDA) supports deployment on Oracle Exadata Racks that already have non-default root passwords. You can perform the deployment without having to change the root password back to the default. In earlier releases, OEDA expected the root password to be the default on all the servers in an Oracle Exadata Rack.

With the new non-default password feature, root passwords can have different values on each node in a rack. If non-default passwords are already set, add the -p option to the existing command line options when you run OEDA. For example: 

./install.sh -cf <config.xml> -s 1 -p

The -p option causes OEDA to prompt for the root password. Make sure you enter the correct password because an incorrect password disables root access for 10 minutes, and after 5 failed login attempts, the server is locked down.

Related Topics

Parent topic: Loading the Configuration Information and Installing the Software

6.17 Using the OEDA changePassword Utility

You can change the password of the grid and oracle users across all clusters by using the changePassword.sh utility included with OEDA.

The utility is located in the same directory as the install and config scripts.
You cannot use this utility if the servers are locked down.
  • To change the password of both oracle and grid users on all database servers and clusters, run the changePassword command.
    ./changePassword.sh -cf config.xml -userName root
    For config.xml, specify the full path name to the deployment file created by OEDA.
    The utility prompts for the current root password and allows you to change the password of any users in the rack.
  • To change the password for a specific user, such as oracle or grid, use the -userName user_name option with the command.
  • To change the password for all users within a specific cluster, use the -clusterName cluster_name option with the command.
  • Use the -h option with the command to display the usage information.

Parent topic: Configuring Oracle Exadata Database Machine

6.18 Installing Oracle Enterprise Manager Cloud Control

Oracle Exadata Database Machine should be monitored using Oracle Enterprise Manager Cloud Control. Ensure that the Enterprise Manager environment is configured for high availability and disaster recovery to ensure sufficient monitoring of the Exadata environments.

Related Topics

Parent topic: Configuring Oracle Exadata Database Machine

6.19 Adding a VM Cluster to Oracle Exadata Database Machine Using OEDA

You can use Oracle Exadata Deployment Assistant (OEDA) to add a virtual machine (VM) cluster to an existing Oracle Exadata Database Machine deployment.

The procedure has the following prerequisites:

  • The system must support an existing VM deployment.

  • You must have free space on your system to accommodate the new VM cluster. Specifically:

    • You must not have the maximum number of VM clusters already deployed.
    • You must have available CPU, memory, and local storage resources on the database servers.
    • You must have available storage space on the storage servers.

  • You require a current copy of the OEDA XML system configuration file.

To add a VM cluster to an existing Oracle Exadata Database Machine deployment:

  1. Start the OEDA Web-based configuration tool.
  2. Import the existing XML system configuration file.

    Use the Import option located in the drop-down menu at the top of the page.

  3. Click Clusters in the navigation menu on the left of the page.

    The Define Clusters section displays the configuration information for the existing clusters.

  4. Click the Add Clusters button.

    The Add Clusters button contains the plus symbol (+) and is adjacent to the existing cluster tabs.

    Clicking the Add Clusters button adds a tab in the Define Clusters section to contain the configuration details for the new cluster.

  5. In the new cluster tab, specify the configuration details for the new cluster. Then, click Apply.

    OEDA now displays the Diskgroups section.

  6. In the Diskgroups section, click the tab for the storage that is associated with the newly defined cluster.
  7. Specify the configuration details for the storage that is associated with the new cluster. Then, click Apply.

    OEDA now displays the Create Database Home section.

  8. Click the Add Database Home button, which contains the plus symbol (+) and is adjacent to the existing tabs.

    A new tab appears in the Create Database Home section to contain the configuration details for a new database home.

  9. In the new tab, specify the configuration details for the new database home. Then, click Apply.

    Ensure that you select the newly defined cluster as the location to contain the new database home.

    Afterward, OEDA displays the Create Database section.

  10. Click the Add Database button, which contains the plus symbol (+) and is adjacent to the existing tabs.

    A new tab appears in the Create Database section to contain the configuration details for a new database.

  11. In the new tab, specify the configuration details for the new database. Then, click Apply.

    Ensure that you select the newly defined database home to support the new database.

    Afterward, OEDA displays the Cluster Networks section.

  12. For each network in the Cluster Networks section, select the tab that is associated with the new cluster and specify the network settings. Then, click Apply.

    As you step through the network settings, ensure that you specify valid IP address settings that are compatible with the existing system configuration.

    Afterward, OEDA displays the Alerting section.

  13. Click Comments in the navigation menu. Then, click Apply and Download to save the updated system configuration information.
  14. Unzip the zip file located in the directory containing the newly saved system configuration information.

    The directory containing the newly saved system configuration information contains multiple files, including the updated XML system configuration file and HTML installation template. However, the zip file contains additional cluster-specific XML configuration files.

  15. Locate the cluster-specific XML configuration file for the new cluster.

    The new cluster-specific XML configuration file is located in the unzipped directory that you created in the previous step. The names of the cluster-specific XML configuration files contain a numeric sequence that reflects the order of cluster creation, with the newest cluster having the highest number.

  16. Use the cluster-specific XML configuration file to deploy the new cluster.

    Use the OEDA deployment utility (install.sh) in conjunction with the cluster-specific XML configuration file for the new cluster.

    Perform all of the deployment steps except for the following, which do not apply. Ensure that you do not perform the following deployment steps as they may impact system performance and availability:

    • Update Nodes for Eighth Rack (For Eighth Rack systems only)
    • Calibrate Cells
    • Create Cell Disks
    • Resecure Machine

    Also, when you perform the Create Installation Summary step, ensure that you use the -nocalibratecell option to ensure that cell calibration is not performed during that step. For example, if the step number for the Create Installation Summary step is 18: 

    $ ./install.sh -cf cluster4.xml -s 18 -nocalibratecell

    To view a complete list of the deployment steps and the corresponding step numbers, use install.sh with the -l option. For example: 

    $ ./install.sh -cf cluster4.xml -l

1. Validate Configuration File
2. Setup Required Files
3. Create Users
4. Setup Cell Connectivity
5. Verify RDMA Network Connectivity
6. Calibrate Cells
7. Create Cell Disks
8. Create Grid Disks
9. Configure Cell Alerting
10. Install Cluster Software
11. Initialize Cluster Software
12. Install Database Software
13. Relink Database with RDS
14. Create ASM Diskgroups
15. Create Databases
16. Apply Security Fixes
17. Setup ASR Alerting
18. Create Installation Summary
19. Resecure Machine

    To run an individual step, use install.sh with the -s option and specify the step number. For example, to run step 1: 

    $ ./install.sh -cf cluster4.xml -s 1

    To run a range of steps, use install.sh with the -r option and specify the step range. For example, to run steps 12-16: 

    $ ./install.sh -cf cluster4.xml -r 12-16

Related Topics

Parent topic: Configuring Oracle Exadata Database Machine