5 Configuring Oracle Exadata Database Machine

This chapter describes how to configure the system, accounts, and software for Oracle Exadata Database Machine.

Note:

  • For ease of reading, the name "Oracle Exadata Rack" is used when information refers to both Oracle Exadata Database Machine and Oracle Exadata Storage Expansion Rack.

  • The procedures in this chapter are applicable to Oracle Exadata Storage Expansion Rack as well as Oracle Exadata Database Machine.

  • The procedures in this chapter use the information from Oracle Exadata Database Machine Deployment Assistant (OEDA). Be sure to run OEDA before starting the procedures in this chapter.

5.1 Configuring Sun Datacenter InfiniBand Switch 36 Switch

The following procedure describes how to configure the Sun Datacenter InfiniBand Switch 36 switches.

  1. Log in to the first database server as the root user. The first database server is the lowest database server in the rack, which is rack position U16.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  2. Use SSH to log in to the Sun Datacenter InfiniBand Switch 36 switch as the ilom-admin user. Find the default IP address for the Sun Datacenter InfiniBand Switch 36 switch for your system in "Default IP Addresses". The following is an example of the command:

    ssh ilom-admin@192.168.1.201
    
  3. Use spsh to open the Integrated Lights Out Manager (ILOM) command line interface.

  4. Configure the network interface using the following commands:

    set /SP/network pendingipdiscovery=static pendingipaddress=pending_ip \
    pendingipgateway=pending_gw pendingipnetmask=pending_nm
    
    set /SP/network commitpending=true 
    

    In the preceding commands, pending_ip, pending_gw, and pending_nm are IP addresses defined by the network administrator.

  5. Use the exit command to exit the interface.

  6. Edit the /etc/hosts file to set the IP address and host name using an editor such as vi. The following is an example of the updated file:

    #Do not remove the following link, or various programs
    #that require network functionality will fail.
    127.0.0.1     localhost.localdomain localhost
    10.7.7.32     dm01sw-ib1-ib2.example.com trnasw-ib2
    

    It is required that the second line has both fully-qualified and non-fully-qualified names.

  7. Use the ILOM interface to configure the IP address, host name, NTP servers, and DNS servers.

  8. Restart the switch.

  9. Examine the firmware version using the following command:

    # version
    

    Refer to My Oracle Support Note 888828.1 for the current firmware version.

  10. Check the health of the switch using the following command:

    # showunhealthy
    
    OK - No unhealthy sensors
    
  11. Run the environment test using the following command:

    # env_test
    
    NM2 Environment test started:
    Starting Voltage test:
    Voltage ECB OK
    Measured 3.3V Main = 3.28 V
    Measured 3.3V Standby = 3.42 V
    Measured 12V =12.06 V
    Measured 5V =5.03 V
    Measured VBAT =3.06 V
    Measured 2.5V =2.53 V
    Measured 1.8V =1.79 V
    Measured I4 1.2V =1.22 V
    Voltage test returned OK
    Starting PSU test:
    PSU 0 present
    PSU 1 present
    PSU test returned OK
    Starting Temperature test:
    Back temperature 30.50
    Front temperature 33.88
    ComEx temperature 34.12
    I4 temperature 56,
    maxtemperature 57
    Temperature test returned OK
    Starting FAN test:
    Fan 0 not present
    Fan 1 running at rpm 12946
    Fan 2 running at rpm 12684
    Fan 3 running at rpm 12558
    Fan 4 not present
    FAN test returned OK
    Starting Connector test:
    Connector test returned OK
    Starting I4 test:
    I4 OK
    All I4s OK
    I4 test returned OK
    NM2 Environment test PASSED
    
  12. Enable the InfiniBand Subnet Manager using the following command:

    # enablesm
    

    Note:

    If you get an error indicating the InfiniBand Subnet Manager is already running, then restart it as follows:

    # disablesm
    
    # enablesm
    
  13. Verify the IP address is correct using the following command:

    # ifconfig eth0
    
    eth0    Link encap:Ethernet HWaddr 00:E0:4B:2A:07:2B
            inet addr:172.16.10.32 Bcast:172.16.10.255
    Mask:255.255.255.0
            inet6 addr:fe80::2e0:3a00:fe2a:61e/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
            RX packets:11927 errors:0 dropped:0 overruns:0 frame:0
            TX packets:89 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:720262 (703.3 KiB)TX bytes:11402 (11.1 KiB)
    
  14. Verify the host name is correct using the following command:

    # hostname
    
    dm01sw-ib2.example.com
    
  15. Set the Subnet Manager Master as described in "Setting the Subnet Manager Master on Oracle Exadata Database Machine Full Rack and Oracle Exadata Database Machine Half Rack". This step is needed for Oracle Exadata Database Machine Half Rack and Oracle Exadata Database Machine Full Rack.

  16. Log out from the InfiniBand switch using the following command:

    # exit
    

Note:

If the network settings did not appear after restart, then power cycle the switch by removing both power cords for one minute.

5.1.1 Setting the Subnet Manager Master on Oracle Exadata Database Machine Full Rack and Oracle Exadata Database Machine Half Rack

Oracle Exadata Database Machine X3-2 systems and Oracle Exadata Database Machine X2-2 systems have three Sun Datacenter InfiniBand Switch 36 switches. Starting with Oracle Exadata Database Machine X4-2, Oracle Exadata Database Machine Two-Socket Systems have two Sun Datacenter InfiniBand Switch 36 switches.

The switch located in rack unit 1 (U1) is referred to as the spine switch. The other two switches are referred to as the leaf switches. The location of the leaf switches is as follows:

  • Oracle Exadata Database Machine Two-Socket Systems (X3-2 and later): rack unit 20 (U20) and rack unit 22 (U22)

  • Oracle Exadata Database Machine X2-2 racks: rack unit 20 (U20) and rack unit 24 (U24)

  • Oracle Exadata Database Machine Eight-Socket Systems (X2-8 and later) Full Racks: Rack unit 21 (U21) and rack unit 23 (U23)

The spine switch is the Subnet Manager Master for the InfiniBand subnet. The Subnet Manager Master has priority 8, and can be verified using the following procedure:

  1. Log in to the spine switch as the root user.

  2. Run the setsmpriority list command.

    The command should show that smpriority has a value of 8. If smpriority has a different value, then do the following:

    1. Use the disablesm command to stop the Subnet Manager.

    2. Use the setsmpriority 8 command to set the priority to 8.

    3. Use the enablesm command to restart the Subnet Manager.

The leaf switches are the Standby Subnet Managers with a priority of 5. This can be verified using the preceding procedure, substituting a value of 5 in the setsmpriority command above.

Note:

Oracle Exadata Database Machine Half Rack with Sun Fire X4170 Oracle Database Servers include two Sun Datacenter InfiniBand Switch 36 switches, which are set to priority 5.

To determine the Subnet Manager Master, log in as the root user on any InfiniBand switch, and run the getmaster command. The location of the Subnet Manager Master is displayed. The following is an example of the output from the getmaster command:

# getmaster
20100701 11:46:38 OpenSM Master on Switch : 0x0021283a8516a0a0 ports 36 Sun DCS 36
QDR switch dm01sw-ib1.example.com enhanced port 0 lid 1 lmc 0

The preceding output shows the proper configuration. The Subnet Master Manager is running on spine switch dm01sw-ib1.example.com.

If the spine switch is not the Subnet Manager Master, then do the following procedure to set the Subnet Manager Master:

  1. Use the getmaster command to identify the current location of the Subnet Manager Master.

  2. Log in as the root user on the leaf switch that is the Subnet Manager Master.

  3. Disable Subnet Manager on the switch. The Subnet Manager Master relocates to another switch.

  4. Use the getmaster command to identify the current location of the Subnet Manager Master. If the spine switch is not Subnet Manager Master, then repeat steps 2 and 3 until the spine switch is the Subnet Manager Master.

  5. Enable Subnet Manager on the leaf switches that were disabled during this procedure.

Note:

  • If the InfiniBand network consists of four or more racks cabled together, then only the spine switches should run Subnet Manager. The leaf switches should have Subnet Manager disabled on them.

  • Oracle Exadata Database Machine Half Racks with Sun Fire X4170 Oracle Database Servers, and Oracle Exadata Database Machine Quarter Racks have two Sun Datacenter InfiniBand Switch 36 switches, and both are set to priority 5. The master is the one with the lowest GUID.

See Also:

5.2 Configuring the Cisco Ethernet Switch

You must perform an initial configuration of the Cisco Ethernet switch.

5.2.1 Configuring the Cisco Catalyst 4948 Ethernet Switch

The Cisco Catalyst 4948 Ethernet switch supplied with Oracle Exadata Rack is minimally configured during installation.

The minimal configuration disables IP routing, and sets the following:

  • Host name

  • IP address setup

  • Subnet mask

  • Default gateway

  • Domain name

  • Name server

  • NTP server

  • Time

  • Time zone

Before configuring the switch, note the following:

  • The Cisco Ethernet switch should not be connected until the running configuration has been verified, and any necessary changes have been made by the network administrator.

  • The Cisco Ethernet switch should not be connected to the customer network until the IP addresses on all components have been configured in Oracle Exadata Rack. This is to prevent any duplicate IP address conflicts which are possible due to the default addresses set in the components when shipped.

Note that the Cisco 4948E-F switch supports multiple uplinks to the customer network by utilizing ports 49 - 52. This is a more complicated switch setup due to the redundant connectivity, and should be performed by the customer's network administrator.

The following procedure describes how to configure the Cisco Ethernet switch. Configuration should be done with the network administrator.

  1. Connect a serial cable from the Cisco switch console to a laptop or similar device. An Oracle supplied rollover cable is pre-installed on the Cisco serial console port. Obtain the appropriate adapter and connect it at the end of the rollover cable. An Oracle P/N 530-3100 RJ45-DB9 adapter as used on ILOM ports will also work, connected at the end of the network cable.

  2. Ensure the terminal session is recorded on the laptop by logging the output. The output can be used as a reference that the switch has been configured correctly. The default serial port speed is 9600 baud, 8 bits, no parity, 1 stop bit, and no handshake.

    Switch con0 is now available
    Press RETURN to get started.
    
  3. Change to the enable mode.

    Switch> enable
    Password: ******
    Switch# 
    

    Note:

    If you do not have the password, then contact Oracle Support Services.

  4. Check the current version on the swtich.

    Switch# show version 
    Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-
    IPBASEK9-M), Version 15.2(3)E2, RELEASE SOFTWARE (fc1)
    Technical Support: http://www.cisco.com/techsupport
    Copyright (c) 1986-2014 by Cisco Systems, Inc.
    Compiled Tue 11-Mar-14 18:28 by prod_rel_team
    
    ROM: 12.2(44r)SG12
    zdlra1sw-ip uptime is 1 minute
    System returned to ROM by reload
    System image file is "bootflash:cat4500e-ipbasek9-mz.152-3.E2.bin"
    Hobgoblin Revision 22, Fortooine Revision 1.40
    ...
    
    Configuration register is 0x2102
    
    Switch#
    

    The version of the Cisco 4948E-F switch firmware purchased and shipped by Oracle with Exadata Database Machine X6 is IPBASEK9-MZ, which includes telnet and ssh support. Currently the full release version string is cat4500e-ipbasek9-mz.152-3.E2.bin.

  5. Configure the network for a single VLAN. The following example assumes you are using IPv4 addressing.

    Switch# configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    Switch(config)# interface vlan 1
    Switch(config-if)# ip address 10.7.7.34 255.255.255.0
    Switch(config-if)# end
    Switch# *Sep 15 14:12:06.309:%SYS-5-CONFIG_I:Configured from console by console
    Switch# write memory
    Building configuration...
    Compressed configuration from 2474 bytes to 1066 bytes [OK ]
    
  6. (Optional) Disable the default IP routing setting, and configure the default gateway. This step is required if IP routing is not used on the switch.

    Switch#configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    Switch(config)#no ip routing
    Switch(config)#ip default-gateway 10.7.7.1
    Switch(config)#end 
    *Sep 15 14:12:46.309:%SYS-5-CONFIG_I:Configured from console by console
    Switch#write memory
    Building configuration...
    Compressed configuration from 2492 bytes to 1070 bytes [OK ]
    

    Note:

    If IP routing is required on the switch, then leave the IP routing setting as the default, and configure the default gateway.

    Switch#configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    Switch(config)#ip route 0.0.0.0 0.0.0.0 10.7.7.1
    Switch(config)#end
    *Sep 15 14:13:26.013:%SYS-5-CONFIG_I:Configured from console by console
    Switch#write memory
    Building configuration...
    Compressed configuration from 2502 bytes to 1085 bytes [OK ]
    
  7. Set the host name of the switch.

    Switch#configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    Switch(config)#hostname exa1sw-ip
    exa1sw-ip(config)#end
    exa1sw-ip#write memory
    Building configuration...
    Compressed configuration from 3789 bytes to 1469 bytes [OK ]
    

    The system host name is used as the prompt name.

  8. Configure up to three DNS servers.

    exa1sw-ip#configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    exa1sw-ip(config)#ip domain-name example.com
    exa1sw-ip(config)#ip name-server 10.7.7.3
    exa1sw-ip(config)#ip name-server 198.51.100.5 
    exa1sw-ip(config)#ip name-server 10.8.160.1
    exa1sw-ip(config)#end 
    *Sep 15 14:26:37.045:%SYS-5-CONFIG_I:Configured from console by console
    exa1sw-ip#write memory
    Building configuration...
    Compressed configuration from 2603 bytes to 1158 bytes [OK ]
    

    If you do not have DNS service available, you must still set the domain-name for the SSH keys to be configurable.

  9. (Optional) Set the password.

    exa1sw-ip #configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    exa1sw-ip(config)#enable password welcome1
    exa1sw-ip(config)#enable secret welcome1 
    The enable secret you have chosen is the same as your enable password.
    This is not recommended.Re-enter the enable secret.
    exa1sw-ip(config)#end
    exa1sw-ip#write memory 
    *Sep 15 14:25:05.893:%SYS-5-CONFIG_I:Configured from console by console
    Building configuration...
    Compressed configuration from 2502 bytes to 1085 bytes [OK ]
    
  10. Verify telnet access is disabled.

    exa1sw-ip#configure terminal
    Enter configuration commands,one per line. End with CNTL/Z.
    exa1sw-ip(config)#line vty 0 15
    exa1sw-ip(config)#login
    % Login disabled on line 1, until 'password' is set
    % Login disabled on line 2, until 'password' is set
     ...
    % Login disabled on line 16, until 'password' is set
    exa1sw-ip(config)#end
    

    If the login command returns output as shown above, then telnet access has been disabled. If instead you get a prompt, then disable telnet access.

    exa1sw-ip(config-line)#no password
    exa1sw-ip(config-line)#end
    exa1sw-ip#write memory 
    Building configuration...
    Compressed configuration from 3786 bytes to 1468 bytes [OK ]
    
  11. (Optional) Configure SSH access on the Cisco switch.

    exa1sw-ip#configure terminal
    Enter configuration commands,one per line. End with CNTL/Z.
    exa1sw-ip(config)#crypto key generate rsa
    % You already have RSA keys defined named exa1sw-ip.example.com.
    % Do you really want to replace them? [yes/no]: yes
    Choose the size of the key modulus in the range of 360 to 2048 for
    your General Purpose Keys. Choosing a key modulus greater than 512
    may take a few minutes.
    
    How many bits in the modulus [512]: 768
    % Generating 768 bit RSA keys, keys will be non-exportable...[OK]
    
    exa1sw-ip(config)#
    exa1sw-ip(config)#username admin password 0 welcome1
    exa1sw-ip(config)#line vty 0 15
    exa1sw-ip(config-line)#transport input ssh
    exa1sw-ip(config-line)#exit
    exa1sw-ip(config)#aaa new-model
    exa1sw-ip(config)#
    exa1sw-ip(config)#ip ssh time-out 60
    exa1sw-ip(config)#ip ssh authentication-retries 3
    exa1sw-ip(config)#ip ssh version 2
    exa1sw-ip(config)#end
    *Sep 15 14:26:37.045: %SYS-5-CONFIG_I: Configured from console by console
    exa1sw-ip#write memory
    Building configuration...
    Compressed configuration from 2603 bytes to 1158 bytes[OK]
    

    If the login command returns output as shown above, then telnet access has been disabled. If instead you get a prompt, then disable telnet access.

    exa1sw-ip(config-line)#no password
    exa1sw-ip(config-line)#end
    exa1sw-ip#write memory 
    Building configuration...
    Compressed configuration from 3786 bytes to 1468 bytes [OK ]
    
  12. Set the clock and time zone. The switch keeps internal time in Coordinated Universal Time (UTC) format.

    • To use UTC, use the following command:

      no clock timezone global configuration
      
    • To use a time zone, use the following command:

      clock timezone zone hours-offset [minutes-offset]
      

      In the preceding command, zone is the time zone to display when standard time in effect, hours-offset is the hours offset from UTC, and minutes-offset is the minutes offset from UTC.

    • Daylight savings time (or summer time) is disabled by default. To set summer time hours, use the following command:

      clock summer-time zone recurring [week day monthhh:mm week day month \
      hh:mm[offset]]
      

      In the preceding command, zone is the time zone to be displayed when summer time is in effect (EDT, for example), week is the week of the month (1 to 5 or last), day is the day of the week (Sunday, Monday, ...), month is the month (January, February, ...), hh:mm is the hours and minutes in 24-hour format, and offset is the number of minutes to add during summer time. The default offset is 60 minutes.

    • To manually set the clock to any time use the following command, where the time specified is relative to the configured time zone:

      clock set hh:mm:ss month day year
      

      In the preceding command, hh:mm:ss is the time in 24-hour format, day is the day by date in the month, month is the name of the month, and year is the 4-digit year.

    To set the local time and time zone, ordering is important. For example, to set the local time to US Eastern time:

    exa1sw-ip#configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    exa1sw-ip(config)#clock timezone EST -5 
    exa1sw-ip(config)#clock summer-time EDT recurring
    exa1sw-ip(config)#end
    exa1sw-ip#clock set 21:00:00 August 09 2017
    exa1sw-ip#write memory
    Building configuration...
    Compressed configuration from 3784 bytes to 1465 bytes [OK ]
    exa1sw-ip#show clock
    21:00:06.643 EST Mon Aug 9 2017
    
  13. After setting the local time zone, you can configure up to two NTP servers.

    exa1sw-ip#configure terminal
    Enter configuration commands,one per line.End with CNTL/Z.
    exa1sw-ip(config)#ntp server 10.7.7.32 prefer
    exa1sw-ip(config)#ntp server 198.51.100.19
    exa1sw-ip(config)#end
    *Sep 15 14:51:08.665:%SYS-5-CONFIG_I:Configured from console by
    console
    exa1sw-ip#write memory
    Building configuration...
    Compressed configuration from 2654 bytes to 1163 bytes [OK ]
    exa1sw-ip#show ntp status
    <output will vary per network>
    exa1sw-ip#show clock
    21:00:23.175 EST Mon Aug 9 2017
    

    The preceding should show the NTP server synchronized to local time if the Cisco switch is connected to the network that has access to the NTP server.

  14. Verify the configuration using the following command:

    exa1sw-ip#show running-config
    
    Building configuration...
    
    Current configuration : 3923 bytes
    !
    !  Last configuration change at 02:40:49 GMT Wed Nov 12 2014
    !
    version 15.2
    no service pad
    service timestamps debug datetime msec
    service timestamps log datetime msec
    no service password-encryption
    service compress-config
    !
    hostname exa1sw-ip
    !
    boot-start-marker
    boot system bootflash:cat4500e-ipbasek9-mz.152-3.E2.bin
    boot-end-marker
    !
    enable secret 5 $1$mS8h$EaJrIECUxavfGH6vLZg1T.
    enable password welcome1
    !
    username admin password 0 welcome1
    !
    aaa new-model
    !
    !
    aaa session-id common
    ip subnet-zero
    ip domain-name example.com
    ip name-server 10.7.7.3
    ip name-server 198.51.100.5
    ip name-server 10.8.160.1
    !
    ip vrf mgmtVrf
    !
    vtp mode transparent
    !
    power redundancy-mode redundant
    !
    !
    spanning-tree mode pvst
    spanning-tree extend system-id
    !
    vlan internal allocation policy ascending
    !
    interface FastEthernet1
     ip vrf forwarding mgmtVrf
     no ip address
     speed auto
     duplex auto
    !
    interface GigabitEthernet1/1
     spanning-tree portfast
    !
    interface GigabitEthernet1/2
     spanning-tree portfast
    !
    ...
    !
    interface GigabitEthernet1/48
     media-type rj45
    !
    interface Vlan1
     ip address 10.7.7.34 255.255.255.0
    !
    interface Vlan48
     no ip address
     shutdown
    !
    ip default-gateway 10.7.7.1
    ip http server
    !
    control-plane
    !
    line con 0
     stopbits 1
    line vty 0 4
     password welcome1
     transport input ssh
    line vty 5 15
     password welcome1
     transport input ssh
    !
    ntp server 10.7.7.32 prefer
    ntp server 198.51.100.19
    end
    

    Note:

    If any setting is incorrect, the repeat the appropriate step. To erase a setting, enter no in front of the same command. For example, to erase the default gateway, use the following commands:

    exa1sw-ip#configure terminal
    Enter configuration commands, one per line. End with CNTL/Z.
    exa1sw-ip(config)#no ip default-gateway 10.7.7.1
    exa1sw-ip(config)#end
    exa1sw-ip# 
    *Sep 15 14:13:26.013: %SYS-5-CONFIG_I: Configured from console by console
    exa1sw-ip#write memory
    Building configuration...
    Compressed configuration from 2502 bytes to 1085 bytes[OK]
    
  15. Save the current configuration.

    exa1sw-ip#copy running-config startup-config
    Destination filename [startup-config]?
    Building configuration...
    Compressed configuration from 2654 bytes to 1189 bytes[OK]
    
  16. Exit from the session using the following command:

    exa1sw-ip#exit
    
    exa1sw-ip con0 is now available
    
    Press RETURN to get started.
    
  17. Disconnect the cable from the Cisco console.

    The Cisco switch must not be connected to the management network at this stage. The switch will be connected later after Oracle has configured the systems with the necessary IP addresses and you have worked with the field service engineer to make any additional changes necessary for connecting to the network.

  18. To check the Cisco swtich, attach a laptop computer to port 48, and ping the IP address of the Exadata internal management network to check the configuration.

5.2.2 Configuring the Cisco Nexus 93108-1G or 9348 Ethernet Switch

The Cisco Nexus 93108-1G or 9348 Ethernet switch supplied with Oracle Exadata Rack is minimally configured during installation.

Note that the Cisco Nexus 93108-1G or 9348 switch supports multiple uplinks to the customer network by utilizing the QSFP+ ports. This is a more complicated switch setup due to the redundant connectivity, and should be performed by the customer's network administrator.

Whether you are configuring the switch for the first time, or configuring a replacement switch, use the following procedures:

5.2.2.1 Performing the Initial Switch Configuration for the Cisco Nexus 93108-1G or 9348 Ethernet Switch

During the initial configuration, you reset the switch and use the Basic System Configuration Dialog to configure the switch.

Before configuring the switch, note the following:

  • The Cisco Ethernet switch should not be connected until the running configuration has been verified, and any necessary changes have been made by the network administrator.

  • The Cisco Ethernet switch should not be connected to the customer network until the IP addresses on all components have been configured in Oracle Exadata Rack. This is to prevent any duplicate IP address conflicts which are possible due to the default addresses set in the components when shipped.

Configuration should be done with the network administrator.

  1. Connect from the Cisco switch serial console to a laptop or similar device using the available RJ45 cable.
  2. Ensure the terminal session is recorded on the laptop by logging the output.
    The output can be used as a reference that the switch has been configured correctly. The default serial port speed is 9600 baud, 8 bits, no parity, 1 stop bit, and no handshake.
  3. Power on the switch.
  4. Log in as the admin user.
    User Access Verification
    exadatax7-adm0 login: admin
    Password: ********
    

    Note:

    If you do not have the password for the admin user, then contact Oracle Support Services.
  5. Erase the existing configuration.
    exadatax7-adm0# write erase
    
    Warning: This command will erase the startup-configuration.
    
    Do you wish to proceed anyway? (y/n)  [n] y
    
  6. Restart the system so you can perform the automated setup.
    exadatax7-adm0# reload
    
    This command will reboot the system. (y/n)?  [n] y
    
    2017 Aug 31 01:09:00 exadatax7-adm0 %$ VDC-1 %$ %PLATFORM-2-PFM_SYSTEM_RESET: Manual system restart from Command Line Interface
    
    
    CISCO SWITCH Ver7.59
    Device detected on 0:1:2 after 0 msecs  
    ...
    
  7. Switch to normal setup and, when asked if you want to enforce secure password standard, enter no, then enter a new password for the admin user.
    Running S93thirdparty-script...
    
    Populating conf files for hybrid sysmgr ...
    Starting hybrid sysmgr ...
    inserting /isan/lib/modules/klm_cisco_nb.o ... done
    
    Abort Auto Provisioning and continue with normal setup ? (yes/no) [n]: yes
    
             ---- System Admin Account Setup ----
    
    Do you want to enforce secure password standard (yes/no) [y]: no
    
      Enter the password for "admin": 
      Confirm the password for "admin": 
    
  8. When the Basic System Configuration Dialog appears, choose to enter the basic configuration dialog.
            ---- Basic System Configuration Dialog VDC: 1 ----
    
    This setup utility will guide you through the basic configuration of
    the system. Setup configures only enough connectivity for management
    of the system.
    
    Please register Cisco Nexus9000 Family devices promptly with your
    supplier. Failure to register may affect response times for initial
    service calls. Nexus9000 devices must be registered to receive 
    entitled support services.
    
    Press Enter at anytime to skip a dialog. Use ctrl-c at anytime
    to skip the remaining dialogs.
    
    Would you like to enter the basic configuration dialog (yes/no): yes
    
  9. In the basic configuration, you can use the default inputs until asked to enter the switch name.

    In this example, the switch has a name of test123sw-adm0.

      Create another login account (yes/no) [n]: 
      Configure read-only SNMP community string (yes/no) [n]: 
      Configure read-write SNMP community string (yes/no) [n]: 
      Enter the switch name : test123sw-adm0
    
    
  10. Respond no when asked to configure Out-of-band management configuration.
    Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: no
    
  11. Respond yes when asked to configure advanced IP options.
    Configure advanced IP options? (yes/no) [n]: yes
    
  12. Respond no when asked to configure static route (this will be configured later).
    Configure static route? (yes/no) [n]: no
    
  13. Enter the destination prefix and mask, and other values as prompted.
       Destination prefix : 10.100.100.0
    
       Destination prefix mask : 255.255.255.0
    
       Next hop IPv4 address : 10.100.100.1
    
  14. Skip configuring the DNS IPv4 addresses (this will be configured later).
    Configure the DNS IPv4 address? (yes/no) [n]: no
    
  15. Skip configuring the default domain name (this will be configured later).
    Configure the default domain name? (yes/no) [n]: no
    
  16. Accept the default responses until asked to configure SSH and the NTP server.
    Enable the telnet service? (yes/no) [n]: no
    Enable the ssh service? (yes/no) [y]: yes
       Type of ssh key you would like to generate (dsa/rsa) [rsa]: rsa
       Number of rsa key bits <1024-2048> [1024]: 1024
     
    Configure the ntp server? (yes/no) [n]: yes
         NTP server IPv4 address : 10.100.100.3
    
  17. Accept the default responses until asked to specify the CoPP system profile. Enter lenient.
     Configure default interface layer (L3/L2) [L2]: 
     Configure default switchport interface state (shut/noshut) [noshut]: 
     Configure CoPP system profile (strict/moderate/lenient/dense) [strict]: lenient
    
  18. After reviewing the configuration, save the configuration.
    The following configuration will be applied:
       no password strength-check
       switchname test123sw-adm0
      ...
    
    Would you like to edit the configuration? (yes/no) [n]: 
    
    Use this configuration and save it? (yes/no) [y]: yes
    
    [########################################] 100%
    Copy complete.
    
  19. Add the VLAN 1 IP address.
    test123sw-adm0(config)# feature interface-vlan
    test123sw-adm0(config)# interface vlan 1
    test123sw-adm0(config-if)# ip address 10.100.100.110/24
    test123sw-adm0(config-if)# no shutdown
    test123sw-adm0(config-if)# exit
    
  20. Set the spanning tree port type for ports 1-47.
    test123sw-adm0(config)# interface E1/1-47
    test123sw-adm0(config-if)# spanning-tree port type edge
    test123sw-adm0(config-if)# exit
    
  21. Set switchport on all 48 ports and set port 48 to a network port (instead of a host port).
    test123sw-adm0(config)# interface E1/1-48
    test123sw-adm0(config-if)# switchport
    test123sw-adm0(config-if)# exit
    test123sw-adm0(config)# interface E1/48
    test123sw-adm0(config-if)# spanning-tree port type network
    test123sw-adm0(config-if)# ip route 0.0.0.0/0 10.100.100.1
    
  22. Configure the DNS information.
    test123sw-adm0(config)# ip domain-name example.com
    test123sw-adm0(config)# ip name-server 10.100.100.2
    test123sw-adm0(config)# exit
    
  23. Save the current configuration.
    test123sw-adm0# copy running-config startup-config
    [########################################] 100%
    Copy complete.
    
  24. Optional: Set the clock, as described in the next topic.

5.2.2.2 Setting the Clock on the Cisco 93108-1G or 9348 Ethernet Switch

After you have performed the initial configuration, you can adjust the time used by the switch.

  1. Log in as the admin user.
  2. View the current time.
    test123sw-adm0(config)# show clock
    20:44:52.986 UTC Thu Aug 31 2017
    Time source is NTP
    
  3. Set the timezone appropriately.
    test123sw-adm0(config)# clock timezone PST -8 0
    
  4. View the modified time.
    test123sw-adm0(config)# show clock
    12:46:22.692 PST Thu Aug 31 2017
    Time source is NTP
    
  5. Save the configuration.
    test123sw-adm0# copy running-config startup-config 
    [########################################] 100%
    Copy complete.
    

5.3 Configuring the Power Distribution Units

The power distribution units (PDUs) are configured with a static IP address to connect to the network for monitoring. Ensure you have the following before connecting the PDU to the network:

  • Static IP address

  • Subnet mask

  • Default gateway

  • Ethernet cables

  • Laptop computer with either Oracle Solaris or Microsoft Windows

For systems less than full racks, the PDU Ethernet connections may use available ports in the Cisco 4948E-F switch. Note that for full rack systems, you cannot connect PDUs to the Cisco 4948E-F because all the ports on the switch are already reserved for other components.

The following procedure describes how to configure and connect the PDU to the network:

  1. Power off the PDU.

  2. Disconnect the power leads from the power source.

  3. Select an unused LAN network connection from the available network connections on the laptop as follows:

    • For Oracle Solaris:

      1. Log in as a super user.

      2. Use the dladm show-link command to find an unused Ethernet interface.

      3. Use the ifconfig -a command to determine which interface is being used.

      4. Use the following command to plumb the unused interface:

        # ifconfig interface plumb up
        

        In the preceding command, interface is the network interface determined in step 3.c.

      5. Using the following command to assign an IPv4 address and netmask to the interface:

        # ifconfig interface IPv4_address netmask + netmask \
          broadcast broadcast up
        

        In the preceding command, interface is the network interface, IPv4_address is the IP address, netmask is the netmask address, and broadcast is the broadcast address.

        Note:

        The network interfaces configured with the ifconfig command do not persist across system restarts.

    • For Microsoft Windows:

      1. Select Settings from the Start menu.

      2. Select Network Connections from the Settings menu.

      3. Select an unused local area connection from the Network Connections menu.

      4. Double-click Internet Protocol (TCP/IP) in the Local Area Connection Properties window. The Internet Protocol (TCP/IP) window appears.

      5. Select Use the following IP address.

      6. Enter the IP address in the IP address field. Do not use 192.168.0.1 because that is the default PDU metering unit address.

      7. Enter the IP address in the Subnet mask field.

      8. Leave the Default gateway field blank.

      9. Click OK.

  4. Connect the PDU and the laptop to two ports on the Cisco switch. The equipment currently using the Cisco switch must be disconnected temporarily.

  5. Connect the PDU power input lead to the power source. Only one PDU power input lead needs to be connected, as follows:

    • On PDUs with one power input lead, connect the one lead to the power source.

    • On PDUs with two power leads, connect the first power lead to the power source. The first power lead is labelled 0.

    • On PDUs with three power input leads, connect the middle power lead to the power source. The middle power lead is labelled 1.

  6. Use a Web browser to access the PDU metering unit by entering the factory default IP address for the unit, 192.168.0.1, in the address line of the browser. The Current Measurement page should appear.

  7. Click Network Configuration in the upper left of the page.

  8. Log in as the admin user on the PDU metering unit. Change the password after configuring the network.

  9. Confirm the DHCP Enable check box is not selected.

  10. Enter the following network settings for the PDU metering unit:

    • IP address

    • Subnet mask address

    • Default gateway

  11. Click Submit to set the network settings, and reset the PDU metering unit.

  12. Disconnect the PDU and laptop from the Cisco switch.

  13. Reconnect the two cables that were originally connected to the Cisco switch.

  14. Connect an Ethernet cable to the PDU metering unit RJ-45 Ethernet port and to the network.

  15. Log in to the PDU metering unit using a Web browser. Use the PDU metering unit's static IP address in the browser's address line. If the configuration was successful, then the Current Measurement page is displayed.

  16. Connect the remaining PDU power input leads to the power source.

5.3.1 Configuring the Threshold Settings for the Power Distribution Units

The PDU current can be monitored directly or with Oracle Enterprise Manager Grid Control. Configure the threshold settings to monitor the PDUs. The threshold settings depend on the size of Oracle Exadata Rack and type PDU. The configurable threshold values for each metering unit module and phase are Info low, Pre Warning, and Alarm.

Note:

The thresholds must be set in the Param Configuration section of the PDU metering unit. Setting threshold values using the Oracle Enterprise Manager Grid Control plug-in has no effect.

See Also:

5.3.1.1 PDU Thresholds for Oracle Exadata Database Machine X5-2 and Later

Starting with Oracle Exadata Database Machine X5-2, Oracle Exadata Configuration Assistant (OECA) is the only source for PDU thresholds. OECA is available on Oracle Technology Network at http://www.oracle.com/technetwork/database/exadata/oeca-download-2817713.html

If for some reason you are unable to utilize OECA, contact Oracle Support.

5.3.1.2 PDU Thresholds for Oracle Exadata Database Machine X4-2

This section contains the PDU thresholds for Oracle Exadata Database Machine X4-2.

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA is available on Oracle Technology Network.

5.3.1.2.1 PDU Thresholds for Oracle Exadata Database Machine X4-2 Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X4-2 Full Rack:

  • Table 5-1 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, Low-voltage PDU

  • Table 5-2 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, Low-voltage PDU

  • Table 5-3 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, High-voltage PDU

  • Table 5-4 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, High-voltage PDU

Table 5-1 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a single-phase, low-voltage PDU.

Table 5-1 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

17

22

A

Module 1, phase 3

0

19

24

B

Module 1, phase 1

0

19

24

B

Module 1, phase 2

0

17

22

B

Module 1, phase 3

0

20

24

Table 5-2 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a three-phase, low-voltage PDU.

Table 5-2 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

42

A and B

Module 1, phase 3

0

32

40

Table 5-3 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a single-phase, high-voltage PDU.

Table 5-3 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

16

20

A

Module 1, phase 3

0

18

23

B

Module 1, phase 1

0

18

23

B

Module 1, phase 2

0

16

20

B

Module 1, phase 3

0

18

23

Table 5-4 lists the threshold values for Oracle Exadata Database Machine X4-2 Full Rack using a three-phase, high-voltage PDU.

Table 5-4 Threshold Values for Oracle Exadata Database Machine X4-2 Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

16

21

5.3.1.2.2 PDU Thresholds for Oracle Exadata Database Machine X4-2 Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine X4-2 Half Rack:

  • Table 5-5 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, Low-voltage PDU

  • Table 5-6 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, Low-voltage PDU

  • Table 5-7 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, High-voltage PDU

  • Table 5-8 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, High-voltage PDU

Table 5-5 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a single-phase, low-voltage PDU.

Table 5-5 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

10

13

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

10

13

B

Module 1, phase 3

0

20

24

Table 5-6 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a three-phase, low-voltage PDU.

Table 5-6 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

17

22

A and B

Module 1, phase 2

0

19

24

A and B

Module 1, phase 3

0

15

20

Table 5-7 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a single-phase, high-voltage PDU.

Table 5-7 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

9

12

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

9

12

B

Module 1, phase 3

0

18

23

Table 5-8 lists the threshold values for Oracle Exadata Database Machine X4-2 Half Rack using a three-phase, high-voltage PDU.

Table 5-8 Threshold Values for Oracle Exadata Database Machine X4-2 Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

15

A and B

Module 1, phase 2

0

9

12

A and B

Module 1, phase 3

0

7

10

5.3.1.2.3 PDU Thresholds for Oracle Exadata Database Machine X4-2 Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack:

  • Table 5-9 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, Low-voltage PDU

  • Table 5-10 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, Low-voltage PDU

  • Table 5-11 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, High-voltage PDU

  • Table 5-12 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, High-voltage PDU

Table 5-9 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a single-phase, low-voltage PDU.

Table 5-9 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

10

13

Table 5-10 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a three-phase, low-voltage PDU.

Table 5-10 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

10

13

A and B

Module 1, phase 3

0

5

7

Table 5-11 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a single-phase, high-voltage PDU.

Table 5-11 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

12

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0.1

0.2

B

Module 1, phase 1

0

0.1

0.2

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

9

12

Table 5-12 lists the threshold values for Oracle Exadata Database Machine X4-2 Quarter Rack using a three-phase, high-voltage PDU.

Table 5-12 Threshold Values for Oracle Exadata Database Machine X4-2 Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

11

A and B

Module 1, phase 2

0

3

4

A and B

Module 1, phase 3

0

3

4

5.3.1.3 PDU Thresholds for Oracle Exadata Database Machine X3-2

This section contains the PDU thresholds for Oracle Exadata Database Machine X3-2.

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA is available on Oracle Technology Network.

5.3.1.3.1 PDU Thresholds for Oracle Exadata Database Machine X3-2 Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X3-2 Full Rack:

  • Table 5-13 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, Low-voltage PDU

  • Table 5-14 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, Low-voltage PDU

  • Table 5-15 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, High-voltage PDU

  • Table 5-16

    Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, High-voltage PDU

Table 5-13 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a single-phase, low-voltage PDU.

Table 5-13 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

14

18

A

Module 1, phase 3

0

19

24

B

Module 1, phase 1

0

19

24

B

Module 1, phase 2

0

14

18

B

Module 1, phase 3

0

20

24

Table 5-14 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a three-phase, low-voltage PDU.

Table 5-14 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

30

38

A and B

Module 1, phase 2

0

32

40

A and B

Module 1, phase 3

0

30

38

Table 5-15 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a single-phase, high-voltage PDU.

Table 5-15 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

18

22

B

Module 1, phase 1

0

18

22

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

18

23

Table 5-16 lists the threshold values for Oracle Exadata Database Machine X3-2 Full Rack using a three-phase, high-voltage PDU.

Table 5-16 Threshold Values for Oracle Exadata Database Machine X3-2 Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

17

21

A and B

Module 1, phase 2

0

17

21

A and B

Module 1, phase 3

0

15

19

5.3.1.3.2 PDU Thresholds for Oracle Exadata Database Machine X3-2 Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine X3-2 Half Rack:

  • Table 5-17 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, Low-voltage PDU

  • Table 5-18 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, Low-voltage PDU

  • Table 5-19 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, High-voltage PDU

  • Table 5-20 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, High-voltage PDU

Table 5-17 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a single-phase, low-voltage PDU.

Table 5-17 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

24

A

Module 1, phase 2

0

9

11

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

9

11

B

Module 1, phase 3

0

20

24

Table 5-18 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a three-phase, low-voltage PDU.

Table 5-18 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

16

21

A and B

Module 1, phase 2

0

19

24

A and B

Module 1, phase 3

0

14

18

Table 5-19 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a single-phase, high-voltage PDU.

Table 5-19 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

8

10

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

8

10

B

Module 1, phase 3

0

18

23

Table 5-20 lists the threshold values for Oracle Exadata Database Machine X3-2 Half Rack using a three-phase, high-voltage PDU.

Table 5-20 Threshold Values for Oracle Exadata Database Machine X3-2 Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

15

A and B

Module 1, phase 2

0

9

11

A and B

Module 1, phase 3

0

6

8

5.3.1.3.3 PDU Thresholds for Oracle Exadata Database Machine X3-2 Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack:

  • Table 5-21 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, Low-voltage PDU

  • Table 5-22 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, Low-voltage PDU

  • Table 5-23 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, High-voltage PDU

  • Table 5-24 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, High-voltage PDU

Table 5-21 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a single-phase, low-voltage PDU.

Table 5-21 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

5

6

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

6

B

Module 1, phase 3

0

10

13

Table 5-22 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a three-phase, low-voltage PDU.

Table 5-22 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

10

13

A and B

Module 1, phase 3

0

4

6

Table 5-23 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a single-phase, high-voltage PDU.

Table 5-23 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

11

A

Module 1, phase 2

0

4

6

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

4

6

B

Module 1, phase 3

0

9

11

Table 5-24 lists the threshold values for Oracle Exadata Database Machine X3-2 Quarter Rack using a three-phase, high-voltage PDU.

Table 5-24 Threshold Values for Oracle Exadata Database Machine X3-2 Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

2

3

A and B

Module 1, phase 3

0

3

4

5.3.1.4 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers)

This section contains the PDU thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers).

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA is available on Oracle Technology Network.

5.3.1.4.1 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack:

  • Table 5-25 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, Low-voltage PDU

  • Table 5-26 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, Low-voltage PDU

  • Table 5-27 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, High-voltage PDU

  • Table 5-28 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, High-voltage PDU

Table 5-25 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, low-voltage PDU.

Table 5-25 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

22

24

A

Module 1, phase 3

0

18

23

B

Module 1, phase 1

0

18

23

B

Module 1, phase 2

0

22

24

B

Module 1, phase 3

0

18

23

Table 5-26 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, low-voltage PDU.

Table 5-26 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

43

A and B

Module 1, phase 3

0

33

42

Table 5-27 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, high-voltage PDU.

Table 5-27 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

20

21

A

Module 1, phase 3

0

16

20

B

Module 1, phase 1

0

16

20

B

Module 1, phase 2

0

20

21

B

Module 1, phase 3

0

16

20

Table 5-28 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, high-voltage PDU.

Table 5-28 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

17

21

5.3.1.4.2 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine Half Rack:

  • Table 5-29 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, Low-voltage PDU

  • Table 5-30 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, Low-voltage PDU

  • Table 5-31 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, High-voltage PDU

  • Table 5-32 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, High-voltage PDU

Table 5-29 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, low-voltage PDU.

Table 5-29 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

18

23

Table 5-30 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, low-voltage PDU.

Table 5-30 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

23

A and B

Module 1, phase 2

0

17

22

A and B

Module 1, phase 3

0

18

23

Table 5-31 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, high-voltage PDU.

Table 5-31 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

12

15

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

12

15

B

Module 1, phase 3

0

16

20

Table 5-32 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, high-voltage PDU.

Table 5-32 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

9

12

A and B

Module 1, phase 3

0

10

13

5.3.1.4.3 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine Quarter Rack:

  • Table 5-33 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, Low-voltage PDU

  • Table 5-34 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, Low-voltage PDU

  • Table 5-35 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, High-voltage PDU

  • Table 5-36 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, High-voltage PDU

Table 5-33 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, low-voltage PDU.

Table 5-33 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

6

8

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

6

8

B

Module 1, phase 3

0

10

13

Table 5-34 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, low-voltage PDU.

Table 5-34 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

12

15

A and B

Module 1, phase 2

0

11

14

A and B

Module 1, phase 3

0

5

7

Table 5-35 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, high-voltage PDU.

Table 5-35 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

12

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

9

12

Table 5-36 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, high-voltage PDU.

Table 5-36 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 M2 and X4270 M2 servers) Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

3

4

A and B

Module 1, phase 3

0

4

5

5.3.1.5 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers)

This section contains the PDU thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers).

Note:

The values listed here are reliable for use on systems in the configuration received upon deployment and delivery from the Oracle Factory. Any changes to this configuration should be modeled in Oracle Exadata Configuration Assistant (OECA), and the PDU thresholds listed in OECA should then be used. OECA can be downloaded from Oracle Technology Network.

5.3.1.5.1 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack

The following tables list the threshold values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack:

  • Table 5-37 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, Low-voltage PDU

  • Table 5-38 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, Low-voltage PDU

  • Table 5-39 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, High-voltage PDU

  • Table 5-40 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, High-voltage PDU

Table 5-37 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, low-voltage PDU.

Table 5-37 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

22

24

A

Module 1, phase 2

0

22

24

A

Module 1, phase 3

0

21

24

B

Module 1, phase 1

0

21

24

B

Module 1, phase 2

0

22

24

B

Module 1, phase 3

0

22

24

Table 5-38 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, low-voltage PDU.

Table 5-38 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

36

45

A and B

Module 1, phase 2

0

39

44

A and B

Module 1, phase 3

0

38

45

Table 5-39 lists the threshold values for Oracle Exadata Database Machine Full Rack using a single-phase, high-voltage PDU.

Table 5-39 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

20

21

A

Module 1, phase 2

0

20

21

A

Module 1, phase 3

0

19

21

B

Module 1, phase 1

0

19

21

B

Module 1, phase 2

0

20

21

B

Module 1, phase 3

0

20

21

Table 5-40 lists the threshold values for Oracle Exadata Database Machine Full Rack using a three-phase, high-voltage PDU.

Table 5-40 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Full Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

20

21

A and B

Module 1, phase 2

0

21

21

A and B

Module 1, phase 3

0

18

21

5.3.1.5.2 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack

The following tables list the threshold values for Oracle Exadata Database Machine Half Rack:

  • Table 5-41 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, Low-voltage PDU

  • Table 5-42 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, Low-voltage PDU

  • Table 5-43 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, High-voltage PDU

  • Table 5-44 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, High-voltage PDU

Table 5-41 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, low-voltage PDU.

Table 5-41 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

21

24

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

21

24

Table 5-42 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, low-voltage PDU.

Table 5-42 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

19

24

A and B

Module 1, phase 2

0

20

25

A and B

Module 1, phase 3

0

19

24

Table 5-43 lists the threshold values for Oracle Exadata Database Machine Half Rack using a single-phase, high-voltage PDU.

Table 5-43 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

19

21

A

Module 1, phase 2

0

12

15

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

12

15

B

Module 1, phase 3

0

19

21

Table 5-44 lists the threshold values for Oracle Exadata Database Machine Half Rack using a three-phase, high-voltage PDU.

Table 5-44 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Half Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

11

14

A and B

Module 1, phase 3

0

10

13

5.3.1.5.3 PDU Thresholds for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack

The following tables list the threshold values for Oracle Exadata Database Machine Quarter Rack:

  • Table 5-45

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, Low-voltage PDU
  • Table 5-46

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, Low-voltage PDU
  • Table 5-47

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, High-voltage PDU
  • Table 5-48

    Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, High-voltage PDU

Table 5-45 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, low-voltage PDU.

Table 5-45 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

12

15

A

Module 1, phase 2

0

6

8

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

6

8

B

Module 1, phase 3

0

12

15

Table 5-46 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, low-voltage PDU.

Table 5-46 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

13

17

A and B

Module 1, phase 2

0

12

15

A and B

Module 1, phase 3

0

5

7

Table 5-47 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a single-phase, high-voltage PDU.

Table 5-47 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

11

14

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

11

14

Table 5-48 lists the threshold values for Oracle Exadata Database Machine Quarter Rack using a three-phase, high-voltage PDU.

Table 5-48 Threshold Values for Oracle Exadata Database Machine X2-2 (with X4170 and X4275 servers) Quarter Rack with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

11

14

A and B

Module 1, phase 2

0

2

3

A and B

Module 1, phase 3

0

3

4

5.3.1.6 PDU Thresholds for Oracle Exadata Database Machine X4-8 and Later

This section contains the PDU thresholds for Oracle Exadata Database Machine Eight-Socket systems for X4-8 and later.

Oracle Exadata Configuration Assistant (OECA) is the only source for PDU thresholds for Oracle Exadata Database Machine X4-8 and later systems. OECA is available on Oracle Technology Network at http://www.oracle.com/technetwork/database/exadata/oeca-download-2817713.html

If for some reason you are unable to utilize OECA, contact Oracle Support.

5.3.1.7 PDU Thresholds for Oracle Exadata Database Machine X3-8 Full Rack

This section contains the PDU thresholds for Oracle Exadata Database Machine X3-8 Full Rack.

  • Table 5-49 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

  • Table 5-50 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

  • Table 5-51 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

  • Table 5-52 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

Table 5-49 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a single-phase, low-voltage 22 kVA PDU.

Table 5-49 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

25

31

A

Module 1, phase 2

0

26

33

A

Module 1, phase 3

0

24

30

B

Module 1, phase 1

0

24

30

B

Module 1, phase 2

0

26

33

B

Module 1, phase 3

0

25

31

Table 5-50 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a three-phase, low-voltage 24 kVA PDU.

Table 5-50 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

21

27

A

Module 1, phase 2

0

23

30

A

Module 1, phase 3

0

22

28

A

Module 2, phase 1

0

20

25

A

Module 2, phase 2

0

21

27

A

Module 2, phase 3

0

23

29

B

Module 1, phase 1

0

20

25

B

Module 1, phase 2

0

21

27

B

Module 1, phase 3

0

23

29

B

Module 2, phase 1

0

21

27

B

Module 2, phase 2

0

23

30

B

Module 2, phase 3

0

22

28

Table 5-51 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a single-phase, high-voltage 22 kVA PDU.

Table 5-51 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

22

28

A and B

Module 1, phase 2

0

24

30

A and B

Module 1, phase 3

0

22

28

Table 5-52 lists the threshold values for Oracle Exadata Database Machine X3-8 Full Rack using a three-phase, high-voltage 24 kVA PDU.

Table 5-52 Threshold Values for Oracle Exadata Database Machine X3-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

12

16

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

10

13

A

Module 2, phase 1

0

10

13

A

Module 2, phase 2

0

13

17

A

Module 2, phase 3

0

11

15

B

Module 1, phase 1

0

10

13

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

11

15

B

Module 2, phase 1

0

12

16

B

Module 2, phase 2

0

13

17

B

Module 2, phase 3

0

10

13

5.3.1.8 PDU Thresholds for Oracle Exadata Database Machine X2-8 Full Rack

This section contains the PDU thresholds for Oracle Exadata Database Machine X2-8 Full Rack.

  • Table 5-53 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

  • Table 5-54 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

  • Table 5-55 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

  • Table 5-56 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

Table 5-53 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a single-phase, low-voltage 22 kVA PDU.

Table 5-53 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, Low-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

36

37

A

Module 1, phase 2

0

35

37

A

Module 1, phase 3

0

35

37

B

Module 1, phase 1

0

35

37

B

Module 1, phase 2

0

35

37

B

Module 1, phase 3

0

36

37

Table 5-54 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a three-phase, low-voltage 24 kVA PDU.

Table 5-54 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, Low-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

43

A and B

Module 1, phase 3

0

33

42

Table 5-55 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a single-phase, high-voltage 22 kVA PDU.

Table 5-55 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Single-phase, High-voltage 22 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

26

32

A and B

Module 1, phase 2

0

25

32

A and B

Module 1, phase 3

0

26

32

Table 5-56 lists the threshold values for Oracle Exadata Database Machine X2-8 Full Rack using a three-phase, high-voltage 24 kVA PDU.

Table 5-56 Threshold Values for Oracle Exadata Database Machine X2-8 Full Rack with Three-phase, High-voltage 24 kVA PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

17

21

5.3.1.9 PDU Thresholds for Oracle Exadata Storage Expansion Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

This section contains the PDU thresholds for Oracle Exadata Storage Expansion Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers.

5.3.1.9.1 PDU Thresholds for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

The following tables list the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers Full Rack:

  • Table 5-57 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

  • Table 5-58 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

  • Table 5-59 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

  • Table 5-60 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

Table 5-57 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, low-voltage PDU.

Table 5-57 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

22

24

A

Module 1, phase 3

0

18

23

B

Module 1, phase 1

0

18

23

B

Module 1, phase 2

0

22

24

B

Module 1, phase 3

0

18

23

Table 5-58 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, low-voltage PDU.

Table 5-58 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

32

40

A and B

Module 1, phase 2

0

34

43

A and B

Module 1, phase 3

0

33

42

Table 5-59 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, high-voltage PDU.

Table 5-59 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

20

21

A

Module 1, phase 3

0

16

20

B

Module 1, phase 1

0

16

20

B

Module 1, phase 2

0

20

21

B

Module 1, phase 3

0

16

20

Table 5-60 lists the threshold values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, high-voltage PDU.

Table 5-60 Threshold Values for Oracle Exadata Storage Expansion Full Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

21

A and B

Module 1, phase 2

0

18

21

A and B

Module 1, phase 3

0

17

21

5.3.1.9.2 PDU Thresholds for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

The following tables list the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers:

  • Table 5-61 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

  • Table 5-62 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

  • Table 5-63 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

  • Table 5-64 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

Table 5-61 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, low-voltage PDU.

Table 5-61 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

18

23

A

Module 1, phase 2

0

13

17

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

13

17

B

Module 1, phase 3

0

18

23

Table 5-62 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, low-voltage PDU.

Table 5-62 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

18

23

A and B

Module 1, phase 2

0

17

22

A and B

Module 1, phase 3

0

18

23

Table 5-63 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, high-voltage PDU.

Table 5-63 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

16

20

A

Module 1, phase 2

0

12

15

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

12

15

B

Module 1, phase 3

0

16

20

Table 5-64 lists the threshold values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, high-voltage PDU.

Table 5-64 Threshold Values for Oracle Exadata Storage Expansion Half Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

9

12

A and B

Module 1, phase 3

0

10

13

5.3.1.9.3 PDU Thresholds for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers

The following tables list the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers:

  • Table 5-65 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

  • Table 5-66 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

  • Table 5-67 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

  • Table 5-68 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

Table 5-65 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, low-voltage PDU.

Table 5-65 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

10

13

A

Module 1, phase 2

0

6

8

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

6

8

B

Module 1, phase 3

0

10

13

Table 5-66 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, low-voltage PDU.

Table 5-66 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, Low-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

12

15

A and B

Module 1, phase 2

0

11

14

A and B

Module 1, phase 3

0

5

7

Table 5-67 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a single-phase, high-voltage PDU.

Table 5-67 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Single-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A

Module 1, phase 1

0

9

12

A

Module 1, phase 2

0

5

7

A

Module 1, phase 3

0

0

0

B

Module 1, phase 1

0

0

0

B

Module 1, phase 2

0

5

7

B

Module 1, phase 3

0

9

12

Table 5-68 lists the threshold values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers using a three-phase, high-voltage PDU.

Table 5-68 Threshold Values for Oracle Exadata Storage Expansion Quarter Rack with Exadata Storage Server with Sun Fire X4270 M2 Servers with Three-phase, High-voltage PDU

PDU Module/Phase Info Low Threshold Pre Warning Threshold Alarm Threshold

A and B

Module 1, phase 1

0

9

12

A and B

Module 1, phase 2

0

3

4

A and B

Module 1, phase 3

0

4

5

5.4 Checking Exadata Storage Servers

The following procedure describes how to check Exadata Storage Servers:

Note:

  • When using the KVM switch, and pressing the ESC key, the BIOS may receive two ESC characters, and prompt to exit. Select CANCEL.

  • If you need to connect to the ILOM serial management port, then the baud rate setting on Exadata Storage Servers is changed from the default ILOM setting 9600 to 115200 baud, 8 bits, no parity, and 1 stop bit.

  1. Log in as the root user to the first Exadata Storage Server.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  2. Verify the SysSN label for the cell matches the product_serial_number value using the following command:

    #ipmitool sunoem cli "show /SYS product_serial_number"
    Connected.Use ^ D to exit.
    ->show /SYS product_serial_number
    /SYS
    Properties:
    product_serial_number =0937XFG03B
    ->Session closed
    Disconnected
    
  3. Check the hardware and firmware using the following command:

    # /opt/oracle.SupportTools/CheckHWnFWProfile
    

    If output does not show success, then examine the output for mismatched hardware or firmware and take steps to correct those as explained in next steps in this section.

  4. Verify the disks on the server are visible and online using the following command. The disks are numbered slot 0 to slot 11.

    #cd /opt/MegaRAID/MegaCli
    #./MegaCli64 -Pdlist -aAll |grep "Slot \|Firmware"
    Slot Number:0
    Firmware state:Online, Spun Up
    Slot Number:1
    Firmware state:Online, Spun Up
    Slot Number:2
    Firmware state:Online, Spun Up
    Slot Number:3
    Firmware state:Online, Spun Up
    Slot Number:4
    Firmware state:Online, Spun Up
    Slot Number:5
    Firmware state:Online, Spun Up
    Slot Number:6
    Firmware state:Online, Spun Up
    Slot Number:7
    Firmware state:Online, Spun Up
    Slot Number:8
    Firmware state:Online, Spun Up
    Slot Number:9
    Firmware state:Online, Spun Up
    Slot Number:10
    Firmware state:Online, Spun Up
    Slot Number:11
    Firmware state:Online, Spun Up
    [root@cell01 ~]#
    
  5. Verify the flash modules are visible using the following command. There are four cards, and four modules per card.

    # lsscsi |grep -i marvel
    # cellcli -e list lun where disktype=flashdisk
    

    The numbering shows the PCI slot number and FMOD number, such as 1_0 is PCI slot 1, FMOD 0 on the card. To confirm the flash disks are mapped that way, use the following command:

    # cellcli -e list physicaldisk attributes name, id, slotnumber where \
    disktype="flashdisk" and 'status! ="not present"'
    

    If any FMODs are missing, then power cycle the system, and wait 10 minutes after starting to check. If this does not resolve the problem, then reseat the module and flash card. If the problem continues, then replace the card with the on-site spare.

  6. Check the rear of the rack to ensure the flash card LEDs are green all the way across. These LEDs are FMOD, 0, 1, 2, 3, and SuperCAP. If any LED is amber, then check again after the next step. If the LED is still amber after restart, then reseat the card and ESM or FMOD. If the problem continues, then replace the card with the on-site spare.

    Note:

    This step can be done when all systems are on. It does not have to be done on a per-server basis.

  7. Log out of the system using the following command:

    # logout
    
  8. Return to the User Interface on the KVM switch by pressing the Ctrl key on the left to return to the KVM switch, and select Disconnect Session, if using the KVM switch to access the server.

  9. Select Target Devices from Unit View, if using the KVM switch to access the server.

  10. Proceed to the next Exadata Storage Server.

5.5 Checking Oracle Database Servers

The following procedure describes how to check the Oracle Database servers. At this point, the operating system is Linux.

  1. Log in as the root user to the first database server.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  2. Verify the SysSN label for the server matches the product_serial_number value using the following command:

    # ipmitool sunoem cli "show /SYS product_serial_number"
    Connected.Use ^ D to exit.
    ->show /SYS product_serial_number
    /SYS
    Properties:
    product_serial_number =093ABCD5001
    ->Session closed
    Disconnected
    
  3. Change to the /opt/oracle.SupportTools/firstconf directory using the following command:

    cd /opt/oracle.SupportTools/firstconf
    
  4. Create a group file with the list of server IP addresses to use in the dcli utility commands, if required, using the following command:

    # ibhosts | awk '/elastic/ {print $8}' | cut -c1-14 > group_file
    

    In the preceding command, group_file is the file with the server IP addresses.

  5. Verify the hardware profile is correct using the following command. The command checks CPU type and count, component firmware, and so on.

    # dcli -l root -g group_file "/opt/oracle.SupportTools/CheckHWnFWProfile  \
    -c  strict" > /tmp/checkhwfw.out
    
    # more /tmp/checkhwfw.out
    

    The following is an example of a check that failed:

    172.108.1.6:[WARNING ]The hardware and firmware are not supported.
    See details below
    [DiskControllerPCIeSlotWidth ]
    Requires:
    x8
    Found:
    x4
    [WARNING ]The hardware and firmware are not supported.See details
    above
    

    If a profile check fails and the firmware needs to be refreshed, then run the following command from the system with the problem.

    # /opt/oracle.SupportTools/CheckHWnFWProfile -U /opt/oracle.cellos/iso/cellbits
    Now updating the ILOM and the BIOS …
    

    The command must be entered on a single line. After running the command, power off the system, reset the ILOM, and wait 10 minutes before powering on the system.

  6. Verify the disks on the server are visible and online using the following command.

    # cd /opt/MegaRAID/MegaCli
    # ./MegaCli64 -Pdlist -aAll |grep "Slot \|Firmware"
    Slot Number: 0
    Firmware state:Online, Spun Up
    Device Firmware Level: 0868
    Slot Number:1
    Firmware state:Online, Spun Up
    Device Firmware Level: 0868
    Slot Number:2
    Firmware state:Online, Spun Up
    Device Firmware Level: A2A8
    Slot Number:3
    ...
    
  7. Examine the output of the following command:

    /opt/MegaRAID/MegaCli/MegaCli64 -LdInfo -LAll -aAll 
    
  8. (Oracle Exadata Database Machine based on Sun Fire X4170 Oracle Database Servers only) Check that the ILOM is configured for the proper fan cooling on each database server using the following command:

    # ipmitool sunoem cli "show /SP/policy"
    
    • If the policy is present and enabled, then no additional steps are needed.

    • If the policy is present and disabled, then enable the policy using the following command:

      # ipmitool sunoem cli "set /SP/policy FLASH_ACCELERATOR_CARD_ \
      INSTALLED=enabled"
      

      The command must be entered as a single line.

  9. Log out of the system using the following command:

    # logout
    
  10. Return to the User Interface on the KVM switch by pressing the Ctrl key on the left to return to the KVM switch, and select Disconnect Session, if using the KVM switch to access the server.

  11. Select Target Devices from Unit View, and proceed to the next database server, if using the KVM switch to access the server.

  12. Verify that NTP servers are configured the same for all servers using the following command. The command is run on each database server individually.

    ntpq -p
    

5.6 Performing Additional Checks and Configuration

The following procedure describes additional checks and configuration steps for Oracle Exadata Rack:

  1. (Oracle Exadata Database Machine only) Log in to the first database server as the root user.

  2. Change to the firstconf directory using the following command:

    # cd /opt/oracle.SupportTools/firstconf
    
  3. Create a group file with the list of IP addresses to use in the dcli utility commands, if needed

  4. Verify the rack master serial number is set correctly using the following command:

    # dcli -l root -g group_file "ipmitool sunoem cli 'show         \
    /SP system_identifier'" > /tmp/show-rack-csn.out
    
    # more /tmp/show-rack-csn.out
    

    If none of the serial numbers are set, then contact Oracle Support Services. If one serial number is not set correctly, then run the following command to correct it:

    ipmitool sunoem cli 'set /SP system_identifier= \
    "Exadata Database Machine rack_type xxxxAKyyyy"'
    

    In the preceding command, rack_type is X2-2 or X2-8, depending on the rack.

  5. Ensure disk cache policy is set to Disabled using the following command as the root user on the first database server in the cluster:

    dcli -g /opt/oracle.SupportTools/onecommand/all_group -l root \
    /opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL | grep -i 'Disk Cache Policy'
    

    The following is an example of the output from the command:

    dm01db01: Disk Cache Policy   : Disabled
    dm01db02: Disk Cache Policy   : Disabled
    dm01db03: Disk Cache Policy   : Disabled
    ...
    dm01cel01: Disk Cache Policy   : Disabled
    dm01cel02: Disk Cache Policy   : Disabled
    ...
    

    If any server shows different output, then identify the LUN and use the following command to reset the policy to Disabled:

    MegaCli64 -LDSetProp -DisDskCache -LUNn -a0
    

    In the preceding command, LUNn is the LUN number, such as L2.

5.7 Verifying the InfiniBand Network

The following procedure describes how to verify the InfiniBand network:

  1. Visually check all the InfiniBand cable connections within the rack. The port lights should be on, and the LEDs should be on. Do not press each connector to verify connectivity.

  2. Log in as the root user on any component in the rack.

  3. Verify the InfiniBand topology using the following commands:

    # cd /opt/oracle.SupportTools/ibdiagtools
    # ./verify-topology [-t rack_size]
    

    The following example shows the output when the network components are correct.

    [DB Machine Infiniband Cabling Topology Verification Tool ]
    Is every external switch connected to every internal switch......[SUCCESS ]
    Are any external switches connected to each other................[SUCCESS ]
    Are any hosts connected to spine switch..........................[SUCCESS ]
    Check if all hosts have 2 CAs to different switches..............[SUCCESS ]
    Leaf switch check:cardinality and even distribution..............[SUCCESS ]
    Check if each rack has an valid internal ring....................[SUCCESS ]
    
    

    In the preceding command, rack_size is the size of the rack. The -t rack_size option is needed if the rack is Oracle Exadata Database Machine Half Rack or Oracle Exadata Database Machine Quarter Rack. Use halfrack for Oracle Exadata Database Machine Half Rack, and quarterrack for Oracle Exadata Database Machine Quarter Rack.

    The following example shows the output when there is a bad InfiniBand switch to InfiniBand cable connection:

    #./verify-topology
    [DB Machine Infiniband Cabling Topology Verification Tool ]
    Is every external switch connected to every internal switch......[SUCCESS ]
    Are any external switches connected to each other................[SUCCESS ]
    Are any hosts connected to spine switch..........................[SUCCESS ]
    Check if all hosts have 2 CAs to different switches..............[SUCCESS ]
    Leaf switch check:cardinality and even distribution..............[SUCCESS ]
    Check if each rack has an valid internal ring....................[ERROR ]
    
    Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them.
    They should have at least 7 links between them
    

    The following example shows the output when there is a bad InfiniBand cable on a database server:

    #./verify-topology
    [DB Machine Infiniband Cabling Topology Verification Tool ]
    Is every external switch connected to every internal switch......[SUCCESS ]
    Are any external switches connected to each other................[SUCCESS ]
    Are any hosts connected to spine switch..........................[SUCCESS ]
    Check if all hosts have 2 CAs to different switches..............[ERROR ]
    Node db01 has 1 endpoints.(Should be 2)
    Port 2 of this node is not connected to any switch
    --------fattree End Point Cabling verification failed-----
    Leaf switch check:cardinality and even distribution..............[ERROR ]
    Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes
    It has only 3 links belonging to compute nodes                  [SUCCESS ]
    Check if each rack has an valid internal ring...................[SUCCESS ]
    

    The following example shows the output when there is a bad connection on the switch and the system:

    #./verify-topology
    [DB Machine Infiniband Cabling Topology Verification Tool ]
    Is every external switch connected to every internal switch......[SUCCESS ]
    Are any external switches connected to each other................[SUCCESS ]
    Are any hosts connected to spine switch..........................[SUCCESS ]
    Check if all hosts have 2 CAs to different switches..............[ERROR ]
    
    Node burxdb01 has 1 endpoints.(Should be 2) 
    Port 2 of this node is not connected to any switch
    --------fattree End Point Cabling verifation failed-----
    Leaf switch check:cardinality and even distribution..............[ERROR ]
    Internal QDR Switch 0x21283a87b8a0a0 has fewer than 4 compute nodes 
    It has only 3 links belonging to compute nodes...................[SUCCESS ]
    Check if each rack has an valid internal ring....................[ERROR ]
    
    Switches 0x21283a87cba0a0 0x21283a87b8a0a0 have 6 connections between them
    They should have at least 7 links between them
    

5.8 Imaging a New System

You have different options for preparing an Oracle Exadata system for deployment.

Using a preconf.csv file with the PXE, USB and ISO options enables an unattended first boot configuration and greatly simplifies the installation but is not mandatory. It is possible to boot from the installation media and follow the prompts to configure the system.

In release 12.1.2.2.0 and above, the installation media has been simplified, and it is no longer necessary to build files using the ImageMaker process.

Refer to My Oracle Support note 2038073.1 for additional details on valid options for imaging 12.1.2.1.0 and above.

5.8.1 Using ISO Images to Image a New System

In versions prior to 12.1.2.2.0, ISO images are available as patches that you can download for bare metal imaging an Exadata system. This is now the default method in 12.1.2.2.0 and above. You are no longer required to build the ISO files using imagemaker. Patches are available for both database nodes and cells. You can use the ISO images to boot the server using a virtual media device on the ILOM (from the Devices menu of the ILOM console). For example, the two ISO images available for 12.1.2.1.1 are listed in the following table.

Table 5-69 ISO Images for 12.1.2.1.1

Version Type Edelivery Part / Notes ARU Bug #/ULN Channel Description
12.1.2.1.1

Cell ISO image and md5sum: 90305b9e2c7e050d7588b8a7d5c731e7 cell_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.iso

Refer to 12.1.2.1.1 Readme (Doc ID 1959143.1) as well as the readme provided with the download.

20757971

Oracle Database Machine Exadata Storage Cell (X5-2L, X4-2L, X4270M3, X4270M2, X4275) Image 12c Release 1 (12.1.2.1.1) for Linux x86_64

12.1.2.1.1

DB ISO image and md5sum: 39f12722cb338b2d4de5acba90adf8fe compute_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.iso

Refer to 12.1.2.1.1 Readme (Doc ID 1959143.1) as well as the readme provided with the download.

20757974

Oracle Database Machine Database Host (X4-8, X4800M2, X4800, X5-2, X4-2, X4170M3, X4170M2, X4170) Image 12c Release 1 (12.1.2.1.1) for Linux x86_64

To prepare a new Exadata system for deployment using ISO images, perform the following steps:

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files. You will need the configuration files when you run applyElasticConfig.sh later.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. After booting from the ISO, follow the prompts and imaging will start automatically. You will be prompted to enter IP addresses.

    You can use a preconf.csv by following the steps in Using ISO to Image a New System to mount the image on an NFS repository. You need to place the preconf.csv file in the same NFS directory.

  5. If you are using Oracle VM, run switch_to_ovm.sh on each database server. The database servers will reboot.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
    2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
    2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3 in /boot/I_am_hd_boot
    2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 system partition
    
  6. Run reclaimdisks.sh on each database server.

    The /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim command reclaims disk space reserved for the deployment type not selected. The command takes less than 5 minutes, approximately. Systems are imaged with disks configured with RAID5; a RAID rebuild is no longer part of the reclaimdisks.sh process.

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh.

    For example:

    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
    Model is ORACLE SERVER X5-2
    Number of LSI controllers: 1
    Physical disks found: 4 (252:0 252:1 252:2 252:3)
    Logical drives found: 1
    Linux logical drive: 0
    RAID Level for the Linux logical drive: 5
    Physical disks in the Linux logical drive: 4 (252:0 252:1 252:2 252:3)
    Dedicated Hot Spares for the Linux logical drive: 0
    Global Hot Spares: 0
    [INFO     ] Check for DOM0 system disk
    [INFO     ] Check for DOM0 with inactive Linux system disk
    [INFO     ] Valid DOM0 with inactive Linux system disk is detected
    [INFO     ] Number of partitions on the system device /dev/sda: 4
    [INFO     ] Higher partition number on the system device /dev/sda: 4
    [INFO     ] Last sector on the system device /dev/sda: 3509759999
    [INFO     ] End sector of the last partition on the system device /dev/sda: 3509759000
    [INFO     ] Unmount /EXAVMIMAGES from ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Remove inactive system logical volume /dev/VGExaDb/LVDbSys1
    [INFO     ] Remove logical volume /dev/VGExaDbOra/LVDbOra1
    [INFO     ] Remove volume group VGExaDbOra
    [INFO     ] Remove physical volume /dev/sda4
    [INFO     ] Remove partition /dev/sda4
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/sda4 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 3300035608
    [INFO     ] Check for existing first boot system image /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Saving /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img in /var/log/exadatatmp ...
    [INFO     ] First boot system image saved in /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Unmount /EXAVMIMAGES from /dev/sda3
    [INFO     ] Remove partition /dev/sda3
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/sda3 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 240132159
    [INFO     ] Create primary ocfs2 partition 3 using 240132160 3509758999
    [INFO     ] Create ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition on /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Restoring /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.img into /EXAVMIMAGES ...
    [INFO     ] Logical volume LVDbSys2 exists in volume group VGExaDb
    [INFO     ] Grub version in /boot/grub/grub.stage.version: 0.97-81.0.1.el6
    [INFO     ] Grub rpm version: 0.97-13.10.0.1.el5
    [INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
    [INFO     ] Create filesystem on device /dev/sda1
    [INFO     ] Tune filesystem on device /dev/sda1
    
    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
    
    [ Minimal BASH-like line editing is supported.  For the first word, TAB
    lists possible command completions.  Anywhere else TAB lists the possible
    completions of a device/filename.]
    grub> root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83
    grub> setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
     Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
    Done.
    grub> quit
    
  7. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with elasticNode in the description.

  8. Run the OEDA tool to deploy.

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
    1. Validate Configuration File
    2. Setup Capacity-on-Demand
    3. Create Virtual Machine  
    4. Create Users  
    5. Setup Cell Connectivity  
    6. Create Cell Disks  
    7. Create Grid Disks  
    8. Configure Alerting  
    9. Install Cluster Software  
    10. Initialize Cluster Software  
    11. Install Database Software  
    12. Relink Database with RDS  
    13. Create ASM Diskgroups  
    14. Create Databases  
    15. Apply Security Fixes  
    16. Install Exachk  
    17. Setup ASR Alerting  
    18. Create Installation Summary  
    19. Resecure Machine
    

5.8.2 Using Elastic Configuration to Image a New System

Elastic configuration is now the standard methodology for all new deployments, and applies to Oracle Exadata X5, X5-2L, and X4-8b (with X5 storage cells) servers. You can also use the same process to add additional database servers or cells to an existing configuration. The process allows initial IP addresses to be assigned to database servers and cells, regardless of the exact customer configuration ordered.

Elastic configuration is only applicable to machines shipped from the factory (or to machines imaged exactly to mimic that) and only for X5 and X4-8b systems or later generations. You can run Elastic configuration at the first time of deployment, and when adding new X5 factory-built components to a rack. You cannot use Elastic configuration to configure or reconfigure systems. It is recommended that you use Elastic configuration only with machines shipped from the factory for first-time deployment.

When a server is shipped from the factory, the elastic configuration code is executed when the server is first powered on. At that time, the machine will be assigned an IP address in the 172 range.

To prepare a new Exadata system for deployment using the elastic configuration method, perform the following steps:

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format “nodeN”, where N is a number.

  4. If you are using Oracle VM, run switch_to_ovm.sh on each database server. The database servers will reboot. If you are using physical deployment and not virtualized deployment, then you can skip this step.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
    2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
    2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3 in /boot/I_am_hd_boot
    2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 system partition
    
  5. Run reclaimdisks.sh on each database server.

    The /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim command reclaims disk space reserved for the deployment type not selected. The command takes less than 5 minutes, approximately. Systems are imaged with disks configured with RAID5; a RAID rebuild is no longer part of the reclaimdisks.sh process.

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh.

    For example:

    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
    Model is ORACLE SERVER X5-2
    Number of LSI controllers: 1
    Physical disks found: 4 (252:0 252:1 252:2 252:3)
    Logical drives found: 1
    Linux logical drive: 0
    RAID Level for the Linux logical drive: 5
    Physical disks in the Linux logical drive: 4 (252:0 252:1 252:2 252:3)
    Dedicated Hot Spares for the Linux logical drive: 0
    Global Hot Spares: 0
    [INFO     ] Check for DOM0 system disk
    [INFO     ] Check for DOM0 with inactive Linux system disk
    [INFO     ] Valid DOM0 with inactive Linux system disk is detected
    [INFO     ] Number of partitions on the system device /dev/sda: 4
    [INFO     ] Higher partition number on the system device /dev/sda: 4
    [INFO     ] Last sector on the system device /dev/sda: 3509759999
    [INFO     ] End sector of the last partition on the system device /dev/sda: 3509759000
    [INFO     ] Unmount /EXAVMIMAGES from ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Remove inactive system logical volume /dev/VGExaDb/LVDbSys1
    [INFO     ] Remove logical volume /dev/VGExaDbOra/LVDbOra1
    [INFO     ] Remove volume group VGExaDbOra
    [INFO     ] Remove physical volume /dev/sda4
    [INFO     ] Remove partition /dev/sda4
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/sda4 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 3300035608
    [INFO     ] Check for existing first boot system image /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Saving /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img in /var/log/exadatatmp ...
    [INFO     ] First boot system image saved in /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Unmount /EXAVMIMAGES from /dev/sda3
    [INFO     ] Remove partition /dev/sda3
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/sda3 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 240132159
    [INFO     ] Create primary ocfs2 partition 3 using 240132160 3509758999
    [INFO     ] Create ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition on /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Restoring /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.img into /EXAVMIMAGES ...
    [INFO     ] Logical volume LVDbSys2 exists in volume group VGExaDb
    [INFO     ] Grub version in /boot/grub/grub.stage.version: 0.97-81.0.1.el6
    [INFO     ] Grub rpm version: 0.97-13.10.0.1.el5
    [INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
    [INFO     ] Create filesystem on device /dev/sda1
    [INFO     ] Tune filesystem on device /dev/sda1
    
    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
    
    [ Minimal BASH-like line editing is supported.  For the first word, TAB
    lists possible command completions.  Anywhere else TAB lists the possible
    completions of a device/filename.]
    grub> root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83
    grub> setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
     Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
    Done.
    grub> quit
    
  6. Apply your specific configuration to the nodes by running the applyElasticConfig.sh script that is supplied with OEDA.

    See My Oracle Support note 1953915.1 for further details on how the elastic configuration process works, and specific examples of how to apply customer IP addresses to the rack.

  7. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with "elasticNode" in the description.

  8. Run the OEDA tool to deploy.

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
    1. Validate Configuration File
    2. Setup Capacity-on-Demand
    3. Create Virtual Machine  
    4. Create Users  
    5. Setup Cell Connectivity  
    6. Create Cell Disks  
    7. Create Grid Disks  
    8. Configure Alerting  
    9. Install Cluster Software  
    10. Initialize Cluster Software  
    11. Install Database Software  
    12. Relink Database with RDS  
    13. Create ASM Diskgroups  
    14. Create Databases  
    15. Apply Security Fixes  
    16. Install Exachk  
    17. Setup ASR Alerting  
    18. Create Installation Summary  
    19. Resecure Machine
    

5.8.3 Using ISO to Image a New System

To prepare a new Exadata system for deployment using ISO, perform the following steps:

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. Run OEDA to generate the preconf.csv file that can be used to image the system.

  5. Connect to the first database node. You can do this by connecting a laptop to the ILOM serial management port (9600, 8, N, 1), and then connecting to the console and log in to the database node.

  6. Obtain the eth0 MAC address for each database node. This is the fru_macaddress field from /SYS/MB/NET0 in the ILOM (or through ip addr if logged into the console).

    1. Connect to the other database nodes using ssh from the console.

    2. Run ibhosts to get a list of node names and IP addresses.

  7. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node. This field should be empty, that is, you should see two consecutive comma characters (,,). For example:

    orhb42-b08-01,us.oracle.com,db,eth0,eth0,Management,00:10:E0:69:34:52,
    10.196.3.106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:ac
    tive-bond-ib,192.168.10.15,255.255.255.0,,,America/Los_Angeles
    
  8. Run ipconf to verify the generated preconf.csv file. This can be done on any Exadata server, and although not required, it is useful to ensure the preconf.csv is valid before proceeding.

    # ipconf -verify -preconf /tmp/preconf.csv
    [Info]: ipconf command line: /opt/oracle.cellos/ipconf.pl -verify -preconf /tmp
    /preconf -nocodes
    [Done]: Pre config verification OK 
    
  9. If you are using versions prior to 12.1.2.2.0, proceed to step 11 to use ImageMaker to create the image files.

    If you are using version 12.1.2.2.0 or later, it is no longer necessary to use imagemaker to create the image files. The relevant PXE, ISO, and USB image files have already been created in the patches themselves. Simply download the patch containing the prebuilt image file from Oracle Software Delivery Cloud and place the downloaded ISO .iso image file in the appropriate location. For 12.1.2.2.0 the patch numbers can be obtained from My Oracle Support note 2038073.1. There are separate ImageMaker files for database nodes and cells.

    Note : For image version 12.1.2.2.0 the format of the downloadable image files has changed.

    There will be an image file in the downloaded zip file. For example, the database node zip file will contain:

    compute_12.1.2.2.0_LINUX.X64_150917-1.x86_64.iso
    

    For cell image, the file will be similar to:

    cell_12.1.2.2.0_LINUX.X64_150917-1.x86_64.iso 
    
  10. (For 12.1.2.2.0 and above) Follow the steps in the patch README.txt to copy the ISO file and optional preconf.csv file into place.

    1. After download, unzip the file as root user for the following ISO image.

      b2d6d9ac61e2e673d05fba11ec65107d compute_12.1.2.2.0_LINUX.X64_150917-1.x
      86_64.iso
      

      Note:

      • ISO + preconf.csv is supported only if the ISO image is mounted on an NFS repository in ILOM and the preconf.csv file is located in the same NFS directory. All other methods of mounting the ISO image in ILOM (SAMBA URI and Java Remote Console) and burning ISO image to DVD will not work with the preconf.csv file. For instructions on how to mount the ISO image from a NFS central repository, see "Configure a Host Storage Device Redirection Session" in the Oracle X5 Series Servers Administration Guide

      • Because of possible late echos of text to the console, be aware that the boot process will be waiting for input to complete the interactive ipconf by prompting for Nameserver:

      • Due to a bug, ipconf will prompt you twice to enter the network settings.

    2. Proceed to step 15, and boot the system with the ISO image and follow installation prompts. The ISO can be made available with a preconf.csv following the steps in the Note box above, or as virtual media on the ILOM of the node to be reimaged when a preconf.csv is not used.

      Steps 11 to 14, which create the image files, can be skipped.

  11. Download the imagemaker kit to prepare and create the files for the USB drive.

    This kit is available publicly on the Oracle Software Delivery Cloud (edelivery.oracle.com) site under “Oracle Database Products -> Linux x86-64”. There are separate ImageMaker files for database nodes and cells.

  12. After download, unzip the first file, called something similar to V75080-01.zip, and then unzip the subsequent file as well. Perform the unzips as the root user. Then untar the file.

    # unzip V75080-01.zip
    
    # unzip cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.zip
    
    # tar -pxvf cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.tar
    

    View the README_FOR_FACTORY.txt file in the resulting directory for additional details.

  13. Run makeImageMedia.sh to build the ISO file for imaging.

    For building images for 12.1.2.1.0 and above, the recommended makeImageMedia.sh options for building ISO images are:

    1. (12.1.2.1.0 up to 12.2.1.1.0) Change directory to the dl180 or dl360 directory.

    2. (12.2.1.1.0 and above) Change directory to the cell or compute directory.

    3. Make the ISO file.

      ./makeImageMedia.sh -factory -stit -reboot-on-success -nodisktests [-preconf
       preconf_file] filename.iso
      

    For ISO imaging, note that you must run makeImageMedia.sh from an Oracle Exadata database server if you are using the -preconf option.

    For example:

    # ./makeImageMedia.sh -factory -stit -reboot-on-success -nodisktests -preconf
     preconf.csv computenode.iso
    [Info]: ipconf command line: /root/imagemaker/compute/initrd/opt/oracle.cellos
    /ipconf.pl -preconf preconf.csv -verify -nocodes
    [Done]: Pre config verification OK
    Please wait. Calculating md5 checksums for cellbits ...
    Calculating md5 checksum for exaos.tbz ...
    Calculating md5 checksum for dbboot.tbz ...
    Calculating md5 checksum for dbfw.tbz ...
    Calculating md5 checksum for kernel.tbz ...
    Calculating md5 checksum for ofed.tbz ...
    Calculating md5 checksum for sunutils.tbz ...
    Calculating md5 checksum for commonos.tbz ...
    Calculating md5 checksum for debugos.tbz ...
    Calculating md5 checksum for dbrpms.tbz ...
    Calculating md5 checksum for exaosovs.tbz ...
    Calculating md5 checksum for dbovsboot.tbz ...
    Calculating md5 checksum for kernelovs.tbz ...
    Calculating md5 checksum for ofedovs.tbz ...
    Calculating md5 checksum for commonovsos.tbz ...
    Calculating md5 checksum for debugosovs.tbz ...
    Calculating md5 checksum for dbovsrpms.tbz ...
    Calculating md5 checksum for sunovsutils.tbz ...
    Calculating md5 checksum for dbvmboot.tbz ...
    Calculating md5 checksum for ofedvm.tbz ...
    Calculating md5 checksum for sunvmutils.tbz ...
    Please wait. Making initrd ...
    395864 blocks
    Please wait. Calculating md5 checksums for boot ...
    Size of boot image is 4 sectors -> No emulation
    0.30% done, estimate finish Wed Nov 29 10:47:27 2016
    0.60% done, estimate finish Wed Nov 29 10:47:27 2016
    0.90% done, estimate finish Wed Nov 29 10:47:27 2016
    1.21% done, estimate finish Wed Nov 29 10:47:27 2016
    1.51% done, estimate finish Wed Nov 29 10:47:27 2016
    1.81% done, estimate finish Wed Nov 29 10:47:27 2016
    2.11% done, estimate finish Wed Nov 29 10:47:27 2016
    2.41% done, estimate finish Wed Nov 29 10:47:27 2016
    2.71% done, estimate finish Wed Nov 29 10:47:27 2016
    3.02% done, estimate finish Wed Nov 29 10:47:27 2016
    3.32% done, estimate finish Wed Nov 29 10:47:27 2016
    3.62% done, estimate finish Wed Nov 29 10:47:27 2016
    3.92% done, estimate finish Wed Nov 29 10:47:27 2016
    4.22% done, estimate finish Wed Nov 29 10:47:27 2016
    4.52% done, estimate finish Wed Nov 29 10:47:27 2016
    4.83% done, estimate finish Wed Nov 29 10:47:27 2016
    5.13% done, estimate finish Wed Nov 29 10:47:27 2016
    5.43% done, estimate finish Wed Nov 29 10:47:27 2016
    5.73% done, estimate finish Wed Nov 29 10:47:27 2016
    <output truncated>
    97.72% done, estimate finish Wed Nov 29 10:47:30 2016
    98.02% done, estimate finish Wed Nov 29 10:47:30 2016
    98.32% done, estimate finish Wed Nov 29 10:47:30 2016
    98.62% done, estimate finish Wed Nov 29 10:47:30 2016
    98.92% done, estimate finish Wed Nov 29 10:47:30 2016
    99.22% done, estimate finish Wed Nov 29 10:47:30 2016
    99.52% done, estimate finish Wed Nov 29 10:47:30 2016
    99.83% done, estimate finish Wed Nov 29 10:47:30 2016
    Total translation table size: 2048
    Total rockridge attributes bytes: 0
    Total directory bytes: 2048
    Path table size(bytes): 26
    Max brk space used 0
    1657890 extents written (3238 MB)
    Done creation of installation ISO for compute node
    
  14. Attach the ISO as virtual media to the ILOM of the database server to be reimaged, and select CDROM as the next boot device.

  15. Reboot the database node or cell to start the imaging process from the ISO.

  16. If you are using Oracle VM, run switch_to_ovm.sh on each database server. The database servers will reboot.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
    2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDb
    Sys3 (/dev/mapper/VGExaDb-LVDbSys3)
    2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys
    1
    2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VG
    ExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3
     in /boot/I_am_hd_boot
    2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 s
    ystem partition
    
  17. Run reclaimdisks.sh on each database server.

    The /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim command reclaims disk space reserved for the deployment type not selected. The command takes less than 5 minutes, approximately. Systems are imaged with disks configured with RAID5; a RAID rebuild is no longer part of the reclaimdisks.sh process.

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh.

    For example:

    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
    Model is ORACLE SERVER X5-2
    Number of LSI controllers: 1
    Physical disks found: 4 (252:0 252:1 252:2 252:3)
    Logical drives found: 1
    Linux logical drive: 0
    RAID Level for the Linux logical drive: 5
    Physical disks in the Linux logical drive: 4 (252:0 252:1 252:2 252:3)
    Dedicated Hot Spares for the Linux logical drive: 0
    Global Hot Spares: 0
    [INFO     ] Check for DOM0 system disk
    [INFO     ] Check for DOM0 with inactive Linux system disk
    [INFO     ] Valid DOM0 with inactive Linux system disk is detected
    [INFO     ] Number of partitions on the system device /dev/sda: 4
    [INFO     ] Higher partition number on the system device /dev/sda: 4
    [INFO     ] Last sector on the system device /dev/sda: 3509759999
    [INFO     ] End sector of the last partition on the system device /dev/sda: 350975
    9000
    [INFO     ] Unmount /EXAVMIMAGES from ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Remove inactive system logical volume /dev/VGExaDb/LVDbSys1
    [INFO     ] Remove logical volume /dev/VGExaDbOra/LVDbOra1
    [INFO     ] Remove volume group VGExaDbOra
    [INFO     ] Remove physical volume /dev/sda4
    [INFO     ] Remove partition /dev/sda4
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/s
    da4 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 330003
    5608
    [INFO     ] Check for existing first boot system image /EXAVMIMAGES/System.first.b
    oot.12.1.2.1.0.141205.2.img
    [INFO     ] Saving /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img in /var/
    log/exadatatmp ...
    [INFO     ] First boot system image saved in /var/log/exadatatmp/System.first.boot
    .12.1.2.1.0.141205.2.img
    [INFO     ] Unmount /EXAVMIMAGES from /dev/sda3
    [INFO     ] Remove partition /dev/sda3
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/s
    da3 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 240132
    159
    [INFO     ] Create primary ocfs2 partition 3 using 240132160 3509758999
    [INFO     ] Create ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition on /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Restoring /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.im
    g into /EXAVMIMAGES ...
    [INFO     ] Logical volume LVDbSys2 exists in volume group VGExaDb
    [INFO     ] Grub version in /boot/grub/grub.stage.version: 0.97-81.0.1.el6
    [INFO     ] Grub rpm version: 0.97-13.10.0.1.el5
    [INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
    [INFO     ] Create filesystem on device /dev/sda1
    [INFO     ] Tune filesystem on device /dev/sda1
    
    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
    
    [ Minimal BASH-like line editing is supported.  For the first word, TAB
    lists possible command completions.  Anywhere else TAB lists the possible
    completions of a device/filename.]
    grub> root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83
    grub> setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
     Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
    Done.
    grub> quit
    
  18. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with elasticNode in the description.

  19. Run the OEDA tool to deploy.

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
    1. Validate Configuration File
    2. Setup Capacity-on-Demand
    3. Create Virtual Machine  
    4. Create Users  
    5. Setup Cell Connectivity  
    6. Create Cell Disks  
    7. Create Grid Disks  
    8. Configure Alerting  
    9. Install Cluster Software  
    10. Initialize Cluster Software  
    11. Install Database Software  
    12. Relink Database with RDS  
    13. Create ASM Diskgroups  
    14. Create Databases  
    15. Apply Security Fixes  
    16. Install Exachk  
    17. Setup ASR Alerting  
    18. Create Installation Summary  
    19. Resecure Machine
    

5.8.4 Using PXE to Image a New System

To prepare a new Exadata system for deployment using PXE, perform the following steps:

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run the OEDA configuration tool to generate the configuration files.

  3. Apply power to all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. Run OEDA to generate the preconf.csv file that you will use to image the system.

  5. Connect to the first database node. You can do this by connecting a laptop to the ILOM serial management port (9600, 8, N, 1), and then connecting to the console and log in to the database node.

  6. Obtain the eth0 MAC address for each database node. This is the fru_macaddress field from /SYS/MB/NET0 in the ILOM (or from ip addr if logged into the console).

    1. Connect to the other database nodes through SSH from the console.

    2. Run ibhosts to get a list of node names and IP addresses.

  7. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node. This field should be empty, that is, you should see two consecutive comma characters (,,). For example:

    orhb42-b08-01,us.oracle.com,db,eth0,eth0,Management,00:10:E0:69:34:52,10.196.3.
    106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:active-bond-ib,192.1
    68.10.15,255.255.255.0,,,America/Los_Angeles
    
  8. Create a PXE configuration file for each node.

    Below are the recommended options for PXE imaging of versions 12.1.2.1.0 and above

    append initrd=img_file pxe factory stit reboot-on-success notests=diskgroup dhcp 
    preconf=n.n.n.n:/directory/preconf.csv sk=n.n.n.n:/directory console=ttyS0,
    115200n8
    

    The command above should be entered as a single line with no line breaks.

  9. For versions prior to 12.1.2.2.0 proceed to step 10 to use ImageMaker to create the image files.

    .

    If you are using version 12.1.2.2.0 or later, it is no longer necessary to use ImageMaker to create the image files. The relevant PXE, ISO, and USB image files have already been created in the patches themselves. Simply download the patch containing the prebuilt PXE image files from Oracle Software Delivery Cloud and place them into the appropriate directories on the PXE server. For 12.1.2.2.0 the patch numbers can be obtained from My Oracle Support note 2038073.1 . There are separate ImageMaker files for database nodes and cells.

    Note : For image version 12.1.2.2.0 the format of the downloadable image files has changed.

    The downloaded zip file contains four image files:

    • For 12.1.2.2.0 up to 12.2.1.1.0:

      kernel: vmlinux-version-ImageName-{DL180|DL360}
      initrd: initrd-version-ImageName-{DL180|DL360}.img
      image: nfsimg-version-ImageName-{DL180|DL360}.tar
      image: nfsimg-version-ImageName-{DL180|DL360}.tar.md5
      
    • For 12.2.1.1.0 and above:

      kernel: vmlinux-version-ImageName-{cell|compute}
      initrd: initrd-version-ImageName-{cell|compute}.img
      image: nfsimg-version-ImageName-{cell|compute}.tar
      image: nfsimg-version-ImageName-{cell|compute}.tar.md5
      

    If you are imaging version 12.1.2.2.0 and later, proceed to step 13. For versions earlier than 12.1.2.2.0, go to step 10 to create the image files using ImageMaker.

  10. Download the ImageMaker kit to prepare and create the files for the PXE server. The kit is available publicly on the Oracle Software Delivery Cloud (edelivery) site under “Oracle Database Products -> Linux x86-64”. There are separate ImageMaker files for database nodes and cells.

  11. After downloading the kit, unzip the first file, called something similar to V75080-01.zip, and then unzip the subsequent file as well. Perform the unzip as the root user. Then untar the file.

    # unzip V75080-01.zip
    
    # unzip cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.zip
    
    # tar -pxvf cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.tar
    

    View the README_FOR_FACTORY.txt file in the resulting directory for additional details.

  12. Run makeImageMedia.sh to build the kernel, initrd, and image files if necessary. Place these in the relevant directories on the PXE server. These files will be sent at boot time to the node being imaged.

    1. (For 12.1.2.2.0 up to 12.2.1.1.0) Change directory to the dl180 or dl360 directory.

    2. (For 12.2.1.1.0 and above) Change directory to the cell or compute directory.

    3. Make PXE build using:

      ./makeImageMedia.sh -pxe -pxeout ImageName
      

      This creates four files in the ./PXE subdirectory:

      • For 12.1.2.2.0 up to 12.2.1.1.0:

        kernel: vmlinux-version-ImageName-{DL180|DL360}
        initrd: initrd-version-ImageName-{DL180|DL360}.img
        image: nfsimg-version-ImageName-{DL180|DL360}.tar
        image: nfsimg-version-ImageName-{DL180|DL360}.tar.md5
        
      • For 12.2.1.1.0 and above:

        kernel: vmlinux-version-ImageName-{cell|compute}
        initrd: initrd-version-ImageName-{cell|compute}.img
        image: nfsimg-version-ImageName-{cell|compute}.tar
        image: nfsimg-version-ImageName-{cell|compute}.tar.md5
        

      For example:

      # ./makeImageMedia.sh -pxe -pxeout test
      
      Please wait. Calculating md5 checksums for cellbits ...
      Calculating md5 checksum for exaos.tbz ...
      Calculating md5 checksum for dbboot.tbz ...
      Calculating md5 checksum for dbfw.tbz ...
      Calculating md5 checksum for kernel.tbz ...
      Calculating md5 checksum for ofed.tbz ...
      Calculating md5 checksum for sunutils.tbz ...
      Calculating md5 checksum for commonos.tbz ...
      Calculating md5 checksum for debugos.tbz ...
      Calculating md5 checksum for dbrpms.tbz ...
      Calculating md5 checksum for exaosovs.tbz ...
      Calculating md5 checksum for dbovsboot.tbz ...
      Calculating md5 checksum for kernelovs.tbz ...
      Calculating md5 checksum for ofedovs.tbz ...
      Calculating md5 checksum for commonovsos.tbz ...
      Calculating md5 checksum for debugosovs.tbz ...
      Calculating md5 checksum for dbovsrpms.tbz ...
      Calculating md5 checksum for sunovsutils.tbz ...
      Calculating md5 checksum for dbvmboot.tbz ...
      Calculating md5 checksum for ofedvm.tbz ...
      Calculating md5 checksum for sunvmutils.tbz ...
      Store filename of nfsimg tarball nfsimg-12.1.2.1.1-test-DL360.tar inside in
      itrd
      Please wait. Making initrd ...
      395853 blocks
      Please wait. Calculating md5 checksums for boot ...
      PXE NFS image:   /root/imagemaker/dl360/./PXE/nfsimg-12.1.2.1.1-test-DL360.
      tar
      PXE NFS md5 sum: /root/imagemaker/dl360/./PXE/nfsimg-12.1.2.1.1-test-DL360.
      tar.md5
      PXE initrd:      /root/imagemaker/dl360/./PXE/initrd-12.1.2.1.1-test-DL360.
      img
      PXE kernel:      /root/imagemaker/dl360/./PXE/vmlinux-12.1.2.1.1-test-DL360
      
  13. Change the boot order using the ILOM to boot from PXE and then reboot to start the imaging process.

    For example, from the Unix prompt on the host:

    # ipmitool chassis bootdev pxe
    
    # reboot
    
  14. If you are using Oracle VM, run switch_to_ovm.sh on each database server. The database servers will reboot.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
    2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LV
    DbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
    2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbS
    ys1
    2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/
    VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSy
    s3 in /boot/I_am_hd_boot
    2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0
     system partition
    
  15. Run reclaimdisks.sh on each database server.

    The /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim command reclaims disk space reserved for the deployment type not selected. The command takes less than 5 minutes, approximately. Systems are imaged with disks configured with RAID5; a RAID rebuild is no longer part of the reclaimdisks.sh process.

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh.

    For example:

    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
    Model is ORACLE SERVER X5-2
    Number of LSI controllers: 1
    Physical disks found: 4 (252:0 252:1 252:2 252:3)
    Logical drives found: 1
    Linux logical drive: 0
    RAID Level for the Linux logical drive: 5
    Physical disks in the Linux logical drive: 4 (252:0 252:1 252:2 252:3)
    Dedicated Hot Spares for the Linux logical drive: 0
    Global Hot Spares: 0
    [INFO     ] Check for DOM0 system disk
    [INFO     ] Check for DOM0 with inactive Linux system disk
    [INFO     ] Valid DOM0 with inactive Linux system disk is detected
    [INFO     ] Number of partitions on the system device /dev/sda: 4
    [INFO     ] Higher partition number on the system device /dev/sda: 4
    [INFO     ] Last sector on the system device /dev/sda: 3509759999
    [INFO     ] End sector of the last partition on the system device /dev/sda: 350
    9759000
    [INFO     ] Unmount /EXAVMIMAGES from ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Remove inactive system logical volume /dev/VGExaDb/LVDbSys1
    [INFO     ] Remove logical volume /dev/VGExaDbOra/LVDbOra1
    [INFO     ] Remove volume group VGExaDbOra
    [INFO     ] Remove physical volume /dev/sda4
    [INFO     ] Remove partition /dev/sda4
    [INFO     ] Re-calculate end sector of the last partition after removing of /de
    v/sda4 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 330
    0035608
    [INFO     ] Check for existing first boot system image /EXAVMIMAGES/System.firs
    t.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Saving /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img in /v
    ar/log/exadatatmp ...
    [INFO     ] First boot system image saved in /var/log/exadatatmp/System.first.b
    oot.12.1.2.1.0.141205.2.img
    [INFO     ] Unmount /EXAVMIMAGES from /dev/sda3
    [INFO     ] Remove partition /dev/sda3
    [INFO     ] Re-calculate end sector of the last partition after removing of /de
    v/sda3 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 240
    132159
    [INFO     ] Create primary ocfs2 partition 3 using 240132160 3509758999
    [INFO     ] Create ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition on /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Restoring /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2
    .img into /EXAVMIMAGES ...
    [INFO     ] Logical volume LVDbSys2 exists in volume group VGExaDb
    [INFO     ] Grub version in /boot/grub/grub.stage.version: 0.97-81.0.1.el6
    [INFO     ] Grub rpm version: 0.97-13.10.0.1.el5
    [INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
    [INFO     ] Create filesystem on device /dev/sda1
    [INFO     ] Tune filesystem on device /dev/sda1
    
    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
    
    [ Minimal BASH-like line editing is supported.  For the first word, TAB
    lists possible command completions.  Anywhere else TAB lists the possible
    completions of a device/filename.]
    grub> root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83
    grub> setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
     Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succee
    ded
    Done.
    grub> quit
    
  16. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with elasticNode in the description.

  17. Run the OEDA tool to deploy.

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
    1. Validate Configuration File
    2. Setup Capacity-on-Demand
    3. Create Virtual Machine  
    4. Create Users  
    5. Setup Cell Connectivity  
    6. Create Cell Disks  
    7. Create Grid Disks  
    8. Configure Alerting  
    9. Install Cluster Software  
    10. Initialize Cluster Software  
    11. Install Database Software  
    12. Relink Database with RDS  
    13. Create ASM Diskgroups  
    14. Create Databases  
    15. Apply Security Fixes  
    16. Install Exachk  
    17. Setup ASR Alerting  
    18. Create Installation Summary  
    19. Resecure Machine
    

5.8.5 Using USB to Image a New System

To prepare a new Exadata system for deployment using USB, perform the following steps:

  1. Obtain the latest Oracle Exadata Deployment Assistant (OEDA).

  2. Run OEDA to generate the preconf.csv file that you can use to image the system.

  3. Power on all the servers.

    1. Database nodes and cells will automatically assign to themselves unused IP addresses on eth0 in the 172.16 range.

    2. Host names of the nodes will be of the format nodeN, where N is a number.

  4. Connect to the first database node. You can do this by connecting a laptop to the ILOM serial management port (9600, 8, N, 1), and then connecting to the console and logging in to the database node.

  5. Obtain the eth0 MAC address for each database node. This is the fru_macaddress field from /SYS/MB/NET0 in the ILOM, or use the command ip addr if logged into the console.

    1. Connect to the other database nodes using SSH from the console.

    2. Run ibhosts to get a list of node names and IP addresses.

  6. Insert the MAC addresses using uppercase characters in the seventh field of the preconf.csv file for each node. This field should be empty, that is, you should see two consecutive comma characters (,,). For example, if the MAC address is 00:10:E0:69:34:52:

    orhb42-b08-01,us.oracle.com,db,eth0,eth0,Management,00:10:E0:69:34:52,
    10.196.3.106,255.255.224.0,10.196.31.250,orhb42-b08-01-priv,Private:active-bond-ib,
    192.168.10.15,255.255.255.0,,,America/Los_Angeles
    
  7. Run ipconf to verify the generated preconf.csv file. This can be done on any Oracle Exadata server, and although not required, it is useful to ensure the preconf.csv file is valid before proceeding.

    # ipconf -verify -preconf /tmp/preconf.csv
    [Info]: ipconf command line: /opt/oracle.cellos/ipconf.pl -verify -preconf /tmp/preconf -nocodes
    [Done]: Pre config verification OK
    
  8. For versions prior to 12.1.2.2.0 proceed to step 11 to use ImageMaker to create the image files.

    If you are using version 12.1.2.2.0 or later, it is no longer necessary to use imagemaker to create the image files in version 12.1.2.2.0. The relevant PXE, ISO, and USB image files have already been created in the patches themselves. Simply download the patch containing the prebuilt image file from Oracle Software Delivery Cloud (edelivery) and place the downloaded USB .img image file onto the USB drive. For 12.1.2.2.0 the patch numbers can be obtained from My Oracle Support note 2038073.1 . There are separate ImageMaker files for database nodes and cells.

    Note : For image version 12.1.2.2.0 the format of the downloadable image files has changed.

    There will be an image file in the downloaded zip file. For example, the database node zip file will contain computeImageMaker_12.1.2.2.0_LINUX.X64_150917.x86_64.img. For cell images, the file will be similar to cellImageMaker_12.1.2.2.0_LINUX.X64_150917.x86_64.img.

  9. (For 12.1.2.2.0 and above) Follow the steps in the patch README.txt file to copy the file and optional preconf.csv file onto the USB drive.

    1. Place the computeImageMaker_12.1.2.2.0_LINUX.X64_150917.x86_64.img file in a directory on a Linux server.

    2. Verify that the md5sums match: f3daae6a9757d4feb4a0f4262e610945 computeImageMaker_12.1.2.2.0_LINUX.X64_150917.x86_64.img

    3. Prepare the USB drive using a command similar to the following, where /dev/sdd is the name of the inserted USB drive:

      # dd if=/dev/zero of=/dev/sdd bs=1M count=100 oflag=direct
      

      You can determine the exact name of the USB drive by looking in /var/log/messages after inserting the USB drive.

    4. Write the .img file to the USB drive. This may take 15 minutes or more and no output is shown during the operation.

      # dd if=filename.img of=/dev/sdd bs=1M oflag=direct
      
    5. Rescan the partition table on Linux to recognize the new partition:

      # partprobe
      
    6. Verify that the USB is mountable as long as the system supports ext4 file system. This must be an Oracle Linux 6 system for this step even if the previous dd command was run on an Oracle Linux 5 system.

      mount /dev/sdd1 /mnt
      
    7. (Optional) Prepare and place the preconf.csv file on the USB drive. The file name must be preconf.csv on the USB drive. The preconf.csv file must contain MAC addresses for each node in the order to be used during the image. If no preconf.csv file is used during imaging, the node will prompt for its network configuration the first time it boots up.

      # cp /path/preconf.csv /mnt/preconf.csv
      
      # umount /mnt
      

      The umount command above ensures that the file system is synchronized.

    8. Proceed to step 15. Steps 11 to 14, which create the image files, can be skipped.

  10. Download the ImageMaker kit to prepare and create the files for the USB drive. This kit is available publicly on the Oracle Software Delivery Cloud (edelivery) site under “Oracle Database Products -> Linux x86-64”. There are separate ImageMaker files for database nodes and cells.

  11. After downloading ImageMaker, unzip the first file, which is named similar to V75080-01.zip, and then also unzip the subsequent file. Perform the unzip as the root user. Then untar the file.

    # unzip V75080-01.zip
    
    # unzip cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.zip
    
    # tar -pxvf cellImageMaker_12.1.2.1.1_LINUX.X64_150316.2-1.x86_64.tar
    

    View the README_FOR_FACTORY.txt file in the resulting directory for additional details

  12. Insert a blank USB device in the slot of the server where ImageMaker was unpacked.

  13. Run makeImageMedia.sh to build the kernel, initrd, and image files for the USB device. makeImageMedia.sh will prompt to confirm creation of the image on the USB device. For building 12.1.2.1.0 and above images, the recommended makeImageMedia.sh options for building USB bootable media are:

    1. (For 12.1.2.1.0 up to 12.2.1.1.0) Change directory to the dl180 or dl360 directory.

    2. (For 12.2.1.1.0 and above) Change directory to the cell or compute directory.

    3. Make files for the USB.

      ./makeImageMedia.sh -factory -stit -reboot-on-success -nodisktests [-preconf path_to_preconf_file]
      

    For USB imaging, note the following:

    • makeImageMedia.sh must be run from an Oracle Exadata database server if using the -preconf option.

    • Storage servers and database servers can hang during boot when attempting a reimage using a bootable USB created with the makeImageMedia.sh script provided with storage cell and Linux database host images available on the Oracle Software Delivery Cloud (edelivery). To work around the issue, see My Oracle Support note 1919001.1

  14. After the USB drive has been prepared, place it into the slot of the server to be imaged. Ensure the image type matches the server type (database node or cell).

  15. Reboot the database node or cell to start the imaging process from USB.

  16. If you are using Oracle VM, run switch_to_ovm.sh on each database server. The database servers will reboot.

    For example:

    # /opt/oracle.SupportTools/switch_to_ovm.sh
    2014-12-07 11:58:36 -0800 [INFO] Switch to DOM0 system partition /dev/VGExaDb/LVDbSys3 (/dev/mapper/VGExaDb-LVDbSys3)
    2014-12-07 11:58:36 -0800 [INFO] Active system device: /dev/mapper/VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Active system device in boot area: /dev/mapper/VGExaDb-LVDbSys1
    2014-12-07 11:58:36 -0800 [INFO] Set active systen device to /dev/VGExaDb/LVDbSys3 in /boot/I_am_hd_boot
    2014-12-07 11:58:36 -0800 [INFO] Reboot has been initiated to switch to the DOM0 system partition
    
  17. Run reclaimdisks.sh on each database server.

    The /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim command reclaims disk space reserved for the deployment type not selected. The command takes less than 5 minutes, approximately. Systems are imaged with disks configured with RAID5; a RAID rebuild is no longer part of the reclaimdisks.sh process.

    Do not skip this step. Skipping this step results in unused space that can no longer be reclaimed by reclaimdisks.sh.

    For example:

    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
    Model is ORACLE SERVER X5-2
    Number of LSI controllers: 1
    Physical disks found: 4 (252:0 252:1 252:2 252:3)
    Logical drives found: 1
    Linux logical drive: 0
    RAID Level for the Linux logical drive: 5
    Physical disks in the Linux logical drive: 4 (252:0 252:1 252:2 252:3)
    Dedicated Hot Spares for the Linux logical drive: 0
    Global Hot Spares: 0
    [INFO     ] Check for DOM0 system disk
    [INFO     ] Check for DOM0 with inactive Linux system disk
    [INFO     ] Valid DOM0 with inactive Linux system disk is detected
    [INFO     ] Number of partitions on the system device /dev/sda: 4
    [INFO     ] Higher partition number on the system device /dev/sda: 4
    [INFO     ] Last sector on the system device /dev/sda: 3509759999
    [INFO     ] End sector of the last partition on the system device /dev/sda: 3509759000
    [INFO     ] Unmount /EXAVMIMAGES from ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Remove inactive system logical volume /dev/VGExaDb/LVDbSys1
    [INFO     ] Remove logical volume /dev/VGExaDbOra/LVDbOra1
    [INFO     ] Remove volume group VGExaDbOra
    [INFO     ] Remove physical volume /dev/sda4
    [INFO     ] Remove partition /dev/sda4
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/sda4 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 3300035608
    [INFO     ] Check for existing first boot system image /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Saving /EXAVMIMAGES/System.first.boot.12.1.2.1.0.141205.2.img in /var/log/exadatatmp ...
    [INFO     ] First boot system image saved in /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.img
    [INFO     ] Unmount /EXAVMIMAGES from /dev/sda3
    [INFO     ] Remove partition /dev/sda3
    [INFO     ] Re-calculate end sector of the last partition after removing of /dev/sda3 partition
    [INFO     ] End sector of the last partition on the system device /dev/sda: 240132159
    [INFO     ] Create primary ocfs2 partition 3 using 240132160 3509758999
    [INFO     ] Create ocfs2 partition on /dev/sda3
    [INFO     ] Mount ocfs2 partition on /dev/sda3 to /EXAVMIMAGES
    [INFO     ] Restoring /var/log/exadatatmp/System.first.boot.12.1.2.1.0.141205.2.img into /EXAVMIMAGES ...
    [INFO     ] Logical volume LVDbSys2 exists in volume group VGExaDb
    [INFO     ] Grub version in /boot/grub/grub.stage.version: 0.97-81.0.1.el6
    [INFO     ] Grub rpm version: 0.97-13.10.0.1.el5
    [INFO     ] Copying /usr/share/grub/x86_64-redhat/* to /boot/grub ...
    [INFO     ] Create filesystem on device /dev/sda1
    [INFO     ] Tune filesystem on device /dev/sda1
    
    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
    
    [ Minimal BASH-like line editing is supported.  For the first word, TAB
    lists possible command completions.  Anywhere else TAB lists the possible
    completions of a device/filename.]
    grub> root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83
    grub> setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
     Running "install /grub/stage1 (hd0) /grub/stage2 p /grub/grub.conf "... succeeded
    Done.
    grub> quit
    
  18. Run ibhosts and verify that all nodes show the correct IP addresses and host names. There should be no nodes with "elasticNode" in the description.

  19. Run the OEDA tool to deploy.

    [root] # ./install.sh -cf ../MAA-dm01.xml -l
    1. Validate Configuration File
    2. Setup Capacity-on-Demand
    3. Create Virtual Machine  
    4. Create Users  
    5. Setup Cell Connectivity  
    6. Create Cell Disks  
    7. Create Grid Disks  
    8. Configure Alerting  
    9. Install Cluster Software  
    10. Initialize Cluster Software  
    11. Install Database Software  
    12. Relink Database with RDS  
    13. Create ASM Diskgroups  
    14. Create Databases  
    15. Apply Security Fixes  
    16. Install Exachk  
    17. Setup ASR Alerting  
    18. Create Installation Summary  
    19. Resecure Machine
    

5.8.6 Updates to Imaging Parameters

Some of the command line options for imaging have changed with 12.1.2.1.0 and above.

The changes are:

  • updfrm

    Do NOT use the updfrm option in any form. It will be removed from the imaging options in the future. Allow the firmware to be updated automatically upon initial boot instead.

  • dualboot

    dualboot is no longer required with 12.1.2.1.0 as there is no support for Solaris. It has no meaning in releases 12.1.2.1.0 and above.

  • factory

    The meaning of the factory keyword has changed as of 12.1.2.1.0. The use of "factory" now enforces additional options such as "ovs=yes". This should be used for all imaging, regardless of the ultimate system configuration (Oracle VM, Physical etc.)

  • OVS=yes|no

    This option defaults to "no". However it is not recommended to use this at all because it may change in the future. In its place, use the "factory" option which will override some default options.

There are some additional command line options which may be useful:

  • To create a hot spare on dbnodes, use the enable_hot_spare option.

    • makeImageMedia.sh option: -enable_hot_spare

    • PXE option: enable_hot_spare

  • To force active bond or no active bond for InfiniBand interfaces (mainly for expansion rack cases), use the active-bond-ib option.

    • makeImageMedia.sh option: -active-bond-ib <yes|no>

    • PXE option: active-bond-ib=<yes|no>

The README_FOR_FACTORY.txt file shipped with the imagemaker software explains each option in more detail.

5.9 Performing Initial Elastic Configuration of Oracle Exadata Database Machine

This topic provides background information on elastic configuration and describes how to perform the initial configuration of Oracle Exadata Database Machine.

You need the configuration file generated by Oracle Exadata Database Machine Deployment Assistant.

Elastic configuration is new in Exadata version 12.1.2.1.0. It applies to all rack configurations, such as the following:

  • Racks ordered with a standard number of database nodes and cells, for example, quarter or half racks

  • Rack configurations that feature additional database nodes and cells

  • Oracle Exadata Database Machine Two-Socket X5 generation and later Storage Servers

Starting with the Oracle Exadata Database Machine Two-Socket X5 generation of Storage Servers, the same process can also be used to add additional database servers or cells to an existing configuration.

The process allows initial IP addresses to be assigned to database servers and cells, regardless of the exact customer configuration ordered. The customer specific configuration can then be applied to the nodes. The procedure can be run from any database node (usually the first node,) and also from a cell in cases where expansion involves adding only cell nodes.

Every Exadata system has a pre-defined method for the cabling of nodes to the InfiniBand switch ports. Therefore, there is a fixed mapping from each node's location in the rack to the ports of the InfiniBand switches. Assuming the rack is always populated following this map, a node's rack unit location can be identified by querying the InfiniBand fabric to determine the InfiniBand switch port the node is connected to. Once found, that information is used to determine the rack unit location of the node. With this information, nodes can be allocated initial IP addresses based on their rack unit location, with nodes lower in the rack getting lower IP addresses.

Note:

  • If a RAID rebuild/reconstruction (that is, a disk replacement or expansion) is in progress, wait until it is finished before performing the steps in this section. The reason is that the reclaimdisks.sh script is blocked from running until the RAID rebuild/reconstruction is complete.

  • Disk space for unused system partitions must be reclaimed before installing software. Use the following script to reclaim the disk space:

    /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim

  • My Oracle Support note 1953915.1 for detailed information about the configuration process.

  1. Open a console session to the first database server. The first database server is the lowest database server in the rack, which is rack position U16. In the case of an expansion involving only cells, the procedure should be run from a cell instead of a database node.

    Note:

    The network connection and configuration for Oracle Exadata Database Machine has not occurred yet.

  2. Log in as the root user on the first database server.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  3. Verify all servers are configured with the 172.16 IP address for the ETH0 interface using the following command:

    # ibhosts
    
    
    Ca  : 0x0010e00001486fb8 ports 2 "node10 elasticNode 172.16.2.46,172.16.2.46 ETH0"
    Ca  : 0x0010e00001491228 ports 2 "node9 elasticNode 172.16.2.45,172.16.2.45 ETH0"
    Ca  : 0x0010e000014844f8 ports 2 "node8 elasticNode 172.16.2.44,172.16.2.44 ETH0"
    Ca  : 0x0010e00001488218 ports 2 "node4 elasticNode 172.16.2.40,172.16.2.40 ETH0"
    Ca  : 0x0010e000014908b8 ports 2 "node2 elasticNode 172.16.2.38,172.16.2.38 ETH0"
    Ca  : 0x0010e0000148ca68 ports 2 "node1 elasticNode 172.16.2.37,172.16.2.37 ETH0"
    Ca  : 0x0010e00001485fd8 ports 2 "node3 elasticNode 172.16.2.39,172.16.2.39 ETH0"
    

    Each server description should be hostname elasticNode. ipaddress ETH0.

  4. (Optional on database servers) Switch to the Oracle VM when using virtualization on the rack using the following command:

    Note that database servers will reboot during this step.

    Note:

    It is not possible or supported to switch between physical and Oracle VM configuration.

    # /opt/oracle.SupportTools/switch_to_ovm.sh
    

    On database servers, run reclaimdisks.sh. This is necessary in all deployment scenarios.

    # /opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim
    

    If Oracle VM was chosen, this step locks in the image as Oracle VM. Going back and forth will not be possible.

    reclaimdisks.sh performs partition reconfiguration and a few other necessary configuration changes that take typically less than a minute to complete.

  5. On database servers, verify disk configuration.

    # /opt/oracle.SupportTools/reclaimdisks.sh -check
    

    If the system is a physical (bare metal) deployment, the last line of output will be the following:

    Valid. Booted: Linux. Layout: Linux.
    

    If the system is an Oracle VM deployment, the last line of output will be the following:

    Valid. Booted: DOM0. Layout: DOM0.
    
  6. Copy Oracle Exadata Database Machine Deployment Assistant configuration files to the database server using one of the following methods. If adding only cell servers, the steps can be run from a cell.

    • Using a USB flash drive on Linux:

      1. Copy the configuration file to a USB flash drive.

      2. Connect the USB drive to the database server or cell.

      3. Locate the USB drive using the following command:

        for x in `ls -1 /sys/block`; do udevadm info --attribute-walk --path=/sys/block/$x | grep \
        -iq 'DRIVERS=="usb-storage"'; if [ $? -eq 0 ] ; then echo /dev/${x}1; \
        fi ; done
        

        The expected output is similar to:

        /dev/sdb1
        

        Note:

        If an X6 or X5 EF cell is used, the command will also return the two internal USB drives, usually /dev/sda1 and /dev/sdb1. The newly inserted USB drive will be the additional device, usually /dev/sdc1.

        The expected output is similar to:

        /dev/sda1
        /dev/sdb1
        /dev/sdc1
        

        The internal USBs can be identified by viewing mdstat:

        # grep md4 /proc/mdstat | awk '{print $5,$6;}'
        
      4. Create a directory on the database server or cell using the following command:

        # mkdir /mnt/usb
        
      5. Create a directory for the installation using the following command:

        # mkdir /opt/oracle.SupportTools/onecommand
        

        If the directory already exists, then rename the existing one to onecommand.old, and then re-create the empty directory.

        Note: Do not use a directory under /u01 or /EXAVMIMAGES because reclaimdidsks.sh will fail if there is content in those directories.

      6. Mount the device. Use the device name given in step 6.c. The following is an example of the command.

        # mount -t vfat /dev/sdb1 /mnt/usb
        
      7. Copy Oracle Exadata Deployment Assistant and the configuration files from the USB flash drive to the onecommand directory on the database server or cell using the following command:

        # cp /mnt/usb/path_to_file/*.* /opt/oracle.SupportTools/onecommand
        

        In the preceding command, path_to_file is the directory path to the configuration file on the USB flash drive.

      8. Unmount the USB flash drive using the following command:

        # umount /mnt/usb
        
      9. Remove the USB flash drive from the database server or cell.

    • Using a temporary network connection:

      This option is only needed if there were IP address conflicts, and Oracle Exadata Database Machine is not already connected to the network.

      1. Connect a network cable to the Ethernet port labeled NET0 on the back of the first database server.

      2. Bring up the eth0:1 interface as follows:

        # ifconfig eth0:1 real_ip netmask real_netmask up
        

        In the preceding command, real_ip is the IP address to assign to the database server, and real_netmask is the subnet mask to assign to the database server.

      3. Add the default route as follows:

        # route add -net 0.0.0.0 netmask 0.0.0.0 gw real_gateway
        

        In the preceding command, real_gateway is the default gateway to use for the database server.

      4. Copy the configuration files to the /opt/oracle.SupportTools/onecommand directory on the database server or cell using a file transfer utility such as scp.

      5. Disconnect the network cable from Ethernet port NET0.

        Note:

        It is not necessary to reconfigure the network because the network is reconfigured automatically during the restart step.

  7. Verify all servers are configured with the 172.16 IP address for the ETH0 interface using the following command:

    # ibhosts
    

    No server description should include elasticNode in its description.

  8. Run the applyElasticConfig.sh script using the following command:

    # cd /opt/oracle.SupportTools/onecommand/linux-x64
    # ./applyElasticConfig.sh -cf customer_name-configFile.xml
    

    The applyElasticConfig.sh script performs network configuration for all database servers and Exadata Storage Servers. All servers restart at the end of the process.

  9. Connect the corporate network cable for the management network to the Cisco Ethernet switch.

  10. Connect the corporate network cables for the client access network to the database servers.

  11. Restart all database servers and Exadata Storage Servers.

  12. Log in to the first database server (or cell, if only cells were added for expansion) to verify network connectivity using the following commands:

    # cd /opt/oracle.SupportTools/onecommand/linux-x64
    #./checkip.sh -cf configuration_file
    

    In the preceding command, configuration_file is the name of the configuration file from Oracle Exadata Deployment Assistant, and includes the full path.

    Note:

    • The checkip.sh script can be run on a server outside of Oracle Exadata Database Machine to verify network connectivity to Oracle Exadata Database Machine. Copy the checkip.sh file to the external system, and run the command as shown in this step.

    • If the command is run from a Microsoft Windows machine, then the command is checkip.cmd.

    • Download the version of Oracle Exadata Deployment Assistant for the relevant platform.

  13. Use Oracle Exadata Deployment Assistant to install and configure the software for Oracle Exadata Database Machine.

See Also:

See Also:

5.10 Adding Additional Elastic Nodes to an Existing Rack

You can add nodes to an existing rack using elastic configuration.

The elastic configuration procedure described in "Performing Initial Elastic Configuration of Oracle Exadata Database Machine" can also be used to add new servers to an existing rack. The new servers and existing servers must be at least X5. The procedure assigns IP addresses to the new database servers and cells, in preparation for adding them into existing setups. The addition of X5, X6, or X7 servers into previous generation racks using this methodology is not supported.

The XML configuration file generated by the Oracle Exadata Deployment Assistant must contain the existing nodes, plus the new nodes to be added. The same elastic configuration procedures described in the previous section should be followed, and the applyElasticConfig.sh script that runs as part of step 8 determines which nodes are new, and assigns IP addresses accordingly.

When adding only storage cells, you can run the applyElasticConfig.sh script from a cell server rather than a database server.

5.11 Loading the Configuration Information and Installing the Software

Oracle Exadata Deployment Assistant loads the network settings, creates the user accounts, installs Oracle Database software, and secures the system based on the information the configuration files. You can also choose to do the process manually.

Note:

  • To enable a supported, non-English language, set the following environment variables before starting the configuration:

    export  LANG=preferred_language
    export  LC_ALL=preferred_language
    
  • To copy the database image to a USB drive, use the procedure in the database maintenance chapter of Oracle Exadata Database Machine Maintenance Guide For releases 12.1.2.1.0 and later, the system used to create the image must be running Oracle Linux 6.

See Also:

Using Oracle Exadata Deployment Assistant for descriptions of the configuration files generated by Oracle Exadata Database Machine Deployment Assistant

5.11.1 Configuring Oracle Exadata Database Machine Using OEDA

Oracle Exadata Deployment Assistant (OEDA)is used to configure Oracle Exadata Database Machine.

The deployment assistant works with the information provided in OEDA.

The following procedure describes how to configure Oracle Exadata Database Machine using OEDA:

  1. Go to My Oracle Support.

  2. Download the latest OEDA, and follow the steps in the associated README file.

  3. Download all necessary Oracle Exadata System Software and Oracle Database patches. See My Oracle Support note 888828.1 for details.

  4. Log in as the root user on the first database server.

    Note:

    If you do not have the password for the root user, then contact Oracle Support Services.

  5. Copy the configuration file from the machine that was used to run OEDA to the /opt/oracle.SupportTools/onecommand directory on the first database server of the Oracle Exadata Rack.

  6. Create a directory named WorkDir on the first database server.

  7. Put the downloaded software and patches in the WorkDir directory.

  8. Apply the OEDA update according to the instructions included in the README file.

  9. Change to the following directory:

    # cd /opt/oracle.SupportTools/onecommand
    
  10. Run the following command to configure Oracle Exadata Rack:

    # ./install.sh -cf configuration_file [-s step_number |  \
        -r step_numberA stepnumberB]
    

    Note:

    If the configuration is run from a Microsoft Windows machine, then the command is install.cmd.

    In the preceding command, configuration_file is the name of the configuration file, and includes the full path. The options for the script are:

    • -cf file : Use configuration file. The full path must be included with the file name.

    • -h : Describe options for the script.

    • -l : List the steps of OEDA.

    • -r {n-N | n } : Run steps n through N of OEDA, or one step.

    • -s n : Run step n of OEDA.

    • -u {n-N | n } : Undo steps n through N, or one step.

The following steps are performed by the utility. However, the actual steps for a specific deployment may differ depending on the deployment options chosen. For example, the "Update Nodes for Eighth Rack" step is performed only if you are doing an eighth rack deployment.

1. Validate Configuration File
2. Update Nodes for Eighth Rack
3. Create Virtual Machine
4. Create Users
5. Setup Cell Connectivity
6. Create Cell Disks
7. Create Grid Disks
8. Configure Alerting
9. Install Cluster Software
10. Initialize Cluster Software
11. Install Database Software
12. Relink Database with RDS
13. Create ASM Diskgroups
14. Create Databases
15. Apply Security Fixes
16. Install Exachk
17. Setup ASR Alerting
18. Create Installation Summary
19. Resecure Machine

Note:

The preceding steps are correct for the current release. However, the steps may differ for other releases. Always use the latest OEDA patch listed in My Oracle Support note 888828.1. Refer to the README in the patch for the latest steps.

See Also:

5.11.2 Configuring Oracle Database and Oracle ASM Instances for Oracle Exadata Database Machine Manually

Oracle Database and Oracle Automatic Storage Management (Oracle ASM) 11g Release 2 (11.2) must be installed on the database server. Follow the instructions for installing Oracle Database and Oracle ASM provided in the Oracle Database Installation Guide for your platform.

Note:

  • During Oracle Database installation, Database Configuration Assistant (DBCA) release 11.2.0.1.0 does not set the COMPATIBLE parameter equal to '11.2.0.2'. You must set the value on the Initialization Parameters page.

  • To fully utilize the Hardware Assisted Resilient Data (HARD) features, ensure the DB_BLOCK_CHECKSUM initialization parameter is set to TYPICAL (default) or FULL.

When installing Oracle Database, you must select the Oracle ASM storage option. If you are installing Oracle Real Application Clusters (Oracle RAC), then refer to the platform-specific Oracle Clusterware and Oracle RAC installation guides, such as Oracle Grid Infrastructure Installation and Upgrade Guide and Oracle Real Application Clusters Installation Guide.

Note:

For Oracle RAC, you must shut down the entire cluster, and then restart the cluster after the database and cell software have been installed, and the cell configuration files are in place.

The recommended procedure is to install Oracle software on Oracle Exadata Database Machine using Oracle Exadata Deployment Assistant (OEDA). If you choose to install the software manually, then do the following:

Note:

Disk space for unused system partitions must be reclaimed before installing software. Use the following script to reclaim the disk space:

/opt/oracle.SupportTools/reclaimdisks.sh -free -reclaim 

If a RAID rebuild or reconstruction (that is, a disk replacement or expansion) is in progress, wait until it is finished before running the reclaimdisks.sh script. The reason is that the reclaimdisks.sh script is blocked from running until the RAID rebuild or reconstruction is complete.

  1. Install Oracle Grid Infrastructure for a cluster as described in Oracle Grid Infrastructure Installation and Upgrade Guide for Linux.

  2. Install Oracle Database as described in Oracle Real Application Clusters Installation Guide for Linux and UNIX.

  3. Apply the latest patches for Oracle Database and Oracle Exadata System Software.

After the cells are configured as described in Oracle Exadata System Software User's Guide, and the database and Oracle ASM instances are installed, the instances must be configured to use the cells as part of an Oracle ASM disk group. The required configuration involves updating the initialization parameter file of both the database and Oracle ASM instances. Additionally, Oracle ASM disk groups must be created to access storage cells.

You can update the initialization parameter file of the Oracle ASM or the database instance while the instance is running, but the instance must be restarted if you manually update the initialization parameter file. If you update an initialization parameter dynamically using the SQL ALTER SYSTEM or ALTER SESSION commands, then there is no need to restart the instance.

In addition, for the database, you can configure a new Oracle Database Resource Manager plan, rather than using the default plans.

See Also:

5.11.2.1 Configuring the Compatible Parameter for a Database Instance

Oracle recommends using the best practices for Oracle Exadata System Software available at My Oracle Support.

To configure a database instance to access cell storage, ensure that the COMPATIBLE parameter is set to 11.2.0.2 or later in the database initialization file.

You can view the initialization parameter file from the operating system or use the SQL*Plus administration command SHOW PARAMETER if the database is running. For example:

SQL> SHOW PARAMETER
SQL> SHOW PARAMETER compatible

If necessary, set the COMPATIBLE initialization parameter in the initialization parameter file to 11.2.0.2 or later. For example:

COMPATIBLE='11.2.0.2'

The COMPATIBLE parameter cannot be changed dynamically. If you change the value in the initialization parameter file, then the database must be shut down and restarted for the change to take effect.

5.11.2.2 Configuring Initialization Parameters for an Oracle ASM Instance

To enable an Oracle ASM instance to discover and access Exadata Storage Server grid disks, you must configure the ASM_DISKSTRING initialization parameter as follows:

  • Set the ASM_DISKSTRING initialization parameter to the empty string ('') to discover all grid disks on the cells listed in the cellip.ora file. This causes Oracle ASM discovery for /dev/raw/* and any ASMLIB disks on any non-Exadata Storage Servers.

  • Set the ASM_DISKSTRING initialization parameter to restrict the discovery by Oracle ASM to only specific sets of disks, as shown in the examples.

    • To discover only a subset of cells, use a pattern for the ASM_DISKSTRING value such as the following:

      o/cell-connect-info-pattern/griddisk-name-pattern. 
      

      For example:

      ASM_DISKSTRING = 'o/*/data*'
      
    • To discover non-Exadata Storage Server disks and all grid disks on all the cells specified in the cellip.ora file, use a wildcard pattern for Exadata Storage Server grid disks. For example:

      ASM_DISKSTRING = '/dev/rdsk/disk*', 'o/*/*'
      

You can change the ASM_DISKSTRING initialization parameter when the Oracle ASM instance is running with the SQL ALTER SYSTEM command. If you edit the ASM_DISKSTRING initialization parameter in the initialization parameter file when the Oracle ASM instance is running, then the Oracle ASM instance must be shut down and restarted for the change to take effect.

See Also: