4 Provisioning Oracle Database Appliance Baremetal System

Understand the process to configure Oracle Database Appliance baremetal system.

The Web Console is the preferred method of deploying your bare metal platform configuration. The Web Console provides all of the fields necessary to provision Oracle Database Appliance, including configuring the system, network, database, and Oracle Auto Service Request (Oracle ASR). If you prefer to use the command-line interface, you must create a JSON file to configure the deployment options.

Plumbing the Network

Plumb the Oracle Database Appliance network with the public internet protocol (IP) information assigned to a node, to enable upload of the Oracle Database Appliance software.

  1. Connect to Oracle ILOM remote console, then log into Oracle Database Appliance as root.
  2. Run the command configure-firstnet on Node0.
    # /opt/oracle/dcs/bin/odacli/configure-firstnet
    Using bonding public interface (yes/no) [yes]:
    Select the Interface to configure the network on (btbond1) [btbond1]:
    Configure DHCP on btbond1 (yes/no) [no]:
    INFO: You have chosen Static configuration
    Use VLAN on btbond1 (yes/no) [no]:
  3. Complete the network configuration as prompted and configure the public network interface. Also provide the netmask and gateway IP address.
You use this network connection to transfer the software to the server node.

Example 4-1 Example of a Bonded Configuration

For a bonded configuration, answer yes to using a bonding public interface and configure the first network to use a btbond1 interface without configuring DHCP.

# opt/oracle/dcs/bin/odacli/configure-firstnet
Using bonding public interface (yes/no) [yes]: yes
Select the Interface to configure the network on (btbond1): btbond1
Configure DHCP on btbond1 (yes/no): no
INFO: You have chosen Static configuration
Enter the IP address to configure: 10.1.10.2
Enter the Netmask address to configure: 255.255.255.0
Enter the Gateway address to configure: 10.1.10.1
INFO: Plumbing the IPs now
INFO: Restarting the network
Shutting down interface btbond1: bonding: btbond1: Removing slave em2. 
bonding: btbond1: releasing active interface em2 
bonding: btbond1: making interface em3 the new active one. 
bonding: btbond1: Removing slave em3. 
bonding: btbond1: releasing active interface em3 
:::::::::: 

Example 4-2 Example of a Non-Bonded Configuration

To create a non-bonded configuration and disable VLAN capabilities, answer no to using a bonding public interface.

# opt/oracle/dcs/bin/odacli/configure-firstnet
Using bonding public interface (yes/no) [yes]: no
INFO: Breaking the bonding on btbond1
INFO: remove bonding module: rmmod bonding
INFO: remove slave setup in /etc/sysconfig/network-scripts/ifcfg-em2
INFO: remove slave setup in /etc/sysconfig/network-scripts/ifcfg-em3
INFO: Restarting the network
Shutting down interface em1:                               [  OK  ]
Shutting down interface p1p1:                              [  OK  ]
Shutting down interface p1p2:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface em1:                                 [  OK  ]
Bringing up interface em2:                                 [  OK  ]
Bringing up interface em3:                                 [  OK  ]
Bringing up interface p1p1:  
Determining if ip address 192.168.16.24 is already in use for device p1p1...
                                                           [  OK  ]
Bringing up interface p1p2:  
Determining if ip address 192.168.17.24 is already in use for device p1p2...
                                                           [  OK  ]
INFO: Restarting the network
Select the Interface to configure the network on (em2 em3) [em2]:
Configure DHCP on em2 (yes/no) [no]:
INFO: You have chosen Static configuration
Enter the IP address to configure : 10.1.10.2
Enter the Netmask address to configure : 255.255.255.0
Enter the Gateway address to configure[10.1.10.1] :
INFO: Plumbing the IPs now
/usr/share/system-config-network/netconfpkg/NCHostsList.py:100: 
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  badlines.append((num, value_exception.message))
/usr/share/system-config-network/netconfpkg/NCHostsList.py:105: 
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  """ % (value_exception.message, num)
/usr/share/system-config-network/netconfpkg/NCProfileList.py:142: 
DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
  self.error = e.message
INFO: Restarting the network
Shutting down interface em1:                               [  OK  ]
Shutting down interface em2:                               [  OK  ]
Shutting down interface em3:                               [  OK  ]
Shutting down interface p1p1:                              [  OK  ]
Shutting down interface p1p2:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface em1:                                 [  OK  ]
Bringing up interface em2:  
Determining if ip address 10.1.10.2 is already in use for device em2...
                                                           [  OK  ]
Bringing up interface em3:                                 [  OK  ]
Bringing up interface p1p1:  
Determining if ip address 192.0.2.24 is already in use for device p1p1...
                                                           [  OK  ]
Bringing up interface p1p2:  
Determining if ip address 192.0.2.25 is already in use for device p1p2...
                                                           [  OK  ]
 

Example 4-3 Example of Configuring VLAN

# configure-firstnet
Using bonding public interface (yes/no) [yes]:
Select the Interface to configure the network on (btbond1) [btbond1]:
Configure DHCP on btbond1 (yes/no) [no]:
INFO: You have chosen Static configuration
Use VLAN on btbond1 (yes/no) [no]:yes
Configure VLAN on btbond1, input VLAN ID [2 - 4094] 122
INFO: using network interface btbond1.122
Enter the IP address to configure : 192.0.2.24
Enter the Netmask address to configure : 192.0.2.24
Enter the Gateway address to configure[192.0.2.24] :
INFO: Restarting the network
Shutting down interface btbond1:                           [  OK  ]
Shutting down interface em1:                               [  OK  ]
Shutting down interface p1p1:                              [  OK  ]
Shutting down interface p1p2:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]
Bringing up loopback interface:                            [  OK  ]
Bringing up interface btbond1:  
Determining if ip address 192.0.2.24 is already in use for device btbond1...
                                                           [  OK  ]
Bringing up interface em1:                                 [  OK  ]
Bringing up interface p1p1:  
Determining if ip address 192.0.2.24 is already in use for device p1p1...                                                           [  OK  ]
Bringing up interface p1p2:  
Determining if ip address 192.0.2.24 is already in use for device p1p2...
                                                           [  OK  ]
Bringing up interface btbond1.122:  
Determining if ip address 192.0.2.24 is already in use for device btbond1.122...
                                                           [  OK 
INFO: Restarting the DCS agent
initdcsagent stop/waiting
initdcsagent start/running, process 32104

Verifying Cabling and Network Connections

After powering on Oracle Database Appliance, verify that the network connections are set up correctly.

Run the validation scripts only on Oracle Database Appliance HA models.
  1. Log into the Oracle Database Appliance console and access the server node with user name as root and password welcome1.
  2. Run validation scripts on both nodes, to verify cabling:
    # /opt/oracle/dcs/bin/odacli validate-storagetopology
    INFO : ODA Topology Verification
    INFO : Running on Node0
    INFO : Check hardware type
    SUCCESS : Type of hardware found : X7-2
    INFO : Check for Environment(Bare Metal or Virtual Machine)
    SUCCESS : Type of environment found : Bare Metal
    INFO : Check number of Controllers
    SUCCESS : Number of ahci controller found : 1
    SUCCESS : Number of External SCSI controllers found : 2
    INFO : Check for Controllers correct PCIe slot address
    SUCCESS : Internal RAID controller :
    SUCCESS : External LSI SAS controller 0 : 3b:00.0
    SUCCESS : External LSI SAS controller 1 : 5e:00.0
    INFO : Check if JBOD powered on
    SUCCESS : 2JBOD : Powered-on
    INFO : Check for correct number of EBODS(2 or 4)
    SUCCESS : EBOD found : 4
    INFO : Check for External Controller 0
    SUCCESS : Cable check for port 0 on controller 0
    SUCCESS : Cable check for port 1 on controller 0
    SUCCESS : Overall Cable check for controller 0
    INFO : Check for External Controller 1
    SUCCESS : Cable check for port 0 on controller 1
    SUCCESS : Cable check for port 1 on controller 1
    SUCCESS : Overall Cable check for controller 1
    INFO : Check for overall status of cable validation on Node0
    SUCCESS : Overall Cable Validation on Node0
    SUCCESS : JBOD0 Nickname set correctly
    SUCCESS : JBOD1 Nickname set correctly
    
  3. If there is a storage expansion shelf connected to the appliance, then power it on. Run the storagetopology command on both nodes to make sure the cabling is correct:
    # /opt/oracle/dcs/bin/odacli validate-storagetopology
    INFO : ODA Topology Verification
    INFO : Running on Node0
    INFO : Check hardware type
    SUCCESS : Type of hardware found : X7-2
    INFO : Check for Environment(Bare Metal or Virtual Machine)
    SUCCESS : Type of environment found : Bare Metal
    INFO : Check number of Controllers
    SUCCESS : Number of ahci controller found : 1
    SUCCESS : Number of External SCSI controllers found : 2
    INFO : Check for Controllers correct PCIe slot address
    SUCCESS : Internal RAID controller :
    SUCCESS : External LSI SAS controller 0 : 3b:00.0
    SUCCESS : External LSI SAS controller 1 : 5e:00.0
    INFO : Check if JBOD powered on
    SUCCESS : 2JBOD : Powered-on
    INFO : Check for correct number of EBODS(2 or 4)
    SUCCESS : EBOD found : 4
    INFO : Check for External Controller 0
    SUCCESS : Cable check for port 0 on controller 0
    SUCCESS : Cable check for port 1 on controller 0
    SUCCESS : Overall Cable check for controller 0
    INFO : Check for External Controller 1
    SUCCESS : Cable check for port 0 on controller 1
    SUCCESS : Cable check for port 1 on controller 1
    SUCCESS : Overall Cable check for controller 1
    INFO : Check for overall status of cable validation on Node0
    SUCCESS : Overall Cable Validation on Node0
    SUCCESS : JBOD0 Nickname set correctly
    SUCCESS : JBOD1 Nickname set correctly
  4. Verify the interconnect network:
    # ethtool p1p1
    Settings for p1p1:
    Supported ports: [ FIBRE ]
    Supported link modes: 1000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: Yes
    Advertised link modes: 1000baseT/Full
    10000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Speed: 25000Mb/s <<<<<<<<<<<<<<<<<<<<<<< check speed
    Duplex: Full
    Port: Direct Attach Copper
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: on
    Current message level: 0x00000000 (0)
    Link detected: yes 
    # ethtool p1p2
    Settings for p1p2:
    Supported ports: [ FIBRE ]
    Supported link modes: 1000baseT/Full
    Supported pause frame use: Symmetric Receive-only
    Supports auto-negotiation: Yes
    Advertised link modes: 1000baseT/Full
    10000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Speed: 25000Mb/s <<<<<<<<<<<<<<<<<<<<<<< check speed
    Duplex: Full
    Port: Direct Attach Copper
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: on
    Current message level: 0x00000000 (0)
    Link detected: yes 
    

Downloading Oracle Database Appliance Software

Download Oracle Database Appliance software and copy to a temporary location before applying updates to your appliance.

  1. Download the software files from My Oracle Support to a temporary location on an external client. Refer to the release notes for details about the software for the latest release.
    For example, download the Oracle Database Appliance GI Clone for ODACLI/DCS stack (patch 27604593) and Oracle Database Appliance RDBMS Clone for ODACLI/DCS stack (patch 27604558) for 18.3:
    p27604593_183000_Linux-x86-64.zip
    p27604558_183000_Linux-x86-64.zip
  2. Unzip the software — it contains README.html and one or more zip files for the patch.
    unzip p27604593_183000_Linux-x86-64.zip
    unzip p27604558_183000_Linux-x86-64.zip

    The GI and RDBMS Clone files for release 18.3 are extracted.

    odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip
    odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
  3. Copy the software files from the external client to Oracle Database Appliance. Use the scp or sftp protocol to copy the bundle.
    Example using scp command:
    scp software_file root@oda_host:/tmp
    Example using sftp command:
    sftp root@oda_host
    Enter the root password, and then copy the file.
    put software_file

    For example, for release 18.3:

    put odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip
    put odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
The GI and RDBMS Clone files are copied to the temporary location on the appliance. You can update the repository, and create the appliance, as explained in the next topic.

Installing Oracle Database Appliance Software

Install Oracle Database Appliance software, before creating the appliance.

  1. Verify the current system version by running the following command on both nodes:
    # /opt/oracle/dcs/bin/odacli describe-component -v
  2. Update the repository with Oracle Grid Infrastructure and Database software.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli update-repository 
    -f /tmp/GI_clone_file,/tmp/DB_clone_file

    For example, for release 18.3

    # /opt/oracle/dcs/bin/odacli update-repository -f 
    /tmp/odacli-dcs-18.3.0.0.0-181205-GI-18.3.0.0.zip,
    /tmp/odacli-dcs-18.3.0.0.0-180905-DB-18.0.0.0.zip
  3. Confirm that the repository update is successful:
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-job -i job_ID
  4. Delete the software zip files from the temporary location on the appliance to save space:
    [root@oda1 opt]# rm software_file
    rm: remove regular file software_file? y
You have now updated the repository with the Oracle Database Appliance software, and can create your appliance, as described in the next topic.

Creating the Appliance

Create the appliance using the Web Console.

Ensure that ports 7070 and 7093 on the network between ODA and the client machine, are open.
  1. Navigate to the Web Console. You are prompted to set the password for the oda-admin user.
    https://ODA-host-ip-address:7093/mgmt/index.html
  2. Log into the Web Console with the new password.
    When you launch the Web Console on an unconfigured appliance, the Appliance page notifies you that the appliance is not configured and provides a link to the pages needed to configure and create the appliance.
  3. Click Create Appliance.
  4. In the Create Appliance page, if you want to create the appliance from a saved configuration, click Browse, and select the JSON file. The information from the saved configuration file is loaded. You can retain or edit the fields. Otherwise, manually, provide the values to create the appliance.
  5. In the System Information section, specify the values as follows.
    1. Host Name: Enter the host name.
      The host name can contain alphanumeric characters and dashes (-), but cannot start with a number or dash (-) or end with dash (-). Do not exceed 30 characters.
    2. Domain Name: Enter the domain name.
    3. Region: Select the region of the world where the Oracle Database Appliance is located.
    4. Time Zone: Select the time zone where the Oracle Database Appliance is located.
    5. (Optional) DNS Servers: Enter addresses for one or more DNS servers.
    6. (Optional) NTP Servers: Enter addresses for one or more NTP servers.
    7. Diskgroup Redundancy: If there are up to two disk groups, then you can select the disk group redundancy as Normal or Flex. A quorum disk must exist, if you want to select Flex redundancy. If there are more than two disk groups, then you can select the redundancy as Normal, High, or Flex. Select Normal, High, or Flex, as per your deployment requirement.
      The Flex parameter defines the disk group redundancy. For Oracle ASM storage, when you select Flex as the disk group redundancy, you can set the database redundancy to either Mirror or High. You can set the disk group redundancy only during appliance creation. The database redundancy can be set anytime you create the database.
      The redundancy level is for DATA, RECO, and FLASH. If you select High redundancy, then DATA, RECO, and FLASH are all High redundancy.

      If the system has less than five (5) NVMe storage devices, then redundancy is automatically set to Normal.

      If the system has more than three (3) NVMe storage devices, then you can choose the Flex redundancy option.

      The redundancy level for REDO can be set to Flex, but the file groups created in REDO are all High redundancy.

    8. Data Storage Percentage: Enter a whole number between 10 and 90 to define the percentage of storage reserved for DATA, the remainder is reserved for RECO. For example, if you enter 80, then 80% of the storage for DATA and 20% for RECO.
    9. System Password and Confirm Password: Enter the system password in both fields.
      The system password is the password set for UNIX and root users. The password must begin with an alpha character and cannot contain quotation marks. Do not exceed 30 characters.
  6. In the Network Information section, configure the primary client access network, virtual networks, and an Oracle Integrated Lights Out Manager (ILOM) network. You are only required to configure the client access network. The ILOM configuration is optional if you already configured the ILOM for the appliance and you do not need to make changes.
    Specify the following, depending on whether it is a single-node or multi-node Oracle Database Appliance deployment:
    1. Client Access Network Host Name: For Node0, enter the host name for the primary client access network.
    2. Client Access Network IP Address: For Node0, enter the IP address for the primary client access network.
    3. VIP Name for Node0: Enter the name of the virtual IP network for Node0.
    4. VIP IP Address for Node0: Enter the virtual IP address that resides on Node0 and is shared between the nodes.
    5. VIP Name for Node1: Enter the name of the virtual IP network for Node0.
    6. VIP IP Address for Node1: Enter the virtual IP address that resides on Node1 and is shared between the nodes.
    7. (Optional) ILOM Host Name: Enter the name of the Oracle ILOM host for Node0 and Node1.
    8. (Optional) ILOM Network IP Address: Enter the IP address for the ILOM for Node0 and Node1.
    9. (Optional) ILOM Network Subnet Mask: Enter the subnet mask address for the ILOM.
    10. (Optional) ILOM Network Gateway: Enter the gateway address for the ILOM.
    11. Client Access Network Subnet Mask: Enter the subnet mask address for the primary client access network.
    12. Client Access Network Gateway: Enter the gateway address for the primary client access network.
    13. Client Access Network Interface: Enter the interface for the primary client access network.
  7. Determine how you want to configure your users and groups and whether or not you want to allow operating system role separation:
    • Two users with six groups: Customize Users and Groups, select No. Allow OS Role Separation, select Yes. This is the default configuration.
    • Two customized users with six customized groups: Customize Users and Groups, select Yes. Allow OS Role Separation, select Yes.
    • Single user with two groups: Customize Users and Groups, select No. Allow OS Role Separation, select No
    • Single user with six groups: Customize Users and Groups, select Yes. Allow OS Role Separation, select No. SAP deployments use this configuration.
  8. Do you want to create an initial database? Select Yes and go to Step 9 or select No and go to Step 10.
  9. Enter the following information to configure an initial database:
    1. DB Name: Enter a name for the database.
      The name must contain alphanumeric characters and cannot exceed 8 characters.
    2. (Optional) DB Unique Name: Enter a globally unique name for the database.
      Databases with the same DB Name within the same domain (for example, copies of a database created for reporting or a physical standby) must have a different DB Unique Name that is unique within the enterprise. The name must contain alphanumeric, underscore (_), dollar ($), and pound (#) characters, but must begin with an alphabetic character. No other special characters are permitted in a database name. The unique name cannot exceed 30 characters.
    3. DB Version: Select a database bundle patch number.
    4. CDB: Select Yes or No to specify whether or not you want a Container Database (CDB).
    5. PDB Name: Enter a name for the pluggable database (PDB).
      The name must begin with an alphanumeric character. The following characters are valid: alphanumeric characters, and underscore (_).
    6. PDB Admin User: Enter an Admin user name for the pluggable database (PDB).
    7. Database Edition: Select the Oracle Database edition, either Standard Edition and Enterprise Edition. Your license determines which database edition you are eligible to create in the appliance.
    8. Password: Provide a password for the database.
    9. Deployment: Select a deployment type from the list. The options are RAC, RAC-One, or SI (single instance database). If you select a single instance database, then select the node for the SI database deployment.
      If you select a single instance database, then you have the option to create the database on either Node0 or Node1. The default is Node0.
    10. Shape: Select a database shape from the list.
    11. In the Database Class field, select a database class from the drop-down list. If an option is not available in the list, it is not supported for the database edition on the Oracle Database Appliance or the version that you selected. The default is OLTP.
    12. In the Storage field, select ACFS or ASM from the drop-down list. The default is Oracle ASM.
      Only Oracle 18c, 12.2, and 12.1 databases can use Oracle ASM storage. Oracle Database 11.2 is only supported on Oracle ACFS.
    13. If you select the storage as ASM, and the Database Version is 12.2 or later, and you specified the disk group redundancy as Flex, then you can select the Database Redundancy value as Mirror or High.
      You can choose Oracle ACFS storage for all database versions, but if you select Oracle ACFS storage, then you cannot specify the database redundancy. If your disk group redundancy is Flex, and you choose Oracle ACFS storage for the database, then the database redundancy is set to Mirror.
    14. Storage: Select ACFS or ASM.
    15. Data Files on Flash Storage: Select Yes or No.
      This option is only available if the HA system has HDD storage drives.
    16. Configure EM Express: Select Yes or No.
      Select Yes to configure the Oracle Enterprise Manager Database Express (EM Express) console for Oracle Database 12.2.1.0 or 12.1.0.2 or the Database Control Console for Oracle Database 11.2.0.4. Selecting Yes enables you to use the console to manage the database.
    17. Characterset: Select a characterset.
    18. National Characterset: Select a national characterset.
    19. Language: Select the database language.
    20. Territory: Select a territory or location from the list.
  10. (Optional) Configure and enable Oracle ASR on the ASR page.

    You can configure and enable Oracle Auto Service Request (Oracle ASR) now or later:

    • To not enable Oracle ASR during deployment, select No and click Submit. After deployment, you can configure an internal Oracle ASR or register with an external Oracle ASR Manager from either the Web Console or command-line interface.
      • Internal Oracle ASR: choose to configure Oracle ASR Manager on Oracle Database Appliance or use Oracle ASR Manager configured on another server in the same network as your appliance.

      • External Oracle ASR: If you already have Oracle ASR Manager configured elsewhere, you can register Oracle Database Appliance with your existing Oracle ASR Manager.

    • To enable Oracle ASR, select Yes and complete the fields:
    1. ASR User Name: Enter the e-mail address associated with the My Oracle Support account under which the server is registered.
    2. Password: Enter the password associated with the My Oracle Support account under which the server is registered.
    3. SNMP Version: Select V2 or V3. V2 is the default and recommended version.
    4. HTTP Proxy used for Upload to ASR: Select Yes or No.
    5. Proxy Server Name: If you are using a proxy for upload, enter the proxy server name.
    6. Proxy Port: If you are using a proxy for upload, enter the proxy port.
    7. (Optional) HTTP Proxy Requires Authentication: If you are using a proxy for upload, select Yes if you require authentication. If you do not require authentication, select No.
    8. Proxy User Name: If you are using a proxy for upload, enter the proxy user name.
    9. (Optional) Proxy Password: If you are using a proxy for upload and require authentication, enter the proxy password.
  11. Click Submit. When prompted, click Yes to confirm that you want to start the job to deploy the appliance.
  12. Verify that the appliance is deployed. Run the odacli describe-appliance command. For multi-node deployments, run the command on both nodes.
    [root@oda1 opt]# /opt/oracle/dcs/bin/odacli describe-appliance
    Appliance Information
    ----------------------------------------------------------------
    ID: 618dfa94-35ff-4f81-8aab-e93810bc2c45
    Platform: Odav6
    Data Disk Count: 48
    CPU Core Count: 36
    Created: July 24, 2018 1:55:57 PM MDT
    System Information
    ----------------------------------------------------------------
    Name: odax7-2ha
    Domain Name: example.com
    Time Zone: America/Denver
    DB Edition: EE
    DNS Servers: 192.1.1.1 192.1.1.2
    NTP Servers: 192.1.1.3 192.1.1.4
    Disk Group Information
    ----------------------------------------------------------------
    DG Name Redundancy Percentage
    ------------------------- ------------------------- ------------
    Data High 80
    Reco High 20
    Redo High 100
    Flash High 100
The job to create the appliance takes time to complete. To monitor the job progress, click the Activity tab. Click the job number to view the tasks.

After deployment, the root and database users SYS, SYSTEM, and PDBADMIN are set to the system password. The oracle and grid passwords are set to the default password. Change these passwords to comply with your user security protocols.