The following are known issues deploying, updating, and managing Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1:
Error: No protocol specified
or Exit Code 6
./OVS
directory is full and ODA_BASE is in read-only mode./var/log/messages
.odacli
commands, when the -u
option is not specified.When upgrading Oracle Database Appliance hardware models with virtualized platforms to 12.2.1.4.0, perform these manual steps before upgrading to release 12.1.2.12.
If you upgrade Oracle Database Appliance hardware models with a virtualized platform, to 12.1.2.12, then there is an error when using the driver domain functionality. To upgrade to Oracle Database Appliance release 12.2.1.4.0, follow the steps documented in the workaround section.
Hardware Models
Oracle Database Appliance X7-2-HA, X6-2-HA, X5-2, X4-2, X3-2, and V1 with a virtualized platform that use driver domain virtual machines
Workaround
Follow these steps to upgrade to Oracle Database Appliance release 12.2.1.4.0:
Deploy Oracle Database Appliance release 12.1.2.11 and create multiple virtual machines with driver domain enabled.
Shut down all running virtual machines and repository.
Apply the release 12.1.2.12 patch bundle.
Apply the release 12.2.1.4.0 patch bundle.
Start the repository. If you encounter the error OAKERR:5015 Start repo operation has been disabled by flag
, then run the following command:
oakcli enable startrepo -node 0/1
Start the virtual machines and confirm that they have started.
When patching Oracle Database 11.2.0.4, the log file may show some errors.
When patching Oracle Database 11.2.0.4 homes, the following error may be logged in alert.log
.
ORA-00600: internal error code, arguments: [kgfmGetCtx0], [kgfm.c], [2840], [ctx], [], [], [], [], [], [], [], []
Once the patching completes, the error will no longer be raised.
Hardware Models
Oracle Database Appliance X7-2-HA Virtualized Platform, X6-2-HA Bare Metal and Virtualized Platform, X5-2, X4-2, X3-2, and V1.
Workaround
There is no workaround for this issue.
This issue is tracked with Oracle bug 28032876.
The FLASH disk group is not mounted after a reboot, including after provisioning, reimaging, or patching the server with Oracle Database Appliance 12.2.1.2.
# oakcli update -patch 12.2.1.2 --server **************************************************************************** ***** For all X5-2 customers with 8TB disks, please make sure to ***** ***** run storage patch ASAP to update the disk firmware to "PAG1". ***** **************************************************************************** INFO: DB, ASM, Clusterware may be stopped during the patch if required INFO: Both Nodes may get rebooted automatically during the patch if required Do you want to continue: [Y/N]?: y INFO: User has confirmed for the reboot INFO: Patch bundle must be unpacked on the second Node also before applying the patch Did you unpack the patch bundle on the second Node? : [Y/N]? : y Please enter the 'root' password : Please re-enter the 'root' password: INFO: Setting up the SSH ..........Completed ..... ... ... INFO: 2017-12-26 00:31:22: -----------------Patching ILOM & BIOS----------------- INFO: 2017-12-26 00:31:22: ILOM is already running with version 3.2.9.23r116695 INFO: 2017-12-26 00:31:22: BIOS is already running with version 30110000 INFO: 2017-12-26 00:31:22: ILOM and BIOS will not be updated INFO: 2017-12-26 00:31:22: Getting the SP Interconnect state... INFO: 2017-12-26 00:31:44: Clusterware is running on local node INFO: 2017-12-26 00:31:44: Attempting to stop clusterware and its resources locally Killed # Connection to server.example.com closed.
The Oracle High Availability Services, Cluster Ready Services, Cluster Synchronization Services, and Event Manager are online. However, when you attempt to create an Oracle Automatic Storage Management Cluster File System (Oracle ACFS) database, you receive an error: flash space is 0
.
Hardware Models
Oracle Database Appliance X5-2, X6-2-HA, and X7-2 HA SSD systems.
Workaround
Manually mount FLASH disk group before creating an Oracle ACFS database.
Perform the following steps as the GRID owner:
Set the environment variables as grid OS user:
on node0 export ORACLE_SID=+ASM1 export ORACLE_HOME= /u01/app/12.2.0.1/grid
Log on to the ASM instance as sysasm
$ORACLE_HOME/bin/sqlplus / as sysasm
Execute the following SQL command:
SQL> ALTER DISKGROUP FLASH MOUNT
This issue is tracked with Oracle bug 27322213.
When patching a virtualized platform, the --local option is not supported.
On a virtualized platform, attempting to use the --local option to patch a single node will result in an error.
# oakcli update -patch 12.2.1.2.0 --server --local ERROR: -local is not supported for server patching, on VM systems.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Workaround
Use the following to update the software on Oracle Database Appliance, which applies the patch to both nodes.
# oakcli update -patch 12.2.1.4.0 --server
An issue with Oracle Database 12.2.1.2 might cause an internal error CRS-01019: THE OCR SERVICE EXITED. If this occurs, the Cluster Ready Services daemon (crsd) is down.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Oracle Database Appliance X7-2-HA virtualized platform
Workaround
Restart the CRS daemon.
Stop crs.
# crsctl stop crs -f
Start crs.
# crsctl start crs -wait
This issue is tracked with Oracle bug 27060167.
Run the database patch, after it finishes patching Node 0, the Clusterware does not run and patching on Node 1 hangs.
The Cluster Ready Services daemon (crsd) process becomes unresponse and the Oracle High Availability Services daemon (OHASD) cannot clean the resource. The server becomes unresponsive when other process cannot complete on the CPU and the load on the server increases. This issue is related to bug 27060167.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Oracle Database Appliance X7-2-HA
Workaround
Kill the crsd
processes running on both nodes, then restart the cluster.
This issue is tracked with Oracle bug 27366345.
When upgrading a database from 11.2.0.4 dbhome to 12.2.0.1, receive Error: No protocol specified
or Exit Code 6
.
Oracle Database Appliance performs pre-checks before applying the upgrade. If one or more of the pre-upgrade checks on the database results in warning conditions that required manual intervention, you should address the warnings as suggested before proceeding with the upgrade. If you receive the No protocol specified
or Exit Code 6
errors, then the warnings were not addressed before the upgrade.
Exit code 6 indicates successful execution with warnings.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Oracle Database Appliance X7-2-HA virtualized platform
Workaround
None.
This issue is tracked with Oracle bug 27381804 and 27074202.
Known issues with Oracle Automatic Storage Management (Oracle ASM) are preventing the REDO diskgroup from mounting for Oracle Database Release 12.1.
Unable to create an Oracle ASM database lower than 12.1.0.2.17814 PSU (12.1.2.12).
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1.
Workaround
There is not a workaround. If you have Oracle Database 11.2 or 12.1 that is using Oracle Automatic Storage Management (Oracle ASM) and you want to upgrade to a higher release of Oracle Database, then you must be on at least Oracle Database Appliance 12.1.2.12.0 and Database Home 12.1.0.2.170814.
The upgrade path for Oracle Database 11.2 or 12.1 Oracle ASM is as follows:
If you are on Oracle Database Appliance version 12.1.2.6.0 or later, then upgrade to 12.1.2.12 or higher before upgrading your database.
If you are on Oracle Database Appliance version 12.1.2.5 or earlier, then upgrade to 12.1.2.6.0, and then upgrade again to 12.1.2.12 or higher before upgrading your database.
This issue is tracked with Oracle bug 21626377, 27682997, and 21780146. The issues are fixed in Oracle Database 12.1.0.2.170814.
Cannot patch an empty Oracle Database Home (dbhome) due to an issue with Oracle Database auto patch.
ERROR: 2017-12-19 18:48:02: Unable to apply db patch on the following Homes : /u01/app/oracle/product/12.1.0.2/dbhome_name
The following is an example excerpt from the dbupdate
log:
OPATCHAUTO-68036: Topology empty. OPATCHAUTO-68036: The topology was empty, unable to proceed. OPATCHAUTO-68036: Check the log for more information. OPatchAuto failed. opatchauto failed with error code 42
Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1.
Workaround
The issue occurs when the dbhome does not have any databases. The workaround is to create a database before patching.
This issue is tracked with Oracle bug 27292674 and 27126871.
In some cases, the dbhome patch update fails due to a timezone issue in opatch.
An error similar to the following appears in the job details:
DCS-10001:Internal error encountered: run datapatch after bundlePatch application on the database home dbhomeID
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Workaround
Open the /u01/app/oracle/product/*/*/inventory/ContentsXML/comps.xml
file.
Search for four (4) character timezone (TZ) information.
For example, HADT and HAST.
Take a backup of those files.
Convert the 4-character timezone to a 3-character timezone.
For example, convert HADT and HAST to HST.
Patch dbhome.
This issue is tracked with Oracle bug 27313653 and 27331844.
When using Oracle Database 12.2.0.1 grid infrastructure (GI) to create an 11.2.0.4 database, the option to configure Oracle Enterprise Manager DB Console is disabled.
An issue with the Enterprise Manager Control (emctl) command line utility and Enterprise Manager Configuration Assistant (emca) occurs when using the 12.2.0.1 GI to create an 11.2.0.4 database.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1 that are using the 12.2.0.1 GI.
Oracle Database Appliance X7-2-HA Virtualized Platform that is using the 12.2.0.1 GI.
Workaround
Manually configure Oracle Enterprise Manager DB Console after creating the database.
If the appliance is a multi-node system, perform the steps on both nodes. The example assumes a multi-node system:
Create a dbconsole.rsp
response file, as follows, based on your environment.
To obtain the cluster name for your environment, run the command $GI_HOME/bin/cemutlo -n
DB_UNIQUE_NAME=pdb_unique_name SERVICE_NAME=db_unique_name.db_domain PORT=scan listener port LISTENER_OH=$GI_HOME SYS_PWD=admin password DBSNMP_PWD=admin password SYSMAN_PWD=admin password CLUSTER_NAME=cluster name ASM_OH=$GI_HOME ASM_SID=+ASM1 ASM_PORT=asm listener port ASM_USER_NAME=ASMSNMP ASM_USER_PWD=admin password
Run the command to configure the dbcontrol using the response file. The command will fail with an error. You will use the steps in the output in Step 4.
$ORACLE_HOME/bin/emca -config dbcontrol db -repos create -cluster -silent -respFile dbconsole.rsp Error securing Database Control. Database Control has not been brought-up on nodes node1 node2 Execute the following command(s) on nodes: node1 node2 1. Set the environment variable ORACLE_UNQNAME to the Database unique name. 2. /u01/app/oracle/product/11.2.0.4/dbhome_1/bin/emctl config emkey -repos -sysman_pwd Password for SYSMAN user -host node -sid Database unique name 3. /u01/app/oracle/product/11.2.0.4/dbhome_1/bin/emctl secure dbconsole -sysman_pwd Password for SYSMAN user -host node -sid Database unique name 4. /u01/app/oracle/product/11.2.0.4/dbhome_1/bin/emctl start dbconsole To secure Em Key, run /u01/app/oracle/product/11.2.0.4/dbhome_1/bin/emctl config emkey -remove_from_repos -sysman_pwd Password for SYSMAN user
Use vi editor to open $ORACLE_HOME/bin/emctl
, then change the setting CRS_HOME=
to CRS_HOME=/u01/app/12.2.0.1/grid
Run the steps reported by emca
in Step 2 with the proper values.
$ORACLE_HOME/bin/emca -reconfig dbcontrol -silent -cluster -EM_NODE node0 host -EM_NODE_LIST node1 host -DB_UNIQUE_NAME db_unique_name -SERVICE_NAME db_unique_name.db_domain
Use vi editor to open $ORACLE_HOME/bin/emctl
, then change the setting CRS_HOME=
to CRS_HOME=/u01/app/12.2.0.1/grid
Check the db console configuration status.
# /u01/app/oracle/product/11.2.0.4/dbhome_1/bin/emctl status agent - https://public IP for Node0:1158/em - https://public IP for Node1:1158/em
This issue is tracked with Oracle bug 27071994.
The /OVS
directory is full and ODA_BASE is in read-only mode.
The vmcore
file in the /OVS/ var
directory can cause the /OVS
directory (Dom 0) to become 100% used. When Dom 0 is full, ODA_BASE is in read-only mode or cannot start.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1.
Oracle Database Appliance X7-2-HA Virtualized Platform.
Workaround
Perform the following to correct or prevent this issue:
Periodically check the file usage on Dom 0 and clean up the vmcore
file, as needed.
Edit the oda_base vm.cfg
file and change the on_crash = 'coredump-restart'
parameter to on_crash = 'restart'
. Especially when ODA_BASE is using more than 200 GB (gigabytes) of memory.
This issue is tracked with Oracle bug 26121450.
When starting a virtual machine (VM), an error message appears that the domain does not exist.
If a VM was cloned in Oracle Database Appliance 12.1.2.10 or earlier, you cannot start the HVM domain VMs in Oracle Database Appliance 12.1.2.11.
This issue does not impact newly cloned VMs in Oracle Database Appliance 12.1.2.11 or any other type of VM cloned on older versions. The vm templates were fixed in 12.1.2.11.0.
When trying to start the VM (vm4
in this example), the output is similar to the following:
# oakcli start vm vm4 -d . Start VM : test on Node Number : 0 failed. DETAILS: Attempting to start vm on node:0=>FAILED. <OAKERR:7007 Error encountered while starting VM - Error: Domain 'vm4' does not exist.>
The following is an example of the vm.cfg
file for vm4
:
vif = [''] name = 'vm4' extra = 'NODENAME=vm4' builder = 'hvm' cpus = '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23' vcpus = 2 memory = 2048 cpu_cap = 0 vnc = 1 serial = 'pty' disk = [u'file:/OVS/Repositories/odarepo1/VirtualMachines/vm4/68c32afe2ba8493e89f018a 970c644ea.img,xvda,w'] maxvcpus = 2 maxmem = 2048
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Oracle Database Appliance X7-2-HA Virtualized Platform.
Workaround
Delete the extra = 'NODENAME=vm_name'
line from the vm.cfg
file for the VM that failed to start.
Open the vm.cfg
file for the virtual machine (vm) that failed to start.
Dom0 : /Repositories/ vm_repo_name /.ACFS/snaps/ vm_name / VirtualMachines/ vm_name
ODA_BASE : /app/sharedrepo/ vm_repo_name /.ACFS/snaps/ vm_name / VirtualMachines/ vm_name
Delete the following line: extra=’NODENAME=vmname’
. For example, if virtual machine vm4
failed to start, delete the line extra = 'NODENAME=vm4'
.
vif = [''] name = 'vm4' extra = 'NODENAME=vm4' builder = 'hvm' cpus = '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23' vcpus = 2 memory = 2048 cpu_cap = 0 vnc = 1 serial = 'pty' disk = [u'file:/OVS/Repositories/odarepo1/VirtualMachines/vm4/68c32afe2ba8493e89f018a 970c644ea.img,xvda,w'] maxvcpus = 2 maxmem = 2048
Start the virtual machine on Oracle Database Appliance 12.1.2.11.0.
# oakcli start vm vm4
This issue is tracked with Oracle bug 25943318.
After applying the server patch and rebooting the node, the kernel version is not updated.
This issue occurs with Oracle Database Appliance 12.1.2.11 and 12.1.2.12.
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Workaround
Update to release 12.2.1.4.0
# oakli update -patch 12.2.1.4.0 --server --local
Remove the kernel rpm.
# rpm -e kernel-uek-4.1.12-103.3.8.1.el6uek.x86_64
Manually install the kernel rpm using rpm -ivh
.
kernel-uek-4.1.12-103.3.8.1.el6uek.x86_64
Modify /boot/grub/grub.conf
to boot from the new kernel. Change default=1
to default=0
# cat /boot/grub/grub.conf timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz #hiddenmenu serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 terminal --timeout=5 serial console default=0 title Oracle Linux Server Unbreakable Enterprise Kernel (4.1.12-103.3.8.1.el6uek.x86_64) root (hd0,0) kernel /vmlinuz-4.1.12-103.3.8.1.el6uek.x86_64 ro root=LABEL=rootfs tsc=reliable nohpet nopmtimer hda=noprobe hdb=noprobe ide0=noprobe numa=off console=tty0 console=ttyS0,115200n8 selinux=0 nohz=off crashkernel=256M@64M loglevel=3 panic=60 ipv6.disable=1 transparent_hugepage=never NODENUM=0 PRODUCT=SUN_SERVER_X4-2 TYPE=V3 pci=noaer initrd /initramfs-4.1.12-103.3.8.1.el6uek.x86_64.img title Oracle Linux Server (4.1.12-61.44.1.el6uek.x86_64) root (hd0,0) kernel /vmlinuz-4.1.12-61.44.1.el6uek.x86_64 ro root=LABEL=rootfs tsc=reliable nohpet nopmtimer hda=noprobe hdb=noprobe ide0=noprobe numa=off console=tty0 console=ttyS0,115200n8 selinux=0 nohz=off crashkernel=256M@64M loglevel=3 panic=60 ipv6.disable=1 transparent_hugepage=never NODENUM=0 PRODUCT=SUN_SERVER_X4-2 TYPE=V3 pci=noaer initrd /initramfs-4.1.12-61.44.1.el6uek.x86_64.img
Reboot the node.
# uname -r 4.1.12-103.3.8.1.el6uek.x86_64
Repeat for Node 1.
This issue is tracked with Oracle bug 26887116.
After updating Oracle Database Appliance, unrecognized token messages appear in /var/log/messages
.
Updating to Oracle Database Appliance 12.1.2.11.0 updates the Oracle VM Server version to 3.4.3. After updating, the following messages appear in /var/log/messages
:
Unrecognized token: "max_seq_redisc" Unrecognized token: "rereg_on_guid_migr" Unrecognized token: "aguid_inout_notice" Unrecognized token: "sm_assign_guid_func" Unrecognized token: "reports" Unrecognized token: "per_module_logging" Unrecognized token: "consolidate_ipv4_mask"
You can ignore the messages for these parameters, they do not impact the InfiniBand compliant Subnet Manager and Administration (opensm) functionality. However, Oracle recommends removing the parameters to avoid flooding /var/log/messages
.
Hardware Models
Oracle Database Appliance X6-2-HA and X5-2 with InfiniBand
Workaround
Perform the following to remove the parameters:
After patching, update the /etc/opensm/opensm.conf
file in bare metal deployments and in Dom0 in virtualized platform environment to remove the parameters.
cat /etc/opensm/opensm.conf | egrep -w 'max_seq_redisc|rereg_on_guid_migr|aguid_inout_notice|sm_assign_guid_func|repo rts|per_module_logging|consolidate_ipv4_mask' | grep -v ^# max_seq_redisc 0 rereg_on_guid_migr FALSE aguid_inout_notice FALSE sm_assign_guid_func uniq_count reports 2 per_module_logging FALSE consolidate_ipv4_mask 0xFFFFFFFF
Reboot. The messages will not appear after rebooting the node.
This issue is tracked with Oracle bug 25985258.
High Availability IP (HAIP) addresses are not supported on Oracle engineered systems.
If you use an HAIP address, then an error message will appear in your operating system log indicating that the address is not supported.
The following error messages might appear in system logs during boot of Oracle Database Appliance systems:
Aug 11 15:31:11 odac1n1 kernel: [ 9932.651622] ** WARNING WARNING WARNING WARNING WARNING ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651624] ** ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651627] ** RDS/IB: Link local address 169.254.165.15 NOT SUPPORTED ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651629] ** ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651631] ** HAIP IP addresses should not be used on ORACLE ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651632] ** engineered systems ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651634] ** ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651636] ** If you see this message, Please refer to ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651638] ** cluster_interconnects in MOS note #1274318.1 ** Aug 11 15:31:11 odac1n1 kernel: [ 9932.651639]
You can ignore these messages. Functionality is not impacted.
Hardware Models
Oracle Database Appliance X6-2S, X6-2M, X6-2L, X6-2-HA, X5-2, X4-2, X3-2, and V1
This issue is tracked with Oracle bug 26623697.
Network information for node0 is always displayed for some odacli
commands, when the -u
option is not specified.
If the -u
option is not provided, then the describe-networkinterface
, list-networks
and the describe-network
odacli
commands always display the results for node0 (the default node), irrespective of whether the command is run from node0 or node1.
Hardware Models
Oracle Database Appliance X7-2-HA, X6-2-HA, X5-2, X4-2, X3-2, and V1
Workaround
Specify the -u
option in the odacli
command, for details about the current node.
This issue is tracked with Oracle bug 27251239.
When attempting to create an DSS database with shape odb-01s, the job may fail with the following error:
CRS-2674: Start of 'ora.test.db' on 'rwsoda609c1n1' failed CRS-5017: The resource action "ora.test.db start" encountered the following error: ORA-03113: end-of-file on communication channel Process ID: 0 Session ID: 0 Serial number: 0 . For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/rwsoda609c1n2/crs/trace/crsd_oraagent_oracle.trc".
Hardware Models
Oracle Database Appliance X6-2-HA, X5-2, X4-2, X3-2, and V1
Workaround
There is no workaround. Select an alternate shape to create the database.
This issue is tracked with Oracle bug 27768012.