Known Issues

The following lists describe the known issues with Oracle Cloud Infrastructure.

Announcements

Currently, there are no known Announcements issues.

Anomaly Detection

Currently, there are no known issues with the Anomaly Detection service.

API Gateway

API gateways do not inherit custom DNS servers from subnets

Details: The default Oracle Cloud Infrastructure Resolver resolves public URL endpoints (and URL endpoints with public hostnames) to IP addresses. Additionally, a subnet can be configured with a custom DNS server that resolves internal hostname (and private hostname) URL endpoints to IP addresses. However, API gateways you create with the API Gateway service do not inherit custom DNS servers from subnets. Instead API gateways use the default Oracle Cloud Infrastructure Resolver, which does not resolve internal/private hostname URL endpoints.

Due to this restriction, if you create an API gateway that has an internal/private hostname URL endpoint as the HTTP or HTTPS URL back end, calls to the API will fail because the hostname cannot be resolved to an IP address.

Workaround: We are aware of the issue and working on a resolution. In the meantime, if you want to create an API gateway that has an internal/private URL endpoint as the HTTP or HTTPS URL back end, you must specify the host's IP address in the URL rather than the hostname. In addition, if the back end is an HTTPS URL, you must also select the Disable SSL Verification option in the Console (or include isSSLVerifyDisabled: true in the API deployment specification JSON file).

Direct link to this issue: API gateways do not inherit custom DNS servers from subnets

Application Performance Monitoring

Browser and Scripted Browser monitors might not run applications that use frames

Details: In Synthetic Monitoring, the Browser and Scripted Browser monitors might fail to run against applications that use frames.

Workaround: We are aware of the issue and working on a resolution. For Scripted Browser monitors, you can work around this issue by replacing index=<frame-index> with either id=<id-of-frame> or name=<name-of-frame> in the .side script.

For example, if this script is the original version:

{
      "id": "23956f51-8812-40e6-ac91-1d608871ee4c",
      "comment": "",
      "command": "selectFrame",
      "target": "index=0",
      "targets": [
        ["index=0"]
      ],
      "value": ""
    }

The following script would be the modified version:

{
      "id": "23956f51-8812-40e6-ac91-1d608871ee4c",
      "comment": "",
      "command": "selectFrame",
      "target": "id=frame1",
      "targets": [
        ["id=frame1"]
      ],
      "value": ""
    }

Direct link to this issue: Browser and Scripted Browser monitors might not run applications that use frames

Issues with the authorization policies based on the apm-domains resource tags

Details: Authorization policies based on the apm-domains resource tags do not work for the Trace Explorer and Synthetic Monitoring APIs, causing authorization failures.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Issues with the authorization policies based on the apm-domains resource tags

Artifact Registry

For known issues with Artifact Registry, see Known Issues.

Audit

Currently, there are no known Audit issues.

Automated CEMLI Execution

For known issues with Automated CEMLI Execution, see Known Issues.

Bastion

Managed SSH session fails for Ubuntu instances

Details: To create a Managed SSH session for a compute instance, the Bastion plugin must be enabled and running. This plugin is available on Oracle Cloud Agent version 1.11 or later. If your Ubuntu instance is running a version that is older than 1.11, the creation of a Managed SSH session fails.

Workaround:

To update an existing Ubuntu compute instance to support Managed SSH sessions:

  1. Add a NAT gateway  to the VCN (virtual cloud network)  in which you created your instance, if not already present. See Setting Up a NAT Gateway.
  2. From your bastion, create a Port Forwarding session (SSH tunnel) to the SSH port (22 by default) on the instance. See Managing Sessions.
  3. Connect to the instance using the Port Forwarding session. See Connecting to Sessions.
  4. To install the latest Oracle Cloud Agent, run the following command on the instance:
    sudo snap refresh oracle-cloud-agent

    For more information, see Installing the Oracle Cloud Agent Software.

  5. Enable the Bastion plugin on the instance. See Managing Plugins Using the Console.
  6. From your bastion, create a Managed SSH session to the instance.

If you're creating another Ubuntu instance, an alternative workaround is to provide a cloud-init script when you launch the instance. In this script, use the same command to install the latest Oracle Cloud Agent:

sudo snap refresh oracle-cloud-agent

For more information about cloud-init scripts, see Installing the Oracle Cloud Agent Software.

Direct link to this issue: Managed SSH session fails for Ubuntu instances

Managed SSH sessions are supported for Arm instances only if they run Oracle Linux

Details: Managed SSH sessions are not supported for compute instances that meet all of these conditions:

  • Created using Arm-based Ampere A1 Compute shapes.
  • Running an operating system other than Oracle Linux, such as Ubuntu.

To create a Managed SSH session, the Bastion plugin must be enabled and running. Because this plugin is not properly enabled on some Arm-based instances, session creation fails.

Workaround: Create a port forwarding session to connect to instances that use Ampere A1 Compute shapes and aren't running Oracle Linux.

Direct link to this issue: Managed SSH sessions are supported for Arm instances only if they run Oracle Linux

Big Data

Synchronize hive databases task fails when specifying wildcard character in Apache Ambari

Details: In Big Data clusters using Oracle Distribution including Apache Hadoop, if you synchronize the hive databases by specifying the wildcard character * for the Synchronize Hive Databases property using Apache Ambari, you receive an error that states that the synchronization of Hive Metadata failed.

Workaround: We are aware of the issue and working on a resolution. Meanwhile, do not use the wildcard character * for the Synchronize Hive Databases property, but explicitly specify the Hive databases that you want to synchronize as a comma separated, no spaces list. For example: db1,db2 .

Direct link to this issue: Synchronize hive databases task fails when specifying wildcard character in Apache Ambari.

Billing

Currently, there are no known Billing issues.

Block Volume

Cross-region replication not supported for volumes encrypted with customer-managed keys

Details: When you try to enable cross-region replication for a volume configured to use a Vault encryption key, the following error message occurs: Edit Volume Error: You cannot enable cross-region replication for volume <volume_ID> as it uses a Vault encryption key.

Workaround: We are aware of the issue and working on a resolution. Cross-region replication is not supported for volumes encrypted with a customer-managed key. As a workaround to enable replication, unassign the Vault encryption key from the volume. In this scenario, the volume is encrypted with an Oracle-managed key.

Direct link to this issue: Cross-region replication not supported for volumes encrypted with customer-managed keys

Paravirtualized volume attachment not multipath-enabled after instance is resized

Details: To achieve the optimal performance level for volumes configured for ultra high performance, the volume attachment must be multipath-enabled. Multipath-enabled attachments to VM instances are only supported for instances based on shapes with 16 or greater OCPUs.

If you have an instance with fewer than 16 OCPUs, you can resize it so that it has 16 or more OCPUs to support multipath-enabled attachments. This step will not work for instances where the original number of OCPUs was less than 8 and the volume attachment is paravirtualized. In this scenario, after the volume is detached and reattached, the volume attachment will still not be multipath-enabled even though the instance now supports multipath-enabled attachments.

Workaround: As a workaround, we recommend that you create a new instance based on a shape with 16 or more OCPUs, and then attach the volume to the new instance.

Direct link to this issue: Paravirtualized volume attachment not multipath-enabled after instance is resized

Attaching the maximum number of block volumes to smaller VM.Standard.A1.Flex instances might fail

Details: When you attempt to attach the maximum number of block volumes to a smaller VM.Standard.A1.Flex instance, in some cases, the volumes might fail to attach. This happens because of limitations with the underlying physical host configuration.

Workaround: We are aware of the issue and working on a resolution. As a workaround, we recommend that you increase the size of the VM by resizing the VM, and then try attaching the volumes again.

Direct link to this issue: Attaching the maximum number of block volumes to smaller VM.Standard.A1.Flex instances might fail

Python SDK composite operations throw a 404 NotAuthorizedOrNotFound error even though operation is a success

Details: The copy_boot_volume_backup_and_wait_for_state and copy_volume_backup_and_wait_for_state from the BlockStorage Client Composite operations throw a 404/NotAuthorizedOrNotFound when copying a backup from one region to another. For more information see: https://github.com/oracle/oci-python-sdk/issues/344.

Workaround: Instead of using the composite operations, use two different clients for this operation; one client in the Source Region to send the request for copying the backup from Region A to Region B, and a second client in Destination region to wait for the backup to become available. See example here: https://github.com/oracle/oci-python-sdk/blob/master/examples/copy_volume_backup_example.py

Direct link to this issue: Python SDK composite operations throw a 404 NotAuthorizedOrNotFound error even though operation is a success

Vault encryption keys not copied to destination region for scheduled cross region backup copies

Details: When you schedule volume and volume group backups using a backup policy that is enabled for cross-region copy for volumes that are encrypted using Vault service encryption keys, the encryption keys are not copied with the volume backup to the destination region. The volume backup copies in the destination region are instead encrypted using Oracle-provided keys.

Workaround: We are aware of the issue and working on a resolution. As a workaround, you can manually copy volume backups and volume group backups across regions, either manually or using a script, and specify the key management key ID in the target region for the copy operation. For more information about manual cross region copy, see Copying a Volume Backup Between Regions.

Direct link to this issue: Vault encryption keys not copied to destination region for scheduled cross region backup copies

Change compartment end event not emitted for block volumes and boot volumes

Details: The com.oraclecloud.blockvolumes.changevolumecompartment.end and com.oraclecloud.blockvolumes.changebootvolumecompartment.end events are not emitted after their corresponding begin events by the Block Volume service even when the operations completed successfully.

Workaround: We are aware of the issue and working on a resolution. Verify directly that your resource was moved to the new compartment.

Direct link to this issue: Change compartment end event not emitted for block volumes and boot volumes

updatevolumekmskey and updatebootvolumekmskey events missing information for block volumes and boot volumes

Details: The com.oraclecloud.blockvolumes.updatevolumekmskey.begin and com.oraclecloud.blockvolumes.updatebootvolumekmskey.begin events are missing the current field, which should contain the KMS key ID of the new key to configure for the volume. Instead, the previous field contains this value, when the previous field should contain the previous KMS key ID.

Workaround: We are aware of the issue and working on a resolution. Verify that your resource has the expected KMS key ID after the update.

Direct link to this issue: updatevolumekmskey and updatebootvolumekmskey events missing information for block volumes and boot volumes

volumeId field format is incorrect in create event with manual volume and boot volume backups

Details:The volumeId field in additionalDetails for the com.oraclecloud.blockvolumes.createvolumebackup.end and com.oraclecloud.blockvolumes.createbootvolumebackup.end events is formatted as an object and not as a string for manually created backups. This means that rules set to trigger on this field will not be triggered for manually created backups. This field is formatted correctly as a string for scheduled backups.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: volumeId field format is incorrect for create event with manual volume and boot volume backups

additionalDetails information missing for copyvolumebackup.begin and copyvolumebackup.end events

Details: The sourceBackupId field and the destinationRegion field are missing in additionalDetails for the com.oraclecloud.blockvolumes.copyvolumebackup.begin and com.oraclecloud.blockvolumes.copyvolumebackup.end events, so rules set to trigger based on these fields will not be triggered.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: additionalDetails information missing for copyvolumebackup.begin and copyvolumebackup.end events

Device path option not available for instances launched before January 11, 2019

Details: When attaching a block volume to an instance launched before January 11, 2019, you cannot specify a device path.

Direct link to this issue: Device path option not available for instances launched before January 11, 2019

409 error occurs when cloning a volume

Details: When you clone a volume that is still attached to an instance, delete the clone, and then clone the volume again, you may encounter the following error:

Volume <volume-OCID> cannot be cloned in parallel while attached

This error may also return with a 409 response code.

Workaround: If you're using the API, CLI, SDK, or Terraform you need to monitor the isHydrated attribute of the deleted clone and not create the second clone until this attribute value is true. If you're using the Console, monitor the Hydrated field on the Block Volume Details page for the deleted clone and not create the second clone until this field value is true.

Direct link to this issue: 409 error occurs when cloning a volume

Attaching a Windows boot volume as a data volume to another instance fails

Details: When you attach a Windows boot volume as a data volume to another instance, when you try to connect to the volume using the steps described in Connecting to a Volume the volume fails to attach and you may encounter the following error:

Connect-IscsiTarget : The target has already been logged in via an iSCSI session.

Workaround: You need to append the following to the Connect-IscsiTarget command copied from the Console:

-IsMultipathEnabled $True

Direct link to this issue: Attaching a Windows boot volume as a data volume to another instance fails

volume-group create operation fails on Windows instances using the CLI

Details: When you use the CLI on Windows to create a volume group and supply inline JSON input for the source-details parameter, the operation fails.

Workaround: We are aware of the issue and working on a resolution. To work around this issue, wrap the inline JSON in double quotes instead of single quotes. You also need to escape the double quotes within the JSON itself. For example, the following code excerpt works on Linux instances:

--source-details '{"type": "volumeIds", 

To get it to work on Windows instances, modify it to:

--source-details "{\"type\": \"volumeIds\", 

Direct link to this issue: volume-group create operation fails on Windows instances using the CLI

Boot Volume resize fails for clone and restore from backup using the CLI

Details: When you use the CLI to clone a boot volume or restore a boot volume from a backup, you cannot resize the volume.

Workaround: We are aware of the issue and working on a resolution. To work around this issue, clone the boot volume or restore it from a backup without resizing it and then you can resize the volume after the clone or restore operation is complete.

Direct link to this issue: Boot Volume resize fails for clone and restore from backup using the CLI

CLI help text is incorrect for Volume and Boot Volume create commands

Details: The help text for the size-in-gbs option and size-in-mbs option are incorrect for the oci bv volume create and the oci bv boot-volume create CLI commands. They incorrectly state that these options cannot be supplied when cloning a volume or restoring a volume from a backup. This is incorrect, they are available to specify when you clone a volume or restore a volume from a backup to a larger size volume than the original source volume. You cannot specify a value smaller than the size of the original source volume.

Workaround: We are aware of the issue and working on a resolution. You can ignore the help text for these command options.

Direct link to this issue: CLI help text is incorrect for Volume and Boot Volume create commands

bootVolumeSizeInGBs attribute is null

Details: When calling GetInstance, the bootVolumeSizeInGBs attribute of InstanceSourceViaImageDetails is null.

Workaround: We are aware of the issue and working on a resolution. To work around this issue, call GetBootVolume, and use the sizeInGBs attribute of BootVolume.

Direct link to this issue: bootVolumeSizeInGBs attribute is null

Blockchain Platform

For known issues with Blockchain Platform, see Known Issues.

Certificates

Currently, there are no known Certificates issues.

Classic Migration

For known issues with Classic Migration Service, see Known Issues.

Cloud Guard

Reporting region cannot be changed

Details: Reporting region is assigned during Cloud Guard enablement. Once assigned, this setting cannot be changed, even upon disable and enable of Cloud Guard.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Reporting region cannot be changed

No value checking for conditional groups

Details: Detector and responder rules apply to a particular resource type. Conditional groups allow you specify particular resources of that type to be included or excluded from applying a rule.

Scenario 1: You can provide resource OCIDs to a conditional group as custom values or in a managed list. Cloud Guard does not check the validity of these values.

Scenario 2: When you add a country or region as a conditional group parameter to an activity detector, Cloud Guard does not check the validity of these values.

Workaround: In both scenarios above, ensure that you provide valid values. For a list of valid country and region values, see Using Conditional Groups with Detectors in Modifying a Cloned Detector Recipe.

Direct link to this issue: No value checking for conditional groups

Cloud Shell

Go SDK cannot automatically find some regions while running in Cloud Shell

Details: Due to some issues with one of its dependencies, the Go SDK feature which allows customers to automatically use new realms which might be unknown to the SDK is not functioning from within Cloud Shell.

Attempting to run code in Cloud Shell that uses this feature will result in the following error message:
can not create client, bad configuration: failed to get security token: failed to renew security token: failed to get security token: failed to call: Post "https://<endpoint>/v1/x509": dial tcp: lookup <endpoint> on 127.0.0.11:53: server misbehaving
panicked while retrying operation. Panic was: runtime error: invalid memory address or nil pointer dereference

Workaround: To resolve this issue, enable resolving regions using the instance metadata service for Go SDK. For more information, see: Adding Regions

Direct link to this issue: Go SDK cannot automatically find some regions while running in Cloud Shell

Compliance Documents

Currently, there are no known Compliance Documents issues.

Compute

Kernel panic when running containers on Ubuntu 20.04, kernel 5.13.0-1033.39~20.04.1

Details: When you run containers on a Compute instance that uses Ubuntu 20.04, kernel version linux-oracle-5.13 5.13.0-1033.39~20.04.1, a kernel panic occurs. The instance crashes and is inaccessible. For more information, see Docker container creation causes kernel oops on linux-aws 5.13.0.1028.31~20.04.22.

Workaround: Upgrade the kernel to a higher version by running the following commands:

sudo apt-get update
sudo apt-get upgrade -y linux-image-oracle

Direct link to this issue: Kernel panic when running containers on Ubuntu 20.04, kernel 5.13.0-1033.39~20.04.1

Older E3/E4 flex shape VM instances fail to start after resizing memory to more than 1,010 GB

Details: E3/E4 flex shape VM instances created before April 5, 2021 fail to start if the memory is resized to more than 1,010 GB. In this case, you see an error that reads "failed to start."

Workaround: We are aware of the issue and working on a resolution. As a workaround, reduce the size of memory to less than 1,010 GB. Or you can create the instance again, and then resize the instance memory up to 1,024 GB.

Direct link to this issue: Older E3/E4 flex shape VM instances fail to start after resizing memory to more than 1,010 GB

Console shows Oracle Autonomous Linux available as an Always Free image

Details: Oracle Autonomous Linux is not supported for Always Free compute instances, but in the Console, Oracle Autonomous Linux appears in the list of supported images for Always Free shapes.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Console shows Oracle Autonomous Linux available as an Always Free image

DNS not working as expected on Oracle Linux instances

Details: In the US East (Ashburn) region, when Oracle Linux instances first boot after provisioning, DNS might not work as expected, and the search field in the /etc/resolv.conf file might be incomplete.

Workaround: We are aware of the issue and working on a resolution. Either reboot the instance or wait for the next DHCP lease renewal. After the DHCP lease renewal, the issue resolves automatically. The standard DHCP lease time is 24 hours but varies depending on network settings.

Direct link to this issue: DNS not working as expected on Oracle Linux instances

PCR values change after reboot on Linux 7.x

Details: When you create a shielded instance using Linux 7.x and then reboot the instance, PCR values might change, causing the red shield to appear.

Workaround: We are aware of the issue and working on a resolution. Some PCR values change at runtime. That change is expected. As a workaround, reset the golden measurements.

Direct link to this issue: PCR values change after reboot on Linux 7.x

BM.Standard.A1.160 instances experience degraded network performance to applications running on socket 1 CPUs

Details: Bare metal instances that use the BM.Standard.A1.160 shape experience reduced network performance for workloads running on socket 1 CPUs.

Workaround: We are aware of the issue and working on a resolution. For applications responsible for processing packets from the network, bind them to CPUs from socket 0.

Direct link to this issue: BM.Standard.A1.160 instances experience degraded network performance to applications running on socket 1 CPUs

Oracle Cloud Agent doesn't post metrics on Windows instances in private subnets with only a service gateway attached

Details: When you provision a compute instance on Windows in a private subnet with a service gateway attached, the Oracle Cloud Agent plugins might not emit metrics.

Workaround: Follow the steps in the Microsoft known issue article: Connectivity issues if the DigiCert Global Root G2 root certificate is not installed.

Direct link to this issue: Oracle Cloud Agent doesn't post metrics on Windows instances in private subnets with only a service gateway attached

Oracle Cloud Agent version 1.11.0 does not update automatically

Details: The Oracle Cloud Agent software does not update itself automatically. This issue affects the following versions of Oracle Cloud Agent, depending on the image that Oracle Cloud Agent is installed on:

  • Windows Server: affects Oracle Cloud Agent version 1.11.0.x
  • Oracle Linux 7, Oracle Linux 8, CentOS 7: affects Oracle Cloud Agent versions 1.11.0.x and 1.11.1.x

Workaround: Manually update Oracle Cloud Agent to the latest version.

Direct link to this issue: Oracle Cloud Agent version 1.11.0 does not update automatically

VM.Standard.A1.Flex instances support only the paravirtualized networking launch option

Details: Instances that use the VM.Standard.A1.Flex shape with hardware-assisted (SR-IOV) networking might face performance issues, and in rare cases, data corruption. To prevent this, platform images for Ampere A1 Compute (aarch64) are configured to use paravirtualized networking only. If you create an instance using a platform image and specify hardware-assisted networking, the launch will fail with a message similar to Failed to validate instance launch options.

For custom images compatible with Ampere A1 Compute, the launch will succeed, but we strongly recommend not selecting hardware-assisted networking to avoid potential performance and data corruption issues.

Workaround: We are aware of the issue and working on a resolution. When you create a VM.Standard.A1.Flex instance using a platform image, let Oracle choose the recommended networking launch type. For custom images, do not use hardware-assisted (SR-IOV) networking.

Direct link to this issue: VM.Standard.A1.Flex instances support only the paravirtualized networking launch option

Invalid shape and image error when creating Intel and AMD instances using Terraform

Details: When you use Terraform to create an Intel or AMD compute instance using a Linux platform image, the operation might fail with the error code InvalidParameter and a message similar to Shape <shape_name> is not valid for image <image_OCID>.

This happens if Terraform identifies the latest image based on the image display_name. Images for Intel and AMD shapes (x86 processor architecture) have similar names to the images for Arm-based shapes (aarch64 processor architecture), but the images are not cross-compatible across processor architectures. If the latest image is an aarch64 image, Terraform selects an aarch64 image for an x86 shape, causing the operation to fail.

Workaround: We are aware of the issue and working on a resolution.

As a workaround, modify the following Terraform files:
  • /home/opc/JDERefArch_InfraProvisioning/TerraformScripts/global/global.datasources.tf
  • /home/opc/JDERefArch_InfraProvisioning/TerraformScripts/pd/pd.datasources.tf
  • /home/opc/JDERefArch_InfraProvisioning/TerraformScripts/nonpd/nonpd.datasources.tf
  • /home/opc/JDERefArch_InfraProvisioning/TerraformScripts/globalDR/globalDR.datasources.tf
  • /home/opc/JDERefArch_InfraProvisioning/TerraformScripts/pdDR/pdDR.datasources.tf

In the files, update the regular expression that identifies the image to filter out all images for Arm-based shapes. Images for Arm-based shapes include "aarch" in the image name.

For example, for Oracle Linux 8 images, make the following update:

  • Current regular expression: values = ["^.*Oracle-Linux-8[.]*[\\d]*-[^G].*$"]
  • Updated regular expression: values = ["^.*Oracle-Linux-8[.][0-9]*-[\\d]{4}.[\\d]{2}.[\\d]{2}-[\\d]*$"]

Direct link to this issue: Invalid shape and image error when creating Intel and AMD instances using Terraform

Oracle Linux Cloud Developer images cannot be managed by the OS Management service

Details: Instances that use the Oracle Linux Cloud Developer image cannot be managed by the OS Management service.

Workaround: We are aware of the issue and working on a resolution. Do not install the OS Management Service Agent (osms-agent) on Oracle Linux Cloud Developer instances.

Direct link to this issue: Oracle Linux Cloud Developer images cannot be managed by the OS Management service

Invalid bucketName error when importing or exporting a custom image

Details: When you try to import or export a custom image from an Object Storage bucket, an error similar to the following might occur:

Invalid bucketName: Specified namespace or bucket to export image does not exist

This error happens for federated users and for users authenticating with instance principals tied to a dynamic group.

Workaround: To avoid this issue, create a pre-authenticated request, and then use the pre-authenticated request to import or export the image. Pre-authenticated requests provide a way to let users access a bucket or an object without having their own credentials. For detailed steps explaining how to create and use pre-authenticated requests, see Using Pre-Authenticated Requests and Pre-Authenticated Requests.

Direct link to this issue: Invalid bucketName error when importing or exporting a custom image

Unable to create instance from boot volume backup

Details: When you try to create an instance from a boot volume backup in the Console, an error similar to the following might occur:

There was an error loading the source image for creating an instance. You might not have permission to access this image, or it might be in a different region. If the image is in a different region, you should still be able to launch your instance.

This error can occur when the compartment that contained the deleted image metadata used for the boot volume backup has also been deleted.

Workaround: If the compartment has been deleted, use the CLI to create the instance. For information about using the CLI, see Command Line Interface (CLI).

To create an instance from a boot volume using the CLI, open a command prompt and run the launch command. To launch an instance using an image or a boot volume, include the --source-details parameter.

oci compute instance launch --availability-domain <AVAILABILITY_DOMAIN> --compartment-id, -c <COMPARTMENT_OCID> --shape <SHAPE> --subnet-id <SUBNET_ID> --source-details <file://path/to/file>

Direct link to this issue: Unable to create instance from boot volume backup

Unable to remove instance from capacity reservation using Terraform

Details: It is not possible to remove an instance from a capacity reservation using Terraform.

Workaround: We are aware of the issue and working on a resolution. As a workaround, use the Console, CLI, or SDK to remove the instance from the capacity reservation.

Direct link to this issue: Unable to remove instance from capacity reservation using Terraform

Creating more than 50 capacity configurations results in an internal error

Details: When you create more than 50 capacity configurations in a capacity reservation, an internal error occurs. After the error occurs, it's not possible to launch instances against the capacity reservation.

Workaround: We are aware of the issue and working on a resolution. To avoid this issue, do not add more than 50 capacity configurations to your capacity reservation.

Direct link to this issue: Creating more than 50 capacity configurations results in an internal error

Capacity reservation service limits inaccurate

Details: The <shape>-core-reserved-count service limit numbers are inaccurate. The number in the Service Limit column might show 1,000,000,000 or N/A. The number in the Available column might show 1,000,000,000 less the number in the Usage column or N/A. The 1,000,000,000 value represents a maximum value and might vary.

Workaround: We are aware of the issue and working on a resolution. For accurate service limits, see Compute Capacity Reservations.

Direct link to this issue: Capacity reservation service limits inaccurate

No service category for capacity reservations when requesting service limit increases

Details: When you request a service limit increase, the Service Category menu does not include a category for capacity reservations.

Workaround: In the Request Service Limit Updates form:

  • For Service Category, select Others.
  • For Resource, select Other Limits.
  • In the Reason for Request field, enter the specific limit to be increased.

Direct link to this issue: No service category for capacity reservations when requesting service limit increases

Windows Server fails on VM.Standard.E3.Flex instances larger than 32 OCPUs

Details: When you create a Windows Server instance using the VM.Standard.E3.Flex shape, if you allocate more than 32 OCPUs to the instance, the instance fails to launch.

If you resize an existing Windows Server instance on a VM.Standard.E3.Flex shape to more than 32 OCPUs, the instance experiences a stop error ("blue screen").

Workaround: We are aware of the issue and working on a resolution. To work around this issue, resize the instance so that it has 32 OCPUs or less.

Direct link to this issue: Windows Server fails on VM.Standard.E3.Flex instances larger than 32 OCPUs

Instance pool creation fails when resources include default tags

Details: When you try to create an instance pool, the instance pool creation fails with the error "Authorization failed or requested resource not found". This happens because resources used by the instance pool contain default tags, and the user does not have permission to the tag namespace.

Workaround: To work around this issue, add a policy statement granting the instance pool user group permission to the tag namespace Oracle-Tags:

Allow group InstancePoolUsers to use tag-namespaces in tenancy where target.tag-namespace.name = 'oracle-tags'

For more information about policies, see Let users manage Compute instance configurations, instance pools, and cluster networks. For more information about default tags, see Understanding Automatic Tag Defaults.

Direct link to this issue: Instance pool creation fails when resources include default tags

Out of host capacity error when creating compute instances

Details: When you try to create an instance, the instance launch fails with the error code InternalError and a message similar to Out of host capacity. This happens because of a lack of physical infrastructure capacity for the shape in the requested fault domain and availability domain.

Workaround: Capacity usually becomes available soon for most shapes. To work around this issue, do the following things:

  • If you’re using a previous generation shape, create the instance using a current generation shape instead. Capacity is limited for previous generation shapes.
  • Create the instance in a different availability domain.
  • Create the instance without specifying a fault domain.
  • Create the instance using a smaller shape, or using a shape in a different series.
  • Wait a few minutes and try again.

Direct link to this issue: Out of host capacity error when creating compute instances

In-transit encryption for a boot volume attachment can be edited when unsupported by the image

Details: When the in-transit encryption value for an image is null, the in-transit encryption value for an instance that's created from the image can be set to a non-null value.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: In-transit encryption for a boot volume attachment can be edited when unsupported by the image

Oracle Cloud Agent plugins are not available on domain controllers

Details: When you use a Windows Server instance as a domain controller, features that depend on Oracle Cloud Agent, such as the Monitoring service and the OS Management service, are not available. This happens because the services installed by Oracle Cloud Agent on Windows run with virtual accounts, but virtual accounts are not supported in the domain controller scope.

Workaround: Disable the Oracle Cloud Agent updater by running the following PowerShell command as an administrator:
net stop OCAU

(To update Oracle Cloud Agent, you can manually download and install the latest version.)

For each Oracle Cloud Agent feature that you want to use, use services.msc to update the user running for the applicable NT service to use a domain service or user account. Then, add the user to the applicable domain local group, as shown in the following table:

Oracle Cloud Agent Feature NT Service User Target Account Type Target Domain Local Group
Oracle Cloud Agent NT service (including the Compute Instance Monitoring plugin) NT Service\OCA Domain service account or user account Performance Monitoring Groups
Compute Instance Run Command plugin NT Service\OCARUN Domain service account or a domain user account that has local administrative privileges Administrators Group
Custom Logs Monitoring plugin NT Service\OCAUM Domain service account or a domain user account that has local administrative privileges Administrators Group
OS Management Service Agent plugin NT Service\OCAOSMS Domain service account or a domain user account that has local administrative privileges Administrators Group
Oracle Cloud Agent updater NT Service\OCAU Domain service account or a domain user account that has local administrative privileges Administrators Group

Direct link to this issue: Oracle Cloud Agent plugins are not available on domain controllers

Boot volume backup size larger than expected

Details: Due to a recent change in how the Compute service handles images, when you create a boot volume backup, the backup is larger than expected. In some cases the boot volume backup may be larger than the boot volume size.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Boot volume backup size larger than expected

Intermittent issues with SSH access, DNS lookups, and access to the metadata service

Details:

You may experience intermittent errors with any of the following for your compute instance:

  • Connecting to the instance using SSH.

  • Performing a DNS lookup

  • Accessing the metadata service at http://169.254.169.254/*.

Workaround: We are aware of the issue and working on a resolution.

To temporarily work around this issue, run the following command on the instance:

sudo ethtool -G ens3 tx 513 && sudo ethtool -G ens3 tx 512

Direct link to this issue: Intermittent issues with SSH access, DNS lookups, and access to the metadata service

iSCSI-attached volumes do not connect on reboot

Details: If you performed a yum update on your instance using the Oracle Linux 7 yum repos between March 22, 2019 and April 9, 2019, you may encounter an issue where iSCSI-attached block volumes are not available after you reboot the instance.

Workaround: This occurs when the instance is not configured to automatically login to iSCSI nodes on reboot. To configure automatic login, update the version of the iscsi-initiator-utils package by running the following command:

sudo yum update -y iscsi-initiator-utils-6.2.0.874-10.0.7.el7

Direct link to this issue: iSCSI-volumes do not connect on reboot

iscsid service should be configured to restart automatically

Details: Oracle Cloud Infrastructure supports iSCSI attached remote boot and block volumes to compute instances. These iSCSI attached volumes are managed by the iscsid service. In scenarios where this service is stopped for any reason, such as the service crashes or a system administrator inadvertently stops the service, it's important that the iscsid service is automatically restarted to increase the stability of your infrastructure.

Workaround: See Updating the Linux iSCSI Service to Restart Automatically for steps on how to configure the iscsid service to restart automatically.

Direct link to this issue: iscsid service should be configured to restart automatically

Virtual machine (VM) instances launch with an iSCSI attached boot volume when you specify a value for the ipxeScript attribute

Details: When you specify a value for the ipxeScript attribute of the Instance for a virtual machine (VM) instance, the instance will launch with an iSCSI attachment for the boot volume instead of a paravirtualized attachment.

Direct link to this issue: Virtual machine (VM) instances launch with an iSCSI attached boot volume when you specify a value for the ipxeScript attribute

Instances experience system hang after running firewall-cmd --reload

Details: A compute instance may experience a system hang after you run the following command to reload the firewall:

firewall-cmd --reload

Reloading the firewall using this command on a running instance may cause the instance’s boot volume to lose its iSCSI connection and crash, based on the order in which firewall rules are reloaded.

Workaround: To prevent this from happening, do not use the reload parameter for firewall-cmd. Instead, run the firewall-cmd command twice, using the permanent parameter the first time you call it to ensure you do not lose iSCSI connectivity.

For example:

firewall-cmd --permanent
firewall-cmd

Direct link to this issue: Instances experience system hang after running firewall-cmd --reload

Network icon on Windows 2016 instances displays incorrect status

Details: On instances running Windows 2016, a red "x" is displayed on the network connection icon in the taskbar even though there is no issue with the instance's network connectivity.

Workaround: We are aware of the issue and working on a resolution. If you recycle the explorer.exe process the icon will display the correct status. However, this is not a permanent fix; the red "x" will reappear when you reboot the instance.

Direct link to this issue: Network icon on Windows 2016 instances displays incorrect status

Instances running October 2018 release of Ubuntu 18.04 experience system hang

Details: iSCSId is disabled by default in the October 2018 release of the Ubuntu 18.04 platform image, so instances using this operating system may experience a system hang if there is a momentary break in the iSCSI communication.

Workaround: To work around this issue, run the following command to enable iSCSId on the instance:

sudo systemctl enable iscsid && sudo systemctl start iscsid

Direct link to this issue: Instances running October 2018 release of Ubuntu 18.04 experience system hang

kmsKeyId attribute is null

Details: When calling the GetInstance or ListInstances operations, the kmsKeyId attribute of InstanceSourceViaImageDetails is null.

Workaround: To work around this issue, call the GetBootVolume operation and retrieve the value from the kmsKeyId attribute of BootVolume.

Direct link to this issue: kmsKeyId attribute is null

Ubuntu instance fails to reboot after enabling Uncomplicated Firewall (UFW)

Details: After you enable UFW on a compute instance running Ubuntu, the instance fails to reboot successfully.

Workaround: Do not use UFW to edit firewall rules. Platform images are preconfigured with firewall rules to enable instances to make outgoing connections to the instance's boot and block volumes. For more information, see Essential Firewall Rules. UFW may remove these rules so that during a reboot the instance is not able to connect to the boot and block volumes.

To modify or add new firewall rules, update the /etc/iptables/rules.v4 file instead. Modifications to firewall rules here will take effect after a reboot. To have the rules take effect immediately, run the following:

$ sudo su -
# iptables-restore < /etc/iptables/rules.v4

Direct link to this issue: Instance fails to reboot after enabling UFW

Unable to log in to instance launched from new generalized Windows custom image

Details: You are unable to log in to an instance launched from a newly created custom Windows image.

Workaround: This is the result of the image generalization process failing due a problem with Sysprep after upgrading to WMF 5.0. To work around this issue, perform the steps described in Sysprep fails after WMF 5.0 installation.

Direct link to this issue: Unable to log into instance launched from new Windows custom image

Custom image created from Windows instance may cause Windows to boot into safe mode

Details: After creating a Windows custom image, the initial instance or instances launched from the image may boot into safe mode or recovery mode. Instances booted into either mode will not respond to RDP. This can occur when the instance is not able to fully shut down prior to the custom image being taken. You can still access the instance by connecting to the VNC console, using the steps described in Connecting to the VNC Console.

Workaround: To work around this issue, prior to creating the custom image, connect and log in to the instance using RDP and initiate the shutdown from there.

Direct link to this issue: Custom image created from Windows instance may cause Windows to boot into safe mode

Instances launched from Ubuntu 16 custom images require custom network configuration

Details: When importing Ubuntu 16 LTS and newer releases of Ubuntu, DHCP fails to get the gateway configuration, and thus fails to set up a default route to the gateway on the VNIC.

Workaround: We are aware of the issue and working on a resolution. To work around this issue, statically configure the default route after import. To do this:

  1. Create the following script:

    #! /bin/bash -e
    								ROUTER_IP=$(/usr/bin/curl --silent http://169.254.169.254/opc/v1/vnics/ | grep "virtualRouterIp" | grep -oP "\d+\.\d+\.\d+\.\d+" | head -n 1)
    								echo "Found Router IP $ROUTER_IP"
    
    							ip route add default via $ROUTER_IP

    and save it to: /usr/local/bin/configure_default_route.sh

  2. Run the following command to make the script executable:

    sudo chmod +x /usr/local/bin/configure_default_route.sh
  3. Add the following to /etc/network/interfaces so that it is launched each time the system boots up:

    # OCI Emulated boot network interface
    								auto ens3
    								iface ens3 inet dhcp
    							post-up /usr/local/bin/configure_default_route.sh

Direct link to this issue: Instances launched from Ubuntu 16 custom images require custom network configuration

Secondary VNIC detachment times out for some instances launched from imported custom images

Details: When you detach a secondary VNIC from instances launched from imported custom images, the operation may time out.

Workaround: The hot plug module, acpiphp, needs to be loaded for secondary VNICs to detach correctly in Linux. If a VNIC fails to detach, run the lsmod command to display the list of loaded modules, and check the list for acpiphp. If you don't see it in the list, load the module by running the following command:

modprobe acpiphp

Retry the detachment operation for the secondary VNIC. You might need to reboot the system for the operation to complete successfully.

Direct link to this issue: Secondary VNIC detachment times out for some instances launched from imported custom images

Secondary VNIC may be non-functional for older CentOS, Oracle Linux, and RHEL images

Details: The secondary VNIC feature is not supported for the following operating systems due to a bug in the kernel:

  • CentOS 4, 5

  • Oracle Linux 4, 5

  • RHEL 4, 5

Secondary VNICs will fail to work after a reboot.

Direct link to this issue: Secondary VNIC may be non-functional for older CentOS, Oracle Linux, and RHEL images

Invalid image error when exporting an image

Details: When you try to export an image, the export fails with an error indicating that the image is invalid. This error only occurs in the US West (Phoenix) region.

Workaround: We are aware of the issue and working on a resolution. To work around this issue:

  1. Launch a new instance based on the image you're trying to export, and specify one of the following shapes for the image:

    • BM.Standard1.36

    • BM.DenseIO1.36

    • VM.DenseIO1.4

    • VM.DenseIO1.8

    • VM.DenseIO1.16

  2. Create a custom image using the steps described in To create a custom image.

After you have created the custom image, you can export this new image.

Direct link to this issue: Invalid image error when exporting an image

Authentication error occurs when connecting to the serial console for a bare metal instance

Details: When establishing an SSH connection to a bare metal instance, your SSH client must send the correct key the first time. If you have more than one SSH key configured under ~/.ssh or in your ~/.ssh/config file, your client may not send the correct key on the first authorization attempt, and you may encounter the following error message:

Received disconnect from UNKNOWN port 65535:2: Too many authentication failures.

Workaround: We are aware of the issue and working on a resolution. To work around this issue, modify the connection string in the SSH command to use the configfile flag, -F to override the default configuration file, the -o IdentitiesOnly=yes option to force the SSH client to use the specified key, and the identity file flag, -i to specify the SSH key to use, as shown in the following example:

ssh -F /dev/null -o IdentitiesOnly=yes -i /<path>/<ssh_key> -o ProxyCommand='ssh -i /<path>/<ssh_key> -W %h:%p -p 443...

Direct link to this issue: Authentication error occurs when connecting to the serial console for a bare metal instance

Incorrect system time on Windows VM instances when you change the default time zone

Details: If you change the time zone from the default setting on Windows VM instances, when the instance reboots or syncs with the hardware clock, the system time will revert back to the time for the default time zone. However, the time zone setting will stay set to the new time zone, so the system clock will be incorrect.

You will also see events in the event log indicating that the system time was changed with the following details:

Change Reason: System time synchronized with the hardware clock.

Workaround: We are aware of the issue and working on a resolution. To work around this issue:

  1. Open Registry Editor and navigate to:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation
  2. Create a new DWORD key named RealTimeIsUniversal and set the value to 1.

  3. Reboot the instance.
  4. Reset the time and time zone manually.

Direct link to this issue: Incorrect system time on Windows VM instances when you change the default time zone

Serial console connections do not work for older instances

VM instance details: You can only create serial console connections to virtual machine (VM) instances launched on August 26, 2017 or later.

Bare metal instance details: You can only create serial console connections to bare metal instances launched on October 21, 2017 or later.

Workaround: If you need serial console access to an instance launched prior to the dates specified for VM and bare metal instances, you can work around this issue by creating a custom image of the instance. When you launch a new instance based on the custom image, the new instance will have serial console access. For details on creating a custom image, see Managing Custom Images.

Direct link to this issue: Serial console connections do not work for older instances

Inactive listImage parameters and missing Image response fields

Details: The Compute API ListImages operation includes parameters for server-side filtering on operatingSystem and operatingSystemVersion. However, these parameters are currently inactive. Also, the Image response object documentation includes the operatingSystem and operatingSystemVersion attributes, but the object currently does not return these fields.

Workaround: The display name for platform images includes the operating system and operating system version, for example "Oracle-Linux-7.2-2016.09.18-0". "Oracle Linux" is the operating system and the version is "7.2".

We are aware of the omission and plan to support these parameters and attributes.

Direct link to this issue: Inactive listImage parameters and missing Image response fields

Instance reboot fails if the Network Manager service is installed

Details: If the Network Manager service is installed, an instance can fail to reboot.

Workaround: If the Network Manager service is not required, you can uninstall it. If the Network Manager service is required, modify the network interface configuration file before you reboot the instance. Set the NM_CONTROLLED configuration key to "no":

NM_CONTROLLED="no"

Usually, the network interface configuration file is located in:

/etc/sysconfig/network-scripts/ifcfg-<interface_name>

Direct link to this issue: Instance reboot fails if the Network Manager service is installed

Non-ASCII characters in the instance name can cause Windows launch failures

Details: When the name of a Windows instance includes non-ASCII characters, the instance might fail to launch. This happens because the instance name is used to set the Windows computer name during instance creation. Windows restricts the characters that are allowed in computer names, and non-ASCII characters can cause Windows instance creation failures.

Workaround: We are aware of the issue and working on a resolution. To temporarily work around this issue, name Windows instances using only these ASCII characters: uppercase letters (A-Z), lowercase letters (a-z), numbers (0-9), and hyphens (-).

Direct link to this issue: Non-ASCII characters in the instance name can cause Windows launch failures

Automatic updates using Oracle Ksplice fail with some FastConnect networking setups

Details: Some FastConnect networking setups prevent automatic patch updates for utilities such as Oracle Ksplice.

Workaround: In the /etc/uptrack/uptrack.conf file, replace all instances of:
oraclecloud-updates-ksplice.oracle.com
with:
updates.ksplice.<region>.oci.oraclecloud.com 
For example, if your home region is US West (Phoenix), replace:
oraclecloud-updates-ksplice.oracle.com
with:
updates.ksplice.us-phoenix-1.oci.oracle.com

This workaround applies to service gateways. It does not apply to private endpoints.

Direct link to this issue: Automatic updates using Oracle Ksplice fail with some FastConnect networking setups

Missing flag is required for the OS Management service for instances created before September 2019

Details: When using the OS Management service on Oracle Linux instances that were created before September 2019, the Instance Details page might incorrectly indicate that the OS Management service is enabled (Oracle Cloud Management Agent: Enabled) when the service is not enabled.

This issue affects instances that were created before the isManagementDisabled flag was defined in the metadata for compute instances. Because this flag is not present, the metadata for these instances is not set properly for the OS Management service.

Workaround: To resolve this issue, set the isManagementDisabled flag to false:

  1. In the agent configuration for the instance, set the isManagementDisabled option to false:

    oci compute instance update --instance-id <instance_OCID> --agent-config '{"isManagementDisabled": false, "isMonitoringDisabled": false}'
  2. Use the CLI to verify that the flag has been updated:

    oci compute instance get --instance-id <instance_OCID>

    In the output, the updated flag appears as "is-management-disabled": false.

    {
      "data":
        "agent-config": {
          "is-management-disabled": false,
          "is-monitoring-disabled": false
        },
    ...
    }
  3. Connect to the instance using SSH, and then use cURL to call the instance metadata service and verify that the flag has been updated within the compute instance:

    curl http://169.254.169.254/opc/v1/instance/

    In the output, the updated flag appears as "managementDisabled" : false.

    {
      ...
      "agentConfig" : {
        "monitoringDisabled" : false,
        "managementDisabled" : false
      }
    }

Direct link to this issue: Missing flag is required for the OS Management service for instances created before September 2019

RESOLVED: BIOS settings for BM.GPU4.8 shape are not applied when configured using the Console

Details: When you use the Console to configure advanced BIOS settings for a Compute instance that uses the BM.GPU4.8 shape, the settings are not applied. Instead, the instance uses the default BIOS settings for the shape.

Workaround: This issue is now resolved.

Direct link to this issue: RESOLVED: BIOS settings for BM.GPU4.8 shape are not applied when configured using the Console

RESOLVED: Unable to create instances with VM.Standard3.Flex and BM.Standard3.64 shapes

Details: When you try to create instances with VM.Standard3.Flex and BM.Standard3.64 shapes, the instance creation might fail, or the available network bandwidth might not match the documented network bandwidth.

Workaround: This issue is now resolved.

Direct link to this issue: RESOLVED: Unable to create instances with VM.Standard3.Flex and BM.Standard3.64 shapes

RESOLVED: Incorrect storage size is displayed for some shapes

Details: For the following list of compute shapes, an incorrect value for the size of the NVMe drives is displayed in the Console and returned by the ListShapes API operation.

  • BM.DenseIO1.36
  • BM.DenseIO2.52
  • BM.GPU4.8
  • BM.HighIO1.36
  • BM.HPC2.36
  • VM.DenseIO1.4
  • VM.DenseIO2.8

Workaround: This issue is now resolved.

Direct link to this issue: RESOLVED: Incorrect storage size is displayed for some shapes

RESOLVED: Marketplace HPC image does not support Optimized3 shapes on RDMA cluster networks

Details: The Marketplace HPC image was built for BM.HPC2 instance shapes and is not compatible with the new BM.Optimized3 shapes.

Workaround: This issue is now resolved.

Direct link to this issue: RESOLVED: Marketplace HPC image does not support Optimized3 shapes on RDMA cluster networks

RESOLVED: Oracle Autonomous Linux images cannot be managed by the OS Management service

Details: Instances that use the Oracle Autonomous Linux image cannot be managed by the OS Management service.

Workaround: This issue is resolved beginning with the August 2021 Oracle Autonomous Linux platform image.

Direct link to this issue: Oracle Autonomous Linux images cannot be managed by the OS Management service

RESOLVED: VNC console connections on Ampere A1 Compute instances are read-only

Details: When you create a VNC console connection to an instance that uses the VM.Standard.A1.Flex shape or the BM.Standard.A1.160 shape, the console connection is read-only.

Workaround: This issue is now resolved.

Direct link to this issue: RESOLVED: VNC console connections on Ampere A1 Compute instances are read-only

Console

Bug in the Firefox browser can cause the Console not to load

Details: When you try to access the Console using Firefox, the Console page never loads in the browser. This problem is likely caused by a corrupted Firefox user profile.

Workaround: Create a new Firefox user profile as follows:

  1. Ensure that you are on the latest version of Firefox. If not, update to the latest version.
  2. Create a new user profile and remove your old user profile. See Mozilla Support for instructions to create and remove user profiles: https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles.
  3. Open Firefox with the new profile.

Alternatively, you can use one of the other Supported Browsers.

Direct link to this issue: Bug in the Firefox browser can cause the Console not to load

Container Engine for Kubernetes

Worker node properties out-of-sync with updated node pool properties

Details: The properties of new worker nodes starting in a node pool do not reflect the latest changes to the node pool's properties. The likely cause is use of the deprecated quantityPerSubnet and subnetIds attributes when using the UpdateNodePoolDetails API operation to update node pool properties.

Workarounds: Do one of the following:
  • Start using the nodeConfigDetails attribute when using the UpdateNodePoolDetails API operation. First, scale the node pool to 0 using quantityPerSubnet. Then stop using the subnetIds and quantityPerSubnet attributes, and use the nodeConfigDetails attribute instead.
  • Contact Oracle Support to restart the back-end component responsible for synchronization (the tenant-agent component).

Direct link to this issue: Worker node properties out-of-sync with updated node pool properties

Unable to launch Kubernetes Dashboard

Details: When you launch the Kubernetes Dashboard, in some situations you might encounter "net/http: TLS handshake timeout" and "connection reset by peer" error messages in your web browser. This issue has only been observed in newly created clusters running Kubernetes version 1.11. For details about a related Kubernetes issue, see https://github.com/kubernetes/dashboard/issues/3038.

Workaround:

  1. In a terminal window, enter:

    $ kubectl -n kube-system port-forward svc/kubernetes-dashboard 8443:443
  2. In your web browser, go to https://localhost:8443

Direct link to this issue: Unable to launch Kubernetes Dashboard

Unable to access in-cluster Helm

Details: When you use a Kubeconfig token version 2.0.0 to access Helm/Tiller versions prior to version 2.11, you will receive one of the following errors:

  • Error: Unauthorized
  • Error: could not get Kubernetes client: exec plugin: invalid apiVersion "client.authentication.k8s.io/v1beta1"

Workaround: Upgrade Helm/Tiller as follows:

  1. In a terminal window, download a Kubeconfig token version 1.0.0 by entering the following command:

    $ oci ce cluster create-kubeconfig --token-version=1.0.0 --cluster-id=<cluster_ocid>
  2. Identify the region key to use to specify the Oracle Cloud Infrastructure Registry registry in the cluster's region (see Availability by Region). For example, if the cluster is in US East (Ashburn), iad is the region key to use to specify the registry in that region.

  3. Upgrade Tiller by entering the following command:

    $ helm init --upgrade -i <region-key>.ocir.io/odx-oke/oke-public/tiller:v2.14.3

    where <region-key> is the key that you identified in the previous step.

  4. In a browser, navigate to https://helm.sh/docs/using_helm/#installing-the-helm-client and follow the instructions to download and install the Helm client binary.

  5. Having upgraded Helm/Tiller, download a Kubeconfig token version 2.0.0 by entering the following command:

    $ oci ce cluster create-kubeconfig --token-version=2.0.0 --cluster-id=<cluster_ocid>

Direct link to this issue: Unable to access in-cluster Helm

Some Kubernetes features (for example, the Metrics Server) cannot communicate with the kubelet via http/2

Details: The Container Engine for Kubernetes 1.8.0 release included a security improvement to improve cipher strength on the kubelet running on customer worker nodes. New worker nodes created between August 20, 2019 and September 16, 2019 include this configuration. The new set of ciphers does not allow connections to the kubelet via http/2. This restriction impacts the Metric Server, and also the Horizontal Pod Autoscaler which depends on the Metrics Server.

Workaround:

For each existing worker node in turn:

  1. Prevent new pods from starting and delete existing pods on the worker node by entering kubectl drain <node_name>. For more information:

    Recommended: Leverage pod disruption budgets as appropriate for your application to ensure that there's a sufficient number of replica pods running throughout the drain operation.

  2. Delete the worker node (for example, by terminating it in the Console).
  3. Wait for a replacement worker node to start.

The replacement worker nodes include include new settings to enable communication with the kubelet.

Direct link to this issue: Some Kubernetes features (for example, the Metrics Server) cannot communicate with the kubelet via http/2

Kubernetes pods fail to mount volumes due to timeouts

Details: When a new pod starts on a worker node in a cluster, in some situations the pod fails to mount all volumes attached to the node due to timeouts and you see a message similar to the following:

Unable to mount volumes for pod "<pod_name>(<pod_uid>)": timeout expired waiting for volumes to attach or mount for pod "<namespace>"/"<pod_name>". list of unmounted volumes=[<failed_volume>]. list of unattached volumes=[<… list of volumes >]

One possible cause identified for this issue is if the pod spec includes an fsGroup field in the securityContext field. If the container is running on a worker node as a non-root user, setting the fsGroup field in the securityContext can cause timeouts due to the number of files to which Kubernetes must make ownership changes (see https://github.com/kubernetes/kubernetes/issues/67014).

If the pod spec does not include an fsgroup field in the securityContext, the cause is unknown.

Workarounds:

If the pod spec includes the fsgroup field in the securityContext and the container is running a non-root user, consider the following workarounds:

  • Remove the fsgroup field from the securityContext.
  • Use the supplementalGroups field in the securityContext (instead of fsgroup), and set supplementalGroups to the volume identifier.
  • Change the pod spec so that the container runs as root.

If the pod spec does not include the fsgroup field in the securityContext, or if the container is already running as root, you have to restart or replace the worker node. For example, by stopping and starting the instance, by rebooting the instance, or by terminating the instance so that a new instance is started. Follow the instructions in Stopping and Starting an Instance or Terminating an Instance as appropriate to use the Console or the API. Alternatively you can use CLI commands, such as the following example to terminate an instance:

$ INSTANCE_OCID=$(kubectl get node <name> -ojsonpath='{.spec.providerID}')
$ oci compute instance terminate --instance-id $INSTANCE_OCID

where <name> is the worker node name, derived from the Private IP Address property of the instance (for example, 10.0.10.5).

Direct link to this issue: Kubernetes pods fail to mount volumes due to timeouts

New labels introduced in Kubernetes version 1.17 are not added to nodes

Details: A number of beta.kubernetes.io node labels are deprecated in Kubernetes version 1.17 in favor of their GA equivalents. Specifically:

Deprecated label Equivalent label in Kubernetes version 1.17
beta.kubernetes.io/instance-type node.kubernetes.io/instance-type
failure-domain.beta.kubernetes.io/region topology.kubernetes.io/region
failure-domain.beta.kubernetes.io/zone topology.kubernetes.io/zone

However, Container Engine for Kubernetes continues to add the deprecated labels to worker nodes, rather than the GA equivalent labels.

Workaround:

We are aware of the issue and working on a resolution.

Direct link to this issue: New labels introduced in Kubernetes version 1.17 are not added to nodes

OS Management causes Kubernetes cluster node pools to fail

Details: When using the OS Management service to manage operating system updates and patches on Oracle Cloud Infrastructure instances, there are some situations in which cluster node pools created by Container Engine for Kubernetes fail to come online.

Workaround:

There are two possible workarounds:

  • Workaround 1: If you want to use OS Management to manage Oracle Cloud Infrastructure instances, enable Oracle Enterprise Linux in OS Management. See Managing Software Sources.
  • Workaround 2: If you don't want to use OS Management to manage Oracle Cloud Infrastructure instances, make sure there are no policies that allow OS Management to run. Specifically, remove the policy that grants a dynamic group of instances access to the OS Management service. See Setting Up Policies for OS Management.

Direct link to this issue: OS Management causes Kubernetes cluster node pools to fail

Volume mount issues in node pools with master nodes running Kubernetes version 1.19 (or later) and worker nodes running Kubernetes version 1.18 (or earlier)

Details: If node pools have master nodes running Kubernetes version 1.19 (or later) and worker nodes running Kubernetes version 1.18 (or earlier), mounting block volumes attached to the cluster using the FlexVolume volume plugin might not work as expected. For example, you might see:

  • A FailedMount warning message in the events of a pod running on a worker node, even though the block volume has been attached successfully.
  • A Volume not attached according to node status for volume error message in the logs of the kubelet running on a worker node.

Workaround:

  1. If there isn't already a node pool in the cluster with worker nodes running Kubernetes version 1.19 (or later), add such a node pool now.
  2. Remove the affected worker node that is running Kubernetes version 1.18 (or earlier), as follows:
    1. Prevent new pods from starting and delete existing pods on the affected worker node by entering kubectl drain <node_name>. For more information:
    2. Delete the affected worker node (for example, by terminating it in the Console).

Direct link to this issue: Volume mount issues in node pools with master nodes running Kubernetes version 1.19 (or later) and worker nodes running Kubernetes version 1.18 (or earlier)

Issues resolving with DNS (nslookup, dig, or curl)
Details: If the Bridge Netfilter kernel module is not enabled, traffic communication with localhost doesn't route correctly. For example:
/ # nslookup www.oracle.com
;; reply from unexpected source: 10.244.0.167#53, expected 10.96.5.5#53
;; reply from unexpected source: 10.244.0.167#53, expected 10.96.5.5#53
;; reply from unexpected source: 10.244.0.167#53, expected 10.96.5.5#53
;; connection timed out; no servers could be reached 
To verify this issue, open a terminal window on the instance and run the following command:
sudo /usr/sbin/lsmod | grep br_netfilter 

If no results are returned, then the Bridge Netfilter kernel module is not enabled. The Bridge Netfilter kernel module is required to masquerade VxLAN traffic for Kubernetes pods.

Workaround: Enable the Bridge Netfilter kernel module. Open a terminal window on the instance and run the following commands:
sudo modprobe br_netfilter 
sudo sh -c 'echo "br_netfilter" > /etc/modules-load.d/br_netfilter.conf'

Direct link to this issue: Issues resolving with DNS (nslookup, dig, or curl)

Source client IP is not preserved for traffic through a LoadBalancer Service using externalTrafficPolicy: Local

Details: When using VCN-native pod networking, the source client IP address of inbound requests to a pod might not be preserved as expected. Instead, inbound requests received via a Kubernetes service of type LoadBalancer that has externalTrafficPolicy: Local set in the manifest file might be shown as originating from the worker node's IP address.

Workaround: For inbound TCP requests received via a Kubernetes service of type LoadBalancer that has the oci.oraclecloud.com/load-balancer-type: "lb" annotation in the manifest file, obtain the source client IP address from the X-Forwarded-For or X-Real-IP header.

Direct link to this issue: Source client IP is not preserved for traffic through a LoadBalancer Service using externalTrafficPolicy: Local

Pod network connectivity issues on bare metal instances

Details: When using VCN-native pod networking, pods might be unable to communicate over the network if you have specified a bare metal shape for worker nodes in one or more of the node pools in the cluster.

Workaround: Specify a VM shape for worker nodes in every node pool in the cluster when using VCN-native pod networking.

Direct link to this issue: Pod network connectivity issues on bare metal instances

Incorrect maximum pods per node limit for flexible shapes

Details: When using VCN-native pod networking, the maximum number of pods per worker node in a node pool might be limited to 31, regardless of the number of OCPUs you specify for the flexible shape you have selected for the node pool.

Workaround: If you want more than 31 pods per worker node in a node pool, select a different shape for worker nodes in the node pool.

Direct link to this issue: Incorrect maximum pods per node limit for flexible shapes

Pod network connectivity issues on VCNs with added CIDR blocks

Details: When using VCN-native pod networking, pods running on worker nodes connected to a pod subnet with a CIDR block outside the first CIDR block specified for the VCN might be unable to communicate with Kubernetes services.

Workaround: Create pod subnets with CIDR blocks within the first CIDR block specified for the VCN.

Direct link to this issue: Pod network connectivity issues on VCNs with added CIDR blocks

Data Catalog

For known issues with Data Catalog, see Known Issues.

Data Flow

For known issues with Data Flow, see Known Issues.

Files required for each application should be in the same region where the application is created.

Details

Applications must be created in the same region as the Object Store bucket that contains all related files, jars and configs required for a successful run of the application. Cross region scenario is not supported.

Workaround
We are aware of this issue. There is no workaround but to ensure that all files, jars, configuration, and so on are in the same region.

Spark UI errors

Details
You might encounter an error accessing the Spark UI. The typical cause for this error is that the Spark application is ending.
Workaround
If you encounter an error, wait for one minute before accessing the Spark UI again.

Data Integration

Success condition evaluation fails during running of a REST task

Details: In a REST task for a long-running API operation, when the success condition uses the ${PARAMETER_NAME} syntax, the evaluation of the success condition during running of the task fails.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Success condition evaluation fails during running of a REST task

SQL task in pipeline fails when an incoming parameter value uses the SYS.RESPONSE_PAYLOAD_JSON output

Details: In a pipeline, a downstream SQL task can use output parameters from an upstream REST task. However, when the incoming parameter value is the JSON output SYS.RESPONSE_PAYLOAD_JSON or SYS.RESPONSE_HEADERS_JSON, the downstream SQL task fails to run.

Workaround: We are aware of the issue and working on a resolution. Meanwhile, use the String output SYS.RESPONSE_PAYLOAD or SYS.RESPONSE_HEADERS to extract a scalar field from a JSON object.

Direct link to this issue: SQL task in pipeline fails when an incoming parameter value uses the SYS.RESPONSE_PAYLOAD_JSON output

Preview data from an Object Storage Excel source doesn't show file metadata

Details: In a data flow where the Object Storage source type is Excel, and the check box to Fetch file metadata as attributes is selected, file metadata attributes such as file_name and object_size are missing on the Attributes and Data tabs of the data flow.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Preview data from an Object Storage Excel source doesn't show file metadata

Preview data fails to load for a data entity in an Object Storage Parquet data asset

Details: When viewing the details of a Parquet data asset, after clicking Get attributes to retrieve the data entity attributes, the attributes display successfully in the Attributes table. The preview sampling data, however, does not display in the Data table. A connectivity error message similar to the following appears after the data fails to load.

DICOM_CONNECTIVITY_0144 - Exception in data-access manager while reading sample-dataset.

Workaround: We are aware of the issue and working on a resolution. Meanwhile, you can add the Parquet data asset to a source operator in a data flow, and use the Data tab of the data flow to preview the sampling data.

Direct link to this issue: Preview data fails to load for a data entity in an Object Storage Parquet data asset

Runtime exception with expressions that reference DATE data attributes

Details: In a previous release, Data Integration made a switch to treat Oracle DATE data as DATETIME in data flow expressions. If a source or target data attribute has the DATE type, a data type mismatch exception is raised during expression parsing.

Workaround: We are aware of the issue and working on a resolution. Meanwhile, edit the expressions and convert the DATE references to DATETIME.

Direct link to this issue: Runtime exception with expressions that reference DATE data attributes

Pipeline with a parameterized request body in a REST task is corrupted

Details: For a pipeline that is created in the May 2022 release: When you open the pipeline immediately after having created it successfully, the configured value for an incoming JSON request body parameter is missing.

For a pipeline that was created in a release before May 2022: When you open the pipeline, the connections between operators are lost, and the configured value for an incoming JSON request body parameter is missing.

Workaround: We are aware of the issue and working on a resolution.

Meanwhile, for a pipeline that is created in the May 2022 release, you can edit the REST task in the pipeline by reconfiguring the incoming JSON parameter value.

Direct link to this issue: Pipeline with a parameterized request body in a REST task is corrupted

Task execution fails when a source table has column names with special characters

Details: When a source table column has special characters in the name (such as 'Name#'), and there is an expression operator downstream that performs a transformation on any of the incoming table columns, task execution fails.

Workaround: We are aware of the issue and working on a resolution. Meanwhile, disable the pushdown option on the source operator:

On the data flow, select the operator.

Select the Advanced options tab in the Properties panel.

Clear the Allow pushdown check box.

Republish the integration task.

Direct link to this issue: Task execution fails when a source table has column names with special characters

Pipeline run fails because the JSON output from a REST task cannot be extracted and consumed downstream

Details: Currently, only scalar outputs from a REST task can be directly consumed by downstream tasks in a pipeline.

Workaround: We are aware of the issue and working on a resolution. Meanwhile, use the following workaround:

To use the JSON output from a REST task as input for a downstream input, insert an expression operator between the REST task and the downstream task.

Then use the REGEXP_SUBSTR function to extract the property's scalar value from the JSON response output.

For example, to extract the value of the 'base' property, write the expression:

REGEXP_SUBSTR(<REST_TASK_IDENTIFIER>.SYS.RESPONSE_PAYLOAD, '"base"\s*:\s*"([^"]+)')

Direct link to this issue: Pipeline run fails because the JSON output from a REST task cannot be extracted and consumed downstream

Data Labeling

Data Science

Currently, there are no known issues with the Data Science service.

Database

Existing PDBs in a new database

Details: Existing PDBs do not appear in a newly created database and it may take up to a few hours before they appear in the Console. This includes the default PDB for a new database and existing PDBs for cloned or restored databases. In case of an in-place restore to an older version, the PDB list is updated similarly and may have some delay.

Workaround: None

Direct link to this issue: Existing PDBs in a new database

PDB in existing Data Guard configuration

Details: Creating and cloning a PDB in the primary database is not allowed via console or the API.

Workaround: You can use sqlplus to create or clone PDBs in the Primary database and they will be synced later in OCI console.

Direct link to this issue: PDB in existing Data Guard configuration

Migrating file-based TDE wallet to customer-managed key-based TDE wallet on Oracle Database 12c R1

Details: Using the Database Service API to migrate a file-based TDE wallet to a customer-managed key-based TDE wallet on Oracle Database 12c release 1 (12.1.0.2) fails with the following error:

[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation

Workaround: Use the DBAASCLI utility with the --skip_patch_check true flag to skip the validation of the patch for bug 30128047. Ensure that you have applied the patch for bug 31527103 in the Oracle home and then run the following dbaascli command:
nohup /var/opt/oracle/dbaascli/dbaascli tde file_to_hsm --dbname <database_name> --kms_key_ocid <kms_key_ocid> --skip_patch_check true &

In the preceding command, <kms_key_ocid> refers to the OCID of the customer-managed key you are using.

Migrating customer-managed key-based TDE wallet to file-based TDE wallet on Oracle Database 12c R1

Details: Using the Database Service API to migrate a customer-managed key-based TDE wallet to a file-based TDE wallet on Oracle Database 12c release 1 (12.1.0.2) fails with the following error:

[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation

Workaround: Use the DBAASCLI utility with the --skip_patch_check true flag to skip the validation of the patch for bug 30128047. Ensure that you have applied the patch for bug 29667994 in the Oracle home and then run following dbaascli command:
nohup /var/opt/oracle/dbaascli/dbaascli tde hsm_to_file --dbname <database_name> --skip_patch_check true &
Migrating file-based TDE wallet to customer-managed key-based TDE wallet on Oracle Database 12c R2

Details: Using the Database Service API to migrate a file-based TDE wallet to customer-managed key-based TDE wallet on Oracle Database 12c release 2 (12.2.0.1) fails with the following error:

[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation

Workaround: Migrate a file-based TDE wallet to a customer-managed key-based TDE wallet, as follows:

  1. Determine whether the database has encrypted UNDO or TEMP tablespaces in any of the Autonomous Databases or in CDB$ROOT, as follows:
    Run the following query from CDB$ROOT, to list all encrypted tablespaces contained within all Autonomous Databases:
    SQL> select tstab.con_id, tstab.name from v$tablespace tstab, v$encrypted_tablespaces enctstab where tstab.ts# = enctstab.ts# and encryptedts = 'YES';

    In then NAME column of the result of the query, search for the names of UNDO and TEMP tablespaces. If there are encrypted UNDO or TEMP tablespaces, then proceed to the next step.

  2. Unencrypt UNDO or TEMP tablespaces, as follows:

    If an UNDO tablespace is encrypted

    Unencrypt existing UNDO tablespaces, as follows:
    SQL> alter tablespace <undo_tablespace_name> encryption online decrypt;

    Repeat this procedure for all encrypted UNDO tablespaces.

    If a TEMP tablespace is encrypted
    1. Check the default TEMP tablespace, as follows:
      SQL> select property_value from database_properties where property_name = 'DEFAULT_TEMP_TABLESPACE';
      If the default TEMP tablespace is not encrypted but other TEMP tablespaces are encrypted, then drop the other TEMP tablespaces, as follows:
      SQL> drop tablespace <temp_tablespace_name>;

      Skip the remainder of the steps in this procedure.

      If the default TEMP tablespace is encrypted, then proceed with the remaining steps to create and set an unencrypted default TEMP tablespace.

    2. Set the encrypt_new_tablespaces parameter to DDL, as follows:
      SQL> alter system set "encrypt_new_tablespaces" = DDL scope = memory;
    3. Create a TEMP tablespace with the specifications of the current TEMP tablespace, as follows:
      SQL> create temporary tablespace <temp_tablespace_name> TEMPFILE size 5000M;
    4. Set the new TEMP tablespace as the default TEMP tablespace for the database, as follows:
      SQL> alter database default temporary tablespace <temp_tablespace_name>;
    5. Drop existing TEMP tablespaces, as follows:
      SQL> drop tablespace <temp_tablespace_name>;

    Repeat this procedure for all encrypted TEMP tablespaces.

    The database is now running with default UNDO and TEMP tablespaces that are not encrypted and any older TEMP and UNDO tablespaces are also decrypted.

    Set encrypt_new_tablespaces to its original value, as follows:
    SQL> alter system set "encrypt_new_tablespaces" = cloud_only;

    Proceed with keystore migration to customer-managed keys.

  3. Once you confirm that there are no UNDO or TEMP tablespaces encrypted in any of the pluggable databases or in CDB$ROOT, use the DBAASCLI utility with the --skip_patch_check true flag to skip the validation of the patch for bug 30128047. Ensure that you have applied the patch for bug 31527103 in the Oracle home and then run following dbaascli command:
    nohup /var/opt/oracle/dbaascli/dbaascli tde file_to_hsm --dbname <database_name> --kms_key_ocid <kms_key_ocid> --skip_patch_check true &

    In the preceding command, <kms_key_ocid> refers to the OCID of the customer-managed key you are using.

Migrating customer-managed key-based TDE wallet to file-based TDE wallet on Oracle Database 12c R2

Details: Using the Database Service API to migrate a customer-managed key-based TDE wallet to a file-based TDE wallet on Oracle Database 12c release 2 (12.2.0.1) fails with the following error:

[FATAL] [DBAAS-11014] - Required patches (30128047) are not present in the Oracle home <ORACLE_HOME>
ACTION: Apply the required patches (30128047) and re-try the operation

Workaround: Migrate a customer-managed key-based TDE wallet to a file-based TDE wallet, as follows:

  1. Determine whether the database has encrypted UNDO or TEMP tablespaces in any of the Autonomous Databases or in CDB$ROOT, as follows:
    Run the following query from CDB$ROOT, to list all encrypted tablespaces contained within all Autonomous Databases:
    SQL> select tstab.con_id, tstab.name from v$tablespace tstab, v$encrypted_tablespaces enctstab where tstab.ts# = enctstab.ts# and encryptedts = 'YES';

    In then NAME column of the result of the query, search for the names of UNDO and TEMP tablespaces. If there are encrypted UNDO or TEMP tablespaces, then proceed to the next step.

  2. Unencrypt UNDO or TEMP tablespaces, as follows:

    If an UNDO tablespace is encrypted

    Unencrypt existing UNDO tablespaces, as follows:
    SQL> alter tablespace <undo_tablespace_name> encryption online decrypt;

    Repeat this procedure for all encrypted UNDO tablespaces.

    If a TEMP tablespace is encrypted
    1. Check the default TEMP tablespace, as follows:
      SQL> select property_value from database_properties where property_name = 'DEFAULT_TEMP_TABLESPACE';
      If the default TEMP tablespace is not encrypted but other TEMP tablespaces are encrypted, then drop the other TEMP tablespaces, as follows:
      SQL> drop tablespace <temp_tablespace_name>;

      Skip the remainder of the steps in this procedure.

      If the default TEMP tablespace is encrypted, then proceed with the remaining steps to create and set an unencrypted default TEMP tablespace.

    2. Set the encrypt_new_tablespaces parameter to DDL, as follows:
      SQL> alter system set "encrypt_new_tablespaces" = DDL scope = memory;
    3. Create a TEMP tablespace with the specifications of the current TEMP tablespace, as follows:
      SQL> create temporary tablespace <temp_tablespace_name> TEMPFILE size 5000M;
    4. Set the new TEMP tablespace as the default TEMP tablespace for the database, as follows:
      SQL> alter database default temporary tablespace <temp_tablespace_name>;
    5. Drop existing TEMP tablespaces, as follows:
      SQL> drop tablespace <temp_tablespace_name>;

    Repeat this procedure for all encrypted TEMP tablespaces.

    The database is now running with default UNDO and TEMP tablespaces that are not encrypted and any older TEMP and UNDO tablespaces are also decrypted.

    Set encrypt_new_tablespaces to its original value, as follows:
    SQL> alter system set "encrypt_new_tablespaces" = cloud_only;

    Proceed with keystore migration to customer-managed keys.

  3. Once you confirm that there are no UNDO or TEMP tablespaces encrypted in any of the pluggable databases or in CDB$ROOT, use the DBAASCLI utility with the --skip_patch_check true flag to skip the validation of the patch for bug 30128047. Ensure that you have applied the patch for bug 29667994 in the Oracle home and then run following dbaascli command:
    nohup /var/opt/oracle/dbaascli/dbaascli tde file_to_hsm --dbname <database_name> --kms_key_ocid <kms_key_ocid> --skip_patch_check true &

    In the preceding command, <kms_key_ocid> refers to the OCID of the customer-managed key you are using.

Billing issue when changing license type

Details: When you change the license type of your Database or DB system from BYOL to license included, or the other way around, you are billed for both types of licenses for the first hour. After that, you are billed according to your updated license type.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Billing issue when changing license type

RESOLVED: Service gateway does not currently support OS updates

Details: If you configure your VCN with a service gateway, the private subnet blocks access to the YUM repositories needed to update the OS. This issue affects all types of DB systems.

Workaround: This issue is now resolved. Here is the workaround that was recommended before the issue's resolution:

The service gateway enables access to the Oracle YUM repos if you use the Available Service CIDR Labels called All <region> Services in Oracle Services Network. However, you still might have issues accessing the YUM services through the service gateway. There's a solution to the issue. For details, see Access issues for instances to Oracle yum services through service gateway.

Direct link to this issue: Service gateway does not currently support OS updates

Bare Metal and Virtual Machine DB Systems Only

Backing up to Object Storage using dbcli or RMAN fails due to certificate change

Details: Unmanaged backups to Object Storage using the database CLI (dbcli) or RMAN fail with the following errors:

-> Oracle Error Codes found:
-> ORA-19554: error allocating device, device type: SBT_TAPE, device name:
-> ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
-> KBHS-00712: ORA-29024 received from local HTTP service
-> ORA-27023: skgfqsbi: media manager protocol error

In response to policies implemented by two common web browsers regarding Symantec certificates, Oracle recently changed the certificate authority used for Oracle Cloud Infrastructure. The resulting change in SSL certificates can cause backups to Object Storage to fail if the Oracle Database Cloud Backup Module still points to the old certificate.

Workaround for dbcli: Check the log files for the errors listed and, if found, update the backup module.

Review the RMAN backup log files for the errors listed above:

  1. Determine the ID of the failed backup job.

    dbcli list-jobs

    In this example output, the failed backup job ID is "f59d8470-6c37-49e4-a372-4788c984ea59".

    root@<node name> ~]# dbcli list-jobs
     
    ID                                       Description                                                                 Created                             Status
    ---------------------------------------- --------------------------------------------------------------------------- ----------------------------------- ----------
    cbe852de-c0f3-4807-85e8-7523647ec78c     Authentication key update for DCS_ADMIN                                     March 30, 2018 4:10:21 AM UTC       Success
    db83fdc4-5245-4307-88a7-178f8a0efa48     Provisioning service creation                                               March 30, 2018 4:12:01 AM UTC       Success
    c1511a7a-3c2e-4e42-9520-f156b1b4cf0e     SSH keys update                                                             March 30, 2018 4:48:24 AM UTC       Success
    22adf146-9779-4a2c-8682-7fd04d7520b2     SSH key delete                                                              March 30, 2018 4:50:02 AM UTC       Success
    6f2be750-9823-4ed5-b5ff-8e49f136dd22     create object store:bV0wqIaoLA4xLT4dGjOu                                    March 30, 2018 5:33:38 AM UTC       Success
    0716f464-1a10-40df-a303-cadee0302b1b     create backup config:bV0wqIaoLA4xLT4dGjOu_BC                                March 30, 2018 5:33:49 AM UTC       Success
    e08b21c3-cd09-4e3a-944c-d1da96cb21d8     update database : hfdb1                                                     March 30, 2018 5:34:04 AM UTC       Success
    1c3d7c58-79c3-4039-8f48-787057ce7c6e     Create Longterm Backup with TAG-DBTLongterm<identity number> for Db:<dbname>    March 30, 2018 5:37:11 AM UTC       Success
    f59d8470-6c37-49e4-a372-4788c984ea59     Create Longterm Backup with TAG-DBTLongterm<identity number> for Db:<dbname>    March 30, 2018 5:43:45 AM UTC       Failure
  2. Use the ID of the failed job to obtain the location of the log file to review.

    
    dbcli describe-job -i <failed_job_ID>

    Relevant output from the describe-job command should look like this:

    Message: DCS-10001:Internal error encountered: Failed to run Rman statement.
    Refer log in Node <node_name>: /opt/oracle/dcs/log/<node_name>/rman/bkup/<db_unique_name>/rman_backup/<date>/rman_backup_<date>.log.

Update the Oracle Database Cloud Backup Module:

  1. Determine the Swift object store ID and user the database is using for backups.

    1. Run the dbcli list-databases command to determine the ID of the database.

    2. Use the database ID to determine the backup configuration ID (backupConfigId).

      dbcli list-databases
      dbcli describe-database -i <database_ID> -j
    3. Using the backup configuration ID you noted from the previous step, determine the object store ID (objectStoreId).

      dbcli list-backupconfigs
      dbcli describe-backupconfig –i <backupconfig_ID> -j
    4. Using the object store ID you noted from the previous step, determine the object store user (userName).

      dbcli list-objectstoreswifts
      dbcli describe-objectstoreswift –i <objectstore_ID> -j
  2. Using the object store credentials you obtained from step 1, update the backup module.

    dbcli update-objectstoreswift –i <objectstore_ID> -p –u <user_name>

Workaround for RMAN: Check the RMAN log files for the error messages listed. If found, log on to the host as the oracle user, and use your Swift credentials to reinstall the backup module.

Note

Swift passwords are now called "Auth tokens." For details, see Using an Auth Token with Swift.
java -jar <opc_install.jar_path> -opcId '<swift_user_ID>' -opcPass '<auth_token>' -container <objectstore_container> -walletDir <wallet_directory> -configfile <config_file> -host https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace> -import-all-trustcerts

For a multi-node DB system, perform the workaround on all nodes in the cluster.

See Oracle Database Cloud Backup Module documentation for details on using this command.

Direct link to this issue: Backing up to Object Storage using dbcli or RMAN fails due to certificate change

Breaking changes in Database service SDKs

Details: The SDKs released on October 18, 2018 introduce code-breaking changes to the database size and the database edition attributes in the database backup APIs.

Workaround: Refer to the following language-specific documentation for more details about the breaking changes, and update your existing code as applicable:

Direct link to this issue: Breaking changes in Database service SDKs

Unable to use Managed Backups in your DB system

Details: Backup and restore operations might not work in your DB system when you use the Console or the API.

Workaround: Install the Oracle Database Cloud Backup Module, and then contact Oracle Support Services for further instructions.

To install the Oracle Database Cloud Backup Module:

  1. SSH to the DB system, and log in as opc.

    
    ssh -i <SSH_key> opc@<DB_system_IP address>
    login as: opc

    Alternatively, you can use opc@<DB_system_hostname> to log in.

  2. Download the Oracle Database Cloud Backup Module from http://www.oracle.com/technetwork/database/availability/oracle-cloud-backup-2162729.html.
  3. Extract the contents of opc_installer.zip to a target directory, for example, /home/opc.
  4. In your tenancy, create a temporary user, and grant them privileges to access the tenancy's Object Storage.
  5. For this temporary user, create an Working with Auth Tokens and note down the password.
  6. Verify that credentials work by running the following curl command:

    Note

    Swift passwords are now called "Auth tokens." For details, see Using an Auth Token with Swift.
    curl -v -X HEAD -u  <user_id>:'<auth_token>' https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace>

    See https://cloud.oracle.com/infrastructure/storage/object-storage/faq for the correct region to use.

    The command should return either the HTTP 200 or the HTTP 204 No Content success status response code. Any other status code indicates a problem connecting to Object Storage.

  7. Run the following command:

    java -jar opc_install.jar -opcid <user_id> -opcPass '<auth_token>' -libDir <target_dir> -walletDir <target_dir> -host https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace> -configFile config.txt

    Note that <target_dir> is the directory to which you extracted opc_installer.zip in step 3.

    This command might take a few minutes to complete because it downloads libopc.so and other files. Once the command completes, you should see several files (including libopc.so) in your target directory.

  8. Change directory to your target directory, and copy the lipopc.so and opc_install.jar files into the /opt/oracle/oak/pkgrepos/oss/odbcs directory.

    cp libopc.so /opt/oracle/oak/pkgrepos/oss/odbcs
    
    
    cp opc_install.jar /opt/oracle/oak/pkgrepos/oss/odbcs

    (You might have to use sudo with the copy commands to run them as root.)

  9. Run the following command to check whether the directory indicated exists:

    
    
    ls /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs

    If this directory exists, perform the following steps:

    1. Back up the files in the /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs directory.
    2. Run these two commands to replace the existing libopc.so and opc_install.jar files in that directory:

      
      cp libopc.so /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs
      cp opc_install.jar /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs
  10. Verify the version of opc_install.jar.

    
    java -jar /opt/oracle/oak/pkgrepos/oss/odbcs/opc_install.jar |grep -i build
    

    If /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs exists, also run the following command:

    
    java -jar /opt/oracle/dcs/commonstore/pkgrepos/oss/odbcs/opc_install.jar |grep -i build

    Both commands should return the following output:

    Oracle Database Cloud Backup Module Install Tool, build MAIN_2017-08-16.
  11. (Optional) Delete the temporary user and the target directory you used to install the backup module.

After you complete the procedure, contact Oracle Support or your tenant administrator for further instructions. You must provide the OCID of the DB system for which you would like to enable backups.

Direct link to this issue: Unable to use Managed Backups in your DB System

Managed Automatic Backups fail on the VM.Standard1.1 shape due to a process crash

Details: Memory limitations of host machines running the VM.Standard1.1 shape can cause failures for automatic database backup jobs managed by Oracle Cloud Infrastructure (jobs managed by using either the Console or the API). You can change the systems' memory parameters to resolve this issue.

Workaround: Change the systems' memory parameters as follows:

  1. Switch to the oracle user in the operating system.

    [opc@hostname ~]$ sudo su - oracle
  2. Set the environment variable to login to the database instance. For example:

    
    [oracle@hostname ~]$ . oraenv
     ORACLE_SID = [oracle] ? orcl
    				
  3. Start SQL*Plus.

    [oracle@hostname ~]$ sqlplus / as sysdba
  4. Change the initial memory parameters as follows:

    
    SQL> ALTER SYSTEM SET SGA_TARGET = 1228M scope=spfile;
    SQL> ALTER SYSTEM SET PGA_AGGREGATE_TARGET = 1228M;
    SQL> ALTER SYSTEM SET PGA_AGGREGATE_LIMIT = 2457M;
    SQL> exit
    							
  5. Restart the database instance.

    
    [oracle@hostname ~]$ srvctl stop database -d db_unique_name -o immediate
    [oracle@hostname ~]$ srvctl start database -d db_unique_name -o open								

Direct link to this issue: Managed Automatic Backups fail on the VM.Standard1.1 shape due to a process crash

Oracle Data Pump operations return "ORA-00439: feature not enabled"

Details: On High Performance and Extreme Performance DB systems, Data Pump utility operations that use compression and/or parallelism might fail and return the error ORA-00439: feature not enabled. This issue affects database versions 12.1.0.2.161018 and 12.1.0.2.170117.

Workaround: Apply patch 25579568 or 25891266 to Oracle Database homes for database versions 12.1.0.2.161018 or 12.1.0.2.170117, respectively. Alternatively, use the Console to apply the April 2017 patch to the DB system and database home.

Note

Determining the Version of a Database in a Database Home

To determine the version of a database in a database home, run either $ORACLE_HOME/OPatch/opatch lspatches as the oracle user or dbcli list-dbhomes as the root user.

Direct link to this issue: Oracle Data Pump operations return "ORA-00439: feature not enabled"

Unable to connect to the EM Express console from your 1-node DB system

Details: You might get a "Secure Connection Failed" error message when you try to connect to the EM Express console from your 1-node DB system because the correct permissions were not applied automatically.

Workaround: Add read permissions for the asmadmin group on the wallet directory of the DB system, and then retry the connection:

  1. SSH to the DB system host, log in as opc, sudo to the grid user.

    [opc@dbsysHost ~]$ sudo su - grid
    [grid@dbsysHost ~]$ . oraenv
    ORACLE_SID = [+ASM1] ?
    The Oracle base has been set to /u01/app/grid
    
  2. Get the location of the wallet directory, shown in red below in the command output.

    [grid@dbsysHost ~]$ lsnrctl status | grep xdb_wallet
    
    (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=dbsysHost.sub04061528182.dbsysapril6.oraclevcn.com)(PORT=5500))(Security=(my_wallet_directory=/u01/app/oracle/admin/dbsys12_phx3wm/xdb_wallet))(Presentation=HTTP)(Session=RAW))
  3. Return to the opc user, switch to the oracle user, and change to the wallet directory.

    [opc@dbsysHost ~]$ sudo su - oracle
    [oracle@dbsysHost ~]$ cd /u01/app/oracle/admin/dbsys12_phx3wm/xdb_wallet
  4. List the directory contents and note the permissions.

    
    [oracle@dbsysHost xdb_wallet]$ ls -ltr
    total 8
    -rw------- 1 oracle asmadmin 3881 Apr  6 16:32 ewallet.p12
    -rw------- 1 oracle asmadmin 3926 Apr  6 16:32 cwallet.sso
    
  5. Change the permissions:

    
    [oracle@dbsysHost xdb_wallet]$ chmod 640 /u01/app/oracle/admin/dbsys12_phx3wm/xdb_wallet/*
  6. Verify that read permissions were added.

    [oracle@dbsysHost xdb_wallet]$ ls -ltr
    total 8
    -rw-r----- 1 oracle asmadmin 3881 Apr  6 16:32 ewallet.p12
    -rw-r----- 1 oracle asmadmin 3926 Apr  6 16:32 cwallet.sso
    

Direct link to this issue: Unable to connect to the EM Express console from your 1-node DB system

Exadata DB Systems Only

Backing up to Object Storage using bkup_api or RMAN fails due to certificate change

Details: Backup operations to Object Storage using the Exadata backup utility (bkup_api) or RMAN fail with the following errors:

* DBaaS Error trace:
-> API::ERROR -> KBHS-00715: HTTP error occurred 'oracle-error'
-> API::ERROR -> ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
-> API::ERROR -> ORA-19554: error allocating device, device type: SBT_TAPE, device name:
-> API::ERROR -> ORA-27023: skgfqsbi: media manager protocol error
-> API::ERROR Unable to verify the backup pieces
-> Oracle Error Codes found:
-> ORA-19554: error allocating device, device type: SBT_TAPE, device name:
-> ORA-19511: non RMAN, but media manager or vendor specific failure, error text:
-> KBHS-00712: ORA-29024 received from local HTTP service
-> ORA-27023: skgfqsbi: media manager protocol error

In response to policies implemented by two common web browsers regarding Symantec certificates, Oracle recently changed the certificate authority used for Oracle Cloud Infrastructure. The resulting change in SSL certificates can cause backups to Object Storage to fail if the Oracle Database Cloud Backup Module still points to the old certificate.

Important

Before using the applicable workaround in this section, follow the steps in Updating Tooling on an Exadata Cloud Service Instance to ensure the latest version of dbaastools_exa is installed on the system.

Workaround for bkup_api: Check the log files for the errors listed above, and if found, reinstall the backup module.

Use the following command to check the status of the failed backup:

/var/opt/oracle/bkup_api/bkup_api bkup_status --dbname=<database_name>

Run the following command to reinstall the backup module:

/var/opt/oracle/ocde/assistants/bkup/bkup -dbname=<database_name>

Workaround for RMAN: Check the RMAN log files for the error messages listed. If found, log on to your host as the oracle user, and reinstall the backup module using your Swift credentials.

Note

Swift passwords are now called "Auth tokens." For details, see Using an Auth Token with Swift.
java -jar <opc_install.jar_path> -opcId '<Swift_user_ID>' -opcPass '<auth_token>' -container <objectstore_container> -walletDir <wallet_directory> -configfile <config_file> -host https://swiftobjectstorage.<region_name>.oraclecloud.com/v1/<object_storage_namespace> -import-all-trustcerts

Perform this workaround on all nodes in the cluster.

See Oracle Database Cloud Backup Module documentation for details on using this command.

Direct link to this issue: Backing up to Object Storage using bkup_api or RMAN fails due to certificate change

Console information not synced for Data Guard enabled databases when using dbaascli

Details: With the release of the shared Database Home feature for Exadata DB systems, the Console now also synchronizes and displays information about databases that are created and managed by using the dbaasapi and dbaascli utilities. However, databases with Data Guard configured do not display correct information in the Console under the following conditions:

  • If Data Guard was enabled by using the Console, and then a change is made to the primary or standby database by using dbaascli (such as moving the database to a different home), the result is not reflected in the Console.
  • If Data Guard was configured manually, the Console does not show a Data Guard association between the two databases.

Workaround: We are aware of the issue and working on a resolution. In the meantime, Oracle recommends that you manage your Data Guard enabled databases by using either only the Console or only command line utilities.

Direct link to this issue: Console information not synced for Data Guard enabled databases when using dbaascli

Grid Infrastructure does not start after offlining and onlining a disk

Details: This is a clusterware issue that occurs only when the Oracle GI version is 12.2.0.1 without any bundle patch. The problem is caused by corruption of a voting disk after you offline then online the disk.

Workaround: Determine the version of the GI, and whether the voting disk is corrupted. Repair the disk, if applicable, and then apply the latest GI bundle.

  1. Verify the GI version is 12.2.0.1 without any bundle patch applied:

    
    [root@rmstest-udaau1 ~]# su - grid
    [grid@rmstest-udaau1 ~]$ . oraenv
    ORACLE_SID = [+ASM1] ? +ASM1
    The Oracle base has been set to /u01/app/grid
    [grid@rmstest-udaau1 ~]$ $ORACLE_HOME/OPatch/opatch lsinventory
    Oracle Interim Patch Installer version 12.2.0.1.6
    Copyright (c) 2018, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/app/12.2.0.1/grid
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/12.2.0.1/grid/oraInst.loc
    OPatch version    : 12.2.0.1.6
    OUI version       : 12.2.0.1.4
    Log file location : /u01/app/12.2.0.1/grid/cfgtoollogs/opatch/opatch2018-01-15_22-11-10PM_1.log
    
    Lsinventory Output file location : /u01/app/12.2.0.1/grid/cfgtoollogs/opatch/lsinv/lsinventory2018-01-15_22-11-10PM.txt
    
    --------------------------------------------------------------------------------
    Local Machine Information::
    Hostname: rmstest-udaau1.exaagclient.sretest.oraclevcn.com
    ARU platform id: 226
    ARU platform description:: Linux x86-64
    
    Installed Top-level Products (1):
    
    Oracle Grid Infrastructure 12c                                       12.2.0.1.0
    There are 1 products installed in this Oracle Home.
    
    
    There are no Interim patches installed in this Oracle Home.
    
    
    --------------------------------------------------------------------------------
    
    OPatch succeeded.
  2. Check the /u01/app/grid/diag/crs/<hostname>/crs/trace/ocssd.trc file for evidence that the GI failed to start due to voting disk corruption:

    ocssd.trc
     
    2017-01-17 23:45:11.955 :    CSSD:3807860480: clssnmvDiskCheck:: configured 
    Sites = 1, Incative sites = 1, Mininum Sites required = 1 
    2017-01-17 23:45:11.955 :    CSSD:3807860480: (:CSSNM00018:)clssnmvDiskCheck: 
    Aborting, 2 of 5 configured voting disks available, need 3 
    ...... 
    . 
    2017-01-17 23:45:11.956 :    CSSD:3807860480: clssnmCheckForNetworkFailure: 
    skipping 31 defined 0 
    2017-01-17 23:45:11.956 :    CSSD:3807860480: clssnmRemoveNodeInTerm: node 4, 
    slcc05db08 terminated. Removing from its own member and connected bitmaps 
    2017-01-17 23:45:11.956 :    CSSD:3807860480: 
    ################################### 
    2017-01-17 23:45:11.956 :    CSSD:3807860480: clssscExit: CSSD aborting from 
    thread clssnmvDiskPingMonitorThread 
    2017-01-17 23:45:11.956 :    CSSD:3807860480: 
    ################################### 
    2017-01-17 23:45:11.956 :    CSSD:3807860480: (:CSSSC00012:)clssscExit: A 
    fatal error occurred and the CSS daemon is terminating abnormally 
     
    ------------
     
    2017-01-19 19:00:32.689 :    CSSD:3469420288: clssnmFindVF: Duplicate voting disk found in the queue of previously configured disks 
    queued(o/192.168.10.18/PCW_CD_02_slcc05cel10|[66223efc-29254fbb-bf901601-21009 
    cbd]), 
    found(o/192.168.10.18/PCW_CD_02_slcc05cel10|[66223efc-29254fbb-bf901601-21009c 
    bd]), is not corrupted 
    2017-01-19 19:01:06.467 :    CSSD:3452057344: clssnmvVoteDiskValidation: 
    Voting disk(o/192.168.10.19/PCW_CD_02_slcc05cel11) is corrupted
  3. You can also use SQL*Plus to confirm that the voting disks are corrupted:

    1. Log in as the grid user, and set the environment to ASM.

      [root@rmstest-udaau1 ~]# su - grid
      [grid@rmstest-udaau1 ~]$ . oraenv
      ORACLE_SID = [+ASM1] ? +ASM1
      The Oracle base has been set to /u01/app/grid
    2. Log in to SQL*Plus as SYSASM.

      $ORACLE_HOME/bin/sqlplus / as sysasm
    3. Run the following two queries:

      SQL> select name, voting_file from v$asm_disk where VOTING_FILE='Y' and group_number !=0;
      SQL> select  CC.name, count(*) from x$kfdat AA JOIN (select disk_number, name from v$asm_disk where VOTING_FILE='Y' and group_number !=0) CC ON CC.disk_number = AA.NUMBER_KFDAT where AA.FNUM_KFDAT= 1048572 group by CC.name;

      If the system is healthy, the results should look like the following example.

      Query 1 Results

      NAME                           VOTING_FILE
      ------------------------------ ---------------
      DBFSC3_CD_02_SLCLCX0788        Y
      DBFSC3_CD_09_SLCLCX0787        Y
      DBFSC3_CD_04_SLCLCX0786        Y

      Query 2 Results

      NAME                           COUNT(*)
      ------------------------------ ---------------
      DBFSC3_CD_02_SLCLCX0788        8
      DBFSC3_CD_09_SLCLCX0787        8
      DBFSC3_CD_04_SLCLCX0786        8

      In a healthy system, every voting disk returned in the first query should also be returned in the second query and the counts for all the disks should be non-zero. Otherwise, one or more of your voting disks are corrupted.

  4. If a voting disks is corrupted, offline the grid disk that contains the voting disk. The cells will automatically move the bad voting disk to the other grid disk and online that voting disk.

    1. The following command offlines a grid disk named DATAC01_CD_05_SCAQAE08CELADM13.

      SQL> alter diskgroup DATAC01 offline disk DATAC01_CD_05_SCAQAE08CELADM13;
           Diskgroup altered.
    2. Wait 30 seconds and then rerun the two queries in step 3c to verify that the voting disk migrated to the new grid disk and that it is healthy.

    3. Verify the grid disk you offlined is now online:

      SQL> select name, mode_status, voting_file from v$asm_disk where name='DATAC01_CD_05_SCAQAE08CELADM13';

      The mode_status should be ONLINE, and the voting_file should NOT be Y.

    Repeat steps 4a through 4c for each remaining grid disk that contains a corrupt voting disk.
    Note

    If the CRS does not start because of the voting disk corruption, start it using Exclusive mode before you execute the command in step 4.

    crsctl start crs -excl
     
  5. If you are using Oracle GI version 12.2.0.1 without any bundle patch, you must upgrade the GI version to the latest GI bundle, whether or not a voting disk was corrupted.

    See Patching Oracle Grid Infrastructure and Oracle Databases Using dbaascli for instructions on how to use the dbaascli utility to perform patching operations for Oracle Grid Infrastructure and Oracle Database on Exadata Database Service on Dedicated Infrastructure.

Direct link to this issue: Grid Infrastructure does not start after offlining and onlining a disk

Managed features not enabled for systems provisioned before June 15, 2018

Details: Exadata DB systems launched on June 15, 2018 or later automatically include the ability to create, list, and delete databases by using the Console, API, or Oracle Cloud Infrastructure CLI. However, systems provisioned before this date require extra steps to enable this functionality.

Attempts to use this functionality without the extra steps result in the following error messages:

  • On creating a database - "Create Database is not supported on this Exadata DB system. To enable this feature, please contact Oracle Support."
  • On terminating a database - "DeleteDbHome is not supported on this Exadata DB system. To enable this feature, please contact Oracle Support."

Workaround: You need to install the Exadata agent on each node of the Exadata DB system.

First, create a service request for assistance from Oracle Support Services. Oracle Support will respond by providing you with a preauthenticated URL for an Oracle Cloud Infrastructure Object Storage location where you can obtain the agent.

Before you install the Exadata agent:

To install the Exadata agent:

  1. Log on to the node as root.
  2. Run the following commands to install the agent:

    [root@<node_n>~]# cd /tmp
    [root@<node_n>~]# wget https://objectstorage.<region_name>.oraclecloud.com/p/1q523eOkAOYBJVP9RYji3V5APlMFHIv1_6bAMmxsS4E/n/dbaaspatchstore/b/dbaasexadatacustomersea1/o/backfill_agent_package_iwwva.tar
    [root@<node_n>~]# tar -xvf /tmp/backfill_agent_package_*.tar -C /tmp
    [root@<node_n>~]# rpm -ivh /tmp/dbcs-agent-2.5-3.x86_64.rpm

    Example output:

    [root@<node_n>~]# rpm -ivh dbcs-agent-2.5-3.x86_64.rpm
    Preparing...                ########################################### [100%]
    Checking for dbaastools_exa rpm on the system
    Current dbaastools_exa version = dbaastools_exa-1.0-1+18.1.4.1.0_180725.0000.x86_64
    dbaastools_exa version dbaastools_exa-1.0-1+18.1.4.1.0_180725.0000.x86_64 is good. Continuing with dbcs-agent installation
       1:dbcs-agent             ########################################### [100%]
    initctl: Unknown instance:
    initctl: Unknown instance:
    initzookeeper start/running, process 85821
    initdbcsagent stop/waiting
    initdbcsadmin stop/waiting
    initdbcsagent start/running, process 85833
    initdbcsadmin start/running, process 85836
    
  3. Confirm that the agent is installed and running.

    [root@<node_n>~]# rpm -qa | grep dbcs-agent
    dbcs-agent-2.5-0.x86_64
    [root@<node_n>~]# initctl status initdbcsagent
    initdbcsagent start/running, process 97832
  4. Repeat steps 1 through 3 on the remaining nodes.

After the agent is installed on all nodes, allow up to 30 minutes for Oracle to complete additional workflow tasks such as upgrading the agent to the latest version, rotating the agent credentials, and so on. When the process is complete, you should be able to use the Exadata managed features in the Console, API, or Oracle Cloud Infrastructure CLI.

Direct link to this issue: Managed features not enabled for systems provisioned before June 15, 2018

Patching configuration file points to wrong region

Details: The patching configuration file (/var/opt/oracle/exapatch/exadbcpatch.cfg) points to the object store of the us-phoenix-1 region, even if the Exadata DB system is deployed in another region.

This problem occurs if the release version of the database tooling package (dbaastools_exa) is 17430 or lower.

Workaround: Follow the instructions in Updating Tooling on an Exadata Cloud Service Instance to confirm that the release version of the tooling package is 17430 or lower, and then update it to the latest version.

Direct link to this issue: Patching configuration file points to wrong region

Various database workflow failures due to Oracle Linux 7 removal of required temporary files

Details: A change in how Oracle Linux 7 handles temporary files can result in the removal of required socket files from the /var/tmp/.oracle directory. This issue affects only Exadata DB systems running the version 19.1.2 operating system image.

Workaround: Run sudo /usr/local/bin/imageinfo as the opc user to determine your operating system image version. If your image version is 19.1.2.0.0.190306, follow the instructions in Doc ID 2498572.1 to fix the issue.

Direct link to this issue: Various database workflow failures due to Oracle Linux 7 removal of required temporary files

Virtual machine DB system storage scaling

If you are scaling either regular data storage or recovery area (RECO) storage from a value less than 10,240 GB (10 TB) to a value exceeding 10,240 GB, perform the scaling in two operations. First, scale the system to 10,240 GB. After this first scaling operation is complete and the system is in the "available" state, perform a second scaling operation, specifying your target storage value above 10,240 GB. Attempting to scale from a value less than 10,240 GB to a value higher than 10,240 GB in a single operation can lead to a failure of the scaling operation. For instructions on scaling, see Scale Up the Storage For a Virtual Machine DB System.

Virtual Machine DB systems shape scaling fails because DB_Cache_nX parameter is not 0 (zero)

When scaling a virtual machine DB system to use a larger system shape, the scaling operation fails if a DB_Cache_nX parameter is not set to 0 (zero).

Workaround

When scaling a virtual DB system, ensure that all DB_Cache_nX parameters (for example, DB_nK_CACHE_SIZE) are set to 0.

Developer Tools

Potential data corruption issue in OCI Java SDK with binary data upload using default retries

Details: If you are using any of the OCI Java SDK synchronous clients that upload streams of data (for example ObjectStorageClient or DataSafeClient) and you do not define the RetryConfiguration at the client level or request level, you may be affected by silent data corruption.

Workaround: We are actively working on fixing this issue. For more information and possible workarounds, see the issue on GitHub.

Direct link to this issue: Potential data corruption issue in OCI Java SDK with binary data upload using default retries

Performance regression in OCI Java SDK versions 2.14.1 and later for all API operations

Details: If you're using versions 2.14.1 and later of the OCI Java SDK, you may encounter performance regressions when using the SDK to call API operations on any of the OCI services. The regression causes a 30% to 60% increase in latency in SDK operations on any of the OCI services.

Workaround: We are actively working on fixing this issue. For more information and possible workarounds, see the issue on GitHub.

Direct link to this issue: Performance regression in OCI Java SDK versions 2.14.1 and later for all API operations

Performance regression with the Apache Connector Provider in OCI SDK for Java

Details: Starting with version 2.0.0, the OCI SDK for Java supports using the Jersey ApacheConnectorProvider instead of the Jersey default HttpUrlConnectorProvider to allow the Apache HttpClient to make OCI service calls.

The ApacheConnectorProvider supports the use of Expect header by default for some Object Storage service operations (see https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/100). This has been observed to cause performance regression in the same Object Storage service operations.

Workaround:You can disable the use of the Expect header by switching back to Jersey Default Connector (see https://docs.oracle.com/iaas/Content/API/SDKDocs/javasdkconfig.htm), or if you're already using the ApacheConnectorProvider, you can disable the Expect header with the ApacheConnectorProvider by doing the following when initializing the client :
final ApacheConnectorProperties apacheConnectorProperties =
        ApacheConnectorProperties.builder()
                .expectContinue(false) // disable the Expect header
                .build();

final ApacheConfigurator configurator =
        new ApacheConfigurator.NonBuffering(apacheConnectorProperties);
        
ObjectStorageClient objectStorageClient =
        ObjectStorageClient.builder()
                .clientConfigurator(configurator)
                .build(provider);

Direct link to this issue: Performance regression with the Apache Connector Provider in OCI SDK for Java

Truncated response for operations that return binary data with the OCI Java SDK

Details: In versions 2.0.0 to 2.13.0 of the OCI Java SDK API, some operations that return a stream of data but don't return the content-length header in the response might return truncated data. This is caused by the SDK prematurely closing the stream before reading all the data.

Workaround:Update the OCI Java SDK client to version 2.13.1 or later. For more information about this issue and workarounds, see https://github.com/oracle/oci-java-sdk/issues/357

Direct link to this issue: Truncated response for operations that return binary data with the OCI Java SDK

Go SDK cannot automatically find some regions while running in Cloud Shell

Details: Due to some issues with one of its dependencies, the Go SDK feature which allows customers to automatically use new realms which might be unknown to the SDK is not functioning from within Cloud Shell.

Attempting to run code in Cloud Shell that uses this feature will result in the following error message:
can not create client, bad configuration: failed to get security token: failed to renew security token: failed to get security token: failed to call: Post "https://<endpoint>/v1/x509": dial tcp: lookup <endpoint> on 127.0.0.11:53: server misbehaving
panicked while retrying operation. Panic was: runtime error: invalid memory address or nil pointer dereference

Workaround: To resolve this issue, enable resolving regions using the instance metadata service for Go SDK. For more information, see: Adding Regions

Direct link to this issue: Go SDK cannot automatically find some regions while running in Cloud Shell

Increased latency issues in operations for some OCI services using the SDKS and other tools

Details: You may encounter an increase in latency for operations made to some OCI services using the SDKs, Terraform, Ansible, and the CLI. This issue has been confirmed to impact the OCI Streaming service, and likely impacts the Email Delivery, Health Checks, NoSQL Database Cloud, Registry, Generic Artifacts, and Web Application Acceleration and Security services as well. This list is not comprehensive, and it is possible you may encounter the issue against other OCI services as well. The issue has been confirmed to NOT affect the OCI Object Storage and Functions services.

The following SDKs and tools are impacted:

  • Go SDK (version 41.2.0 and later)
  • .NET SDK (version 14.2.0 and later)
  • Java SDK (version 2.0.0 and later)
  • Python SDK (version 2.38.4 and later)
  • CLI (version 2.25.0 and later)
  • PowerShell Modules (version 9.2.0 and later)
  • Ansible Modules (version 2.23.0 and later)
  • Terraform Provider (version 4.30.0 and later)

Workarounds and more information: We are actively working on fixing this issue. For more information and possible workarounds, please see the following:

Direct link to this issue: Increased latency issues in operations for some OCI services using the SDKS and other tools

Python SDK composite operations throw a 404 NotAuthorizedOrNotFound error even though operation is a success

Details: The copy_boot_volume_backup_and_wait_for_state and copy_volume_backup_and_wait_for_state from the BlockStorage Client Composite operations throw a 404/NotAuthorizedOrNotFound when copying a backup from one region to another. For more information see: https://github.com/oracle/oci-python-sdk/issues/344.

Workaround: Instead of using the composite operations, use two different clients for this operation; one client in the Source Region to send the request for copying the backup from Region A to Region B, and a second client in Destination region to wait for the backup to become available. See example here: https://github.com/oracle/oci-python-sdk/blob/master/examples/copy_volume_backup_example.py

Direct link to this issue: Python SDK composite operations throw a 404 NotAuthorizedOrNotFound error even though operation is a success

Potential data rounding issue for big numbers with OCI SDK for TypeScript and JavaScript

Details: The OCI SDK for TypeScript and JavaScript have a known issue with big numbers greater than JavaScript's Number.MAX_SAFE_INTEGER. Any Numbers greater than Number.MAX_SAFE_INTEGER will result in rounding issue.

Workaround: We are aware of the issue where an API response may send back a number greater than JavaScript's Number.MAX_SAFE_INTERGER. At the moment the number rounding issue is not an impact to calling any APIs.

Direct link to this issue: Potential data rounding issue for big numbers with OCI SDK for TypeScript and JavaScript

Potential data corruption issue with OCI Java SDK on binary data upload with RefreshableOnNotAuthenticatedProvider

Details: When using version 1.25.1 or earlier of the OCI Java SDK clients that upload streams of data (for example ObjectStorageClient or FunctionsInvokeClient), either synchronously and asynchronously, and you use a RefreshableOnNotAuthenticatedProvider (for example, for Resource Principals or Instance Principals) you may be affected by silent data corruption.

Workaround: Update the OCI Java SDK client to version 1.25.2 or later. For more information about this issue and workarounds, see Potential data corruption issue for OCI Java SDK on binary data upload with RefreshableOnNotAuthenticatedProvider.

Direct link to this issue: Potential data corruption issue with OCI Java SDK on binary data upload with RefreshableOnNotAuthenticatedProvider

Potential data corruption issue with OCI HDFS Connector on binary data upload with RefreshableOnNotAuthenticatedProvider

Details: If you are using version 3.2.1.1 or earlier of the OCI HDFS Connector clients and you use a RefreshableOnNotAuthenticatedProvider (e.g. InstancePrincipalsCustomAuthenticator, or generally for Resource Principals or Instance Principals) you may be affected by silent data corruption.

Workaround: Update the OCI HDFS Connector client to version 3.2.1.3 or later. For more information about this issue and workarounds, see Potential data corruption issue for OCI HDFS Connector with RefreshableOnNotAuthenticatedProvider.

Direct link to this issue: Potential data corruption issue with OCI HDFS Connector on binary data upload with RefreshableOnNotAuthenticatedProvider

Potential data corruption with SDK for Python on binary upload

Details: When using the SDK for Python to perform binary upload operations you may encounter an issue with data corruption if retries are enabled or if you are using UploadManager.upload_file.

Workaround: We are aware of the issue and working on a resolution. For more information about this issue and workarounds, see Potential data corruption issue for PythonSDK retry on binary data upload.

Direct link to this issue: Potential data corruption with SDK for Python on binary upload

Potential data corruption issue with SDK for Python and upload streams

Update: The root cause of the issue causing data corruption has been fixed with the release of v2.54.0. Please use oci v2.54.0 or above to avoid data corruption. The behavior of older versions of the OCI Python SDK regarding this issue has been explained below.

Details: Customers using the OCI SDK for Python and oci.object_storage.UploadManager.upload_stream in FIPS mode might be vulnerable to silent data corruption. If the circumstances to produce the issue are true for your environment, the SDK reports success for the upload operation, but a 0-length object is uploaded.

The resolution differs depending on the state of your environment:

  1. Using UploadManager.upload_stream() in an environment which uses a FIPS-compliant OpenSSL version where the SDK for Python is not operating in FIPS mode as described in Using FIPS validated libraries.

    To determine if you fall under this scenario:

    • Verify that you are using a FIPS-compliant OpenSSL version by running the command openssl version . If "fips" is part of the version name, and you are not operating the SDK in FIPS mode, then you would fall under this scenario.

    • If oci.fips.is_fips_mode() does not return True, then the SDK is not operating in FIPS mode.

    Workaround: Upgrade the OCI SDK for Python to version 2.53.1 or greater and operate the SDK for Python in FIPS mode as described in Using FIPS validated libraries.
    Important

    Not operating the SDK in FIPS mode while using a FIPS-compliant OpenSSL version will still result in data corruption while using UploadManager.upload_stream().
  2. Using UploadManager.upload_stream() when the SDK for Python is operating in FIPS mode as described in Using FIPS validated libraries and the SDK for Python is v2.53.0 or lower.

    If oci.fips.is_fips_mode() returns True, then the SDK is operating in FIPS mode.

    Resolution: Upgrade the OCI SDK for Python to version 2.53.1 or greater.

For more information about this issue, see Potential data corruption issue for multipart stream upload for OCI Python SDK on GitHub.

Direct link to this issue: Potential data corruption issue with SDK for Python and upload streams

DNS

Currently, there are no known DNS issues.

Events

Currently, there are no known Events issues.

File Storage

File Storage does not currently support Access Control Lists (ACLs)
Details: File Storage does not support file level Access Control Lists (ACLs). Only user, group, and world permissions are supported. File Storage uses the NFSv3 protocol, which doesn't include support for ACLs. setfacl fails on mounted file systems. getfacl returns only standard permissions.
Note

Some implementations might extend the NFSv3 protocol and add support for ACLs as part of a separate rpc program.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue:File Storage does not currently support Access Control Lists (ACLs)

Semaphore timeout error when creating a snapshot with the Windows command line

Details: When using the mkdir command in Windows CMD to create a snapshot of a mounted file system, an error appears. For example: 

C:\>mkdir X:\.snapshot\snapshot1

The semaphore timeout period has expired.

Although the error appears, the snapshot is successfully created.

Workaround: Use the Console, API or CLI to create snapshots.

Direct link to this issue: Semaphore timeout error when creating a snapshot with Windows command line

Unable to move file storage resources to a different compartment

Details: When moving a file system or mount target from one compartment to another, the operation fails. Users are required to be members of the Administrators group.

Workaround: We are aware of the issue and working on a resolution. To work around this problem, be sure the user is a member of the Administrators group. For more information, see Managing Groups.

Direct link to this issue:Unable to move file storage resources to a different compartment.

409 error occurs when creating or moving a file system or mount target

Details: When creating or moving a file system or mount target from one compartment to another, you might encounter one of the following 409 API errors:

Create File System:

oci.exceptions.ServiceError: {'opc-request-id': <<OPC REQUEST ID>>, 'code': 'Conflict', 'message': 'Another filesystem is currently being provisioned, try again later', 'status': 409}

Move File System:

oci.exceptions.ServiceError: {'opc-request-id': <<OPC REQUEST ID>>, 'code': 'Conflict', 'message': 'filesystem <<FILE SYSTEM OCID>> is currently being modified, try again later', 'status': 409}

Create Mount Target:

oci.exceptions.ServiceError: {'opc-request-id': <<OPC REQUEST ID>>, 'code': 'Conflict', 'message': 'Another mount target is currently being provisioned, try again later', 'status': 409}

Move Mount Target:

oci.exceptions.ServiceError: {'opc-request-id': <<OPC REQUEST ID>>, 'code': 'Conflict', 'message': 'mount target<<MOUNT TARGET OCID>> is currently being modified, try again later', 'status': 409}

The Overview of Compartment Quotas feature introduces constraints that limit the number of concurrent operations that a tenancy can perform on file system and mount target resources in a region:

  • Each tenancy in a region can have 1 CreateFileSystem or ChangeFilesystemCompartment operation in progress at a time.
  • Each tenancy in a region can have 1 CreateMountTarget or ChangeMountTargetCompartment operation in progress at a time.

If a tenancy attempts to do more than one simultaneous operation, one operation succeeds and the others receive the 409 error response code. The default retry strategy for the OCI SDK is to not retry 409 conflicts. See SDK Behaviors - Retries.

Workaround: We are aware of the issue and working on a resolution. To work around this problem, create a custom retry strategy that retries on 409. Several examples of building a custom retry strategy are provided at https://github.com/oracle/oci-python-sdk/blob/master/examples/retries.py.

Direct link to this issue: 409 error occurs when creating or moving a file system or mount target

File Storage in-transit encryption does not currently support DNS hostnames

File systems that use in-transit encryption can't be mounted using a DNS hostname. Only IP addresses can be used to mount file systems with in-transit encryption.

Details: The oci-fss-utils in-transit encryption tool does not currently support the use of DNS hostnames for mounting file systems.

Workaround: We are aware of the issue and working on a resolution. Until this issue is resolved, use the IP address of the mount target in the oci-fss-utils mount command. For example:
sudo mount -t oci-fss 10.x.x.x:/fs-export-path /mnt/yourmountpoint
Replace 10.x.x.x: with the local subnet IP address assigned to your mount target, fs-export-path with the export path you specified when associating the file system with the mount target, and yourmountpoint with the path to the local mount point. The export path is the path to the file system (relative to the mount target IP address). See Using In-transit Encryption for more information.

Direct link to this issue: File Storage in-transit encryption does not currently support DNS hostnames

Functions

Currently, there are no known Functions issues.

Health Checks

Currently, there are no known Health Checks issues.

IAM

Security questions are not available as MFA factors

Details: Security questions as MFA factors are causing region subscription issues for cloud accounts that use IAM identity domains.

Workaround: For cloud accounts that use IAM identity domains, do not use security questions as MFA factors.

Direct link to this issue: Security questions are not available as MFA factors

Permissions granted through policies that specify groups or dynamic groups by name persist through name changes

Details: Policy statements that reference a group or dynamic group by name remain valid even through changes to the group or dynamic group name. Any access granted to the group or dynamic group by its previous name persists. The policy continues to grant the members of the group or dynamic group access to resources without any changes to the policy statement itself. This happens because IAM applies the policy to the subject OCID rather than its name.

Workaround: Oracle strongly recommends that you update policy statements to stay current with intended group or dynamic group names or to reference subject OCIDs instead. Also delete any policy statements with outdated references that you no longer need.

Direct link to this issue: Permissions granted through policies that specify groups or dynamic groups by name persist through name changes

New permissions in resource-types are not propagated

Details: When a new permission is added to an existing resource-type, the permission is not propagated to any policies that include the resource-type. This happens because IAM does not recompile a policy unless there is a change to the policy statement.

Workaround: For any existing policies that use resource-types, when new permissions are added to the resource-type, edit the policy by adding a blank space. Then, save the policy.

Direct link to this issue: New permissions in resource-types are not propagated

Unable to set up new federations with Microsoft Active Directory

Details: During the setup process, after you upload the Oracle Cloud Infrastructure federation metadata document, Microsoft AD FS automatically enables assertion encryption. Oracle Cloud Infrastructure does not support encryption at this time, so the setup fails.

Workaround: This issue is resolved. The IAM service now supports assertion encryption. See Federating with Microsoft Active Directory.

Direct link to this issue: Unable to set up new federations with Microsoft Active Directory

Deleted compartments continue to count against service limits

Details: Deleted compartments continue to count against the compartment service limit for your tenancy. A deleted compartment is removed from the count after 90 days. This is also the setting that specifies the time period for deleted compartments to remain displayed in the Console.

Workaround: Until this issue is resolved, you can request to have your service limit increased for compartments. See Requesting a Service Limit Increase.

Direct link to this issue: Deleted compartments continue to count against service limits

Integration

For known issues with Integration Generation 2, see Known Issues.

For known issues with Integration 3, see Known Issues.

Java Management

For details about known issues in the Java Management service, see Known Issues.

Language

Currently, there are no known issues with the Language service.

Logging

Some agent warnings can be ignored

Details: Benign warnings may occur for the Oracle fluentd-based agent, similar to the following:

Sep 22 05:47:43 ociutv3mgftp02 ruby[1278962]: /opt/unified-monitoring-agent/embedded/lib/ruby/gems/2.6.0/gems/oci-2.9.0.1125/lib/oci/identity/models/base_tag_definition_validator.rb:23: warning: already initialized constant OCI::Identity::Models::BaseTagDefinitionValidator::VALIDATOR_TYPE_ENUM
Sep 22 05:47:43 ociutv3mgftp02 ruby[1278962]: /opt/unified-monitoring-agent/embedded/lib/ruby/gems/2.6.0/gems/oci-2.9.0.1125/lib/oci/identity/models/base_tag_definition_validator.rb:24: warning: previous definition of VALIDATOR_TYPE_ENUM was here

You can ignore benign warnings. These warnings have no impact on agent functionality.

Direct link to this issue: Some agent warnings can be ignored

Unified Monitoring Agent cannot parse the Logrus format

Details: Container Engine for Kubernetes (OKE) versions greater than 1.20 are not supported by the Unified Monitoring Agent. Oracle Cloud Infrastructure Logging cannot parse the Logrus format that CRI-O uses. For more information, see Log Inputs and Parsers and Currently Supported Kubernetes Versions.

Direct link to this issue: Unified Monitoring Agent cannot parse the Logrus format

Logging Analytics

On-demand upload from a Windows machine using a zip file

Details: The on-demand upload of a zip file which is created on a Windows machine might sometimes fail to upload the log content. The reason for the failure is that the zip created on Windows has the same last modification time as the file's creation time. So, when the file is unzipped, the file's last modification time is set as the file's creation time which might be older than the timestamp of the log entries in the log file. In such a case, the log entries with the timestamp more recent than the file's last modification time are not uploaded.

An example of the issue:

Timestamp on the log entry: 2020-10-12 21:12:06

File last modification time of the log file: 2020-10-10 08:00:00

Workaround: Copy the log files to a new folder and create a zip file. This action makes the file's last modification time more recent than the timestamp of the log entries. Use this new zip file for on-demand upload.

Using the previous example, after the workaround is implemented:

Timestamp on the log entry: 2020-10-12 21:12:06

File last modification time of the log file: 2021-03-31 08:00:00

Direct link to this issue: On-demand upload from a Windows machine using a zip file

Special handling when monitoring logs in large folders

Details: Folders containing more than 10,000 files can cause log collection issues (as well as operating system issues).

When large folders are encountered by the Management Agent Logging Analytics plug-in, a message similar to the following example message is added to the Management Agent mgmt_agent.log file:

2020-07-30 14:46:51,653 [LOG.Executor.2388 (LA_TASK_os_file)-61850] INFO - ignore large dir /u01/service/database/logs. set property loganalytics.enable_large_dir to enable.

Resolution: We recommend avoiding large folders.

However, if you want to continue monitoring logs in large folders, then you can enable the property indicated in the mgmt_agent.log file by performing the following action:

sudo -u mgmt_agent echo "loganalytics.enable_large_dir=true" >> INSTALL_DIRECTORY/agent_inst/config/emd.properties

Replace INSTALL_DIRECTORY with the path to the agent_inst folder.

Direct link to this issue: Special handling when monitoring logs in large folders

Managed Cloud Self Service Platform

For known issues with Managed Cloud Self Service Platform, see Known Issues.

Management Agent

Currently, there are no known Management Agent issues.

Marketplace

Currently, there are no known Marketplace issues.

Media Flow

For known issues with Media Flow, see Known Issues.

Media Streams

For known issues with Media Streams, see Known Issues.

Monitoring

Alarm Messages are not Received in Oracle Platform Services Managed Compartments

Details: Alarm messages sent to topics  in Oracle Platform Services managed compartments (named "ManagedCompartmentForPaas") are not received. This issue occurs when the Monitoring service does not have permission to use topics in that compartment.

Workaround: We are aware of the issue and working on a resolution. To work around this issue, move the alarm to a non-managed compartment and update the alarm's notification destination  to use a topic  in a non-managed compartment.

Direct link to this issue: Alarm Messages are not Received in Oracle Platform Services Managed Compartments

Notifications

Currently, there are no known Notifications issues.

Object Storage

Currently, there are no known Object Storage issues.

Operations Insights

Currently, there are no known Operations Insights issues.

OS Management

For details about known issues in the OS Management service, see Known Issues.

Process Automation

For details about known issues in the Process Automation service, see Known Issues.

Registry

Use Tenancy Namespace instead of Tenancy Name in image tags and Docker login credentials on or before September 30, 2019

Details: Up to now, you might have been using either the tenancy name or the tenancy namespace when logging in to Oracle Cloud Infrastructure Registry and when performing operations on images in the Container Registry.

After September 30, 2019, you will have to use the tenancy namespace rather than the tenancy name when using Oracle Cloud Infrastructure Registry.

Background: After September 30, 2019, you will not be able to:

  • Specify the tenancy name when logging in to Oracle Cloud Infrastructure Registry.
  • Perform operations on images that include tenancy name in the repository path.

Instead, you will have to use the tenancy namespace rather than the tenancy name when using Oracle Cloud Infrastructure Registry.

A tenancy namespace is an auto-generated and immutable random string of alphanumeric characters. For example, the namespace of the acme-dev tenancy might be ansh81vru1zp. You can see the tenancy namespace on the Container Registry page of the Console.

Note that for some older tenancies, the tenancy namespace might be the same as the tenancy name. If that is the case, no action is required.

On or before September 30, 2019, if the tenancy namespace and the tenancy name are different, you must:

  • Start specifying the tenancy namespace when logging in to Oracle Cloud Infrastructure Registry, instead of the tenancy name.
  • Start specifying the tenancy namespace when pushing new images to Oracle Cloud Infrastructure Registry, instead of the tenancy name.
  • Migrate any existing images in Oracle Cloud Infrastructure Registry that include the tenancy name in the path.

The following workarounds and examples assume:

  • tenancy name is acme-dev
  • tenancy namespace is ansh81vru1zp
  • username is jdoe@acme.com

Workaround for logging into Oracle Cloud Infrastructure Registry: Previously, when you logged in to Oracle Cloud Infrastructure Registry and were prompted for a username, you could have entered it in the format <tenancy-name>/<username>.

For example:

$ docker login phx.ocir.io

Username: acme-dev/jdoe@acme.com
Password:

On or before September 30, 2019, you must start using the tenancy namespace instead of the tenancy name when logging in to Oracle Cloud Infrastructure Registry. When you are prompted for username, enter it in the format <tenancy-namespace>/<username>.

For example:

$ docker login phx.ocir.io

Username: ansh81vru1zp/jdoe@acme.com
Password:

Workaround for pushing new images to Oracle Cloud Infrastructure Registry: Previously, when you pushed a new image to Oracle Cloud Infrastructure Registry, you could have specified the tenancy name as part of the repository path in the docker push command. You could have entered the command in the format:

$ docker push <region-key>.ocir.io/<tenancy-name>/<image-name>:<tag>

For example:

$ docker push phx.ocir.io/acme-dev/helloworld:latest

On or before September 30, 2019, you must start using the tenancy namespace instead of the tenancy name in the docker push command when you push new images. Enter the command in the format:

$ docker push <region-key>.ocir.io/<tenancy-namespace>/<image-name>:<tag>

For example:

$ docker push phx.ocir.io/ansh81vru1zp/helloworld:latest

Workaround for existing images in Oracle Cloud Infrastructure Registry that include the tenancy name in the repository path: If you have previously pushed images to Oracle Cloud Infrastructure Registry, those existing images could have included the tenancy name as part of the repository path. For example, phx.ocir.io/acme-dev/helloworld:latest.

After September 30, 2019, you will not be able to perform operations on existing images in the Container Registry that include the tenancy name in the repository path.

So on or before September 30, 2019, for every existing image that contains the tenancy name in the repository path, you must replace tenancy name with tenancy namespace.

To replace tenancy name with tenancy namespace in the repository path of an existing image:

  1. Pull the image by entering:

    $ docker pull <region-key>.ocir.io/<tenancy-name>/<image-name>:<tag>

    For example:

    $ docker pull phx.ocir.io/acme-dev/helloworld:latest
  2. Use the docker tag command to change the repository path by entering:

    $ docker tag <region-key>.ocir.io/<tenancy-name>/<image-name>:<tag> <region-key>.ocir.io/<tenancy-namespace>/<image-name>:<tag>

    For example:

    $ docker tag phx.ocir.io/acme-dev/helloworld:latest phx.ocir.io/ansh81vru1zp/helloworld:latest
  3. Push the image with the new repository path to the Container Registry by entering:

    $ docker push <region-key>.ocir.io/<tenancy-namespace>/<image-name>:<tag>

    For example:

    $ docker push phx.ocir.io/ansh81vru1zp/helloworld:latest
  4. Repeat the above steps for every existing image that has tenancy name in the repository path.

Direct link to this issue: Use Tenancy Namespace instead of Tenancy Name in image tags and Docker login credentials on or before September 30, 2019

Resource Manager

Error: Circuit breaker is open

Details: The logs for a job show the following: Error: Circuit breaker is open. This error usually indicates an error with a downstream service.

Workaround: We are aware of the issue and working on a resolution. Follow these instructions to identify the downstream service causing the error, then contact that service to determine resolution.

  1. Gather debugging information by running a new job on the stack:

    1. Display the job panel for the type of job you want to run (Plan, Apply, or Destroy).

    2. In the job panel, click Show Advanced Options.

    3. Set Detailed Log Level to Debug.

      Wait for the job to finish running.

      For more information about debugging Terraform job logs, see Debugging Terraform.

  2. Download the detailed log: On the Job Details page for the job you just ran, click Download Detailed Log File (in the Job Information tab, to the right of Detailed Log Level).

  3. Review the downloaded log to identify the error-associated downstream services.

  4. Contact the associated downstream service to determine resolution.

Direct link to this issue: Error: Circuit breaker is open

Object Storage buckets may not be available in the Console for new stacks

Details: When using the Console to create a stack from a bucket in Object Storage, the list of values for Object Storage Buckets is only available when the bucket namespace is identical to the tenancy name.

Workaround: Create the stack using the SDK, CLI, or API instead. We are aware of the issue and working on a resolution.

Direct link to this issue: Object Storage buckets may not be available in the Console for new stacks

Resource Discovery fails (permissions issue)

Details: When using Resource Discovery to create a stack from a compartment, the work request fails.

Possible cause: The user who is creating the stack lacks permissions to inspect compartments for the tenancy.

Workaround: To work around this issue, make sure that the user who is creating the stack has permissions to inspect compartments for the tenancy. For the group that the user belongs to, create the following policy.

Allow group <group name> to inspect compartments in tenancy

Direct link to this issue: Resource Discovery fails

Missing attributes in some discovered resources

Details: Attributes are missing from some supported resources captured using resource discovery.

Service Resource type Missing fields (with links to oci Terraform provider documentation)
Big Data Service Instances

cluster_admin_password

cluster_public_key

Block Volume (core) Volumes volume_backup_id
Compute (core) Images instance_id

image_source_details

Compute (core) Instance Configurations instance_id

source

Compute (core) Instance Console Connections public_key
Compute (core) Instances

hostname_label (deprecated)

is_pv_encryption_in_transit_enabled

subnet_id (deprecated)

Compute (core) Volume Attachments use_chap
Container Engine for Kubernetes Node Pools node_source_details
Data Catalog Connections enc_properties
Database Autonomous Container Databases maintenance_window_details
Database Autonomous Databases

admin_password

autonomous_database_backup_id

autonomous_database_id

clone_type

is_preview_version_with_service_terms_accepted

source

source_id

timestamp

Database Autonomous Exadata Infrastructures maintenance_window_details
Database Databases

admin_password

backup_id

backup_tde_password

db_version

source

Database Db Homes

admin_password

backup_id

backup_tde_password

source

Database Db Systems

admin_password

backup_id

backup_tde_password

maintenance_window_details

IAM Identity Providers metadata
Load Balancer Load Balancers ip_mode
Marketplace Accepted Agreements signature
Networking (core) Cross Connects

far_cross_connect_or_cross_connect_group_id

near_cross_connect_or_cross_connect_group_id

NoSQL Database Cloud Indexes is_if_not_exists
Object Storage Objects

cache_control

content

content_disposition

content_encoding

content_language

source

source_uri_details

Web Application Acceleration and Security Certificates

certificate_data

is_trust_verification_disabled

private_key_data

Web Application Acceleration and Security Policies

are_redirects_challenged

is_case_sensitive

is_nat_enabled (human_interaction_challenge)

is_nat_enabled (js_challenge)

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Missing attributes in some discovered resources

Roving Edge Infrastructure

Currently, there are no known Roving Edge Infrastructure issues.

Security Zones

For known issues with Security Zones, see Known Issues.

Service Connector Hub

Multiple SMS Messages for a Single Notification

Details: A notification that exceeds the maximum characters supported by a single SMS message is split and sent as multiple SMS messages, which has billing implications.

This issue occurs when the Streaming source is specified for a service connector that uses the Notifications target, referencing a topic  that contains SMS subscriptions.

Workaround: We are aware of the issue and working on a resolution.

Direct link to this issue: Multiple SMS Messages for a Single Notification

Service Mesh

For known issues with Service Mesh, see Known Issues.

Speech

Currently, there are no known issues with the Speech service.

Storage Gateway

Exceptions to POSIX compliance

Details: The following file to object translations are not supported:

  • ACLs
  • Symlinks, hard links, named pipes, and special devices
  • Sticky bits

Workaround: If you need to copy special files to Object Storage, create a tar archive of the files.

Direct link to this issue: Exceptions to POSIX compliance

df command cannot report accurate size and capacity

Details: If you run the df command on a filesystem in an NFS client, df reports a filesystem size of 0 (zero) bytes and a capacity of 8 EB (maximum capacity). Because Object Storage does not have quotas and can store an unlimited amount of data, there is not a way to report filesystem size. Because the Object Storage bucket does not report storage usage, there is not a way to report capacity.

Workaround: You can run the du command to get usage, however this command is metadata intensive and takes longer to report usage. You could also list all objects in Object Storage and add the object size to determine current Object Storage usage. However, this method doesn’t take into account the amount of data stored in the filesystem cache. You can also explore out‑of‑band mechanisms that approximate storage usage.

Direct link to this issue: df command cannot report accurate size and capacity

Threat Intelligence

For known issues with Threat Intelligence, see Known Issues.

Traffic Management Steering Policies

Currently, there are no known Traffic Management Steering Policies issues.

Vault

Currently, there are no known Vault service issues.

Web Application Firewall (WAF)

Unable to add default origin to WAF policy created with the API

Details: When creating a WAF policy using the API, if you do not specify a default origin, you cannot add the default origin later using the Console or API. This issue does not apply to policies created using the Console.

Workaround: Delete the policy that was created without a default origin and create a new policy with the default origin specified.

Direct link to this issue: Unable to add default origin to WAF policy created with the API

TLS versions TLS_V1 and TLS_V1_1 have been deprecated

Details: TLS versions TLS_V1 and TLS_V1_1 have been deprecated and cannot be used in policy configurations. If you use these versions, a validation might occur.

Workaround: To work around this issue, update your policy configuration to use versions TLS_V1_2 or TLS_V1_3, or both.

Direct link to this issue: TLS versions TLS_V1 and TLS_V1_1 have been deprecated

Global DNS change will cause service disruption if new subnets are not whitelisted

Details: Global DNS changes will be made for all Oracle Web Application Firewall (WAF) customers beginning in December 2019. All customers that have an origin lock-down (using an explicit IP whitelisting) and will not whitelist the new subnets will have downtime and service degradation.

Workaround: (Action Required) Customers must whitelist the new subnets to avoid service disruption. For the API documentation, see ListEdgeSubnets.

OCI WAF Expansion Whitelist

130.35.0.0/20

130.35.128.0/20

130.35.240.0/20

138.1.32.0/21

138.1.128.0/19

147.154.96.0/19

192.29.96.0/20

130.35.16.0/20

130.35.48.0/20

130.35.64.0/19

130.35.96.0/20

130.35.120.0/21

130.35.144.0/20

130.35.176.0/20

130.35.192.0/19

130.35.224.0/22

130.35.232.0/21

138.1.48.0/21

147.154.0.0/18

147.154.64.0/20

147.154.80.0/21

130.35.112.0/22

138.1.16.0/20

138.1.80.0/20

138.1.208.0/20

138.1.224.0/19

147.154.224.0/19

138.1.0.0/20

138.1.40.0/21

138.1.64.0/20

138.1.96.0/21

138.1.104.0/22

138.1.160.0/19

138.1.192.0/20

147.154.128.0/18

147.154.192.0/20

147.154.208.0/21

192.29.0.0/20

192.29.64.0/20

192.29.128.0/21

192.29.144.0/21

192.29.16.0/21

192.29.32.0/21

192.29.48.0/21

192.29.56.0/21

Direct link to this issue: Global DNS change will cause service disruption if new subnets are not whitelisted