7 Troubleshoot

Identify common problems in Oracle WebLogic Server for OCI and learn how to diagnose and solve them.

Check Known Issues

Learn about known problems in Oracle WebLogic Server for OCI and how to work around them.

Clone Script Failed

Issue: When you run the clone script it might fail at a particular stage.

Workaround: By using the error message, you can identify the stage where the error occurred, fix the error, and then run the following command to continue the cloning script from the required stage.
python3 /opt/scripts/cloning/create_clone.py -p <stage_name>

AD Mismatch:

In an Availability Domain (AD), if the threshold of set limits are reached, then when you create the Cloning instance the Compute instances might be placed in another Availability Domain that does not match the Availability Domain of the Original instance. Due to this mismatch, the cloned volumes cannot be attached.

Workaround: Use the Oracle WebLogic Server for OCI console to create backup of data volumes, and optionally, the Middleware volumes. Make a note of the OCIDs of backed up volumes and then complete the steps in Method 2: Manually create cloned volumes and destroy the source stack.

Cleanup Resources of a Deleted Instance

If you create an Oracle WebLogic Server for OCI instance and delete some resources outside of terraform, the terraform destroy may fail.

Issue:

You might have deleted an instance without destroying the instance. In this scenario, some of the resources you had created for the instance are not deleted.

Workaround:

In such scenarios, you can run the following script to remove all the resources of the instance.

  1. Copy the following script in Cloud Shell.

    For example, copy the script and save the file as remove_resources.py.

    """
    #
    # Copyright (c) 2021, Oracle Corporation and/or its affiliates.
    # Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.
    """
    
    import os
    import sys
    import oci
    
    """
    Lists and deletes the resources like instances, policies, volumes, VCN related resources, logs and tags etc..
    """
    
    
    class CleanUpResources:
    
        def __init__(self):
            # delegate token should be present at /etc/oci/delegation_token in cloud shell
            if os.path.exists('/etc/oci/delegation_token'):
                with open('/etc/oci/delegation_token', 'r') as file:
                    delegation_token = file.read()
                self.signer = oci.auth.signers.InstancePrincipalsDelegationTokenSigner(delegation_token=delegation_token)
            else:
                print("ERROR: In the Cloud shell the delegation token does not exist at location /etc/oci/delegation_token."
                      "Run the script from the Cloud shell, where you need to delete the resources.")
                sys.exit(1)
            self.vcn_client = oci.core.VirtualNetworkClient(config={}, signer=self.signer)
            self.virtual_network_composite_operations = oci.core.VirtualNetworkClientCompositeOperations(self.vcn_client)
            self.log_client = oci.logging.LoggingManagementClient(config={}, signer=self.signer)
            self.log_composite_operations = oci.logging.LoggingManagementClientCompositeOperations(self.log_client)
            self.identity_client = oci.identity.IdentityClient(config={}, signer=self.signer)
            self.identity_client_composite_operations = oci.identity.IdentityClientCompositeOperations(self.identity_client)
    
        # Lists all the resources based on the service name prefix
        def list_all_resources(self, service_name_prefix):
            search_client = oci.resource_search.ResourceSearchClient(config={}, signer=self.signer)
            running_resources = ["RUNNING", "Running", "AVAILABLE", "STOPPED", "Stopped", "ACTIVE", "CREATED", "INACTIVE"]
            resource_not_required = ["PrivateIp", "Vnic"]
            structured_search = oci.resource_search.models.StructuredSearchDetails(
                query="query all resources where displayname =~ '{}'".format(service_name_prefix),
                type='Structured',
                matching_context_type=oci.resource_search.models.SearchDetails.MATCHING_CONTEXT_TYPE_NONE)
    
            resources = search_client.search_resources(structured_search)
            resources_details = []
            no_of_resources = 0
            tagname_resource = "wlsoci-" + service_name_prefix
            default_rt = "Default Route Table for " + service_name_prefix
            print(
                "Resource Name                              Resource Type                        Resource Lifecycle State                 OCID         DOC")
            print(
                "=================================================================================================================================================")
            for resource in resources.data.items:
                resource_name = resource.display_name
                if (resource_name.startswith(service_name_prefix) or tagname_resource in resource_name or default_rt in resource_name) and (
                        resource.lifecycle_state in running_resources) and (
                        resource.resource_type not in resource_not_required):
                    resources_details.append(resource)
                    no_of_resources = no_of_resources + 1
                    print("{}             {}          {}          {}           {}".format(resource.display_name,
                                                                                          resource.resource_type,
                                                                                          resource.lifecycle_state,
                                                                                          resource.identifier,
                                                                                          resource.time_created))
            print(
                "================================================================================================================================================")
            print("Total number of resources {}".format(len(resources_details)))
            return resources_details
    
        # Removes all resources based on the service name prefix
        def cleanup_resources(self, delete_list):
            print("Deleting the resources")
            self.delete_policies(delete_list)
            self.delete_instance(delete_list)
            self.delete_block_volumes(delete_list)
            self.delete_load_balancer(delete_list)
            self.delete_subnet(delete_list)
            self.delete_sec_list(delete_list)
            self.delete_route_table(delete_list)
            self.delete_dhcp_options(delete_list)
            self.delete_internet_gateway(delete_list)
            self.delete_service_gateway(delete_list)
            self.delete_local_peering_gateway(delete_list)
            self.delete_nat_gateway(delete_list)
            self.delete_vcn_resources(delete_list)
            self.delete_unified_agent_configuration(delete_list)
            self.delete_log(delete_list)
            self.delete_log_group(delete_list)
            self.delete_mount_targets(delete_list)
            self.delete_fss(delete_list)
            self.delete_tag_namespace(delete_list)
            self.delete_boot_volumes(delete_list)
    
        # Delete Policies
        def delete_policies(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "Policy":
                    policy_ocid = resource.identifier
                    print("Deleting policy: {0}, with ocid: {1}".format(resource.display_name, policy_ocid))
                    try:
                        self.identity_client_composite_operations.delete_policy_and_wait_for_state(
                            policy_ocid,
                            wait_for_states=[oci.identity.models.Policy.LIFECYCLE_STATE_DELETED])
                        print("Deleted policy successfully!")
                    except Exception as e:
                        print("Error while deleting the policy {0}, policy id {1}, Error message {2}".format(
                            resource.display_name, policy_ocid, str(e)))
    
        # Delete Dynamic Group
        def delete_dynamic_group(self, service_name_prefix):
            tenancy = os.environ['OCI_TENANCY']
            dynamic_group_list = self.identity_client.list_dynamic_groups(tenancy).data
            for d_group in dynamic_group_list:
                if service_name_prefix in d_group.name:
                    print("Deleting the dynamic group: {0}, with ocid: {1}".format(d_group.name, d_group.id))
                    try:
                        self.identity_client_composite_operations.delete_dynamic_group_and_wait_for_state(
                            d_group.id, wait_for_states=[oci.identity.models.DynamicGroup.LIFECYCLE_STATE_DELETED])
                        print("Deleted the dynamic group successfully!")
                    except Exception as e:
                        print("Error while deleting the dynamic group name {}, ocid {}, Error message {}".format(
                            d_group.name, d_group.id, str(e)))
    
        # Delete Block Volumes
        def delete_block_volumes(self, delete_list):
            bv_client = oci.core.BlockstorageClient(config={}, signer=self.signer)
            bv_composite_operations = oci.core.BlockstorageClientCompositeOperations(bv_client)
            for resource in delete_list:
                if resource.resource_type == "Volume":
                    bv_ocid = resource.identifier
                    try:
                        print(
                            "Deleting the block volume: {0}, with ocid {1}".format(resource.display_name, bv_ocid))
                        bv_composite_operations.delete_volume_and_wait_for_state(
                            bv_ocid, wait_for_states=[oci.core.models.Volume.LIFECYCLE_STATE_TERMINATED])
                        print("Deleted the block volume successfully!")
                    except Exception as e:
                        print(
                            "Error while deleting the block volume {0}, ocid {1}, Error message {2}".format(
                                resource.display_name, bv_ocid, str(e)))
    
        # Delete all compute instances
        def delete_instance(self, delete_list):
            compute_client = oci.core.ComputeClient(config={}, signer=self.signer)
            compute_composite_operations = oci.core.ComputeClientCompositeOperations(compute_client)
            for resource in delete_list:
                if resource.resource_type == "Instance":
                    instance_ocid = resource.identifier
                    instance_name = resource.display_name
                    print("Deleting the compute instance: {0}, with ocid {1}".format(instance_name, instance_ocid))
                    try:
                        compute_composite_operations.terminate_instance_and_wait_for_state(
                            instance_ocid, wait_for_states=[oci.core.models.Instance.LIFECYCLE_STATE_TERMINATED])
                        print("Deleted the compute instance successfully!")
                    except Exception as e:
                        print(
                            "Error while deleting the instance {0}, ocid {1}, Error message {2}".format(
                                instance_name, instance_ocid, str(e)))
    
        # Delete all Subnets in the VCN
        def delete_subnet(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "Subnet":
                    subnet_ocid = resource.identifier
                    print(
                        "Deleting subnet: {0}, with ocid {1}".format(resource.display_name, resource.identifier))
                    try:
                        self.virtual_network_composite_operations.delete_subnet_and_wait_for_state(
                            subnet_ocid,
                            wait_for_states=[oci.core.models.Subnet.LIFECYCLE_STATE_TERMINATED])
                        print("Deleted subnet successfully!")
                    except Exception as e:
                        print("Error while deleting the subnet {0}, ocid {1}, Error message {2}".format(resource.display_name,
                                                                                                  subnet_ocid, str(e)))
    
        # Delete Security lists
        def delete_sec_list(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "SecurityList":
                    sec_list_name = resource.display_name
                    sec_list_ocid = resource.identifier
                    if not ("Default" in sec_list_name):
                        print(
                            "Deleting the security list: {0}, with ocid {1}".format(resource.display_name,
                                                                                   resource.identifier))
                        try:
                            self.virtual_network_composite_operations.delete_security_list_and_wait_for_state(
                                sec_list_ocid,
                                wait_for_states=[oci.core.models.SecurityList.LIFECYCLE_STATE_TERMINATED])
                            print("Deleted the security list successfully!")
                        except Exception as e:
                            print(
                                "Error while deleting the security list {0}, ocid {1}, Error message {2}".format(
                                    resource.display_name, sec_list_ocid, str(e)))
    
        # Delete Load balancers
        def delete_load_balancer(self, delete_list):
            lb_client = oci.load_balancer.LoadBalancerClient(config={}, signer=self.signer)
            lb_composite_operations = oci.load_balancer.LoadBalancerClientCompositeOperations(lb_client)
            for resource in delete_list:
                if resource.resource_type == "LoadBalancer":
                    lb_name = resource.display_name
                    lb_ocid = resource.identifier
                    print("Deleting Load balancer {0} with ocid {1}".format(lb_name, lb_ocid))
                    try:
                        lb_composite_operations.delete_load_balancer_and_wait_for_state(
                            lb_ocid,
                            wait_for_states=[oci.load_balancer.models.WorkRequest.LIFECYCLE_STATE_SUCCEEDED])
                        print("Load balancer deleted successfully!")
                    except Exception as e:
                        print(
                            "Error while deleting the loadbalancer {0}, ocid {1}, Error message {2}".format(
                                lb_name, lb_ocid, str(e)))
    
        # Delete Route tables
        def delete_route_table(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "RouteTable":
                    route_table_name = resource.display_name
                    route_table_ocid = resource.identifier
                    # Removing the route rules from the tables
                    rt_details = oci.core.models.UpdateRouteTableDetails()
                    rt_details.route_rules = []
                    self.virtual_network_composite_operations.update_route_table_and_wait_for_state(
                        route_table_ocid, rt_details,
                        wait_for_states=[oci.core.models.RouteTable.LIFECYCLE_STATE_AVAILABLE])
                    self.vcn_client.update_route_table(route_table_ocid, rt_details)
                    # Default route table can't be deleted from VCN
                    if not ("Default" in route_table_name):
                        print(
                            "Deleting the route table: {0}, with ocid {1}".format(resource.display_name,
                                                                                 resource.identifier))
                        try:
                            self.virtual_network_composite_operations.delete_route_table_and_wait_for_state(
                                route_table_ocid,
                                wait_for_states=[oci.core.models.RouteTable.LIFECYCLE_STATE_TERMINATED])
    
                            print("Deleted the route table successfully!")
                        except Exception as e:
                            print("Error while deleting the route table {0}, ocid {1}, Error message {2}".format(
                                resource.display_name, route_table_ocid, str(e)))
                            if "associated with Subnet" in str(e):
                                try:
                                    self.delete_subnet_route_table_association(route_table_ocid)
                                    # After removing the association again retrying the removal of route table
                                    # This is for Db subnet route table
                                    self.virtual_network_composite_operations.delete_route_table_and_wait_for_state(
                                        route_table_ocid,
                                        wait_for_states=[oci.core.models.RouteTable.LIFECYCLE_STATE_TERMINATED])
                                    print("Deleted the route table successfully!")
                                except Exception as e:
                                    print("Error while deleting the route table after removing the association "
                                          "{0}, ocid {1}, Error message {2}".format
                                          (resource.display_name, route_table_ocid, str(e)))
    
        # Delete Subnet and route table association to remove route table
        def delete_subnet_route_table_association(self, route_table_ocid):
            default_rt_id_in_vcn = ""
            print("Route table is associated with a subnet. Removing the association between the subnet and route table")
            rt_res = self.vcn_client.get_route_table(route_table_ocid).data
            vcn_id = rt_res.vcn_id
            compartment_id = rt_res.compartment_id
            list_route_rables_vcn = self.vcn_client.list_route_tables(compartment_id=compartment_id,
                                                                      vcn_id=vcn_id).data
            for rt in list_route_rables_vcn:
                if "Default Route" in rt.display_name:
                    default_rt_id_in_vcn = rt.id
            list_subnets = self.vcn_client.list_subnets(compartment_id=compartment_id, vcn_id=vcn_id).data
            for subnet in list_subnets:
                subnet_ocid = subnet.id
                if subnet.route_table_id == route_table_ocid:
                    subnet_details = oci.core.models.UpdateSubnetDetails()
                    subnet_details.route_table_id = default_rt_id_in_vcn
                    try:
                        self.virtual_network_composite_operations.update_subnet_and_wait_for_state(
                            subnet_ocid, subnet_details,
                            wait_for_states=[oci.core.models.Subnet.LIFECYCLE_STATE_AVAILABLE])
                        print("Removed the association between the subnet and route table.")
                    except Exception as e:
                        print("Error while removing the association between the subnet and route table {}".format(str(e)))
    
        # Delete DHCP Options
        def delete_dhcp_options(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "DHCPOptions":
                    dhcp_name = resource.display_name
                    dhcp_ocid = resource.identifier
                    if not ("Default" in dhcp_name):
                        print(
                            "Deleting the DHCP options: {0}, with ocid {1}".format(resource.display_name, dhcp_ocid))
                        try:
                            self.virtual_network_composite_operations.delete_dhcp_options_and_wait_for_state(dhcp_ocid,
                                                                                                             wait_for_states=[
                                                                                                                 oci.core.models.DhcpOptions.LIFECYCLE_STATE_TERMINATED])
                            print("Deleted the DHCP options successfully!")
                        except Exception as e:
                            print(
                                "Error while deleting the DHCP options {0}, ocid {1}, Error message {2} ".format(
                                    resource.display_name, dhcp_ocid, str(e)))
    
        # Delete Internet Gateway
        def delete_internet_gateway(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "InternetGateway":
                    ig_ocid = resource.identifier
                    print("Deleting the Internet Gateway: {0}, with ocid {1}".format(resource.display_name,
                                                                                    ig_ocid))
                    try:
                        self.virtual_network_composite_operations.delete_internet_gateway_and_wait_for_state(ig_ocid,
                                                                                                             wait_for_states=[
                                                                                                                 oci.core.models.InternetGateway.LIFECYCLE_STATE_TERMINATED])
    
                        print("Deleted the Internet Gateway successfully!")
                    except Exception as e:
                        print("Error while deleting the Internet Gateway {0}, ocid {1}, Error message {2}".format(
                            resource.display_name,
                            ig_ocid, str(e)))
    
        # Delete Service Gateway
        def delete_service_gateway(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "ServiceGateway":
                    svc_gateway_ocid = resource.identifier
                    print("Deleting the service gateway: {0}, with ocid {1}".format(resource.display_name,
                                                                                   svc_gateway_ocid))
                    try:
                        self.virtual_network_composite_operations.delete_service_gateway_and_wait_for_state(
                            svc_gateway_ocid, wait_for_states=[oci.core.models.ServiceGateway.LIFECYCLE_STATE_TERMINATED])
    
                        print("Deleted the service gateway successfully!")
                    except Exception as e:
                        print("Error while deleting the service gateway {0}, ocid {1}, Error message {2}".format(
                            resource.display_name,
                            svc_gateway_ocid, str(e)))
    
        # Delete Local Peering Gateway
        def delete_local_peering_gateway(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "LocalPeeringGateway":
                    lpg_ocid = resource.identifier
                    print("Deleting the local peering gateway: {0}, with ocid {1}".format(resource.display_name,
                                                                                         lpg_ocid))
                    try:
                        self.virtual_network_composite_operations.delete_local_peering_gateway_and_wait_for_state(
                            lpg_ocid, wait_for_states=[oci.core.models.LocalPeeringGateway.LIFECYCLE_STATE_TERMINATED])
    
                        print("Deleted local peering gateway successfully!")
                    except Exception as e:
                        print("Error while deleting the local peering gateway {0}, ocid {1}, Error message {2}".format(
                            resource.display_name, lpg_ocid, str(e)))
    
        # Delete Nat Gateway
        def delete_nat_gateway(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "NatGateway":
                    nat_ocid = resource.identifier
                    print("Deleting the NAT gateway: {0}, with ocid {1}".format(resource.display_name,
                                                                               nat_ocid))
                    try:
                        self.virtual_network_composite_operations.delete_nat_gateway_and_wait_for_state(
                            nat_gateway_id=nat_ocid,
                            wait_for_states=[oci.core.models.NatGateway.LIFECYCLE_STATE_TERMINATED]
                        )
                        print("Deleted the NAT gateway successfully!")
                    except Exception as e:
                        print("Error while deleting the NAT gateway {0}, ocid {1}, Error message {2}".format(
                            resource.display_name, nat_ocid, str(e)))
    
        # Delete VCN
        def delete_vcn_resources(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "Vcn":
                    vcn_ocid = resource.identifier
                    vcn_name = resource.display_name
                    print("Deleting the VCN: {0}, with ocid {1}".format(vcn_name, vcn_ocid))
                    try:
                        self.virtual_network_composite_operations.delete_vcn_and_wait_for_state(vcn_ocid,
                                                                                                oci.core.models.Vcn.LIFECYCLE_STATE_TERMINATED)
                        print("Deleted the VCN successfully!")
                    except Exception as e:
                        print("Error while deleting the VCN {0}, VCN id {1}, Error message {2}".format(vcn_name, vcn_ocid,
                                                                                                      str(e)))
    
        # Deleting the Unified Agent Configuration
        def delete_unified_agent_configuration(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "UnifiedAgentConfiguration":
                    uac_ocid = resource.identifier
                    print("Deleting the unified agent configuration: {}, with ocid {}".format(resource.display_name, uac_ocid))
                    try:
                        self.log_composite_operations.delete_unified_agent_configuration_and_wait_for_state(
                            uac_ocid,
                            wait_for_states=[oci.logging.models.WorkRequest.STATUS_SUCCEEDED])
                        print("Deleted the unified agent configuration successfully!")
                    except Exception as e:
                        print("Error while deleting the unified agent configuration name {0}, ocid {1} - Error message {2}".format(
                            resource.display_name, uac_ocid, str(e)))
    
        # Delete logs in a Log groups
        def delete_log(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "LogGroup":
                    log_group_ocid = resource.identifier
                    list_logs = self.log_client.list_logs(log_group_ocid).data
                    for log in list_logs:
                        print("Deleting the log name {0}, with log ocid {1}".format(log.display_name, log.id))
                        try:
                            self.log_composite_operations.delete_log_and_wait_for_state(
                                log_group_ocid, log.id, wait_for_states=[oci.logging.models.WorkRequest.STATUS_SUCCEEDED])
                            print("Deleted the log successfully!")
                        except Exception as e:
                            print("Error while deleting the log name {}, log ocid {}, Error message {}".format(
                                log.display_name, log.id, str(e)))
    
        # Delete Log Group
        def delete_log_group(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "LogGroup":
                    log_group_ocid = resource.identifier
                    print("Deleting the log group: {0}, with ocid {1}".format(resource.display_name,
                                                                             log_group_ocid))
                    try:
                        self.log_composite_operations.delete_log_group_and_wait_for_state(
                            log_group_ocid, wait_for_states=[oci.logging.models.WorkRequest.STATUS_SUCCEEDED])
    
                        print("Deleted log group successfully!")
                    except Exception as e:
                        print("Error while deleting the log group {0}, ocid {1}, Error message {2}".format(
                            resource.display_name, log_group_ocid, str(e)))
    
        # Delete the Mount targets
        def delete_mount_targets(self, delete_list):
            mt_client = oci.file_storage.FileStorageClient(config={}, signer=self.signer)
            mt_composite_operations = oci.file_storage.FileStorageClientCompositeOperations(mt_client)
            for resource in delete_list:
                if resource.resource_type == "MountTarget":
                    mt_ocid = resource.identifier
                    print("Deleting the mount target {0}, with ocid {1}".format(resource.display_name, mt_ocid))
                    try:
                        mt_composite_operations.delete_mount_target_and_wait_for_state(
                            mt_ocid, wait_for_states=[oci.file_storage.models.MountTarget.LIFECYCLE_STATE_DELETED])
                        print("Deleted the mount target successfully!")
                    except Exception as e:
                        print("Error while deleting the mount target {0}, ocid {1}, Error message {2}".format(
                            resource.display_name, mt_ocid, str(e)))
    
        # Delete FSS
        def delete_fss(self, delete_list):
            fss_client = oci.file_storage.FileStorageClient(config={}, signer=self.signer)
            fss_composite_operations = oci.file_storage.FileStorageClientCompositeOperations(fss_client)
            for resource in delete_list:
                if resource.resource_type == "FileSystem":
                    fss_ocid = resource.identifier
                    try:
                        # Get the list of exports to delete
                        list_exports = fss_client.list_exports(file_system_id=fss_ocid).data
                        for export in list_exports:
                            export_ocid = export.id
                            print("Deleting the export id {}".format(export_ocid))
                            fss_composite_operations.delete_export_and_wait_for_state(
                                export_id=export_ocid,
                                wait_for_states=[oci.file_storage.models.Export.LIFECYCLE_STATE_DELETED])
                            print("Deleted the exports successfully!")
                    except Exception as e:
                        print("Error while deleting the export, Error message {}".format(str(e)))
                    try:
                        print("Deleting the FSS: {0}, with ocid {1}".format(resource.display_name, fss_ocid))
                        fss_composite_operations.delete_file_system_and_wait_for_state(
                            fss_ocid, wait_for_states=[oci.file_storage.models.FileSystem.LIFECYCLE_STATE_DELETED])
                        print("Deleted the FSS successfully!")
                    except Exception as e:
                        print("Error while deleting the FSS name {0}, ocid {1}, Error message {2}".format(
                            resource.display_name, fss_ocid, str(e)))
    
        # Deletion of TagNamespace
        def delete_tag_namespace(self, delete_list):
            for resource in delete_list:
                if resource.resource_type == "TagNamespace":
                    tag_ns_name = resource.display_name
                    tag_ns_ocid = resource.identifier
                    print("Deleting the tag namespace {0}, with ocid {1}".format(tag_ns_name, tag_ns_ocid))
                    try:
                        # Retiring the tag namespace
                        tag_status = self.identity_client.get_tag_namespace(tag_namespace_id=tag_ns_ocid).data
                        print("Tag namespace: {} and isRetired: {}".format(tag_ns_name, tag_status.is_retired))
    
                        if not tag_status.is_retired:
                            print("Retiring the tag namespace {}".format(tag_ns_name))
                            tag_ns_details = oci.identity.models.UpdateTagNamespaceDetails()
                            tag_ns_details.is_retired = True
                            self.identity_client_composite_operations.update_tag_namespace_and_wait_for_state(
                                tag_namespace_id=tag_ns_ocid,
                                update_tag_namespace_details=tag_ns_details,
                                wait_for_states=[
                                    oci.identity.models.TagNamespace.LIFECYCLE_STATE_INACTIVE])
                            tag_status = self.identity_client.get_tag_namespace(tag_namespace_id=tag_ns_ocid).data
                            print("Tag status before deleting {}".format(tag_status.is_retired))
                        print("Deleting the tag namespace {}".format(tag_ns_name))
                        # Tag namespace deletion is taking too long time. So not waiting for the completion.
                        self.identity_client.cascade_delete_tag_namespace(tag_namespace_id=tag_ns_ocid)
                        print("Asynchronous deletion of Tag namespaces is enabled."
                              "Check the deletion status manually. Tag name {0} with ocid {1}".format(tag_ns_name,
                                                                                                     tag_ns_ocid))
                    except Exception as e:
                        print("Error while deleting the Tag namespace {0}, ocid {1}, Error message {2} "
                              .format(tag_ns_name, tag_ns_ocid, str(e)))
    
        # Deleting the unattached boot volumes
        def delete_boot_volumes(self, delete_list):
            bv_client = oci.core.BlockstorageClient(config={}, signer=self.signer)
            bv_composite_operations = oci.core.BlockstorageClientCompositeOperations(bv_client)
            for resource in delete_list:
                if resource.resource_type == "BootVolume" and resource.lifecycle_state == "AVAILABLE":
                    bv_ocid = resource.identifier
                    bv_name = resource.display_name
                    print("Deleting the boot volume {}, with ocid {} ".format(bv_name, bv_ocid))
                    try:
                        bv_composite_operations.delete_boot_volume_and_wait_for_state(
                            boot_volume_id=bv_ocid,
                            wait_for_states=[oci.core.models.BootVolume.LIFECYCLE_STATE_TERMINATED])
                        print("Deleted the boot volume successfully!")
                    except Exception as e:
                        print("Error while deleting the boot volume name {}, ocid {}, Error message {}".format(bv_name,
                                                                                                               bv_ocid,
                                                                                                               str(e)))
    
    
    if __name__ == '__main__':
        no_of_args = len(sys.argv)
        if no_of_args < 2:
            print("Usage: ")
            print("To list all the resources based on service name prefix:")
            print("python3 remove_resources.py <service_name_prefix>")
            print("To remove all the resources based on service name prefix:")
            print("python3 remove_resources.py <service_name_prefix> delete")
            sys.exit(1)
    
        service_prefix = sys.argv[1]
        print("Service prefix name:" + service_prefix)
        cleanup_util = CleanUpResources()
        if len(service_prefix) >= 16:
            service_prefix = service_prefix[0:16]
        service_prefix = service_prefix + "-"
        if no_of_args < 3:
            print("Listing all resources with service prefix name" + service_prefix)
            cleanup_resources = cleanup_util.list_all_resources(service_prefix)
        elif no_of_args < 4 and sys.argv[2] == "delete":
            print("Deleting all resources with service prefix name" + service_prefix)
            cleanup_resources = cleanup_util.list_all_resources(service_prefix)
            cleanup_util.cleanup_resources(cleanup_resources)
            cleanup_util.delete_dynamic_group(service_prefix)
    
  2. Run the following command to list all the resources:
    python3 remove_resources.py <full_service_prefix_name>
  3. Check that list and ensure that you want to delete these resources.
  4. Run the following command to deleted all the resources of the instance:
    python3 remove_resources.py <full_service_prefix_name> delete

Stack Creation Failed

Troubleshoot a failed Oracle WebLogic Server domain that you attempted to create with Oracle WebLogic Server for OCI.

View the stack log files

Use the Terraform job logs in Resource Manager to identify the cause of the failure.

  1. Click the navigation menu Navigation Menu icon, select Developer Services. Under the Resource Manager group, click Jobs.
  2. Identify and click the job for your stack.
    • The Type is Apply.
    • The State is Failed.
    • The Stack is the name of your Oracle WebLogic Server for OCI stack.
  3. From the Logs section, search the log for error messages.

    You can optionally Download the log files and search the files offline.

  4. See below for details about specific error messages.

Modify the stack configuration

If necessary, delete the current stack resources, modify your stack configuration, and then apply the changes.

  1. Click the navigation menu Navigation Menu icon, select Developer Services. Under the Resource Manager group, click Stacks.
  2. Click the name of your stack.
  3. Click Terraform Actions and select Destroy.

    Wait for the destroy job to complete.

  4. Click Edit Stack.
  5. When done, click Save Changes.
  6. Click Terraform Actions and select Apply.

Cannot launch a stack in Marketplace

Example message: Unable to accept Terms of Use

In Marketplace, you might see the message when you click Launch Stack, after you've selected a stack version and compartment, and checked the Oracle Standard Terms and Restrictions box.

You likely don't have permission to:

  • Create Marketplace applications in the selected compartment. Verify that this policy exists in the compartment where you want to create the stack.

    Allow group Your_Group to manage app-catalog-listing in compartment Your_Compartment

  • Access the selected compartment. Choose another compartment or ask your administrator.

Cannot determine home region

Example message:

data.oci_core_app_catalog_subscriptions.mp_image_subscription[0]: Refreshing state...
Error: Null value found in list ... "oci_identity_regions" "home-region"

If you are not an administrator, ask them to verify that the following root-level policy exists in your tenancy:

Allow group Your_Group to inspect tenancies in tenancy

Cannot find dynamic group and secrets policy

Example messages:

Error: Service error:NotAuthorizedOrNotFound. Authorization failed or requested resource not found. http status code: 404.
 Opc request id: request_id on modules/policies/groups.tf line 8, in resource...
 "oci_identity_dynamic_group" "wlsc_instance_principal_group" {
Error: Service error:NotAuthorizedOrNotFound. Authorization failed or requested resource not found. http status code: 404.
 Opc request id: request_id on wlsc-policies.tf line 10, in resource...
 "oci_identity_policy" "wlsc_secret-service-policy" {

When the OCI Policies check box is selected (by default), Oracle WebLogic Server for OCI creates a dynamic group and one or more root-level policies in your tenancy.

You must be an Oracle Cloud Infrastructure administrator, or be granted root-level permissions to create domains. If you are not an administrator, ask them to verify that root-level policies exist in your tenancy. For example:

Allow group Your_Group to manage dynamic-groups in tenancy
Allow group Your_Group to manage policies in tenancy
Allow group Your_Group to use secret-family in tenancy

See:

Maximum number of dynamic groups has exceeded

Example message:

<WLSC-VM-ERROR-0119> : Failed to get secret content for [ocid1.vaultsecret.oc1.iad.alongstring123]: [{'status': 400, 'message': "This instance principal matches more than '5' dynamic groups, update your dynamic groups' matching rules"...'}]>

When the OCI Policies check box is selected (by default), Oracle WebLogic Server for OCI creates a dynamic group and one or more root-level policies in your tenancy. The maximum number of dynamic groups allowed is 5.

Solution:

  1. Add the following policy that uses the existing dynamic group to access the new secrets for the new stack:
    Allow dynamic-group <existing-dyanmic-group-name> to read secret-bundles in tenancy where target.secret.id = '<OCID_of_the_secret>'
  2. Deselect the OCI Policies check box and try to create the stack again.

Unable to get secret content or decrypted credential

Example messages:

  • Failed to get secret content for Your_vault_secret_OCID
  • Authorization failed or requested resource not found
  • Error retrieving %s password from Secret Vault
  • Failed in create domain due to exception [Failed to retrieve WebLogic Password from Secrets Vault]
  • Failed to retrieve IDCS Client Secret from Secrets Vault
  • Unable to get decrypt credential
  • Key or Vault does not exist or you are not authorized to access them.

When you create a domain with Oracle WebLogic Server for OCI, you provide the OCID values of the secrets that contain the passwords to use for the domain and during provisioning. The compute instances use this information to decrypt the passwords. The compute instances are granted access to vault secrets using policies.

You must be an Oracle Cloud Infrastructure administrator, or be granted root-level permissions to create domains. If you are not an administrator, ask them to verify that relevant vault secret policies exist in your tenancy and compartment. For example:

Allow group Your_Group to use secret-family in tenancy
Allow dynamic-group Your_DynamicGroup to use secret-family in compartment MyCompartment
Allow dynamic-group Your_DynamicGroup to use keys in compartment MyCompartment
Allow dynamic-group Your_DynamicGroup to use vaults in compartment MyCompartment

If the policies exist, check that the OCID of the compartment in listed in dynamic group.

See:

Unable to get decrypted credential when creating a stack in a private subnet

Example message: <WLSC-VM-ERROR-001> Unable to get decrypt credential [HTTPSConnectionPool(host='auth.us-phoenix-1.oraclecloud.com', port=443): Max retries exceeded with url: /v1/x509 (Caused by ConnectTimeoutError(<oci._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x1e5110>, 'Connection to auth.us-phoenix-1.oraclecloud.com timed out. (connect timeout=10)'))]>

When you create a domain with Oracle WebLogic Server for OCI in an existing private subnet, provisioning fails if the WebLogic Server subnet is using a route table that does not include a service gateway or a Network Address Translation (NAT) gateway.

Modify the private subnet, and select a route table that uses a service gateway or NAT gateway. Or select a virtual cloud network (VCN) whose default route table uses a service gateway or NAT gateway. Refer to these topics:

Failed to download Oracle Autonomous Database wallet

Example message: module.provisioners.null_resource.status_check[0] (remote-exec): <Nov 23, 2019 09:37:17 PM GMT> <ERROR> <oci_api_utils> <(host:stackname-wls-0.subnetxxx.stacknamevcn.oraclevcn.com) - <WLSC-VM-ERROR-0052> : Unable to download atp wallet. [{'status': 403, 'message': u'Forbidden', 'code': u'Forbidden', 'opc-request-id': 'FA6C16D8B'}]

You must be an Oracle Cloud Infrastructure administrator, or be granted root-level and compartment-level permissions to create domains. Access to the database wallet is needed when you create a JRF-enabled domain that uses an autonomous database. If you are not an administrator, ask them to verify that relevant policies for autonomous databases exist in your tenancy and compartment. For example:

Allow group Your_Group to inspect autonomous-transaction-processing-family in compartment Your_ATP_Compartment
Allow dynamic-group Your_DynamicGroup to inspect autonomous-transaction-processing-family in compartment Your_ATP_Compartment

See:

Failed to validate DB connectivity

When you create a domain that includes the Java Required Files (JRF) components, you must select an existing database and provide connection details. The compute instances use this information to connect to the database and provision the JRF database schemas.

Possible causes for this error include:

  • You entered the wrong database password or a plain text password.
  • The database does not allow the compute instances to access its listen port (1521 by default).
    • Oracle Autonomous Database - Check your access control list (ACL).
    • Oracle Cloud Infrastructure Database - Check the network security group that was assigned to the database, and the security lists for the subnet on which the database was created.
  • You selected an Oracle Cloud Infrastructure Database running Oracle Database 12c or later, and you did not provide the name of a pluggable database (PDB).

Invalid or overlapping network CIDR

Stack provisioning fails if you specify subnets with overlapping CIDRs or use the same subnet for WebLogic Server and the load balancer.

Example messages:

Error: module.network-wls-public-subnet.oci_core_subnet.wls-subnet: 1 error(s) occurred: oci_core_subnet.wls-subnet: Service error:InvalidParameter. The requested CIDR 10.0.3.0/24 is invalid: subnet ocid1.subnet.oc1.iad.aaan4a with CIDR 10.0.3.0/24 overlaps with this CIDR.. http status code: 400.

Error: module.validators.null_resource.duplicate_lb2_subnet_cidr: : invalid or unknown key: WLSC-ERROR: Load balancer subnet 2 CIDR has to be unique value.

Error: module.validators.null_resource.duplicate_wls_subnet_cidr: : invalid or unknown key: WLSC-ERROR: Weblogic subnet CIDR has to be unique value.

Possible causes for these errors include:

  • You chose to create new subnets for WebLogic Server, the load balancer, or the bastion, and the CIDR you specified for these subnets overlaps with the CIDRs for existing subnets in the same virtual cloud network (VCN).
  • You chose to use an existing subnet when provisioning a stack with a load balancer, and you specified the same subnet for WebLogic Server and the load balancer.
  • You created a JRF-enabled domain, your Oracle Cloud Infrastructure Database and WebLogic domain are in different VCNs, and the VCNs have overlapping CIDRs. For example, you cannot create a WebLogic domain on VCN 10.0.0.0/16 that uses a database on VCN 10.0.0.1/24.

Job is still running or has timed out

Most stack creation jobs for Oracle WebLogic Server for OCI should complete within an hour. Some internal provisioning problems might cause the job to run indefinitely until it eventually times out after 24 hours.

After the current Apply job times out, run a new Apply job on the same stack. This will destroy any resources that were created, and then attempt to create the resources again. If the problem occurs again, contact support.

Failed to check database port is open for Exadata DB system

When you create a domain that includes Java Required Files (JRF) components, for Exadata DB systems, the database port open check does not work if the Create DB Security List checkbox is selected. In this case, the provisioning fails if the database subnet has more than five security lists.

So, when provisioning, deselect the Create DB Security List check box to avoid creating an additional security list for the database port in the VCN, and manually open the database port (1521 by default).

Error in Mounting the Volume During Provisioning

Troubleshoot a failed Oracle WebLogic Server for OCI domain due to Oracle Cloud Infrastructure (OCI) policy not in effect.

Issue:

When you create a domain with the OCI Policies check box selected, at times, the policy creation takes longer time than it takes to reach the first OCI API call, that is, the call to create a volume mount, and an error message is displayed in the bootstrap log.

Example message:

Exception stacktrace [
{'opc-request-id': '<GUID>', 'code': 'NotAuthorizedOrNotFound', 'message': 'Authorization failed or requested resource not found.', 'status': 404} 
]
Error executing volume mounting.. Exiting provisioning

Workaround:

  1. Create a dynamic group using the following rule:

    All { instance.compartment.id = '<resource_compartment ocid>') }

    To create a dynamic group, see Create a Dynamic Group.

  2. Create the policy for the dynamic group as follows:

    Example policy:

    Allow dynamic-group MyInstancesPrincipalGroup to use volumes in compartment MyCompartment where any { request.operation = 'AttachVolume', request.operation = 'DetachVolume'}" 

    To create other policies for the dynamic group, see Create Policies for the Dynamic Group.

  3. Wait for 10 to 15 minutes and then create a domain without selecting the OCI Policies check box.
  4. Delete the dynamic group and policy created manually after the domain is created.

Unable to Access the Domain

Troubleshoot problems accessing an Oracle WebLogic Server domain after it's successfully created.

Cannot access the WebLogic Console from the Internet

By default the WebLogic Server Administration Console is accessed through port 7001 or 7002.

To check port access:

  1. Access the Oracle Cloud Infrastructure console.
  2. From the navigation menu, select Networking, and then click Virtual Cloud Networks.
  3. Select the compartment in which you created the domain.
  4. Select the virtual cloud network in which the domain was created.
  5. Select the subnet where the WebLogic Server compute instance is provisioned.
  6. Select the security list assigned to this subnet.
  7. For a domain that's not on a private subnet, make sure the following ingress rules exist:
    Source: 0.0.0.0/0
    IP Protocol: TCP
    Source Port Range: All
    Destination Port Range: 7002
    Source: 0.0.0.0/0
    IP Protocol: TCP
    Source Port Range: All
    Destination Port Range: 7001

    For a domain on a private subnet, set the Source to the CIDR of the bastion instance subnet.

Cannot access the sample application using the load balancer: Not Found

On a domain running Oracle WebLogic Server Standard Edition, the sample application is deployed only to the first Managed Server. If your Standard Edition domain has multiple Managed Servers and you access the sample application using a load balancer, the Managed Servers that aren't hosting the sample application will respond with the code 404 (Not Found).

You can use the WebLogic Server Administration Console to update the targets for the sample application, and add the remaining Managed Servers.

Cannot access applications using the load balancer: Bad Gateway

If you restart the compute instances running your Managed Servers, or you restart the compute instances running the App Gateway, the backend set of the load balancer will temporarily be in an unhealthy state. By default, a load balancer in this state will respond with the code 502 (Bad Gateway). After the WebLogic Server and App Gateway processes are running, the load balancer should return to the OK state.

To check the status of the load balancer and backend servers:

  1. Access the Oracle Cloud Infrastructure console.
  2. From the navigation menu, select Networking, and then click Load Balancers.
  3. Click the load balancer that was created for your domain, prefix-lb.
  4. Click Backend Sets, and then click prefix-lb-backendset.
  5. Click Backends, and then check the state of each backend.
  6. Access the WebLogic compute instances using a secure shell (SSH) client. Check that the Managed Server process is listening on its assigned port (the default is 7003).
    curl -s -o /dev/null -w "%{http_code}\n" http://private_ip:7003

    A 404 response indicates that the Managed Server is running.

  7. If you enabled authentication with Oracle Identity Cloud Service, then access the App Gateway compute instances using an SSH client. Check that the App Gateway process is listening on its assigned port (the default is 9999).
    curl -s -o /dev/null -w "%{http_code}\n" http://private_ip:9999

    A 404 response indicates that the App Gateway is running.

See Managing Backend Servers in the Oracle Cloud Infrastructure documentation.

Cannot access the Fusion Middleware Control Console from the Internet

If you enabled authentication with Oracle Identity Cloud Service on a WebLogic Server 12.2.1.4 domain, you might be redirected to an error page when you try to log in to the Fusion Middleware Control Console.

Example message:

<Error> <oracle.help.web.rich.OHWFilter>
<BEA-000000> <ADFSHARE-00120: Error encountered while creating the MDS
Session. Application state will be reset. Please logout and log back in if
problem persists.
oracle.adf.share.ADFShareException: ADFSHARE-00120: Error encountered while
creating the MDS Session. Application state will be reset. Please logout and
log back in if problem persists.

To access the Fusion Middleware Control Console:

  1. Add the Cloud Gate App Role to your confidential application that you created for the domain.
    1. Access the Oracle Identity Cloud Service console.
    2. From the navigation menu, click Applications.
    3. Click the confidential application that was created for your domain.
    4. Click the Configuration tab.
    5. Under Client Configuration, locate Grant the client access to Identity Cloud Service Admin APIs, and then click Add.
    6. Select the Cloud Gate App Role and click Add.
    7. Click Save.
  2. Restart your WebLogic Server domain and log in to the Fusion Middleware Control Console again.

See Create a Confidential Application.

Load Balancer does not send Cookie X-Oracle-BMC-LBS-Route

When you setup Oracle WebLogic Server for OCI with a load balancer, the load balancer does not send cookie X-Oracle-BMC-LBS-Route.

Scenario:
  1. Create a 2-node WebLogic instance with load balancer by using a Oracle WebLogic Server for OCI listings in marketplace.
  2. Access the sample app through load balancer.
  3. In your web browser, go to WebDeveloper > Web Console > Network.
  4. Click Reload.
  5. Click on Get request of the sample app, then select the Cookies tab

    The cookies tab is empty.

Workaround:
  1. Sign in to the Oracle Cloud Infrastructure console.
  2. From the navigation menu, click Networking, and then click Load Balancers.
  3. Click the name of the Compartment that contains the load balancer you want to modify, and then click the load balancer's name.
  4. , and then click the name of the backend set you want to modify.
  5. In the Resources menu, click Backend Sets. Deselect HTTP Only.
  6. Save the changes.
  7. Undo the changes you did in step 5 and then save the changes.
  8. Reload the sample app browser.

    Now you can view that the cookie with name X-Oracle-BMC-LBS-Route is passed properly.

Autoscaling Failed to Create Functions

When you create an Oracle WebLogic Server for OCI domain with autoscaling enabled, functions may not be created during provisioning.

Issue:

During provisioning, if autoscaling failed to create the functions due to invalid OCIR auth token value, provisioning is successful but the following error is displayed in the log:

module.provisioners.null_resource.print_service_info[0] (remote-exec): ************************************************************
module.provisioners.null_resource.print_service_info[0] (remote-exec): This service is configured with the following options .....
module.provisioners.null_resource.print_service_info[0] (remote-exec): WebLogic Server for OCI Version : 22.1.1-220208100422
module.provisioners.null_resource.print_service_info[0] (remote-exec): WebLogic Server Version: 12.2.1.4
module.provisioners.null_resource.print_service_info[0] (remote-exec): WebLogic Server Edition: SUITE
module.provisioners.null_resource.print_service_info[0] (remote-exec): Virtual Cloud Network : NEW VCN
module.provisioners.null_resource.print_service_info[0] (remote-exec): Network Type: PRIVATE Network with BASTION
module.provisioners.null_resource.print_service_info[0] (remote-exec): Domain Type: Plain WebLogic Server Domain (non-JRF)
module.provisioners.null_resource.print_service_info[0] (remote-exec): APM agent enabled : [True]
module.provisioners.null_resource.print_service_info[0] (remote-exec): APM agent installed : [True]
module.provisioners.null_resource.print_service_info[0] (remote-exec): Failed to create autoscaling resources, Check provisioning logs for details
module.provisioners.null_resource.print_service_info[0] (remote-exec): User can invoke remove_resources.py script and then rerun the configure_autoscaling.sh script on Admin VM to recreate the resources.
module.provisioners.null_resource.print_service_info[0] (remote-exec): ************************************************************

Workaround:

Perform the following steps to create the function resources:
  1. Update the OCIR auth token secret to the valid OCIR auth token as follows:
    • In the script /opt/scripts/observability/autoscaling/configure_autoscaling.sh, replace the line ocir_auth_token=$(python3 /opt/scripts/wls_credentials.py ocirAuthToken)with ocir_auth_token='<VALID_AUTH_TOKEN>'.
  2. Run the script to delete resources to clean up any resources created by autoscaling during provisioning from WebLogic administration instance.

    python3 remove_resources.py pre-destroy <service_name_prefix> -f autoscaling

  3. Log in as a root user to the Administration server and run the configure_autoscaling.sh script.

    /opt/scripts/observability/autoscaling/configure_autoscaling.sh

    This script creates the autoscaling functions using the valid OCIR auth token from step 1. If you encounter any errors when you run the script, see /u01/logs/provisioning.log.

  4. In the OCI console, verify if:
    • Autoscaling functions are created under the function application.
    • Notification subscriptions are created for Scale Out and Scale In notification topics.
    • Event rule is created for the stack.

Management Agents Are Not Deleted on Instance Termination

In case of an Oracle WebLogic Server for OCI domain with autoscaling enabled, when you destroy the stack, the management agent resources associated with the compute instance are not destroyed.

Issue:

When you destroy the stack, the compute instance for the domain is terminated but the associated management agent resources are active.

Workaround:

Perform the following steps to manually delete the management agent resources:

Note:

You must run the following commands from the Cloud Shell. You can also install OCI CLI and create the config file on your host, and then run the following commands. See Installing the CLI.
  1. List the management agents in the stack compartment.
    oci management-agent agent list --compartment-id <stack_compartment_ocid>
    oci management-agent agent list --compartment-id <stack_compartment_ocid>  | grep ocid1.managementagent | grep '"id":'
    
  2. Delete the management agents associated with your service.
    oci management-agent agent delete --agent-id <OCID_of_the_managementagent_from_step1> --force

Enterprise Manager Console Is Not Loading

Issue: After you create a JRF-enabled domain without Oracle Identity Cloud Service in Oracle WebLogic Server for OCI, you are unable to log into the Enterprise Manager console.

Note:

This issue is applicable for stacks created between 31st August, 2022 and 20th September, 2022 (22.3.2 release).

Workaround:

  1. Edit the jps-config.xml located in /u01/data/domain/<domain-name>/config/fmwconfig/. Replace idstore.scim with idstore.ldap.
    <serviceInstanceRef ref="idstore.ldap"/>
  2. Execute the restart_domain.sh script.
    /opt/scripts/restart_domain.sh -o restart

Scale Out Fails on the Administration Compute Instance

Scale out fails due to high CPU usage on the administration compute instance.

Issue:

If the administration compute instance is stressed to 100 percent utilization, the scale out fails as the pack commands that are run during scale out on the administration compute instance do not give any results and time out.

Workaround:

Verify if the control group wlsmcg exists under /sys/fs/cgroup/cpu. If it exists, assign process IDs for the administration server and managed server to the control group, and assign CPU shares to the control group to manage the CPU usage. So, you can successfully run the pack command.

Note:

If you start the servers on the administration compute instance using scripts or through the WebLogic console, the new process IDs for the server are not added to the control group.
To reassign the process ID for the administration server and managed server to the control group:
  1. Check if current server process ID is part of the control group wlsmcg.
    cat /sys/fs/cgroup/cpu/wlsmscg/tasks | grep "<processID for managed server>"
  2. If the process ID is not found in the step 1, run the following script as the opc user to create control groups and assign process IDs for the administration server and managed server.
    sudo /opt/scripts/create_control_groups.sh
  3. Verify that managed server process ID is assigned to the control group, wlsmscg.
     cat /sys/fs/cgroup/cpu/wlsmscg/tasks | grep "<processID for managed server>"

Enable OS Management to Install Patches

For an existing Oracle WebLogic Server for OCI instance, you might encounter issues when you use the OS Management to apply patches.

So, to enable the OS Management, create the following policies:

Allow dynamic-group MyInstancesPrincipalGroup to use osms-managed-instances in compartment MyCompartment
Allow dynamic-group MyInstancesPrincipalGroup to use osms-managed-instances in compartment MyCompartment

Security Checkup Tool Warnings

Learn about the security check warnings that are displayed in the Oracle WebLogic Server Administration console and how to troubleshoot them.

At the top of the WebLogic Server Administration console, the message Security warnings detected. Click here to view the report and recommended remedies is displayed for Oracle WebLogic Server for OCI instances created after July 20, 2021, or the instances on which the July 2021 PSUs are applied.

When you click the message, a list of security warnings are displayed as listed in the following table.

The warning messages listed in the table are examples.

Security Warnings

Warning Message Resolution

The configuration for key stores for this server are set to Demo Identity and Demo Trust. Trust Demo certificates are not supported in production mode domains.

Configure the identity and trust keystores for each server and the name of the certificate in the identity keystore that the server uses for SSL communication. See Configure Keystore Attributes for Identity and Trust.

Note: This warning is displayed for Oracle WebLogic Server for OCI instances created after October 20, 2021, or the instances on which the October PSUs are applied.

SSL hostname verification is disabled by the SSL configuration.

You see the SSL host name verification warnings in case of existing Oracle WebLogic Server for OCI instances created before release 21.3.2 (August 17, 2021).

Review your applications before you make any changes to address these SSL host name security warnings.

For applications that connect to SSL endpoints with a host name in the certificate, which does not match the local machine's host name, the connection fails if you configure the BEA host name verifier in Oracle WebLogic Server. See Using the BEA Host Name Verifier in Administering Security for Oracle WebLogic Server.

For applications that connect to Oracle provided endpoints such as Oracle Identity Cloud Service (for example,*.identity.oraclecloud.com), the connection fails if you did not configure the wildcard host name verifier or a custom host name verifier that accepts wildcard host names. If you are not sure of the SSL configuration settings you should configure to address the warning, Oracle recommends that you configure the wildcard host name verifier. See Using the Wildcard Host Name Verifier in Administering Security for Oracle WebLogic Server.

Note: For WebLogic Server 14.1.1.0.0, the default host name verifier is set to the wildcard host name verifier.

Remote Anonymous RMI T3 or IIOP requests are enabled. Set the RemoteAnonymousRMIT3Enabled and RemoteAnonymousRMIIIOPEnabled attributes to false.

Disable the anonymous RMI T3 and IIOP requests in the WebLogic Server Administration Console as soon as possible unless your deployment requires anonymous T3 or IIOP (not typical). See Disable Remote Anonymous RMI T3 and IIOP Requests.

After you address the warnings, you must click Refresh Warnings to see the warnings removed in the console.

For Oracle WebLogic Server for OCI instances created after July 20, 2021, though the java properties to disable anonymous requests for preventing anonymous RMI access are configured, the warnings still appear. This is a known issue in Oracle WebLogic Server.

If you want to perform anonymous RMI requests, you must disable the java properties. Go to the nodemanager.properties file located under DOMAIN_HOME/nodemanager and remove the weblogic.startup.Arguments property.

Disable Remote Anonymous RMI T3 and IIOP Requests

To disable the remote anonymous RMI T3 and IIOP requests in the WebLogic Server Administration console:

  1. Locate the Change Center and click Lock & Edit to lock the editable configuration hierarchy for the domain.

  2. Under Domain structure, select the domain name, and then select the Security tab.

  3. Expand Advanced and deselect Remote anonymous RMI access via IIOP and Remote anonymous RMI access via T3.

After saving the changes, return to Change Center and click Activate Changes.

Configure Keystore Attributes for Identity and Trust

To configure the identity and trust keystore files and the name of the certificate in the identity keystore in the WebLogic Server Administration console:

  1. Locate the Change Center and click Lock & Edit to lock the editable configuration hierarchy for the domain.

  2. Under Domain structure, select Environment and then select Servers.

  3. In the Servers table, select the server you want to configure.

  4. On the Configuration tab, click Keystores, and then click Change.

  5. Select Custom Identity and Custom Trust, and then click Save.

  6. Under Identity, provide the following details:

    1. Enter the full path of your identity keystore.

      For example: /u01/data/keystores/identity.jks

    2. For Custom Identity Keystore Type, enter JKS.

    3. For Custom Identity Keystore Passphrase, enter your keystore password. Enter the same value for Confirm Custom Identity Keystore Passphrase.

  7. Under Trust, provide the following details:

    1. Enter the full path of your identity keystore.

      For example, /u01/data/keystores/trust.jks

    2. For Custom Trust Keystore Type, enter JKS.

    3. For Custom Trust Keystore Passphrase, enter your keystore password. Enter the same value for Confirm Custom Trust Keystore Passphrase.

  8. Click Save.

  9. Click the SSL tab.

  10. Under Identity, provide the following details:

    1. For Private Key Alias, enter the name of the certificate (private key) in the identitykeystore, server_cert.

    2. For Private Key Passphrase, enter the password for this certificate in the keystore. Enter the same value for Confirm Private Key Passphrase.

      By default, the password for the certificate is the same as the identity keystore password.

  11. Click Save.

    After saving the changes, return to Change Center and click Activate Changes.

  12. Repeat steps 3 to 9 to configure each server in the domain.

Running python3 Command Fails

Issue: When you run a command that uses python3, it might fail on the stacks that were created earlier than May 25, 2022.

Workaround: If your stack is created earlier than May 25, 2022, then in the command use python. For stacks created after May 25, 2022, use python3.

Get Additional Help and Contact Support

Use online help, email, customer support, and other tools if you have questions or problems with Oracle WebLogic Server for OCI.

For customer support, you can create support tickets using the Oracle Cloud Infrastructure (OCI) console or My Oracle Support.

Create Support Ticket Using OCI Console

Use the Support Center in Oracle Cloud Infrastructure console to create a support ticket for your technical issues for Oracle WebLogic Server for OCI service in the Marketplace.

Note:

Make sure to provision your support account before you create a support request. See Configuring Your Oracle Support Account in Oracle Cloud Infrastructure documentation.

To create a support ticket:

  1. Sign in to the Oracle Cloud Infrastructure console.
  2. Click the navigation menu Navigation Menu icon, and select Governance & Administration. Under Support, click Support Center.
  3. Click Create Support Request.

    The Technical Support tab on the Support Options page is displayed.

  4. For Issue Summary, enter the a title that summarizes your issue.
  5. For Describe Your Issue, enter a brief description of your issue.
  6. Select the severity level of the issue based on the impact of service.
  7. Select Marketplace from the Select Service list.
  8. Select Oracle WebLogic Server for OCI from the Select Category list.
  9. Select the type of issue you are experiencing.
  10. Click Create Support Request.

After you submit the request, My Oracle Support sends a confirmation email to the address provided in the primary contact details. A follow-up email is sent if additional information is required.

Optionally, you can create a support ticket using the Help menu Help Menu Icon and the Support button Support Iconin Oracle Cloud Infrastructure console. See Support Ticket Management in Oracle Cloud Infrastructure documentation.

However, when you a create support ticket using these options, the support ticket may not be assigned to a specific service or component for resources like compute instances, networks and load balancers. So, it is recommended to use Support Center in Oracle Cloud Infrastructure console to create support tickets.

Create Support Ticket Using My Oracle Support

Use the Service Request in My Oracle Support to create a support ticket for your technical issues for Oracle WebLogic Server for OCI service in the Marketplace.

Note:

Make sure you have a Support Identifier which verifies your eligibility for Support services, and an account at My Oracle Support.

To create a support ticket:

  1. Sign in to My Oracle Support.
  2. On the Service Requests tab, click Create Technical SR.
  3. Enter the Problem Summary and the Problem Description.
  4. Under Where is the Problem, click Cloud.
  5. Select Oracle WebLogic Server for OCI from the Service Type list.
  6. Select the tenancy from the Service list.
  7. Select a Problem Type and provide the Support Identifier details.
  8. Click Next until you have provided all the mandatory information.
  9. Click Submit.

    Your service request is created.

For general help with Oracle Cloud Marketplace, see How Do I Get Support in Oracle Cloud Infrastructure documentation.