Skip Headers
Oracle® Fusion Applications Administrator's Troubleshooting Guide
11g Release 6 (11.1.6)

Part Number E25450-05
Go to Documentation Home
Go to Book List
Book List
Go to Table of Contents
Go to Feedback page
Contact Us

Go to previous page
Go to next page
PDF · Mobi · ePub

13 Troubleshooting Moving Components for Oracle Fusion Applications Across Environments

This chapter describes common problems that you might encounter when moving Oracle Fusion Applications components across environments and explains how to solve them.

This chapter includes the following topics:

13.1 Getting Started with Logging Basics

To troubleshoot moving Oracle Fusion Applications across environments, use the following tools:

13.2 Avoiding Issues with Moving Components for Oracle Fusion Applications Across Environments

To avoid movement failures, check the following:

13.2.1 Oracle Inventory Verification

One of the most common reasons for a failed movement session is due to the use of an incorrect or invalid Oracle Inventory (oraInventory) when running the copy or paste commands.

To ensure your environment has the correct Oracle Inventory:

  1. Verify the Oracle Inventory location.

    On UNIX systems, verify that the inventory location referenced in the default inventory location (/etc/oraInst.loc on Linux systems) refers to the actual inventory your source Oracle Fusion Applications install is registered into. If it does not, you will need to use the -invPtrLoc (-ipl) argument on all copy and paste operations. This argument should contain the path to the oraInst.loc file that matches the inventory your Oracle Fusion Applications install is registered into.

    On Windows systems, the Oracle Inventory location is stored in the Windows registry under the HKEY_LOCAL_MACHINE\Software\Oracle location, in the inst_loc key. This value is global across the machine. On the Windows platform, the -invPtrLoc (-ipl) argument is not supported. You must use the inventory specified in the registry.

  2. On Windows systems, the Oracle Inventory location is stored in the Windows registry under the HKEY_LOCAL_MACHINE\Software\Oracle location, in the inst_loc key. This value is global across the machine. On the Windows platform, the -invPtrLoc (-ipl) argument is not supported. You must use the inventory specified in the registry.

  3. Verify that the user ID performing the movement actions has full read and write permissions to the Oracle Inventory location being used.

  4. Verify that all Oracle home locations within the fusionapps Middleware directory are using the same oraInventory location. Failure to do so will result in some Oracle homes not being present in the archive.

Failure to adhere to the requirements listed above can manifest itself in failures such as:

  • Not all Oracle homes present in the source location being included in the archive.

  • Not all Oracle homes present in the archive being processed by the paste* commands.

  • NullPointerException during copyBinary operation.

13.2.2 Java Issues

To avoid Java issues:

  1. Do not use a JDK from a directory location containing spaces (if on Windows, please use the short form directory names if you must use a JDK in a directory location containing spaces).

  2. Ensure that your JDK "bitness" matches that of your Oracle Fusion Applications install. If you are using a 64-bit JDK, make sure that you are using a 64-bit Oracle Fusion Applications installation.

    On Windows systems, failure to ensure this can result in T2P getting confused regarding the Oracle Inventory on 64 bit boxes. On 64 bit Windows machines, both the Program Files and Program Files (x86) directories can contain inventories: Program Files is used for 64-bit installations.

13.2.3 Running Commands from the Wrong Directory

On the target machine, after running the pasteBinary operation, be sure to execute the related commands, pasteConfig, extractMovePlan, and so on) from the following extracted locations:

(UNIX) MW_HOME/oracle_common/bin
(Windows) MW_HOME\oracle_common\bin

13.2.4 Oracle Web Services Manager Policy Migration Failed During pasteconfig

If Oracle Web Services Manager policy migration fails when using the pasteConfig script, the issue can occur if you do not use the proper Oracle Identity Management database service name. The SID value should not be used.

13.2.5 Oracle Platform Security Services JPS Root fusionappspolicies Not Created In Oracle Identity Management Delta Movement

Oracle Platform Security Services JPS root fusionappspolicies not created during an Oracle Identity Management delta movement.

Ensure that the instructions given in the "Update the ADF Credentials" section in the Oracle Fusion Applications Administrator's Guide for the Identity Store have been followed and that the resulting data is properly populated.

13.2.6 Move Plan Validation Error

During pasteConfig operations, invalid editing of the move plans can result in validation errors. Please ensure the following:

  1. Use a syntax validating editor to edit the move plan files. This ensures that simple XML formatting rules within the file are followed.

  2. Ensure that no READ_ONLY elements have been edited and modified.

  3. Ensure that no new READ_WRITE elements have been added.

13.2.7 pasteConfig for Oracle Internet Directory Is Failing

When performing the pasteConfig operation for Oracle Internet Directory, the operations fails with an invalid database error.

During a pasteConfig operation, if the target database is the same as the source database, then the Oracle Internet Directory component name should be different.

If the target database is different from the source database, then the Oracle Internet Directory component name needs to be the same.


Target database also needs to be a clone of source database.

13.2.8 pasteBinary May Incorrectly Calculate Free Disk Space

The pasteBinary operation may incorrectly calculate the free disk space. This can be due to the target location being on a Network File System (NFS) share or a different type of file system (such as data ONTAP).

If you are sure that there is enough free disk space for the archive to be extracted and configured, use the -ignoreDiskWarning (-idw) argument to ignore the disk space check error

13.2.9 Node Manager Movement Operations

Ensure that the copyConfig and pasteConfig operations for Node Manager operations are performed. The copyConfig and pasteConfig operations for the domains do not configure the Node Manager properties. Failure to do so can result in defaulted Node Manager configuration rather than the properties from the source environment.

See the "Running the copyConfig for Node Manager" section and the "Running the pasteConfig for Node Manager" sections for command line usage.

13.3 Problems and Solutions

This section describes common problems and solutions. It includes the following topics:

For information about the error messages you may encounter when moving Oracle Fusion Applications components across environments, see Oracle Fusion Middleware Error Messages Reference.

13.3.1 Running Fusion Applications Role Category Seeding Task Returns Exception

An exception is raised when you run the Fusion Applications Role Category Seeding task in Oracle Identity Manager.


Oracle Virtual Directory was not configured properly for the change log adapter.


Set the Remote Base value to an empty value.

  1. In Oracle Directory Services Manager, select the Adapter tab, then General.

  2. Remove the value in the Remote Base field.

  3. Restart Oracle Virtual Directory and Oracle Identity Manager.

13.3.2 pasteBinary Returns Permission Errors

While troubleshooting binary movement issues, you may run across some permission-related messages. These messages are expected due to the nature of transient files between the source and target machines. In most cases, you can safely ignore these message if there are no other movement failures. Some examples of permission errors are:

/bin/chmod: cannot access
 /webtier/cfgtoollogs/configToolAllCommands' : No such file or directory
'/bin/chmod: cannot access
/webtier/network/lib/' : No such file or directory

13.3.3 pasteBinary Returns Error for Middleware Home Directory

The pasteBinary script returns an error when you apply the Middleware home directory to the target environment.


Incorrect information was entered.


Take the following actions to recover:

  1. Delete the target Middleware home.

  2. Remove the Oracle home entry from the Oracle inventory, if it is present.

  3. For Windows, remove the shortcut for the Middleware home and Oracle home.

13.3.4 pasteBinary Fails With deriveNewWLSProperties Error

You encounter the deriveNewWLSProperties error when you run the pasteBinary script.


The target Middleware home location is inside another Middleware home.


Specify a target Middleware home location that is not inside another Middleware home when you run the pasteBinary script.

13.3.5 copyConfig Fails When Checking Oracle Homes

The copyConfig script fails when it checks Oracle homes.


The inventory configuration is invalid.


Make sure that inventory.xml is valid. Do not update the inventory manually.

13.3.6 copyConfig Takes Too Long to Complete

The copyConfig script takes an unusually long time to complete.


The metadata export is requiring a lot of time.


Check the logs to make sure it is taking time to export. Clean up the database cache or purge the database. You can also collect the database statistics.

13.3.7 copyConfig Fails With CLONE-20408 Error

The copyConfig script fails with the error CLONE-20408.


The machine is specified as unix-machineType in the config.xml file.


Edit the machine type in DOMAIN_HOME/config/config.xml to remove xsi:type="unix-machineType" in:

<machine xsi:type="unix-machineType">   <name>machine_name</name>

For example:

<machine>   <name>machine_name</name>

Restart the servers after you update the config.xml file.

13.3.8 copyConfig Fails with MaxMessageSizeExceededException

The copyConfig script fails on the WebLogic Server domains.


You encounter the following error when you are executing copyConfig of Oracle WebLogic Server Domains:

weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '1000080' bytes exceeds the configured maximum of:'1000000' bytes for protocol: 't3'. 


Change the message size limit by using the T2P_JAVA_OPTIONS environment variable to pass the -Dweblogic.MaxMessageSize=2000000 property to both the copyConfig and pasteConfig scripts.

13.3.9 Cannot Run pasteConfig for Node Manager After Initial Failure

You cannot run the pasteConfig script for Node Manager after running it failed the first time.


The pasteConfig script for Node Manager creates the file and other files, which are left out when there is a failure.


Clean up all the files in the target Node Manager home location before re-running the pasteConfig script.

13.3.10 Copy or Paste Fails with a DiskQuotaExceeded Error

A user block limit/disk quota error causes either a copy or a paste operation to fail.


You faced the following exception during a copy or paste operation: write failed, user block limit reached. SEVERE: Disk quota exceeded.


To ensure that enough disk space is allocated to this user, you can either expand the disk quota for that user or remove the disk quota limit for that user.

13.3.11 Exceptions Are Raised for Oracle SES During pasteConfig

The pasteConfig script returns exceptions for Oracle SES when you apply system components to the target environment.


pasteConfig was invoked a second time on the same Oracle SES instance.


Clean up the Oracle SES instance in the target environment using the searchadmin command-line interface before invoking pasteConfig again.

For more information about the searchadmin commands, see Oracle Secure Enterprise Search Administration API Guide.

To clean up the Oracle SES instance:

  1. At the operating system prompt, log in to the searchadmin command-line interface using the following command:

    SES_ORACLE_HOME/bin/searchadmin -p adminPwd -c adminWebServiceUrl

    For example:

    >/scratch/yummy/wls/Oracle_SES1/bin/searchadmin -p welcome2SES -c

    The prompt changes to SES>.

  2. Enter the following searchadmin commands in the order specified:

    SES>deleteAll altWord
    SES>deleteAll proxyLogin
    SES>deleteAll facetTree
    SES>deleteAll suggLink
    SES>deleteAll sourceGroup
    SES>deleteAll schedule
    SES>deleteAll docServicePipeline
    SES>deleteAll docServiceInstance
    SES>deleteAll source
  3. Delete any remaining sources:

    1. Log in to the Oracle SES Administration GUI.

    2. Select the Sources secondary tab.

    3. On the Sources page, delete any remaining sources that are listed.

  4. Return to the Oracle SES searchadmin command line and enter the following commands:

    SES>deleteAll searchAttr
    SES>delete thesaurus -n DEFAULT


    The thesaurus may not exist. You can ignore the error message EQA-11000: The object with key "[name=DEFAULT]" and type "thesaurus" was not found.

  5. Check whether or not an active identity plug-in exists:

    1. Enter the following command:

      SES>getAllStates identityPlugin -o idPluginState.xml

      The idPluginState.xml file is generated in SES_ORACLE_HOME/bin/idPluginState.xml.

    2. Open the idPluginState.xml file and search for the phrase <search:value>ACTIVE</search:value>.

    3. If <search:value>ACTIVE</search:value> does not exist, then there is no active identity plug-in. Skip to Step 8.

      If <search:value>ACTIVE</search:value> exists, continue to Step 6.


    There will be at most only one active identity plug-in at a time.

  6. Scroll up the page to locate the jarFilePath and managerClassName of this active identity plug-in.

  7. Deactivate the plug-in by entering the following command:

    SES>deactivate identityPlugin --JAR_FILE=jarFilePath --MANAGER_CLASS=managerClassName
  8. Check whether or not partitionConfig is active:

    1. Enter the following command:

      SES>getState partitionConfig -o partitionConfigState.xml

      The partitionConfigState.xml file is generated in SES_ORACLE_HOME/bin/partitionConfigState.xml.

    2. Open the partitionConfigState.xml file and search for the phrase <search:value>ACTIVE</search:value>.

    3. If <search:value>ACTIVE</search:value> does not exist, then there is no active partitionConfig. Skip to Step 10.

      If <search:value>ACTIVE</search:value> exists, continue to Step 9.

  9. Enter the following two commands:

    SES>export partitionConfig -o partitionConfig.xml
    SES>update partitionConfig -i partitionConfig.xml -a remove
  10. Delete the storage areas by entering the following command:

    SES>deleteAll storageArea

13.3.12 SOA-infra Application Is in Failed State

The SOA server fails to start after you have completed the procedures for moving the Oracle Fusion Applications components.


An incorrect configuration of the Oracle Coherence framework that is used for deployment is preventing the SOA system from starting. The deployment framework must be properly customized for the network environment on which the SOA system runs.


Modify the Oracle Coherence cluster configuration by editing the values for the following startup parameters:

  • tangosol.coherence.wka1

  • tangosol.coherence.localhost

  • tangosol.coherence.localport

  • tangosol.coherence.wka1.port

For information, see the "Configuring Oracle Coherence for Deploying Composites" section in Oracle Fusion Middleware Enterprise Deployment Guide for Oracle SOA Suite.

13.3.13 No Oracle Fusion Applications Targets Display in Oracle Enterprise Manager

After moving the Oracle WebLogic Server domain configuration, the Oracle Fusion Middleware targets display in Oracle Enterprise Manager, but Oracle Fusion Applications targets (such as product families, products, and J2EE applications) do not.


The ASK tables in the database (such as ASK_DEPLOYED_APPLICATIONS and ASK_DEPLOYED_DOMAINS) are not populated.


Check to make sure that Topology Manager functions properly when you run the copyConfig and pasteConfig scripts. Also check to make sure that the move plan values are properly modified.

13.3.14 OutOfMemory (Heap Size) Error while Unpacking the Domain in pasteConfig

The pasteConfig script hangs, then displays a java.lang.OutOfMemoryError:Java heap space error while the unpack command was executed as part of the pasteConfig operation.


There is insufficient memory to process the large files (either spurious soft links to directories with many large files or large log files) of the Oracle WebLogic Server domain.


Remove the large, unnecessary files, and run the copyConfig and pasteConfig scripts again. You can also increase the JVM heap size by using the -Xmx option for maximum heap size, and the -Xms option for initial heap size. For example:

export CONFIG_JVM_ARGS="-Xms512m -Xmx1024m"

13.3.15 OutOfMemory Error with pasteConfig while Starting the Administration Server or Managed Server

You cannot run the pasteConfig script because of an OutOfMemory error.


You encounter an OutOfMemory error while the pasteConfig operation is trying to start the server.


Set the JVM system variable USER_MEM_ARGS, as described in the "Setting the JVM System Variables on Windows" section of the Oracle Fusion Applications Administrator's Guide.

13.3.16 wf_client_config.xml Refers to Files in the Source Environment

After moving your Oracle Fusion Applications environment from one host to another, the wf_client_config.xml files still refers to the files in source environment.


The wf_client_config.xml files were not edited to change the value of the <rootEndPointURL> element.


Edit the wf-client-config.xml file for each application to change the value of the <rootEndPointURL> element.

To locate each wf_client_config.xml file and edit the <rootEndPointURL> element:

  1. Run the following command from the fusionapps Middleware directory (for example, /net/mount1/appbase/fusionapps):

    find . -name wf_client_config.xml
  2. For each wf_client_config.xml file (one for each application), change the value of <rootEndPointURL> to point to the internal virtual host port for the SOA cluster on the target environment. For example:


13.3.17 PolicyManagerValidator Fails to Preload

PolicyManagerValidator fails to preload on startup in the /wsm-pm web application.


An MDSConfiguration exception was encountered in parseADFConfiguration at oracle.adf.share.config.ADFMDSConfig.parseADFConfiguration.


Manually make the following targeting changes:

  • Add the Oracle Web Services Manager targets: admin, ess, and spaces.

  • Remove the wsm-pm target: admin.

13.3.18 Issues While Running pasteConfig

You encounter issues while running the pasteConfig script.


The custom scripts in the ORACLE_HOME/COMMON/WLST directory are invalid.


Validate the custom scripts in the ORACLE_HOME/COMMON/WLST directory.

13.3.19 Cluster Validation Fails

You receive a setClusterAddress() failed error while configuring the machine in the target environment.


The cluster member host address contains an illegal character in the domain or host name or address.


Replace the domain or host name or address with a valid value.

13.3.20 Cannot Enable the SSL Connection Type After Node Manager Movement

After you move Node Manager from the source environment to the target environment, you encounter a exception because the connection between the Administration Server and Node Manager is of connection type plain socket, instead of SSL.


The keystores were not created properly.


Complete the following tasks to properly create the keystores:

  1. Create the identity keystores and trust keystores for the middle tier Managed Servers and Node Manager. See the "Ensure Source and Destination Environments Fulfill Requirements" section in Oracle Fusion Applications Administrator's Guide.

  2. Perform Node Manager movement, editing the Node Manager properties in the move plan. See the "Move Plan Properties for Node Manager" section in Oracle Fusion Middleware Administrator's Guide.

13.3.21 Cannot Retrieve Documents from Oracle WebCenter Content: Imaging Viewer

After moving the Oracle Fusion Applications components, you cannot retrieve documents from the Oracle WebCenter Content: Imaging Viewer.


The ixTransformer no longer has execute rights when bundled with and extracted with The server must be able to execute this file in order to transform documents into TIF format for viewing.


Set the permissions to 750 (on UNIX systems) for files in the unpacked domain with the extension .sh. For information, see "Task 3 Modify Oracle Information Rights Management Settings" in Oracle Fusion Middleware Administrator's Guide.

13.3.22 OHS Component Will Not Start After pasteConfig

Upon completion of the pasteConfig script for an OHS component, the script automatically attempts to start the new component, but it will not start.


SSL was previously configured on the source environment, but upon being cloned to the target, ssl.conf still points to the old wallet location, which no longer exists. The OHS cannot find the wallet, which results in startup failure.


If the non-default wallet was used at the source environment, then export the wallet from the source environment and import it to the target environment. Then, update the wallet location (if changed) in the ssl.conf file in the target environment.

13.3.23 Application Scope Data Sources Are Not Copied

After moving the Oracle Fusion Applications components, all Oracle ADF applications that use the application scope data sources fail to start.


The copyConfig script handles only global data sources defined in each Oracle WebLogic Server domain.


For application level data sources, you must deploy the Oracle ADF application configured with the application level data sources to a server in the target domain, and manually configure the data sources on the target domain.

13.3.24 Java Runtime Environment Was Not Found During Cloning Operations

When performing cloning operations as a part of the paste operations, the Java Runtime Environment was not found when runInstaller / setup.exe is executed from the cloned Oracle Home.


While invoking runInstaller / setup.exe from cloned Oracle Home, the following message is displayed:

The Java RunTime Environment was not found when runInstaller / setup.exe is executed from the cloned Oracle Home.

The error occurs because the JRE was not found at /tmp/OraInstalltime_stamp/jre/bin/java. Therefore, the Oracle Universal Installer cannot be run.


To resolve this issue:

  1. Visit and install JRE version 1.3.1 or higher and try again.

  2. Ensure the JRE_LOCATION setting in ORACLE_HOME/oui/oraparam.ini points to a valid JRE location.

13.4 Diagnosing Problems

This section describes general approaches for diagnosing moving Oracle Fusion Applications components across environments. It contains the following topic:

13.4.1 Diagnosing Issues with Oracle Metadata Repository Export and Import

The copyConfig process tries to export the Oracle Metadata Repository (MDS) data of all applications. Some applications may have MDS data, while some may not have any data—or in some cases, the Managed Server may be down or the application may not be in active state.

To diagnose issues related to MDS export, check the copyConfig logs after the copyConfig process is complete to make sure that all the applications with MDS data are exported successfully.

In the section of the copyConfig log, shown in Example 13-1, the listed applications do not have any MDS data, which causes the failure.

Example 13-1 Sample copyConfig Log

FINE : [J2EEComponentCreateCloner:doClone] ##################################### MDS Export Summary #############################################
FINE : [J2EEComponentCreateCloner:doClone]
MDS Export of some applications failed. Check the following list of applications and make sure that MDS data is not associated with these applications. - FusionEsdApp, FusionCtdApp, FMW Welcome Page Application, em, ESSAPP, ESSJobTypesAPP, DiagnosticsUI-Ess, SalesApp, oraclediagent, odiconsole, wsil-wls, DMS Application, AppsLoggerService, FileAdapter, DbAdapter, JmsAdapter, AqAdapter, FtpAdapter, SocketAdapter, MQSeriesAdapter, OracleAppsAdapter, OracleBamAdapter, usermessagingserver, usermessagingdriver-email, usermessagingdriver-extension, worklistapp, b2bui, DefaultToDoTaskFlow, composer, OracleBPMProcessRolesApp, OracleBPMComposerRolesApp, OracleBPMWorkspace, SimpleApprovalTaskFlow, SetTransformService, DiagnosticsUI-Assembly, Diagnostics-JmxApi, 

FINE : [J2EEComponentCreateCloner:doClone] #########################################################################################################

You can check the reason for failure for each of these applications from the copyConfig log.

Similarly, the pasteConfig process tries to import the MDS data that was exported during copyConfig. To diagnose issues related to MDS import, check the pasteConfig logs to make sure that the MDS import is successful.

Import failed for ProductManagementApp, ProductManagementCommonApp, and CostManagementApp in the pasteConfig log, shown in Example 13-2.

Example 13-2 Sample pasteConfig Log

FINE : [J2EEComponentApplyCloner:doClone] ##################################### MDS Import Summary #############################################
FINE : [J2EEComponentApplyCloner:doClone] MDS Import of following applications failed.


. Check managed server and clone logs for details.
FINE : [J2EEComponentApplyCloner:doClone] #########################################################################################################

Resolve any import failures after the pasteConfig is complete. You can use the importMetadata command to import the MDS after pasteConfig is complete. For information, see Oracle Fusion Middleware WebLogic Scripting Tool Command Reference.

13.5 Recovering from pasteConfig Errors

To recover from a pasteConfig error:

  1. Check the logs to identify the issue.

  2. Stop all processes related to the domain.

  3. Delete the following directories:

  4. For LDAP, drop the schemas and re-create them using the Oracle Fusion Middleware Metadata Repository Creation Utility, and delete the domain node.

    For Metadata Services (MDS), you can overwrite the schemas without dropping the domain node.

When you execute the pasteBinary or pasteConfig scripts and enter incorrect information in the move plan, the scripts return an error. In some cases, the scripts may have partially completed the paste operation.

To recover from a pasteConfig error when applying system components to the target:

  1. Uninstall the instance.

  2. If you cannot uninstall the instance, stop all processes related to that instance and delete the Oracle instance.