| Oracle® Fusion Applications Administrator's Troubleshooting Guide 11g Release 6 (11.1.6) Part Number E25450-05 |
|
|
PDF · Mobi · ePub |
| Oracle® Fusion Applications Administrator's Troubleshooting Guide 11g Release 6 (11.1.6) Part Number E25450-05 |
|
|
PDF · Mobi · ePub |
This chapter describes common problems that you might encounter when moving Oracle Fusion Applications components across environments and explains how to solve them.
This chapter includes the following topics:
To troubleshoot moving Oracle Fusion Applications across environments, use the following tools:
For movement issues, review issues in the log files.
Use the -logDirLoc option with the copyBinary, pasteBinary, copyConfig, pasteConfig, and extractMovePlan scripts to specify a directory for log files. If you do not specify this option for the scripts, log files are stored in the temp directory.
For component-specific or user errors, review the following logs:
Oracle Fusion Applications logs:
(UNIX) DOMAIN_HOME/config/fmwconfig/servers/Managed_Server/logs/apps (Windows) DOMAIN_HOME\config\fmwconfig\servers/Managed_Server/logs/apps
See the "Viewing and Searching Log Files During Normal Operation" section in the Oracle Fusion Applications Administrator's Guide.
Oracle WebLogic Server and instance logs:
(UNIX) DOMAIN_HOME/servers/server_name/logs (Windows) DOMAIN_HOME\servers\server_name\logs
See the "Viewing Log Files and Their Messages Using Fusion Middleware Control" section in the Oracle Fusion Middleware Administrator's Guide.
To avoid movement failures, check the following:
Section 13.2.4, "Oracle Web Services Manager Policy Migration Failed During pasteconfig"
Section 13.2.7, "pasteConfig for Oracle Internet Directory Is Failing"
Section 13.2.8, "pasteBinary May Incorrectly Calculate Free Disk Space"
One of the most common reasons for a failed movement session is due to the use of an incorrect or invalid Oracle Inventory (oraInventory) when running the copy or paste commands.
To ensure your environment has the correct Oracle Inventory:
Verify the Oracle Inventory location.
On UNIX systems, verify that the inventory location referenced in the default inventory location (/etc/oraInst.loc on Linux systems) refers to the actual inventory your source Oracle Fusion Applications install is registered into. If it does not, you will need to use the -invPtrLoc (-ipl) argument on all copy and paste operations. This argument should contain the path to the oraInst.loc file that matches the inventory your Oracle Fusion Applications install is registered into.
On Windows systems, the Oracle Inventory location is stored in the Windows registry under the HKEY_LOCAL_MACHINE\Software\Oracle location, in the inst_loc key. This value is global across the machine. On the Windows platform, the -invPtrLoc (-ipl) argument is not supported. You must use the inventory specified in the registry.
On Windows systems, the Oracle Inventory location is stored in the Windows registry under the HKEY_LOCAL_MACHINE\Software\Oracle location, in the inst_loc key. This value is global across the machine. On the Windows platform, the -invPtrLoc (-ipl) argument is not supported. You must use the inventory specified in the registry.
Verify that the user ID performing the movement actions has full read and write permissions to the Oracle Inventory location being used.
Verify that all Oracle home locations within the fusionapps Middleware directory are using the same oraInventory location. Failure to do so will result in some Oracle homes not being present in the archive.
Failure to adhere to the requirements listed above can manifest itself in failures such as:
Not all Oracle homes present in the source location being included in the archive.
Not all Oracle homes present in the archive being processed by the paste* commands.
NullPointerException during copyBinary operation.
To avoid Java issues:
Do not use a JDK from a directory location containing spaces (if on Windows, please use the short form directory names if you must use a JDK in a directory location containing spaces).
Ensure that your JDK "bitness" matches that of your Oracle Fusion Applications install. If you are using a 64-bit JDK, make sure that you are using a 64-bit Oracle Fusion Applications installation.
On Windows systems, failure to ensure this can result in T2P getting confused regarding the Oracle Inventory on 64 bit boxes. On 64 bit Windows machines, both the Program Files and Program Files (x86) directories can contain inventories: Program Files is used for 64-bit installations.
On the target machine, after running the pasteBinary operation, be sure to execute the related commands, pasteConfig, extractMovePlan, and so on) from the following extracted locations:
(UNIX) MW_HOME/oracle_common/bin (Windows) MW_HOME\oracle_common\bin
If Oracle Web Services Manager policy migration fails when using the pasteConfig script, the issue can occur if you do not use the proper Oracle Identity Management database service name. The SID value should not be used.
Oracle Platform Security Services JPS root fusionappspolicies not created during an Oracle Identity Management delta movement.
Ensure that the instructions given in the "Update the ADF Credentials" section in the Oracle Fusion Applications Administrator's Guide for the Identity Store have been followed and that the resulting data is properly populated.
During pasteConfig operations, invalid editing of the move plans can result in validation errors. Please ensure the following:
Use a syntax validating editor to edit the move plan files. This ensures that simple XML formatting rules within the file are followed.
Ensure that no READ_ONLY elements have been edited and modified.
Ensure that no new READ_WRITE elements have been added.
When performing the pasteConfig operation for Oracle Internet Directory, the operations fails with an invalid database error.
During a pasteConfig operation, if the target database is the same as the source database, then the Oracle Internet Directory component name should be different.
If the target database is different from the source database, then the Oracle Internet Directory component name needs to be the same.
Note:
Target database also needs to be a clone of source database.
The pasteBinary operation may incorrectly calculate the free disk space. This can be due to the target location being on a Network File System (NFS) share or a different type of file system (such as data ONTAP).
If you are sure that there is enough free disk space for the archive to be extracted and configured, use the -ignoreDiskWarning (-idw) argument to ignore the disk space check error
Ensure that the copyConfig and pasteConfig operations for Node Manager operations are performed. The copyConfig and pasteConfig operations for the domains do not configure the Node Manager properties. Failure to do so can result in defaulted Node Manager configuration rather than the properties from the source environment.
See the "Running the copyConfig for Node Manager" section and the "Running the pasteConfig for Node Manager" sections for command line usage.
This section describes common problems and solutions. It includes the following topics:
Section 13.3.1, "Running Fusion Applications Role Category Seeding Task Returns Exception"
Section 13.3.3, "pasteBinary Returns Error for Middleware Home Directory"
Section 13.3.4, "pasteBinary Fails With deriveNewWLSProperties Error"
Section 13.3.5, "copyConfig Fails When Checking Oracle Homes"
Section 13.3.8, "copyConfig Fails with MaxMessageSizeExceededException"
Section 13.3.9, "Cannot Run pasteConfig for Node Manager After Initial Failure"
Section 13.3.10, "Copy or Paste Fails with a DiskQuotaExceeded Error"
Section 13.3.11, "Exceptions Are Raised for Oracle SES During pasteConfig"
Section 13.3.13, "No Oracle Fusion Applications Targets Display in Oracle Enterprise Manager"
Section 13.3.14, "OutOfMemory (Heap Size) Error while Unpacking the Domain in pasteConfig"
Section 13.3.16, "wf_client_config.xml Refers to Files in the Source Environment"
Section 13.3.20, "Cannot Enable the SSL Connection Type After Node Manager Movement"
Section 13.3.21, "Cannot Retrieve Documents from Oracle WebCenter Content: Imaging Viewer"
Section 13.3.22, "OHS Component Will Not Start After pasteConfig"
Section 13.3.23, "Application Scope Data Sources Are Not Copied"
Section 13.3.24, "Java Runtime Environment Was Not Found During Cloning Operations"
For information about the error messages you may encounter when moving Oracle Fusion Applications components across environments, see Oracle Fusion Middleware Error Messages Reference.
An exception is raised when you run the Fusion Applications Role Category Seeding task in Oracle Identity Manager.
Problem
Oracle Virtual Directory was not configured properly for the change log adapter.
Solution
Set the Remote Base value to an empty value.
In Oracle Directory Services Manager, select the Adapter tab, then General.
Remove the value in the Remote Base field.
Restart Oracle Virtual Directory and Oracle Identity Manager.
While troubleshooting binary movement issues, you may run across some permission-related messages. These messages are expected due to the nature of transient files between the source and target machines. In most cases, you can safely ignore these message if there are no other movement failures. Some examples of permission errors are:
/bin/chmod: cannot access /webtier/cfgtoollogs/configToolAllCommands' : No such file or directory '/bin/chmod: cannot access /webtier/network/lib/env_network.mk.32' : No such file or directory
The pasteBinary script returns an error when you apply the Middleware home directory to the target environment.
Problem
Incorrect information was entered.
Solution
Take the following actions to recover:
Delete the target Middleware home.
Remove the Oracle home entry from the Oracle inventory, if it is present.
For Windows, remove the shortcut for the Middleware home and Oracle home.
You encounter the deriveNewWLSProperties error when you run the pasteBinary script.
Problem
The target Middleware home location is inside another Middleware home.
Solution
Specify a target Middleware home location that is not inside another Middleware home when you run the pasteBinary script.
The copyConfig script fails when it checks Oracle homes.
Problem
The inventory configuration is invalid.
Solution
Make sure that inventory.xml is valid. Do not update the inventory manually.
The copyConfig script takes an unusually long time to complete.
Problem
The metadata export is requiring a lot of time.
Solution
Check the logs to make sure it is taking time to export. Clean up the database cache or purge the database. You can also collect the database statistics.
The copyConfig script fails with the error CLONE-20408.
Problem
The machine is specified as unix-machineType in the config.xml file.
Solution
Edit the machine type in DOMAIN_HOME/config/config.xml to remove xsi:type="unix-machineType" in:
<machine xsi:type="unix-machineType"> <name>machine_name</name>
For example:
<machine> <name>machine_name</name>
Restart the servers after you update the config.xml file.
The copyConfig script fails on the WebLogic Server domains.
Problem
You encounter the following error when you are executing copyConfig of Oracle WebLogic Server Domains:
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '1000080' bytes exceeds the configured maximum of:'1000000' bytes for protocol: 't3'.
Solution
Change the message size limit by using the T2P_JAVA_OPTIONS environment variable to pass the -Dweblogic.MaxMessageSize=2000000 property to both the copyConfig and pasteConfig scripts.
You cannot run the pasteConfig script for Node Manager after running it failed the first time.
Problem
The pasteConfig script for Node Manager creates the nodemanager.properties file and other files, which are left out when there is a failure.
Solution
Clean up all the files in the target Node Manager home location before re-running the pasteConfig script.
A user block limit/disk quota error causes either a copy or a paste operation to fail.
Problem
You faced the following exception during a copy or paste operation: write failed, user block limit reached. SEVERE: Disk quota exceeded.
Solution
To ensure that enough disk space is allocated to this user, you can either expand the disk quota for that user or remove the disk quota limit for that user.
The pasteConfig script returns exceptions for Oracle SES when you apply system components to the target environment.
Problem
pasteConfig was invoked a second time on the same Oracle SES instance.
Solution
Clean up the Oracle SES instance in the target environment using the searchadmin command-line interface before invoking pasteConfig again.
For more information about the searchadmin commands, see Oracle Secure Enterprise Search Administration API Guide.
To clean up the Oracle SES instance:
At the operating system prompt, log in to the searchadmin command-line interface using the following command:
SES_ORACLE_HOME/bin/searchadmin -p adminPwd -c adminWebServiceUrl
For example:
>/scratch/yummy/wls/Oracle_SES1/bin/searchadmin -p welcome2SES -c http://donuts02.us.example.com:8001/search/api/admin/AdminService
The prompt changes to SES>.
Enter the following searchadmin commands in the order specified:
SES>deleteAll altWord SES>deleteAll proxyLogin SES>deleteAll facetTree SES>deleteAll suggLink SES>deleteAll sourceGroup SES>deleteAll schedule SES>deleteAll docServicePipeline SES>deleteAll docServiceInstance SES>deleteAll source
Delete any remaining sources:
Log in to the Oracle SES Administration GUI.
Select the Sources secondary tab.
On the Sources page, delete any remaining sources that are listed.
Return to the Oracle SES searchadmin command line and enter the following commands:
SES>deleteAll searchAttr SES>delete thesaurus -n DEFAULT
Note:
The thesaurus may not exist. You can ignore the error message EQA-11000: The object with key "[name=DEFAULT]" and type "thesaurus" was not found.
Check whether or not an active identity plug-in exists:
Enter the following command:
SES>getAllStates identityPlugin -o idPluginState.xml
The idPluginState.xml file is generated in SES_ORACLE_HOME/bin/idPluginState.xml.
Open the idPluginState.xml file and search for the phrase <search:value>ACTIVE</search:value>.
If <search:value>ACTIVE</search:value> does not exist, then there is no active identity plug-in. Skip to Step 8.
If <search:value>ACTIVE</search:value> exists, continue to Step 6.
Note:
There will be at most only one active identity plug-in at a time.
Scroll up the page to locate the jarFilePath and managerClassName of this active identity plug-in.
Deactivate the plug-in by entering the following command:
SES>deactivate identityPlugin --JAR_FILE=jarFilePath --MANAGER_CLASS=managerClassName
Check whether or not partitionConfig is active:
Enter the following command:
SES>getState partitionConfig -o partitionConfigState.xml
The partitionConfigState.xml file is generated in SES_ORACLE_HOME/bin/partitionConfigState.xml.
Open the partitionConfigState.xml file and search for the phrase <search:value>ACTIVE</search:value>.
If <search:value>ACTIVE</search:value> does not exist, then there is no active partitionConfig. Skip to Step 10.
If <search:value>ACTIVE</search:value> exists, continue to Step 9.
Enter the following two commands:
SES>export partitionConfig -o partitionConfig.xml SES>update partitionConfig -i partitionConfig.xml -a remove
Delete the storage areas by entering the following command:
SES>deleteAll storageArea
The SOA server fails to start after you have completed the procedures for moving the Oracle Fusion Applications components.
Problem
An incorrect configuration of the Oracle Coherence framework that is used for deployment is preventing the SOA system from starting. The deployment framework must be properly customized for the network environment on which the SOA system runs.
Solution
Modify the Oracle Coherence cluster configuration by editing the values for the following startup parameters:
tangosol.coherence.wka1
tangosol.coherence.localhost
tangosol.coherence.localport
tangosol.coherence.wka1.port
For information, see the "Configuring Oracle Coherence for Deploying Composites" section in Oracle Fusion Middleware Enterprise Deployment Guide for Oracle SOA Suite.
After moving the Oracle WebLogic Server domain configuration, the Oracle Fusion Middleware targets display in Oracle Enterprise Manager, but Oracle Fusion Applications targets (such as product families, products, and J2EE applications) do not.
Problem
The ASK tables in the database (such as ASK_DEPLOYED_APPLICATIONS and ASK_DEPLOYED_DOMAINS) are not populated.
Solution
Check to make sure that Topology Manager functions properly when you run the copyConfig and pasteConfig scripts. Also check to make sure that the move plan values are properly modified.
The pasteConfig script hangs, then displays a java.lang.OutOfMemoryError:Java heap space error while the unpack command was executed as part of the pasteConfig operation.
Problem
There is insufficient memory to process the large files (either spurious soft links to directories with many large files or large log files) of the Oracle WebLogic Server domain.
Solution
Remove the large, unnecessary files, and run the copyConfig and pasteConfig scripts again. You can also increase the JVM heap size by using the -Xmx option for maximum heap size, and the -Xms option for initial heap size. For example:
export CONFIG_JVM_ARGS="-Xms512m -Xmx1024m"
You cannot run the pasteConfig script because of an OutOfMemory error.
Problem
You encounter an OutOfMemory error while the pasteConfig operation is trying to start the server.
Solution
Set the JVM system variable USER_MEM_ARGS, as described in the "Setting the JVM System Variables on Windows" section of the Oracle Fusion Applications Administrator's Guide.
After moving your Oracle Fusion Applications environment from one host to another, the wf_client_config.xml files still refers to the files in source environment.
Problem
The wf_client_config.xml files were not edited to change the value of the <rootEndPointURL> element.
Solution
Edit the wf-client-config.xml file for each application to change the value of the <rootEndPointURL> element.
To locate each wf_client_config.xml file and edit the <rootEndPointURL> element:
Run the following command from the fusionapps Middleware directory (for example, /net/mount1/appbase/fusionapps):
find . -name wf_client_config.xml
For each wf_client_config.xml file (one for each application), change the value of <rootEndPointURL> to point to the internal virtual host port for the SOA cluster on the target environment. For example:
<rootEndPointURL>http://abcdef14.us.example.com:10615/soa-infra</rootEndPointURL>
PolicyManagerValidator fails to preload on startup in the /wsm-pm web application.
Problem
An MDSConfiguration exception was encountered in parseADFConfiguration at oracle.adf.share.config.ADFMDSConfig.parseADFConfiguration.
Solution
Manually make the following targeting changes:
Add the Oracle Web Services Manager targets: admin, ess, and spaces.
Remove the wsm-pm target: admin.
You encounter issues while running the pasteConfig script.
Problem
The custom scripts in the ORACLE_HOME/COMMON/WLST directory are invalid.
Solution
Validate the custom scripts in the ORACLE_HOME/COMMON/WLST directory.
You receive a setClusterAddress() failed error while configuring the machine in the target environment.
Problem
The cluster member host address contains an illegal character in the domain or host name or address.
Solution
Replace the domain or host name or address with a valid value.
After you move Node Manager from the source environment to the target environment, you encounter a javax.net.ssl.SSLKey exception because the connection between the Administration Server and Node Manager is of connection type plain socket, instead of SSL.
Problem
The keystores were not created properly.
Solution
Complete the following tasks to properly create the keystores:
Create the identity keystores and trust keystores for the middle tier Managed Servers and Node Manager. See the "Ensure Source and Destination Environments Fulfill Requirements" section in Oracle Fusion Applications Administrator's Guide.
Perform Node Manager movement, editing the Node Manager properties in the move plan. See the "Move Plan Properties for Node Manager" section in Oracle Fusion Middleware Administrator's Guide.
After moving the Oracle Fusion Applications components, you cannot retrieve documents from the Oracle WebCenter Content: Imaging Viewer.
Problem
The ixTransformer no longer has execute rights when bundled with copyConfig.sh and extracted with pasteConfig.sh. The server must be able to execute this file in order to transform documents into TIF format for viewing.
Solution
Set the permissions to 750 (on UNIX systems) for files in the unpacked domain with the extension .sh. For information, see "Task 3 Modify Oracle Information Rights Management Settings" in Oracle Fusion Middleware Administrator's Guide.
Upon completion of the pasteConfig script for an OHS component, the script automatically attempts to start the new component, but it will not start.
Problem
SSL was previously configured on the source environment, but upon being cloned to the target, ssl.conf still points to the old wallet location, which no longer exists. The OHS cannot find the wallet, which results in startup failure.
Solution
If the non-default wallet was used at the source environment, then export the wallet from the source environment and import it to the target environment. Then, update the wallet location (if changed) in the ssl.conf file in the target environment.
After moving the Oracle Fusion Applications components, all Oracle ADF applications that use the application scope data sources fail to start.
Problem
The copyConfig script handles only global data sources defined in each Oracle WebLogic Server domain.
Solution
For application level data sources, you must deploy the Oracle ADF application configured with the application level data sources to a server in the target domain, and manually configure the data sources on the target domain.
When performing cloning operations as a part of the paste operations, the Java Runtime Environment was not found when runInstaller / setup.exe is executed from the cloned Oracle Home.
Problem
While invoking runInstaller / setup.exe from cloned Oracle Home, the following message is displayed:
The Java RunTime Environment was not found when runInstaller / setup.exe is executed from the cloned Oracle Home.
The error occurs because the JRE was not found at /tmp/OraInstalltime_stamp/jre/bin/java. Therefore, the Oracle Universal Installer cannot be run.
Solution
To resolve this issue:
Visit http://www.javasoft.com and install JRE version 1.3.1 or higher and try again.
Ensure the JRE_LOCATION setting in ORACLE_HOME/oui/oraparam.ini points to a valid JRE location.
This section describes general approaches for diagnosing moving Oracle Fusion Applications components across environments. It contains the following topic:
The copyConfig process tries to export the Oracle Metadata Repository (MDS) data of all applications. Some applications may have MDS data, while some may not have any data—or in some cases, the Managed Server may be down or the application may not be in active state.
To diagnose issues related to MDS export, check the copyConfig logs after the copyConfig process is complete to make sure that all the applications with MDS data are exported successfully.
In the section of the copyConfig log, shown in Example 13-1, the listed applications do not have any MDS data, which causes the failure.
Example 13-1 Sample copyConfig Log
FINE : [J2EEComponentCreateCloner:doClone] ##################################### MDS Export Summary ############################################# FINE : [J2EEComponentCreateCloner:doClone] MDS Export of some applications failed. Check the following list of applications and make sure that MDS data is not associated with these applications. - FusionEsdApp, FusionCtdApp, FMW Welcome Page Application, em, ESSAPP, ESSJobTypesAPP, DiagnosticsUI-Ess, SalesApp, oraclediagent, odiconsole, wsil-wls, DMS Application, AppsLoggerService, FileAdapter, DbAdapter, JmsAdapter, AqAdapter, FtpAdapter, SocketAdapter, MQSeriesAdapter, OracleAppsAdapter, OracleBamAdapter, usermessagingserver, usermessagingdriver-email, usermessagingdriver-extension, worklistapp, b2bui, DefaultToDoTaskFlow, composer, OracleBPMProcessRolesApp, OracleBPMComposerRolesApp, OracleBPMWorkspace, SimpleApprovalTaskFlow, SetTransformService, DiagnosticsUI-Assembly, Diagnostics-JmxApi, FINE : [J2EEComponentCreateCloner:doClone] #########################################################################################################
You can check the reason for failure for each of these applications from the copyConfig log.
Similarly, the pasteConfig process tries to import the MDS data that was exported during copyConfig. To diagnose issues related to MDS import, check the pasteConfig logs to make sure that the MDS import is successful.
Import failed for ProductManagementApp, ProductManagementCommonApp, and CostManagementApp in the pasteConfig log, shown in Example 13-2.
Example 13-2 Sample pasteConfig Log
FINE : [J2EEComponentApplyCloner:doClone] ##################################### MDS Import Summary ############################################# FINE : [J2EEComponentApplyCloner:doClone] MDS Import of following applications failed. ProductManagementApp, ProductManagementCommonApp, CostManagementApp, . Check managed server and clone logs for details. FINE : [J2EEComponentApplyCloner:doClone] #########################################################################################################
Resolve any import failures after the pasteConfig is complete. You can use the importMetadata command to import the MDS after pasteConfig is complete. For information, see Oracle Fusion Middleware WebLogic Scripting Tool Command Reference.
To recover from a pasteConfig error:
Check the logs to identify the issue.
Stop all processes related to the domain.
Delete the following directories:
/net/mount2/instance/domains/host/domain_name /net/mount2/instance/applications/domain_name
For LDAP, drop the schemas and re-create them using the Oracle Fusion Middleware Metadata Repository Creation Utility, and delete the domain node.
For Metadata Services (MDS), you can overwrite the schemas without dropping the domain node.
When you execute the pasteBinary or pasteConfig scripts and enter incorrect information in the move plan, the scripts return an error. In some cases, the scripts may have partially completed the paste operation.
To recover from a pasteConfig error when applying system components to the target:
Uninstall the instance.
If you cannot uninstall the instance, stop all processes related to that instance and delete the Oracle instance.
|
 Copyright © 2012, 2013, Oracle and/or its affiliates. All rights reserved. Legal Notices |
|
