21 Troubleshoot Oracle Enterprise Performance Management

This section describes common problems that might be encountered when using Oracle Enterprise Performance Management, and explains how to solve them.

The following topics are discussed:

21.1 Introduction to Reporting with Enterprise Performance Management

Oracle Enterprise Performance Management System products comprise a high-end financial reporting system separate from the Oracle Business Intelligence products described in Troubleshoot and Tune Oracle Fusion Reports. This system is best for more complex financial calculations, and handling drill-down through complex financial accounting hierarchies. It includes no seeded reports. A seeded and auto-populated Essbase database is available for General Ledger.

The components of the Enterprise Performance Management System that are integrated with Fusion Applications include the following:

Oracle Hyperion Financial Reporting
Oracle Hyperion Enterprise Performance Management Workspace
Oracle Smart View for Office
Oracle Hyperion Calculation Manager
Oracle Essbase
Oracle Essbase Administration Services
Oracle Hyperion Provider Services
Oracle Essbase Studio
Oracle Data Relational Management
Oracle Smart View for Office

21.2 Problems and Solutions for Foundation Services

This section describes some common problems and solutions for Foundation Services. The following topics are discussed:

21.2.1 Get Started with Logging Basics for Foundation Services

Log files are the best tools for analyzing what might be wrong with the system configuration. The logging configuration file, the defaults it ships with, and instructions for changing change those defaults will help with the analysis.

Configuration: The logging.xml file in the following directories contains the loggers and their default levels:
```
(UNIX) DOMAIN_HOME/servers/server_name/logs
```
The DOMAIN_HOME is named BIDomain.
Output: The default locations of the loggers are rooted in the following directories:
```
(UNIX) DOMAIN_HOME/servers/server_name/logs
```
These files include the following:
- /registry/registry.log
- /css/css.log
- /workspace/Framework.log
- /workspace/workspace.log
- /financialreporting/fr.log
- CalcManager.log
Loggers: Increase or decrease the levels of the following loggers:
- oracle.EPMCSS
- oracle.bi.bifndnepm.epmreg
- oracle.bi.bifndnepm.bpmui
- oracle.bi.bifndnepm.workspace
- oracle.EPMFR
- oracle.EPMAnnotations
- oracle.EPMJCR
- oracle.EPMADM
- oracle.bi.bifndnepm.calcmgr

21.2.1.1 Change BI Logger Levels

To change logger levels, perform the following steps:

Log in to Fusion Applications Control.
From the navigation pane, expand the farm and then WebLogic Domain.
Right-click a Managed Server from within the domain (each server's log levels can be independently set).
Choose Logs > Log Configuration.
In the Logger Name column, expand the oracle runtime loggers to display loggers.
Change the logging level as appropriate.

21.2.2 Web Traffic Snooping to Resolve Issues

Problem

Some components are not operating correctly, or appear to have some minor issues. This is a broad category, but snooping the web traffic can help development resolve some issues by looking for irregular status, content, and redirection, and just providing the flow of calls up to the problem area.

Solution

To solve this issue, perform the following steps:

Connect to http://www.telerik.com/fiddler and install the appropriate version.
Launch Fiddler through one of the following methods:
- Start Menu: go to All Programs, and then click Fiddler.
- Microsoft Internet Explorer: go to Tools, and then click Fiddler.
Connect to the starting URL and continue normally until the problem occurs.
When the problem occurs, in Fiddler, choose File, then Save, and then All Sessions.
Enter a name for the session archive (.saz) file.

CAUTION: Manage the file carefully. The file may contain sensitive information, such as user ID and password information.

21.2.3 Enterprise Performance Management Registry Problems

This section describes common problems and solutions for Enterprise Performance Management (EPM) Registry. It contains the following topics:

21.2.3.1 Enterprise Performance Management (EPM) Registry Initialization Fails

Problem

The logs indicate that the Enterprise Performance Management (EPM) Registry initialization failed in the BIDomain domain. Oracle Fusion General Ledger and Oracle Fusion Customer Relationship Management (CRM) code running within the FinancialsDomain and CRMDomain domains, respectively, has direct integration with the EPM Registry API and consumes EPM Security Component, which also tightly integrates with the EPM Registry API. When EPM Registry initialization fails, it greatly affects communication with downstream EPM products in the BI domain.

Solution

To resolve this problem, perform the following steps:

Login to Oracle WebLogic Server Console as the Oracle WebLogic Server administrative user.
From Domain Structure portlet, navigate to Services then Data Sources, and then select EPMSystemRegistry Datasource from the available list of data sources.
Click on the Connection Pool tab and confirm that the correct information is available for URL property.
Click on the Targets tab and confirm that the data source is targeted to all of the required Oracle WebLogic Servers and clusters running in the Financials domain.
Click on the Monitoring tab and confirm that the data source for all of the targeted Oracle WebLogic Servers and clusters is in Running state. Optionally, try clicking Test Data Source button to test the data source for a given server.
If, in Step 5, the data source was not in the Running state, click the Control tab and attempt to start the data source using the Start button.
If any errors were encountered during Step 5 or 6, consider restarting the Oracle WebLogic Server domain.

21.2.3.2 Connection to Essbase Server Fails

Problem

An error is returned when connecting to Essbase Server from the Fusion Applications domain, using the Essbase Cluster lookup URL that is built using the host and port information retrieved for Essbase_FA_Cluster, from EPM Registry using the Registry API.

Solution

To resolve this issue, perform the following steps:

Verify that the host and port values for the LOGICAL_WEB_APP component in EPM System Registry, where webAppType is PROVIDER_SERVICES_WEB_APP, exactly matches the host and port that the APS Web application is running on in the BI domain. Follow these steps:
1. Run the EPM Registry Editor utility using the following command:
```
(UNIX)  APPLICATIONS_CONFIG/BIInstance/config/foundation/11.1.2.0/epmsys_registry.sh view LOGICAL_WEB_APP | tee -a logical_webapp_report.txt
```
2. In the logical_webapp_report.txt file that is generated as a result of Step 1, search for the following string:
```
*"webAppType = PROVIDER_SERVICES_WEB_APP"  
```
3. For the COMPONENT containing the matching string, verify that HOST/port/SSL_PORT exactly match what the APS Web application is actually running on in the BI domain. If any of these values are different in the Essbase cluster lookup URL, they should be appropriately updated in EPM Registry for the given Logical Webapp Component. This solution is discussed in Host and Ports Do Not Match.
4. Confirm that the APS Web application is in an Active state in the BI domain, by logging into the BI Administration Server console and reviewing the deployment profile of APS.
If verification of Step 1 successfully passes, verify the Oracle Access Manager protection policies and confirm that the Essbase cluster lookup URL is excluded from those policies

21.2.3.3 Host and Ports Do Not Match

Problem

If the host or port for Logical Web App components in the EPM Registry do not exactly match the actual host and port that EPM Web applications are running on in the BIDomain domain, there may be connection or launch issues for EPM Web applications from within the Fusion user interface. In this case, you must update the host and port for Logical Web App components in EPM Registry.

Solution

To resolve the issue, perform the following steps:

Execute the following command so that respective component IDs are available in the file for later use:
```
(UNIX)  APPLICATIONS_CONFIG/BIInstance/config/foundation/11.1.2.0/epmsys_registry.sh view LOGICAL_WEB_APP | tee -a logical_webapp_report.txt
```
The EPM Registry Dump (.html report) that is created by running epmsys_registry.sh and epmsys_registry.bat does not include the component IDs that are required to update the host and port properties for individual components.

After successfully executing this command, the logical_webapp_report.txt file now contains exported data for five Logical Web App components - something similar to the following:
```
COMPONENT - 1
 NAME -  Default
 ID -  a01b453873d1e7b5S7b70c01f12e34faed1bS7fdc
 TYPE -  LOGICAL_WEB_APP
 HOST -  hostname
 HYPERION HOME -  /scratch/aime1/work/mw8747/Oracle_BI1
 PROPERTIES -
           webAppType = CALC_WEBAPP
          context = calcmgr
```
The ID highlighted above in bold will be available for COMPONENT - [1-5]; this is what will be required to use in subsequent steps.

For the individual ID of COMPONENT - [1-5], execute the following EPM Registry update command:

(UNIX)  APPLICATIONS_CONFIG/BIInstance/config/foundation/11.1.2.0/epmsys_registry.sh updateproperty \#ID/@port port

For example:

./epmsys_registry.sh updateproperty \#a01b453873d1e7b5S7b70c01f12e34faed1bS7fdc/@port 16050

Validate that the host and port for Logical Web App components have been successfully updated:

(UNIX)  APPLICATIONS_CONFIG/BIInstance/config/foundation/11.1.2.0/epmsys_registry.sh view LOGICAL_WEB_APP | tee -a logical_webapp_update_report.txt

After these updates, restart the BIDomain, CRMDomain and FinancialsDomain domains, so that new changes take effect. See the section Start an Oracle WebLogic Server Domain for a Product Family.

21.2.4 Hyperion Security Username/Password Authentication Fails during Enterprise Scheduler Services Essbase Cube Creation in the Domain

Problem

The following error about Oracle Enterprise Scheduler Essbase Cube creation in the BIDomain domain is returned because of a Hyperion security username and password authentication failure:

EPMCSS-00301: Failed to authenticate user. Invalid credentials. Enter valid credentials.

Solution

To resolve this issue, verify that the password stored in credential store for Application Identity (App ID) used by the Oracle Enterprise Scheduler job to connect to Essbase exactly matches the password for the given Application Identity in the underlying LDAP Identity Store by performing the following steps:

Identify which App ID is used by the Oracle Enterprise Scheduler job in the given domain, to connect to an Essbase single cluster.
The following application identities are used for the respective domains:
- Oracle Fusion Customer Relationship Management: FUSION_APPS_CRM_ADF_APPID
- Oracle Fusion Projects: FUSION_APPS_PRJ_ESS_APPID
- Oracle Fusion General Ledger: FUSION_APPS_GL_ESSBASE_APPID
Retrieve the credential with a given map and key, use scripting to invoke the MBean operation JpsCredentialMXBean.getPortableCredential(map, key).
For example:
```
(map="oracle.wsm.security", key="FUSION_APPS_AMX_APPID-KEY")
```
The credential obtained in Step 2 can then be used to compare the existing password for the given App ID in the underlying LDAP ID Store, using the following command line ldapcompare:
```
ldapcompare -h hostname -p port -D "bind_user_dn" -w bind_user_password -a userpassword -v appid_password_from_credstore -b "appid_dn"
```
If Step 3 does not run with positive results, it implies that the credential store password and the LDAP store password failed the comparison test. In this case, passwords in credential store and LDAP ID Store should be synchronized.

21.2.5 EPM Workspace Problems

This section describes common problems and solutions for EPM Workspace. It contains the following topics:

21.2.5.1 Set Up Client Debugging Tool

Problem

Errors occur with certain functionality.

Solution

To begin debugging, perform the following steps:

Check the status of Workspace by entering the following URL:
```
http://server_name:19000/workspace/status 
```
This URL displays a summary of any fatal errors that prevent Workspace from running, and a list of integrated products.
Enable the client debugging feature by selecting Navigate , Administer , Workspace Settings , Server Settings, and by then selecting Client Debug Enabled.
All users who log on after selecting this option will be affected. The URLs that can be accessed include the following:
```
http://server_name:19000/workspace/debug/configInfo.jsp 
http://server_name:19000/workspace/debug/userInfo.jsp 
```
The first URL shows information about the system including metadata from integrated products (for example, menus, preferences panels, and so on); the second URL shows information about the current user, including groups, effective roles, and Workspace menu items as interpreted and filtered for the current user.

21.2.5.2 Startup Fails to Respond or Error Occurs

Problem

If the http://server_name:19000/workspace/ URL fails to respond or an error occurs, the following information explains the task flow that the software requires, to assist in troubleshooting.

Workspace initialization happens at the first request, not at server startup. If initialization fails, it is reattempted at every subsequent request. Any error during initialization is logged in the workspace.log file. The error message displays a page that replaces the normal Workspace Log on screen. Finally, the error is reported in the status report servlet, /workspace/status. Only the log files include Exception stack traces.

Solution

The following tasks must successfully complete in order before Workspace allows log ons, and can be matched up in the workspace.log file:

Parse and validate the file /conf/WSProducts.xml in the Workspace Web application.
Initialize the Shared Services Registry:
1. Connect to the Shared Services Registry.
2. Find the unique WORKSPACE component in the Shared Services Registry.
3. Find the unique Workspace LOGICAL_WEB_APP component in the Shared Services Registry.
4. View the Workspace configuration properties from LOGICAL_WEB_APP.
5. Find at least one WEB_SERVER component that is a child of WORKSPACE.
Update LOGICAL_WEB_APP with host and port information from the Oracle WebLogic Server MBean.
Update LOGICAL_WEB_APP with any missing configuration properties.
Initialize Shared Services.
Scan the Shared Services Registry for integrated products, and match every discovered product with a configuration file from the following resources:
- /conf/WSProducts.xml (Workspace)
- WorkspaceConfig file attribute of LOGICAL_WEB_APP (all other products)

21.2.6 Calculation Manager

This section describes common problems and solutions for Calculation Manager. The following topics are discussed:

21.2.6.1 Essbase Applications Are Not Listed Under the Essbase Node

If no Essbase applications are listed under the Essbase node, it could be due to any of the following reasons:

You are not provisioned to work with Calculation Manager. There must be at least one of the following access privileges to work with Calculation Manager:
- Create General Ledger Allocation Rules
- Administer Allocation Rules
- Generate General Ledger Allocation Rules.
The Essbase cluster name is not Essbase_FA_Cluster. If the Essbase server is not under this cluster, it is ignored by Calculation Manager.
The stripe ID for General Ledger is different than the stripe ID for Workspace, so different security may be applied when Calculation Manager communicates with Essbase. Check the Calculation Manager log file to ensure that the stripe ID (or the application ID) for Calculation Manager is the same as the stripe ID for General Ledger.
The Essbase application contains an empty Comment field. Each Essbase application must contain at least one character in the Comment field, or it is ignored by Calculation Manager.
The application is not an Essbase aggregate storage application; only Essbase aggregate storage applications may be used with General Ledger.

21.2.6.2 Business Rules Cannot be Deployed to General Ledger

Problem

Business rules cannot be deployed from Calculation Manager to General Ledger, and the following error message is received:

no WSDLLOCATION set

Solution

In the EPM registry, ensure that the WSDLLOCATION property is defined for Calculation Manager. The WSDLLOCATION property contains the General Ledger web services URL.

21.2.7 Smart View Problems

This section describes common problems and solutions for Smart View. The following topics are discussed:

21.2.7.1 Enable Advanced Logging to Troubleshoot

Problem

Smart View collects and records events, errors, and other information in a log file, typically SmartViewLogs.log. When experiencing performance or other issues, enable the logging in this file of additional information about profiling and requests and responses between Smart View and the server. This additional information can help Oracle Support to troubleshoot the issues.

Solution

To enable additional troubleshooting information, perform the following steps:

Close all Microsoft Office applications.
Select Start, and then Run.
Enter regedit and click OK to open the registry.
Navigate to HKEY_CURRENT_USER\Software\Hyperion Solutions\HyperionSmartView\Options.
Right click Options, select New, and then String Value.
Name the new string value Profile.
Double-click Profile to open Edit String.
From Edit String, under Value Data, enter one of the following values:
- 0: Logging is not enabled. Initialized value bEnableProfiling is set to false.
- 1: Profile entries are logged when an event completes. This setting provides little information about abrupt terminations, but performance is better than with 2.
- 2: Profile entries are logged immediately. Use this setting to determine the function in which a termination occurred. This setting provides the most detailed information but has the greatest negative impact on performance.
Close the registry.
Restart Excel.
From the Smart View ribbon, select Options.
Go to the Advanced page and ensure that Route Messages to File is selected. The log file, typically SmartViewLogs.log, will now begin recording profiling and request/response information between Smart View and the server in addition to the other information it records.

To improve performance, when there is no longer a need to log profiling and request/response information, delete the Profile entry that was created in the registry.

21.2.7.2 Re-enable Smart View in Microsoft Office Applications

Problem

Smart View may become disabled in Microsoft Office applications.

Solution

Smart View is a COM add-in, which Microsoft Office applications can disable. To re-enable Smart View, follow instructions in Excel Help for re-enabling COM add-ins.

21.2.7.3 Timeout Errors: Increase the Timeout Limit for Smart View Client Computers

Problem

The server takes longer to process a Smart View operation than the timeout value set on the client computer, and users receive a connection timeout error, or zero values may be displayed for Smart View functions.

Solution

To resolve this issue, increase the timeout limit for Smart View client computers. Smart View uses Win-Inet APIs to communicate with the provider. These are the same modules that Internet Explorer uses. To increase the timeout value for a Windows client computer, perform the following steps:

In the Windows registry of the client computer, navigate to the following location:
```
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\InternetSettings
```

Set the following values:

"ReceiveTimeout"=dword:00dbba00
"KeepAliveTimeout"=dword:0002BF20
"ServerInfoTimeout"=dword:0002BF20

21.2.7.4 Shared Connections Panel Does Not Display Server Names

Problem

The Shared Connections panel of the Smart View Panel displays no server names. This typically happens when the applications are not properly registered during installation and configuration.

Solution

Review the information in the Oracle Hyperion Enterprise Performance Management System Installation and Configuration Guide to ensure that applications and servers are properly configured.

21.3 Problems and Solutions for Financial Reporting

This section describes common problems and solutions for Financial Reporting. The following topics are discussed:

21.3.1 Set Up Data Sources and Debug Setup and Connectivity Issues

This section describes the following common problems and solutions for setting up data sources and debugging setup and connectivity issues:

21.3.1.1 Unable to Connect to Essbase Cube

Problem

It is not possible to connect to an Essbase cube.

Solution

To resolve this issue, verify that the Essbase Server name and port are correct. Verify the user credentials used to make the connection are correct and the User has at least read rights to the cube.

21.3.1.2 Error While Creating Database Connection

Problem

An error occurred while creating a database connection.

Solution

To resolve this issue, check the Financial Reporting log file. If the log file has the following error, a Catalog GUID refresh is required:

"Caused by: javax.jcr.LoginException: access denied for user to path /users/userid"

For more information, see the Refreshing User GUIDs section in the Security Guide for Oracle Business Intelligence Enterprise Edition.

21.3.1.3 Cannot Set Up a Database Connection

Problem

When creating a database connection, it is appended to the list in the Database Manager dialog box.

Solution

To add a database connection, perform the following steps:

In Workspace, select Tools, then Database Connection Manager.
Click New.
In Database Connection Name, enter a name for the database connection.
Select a data source type of Essbase.
In Server, add the Essbase Server Name and Port Number.
Re-enter the User ID and Password.
To add application and database names, click the magnifying glass, and make the selections.

The Application Lookup button displays a tree view of the applications and corresponding databases; the applications are listed as parents and the databases are listed as children. Search on an application or database. For data sources that are not associated with a database, only applications are listed.
Click OK.

The database connection profile is appended to the list in Database Connection Manager dialog box.

21.3.2 Common Administrative Tasks and How to Debug Issues

This section describes common problems and solutions for setting up data sources and debugging setup and connectivity issues:

21.3.2.1 Invalid Grid Error When Running a Report: Re-Save to the Database

Problem

When running a report, following error is displayed:

Error 1012:Report contains an invalid grid. The following dimensions could not be found: <i>xxx</i>

This error may occur if the database was recently changed on the report.

Solution

To resolve the issue, open and save the report that has mismatched dimensions. This causes the dimensions that existed in the old database connection but not in the new database connection to be removed. The dimension and its members that existed in the rows and columns are removed from the grid. If, as a result of the removal, no dimension exists in the row or column, add a valid dimension to the cleared row or column in order for the report to run. Dimensions that exist in the new database connection but not in the old one, are added to the POV.

21.3.2.2 Financial Reporting Cannot be Accessed from Workspace: Check Application Status Using Log Files

Problem

Financial Reporting cannot be accessed from Workspace.

Solution

To resolve this issue, perform the following steps:

Check if the Financial Reporting web application is running. In Internet browser, enter the protocol //server:port/hr/status.jsp. If the web application is running, the following message is displayed:
```
The Oracle© Hyperion Financial Reporting, Fusion Edition Web application is available.
```
View the Financial Reporting log file fr.log from the following directories to see if there are any com.sun.xml.ws.wsdl.parser related errors:
```
(UNIX) DOMAIN_HOME/servers/server_name/logs/financialreporting
(Windows) DOMAIN_HOME\servers\server_name\logs\finsncialreporting
```
In this case, it is possible that analytics web application is not running. If fr.log contains multiple instances of Caused by: javax.jcr.LoginException: access denied for user to path /users/userid, then a Catalog GUID refresh is required.

For more information, see the Refreshing User GUIDs section in the Security Guide for Oracle Business Intelligence Enterprise Edition.

21.3.2.3 Unable to Connect to Financial Reporting Through a Firewall: Find and Modify MBean Properties

Problem

It is not possible to connect to Financial Reporting through a firewall from the Financial Reporting Studio.

Solution

By default, Financial Reporting components communicate with each other through Remote Method Invocation (RMI) on dynamically assigned Transmission Control Protocol (TCP) ports. To communicate through a firewall, you must specify the port for each Financial Reporting component separated by the firewall in the JConsole.exe file, and then open the necessary ports in your firewall. In addition, you may need to open ports for the Reports Server RDBMs, for data sources that you report against, and for LDAP/NTMLM for external authentication. Once connected, all RMI Services can use anonymous ports by default for communication. Alternatively a range of ports can be configured for communication by setting RMIPortRangeLower and RMIPortRangeUpper within the Financial Reporting configuration. You can change the port assignments to use in a firewall environment for servers in the Financial Reporting Mbeans called RMIPortRangeUpper and RMIPortRangeLower.

To locate the Financial Reporting MBeans properties to modify in Fusion Applications Control, perform the following steps:

From the navigation pane, expand the BIDomain farm, Application Deployments, and then Internal Applications.
Expand FinancialReporting(11.1.1)(bi_cluster_name).
Right-click FinancialReporting(11.1.1)(bi_server_name) and choose System MBean Browser.
In the System MBean Browser page, expand Application Defined MBeans.
Expand Application Defined MBeans, com.hyperion, Server:bi_server_name.

Figure 21-1 System MBean Browser

Description of "Figure 21-1 System MBean Browser"
Expand Financial Reporting.
Click Financial Reporting.
In the Application Defined MBeans: Financial Reporting page, scroll down to PrintServers, RMIPortRangeUpper, and RMIPortRangeLower.
Click on each attribute and add a value in the Value field and click Apply.

21.3.2.4 Error “Application HReports Is not Defined": Start the fressclient.ear Application

Problem

When the Batch Scheduler is opened, the following error displays:

application HReports is not defined

Solution

This may happen if the fressclient.ear application is not started in the General Ledger's Oracle Enterprise Scheduler domain.

To resolve this issue, from the Oracle WebLogic Server Console for the Oracle Enterprise Scheduler domain, perform the following to steps:

Navigate to Deployments and select fressclient.
Select Start, then Servicing all Requests.

21.3.2.5 Permission Errors in Batch Scheduler: Check User Permissions

Problem

When the Batch Scheduler is opened, or at the time of job submission, permission errors are shown.

Solution

This may happen if the user is not granted the oracle.as.scheduler.security.MetadataPermission permission. To resolve this issue, perform the following steps:

Check if the user has oracle.as.scheduler.security.MetadataPermission permission in the fscm stripe using Fusion Applications Control.

21.3.2.6 All the Jobs Fail: Restart Server if Needed

Problem

All the jobs fail. This may happen if the Essbase server is not running.

Solution

To check if the Essbase server is running and restart (if needed), perform the following steps:

Determine the current status as follows:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl status
(Windows) APPLICATIONS_CONFIG\BIInstance\bin/opmnctl status

OPMN generates a list of the running components and processes. The following message indicates that the Essbase Server (essbaseserver1) is already running:

Processes in Instance: BIInstance
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status 
---------------------------------+--------------------+---------+---------
essbaseserver1                   | Essbase            |   27879 | Alive  
coreapplication_obiccs1          | OracleBIClusterCo~ |   10828 | Alive  
coreapplication_obisch1          | OracleBIScheduler~ |   18308 | Alive  
coreapplication_obijh1           | OracleBIJavaHostC~ |   18337 | Alive  
coreapplication_obips1           | OracleBIPresentat~ |   26455 | Alive  
coreapplication_obis1            | OracleBIServerCom~ |   21716 | Alive

Restart the Essbase server as follows:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl restartproc ias-component=component_name  
(Windows) APPLICATIONS_CONFIG\BIInstance\bin\opmnctl restartproc ias-component=component_name

For example:

APPLICATIONS_CONFIG/BIInstance/bin/opmnctl restartproc ias-component=essbaseserver1

21.4 Problems and Solutions for Hyperion Provider Services

This section describes common problems and solutions for Oracle Hyperion Provider Services. The following topics are discussed:

21.4.1 Provider Services Server Not Functional: Check Status, Ports, Restart if Needed

Problem

Sometimes Provider Services appears not to be functional.

Solution

Determine whether Provider Services is running, and restart if necessary. In addition, check port connectivity.

To determine whether Provider Services is running, perform the following steps:

Launch a web browser.
Enter the connection URL for the corresponding products:
- Smart View Provider: https://Provider_Services_server:Provider_Services_port/aps/SmartView
- Java API: https://Provider_Services_server:Provider_Services_port/aps/JAPI
If an HTTP error is returned, the Provider Services server is not running.
Find out if you are connecting to the right port.
- If Provider Services is running with Oracle Access Manager, then the default Provider Services port number is 9704.
- If Provider Services is not running with Oracle Access Manager, then the default Provider Services port number is 4443.

To start the Provider Services server, use the following script from the fusionapps Middleware directory:

(UNIX) FA_MW_HOME/user_projects/domains/bi_foundation_domain_name/bin/startWebLogic.sh
(Windows) FA_MW_HOME\user_projects\domains\bi_foundation_dmain_name\bin\startWebLogic.cmd

21.4.2 Provider Services Version Information Not Showing in Interface: Where to Look

If Provider Services version information is not provided in a user interface, it is possible to find the information in the following locations:

Operating system console window that is displayed when the Provider Services server is running
Provider Services log files

21.4.3 Monitor Provider Sessions During Troubleshooting

It might be necessary to monitor the number of sessions on a Provider Services server.

To do so from Administration Services Console, perform the following steps:

From Enterprise View or a custom view, under the Provider Services node, select a provider.
Under the provider node, select Action, and then Sessions.

21.4.4 Monitor Active-Active Essbase Clusters

It might be necessary to monitor active-active Essbase clusters.

To do so from Administration Services Console, perform the following steps:

From Enterprise View or a custom view, under the Provider Services node, select a provider.
Under the provider node, expand Essbase Clusters and select the active-active Essbase cluster node you want to monitor.
Choose Action, and then edit the active-active Essbase cluster.

21.4.5 Enable and Disable Active-Active Essbase Cluster Components

It might be necessary to enable or disable individual active-active Essbase cluster components. To do so, follow the steps in the Oracle Hyperion Enterprise Performance Management System High Availability and Disaster Recovery Guide.

21.5 Problems and Solutions for Essbase

This section describes common problems and solutions. It contains the following topics:

21.5.1 Get Started with Logging Basics for Essbase

Log files are the best tools for analyzing what might be wrong with the system configuration. The log file, the defaults it ships with, and instructions for changing change those defaults will help with the analysis. The following list includes the log files for Essbase:

Essbase log files

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/Essbase.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\Essbase.log

Oracle Diagnostic Logging (ODL) logs

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/ESSBASE_ODL.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\ESSBASE_ODL.log

Lease Manager logs

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/leasemanager_essbase_machine_name.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\leasemanager_essbase_machine_name.log

Shared Services logs

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/SharedServices_Security_Client.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\SharedServices_Security_Client.log

OPMN logs

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/OPMN/opmn/opmn.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\OPMN\opmn\opmn.log

Essbase ping log

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/OPMN/opmn/EssbasePing.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\OPMN\opmn\EssbasePing.log

OPMN debug log

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/OPMN/opmn/debug.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\OPMN\opmn\debug.log

The system will generate the OPMN debug log only if DEBUG mode is turned on in opmn.xml.

OPMN console log

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/OPMN/opmn/console*.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\OPMN\opmn\console*.log

The system will generate the OPMN console log only if DEBUG mode is turned on in opmn.xml.

21.5.2 Essbase Agent Startup Fails with "Error: FUSIONAPPID not Specified in the cfg File"

Problem

The Essbase Agent startup fails with the following error:

Fatal Error: FUSIONAPPID not specified in the cfg file

Solution

To modify essbase.cfg and restart the Essbase server, perform the following steps:

Locate the essbase.cfg file in the following directories:

(UNIX) APPLICATIONS_CONFIG/BIInstance/Essbase/essbaseserver_name/bin
(Windows) APPLICATIONS_CONFIG\BIInstance\Essbase\essbaseserver_name\bin

Make sure that the following entry is present in essbase.cfg:
```
FUSIONAPPID appidname
```

Restart the Essbase server.

To restart the Essbase server using opmnctl, perform the following steps:

Determine the current status as follows:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl status
(Windows) APPLICATIONS_CONFIG\BIInstance\bin/opmnctl status

OPMN generates a list of the running components and processes. The following message indicates that the Essbase Server (essbaseserver1) is already running:

Processes in Instance: BIInstance
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status 
---------------------------------+--------------------+---------+---------
essbaseserver1                   | Essbase            |   27879 | Alive  
coreapplication_obiccs1          | OracleBIClusterCo~ |   10828 | Alive  
coreapplication_obisch1          | OracleBIScheduler~ |   18308 | Alive  
coreapplication_obijh1           | OracleBIJavaHostC~ |   18337 | Alive  
coreapplication_obips1           | OracleBIPresentat~ |   26455 | Alive  
coreapplication_obis1            | OracleBIServerCom~ |   21716 | Alive

Restart the Essbase server as follows:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl restartproc ias-component=component_name  
(Windows) APPLICATIONS_CONFIG\BIInstance\bin\opmnctl restartproc ias-component=component_name

For example:

APPLICATIONS_CONFIG/BIInstance/bin/opmnctl restartproc ias-component=essbaseserver1

To start Essbase server using Fusion Applications Control, perform the following steps:

From the navigation pane, expand the farm and then WebLogic Domain for the BIDomain domain.
Expand Essbase Servers, and then select the Essbase server.
From the Essbase Server menu, choose Administration, then Ports Configuration.
Select the Listen port, and then click Edit.
Change the port number, and then click OK.
From the Essbase Server menu, choose Control, then Restart.

21.5.3 Essbase Agent Startup Fails with "Error While Loading Shared Libraries"

Problem

The Essbase Agent startup fails with the following error:

Error while loading shared libraries: libARicu24.so: cannot open shared object file: No such file or directory

Solution

To modify opmn.xml file and restart the Essbase server, perform the following steps:

Locate the opmn.xml file in the following directories:

(UNIX) APPLICATIONS_CONFIG/config/OPMN/opmn
(Windows) APPLICATIONS_CONFIG\BIInstance\config\OPMN\opmn

Update the following ODBC-related entries in opmn.xml as follows:

<variable append="true" id="LD_LIBRARY_PATH" value="$ORACLE_HOME/common/ODBC/Merant/7.1.4/lib$:$ORACLE_HOME/jdk/jre/lib/i386/server$:$ESSBASEPATH/bin"/>

<variable id="ODBCINI" value="$ORACLE_HOME/common/ODBC/Merant/7.1.4/odbc.ini"/>

<variable id="ODBCINST" value="$ORACLE_HOME/common/ODBC/Merant/7.1.4/odbcinst.ini"/>

Restart the Essbase server by performing the following steps:

Determine the current status as follows:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl status
(Windows) APPLICATIONS_CONFIG\BIInstance\bin/opmnctl status

OPMN generates a list of the running components and processes. The following message indicates that the Essbase Server (essbaseserver1) is currently running:

Processes in Instance: BIInstance
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status 
---------------------------------+--------------------+---------+---------
essbaseserver1                   | Essbase            |   27879 | Alive  
coreapplication_obiccs1          | OracleBIClusterCo~ |   10828 | Alive  
coreapplication_obisch1          | OracleBIScheduler~ |   18308 | Alive  
coreapplication_obijh1           | OracleBIJavaHostC~ |   18337 | Alive  
coreapplication_obips1           | OracleBIPresentat~ |   26455 | Alive  
coreapplication_obis1            | OracleBIServerCom~ |   21716 | Alive

Restart the Essbase server as follows:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl restartproc ias-component=component_name  
(Windows) APPLICATIONS_CONFIG\BIInstance\bin\opmnctl restartproc ias-component=component_name

For example:

APPLICATIONS_CONFIG/BIInstance/bin/opmnctl restartproc ias-component=essbaseserver1

21.5.4 opmnctl Commands Fail to Execute

Problem

A communication error occurs when attempting to issue commands using the opmnctl command line for OPMN.

For example:

qtfhp3:/vol1/nnguyen/rc11/Oracle/Middleware/user_projects/epmsystem1/bin]$ opmnctl status RCV: No such file or directory Communication error with the OPMN server local port. Check the OPMN log files opmnctl status: opmn is not running.

Solution

To modify opmn.xml, perform the following steps:

Edit opmn.xml to assign a different local and a remote port to OPMN, or just a remote port to OPMN. The currently assigned ports may already be used by another process. See the following example:

<notification-server interface="any"> <ipaddr remote="<hostname>"/> <port local="6711" remote="6712"/>
Restart OPMN and try the opmnctl command again.

21.5.5 An Application Stops Responding: Search Logs for Error Messages

Problem

An Essbase application stops responding or shuts down abnormally.

Solution

To determine why the application is not responding, perform the following steps:

Check the Essbase.log file for the following error message:

1002089 RECEIVED ABNORMAL SHUTDOWN COMMAND - APPLICATION TERMINATING
Check the Essbase.log file for the following message:

Exception error log [log00001.xcp] is being created...

If either of the above messages are found, contact Technical Support.

If the log00001.xcp file is found, save it. Oracle Support will need this file to troubleshoot the issue. For example:

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/app/appname/log000001.xcp
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\app\appname\log000001.xcp

21.5.6 An Essbase Application Will Not Start or Network Error Displayed: Check for Port Conflict

Problem

An application does not respond after being started, or an error message like the following is displayed in the Essbase log file:

Network Error 10048 : Unable to Bind Host Server Socket On Port 4213

Solution

A port conflict may be occurring. Correct the port-related entries in opmn.xml.

Update the Essbase port range in opmn.xml to match the port specified in the configuration file as follows:

<port id="essbase-port-range" range="32768-33768"/>

21.5.7 ASO Database Corruption Error: Clear Database and Reload

Problem

During an operation on an Essbase aggregate storage database, the following error is displayed:

Persistent data does not match the outline. There is no member in dimension [Abc] for member number [12345]. Data is corrupted.

Solution

To clear the database and reload it from the original sources or from a saved exported backup, perform the following steps:

Save the list of current aggregate views.
Clear all data in the database.
Reload the data from original sources or from a backup.
Rebuild the aggregate views using the list specified.

21.5.8 Cannot Stop an Application Process: Check Logs and Forcefully Terminate if Needed

Problem

An Essbase application process cannot be shut down.

Solution

To resolve this issue, check the log messages to see if the application is still involved with cleanup or processing. If required, forcefully terminate the application process.

To check the application log for active processing, perform the following steps:

Access the application log. Note the following locations:

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/app/appname/appname.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\app\appname\appname.log

Check the application log and look for the following message:

RECEIVED SHUTDOWN COMMAND - SERVER TERMINATING.
If the proceeding message is seen, wait a few minutes to allow the application to self-terminate. The application may be doing cleanup tasks associated with the shutdown process.

If the following message is seen, the application is still processing user requests. Wait for user requests or any other operation that is still in progress to terminate, and then try shutting down the application.

Cannot unload database dbname while user username is performing database operation. Wait for the user to complete the operation, or ask the user to abort it. Log out all users and then unload the database. Cannot unload database dbname when it is still in use

If there is a need to terminate the application, perform the following steps:
1. If there is a need to terminate the application even when there are user requests in progress on the database, run the following sequence MaxL statements to forcefully terminate user requests:
```
#Display active sessions to see current requests display session on application appname
#Disallow new connections to the application alter application appname disable connects; 
#Force logout of all, and terminate requests alter system logout session on database appname.dbname force;
```
2. After forcefully logging out all users, wait for a few minutes, and then use the following MaxL statement to check if any user requests are still running:
```
#Display active sessions display session on application <appname>;
```
3. If there are no requests running, attempt to stop the application process.
4. After the application starts again, enable connects.

21.5.9 Change the Essbase Ports (High-Availability Mode)

Problem

The Essbase port numbers need to be changed when Essbase is installed in high-availability mode.

Solution

To run the Essbase failover automation script to change the Essbase port, perform the following steps:

Go to the following location:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/essbase_ha
(Windows) APPLICATIONS_CONFIG\BIInstance\bin\essbase_ha

Look for the %SHARED_FOLDER% location specified in the file essfoenv.properties.
Edit the %SHARED_FOLDER%\EssFoConfig.properties file or %SHARED_FOLDER%/EssFoConfig.properties file.
Look for SYSTEM_AGENT_PORTNUMBER(1|2), and change the port number. There are two Essbase instances, so make sure to change the one that you need.
Execute essfoconfig.sh (UNIX) or essfoconfig.bat (Windows) without any parameter. The Help prints so you can see a list of options for parameters.
Execute the following command:
```
essfoconfig.sh update prpName
```
Where prpName is the port number property in Step 4.

For example, the following command updates the agent port of the first Essbase instance:
```
essfoconfig.sh update SYSTEM_AGENT_PORTNUMBER1
```

21.5.10 Change the Essbase Ports (Non-High-Availability Mode)

Problem

The Essbase port numbers need to be changed when Essbase is installed in non-high-availability mode.

Solution

Go to Fusion Applications Control and change the Essbase port by performing the following steps:

From the navigation pane, expand the farm and then WebLogic Domain for the BIDomain domain.
Expand Essbase Servers, and then select the Essbase server.
From the Essbase Server menu, choose Administration, then Ports Configuration.
Select the Listen port, and then click Edit.
Change the port number, and then click OK.
Restart Essbase server. From the Essbase Server menu, choose Control, then Restart.

21.5.11 Data Load Fails with the "Load Buffer Does Not Exist" Error

Problem

Following a data load failure to a specified load buffer, the following error displays for subsequent loads to the same load buffer:

Data load buffer [123] does not exist.

This error may occur when:

The user uses the MaxL command that creates and commits buffer in one step, but then commit again using a separate step. The load buffer commit must be used with another MaxL command pair.
Data loads initialize a load buffer and load multiple data files to it before committing it to the cube. If an error occurs that causes one of the steps to fail, then the load buffer is automatically destroyed, causing all subsequent steps to fail with the previous error.

Solution

Use one of the following methods to correct this error:

Determine the cause of the original data load error and try to resolve it.
Set the Essbase data load options to indicate that the data load should not be aborted on error; instead, have data load errors written or appended to the log file

21.5.12 Data Load Fails with "Load Buffer Resource Usage" Error

Problem

Upon attempting to load data into an Essbase aggregate storage database, the following error displays:

Specified load buffer resource usage [100] is above currently available value [0].

Other ongoing data load operations have reserved part of the cache for their load buffers.

Solution

Use one of the following methods to correct this error:

Reduce the resource usage for this data load, and try again. If you are using MaxL, you must explicitly create the load buffer, using the optional resource_usage argument. For example:

alter database AsoSamp.Sample initialize load_buffer with buffer_id 1 resource_usage .5;
Wait for other operations to finish.
To see what reservations have been made to the cache resources, run the following MaxL statement:
```
query database "app"."db" list load_buffers;
```
For more information about the query database, alter database, and other MaxL statements, see the Oracle Essbase Technical Reference.

21.5.13 Essbase Fails to Start in Cluster Mode: Check for Hostname Qualification Mismatch

Problem

Sometimes Essbase does not start when it is in cluster mode and Oracle Business Intelligence domain is installed with a non-domain-qualified host name.

Solution

The error occurs when the Essbase host specified in the cluster configuration properties file, EssFOConfig.properties, is domain-qualified, but is not in the EPM Registry, leading to a mismatch. For more information about viewing the EPM Registry, see the Viewing the Components in the Shared Services Registry section in the Oracle Hyperion Enterprise Performance Management System Installation and Configuration Guide.

To check if there is an Essbase host name qualification mismatch, perform the following steps:

Use the following command to return all the Essbase clusters that were configured in the registry:

(UNIX) APPLICATIONS_CONFIG/BIInstance/config/foundation/11.1.2/epmsys_registry.sh view CLUSTER
(Windows) APPLICATIONS_CONFIG\BIInstance\config\foundation\11.1.2\epmsys_registry.bat view CLUSTER

Drill into the specific Essbase Server instance underneath this cluster, and view that node to see the host name as follows:
```
(UNIX) APPLICATIONS_CONFIG/BIInstance/config/foundation/11.1.2/epmsys_registry.sh view #Essbase_Server_GUID
(Windows) APPLICATIONS_CONFIG\BIInstance\config\foundation\11.1.2\epmsys_registry.bat view #Essbase_Server_GUID
```
Note that GUID is the global unique ID of an Essbase Server instance as returned from the full viewing of the EPM Registry.
Access the EPM registry and confirm that the host in the cluster configuration properties file displays exactly how Essbase is configured in the EPM Registry.
Restart Essbase.

21.5.14 Essbase Login Failed Due to Invalid Credentials

Problem

When the user attempts to log in to Essbase, the login attempt fails. The problem may be an authentication failure. When the Essbase login fails due to an authentication problem, it reports the following errors:

ERROR - 103 - Unexpected Essbase error 1051440.
ERROR - 1051440 - Essbase user [bi-001] Authentication Fails against the Shared Services Server with Error [EPMCSS-1009004: Failed to read data from the policy store.].

Solution

To resolve this issue, perform the following steps:

21.5.14.1 Gather Detailed Error Messages Using essbase.cfg File

To determine login failure problems by gathering detailed error messages, perform the following steps:

Locate the essbase.cfg file in the following directories:

(UNIX) APPLICATIONS_CONFIG/BIInstance/Essbase/essbaseserver_name/bin
(Windows) APPLICATIONS_CONFIG\BIInstance\Essbase\essbaseserver_name\bin

Using a text editor, add the following text on its own line:

LOGINFAILUREMESSAGEDETAILED

For more information about this and other essbase.cfg configuration settings, see the Oracle Essbase Technical Reference.
Stop and restart the Essbase server. For detailed procedures, see the Starting and Stopping Essbase Server section in the Oracle Essbase Database Administrator's Guide.

Use the following paths to access the Essbase.log file, which contains any error messages:

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/Essbase.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\Eessbase.log

In the same path where Essbase.log is found, see the SharedServices_Security_Client.log to locate security-related error messages.

21.5.14.2 Gather Debug-Level Messages Using essbase.cfg File

To gather debug-level error messages, perform the following steps:

Keep the debug statistics available for diagnostic purposes, in case there is a need to contact Technical Support.

Locate the essbase.cfg file in the following directories:

(UNIX) APPLICATIONS_CONFIG/BIInstance/Essbase/essbaseserver_name/bin
(Windows) APPLICATIONS_CONFIG\BIInstance\Essbase\essbaseserver_name\bin

Using a text editor, add the following text on its own line:

AGENTLOGMESSAGELEVEL DEBUG

For more information about this and other essbase.cfg configuration settings, see the Oracle Essbase Technical Reference.
Stop and restart the Essbase server. For detailed procedures, see the Starting and Stopping Essbase Server section in Oracle Essbase Database Administrator's Guide.

Use the following paths to access the Essbase.log file, which contains any error messages:

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/Essbase.log
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\Eessbase.log

21.5.14.3 Check for BI Domain Login Errors and Add a Valid Essbase User Name

To check for and correct BI domain login errors, perform the following steps:

The following table provides the names of each file along with the corresponding error message to identify if there is a debug login failure problem. Go to each file listed in the table and search for the corresponding error message. If one or both of the files contain errors, use the following procedure to correct them:

Table 21-1 File Names and Error Messages

File Name Error Message

File Name	Error Message
`AdminServer.out`	`<oracle.epm.oem.essbase.mbeans.EssbaseServerRegistrationMBeanImpl>` `<BEA-000000> <Child MBean registration failed due to some reason Not able to get Credential Store Factory from EM java.lang.Exception: Not able to get Credential Store Factory from EM`
`BIDomain.log`	`Exception while retrieving data from Essbase Cannot connect to olap service.` `Cannot connect to Essbase Server. Error:EssbaseError(1051293): Login fails due to invalid login credentials com.essbase.api.base.EssException: Cannot connect to olap service.` `Cannot connect to Essbase Server. Error:Essbase`

AdminServer.out

<oracle.epm.oem.essbase.mbeans.EssbaseServerRegistrationMBeanImpl>

<BEA-000000> <Child MBean registration failed due to some reason Not able to get Credential Store Factory from EM java.lang.Exception: Not able to get Credential Store Factory from EM

BIDomain.log

Exception while retrieving data from Essbase Cannot connect to olap service.

Cannot connect to Essbase Server. Error:EssbaseError(1051293): Login fails due to invalid login credentials com.essbase.api.base.EssException: Cannot connect to olap service.

Cannot connect to Essbase Server. Error:Essbase

To add a valid Essbase user name to the credential store from Fusion Applications Control, perform the following steps:

From the navigation pane, expand the farm and then WebLogic Domain for the BIDomain domain.
Right-click BIDomain and click Security.
Click Credentials. The Credentials page displays.
In the Credential table, expand essbaseserver.
Select the Essbase server and click the Edit the selected credential key button.
The Edit Key page displays.
Verify that the user name and password are correct.

21.5.15 System Error "Failed to Open File" on UNIX: Increase Maximum Number of Open File Descriptors

Problem

The following error occurs on a UNIX platform:

Failed to open file [filename]: a system file error occurred. Please see application log for details.

Solution

To open a file on UNIX, perform the following steps:

Confirm that the specified file exists.
If the file exists, increase the maximum number of open file descriptors. To do this, consult the UNIX operating system's documentation for the ulimit command.

21.5.16 GL Writeback Fails with "Accounting Date Conversion" Error: Check Date String Format

Problem

GL writeback fails with the following error:

The accounting date conversion failed for %s to %s in the ACCOUNTING_DATE column. Unable to proceed with GL export.

Solution

This error may occur when GL writeback cannot format the date string in the way that GL tables expect.

To fix this issue, make sure that the <DATE_FORMAT> tag in the cubeMap.xml file has date format represented in one of the following acceptable formats, and that it matches with the date value stored in the accounting date alias table of the outline. The following are the acceptable date formats:

mon dd yyyy
Month dd yyyy
mm/dd/yy
mm/dd/yyyy
yy.mm.dd
dd/mm/yy
dd.mm.yy
dd-mm-yy
dd Month yy
dd mon yy
Month dd, yy
mon dd, yy
mm-dd-yy
yy/mm/dd
yymmdd
dd Month yyyy
dd mon yyyy
dd/mon/yy
yyyy-mm-dd
yyyy/mm/dd
Day, Month dd, yyyy

21.5.17 GL Writeback Fails with "Group ID Node" Error: Check User Group IDs

Problem

GL writeback fails with the following error:

The group id node has active GL operations in state [<state_name>]. Use the appropriate API call in sequence. Unable to proceed with [api name] call.

Solution

This error may occur if more than one user tries to use the GL writeback-related API calls with the same group ID. To fix this issue, ensure that all users using GL writeback operations are using different group IDs.

21.5.18 GL Writeback Fails with "Not a Valid GL Application" Error: Check cubeMap.xml

Problem

GL writeback fails with the following error:

This application is not a valid GL application for Essbase.

Solution

The error may occur if the cubeMap.xml file is either not present in the app/ db directory, or is not parsed properly. For the GL writeback to work properly, the cubeMap.xml file must be present and contain the required information to be parsed successfully.

To confirm that cubeMap.xml was parsed correctly, perform the following steps:

Ensure that cubeMap.xml file exists in the app/ db directory and contains the correct information.
Ensure that cubeMap.xml was found and parsed successfully by checking the Essbase application log for the following entry:
```
Parsing of cubeMap.xml file succeeded
```
If the parsing of cubeMap.xml fails, verify that the file exists in the app/ db directory and that it contains the appropriate information.

21.5.19 GL Writeback Fails with "SQL Database Connection" Error: Check SQL Driver Setup

Problem

GL writeback fails with the following error:

Failed to Establish Connection With SQL Database Server. See log for more information.

Solution

The error may occur if the SQL drivers are not set up correctly. To confirm that the SQL drivers are set up correctly, perform the following steps:

Ensure that the ODBC Merant drivers path in odbcinst.ini is correct.
Ensure that the ODBCINST variable in opmn.xml is pointing to the correct odbcinst.ini file.
Ensure that an Oracle GL target database is running. For the GL writeback to work, a GL system is required, with cubeMap.xml correctly set up with the GL system information as follows:
```
<HOST>somemachine.company.com</HOST> <PORT>port</PORT> <SID>SID</SID>
```
Ensure that Essbase.cfg contains driver descriptors information in the following format, so that Essbase can connect to drivers in the odbcinst.ini as follows:

BPM_Oracle_DriverDescriptor "DataDirect 6.0 Oracle Wire Protocol"

21.5.20 Network Error "Cannot Send Data"/"Cannot Receive Data": Check xcp File and Timeout Parameters

Problem

When performing an operation against an Essbase cube, one of the following errors occurs:

Network error [12345]: Cannot Send Data
Network error [12345]: Cannot Receive Data

Solution

The error may indicate that Essbase has terminated abnormally. If an xcp file is found in the following location, save it and contact Oracle Support:

(UNIX) APPLICATIONS_CONFIG/BIInstance/diagnostics/logs/Essbase/essbaseserver_name/essbase/app/appname/log000001.xcp
(Windows) APPLICATIONS_CONFIG\BIInstance\diagnostics\logs\Essbase\essbaseserver_name\essbase\app\appname\log000001.xcp

In essbase.cfg, increase the network timeout parameters using the settings NETDELAY and NETRETRYCOUNT. For more information, see Essbase.cfg Configuration Settings section in Oracle Essbase Technical Reference.

21.5.21 OPMN Fails to Start Essbase in High-Availability Mode: Check ARBORPATH Variables

Problem

Sometimes OPMN fails to start Essbase in high-availability mode.

Solution

For Essbase clustering to work, all Essbase failover cluster data must be on shared storage with the ARBORPATH variables set correctly. To confirm that Essbase can start in high-availability mode, perform the following steps:

Ensure that OPMN is started by a network or domain user.
Ensure that ARBORPATHs are specified as mapped drives, and not as UNC paths.

21.5.22 Restructure Failure

The following are three primary restructure failure errors:

21.5.22.1 Memory Error 1130203: Increase Virtual Memory and Use 64-bit Installation

Problem

During a dimension build or outline restructure, error code 1130203 occurs.

Solution

The error indicates that the operation failed due to insufficient memory. Use one of the following methods to correct this error:

Increase the amount of virtual memory available to the operating system.
If Essbase is running on a 32-bit platform, keep in mind that the maximum memory available to Essbase is between 2 GB and 4 GB, regardless of how much RAM is installed on the machine. Because Essbase loads both the old and new outlines in memory during the restructure, there is a restriction on the largest outline that can be modified on such a machine. For example, if the platform only allows 2 GB of memory to be used by a process, and an Essbase application process is already using 200 MB memory, then the maximum-size outline that can be modified is 900 MB. Switching to a 64-bit installation of Essbase will lift this limitation.

21.5.22.2 "Errors Validating Outline"/"Outline Has Errors": Check Log for Hints

Problem

During dimension build or outline restructure, outline validation fails with one of the following errors:

There were errors validating the outline. Please check the error file.

Outline has errors.

Solution

Review the application log or the error file (if available) to see what specific errors are causing the validation error.

21.5.22.3 "Cannot Write New Outline"/"Error Writing outline Change Log File": Check for Space on File System

Problem

During dimension build or outline restructure, one of the following errors occurs:

Cannot write the new outline file during the restructuring of [%s]

Error writing outline change log file for database [%s]

Solution

Check the application log for other errors. If none are found, check to see if the file system is full. Remember that making a change to an Essbase outline requires at least as much free space as the existing outline.

21.5.23 "Invalid Item Index in Security File": Fix Corrupt Security File

Problem

The Console.log displays the following error:

Fatal Error: Invalid item index in security file

Solution

In this case, the Essbase security file, essbase.sec, is invalid or corrupt. Use one of the following methods to correct this error:

Restart Essbase using the latest backup security file by copying essbase_timestamp.bak to essbase.sec. Both files are located in the following directories:
```
(UNIX) APPLICATIONS_CONFIG/BIInstance/Essbase/essbaseserver_name/bin
(Windows) APPLICATIONS_CONFIG\BIInstance\Essbase\essbaseserver_name\bin
```
For more information, see the Managing the Essbase Security File section in the Oracle Essbase Database Administrator's Guide.
Open the essbase.cfg file and set the ENABLESWITCHTOBACKUPFILE setting to TRUE. This setting allows Essbase to automatically use a backup security file. The essbase.cfg file is located in the following directories:
```
(UNIX) APPLICATIONS_CONFIG/BIInstance/Essbase/essbaseserver_name/bin
(Windows) APPLICATIONS_CONFIG\BIInstance\Essbase\essbaseserver_name\bin
```
If Essbase.cfg is edited, restart Essbase to enable the change.

For more information, see the Essbase.cfg Configuration Settings section in the Oracle Essbase Technical Reference.

21.5.24 Find Out if Essbase is Running by Checking Essbase Agent Status

Problem

When the Administrator does not know if Essbase is running, the following OPMN command can be used to determine if the Essbase agent is running:

(UNIX) APPLICATIONS_CONFIG/BIInstance/bin/opmnctl status
(Windows) APPLICATIONS_CONFIG\BIInstance\bin/opmnctl status

Solution

To start Essbase server using Fusion Applications Control, perform the following steps

From the navigation pane, expand the farm and then WebLogic Domain for the BIDomain domain.
Expand Essbase Servers, and then select the Essbase server to see if it is running.
If it is not running, From the Essbase Server menu, choose Control, then Restart.

21.5.25 "Failed to Extend File" Error During Data Load or Building Aggregate Views: Resolve Insufficient File Space Problem

Problem

When attempting to load data into an Essbase aggregate storage database or while building aggregate views, the following error is displayed:

Failed to extend file [<path>/ess0001.dat]: a system file error occurred. Please see application log for details.

Solution

This error indicates that the file system is out of space. Essbase can require significant temporary disk storage while performing a data load or aggregate view build. The "default" tablespace is the location where cube data is stored. The "temp" tablespace is where Essbase writes temporary data while building the cube. By default, both tablespaces are on the disk drive where Essbase is installed.

During the operation, the space required by the temp tablespace is at least as big as the resulting change in the database size, so the total free space required is at least twice as big as the resulting change to the database size. For example, loading a database that is 1GB will require at least 2GB of free space. Building 10GB worth of aggregate views will require at least 20GB of free space. Note that after the operation is complete, the files created in the temp tablespace are deleted.

Use one of the following methods to correct this error:

To correct this error, use MaxL statements to view or change the tablespace settings.
If the tablespace locations do not have enough free space, consider moving one or both tablespaces to different disk drives, or limit the size of the existing file locations and add new locations on other disk drives. It is not possible to remove a tablespace location that already contains data.