This chapter explains how to set up and use Oracle Endeca with Oracle BI Applications.
This chapter contains the following sections:
Section 5.2, "Setting Up Endeca Integration with Oracle BI Applications"
Section 5.3, "Tasks for Setting Up Oracle Endeca with Oracle BI Applications"
Section 5.4, "ODI Package, Interface, and Procedure Design Overview"
Section 5.6, "Applying the Custom BI Applications Security Manager"
Oracle BI Applications can optionally be used with Oracle Endeca Information Discovery, which can use Oracle BI Applications data for its data visualizations. Oracle Endeca Information Discovery is an enterprise discovery platform for the analysis of information from both structured and unstructured sources, providing the ability to search for and visualize data in many different ways, including tag clouds.
This section provides instructions for setting up Oracle Endeca with Oracle BI Applications.
List of Steps for Setting Up Oracle Endeca Integration with Oracle BI Applications
To install and set up Endeca integration with Oracle BI Applications, you must complete the following tasks, in order. Each high-level task breaks down into a list of detailed steps provided in the next section.
Install Oracle Endeca Server 7.6.0. For information about requirements and installation of the Endeca Server, refer to the Oracle Endeca Server Installation Guide. Install all available patches.
Install Endeca 3.1 Studio. For information about requirements and installation of the Studio, refer to the Oracle Endeca Information Discovery Studio Installation Guide. Install available patches.
In Oracle Data Integrator (ODI) Studio, set up the ODI topology to include Oracle Business Intelligence Enterprise Edition as a source, the Endeca Server as a target, and access to required, as described in Section 5.3.3, "Setup Step: Set Up ODI Connectivity for Oracle BI and Endeca."
Load data by running ODI scenarios individually or by running the ODI load plan, as described in Section 5.3.4, "Setup Step: Load Data."
Create a new Endeca Studio Application and deploy the Sample Application, as described in Section 5.3.5, "Setup Step: Create a New Endeca Application."
This section provides detailed tasks for setting up Oracle Endeca with Oracle BI Applications, and contains the following sections:
Section 5.3.3, "Setup Step: Set Up ODI Connectivity for Oracle BI and Endeca"
Section 5.3.5, "Setup Step: Create a New Endeca Application"
Note: You must perform the tasks in this section in the sequence described in Section 5.2, "Setting Up Endeca Integration with Oracle BI Applications".
For information about requirements and installation of the Endeca Server, refer to the Oracle Endeca Server Installation Guide. Install all available patches. Optionally, you can take some steps to optimize the Endeca Server described below.
For Endeca Server hardware configured for six or less threads, it is recommended that you limit the data size as follows:
Limit data to 300 attributes and 1,000,000 rows.
The less attributes included, the more rows can be loaded in the same amount of time, up to 300 million attributes per hour.
The more processing available, the higher the number of rows that can be loaded in a given time.
Creating Endeca Data Domain Profile to Optimize CPU Core Count
Endeca Server's Data Domain Profile has its numComputeThreads property set to 2, representing two CPU Cores. Create a new Data Domain Profile to set the numComputeThreads value appropriately for the CPU cores available on the server when creating new Data Domains. For information about setting this property and creating domains using the endeca-cmd utility, refer to the chapter titled, "Endeca Server Command Reference" in the Oracle Endeca Server Administrator's Guide.
After the new Data Domain Profile is created, set the new profile name to the ODI parameter ENDECA_DATADOMAIN_PROFILE when running the Endeca Load Plan or scenarios. For more information about running the Endeca Load Plan and scenarion, refer to Section 5.3.4, "Setup Step: Load Data".
Disabling, Enabling, or Deleting Data Domains
During data ingest, data domains reach their highest memory usage and do not release memory. Domains tend to page memory as needed to make room for other memory requests made to the operating system. If the memory usage nears the maximum amount available, disabling and then enabling each data domain can reduce their claim on memory.
The below endeca-cmd commands can be run on the server to disable an existing domain and then enable it again. The location of endeca-cmd command line utility is $MWHOME\EndecaServer7.6.1\endeca-cmd:
To disable a data domain: endeca-cmd disable-dd <data-domain>
To enable a data domain: endeca-cmd enable-dd <data-domain>
A data domain can be left disabled to preserve the existing data and not required to be online for a period of time. If a data domain is no longer needed, it can also be deleted to free resources. No backup is performed, so when a data domain is deleted it cannot be restored. A shell or batch file script can be used to disable, enable, and delete multiple data domains.
Install Endeca 3.1 Studio. For information about requirements and installation of the Studio, refer to the Oracle Endeca Information Discovery Studio Installation Guide. Install available patches.
In this step, you create source and target server definitions in Oracle Data Integrator (ODI) to support Endeca ETL in the BI Applications data warehouse, as well as a definition for required Endeca view XML files.
Create Oracle BI Enterprise Edition Source
Endeca when integrated with Oracle BI Applications sources data from the Oracle Business Analytics Warehouse using an Endeca load plan, which denormalizes the OBAW data and loads it into the Endeca data domain, before using BI Server metadata such as data type, column name, attribute group name, and so on to load the Endeca schema. To support this integration, you create an ODI Data Server definition in the ODI repository metadata to support connections to the Oracle BI Server.
In ODI Designer's Topology tab, expand the Oracle BI Technology in the Physical Architecture.
In the Oracle BI Technology, create a new BI Applications data server.
In the Definition tab of the new server, enter a name and, in the Connection details, provide User and Password credentials for an Oracle BI EE Server administrative user.
In the JDBC tab, enter 'oracle.bi.jdbc.AnaJdbcDriver' as the JDBC Driver.
Enter 'jdbc:oraclebi://<IP address or the Oracle BI EE Server hostname>:9703/' as the JDBC URL.
Note:
Ensure that there are no leading or trailing spaces in the JDBC URL.In the BI Applications data server, create a new Physical Schema. In its Definition tab, specify Core for the Catalog (Catalog) and for Catalog (Work Catalog).
In ODI Designer's Topology tab, expand the Logical Architecture and assign the OBI_BIAPPS11G logical schema to the physical schema you created.
To support Endeca Server integration, you create a target ODI Data Server definition in the ODI Physical Architecture for the Endeca Server.
In ODI Designer's Topology tab, expand the Endeca Server Technology in the Physical Architecture.
In the Endeca Server Technology, create a new BI Applications data server.
In the Definition tab of the new server, enter a name. In the Connection details, leave the User and Password fields empty.
In the JDBC tab, leave the JDBC Driver empty.
Enter 'http://<IP address or the Endeca Server hostname>:7001/endeca-server' as the JDBC URL.
Note:
Ensure that there are no leading or trailing spaces in the JDBC URL.In the BI Applications data server, create a new Physical Schema. In its Definition tab, you can leave Catalog (Catalog) and Catalog (Work Catalog) empty.
In ODI Designer's Topology tab, expand the Logical Architecture and assign the OEID_BIAPPS11G logical schema to the physical schema you created.
Create Data Server for Endeca Files
To support Endeca Server integration, you create a target ODI Data Server definition in the ODI Physical Architecture for Endeca files used by the Endeca web service to load view XML, including Sample Applications.
In ODI Designer's Topology tab, expand the File Technology in the Physical Architecture.
In the File Technology, create a new BI Applications data server.
In the Definition tab of the new server, enter a name. In the Connection details, leave the User and Password fields empty.
In the JDBC tab, enter 'com.sunopsis.jdbc.driver.file.FileDriver' as the JDBC Driver.
Enter 'jdbc:snps:dbfile?OPT=TRUE' as the JDBC URL.
Note:
Ensure that there are no leading or trailing spaces in the JDBC URL.In the BI Applications data server, create a new Physical Schema. Use the directory containing the view XML files (for example, C:\Temp) for Directory (Schema) and Directory (Work Schema). This directory should be relative to the server where the ODI Agent is installed and can access the view XML files.
Note:
The Sample Apps View XMLs can be found in the BIAPPS installation under <Middleware Home>\Oracle_BI1\biapps\admin\provisioning\endeca\OracleBIApps_Endeca.zip.In ODI Designer's Topology tab, expand the Logical Architecture and assign the DW_OEID_SRCFILES logical schema to the physical schema you created.
Exporting and Importing Views Using Endeca Web Service Procedure
The Endeca Web Service Procedure supports exporting views. Beyond sample application view XML, you can export and import views for other subject areas.Endeca Views facilitate additional data manipulation in Endeca Server using Endeca Query Language (EQL), SQL-like views which are typically created in Studio but are stored in the server and need to be exported and then imported. Eight Subject Areas, or sample applications, have the Endeca Web Service Procedure included in the ODI package to load Endeca views.
To export views:
Create a new ODI package and add the Endeca Web Service Procedure. Give the package a name, for example Export Views.
Set the following Procedure options:
ENDECA_WS_RELATIVE_PATH: /ws/sconfig/<Data Domain Name>
REQUEST_FILE_NAME: Export_Views.xml
RESPONSE_FILE_NAME: Name of the response message file views export. The naming convention is <Data Domain Name>_Views.xml.
Place or save the Export_Views.xml file into the directory entered in the BIAPPS_OEID_FILE physical schema. This is also the directory where the procedure outputs the response file (RESPONSE_FILE_NAME).
Run the ODI Package. If successful, the views are exported into the response file.
To import views:
The exported views xml requires formatting before it can be imported or re-imported. Open the xml in a text editor and search for validatedSemanticEntity and replace it with semanticEntity.
Search for listEntitiesResponse and replace it with putEntities.
Use a new or existing ODI package to load the Endeca Data Domain. In the package, add the Endeca Web Service Procedure as the final step.
Set the following Procedure options:
ENDECA_WS_RELATIVE_PATH: /ws/sconfig/<Data Domain Name>
REQUEST_FILE_NAME: Name of the response message file from the views export. The naming convention is <Data Domain Name>_Views.xml.
Place or save the Export_Views.xml file into the directory entered in the BIAPPS_OEID_FILE physical schema. This is also the directory where the procedure outputs the response file (RESPONSE_FILE_NAME).
Run the ODI Package. If successful, the views are imported into the target data domain.
Verify Connection to the Data Warehouse
Verify that the Oracle Business Analytics Warehouse data server has correct connection information.
In ODI Designer's Topology tab, expand the Technology in the Physical Architecture corresponding to the data warehouse.
Select the BIAPPS_DW data server.
In the Definition tab, enter the connection user name and password for the OBAW database.
In the JDBC tab, enter the correct JDBC Driver information for the database.
ODI Agents exist as either a standalone agent or a Java EE agent. The Endeca IKM jar files, which must be included with the ODI Agent, are available in the BI Applications installation in <Middleware Home>\Oracle_BI1\biapps\admin\provisioning\endeca\OracleBIApps_Endeca.zip (Import/Lib directory).
If your ODI agent is a standalone agent, copy the JAR files into the ODI drivers directory located in <install path>\Oracle\Middleware\Oracle_ODI1\oracledi\agent\drivers (or the equivalent path on UNIX). Restart the ODI Agent afterwards.
If your ODI agent is a Java application running on the WebLogic Server, do the following:
Create a jlib directory under <ORACLE_HOME>biapps/odi/bia_odiagent/.
Copy all the required Endeca IKM jars in the odi agent classpath, in <ORACLE_HOME>biapps/odi/bia_odiagent/jlib.
Restart the ODI Server.
For more information about the IKM and jar files, refer to the Endeca SQL to Endeca Server Installation and Usage Guide.
Disable Applying Schemas in the Load Target Schema ODI Interface
Endeca Server schema metadata can be modified using Endeca Studio rather than editing ODI flexfields after the schema is created during the initial Endeca ETL. To prevent overwriting a customized schema, you set the APPLY_SCHEMA option in the Load Target Schema Interface to FALSE.
In ODI Designer, under Projects, open the Package of the subject area and select the Diagram tab.
Right-click the TLP_OEID_<Subject Area Name>Load_Tgt_Schema step and select Edit Linked Object. The Load Target Schema Interface opens in a new tab.
In the Flow tab, click the Target (BIAPPS_OEID). Change the APPLY_SCHEMA option in the Properties Inspector options area to 'false'.
Save and close the interface, then the package.
Generate the scenario. In Projects, right-click the Package and select Generate Scenario, and click OK to accept the defaults.
Note the version of the new scenario.
You can now run the scenario directly using the noted version number, selecting the Context, Logical Agent, and Log Level. Alternately, you can run the scenario from the Endeca load plan. For each of the scenario steps, the load plan sets the version -1, which uses the latest generated scenario. Scenarios are run directly in ODI Designer.
You can load data into the Endeca domain either by running ODI scenarios directly or by running the Endeca load plan. Source and target data stores delivered with the BI Applications installation include all Oracle BI Applications repository (RPD) logical columns. These may be customized to add or remove mappings to meet business requirements.
Setting and Declaring Variables
When running scenarios directly, you can set session variables at runtime. The following ODI variables can be overridden at runtime.
ENDECA_DATADOMAIN_PROFILE: This variable is the name of the Data Domain Profile if one was created. For information about creating a Data Domain Profile to maximize performance, refer to "Creating Endeca Data Domain Profile to Optimize CPU Core Count".
ENDECA_OBIEE_SQL_PREFIX: This variable is used to override the default preferred currency, which is GLOBAL1.
ETL_PREDICATE_EXTRACT: This variable is used to enter filters.
The following ODI variables require modifications to the ODI package.
ENDECA_DATADOMAIN
ENDECA_DATALOAD_LOG_FILE_PATH
To change the ENDECA_DATADOMAIN and/or ENDECA_DATALOAD_LOG_FILE_PATH values:
In the Designer under Projects, open the package of the subject area and select the Diagram tab.
Select the ENDECA_DATADOMAIN or ENDECA_DATALOAD_LOG_FILE_PATH variable. In the Properties pane, in the General tab, enter the desired value.
Save and close the Package.
Save and close the interface, then the package.
Generate the scenario. In Projects, right-click the Package and select Generate Scenario, and click OK to accept the defaults. Select Startup Parameters "Use All" when prompted, then click OK.
To run ODI scenarios directly:
In ODI Designer, open the scenario to run.
Click Run.
Select the Context, Logical Agent, and Log Level.
Use the default or modify the session variables as needed. For example, BIAPPS.ETL_PREDICATE_EXTRACT is a customizable filter variable.
Click OK.
A predefined load plan is provided to run the Endeca packages. The load plan contains steps to evaluate whether the IS_ENDECA_DEPLOYED variable is set to 'Y' or 'N' for each of the OEID_ scenarios, which correlate with Endeca Fact Groups. The frequency with which you run the load plan for data refresh is dependent on your use. Depending on the cost of a typical load, nightly execution may be appropriate, but if you do not require real-time data, you may run it less frequently. In the case that you are performing only historical analysis, the first load may be the only required.
Note:
Configuration Manager does not include an offering for setting up the ODI repository to load Endeca. The load plan is executed directly using the ODI Web Console or Client.Set IS_ENDECA_DEPLOYED Variable and Run ODI Load Plan
To select which scenarios are run, set the IS_ENDECA_DEPLOYED variable, then run the load plan.
In ODI Client, open the BI Apps DW to Endeca Load Plan under Load Plan and Scenarios > Predefined Load Plans/Endeca.
Select the Steps tab and expand the Parallel step.
Select the substep with an "OEID_" prefix containing the scenario to be executed, for example the OEID_COSTING_INVENTORY_VALUATION_FG, which represents the Costing Inventory Valuation fact group.
In the Properties Inspector, under the Variables section, set the value for BIAPPS.IS_ENDECA_DEPLOYED to Y. Ensure that the Y is capitalized and that the Refresh checkbox is not selected after entering the value.
Repeat the above steps for each OEID_ substep which contains a scenario to be executed.
Save the load plan.
Run and monitor the load to execute the substeps which were set to Y.
Set Filters and Currency Setting
Within the predefined Endeca load plan, variables such as ENDECA_OBIEE_SQL_PREFIX and ETL_PREDICATE_EXTRACT may be overridden and saved in the load plan. Filter and currency modifications can be done directly in the load plan under Scenario Variables by selecting the scenario step, and checking and providing a new value.
To create a new Endeca Application:
Log in to Endeca Studio as an Administrator.
Click the gear icon and, in the Control Panel, select Endeca Servers.
Click New Connection and enter the required parameters.
Validate the connection and, optionally, test it.
Click Back to Home to return to the home page.
Click New Application.
Enter an Application name and Application description.
Select a pre-built Endeca Server under Select a Data Source.
Select a managed data connection.
Click Done. A default template is created.
This section describes unique design details about the Endeca ODI package, interface, and procedures. Source and target data stores delivered with the BI Applications installation include all Oracle BI Applications repository (RPD) logical columns. These may be customized to add or remove mappings to meet business requirements.
The flexfield properties in the ODI Data Store columns configure the Endeca Server column settings. For example, the Endeca properties define whether the value of a column is text searchable, value searchable, and so on. These properties all begin with the "Endeca Property" as a prefix.
Each Subject Area (SA) has the following package design pattern:
ENDECA_DATADOMAIN variable: Name of SA Data Domain, for example, OEID_Project_Cost.
ENDECA_DATADOMAIN_PROFILE variable: Data Domain profile. If blank, the default Data Domain profile is used.
ENDECA_DATALOAD_LOG_FILE_PATH variable: Path/name for Log File, for example OEID_Project_Cost_ENDECA_DATALOAD.LOG.
ENDECA_OBIEE_SQL_PREFIX variable: Optional Oracle BI EE prefix, usually set for currency, for example, set variable PREFERRED_CURRENCY='Global Currency 4'.
ETL_PREDICATE_EXTRACT variable: Optional Oracle BI EE where clause (filter) condition. The default is 1=1.
TLP_OEID_<SA>_Load_Tgt_Schema interface, for example, TLP_OEID_Project_Cost_Load_Tgt_Schema
Interface does not load data (has filter set to 1=2).
Interface creates the Data Domain if it does not already exist.
Mapping contains all of the columns for the Subject Area and loads only the schema.
Caution:
If any columns are modified in the data loading interfaces, this load target schema interface's columns needs to be updated and maintained.There is an IKM option to disable the "APPLY SCHEMA" option to preserve attribute configurations made in Studio.
TLP_OEID_<SA>_<Fact Table Name> interfaces, for example, TLP_OEID_Project_Cost_Fact_Project_Cost.
Interface or interfaces (some Subject Areas may have more than one) extracts data from Oracle BI EE and loads Endeca.
One interface exists per dataset collection name, for example, "Fact_Project_Cost"
Each interface loads a single denormalized fact star schema.
Each interface truncates the collection data before each load, so each load is a destructive load and not incremental.
Each interface uses the following:
LKM: LKM BIAPPS OBIEE to SQL BMMFETCH
IKM: IKM SQL to Endeca Server
Source: Oracle BI Applications\Oracle BI Applications Business Model\Core
Target: Oracle BI Applications\Oracle BI Applications Endeca\Core
The procedure requires the same jars used by the SQL to Endeca Server IKM.
Only Subject Areas that are sample apps have the Endeca Web Service Procedure added to load views as the final step in the Package.
The Endeca Web Service Procedure can optionally be included in any Subject Area, or even be run standalone in its own ODI Package.
This section describes how to deploy Endeca sample applications, which can be downloaded from the BI Applications 11.1.1.8.1 Media Pack from Oracle Software Delivery Cloud, and are used as a basis to create your own applications. There are eight sample applications, with Data Domains and Studio Applications (.lar files) delivered in the Media Pack under Supporting Files for Endeca Sample Applications for BI Applications 11.1.1.8.1, and View XML files deployed by the BI Applications installer. Notice that some sample applications contain tag clouds based on enriched text, so if the Data Domain is reloaded, rules need to be re-run. If the Data Domain is reset, rule need to be recreated.
The data domain files included with the installation should be placed in: $DOMAIN_HOME\EndecaServer\offline.
The endeca-cmd utility can import the data domains and enable them using the following commands:
CALL endeca-cmd import-dd my_dd1 --offline-name my_dd_offline1 CALL endeca-cmd enable-dd my_dd1
You can use a script to import and enable multiple data domains. The below is an example of a DOS .bat script:
CALL endeca-cmd import-dd OEID_Sales_Order_Lines \--offline-name OEID_Sales_Order_Lines CALL endeca-cmd enable-dd OEID_Sales_Order_Lines CALL endeca-cmd import-dd OEID_Project_Performance \--offline-name OEID_Project_Performance CALL endeca-cmd enable-dd OEID_Project_Performance CALL endeca-cmd import-dd OEID_SIA_Admissions_and_Recruiting_Student_Response \--offline-name OEID_SIA_Admissions_and_Recruiting_Student_Response CALL endeca-cmd enable-dd OEID_SIA_Admissions_and_Recruiting_Student_Response CALL endeca-cmd import-dd OEID_SIA_Admissions_and_Recruiting_Application_Evaluation \--offline-name OEID_SIA_Admissions_and_Recruiting_Application_Evaluation CALL endeca-cmd enable-dd OEID_SIA_Admissions_and_Recruiting_Application_Evaluation CALL endeca-cmd import-dd OEID_Employee_Expenses_Overview \--offline-name OEID_Employee_Expenses_Overview CALL endeca-cmd enable-dd OEID_Employee_Expenses_Overview CALL endeca-cmd import-dd OEID_Manufacturing_Work_Order_Performance \--offline-name OEID_Manufacturing_Work_Order_Performance CALL endeca-cmd enable-dd OEID_Manufacturing_Work_Order_Performance CALL endeca-cmd import-dd OEID_Human_Resources_Recruitment \--offline-name OEID_Human_Resources_Recruitment CALL endeca-cmd enable-dd OEID_Human_Resources_Recruitment CALL endeca-cmd import-dd OEID_Manufacturing_Actual_Production \--offline-name OEID_Manufacturing_Actual_Production CALL endeca-cmd enable-dd OEID_Manufacturing_Actual_Production
For information about using the endeca-cmd utility, refer to the chapter titled, "Endeca Server Command Reference" in the Oracle Endeca Server Administrator's Guide. For information about importing, exporting, enabling, or disabling data domains, refer to the chapter titled, "Managing Data Domains" in the Oracle Endeca Server Administrator's Guide.
Importing Endeca Studio Application Sample Applications
To import Endeca Studio sample applications, you are required to have created a new application prior to import. To create a new application using the default template:
Log in to Endeca Studio as an Administrator.
Click the gear icon on the top right to navigate to the Control Panel, then Endeca Servers.
Click the New Connection button and input the necessary parameters. Below is an example of typical parameters:
Connection ID: OEID_Employee_Expenses_Overview
{
"dataDomainName": "OEID_Employee_Expenses_Overview",
"name": "OEID_Employee_Expenses_Overview",
"port": "7001",
"server": "hostname"
}
Note:
If applying the custom OBIA Studio security manager for the sample apps, the Connection ID needs to match the Data Domain name to be in sync with the Connection ID specified in the delivered security columns configuration file. For more information about the security columns configuration file, refer to Section 5.6, "Applying the Custom BI Applications Security Manager".Validate the connection, then click Save. Optionally, confirm connectivity using the Test Connection button.
Click the Back to Home button to return to the home page.
Click the New Application button.
Provide an Application name, for example, Employee Expenses, and an Application description.
Under Select a Data Source, choose Use a Pre-built Endeca Server.
Select a managed data connection. For example, OEID_Employee_Expenses_Overview.
Click Done to create a default template.
Import Sample Application (.lar file)
To import a sample application provided in the Media Pack:
Click the gear icon on the top right to navigate to the Control Panel, then select Applications.
On the line of the application, to import the .lar file, select Actions > Manage Pages.
Under All Pages, in the Export/Import tab, select the Import tab.
Click Browse or Choose File, and select the appropriate .lar file.
Under the section titled What would you like to import, leave the defaults. To delete any pages in the destination environment that do not exist in the ,lar file, check the Delete Missing Pages checkbox. To ensure that the import includes all of the component configuration, check the User Preferences checkbox.
Click Import.
Text Enrichment (Term Extraction) / Enrichment Rules
Some of the sample applications contain tag clouds based on enriched text. The tag clouds should work after importing sample data domains, because the text enrichment pipelines are stored in the data domains and not in Studio. If the sample application data domains are loaded without being imported, the enriched tag clouds display an error message due to missing enriched attributes. The text enrichment pipelines need to be recreated. To do this:
In Application Settings > Data Sets > Enrichments, click Add Enrichment, extract terms, and select the attribute or attributes containing free form text (unstructured content) to be enriched
Provide an output name, for example Attribute Name (terms). The rest of the settings can remain defaults.
Click the Run button to process the enrichment.
Edit the tag clouds settings to replace the invalid output attributes with the new ones.
Whenever data is replaced or reloaded into the Endeca Data Domain, the tag clouds and enrichment pipelines should still work and do not need to be recreated, but the enrichment pipeline needs to be run again by clicking on the run button as described above. This recreates the enriched data. Otherwise, the enriched tag clouds do not show any data. To rerun enrichment:
Navigate to Application Settings > Data Sets and select the datasets which have enrichment pipelines.
Select the Enrichment tab and run the Enrichment or Enrichments by clicking the Run button.
Click the Run button to process the enrichment.
For applications that contain multiple data sets, refinement rules allow you to connect attributes from the different data sets. For example, a sales data set for automotive data includes the make and vehicle ID number (VIN) of cars that were sold. Another data set containing warranty claims also includes the make and VIN of cars for which warranty claims were filed. If you then create refinement rules for the make and VIN attributes, then when users refine one data set by a make or VIN, the other data set also is refined by that make or VIN.
For more information about using refinement rules, refer to the chapter titled, "Using Refinement Rules to Link Attributes from Different Data Sets," in the Oracle Endeca Information Discovery Studio User's Guide.
This section describes how to set up Endeca security for interoperability with Oracle BI Applications and Oracle WebLogic. It includes information about assigning required BI roles, creating the Endeca credential store and including the files required by the Oracle BI Applications Security Manager, defining new users in Endeca Studio, and other optional security configurations you can make to enhance Endeca Server performance or user experience.
Assigning BIImpersonator Role to an OBIEE user
The Oracle BI Applications Security Manager requires an Oracle BI Enterprise Edition user with administrator and impersonator roles, which is used to obtain information necessary to apply security filters in an Endeca Application. To add the impersonator role to an existing Oracle BI EE administrator account:
Log in to Oracle Business Intelligence's Enterprise Manager with Administrator privileges.
Expand Business Intelligence in the left-hand pane.
Right-click coreapplication and select Security > Application Roles to navigate to the Application Roles page.
By default, the obi application stripe is selected and the default application roles are displayed. Search for the Role Names with a prefix of "BI".
In the Members list, click the BIImpersonator role, then click Edit.
Click Add and, in the Add Principal dialog box, search for Type of User and locate an administrative user.
Creating and Setting Up the Credential Store in Endeca Studio
The Oracle BI Enterprise Edition account credentials with administrator and impersonator roles is saved locally in the Endeca Studio Domain's Credential Store. The BI Applications Security Manager obtains the password information from the Credential Store to connect to Oracle BI Enterprise Edition using JDBC. To set up the credential store, follow the steps below, which assume WebLogic has already been installed and occur before creating an Endeca Studio domain. It is possible to extend an existing domain to include Enterprise Manager.
Verify that Oracle Application Development Framework was installed as part of the Endeca Server installation. ADF may already be available if Endeca Studio is installed on the same WebLogic Server.
When creating the Endeca Studio Domain for the first time, check "Oracle Enterprise Manager". JRF is automatically included. If a Studio Domain has already been created, the existing domain can be extended to include Enterprise Manager.
Start WebLogic and create a Credential Store to save the OBIEE credential information using Enterprise Manager.
Log in to Enterprise Manager and, in the WebLogic Domain, right-click endeca_studio_domain > Security > Credentials. Create a new map (for example, oracle.bi.enterprise) and key (for example, repository.OBIA), then save the OBIEE username and password information.
Note:
The password assigned to the map and key must match the OBIEE account with administrator and impersonator roles assigned to it.Including the Oracle BI Applications Security Manager Related Files
The Oracle BI Enterprise Edition account credentials with administrator and impersonator roles are saved locally in the Endeca Studio Domain's Credential Store. The BI Applications Security Manager obtains the password information from the Credential Store to connect to Oracle BI Enterprise Edition using JDBC. To set up the credential store, follow the steps below, which assume WebLogic has already been installed and occur before creating an Endeca Studio domain. It is possible to extend an existing domain to include Enterprise Manager.
For the WebLogic version of Endeca Studio 3.1, the custom BI Applications Security Manager .jar files need to be added to the .ear installation file. Decompress the .ear file using a utility, then copy the .jar files into the \APP-INF\lib\ directory. To copy the files:
Save a copy of the Endeca Studio installation .ear file with a different file name, for example, OBIA-endeca-portal-weblogic-3.1.13849.ear.
Add OBIAMDEXSecurityManager.jar and bijdbc.jar to Endeca Studio's .ear file under \APP-INF\lib\ using a compression utility.
Deploy the .ear into WebLogic following the installation instructions. For information about deploying the .ear, refer to the setion titled, "Deploying Studio to the WebLogic domain" in the Oracle Endeca Information Discovery Studio Installation Guide. If there is an existing Endeca Studio deployment, be sure to undeploy it first. Note the name used for the deployment, as this value will be used when modifying the system-jazn-data.xml. A typical name for the deployment is OBIA-endeca-portal-weblogic-3.
Add directories on the WebLogic server named XML and User_Input under $MW_HOME\user_projects\domains\endeca_studio_domain.
Add or create the config.properties file in the User_Input directory. Set the parameters as follows. A sample file is available in <Middleware Home>\Oracle_BI1\biapps\admin\provisioning\endeca\OracleBIApps_Endeca.zip (OBIAMDEXSecurityManager/User_Input/.
OBIEE_HOST=<OBIEE hostname> OBIEE_USERID=<OBIEE username with Admin and Impersonator Roles assigned> OBIEE_JDBC_PORT=<port number, usually 9703> OBIEE_USERID_MAP=<Credential Store Map Name> OBIEE_USERID_KEY=<Credential Store Key Name>
Add or create the securitycolumns.csv file in the User_Input directory. A sample file is available in <Middleware Home>\Oracle_BI1\biapps\admin\provisioning\endeca\OracleBIApps_Endeca.zip (OBIAMDEXSecurityManager/User_Input/. This .csv file has three columns, Endeca Server Connection ID, Collection Name, and Security Columns, used to relate the security columns.
Start and log in to Endeca Studio. From the Control Panel, select Framework Settings and set df.mdexSecurityManager from "com.endeca.portal.data.security.DefaultMDEXSecurityManager" to "com.endeca.portal.extensions.OBIAMDEXSecurityManager".
Click Update Settings.
Shut down the WebLogic server.
Add the following entry to the system-jazn-data.xml file found in $DOMAIN_HOME\config\fmwconfig, under the <system-policy> and <jazn-policy> tags. This includes permissions for Studio to access the Credential Store. Examples of
<grant>
<grantee>
<codesource>
<url>file:${oracle.deployed.app.dir}/<appName>${oracle.deployed.app.ext}</url>
</codesource>
</grantee>
<permissions>
<permission>
<class>oracle.security.jps.service.credstore.CredentialAccessPermission</class>
<name>context=SYSTEM,mapName=<mapName>,keyName=<keyName></name>
<actions>*</actions>
</permission>
</permissions>
</grant>
Examples of values include:
appName=OBIA-endeca-portal-weblogic-3
mapName=oracle.bi.enterprise
keyName=repository.OBIA
Restart the WebLogic Endeca Studio Server with the new custom Security Manager applied and in use by Endeca Studio.
Optional: Increasing WebLogic Server Heap Space to Improve Endeca Studio Performance
Update the setDomainEnv script file, which is named setDomainEnv.cmd in Windows environments and setDomainEnv.sh in Linux. The file is located in the bin subdirectory of the domain directory, <MiddlewareHomeDirectory>/user_projects/domains/endeca_studio_domain/bin/.
Search for the following in the setDomainEnv.cmd file:
if NOT "%USER_MEM_ARGS%"=="" (
s et MEM_ARGS=%USER_MEM_ARGS%
)
Before the above "if" statement, add the following, which sets a higher -Xmx or maximum memory heap size):
set MEM_ARGS=-Xms128m -Xmx1280m %MEM_DEV_ARGS% %MEM_MAX_PERM_SIZE%
Optional: Enabling Verbose Debugger Logging
To enable logging of debugging messages in the log file located in $MW_HOME\user_projects\domains\endeca_studio_domain\eid-studio.log, select Server Administration in the Control Panel, then select the Log Levels tab. Select the Add Category tab and enter the following:
com.endeca.portal.extensions.OBIAMDEXSecurityManager (DEBUG)
com.endeca.portal.extensions.OBIAMDEXSecurityManager.BIHandlers (DEBUG)
Log out, then log back in to enable the changes. This step has to be repeated if the Endeca Studio server is restarted.
Overriding Screen Name Validator in Endeca Studio
By default, Endeca Studio does not allow screen names to contain underscores. The screen name validator must be changed from the DefaultScreenNameValidator to the LiberalScreenNameValidator.
Shut down Endeca Studio.
Open the portal-ext.properties file under <%WLS_HOME>\user_projects\domains\endeca_studio_domain\eid\studio. Back up the existing portal-ext.properties file.
Add the following at the bottom of the file and save it: users.screen.name.validator=com.liferay.portal.security.auth.LiberalScreenNameValidator.
Start Endeca Studio.
Defining New Users in Endeca Studio (Must Match OBIEE Userid to Obtain Role Security Info) and Adding Users to Studio Applications
The Custom Oracle BI Applications Security Manager applies filters in Endeca based on the user's application role information set in Oracle Business Intelligence Enterprise Edition and the security columns defined in securitycolumns.csv.
For an Oracle BI EE user, a new user account must be created in Endeca Studio where the screenname matches their BI EE user ID. For information about how to create a new user in Endeca Studio, refer to the chapter titled, "Creating and Editing Users in Studio" in the Oracle Endeca Information Discovery Studio Studio Administration and Customization Guide.
Optionally, you can also change the default behavior, which is to have Endeca users log in using their email addresses, so that logins are consistent with Oracle BI Enterprise Edition. For information about changing this default behavior, refer to the section titled, "Configuring the type of user name for Studio" in the Oracle Endeca Information Discovery Studio Administration and Customization Guide. For more information about users, roles, and application permissions in Endeca Studio, refer to the "Configuring and Removing Applications" chapter in the Oracle Endeca Information Discovery Studio Studio Administration and Customization Guide.
Adding Users and Managing Studio Applications Permissions in Endeca Studio
Users must be added to a Studio Application to enable them to view it. For information about how add users to Studio Applications in Endeca Studio, refer to the section titled, "Adding and removing application members" in the Oracle Endeca Information Discovery Studio Studio Administration and Customization Guide.
By default, users are able to create new applications. To prevent this ability for a user, remove the Power User role. For information about removing this role, refer to the section titled, "Preventing a user from creating applications" in the Oracle Endeca Information Discovery Studio Studio Administration and Customization Guide.
The application type determines whether an application is visible to users on the Discovery Applications page, and can be set to either Public or Private. To change this value, refer to the section titled, "Configuring the application type" in the Oracle Endeca Information Discovery Studio Studio Administration and Customization Guide. You can also control the visibility of pages within an application. To manage page visibility, refer to the section titled, "Configuring the visibility type for a page" in the Oracle Endeca Information Discovery Studio Studio Administration and Customization Guide.
This section provides troubleshooting notes and guidelines.
To troubleshoot ETL, view logging information in the ODI Operator tab. Key areas of interest include row counts, logical SQL for the business model mapping, and the timing of steps in the ETL. To troubleshoot logical SQL, obtain the SQL from the Operator, then run it in IssueSQL in Log Level 2 and obtain the SQL query from the Oracle BI Server's log, nqquery.log.
Note that repository metadata changes may impact ETL. If columns of joins are modified in the RPD, this may impact data extracted by ODI. Also, ROW_WID = 0 rows may cause less or no data to be returned if a fact table is joined to a dimension table missing a ROW_WID = 0 row.