1 Setting Up Services

The Oracle Life Sciences Data Hub (Oracle LSH) system administrator role is required to define services and service locations.

This chapter describes how to set up the distributed processing server and define service locations and services. It contains the following topics:

• About Services
  Oracle Life Sciences Data Hub includes adapters that allow users to define and run objects in external systems for manipulating and reporting Oracle LSH data.
• About the Distributed Processing Server
  Oracle Life Sciences Data Hub uses its Distributed Processing Server (DP Server) to pass jobs to processing engines of different technologies to execute user-defined Programs, Load Sets, and Data Marts. The DP Server also passes jobs to XML Publisher to produce predefined reports and to handle Report Set post-processing.
• Setting Up the Distributed Processing Server
  You must install the Oracle Life Sciences Data Hub Distributed Processing (DP) Server on each computer where you installed an external processing engine and also on the computer where you have installed XML Publisher.
• Defining Service Locations
  A service location is a computer where one or more external processing engines or IDEs used by are installed. You must register each service location by defining it.
• Defining Services
  This section includes the topics on service types, priorities, and instances.

About Services

Oracle Life Sciences Data Hub includes adapters that allow users to define and run objects in external systems for manipulating and reporting Oracle LSH data.

Oracle LSH includes adapters that allow users to define and run objects in external systems for manipulating and reporting data.

To integrate Oracle LSH with these external systems in your company, you must define a service location for each computer where an external system you are using is installed. Within the service location definition you must define a service for each service type required by each external system installed there.

Some external systems have more than one predefined service type; for example, one service type to integrate the system for use as an integrated development environment (IDE) and another service type to integrate the external system for use as a processing engine. See Service Types for further information.

For each service, you must define one or more service instances. The number of service instances required, and their exact function, depends on the external technology and the service type. See Service Instances for further information.

About the Distributed Processing Server

Oracle Life Sciences Data Hub uses its Distributed Processing Server (DP Server) to pass jobs to processing engines of different technologies to execute user-defined Programs, Load Sets, and Data Marts. The DP Server also passes jobs to XML Publisher to produce predefined reports and to handle Report Set post-processing.

The DP Server is designed to make the Oracle LSH execution platform engine scalable. You can install one or more instances of the DP Server on one or more computers. A single DP Server instance can handle jobs for one or more processing engines. You can install the DP Server on the same computer as the Oracle LSH database or application server or on another network node.

The DP Server requires access to XML Publisher for the purpose of producing coversheets for outputs and for merging outputs into single- or multi-volume PDF report sets. The DP Server also requires access to each processing engine that you install to use with Oracle LSH.

Conversely, each processing engine you install for use with Oracle LSH requires access to its DP Server instance.

You must also define one service location in the Oracle LSH user interface for each instance of the DP Server (usually one instance per computer) and define one or more services for each processing engine on that computer. For each service, you specify the number of service instances that can run simultaneously. A service instance handles one job at a time.

Oracle LSH tracks the availability of each service on each location and assigns new jobs according to their priority and the availability of appropriate service instances in different locations; see Job Allocation Algorithm.

The DP Server runs as a standalone Java program. It includes two dedicated threads:

• Listener. At each location, the listener listens on a predefined Advanced Queue message queue called the DP Server job queue for jobs that have been assigned to local services. When the listener detects an XML job message request for a local service on the job queue, it dequeues the message. The message contains all the necessary details of one or more jobs.
• Job Processor. The job processor thread uses the information in the XML message to connect back to the Oracle LSH database. Once connected, it downloads all the necessary data, including the executable program script, into a target directory identified in the XML message. The job processor then executes the job-specific script by spawning a new O/S-level process. This spawned process runs the processing engine, which in turn carries out the actual job execution using the downloaded job input data. After the spawned external process completes, the job processor connects back to Oracle LSH and uploads any output results into Oracle LSH.

This section contains the following topics:

• Log Files
  The log directory in your DP Server Home directory holds DP Server log files.
• Job Allocation Algorithm
  The description of the algorithm for job allocation

Log Files

The log directory in your DP Server Home directory holds DP Server log files.

Each time you start the DP Server it creates one log file. The DP Server adds log information to that log file each time it runs a job.

Job Allocation Algorithm

The description of the algorithm for job allocation

If more than one service of the same technology type and priority exists, including in different locations, Oracle Life Sciences Data Hub allocates each job using the following algorithm:

• Oracle LSH allocates the job to the service of the correct type and priority that has been least recently used, if the least-recently-used location has an available service instance.
• If there are no available service instances, Oracle LSH checks jobs currently using the appropriate service instances on the least-recently-used service and determines whether the jobs are active. If a job is not active, Oracle LSH sets its status to Failed and allocates the new job to the service instance. If all the jobs are active, Oracle LSH checks the next-least-recently-used service, and so on until all services of the correct technology type and priority have been checked. If all services are unavailable, Oracle LSH waits for a configurable interval and then checks the job activity again.

Oracle LSH uses the following algorithm to determine which available service instance to use if a service instance of the priority requested for a particular job is not available:

• If a high-priority service is requested but not available, run a normal-priority service; if a normal-priority service is not available, run a low-priority service.
• If a normal-priority service is requested but not available, run a low-priority service; if a low-priority service is not available, run a high-priority service.
• If a low-priority service is requested but not available, run a normal-priority service; if a normal-priority service is not available, run a high-priority service.

Setting Up the Distributed Processing Server

You must install the Oracle Life Sciences Data Hub Distributed Processing (DP) Server on each computer where you installed an external processing engine and also on the computer where you have installed XML Publisher.

The following steps are required:

1. Install the Distributed Processing Server. See "Install the Distributed Processing Server" in the Oracle Life Sciences Data Hub Installation Guide.
2. Copy files. See "Copy and Edit Files" in the Oracle Life Sciences Data Hub Installation Guide.
3. Define Service Locations and Services; see Defining Service Locations and Defining Services
4. Edit the files. See "Copy and Edit Files" in the Oracle Life Sciences Data Hub Installation Guide.
5. Start the DP Server. See Starting the Distributed Processing Server Service.

When you start the DP Server code you must specify a location for DP server log files. You may need to read and analyze these log files if problems occur with jobs run on external servers. Instructions for creating appropriate directories for this purpose are included in the Oracle Life Sciences Data Hub Installation Guide.

Defining Service Locations

A service location is a computer where one or more external processing engines or IDEs used by are installed. You must register each service location by defining it.

Note:

• Before defining a service location or services for use with Oracle Business Intelligence visualizations, read Setting Up Oracle Business Intelligence Visualizations.
• The Oracle Life Sciences Data Hub installation process automatically creates one service location with one normal-priority PL/SQL service with three instances allowed, for internal use. This enables batch processing and many functions in Oracle LSH Definition.

To define a Service Location:

1. In the Administration tab, click the Service Locations subtab.
2. Click Create Service Location. The Service Location screen opens.
3. Enter values in the following fields:
   • Location Name. Enter a name for your service location.

     Note:

     • For processing engine services including XML Publisher and SAS, this name must match (including case-sensitivity) the service location name defined in the DP Server Start script that you run on the computer that corresponds to this defined service location; see Starting the Distributed Processing Server Service. When you run the script, you pass the actual values required to identify the physical location.
     • For OBIEE Service Locations where the BI Server is not installed, the name is very important. See Setting Up Oracle Business Intelligence Visualizations for details on OBIEE services.
   • Description. Enter additional information about the service location.
   • Machine. Enter the name of the computer that contains the location. This is for your own records; it has no effect.
   • Max Simultaneous Jobs. Enter the maximum number of jobs that this location should handle at the same time, across all services of all technology types.
4. Click Apply. The system creates the Service Location definition and returns you to the View Service Locations screen. (In Oracle LSH 2.4.8, the DP Server Status field always displays the status "Running.")

Defining Services

This section includes the topics on service types, priorities, and instances.

For each service location, define services as follows:

• One or more services for each XML Publisher, SAS, Informatica, Oracle Reports, Oracle Business Intelligence Presentation Services instance, and Oracle Business Intelligence Publisher (BIP) engine installed on that computer
• One and only one service for PL/SQL as a processing engine
• One and only one service for each IDE service type

For the service location you specify the maximum number of jobs (across all services) that can run simultaneously. DESCOPED

For non-PL/SQL processing engine service types, you may want to define more than one service in order to allow jobs with different priorities to run on the same computer. For example, if you have both SAS and Oracle Reports installed on the same computer, you might define the following services for that service location:

• One high-priority SAS service with two instances allowed
• One normal-priority SAS service with seven instances allowed
• One normal-priority Oracle Reports service with five instances allowed

To define a Service on a Service Location:

1. In the Administration tab, click the Service Locations subtab. The Service Locations screen opens.
2. Click Service Location. The Service Location screen opens.
3. Click the Location Name of the Service Location where you want to create a Service. The Maintain Service Locations screen opens.
4. Click Create Service. The Services screen opens.
5. Enter values in the following fields:
   • Service Type. Select a value from the drop-down list. For a description of each type, see Service Types.
   • Description. Enter a description of the service.
   • Service Priority. Select a value from the drop-down list. You must define each service as High, Normal, or Low priority. See Service Prioritization for further information. IDE service types do not use this attribute.
   • Number of Instances. Enter the number of jobs you want to allow to use this service at the same time. See Service Instances for further information.
   • Launch Wait Time. Enter the length of time (in seconds) for the program to start executing after the message to start the program is sent to the DP Server Listener. Set to a value long enough that even a busy service will have time to receive and process the message —but not too long or error recovery is adversely affected. 30 - 120 seconds is recommended.
   • Reuse Wait Time. Enter the length of time (in minutes) that you want the system to wait after the start of a program's execution until it starts checking whether a program is still executing when the system is looking for available service instances. Set to an interval large enough to allow most programs of the service's technology type to complete.
   • Default Resource Group. Enter the name of an Oracle database resource group to set the Oracle database priority for jobs using this service. For more information, see the Oracle® Database Administrator's Guide at https://docs.oracle.com/cd/E11882_01/server.112/e25494/toc.htm.
   • Execution Command. Enter one of the following:
     • For services used to launch an IDE, enter the URL of the development or visualization environment.
     • For services used to execute a job, enter the complete directory path and filename for the UNIX or Windows execution command script required for the service type. The service instance launches this script in order to run a job. The script must be located in the DP Server Home directory or a subdirectory of it. Instructions for copying and editing the sample execution command scripts, which are located in $CDR_TOP/admin/template, are included in the "Install the Distributed Processing Server" section of the Oracle Life Sciences Data Hub Installation Guide.
     You can customize the script to change the OS execution priority of the program to correspond to the service's priority; see Service Prioritization.

     The sample scripts include:

     • cdrzip-1.sh for Text Data Marts
     • sasNormal.sh for SAS Programs
     • oraexp.sh for Oracle Export Data Marts
     • orareprunner.sh for Oracle Reports Programs
     • sqlplus for PL/SQL Programs
     • txtNormal for Text Load Sets
     • xmlprunner.sh for post-processing Report Sets
     • xmlpreprunner.sh for the Oracle LSH system reports and for coversheets for outputs
     • obieeinstall.cmd for Oracle Business Intelligence Business Areas based on OBIEE 11g, or obieeinstall10g.cmd for Oracle Business Intelligence Business Areas based on OBIEE 10g; see Setting Up Oracle Business Intelligence Visualizations.
     • obieedeploy.cmd for Oracle Business Intelligence Business Areas based on OBIEE 11g, or obieedeployl0g.cmd for Oracle Business Intelligence Business Areas based on OBIEE 10g; see Setting Up Oracle Business Intelligence Visualizations.
     • obieepsrestart10g.cmd or obieepsrestart10g.sh for Oracle Business Intelligence Business Areas based on OBIEE 10g that are using a multiple-computer configuration; see Setting Up Oracle Business Intelligence Visualizations.
     • infarunner.sh for Informatica Programs
     • cdrunzip.sh for creating Tables from a zipped file and Text Load sets
   • Root Directory. Enter the directory to be used as the ROOT of the directory structure where jobs will execute. The DP Server creates a subdirectory in the root directory for each job, named with the job ID; for example:

     root dir/3233210001

     where the job ID is 3233210001. This directory will hold all the files and/or additional directories needed for the job to run.

   • Enabled. If set to Yes, this service is available for use.
   • Grant Connect. If set to Yes, the service can connect to the database to retrieve and manipulate data. If the service does not need to connect to the database, set to No.

     Note:

     Set Grant Connect to No for the Workflow service.
   • Grant Resource. If set to Yes, the execution accounts can create tables in the Service Instance account.

     Note:

     Set Grant Resource to No for the Workflow service and to Yes for the Oracle Export service.
   • Set Synonym. If set to Yes, synonyms are created when the user connects to the database to run the job or the development environment.

     Note:

     Set the value of Set Synonym to No for the Workflow and Oracle Export services.
   • Details. This field takes different values for different adapters. Not all adapters use this field but it is mandatory for the adapters that do. The following adapters use it as follows:
     • Informatica. Enter the Informatica domain name, repository name, and integration service name, separated by a vertical bar; for example: Domain_infa444dom|PowerCenter_abcd|PowerCenter_abcd_Integration_Service.
     • OBIEE Business Area Install Service. If the service location is a UNIX computer, enter: os=unix

       See Setting Up Oracle Business Intelligence Visualizations for more information.

     • OBIEE Business Area Deploy Service. Enter the values described below, delimited by commas:

       DSN. Enter the DSN name if you are using an ODBC system DSN, or enter the TNS name of the source Oracle LSH database server name if you are using OCI10g.

       Call Interface. Enter ODBC if you are using an ODBC system DSN, or OCI10g if you are using OCI10g.

       RPD. Enter the name of the OBI repository file you defined for this computer in the NQSConfig.ini file on the BI Server. The name must be an exact match of the value there, except the value here should not include the .rpd extension. Oracle recommends using the computer name to help ensure consistency.

       Version. If you are using OBIEE 11g, add the string ver=11g at the end.

       Examples

       If you are using an ODBC DSN with OBIEE 11g:

       dsn=name_of_DSN,callinterface=ODBC,rpd=name_of_the_OBI_Repository_file_to_be_deployed_on_this_Oracle_BI_Presentation_Server,ver=11g

       If you are using OCI10g and OBIEE 10g:

       dsn=tns_name_of_source_LSH_database_server ,callinterface=OCI10G,rpd=name_of_the_OBI_Repository_file_to_be_deployed_on_this_Oracle_BI_Presentation_Server

     • SAS with SQL*Loader. If you are running the SAS and the SQL*Loader services on the same DP server computer for the purpose of bulk loading SAS data sets, enter the absolute path of the SQL*Loader executable in this field for the SAS service.
   This section contains the following topics:

• Service Types
  Some service types use the Distirbuted Processing (DP) Server and others do not. It depends on their definition requirements.
• Service Prioritization
  You must specify a priority for each service: High, Normal, or Low.
• Service Instances
  When you create a service, you must specify how many instances of the service are allowed to run at the same time.

Service Types

Some service types use the Distirbuted Processing (DP) Server and others do not. It depends on their definition requirements.

See the following topics for details:

• Service Types that Do Not Use the DP Server
  Development (IDE) service types do not use the DP Server. The PL/SQL processing engine also does not use the DP Server.
• Service Types that Use the DP Server
  Processing engines integrated with Oracle Life Sciences Data Hub use the DP Server.

Service Types that Do Not Use the DP Server

Development (IDE) service types do not use the DP Server. The PL/SQL processing engine also does not use the DP Server.

For each of the technologies you plan to use, you must define a single service of the corresponding service type. You cannot control where these services are used. The system does not use the service location you define to determine where to run them. However, you must define them within a service location. You can define them on the service location where they are actually installed, or you may want to define a service location specifically for the purpose of defining these services, that include:

• Oracle Reports for Development. Oracle Life Sciences Data Hub uses these services to support the Oracle Reports integrated development environment (IDE).
• PLSQL for Development. Oracle LSH uses this service to support the SQL*Plus integrated development environment (IDE) for PL/SQL Programs.
• SAS for Development. Oracle LSH uses this service to support the SAS integrated development environment (IDE).
• PLSQL. Oracle LSH uses PLSQL services to run internal jobs and user-defined PL/SQL Programs and Text Data Marts.
• OBIEE Business Area IDE Service. This is the service that takes the OBIEE Presentation Server's URL as input for the Execution Command field. Oracle LSH uses additional OBIEE service types to support running visualizations through the OBIEE Presentation Server. For more details, see Setting Up Oracle Business Intelligence Visualizations. The other OBIEE Services are listed under Service Types that Use the DP Server.
• BI Publisher Development. Oracle LSH uses this service to support the BI Publisher integrated development environment (IDE).
• Informatica for Development. Oracle LSH uses this service to support the Informatica integrated development environment (IDE).
• View for Development Oracle LSH uses this service to support the SQL*Developer integrated development environment (IDE) for View Programs.

Service Types that Use the DP Server

Processing engines integrated with Oracle Life Sciences Data Hub use the DP Server.

You can control the number of service locations and services for service types that use the DP Server. Service locations correspond to physical locations, and each one can have any number of services defined. Each service can have any number of service instances defined. For more details, see Service Instances.

For each technology you plan to use, define at least one service of the following service types:

• Oracle Export. Oracle LSH uses Oracle Export services for creating Oracle Export-type Data Marts.
• Oracle Reports. Oracle LSH uses these services to execute Oracle Reports Programs.
• SAS. Oracle LSH uses SAS services to execute SAS Load Sets, SAS Programs, and SAS Data Marts on a SAS server.
• Text for SQLLoader. Oracle LSH uses Oracle SQL Loader to execute Text Load Sets.
• Workflow. Oracle LSH uses Workflow services to execute user-defined and internal Workflows.
• XML Publisher. Oracle LSH uses XML Publisher services to generate PDF-format Report Sets.
• XML Reports. Oracle LSH uses these services to produce predefined reports and the coversheets for reports and report sets.
• Zip Utility. Oracle LSH uses Zip Utility services to produce zipped Text and SAS Data Marts. You can zip the text files if you want to.
• Unzip Utility. Oracle LSH uses Unzip Utility services to unzip uploaded zip files while creating Tables and Text Load Sets.
• OBIEE Business Area Install Service. This service is required to support running visualizations through the OBIEE Presentation Server, and using the OBIEE Admin Tool. For more details, see Setting Up Oracle Business Intelligence Visualizations

• OBIEE Business Area Deploy Service. This service is required to support running visualizations through the OBIEE Presentation Server. For more details, see Setting Up Oracle Business Intelligence Visualizations.

• BI Publisher. Oracle LSH uses this service to execute BI Publisher Programs.

• Informatica. Oracle LSH uses this service to execute Informatica Programs.

Note:

The following service types are inactive: Oracle Warehouse Builder and SQL*Plus. Do not define services of these types.

Service Prioritization

You must specify a priority for each service: High, Normal, or Low.

To ensure that services are given the appropriate priority, you should also do the following:

• Specify an Oracle Database Resource Group to set the Oracle database priority for jobs using this service. For more information, see the Oracle® Database Administrator's Guide at https://docs.oracle.com/cd/E11882_01/server.112/e25494/toc.htm.
• In the Execution Command, set the operating system priority for jobs using this service.

Service Instances

When you create a service, you must specify how many instances of the service are allowed to run at the same time.

Since each instance can service one job (or Program or Business Area definition) at a time, this effectively defines the number of jobs that can run on a particular service at a particular time.

For processing engine service types, service instances serve as the interface between the requested job and the Oracle Life Sciences Data Hub execution system. They track the association of a particular job to a particular service and manage many aspects of the job's execution.

IDE service instances control the association between the use of a particular IDE for a particular Program or Business Area (which are used for visualizations) and the security and data access needed by the particular IDE. When a user launches an IDE from the Program or Business Area user interface, the system assigns a service instance to the Program or Business Area. After a service instance is allocated, it remains assigned to that Program or Business Area until it is explicitly released by a Checkin or Undo Checkout.

You can modify the number of service instances for each service.

Note:

For the OBIEE IDE Service, specify a number of Service Instances equal to the maximum number of users you expect to query the repository at the same time, plus a buffer.

If you do not create enough service instances, users may lose access to the database in the middle of a task. For example, if you define 10 service instances and 11 people log in to the same OBIEE URL, when the last person logs in, the system looks for a session that has been running longer than the defined Reuse Wait time for the service. If there is such a session, the system recycles it even if the user is active, and the original user loses his or her connection to the database.

It is better to solve this problem by defining a large number of service instances than a high reuse wait time because the same user cannot open a new session—for example, if he or she wants to select different launch settings—until his or her original session reaches the reuse wait time.