Data Collection Steps

A data collection step defines the way in which an integration obtains data from a source system. A specific integration may contain one or more data collection steps.

Query Step

This step makes it possible to get a real-time response from Oracle Health Insurance applications e.g Claims, using Query API. Query API is an HTTP API framework supported by Oracle Health Insurance applications.

The format in which the data gets queried from the Oracle Health Insurance source application is JSON and is Oracle Health Insurance driven. The response can be transformed before delivery by configuring a subsequent Delivery step or Transform step within the integration.

The following table lists the fields valid for this type of step.

Field	Description	Mandatory?
Code	Code of the Query step	Yes
Sequence	The execution sequence for this step. The integration steps get executed in an ascending order of their sequence.	Yes
Subtype	QUERY	Yes
Condition Invocation Check	A dynamic logic condition to check if this step should be invoked. The step gets skipped if the condition returns false.	No
Function Post Process	A dynamic logic function to post process the results of the step.	No
Destination	Reference to the REST destination. For more details on REST destinations refer to the "Destinations" page in the Building Blocks chapter of the Configuration Guide. For the Query step, the destination address key must specify the URL value only till the context root of the application.	Yes
Output Name	Dynamic logic bind variable name. This variable holds the queried results as raw payload and is available as a bind variable to all the subsequent steps and dynamic logic. If the output name is not specified, then the integration step code is used as the bind variable.	No
Type Config	The configuration of the query includes supplying information about: resourceName: The root resource to query. List of resources can be accessed using generic GET operation on /generic/ohiresources locationHeader: A location header of a resource that has been determined earlier in the integration invocation flow. This takes precedence over the q element mentioned next. q: The query string for the root resource, this provides the row selection criteria and follows the Query API semantics. For more details, refer to the "Query API" page in HTTP API chapter of the Developer Guide. It is also possible to inject parameters in the query expression. Example: servicedMember.code.eq({serviced_member_code}). Here the service member code is parameterized. Refer to the section parameter and exchange properties for more details. fields: The column selection criteria for the resource. This influences on what gets packaged in the payload for the resource. For more details, refer to the chapter "Influence Resource Representation" page of the HTTP API chapter in the Developer Guide. expand: The column selection criteria for the resource. This influences on what gets packaged in the payload for the resource. For more details, refer to the "Influence Resource Representation" page of the HTTP API chapter in the Developer Guide.</li> </ul>	Yes

Query API response size is limited to a maximum of 200 records. For large data sets, Oracle recommends the use of Extract step to gather data.

Sample Configuration: Query member data from an Oracle Health Insurance application for a given member code.

Configure the Query step as follows to query member data from Oracle Health Insurance application.

Field	Configuration	Remarks
code	getMemberData
sequence	1	Assuming this is the first step
destination	policies	Refers to a rest destination with code "policies" (assuming the person data gets queried from Policies application)
outputName	memberData	The Query API response can be accessed in the dynamic logic of any subsequent step using "memberData" bind variable.
type config	resourceName: persons q: code.eq("{memberCode}") expand:addressList	maritalStatusList

An example of the JSON payload for the creation of a Query step using API is given below. Note that this JSON is part of the larger payload needed for the creation of the Integration.

{
  "code"        : "getMemberData",
  "sequence"    : 1,
  "subtype"     : "QUERY",
  "destination" : { "code" : "policies"},
  "outputName"  : "memberData",
  "typeConfig"  : {
     "resourceName" : "persons",
     "q"            : "code.eq("{memberCode}")",
     "expand"       : "addressList|maritalStatusList"
   }
}

Configure the REST destination as follows :

Field	Configuration	Remarks
code	policies
credentialKey	policy_user	The following must be done to set up credential key "policy_user" Step 1:: Create credential using credential IP or Credential Page Example: To setup credential key "policy_user" using Credential IP the following must be done ---- PUT: http(s)://{host:port}/{oig application context}/credentials/credential/policy_user with payload { "username" : "user that OIG should use to access policy application", "password": "password to access policy application" } ---- Step 2:: Set the authentication mechanism using Generic API - properties or using property configuration page Example: To set up authentication using Generic API - /properties (assuming the polices application uses basic authentication) the following must be done ---- POST : http(s)://{host:port}/{oig application context}/generic/properties with payload { "name": "ohi.service.policy_user.client.authentication", "value": "BasicAuthentication" } ---- TIP: For details on "Generic API" refer to Developer Guide, section Integration Generic API and Operations.
addressKey	address.key.policies.baseurl	The address key can be setup by Generic API /properties or properties page Example : To setup address key through Generic API - /properties the following must be done POST: http(s)://{host:port}/{oig application context}/generic/properties with payload { "name": "address.key.policies.baseurl", "value": "URL till {policiesComponent} context root "e.g." http://{host"port}/{context-root}" }
typeConfig	path: {path} httpMethod: {httpMethod}	NOTE: For Query step, the system sets the path internally to 'generic/{resource}' and method as 'GET' Here, the path and http method are parameterized (recommended). This way a single rest destination can be used by multiple integrations with the policies application. For more details on parameterized configurations, refer to the section Integration Parameters and Exchange Properties.

An example of the JSON payload for the creation of polices destination though API /restdestinations is given below.

{
    "code": "policies",
    "credentialKey": "policy_user",
    "addressKey": "address.key.policies.baseurl",
    "typeConfig": {
        "path": "{path}",
        "httpMethod": "{httpMethod}"
    }
}

Extract Step

Extract step enables the users to collect data from the source Oracle Health Insurance application as a file extract This step triggers Extract Integration Point in the end applications and therefore offers all the possibilities supported by the extract integration point. For more details on extract integration point, please refer to the Developer Guide in the core application.

The following table lists the fields valid for this type of step.

Field	Description	Mandatory?
Code	Code of the Extract step	Yes
Sequence	The execution sequence for this step. The integration steps get executed in an ascending order of their sequence.	Yes
Subtype	EXTRACT	Yes
Condition Invocation Check	A dynamic logic condition to check if this step should be invoked. The step gets skipped if the condition returns false	No
Function Post Process	A dynamic logic function to post process the results of the step.	No
Destination	Reference to the REST destination. For more details on REST destinations refer to the "Destinations" page in the Building Blocks chapter of the Configuration Guide. For the Extract step, Oracle Insurance Gateway constructs URL to access the extract endpoint in the end application based on the context information as given by the destination address key. The destination address key must specify the URL value only till the context root of the application.	Yes
Output Name	Dynamic logic bind variable name. This variable holds the data files extracted (when exists) or the response payload as received from the external system and is available as a bind variable to all the subsequent dynamic logic/steps. If the output name is not specified, then the integration step code is used as the bind variable.	No
Type Config	The configuration of the extract includes supplying information about: resourceName: The root resource on which the extract is needed. Note that the resourceName must be a constant value. Use of a parameter results in validation exception OIG-GATE-057. List of resources can be accessed through GET operation on Generic API /ohiresources q: The query string for the root resource, this provides the row selection criteria and follows the Query API semantics. It is also possible to inject parameters in the query expression. Example: servicedMember.code.eq({serviced_member_code}). Here the value for the parameter in curly brackets is parameterized. For more details on parameterized configurations, refer to the section Integration Parameters and Exchange Properties. fields/expand: The column selection criteria for the root resource. This influences on what gets packaged in the payload for the root resource. downloadMessages: true or false - Indicates if messages should to be auto downloaded. Default value is true. timeAllowed: The time allowed in minutes for this extract step to finish, before the Integration Gateway tries to fetch the status of the extract from target application. This can also be overwritten at the time of invocation. See below. linkedResources: This facilitates mentioning extra resources that also need to be extracted but are not available in the root resource object graph. Example, financial information along with claims information. This is a list of linkedResource configuration to extract multiple resources along with the root resource. The configuration related to linked resource is similar to the root resource resourceName: The linked resource name. List of resources can be access using generic GET operation on Generic API /ohiresources join: The linked resource join criteria with the root resource. See below for grammar. q: Other selection criteria for the linked resource. fields/expand: The column selection criteria for the linked resource. For details on fields/expand, refer to the "Influence Resource Representation" page of the HTTP API chapter in the Developer Guide.. For more details on Oracle Health Insurance extracts functionality, refer to Extracts Integration Point in the Oracle Health Insurance application’s Developer Guide.	Yes

Configuring multiple Extract steps for merging is not recommended. From a performance standpoint, considering the high volume nature of the extract, transforming/merging two high volume files is not recommended.

Join

The grammar used for the join criteria is as follows. As a convention the join criteria left-hand side expression is the root resource and right side expression is linked resource.

 join := expression
 expression := [[(]simple_expression | complex_expression[)]]
 simple_expression := field_name.operator(linked_resource_name.field_name)
 operator := eq | neq
 field_name := attribute[(+)]?[.field_name]* -- for joins
 complex_expression := expression.complex_operator.expression
 complex_operator := and | or

Example: Simple Expression

The extract must fetch the claim and linked ctrClaim. The join between ctrClaim and claim is on code so the join criteria is a simple expression.

code.eq(code)

The left hand side refers to the claims root resource and the right hand side refers to the ctrClaims linkedResource.

Example: Complex Expression

The extract must fetch policies and other versions of the policies as linkedResources. The join between policy and its versions is on gid and the id of the two is not the same, so the join criteria is a complex expression.

gid.eq(gid).and.id.neq(id)

The left-hand side refers to the policies root resource which is the last version and the right-hand side refers to policies as the linked resource to fetch the previous versions.

Incremental or Full Extract?

Extract provides the ability to do full or incremental data collection. All extracts are by default incremental in nature. The integration invocation can control whether it is full or incremental, this is done by using the invocation parameter "extractIncremental" which takes a boolean value. Setting this parameter to 'false' results in a full extract. For details on setting the parameters while invoking an integration, refer to the Exchange Integration Point in the Developer Guide.

Incremental

By specifying the parameter "incrementalField" during integration invocation, it is possible to specify which field to add watermark check on. By default, this field is the root resource’s lastUpdatedDate. It can be overridden during the invocation of the extract to a different value. This field can also specify an attribute on a dependent/linked resource for example: when an incremental extract of claims is desired based on the lastUpdatedDate of servicedMember, the incrementalField is required to be sent as servicedMember.lastUpdatedDate.

Every integration invocation start time is logged with a timestamp. This current invocation’s timestamp acts as the high watermark for the incremental extract and the previous invocation’s timestamp (if any) acts as the low watermark.

If the previous invocation was not successful, the current invocation still incrementally collects data, and it is envisioned that the operator recovers/restarts the previous invocation.

The q-part or query part of the configuration in which all parameters are resolved into actual values is used as part of the criteria for retrieving the previous exchange. The following example, which periodically extracts data from Policies resource "policygroupaccounts" using the following query:

{
  "resource": {
    "q": "groupAccount.code.eq({code})"
  }
}

demonstrates how the system takes query parts of previous invocations into account.

At time 1, Policy Group Account data is extracted for group account "LARGE_ACCOUNT" using query

{
  "resource": {
    "q": "groupAccount.code.eq(LARGE_ACCOUNT)"
  }
}

As this is the first time an extract runs for this specific query it is not done in an incremental way.

At time 2, Policy Group Account data is extracted for group account "SMALL_ACCOUNT" using query

{
  "resource": {
    "q": "groupAccount.code.eq(SMALL_ACCOUNT)"
  }
}

Again, as this is the first time an extract runs for this specific query it is not done in an incremental way.

At time 3, Policy Group Account data is again extracted for group account "LARGE_ACCOUNT" using query

{
  "resource": {
    "q": "groupAccount.code.eq(LARGE_ACCOUNT)"
  }
}

As the system retrieves the invocation that ran at time 1 it applies that invocation’s timestamp as the low watermark, effectively executing query

{
  "resource": {
    "q": "roupAccount.code.eq(LARGE_ACCOUNT).and.groupAccount.lastUpdatedDate.gte(time 1)"
  }
}

Timeout

When the extract process is invoked, the target application replies back with the location header of the extract. If a timeout is configured and when the timeout expires, the Integration Gateway framework tries to fetch the status of the extract through the location header of the invoked extract process. If the status is a final state, the process at the gateway end continues in the same way as when a notification is received. If the status is not in a final state, the process sleeps again and retries again for a maximum of three times (this can be overridden by system property ohi.timeout.maxRepeatAttempts). The timeout value can be overwritten at the time of invocation by specifying the invocation parameter < Integration Step Code>-timeAllowed (for below sample, it would be: claimsExtract-timeAllowed). This overwritten value is in milliseconds. If there is no configuration present and there is no invocation time parameter, the default value of 5 minutes applies.

Please note that if the process resumes through a timeout and at a later point in time, notification of the same process also arrives, the notification would fail as the process has already resumed (and further resuming the process is not possible anymore).

Sample Configuration: Extract incremental claims data periodically for the data warehouse

The integration can be configured as follows:

	Field	Configuration
Remarks	Integration	code
claimsExtract
schedule	0 0 23 * * *	Every night at 11 PM
Integration Step	code	extractStep
Extract of all claims periodically for the data warehouse
by default the extract process informs the Oracle Insurance Gateway when it completes, as extracts is a long running process.	Set the 'extract notification endpoint' in the claims application to 'Oracle Insurance Gateway notification endpoint'. This requires the system property ohi.extract.gateway.notification.endpoint to be set to: http(s)://{host:port}/{oig application context}/notification In the claims application, the credential and authentication property must be set for the key gateway. Here, 'gateway' is the credential key/client id based on which all endpoint and authentication information must be set in Claims Application. This is similar to the setting of credential and authentication for the key policies_user as explained in the Query step example
destination	claims	Refers to a rest destination with code "claims" (for details refer to "polices" destination explained with Query step)
	sequence	1
		outputName
claimsData
type config	resourceName: claims expand: dynamicData	subResources

The runtime invocation of the above configuration results in an integration invocation (exchange) log with timestamp.

Id	Integration ID	Timestamp
1	1	11 Jan 2019 23:00:45
2	1	12 Jan 2019 23:00:35

• First invocation of such an extract, (id=1), takes the high watermark as 2019-01-11 23:00:45, hence it adds a selection clause to the extract query as lastUpdatedDate < {invocation_timestamp}

• Second invocation of such an extract, (id=2), takes the high watermark as 2019-01-12 23:00:35 and low watermark as the previous invocation timestamp, hence it adds a selection clause to the extract query as lastUpdatedDate < {invocation_timestamp} and lastUpdatedDate >= {previous_invocation_timestamp}

An example of the JSON payload for the creation of such an Extract step though API is given below. Note that this JSON is part of the larger payload for the creation of the Integration.

{
  "code": "extractStep",
  "sequence": 1,
  "subtype" : "EXTRACT",
  "outputName" : "claimsData",
  "typeConfig": {
    "resourceName" : "claims",
    "expand" : "dynamicData|subResources"
  }
}

Custom Step

This step enables a user to collect data from the source applications using the Dynamic Logic. The dynamic logic provides an ability to perform external callouts using Callout Mechanism.

This step is useful to gather data when the source application is not an Oracle Health Insurance application and therefore the Query or Extract types cannot be used. This step can also be used to process data (perform DMLs) within Oracle Health Insurance using HTTP API framework, for example, process a request from a customer relationship management system for a specific member and update member data using persons API.

The following table lists the fields valid for this type of step.

Field	Description	Mandatory?
Code	Code of the Custom step	Yes
Sequence	The execution sequence for this step. The integration steps get executed in an ascending order of their sequence..	Yes
Subtype	CUSTOM	Yes
Condition Invocation Check	A dynamic logic condition to check if this step should be invoked. The step gets skipped if the condition returns false	No
Function Post Process	A dynamic logic function to post process the results of the step.	No
Destination	Reference to REST destination. For more details on destinations refer to the "Destinations" page in the Building Blocks chapter of the Configuration Guide.	Yes
Output Name	Dynamic logic bind variable name. This variable holds the raw payload returned by the dynamic logic function Data Collection Custom and is available as a bind variable to all subsequent dynamic logic/steps. If the output name is not specified, then the integration step code is used as the bind variable.	No
Type Config	The configuration of the custom step includes supplying information about customFunctionCode: The dynamic logic code with subtype FUNC and signature Data Collection Custom that provides the custom code to collect data. The dynamic logic provides the ability to perform external callouts using Callout Mechanism.	Yes

Sample Configuration: Getting payment status from an external financial system

Lets assume that a POST request with query parameters - member code, start date and end date - to a financial system endpoint 'http://{external application host:port}/{external application context}/{URI path}' returns member’s payment status history.

To achieve this the Custom step is configured as follows:

Field	Configuration	Remarks
code	getPaymentStatus
sequence	1	Assuming this is the first step in the Integration
destination	financials	Refers to a rest destination with code "financials"
outputName	paymentStatus	The response can be accessed in the dynamic logic of the subsequent step using "paymentStatus" bind variable.
type config	customFunctionCode : paymentStatusCallout

An example of the JSON payload for the creation of a custom step using API is given below. Note that this JSON is part of the larger payload needed to create an integration

{
  "code"        : "getPaymentStatus",
  "sequence"    : 1,
  "subtype"     : "CUSTOM",
  "destination" : { "code" : "financials"},
  "outputName"  : "paymentStatus",
  "typeConfig"  : {
     "customFunctionCode":"paymentStatusCallout"
   }
}

Dynamic Logic - paymentStatusCallout is configured as:

String paymentStatus = webTarget("financials")
        .queryParam("memberCode", properties.get("memberCode"))
        .queryParam("startDate", properties.get("startDate"))
        .queryParam("endDate", properties.get("endDate"))
        .request()
        .header("Accept", "application/json")
        .buildGet()
        .invoke()
        .readEntity(String.class)

return paymentStatus

The rest destination "financials" is configured as :

Field	Configuration	Remarks
code	financials
credentialKey	financials_user	The following must to done to set up credential key "financials_user" Create credential using credential IP or Credential Page Example: To setup credential key "financials_user" using Credential IP the following must be done PUT http(s)://{host:port}/{oig application context}/credentials/credential/_financials_user_ with payload {"username" : "user that OIG should use to access financial application", "password": "password to access financial application" } Set the authentication mechanism using Generic API- properties or using property configuration page Example: To setup authentication using Generic API - properties (assuming the external financial application uses basic authentication) the following must be done POST http(s)://{host:port}/<oig application context>/generic/properties with payload { "name": "ohi.service.**financials_user**.client.authentication", "value": "BasicAuthentication" } For details on Generic API refer to Developer Guide, section Integration Generic API and Operations.
addressKey	address.key.financials.baseurl	The address key can be setup using generic properties resource or properties page Example: To setup address key using Generic API - properties the following must be done: POST http(s)://{host:port}/{oig application context}/generic/properties with payload { "name": "address.key.financials.baseurl", "value": "URL of the financials applications" }
typeConfig	path: must be set to the URI Path (relative to addressKey value) httpMethod: POST