Extract Integration Point

Introduction

This integration point enables OHI Application(s) to do bulk data extraction. It exposes APIs to submit and monitor extract process. The submit API accepts an extract configuration and triggers an asynchronous process and once the process is completed a notification is sent out to the configured destination.

Extract Configuration

An extract configuration defines the definition of data that need to be extracted out. As a part of the configuration, it is possible to provide the row selection criteria, property selection, any linked data that is not directly part of resource itself.

resourceName: The root resource for extraction, example: policies, ctrclaims.
q: The query string for the root resource, this provides the row selection criteria and follows the Query API semantics.
fields: The basic and reference field properties selection for the root resource, with the fields configuration it is possible to include/exclude properties, check the below examples.
expand: The subResource property selection criteria for the root resource. To make configuration less verbose, it is possible to use shorthand keys dynamicData, rootDynamicData and subResources to include certain type of data. Check below examples
linkedResources: This allows to include extra resources in the extract that are not part of root resource. Example: calculation results along with policies. This is a list of linkedResource configuration to extract multiple resources along with the root resource. The configuration related to linked resource is similar to the root resource
- resourceName: The linked resource name.
- join: The linked resource join criteria with the root resource. Only supports subset of Query API operators, check below examples.
- q: Other selection criteria for the linked resource.
- fields: The basic and reference type properties selection for the root resource.
- expand: The subResource properties selection criteria for the linked resource.

Extract Payload

The extract process generates a DataFileSet with multiple files, each file containing an array of records and each record is a representation of a row selected by the root resource query criteria including all the linked resources for the row selection.

Default Row Payload

The payload for each selected row contains the root resource and the properties of all the linked resources. The resource properties can be found in the View Objects page. Object properties in OHI Components can have different data types. Each datatype has a different representation which is defined in the section Property Type Representation.

The property name of a object property matches the name in the View Objects page: for fixed properties, just the name is used. For dynamic properties, the Field Usage Name is used as property name.

Default representation of a resource include following elements:

Element	Included?
Basic Properties (single value)	Yes
Reference Properties (single value)	No
Dynamic Record (single value)	No
Sub resources	No
Basic Properties (multi value)	Yes
Reference Properties (multi value)	No
Dynamic Record (multi value)	No
Basic Properties (time valid)	No
Reference Properties (time valid)	No
Dynamic Record (time valid)	No

Element

Included?

Basic Properties (single value)

Yes

Reference Properties (single value)

Dynamic Record (single value)

Sub resources

Basic Properties (multi value)

Yes

Reference Properties (multi value)

Dynamic Record (multi value)

Basic Properties (time valid)

Reference Properties (time valid)

Dynamic Record (time valid)

Influence Row Payload

The default presentation can be influenced by fields/expand parameters. Expand is used to control the subResource and fields is used to control the basic and reference properties.

Include Sub Resources using "expand"

"expand" value	claimLine included?	claimLineDiagnosis included?	claimDiagnosis included?	claimPendReason included?	claimLinePendReason included?
not specified	No	No	No	No	No
claimLineList	Yes	No	No	No	No
claimLineList	claimDiagnosesList	Yes	No	Yes	No
No	subResources	Yes	No	Yes	Yes
No	claimLineList.subResources	Yes	Yes	No	No

"expand" value

claimLine included?

claimLineDiagnosis included?

claimDiagnosis included?

claimPendReason included?

claimLinePendReason included?

not specified

claimLineList

Yes

claimLineList

claimDiagnosesList

Yes

subResources

Yes

claimLineList.subResources

Yes

Include DynamicData

The "expand" also influences the dynamicData.

"expand" value	claim dynamicData included?	claimLine dynamicData included?	claimDiagnosis dynamic data included?	brand dynamicData included?	claimLinePendReason dynamicData included?
not specified	No	No	No	No	No
rootDynamicData	Yes	No	No	No	No
subResources	Yes	Yes	Yes	No	No
dynamicData	Yes	Yes*	Yes*	No	Yes*
brand.dynamicData	No	No	No	Yes	No
claimLineList.subResources	No	Yes	No	No	Yes
claimLineList.dynamicData	No	Yes	No	No	No

"expand" value

claim dynamicData included?

claimLine dynamicData included?

claimDiagnosis dynamic data included?

brand dynamicData included?

claimLinePendReason dynamicData included?

not specified

rootDynamicData

Yes

subResources

Yes

dynamicData

Yes

Yes*

brand.dynamicData

Yes

claimLineList.subResources

Yes

claimLineList.dynamicData

Yes

Dynamic data is only included, if the sub resource itself is included.

Include/Exclude properties using "fields"

The "fields" parameter can be used to control which properties are to be included in the resource representation. There is no way to explicitly exclude properties, that works together with include, if any one of the basic property is included on certain level then other basic properties are excluded.

"fields" value	claim included?	servicedMember included?	servicedMember.addressList included?	claimLineList included?
not specified	All the properties specified in default representation	No	No	No
code	id	Only code and id are included	No	No
No	servicedMember	Default representation + servicedMember reference property	Default representation properties for servicedMember	No
No	servicedMember.addressList	Default representation + servicedMember reference property	Default representation properties for servicedMember addressList sub resource	Default representation for all the addresses.
No	claimLineList.id	claimLineList.code	All the properties specified in default representation	No

"fields" value

claim included?

servicedMember included?

servicedMember.addressList included?

claimLineList included?

not specified

All the properties specified in default representation

code

Only code and id are included

servicedMember

Default representation + servicedMember reference property

Default representation properties for servicedMember

servicedMember.addressList

Default representation + servicedMember reference property

Default representation properties for servicedMember
addressList sub resource

Default representation for all the addresses.

claimLineList.id

claimLineList.code

All the properties specified in default representation

Joining Linked Resources

The grammar used for the join criteria is as follows. As a convention the join criteria left hand side expression is root resource and right side expression is linked resource.

join := expression
expression := [[(]simple_expression | complex_expression[)]]
simple_expression := field_name.operator(linked_resource_name.field_name)
operator := eq | neq
field_name := attribute[(+)]?[.field_name]* -- for joins
complex_expression := expression.complex_operator.expression
complex_operator := and | or

Join expressions does not support joining on dynamic fields.

Example: Simple Expression

The extract is on claims and it is required to fetch ctrClaims along with it as linkedResources. The join between ctrClaim and claim is on code so the join criteria is a simple expression.

code.eq('code')

Here, left hand side is referred to claims root resource and right hand side refers to ctrClaims linkedResource.

Example: Complex Expression

The extract is on policies and it is required to fetch all other versions of policies as linkedResources. The join between policies and policies versions is on gid and the id of the two is not the same, so the join criteria is a complex expression.

gid.eq('gid').and.id.neq('id')

Here, left hand side is referred to policies root resource where it is the last version and right hand side refers to policies as linkedResource to fetch the versions.

Extract Notification

The extract process supports the notification mechanism. A notification key can be provided during the extract submission process. If the notification key is not provided, notification step is skipped. And If the notification key is provided, after the extract process is completed successfully or unsuccessfully a notification is sent out to the configured endpoint.

ohi.extract.{0}.notification.endpoint property is used to configure the notification endpoint. {0} should be replaced with the notification key defined on the extract. The configured endpoint url should accept the POST requests.

Gateway extract requests use "gateway" as the notification key. It points to the Notification Integration Point in the Gateway

JSON is the default media type used for the notifications. It is possible to configure other media types with rest client service media type properties. ohi.service.{0}.media.type property is used to configure the media type of the notification. {0} should be replaced with the notification key defined on the extract.

Following message is used for notification:

{
    "status": '',
    "correlationId": '',
    "links": [
        {
            "rel": 'file',
            "href": 'href='http://host:port/contextroot/datafilesets/{datafilesetcode}/datafiles/{datafilecode}/data',
        }
    ]
}

Authentication

The extract notification supports different authentication types for the notification clients, the default value is BasicAuthentication. ohi.service.{0}.client.authentication property can be used to set the authentication feature, {0} should be replaced with the notification key on the extract, same notification key is also used for credentialKey.

Authorization

This integration point is protected by "extracts IP" access restriction.

API Conversation & Interaction

This integration point uses Long Running Operations through REST API conversation style. This section provides a complete example of a single extract conversation.

This example covers the use case of extracting all the claims with person as linked resource, and claimLine serviced member is used for join criteria. The extract payload includes all the claim and claimline sub resources and for person linked resource all the sub resources for the person. The extract notification is sent out to the configured endpoint with 'gateway' notification key.

The submit API accepts a optional "correlationId" header that can be used by external systems to match the request and notification message. When provided the header is stored on the extract and later included as a part of the notification message.

Starting Extract Process

To start the extract process, following request is submitted to /extracts using the POST method.

Request

header - correlationId: 123
{
   "notificationKey": "gateway",
   "resourceName": "claims",
   "expand": "subResources|claimLineList.subResources",
   "linkedResources": [
    {
       "resourceName": "persons",
       "join": "claimLineList.servicedMember.code.eq('code')",
       "expand": "subResources"
    }
   ]
}

Response

201
Content-Type: application/vnd.oracle.insurance.resource+json
Vary: Accept,Accept-Encoding,Accept-Language,Origin
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-OHI-APP-ID: ALI
X-OHI-APP-VERSION: 6.0.0
X-XSS-Protection: 1; mode=block
Location: http://host:port/api/generic/extracts/12324

The system responds with HTTP 201 (Created) and a location header for the extract.

Monitor Extract Process

A monitor link from the starting response can be used for monitoring the extract process. The following request can be used to get the extract status.

Request

GET /extracts/1

Response

200
{
    "notificationKey": "gateway",
    "status": "InProcess",
    "links": [
        {
            "href": "http://hostname:port/contextroot/api/activities/{id}",
            "rel": "operator",
            "httpMethod": "GET"
        },
        {
            "href": "http://hostname:port/contextroot/api/extracts/{id}",
            "rel": "monitor",
            "httpMethod": "GET"
        }
    ]
}

200 response is returned with the payload containing links and status of the extract. In the event of extract process failure additional links operator:restart, operator:messages are included in the response. If the same request as above is used in failure scenario the response looks like below:

{
    "notificationKey": "gateway",
    "status": "TechnicalError",
    "links": [
        {
            "href": "http://host:port/contextroot/api/activities/234",
            "rel": "operator",
            "httpMethod": "GET"
        },
        {
            "href": "http://host:port/contextroot/api/activities/234/recover",
            "rel": "operator:restart",
            "httpMethod": "POST"
        },
        {
            "href": "http://host:port/contextroot/api/activities/234/messages",
            "rel": "operator:messages",
            "httpMethod": "GET"
        },
        {
            "href": "http://host:port/contextroot/api/extracts/1",
            "rel": "monitor",
            "httpMethod": "GET"
        }
    ]
}

Completion of Extract Process

After the extract process is completed, a notification is sent out to the configured endpoint. The location header link returned in the first step can be used to get the extract results.

GET on location header link returns following response:

{
    "notificationKey": "gateway",
    "status": "Completed",
    "links": [
        {
            "href": "http://host:port/contextroot/api/activities/2572368306",
            "rel": "operator",
            "httpMethod": "GET"
        },
        {
            "href": "http://host:port/contextroot/api/datafilesets/EXT12302",
            "rel": "datafileset:input",
            "httpMethod": "GET"
        }
    ]
}

Response contains a link to datafileset, this link can be used to download the extract results.