3 Processing Transfers with Custom Callouts

This chapter describes how to create and use custom callouts for file transfer preprocessing and postprocessing in Oracle Managed File Transfer (MFT).

To create callouts, you must be familiar with Java and XML code and with creating transfers as described in Designing Artifacts: Transfers, Sources, and Targets.

This chapter includes the following sections:

Understanding Custom Callouts

Oracle Managed File Transfer provides built-in compression, decompression, encryption, and decryption actions for transfer preprocessing and postprocessing. See Setting Up Transfer Preprocessing and Postprocessing Actions and Setting Up Source Processing Actions for details.

You can create new preprocessing and postprocessing actions, which are called custom callouts. Examples of custom callout uses are as follows:

Adding copyright headers or footers
Validating content
Notifying a separate application in case of delivery success or failure
Invoking a separate application in case of delivery success or failure
Adding custom headers or properties, which are persisted in the MFT database
Adding an alternative encryption technique

This chapter uses a Newline Conversion callout example.

Custom callouts can be associated with either the source or the target. The sequence of processing action execution during a transfer is as follows:

Source preprocessing actions
Target preprocessing actions
Payload delivery
Target postprocessing actions

Note:

In the MFT console, target processing actions are configured at the transfer. In Designing Artifacts: Transfers, Sources, and Targets, these actions are referred to as transfer processing actions. In this chapter, these actions are referred to as target processing actions.

Note:

Postprocessing occurs after file delivery. Therefore, the Active Deliveries and File Finder views in the Dashboard tab on the Monitoring page show different statuses if file delivery succeeds but postprocessing fails. Specifically, the Active Deliveries view displays a Completed status but the File Finder view displays a Failed status.

Table 3-1 summarizes the types of callouts you can create.

Table 3-1 Callout Types

Callout Type	Interface Name	Payload Modification	Header and Property Modification
Preprocessing	PreCalloutPlugin	Allowed	Allowed
Postprocessing	PostCalloutPlugin	Not Allowed	Allowed

Creating a Custom Callout: High-Level Steps

The high-level steps for creating a custom callout are:

Create the Java source code based on the appropriate interface and package it in a JAR file. See Creating the Code.
Create a callout definition XML file based on the callout schema. See Creating the Callout Definition File.
Copy the JAR file and the XML file into the callout directory. See Locating the Callout Directory.
Run the createCallouts WLST command to configure the callout in MFT. See Running the createCallouts Command.
Test the callout by using it in a file transfer. See Testing the Callout.

The following sections describe these steps in detail using a Newline Conversion callout example. A newline, also known as a line break or end-of-line (EOL) marker, is a special character or sequence of characters signifying the end of a line of text. Unix operating systems use different newline characters than DOS (and Windows) operating systems. Using this callout, you can convert newline characters from DOS to Unix or from Unix to DOS.

Creating the Code

This section includes the following topics:

Java Code Requirements and Tips

The Java code defines the callout actions. It must meet these minimum requirements:

It must import the java.io.IOException, java.io.InputStream, java.io.OutputStream, and java.util.Map classes.
It must import the PluginOutput and PluginContext classes in the com.oracle.tip.mft.engine.processor.plugin package.
It must implement the PreCalloutPlugin or PostCalloutPlugin interface in the com.oracle.callout package.

See PreCalloutPlugin Interface or PostCalloutPlugin Interface for the contents of the interface.
It must be packaged in a JAR file.

You can include multiple callouts in the same JAR file or package them in separate JAR files.

To make a callout set header variables, use code like this:

context.getCustomPropertyMap().put("name", "value");
context.getTransportHeaderMap().put("name", "value");

To make a callout change the payload file name, use code like this:

PluginOutput pOutput = new PluginOutput();
    pOutput.setNewFileName("abc.xyz");
  return pOutput;

To make a callout perform different actions for source preprocessing, source postprocessing, target preprocessing, and target postprocessing, use code like this:

if(message instanceof oracle.tip.mft.bean.SourceMessage){
    // in case of source pre and post processing
} else if(message instanceof oracle.tip.mft.bean.Instance){
    // in case of target pre
} else if(message instanceof oracle.tip.mft.bean.TargetMessage ){
    // in case of target post
}

To compile the code, use a command like this, with the core MFT JAR file in the classpath:

javac -classpath $MW_HOME/mft/modules/oracle.mft/core.jar <class>

To create the JAR file, use a command like this:

jar cf newlineConversion.jar

Java Code for the Newline Conversion Example

The Java class for the Newline Conversion example performs these actions:

Specifies that the callout changes the payload.
Accepts the Type parameter value and determines whether it is Dos2Unix or Unix2Dos.
Returns null if no Type value is provided.
Uses the doLineConversion method to rewrite each line in the payload with the chosen newline characters.
Returns a copy of the payload file with the newlines changed.

Example - Newline Conversion Code shows the Java class code for the Newline Conversion example.

package com.oracle.callout.sample;

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.util.Map;

import oracle.tip.mft.engine.processsor.plugin.PluginContext;
import oracle.tip.mft.engine.processsor.plugin.PluginOutput;
import oracle.tip.mft.engine.processsor.plugin.PreCalloutPlugin;

public class NewlineConversion implements PreCalloutPlugin {

    @Override
    public boolean isPayloadChangeRequired(PluginContext context,
            Map<String, String> calloutParams) {
        return true;
    }

    @Override
    public PluginOutput process(PluginContext context, InputStream input,
            OutputStream out, Map<String, String> calloutParams) {
        String type = calloutParams.get("Type");
        boolean isDos = false;
        if ("Unix2Dos".equals(type)) {
            isDos = true;
        }
        doLineConversion(input, out, isDos);
        return new PluginOutput();
    }

    @Override
    public PluginOutput process(PluginContext context, InputStream input,
            Map<String, String> calloutParams) {
        return null;
    }

    public static final String DOS_NEW_LINE = "\r\n";
    public static final String UNIX_NEW_LINE = "\n";

    private void doLineConversion(InputStream in, OutputStream out,
            boolean isDos) {
        String newLineChar = null;
        if (isDos) {
            newLineChar = DOS_NEW_LINE;
        } else {
            newLineChar = UNIX_NEW_LINE;
        }
        BufferedReader bufferIn = null;
        BufferedWriter bufferOut = null;
        try {
            DataInputStream dataIn = new DataInputStream(in);
            bufferIn = new BufferedReader(new InputStreamReader(dataIn));
            DataOutputStream dataOut = new DataOutputStream(out);
            bufferOut = new BufferedWriter(new OutputStreamWriter(dataOut));
            // For each line in the un-normalized file
            String line;
            while ((line = bufferIn.readLine()) != null) {
                // Write the original line plus the operating-system dependent
                // newline
                bufferOut.write(line);
                bufferOut.write(newLineChar);
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            try {
                bufferIn.close();
                bufferOut.close();
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        }
    }
}

Creating the Callout Definition File

The definition file specifies the callout structure in the MFT configuration. It must meet these requirements:

It must conform to the XML format specified in the callout.xsd schema file. See Callout Definition Schema for the contents of the schema file.

You can include multiple callouts in the same definition file or package them in separate definition files.
The timeout attribute of the Callout element must have a value (in seconds) to prevent an insufficiently debugged callout with an infinite loop from running too long.
The implementationClass attribute of the Callout element must reference the Java class name, including the package name.
The libraryName attribute of the Callout element must reference the JAR file name.
If the Java class has input parameters, corresponding Parameter elements must be specified.

Example - Newline Conversion Callout Definition File shows the callout definition file for the Newline Conversion example.

Example - Newline Conversion Callout Definition File

<?xml version="1.0" encoding="UTF-8"?>
<mft:Callouts xmlns:mft="http://xmlns.oracle.com/mft" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://xmlns.oracle.com/mft callout.xsd ">
    <mft:Callout description="New line conversion"
        helpText="New line conversion"
        groupName="Source-pre,Target-pre" timeout="300" 
        implementationClass="com.oracle.callout.sample.NewlineConversion"
        libraryName="newlineConversion.jar" name="New line conversion">
        <mft:Parameter description="Type" mandatory="true" helpText="Type"
            name="Type" listValue="Dos2Unix,Unix2Dos"
            parameterType="string" defaultValue="Dos2Unix"/>
    </mft:Callout>
</mft:Callouts>

Locating the Callout Directory

After you have created the JAR file and the definition file, you must copy them into the MFT callout directory. The default location of this directory is as follows:

FMW_HOME/user_projects/domains/soainfra/mft/callouts

To verify the location, you can look at the MFT configuration.

The steps for this process are:

Go to the Administration page.
In the left navigation pane, click Server Properties.
Look at the Callout Directory setting near the top of the page.

Running the createCallouts Command

After you have copied the JAR file and the definition file into the MFT callout directory, you must use the createCallouts command to upload the callout definition to the MFT configuration.

The steps for this process are:

Start WLST using the steps described in Running WLST Commands.
Specify the full path to the callout definition file using the createCallouts command:
```
crtCalls('/home/user_projects/domains/soainfra/mft/callouts/Newline.xml')
```
The example uses the crtCalls abbreviation.
Verify that the callout has been added using the listCallouts command:
```
listCallouts()
```
Exit WLST using the steps described in Running WLST Commands.

For more information about MFT WLST commands, see MFT WLST Command Summary and MFT Custom WLST Commands in WLST Command Reference for SOA Suite.

Testing the Callout

After you create the callout and upload its definition, you must test it to be sure it works as designed.

The high-level steps for testing a custom callout are:

Add the callout to a source or target and deploy the associated transfer. See Adding the Callout to a Source or Adding the Callout to a Target.
After the transfer runs, look at the source or target report. See Viewing the Report to Verify the Callout Action.
Update the callout if you need to correct errors or make improvements. See Updating the Callout.

Note:

If you were logged into the MFT console when you ran the createCallouts command, you must log out and log in again before you can add the callout to a source or target.

Adding the Callout to a Source

The steps for adding the Newline Conversion callout to a source are:

Click the arrow to the left of Sources in the left pane navigator.

The sources are listed.
Click the source name or right-click it and then select the Open menu item.

The source tab opens.
Click the arrow to the left of Actions.

The Actions section opens.
Click add processing actions.

The Processing Actions dialog opens.
Select New line conversion from the All Actions drop-down list.
Click Add to List.

To remove an action from the list, click the Delete icon to the right of the action.
Type Dos2Unix or Unix2Dos in the Type text box.
Click OK.

To cancel adding actions, click Cancel.
Save and optionally Deploy the source.
Add the source to a transfer and Deploy the transfer.

See Deploying and Testing Transfers for more information.

Adding the Callout to a Target

The steps for adding the Newline Conversion callout to a target are:

Click the arrow to the left of Transfers in the left pane navigator.

The transfers are listed.
Click the transfer name or right-click it and then select the Open menu item.

The transfer tab opens.
Click the arrow to the left of the target.

The target settings are displayed.
Click add preprocessing actions.

The Pre-Processing Actions dialog opens.
Select New line conversion from the All Actions drop-down list.
Click Add to List.

To remove an action from the list, click the Delete icon to the right of the action.
Type Dos2Unix or Unix2Dos in the Type text box.
Click OK.

To cancel adding actions, click Cancel.
Save and Deploy the transfer.

See Deploying and Testing Transfers for more information.

Viewing the Report to Verify the Callout Action

To access the report for the source or target, go to the Monitoring page and see Interpreting Source_ Transfer_ and Target Reports.

The New line conversion callout is listed in the Source Pre-Processing table in the source report or the Target Pre-Processing table in the target report. The Status column of either table shows the value Processed if the callout action was successful.

In addition, the Advanced section of the report includes payloadModified=yes in the Message Property field if the callout action was successful.

See Source Reports or Target Reports for more information.

Updating the Callout

Keep these points in mind when updating callouts:

No MFT server restart is needed when creating or updating callouts.
If you change only the Java code, and the parameters are unchanged, all you need to do is copy the new JAR file into the callouts directory.
If you change the definition file, and the parameters are unchanged, all you need to do is use the updateCallouts command to reload the callout definition.
If you add, delete, or modify callout parameters, you must perform these steps:
1. Delete the callout using the deleteCallout command.
2. Recreate the callout using the createCallouts command.
3. Reconfigure the source or target to use the new callout version.

For more information about MFT WLST commands, see MFT WLST Command Summary and MFT Callout Commands in WLST Command Reference for SOA Suite.

Reference Files

This section lists the contents of the following files:

PreCalloutPlugin Interface

Example - PreCalloutPlugin Interface shows the Java code for the com.oracle.callout.PreCalloutPlugin interface.

Example - PreCalloutPlugin Interface

 public interface PreCalloutPlugin {
 
    /**
     * Depending on the return of this function output stream for changing the
     * pay load content will be passed. Only if the payload change is required
     * return true.
     *
     * @param context
     * @param calloutParams
     * @return
     */
    public boolean isPayloadChangeRequired(final PluginContext context,
            final Map<String, String> calloutParams);
 
    /**
     * This function will be called only if the isPayloadChangesRequired returns
     * true. This method must write final pay load content to the output stream.
     * Any custom properties or the header properties can be set at the context
     * level.
     *
     * @param context
     * @param input
     * @param out
     * @param calloutParams
     * @return
     */
    public PluginOutput process(final PluginContext context,
            final InputStream input, OutputStream out,
            final Map<String, String> calloutParams);
 
    /**
     * This function will be called only if the isPayloadChangesRequired returns
     * false. Any custom properties or the header properties can be set at the
     * context level.
     *
     * @param context
     * @param input
     * @param out
     * @param calloutParams
     * @return
     */
    public PluginOutput process(final PluginContext context,
            final InputStream input, final Map<String, String> calloutParams);
}

PostCalloutPlugin Interface

Example - PostCalloutPlugin Interface shows the Java code for the com.oracle.callout.PostCalloutPlugin interface.

Example - PostCalloutPlugin Interface

public interface PostCalloutPlugin {
 
    /**
     * Any custom properties or the header properties can be set at the
     * context level.
     *
     * @param context
     * @param input
     * @param out
     * @param calloutParams
     * @return
     */
    public PluginOutput process(final PluginContext context,
            final InputStream input, final Map<String, String> calloutParams);
}

Callout Definition Schema

The callout.xsd file defines the format of the callout definition XML file. Example - Callout Definition Schema shows its contents.

Example - Callout Definition Schema

<?xml version="1.0" encoding="UTF-8"?>
<xsd:schema targetNamespace="http://xmlns.oracle.com/mft"
    xmlns="http://www.w3.org/2001/XMLSchema"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:mft="http://xmlns.oracle.com/mft"
    elementFormDefault="qualified">
 
<xsd:element name="Callouts">
    <xsd:complexType>
        <xsd:sequence>
            <xsd:element name="Callout" minOccurs="0" maxOccurs="unbounded">
                <xsd:complexType>
                    <xsd:sequence>
                        <xsd:element minOccurs="0" maxOccurs="unbounded" ref="mft:Parameter"/>
                    </xsd:sequence>
                    <xsd:attribute name="name" use="required">
                        <xsd:simpleType>
                            <xsd:restriction base="mft:non-empty-string">
                                <xsd:maxLength value="128"/>
                            </xsd:restriction>
                        </xsd:simpleType>
                    </xsd:attribute>
                    <!-- multiple values separated by comma are supported, possible values
                         Source-pre, Target-pre, Target-post -->
                    <xsd:attribute name="groupName" type="mft:non-empty-string" use="optional"/>
                    <xsd:attribute name="implementationClass" type="mft:non-empty-string"/>
                    <!-- jar name -->
                    <xsd:attribute name="libraryName" type="mft:non-empty-string"/>
                    <xsd:attribute name="timeout" type="xsd:integer" use="optional"
                               default="30"/>
                    <xsd:attribute name="description" type="mft:non-empty-string"/>
                    <xsd:attribute name="helpText" type="mft:non-empty-string"/>
                </xsd:complexType>
                <xsd:unique name="Callout_UK">
                    <xsd:selector xpath="mft:Callout"/>
                    <xsd:field xpath="@name"/>
                </xsd:unique>
            </xsd:element>
        </xsd:sequence>
    </xsd:complexType>
</xsd:element>
        
<xsd:element name="Parameter">
    <xsd:complexType>
      <xsd:attribute name="name" type="mft:non-empty-string" use="required"/>
      <xsd:attribute name="parameterType" type="mft:parameterTypeLOV"
                 use="required"/>
      <xsd:attribute name="basic" type="boolean" use="optional" default="false"/>
      <xsd:attribute name="mandatory" type="boolean" use="optional"
                 default="false"/>
      <xsd:attribute name="hidden" type="boolean" use="optional" default="false"/>
      <!-- multiple values separated by comma is supported -->
      <xsd:attribute name="groupName" type="mft:non-empty-string"
                 use="optional"/>
      <!-- sub-groups can have multiple values separated by comma. -->
      <xsd:attribute name="sub-groups" type="mft:non-empty-string"
                 use="optional"/>
      <!-- VRule: defaultValue and parameterTypeLOV MUST match -->
      <xsd:attribute name="defaultValue" type="mft:non-empty-string"
                 use="optional"/>
      <xsd:attribute name="listValue" type="mft:non-empty-string"
                 use="optional"/>
      <xsd:attribute name="mapValue" type="mft:non-empty-string"
                 use="optional"/>
      <!-- changed refValue from IDREF to non-empty-string as IDREF does not allow ",". In the
      future refValue can have value separated by ",". Ex: "xPathaParams, someOtherParams" -->
      <xsd:attribute name="refValue" type="mft:non-empty-string" use="optional"/>
      <!--attribute name="tpOverrideable" type="boolean" use="optional" default="true"/-->
      <xsd:attribute name="overrideLevel" type="mft:overrideLevelLOV"
                 use="optional" default="all"/>
      <xsd:attribute name="displayName" type="mft:non-empty-string"/>
      <xsd:attribute name="description" type="mft:non-empty-string"/>
      <xsd:attribute name="helpText" type="mft:non-empty-string"/>
    </xsd:complexType>
  </xsd:element>
 
<xsd:simpleType name="overrideLevelLOV">
    <xsd:restriction base="string">
      <xsd:enumeration value="admin"/>
      <xsd:enumeration value="tp"/>
      <xsd:enumeration value="host"/>
      <xsd:enumeration value="tpa"/>
      <xsd:enumeration value="all"/>
    </xsd:restriction>
</xsd:simpleType>
 
<xsd:simpleType name="non-empty-string">
    <xsd:restriction base="string">
      <xsd:minLength value="1"/>
    </xsd:restriction>
</xsd:simpleType>
 
<xsd:simpleType name="parameterTypeLOV">
    <xsd:restriction base="string">
      <xsd:enumeration value="float"/>
      <xsd:enumeration value="integer"/>
      <xsd:enumeration value="string"/>
      <xsd:enumeration value="boolean"/>
      <xsd:enumeration value="date"/>
      <xsd:enumeration value="SO"/>
      <xsd:enumeration value="MO"/>
      <xsd:enumeration value="credential"/>
      <xsd:enumeration value="list"/>
      <xsd:enumeration value="map"/>
      <xsd:enumeration value="ref"/>
      <xsd:enumeration value="hex"/>
    </xsd:restriction>
</xsd:simpleType>
</xsd:schema>

Validating using Custom Callout

You can validate a custom callout action. For example, when validating checksum, you can validate by setting the exception object in the plugin output in the callout implementation.

The code for validation is as follows:

public PluginOutput process(PluginContext context, InputStream input,
Map<String, String> calloutParams) { 
PluginOutput res = new PluginOutput() ; 
boolean isValid = validateChecksum(context, input, calloutParams) ; 
if( ! isValid ){res.setException(new Exception("Checksum mismatch, corrupted payload.")) ; 

} 
return res ; 
}

The checksum action status for the above example will be failed for the instance report page. The message status will be Failed[Pre-processing].