16 Using Web Services

This chapter describes how to work with Web services in Oracle Data Integrator.

This chapter includes the following sections:

Introduction to Web Services in Oracle Data Integrator
Oracle Data Integrator Run-Time Services and Data Services
Invoking Third-Party Web Services

Introduction to Web Services in Oracle Data Integrator

Oracle Data Integrator provides the following entry points into a service-oriented architecture (SOA):

Data services
Oracle Data Integrator run-time services
Invoking third-party Web services

Figure 16-1 shows an overview of how the different types of Web services can interact.

Figure 16-1 Web Services in Action

Description of "Figure 16-1 Web Services in Action"

Figure 16-1 shows a simple example with the Data Services, Run-Time Web services (Public Web service and Agent Web service) and the OdiInvokeWebService tool.

The Data Services and Run-Time Web services components are invoked by a third-party application, whereas the OdiInvokeWebService tool invokes a third-party Web service:

The Data Services provides access to data in data stores (both source and target data stores), as well as changes trapped by the Changed Data Capture framework. This Web service is generated by Oracle Data Integrator and deployed in a Java EE application server.
The Agent Web service commands the Oracle Data Integrator Agent to: start and monitor a scenario; restart a session; get the ODI version; start, stop or restart a load plan; or refresh agent configuration. Note that this Web service is built in the Java EE or Standalone Agent.
The OdiInvokeWebService tool is used in a package and invokes a specific operation on a port of the third-party Web service, for example to trigger a BPEL process.

Oracle Data Integrator Run-Time Services and Data Services

Oracle Data Integrator Run-Time Web services and Data Services are two different types of Web services:

Oracle Data Integrator Run-Time Services (Agent Web service) are Web services that enable users to leverage Oracle Data Integrator features in a service-oriented architecture (SOA).

Oracle Data Integrator Run-Time Web services enable you to access the Oracle Data Integrator features through Web services. These Web services are invoked by a third-party application and manage execution of runtime artifacts developed with Oracle Data Integrator.

How to perform the different ODI execution tasks with the ODI Run-Time Services, such as executing a scenario and restarting a session, is detailed in "Managing Executions Using Web Services". That topic also provides examples of SOAP requests and responses.
Data Services are specialized Web services that provide access to data in datastores, and to changes captured for these datastores using Changed Data Capture. Data Services are generated by Oracle Data Integrator to give you access to your data through Web services. These Web services are deployed to a Web services container in an application server.

For more information on how to set up, generate and deploy Data Services refer to Chapter 8, "Creating and Using Data Services."

Invoking Third-Party Web Services

This section describes how to invoke third-party Web services in Oracle Data Integrator.

This section includes the following topics:

Introduction to Web Service Invocation
Using HTTP Analyzer
Using the OdiInvokeWebService Tool

Introduction to Web Service Invocation

You can invoke Web services:

In Oracle Data Integrator packages or procedures using the HTTP Analyzer tool. This tool allows you to invoke any third party Web service, and save the response in a SOAP file that can be processed with Oracle Data Integrator.

You can use the results to debug a locally or remotely deployed Web service.
For testing Data Services. The easiest way to test whether your generated data services are running correctly is to use the HTTP Analyzer tool.
In Oracle Data Integrator packages or procedures using the OdiInvokeWebService tool. This tool allows you to invoke any third party Web service, and save the response in an XML file that can be processed with Oracle Data Integrator.

The three approaches are described in the sections that follow.

Using HTTP Analyzer

The HTTP Analyzer allows you to monitor request/response traffic between a Web service client and the service. The HTTP Analyzer helps you to debug your Web service in terms of the HTTP traffic sent and received.

When you run the HTTP Analyzer, there are a number of windows that provide information for you.

The HTTP Analyzer enables you to:

Observe the exact content of the request and response TCP packets of your Web service.
Edit a request packet, re-send the packet, and see the contents of the response packet.
Test Web services that are secured using policies; encrypted messages will be decrypted.

This section describes the following topics:

Using HTTP Analyzer: Main Steps
What Happens When You Run the HTTP Analyzer
How to Specify HTTP Analyzer Settings
How to Use the Log Window
How to Use the Test Window
How to Use the Instances Window
How to Use Multiple Instances
Using Credentials With HTTP Analyzer
Using SSL With HTTP Analyzer
How to Debug Web Pages Using the HTTP Analyzer
How to Use Rules to Determine Behavior
How to Set Rules
Reference: Troubleshooting the HTTP Analyzer

Using HTTP Analyzer: Main Steps

To examine the packets sent and received by the client to a Web service:

Note:

In order to use the HTTP Analyzer, you may need to update the proxy settings.

Create and run the Web service.
Start the HTTP Analyzer by selecting Tools > HTTP Analyzer.

You can also start it from the HTTP Analyzer button on the General tab of the OdiInvokeWebService tool step.

It opens in its own window.
Click the Create New Soap Request button in the HTTP Analyzer Log window.
Enter the URL of a Web service, or open the WSDL for a Web service, to get started.
Run the client proxy to the Web service. The request/response packet pairs are listed in the HTTP Analyzer Test window.

The test window allows you examine the headers and parameters of a message. You can test the service by entering a parameter that is appropriate and clicking Send Request.
You can examine the contents of the HTTP headers of the request and response packets to see the SOAP structure (for JAX-WS Web services), the HTTP content, the Hex content or the raw message contents by choosing the appropriate tab at the bottom of the HTTP Analyzer Test window.

Note:

The WADL structure (for RESTful services) is not supported by Oracle Data Integrator.
You can test Web services that are secured using policies by performing one of the following tasks:
- Select an existing credential from the Credentials list.
  
  Oracle Data Integrator delivers with a set of preconfigured credentials, HTTPS Credential.
- Click New to create a new credential. In the Credential dialog, define the credentials to use in the HTTP Analyzer Test window.

What Happens When You Run the HTTP Analyzer

When you start the HTTP Analyzer and test a Web service, the Web service sends its traffic via the HTTP Analyzer, using the proxy settings in the HTTP Analyzer Preferences dialog.

By default, the HTTP Analyzer uses a single proxy on an analyzer instance (the default is 8099), but you can add additional proxies of your own if you need to.

Each analyzer instance can have a set of rules to determine behavior, for example, to redirect requests to a different host/URL, or to emulate a Web service.

How to Specify HTTP Analyzer Settings

By default, the HTTP Analyzer uses a single proxy on an analyzer instance (the default is 8099), but you can add additional proxies of your own if you need to.

To set HTTP Analyzer preferences:

Open the HTTP Analyzer preferences dialog by doing one of the following:
- Click the Start HTTP Analyzer button in the HTTP Analyzer Instances window or Log window.
- Choose Tools > Preferences to open the Preferences dialog, and navigating to the HTTP Analyzer page.
For more information at any time, press F1 or click Help from the HTTP Analyzer preferences dialog.
Make the changes you want to the HTTP Analyzer instance. For example, to use a different host and port number, open the Proxy Settings dialog by clicking Configure Proxy.

How to Use the Log Window

When you open the HTTP Analyzer from the Tools menu, the HTTP Analyzer Log window opens, illustrated in Figure 16-2.

Figure 16-2 HTTP Analyzer Log Screen

Description of "Figure 16-2 HTTP Analyzer Log Screen"

When HTTP Analyzer runs, it outputs request/response messages to the HTTP Analyzer log window. You can group and reorder the messages:

To reorder the messages, select the Sequence tab, then sort using the column headers (click on the header to sort, double-click to secondary sort).
To group messages, click the Correlation tab.
To change the order of columns, grab the column header and drag it to its new position.

Table 16-1 HTTP Analyzer Log Window Toolbar Icons

Icon	Name	Function
	Analyzer Preferences	Click to open the HTTP Analyzer Preferences dialog where you can specify a new listener port, or change the default proxy. An alternative way to open this dialog is to choose Tools > Preferences, and then navigate to the HTTP Analyzer page. For more information, see
	Create New Request	Click to open the HTTP Analyzer Test window, where you enter payload details, and edit and resend messages.
	Start HTTP Analyzer	Click to start the HTTP Analyzer running. The monitor runs in the background, and only stops when you click Stop or exit JDeveloper. If you have more than one listener defined clicking this button starts them all. To start just one listener, click the down arrow and select the listener to start.
	Stop HTTP Analyzer	Click to stop the HTTP Analyzer running. If you have more than one listener running, clicking this button stops them all. To stop just one listener click the down arrow and select the listener to stop.
	Send Request	Click to resend a request when you have changed the content of a request. The changed request is sent and you can see any changes in the response that is returned.
	Open WS-I log file	Click to open the Select WS-I Log File to Upload dialog, where you can navigate to an existing WS-I log file.
	Save Packet Data	Click to save the contents of the HTTP Analyzer Log Window to a file.
	WS-I Analyze	This tool does not apply to Oracle Data Integrator.
	Select All	Click to select all the entries in the HTTP Analyzer Log Window.
	Deselect All	Click to deselect all the entries in the HTTP Analyzer.
	Clear Selected History (Delete)	Click to clear the entries in the HTTP Analyzer.

How to Use the Test Window

An empty HTTP Analyzer test window appears when you click the Create New Soap Request button in the HTTP Analyzer Log window.

Enter the URL of a Web service, or open the WSDL for a Web service, and then click Send Request. The results of the request are displayed in the test window, as shown in Figure 16-3.

Figure 16-3 HTTP Analyzer Test Window

Description of "Figure 16-3 HTTP Analyzer Test Window"

You can examine the contents of the HTTP headers of the request and response packets to see the SOAP structure, the HTTP content, the Hex content or the raw message contents by choosing the appropriate tab at the bottom of the HTTP Analyzer Test window.

The test window allows you examine the headers and parameters of a message. You can test the service by entering a parameter that is appropriate and clicking Send Request.

The tabs along the bottom of the test window allow you choose how you see the content of the message. You can choose to see the message as:

The SOAP structure, illustrated in Figure 16-3.

The HTTP code, for example:

<?xml version = '1.0' encoding = 'UTF-8'?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.example.com/wls">
   <env:Header/>
   <env:Body>
      <ns1:sayHello>
         <arg0/>
      </ns1:sayHello>
   </env:Body>

The hex content of the message, for example:

[000..015] 3C 65 65 20 78 6D ...   <env:Envelope xm
[016..031] 6C 6E 70 3A 2F 2F ...   lns:env="http://
[032..047] 73 63 6F 61 70 2E ...   schemas.xmlsoap.
[048..063] 6F 72 65 6C 6F 70 ...   org/soap/envelop
[064..079] 65 2F 22 20 78 6D ...   e/" xmlns:ns1="h
[080..095] 74 74 70 3A 2F 2F ...   ttp://www.bea.co
[096..111] 6D 2F 77 6C 73 22 ...   m/wls"><env:Head
[112..127] 65 72 2F 3E 3C 65 ...   er/><env:Body><n
[128..143] 73 31 3A 73 61 79 ...   s1:sayHello><arg
[144..159] 30 3E 3C 2F 61 72 ...   0></arg0></ns1:s
[160..175] 61 79 48 65 6C 6C ...   ayHello></env:Bo
[176..191] 64 79 3E 3C 2F 65 ...   dy></env:Envelop
[192..193] 65 3E             ...   e>

The raw message, for example:

POST http://localhost:7001/MySimpleEjb/MySimpleEjbService HTTP/1.1
Content-Type: text/xml; charset=UTF-8
SOAPAction: ""
Host: localhost:7001
Content-Length: 194
X-HTTPAnalyzer-Rules: 3@localhost:8099
 
<?xml version = '1.0' encoding = 'UTF-8'?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.example.com/wls">
   <env:Header/>
   <env:Body>
      <ns1:sayHello>
         <arg0/>
      </ns1:sayHello>
   </env:Body>
</env:Envelope>

How to Use the Instances Window

When you open the HTTP Analyzer from the Tools menu, the HTTP Analyzer tab appears by default.

Click the HTTP Analyzer Instances tab. The HTTP Analyzer Instances window appears, as shown in Figure 16-4.

This window provides information about the instances of the HTTP Analyzer that are currently running, or that were running and have been stopped. The instance is identified by the host and port, and any rules are identified. You can start and stop the instance from this window.

Figure 16-4 HTTP Analyzer Instances Window

Description of "Figure 16-4 HTTP Analyzer Instances Window"

You create a new instance in the HTTP Analyzer dialog, which opens when you click the Create New Soap Request button.

Table 16-2 HTTP Analyzer Instances Window Toolbar Icons

Icon	Name	Function
	Analyzer Preferences	Click to open the HTTP Analyzer dialog where you can specify a new listener port, or change the default proxy.
	Create New Request	Click to open a new instance of the HTTP Analyzer Test window, where you enter payload details, and edit and resend messages.
	Start HTTP Analyzer	Click to start the HTTP Analyzer running. The monitor runs in the background, and only stops when you click Stop or exit JDeveloper. If you have more than one listener defined clicking this button starts them all. To start just one listener, click the down arrow and select the listener to start.
	Stop HTTP Analyzer	Click to stop the HTTP Analyzer running. If you have more than one listener running, clicking this button stops them all. To stop just one listener click the down arrow and select the listener to stop.

How to Use Multiple Instances

You can have more than one instance of HTTP Analyzer running. Each will use a different host and port combination, and you can see a summary of them in the HTTP Analyzer Instances window.

To add an additional HTTP Analyzer Instance:

Open the HTTP Analyzer preferences dialog by doing one of the following:
- Click the Analyzer Preferences button in the HTTP Analyzer Instances window or Log window.
- Choose Tools > Preferences to open the Preferences dialog, and navigating to the HTTP Analyzer page.
For more information at any time, press F1 or click Help from the HTTP Analyzer preferences dialog.
To create a new HTTP Analyzer instance, that is a new listener, click Add. The new listener is listed and selected by default for you to change any of the values.

Using Credentials With HTTP Analyzer

You can use the HTTP Analyzer to test Web services that are secured using policies. You choose the credentials to use in the HTTP Analyzer Test window.

HTTP Analyzer supports the following credentials for this purpose:

HTTPS. The message is encrypted prior to transmission using a public key certificate that is signed by a trusted certificate authority. The message is decrypted on arrival.
Username token. This token does not apply to Oracle Data Integrator.

This is a way of carrying basic authentication information using a token based on username/password.
X509. This token does not apply to Oracle Data Integrator.

This is a PKI standard for single sign-on authentication, where certificates are used to provide identity, and to sign and encrypt messages.
STS. This token does not apply to Oracle Data Integrator.

Security Token Service (STS) is a Web service which issues and manages security tokens.

Using SSL With HTTP Analyzer

You can use the HTTP Analyzer with secured services or applications, for example, Web services secured by policies. Oracle Data Integrator includes a credential, HTTPS Credential, for this purpose.

Once you have configured the credentials, you can choose which to use in the HTTP Analyzer Test window.

HTTPS encrypts an HTTP message prior to transmission and decrypts it upon arrival. It uses a public key certificate signed by a trusted certificate authority. When the integrated application server is first started, it generates a DemoIdentity that is unique, and the key in it is used to set up the HTTPS channel.

For more information about keystores and keystore providers, see Understanding Security for Oracle WebLogic Server. When the default credential HTTPS Credential is selected, you need to specify the keystores that the HTTP Analyzer should use when handling HTTPS traffic.

Two keystores are required to run the HTTP Analyzer:

The "Client Trusted Certificate Keystore," containing the certificates of all the hosts to be trusted by the Analyzer (client trust) when it makes onward connections. The server's certificate must be in this keystore.

The "Client Keystore" is required only when mutual authentication is required.
The "Server Keystore," containing a key that the Analyzer can use to authenticate itself to calling clients.

To configure the HTTP Analyzer to use different HTTPS values:

From the main menu, choose Tools > Preferences.
In the Preferences dialog, select the Credentials node. For more information, press F1 or click Help from within the dialog page.
Enter the new keystore and certificate details you want to use.

How to Debug Web Pages Using the HTTP Analyzer

You can use the HTTP Analyzer when you are debugging Web pages, such as HTML, JSP, or JSF pages. This allows you to directly examine the traffic that is sent back and forth to the browser.

To debug Web pages using the HTTP Analyzer:

Configure a browser to route messages through the HTTP Analyzer so that you can see the traffic between the web browser and client.
Start the HTTP Analyzer running.
Run the class, application, or Web page that you want to analyze in the usual way.

Each request and response packet is listed in the HTTP Analyzer Log window, and detailed in the HTTP Analyzer Test Window.

How to Use Rules to Determine Behavior

You can set rules so that the HTTP Analyzer runs using behavior determined by those rules. You can set more than one rule in an HTTP Analyzer instance. If a service's URL matches a rule, the rule is applied. If not, the next rule in the list is checked. If the service does not match any of the rules the client returns an error. For this reason, you should always use a Pass Through rule with a blank filter (which just passes the request through) as the last rule in a list to catch any messages not caught by the preceding rules.

The types of rule available are:

Pass Through Rule
Forward Rule
URL Substitution Rule
Tape Rule

Using the Pass Through Rule

The Pass Through simply passes a request on to the service if the URL filter matches. When you first open the Rule Settings dialog, two Pass Through Rules are defined:

The first has a URL filter of http://localhost:631 to ignore print service requests.
The second has a blank URL filter, and it just which just passes the request to the original service. This rule should normally be moved to end of the list if new rules are added.

Using the Forward Rule

The Forward rule is used to intercept all URLs matched by the filter and it forwards the request on to a single URL.

Using the URL Substitution Rule

The URL Substitution rule allows you to re-host services by replacing parts of URL ranges. For example, you can replace the machine name when moving between the integrated application server and Oracle WebLogic Server.

Using the Tape Rule

The tape rule allows you to run the HTTP Analyzer in simulator mode, where a standard WS-I log file is the input to the rule. When you set up a tape rule, there are powerful options that you can use:

Loop Tape, which allows you to run the tape again and again.
Skip to matching URL and method, which only returns if it finds a matching URL and HTTP request method. This means that you can have a WSDL and an endpoint request in the same tape rule.
Correct header date and Correct Content Size, which allow you change the header date and content size of the message to current values so that the request does not fail.

An example of using a tape rule would be to test a Web service client developed to run against an external Web service.

To test a Web service client developed to run against an external Web service:

Create the client to the external Web service.
Run the client against the Web service with the HTTP Analyzer running, and save the results as a WS-I log file.

You can edit the WS-I file to change the values returned to the client.
In the HTTP Analyzer page of the Preferences dialog, create a tape rule.

Ensure that it is above the blank Pass Through rule in the list of rules.
In the Rule Settings dialog, use the path of the WS-I file as the Tape path in the Rule Settings dialog.

When you rerun the client, it runs against the entries in the WS-I file instead of against the external Web service.

There are other options that allow you to:
- Correct the time and size of the entries in the WS-I log file so the message returned to the client is correct.
- Loop the tape so that it runs more than once.
- Skip to a matching URL and HTTP request method, so that you can have a WSDL and an endpoint request in the same tape rule.

Note:

Tape Rules will not work with SOAP messages that use credentials or headers with expiry dates in them.

How to Set Rules

You can set rules so that the HTTP Analyzer runs using behavior determined by those rules. Each analyzer instance can have a set of rules to determine behavior, for example, to redirect requests to a different host/URL, or to emulate a Web service.

To set rules for an HTTP Analyzer instance:

Open the HTTP Analyzer by choosing Tools > HTTP Analyzer. The HTTP Analyzer docked window opens.

Alternatively, the HTTP Analyzer automatically opens when you choose Test Web Service from the context menu of a Web service container in the Applications window.
Click the Analyzer Preferences button to open the HTTP Analyzer preferences dialog, in which you can specify a new listener port, or change the default proxy.

Alternatively, choose Tools > Preferences, and then navigate to the HTTP Analyzer page.
Click Configure Rules to open the Rule Settings dialog in which you define rules to determine the actions the HTTP Analyzer should take. For more help at any time, press F1 or click Help in the Rule Settings dialog.
In the Rule Settings dialog, enter the URL of the reference service you want to test against as the Reference URL. This will help you when you start creating rules, as you will be able to see if and how the rule will be applied.
Define one or more rules for the service to run the client against. To add a new rule, click the down arrow next to Add, and choose the type of rule from the list. The fields in the dialog depend on the type of rule that is currently selected.
The rules are applied in order from top to bottom. Reorder them using the up and down reorder buttons. It is important that the last rule is a blank Pass Through rule.

Reference: Troubleshooting the HTTP Analyzer

This section contains information to help resolve problems that you may have when running the HTTP Analyzer.

Running the HTTP Analyzer While Another Application is Running

If you have an application waiting for a response, do not start or stop the HTTP Analyzer. Terminate the application before starting or stopping the HTTP Analyzer.

The HTTP Analyzer can use one or more different sets of proxy settings. These settings are specific to the IDE only. If enabled, Oracle Data Integrator uses these settings to access the Internet through your organization proxy server. If you do not enable the proxy server setting, then your Web application may not be able to access the Internet. Proxy server settings are visible in the preferences settings for your machine's default browser.

When you run the HTTP Analyzer, it can use one or more different sets of proxy settings. These proxy settings override the HTTP Proxy Server settings when the HTTP Analyzer is running.

Changing Proxy Settings

When you use the HTTP Analyzer, you may need to change the proxy settings in Oracle Data Integrator. For example:

If you are testing an external service and your machine is behind a firewall, ensure that Oracle Data Integrator is using the HTTP proxy server.
If you are testing a service in the integrated application server, for example when you choose Test Web Service from the context menu of a Web service in the Applications window, ensure that Oracle Data Integrator is not using the HTTP proxy server.

If you run the HTTP Analyzer, and see the message

500 Server Error
The following error occurred: [code=CANT_CONNECT_LOOPBACK] Cannot connect due to potential loopback problems

you probably need to add localhost|127.0.0.1 to the proxy exclusion list.

To set the HTTP proxy server and edit the exception list:

Choose Tools > Preferences, and select Web Browser/Proxy.
Ensure that Use HTTP Proxy Server is selected or deselected as appropriate.
Add any appropriate values to the Exceptions list, using | as the separator.

In order for Java to use localhost as the proxy ~localhost must be in the Exceptions list, even if it is the only entry.

Using the OdiInvokeWebService Tool

The OdiInvokeWebService tool invokes a Web service using the HTTP or HTTPS protocol and is able to write the returned response to an XML file, which can be an XML payload or a full-formed SOAP message including a SOAP header and body.

You can configure OdiInvokeWebService tool parameters using Http Analyzer. To do this, click the Http Analyzer button on the General tab of the OdiInvokeWebService step in the package editor. This opens OdiInvokeWebServiceAdvance, which you can use to configure command parameters.

See "OdiInvokeWebService" for details on the OdiInvokeWebService tool parameters.

The OdiInvokeWebService tool invokes a specific operation on a port of a Web service whose description file (WSDL) URL is provided. If this operation requires a SOAP request, it is provided either in a request file or in the tool command. The response of the Web service request is written to an XML file that can be used in Oracle Data Integrator.

Note:

When using the XML payload format, the OdiInvokeWebService tool does not support the SOAP headers of the request. In order to work with SOAP headers, for example for secured Web service invocation, use a full SOAP message and manually modify the SOAP headers.

This tool can be used as a regular Oracle Data Integrator tool in a tool step of a package and also in procedures and knowledge modules. See "Adding Oracle Data Integrator Tool Steps" for information on how to create a tool step in a package.

You can process the information from your responses using regular Oracle Data Integrator interfaces sourcing for the XML technology. Refer to the Connectivity and Modules Guide for Oracle Data Integrator for more information on XML file processing.

Note:

Each XML file is defined as a model in Oracle Data Integrator. When using XML file processing for the request or response file, a model will be created for each request or response file. It is recommended to use model folders to arrange them. See "Organizing Models with Folders" for more information.

Oracle Data Integrator provides the OdiXMLConcat and OdiXMLSplit tools for processing the Web service response. Refer to the XML section of "Oracle Data Integrator Tools by Category" for details on how to use these tools.

Using the Binding Mechanism for Requests

It is possible to use the Binding mechanism when using a Web service call in a Procedure. With this method, it is possible to call a Web service for each row returned by a query, parameterizing the request based on the row's values. Refer to "Binding Source and Target Data" for more information.