Client Application Developer's Guide

Advanced Topics

This chapter provides information on miscellaneous topics related to client programming with BEA AquaLogic Data Services Platform (DSP). It covers the following topics:

Using Catalog Services to Obtain Data Services' Metadata

BEA AquaLogic Data Services Platform (DSP) maintains metadata about all data services through a system catalog-type data service, known as Catalog Services. Catalog Services are available to client application developers to use in the same way they use any other data service in DSP.

Catalog services provide a convenient way for client-application developers to programmatically obtain information about the data services that are running on the server. The primary benefit of Catalog Services to developers is that they can create dynamic applications based on the metadata underlying the data service applications that have been deployed. Enterprise, third-party, and other developers who want to build dynamic, metadata driven query-by-form (QBF) applications can leverage DSP's Catalog Services to do just that. In addition, Catalog Services enables interoperability with other metadata repositories.

By querying Catalog Services, developers can obtain all the information they need about data services. For example, you can obtain information about:

To develop a metadata-driven application, developers can use the client Mediator API and invoke the Catalog Service's methods (see Listing 10-2) as needed populate the page they present to users of their application, for example.

Since the Catalog Services are data services, just as with any other data service you can also view these data services in three other ways, specifically through the:

However, given the typical use case for the catalog services—metadata driven QBF applications—it is far more likely that application developers will invoke Catalog Services methods by using the Mediator API.

Generally speaking, to create a QBF application your code can leverage Catalog Services as follows:

1) Call Folder.getFolder()

2) Select a data service

3) Call DataService.getDataServiceById( dsId )

4) [optional] call Schema.getSchemaByDataServiceRef( ds.getRef( ) )

5) Select a function from the selected dataservice

6) Provide a form to enter the arguments. The more complex the arguments, the more complex the code you must write.

Obtain a schema for parameters using the Catalog I think you can get the schema for parameters from Catalog Services - see Test View code on how to generate a `template'

Installing Catalog Services

The ability to build Catalog Services within any DSP-enabled application is available by default when you install the product. DSP Catalog Services are installed easily on a per-application basis, by selecting Install Catalog Services from the WebLogic Workshop main menu.

Installing the Catalog Services creates a _metadata.jar file in the application's Library folder, and creates the various CLASS files that provide the typed accessors (see Table 10-2).

After installing Catalog Services, you will have access to all application metadata. For any DSP-enabled application that you want to leverage in this way, simply install the Catalog Services into the application.

In addition to the methods shown in Table 10-2, you will also see several extraneous methods that define relationships among data services. However, the methods shown are the only methods you need to develop a metadata-driven client application.

Creating a Query-by-Form (QBF) Application Using Catalog Services

You can create a Query-by-Form (QBF) application using DSP's Catalog Services and the Mediator APis. Your application can leverage the Catalog Service in the same way you might leverage any data service using the Mediator APIs.

Filtering, Sorting, and Fine-tuning Query Results

The Filter API enables client applications to apply filtering conditions to the information returned by data service functions. In a sense, filtering allows client applications to extend a data service interface by allowing them to specify more about how data objects are to be instantiated and returned by functions.

The Filter API alleviates data service designers from having to anticipate every possible data view that their clients may require and to implement a data service function for each view. Instead, the designer may choose to specify a broader, more generic interface for accessing a business entity and allow client applications to control views as desired through filters.

Only objects in the function return set that meet the condition are returned to the client. (The evaluation occurs at the server, so objects that are filtered are not passed over the network. Often, objects that are filtered out are not even retrieved from the underlying sources.) A filter is similar to a WHERE clause in an XQuery or SQL statement—it applies conditions to a possible result set. You can apply multiple filter conditions using AND and OR operators. Other operators that be applied to filter conditions are listed in Table 10-3.

Using Filters

Filtering capabilities are available to Mediator and Data Service control client applications. You use filter conditions to specify the data you want returned, sort the data, or limit the number of records returned. To use filters in a mediator client application, import the appropriate package and use the supplied interfaces for creating and applying filter conditions. Data service control clients get the interface automatically. When a function is added to a control, a corresponding "WithFilter" function is added as well.

The filter package is named as follows:

	com.bea.ld.filter.FilterXQuery; 

To use a filter, perform the following steps:

Specifying Filter Effects

If a filter condition applied to a specified element value resolves to false, an element is not included in the result set. The element that is filtered out is specified as the first argument to the addFilter() function.

The effects of a filter can vary, depending on the desired results. For example, consider the CUSTOMERS data object shown in Figure 10-1. It contains several complex elements (CUSTOMER and ORDERS) and several simple elements, including ORDER_AMOUNT. You can apply a filter to any elements in this hierarchy.

In general, with nested XML data, a condition such as "CUSTOMER/ORDER/ORDER_AMOUNT > 1000" can affect what objects are returned in several ways. For example, it can cause all CUSTOMER objects to be returned, but filter ORDERS that have an amount less than 1000.

Alternatively, it can cause only CUSTOMER objects to be returned that have at least one large order, and all ORDER objects are returned for every CUSTOMER. Further, it can cause only CUSTOMER objects to be returned for which every ORDER is greater than 1000. For example,

	XQueryFilter myFilter = new XQueryFilter();
	myFilter.addFilter(			"CUSTOMERS/CUSTOMER",
				"CUSTOMERS/CUSTOMER/ORDER/ORDER_AMOUNT",
				FilterXQuery.GREATER_THAN,"1000",true);

Note that in the optional fourth parameter everyChild = true, by default this attribute is false. By setting this parameter to true, only those CUSTOMER objects for which every ORDER is greater than 1000 will be returned.

The following examples show how filters can be applied in several different ways:

The last example is a compound filter; that is, a filter with two conditions. Listing 10-1 uses the AND operator to apply a combination of filters to a result set, given a data service instance customerDS.

FilterXQuery myFilter = new FilterXQuery();
Filter f1 = myFilter.createFilter("CUSTOMER_PROFILE/ADDRESS/ISDEFAULT",
		 			FilterXQuery.NOT_EQUAL,"0");
Filter f2 = myFilter.createFilter("CUSTOMER/ADDRESS/STATUS",
					FilterXQuery.EQUAL,
					"\"ACTIVE\"");
Filter f3 = myFilter.createFilter(f1,f2, FilterXQuery.AND);
Customer customerDS = Customer.getInstance(ctx, "RTLApp");
CustomerDS.setFilterCondition(myFilter);

Ordering and Truncating Data Service Results

Another type of filter you can use in client application code is an ordering condition—you specify the order (descending, ascending) in which results should be returned from the data service. The method (addOrderBy(), in the FilterXQuery class), takes a property name as the criterion upon which the ascending or descending decision is based. Listing 10-2 provides an example of creating a filter that will return customer profiles in ascending order, based on the date each person became a customer.

FilterXQuery myFilter = new FilterXQuery();
myFilter.addOrderBy("CUSTOMER_PROFILE",
			"CustomerSince" ,FilterXQuery.ASCENDING);
ds.setFilterCondition(myFilter);
DataObject objArrayOfCust = (DataObject) ds.invoke("getCustomer", null);

Similarly, you can set the maximum number of results that can be returned from a function. The setLimit() function limits the number of elements in an array element to the specified number. And on a repeating node, it makes sense to specify a limit on the results to be returned. (Setting the limits on non-repeating nodes does not truncate the results.)

Listing 10-3 shows how to use the setLimit() method. It limits the number of active address in the result set (filtering out active addresses) to 10 given a data service instance ds.

FilterXQuery myFilter = new FilterXQuery();
Filter f2 = myFilter.createFilter("CUSTOMER_PROFILE/ADDRESS",
                                   FilterXQuery.EQUAL,"\"INACTIVE\"");
myFilter.addFilter("CUSTOMER_PROFILE", f2);
myFilter.setLimit("CUSTOMER_PROFILE", "10");
ds.setFilterCondition(myFilter);

Using Ad Hoc Queries to Fine-tune Results from the Client

An ad hoc query is an XQuery function that is not defined as part of a data service, but is instead defined in the context of a client application. Ad hoc queries are typically used in client applications to invoke data service functions and refine the results in some way. You can use an ad hoc query to execute any valid XQuery expression against a data service. The expression can target the actual data sources that underlie the data service, or can use the functions and procedures hosted by the data service.

To execute an XQuery expression, use the PreparedExpression interface, available in the Mediator API. Similar to JDBC's PreparedStatement interface, the PreparedExpression interface takes the XQuery expression as a string in its constructor, along with the JNDI server context and application name. After constructing the prepared expression object in this way, you can call the executeQuery() method on it. If the ad hoc query invokes data service functions or procedures, the data service's namespace must be imported into query string before you can reference the methods in your ad hoc query. Listing 10-4 shows a complete example; the code returns the results of a data service function named getCustomers(), which is in the namespace:

ld:DataServices/RTLServices/Customer

String queryStr = 
      "declare namespace ns0=\"ld:DataServices/RTLServices/Customer\";" +
      "<Results>" +
      "  { for $customer_profile in ns0:getCustomer()" +
      "     return $customer_profile }" +
      "</Results>";
PreparedExpression adHocQuery = 
      DataServiceFactory.prepareExpression(context,"RTLApp",queryStr );
XmlObject objResult = (XmlObject) adHocQuery.executeQuery();

DSP passes information back to the ad hoc query caller as an XMLObject data type. Once you have the XMLObject, you can downcast to the data type of the deployed XML schema. Since XMLObject has only a single root type, if the data service function returns an array, your ad hoc query should include a root element as a container for the array.

For example, the ad hoc query shown in Listing 10-4 specifies a <Results> container object to hold the array of CUSTOMER_PROFILE elements that will be returned by the getCustomer() data service function.

Security policies defined for a data service apply to the data service calls in an ad hoc query as well. If an ad hoc query uses secured resources, the appropriate credentials must be passed when creating the JNDI initial context. (For more information, see Obtaining a WebLogic JNDI Context for Data Services Platform.)

As with the PreparedStatement interface of JDBC, the PreparedExpression interface supports dynamically binding variables in ad hoc query expressions. PreparedExpression provides several methods (bindType( ) methods; see Table 10-4), for binding values of various data types.

To use the bindType methods, pass the variable name as an XML qualified name (QName) along with its value; for example:

adHocQuery.bindInt(new QName("i"),94133);

Listing 10-5 shows an example of using a bindInt() method in the context of an ad hoc query.

PreparedExpression adHocQuery = DataServiceFactory.preparedExpression(
                 context, "RTLApp", 
                "declare variable $i as xs:int external;
                 <result><zip>{fn:data($i)}</zip></result>");
adHocQuery.bindInt(new QName("i"),94133);
XmlObject adHocResult = adHocQuery.executeQuery();

http://www.w3.org/TR/xmlschema-2/#QName

Listing 10-6 shows a complete ad hoc query example, using the PreparedExpression interface and QNames to pass values in bind methods.

import com.bea.ld.dsmediator.client.DataServiceFactory;
import com.bea.ld.dsmediator.client.PreparedExpression;
import com.bea.xml.XmlObject;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import javax.xml.namespace.QName;
import weblogic.jndi.Environment;

public class AdHocQuery
{
   public static InitialContext getInitialContext() throws NamingException {
    Environment env = new Environment();
    env.setProviderUrl("t3://localhost:7001");
    env.setInitialContextFactory("weblogic.jndi.WLInitialContextFactory");
    env.setSecurityPrincipal("weblogic");
    env.setSecurityCredentials("weblogic");
    return new InitialContext(env.getInitialContext().getEnvironment());
   }

   public static void main (String args[]) {
     System.out.println("========== Ad Hoc Client ==========");
     try {
      StringBuffer xquery = new StringBuffer();
      xquery.append("declare variable $p_firstname as xs:string external; \n");
      xquery.append("declare variable $p_lastname as xs:string external;  \n");
 
      xquery.append(
       "declare namespace ns1=\"ld:DataServices/MyQueries/XQueries\"; \n");
      xquery.append(
       "declare namespace ns0=\"ld:DataServices/CustomerDB/CUSTOMER\"; \n\n");

      xquery.append("<ns1:RESULTS>                                    \n");
      xquery.append("{                                                \n");
      xquery.append("    for $customer in ns0:CUSTOMER()              \n");
      xquery.append("    where ($customer/FIRST_NAME eq $p_firstname  \n");
      xquery.append("       and $customer/LAST_NAME eq $p_lastname)   \n");
      xquery.append("    return                                       \n");
      xquery.append("      $customer                                  \n");
      xquery.append(" }                                               \n");
      xquery.append("</ns1:RESULTS>                                   \n");

         PreparedExpression pe = DataServiceFactory.prepareExpression(
                        getInitialContext(), "RTLApp",  xquery.toString());
         pe.bindString(new QName("p_firstname"), "Jack");
         pe.bindString(new QName("p_lastname"),  "Black");
         XmlObject results = pe.executeQuery();
         System.out.println(results);

       } catch (Exception e) {
           e.printStackTrace();
       }
}

Handling Large Result Sets with Streaming APIs

This section discusses further programming topics related to client programming with the Data Service Mediator API. It includes the following topics:

Using the Streaming Interface

When a function in the standard data service interface is called, the requested data is first materialized in the system memory of the server machine. If the function is intended to return a large amount of data, in-memory materialization of the data may be impractical. This may be the case, for example, for administrative functions that generate "inventory reports" of the data exposed by DSP. For such cases, DSP can serve information as an output stream.

DSP leverages the WebLogic XML Streaming API for its streaming interface. The WebLogic Streaming API is similar to the standard SAX (Streaming API for XML) interface. However, instead of contending with the complexity of the event handlers used by SAX, the WebLogic Streaming API lets you use stream-based (or pull-based handling of XML documents in which you step through the data object elements. As such, the WebLogic Streaming API affords more control than the SAX interface, in that the consuming application initiates events, such as iterating over attributes or skipping ahead to the next element, instead of reacting to them.

It is important to note that although serving data as a stream relieves the server from having to materialize large objects in memory, the server is using the request thread while output streaming occurs. This can tie up a thread for quite a while and affect the server's ability to respond to other service requests in a timely fashion. The streaming API is intended for use only for administrative sorts of uses, and should be avoided except at off-peak times or in non-production environments.

Data Services Platform streaming API can only be invoked from Java code that is part of the same application from which you are streaming data. That is, the client code needs to be in the same EAR application file in which the data services are hosted.

You can get DSP information as a stream by using either an ad hoc or an untyped data service interface.

The streaming interface is in these classes in the com.bea.ld.dsmediator.client package:

Using these interfaces is very similar to using their SDO mediator client API equivalents. However, instead of a document object, they return data as an XMLInputStream. For functions that take complex elements (possibly with a large amount of data) as input parameters, XMLInputStream is supported as an input argument as well. The following is a example:

StreamingDataService ds = StreamingDataServiceFactory.getInstance(
                             context, 
                             "ld:DataServices/RTLServices/Customer");
XMLInputStream stream = ds.invoke("getCustomerByCustID", "CUSTOMER0");

The previous example shows the dynamic streaming interface. The following example uses an ad hoc query:

String adhocQuery = 
     "declare namespace ns0=\"ld:DataServices/RTLServices/Customer\";\n" +
     "declare variable $cust_id as xs:string external;\n" + 
     "for $customer in ns0:getCustomerByCustID($cust_id)\n" + 
     "return\n" + 
     "   $customer\n"; 
StreamingPreparedExression expr = 
                DataServiceFactory.prepareExpression(context, adhocQuery);

If you have external variables in the query string (adhocQuery in the above example), you will also need to do the following:

	expr.bindString("$cust_id","CUSOMER0");
	XMLInputStream xml = expr.executeQuery();

Listing 10-7 shows an example of a method that reads the XML input stream. This method uses an attribute iterator to print out attributes and namespaces in an XML event and throws an XMLStream exception if an error occurs.

import weblogic.xml.stream.Attribute;
import weblogic.xml.stream.AttributeIterator;
import weblogic.xml.stream.ChangePrefixMapping;
import weblogic.xml.stream.CharacterData;
import weblogic.xml.stream.XMLEvent;
import weblogic.xml.stream.EndDocument;
import weblogic.xml.stream.EndElement;
import weblogic.xml.stream.EntityReference;
import weblogic.xml.stream.Space;
import weblogic.xml.stream.StartDocument;
import weblogic.xml.stream.XMLInputStream;
import weblogic.xml.stream.XMLInputStreamFactory;
import weblogic.xml.stream.XMLName;
import weblogic.xml.stream.XMLStreamException;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
public class ComplexParse {

public void parse(XMLEvent event)throws XMLStreamException
{
  switch(event.getType()) {
  case XMLEvent.START_ELEMENT:
  StartElement startElement = (StartElement) event;
  System.out.print("<" + startElement.getName().getQualifiedName() );
  AttributeIterator attributes = startElement.getAttributesAndNamespaces();
  while(attributes.hasNext()){
    Attribute attribute = attributes.next();
    System.out.print(" " + attribute.getName().getQualifiedName() + 
    "='" + attribute.getValue() + "'");
  }
  System.out.print(">");
    break;
  case XMLEvent.END_ELEMENT:
    System.out.print("</" + event.getName().getQualifiedName() +">");
    break;
  case XMLEvent.SPACE:
  case XMLEvent.CHARACTER_DATA:
    CharacterData characterData = (CharacterData) event;
    System.out.print(characterData.getContent());
    break;
  case XMLEvent.COMMENT:
    // Print comment
    break;
  case XMLEvent.PROCESSING_INSTRUCTION:
    // Print ProcessingInstruction
    break;
  case XMLEvent.START_DOCUMENT:
    // Print StartDocument
    break;
  case XMLEvent.END_DOCUMENT:
    // Print EndDocument
    break;
  case XMLEvent.START_PREFIX_MAPPING:
    // Print StartPrefixMapping
    break;
  case XMLEvent.END_PREFIX_MAPPING:
    // Print EndPrefixMapping
    break;
  case XMLEvent.CHANGE_PREFIX_MAPPING:
    // Print ChangePrefixMapping
    break;
  case XMLEvent.ENTITY_REFERENCE:
    // Print EntityReference
    break;
  case XMLEvent.NULL_ELEMENT:
      throw new XMLStreamException("Attempt to write a null event.");
  default:
   throw new XMLStreamException("Attempt to write unknown event["
                                 +event.getType()+"]");
  }
}

Writing Data Service Function Results to a File

You can write serialized results of a data service function to a file using a WriteOutputToFile method. Such a function is generated automatically for each function defined in the data service. For security reasons it writes only to a file on the server's file system.

These functions provide services that are similar to streaming APIs. They are intended for creating reports or an inventory of data service information. However, the writeOutputToFile method can be invoked from a remote mediator API (in contrast with the streaming API described in Using the Streaming Interface on page 10-16).

The following example shows how to write to a file from the untyped interface.

 StreamingDataService sds =
  	DataServiceFactory.newStreamingDataService(
     		context,"RTLApp","ld:DataServices/RTLServices/Customer""); 
 sds.writeOutputToFile("getCustomer", null, "streamContent.txt");
 sds.closeStream();

Providing Role-based Access to DSP Relational Sources

When you import metadata from relational sources, you can provide logic in your application that maps users to different data sources depending on the user's role. This is accomplished by creating an intercepter and adding an attribute to the RelationalDB annotation for each data service in your application.

The interceptor is a Java class that implements the SourceBindingProvider interface. This class provides the logic for mapping a users, depending on their current credentials, to a logical data source name or names. This makes it possible to control the level of access to relational physical source based on the logical data source names.

For example, you could have the data source names cgDataSource1, cgDataSourc2, and cgDataSource3 defined on your WebLogic Server and define the logic in your class so that an user who is an administrator can access all three data sources, but a normal user only has access to the data source cgDataSource1.

To implement the interceptor logic, do the following:

Setting Up Handlers for Web Services Accessed by DSP

When you import metadata from web services for DSP, you can create SOAP handler for intercepting SOAP requests and responses. The handler will be invoked when a web service method is called. You can chain handlers that are invoked one after another in a specific sequence by defining the sequence in a configuration file.

To create and chain handlers, follow these two steps: