Skip Headers
Oracle® Application Server Wireless Developer's Guide
10g Release 2 (10.1.2)
B13819-02
  Go To Documentation Library
Home		Go To Product List
Solution Area		Go To Table Of Contents
Contents		 Go To Index
Index
Previous
Previous 		Next
Next 		 

14 Web Scraping

This chapter describes how to clip and scrape Web content and create Wireless Web Clipping applications for use by wireless devices. The sections in this chapter include:

14.1 Overview of Web Scraping

The majority of applications available on the Web render content in format specific to certain types of devices. Web scraping allows applications developed for a particular markup language to be reformatted for use with other devices. Specifically, Web applications rendered in HTML and developed for PC browsers can be scraped to deliver to wireless devices through OracleAS Wireless.

To reuse WML applications, you can use the WML Translator.

14.2 Introducing Oracle Application Server Web Clipping

The Wireless Web Clipping server enables OracleAS Wireless administrators to clip, then scrape Web content and create Web Clipping applications that are stored persistently in the Wireless Web Clipping server repository. Then, the server serves Web Clipping applications based on these definitions, which are delivered by OracleAS Wireless.

The Wireless Web Clipping server architecture as it relates to OracleAS Wireless is shown in Figure 14-1.

Figure 14-1 Web Clipping Server Architecture Relative to OracleAS Wireless Core

Description of Figure 14-1 follows
Description of "Figure 14-1 Web Clipping Server Architecture Relative to OracleAS Wireless Core"

The Web Clipping definitions are comprised of URLs and scraping instructions so that they can be used to dynamically play back the scraping action to the repurposed Web sites. The Wireless Web Clipping server repository stores these definitions, while the Wireless Repository stores information to access the Wireless Web Clipping application.

The Wireless Web Clipping server serves Web Clipping applications based on these definitions, which are delivered by OracleAS Wireless.

When a request is initiated by a user using a mobile wireless device to view a particular Web Clipping application, that application is retrieved by the HTTP Adapter and delivered to OracleAS Wireless for processing and delivery to the mobile device, as shown in Figure 14-2.

Figure 14-2 Web Clipping Application

Description of Figure 14-2 follows
Description of "Figure 14-2 Web Clipping Application"

Web clipping content can be clipped and scraped from Internet or Intranet Web sites scattered throughout a large organization. Using OracleAS Wireless, you can generate Wireless Web Clipping applications using Web clippings and store them in the Wireless Repository.

In addition, you can create customized wireless applications that use Web clippings to their best advantage. As a developer, you first identify the development environment that best fits your needs. Depending upon the development environment you select, you must download and deploy the required archive files into that development environment. Next, you clip the desired content. Then, you download either the Java or JSP stub files to start developing.

Figure 14-3 shows a more detailed look at the Web Clipping server.

Figure 14-3 Detailed Web Clipping Server Architecture

Description of Figure 14-3 follows
Description of "Figure 14-3 Detailed Web Clipping Server Architecture"

OracleAS Wireless provides archive files for the following development environments are:

Applications can be customized and deployed in J2EE or J2SE environments as standalone Java applications or part of other JavaServer Pages (JSP) applications, or these applications can be created into wireless applications and stored in the wireless repository, or this content can be clipped.

To create a Web Clipping application, the OracleAS Wireless administrator uses a Web browser to navigate to the Web page containing the desired content, then selects the portion of the page to clip and scrape, sets some attributes, and if the Web clipping uses form-based submission, exposes input parameters, saves the application, and tests the application.

Web Clipping applications support:

The following are known limitations of Web clipping:

The Web Clipping server provides and renders clipped Web content as Web Clipping application content. The Web Clipping Studio allows OracleAS Wireless administrators and wireless application users to do the following:

By default, all Web Clipping application definitions are stored persistently in the OracleAS Infrastructure database. Any secure information, such as passwords, are stored in encrypted form, according to the Data Encryption Standard (DES), using Oracle encryption technology.

14.3 Creating an OracleAS Web Clipping Application

The Web Clipping server is automatically installed as part of OracleAS Wireless. As an OracleAS Wireless administrator, you can access the Web Clipping server in the repository under the Applications: Browse Folder: page in the OracleAS Wireless Service Manager.

You may need to perform some configuration tasks before getting started. See Section 14.6, "Administering Web Scraping" for more information.

To create a Web Clipping, perform the following steps:

  1. Log into OracleAS Wireless Service Manager as an OracleAS Wireless administrator.

  2. Select the Services tab.

  3. On the Applications tab, click Create Application.

  4. On the Select the Application To Be Created page, select Web Clipping Application, then click Create.

  5. On the Manage Web Clippings page, click Add Web Clipping. The Find Web Clipping page in the Web Clipping Studio is displayed.

  6. In the URL Location field on the Find Web Clipping page, enter the location of the first Web page that leads to the Web page you want to clip. This example starts with http://oraclestore.oracle.com, as shown in Figure 14-4.

    Figure 14-4 Find a Web Clipping

    Description of Figure 14-4 follows
    Description of "Figure 14-4 Find a Web Clipping"


    Note:

    To import a WIDL file and create a Web clipping, select the Import from WIDL tab. See Section 14.4, "Migrating from Existing Transcoding Technologies" for more information.

  7. Click Start.

    Web Clipping Studio displays the target page, which asks you to choose your store.

  8. Select a country and language. For example, for United States, click English.

    The Oracle Store Web page for the country you chose appears in the Web Clipping Studio, as shown in Figure 14-5.

    Figure 14-5 Browse ORACLESTORE.ORACLE.COM in the Web Clipping Studio

    Description of Figure 14-5 follows
    Description of "Figure 14-5 Browse ORACLESTORE.ORACLE.COM in the Web Clipping Studio"

    As you click hyperlinks in the Web page to browse to the content you want to clip, Web Clipping Studio records your navigation links.


    Note:

    Any browsing operations that do not contribute to the eventual Web clipping will be discarded. Thus, only the significant browsing operations are recorded for later playback during the test mode; any discarded links are not visited.


    Note:

    For any Web sites that require HTTP Basic or Digest Authentication, a form is displayed that requests user name and password information. This encoded authentication information is recorded as part of the browsing information.

    For this example, in the Quick Search field, enter J2EE and click Go. All products containing that name appear in the search results, as shown in Figure 14-6.

    Figure 14-6 Browse to the Content to Be Clipped

    Description of Figure 14-6 follows
    Description of "Figure 14-6 Browse to the Content to Be Clipped"

  9. In the Web Clipping Studio in browse mode, after you find the content you want to clip, click Section. This divides the target Web page into its clippable sections, as shown in Figure 14-7.

    Figure 14-7 Divide the Page into Clippable Sections

    Description of Figure 14-7 follows
    Description of "Figure 14-7 Divide the Page into Clippable Sections"


    Note:

    After you click Section, you are no longer able to browse links in the displayed page. If you want to continue navigation, click Unsection.

  10. In the Web Clipping Studio in section mode, find the section of the Web content you want to clip, then click Choose.

    To increase the number of sections available from which to choose, click Section Smaller. For example, you click Section Smaller to drill down one level of nested tables. To decrease the number of sections available from which to choose, click Section Larger.


    Note:

    You can click only one Choose section at a time. Only the content immediately below the Choose section icon you selected is chosen as a clipping. Continuing to preview mode will let you view the clipped content. Clicking Unselect will return you to section mode again to choose another section to clip if the section you had chosen was not the one you wanted.

    In the Web Clipping Studio, a preview of the Item section is displayed with the search results, as shown in Figure 14-8.

    Figure 14-8 Preview the Web Clipping in the Web Clipping Studio

    Description of Figure 14-8 follows
    Description of "Figure 14-8 Preview the Web Clipping in the Web Clipping Studio"


    Note:

    Some sections may contain no data, only whitespace. For example, a Web page may contain an HTML <DIV> tag that contains no text or images. If you clicked Choose on a section that contains no data, Web Clipping displays a preview, but the preview correctly shows only whitespace. In this case, click Unselect to return to the sectioned page. Then, select a section containing data.

    If you do not want to use the section you clipped, click Unselect to return to the page containing the section. You can choose another section on the page, or click Unsection to navigate to another page.

  11. If it is the section you want, click Scrape to continue to scrape mode.

  12. In scrape mode, a check box following anything, such as text, indicates that you can select it as output that can be scraped. One or more items of output can make up a collection of similar output. In tables with cells, a special stacked check box is displayed to allow selection of all text in the cell as output.

    Click the check box at the end of the first output. For each output you select, a row of values appears under the Value and Name table in the bottom frame, where you can name your output. In the Name column, enter a meaningful name for your output, such as Books.


    Note:

    The name cannot contain any spaces.

    If the content you select contains an associated hyperlink, the value of the hyperlink will appear as an additional column in the resulting data source output. In addition, if you select an image, the value of the source (SRC) of the image will be output.

    After you select the output, you will have created a group that can be repeated as a whole as long as the first output you selected represents the first item in the collection of similar output. This means that you can only apply repeatability to a collection of similar output. Click More once to increase the repeat level to encompass the entire collection of similar output. Every item of output is checked, as shown in Figure 14-9.

    Figure 14-9 Scrape the Web Clipping

    Description of Figure 14-9 follows
    Description of "Figure 14-9 Scrape the Web Clipping"

    Clicking More lets you quickly select the entire collection of similar output to be scraped. As you click More, check boxes of successive groups of selected output are checked and designated to be scraped. This is a quick way of selecting successive groups of similar output to be scraped as output, rather than individually checking each one.

    Click Continue.

  13. In the Edit Clipping Attributes section of the Find Web Clipping page, you can set the following attributes of the Web Clipping application:

    • Title: The name of the Wireless Web Clipping application.

    • Description: A brief description of the Wireless Web Clipping application. Enter Search Oracle Store.

    • Time Out (seconds): The connection timeout or length of time in seconds that the Wireless Web Clipping server will wait to establish a connection to the source of the Web clipping application before it returns a timeout message. The default value is generated from the actual time measured to connect to the intranet Web site being clipped using the Web Clipping Studio, and it will be between 0 and 60 seconds.

    Figure 14-10 shows the top third of the Find Web Clipping page.

    Figure 14-10 Edit Clipping Attributes

    Description of Figure 14-10 follows
    Description of "Figure 14-10 Edit Clipping Attributes"

  14. If you went through form-based submission while clipping content for the Web Clipping application, a heading titled Select the Clipping Customizable Parameters appears in the Find Web Clipping page. A table lists the parameters.

    For each parameter that you want to page viewers to be able to customize, select the desired parameter from the drop-down box in the Parameters column, then select the Customizable column. Doing this lets OracleAS Wireless administrators customize the input value to personalize each parameter selected.


    Note:

    Do not make the user name and password fields into parameters that could be customized for Web sites that use form-based submission login.

    In addition, to enforce stronger security, the Web clipping does not support capturing JavaScript from Web sites that may perform additional JavaScript processing of the user name and password each time a login takes place.


    Figure 14-11 shows the Select the Clipping Customizable Parameters section of the Find Web Clipping page.

    Figure 14-11 Customize Form Input for a Web Clipping Application

    Description of Figure 14-11 follows
    Description of "Figure 14-11 Customize Form Input for a Web Clipping Application "

  15. In the row whose Index value is 2, select kw from the Parameters list. Then, change its Display Name from kw to keyword to make this name more recognizable for wireless application users.

  16. Click the Customizable box to let wireless application users customize the parameter input value.

  17. To test the Web clipping application, click Test in the Test the Web Clipping section as shown in Figure 14-12.

    Figure 14-12 Test the Web Clipping Application

    Description of Figure 14-12 follows
    Description of "Figure 14-12 Test the Web Clipping Application"

    Results of the Web clipping test appear in a browser page as an XML document. Inspect the contents of the XML document to see if it contains the desired content. In a wireless device, this XML document would appear formatted.

  18. Click Apply to save your changes when you have finished choosing the options.


    Note:

    Selecting Apply saves all edits and options for the Web Clipping application, which makes these changes immediately accessible to wireless application users while letting the OracleAS Wireless administrator make more edits as necessary on the Web Clipping application attributes page. On the other hand, selecting OK both saves all edits to the Web Clipping application attributes, and makes these changes immediately accessible to wireless application users, but exits the Web Clipping application attributes page and returns to the Applications page.

  19. After you click Apply to save your changes, the Test the Web Clipping section of the Find Web Clipping page includes an Inputs heading, which lists the parameters that were customized, along with their initial input values, as shown in Figure 14-13.

    Figure 14-13 Test the Web Clipping Application Input Value

    Description of Figure 14-13 follows
    Description of "Figure 14-13 Test the Web Clipping Application Input Value"

  20. Notice the parameter named Keyword and its input value J2EE. To test another input value, replace the initial value and enter a new initial value, such as cache, as shown in Figure 14-14. Click Test to test the new input value.

    Figure 14-14 Change the Input Value

    Description of Figure 14-14 follows
    Description of "Figure 14-14 Change the Input Value"

  21. Results of this Web Clipping application test appear in a browser page. Inspect the contents of the XML document to see if it contains the desired content.

  22. Click OK to save your changes and return to the Manage Web Clippings page as shown in Figure 14-15.

    Figure 14-15 Manage Web Clippings

    Description of Figure 14-15 follows
    Description of "Figure 14-15 Manage Web Clippings"

    On the Manage Web Clipping page, you can:

    • Search for a Web clipping: In the Search field, enter a search string that matches any part of the name of a Web clipping, then click Go to initiate the search. The list of Web clippings whose names match this search string is displayed.

    • Edit a Web clipping: Select the Web clipping you want to edit by selecting it in the Select column. Then, click Edit. The Find Web Clipping page with the Edit Attributes section, as shown in Figure 14-10, appears. From this page, you can edit the Web clipping attributes.

    • Create a new Web clipping: Click Add Web Clipping. The Find Web Clipping page appears in which you can enter a starting-point URL and begin browsing a Web site, and looking for content to clip.

    • Delete a Web clipping: Select the Web clipping you want to delete by selecting it in the Select column and then click Delete. A confirmation message is displayed indicating the Web clipping is deleted.

    • Create a default application: Click Create Default Application to see a default rendering of your newly created Web clipping.

14.3.1 Customizing an OracleAS Web Clipping Service

Wireless users can click Change Input on the test page to customize values for any input parameters of a form if the OracleAS Wireless administrator parameterized these input parameters and made them customizable. See Steps 14 through 16 in Section 14.3 for more information.

14.4 Migrating from Existing Transcoding Technologies

Previously in OracleAS Wireless, transcoding or Web content reuse was performed using Web Methods' Web Integration technologies. The recorded instructions on how and where to fetch external Web content was captured in a Web Integration Definition Language (WIDL) file. WIDL is a meta-language that implements a service-based architecture over the document-based resources of the World Wide Web. In this release, the responsibility of transcoding falls on Web Clippings.

You do not need to recreate the same set of instructions on Web Clippings terms. Web Clipping provides a migration path that allows for importing WIDL mapping files into Web Clippings so that you may reuse what you have captured previously. Figure 14-16 shows the augmented Web Clipping Server architecture, with the added WIDL Source as an alternate means of creating a Web Clipping.

Figure 14-16 Web Clipping Server Architecture Showing Importing a WIDL Definition File

Description of Figure 14-16 follows
Description of "Figure 14-16 Web Clipping Server Architecture Showing Importing a WIDL Definition File"

To migrate to Web Clipping:

  1. To import a WIDL definition file and create a Web clipping from it, click the Web Clipping Studio tab. On the Find Web Clipping page, select the Import from WIDL subtab.

    Figure 14-17 shows the Import Web Clippings from WIDL page.

    Figure 14-17 Import from a WIDL Definition to a Web Clipping

    Description of Figure 14-17 follows
    Description of "Figure 14-17 Import from a WIDL Definition to a Web Clipping"

  2. At the Import Web Clippings from WIDL page, you can import an existing WIDL file, create a Web clipping, and save it in the repository.

    To import a WIDL definition file, you can either:

    • Enter the starting-point URL in the WIDL Location field and click Start.

    • Click Browse to browse to the directory on your system where your WIDL definition file is located and click the WIDL file to populate the WIDL Location field. Then, click Start to begin importing the WIDL definition file.

    The Import from WIDL page reappears, containing a new section Choose the WIDL service to import, as shown in Figure 14-18.

    Figure 14-18 Choose the WIDL Service to Import

    Description of Figure 14-18 follows
    Description of "Figure 14-18 Choose the WIDL Service to Import"

  3. If your WIDL file contains more than one service to be imported, select the service that you want to import from the Choose WIDL Service box.

  4. If the WIDL service went through form-based submission (that is, it contains input bindings), the Import from WIDL page contains the section Set Parameters. In this section, you can specify default values for any parameter. In the Default Value column, and enter a default value for each chosen parameter, as shown in Figure 14-19.

    Figure 14-19 Specify Default Values for Parameters

    Description of Figure 14-19 follows
    Description of "Figure 14-19 Specify Default Values for Parameters"

  5. Click Continue. The WIDL service is imported and made into a Web clipping. The Find Web Clipping page appears.

  6. In the Find Web Clipping page, you can edit the attributes of the new Web clipping, including customizing form input information if the Web clipping has form-based submission. Edit the clipping attributes, such as the Description field, then click OK to save your changes and return to the Manage Web Clippings page.

At the Manage Web Clippings page, you are ready to create a wireless application using the Web clipping you created.

14.5 Creating a Wireless Application

To create a wireless application from a Web clipping, you use either of the following approaches:

14.5.1 Creating a Default Application

On the Manage Web Clippings page, as shown in Figure 14-15, you can create a default wireless application.

From the Select table, select the Web clipping you want to make into a default wireless application, and then click Create Default Application. A default wireless application is created to deliver a default rendering of the scraping just performed, in OracleAS Wireless XML.

After creating the default application, you are returned to the OracleAS Wireless Applications page, where you can create new Web Clipping applications.

Continuing with the example, the Oracle Store Search Result application is added to the list of applications, as shown in Figure 14-20.

Figure 14-20 Applications Page with New Wireless Application Listed

Description of Figure 14-20 follows
Description of "Figure 14-20 Applications Page with New Wireless Application Listed"

After you create a default application from a Web clipping, you can publish a new application. To create a new application link for a new wireless application, take the following steps:

  1. Click the Content tab and then click Add Application Link to add this new application to the Applications Links page.

  2. Follow the steps provided by the wizard. At the Application step, click the application on which you want to base this application. The page now shows a list of applications. Select your application, Oracle Store Search Result. Click Next.

  3. At the General step, enter a name, such as ORACLESTORE, for your application, then click Next.

  4. At the Input Parameters step, accept the default settings for your new application by clicking Next.

  5. At the Async Application step, click Next.

  6. At the Additional step, in the Description field, enter the word Books, then click Finish.

Your new application appears in the Applications Links list of applications, as shown in Figure 14-21. This enables an application to be published to user groups and made accessible to mobile users of wireless devices.

Figure 14-21 Application Links Page with New Application Published

Description of Figure 14-21 follows
Description of "Figure 14-21 Application Links Page with New Application Published"

From this page, you can also:

  • Delete a Web Clipping application: Select the application to be deleted by selecting it in the Select column, and then click Delete. A confirmation message appears indicating that the selected Web Clipping application is deleted.

  • Edit an existing Web Clipping application's attributes: Select the application to be edited by selecting it in the Select column, then click Edit in the Edit Clip in Web Clipping Studio section.

  • Categorize an application: Selecting it in the Select column, then click Categorize.

14.5.2 Testing an Existing OracleAS Web Clipping Application

You can test an application either from the Services tab or if you have published it, from the Content tab:

  1. At the Content tab, select the application to be tested, in this case your new application link ORACLESTORE, by clicking its Test icon in the Test column. An OracleAS Wireless PDA Simulator page appears in a new browser window, showing the contents of the Web Clipping application, as shown in Figure 14-2.

  2. On the OracleAS Wireless PDA Simulator display, scroll down to the bottom of the page and click Change Input to change the value of the initial input value for the parameter named keyword, as shown in Figure 14-22.

    Figure 14-22 Wireless PDA Simulator Showing an Initial Value

    Description of Figure 14-22 follows
    Description of "Figure 14-22 Wireless PDA Simulator Showing an Initial Value"

  3. Delete the initial input value, J2EE, and enter the value cache. Then click Submit.

  4. The OracleAS Wireless PDA Simulator now displays the results of the search for items in the Oracle Store relating to the value cache, as shown in Figure 14-23.

    Figure 14-23 Wireless PDA Simulator Shows the Results of a New Search

    Description of Figure 14-23 follows
    Description of "Figure 14-23 Wireless PDA Simulator Shows the Results of a New Search "

14.5.3 Building a Custom Application

You can build a custom wireless application using the Manage Web Clippings page, as shown in Figure 14-15. (To navigate to this page, from the Services tab, click Web Clipping Manager.) Select the Custom Applications tab. The Custom Applications page appears, as shown in Figure 14-24.

Figure 14-24 Custom Applications Downloads Page

Description of Figure 14-24 follows
Description of "Figure 14-24 Custom Applications Downloads Page"

From the Downloads section of the Custom Applications page, you can:

  • Download (for J2EE development) the Java Connector Architecture (JCA) Resource Adaptor Archive (RAR) for development within an instance of Oracle Application Server Containers for J2EE (OC4J) using JCA's Common Client Interface (CCI). Take the following steps:

    1. Click the RAR link in the J2EE Development section to download the JCA RAR file, webclipping.rar.

    2. Follow the instructions about deploying resource adapters in Oracle Application Server Containers for J2EE Services Guide to deploy an RAR file into your OC4J. See Oracle Application Server Containers for J2EE User's Guide for information about the correct version of OC4J.

      You may need to restart OC4J for the library to take effect.

    3. Return to the Web Clippings tab and find your desired Web clipping in the form of a JSP file to start developing using the RAR file you have just deployed.

  • Download (for J2SE development) the Java library (JAR) for development using a standalone Java 2 SDK. Take the following steps:

    1. Click the JAR link in the J2SE Development section to download the Java library file, wcbean.jar.

    2. For a standalone Java 2 SDK environment, place the JAR file in the classpath during compilation and running of your code.

      For J2EE (OC4J) development, place this JAR file where your Web application can access it, either in the root classpath of OC4J, in the application's own library path, or in the WEB-INF/lib directory of the Web Archive (WAR) file containing your Web application. Depending on the resource adaptor you use, you may need to restart your OC4J instance to be able to use the Web Clipping Bean API.

      The other files that are needed in order to compile and run the stub Java files are http_client.jar, Javax-ssl-1_2.jar, jssl-1_2.jar, and xmlparserv2.jar. Look for these files in your installation of OC4J.

    3. Return to the Web Clippings tab and find your desired Web clipping in the form of a Java source file (.Java) to start developing using the JAR file you just deployed.

At the Web Clippings tab, you can do one of the following:

  • Generate a Java file.

    Click Download Java (see Figure 14-15) to generate a Java (.Java) file from the Web clipping. This file can be compiled with other Java classes into an application, and deployed in J2SE as a standalone Java 2 SDK application.

  • Generate a JSP file.

    Click Download JSP (see Figure 14-15) to generate a JSP (.jsp) file from the Web clipping. This file can be deployed into J2EE OC4J so you can execute the Web clipping as a JSP file.

  • Create your own custom HTTP application using the downloaded Java file or JSP file.

    • Using the Java file

      Starting with the downloaded Java file, which contains the sample code of how to use the Web Clipping Bean APIs, you can create your own HTTP application using any standard J2EE Container (such as OC4J). You must first include the wcbean.jar file, as well as its dependency libraries, into the library path of the HTTP Server.

    • Using the JSP file

      With the downloaded JSP file, which contains the sample code of how to use the Java Connector Architecture APIs, you can create your own HTTP application using OC4J (this is a requirement as the RAR file is compliant with OC4J).

    • Building the Mobile UI

      To be compliant with OracleAS Wireless, the HTTP application must render its output in either OracleAS Wireless XML, XHTML/MP, or XHTML/XForms. You have full control over the Mobile UI of this HTTP application including the addition of mobile links or images. Note that the default JSP code renders the data as an HTML unordered list; you must change that rendering to one that is compliant with OracleAS Wireless.

    • Creating a Wireless Application

      After the creation of the HTTP application, you can use it to create a Wireless application using the HTTP Adaptor.

14.6 Administering Web Scraping

The administrative tasks related to Web Scraping that must be performed by the OracleAS Wireless administrator include:

14.6.1 Configuring Security

When you navigate to a secure site, the Web site typically returns a certificate identifying itself to the user when it asks for secure information. If you accept the certificate, the certificate is placed into the list of trusted certificates of the browser so that a secure channel can be opened between the browser and that server. Like a Web browser, the Web Clipping server behaves as an HTTP client to external Web sites. In order for the Web Clipping server to keep track of trusted sites, it makes use of a file that stores the certificates of those sites, namely the trusted server certificate file, ca-bundle.crt. This file is located in ORACLE_HOME/portal/conf on UNIX or in ORACLE_HOME\portal\conf on Windows.

The trusted server certificate file, generated from Oracle Wallet Manager, is shipped with OracleAS Portal. It contains an initial list of trusted server certificates that might be used for navigating to some secure servers using HTTPS. However, this is not a complete list of all possible server certificates that exist on the Web. For this reason, when an OracleAS Wireless administrator navigates to a secure server using HTTPS, the administrator may get an "SSL Hand-shake failed" exception in the Web Clipping Studio. To solve this problem, the ca-bundle.crt file needs to be augmented with new trusted sites that are visited. As an OracleAS Wireless administrator, you must do the following to extend the shipped ca-bundle.crt file:

  1. Use a browser (preferably Internet Explorer) to download the root server certificate from each external HTTPS Web site in BASE64 format that is visited, and is missing from the trusted certificate file.

  2. Use Oracle Wallet Manager to import each certificate.

  3. Export the trusted server certificates into a file, and replace the ca-bundle.crt file with that file.


Note:

The ca-bundle.crt file will still be present and functional, even if OracleAS Portal has not been configured.

For more information about Oracle Wallet Manager, see the Oracle Application Server Security Guide.

14.6.2 Rendering Events to Be Logged and Generating Reports

Web Clipping allows rendering events to be logged so that an administrator can query the event log and generate useful reports, such as those used for billing purposes. To enable event logging, as administrator, you must manually modify a context-param within the web.xml file located at: 

ORACLE_HOME/j2ee/OC4J_Wireless/applications/webclipping/ webclipping-web/WEB-INF/web.xml

Look for the <context-param> with <param-name> equal to oracle.webclipping.LogBusiness. To enable the logging of rendering events, change the <param-value> to true. After setting this parameter, restart the OC4J_Wireless instance to refresh this change. Refer to the Oracle Application Server Containers for J2EE User's Guide for more information.

After the logging is enabled, you can make use of a set of PL/SQL procedures in the Infrastructure database to operate on the logged events. First, connect to the Infrastructure database as SYSDBA, then execute the following line to change the user to WCRSYS: 

ALTER SESSION SET CURRENT_SCHEMA=WCRSYS;

As user WCRSYS, you can use the following PL/SQL procedures and functions to operate on the events logged. The record type that you will be using in most of these procedures and functions is WWWCP_API_REGISTRY.REC_RENDER_EVENT: 

/**
* This describes a record type used to return a single clipping rendering
* event.
*
* This structure is used by the lookup APIs to encapsulate the 
* information that is retrieved from the wwwcp_render_log$ table. 
* It is used to describe the cursor that will be returned as an OUT 
* parameter of lookup_render_events.
*
* @field clip_id           The clip id that allows the fetching of the 
*                          other facets of the clipping definition to 
*                          populate the events table.
* @field clip_description  Textual description of the clip rendered.
* @field clip_title        Title of the clip that was rendered.
* @field clip_timeout      Timeout in milliseconds that allows the clipping
*                          that was rendered to be timed out. This could be
*                          an indication of the quality of service.
* @field effective_url     The url where the clip resides. This is usually
*                          the last url declared in the clipping definition
*                          clipping definition, where the clip would reside.
* @field render_status     A number that indicates either success or 
*                          failure of the rendering call.
* @field render_type       Tells what type of rendering is in question,
*                          whether it be for Portal Show Mode, for 
*                          Wireless default SimpleResult show mode, or 
*                          for the Wireless connector show mode.
* @field render_start      Starting time of the rendering.
* @field render_end        Ending time of the rendering.
* @field fuzzy_used        Indicates whether fuzzy match was kicked in.
* @field db_lookup         The time it takes in milliseconds to look up
*                          the clipping definition from the database.
* @field http_latency      The time it takes to reach the first byte to
*                          read from the http remote site.
* @field render_user       The user (together with a possible realm) for
*                          which the rendering was done.
* @field error_cause       The cause of the error if the status indicated
*                          a failure of the rendering.
* @field event_description This field provides more information about
*                          the logged event.
* @field clip_input        The input provided to render this clipping. It
*                          is usually provided in the form of an HTTP URL
*                          query string like abc=def.
* @field clip_output       The possible output (only if it's small) of 
*                          rendering, if it gives any more hints on the
*                          billing process.
*/
type rec_render_event is record (
    clip_id           integer,
    clip_description  varchar2(1024),
    clip_title        varchar2(512),
    clip_timeout      integer,
    effective_url     varchar2(2048),
    render_status     integer,
    render_type       integer,
    render_start      date,
    render_end        date,
    fuzzy_used        integer,
    db_lookup         integer,
    http_latency      integer,
    render_user       varchar2(512),
    error_cause       varchar2(256),
    event_description varchar2(256),
    clip_input        varchar2(128),
    clip_output       varchar2(256));

The Cursor type that is used with this record type is WWWCP_API_REGISTRY.RENDER_EVENT_CURSOR. The following procedures within the WWWCP_API_REGISTRY have been defined to allow lookups on the render event log: 

/**
 * This API looks up rows of render events from the wwwcp_render_log$
 * table, filtered by a specific clip id.
 *
 * The full rendering event information structure is returned as iterable
 * by the cursor returned.
 *
 * @param p_clip_id          The clip id as a filtering mechanism of the
 *                           events that were logged for this particular
 *                           clip.
 * @param p_cv_render_events The list of render events that are associated
 *                           with this clip.
 */
 procedure lookup_render_events(
     p_clip_id          in  integer,
     p_cv_render_events out render_event_cursor
 );

/**
 * This API looks up rows of render events from the wwwcp_render_log$
 * table, filtered by a specific effective url.
 *
 * This is useful for business queries that center on not the clip id but by
 * the actual location of where the clips are found. For example, if Yahoo's
 * partner, namely someone using Web Clippings, can do a search of all the
 * events that went to http://my.yahoo.com, it can find out how to charge
 * each user for what they have rendered.
 *
 * The full rendering event information structure is returned as iterable
 * by the cursor returned.
 *
 * @param p_effective_url    The effective url, starting with which points
 *                           to the Web page containing the clip.
 * @param p_cv_render_events The list of render events that are associated
 *                           with this clip.
 */
 procedure lookup_render_events(
     p_effective_url    in  varchar2,
     p_cv_render_events out render_event_cursor
 );
 
/**
 * This API looks up rows of render events from the wwwcp_render_log$
 * table, filtered by the domain.
 *
 * This is useful for business queries that center on not the clip id but by
 * the domain of where the clips are found. For example, if Yahoo can do a
 * search of all the events that went to *.yahoo.com, it can find out how to
 * charge each user for what they have rendered.
 *
 * The full rendering event information structure is returned as iterable
 * by the cursor returned.
 *
 * @param p_effective_domain The domain of the effective url, starting with
 *                          which points to the Web page containing the
 *                          clip. This domain is provided in the format
 *                          "oracle.com" or "yahoo.com".
 * @param p_cv_render_events The list of render events that are associated
 *                          with this clip.
 */
 procedure lookup_render_events(
     p_effective_domain in  varchar2,
     p_cv_render_events out render_event_cursor
 );
 
 /**
 * This API looks up rows of render events from the wwwcp_render_log$
 * table, filtered by the user that requested the render.
 *
 * This is useful for business queries that center on not the clip id but by
 * the user to visualize how much a user has rendered, regardless of where
 * the clipping is from, so it can charge for a flat fee based on for 
 * example, the number of render events for a particular user.
 *
 * The full rendering event information structure is returned as iterable
 * by the cursor returned.
 *
 * @param p_render_user      The user on behalf of whom the render request
 *                           is made.
 * @param p_cv_render_events The list of render events that are associated
 *                           with this clip.
 */
 procedure lookup_render_events(
     p_render_user      in  varchar2,
     p_cv_render_events out render_event_cursor
);

Make sure that the logged events do not exceed the allotted database table space. The following procedures have been defined to allow removal of entries from the render event log: 

/**
 * This API removes all render events from the wwwcp_render_log$ table.
 *
 * This is useful as a cleansing measure that an organization can do between
 * the different phases of rolling out their Web Clipping usage initiative.
 * It can be used to reset the stage before going into development testing
 * phase, or when the rollout is going live and the company wishes to begin
 * charging its subscribers for their usages, or another possible cleansing
 * may occur at for example, every month end or year end when the data
 * recorded is no longer useful.
 */
 procedure remove_all_render_events;
 
/**
 * This API removes all render events from the wwwcp_render_log$ table for
 * a given rendering user.
 *
 * This is useful as a cleansing measure that an organization can do for
 * example, labeling the development user to be some constant user id and
 * then removing all rendering events when it goes live.
 *
 * @param p_render_user The user on behalf of whom the render request is
 *                      made.
 */
 procedure remove_render_events(p_render_user in varchar2);
 
/**
 * This API removes all render events from the wwwcp_render_log$ table for
 * a given starting or stopping time before or after the query time.
 *
 * This is useful as a cleansing measure that an organization can do for
 * example, all rendering events that started or stopped before a certain 
 * date are deemed development efforts and therefore should be disregarded.
 *
 * @param p_query_time   The date/time with which to query.
 * @param p_query_before Whether the query is for before or after. A value
 *                       of 1 signifies that the query is for comparing 
 *                       times before the p_query_time while a 0 value.
 *                       implies comparing times after the p_query_time.
 * @param p_query_start  Whether or not the query is for start time.
 *                       A value of 1 means true while 0 means false.
 */
 procedure remove_render_events(
     p_query_time   in date,
     p_query_before in number,
     p_query_start  in number
 );

14.7 Understanding the WML Translator

The OracleAS Wireless WML Translator reformats applications developed in WML for all wireless Web-enabled devices. At runtime, content developed in WML is converted to OracleAS Wireless XML, which is then transformed into the appropriate device specific markup language.

Generally, source documents in WML 1.x (up to WML 1.3) are supported. Each source document is processed in the following order:

Table 14-1 shows a rough comparison between source elements in WML and translated elements in OracleAS Wireless XML.

Table 14-1 Relationship Between Source and Translated Elements

Source Element in WML Translated Element in Wireless XML Notes

<wml>

<SimpleResult>

None

<head>

NA

Child element <meta> is translated into <SimpleMeta>

<template>

NA

Attributes onenterforward, onenterbackward, ontimer, and child elements <do> and <onevent> are honored based on WML specification (see <card>)

<card>

<SimpleContainer>

Attributes onenterforward, onenterbackward, ontimer, and child elements <do>, <onevent> are processed in combination with those corresponding attributes or child elements in the <template> element. See <card onenterforward>, <card onenterbackward>, and <card ontimer> for details.

< card onenterforward> (or <template onterforward>, if applicable)

<SimpleBind>

Child element <SimpleMatch> (with child element <SimpleEvent type="onenterforward">), and child element <SimpleTask> (with child element <SimpleGo target> assigned to <card onenterforward>.)

<card onterbackward> (or <template onenterbackward>, if applicable)

<SimpleBind>

Child element <SimpleMatch> (with child element <SimpleEvent type="onenterbackward">), and child element <SimpleTask> (with child element <SimpleGo target> assigned to <card onenterbackward>.)

<card ontimer> (or <template ontimer>, if applicable)

SimpleTimer

None

<card> <onevent> (or <template> <onevent>, if applicable)

<SimpleBind> or <SimpleTimer>

If <onevent type="ontimer">, <SimpleTimer> is generated; otherwise, <SimpleBind> with corresponding child element (<SimpleEvent> and <SimpleTask>) is generated

<card> <do> (or <template> <do>, if applicable)

<SimpleBind>

If <do> has a child <go> element, <SimpleBind> with a child element (<SimpleMatch> and <SimpleKey>, <SimpleTask>, <SimpleSubmit> and <SimpleDisplay>) is generated.

If <do> has a child <prev> element, <SimpleBind> with a child element (<SimpleMatch> and <SimpleKey>, <SimpleTask> and <Prev>, and <SimpleDisplay>) is generated.

If <do> has a child <refresh> element, <SimpleBind> with child element (<SimpleMatch> and <SimpleKey>, <SimpleTask>, <SimpleRefresh>, and <SimpleDisplay>) is generated.

<p>

<SimpleForm>, <SimpleMenu>, or <SimpleText>

If <p> has an <input> element or a <select> element with <option value>, <SimpleForm> is generated.

If <p>" has a <select> element with a <option onpick>, <SimpleMenu> is generated; otherwise, <SimpleText> is generated.

<pre>


Only child elements <a>, <anchor>, <do>, <u>, <br>, <b>, <i>, <em>, <strong> are processed.

<input>

SimpleFormItem

Title which normally resides outside the <input> element is copied into <SimpleFormItem>.

<select>

<SimpleMenuItem>, <SimpleHref>, or <SimpleFormOption>

<select> <option onpick> (without <event> child) is translated to <SimpleMenu>, <SimpleMenuItem>.

<select> <option onpick> (with <event> child) is translated to <SimpleMenu>, <SimpleBind>, <SimpleMatch>, <SimpleMenuItem>.

<select value> is translated to <SimpleForm>, <SimpleFormSelect>, <SimpleFormOption>.

<a>

<SimpleHref>

None

<anchor>

<SimpleHref>

If <anchor> has a <go> element, <SimpleHref> is generated.

If <anchor> has a <prev< or <refresh> element, nothing is generated in current release.

<img>

<SimpleImage>

Image adaptation is utilized from 904 release.

<br>

<SimpleBreak>

None

<b>, <strong>, <em>, <big>

<SimpleStrong>

None

<i>, <small>

<SimpleEm>

None

<u>

<SimpleUnderline>

None

<text node>

<SimpleTextItem>

Generally the text node is copied as it is, except for the following two cases:

  • The node is an immediate preceding sibling of <input> or <select>.

  • The node is not the only child of a formatting element (such as <b>, <strong>, <em>, <big>.)

<table>

SimpleTable

None

<tr>

SimpleRow

None

<td>

SimpleCol

None


The WML Translator does not support pass-through mode. Even if the end user's device supports WML, the WML Translator still translates source WML into OracleAS Wireless XML format.

URLs are generally rewritten to point to the Wireless and Voice Portal. However, URLs to images are not rewritten. Image adaptation is utilized in this release. For example, suppose you have the following WML element: 

<img src="/images/yahooicon.wbmp" alt="Yahoo!"/>

It is translated into: 

<SimpleImage src="http://wap.yahoo.com/images/yahooicon.wbmp" vspace="0" hspace="0" valign="bottom" alt="Yahoo!" addImageExtension="auto"/>

This image is adapted and rendered based on end user's device model.

There are certain constraints in current release of WML Translator:

14.7.1 Deploying and Configuring the WML Translator

The WML Translator is deployed as an OracleAS Wireless application. It can be accessed via Commerce->Translator by end users. However, the following three service input parameters are no longer supported: 

ORACLE_SERVICES_COMMERCE_TRANSLATOR_DEFAULT_CONNECTION
ORACLE_SERVICES_COMMERCE_TRANSLATOR_HELPER_WML
ORACLE_SERVICES_COMMERCE_TRANSLATOR_XSL_WML_FILENAME

The following service input parameter is still supported: 

ORACLE_SERVICES_COMMERCE_TRANSLATOR_SHOW_GOHOME

The following service input parameter is added to enhance navigation support: 

ORACLE_SERVICES_TRANSCODER_NAVIGATION

If ORACLE_SERVICES_TRANSCODER_NAVIGATION has a valid value, ORACLE_SERVICES_COMMERCE_TRANSLATOR_SHOW_GOHOME is ignored.

ORACLE_SERVICES_TRANSCODER_NAVIGATION should point to an XML file that can be accessed by either a URL or a file on server's local file system. The XML contains the navigation specification. The following is sample navigation XML: 

<?xml version="1.0" encoding="UTF-8"?>
<Navigation>
<NavigationItems>
<Item target="%value home.url%" 
label="Home"
showAs="Link" 
preferredLocation="Header" />
<Item target="%value service.parent.url%" 
label_prefix="Back"
showAs="Link" />
<Item target="http://www.oraclemobile.com" 
label="OracleMobile" 
showAs="Button" 
preferredLocation="Footer" />
</NavigationItems>
</Navigation>

Each navigation item has the attributes described in Table 14-2.

Table 14-2 Navigation Item Attributes

Attribute Name Meaning Required? Accepted values Default value

target

Where to go

Required

Either a fully-qualified URL, or a placeholder for mobile context, for example, portal home, service home.

NA

label

Label shown to end user

Optional

String

NA

label_prefix

Prefix to the label

Optional

Only meaningful for mobile context, for example, portal home

NA

label_suffix

Suffix to the label

Optional

Only meaningful for mobile context, for example, portal home

NA

showAs

How to show the label

Optional

Menu item, Link, or Button

Button

preferredLocation

Where to show the label

Optional

Header or Footer

Header


Here is the schema for navigation XML: 

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" 
elementFormDefault="qualified" attributeFormDefault="unqualified">
<xs:element name="Navigation">
<xs:complexType>
<xs:all>
<xs:element ref="NavigationItems" minOccurs="0"/>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="NavigationItems">
<xs:complexType>
<xs:sequence>
<xs:element ref="Item" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Item">
<xs:complexType>
<xs:attribute name="target" type="xs:string" use="required"/>
<xs:attribute name="label" type="xs:string" use="optional"/>
<xs:attribute name="label_prefix" type="xs:string" use="optional"/>
<xs:attribute name="label_suffix" type="xs:string" use="optional"/>
<xs:attribute name="showAs" type="xs:string" use="optional"/>
<xs:attribute name="preferredLocation" type="xs:string" use="optional"/>
</xs:complexType>
</xs:element>
</xs:schema>

14.7.2 Using the WML Translator

The WML Translator is deployed as an OracleAS Wireless Application at the URL omp://oracle/services/commerce/translator. The application can be invoked by passing WML source URL in the request parameter XLTORSITE. For example, to invoke www.oraclemobile.com, you can use the following URL in your OracleAS Wireless XML: 

omp://oracle/services/commerce/translator?XLTORSITE=http%3A%2F%2Fwww.oraclemobile.com
  Previous
Previous 		Next
Next
  Go To Documentation Library
Home		Go To Product List
Solution Area		Go To Table Of Contents
Contents		 Go To Index
Index