4 Building Content-Based Portlets with Web Clipping

The Web Clipping provider is a portlet provider to OracleAS Portal. It provides and renders clipped Web content as portlet content through the Web Clipping portlet.

The Web Clipping portlet enables you to collect Web content into a single centralized portlet. You can use it to consolidate content from hundreds of Web sites scattered throughout a large organization.

The Web Clipping portlet supports:

• Navigation through various styles of login mechanisms, including form- and JavaScript-based submission and HTTP Basic and Digest Authentication with cookie-based session management.

• Fuzzy matching of clippings. If a Web clipping gets reordered within the source page or if its character font, size, or style changes, it will still be identified correctly by OracleAS Portal and delivered as the portlet content.

• Reuse of a wide range of Web content, including basic support of pages written with HTML 4.0.1, JavaScript, applets, and plug-in enabled content, retrieved through HTTP GET and POST (form submission).

• Customization, allowing a page designer to make a Web Clipping customizable, exposing input parameters that page viewers can modify when they customize the portlet. In addition, these parameters are mapped to portlet parameters that a page designer can further map to Page Parameters.

• Use of information from external Web sites, including integration with external applications, which enables you to leverage Oracle Application Server Single Sign-On and to clip content from authenticated external Web sites.

• Inline rendering, enabling you to set up your Web Clipping portlets to display links within the context of the portlet. So, when a user clicks a link in your Web Clipping portlet, the results display within the same portlet. You can use this feature with internal and external Web sites.

• Migrate from URL-based portlets, enabling you to migrate to Web Clipping portlets from URL-based portlets. For more information, refer to Portal Center (http://portalcenter.oracle.com).

All Web clipping definitions are stored persistently in the Oracle Application Server infrastructure database. Any secure information, such as passwords, are stored in encrypted form, according to the Data Encryption Standard (DES), using Oracle encryption technology.

Instructions for installing, configuring, and registering the Web Clipping provider are provided in the following file:

http://host:port/portalTools/webClipping/htdocs/help/configuring.webclipping.html

This chapter contains the following sections:

• Adding a Web Clipping Portlet to a Page

• Selecting a Section of a Web Page to Display in the Web Clipping Portlet

• Setting Web Clipping Portlet Properties

• Integrating an External Application into a Portlet

• Example: Adding Portal Center Information to a Page

• Current Limitations for Web Clipping

• Troubleshooting Web Clipping

4.1 Adding a Web Clipping Portlet to a Page

To add Web page content to a portal page, you simply add the Web Clipping portlet to the portal page, use the Web Clipping Studio to navigate to the Web page containing the desired content, then select the portion of the page to clip.

To add a Web Clipping portlet to a page:

1. In the Edit a Page section of the Page Groups portlet, click the Browse Pages icon and select the page to which you want to add the Web Clipping portlet.

   By default, the Page Groups portlet is located on the Build tab of the Portal Builder page. Figure 4-1 shows the page.

   Figure 4-1 Selecting a Page

   
   Description of the illustration pdg_wc_selpage.gif
   
   

2. Click Edit.

3. In the region in which you want to add the Web Clipping portlet, click the Add Portlet icon.

   Figure 4-2 shows a portion of the page.

   Figure 4-2 Adding a Portlet to a Page

   
   Description of the illustration pdg_wc_addportlet.gif
   
   

4. In the Add Portlets page, navigate to the Web Clipping Portlet link and click it.

   By default, the Web Clipping portlet is located in the Portal Content Tools page of the Portlet Repository. If you cannot find this page, use the Search field to find the portlet.

5. Click OK to add a Web Clipping portlet to your page.

   Figure 4-3 shows the Web Clipping portlet added to your page.

   Figure 4-3 Web Clipping Portlet Added to a Page

   
   Description of the illustration pdg_wc_result1.gif
   
   

4.2 Selecting a Section of a Web Page to Display in the Web Clipping Portlet

Using the Web Clipping Studio, you can:

• Browse for Web content

• Section the chosen target page

• Choose the exact portion of the Web content to clip

• Preview the clipped content of the portlet

• Save the clipped content of the portlet

• Set portlet properties and save the updated portlet information

To select a section of a Web page to display in the Web Clipping portlet:

1. Above the Web Clipping portlet, click the Edit Defaults icon, as shown in Figure 4-4.

   Figure 4-4 Editing Default Settings

   
   Description of the illustration pdg_wc_editdef.gif
   
   

   The Find a Web Clipping page is displayed.

2. In the URL Location field, enter the location of the starting Web page that links to the content you want to clip.

3. Click Start, as shown in Figure 4-5.

   Figure 4-5 Specifying a URL

   
   Description of the illustration pdg_wc_url.gif
   
   

   The Web Clipping Studio displays the page you specified. Figure 4-6 shows how the page is displayed in Web Clipping Studio.

   Figure 4-6 Browsing to a Page Containing Content for a Web Clipping

   
   Description of the illustration pdg_wc_url2.gif
   
   

   Note that the URL in the browser bar changes from:

   http://hostname:port/portal/page?_dad=portal&_schema=PORTAL...
   
   

   To:

   http://hostname:port/portalTools/webClipping...
   
   

4. Browse to the page that contains the content you want to clip.

   As you click hyperlinks in the Web page, your navigation links are recorded.

   
   

   Note: Any browsing operations that do not contribute to the eventual Web clipping will be discarded. Only the significant browsing operations are recorded for later playback during the show mode; any discarded links are not visited. For any Web sites that require HTTP Basic or Digest Authentication, a form is displayed that requests user name and password information. This encoded authentication information is recorded as part of the browsing information.

   
   

5. Once you display the page that contains the content you want to clip, in the Web Clipping Studio banner, click Section, as shown in Figure 4-7.

   Figure 4-7 Sectioning the Target Web Page

   
   Description of the illustration pdg_wc_selsect.gif
   
   

   Sectioning divides the target Web page into its clippable sections, as shown in Figure 4-8. After you click Section, you are no longer able to browse links in the displayed page. If you want to continue navigation, click Unsection in the Web Clipping Studio banner.

   Figure 4-8 Sectioned Target Web Page

   
   Description of the illustration pdg_wc_section.gif
   
   

6. At the top left of the section of the Web content you want to clip, click Choose.

   You can choose only one section as a clipping at a time.

   
   

   Note: To increase the number of sections available from which to choose, click Zoom In in the Web Clipping Studio banner. For example, you would click Zoom In to drill down one level of nested tables. To decrease the number of sections available from which to choose, click Zoom Out.

   
   

7. A preview of your chosen section displays. If it is the section you want, click Select in the Web Clipping Studio banner to return to the Web Clipping Portlet Edit Defaults page.

   If you do not want to use the section you clipped in your portlet, click Unselect to return to the page containing the section. You can choose another section on the page, or click Unsection to navigate to another page.

8. In the Find a Web Clipping page, click OK to display the selected Web clipping in the Web Clipping portlet on your page. (You can edit default properties in the page. See Section 4.3 for more information.)

   Figure 4-9 shows the content added to the Web Clipping portlet.

   Figure 4-9 Clipped Content Added to the Web Clipping Portlet on a Portal Page

   
   Description of the illustration pdg_wc_result2.gif
   
   

4.3 Setting Web Clipping Portlet Properties

You can edit various portlet settings to change the appearance of the Web Clipping portlet and to specify how end users can interact with the portlet.

To set Web Clipping portlet properties:

1. Above the Web Clipping portlet, click the Edit Defaults icon.

2. From the URL Rewriting list, choose Inline if you want link targets to be displayed inside the portlet, or choose None if you want link targets to be displayed in a new window.

3. In the Title field, enter a title to display in the portlet banner.

4. In the Description field, enter a description of the portlet.

5. In the Time Out (seconds) field, enter the amount of time (in seconds) for the Web Clipping provider to attempt to contact the Web page from which the content was clipped.

6. In the Expires (minutes) field, enter the amount of time (in minutes) that cached content is valid. Any requests for portlet content that occur within the time period you specify will be satisfied from the cache.

   Once the cache period is exceeded, requests for portlet content will be satisfied by retrieving content from the portlet's Web Clipping data source. The cache will also be refreshed with this content.

7. If you entered any information in a form while clipping content for the Web Clipping portlet, the Parameterize Inputs section is available. Select the Click to start parameterizing check box to customize parameters associated with the Web Clipping portlet content. Then:

   1. From the Parameters list, choose the parameters that you want to customize.

   2. From the Customizable list, select a parameter if you want to allow end users to provide their own values for the parameters when they customize the portlet. Select None if you do not want to allow this.

   3. In the Display Name field, enter a name to be displayed for the parameter.

   4. In the Default Value field, enter a value to use by default for the parameter.

8. Click OK.

4.4 Integrating an External Application into a Portlet

You can integrate an external application into a Web Clipping portlet, leveraging Oracle Application Server Single Sign-On to clip content from authenticated external Web sites. For example, if you have an account with finance.yahoo.com (an external application) that requires a login to access a particular page, you can incorporate clips from that page into a Web Clipping portlet.

To integrate an external application, take the following steps:

1. Set up the external application in OracleAS Portal, specifying the authentication information:

   1. Navigate to the Administer External Applications portlet. (Select the Administer tab, then select the Portlets tab. Select Administration Portlets. Select SSO/OID. Then, select SSO Server Administration. Select Administer External Applications.)

   2. Click Add External Application.

   3. For Application Name, enter a name for the application. For example: username Yahoo.

   4. For Login URL, enter the URL to log into the application, for example, http://login.yahoo.com.

   5. For User Name/ID Field Name, enter your username for the application.

   6. For Password, enter your password for the application. OracleAS Portal uses this information in connecting to the application.

   7. Select POST as the authentication method.

   For more information about OracleAS Server Single Sign-On and External Applications, see the Oracle Application Server Single Sign-On Administrator's Guide.

2. For the Web Clipping portlet, register the external application:

   1. Select the Administer tab, then select the Portlets tab.

   2. Select Register a Provider.

   3. In the Register Provider page, enter webClippingYahoo for the Name and Web Clipping Provider Yahoo for the Display Name. Select Web for the Implementation Style.

   4. Click Next.

   5. In the General Properties section of the Define Connection page, for the URL, enter:

      http://server:port/portalTools/webClipping/providers/webClipping

   6. For the user's identity, select The user's identity needs to be mapped to a different name in the Web provider's application, and/or the Web provider requires an external application login for establishment of a browser session. If selecting this option, specify the external application ID below.

   7. For External Application ID, click the List of Values icon and select the external application you registered for Yahoo.

      Figure 4-10 shows the Define Connections page.

      Figure 4-10 Specifying an External Application for a Web Clipping Provider

      
      Description of the illustration pdg_wc_ea.gif
      
      

   8. Check that the proxy settings are correct. Click Finish.

3. Add a portlet to a page, as described in Section 4.1.

4. Select a section of the finance.yahoo.com section to display in the Web Clipping portlet, as described in Section 4.2.

Now, the clip, even though it is from a page requiring authentication, is available in your portlet.

Note that you can only associate one external application with a provider. For each external application, you must register a new provider.

4.5 Example: Adding Portal Center Information to a Page

In this example, you will use the Web Clipping portlet to add information from Portal Center to a portal page.

This example contains the following exercises:

• Exercise: Adding a Web Clipping Portlet to a Page

• Exercise: Selecting a Clipping in Portal Center

• Exercise: Setting Web Clipping Portlet Properties

4.5.1 Exercise: Adding a Web Clipping Portlet to a Page

Administrators can set up personal pages for all users. This exercise assumes that the administrator has enabled this functionality. In this exercise, you add the Web Clipping portlet to your personal page.

1. In the Edit a Page section of the Page Groups portlet, click the Browse Pages icon.

   By default the Page Groups portlet is located on the Build tab of the Portal Builder page.

2. In the Page Group Map, expand the Personal Pages node, then expand the node for the first letter of your user name. Figure 4-11 shows the Personal Pages node.

   Figure 4-11 Expanding Page Group Map Nodes

   
   Description of the illustration pdg_wc_pgmap.gif
   
   

3. Click Return Object next to your user name. Your personal page is displayed.

4. Click Edit Page.

5. In any portlet region, click the Add Portlets icon.

6. In the Add Portlets page, click the Web Clipping Portlet link.

   By default, the Web Clipping portlet is located in the Portal Tools page of the Portlet Repository. If you cannot find this page, use the Search field to find the portlet.

7. The Web Clipping portlet is added to the Selected Portlets list. Click OK.

4.5.2 Exercise: Selecting a Clipping in Portal Center

In this exercise, you navigate to Portal Center and search for specific information, then select the results as the clipping for your portlet.

1. Above the Web Clipping portlet, click the Edit Defaults icon.

2. In the Web Clipping Studio's Find a Web Clipping page, in the URL Location field, enter http://portalcenter.oracle.com. Click Start.

3. Enter a search string in the Search field on the left of the Portal Center home page, as shown in Figure 4-12. For this exercise, enter Web Clip, then click Go.

   Figure 4-12 Searching for Information on Portal Center Home Page

   
   Description of the illustration pdg_wc_search.gif
   
   

4. The Search result is displayed in the Web Clipping Studio. Click Section. Web Clipping Studio divides the target Web page into its clippable sections, as shown in Figure 4-13

5. Figure 4-13 Sectioning the Target Web Page

   
   Description of the illustration pdg_wc_choose.gif
   
   

6. At the top left corner of the search result, click Choose.

   A preview of the search result section displays.

7. Click Select to confirm that the search result section is the one you want to clip.

8. Click OK to display the selected Web clipping in the Web Clipping portlet on your page. Figure 4-14 shows the Web Clipping displayed in the page.

Figure 4-14 Selected Web Clipping Displayed in Web Clipping Portlet

Description of the illustration pdg_wc_wc2003.gif

4.5.3 Exercise: Setting Web Clipping Portlet Properties

In this exercise, you edit the properties of the Web Clipping portlet to allow end users to display a different product in the portlet.

1. Above the Web Clipping portlet you just added, click the Edit Defaults icon, as shown in Figure 4-15.

   Figure 4-15 Clicking Edit Defaults for the Web Clipping Portlet

   
   Description of the illustration pdg_wc_wc2003ed.gif
   
   

2. In the page that displays, modify the following items in the Properties section:

   • From the URL Rewriting list, choose Inline to specify that you want link targets displayed inside the portlet, rather than in a new window.

   • In the Title field, enter Portal Center Search. This title displays in the banner of your Web Clipping portlet, as well as the pages where users can customize parameters for the Web clipping.

   Figure 4-16 Setting Properties

   
   Description of the illustration pdg_wc_prop.gif
   
   

3. Because the content displayed in the portlet was reached by entering information in the Search field on Portal Center, you can customize the parameters used by the search to allow end users to specify their own search string.

   In the Parameterize Inputs section, select the Click to start parameterizing check box.

4. On the last line of the parameters table, make the following changes:

   • In the Parameters column, choose p_mainsearch from the list.

   • In the Customizable column, choose Param1 from the list.

   • In the Display Name column, enter Portal Center Search.

   • Make sure that Default Value displays Web Clip, to be sure you have selected the right parameter.

   Figure 4-17 shows the Parameterize Inputs section.

5. Figure 4-17 Specifying Parameter for User Input

   
   Description of the illustration pdg_wc_parms.gif
   
   

6. Click OK to display the default search results in the Web Clipping portlet on your page.

7. In the Web Clipping portlet banner, click Customize, as shown in Figure 4-18.

   Figure 4-18 Clicking Customize in the Web Clipping Portlet Banner

   
   Description of the illustration pdg_wc_wc2003cust.gif
   
   

8. In the page that displays, scroll down to the Inputs section. Notice that the parameter field for the search string is labeled Portal Center Search, as you specified for the Display Name for this parameter. In the Portal Center Search field, enter a different search string. For example, enter OmniPortlet 2003, as shown in Figure 4-19.

   Figure 4-19 Specifying Input for Parameters

   
   Description of the illustration pdg_wc_input.gif
   
   

9. Click OK.

   The Web Clipping portlet now displays the results of performing a search on Portal Center for OmniPortlet 2003 information, as shown in Figure 4-20.

   Figure 4-20 New Web Clipping Result Based on Customer Input Parameter

   
   Description of the illustration pdg_wc_newsearch.gif
   
   

4.6 Current Limitations for Web Clipping

This section describes limitations for Release 9.0.4.1 of Web Clipping. For information about the latest features and limitations in a release, be sure to read the Release Notes.

• The export/import utilities of OracleAS Portal, which enable you to move content between portal installations, do not include the export/import of provider customizations. Specifically, the clippings defined for each instance of the Web Clipping portlet are not exported or imported along with the portlet instances. The result is, on the target OracleAS Portal where the portlet instances are imported, the instances will not have their clipping defined.

• If the site to which you are connecting uses a lot of JavaScript to manipulate cookies or uses the JavaScript method document.write to modify the HTML document being written out, you may not be able to clip content from the site.

• Proxy authentication. Web Clipping does not fully support proxy authentication. It supports only global authentication, not per-user authentication. If you use proxy servers to connect to Web sites outside of a firewall and the proxy servers are configured for proxy authentication, you cannot clip a page with the Web Clipping Studio if the page is outside the firewall unless you manually configure proxy authentication. For more information, see Section 4.7.2.2.

4.7 Troubleshooting Web Clipping

This section provides information to help you troubleshoot problems you may encounter while using the Web Clipping provider or Web Clipping Studio.

4.7.1 Checking the Status of the Provider with the Test Page

You can use the Web Clipping Provider Test Page to determine whether or not the provider is functioning properly. To access the Test Page, click Web Clipping Provider from the Portal Tools Application Welcome Page, which is located at:

http://Hostname:Port/portalTools

The Provider Test Page: Web Clipping is displayed. It provides the following information:

• Portlet information: Information about the Web Clipping portlet (The Web Clipping provider contains only one portlet.)

• Provider initialization parameters and values.

• Provider status, with links to pages to edit the configuration.

For more information about using the Test Page, see the "Administering Web Clipping" appendix in the Oracle Application Server Portal Configuration Guide.

4.7.2 Solving Problems with Connections

If you encounter difficulties making or maintaining connections to the site to be clipped or a site that was clipped, note the following:

• If a proxy server is needed to connect to HTTP servers outside of a firewall, make sure that the proxy servers are configured correctly. See Section 4.7.2.1 for more information.

• If the proxy servers are configured for proxy authentication, you will receive HTTP error code 407 when you attempt to clip a page outside the firewall unless you have manually configured proxy authentication. See Section 4.7.2.2 for information about manual configuration.

• If a reverse proxy is used, make sure that the reverse proxy server is configured correctly. See the "Performing Advanced Configuration" chapter of the Oracle Application Server Portal Configuration Guide for information about configuring a reverse proxy.

• If you are attempting to add a clip to a Web Clipping portlet, and experience difficulty making or maintaining connections, the cause may be that the configuration includes a load balancer, but the configuration was not set correctly:

  • If multiple OC4J instances are set up behind a load balancer, the Web Clipping Repository and HTTP proxy must be configured to be identical on all OC4J instances before you join the OC4J instances to the Load Balancer.

    Web clippings have definitions that must be stored persistently in the Web Clipping Repository hosted by an Oracle Database server. In a multiple middle-tier environment, all instances of OC4J must be configured to store definitions in the same repository.

    In addition, all instances of OC4J must have identical configurations for the HTTP proxy.

  • The Load Balancer must be session-enabled. If it is not, the first request connects, but subsequent requests, which may be routed to a different instance, fail.

  For more information about configuring with a load balancer, see the "Performing Advanced Configuration" chapter of the Oracle Application Server Portal Configuration Guide.

• To be sure a URL is correct, test the URL that you want to clip in a browser before you attempt to clip it. Also test the URL from the provider middle tier to be sure that it is accessible from there.

• If you cannot clip a page, make sure that the page is not overpopulated with IFrames. View the page in a browser, looking at the page source. If it contains IFrames, start with the URL pointed to by the IFrame "src" attribute.

• If images in a clipping are not retrieved when the rest of the clipping is retrieved, check your browser proxy settings. Because images are treated as links (using the "src" attribute of the IMG tag), images from clipped sites are served directly from the original sites. If the images required that the proxy setting be enabled during creation of the clipping or Show mode, disabling the browser proxy setting disables viewing of the images in a clipping. Enable the browser proxy setting.

• If you cannot connect, check the error log. (See Section 4.7.3 for information about the error log.) Check to see if the log contains a message about logon to the database being denied. If this is the case, the PORTAL schema password in the infrastructure database may have been modified manually and no longer matches the password stored in Oracle Internet Directory. Refer to the Oracle Internet Directory Administrator's Guide for more information about setting the password.

4.7.2.1 Configuring Proxy Servers

If a proxy server is needed to connect to HTTP servers outside of a firewall, make sure that the proxy servers are configured correctly.

To configure the proxy servers, go to the Web Clipping Provider Test Page, as described in Section 4.7.1. In the Web Clipping Provider Test Page, click Edit in the Actions column of the HTTP Proxy row. In the Edit Provider page, specify the HTTP Proxy Host and the HTTP Proxy Port for the HTTP Proxy.

For access to servers that are inside the firewall, you can specify a list of domain names that do not require going through the firewall by selecting No Proxy for Domains beginning with and entering the URL. You do not need to restart OC4J for the new settings to take effect.

For more information about configuring proxy servers, see the "Administering Web Clipping" appendix of the Oracle Application Server Portal Configuration Guide.

4.7.2.2 Proxy Authentication

Web Clipping does not fully support proxy authentication. It supports global proxy authentication, not per-user authentication. You must manually configure the proxy authentication, by editing the provider.xml file, which is located at:

OracleHome/j2ee/OC4J_Portal/applications/portalTools/webClipping/WEB-INF/providers/ webClipping

In the provider.xml file, specify the following in the proxyInfo section:

• For proxyUser, specify the user name.

• For proxyPassword, specify the password for the user.

• For proxyType, specify the type of authentication, either Basic or Digest.

• For proxyRealm, specify the realm name.

The following shows an example of proxy authentication settings:

<proxyUser>root</proxyUser>
<proxyPassword>HfnkWjuXsWbB</proxyPassword>
<proxyType>Basic</proxyType>
<proxyRealm>webcl</proxyRealm>

4.7.3 Setting Logging Levels

By default, the logging level of Web Clipping is set to level 3, which provides information about configuration, severe errors, and warnings. This is a reasonable level for day-to-day operation. To view information that is useful for debugging, set the logging level to 7.

To set the logging level, edit the web.xml file and specify the level for the oracle.portal.log.LogLevel parameter. The web.xml is located at:

OracleHome/j2ee/OC4J_Portal/applications/portalTools/webClipping/WEB-INF

For example, to set the level to display debugging information, set the value of the parameter oracle.portal.log.LogLevel to 7, as shown in the following example:

<context-param>
        <param-name>oracle.portal.log.LogLevel</param-name>
        <param-value>7</param-value>
</context-param>

Errors that occur when accessing the Test Page or during the execution of the Web Clipping portlet are written to one of the following files:

OC4J_HOME/j2ee/home/application-deployments/portalTools/application.log
IAS_HOME/j2ee/OC4J_instance/application-deployments/portalTools/OC4J_instance_default_island_1/application.log

For information about errors that can occur during configuration of the Web Clipping provider, see:

http://host:port/portalTools/webClipping/htdocs/help/configuring.webclipping.html