This chapter contains the following topics:
Consumer search engines, like Google and Yahoo, search HTML pages. An enterprise search engine, however, must also search databases, e-mail systems, intranet portals, document management systems, and custom applications. Oracle SES ships plug-ins to the most popular of these systems in use today.
Some of the plug-ins shipped with Oracle SES require extra licensing fees. Contact Oracle sales for details.
Individual client libraries may need to be installed (and licensed from the vendor) for some content sources to work. For example, EMC Documentum requires a compatible version of Documentum Foundation Classes (DFC), a Java library, to be installed on the machine running Oracle SES. Oracle SES does not ship with DFC.
See Also:
Oracle Secure Enterprise Search Release Notes for a list of supported platformsOracle SES lets you register an identity plug-in as an interface to any identity management system. Oracle SES provides registered identity plug-ins for Oracle Internet Directory, Active Directory, and other identity management systems. The plug-in that you activate is responsible for all authentication and validation activity in Oracle SES. This is done on the Global Settings - Identity Management Setup page.
See Also:
"Authorization and Authentication" for information about identity plug-insThe following table lists which identity plug-ins are available for each enterprise content source.
Table 5-1 Identity Plug-ins for Enterprise Content Sources
Source Type | Versions Supported | Identity Plug-in |
---|---|---|
EMC Documentum Content Server |
5.3 SP2 |
Active Directory, Oracle Internet Directory, Native |
FileNet Content Engine |
3.5 |
Active Directory |
FileNet Image Services |
4.0 (ISRA 3.2) |
Active Directory, Oracle Internet Directory, Native |
Lotus Notes |
5.0.9, 6.5.4,7.0 |
Active Directory, Oracle Internet Directory, Native |
NTFS |
Windows 2000, Windows 2003 |
Active Directory |
Open Text Livelink |
9.2, 9.5, 9.5.5 |
Active Directory, Native |
Oracle Calendar |
10.1.2 or later |
Oracle Internet Directory |
Oracle Content Database |
Oracle Content Services 10.1.2 or later, Oracle Content Database 10.2 |
Native, Query-time authorization |
Oracle E-Business Suite 11i |
11i |
Native |
Siebel 8 |
8 |
Native |
Microsoft Exchange |
Windows 2000, Windows 2003 |
Active Directory |
Tip:
"Re-registering Preinstalled Identity Plug-ins" for a list of identity plug-ins native to enterprise content sourcesDocumentum data is stored in DocBases, which can contain cabinets and folders. A Documentum Content Server instance can have one or more DocBases crawled with an EMC Documentum Content Server source. The Documentum Content Server source navigates through the DocBases and the inline cabinets to crawl all the documents in Documentum Content Sever. Oracle SES creates an index, stores the metadata, and accesses information in Oracle SES to provide search according to the end user permissions.
Oracle SES supports incremental crawling; that is, it crawls and indexes only those documents that have changed after the most recent crawling was scheduled. A document is re-crawled if either the content or metadata or the direct security access information of the document has changed. A document is also re-crawled if it is moved within Documentum Content Server and the end user has to access the same document with a different URL. Documents deleted from a DocBase will be removed from the index during incremental crawling.
The admin account of a DocBase should be used by the Documentum source in Oracle SES for crawling and indexing documents of that DocBase.
Documentum Content Server DA (Documentum Administrator) or Documentum Content Server WebTop application must be installed and configured.
Documentum Foundation Classes (DFC) must be installed on the server running Oracle SES.
Because EMC Documentum Content Server software is not included with Oracle SES, certain files must be copied manually into Oracle SES.
The DFC installation asks for destination directory and user directory. For Windows, the default destination directory is C:\Program Files\Documentum
and default user directory is C:\Documentum
. For UNIX, it is a prerequisite to create DFC program root and DFC user root. For example, DFC program root can be <USER HOME>/documentum_shared
and DFC user root can be <USER HOME>/documentum
.
Copy the dfc.properties
and DFC jar files from the following locations into ORACLE_HOME/search/lib/plugins/dcs
.
dfc.jar
Windows: <DFC destination directory>/shared/
UNIX: <DFC destination directory>/dfc
dfcbase.jar
Windows: <DFC destination directory>/shared/
UNIX: <DFC destination directory>/dfc
dfc.properties
Windows: <DFC user directory>/config/
UNIX: <DFC user directory>/config/
For Windows 2003 Server, copy dmcl40.dll
from <DFC destination directory>/shared/
to ORACLE_HOME/bin
.
For UNIX, copy libdmcl40.so
from <DFC destination directory>/dfc
to ORACLE_HOME/lib
.
Note:
The environment variable$DOCUMENTUM_SHARED
(DFC Program root) and $DOCUMENTUM
(DFC user directory) must be created before installing DFC on UNIX. See the DFC installation guide for more information.Push the DCS libraries to global libraries by adding the following lines to the oc4j/j2ee/OC4J_SEARCH/config/application.xml
file:
<library path="../../../../search/lib/plugins/dcs/dfcbase.jar" /> <library path="../../../../search/lib/plugins/dcs/dfc.jar" /> <library path="../../../../search/lib/plugins/dcs" /> <library path="../../../../search/lib/log4j.jar" />
This assumes that the directory search/lib/plugins/dcs
contains the Documentum Server configuration file dfc.properties
.
Activate the identity plug-in on the Global Settings - Identity Management Setup page. Select Oracle Internet Directory identity plug-in and click Activate.
Enter values for the following parameters:
For Authentication Attribute, select nickname.
For Host name, enter the host name of the machine where Oracle Internet Directory is running.
For Port, enter the value 389 (the default LDAP port number).
For Use SSL, enter true or false.
For Realm, enter the Oracle Internet Directory realm; for example, dc=us,dc=oracle,dc=com.
For User name, enter the Oracle Internet Directory Administrator user name; for example, cn=orcladmin.
For Password, enter the password for the user name.
Compatible version of Documentum Foundation Classes (DFC) must be installed on the machine running Oracle SES.
Import users/groups from Oracle Internet Directory to Documentum. First, create an LDAP Configuration Object in Documentum Administrator (DA):
Login to DA.
Navigate to Administration - User Management - LDAP.
Click File - New - LDAP Configuration Object.
For Name, enter a name for the ldap configuration object.
For User Subtype, select dm_user.
For Communication Mode, select Regular.
For Import, select Users and Groups.
Use this configuration object in the server field select Default Configuration Object.
Click Next.
For Directory Type, select Oracle Internet Directory Server.
For Bind Type, select Bind by Searching for Distinguished Name.
For Binding Name, enter the Administrator user name of Oracle Internet Directory, normally cn=orcladmin.
For Binding Password, enter the Administrator password of Oracle Internet Director.
For Host Name, enter the Oracle Internet Directory host name.
For Port, it shows the default value 389 (the default port number of Oracle Internet Directory).
For Person Object Class, enter the information of Base Person Object, typically the value is inetOrgPerson.
For Person Search Base, enter the person search base defined in Oracle Internet Directory; for example, dc=Users,dc=us,dc=oracle,dc=com.
For Person Search Filter, specify the cn=*.
For Group Object Class, enter the Group Object; typically, its value is groupOfUniqueNames.
For Group Search Base, enter the Group Search base defined in Oracle Internet Directory; for example, cn=Groups,dc=us,dc=oracle,dc=com.
For Group Search Filter, specify the cn=*.
Click Next.
Attribute Map information is displayed. Click Finish.
Run the LDAP_Synchronization job:
Login to DA.
Navigate to Administration - Job Management - Jobs.
Open the job dm_LDAPsynchronization.
For state, select Active.
Check the Deactivate On Failure check box.
For Designated Server, select the host name of Documentum Server.
Check the Run After Update check box.
Go to the Schedule tab.
For Start Date And Time, set the current date and time.
Select Repeat time from the Repeat list.
Set Frequency to any numeric value.
Select the End Date And Time radio button and specify how long the synchronization job should run.
Go to the Method tab.
Check the Pass Standard Argument check box.
Go to the SysObject info tab.
Click OK.
Add permission to each folder and file to make them accessible by the search user. (Adding permissions to a folder automatically adds the same permissions to all files and sub-folders in the folder.) The following steps create a permission set and assign users/groups to that set. The same permission is assigned to documents. If the documents are not stamped with permission, then it won't get crawled.
Create Access Control Lists (ACLs):
Login to DA.
Navigate to Administration - Security.
In the File menu click File - New - Permission set.
For Name, enter a name for the permission set.
Click Next.
Click Add to add more users/groups to the permission set.
Select LDAP users/groups that are to made a part of the permission set and move them to the right frame using the arrow keys. Click OK.
Click Finish.
Assign ACLs to documents:
Login to DA.
Navigate to the document where the permission set is to be applied.
Select the Properties icon of this document.
Go to the Permissions tab.
Click Select in front of Permission set name.
Search and select the permission set to be applied to the document.
Click OK.
It is important that the users/groups in the permission sets that are applied to the documents are LDAP users/groups. Those documents that do not have permission sets with LDAP users/groups will not be crawled.
Create an EMC Documentum Content Server source on the Home - Sources page. Select EMC Documentum Content Server from the Source Type list, and click Create. Enter values for the following parameters:
User name: Enter the user name of a valid Documentum Content Server user. The user should be an administrator user or a user who has access to all cabinets/folders and documents of the DocBases configured in the Container name parameter. The user should be able to retrieve content, metadata, and ACL from cabinets, folders, documents and other custom sub classes of all DocBases configured in Container name parameter. This is a required parameter.
Password: Password of the Documentum user. This is a required parameter.
Container name: The names of the containers to be crawled by Oracle SES. You can crawl an entire Documentum DocBase or a specific cabinet/folder. The format is <DocBase Name>/<Cabinet Name>/<Folder Name>/<Sub Folder Name>. Multiple comma-delimited container names can be entered. This parameter is case-sensitive; hence, the same cabinet name as in Documentum repository should be entered. This is a required parameter. For example:
Container name: DocBase1: The entire DocBase1 will be crawled.
Container name: DocBase2/Cabinet21: Cabinet21 and its sub-folders within DocBase2 will be crawled.
Container name: DocBase2/Cabinet21/Folder11: Folder11 and its sub-folders will be crawled.
Container name: DocBase1, DocBase2/Cabinet21/Folder11: The entire DocBase1 and Folder 11 in DocBase2/Cabinet21 will be crawled.
Crawl folder attributes: Indicate whether folder attributes need to be crawled, either true or false. This is an optional parameter. The default value is false. If any other value is provided, it is assumed to be false.
Crawl versions: Indicate whether multiple versions of documents should be crawled, either true or false. This is an optional parameter. The default value is false. If any other value is provided, it is assumed to be false and only the latest versions of a document will be crawled.
Attribute list: The comma-delimited list of Documentum attributes along with their data types to be searchable. The format is <Attribute Name>:<Attribute Type>, <Attribute Name:Attribute Type>. Valid values are String, Number, and Date.
Table 5-2 Documentum Data Type Mapping
Sr. No | Documentum Data Type | Oracle SES Data Type |
---|---|---|
1 |
Boolean |
Number |
2 |
Integer |
Number |
3 |
String |
String |
4 |
ID |
String |
5 |
Time or Date |
Date |
6 |
Double |
Number |
While crawling a DocBase, an attribute is indexed only if both name and type match the configured name and type; otherwise, it is ignored. This is an optional parameter. For example: To make the following Documentum attributes searchable:
Attribute Name: Account Name Attribute Type: String
Attribute Name: Account Id Attribute Type: Integer
Attribute Name: Creation Date Attribute Type: Date
The value of Attribute list should be the following:
Account Name: String, Account ID: Number, Creation Date:Date
The default searchable attributes for Documentum Content Server are Modified Date, Title, and Author.
Multiple attributes with same name are not allowed. For example, Emp_ID:String, Emp_ID:Number
URL for Viewing the Document: A valid URL for Documentum WebTop or DA application used for viewing the Oracle SES search results. For example, http://<IP address>:<Port No>/da or http://<IP address>:<Port No>/webtop.
Authentication Attribute: This parameter is used to set ACLs. This parameter lets you set multiple LDAP servers. If Oracle SES and Documentum Content Server are synchronized with Active Directory, then enter the value USER_NAME
. If Oracle Internet Directory is used, then enter nickname
.
FileNet Content Engine data is stored in object stores, which can be further contained inside folders on a server. A FileNet Content Engine instance can have one or more object stores that can be crawled by specifying the Object Store details in the Container name parameter in Oracle SES. The Content Engine source navigates the object store to crawl all the documents in the configured Content Engine Object Store. It stores the metadata and accesses information in Oracle SES to provide search according to the end user permissions.
Any user having administrative privileges can be used to access FileNet Content Engine Crawler plug-in for crawling and indexing documents.
Because FileNet Content Engine software is not included with Oracle SES, certain files must be copied manually into Oracle SES:
Copy javaapi.jar
, soap.jar
, xercesImpl.jar
and xml-apis.jar
files from <FileNet installed Folder>/Workplace/WEB-INF/lib
to ORACLE_HOME/search/lib/plugins/fnetce
.
Copy the WCMConfig.properties
file from <FileNet installed Folder>/Workplace/WEB-INF
, into ORACLE_HOME/search/lib/plugins/fnetce.
If any of the parameters are updated after initial crawl, then you must update the crawler re-crawl policy to Process All Documents on the Home - Schedules - Edit Schedules page, and re-crawl the source.
If additional document types are configured after first time crawl, these document types are not indexed on subsequent re-crawls. Same is the case if Document Size parameter is changed after first crawl, for example if the Document Size was 10 MB at the time of first crawl and it is changed to 20 MB before re-crawl, documents greater than 10 MB are be rejected. Workaround is to create the source again and then make the changes.
If a FileNet Content Engine source is used, Oracle recommends that Active Directory be used as identity management system for the Oracle SES instance. The Active Directory instance must be the same one that FileNet Content Engine is using to authenticate users on the file system.
See Also:
"Activating an Identity Plug-in" for information on activating the Active Directory identity plug-inCreate a FileNet Content Engine source on the Home - Sources page. Select FileNet Content Engine from the Source Type list, and click Create. Enter values for the following parameters:
User name: A valid FileNet Content Engine user. The user should be an Administrator user or a user who has access to all Folders and Documents present in the configured container. The user should be able to retrieve content, metadata, and ACL from folders, documents of all containers configured in Container name parameter. This is a required parameter.
Password: Password of the Content Engine user. This is a required parameter.
Container name: The names of the containers to be crawled by Oracle SES. You can crawl a complete objectstore or a specific Folder. The format for specifying container is <ObjectStore>/<Folder Name>/<Sub Folder Name>. Multiple comma-delimited containers can be specified. This is a required parameter. For example:
Container name: ObjectStore1: The entire ObjectStore1 will be crawled.
Container name: ObjectStore1/Folder1/Folder12: The documents inside Folder12 and its sub-folders will be crawled.
Container name: ObjectStore1, ObjectStore2/Folder1/Folder12: The entire ObjectStore1 and contents of Folder12 in ObjectStore2 will be crawled.
Attribute list: Attribute list corresponds to the comma-delimited list of Content Engine attributes along with their data types that the administrator wants to be searchable. The format is <Attribute Name>:<Attribute Type>, <Attribute Name:Attribute Type>. The valid values are String, Number, and Date.
Table 5-3 FileNet Content Engine Data Type Mapping
Sr. No | FileNet Content Engine Data Type | Oracle SES Data Type |
---|---|---|
1 |
Boolean |
String |
2 |
float, int, byte, and other numeric values |
Number (Big Decimal) |
3 |
String |
String |
4 |
DateTime, Date |
Date |
5 |
Others |
String |
While crawling from object store an attribute will be indexed only if a valid attribute name and data type should be matched with the configured name and type, else it will be ignored. This is an optional parameter. For example, to make the following Content Engine attributes searchable:
Attribute Name: DocumentTitle Attribute Type: String
Attribute Name: Id Attribute Type: Number
Attribute Name: DateCreated Attribute Type: Date
The value of Attribute List should be: Document Title: String, Id: Number, DateCreated: Date
The default searchable attributes for FileNet Content Engine are Title, Author, and Last Modified Date. Multiple attributes with same name are not allowed. For example: Emp_ID: String, Emp_ID: Number is not allowed.
Crawl versions: Indicate multiple versions of documents to be crawled with true. By default, this value is false; that is, only the latest version of documents will be crawled. If any value other than true is specified, it is considered false.
URL for viewing the documents: The URL for FileNet Workplace application used for viewing the search results. Workplace is a part of FileNet P8 AE. For example: http://<IP address> :< Port No.>/Workplace
Remove deleted documents from index: This parameter determines whether documents deleted from CE object stores should be removed from the index as well, either true or false. The default value is false, as this would be a costly operation in terms of performance. If any value other than true is specified, it is considered false.
Crawl folder attributes: Specify whether or not folder metadata should be indexed, either true or false. The default value is false. Any other value for this parameter is considered false.
Documents in FileNet Images Services are organized into Folders. A FileNet Image Services source navigates through the folder hierarchy to crawl all documents in FileNet Image Services (IS). Oracle SES creates the index and stores the metadata of the documents retrieved from FileNet Images Services in Oracle SES to provide search according to the end users' permissions.
A FileNet Image Server instance can have one or more Libraries. A Library is the document repository and contains documents within Folders and sub-Folders. A FileNet Image Services source can crawl multiple Libraries.
Images stored in Image Services can have annotations. Some of the annotations contain text, and these annotations will be crawled. The annotations crawled are:
Stamp
Transparent Text
Stick note
You can search on the content of these annotations after the IS library has been crawled.
A user belonging to IS SysAdmin group should be used to crawl documents and metadata in IS.
FileNet Image Services Server version 4.0 or 3.6 SP2
Image Services Resources Adapter version 3.2.1
Because FileNet Image Services software is not included with Oracle SES, certain tasks must be performed manually to integrate with Oracle SES:
Deploy the ISCrawlerWeb.war
file in the same application server on which ISRA has been deployed.
For application servers that require context root to be specified while deploying a WAR file, specify Context Root as ISCrawlerWeb
.
If the application server is WebSphere Application Server, then activate URL rewriting: Click Servers - Application Servers - name of the server - Web Container - Session Management - Enable URL Rewriting.
If additional document types are configured after the first crawl, these document types are not indexed on subsequent re-crawls. The same applies if the Document Size parameter is changed after first crawl. For example, Document Size was 10 MB at the time of first crawl and it is changed to 20 MB before re-crawl, then documents with greater than 10 MB are rejected. As a workaround: update the crawler re-crawl policy to Process All Documents on the Home - Schedules - Edit Schedules page, and re-crawl the source.
XML documents are crawled by default without configuring the source for XML documents: Oracle SES provides an option of configuring the documents types, including XML, to be crawled. Currently, even if XML document type is not configured, XML documents still are crawled.
Activate the identity plug-in on the Global Settings - Identity Management Setup page.
Configure Oracle SES to Active Directory:
On the Global Settings - Identity Management Setup page, click Register new Identity Plug-in.
For Plug-in Manager Class Name, enter oracle.search.plugin.security.idm.IdentityPluginManagerADImpl
For Plug-in Manager Jar File Name, enter idm/idmPlugin.jar
.
Click Finish.
Select the radio button for The Active Directory Identity Plug-in Manager implemented based on Oracle User & Role API and click Activate.
For Authentication Attribute, select USER_NAME.
For Directory URL, enter the host name and port number; for example, ldap://ldapserverhost:port.
For Directory account name, enter the Active Directory user; for example, Administrator.
For Directory account password enter the password of the Active Directory user.
For Directory subscriber, enter the Active Directory information like Directory subscriber (ldap base) like 'dc=us,dc=oracle,dc=com'.
For Directory security protocol, enter none or the port number.
Click Finish.
Configure the identity plug-in for Image Services
Go to the Global Settings - Identity Management Setup page in Oracle SES.
Create a new directory under [oracle_home]/product/[version]/ [SES Instance Name]/search/lib/plugin/Identity/ for example IdentityPlugin_folder.
Copy the FileNet Image Services identity plug-in jar to that folder.
Click Register new Identity Plug-in.
For Plug-in Manager Class Name, enter oracle.search.plugin.security.identity.fnis.FNISIdentityPluginManager
For Plug-in Manager Jar File Name, enter identity/fnis/FNISIdentityPlugin.jar.
Click Finish.
Select the radio button for The Image Services Identity Plug-in Manager implemented based on Oracle User & Role API and click Activate.
For Authentication Attribute, select NATIVE.
For Web Component URL enter the host name and port number of the Web component URL; for example, http://webserverhost:port/ISCrawlerWeb.
For Administrator user name, enter Image Services user name.
For Administrator password, enter the password of the Image Services user.
For Library name of IS Server, enter the name of the Image Services library like 'ISCF'. Library Name is the ISRA connection factory name that is created when ISRA is deployed.
Click Finish.
Image Services Resources Adapter (ISRA) must be deployed on a supported application server. See the ISRA documentation for supported application servers.
Connection Factory must be created for ISRA, the connection factory should be configured for the target IS libraries. See the ISRA documentation for deployment instructions.
ISRA comes with a viewer application for viewing images and annotations, the FNImageViewer.ear
application should be deployed on the same application server as ISRA. This viewer would be invoked to display images for example jpeg, tiff, bmp, gif, and annotations. See the ISRA documentation for deployment instructions.
To support secure search, the Image Services server must be synchronized with the Active Directory server. See the section 'LDAP configuration' in ISRA deployment guides for importing Microsoft Active Directory users/groups to Image Services.
After Active Directory users/groups have been imported into Image Services, ISRA must be configured to authenticate with Active Directory. See the section 'LDAP configuration' in ISRA deployment guide for details.
Create a FileNet Image Services source on the Home - Sources page. Select FileNet Image Services from the Source Type list, and click Create. Enter values for the following parameters:
User name: Enter the user name of a valid FileNet Image Services user. The user should be a SysAdmin user or a user who has access to all Folders and Documents of the Libraries configured in the Container name parameter. The user should be able to retrieve content, metadata and ACL from folders, documents and other custom sub classes. The user should be defined in the configured LDAP server and should be imported into IS. This is a required parameter.
Password: The FileNet Image Services user password. This is a required parameter.
Container name: The names of the containers to be crawled by Oracle SES. You can crawl an entire FileNet Image Services Library or a specific Folder. The format is <Library Name>/<Folder Name>/<Sub Folder Name>(cache name). Library name is the ISRA connection factory name created when ISRA is deployed. Cache name is in which the document content can be found. Multiple comma-delimited container names can be entered. This is a required parameter. For example:
Container Name: LibraryName1(cache name): The entire LibraryName1 will be crawled
Container Name: LibraryName2/Folder1/(cache name): Folder1 and its sub-folders will be crawled.
Container Name: LibraryName1, LibraryName2/Folder1(cache name): The entire LibraryName1 and Folder 1 in LibraryName2 will be crawled
Cache name: The format is cache name:DomainName:Oraganization. This is an optional parameter, if the cache name is not provided the plug-in tries to retrieve document content from the default page cache. However, the plug-in throws an error if an invalid page cache or empty brackets () is specified. Ask IS administrator for cache details.
Attribute names: The comma-delimited list of Image Services attributes along with their data types to search. The format is <Attribute Name> :<Attribute Type>, <Attribute Name: Attribute Type>. Valid values are String, Number, and Date.
Table 5-4 FileNet Image Services Data Type Mapping
Sr. No | FileNet Image Services Data Type | Oracle SES Data Type |
---|---|---|
1 |
BOOLEAN |
String |
2 |
BYTE |
Number |
3 |
UNSBYTE |
Number |
4 |
SHORT |
Number |
5 |
UNSSHORT |
Number |
6 |
LONG |
Number |
7 |
UNSLONG |
Number |
8 |
ASCII |
String |
9 |
TIME |
Date |
10 |
DATE |
Date |
11 |
MENU |
Number |
12 |
FP_NUM |
Number |
While crawling a Library an attribute will be indexed only if both name and type of the attribute in the library match the configured name and type; otherwise, it is ignored. This is an optional parameter. For example, to make the following FileNet Image Services attributes searchable:
Attribute Name: Account Name Attribute Type: String
Attribute Name: Account Id Attribute Type: Integer
Attribute Name: Creation Date Attribute Type: Date
The value of Attribute List should be
Account Name: String, Account Id: Number, Creation Date: Date
Set source hierarchy: Indicate whether the source should set the source hierarchy of the document, either true or false. The default value is false. If any other value is provided, it is assumed to be false.
A document in Image Services can be filed in multiple folders, it is possible that a user could have READ permissions on a document but not on all the folders in which the document is filed. If Set Source Hierarchy is 'True', then there is a possibility that a user could view a source hierarchy on which he does not have permissions in IS. However, he would not be able to view the documents on which he does not have READ permissions.
Web component URL: The URL of J2EE application server where the crawler plug-in Web component module is deployed. The format of the URL is http://<host name> :< Port Number>. This is a required parameter.
The Web component is also used to view the search results, on clicking an Oracle SES search result the user is prompted for login. On successful login, the document is displayed. To view images and annotations the FileNet Image viewer FNImageViewer.ear
should be deployed. FNImageViewer.ear
is a part of ISRA CD. If the viewer is not deployed, the images will be displayed in native viewer or the user is prompted to download the document.
Set public access: Indicate whether the source should set the public access of the documents whose ACL is Anyone, either true or false. The default value is false. If any other value is provided, it is assumed to be false.
Authentication attribute: This parameter is used to get the LDAP authentication attribute. This parameter will vary based on the identity plug-in used for authentication. For Microsoft Active Directory, it should be USER_NAME
. For FileNet Image Services identity plug-in, it should be NATIVE
.
Lotus Notes data is stored in notes-databases, which can be further contained inside directories on a server. A Lotus Domino Server instance can have one or more databases that can be crawled using the Lotus Notes source. The Lotus Notes source navigates through the databases to crawl all the documents in the specified databases. It stores the metadata, and accesses information in Oracle SES to provide search according to the end users' credentials.
The Lotus Notes source supports incremental crawling; that is, it crawls and indexes only those documents that have changed after recent most crawling was scheduled. A document is re-crawled if either the content, metadata, display URL or the direct security access information of the document has changed. Documents deleted from a database will be removed from the index during incremental crawling.
The user-account used to crawl Lotus Notes databases should preferably be an Administrator account, such that it has access on all databases and is able to retrieve and crawl all documents in the specified databases.
The following tasks must be performed before installing the Lotus Notes source:
HTTP and DIIOP tasks must be running on Domino Server.
If the Active Directory identity plug-in is used, then the users and user-groups in the Domino Directory must be synchronized with Active Directory. While using the Active Directory identity plug-in, the short-name in the Lotus Notes person document is used for validating the user in Active Directory, so it should be a resolvable logon name in Active Directory.
Configure the server document:
Open the server document on the Lotus Notes server that needs to be crawled.
On the Configuration page, expand the server section.
On the Security page, in the Programmability Restrictions area, specify the appropriate security restrictions for your environment in the following fields:
Run restricted Lotus Script/Java agents
Run restricted Java/Javascript/COM
Run unrestricted Java/Javascript/COM
For example, you might specify an asterisk (*) to allow unrestricted access by Lotus Script/Java agents, and specify user names that are registered in the Domino Directory for the Java/Javascript/COM restrictions.
Note:
The crawler that you configure to crawl this server with the DIIOP protocol must be able to use the user names that you specify in these fields.Open the Internet Protocol page, then open the HTTP page, and set the Allow HTTP Clients to Browse Database option to Yes.
Configure the user document:
Open the user document on the Lotus Notes server that needs to be crawled. This document is stored in the Domino directory.
On the Basics page, for Internet password, specify a password.
Restart the DIIOP task on the server.
A Lotus Notes source does not index encrypt fields, and the content of attachments with encrypted documents, for searching. With encrypted documents, the URL of the search result launches the Notes document in place of the attachment file, which is the case when non-encrypted documents are crawled.
Oracle SES currently does not support crawling inside specific folders/views of the Notes custom-applications or mail-databases.
Oracle SES currently launches the search result documents on the Web browser only and does not yet support the launch for Notes thick client.
A user cannot login through the Oracle SES search page, when working with Lotus Notes Release 6 identity plug-in. However, this scenario works fine when using Active Directory plug-in.
During source configuration, if you enter multiple attributes with the same name, the crawler considers the first attribute and ignores the others with the same name.
Activate an identity plug-in on the Global Settings - Identity Management Setup page. Select the identity plug-in for Microsoft Active Directory click Activate.
The users/groups on Active Directory can be synchronized with Lotus Domino Directory such that all users/groups in Active Directory get registered in Domino as well. Thus, any ACL entry in a notes database or notes document can be validated in Active Directory also, and vice versa.
Oracle SES also provides a Lotus Notes identity plug-in so the Lotus Domino Directory can be used to authenticate and validate the notes native users and groups in Oracle SES. To use the Lotus Notes identity plug-in:
Register the Lotus Notes identity plug-in by providing the following parameters:
Plug-in Manager Class Name = oracle.search.plugin.security.identity.ln.LNIdentityPluginManager
Plug-in Manager Jar File Name = <lotus notes identity plug-in folder>/ ln/LNIdentityPlugin.jar
Activate the Lotus Notes identity plug-in with the following parameters:
Server name: The Domino server fully qualified host name/IP address. If the HTTP port on the Domino server is not 80, then the host name should be "<server-name> :< HTTP port number>".
User name: Enter user name of a valid Lotus Domino Server user. This is a required parameter.
Password: Internet password of the Lotus Notes user. This is a required parameter.
Create a Lotus Notes source on the Home - Sources page. Select Lotus Notes from the Source Type list, and click Create. Enter values for the following parameters:
User name: Enter the user name of a valid Lotus Domino Server user. The user should be an Administrator user or a user who has access to all Folders and Documents of the databases configured in the Container name parameter. The user should be able to retrieve content, metadata, and ACL from documents of all databases configured in Container name parameter. This is a required parameter.
Password: Internet password of the Lotus Notes user. This is a required parameter.
Container name: The comma-delimited names of the containers to be crawled by Oracle SES. These containers could be one or many specific databases or Directory-names if all databases in the particular directories need to be crawled. Multiple database or directory names should be separated by a comma. This is a required parameter.
Attribute list: The comma-delimited list of Lotus Notes attributes along with their data types to search. The format is <Attribute Name> :< Attribute Type>, <Attribute Name: Attribute Type>. The valid values are String, Number, and Date.
Table 5-5 Lotus Notes Data Type Mapping
Sr. No | Lotus Notes Data Type | Oracle SES Data Type |
---|---|---|
1 |
Boolean |
String |
2 |
Integer |
Number (Big Decimal) |
3 |
String |
String |
4 |
Date |
Date |
While crawling a database, an attribute is indexed only if both name and type match the configured name and type; otherwise, it is ignored. This is an optional parameter.
The default searchable attributes for Lotus Domino Server are Modified Date, Title, and Author. Multiple attributes with same name are not allowed.
Server name: The Domino server fully qualified host name/IP address. If the HTTP port on the Domino server is not 80, then the host name should be "<server-name> :< HTTP port number>". This is a required parameter.
Crawl public documents: Indicate whether the public documents on notes databases need to be crawled such that they are available to anonymous users in Oracle SES, either true or false. This is a required parameter.
Authentication attribute: The attribute used to validate the ACL. With Active Directory identity plug-in, the value should be USER_NAME
. With the Lotus Notes identity plug-in, the value should be NATIVE
. This is a required parameter.
Mail template name: This parameter is specific to the mail-databases and the mail template's name should be specified here if any/all of the databases being crawled are mail databases. This is a mandatory parameter if either the Past Days or Future Days parameter is specified.
Past days: If the user is crawling calendar entries, then this parameter specifies the number of days in the past for which the calendar entries are picked. The date of reference here is the start date of the event. This accounts for the number of days in the past, and it does not filter the search by time.
Future Days: If the user is crawling calendar entries, then this parameter specifies the number of days in the future for which the calendar entries are picked. The date of reference here is the end date of the event. This accounts for the number of days in the future, and it does not filter the search by time.
Notes title: Because in Lotus Notes custom applications it is not mandatory to maintain a Title field, this parameter has been provided where the administrator can specify those text fields that should be parsed to retrieve the title field. In case of multiple field names, the first field available on the document would be picked for the title. This is a required parameter.
This section includes information for Windows NT File System (NTFS) source on Windows. There is a separate source type for NTFS on UNIX.
The NTFS connector enables Oracle SES to search file repositories in Microsoft NTFS. An Oracle SES NTFS source collects the content, metadata attributes and ACLs of files in NTFS. An NTFS source supports incremental crawl. After the initial crawl is performed, subsequent crawls only collect those documents that have changed since the last crawl. A document is re-crawled if the content, metadata, or the ACL information of the document has changed. A file is also re-crawled if it is moved between folders. Files deleted from NTFS are removed from the index during incremental crawls.
The operating system user running the Oracle SES instance must have read permission on the NTFS file share being crawled. For example, if the remote file share \\machine1\share1\directory1\ is crawled by the NTFS source, then the SES instance must be run as a domain user who has access to the file share.
If you get the ACL in the form <encrypted acl>@domain for a folder on a remote machine, it probably means that the machine running the Oracle SES instance and the remote machine are on different domains and your machine cannot interpret the ACLs appropriately.
Windows .NET Framework 2.0
Microsoft Developer Support OLE File Property Reader (dsofile)
If not already installed, download and install the Windows .Net 2.0 Framework:
If not already installed, download and install Microsoft Developer Support OLE File Property Reader.
Register dsofile.dll
in the Windows operating system using regsvr32.exe
.
The Oracle SES process needs to be run as domain administrator to crawl remote machines on the domain This is an important pre-requisite to crawl the remote machines for NTFS. Follow these steps to run Oracle SES process as the domain administrator:
Navigate to Control Panel - Administrative Tools - Services.
Select the process OracleService<db sid>
.
Stop this process.
Right click and select Properties.
Select the Log on tab.
Select the option This account, and enter the domain administrator name and password.
Start this process.
Note:
If the Oracle SES instance fails to start after the preceding change, then follow these steps:Navigate to the $ORACLE_HOME/NETWORK/ADMIN
directory.
Edit sqlnet.ora
by changing SQLNET.AUTHENTICATION_SERVICES=(NTS) to SQLNET.AUTHENTICATION_SERVICES=(NONE)
.
If an NTFS source is used, Oracle recommends that Active Directory be used as identity management system for the Oracle SES instance. The Active Directory instance must be the same one that NTFS is using to authenticate users on the file system.
For the Oracle SES instance to read the files during crawling, add permission to each folder and file to make them accessible by the operating system user that runs the Oracle SES instance. (Adding permissions to a folder will automatically add the same permissions to all the files and sub-folders in the folder.)
See Also:
"Activating an Identity Plug-in" for information on activating the Active Directory identity plug-inCreate an NTFS source on the Home - Sources page. Select NTFS from the Source Type list, and click Create. Enter the values for the following parameters:
Suppose you want to crawl \\myserver\test1
and \\myserver\test2
on an NTFS box. Specify the UNC PATH as follows: \\myserver\test1
and \\myserver\test2
. The domain user must have read privileges on the shared folders.
Use boundary rules on the NTFS source to restrict the Oracle SES crawler to URLs that match the indicated rules. This is set on the Home - Sources - Boundary Rules page.
For simple rules, Oracle SES supports the *, ^, and $ special characters:
SIMPLE_INC <simple boundary rule string>
SIMPLE_EXC <simple boundary rule string>
This is a set of user-friendly, simplified regular expression rules. Specify an inclusion rule that a URL contain, start with, or end with a term. Use an asterisk (*) to represents a wildcard. Use a caret (^) to denote the beginning of a URL, and use a dollar sign ($) to denote the end of a URL. For example:
^https://*.oracle.com/ .jpg$
For regexp rules, Oracle SES supports all regexp patterns:
REGEXP_INC <regular expression boundary rule string>
REGEXP_EXC <regular expression boundary rule string>
This is a set of regular expression rules using the java.util.regex package.
For example:
^https://.*\.oracle(?:corp){0,1}\.com
For any of these parameters, you can specify up to 50 rules. Use a semi-colon to separate strings and specify multiple rules. For example:
/^https://.*\.oracle(?:corp){0,1}\.com;^https://*.oracle.com/;https://*.oracle.com/*/
This section includes information for Windows NT File System (NTFS) source on UNIX. NTFS sources for UNIX have additional setup steps not required on Windows.
An NTFS source collects the content, metadata attributes, and ACLs of files in NTFS. An NTFS source supports incremental crawl. After the initial crawl is performed, subsequent crawls only collect those documents that have changed since the last crawl. A document is re-crawled if the content, metadata or the ACL information of the document has changed. A file is also re-crawled if it is moved between folders. Files deleted from NTFS are removed from the index during incremental crawls.
On the Windows server, the Super User must have permissions to read the NTFS file share
The Super User must be the impersonate user in the IIS Server
Microsoft Internet Information Server (IIS)
NET 2.0 Framework
Microsoft Developer Support OLE File Property Reader (dsofile)
NTFS sources on UNIX requires an NTFS Agent to be installed and configured on the Windows domain where the NTFS files are to be crawled. The NTFS Agent collects and sends content and meta data to the crawler plug-in on the Oracle SES machine in a crawl session. The communication protocol between Oracle SES and the NTFS Agent is HTTP or HTTPS.
The NTFS Agent needs to be installed on a Windows machine where IIS is present and the machine needs to be in the same Windows domain where the NTFS file share to be crawled resides.
Typically, a remote file share is crawled with the permission of a domain Administrator or a domain user with read privileges on the file share. The easiest way to configure this is to add the domain admin group to the 'administrators' group of the target machine.
The Oracle SES instance needs to connect to the same Active Directory instance that the MS NTFS domain connects to.
Install NTFS Agent on the Windows machine
If not already installed, download and install the Windows .Net 2.0 Framework.
If not already installed, download and install Microsoft Developer Support OLE File Property Reader.
Copy dsofile.dll
to a Windows system folder on the machine where the IIS is installed. Register dsofile.dll
file using regsvr32.exe
. This machine will be where the NTFS Agent resides.
Configure NTFS Agent in IIS:
Unzip $ORACLE_HOME/search/lib/plugin/ntfsLinWin/NTFSWebService.zip
into a temporary directory
Create a Virtual Directory in IIS and copy all the files unzipped from NTFSWebService.zip
into the Virtual Directory, or copy the files into an existing Virtual Directory on IIS.
For help in Creating Virtual Directories in IIS (IIS 6.0) see http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/5adfcce1-030d-45b8-997c-bdbfa08ea459.mspx?mfr=true
(Optional) Configure IIS Web site to use SSL
See Also:
Configuring IIS Web site to use SSL: http://www.petri.co.il/configure_ssl_on_your_website_with_iis.htm
How to implement SSL in IIS: http://support.microsoft.com/kb/299875
Configure the NTFS Agent to connect to the NTFS store in IIS:
Right-click your Web site (The IIS virtual directory with NTFSWebService Folder/files
)
Click the Properties tab.
Click the ASP.NET button and Click Edit Configurations.
ASP.NET Configuration/Application settings Parameters needs to be given
Service UserName: User name to authenticate between Oracle SES and NTFS Agents. This user name is required in Oracle SES source configuration.
Service Password: Password to authenticate between Oracle SES and NTFS Agents. This password is required in the Oracle SES source configuration.
Configure ASPNET impersonation: Impersonation is performed when ASP.NET executes code in the context of an authenticated and authorized client. Using impersonation, ASP.NET applications can optionally execute the processing thread using the identity of the client on whose behalf they are operating.Configure IIS virtual Directory as follows:
Right-click your IIS Web site (virtual directory), and then click Properties.
Click the ASP.NET button and click Edit Configurations.
Click the Application tab of ASP.NET Configuration Settings for Location Impersonation settings User Name: DOMAIN\<domain user>Password: password for <domain user>.
NTFS Agent can be deployed in any IIS instance in the same Windows domain. Application user or super user (Impersonate User) must have read permissions on the file share to be crawled. To enable read permissions do the following:
Right-click the file folder
Click Properties
Click security and then click Advanced tab.
Click effective permissions.
Enable read permissions for the user entered in the NTFS agent configuration.
If an NTFS source is used, Oracle recommends that Active Directory be used as identity management system for the Oracle SES instance. The Active Directory instance must be the same one that NTFS is using to authenticate users on the file system.
For the Oracle SES instance to read the files during crawling, add permission to each folder and file to make them accessible by the operating system user that runs the Oracle SES instance. (Adding permissions to a folder will automatically add the same permissions to all the files and sub-folders in the folder.)
See Also:
"Activating an Identity Plug-in" for information on activating the Active Directory identity plug-inCreate an NTFS source on the Home - Sources page. Select NTFS from the Source Type list, and click Create. Enter the values for the following parameters:
UNC PATH: UNC path for the NTFS system to crawl; for example, \\MYSERVER\mysharedfolder
EndPoint: Target end point (HTTP or HTTPS); for example, http(s)://NTFS Domain server (mail.doklet.com in this fig.)/virtual directory (NTFSWebService in the fig.)/NTFSWebService.asmx
USER NAME: User name to authenticate between Oracle SES and Microsoft Exchange: (configuration parameters similar to Exchange Agent in IIS)
PASSWORD: Password to authenticate between Oracle SES and Microsoft Exchange: (configuration parameters similar to Exchange Agent in IIS)
Use boundary rules on the NTFS source to restrict the Oracle SES crawler to URLs that match the indicated rules. This is set on the Home - Sources - Boundary Rules page.
For simple rules, Oracle SES supports the *, ^, and $ special characters:
SIMPLE_INC <simple boundary rule string>
SIMPLE_EXC <simple boundary rule string>
This is a set of user-friendly, simplified regular expression rules. Specify an inclusion rule that a URL contain, start with, or end with a term. Use an asterisk (*) to represents a wildcard. Use a caret (^) to denote the beginning of a URL, and use a dollar sign ($) to denote the end of a URL. For example:
^https://*.oracle.com/ .jpg$
For regexp rules, Oracle SES supports all regexp patterns:
REGEXP_INC <regular expression boundary rule string>
REGEXP_EXC <regular expression boundary rule string>
This is a set of regular expression rules using the java.util.regex package.
For example:
^https://.*\.oracle(?:corp){0,1}\.com
For any of these parameters, you can specify up to 50 rules. Use a semi-colon to separate strings and specify multiple rules. For example:
/^https://.*\.oracle(?:corp){0,1}\.com;^https://*.oracle.com/;https://*.oracle.com/*/
Livelink data is stored in Workspaces, which in turn can contain folders, files, projects, and task lists. A Livelink Enterprise Server instance can have one or more Workspaces that can be crawled using the Livelink Enterprise Server plug-in by configuring the configuration parameter in Oracle SES. The Livelink Enterprise Server plug-in navigates through the Workspaces to crawl all the objects in Livelink Enterprise Server. It creates an index, stores the metadata, and accesses information in Oracle SES to provide search according to the end user permissions.
The admin account should be used by the Livelink crawler plug-in for the container for crawling and indexing documents.
The Livelink Enterprise Server version must be 9.2, 9.5.0, 9.5.5
Because Open Text Livelink software is not included with Oracle SES, certain files must be copied manually into Oracle SES. Copy the lapi.jar
file from LAPI installation folder into ORACLE_HOME/search/lib/plugins/llcs
.
The Directory Services module of Livelink should be installed with Livelink (if users/groups are importing from LDAP server and you want to use the Active Directory identity plug-in).
To import users/groups of Active Directory in Livelink, follow these steps to import users/groups of Active Directory in Livelink Server.
Importing Users/Groups from LDAP to Livelink
Create an LDAP user that has permissions in Active Directory to administer users and groups. This user is used to synchronize the Active Directory with Livelink.
To extend the schema of Active Directory, install the Active Directory Schema snap-in as under:
Select Run from Windows Start menu.
Type mmc /a
in the Open field and click OK.
On the Console menu, choose Add/Remove Snap-in and click Add.
Under Snap-in, double-click Active Directory Schema. Click Close, then OK. Save the console (for example, as "Active Directory Schema.msc"). If the new snap-in does not appear under Snap-in, then you may have to re-install the Windows 2003 Administrative Tools and start again at step 2.
Open the file ot-livelink-schema.conf
(it is in the directory <livelink_home>/ module/directory_2_3_0
) in a text editor.
Open the Active Directory Schema console by clicking the Windows Start button, pointing to Programs - Administrative Tools and selecting (based on the sample name given) Active Directory Schema.msc.
Right-click Active Directory Schema and select Operations Master.
Right click the Attributes folder and select Create Attribute.
Create the attribute llserverinfo
using the information from ot-livelink-schema.conf
as under:
Create the attribute llquery
using the information from ot-livelink-schema.conf
as under:
Browse through the Directory Services Administration section of the Livelink Administration page for the enabling the following configuration:
Enabling the Synchronization Features:
Click the Choose Directory Services link.
Select LDAP Synchronization (Read-Only LDAP) from the Synchronization list.
For Livelink CGI Hosts, specify 127.0.0.1,<LIVELINK_SERVER_IP>
Click Save Changes.
Configuring LDAP Read-Only Parameters:
New User Password Policy |
Hidden |
User name Case Sensitivity |
Preserve Case |
Livelink Server Name |
Machine name on which Livelink Server is running |
LDAP Server |
Machine name or IP Address on which LDAP Server is running |
LDAP Server Port |
389 |
Search Root |
cn=Users,dc=otdomain,dc=com |
LDAP User name |
cn=<LDAP_User_Name>,cn=Users, dc=otdomain,dc=com |
LDAP Password |
<LDAP_User_Password> |
Log-in Name |
sAMAccountName or cn |
First Name |
givenname |
Last Name |
sn |
Title |
title |
|
|
Contact |
telephonenumber |
Department Mapping |
disable |
Group Name |
cn |
Group Leader |
managedBy |
Group Member |
Member |
Group Member Query |
llquery |
Privileges |
Select Log-in enabled, Public Access |
Group Search Filter |
objectclass=group |
Synchronize Group |
checked |
Click Save Changes.
Click Synchronize LDAP Read-only.
Click Synchronize.
The Livelink Enterprise Server identity plug-in authenticates native users of Livelink Enterprise Server. The identity plug-in communicates with the directory to authenticate a user's credentials, validate a user or group and return the associated canonical form, and return the groups associated with a given user.
Activate the identity plug-in on the Global Settings - Identity Management Setup page:
For the Active Directory identity plug-in, activate the oracle.search.plugin.security.idm.IdentityPluginManagerADImpl plug-in.
For the Livelink identity plug-in, activate the Livelink identity plug-in manager.
Create an Open Text source on the Home - Sources page. Select Open Text from the Source Type list, and click Create. Enter values for the following parameters:
User name: Name of a valid Livelink Enterprise Server user. The user must be an Administrator user or a user who has access to all folders and documents of the workspaces configured in the Container name parameter. The user should be able to retrieve content, metadata, and ACL from folders, documents and other custom sub classes of all workspaces configured in Container name parameter. This is a required parameter.
Password: Password of the Livelink user. This is a required parameter.
Container name: The names of the containers to be crawled by Oracle SES. You can crawl an entire Livelink Workspace or a specific folder. The format for is: <Workspace Name>/<Folder Name>/<Sub Folder Name>. Multiple comma-delimited container names can be entered. This is a required parameter. For example:
Container name: Workspace1: The entire Workspace1 will be crawled.
Container name: Workspace2/Folder21: Folder21 and its sub-folders within Workspace2 will be crawled.
Crawl folder attributes: Indicate whether folder attributes need to be crawled, either true or false. This is an optional parameter. The default value is false. If any other value is provided, it is assumed to be false.
Crawl versions: Indicates whether multiple versions of documents should be crawled, either true or false. This is an optional parameter and the default value is false. If any other value is provided, it is assumed to be false; in this case, only latest versions of a document will be crawled.
Attribute list: The comma-delimited list of Livelink attributes along with their data types to be searchable. The format for attribute list is <Attribute Name>:<Attribute Type>, <Attribute Name:Attribute Type>. Valid values are String, Number, and Date.
Table 5-9 Open Text Data Types
Sr. No | Open Text Data Type | Oracle SES Data Type |
---|---|---|
1 |
Boolean |
String |
2 |
Integer |
Number (Big Decimal) |
3 |
String |
String |
4 |
Date |
Date |
While crawling a Workspace an attribute is indexed only if both name and type match with configured name and type; otherwise, it will be ignored. This is an optional parameter. For example: If the administrator wants to make the following Livelink attributes searchable:
Attribute Name: Account Name Attribute Type: String
Attribute Name: Account Id Attribute Type: Integer
Attribute Name: Creation Date Attribute Type: Date
The value of Attribute list should be
Account Name: String, Account Id: Number, Creation Date:Date
The default searchable attributes for Livelink Enterprise Server will be Modified Date, Title, and Author.
Multiple attributes with same name are not allowed. For example Emp_ID:String, Emp_ID:Number
Server Name and Port Number for Livelink: The machine name/IP address and the port number on which Livelink server is running. The format is <Server Name>:<Port Number>.
Authentication attribute: The attribute used to set ACL. With Active Directory, the value is USER_NAME
. With the Livelink identity plug-in, the value is NATIVE
. This is a required parameter. This parameter is case-sensitive.
Crawl objects with public access: This parameter indicates whether objects with public access should be crawled without any ACL. Valid values are true or false. If false, then all objects having this ACL will be ignored.
SSL Enabled for Livelink: Specify if Livelink is running on SSL. If it is running on SSL, then this is true; otherwise, false.
Oracle recommends creating one source group for archived calendar data and another source group for active calendar data. One instance for the archived source can run less frequently, such as every week or month. This source should cover all history. A separate instance for the active source can run daily for only the most recent period.
The Oracle SES instance and the Oracle Calendar instance must be connected to the same Oracle Internet Directory system. Follow these steps to set up a secure Oracle Calendar source:
On the Global Settings - Identity Management Setup page in the Oracle SES administration tool, select the Oracle Internet Directory identity plug-in manager, and click Activate.
Use the following LDIF file to create an application entity for the plug-in. (An application entity is a data structure within LDAP used to represent and keep track of software applications accessing the directory with an LDAP client.)
$ORACLE_HOME/bin/ldapmodify -h oidHost -p OIDPortNumber -D "cn=orcladmin" -w password -f calPlugin.ldif
Where $ORACLE_HOME
is the Oracle Calendar infrastructure installation and calPlugin.ldif
is the current directory.
This defines the entity that will be used for the plug-in: orclApplicationCommonName=ocsCsPlugin,cn=ifs,cn=Products,cn=OracleContext
. The entity will have the password welcome1
.
Create an Oracle Calendar source on the Home - Sources page. Select Oracle Calendar from the Source Type list, and click Create. Enter values for the following parameters:
Table 5-10 Calendar Source Parameters
Parameter | Value |
---|---|
Calendar server |
http://host name:port |
Application entity name |
orclApplicationCommonName=ocsCsPlugin,cn=ifs,cn=Products,cn=OracleContext |
Application entity password |
welcome1 |
OID server hostname |
host name |
OID server port |
389 |
OID server SSL port |
636 |
OID server ldapbase |
dc=us,dc=oracle,dc=com |
uid |
|
User query |
(objectclass=ctCalUser) |
Past days |
30 |
Future days |
60 |
Rollover |
true |
Document in Oracle Content Database are organized into folders. Oracle SES navigates the folder hierarchy to crawl all documents in Oracle Content Database. It creates an index, stores the metadata, and accesses information in Oracle SES to provide search according to the end users' permissions.
Oracle SES supports incremental crawling; that is, it only crawls and indexes documents that have changed since the last crawling. A document is re-crawled if either the content or the direct security access information of the document changes. A document is also re-crawled if it is moved within Oracle Content Database and the end user has to access the same document with a different URL. Deleted documents are removed from the index during incremental crawling.
Oracle Content Database and Oracle Content Services are the same product. This section uses the product name Oracle Content Database to mean Oracle Content Database and Oracle Content Services.
The administrator account used by the Oracle Content Database source must have the ContentAdministrator
role on the site that is being crawled and indexed. Also, end-users searching documents in Oracle Content Database must have the GetContent
and GetMetadata
permissions.
By default, Oracle Content Database has a limit of three concurrent requests (simultaneous operations) for each user. However, Oracle SES has a default of five concurrent crawler threads. When crawling Oracle Content Database, only three of the five threads can successfully crawl, which causes the crawl to fail.
Workaround: For an Oracle Content Database source, change the Number of Crawler Threads on the Home - Sources - Crawling Parameters page to a value less than or equal to three.
Or, modify the Oracle Collaboration Suite configuration in Oracle Enterprise Manager to allow more than three concurrent requests. For example:
Access the Enterprise Manager page for the Collaboration Suite Midtier. For example: http://machine.domain:1156/
.
Click the Oracle Collaboration Suite midtier standalone instance name. For example: ocsapps.machine.domain
.
In the System Components table, click Content.
From Administration, click Node Configurations.
In the Node Configurations table, click HTTP_Node. For example: ocsapps.machine.domain_HTTP_Node
.
On Properties, change the value for Maximum Concurrent Requests Per User. Enter a value larger than or equal to the number of crawling threads used by Oracle SES. This value is listed on the Global Settings - Crawler Configuration page.
The Oracle SES instance and the Oracle Content Database instance must be connected to the same Oracle Internet Directory system. The groups in Oracle Content Database must also be synchronized with Oracle Internet Directory. Follow these steps to set up a secure Oracle Content Database source:
Read Known Limitations and confirm that the number of crawler threads does not exceed the available concurrent connection settings for each user in Oracle Content Database.
Activate the Oracle Internet Directory identity plug-in for the Oracle Content Database instance. This is done on the Global Settings - Identity Management Setup page in the Oracle SES administration tool.
Use the following LDIF file to create an application entity for the plug-in. (An application entity is a data structure within LDAP used to represent and keep track of software applications accessing the directory with an LDAP client.)
$ORACLE_HOME/bin/ldapmodify -h oidHost -p OIDPortNumber -D "cn=oracle" -w password -f csPlugin.ldif
Where $ORACLE_HOME
is the Oracle Content Database infrastructure installation and csPlugin.ldif
is the current directory.
This defines the entity that will be used for the plug-in: orclapplicationcommonname=ocscsplugin, cn=ifs,cn=products,cn=oraclecontext
. The entity will have the password welcome1
.
Create an Oracle Content Database source on the Home - Sources page. Select Oracle Content Database from the Source Type list, and click Create. Enter values for the following parameters:
Table 5-11 Oracle Content Database Source Parameters
Parameter | Value |
---|---|
Oracle Content Database URL |
http://host name:port/content |
Starting paths |
/ |
Depth |
-1 |
Oracle Content Database admin user |
orcladmin |
Entity name |
|
Entity password |
welcome1 |
Crawl only |
false |
Use e-mail for authorization |
false |
Table 5-12 Oracle Content Database Authorization Manager Plug-in Parameters
Parameter | Value |
---|---|
Oracle Content Database URL |
http://host name:port/content |
Oracle Content Database admin user |
orcladmin |
Entity name |
|
Entity password |
welcome1 |
Use e-mail for authorization |
false |
An Oracle E-Business Suite 11i source crawler is based on crawling a view or query in a database. Each record in the view or query is considered a document.
The view or query to be crawled for this source should contain the following columns:
Table 5-13 Oracle E-Business Suite 11i Source Required Columns
Name | Type | Description |
---|---|---|
|
varchar2 |
Display URL for the document |
|
varchar2/clob |
Document content |
|
date |
Last modified date for crawls |
|
varchar2 |
Key to the record |
|
varchar2 |
Document language |
The view or query can contain the following optional columns:
Table 5-14 Oracle E-Business Suite 11i Source Optional Columns
Name | Type | Description |
---|---|---|
|
varchar2 |
Path to the document. This is used in the browse feature. |
|
varchar2 |
HTTP link to the attachment for the document. This attachment will be indexed instead of the |
|
blob |
Binary attachments for the document. This will be indexed instead of the |
|
varchar2 |
Content type of the text content (text/plain or text/HTML). This column can also be used to indicate the content type (if known) for the binary content. |
Any other column in the view or query is considered an attribute of the document.
Activate the identity plug-in on the Global Settings - Identity Management Setup page. Select Identity Plugin Manager for Oracle E-Business Suite 11i and click Activate. Enter the values for the following parameters:
User Validation Database Connection String: JDBC connection string for the database, used for validating a user.
User ID: User ID to login to the user validation database.
Password: Password to login to the user validation database.
User Authentication Query: SQL query to authenticate a user. The query should return a single record with a single column with a string value of 'Y' or 'N' based on successful or unsuccessful authentication, respectively. The placeholder for user name and password should be specified as '?'. The default query (which can be changed if needed) is:
SELECT fnd_web_sec.Validate_login(upper(?),?) FROM dual
User Validation Query: SQL query to validate a given user. The query should return 1 if the user is valid. Else, no rows should be returned. The placeholder for the user name should be specified as '?'. The default query (which can be changed if needed) is:
SELECT 1 FROM fnd_user WHERE user_name = upper(?)
Click Finish.
Create an Oracle E-Business Suite 11i source on the Home - Sources page. Select Oracle E-Business Suite 11i from the Source Type list, and click Create. Enter values for the following parameters:
Database Connection String: JDBC connection string for the E-Business Suite database from which the content will be crawled.
User ID: User ID to login to the E-Business Suite database. This user ID should have access to the schema owning the view specified in the View parameter.
Password: Password to login to the E-Business Suite database.
View: Table or view containing the required set of columns
Document Count: Maximum number of documents to be crawled and indexed. Enter -1
if all documents should be crawled before indexing.
Query: Query projecting the required set of columns. This query should be used if the view defined in the View parameter is not available. Only one of these - View or Query – should be specified.
URL Prefix: String to prefix the content of URL column to form a display URL for the document
Cache File: Local file to which the contents can be temporarily cached while crawling.
Path Separator: Path separator character in the document path string
Parse Attributes: Enter true if the values of the attributes should be extracted from the document content specified in SOLUTION
column. Otherwise, enter false.
Grant Security Attributes: Space-delimited list of grant security attributes
Deny Security Attributes: Space-delimited list of deny security attributes
Click Next.
Click Get Parameters to obtain a list of parameters for the authorization manager plug-in.
Enter the values for the authorization manager plug-in parameters:
Authorization Database Connection String: JDBC connection string for the authorization database. The values of the security attributes to which a given user is authorized will be retrieved from this database.
User ID: User ID to login to the authorization database
Password: Password to login to the authorization database
Authorization Query: SQL query to retrieve the values of security attributes to which a given user is authorized. The SELECT
clause of this query should have all the security attributes specified in the Grant Security Attributes and Deny Security Attributes parameters with identical names. This query can be of two types:
The query can return a single record for a given user. The value in each security attribute column should be a space-delimited list of values to which the user is authorized.
The query can return multiple records for a given user. The value in each security attribute column of every row of the result set of this query will be interpreted as a single value.
The placeholder for the user name in the query should be specified as '?'.
Single Record Query: Enter true if the authorization query returns a single record. Enter false if the query can return multiple records.
Click Create.
For Siebel sources, searching is based on Siebel data available as RSS feeds. This section provides the instructions to create a secure Siebel 8 source.
Activate the identity plug-in on the Global Settings - Identity Management Setup page. Select Identity Plugin Manager for Siebel 8 and click Activate.
Enter values for the following parameters:
Siebel 8 authentication Web service endpoint: HTTP endpoint of the Siebel Web service that provides the authentication service
Siebel 8 validation Web service endpoint: HTTP endpoint of the Siebel Web service that provides the user validation service
User ID: Admin user ID for accessing the user validation service
Password: Admin password for accessing the user validation service
Click Finish.
Create a Siebel 8 source on the Home - Sources page. Select Siebel 8 from the Source Type list, and click Create.
Enter the values for the following parameters:
Configuration URL: File URL of the XML configuration file providing details about the source, such as the data feed type, location, security attributes, and so on.
Obtain this file from Siebel administrator and save it on the machine on which Oracle SES is installed. Enter the configuration URL as file://localhost/<Absolute path of the configuration file>
. For example: file://localhost/private/oracle/config.xml/
.
User ID: User ID to login to the FTP server, if the data feeds are to be accessed over FTP. The access details of the data feed are specified in the configuration file. This can be obtained from Siebel administrator.
Password: Password to login to the FTP server. This can be obtained from Siebel administrator.
Scratch Directory: A directory, in the machine where Oracle SES is installed to temporarily write the status logs.
Maximum number of connection attempts: Maximum number of attempts to connect to the target server to access the data feed.
Click Next.
Enter the values for the authorization manager plug-in parameters:
Siebel 8 authorization Web service endpoint: Webs service endpoint of the Siebel Web service that provides the authorization service
User ID: Admin user ID for accessing the authorization service
Password: Admin password for accessing the authorization service
Click Create.
Oracle SES can crawl through the e-mail and calendar items, related metadata, attributes, ACLs and attachments in Exchange and provide secure search. It also provides attribute search and browse functionality, which allows search to be done against a specific subfolder in the hierarchy.
The Microsoft Exchange plug-in supports incremental crawling; that is, it crawls and indexes only those documents that have changed after the last crawl was scheduled. A document is re-crawled if either the content or metadata or the direct security access (permissions) information of the document has changed. A document is also re-crawled if it is moved within Microsoft Exchange. Documents deleted from Exchange are removed from the index during incremental crawls.
A Microsoft Exchange source covers the following objects in Exchange:
E-mail attachments
Calendar events
On the Exchange server, the super user needs to grant himself the Send as
and Receive as
privileges. You can enable privileges globally for all users in the system. No user-specific privilege grants are required.
See Also:
Microsoft Exchange 2003 Technical Reference Guide and information about permissions in Microsoft Exhange: http://www.microsoft.com/technet/prodtechnol/exchange/default.mspx
Oracle Secure Enterprise Search Release Notes for supported platforms
Proper permissions on the Exchange server need to be granted to the Exchange administrator. The Exchange server is crawled with the permission of a super user with the Send as
and Receive as
privileges. The easiest way to configure this is to use an administrator as super user or create a super user with the administrator privilege and the Send as
and Receive as
privileges targeting Exchange inbox store and public folders.
The Microsoft Exchange source requires an Exchange Agent to be installed and configured on the Windows domain where the Exchange server is to be crawled. The Exchange Agent collects and sends content and metadata to the crawler plug-in on the Oracle SES machine in a crawl session. The communication protocol between Oracle SES and the Exchange Agent is HTTP or HTTPS.
The Exchange Agent must be installed on a Windows machine where IIS is present, and the machine needs to be in the same Windows domain where the Exchange server to be crawled resides.
Install the Exchange Agent on the Exchange server:
Unzip $ORACLE_HOME/search/lib/plugin/msexchange/ExchangeWebService.zip
into a temporary directory.
Create a virtual directory in IIS (IIS 6.0) and copy all the files unzipped from ExchangeWebService.zip
into the virtual directory, or copy the files into an existing virtual directory on IIS.
(Optional) Configure IIS Web site to use SSL:
See Also:
Configuring IIS Web site to use SSL: http://www.petri.co.il/configure_ssl_on_your_website_with_iis.htm
How to implement SSL in IIS: http://support.microsoft.com/kb/299875
Configure the Exchange Agent to connect to native Exchange Server store:
Right-click your Web site (the IIS virtual directory with Exchange Agent files).
Click the Properties tab.
Click the ASP.NET button, and click Edit Configurations.
Application settings parameters must be entered:
Service UserName: User name to authenticate between Oracle SES and Exchange Agent. This user name is required in Oracle SES source configuration.
Service Password: Password to authenticate between Oracle SES and Exchange Agent. This password is required in the Oracle SES source configuration.
Enter impersonation settings. Impersonation is when ASP.NET executes code in the context of an authenticated and authorized client. Using impersonation, ASP.NET applications can optionally execute the processing thread using the identity of the client on whose behalf they are operating. Configure IIS virtual Directory as follows:
Right-click your IIS Web site (virtual directory), and then click Properties.
Click the ASP.NET button, and click Edit Configurations.
Click the Application tab of ASP.NET Configuration Settings for Location Impersonation settings:
User Name: DOMAIN\SuperUser
Password: Password for SuperUser
The Exchange Agent can be deployed in any IIS in the same Windows domain.
If a Microsoft Exchange source is used, Oracle recommends that Active Directory be used as identity management system for the Oracle SES instance. The Active Directory instance must be the same one that Microsoft Exchange is using to authenticate users on the file system.
For the Oracle SES instance to read the files during crawling, add permission to each folder and file to make them accessible by the operating system user that runs the Oracle SES instance. (Adding permissions to a folder will automatically add the same permissions to all the files and sub-folders in the folder.)
See Also:
"Activating an Identity Plug-in" for information on activating the Active Directory identity plug-inCreate a Microsoft Exchange source on the Home - Sources page. Select Microsoft Exchange from the Source Type list, and click Create.
Enter values for the following parameters:
USER NAME: User name to authenticate between Oracle SES and Exchange (configuration parameters consistent with that for Exchange Agent in IIS).
PASSWORD: password to authenticate between Oracle SES and Exchange (configuration parameters consistent with that for Exchange Agent in IIS).
ENDPOINT: Target end point (HTTP or HTTPS); for example, http(s)://exchange server (mail.doklet.com in the example)/virtual directory (Web site in the example)/ExchangehttpsService.asmx.
Use boundary rules on the Microsoft Exchange source to restrict the Oracle SES crawler to URLs that match the indicated rules. This is set on the Home - Sources - Boundary Rules page.
For simple rules, Oracle SES supports the *, ^, and $ special characters:
SIMPLE INCLUDE <simple boundary rule string>
SIMPLE EXCLUDE <simple boundary rule string>
This is a set of user-friendly, simplified regular expression rules. Specify an inclusion rule that a URL contain, start with, or end with a term. Use an asterisk (*) to represents a wildcard. Use a caret (^) to denote the beginning of a URL, and use a dollar sign ($) to denote the end of a URL. For example:
^https://*.oracle.com/ .jpg$
For regexp rules, Oracle SES supports all regexp patterns:
Regular Expression INCLUDE <regular expression boundary rule string>
Regular Expression: EXCLUDE <regular expression boundary rule string>
This is a set of regular expression rules using the java.util.regex
package.
For example:
^https://.*\.oracle(?:corp){0,1}\.com
For any of these parameters, you can specify up to 50 rules. Use a semi-colon to separate strings and specify multiple rules. For example:
/^https://.*\.oracle(?:corp){0,1}\.com;^https://*.oracle.com/;https://*.oracle.com/*/
See Also:
"Tips for Using Federated Sources"Secure federated search enables searching secure content across distributed Oracle SES instances. An end user is authenticated to the Oracle SES federation broker. Along with querying the secure content in its own index, the federation broker federates the query to each federation endpoint on behalf of the authenticated end user. This mechanism necessitates propagation of user identity between the Oracle SES instances. In building a secure federated search environment, an important consideration is the secure propagation of user identities between the Oracle SES instances. This section explains how Oracle SES performs secure federation.
See Also:
When performing a secure search on a federation endpoint, the federation broker must pass the identity of the logged in user to the federation endpoint. If the endpoint instance trusts the broker instance, then the broker instance can proxy as the end user. To establish this trust relationship, Oracle SES instances should exchange some secret. This secret is exchanged in the form of a trusted entity. A trusted entity consists of two values: entity name and entity password. Each Oracle SES instance can have one or more trusted entities that it can use to participate in secure federated search. (A trusted entity is also referred to as a proxy user.)
Create trusted entities on the Global Settings - Federation Trusted Entities page of Oracle SES administration tool.
An Oracle SES instance can connect to an identity management (IDM) system for managing users and groups. An IDM system can be an LDAP compliant directory, such as Oracle Internet Directory or Active Directory.
Each trusted entity can be authenticated by either an IDM system or by the Oracle SES instance directly, independent of an IDM system. For authentication by an IDM system, check the box Use Identity Plug-in for authentication when creating a trusted entity. In this case, the entity password is not required. This is useful when there is a user configured in the IDM system that can be used for proxy authentication. Make sure that the entity name is the name of the user that exists in the IDM system and is going to be used as the proxy user.
For authentication of the proxy user by Oracle SES, clear (uncheck) the box Use Identity Plug-in for authentication when creating a trusted entity. Then use any name and password pair to create a trusted entity.
Use Authentication Attribute to specify the format of the user credential that the Oracle SES federation endpoint expects for this particular trusted entity in proxy authentication. The identity plug-in registered on the federation endpoint should be able to map this user identity to the default authentication format used on the federation endpoint. This is useful when a federation broker cannot send user identity in the default authentication format used on the federation endpoint for proxy authentication, but the identity plug-in registered on the federation endpoint can map the value from the attribute in which it receives the user identity during proxy authentication to the default authentication format used on the federation endpoint.
To use a proxy entity, use the Web services API proxyLogin(
) user name and password for the entity name and entity password. The identity plug-in can validate the password instead of storing it. When a request is sent for proxyLogin()
, Oracle SES calls the identity plug-in (which returns the call) to authenticate the entity. The proxyLogin()
must supply one of the valid trusted entities registered in the federation trusted entities.
To perform secure federated search, both the broker and the endpoint instances involved in the federation must have identity plug-ins registered. The identity plug-ins may or may not talk to the same IDM system. Carefully specify the following parameters under the section Secure Federated Search when creating a federated source on the broker instance:
Remote Entity Name: This is the name of the federation trusted entity on the federation endpoint. It is provided by the administrator of the endpoint instance.
Remote Entity Password: This is the password of the federation trusted entity on the federation endpoint. It is provided by the administrator of the endpoint instance.
Search User Attribute: This attribute identifies, and is used to authenticate, a user on the federation endpoint instance. This parameter is an optional parameter, except when the broker and endpoint use different authentication attributes to identify end users. (For example, on the broker instance, an end user can be identified by user name; on the endpoint instance, the end user can be identified by e-mail address.)
The identity plug-in registered on the broker instance should be able to map the user identity to this attribute based on the authentication attribute used during the registration of the identity plug-in. If this attribute is not specified during creation of the federation source, then the user identity on the broker instance is used to search on the endpoint instance.
Note:
If these parameters are not specified during the creation of the federated source, then the federated source is treated as a public source (that is, only public content is available to the search users).Secure Oracle HTTP Server-Oracle SES channel: Because any Oracle HTTP Server can potentially connect to the AJP13 port on the Oracle SES instances and masquerade as a specific person, either the channel between the Oracle HTTP Server and the Oracle SES instance must be SSL-enabled or the entire Oracle HTTP Server and Oracle SES instance machines must be protected by a firewall.
Notes:
In a secure federated search environment, the broker or the endpoint instance might or might not be using single sign-on (SSO). However, the Web service URL of the endpoint should not be behind SSO.
Oracle strongly recommends that you SSL-protect the channel between Oracle HTTP Server and Oracle SES for secure content. The endpoint instance should be SSL-enabled, or you should be able to access the Web service using HTTPS.
See Also:
"Tips for Using Federated Sources"