This chapter provides an overview and describes how to install Oracle Enterprise Repository into a clustered environment
This chapter includes the following sections:
Step 2: Deploy and Validate Oracle Enterprise Repository on One Cluster Member
Step 5: Configure the cluster.properties File on Each Cluster Member
Oracle Enterprise Repository uses a server-side cache on each application server. Cached data is used only if it is available, otherwise, the database delivers data content to the cache and to the application.
When Oracle Enterprise Repository runs in a cluster, the cluster members must communicate with each other using HTTP. An edit that occurs on one cluster member invalidates the cached element on that cluster member, and communicates the edit to other cluster members. This is accomplished by a system property called cachesyncurl, which accepts a URL to the application as a valid value.
On start, the system writes the cachesyncurl to the database and fetches a list of other server's URLs from the database. A message is sent to all discovered URLs that announces the presence of a new member of the cluster. Each server then refreshes the server list from the database. On a clean server shutdown, the value is removed from the list and a cache-refresh notification is broadcast to the server list.
When edits invalidate an element in the local cache, a message is sent to all the other servers noting which cached elements must be invalidated. Upon receipt of the message, the designated element is removed from the cache. On subsequent data request, the cache contains no data, so it first caches and then the database delivers data to the application.
Clustering and Installation Requirements
Session affinity
Server-side HTTP cache communication
Failover
Requires session management and persistent sessions
Load Balancing
Installing Oracle Enterprise Repository on a Clustered Environment
To install Oracle Enterprise Repository on a clustered environment:
Install and configure Oracle Enterprise Repository.
Create the clustered environment that hosts Oracle Enterprise Repository.
Install and deploy the Oracle Enterprise Repository application on one member of the clustered application servers.
Validate the deployment of the application on one member of the cluster.
Move the application properties to the database.
Shut down the cluster.
Install the application on all of the other cluster members.
Configure a cluster.properties file on each cluster member.
Start the cluster and all members.
Validate the cluster.
Before planning your clustered environment, review Administering Clusters for Oracle WebLogic Server for important information about clustering with WebLogic server.
For information regarding the installation of Oracle Enterprise Repository, see Chapter 2, "Installing the Oracle Enterprise Repository Software".
For information on how to configure and deploy Oracle Enterprise Repository on WebLogic Server, see Chapter 3, "Configuring the Oracle Enterprise Repository Domain".
Note:
Before using the Move settings to database option, you must enablecmee.eventframework.clustering.enabled, if JMS clustering is to be achieved.Use the pack command to pack the contents of the managed server created in Section 4.3. Use the unpack command to unpack a duplicate of the domain onto any number of machines that had been configured for inclusion in the clustered environment. See Creating Templates and Domains using the Pack and Unpack Commands for more information.
Note:
When packing the original domain, the properties files in the<DOMAIN_HOME>/config_oer or <DOMAIN_HOME>/config_oac directories are NOT included in the process. You must manually edit the properties files and configuration files on the initial machine and copy them over to the target servers manually. This procedure should be completed while the managed servers are offline.Next, follow the instructions in Chapter 5, "Configuring the Oracle Enterprise Repository Domain for a Clustered Environment" to complete the creation of the clustered environment.
After the domain creation is completed, you must edit the <DOMAIN_HOME>/config/config.xml file to point the OERDataSource and the oer_server deployments to target the cluster rather than the managed servers. You must also edit the cmee.properties file as appropriate for your deployment for the proxy server being used to access the managed servers and clustered environment.
Here is an example of elements in the config.xml file before editing:
<app-deployment>
<name>oer_server</name>
<target>oer_server1</target>
<module-type>ear</module-type>
<source-path>/users/oracle/Oracle/Middleware/Oracle_Home/oer/applications/oer</source-path>
<deployment-order>45</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
<staging-mode>nostage</staging-mode>
</app-deployment>
<jdbc-system-resource>
<name>OERDataSource</name>
<target>oer_server</target>
<descriptor-file-name>jdbc/OERDataSource-jdbc.xml</descriptor-file-name>
</jdbc-system-resource>
Here is an example of the same file after editing:
<app-deployment>
<name>oer_server</name>
<target>oer_cluster1</target>
<module-type>ear</module-type>
<source-path>/users/oracle/Oracle/Middleware/Oracle_Home/oer/applications/oer</source-path>
<deployment-order>45</deployment-order>
<security-dd-model>DDOnly</security-dd-model>
<staging-mode>nostage</staging-mode>
</app-deployment>
Here is an example of a cmee.properties file before editing:
cmee.enterprisetab.homepage=http\://www.example.com \:8111/oer/custom/home_oac.jsp cmee.server.paths.resource=http\://www.example.com \:8111/oer -web cmee.assettab.asset-detail-page=http\://www.example.com \:8111/oer /cmee/index.jsp cmee.server.paths.image=http\://www.example.com \:8111/oer -web/images cmee.server.defaultencoding=UTF-8 cmee.server.paths.servlet=http\://www.example.com \:8111/oer cmee.server.paths.jnlp-tool=http\://www.example.com \:8111/oer -web/webstart cmee.server.paths.jsp=http\://www.example.com \:8111/oer
Here is an example of the same file after editing:
cmee.enterprisetab.homepage=http\://ohs.example.com \:7777/oer/custom/home_oac.jsp cmee.server.paths.resource=http\://ohs.example.com \:7777/oer -web cmee.assettab.asset-detail-page=http\://ohs.example.com \:7777/oer /cmee/index.jsp cmee.server.paths.image=http\://ohs.example.com \:7777/oer -web/images cmee.server.defaultencoding=UTF-8 cmee.server.paths.servlet=http\://ohs.example.com \:7777/oer cmee.server.paths.jnlp-tool=http\://ohs.example.com \:7777/oer -web/webstart cmee.server.paths.jsp=http\://ohs.example.com \:7777/oer
Property files always take precedence when reading properties into the Oracle Enterprise Repository application. The application looks for properties and their corresponding values, first within the database, and then within the property files. Any properties read from the database are overwritten by the corresponding properties in the files.
This procedure begins with deploying one application.
In the Admin screen, click System Settings in the left pane.
Scroll to the bottom and click the Move settings to database button.
A confirmation message appears.
Remove the properties files from the classpath.
Shutdown the managed servers.
Locate the configuration files folder:
<DOMAIN_HOME>/oer_config
Remove the property files listed below from the configuration folder:
enterprise.properties
ldap.properties
containerauth.properties
eventing.properties
juddi.properties
openapiserverlog.properties
These properties are written to the entSettings table within the database.
Modify the cmee.properties file. Remove all property values except those containing URL values. Update the URL references to point to the proxy server path being used to load balance access to the cluster members.
Now, redeploy the application directly to the Oracle Enterprise Repository cluster.
Note:
Any properties enabled after this procedure are written to the database, not to the properties files.To configure the cluster.properties file on each cluster member:
Stop each cluster member.
On each cluster member create a file called cluster.properties, which resides in the same place as all other .properties files:
<DOMAIN_HOME>/oer_config
The contents of cluster.properties is based on the property cmee.server.paths.servlet in the cmee.properties file. However, the host name in the path should refer to the host name of the cluster member, not the proxy host name of the entire cluster.
cluster.properties #cluster.properties Example: #cluster.properties Other properties that are optional # alias is used as an alternate/convenient name to refer # to the server # example: server1 # default: same value as =cachesyncurl= alias=EclipseServer # registrationIntervalSeconds is the number of seconds between # attempts to update the server's registration record in the database # default: 120 registrationIntervalSeconds=120 # registrationTimeoutSeconds is the number of seconds before a server # is considered to be inactive/not running # make sure this value is higher than the registrationIntervalSeconds # default: 240 registrationTimeoutSeconds=240 # maxFailures is the number of consecutive attempts that are made # to deliver a message to another server after which it is determined # to be unreachable # default: 20 maxFailures=20 # maxQueueLength is the number of messages that are queued up to # send to another server after which server are determined to be # unreachable # default: 4000 maxQueueLength=5000 # email.to is the address of the email recipient for clustering status # messages email.to=jsmith@company.com # email.from is the address of the sender for clustering status messages email.from=jsmith@company.com # email.subject is the subject line of the message for clustering status # messages email.subject=Oracle Enterprise Repository Clustering communication failure # email.message is the body of the message for clustering status messages email.message=This is an automated message from the Oracle Enterprise Repository informing you of a cluster member communication failure.
Example of a cluster.properties file
registrationIntervalSeconds=120registrationTimeoutSeconds=240 maxFailures=20 maxQueueLength=5000 email.to=jsmith@example.com email.from=jsmith@example.com email.subject=Oracle Enterprise Repository Clustering communication failure email.message=This is an automated message from the Oracle Enterprise Repository informing you of a cluster member communication failure
Note:
The time delay should not be more than 120 seconds between the application server and the database server. Network Time Protocol is recommended to keep these servers in sync. The clustering process calculates the difference of time between messaging between the nodes of the cluster.
Before restarting the server, you must add eventing.properties, if JMS Clustering is enabled, and this should contain cmee.eventframework.jms.producers.client.id property with unique value on each of the cluster member. For example, cmee.eventframework.jms.producers.client.id=OER_JmsProducer1
See Section 4.8 for important information about setting the cmee.cachesyncurl property as a JVM property for each managed server within the cluster.
Restart each cluster member.
Note:
After a cluster member is inactivated due to exceeding maxFailures, then the only way to activate is by restarting the server.Messages are sent to the standard out log of each cluster member.
"running in single server mode"
Indicates that Oracle Enterprise Repository clustering is not configured and the application is running in single server mode.
"running in multi server mode with a sync-url of..."
Indicates the Oracle Enterprise Repository clustering is functioning and the application is running in clustered mode.
cachesyncurl
The value of the cachesyncurl in the cluster.properties file, which references the same URL as the individual node's instance with the path of /cachesync appended. Most cluster configurations have a proxy server load-balancing each node within the clustered server.
Example:
Node1: cachesyncurl=http://node1.example.com:7101/oer
Node2: cachesyncurl=http://node1.example.com:7101/oer
It is also possible to validate the clustering installation by viewing the clustering diagnostic page from the Oracle Enterprise Repository Diagnostics screen. Click Cluster Info on the Diagnostics screen to view the Cluster Diagnostic page. This page lists information about all servers registered in the cluster, as well as information about inter-server communications.
Oracle HTTP Server Config File
If Oracle HTTP Server is used to route from the load balancer to the nodes, the HTTP Server Config (httpd.conf) file should include two entries: /oer and /oer-web. A sample httpd.conf file is shown below:
<VirtualHost 10.0.0.1:80>
ServerName 10.0.0.1:80
<IfModule mod_weblogic.c>
SecureProxy OFF
# TrustedCAFile /usr/local/apache2/ca.pem
RequireSSLHostMatch false
Debug ERR
DebugConfigInfo ON
ErrorPage /error.html
WLLogFile logs/worker_matchproxy.log
WLTempDir /tmp
KeepAliveEnabled ON
KeepAliveSecs 15
HungServerRecoverSecs 30
MaxSkipTime 10
ConnectRetrySecs 5
</IfModule>
<Location /console>
SetHandler weblogic-handler
WebLogicCluster 10.0.0.2:7001
</Location>
<Location /oer_fa_cluster>
SetHandler weblogic-handler
WebLogicCluster 10.0.0.3:7101,10.0.0.4:7101
</Location>
<Location /oer_fa_cluster-web>
SetHandler weblogic-handler
WebLogicCluster 10.0.0.3:7101,10.0.0.4:7101
</Location>
</VirtualHost>
In this example, host 10.0.0.1 is the HTTP server proxying for the cluster machines, host 10.0.0.2 is the Admin Server, hosts 10.0.0.3 and 10.0.0.4 are the Oracle Enterprise Repository managed servers.
See Understanding Oracle WebLogic Server for more information.
If cluster nodes are deployed using a centralized administration console, you must apply a JVM Parameter to allow the appropriate Oracle Enterprise Repository clustering operation in the absence of the cluster.properties file.
This JVM parameter should be applied statically for each member of the cluster or within the managed server startup command file. This is most easily done using a server definition in the DOMAIN_HOME/bin/setStartupEnv.sh or setStartupEnv.bat files. Add the following code (edited to match your environment) to the setStartupEnv.sh or .bat file:
case "${SERVER_NAME}" in
oer_server1)
EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} -Dcmee.cachesyncurl=http://server1:8111/oer"
;;
oer_server2)
EXTRA_JAVA_PROPERTIES="${EXTRA_JAVA_PROPERTIES} -Dcmee.cachesyncurl=http://server2:8111/oer"
;;
esac
This JVM parameter can also be set within the JAVA_OPTIONS environment variable for WebLogic Server. The JVM Parameter is as follows:
-Dcmee.cachesyncurl=http://<member host name>:<port>/<APP_PATH>
Each managed server within the cluster domain will communicate with one another directly and not through the HTTP proxy server. The cmee.cachesyncurl property represents the managed server hostname/IP address and port number.
Note:
Each managed server in the cluster must be started with one of these JVM parameters in order to coexist and cooperate within the cluster.