7 Managing Database Workload Using Services

Using workload management, you can distribute the workload across database instances to achieve optimal database and cluster performance for users and applications. This chapter contains the following sections:

About Workload Management

Applications using a clustered database generally want to load balance their workload across the cluster. Oracle Real Application Clusters (Oracle RAC) includes a highly available (HA) application framework that provides the necessary service and integration points between Oracle RAC and custom enterprise applications.

You can deploy Oracle RAC and single-instance Oracle database environments to use workload management features in many different ways. Depending on the number of nodes and your environment's complexity and objectives, your choices for the optimal workload management and high availability configuration depend on a variety of considerations, as described in this chapter.

To implement workload management for an Oracle Real Application Clusters (Oracle RAC) database, you can use a number of different features. This section contains the following topics:

About Oracle Services

Oracle Database 10g introduced an automatic workload management facility, called services. A service represents the workload of applications with common attributes, performance thresholds, and priorities. A single service can represent an application, multiple applications or a subset of a single application. A single service can be associated with one or more instances of an Oracle RAC database, and a single instance can support multiple services. Services provide the following benefits:

  • A single entity for managing applications that compete for the same resources

  • Allow each workload to be managed as a single unit

  • Hide the complexity of the cluster from the client

To manage workloads, you can define services that you assign to a particular application or to a subset of an application's operations. You can also use services to manage the workload for different types of work. For example, online users can use one service while batch processing can use a different service and reporting can use yet another service type.

Traditionally an Oracle database provided a single service and all users connected to the same service. A database will always have this default database service that is the database name. This service cannot be modified and will always allow you to connect to the database.

Note:

Do not use the default database service for application workloads. Create at least one service as described in "Creating Services".

When a user or application connects to a database, Oracle recommends that you use a service for the connection. Applications and mid-tier connection pools select a service by using the service name in their connection data. For more flexibility in the management of the workload using the database, Oracle Database enables you to create multiple services and specify which database instances offer the services.

Services are integrated with the Database Resource Manager, which enables you to restrict the resources that are used by a service within an instance. In addition, Oracle Scheduler jobs can run using a service, as opposed to using a specific instance.

See Also:

About Configuring Services for High Availability

When you create a service, you define which instances typically support that service. These are known as the preferred instances for that service. You can also define other instances to support a service if the service's preferred instances fail. These are known as available instances for a service.

When you specify a preferred instance for a service, the service runs on that instance during standard operation. Oracle Clusterware attempts to ensure that the service always runs on all the preferred instances that have been configured for a service. If the instance fails, the service is randomly relocated to another preferred instance or one of the available instances. You can also manually relocate the service to an available instance. If you do not specify preferred or available instances when you create a service, then by default, every instance in the Oracle RAC database is a preferred instance for that service.

If a service fails over to an available instance, the service is not moved back to its preferred instance automatically. However, you can automate the relocation of a service to its preferred instance by using a callout. For more information about callouts, see "About FAN Callouts". An example callout script is available in the Oracle Real Application Clusters section on Oracle Technology Network at http://www.oracle.com/technetwork/database/clustering/overview/awmrac11g-133673.pdf.

You do not have to specify available instances for a service. However, if you configure a preferred instance for a service, but do not specify at least one available instance for the service, then the service does not relocate to another instance if the preferred instance fails.

You can also specify an instance as Not Used. This setting means the service does not run on the instance, even if the preferred instance for the service fails.

See Also:

About the Database Resource Manager

The Database Resource Manager controls database resources allocated to users, applications, and services. This approach ensures that users, applications, and services receive their share of the available database resources. The Database Resource Manager enables an Oracle RAC database running on one or more nodes to support multiple applications and mixed workloads with optimal efficiency.

The Database Resource Manager provides the ability to prioritize work within an Oracle database or your Oracle RAC environment. For example, high priority users, such as online workers, would get more resources to minimize response time, while lower priority users, such as batch jobs or reports, would get fewer resources, and could take longer to run. This allows for more granular control over resources.

Resources are allocated to users according to a resource plan specified by the database administrator. The following terms are used in specifying a resource plan:

  • A resource plan specifies how the resources are to be distributed among various users (resource consumer groups).

  • Resource consumer groups allow the administrator to group user sessions together by resource requirements. Resource consumer groups are different from user roles; one database user can have different sessions assigned to different resource consumer groups.

  • Resource allocation methods are the methods or policies used by the Database Resource Manager when allocating for a particular resource. Resource allocation methods are used by resource consumer groups and resource plans. The database provides the resource allocation methods that are available, but the DBA determines which method to use.

  • Resource plan directives are a means of assigning consumer groups to particular plans and partitioning resources among consumer groups by specifying parameters for each resource allocation method.

  • Subplans, which the DBA can create within a resource plan, allow further subdivision of resources among different users of an application.

  • Levels provide a mechanism to specify distribution of unused resources among available users. Up to eight levels of resource allocation can be specified.

The Database Resource Manager enables you to map a resource consumer group to a service so that users who connect using that service are members of the specified resource consumer group, and thus restricted to the resources available to that resource consumer group.

See Also:

About Oracle RAC High Availability Framework

The Oracle RAC high availability framework enables Oracle RAC to maintain the database, components, and applications in a running state at all times. If an instance, component, or application fails, it can be automatically restarted to keep Oracle Database operating at full capacity.

Oracle Database focuses on maintaining service availability. In Oracle RAC, Oracle services are designed to be continuously available with workloads shared across one or more instances. The Oracle RAC high availability framework maintains service availability by storing the configuration information for each service in the Oracle Cluster Registry (OCR). Oracle Clusterware recovers and balances services across instances based on the service definition.

See Also:

About Fast Application Notification (FAN)

One of the main requirements of a highly available application is for it to be quickly notified when something happens to critical system components. This allows the application to execute event-handling programs. The timely execution of such programs minimizes the time it takes to react to cluster resource organizations and the impact of cluster component failures by avoiding costly connection timeouts and application timeouts.

Fast Application Notification is a notification mechanism that Oracle RAC uses to notify other processes about cluster configuration and service-level information, including status changes such as UP or DOWN events. FAN UP and DOWN events can apply to instances, services, and nodes. FAN also publishes Load Balancing Advisory events.

FAN enables the automated recovery of applications when cluster components fail. For cluster configuration changes, the Oracle RAC high availability framework publishes a FAN event immediately when a change occurs regarding the state of the instances in the cluster. Instead of waiting for the application to query the database and detect a problem, applications can receive FAN events and react immediately.

FAN UP and DOWN events provide the following benefits:

  • For DOWN events, the disruption to the application can be minimized because sessions that are connected to the failed instance or node can be terminated. Incomplete transactions can be terminated and the application user notified immediately. Application users who request connections are directed to instances that are started and are providing the requested service.

  • For UP events, when services and instances are started, new connections can be created so that the application can immediately take advantage of the extra resources.

Oracle Clusterware and Oracle RAC utilize Oracle Notification Service (ONS) to propagate FAN messages both within the Oracle RAC cluster and to client or mid-tier machines. ONS is installed with Oracle RAC and the Oracle Clusterware resources to manage the ONS daemon are created automatically during the installation process. ONS daemons run locally sending messages to and receiving messages from a configured list of nodes (where other ONS daemons are active).

About FAN Callouts

FAN callouts are server-side executable files that Oracle RAC runs immediately when high availability events occur. A callout is essentially a shell script or precompiled executable written in any programming language. Some examples of how you can use FAN callouts to automate the actions performed when events occur in a cluster configuration are as follows:

  • Starting and stopping server-side applications

  • Relocating low-priority services when high-priority services come online

  • Sending text or numeric messages to pagers

  • Executing shell scripts

The executable files for FAN callouts are stored in the CRS_home/racg/usrco subdirectory. If this subdirectory does not exist in your CRS home, then you must create this directory with the same permissions and ownership as the CRS_home/racg/tmp subdirectory.

All executables in the directory CRS_home/racg/usrco are executed immediately, in an asynchronous fashion, when a FAN event received through the ONS. A copy of the executable files used by FAN callouts should be available on every node that runs Oracle Clusterware. Example callout scripts are available in the Oracle Real Application Clusters white papers on Oracle Technology Network at http://www.oracle.com/technetwork/database/clustering/overview/awmrac11g-133673.pdf.

See Also:

About the Load Balancing Advisory

The Load Balancing Advisory provides information to applications or clients about the current service levels that the Oracle RAC database instances are providing. Applications can take advantage of the load balancing Fast Application Notification (FAN) events to direct work requests to the instance in the cluster that will provide the best performance based on the workload management directives that you have defined for that service. Also, when an instance is restarted, Oracle RAC uses FAN events to notify the application's connection pool so that the connection pool can create connections to the recently started instance and take advantage of the additional resources that this instance provides.

The load balancing advisory is integrated with the Automatic Workload Repository built into Oracle Database 11g. The Automatic Workload Repository measures response time and CPU consumption for each service.

The advice given by the Load Balancing Advisory takes into account the power of the server as well as the current workload of the service on the server. Enabling the Load Balancing Advisory helps improve the throughput of applications by not sending work to instances that are overworked, running slowly, not responding, or have failed.

Your application can take advantage of the Load Balancing Advisory without any programmatic changes if you use an integrated Oracle client, one that has the Runtime Connection Load Balancing feature. Due to the integration with FAN, Oracle integrated clients are more aware of the current status of an Oracle RAC cluster. This prevents client connections from waiting or trying to connect to an instance that is no longer available. The integrated clients for FAN events include Oracle Database 11g JDBC, Oracle Database 11g ODP.NET, and Oracle Database 11g Oracle Call Interface (OCI).

You configure your Oracle RAC environment to use the Load Balancing Advisory by defining service-level goals for each service used. This enables the Load Balancing Advisory for that service and enables the publication of FAN load balancing events. There are two types of service-level goals for Runtime Connection Load Balancing:

  • Service Time—The Load Balancing Advisory attempts to direct work requests to instances according to their response time. Load Balancing Advisory data is based on the elapsed time for work done by connections using the service, as well as available bandwidth to the service. This goal is best suited for workloads that require varying lengths of time to complete, for example, an internet shopping system.

  • Throughput—The Load Balancing Advisory measures the percentage of the total response time that the CPU consumes for the service. This measures the efficiency of an instance, rather than the response time. This goal is best suited for workloads where each work request completes in a similar amount of time, for example, a trading system.

If you do not select the Enable Load Balancing Advisory option, the service-level goal is set to None, which disables load balancing for that service.

See Also:

About Connection Load Balancing

Oracle Net is a software component that resides on the client and on the Oracle database server. It establishes and maintains the connection between the client application and the server, and exchanges messages between them using industry standard protocols. For the client application and a database to communicate, the client application must specify location details for the database it wants to connect to, and the database must provide some sort of identification or address.

On the database server, the Oracle Net Listener, commonly known as the listener, is a process that listens for client connection requests. The configuration file for the listener is the listener.ora.

You can use Net Configuration Assistant (NETCA) to create a net service name, a simple name for the database service. The net service name resolves to the connect descriptor, which is the network address of the database and the name of the database service. The address portion of the connect descriptor is actually the protocol address of the listener. The client uses a connect descriptor to specify the database or instance to which the client wants to connect.

When a net service name is used, establishing a connection to a database instance takes place by first mapping the net service name to the connect descriptor. This mapped information is stored in one or more repositories of information that are accessed using naming methods. The most commonly used naming method is Local Naming, where the net service names and their connect descriptors are stored in a localized configuration file named tnsnames.ora.

When the client connects to the cluster database using a service, you can use the Oracle Net connection load balancing feature to spread user connections across all the instances that are supporting that service. There are two types of load balancing that you can implement: client-side and server-side load balancing. In an Oracle RAC database, client connections should use both types of connection load balancing. When you create an Oracle RAC database using Oracle Database Configuration Assistant (DBCA), DBCA configures and enables server-side load balancing by default.

See Also:

Client-Side Load Balancing

Client-side load balancing balances the connection requests across the listeners. When the listener receives the connection request, the listener connects the user to an instance that the listener knows provides the requested service.

Client-side load balancing is defined in your client connection definition by setting the parameter LOAD_BALANCE=yes in the tnsnames.ora file. When you set this parameter to yes, the Oracle client randomly selects an address from the address list, and connects to that node's listener. This balances client connections across the available listeners in the cluster.

When you create an Oracle RAC database using DBCA, the assistant creates a sample load balancing connection definition in the tnsnames.ora file.

Client-side load balancing includes connection failover. With connection failover, if an error is returned from the chosen address, Oracle Net Services will try the next address in the address list until either a successful connection is made or it has exhausted all the addresses in the list.

Server-Side Load Balancing

With server-side load balancing, the listener directs a connection request to the best instance currently providing the service by using information from the Load Balancing Advisory.

For each service, you can define the method that you want the listener to use for load balancing by setting the connection load balancing goal. You can use a goal of either long or short for connection load balancing. These goals have the following characteristics:

  • Short—Connections are distributed across instances based on the amount of time that the service is used. Use the Short connection load balancing goal for applications that have connections of brief duration.

  • Long—Connections are distributed across instances based on the number of sessions in each instance, for each instance that supports the service. Use the Long connection load balancing goal for applications that have connections of long duration. This is typical for connection pools and SQL*Forms sessions. Long is the default connection load balancing goal.

Any services created by using DBCA use the Long connection load balancing goal by default.

Note:

If you did not use DBCA to create your database, or if you are using listener ports other than the default of 1521, then you must configure the LOCAL_LISTENER and REMOTE_LISTENER database initialization parameters for your cluster database.

About Runtime Connection Load Balancing

Runtime Connection Load Balancing is a feature of Oracle connection pools that can distribute client work requests across the instances in an Oracle RAC database based on the Load Balancing Advisory information. The connection allocation is based on the current performance level provided by the database instances as indicated by the Load Balancing Advisory FAN events. This provides load balancing at the transaction level, instead of load balancing at the time of the initial database connection.

With Runtime Connection Load Balancing, applications use Load Balancing Advisory information to provide better performance to users. The Oracle JDBC and Oracle Data Provider for .NET (ODP.NET) client connection pools are integrated to take advantage of Load Balancing Advisory information. You must enable the client data source for Runtime Connection Load Balancing with a service that has the following configuration:

  • The Load Balancing Advisory is enabled and the service-level goal is set to either Service Time or Throughput.

  • The service connection load balancing goal is set to Short.

Figure 7-1, "Runtime Connection Load Balancing" illustrates Runtime Connection Load Balancing. In this illustration, the Oracle RAC database has three instances. Suppose that the Load Balancing Advisory indicates that Instance1 and Instance3 have the best performance, while Instance2 currently has less than optimal performance. When Runtime Connection Load Balancing is enabled on the implicit connection cache, the following process occurs:

  1. A client requests a connection from the connection cache.

  2. Runtime Connection Load Balancing selects the connection that belongs to the most efficient (best) instance from the connection cache. In Figure 7-1, there are three possible nodes to which the connection can be routed. Instance1, which has the least amount of CPU workload, is currently being assigned about 60 percent of the incoming connections. Instance2, which is currently overloaded, is only being assigned around 10 percent of the incoming connections. Instance3, which has a high workload, is being assigned around 30 percent of the incoming connections. The best instance to handle the connection request in this case would be Instance1.

  3. The client receives the connection that would process the work request with the best response time.

Figure 7-1 Runtime Connection Load Balancing

Description of Figure 7-1 follows
Description of "Figure 7-1 Runtime Connection Load Balancing"

Oracle Database 11g introduces an additional flag in the load balancing advisory event called affinity hint. The affinity hint is automatic when load balancing advisory is turned on through setting the goal on the service. This flag is for temporary affinity that lasts for the duration of a web session. Web conversations often connect and disconnect a number of times during the entire session. During each of these connects, it may access the same or similar data, for example, a shopping cart, Siebel, and so on. Affinity can improve buffer cache efficiency, which lowers cpu usage and transaction latency. The Affinity Hint is a flag that indicates if Affinity is active or inactive for a particular instance and service combination. Different instances offerin the same service can have different settings for the Affinity Hint.

Oracle Database 11g Patchset 1 (11.1.0.7) introduces a new connection pool for Java called the Universal Connection Pool (UCP). UCP can be used against Oracle Database 10g or Oracle Database 11g. Applications using Oracle Database 11g and UCP, can take advantage of this new affinity feature. If the affinity flag is turned on in the Load Balancing Advisory event, then UCP will create an Affinity Context for the Web session such that when that session does a get connection from the pool, the pool will always try to give it a connection to the instance it connected to the first time it acquired a session. The choice of instance for the first connection is based on the current load balancing advisory information.

Creating Services

You can create a service using Oracle Enterprise Manager Database Control.

To create a service:

  1. On the Cluster Database Home page, click Availability.

    The Availability page appears.

    Description of wlm001.gif follows
    Description of the illustration wlm001.gif

  2. Click Cluster Managed Database Services in the Services section. Enter or confirm the credentials for the Oracle RAC database and host operating system.

    The Cluster Managed Database Services page appears.

    Description of wlm005.gif follows
    Description of the illustration wlm005.gif

  3. Click Create Service.

    The Create Service page appears.

  4. Enter the name of your service in the Service Name field, for example, DEVUSERS.

  5. Select Start Service after creation if you want the service to be started after it is created.

    Description of wlm006.gif follows
    Description of the illustration wlm006.gif

  6. Choose whether the instance is a Preferred or Available instance for each instance displayed in the High Availability Configuration section for this service. If you do not want the service to run on a particular instance, select Not Used for the Service Policy.

  7. Select Short for Connection Load Balancing Goal if you want to distribute the connection workload based on elapsed time instead of the overall number of connections. Otherwise, choose Long.

  8. Select Enable Load Balancing Advisory under the heading Notification Properties on the Create Service page to enable the Load Balancing Advisory for this service, as shown in the following screenshot. Choose a service-level goal of either Service Time or Throughput.

    Description of wlm007.gif follows
    Description of the illustration wlm007.gif

  9. Select Enable Fast Application Notification under the heading Notification Properties if this service is used by an Oracle Call Interface (OCI) or ODP.NET application, and you want to enable FAN.

  10. In the Service Threshold Levels section, you can optionally set the service-level thresholds by entering a value in milliseconds for Warning and Critical thresholds for the Elapsed Time and CPU Time metrics.

  11. If you want to use a Resource Plan to control the resources used by this service, select the name of the consumer group from the Consumer Group Mapping list in the Resource Management Properties section. For example, you might choose the LOW_GROUP consumer group to give development users low priority to database resources.

    Note:

    You cannot change the consumer group name for a service on the Edit Service page. This is because there may be several consumer groups associated with a given service. However, the Edit Service page contains a link to the Resource Consumer Group Mapping page, where you can modify the consumer group mapping for the service.

  12. If this service is used by a specific Oracle Scheduler job class, you can specify the mapping by selecting the name from the Job Scheduler Mapping list in the Resource Management Properties.

  13. Click OK to create the service.

See Also:

Configuring Oracle Net to Support Services

Although Enterprise Manager configures Oracle Clusterware resources for the newly created service, it does not generate the corresponding entries in your tnsnames.ora file.

To configure Oracle Net Services to support the newly created service:

  1. Determine if the listener on the local node recognizes the new service by using the following command:

    lsnrctl status

    You should see a list for the new service, similar to the following:

    Service "DEVUSERS.oracle.com" has 1 instance(s).
  Instance "sales1", status READY, has 2 handler(s) for this service...

    The displayed name for your newly created service, for example DEVUSERS.oracle.com, is the value you will use for the SERVICE_NAME parameter in the tnsnames.ora file.

  2. Use a text editor to modify the tnsnames.ora file in the Oracle_home/network/admin directory on each node that contains an instance listed as a Preferred or Available instance for the service, and on each client that uses the service to connect to the database. Add an entry similar to the following, specifying the VIP address for each node:

    DEVUSERS = 
  (DESCRIPTION = 
    (ADDRESS_LIST = Service
      (ADDRESS = (PROTOCOL = TCP)(HOST = docrac1-vip)(PORT = 1521))
      (ADDRESS = (PROTOCOL = TCP)(HOST = docrac2-vip)(PORT = 1521))
    (LOAD_BALANCE = yes)
    )
  (CONNECT_DATA = (SERVICE_NAME = DEVUSERS.oracle.com))
   )

    In the previous example, the ADDRESS_LIST parameter contains one ADDRESS for each node that contains an instance configured as either Preferred or Available for the service.

  3. Test the Oracle Net Services configuration by attempting to connect to the Oracle RAC database using SQL*Plus and the service name, for example:

    $ sqlplus system@DEVUSERS
Enter password: password

    After you enter the password, you should see a message indicating you are successfully connected to the Oracle RAC database. If you get an error message, examine the tnsnames.ora file and verify the user name, password, and service name were typed in correctly and all the information is correct for your Oracle RAC environment.

  4. Repeat these steps on the other nodes in your cluster that contain instances specified as either Preferred or Available for the newly created service.

Administering Services

You can create and administer services using Enterprise Manager. You can also use the DBMS_SERVICE PL/SQL package and the SRVCTL utility to perform most service administration tasks.

The following sections describe how to manage services for your cluster database:

About Service Administration Using Enterprise Manager

The Cluster Managed Database Services page is the master page for beginning all tasks related to services. To access this page, go to the Cluster Database Maintenance page, then click Cluster Managed Database Services in the Services section. You can use this page and links from this page to do the following:

  • View a list of services for the cluster.

  • View the instances on which each service is currently running.

  • View the status for each service.

  • Create or edit a service.

  • Start or stop a service.

  • Enable or disable a service.

  • Perform instance-level tasks for a service.

  • Delete a service.

See Also:

Using the Cluster Managed Database Services Page

When managing services using Enterprise Manager, you use the Cluster Managed Database Services page.

On the Cluster Managed Database Services page you can perform the following tasks:

  • View a list of services for the cluster, the instances on which each service is currently running, and the status for each service.

  • Start or stop a service, or enable or disable a service.

  • Access the Create Service and Edit Service pages.

  • Access the Services Detail page to perform instance-level tasks for a service.

  • Test the connection for a service.

To access the Cluster Managed Database Services page:

  1. On the Cluster Database Home page, click the Availability tab.

  2. On the Availability subpage, under the Services heading, click Cluster Managed Database Services.

    The Cluster and Database Login page appears.

  3. Enter credentials for the database and for the cluster that hosts the Oracle RAC database, then click Continue.

    The Cluster Managed Database Services page appears and displays the services that are available on the cluster database instances.

See Also:

Using the Cluster Managed Database Services Detail Page

On the Cluster Managed Database Services detail page for a service you can perform the following tasks:

  • View the status of a service on all of its Preferred and Available instances; the status can be Running, Stopped, or Disabled.

  • Stop or start a service for an instance of a cluster database.

  • Disable or enable a service for an instance of a cluster database.

  • Relocate a service to manually rebalance the services across database instances.

To access the Cluster Managed Database Services detail page:

  1. On the Cluster Database Home page, click the Availability tab.

  2. On the Availability subpage, under the Services heading, click Cluster Managed Database Services.

    The Cluster and Database Login page appears.

  3. Enter credentials for the database and for the cluster that hosts the Oracle RAC database, then click Continue.

    The Cluster Managed Database Services page appears and displays the services that are available on the cluster database instances.

  4. Click the name of the service for which you want to view the details.

    The Cluster Managed Database Service detail page for that service appears. In the following screenshot, the detail page for the DEVUSERS service is displayed.

Description of wlm008.gif follows
Description of the illustration wlm008.gif

See Also:

Configuring Service-Level Thresholds

When you create a service, you can specify thresholds for measuring the performance of the service. If the specified threshold value is exceeded by the service, the Automatic Workload Repository (AWR) raises an alert that is displayed by Enterprise Manager.

Performance-related statistics, wait events, and active sessions are monitored at the service level. Also, the AWR enables you to monitor performance using services. It records the service performance, including SQL execution times, wait classes, and resources consumed by a service.

You can specify values for the Elapsed Time Threshold or the CPU Time Threshold when you create a service. You can specify Warning and Critical threshold values for these metrics.

To modify service-level thresholds:

  1. On the Cluster Database Home page, scroll down to the Instances section.

  2. Click the name of the instance for which you want to modify the threshold values for these metrics.

    The Cluster Database Instance Home page appears.

  3. In the Related Links section at the bottom of the page, click Metric and Policy Settings.

    The Metric and Policy Settings page appears.

  4. Set the View to All metrics, then locate either the Service Response Time or the Service CPU Time threshold. Click the Edit icon for that threshold.

    The Edit Advanced Settings page appears.

  5. Enter a threshold value in milliseconds in the Warning Threshold or Critical Threshold field, then click Continue.

    The Metric and Policy Settings page appears.

  6. Edit the threshold values for another metric, or, if done, click OK.

See Also:

Configuring Clients for High Availability

There are two central elements to consider when automating failover for application clients. First, clients that are connected at the time of failure must be quickly and automatically notified that a failure has occurred to avoid waiting for TCP/IP network timeouts before attempting to connect to the new production database (such timeouts range anywhere from 8 minutes to 2 hours, depending on operating system). Oracle RAC configurations utilize Fast Application Notification (FAN) to notify JDBC clients, OCI clients, and ODP.NET clients. FAN event notifications and callouts enable automatic and fast redirection of clients in the event of primary site failure.

The second central element of client failover, is the redirection of clients to the new instance after a failover has occurred, which can be implemented using services. When you create services in an Oracle RAC database, if an instance to which you have assigned a service becomes unavailable, Oracle RAC relocates the service to an available instance in the database. Users will be able to access the service independent of the instance providing it because, using listener registration, all listeners in the cluster are aware of which instances are currently providing a service when a connection request comes in.

This section deals with configurating FAN notificaton for application clients, and contains the following topics:

See Also:

Configuring JDBC Clients

Your application can use the JDBC development environment for either thick or thin JDBC clients. You must use the JDBC Implicit Connection Cache to enable the FAN features of Fast Connection Failover and Runtime Connection Load Balancing.

The JDBC connection pool subscribes to the FAN Load Balancing events automatically when you configure fast connection failover. Instead of randomly assigning a free connection to a work request, the connection pool chooses the connect that will give the best service according to the latest information it has received. If a node becomes hung, the connection pool gradually shifts connections from the hung node to other nodes in the cluster.

To configure JDBC clients for Fast Connection Failover:

  1. Use the Cluster Managed Services page in Oracle Enterprise Manager Database Control or Oracle Enterprise Manager Grid Control to create new services. See "Creating Services" for more information about creating services.

  2. Enable fast connection failover for JDBC clients by setting the DataSource property FastConnectionFailoverEnabled to TRUE, as demonstrated in the following example:

    OracleDataSource ods;
ods = new OracleDataSource();
...
ods.setConnectionCachingEnabled(True);
ods.setFastConnectionFailoverEnabled(True);
ods.setConnectionCacheName("MyCache");
ods.setConnectionCacheProperties(cp);

  3. Set the oracle.net.ns.SQLnetDef.TCP_CONNTIMEOUT_STR property to a nonzero value on the data source (not on the implicit connection cache). When this property is set, if the JDBC client attempts to connect to a host that is unavailable, the connection attempt is bounded to the time specified for oracle.net.ns.SQLnetDef.TCP_CONNTIMEOUT_STR. After the specified time has elapsed and a successful connection has not been made, the client attempts to connect to the next host in the address list. Setting this property to a value of 3 seconds is sufficient for most installations.

  4. Configure JDBC clients to use a connect descriptor that includes a list of the VIP addresses for each node in the cluster, and that connects to an existing service. The following example is for a two-node Oracle Real Application Clusters (Oracle RAC) cluster:

    Sales_JDBC =
  (DESCRIPTION =
    (ADDRESS=(PROTOCOL=TCP)(HOST=docrac1_vip)(PORT=1521))
    (ADDRESS=(PROTOCOL=TCP)(HOST=docrac2_vip)(PORT=1521))
    (LOAD_BALANCE = yes)
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = Sales_JDBC)
    )
  )

    Note:

    Do not configure Transparent Application Failover (TAF) with Fast Connection Failover for JDBC thick clients as TAF processing will interfere with FAN ONS processing.

    If you are using a JDBC thin driver, you must include the complete connect descriptor in the URL because the JDBC thin driver does not use Oracle Net.

  5. Configure a remote Oracle Notification Services (ONS) subscription on the JDBC client so that an ONS daemon is not required on the client, as in the following example:

    ods.setONSConfiguration("docrac1_vip:6200,docrac2_vip:6200");

    The remote ONS subscription must contain every host that the client application can use for failover. In addition, use Secure Sockets Layer (SSL) for all ONS communications, as in the following example:

    ods.setONSConfiguration("nodes=docrac1_vip:6200,
docrac2_vip:6200 walletfile=/mydir/conf/Wallet");

  6. When starting the JDBC application, ensure the ons.jar file (part of the Oracle Client installation) is located in the application CLASSPATH.

See Also:

Universal Connection Pool

Oracle Database 11g (11.1.0.7) includes a new Java connection pool called the Universal Connection Pool (UCP). The Universal Connection Pool is a Java-based connection pool that supports any type of connection (JDBC, LDAP, JCA), to any type of database (Oracle or non-Oracle) with any middle tier (Oracle or non-Oracle). It also supports standalone deployments such as TopLink or BPEL. UCP includes integration features of Oracle Database such as Oracle RAC, including Fast Connection Failover, Runtime Connection Load Balancing, and Connection Affinity for Oracle RAC instances.

To take advantage of FCF and Runtime Connection Load Balancing, you must have both ucp.jar and ons.jar in the application CLASSPATH, and you need to set the UCP Datasource property for FastConnectionFailoverEnabled and ONSConfiguration as shown in the following code example:

PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
Pds.setConnnectionPoolName(“FCFSampleUCP”);
pds.setONSConfiguration(“nodes=docrac1:4200,docrac2:4200”);
pds.setFastConnectionFailoverEnabled(true);
pds.setConnectionFactoryClassName(“oracle.jdbc.pool.OracleDataSource”);
pds.setURL(“jdbc:oracle:thin:@DESCRIPTION=
    (LOAD_BALANCE=on)
    (ADDRESS=(PROTOCOL=TCP)(HOST=host1)(PORT=1521))
    (ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521))
    (ADDRESS=(PROTOCOL=TCP)(HOST=host3)(PORT=1521))
    (ADDRESS=(PROTOCOL=TCP)(HOST=host4)(PORT=1521))
      (CONNECT_DATA=(SERVICE_NAME=service_name)))”);

See Also:

Oracle Universal Connection Pool for JDBC Developer's Guide for more information

Configuring OCI Clients

The Oracle Call Interface (OCI) provides integration with FAN and Load Balancing Advisory events. To take advantage of the Load Balancing Advisory, you need to enable the OCI Session pool. OCI clients can register to receive notifications about Oracle RAC high availability events and respond when events occur. This improves the connection failover response time in OCI and also removes terminated connections from connection and session pools. This feature works for all OCI client applications.

To configure OCI clients to receive FAN notifications:

  1. Use the Cluster Managed Services page in Oracle Enterprise Manager Database Control or Oracle Enterprise Manager Grid Control to a create services for the OCI clients. See "Creating Services" for more information about creating services.

    You should configure the primary instance as preferred for that service. Under Service Properties, set the Transparent Application Failover Policy to Basic. For Notification Properties, choose "Enable Fast Application Notification for OCI and ODP.NET Applications".

    See "Administering Services" for more information about modifying services using Enterprise Manager.

  2. Enable FAN for OCI clients by initializing the environment with the OCI_EVENTS parameter, as in the following example:

    OCIEnvCreate(...OCI_EVENTS...)

  3. Link the OCI client applications with thread library libthread or libpthread.

  4. In your application, you will need to check if an event has occured, using code similar to the following example:

    void evtcallback_fn(ha_ctx, eventhp)
...  
printf("HA Event received.\n");
  if (OCIHandleAlloc( (dvoid *)envhp, (dvoid **)&errhp, (ub4) OCI_HTYPE_ERROR, 
                      (size_t) 0, (dvoid **) 0))
    return;
  if (retcode = OCIAttrGet(eventhp, OCT_HTYPE_EVENT, (dvoid *)&srvhp, (ub4 *)0,
                           OCI_ATTR_HA_SRVFIRST, errhp))
    checkerr (errhp, (sword)retcode;
  else {
     printf("found first server handle.\n");
     /*get associated instance name */
     if (retcode = OCIAttrGet(srvhp, OCI_HTYPE_SERVER, (dvoid *)&instname,
                           (ub4 *)&sizep, OCI_ATTR_INSTNAME, errhp))
       checkerr (errhp, (sword)retcode);
     else
       printf("instance name is %s.\n", instname);

  5. After a HA event is received, clients and applications can register a callback that is invoked whenever a high availability event occurs, as shown in the following example:

    /*Registering HA callback function */
  if (checkerr(errhp, OCIAttrSet(envhp, (ub4) OCI_HTYPE_ENV, 
                             (dvoid *)evtcallback_fn, (ub4) 0,
                             (ub4)OCI_ATTR_EVTCBK, errhp)))
  {
    printf("Failed to set register EVENT callback.\n");
    return EX_FAILURE;
  }
  if (checkerr(errhp, OCIAttrSet(envhp, (ub4) OCI_HTYPE_ENV,
                                (dvoid *)evtctx, (ub4) 0, 
                                (ub4)OCI_ATTR_EVTCTX, errhp)))
  {
    printf("Failed to set register EVENT callback context.\n");
    return EX_FAILURE;
  }
return EX_SUCCESS;

    After registering an event callback and context, OCI will call the registered function once for each high availability event.

See Also:

Configuring ODP.NET Clients

Oracle Data Provider for .NET (ODP.NET) connection pools subscribe to FAN notifications from Oracle RAC that indicate when nodes are down and when services are up or down. Based on these notifications, ODP.NET connection pools make idle connections, connections that were previously connected to node that failed, available again. It also creates new connections to healthy nodes if possible.

ODP.NET provides Runtime Connection Load Balancing to provide enhanced load balancing of the application workload. Instead of randomly selecting an available conenction from the connection pool, it will choose the connection that will provide the best service based on the current workload information.

The procedures for enabling ODP.NET are similar to the procedures for enabling JDBC in that you must set parameters in the connection string to enable FCF. This section explains how to configure Oracle Data Provider for .NET (ODP.NET) clients for failover using FAN events.

To configure ODP.NET clients to receive FAN notifications:

  1. Use the Cluster Managed Services page in Oracle Enterprise Manager Database Control or Oracle Enterprise Manager Grid Control to a create services for the ODP.NET clients. See "Creating Services" for more information about creating services.

    You should configure the primary instance as preferred for that service. Under Service Properties, set the Transparent Application Failover Policy to Basic. For Notification Properties, choose "Enable Fast Application Notification for OCI and ODP.NET Applications". Set the Connection Load Balancing Goal to Long.

  2. Enable Fast Connection Failover for ODP.NET connection pools by subscribing to FAN high availability events. Do this by setting the ha events connection string attribute to true either at connection time or in the data source definition. Note that this only works if you are using connection pools (the pooling attribute is set to true).

    You can also enable Runtime Connection Load Balancing by setting the load balancing connection string attribute to true.

    Use code similar to the following, where username is the name of the database user to which you connect, password is the database password for that user, and the service name is odpserv:

    // C#
using System;
using Oracle.DataAccess.Client;
class HAEventEnablingSample
{
  static void Main()
  {
    OracleConnection con = new OracleConnection();

    // Open a connection using connection pooling
    // Also, enable "load balancing"
    con.ConnectionString =
      "User Id=username;Password=password;Data Source=odpserv;" +
      "Min Pool Size=10;Connection Lifetime=120;Connection Timeout=60;" +
      "HA Events=true";"Incr Pool Size=5;Decr Pool Size=2";

    con.Open();
    // Carry out work against the database here.
    con.Close();
    // Dispose OracleConnection object
    con.Dispose();
    }
 }

  3. The HA events are published as messages in the SYS.SYS$SERVICE_METRICS queue. You must grant dequeue permission on this queue to the database user that the application uses to connect to the database.

    Use a command similar to the following, where username represents the database user that the .NET application uses to connect to the database:

    execute
dbms_aqadm.grant_queue_privilege('DEQUEUE','SYS.SYS$SERVICE_METRICS', username);

    The username specified in this step is the same as the username used for the User Id argument in the previous step.

See Also: