Chapter 6 Configuring High Availability Session Persistence and Failover

This chapter explains how to enable and configure high availability session persistence.

• Overview of Session Persistence and Failover

• How SIP Session Replication Works

• Setting Up High Availability Session Persistence

• HTTP/SIP Session Failover

• Stateful Session Bean Failover

Overview of Session Persistence and Failover

Communications Server provides high availability session persistence through failover of HTTP/SIP session data and stateful session bean (SFSB) session data. Failover means that in the event of an server instance or hardware failure, another server instance takes over a distributed session.

Requirements

A distributed session can run in multiple Sun GlassFish Communications Server instances, if:

• Each server instance has access to the same session state data. Communications Server provides the following types of high availability storage for HTTP/SIP session and stateful session bean data:

  • In-memory replication on other servers in the cluster. In-memory replication is enabled by default with the cluster profile.

    The use of in-memory replication requires the Group Management Service (GMS) to be enabled. For more information about GMS, see Group Management Service.

    If server instances in a cluster are located on different machines, ensure that the following prerequisites are met:

    • To ensure that GMS and in-memory replication function correctly, the machines must be on the same subnet.

    • To ensure that in-memory replication functions correctly, the system clocks on all machines in the cluster must be synchronized as closely as possible.

• Each server instance has the same distributable web application deployed to it. The web-app element of the web.xml deployment descriptor file or the sip-app element of the sip.xml deployment descriptor file must contain the distributable element.

• The web application uses high-availability session persistence. If a non-distributable web application is configured to use high-availability session persistence, the server writes an error to the log file.

• The web application must be deployed using the deploy or deploydir command with the --availabilityenabled option set to true. For more information on these commands, see deploy(1) and deploydir(1).

How SIP Session Replication Works

The SIP Session Replication scheme enables Communications Server server to provide high availability and failover.

Consider a cluster with four instances. Each instance has an active cache of SIP Dialog Structures and a replica cache of SIP Dialog Structures. Each Dialog Structure is replicated based on its BEKey. Dialog Structure replicas are created based on the Converged Load Balancer's routing algorithm.

A SIP Dialog Structure is a peer-to-peer SIP relationship between two user agents that persists for a specific time.

In this scheme, consider a scenario in which instance4 fails. The converged load balancer routes the request for DialogStructure1 to instance3, and the request for DialogStructure2 gets routed to instance4.

When the request for DialogStructure1 arrives in instance3, the replica cache of instance3 will already have DialogStructure1 in its replica cache. Therefore, DialogStructure1 is moved to the active cache of instance3 without requiring a network load for DialogStructure1. After DialogStructure1 is loaded to instance3's active cache, DialogStructure1 is replicated to its new partner as determined by the CLB algorithm (say instance2, as shown in the following figure.

The same replication approach is used for DialogStructure2 as well. instance2 is shown as the new replica partner for DialogStructure2 , but it could be instance3 or any other instance as computed by the converged load balancer algorithm.

When the failed node, instance1, gets restarted, the requests for DialogStructure1 and DialogStructure2 get remapped to instance1, as shown in the following figure:

When the instance1 gets the request for DialogStructure1, it moves the DialogStructure1 from instance3's active cache to its active cache. After DialogStructure1 is moved, instance1 replicates DialogStructure1to its replica partner as determined by the converged load balancer routing algorithm (which is instance3, as shown in the figure). Similarly, when the request for DialogStructure2 remaps to instance1, DialogStructure2 is moved from instance4 to instance1. DialogStructure2 is then replicate in instance4.

After instance1 is restarted, the converged load balancer does not route the full traffic immediately. Instead, it routes the incremental traffic by using load-factor settings. After a specific interval (as determined by the load-factor setting), the restarted instance starts receiving full traffic.

Setting Up High Availability Session Persistence

This section explains how to set up high availability session persistence, with the following topics:

• To Set Up High Availability Session Persistence

• Enabling Session Availability

To Set Up High Availability Session Persistence

Before You Begin

High availability session persistence is incompatible with dynamic deployment, dynamic reloading, and autodeployment. These features are for development, not production environments, so you must disable them before enabling HA session persistence. For information about how to disable these features, see Sun GlassFish Communications Server 2.0 Application Deployment Guide.

1. Create an Communications Server cluster.

   For more information, see To Create a Cluster .

2. Set up load balancing for the cluster.

   For more information , see or Chapter 2, Configuring Converged Load Balancing for SIP or converged web/SIP applications.

3. Enable availability for the desired application server instances and web or EJB containers.

   Then configure the session persistence settings. Choose one of these approaches:

   • Use Admin Console. See Enabling Availability for a Server Instance.

   • Use the asadmin command-line utility. See set(1).

4. Restart each server instance in the cluster.

5. Enable availability for any specific SFSB that requires it.

   Select methods for which checkpointing the session state is necessary. See Configuring Availability for an Individual Bean

6. Make each web/SIP module distributable if you want it to be highly available.

7. Enable availability for individual applications, web/SIP modules, or EJB modules during deployment.

   See Configuring Availability for an Individual Application or EJB Module

   In the Administration Console, check the Availability Enabled box, or use the asadmin deploy command with the --availabilityenabled option set to true.

Enabling Session Availability

You can enable session availability at five different scopes (from highest to lowest):

1. Server instance, enabled by default. Enabling session availability for the server instance means that all applications running on the server instance can have high-availability session persistence. For instructions, see next section, Enabling Availability for a Server Instance .

2. Container (web/SIP, or EJB), enabled by default. For information on enabling availability at the container level, see:

   • Configuring Availability for the Web Container

   • Configuring Availability for the SIP Container

   • Configuring Availability for the EJB Container

3. Application, disabled by default.

4. Standalone web/SIP or EJB module, disabled by default.

5. Individual SFSB, disabled by default.

To enable availability at a given scope, you must enable it at all higher levels as well. For example, to enable availability at the application level, you must also enable it at the server instance and container levels.

The default for a given level is the setting at the next level up. For example, if availability is enabled at the container level, it is enabled by default at the application level.

When availability is disabled at the server instance level, enabling it at any other level has no effect. When availability is enabled at the server instance level, it is enabled at all levels unless explicitly disabled.

Enabling Availability for a Server Instance

To enable availability for a server instance, use the asadmin set command to set the configuration’s availability-service.availability-enabled property to true.

For example, if config1 is the configuration name:

asadmin set --user admin --passwordfile password.txt 
--host localhost 
--port 4849 
config1.availability-service.availability-enabled="true"

To Enable Availability for the Server Instance with Admin Console

1. In the tree component, expand the Configurations node.

2. Expand the node for the configuration you want to edit.

3. Select the Availability Service node.

4. In the Availability Service page, enable instance level availability by checking the Availability Service box.

   To disable it, uncheck the box.

5. Click on the Save button.

6. Stop and restart the server instance.

HTTP/SIP Session Failover

Java EE and SIP applications typically have significant amounts of session state data. A web shopping cart is the classic example of session state. Also, an application can cache frequently-needed data in the session object. In fact, almost all applications with significant user interactions need to maintain session state.

Configuring Availability for the Web Container

If you are using in-memory replication to store session state data, you must use the asadmin set command to enable web container availability and to set properties.

For example, use the set command as follows, where config1 is the configuration name:

asadmin set 
config1.availability-service.web-container-availability.availability-enabled="true"

asadmin set 
config1.availability-service.web-container-availability.persistence-frequency="time-based"

To Enable Availability for the Web Container with Admin Console

1. In the tree component, select the desired configuration.

2. Click on Availability Service.

3. Select the Web Container Availability tab.

   Check the Availability Service box to enable availability. To disable it, uncheck the box.

4. Change other settings, as described in the following section, Web Container Availability Settings

5. Restart the server instance.

Web Container Availability Settings

The Web Container Availability tab of the Availability Service enables you to change these availability settings:

Persistence Type: Specifies the session persistence mechanism for web applications that have availability enabled. Allowed values are memory (no persistence) file (the file system), and replicated (memory on other servers).

If web container availability is enabled, the default persistence type depends on the profile, as shown in the following table.

For production environments that require session persistence, use replicated. The memory persistence type and the file persistence type do not provide high availability session persistence.

If web container availability is disabled, the default persistence type memory.

Persistence Frequency: Specifies how often the session state is stored. Applicable only if the Persistence Type is replicated. Allowed values are:

• web-method - The session state is stored at the end of each web request prior to sending a response back to the client. This mode provides the best guarantee that the session state is fully updated in case of failure. This is the default.

• time-based - The session state is stored in the background at the frequency set by the reapIntervalSeconds store property. This mode provides does not guarantee that session state is fully updated. However, it can provide a significant performance improvement because the state is not stored after each request.

Persistence Scope : Specifies how much of the session object and how often session state is stored. Applicable only if the Persistence Type is replicated. Allowed values are as follows:

• session - The entire session state is stored every time. This mode provides the best guarantee that your session data is correctly stored for any distributable web application. This is the default.

• modified-session - The entire session state is stored if it has been modified. A session is considered to have been modified if HttpSession.setAttribute() or HttpSession.removeAttribute() was called. You must guarantee that setAttribute() is called every time an attribute is changed. This is not a Java EE specification requirement, but it is required for this mode to work properly.

• modified-attribute - Only modified session attributes are stored. For this mode to work properly, you must follow a few guidelines:

  • Call setAttribute() every time the session state is modified.

  • Make sure there are no cross-references between attributes. The object graph under each distinct attribute key is serialized and stored separately. If there are any object cross references between the objects under each separate key, they are not serialized and deserialized correctly.

  • Distribute the session state across multiple attributes, or at least between a read-only attribute and a modifiable attribute.

Single Sign-On State: Check this box to enable persistence of the single sign-on state. To disable it, uncheck the box. For more information, see Using Single Sign-on with Session Failover

Configuring Availability for Individual Web Applications

To enable and configure availability for an individual web or converged web or SIP application, edit the application deployment descriptor file, sun-web.xml. The settings in an application’s deployment descriptor override the web container’s availability settings.

The session-manager element’s persistence-type attribute determines the type of session persistence an application uses. It must be set to replicated to enable high availability session persistence.

For more information about the sun-web.xml file, see The sun-web.xml File in Sun GlassFish Communications Server 2.0 Application Deployment Guide.

Example

<sun-web-app> ... 
  <session-config> 
    <session-manager persistence-type="replicated"> 
      <manager-properties> 
        <property name="persistenceFrequency" value="web-method" /> 
      </manager-properties> 
      <store-properties> 
        <property name="persistenceScope" value="session" /> 
      </store-properties> 
    </session-manager> ... 
</session-config> ...

Configuring Availability for the SIP Container

Use the asadmin set command to enable web container availability and to change availability settings. For example, use the set command as follows, where config1 is the configuration name:

asadmin set 
config1.availability-service.sip-container-availability.availability-enabled="true"

asadmin set 
config1.availability-service.sip-container-availability.persistence-frequency="time-based"

To Enable Availability for the SIP Container with Admin Console

1. In the tree component, select the desired configuration.

2. Click on Availability Service.

3. Select the SIP Container Availability tab.

   Check the Availability Service box to enable availability. To disable it, uncheck the box.

4. Change other settings, as described in the following section, SIP Container Availability Settings.

5. Restart the server instance.

SIP Container Availability Settings

The SIP Container Availability tab of the Availability Service enables you to change these availability settings:

Persistence Type: Specifies the session persistence mechanism for web applications that have availability enabled. Allowed values are memory (no persistence) and replicated (memory on other servers). If SIP container availability is disabled, the default persistence type memory. If SIP container availability is enabled, the default persistence type is replicated. For production environments that require session persistence, use replicated. The memory persistence type does not provide high availability session persistence.

Persistence Frequency: Specifies how often the session state is stored. Applicable only if the Persistence Type is replicated. Allowed values are:

• sip-transaction - The session state is stored at the end of each request prior to sending a response back to the client. This mode provides the best guarantee that the session state is fully updated in case of failure. This is the default.

• time-based - The session state is stored in the background at the frequency set by the reapIntervalSeconds store property. This mode provides does not guarantee that session state is fully updated. However, it can provide a significant performance improvement because the state is not stored after each request.

Persistence Scope : Specifies how much of the session object and how often session state is stored. Applicable only if the Persistence Type is replicated. Allowed values are as follows:

• session - The entire session state is stored every time. This mode provides the best guarantee that your session data is correctly stored for any distributable web application. This is the default.

• modified-session - The entire session state is stored if it has been modified. A session is considered to have been modified if SipApplicationSession.setAttribute() or SipApplicationSession.removeAttribute() was called. You must guarantee that setAttribute() is called every time an attribute is changed. This is not a Java EE specification requirement, but it is required for this mode to work properly.

• modified-attribute - Only modified session attributes are stored. For this mode to work properly, you must follow a few guidelines:

  • Call setAttribute() every time the session state is modified.

  • Make sure there are no cross-references between attributes. The object graph under each distinct attribute key is serialized and stored separately. If there are any object cross references between the objects under each separate key, they are not serialized and deserialized correctly.

  • Distribute the session state across multiple attributes, or at least between a read-only attribute and a modifiable attribute.

Single Sign-On State: Check this box to enable persistence of the single sign-on state. To disable it, uncheck the box. For more information, see Using Single Sign-on with Session Failover.

Configuring Availability for Individual SIP Applications

To enable and configure availability for an individual SIP application, edit the application deployment descriptor file, sun-sip.xml. The settings in an application’s deployment descriptor override the SIP container’s availability settings.

For a converged web/SIP application, edit sun-web.xml to configure the web container and sun-sip.xml to configure the SIP container.

The session-manager element’s persistence-type attribute determines the type of session persistence an application uses. It must be set to replicated to enable high availability session persistence.

For more information about the sun-sip.xml file, see The sun-sip.xml File in Sun GlassFish Communications Server 2.0 Application Deployment Guide.

Example

<sun-web-app> ... 
  <session-config> 
    <session-manager persistence-type="replicated"> 
      <manager-properties> 
        <property name="persistenceFrequency" value="sip-transaction" /> 
      </manager-properties> 
      <store-properties> 
        <property name="persistenceScope" value="session" /> 
      </store-properties> 
    </session-manager> ... 
</session-config> ...

Using Single Sign-on with Session Failover

In a single application server instance, once a user is authenticated by an application, the user is not required to re-authenticate individually to other applications running on the same instance. This is called single sign-on. For more information, see User Authentication for Single Sign-on in Sun GlassFish Communications Server 2.0 Developer’s Guide.

For this feature to continue to work even when an HTTP/SIP session fails over to another instance in a cluster, single sign-on information must be persisted using in-memory replication. To persist single sign-on information, first, enable availability for the server instance and the web container, then enable single-sign-on state failover.

You can enable single sign-on state failover with the Admin Console in the Web Container Availability or SIP Container Availability tab of the Availability Service, as described in Configuring Availability for the Web Container. You can also use the asadmin set command to set the configuration’s availability-service.web-container-availability.sso-failover-enabled property to true.

For example, use the set command as follows, where config1 is the configuration name:

asadmin set --user admin --passwordfile password.txt 
--host localhost --port 4849 
config1.availability-service.web-container-availability.
sso-failover-enabled="true"

Single Sign-On Groups

Applications that can be accessed through a single name and password combination constitute a single sign-on group. For HTTP/SIP sessions corresponding to applications that are part of a single sign-on group, if one of the sessions times out, other sessions are not invalidated and continue to be available. This is because time out of one session should not affect the availability of other sessions.

As a corollary of this behavior, if a session times out and you try to access the corresponding application from the same browser window that was running the session, you are not required to authenticate again. However, a new session is created.

Take the example of a shopping cart application that is a part of a single sign-on group with two other applications. Assume that the session time out value for the other two applications is higher than the session time out value for the shopping cart application. If your session for the shopping cart application times out and you try to run the shopping cart application from the same browser window that was running the session, you are not required to authenticate again. However, the previous shopping cart is lost, and you have to create a new shopping cart. The other two applications continue to run as usual even though the session running the shopping cart application has timed out.

Similarly, suppose a session corresponding to any of the other two applications times out. You are not required to authenticate again while connecting to the application from the same browser window in which you were running the session.

This behavior applies only to cases where the session times out. If single sign-on is enabled and you invalidate one of the sessions using HttpSession.invalidate() or SipApplicationSession.invalidate(), the sessions for all applications belonging to the single sign-on group are invalidated. If you try to access any application belonging to the single sign-on group, you are required to authenticate again, and a new session is created for the client accessing the application.

Stateful Session Bean Failover

Stateful session beans (SFSBs) contain client-specific state. There is a one-to-one relationship between clients and the stateful session beans. At creation, the EJB container gives each SFSB a unique session ID that binds it to a client.

An SFSB’s state can be saved in a persistent store in case a server instance fails. The state of an SFSB is saved to the persistent store at predefined points in its life cycle. This is called checkpointing. If enabled, checkpointing generally occurs after the bean completes any transaction, even if the transaction rolls back.

However, if an SFSB participates in a bean-managed transaction, the transaction might be committed in the middle of the execution of a bean method. Since the bean’s state might be undergoing transition as a result of the method invocation, this is not an appropriate time to checkpoint the bean’s state. In this case, the EJB container checkpoints the bean’s state at the end of the corresponding method, provided the bean is not in the scope of another transaction when that method ends. If a bean-managed transaction spans across multiple methods, checkpointing is delayed until there is no active transaction at the end of a subsequent method.

The state of an SFSB is not necessarily transactional and might be significantly modified as a result of non-transactional business methods. If this is the case for an SFSB, you can specify a list of checkpointed methods, as described in Specifying Methods to Be Checkpointed

If a distributable web application references an SFSB, and the web application’s session fails over, the EJB reference is also failed over.

If an SFSB that uses session persistence is undeployed while the Communications Server instance is stopped, the session data in the persistence store might not be cleared. To prevent this, undeploy the SFSB while the Communications Server instance is running.

Configuring Availability for the EJB Container

To Enable Availability for the EJB Container

1. Select the EJB Container Availability tab.

2. Check the Availability Service box.

   To disable availability, uncheck the box.

3. Change other settings as described in Availability Settings

4. Click on the Save button.

5. Restart the server instance.

Equivalent asadmin command

To enable availability for the EJB container use the asadmin set command to set the following three properties for the configuration:

• availability-service.ejb-container-availability.availability-enabled

• availability-service.ejb-container-availability.sfsb-persistence-type

• availability-service.ejb-container-availability.sfsb-ha-persistence-type

For example, if config1 is the configuration name, use the following commands:

asadmin set --user admin --passwordfile password.txt --host localhost --port 4849config1.availability-service.ejb-container-availability.availability-enabled="true"

asadmin set --user admin --passwordfile password.txt --host localhost --port 4849config1.availability-service.ejb-container-availability.sfsb-persistence-type="file"

asadmin set --user admin --passwordfile password.txt --host localhost --port 4849config1.availability-service.ejb-container-availability.sfsb-ha-persistence-type="replicated"

Availability Settings

The EJB Container Availability tab of the Availability Service enables you to change these settings:

HA Persistence Type: Specifies the session persistence and passivation mechanism for SFSBs that have availability enabled. Allowed values are file (the file system), and replicated (memory on other servers). The default value is replicated. For production environments that require session persistence, use replicated.

SFSB Persistence Type: Specifies the passivation mechanism for SFSBs that do not have availability enabled. Allowed values are file (the default) and replicated.

If either Persistence Type is set to file, the EJB container specifies the file system location where the passivated session bean state is stored. Checkpointing to the file system is useful for testing but is not for production environments. For information about configuring store properties, see the Admin Console online help.

HA persistence enables a cluster of server instances to recover the SFSB state if any server instance fails. The HA store is also used as the passivation and activation store. Use this option in a production environment that requires SFSB state persistence.

Configuring the SFSB Session Store When Availability Is Disabled

If availability is disabled, the local file system is used for SFSB state passivation, but not persistence. To change where the SFSB state is stored, change the Session Store Location setting in the EJB container. For information about configuring store properties, see the Admin Console online help.

Configuring Availability for an Individual Application or EJB Module

You can enable SFSB availability for an individual application or EJB module during deployment:

• If you are deploying with the Admin Console, check the Availability Enabled checkbox.

• If you are deploying using use the asadmin deploy or asadmin deploydir commands, set the --availabilityenabled option to true. For more information, see deploy(1) and deploydir(1).

Configuring Availability for an Individual Bean

To enable availability and select methods to be checkpointed for an individual SFSB, use the sun-ejb-jar.xml deployment descriptor file. .

To enable high availability session persistence, set availability-enabled="true" in the ejb element. To control the size and behavior of the SFSB cache, use the following elements:

• max-cache-size : specifies the maximum number of session beans that are held in cache. If the cache overflows (the number of beans exceeds max-cache-size), the container then passivates some beans or writes out the serialized state of the bean into a file. The directory in which the file is created is obtained from the EJB container using the configuration APIs.

• resize-quantity

• cache-idle-timeout-in-seconds

• removal-timeout-in-seconds

• victim-selection-policy

For more information about sun-ejb-jar.xml, see The sun-ejb-jar.xml File in Sun GlassFish Communications Server 2.0 Application Deployment Guide.

Example 6–1 Example of an EJB Deployment Descriptor With Availability Enabled

<sun-ejb-jar>
    ...
    <enterprise-beans>
        ...
        <ejb availability-enabled="true">
            <ejb-name>MySFSB</ejb-name>
        </ejb>
        ...
    </enterprise-beans>
</sun-ejb-jar>

Specifying Methods to Be Checkpointed

If enabled, checkpointing generally occurs after the bean completes any transaction, even if the transaction rolls back. To specify additional optional checkpointing of SFSBs at the end of non-transactional business methods that cause important modifications to the bean’s state, use the checkpoint-at-end-of-method element in the ejb element of the sun-ejb-jar.xml deployment descriptor file.

The non-transactional methods in the checkpoint-at-end-of-method element can be:

• create() methods defined in the home interface of the SFSB, if you want to checkpoint the initial state of the SFSB immediately after creation

• For SFSBs using container managed transactions only, methods in the remote interface of the bean marked with the transaction attribute TX_NOT_SUPPORTED or TX_NEVER

• For SFSBs using bean managed transactions only, methods in which a bean managed transaction is neither started nor committed

  Any other methods mentioned in this list are ignored. At the end of invocation of each of these methods, the EJB container saves the state of the SFSB to persistent store.

If an SFSB does not participate in any transaction, and if none of its methods are explicitly specified in the checkpoint-at-end-of-method element, the bean’s state is not checkpointed at all even if availability-enabled="true" for this bean.

For better performance, specify a small subset of methods. The methods should accomplish a significant amount of work or result in important modification to the bean’s state.

Example 6–2 Example of EJB Deployment Descriptor Specifying Methods Checkpointing

<sun-ejb-jar>
    ...
    <enterprise-beans>
        ...
        <ejb availability-enabled="true">
            <ejb-name>ShoppingCartEJB</ejb-name>
            <checkpoint-at-end-of-method>
                <method>
                    <method-name>addToCart</method-name>
                </method>
            </checkpoint-at-end-of-method>
        </ejb>
        ...
    </enterprise-beans>
</sun-ejb-jar>