Chapter 11. Remote and Offline JDO

11.1. Detach and Attach
11.1.1. Declaring Detachability
11.1.2. Detach and Attach Behavior
11.1.3. Defining the Detached Object Graph
11.1.4. Detach and Attach Callbacks
11.1.5. Automatic Detachment
11.1.5.1. Detach on Close
11.1.5.2. Detach on Serialize
11.2. Remote Persistence Managers
11.2.1. Standalone Persistence Manager Server
11.2.2. HTTP Persistence Manager Server
11.2.3. Client Persistence Managers
11.2.4. Data Compression and Filtering
11.2.5. Remote Persistence Manager Deployment

The standard JDO runtime environment is local and online. It is local in that JDO components such as persistence managers and queries connect directly to the datastore and execute their actions in the same JVM as the code using them. It is online in that all changes to persistent objects must be made in the context of an active persistence manager. These two properties, combined with the fact that persistence managers cannot be serialized for storage or network transfer, make the standard JDO runtime difficult to incorporate into some enterprise and client/server program designs.

Kodo extends the standard JDO runtime to add remote and offline capabilities in the form of Remote Persistence Managers and Detach and Attach APIs. The following sections explain these capabilities in detail.

11.1. Detach and Attach

A common use case for an application running in a servlet or application server is to "detach" objects from all server resources, modify them, and then "attach" them again. For example, a servlet might store persistent data in a user session between a modification based on a series of web forms. Between each form request, the web container might decide to serialize the session, requiring that the stored persistent state be disassociated from any other resources. Similarly, a client/server or EJB application might transfer persistent objects to a client via serialization, allow the client to modify their state, and then have the client return the modified data in order to be saved. This is sometimes referred to as the "data transfer object" or "value object" pattern, and it allows fine-grained manipulation of data objects without incurring the overhead of multiple remote method invocations (which is one of the things that often makes entity bean-based solutions slow). Version 1 of the JDO specification does not provide direct support for this pattern or any offline modification, since persistent object updates can only take place when the instance is associated with a persistence manager that has a transaction running.

Kodo provides support for this pattern by introducing detach and attach APIs that allow a user to detach a persistent instance, modify the detached instance, and attach the instance back into a persistence manager (either the same one that detached the instance, or a new one). The changes will then be applied to the existing instance from the datastore. Very similar APIs are being considered for inclusion in the JDO 2 specification, so they may become standard in the future.

11.1.1. Declaring Detachability

In order to be able to detach a persistent instance, the metadata for the class must declare that it is eligible for detachment using the detachable extension. Changes to this extension require that the class be re-enhanced. This is because detachability requires that the enhancer add additional fields to the class to hold information about the persistent instance's object id and state.

[Note]Note

Kodo uses reflection to access the detached state, so any detachable classes must be declared public.

The simplest example of the detachable extension is:

public class DetachExample
{
    private String someField;
}


<?xml version="1.0"?>
<jdo>
  <package name="com.somecompany">
    <class name="DetachExample">
      <extension vendor-name="kodo" key="detachable" value="true"/>
    </class>
  </package>
</jdo>

As mentioned previously, when a class is declared to be detachable, the Kodo enhancer adds additional fields to the enhanced version of the class. One of these fields is of type Object, and holds an object that refers to the class' state. Kodo uses this field for bookkeeping information, including the versioning data needed to detect optimistic concurrency violations when the object is re-attached. If the persistent class uses datastore identity, the enhancer adds a second String field used to store the stringified object id of the detached instance.

It is possible to define one or both of these fields yourself. Declaring your own state and identity fields in your class metadata prevents the enhancer from adding any extra fields to the class, and keeps the enhanced class serialization-compatible with the unenhanced version (in case the client tier only has the unenhanced class definition to work with). The detached-objectid-field and detached-state-field class-level metadata extensions name these fields. The fields must not be managed by JDO (they must have their persistence-modifier attribute set to "none").

public class DetachExample
    implements Serializable
{
    private String someField;
    private String detachedObjectId;
    private Object detachedState;
}


<?xml version="1.0"?>
<jdo>
  <package name="com.somecompany">
    <class name="DetachExample">
      <extension vendor-name="kodo" key="detachable" value="true"/>
      <extension vendor-name="kodo" key="detached-objectid-field"
        value="detachedObjectId"/>
      <extension vendor-name="kodo" key="detached-state-field"
        value="detachedState"/>

      <!-- string fields are normally managed by default, so explicitly set -->
      <!-- this field to unmanaged; we don't need to worry about the        -->
      <!-- detachedState fields because fields of type java.lang.Object     -->
      <!-- are not managed by default -->
      <field name="detachedObjectId" persistence-modifier="none"/>
    </class>
  </package>
</jdo>

11.1.2. Detach and Attach Behavior

The KodoPersistenceManager exposes three methods to detach objects:

  • Object detach(Object ob)

  • Object[] detachAll(Object[] obs)

  • Collection detachAll(Collection obs)

Each detach method returns unmanaged copies of the given instances. The copy mechanism is similar to serialization, except that only certain fields are traversed. We will see how to control which fields are detached in a later section.

When detaching an instance that has been modified in the current transaction (and thus made dirty), the current transaction is flushed. This means that when subsequently re-attaching the detached instances, Kodo assumes that the transaction from which they were originally detached was committed; if it has been rolled back, then the re-attachment process will throw a JDOOptimisticVerificationException. You can stop Kodo from assuming the transaction will commit by invoking KodoPersistenceManager.setRollbackOnly prior to detaching your objects. Setting the RollbackOnly flag prevents Kodo from flushing when detaching dirty objects; instead Kodo just runs its pre-store actions (see the KodoPersistenceManager.preStore Javadoc for details). This allows you to use the same instances in multiple attach/modify/detach/rollback cycles. Alternatively, you might also prevent a flush by making your modifications outside of a transaction (with NontransactionalWrite enabled) before detaching.

For every detach method, there is a corresponding attach version:

  • Object attach(Object ob)

  • Object[] attachAll(Object[] obs)

  • Collection attachAll(Collection obs)

Each attach method returns managed copies of the given detached objects. Any changes made to the detached objects are applied to these managed instances. Because attaching involves changing persistent state, you can only attach within a transaction.

If you attempt to attach an instance whose representation has changed in the datastore since detachment, Kodo will throw a JDOOptimisticVerificationException upon commit or flush, just as if a normal optimistic conflict was detected. When attaching an instance whose database record has been deleted since detaching, or when attaching a detached instance into a persistence manager that has a stale version of the object, Kodo will throw a JDOOptimisticVerificationException from the attach method. In these cases, Kodo sets the RollbackOnly flag on the transaction.

Example 11.1. Detaching and Attaching a Single Instance

This example demonstrates a common client/server scenario. The client requests objects and makes changes to them, while the server handles the object lookups and transactions.

import kodo.runtime.*;

...

// CLIENT:
// requests an object with a given oid
DetachExample detached = (DetachExample) getFromServer (oid);

...

// SERVER:
// detaches object and returns to client
Object oid = processClientRequest ();
KodoPersistenceManager kpm = (KodoPersistenceManager) pm;
DetachExample example = (DetachExample) kpm.getObjectById (oid, false);
DetachExample detached = (DetachExample) kpm.detach (example);
sendToClient (detached);

...

// CLIENT:
// makes some modifications and sends back to server
detached.setSomeField ("bar");
sendToServer (detached);

...

// SERVER:
// re-attaches the instance and commit the changes
DetachExample modified = (DetachExample) processClientRequest ();
kpm.currentTransaction ().begin ();
kpm.attach (modified);
kpm.currentTransaction ().commit ();

11.1.3. Defining the Detached Object Graph

After viewing the example above, you may be wondering exactly what state of the object being transferred back and forth is available to the client. Can the client traverse into relations? If so, how deep? Kodo supports three modes for determining which fields and relations of an object are detached. All of the modes are recursive.

  1. KodoPersistenceManager.DETACH_FGS: Detach all fields and relations in the default fetch group, and any other fetch groups that you have added to the persistence manager's fetch configuration. This is the default mode. For more information on custom fetch groups, see Section 14.5, “Fetch Groups”.

  2. KodoPersistenceManager.DETACH_LOADED: Detach all fields and relations that are already loaded, but don't include unloaded fields in the detached graph.

  3. KodoPersistenceManager.DETACH_ALL: Detach all fields and relations. Be very careful when using this mode; if you have a highly-connected domain model, you could end up bringing every object in the database into memory!

Any field that is not included in the set determined by the detach mode is set to its Java default value in the detached instance. You can set the detach mode at any time using the KodoPersistenceManager.setDetachFields(int) method:

KodoPersistenceManager kpm = (KodoPersistenceManager) pm;
kpm.setDetachFields (kpm.DETACH_LOADED);

You can also change the mode persistence managers are initialized with using the kodo.PersistenceManagerImpl configuration property:

# available settings are "fgs", "loaded", "all"
kodo.PersistenceManagerImpl: DetachFields=loaded

Here is an example using the default detach mode with a custom fetch group to control the detached graph declaratively.

Example 11.2. Using Custom Fetch Groups for Detach

public class DetachExample
    implements Serializable
{
    private String someField;
    private DetachExampleRelation someRelation;
}


<?xml version="1.0"?>
<jdo>
  <package name="com.somecompany">
    <class name="DetachExample">
      <extension vendor-name="kodo" key="detachable" value="true"/>
      <field name="someRelation">
        <extension vendor-name="kodo" key="fetch-group" value="mygroup"/>
      </field>
    </class>
    <class name="DetachExampleRelation">
      <extension vendor-name="kodo" key="detachable" value="true"/>
    </class>
  </package>
</jdo>


import kodo.runtime.*;

...

// CLIENT:
// requests an object with a given oid
DetachExample detached = (DetachExample) getFromServer (oid);

...

// SERVER:
// detaches object and returns to client
Object oid = processClientRequest ();
KodoPersistenceManager kpm = (KodoPersistenceManager) pm;
kpm.getFetchConfiguration ().addFetchGroup ("mygroup");
DetachExample example = (DetachExample) kpm.getObjectById (oid, false);
DetachExample detached = (DetachExample) kpm.detach (example);
sendToClient (detached);

...

// CLIENT:
// makes some modifications and sends back to server
detached.setSomeField ("bar");
detached.getRelation ().setSomeOtherField ("baz");
sendToServer (detached);

...

// SERVER:
// re-attaches the instance and commit the changes
DetachExample modified = (DetachExample) processClientRequest ();
kpm.currentTransaction ().begin ();
kpm.attach (modified); // will recursively attach relation and apply mods
kpm.currentTransaction ().commit ();

11.1.4. Detach and Attach Callbacks

Persistent instances can be notified when they are being detached or attached by implementing the kodo.runtime.PreDetachCallback kodo.runtime.PostDetachCallback kodo.runtime.PreAttachCallback and kodo.runtime.PostAttachCallback interfaces. These interfaces are analogous to the notification mechanisms provided by the javax.jdo.InstanceCallbacks interface (see Section 4.4, “InstanceCallbacks”).

  • PreDetachCallback.jdoPreDetach is called on the managed persistent instance before it be to be detached.

  • PostDetachCallback.jdoPostDetach is called on the detached copy of the persistent instance after the entire detach process is complete. The method takes a single java.lang.Object argument, which is the managed instance that the detached copy is the clone of. It can be used to transfer transient or unmanaged state between the managed instance and the detached instance.

  • PreAttachCallback.jdoPreAttach is called on a detached instance before it is to be attached.

  • PostAttachCallback.jdoPostAttach is called on the managed instance after it has been attached. It is only invoked after the entire attach process is complete. The method takes a single java.lang.Object argument, which is the corresponding detached instance. It can be used to transfer transient or unmanaged state between the detached instance and the attached instance.

11.1.5. Automatic Detachment

Automatic detachment encompasses two features: detach-on-close and detach-on-serialize. Each feature is designed to simplify certain patterns of enterprise application development.

11.1.5.1. Detach on Close

Normally when you close a persistence manager, all of its managed objects become invalid. Attempting to manipulate these objects results in undefined behavior; often you will get an exception. With the detach-on-close feature, however, you can configure persistence managers to automatically detach all of their objects when they close. This allows you to continue working with the objects in their detached state, and to later re-attach them to apply any changes you might make. Detach on close is designed for the following scenarios in particular:

  • Client/Server applications with short-lived persistence managers. A common server architecture handles each request by obtaining a persistence manager, looking up persistent objects, closing the persistence manager, and returning the objects to the client. Using detach-on-close ensures that persistent objects remain valid for return without forcing you to detach them explicitly.

  • Servlets and JSPs. A very common pattern in servlet and JSP architectures is to allocate a persistence manager at the beginning of a web request, then close it at the end of the request. The objects used by the persistence manager during the request are usually discarded. Sometimes, though, you need to store persistent objects in the HTTP session for use in later requests. With detach-on-close, you no longer need to explicitly detach the objects you want to retain. Simply place persistent objects into the session, and when you close the persistence manager at the end of the request the objects will detach transparently.

  • Session Beans. Each business method of a session bean is supposed to obtain a persistence manager, use it for the necessary persistence operations, then close it. But you often need to return persistent objects from your business methods, and closing the persistence manager invalidates the objects you're trying to return! The standard workaround is to use makeTransient or detach to separate the return objects from the persistence manager. With detach-on-close, though, this step is no longer needed. If you configure the persistence manager to detach all of its objects when it closes, the return objects will remain valid. The code calling your session bean can access their state, or even modify them and send them back to your bean for re-attachment.

[Note]Note

In transactional enterprise bean methods, calling PersistenceManager.close may not actually close the persistence manager. Rather, the persistence manager stays open until the managed transaction in progress completes. Thus, your return objects may be serialized before the detach-on-close feature is activated. In transactional EJB code, therefore, you should combine detach-on-close with detach-on-serialize, discussed in Section 11.1.5.2, “Detach on Serialize”.

The detach-on-close feature is off by default. You can toggle it at runtime though the KodoPersistenceManager.setDetachOnClose method. You can also set the value new persistence managers are initialized with using the kodo.PersistenceManagerImpl configuration property:

kodo.PersistenceManagerImpl: DetachOnClose=true

11.1.5.2. Detach on Serialize

JDO serialization is designed to be transparent. When a managed object is serialized, it loads all of its persistent fields and relations so that the serialized graph contains the full state of the object. This is in keeping with standard Java serialization, but often it isn't the behavior you want. With a highly connected persistent graph, the recursive serialization process might end up pulling a large percentage of the entire database into memory!

Kodo's detach-on-serialize feature stresses practicality over strict transparency. A class that is configred to use detach on serialize writes a detached version of itself to the serialization stream. This has two significant benefits:

  1. You have control over the size of the serialized graph. Rather than each object reading in its entire state, your detach mode determines which fields are serialized. See Section 11.1.3, “Defining the Detached Object Graph”.

  2. The resulting deserialized objects are detached instances. They retain their persistent identity and version, so you can re-attach them later without creating new datastore records for them.

The detach-on-serialize feature is particulary useful in the following use cases:

  • Client/Server applications with long-lived persistence managers. In a client/server architecture in which the server uses one persistence manager per request, you should use detach-on-close, because the persistence manager will close before any return objects are serialized to the client. But if the server keeps persistence managers open for mulitple requests, detach-on-serialize allows you to pass persistent objects over the wire from the server to the client without explicit detach calls.

  • Session Beans. Each business method of a session bean should close the persistence manager at the end of the method. When there is a managed transaction in progress, however, the persistence manager cannot actually close until the transaction completes. Some application servers will serialize the return value of your business method before completing the transaction, meaning that the persistence manager is still open during serialization. Using detach-on-serialize ensures that the code calling your session bean receives detached copies of the return objects without any explicit detach calls. You should typically combine detach-on-serialize with detach-on-close, covered in Section 11.1.5.1, “Detach on Close”.

Serializing objects to a detached version involves changes to how classes are enhanced. Therefore, the detach-on-serialize feature isn't controlled by a runtime flag or configuration setting. Instead, you tag classes that should serialize to a detached instance by setting the detachable metadata extension to serialize instead of true:

<?xml version="1.0"?>
<jdo>
  <package name="com.somecompany">
    <class name="DetachExample">
      <extension vendor-name="kodo" key="detachable" value="serialize"/>
    </class>
  </package>
</jdo>
[Note]Note

The enhancer implements the java.io.Externalizable interface for classes that detach on serialize. This means that you cannot combine detach-on-serialize with your own custom serialization code.

Also, the detach callbacks discussed in Section 11.1.4, “Detach and Attach Callbacks” are not performed on instances that are detached through serialization.