Using Portable Object Format

27 Using Portable Object Format

You can use Portable Object Format (POF) to serialize Java objects for use in Coherence.

For information on how to work with POF when building .NET and C++ extend clients, see Building Integration Objects for .NET Clients and Building Integration Objects for C++ Clients in Developing Remote Clients for Oracle Coherence..

This chapter includes the following sections:

Overview of POF Serialization

Serialization is the process of encoding an object into a binary format. It is a critical component to working with Coherence because data must be moved around the network. POF is a language agnostic binary format. POF is efficient in both space and time and is a cornerstone technology in Coherence. See The PIF-POF Binary Format.

There are several options available for serialization including standard Java serialization, POF, and your own custom serialization routines. Each has their own trade-offs. Standard Java serialization is easy to implement, supports cyclic object graphs and preserves object identity. Unfortunately, it's also comparatively slow, has a verbose binary format, and is restricted to only Java objects.

POF has the following advantages:

It's language independent with current support for Java, .NET, and C++.
It's very efficient. In a simple test class with a String, a long, and three ints, (de)serialization was seven times faster, and the binary produced was one sixth the size compared with standard Java serialization.
It's versionable. Objects can evolve and have forward and backward compatibility.
It supports the ability to externalize your serialization logic.
It's indexed to allow for extracting values without deserializing the whole object. See Using POF Extractors and POF Updaters.

About Portable Types

Coherence Feature Pack release 14.1.1.2206 introduces Portable Types, which provide a way to add support for POF serialization to your classes by using annotations and without the requirement to implement serialization code by hand.

POF is the preferred serialization format for writing pure Java applications, for the following reasons:

It is significantly faster than other supported serialization formats, such as Java serialization and ExternalizableLite.
It is significantly more compact that other supported serialization formats, allowing you to store more data in a cluster of a given size, and to move less data over the wire.
It supports seamless evolution of data classes, allowing you to upgrade various parts of the application (both storage members and clients) independently of one another, without the risk of losing data in the process.

Over the years, although POF remained largely unchanged, POF Reflection was introduced in Coherence 3.5 (2009), allowing you to extract individual attributes from the POF stream through PofNavigator. See Using POF Extractors and POF Updaters.

The implementation of POF annotations is dependent on Java reflection which impacts the performance benefits of POF. Because of these limitations, legacy POF Annotations have been deprecated in this release. This feature is now replaced with Portable Types.

This section includes the following topics:

Features and Benefits of Portable Types

Unlike POF annotations, Portable Types:

Provide a way to add support for POF serialization to your classes by using annotations and without the need to implement serialization code by hand, just as POF Annotations did.
Implement serialization code at compile-time using byte code instrumentation, and do not rely on Java reflection at runtime. This non-dependency on Java makes them just as fast, but less error-prone, as manually implemented serialization code.
Support, but do not require explicit registration through the POF configuration file because all the metadata required for POF type registration, such as type identifier, and the serializer class to use, are already available in the @PortableType annotation.
Fully support class evolution.

In fact, Portable Types provide a better and more complete evolution support than the implementation of the Evolvable interface by hand.

One of the limitations of the Evolvable interface is that it only supports the evolution of the leaf classes in the class hierarchy. Portable Types do not have this limitation, and allow you to not only evolve any class in the hierarchy, but also to evolve the class hierarchy itself, by adding new classes to any level of the class hierarchy.

Understanding Usage Basics

There are only two basic requirements for Portable Types:

The class must be annotated with the @PortableType annotation

The fields that should be serialized must be annotated with @Portable or one of the related annotations (@PortableDate, @PortableArray, @PortableSet, @PortableList, or @PortableMap)

@PortableType(id = 1)
public class Pet
    {
    @Portable
    protected String name;

    // constructors, accessors, etc.
    }

@PortableType(id = 2)
public class Dog extends Pet
    {
    @Portable
    private String breed;

    // constructors, accessors, etc.
    }

Note:

Non-annotated fields are considered transient.

Additional attribute-level annotations allow you to control certain serialization behaviors that are specific to the type of the attribute.

For example, @PortableDate allows you to control whether you want to serialize date, time, or both when serializing java.util.Date instances (by using the mode property), and whether time zone information should be included (by using the includeTimezone property).

If you are using Java 8 (or later) java.time classes, you can derive this information from the class itself by using the @Portable annotation instead. For example, LocalTime will be serialized as time only, with no time zone information, while the OffsetDateTime will be serialized as both date and time, with time zone information.

Similarly, when serializing arrays, collections and maps, POF allows you to use uniform encoding, where the element type (or key and/or value type, in case of maps) is written into the POF stream only once, instead of once for each element of the collection, resulting in a more compact serialized form.

public class MyClass
    {
    @PortableArray(elementClass = String.class)
    private String[] m_stringArray;

    @PortableSet(elementClass = String.class, clazz = LinkedHashSet.class)
    private Set<String> m_setOfStrings;

    @PortableList(elementClass = String.class)
    private List<String> m_listOfStrings;

    @PortableMap(keyClass = Integer.class, valueClass = String.class, clazz = TreeMap.class)
    private Map<Integer, String> m_uniformMap;
    }

As you can see from the examples above, these annotations also allow you to specify the concrete class that should be created during deserialization for a given attribute. If you do not specify the clazz property, HashSet will be used as the default set type, ArrayList as the default list type, and HashMap as the default map type.

Class Versioning and Evolution

Coherence is a distributed system, and there is no guarantee that every cluster member, and every client process that connects to the cluster, will have the same version of each and every class. In fact, it is certainly true for systems that use rolling upgrades to avoid any downtime.

It is also neither safe nor practical to upgrade the cluster and all the clients at the same time. Therefore, being able to tolerate different versions of the same class across cluster members and clients is not only desirable, but a necessity for many.

The issue is that when a process that has an older version of the class reads serialized data created from the newer version of the same class, it may encounter some attributes that it knows nothing about. Ideally, it should be able to ignore them and read the attributes it needs and knows about, instead of crashing, but that only solves part of the problem. If the process ignores the unknown attributes completely, when it writes the same data back by serializing an older version of the class that is only aware of some attributes, the process will lose the data it previously received but knows nothing about.

Obviously, this is not a desirable scenario for a system that is intended for long-term data storage. POF supports class evolution in a way that ensures that no data is lost, regardless of how many versions of the same class are present across the various cluster and client processes, and regardless of which of those processes read or write the data. The support for class evolution has been in POF from the very beginning, through the Evolvable interface, but Portable Types remove some of the limitations and make the whole process significantly simpler.

Both the class annotation (@PortableType) and the attribute annotations (@Portable and related annotations) provide a way to specify versioning information that is necessary for class evolution.

At the class level, whenever you modify a class by introducing a new attribute, you should increment the version property of the @PortableType annotation.

At the same time, you should specify the since attribute that matches the new class version number for any new class attribute. For example, to add the age attribute to the Pet class, and the color attribute to the Dog class, change the code provided earlier (see Understanding Usage Basics):

@PortableType(id = 1, version = 1)
public class Pet
    {
    @Portable
    protected String name;

    @Portable(since = 1)
    protected int age;

    // constructors, accessors, etc.
    }

@PortableType(id = 2, version = 1)
public class Dog extends Pet
    {
    @Portable
    private String breed;

    @Portable(since = 1)
    private Color color;

    // constructors, accessors, etc.
    }

Notice that both version and since properties are zero-based, which allows you to omit them completely in the initial implementation. It also means that for the first subsequent revision, they should be set to 1.

These are just the defaults. You can certainly set the class and attribute version explicitly to any value even for the initial implementation, if required. The only thing that matters is that you increment the version and set the since property to the latest version number whenever you make changes to the class in future.

For example, if in future, you want to add the height and weight attributes to the Pet class, you should simply increment the version to 2 and set the since property for the new attributes accordingly, as shown below:

@PortableType(id = 1, version = 2)
public class Pet
    {
    @Portable
    protected String name;

    @Portable(since = 1)
    protected int age;

    @Portable(since = 2)
    protected int height;

    @Portable(since = 2)
    protected int weight;

    // constructors, accessors, etc.
    }

Note:

Class evolution allows you to add attributes to the new version of the class, but you should never remove the existing attributes because removing them will break the serialization across the class versions.

However, you can remove or deprecate the attribute accessors from the class, but you should leave the field itself as is to preserve the backwards compatibility of the serialized form.

Similarly, you should avoid renaming the fields because the default serialization order of fields is determined based on the alphabetical order of field names within a given class version (all fields with the same since value).

Using POF Extractors

When using Portable Types, the recommended way to create ValueExtractors is to use the Extractors.fromPof factory method. The general usage is:

Extractors.fromPof(rootClass, propertyPath)

If you have a Pet class and want to extract the name, you can use:

ValueExtractor<Person, String> nameExtractor = Extractors.fromPof(Pet.class, "name");

If you have a Person class with Address property, you can create an extractor for a nested `city` property like this:

ValueExtractor<Person, String> cityExtractor = Extractors.fromPof(Person.class, "address.city");

Instrumenting the Classes at Compile-Time

Annotating the classes is the first step in the implementation of Portable Types, but that alone is not sufficient. To implement the necessary serialization logic, the classes also need to be instrumented at compile time.

The following options are available for instrumentation:

Using the Maven POF Plug-In

You can complete this task by using the pof-maven-plugin plug-in. You should configure this plug-in in the pom.xml file, as shown below:

<plugin>
  <groupId>com.oracle.coherence.ce</groupId>
  <artifactId>pof-maven-plugin</artifactId>
  <version>22.06</version>
  <executions>
    <execution>
      <id>instrument</id>
      <goals>
        <goal>instrument</goal>
      </goals>
    </execution>
    <execution>
      <id>instrument-tests</id>
      <goals>
        <goal>instrument-tests</goal>
      </goals>
    </execution>
  </executions>
</plugin>

The above configuration will discover and instrument all project classes annotated with the @PortableType annotation, including test classes. If you do not need to instrument test classes, you can omit the instrument-tests execution from the plug-in configuration.

The pof-maven-plugin uses the Schema support to define the type system that contains all reachable portable types. This type system includes not only project classes that need to be instrumented, but also all portable types that exist in project dependencies. This is necessary because those dependent types may be used as attributes within the project classes, and therefore, need to be serialized appropriately.

In some cases, it may be necessary to expand the type system with the types that are not annotated with the @PortableType annotation, and are not discovered automatically. This is typically the case when some of your portable types have 'enum' values, or existing classes that implement the PortableObject interface explicitly as attributes.

You can add those types to the schema by creating a META-INF/schema.xml file and specifying them explicitly. For example, if you assume that the Color class from the earlier code examples (see Class Versioning and Evolution is of 'enum' type, then you will need to create the following META-INF/schema.xml file to register it and allow pof-maven-plugin plug-in to instrument the Dog class correctly:

<?xml version="1.0"?>

<schema xmlns="http://xmlns.oracle.com/coherence/schema"
       xmlns:java="http://xmlns.oracle.com/coherence/schema/java"
       external="true">

  <type name="Color">
    <java:type name="petstore.Color"/>
  </type>

</schema>

After all these bits and pieces are in place, you can simply run the build as usual:

$ mvn clean install

You can verify whether the classes are instrumented successfully by checking the Maven output log. You should see something similar to the following:

[INFO] --- pof-maven-plugin:21.12:instrument (instrument) @ petstore ---
[INFO] Running PortableTypeGenerator for classes in /projects/petstore/target/classes
[INFO] Instrumenting type petstore.Pet
[INFO] Instrumenting type petstore.Dog

After you have successfully instrumented the classes, they are ready to be registered and used.

Using the Gradle POF Plug-In

The POF Gradle Plug-in provides automated instrumentation of classes with the @PortableType annotation to generate consistent (and correct) implementations of Evolvable POF serialization methods.

It is not a trivial exercise to manually write serialization methods that support serializing inheritance hierarchies that support the Evolvable concept. However, with static type analysis, these methods can be generated deterministically.

Generating methods deterministically enables developers to focus on business logic rather than implementing boilerplate code for Evolvable POF serialization methods. For more information about portable types, see About Portable Types.

To use the POF Gradle Plug-in, you need to declare it as a plug-in dependency in the build.gradle file:

plugins {
    id 'java'
    id 'com.oracle.coherence.ce' version '23.03'
    }

Note:

Only the open source Gradle POF plug-in will work with Coherence 14.1.1.2206.

Without any further configuration, the plug-in will add a task named coherencePof to your project. The coherencePof task will be executed at the end of the compileJava task. At the same time, coherencePof also depends on the compileJava task.

Therefore, calling gradle compileJava will execute the coherencePof task. Similarly, calling gradle coherencePof will execute the compileJava task first. By default, the coherencePof task will take the build output directory as input for classes to be instrumented, excluding any test classes.

By just adding the plug-in as a dependency, the POF Gradle Plug-in will discover and instrument all project classes annotated with the @PortableType annotation, excluding test classes. If you do need to instrument test classes, you can add the coherencePof closure and provide additional configuration properties.

This section includes the following topics:

Customizing the Gradle Configuration

You can customize the default behavior of the Coherence Gradle Plug-in by using several optional properties. Simply provide a coherencePof closure to the build.gradle script containing any additional configuration properties, as shown in the following example:

coherencePof {
  debug=true 
}

In this example, debug=true will instruct Coherence to provide more logging output for the instrumented classes.

Available Configuration Properties

The following configuration properties are available for use:

Enable Debugging: Set the Boolean debug property to true to instruct the underlying PortableTypeGenerator to generate the debug code for the instrumented classes. If not specified, this property defaults to false.
Instrumentation of Test Classes: Set the Boolean instrumentTestClasses property to true to instrument test classes. If not specified, this property defaults to false.
Set a Custom TestClassesDirectory: Provide a path to a custom test classes directory using the testClassesDirectory property. If not set, it will default to the default test output directory.
Set a Custom MainClassesDirectory: Provide a path to a custom classes directory using the mainClassesDirectory property. If not set, it will default to the default output directory.

Using Classes Without the @PortableType Annotation

In some cases, it may be necessary to expand the type system with the types that are not annotated with the @PortableType annotation, and are not discovered automatically. This is typically the case when some of the portable types have 'enum' values or existing classes that implement the PortableObject interface explicitly as attributes.

You can add those types to the schema by creating a META-INF/schema.xml file and specifying them explicitly. For example, if you are using the Color class:

<?xml version="1.0"?>

<schema xmlns="http://xmlns.oracle.com/coherence/schema"
        xmlns:java="http://xmlns.oracle.com/coherence/schema/java" external="true">

  <type name="Color">
    <java:type name="petstore.Color"/>
  </type>
</schema>

Adding the Jandex Gradle Plug-In

The portable type discovery feature of Coherence depends on the availability of a Jandex index within the modules that provide the portable types that need to be registered.

Therefore, to use the POF instrumented classes at compile-time in the Gradle project, you should also add a Jandex Gradle plug-in. As Oracle Coherence uses Jandex 2 under the covers, there are two options:

Checksum Dependency Plugin

plugins {
  id 'java'
  id 'com.oracle.coherence.ce' version '23.03'
  id 'com.github.vlsi.jandex' version '1.86'
}

jandex-gradle-plugin (only for versions

<
                    1.0.0

as recent versions use Jandex 3.

plugins {
  id 'java'
  id 'com.oracle.coherence.ce' version '23.03'
  id 'org.kordamp.gradle.jandex' version '0.13.2'
}

Processing the Person Class with the Plug-In - An Example

An example Person class (source code shown below) when processed with the plug-in, results in the bytecode shown as the output below:

Example 27-1 Processing the Person Class with the Plug-In

@PortableType(id=1000)
public class Person
    {
    public Person()
        {
        }

    public Person(int id, String name, Address address)
        {
        super();
        this.id = id;
        this.name = name;
        this.address = address;
        }

    int id;
    String name;
    Address address;

    // getters and setters omitted for brevity
    }

Inspect the generated bytecode:

javap Person.class

This should yield the following output:

public class demo.Person implements com.tangosol.io.pof.PortableObject,com.tangosol.io.pof.EvolvableObject {
  int id;
  java.lang.String name;
  demo.Address address;
  public demo.Person();
  public demo.Person(int, java.lang.String, demo.Address);
  public int getId();
  public void setId(int);
  public java.lang.String getName();
  public void setName(java.lang.String);
  public demo.Address getAddress();
  public void setAddress(demo.Address);
  public java.lang.String toString();
  public int hashCode();
  public boolean equals(java.lang.Object);

  public void readExternal(com.tangosol.io.pof.PofReader) throws java.io.IOException; 
  public void writeExternal(com.tangosol.io.pof.PofWriter) throws java.io.IOException;
  public com.tangosol.io.Evolvable getEvolvable(int);
  public com.tangosol.io.pof.EvolvableHolder getEvolvableHolder();
}

The last part of the output shows the additional methods generated by Coherence POF plug-in.

Skipping the Execution of the Task

You can skip the execution of the coherencePof task by running the Gradle build using the -x flag, as shown in the following example:

gradle clean build -x coherencePof

Testing the Plug-In Code During Development

During development, it is extremely useful to rapidly test the plug-in code against separate example projects. For this, you can use Gradle’s composite build feature. See Composing builds. Therefore, the Coherence POF Gradle Plug-in module itself provides a separate sample module. From within the sample directory, run the following command:

gradle clean compileJava --include-build ../plugin

This command will not only build the sample but will also build the plug-in. Thus, developers can make plug-in code changes and see changes reflected rapidly in the execution of the sample module.

Alternatively, you can build and install the Coherence Gradle plug-in to the local Maven repository using the following command:

gradle publishToMavenLocal

For projects to pick up the local changes, ensure the following configuration:

Build.gradle

plugins {
  id 'java'
  id 'com.oracle.coherence.ce' version '23.03'
  id 'com.github.vlsi.jandex' version '1.86'
}

Settings.gradle

pluginManagement {
  repositories {
    mavenLocal()
    gradlePluginPortal()
  }
}

Registration and Discovery

Portable Object Format is not a self-describing serialization format. It replaces platform-specific class names with integer-based type identifiers. Therefore, it needs a way of mapping those type identifiers back to the platform-specific classes. This mapping enables portability across platforms, which is the main objective of POF.

To manage the mappings between the type identifiers and concrete types, POF uses com.tangosol.io.pof.PofContext:

public interface PofContext extends Serializer
    {
    PofSerializer getPofSerializer(int nTypeId);

    int getUserTypeIdentifier(Object o);
    int getUserTypeIdentifier(Class<?> clz);
    int getUserTypeIdentifier(String sClass);

    String getClassName(int nTypeId);
    Class<?> getClass(int nTypeId);

    boolean isUserType(Object o);
    boolean isUserType(Class<?> clz);
    boolean isUserType(String sClass);
    }

It is worth noting that PofContext extends the com.tangosol.io.Serializer interface, which means that you can use any PofContext implementation wherever Coherence expects a Serializer to be specified, that is, within cache services as a storage-level serializer for data classes, as a transport-level serializer between thin clients and the proxy servers, and so on. The PofContext performs the actual serialization by delegating to the appropriate PofSerializer, which is obtained through the PofContext.getPofSerializer method, based on a type identifier.

There are several built-in implementations of PofContext. The SimplePofContext allows you to programmatically register type mappings by providing all the metadata needed for serialization, such as type identifier, class, and the PofSerializer to use:

SimplePofContext ctx = new SimplePofContext();
ctx.registerUserType(1, Pet.class, new PortableTypeSerializer<>(1, Pet.class));
ctx.registerUserType(2, Dog.class, new PortableTypeSerializer<>(2, Dog.class));
ctx.registerUserType(3, Color.class, new EnumPofSerializer());

Notice that a lot of this information is somewhat repetitive and unnecessary when working with Portable Types, as all the metadata you need can be obtained from the class itself or from the @PortableType annotation.

SimplePofContext also provides several convenient methods, specifically for Portable Types:

ctx.registerPortableType(Pet.class);
ctx.registerPortableType(Dog.class);

Or even simpler:

ctx.registerPortableTypes(Pet.class, Dog.class);

While the SimplePofContext is useful for testing and quick prototyping, a PofContext implementation that is much more widely used within Coherence applications is ConfigurablePofContext.

The ConfigurablePofContext allows you to provide type mappings through an external XML file:

<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config coherence-pof-config.xsd">

  <user-type-list>

    <user-type>
      <type-id>1</type-id>
      <class-name>petstore.Pet</class-name>
    </user-type>

    <user-type>
      <type-id>2</type-id>
      <class-name>petstore.Dog</class-name>
    </user-type>

    <user-type>
      <type-id>3</type-id>
      <class-name>petstore.Color</class-name>
      <serializer>
        <class-name>com.tangosol.io.pof.EnumPofSerializer</class-name>
      </serializer>
    </user-type>

  </user-type-list>

</pof-config>

Notice that you did not specify a serializer explicitly for the Pet and Dog classes. This is because ConfigurablePofContext has the logic to determine which of the built-in PofSerializer implementations to use depending on the interfaces implemented by, or the annotations present on the specified class. In this case, it will automatically use PortableTypeSerializer because the classes have the @PortableType annotation.

However, you can make the configuration even simpler by enabling portable type discovery:

<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config coherence-pof-config.xsd">

  <user-type-list>

    <user-type>
      <type-id>3</type-id>
      <class-name>petstore.Color</class-name>
      <serializer>
        <class-name>com.tangosol.io.pof.EnumPofSerializer</class-name>
      </serializer>
    </user-type>

  </user-type-list>

  <enable-type-discovery>true</enable-type-discovery>

</pof-config>

After you set the enable-type-discovery flag to true, the ConfigurablePofContext will discover all the classes annotated with @PortableType and register them automatically, based on the annotation metadata. If you do not use the Color enum that has to be registered explicitly, you can even omit the configuration file completely, as the default pof-config.xml file that is built into Coherence looks like this:

<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config coherence-pof-config.xsd">

  <user-type-list>
    <!-- by default just include coherence POF user types -->
    <include>coherence-pof-config.xml</include>
  </user-type-list>

  <enable-type-discovery>true</enable-type-discovery>

</pof-config>

Note:

The portable type discovery feature depends on the availability of a Jandex index within the modules that provide portable types that need to be registered.

Ensure that you configure the Jandex Maven Plug-in to index classes in your modules at build time:

<plugin>
  <groupId>org.jboss.jandex</groupId>
  <artifactId>jandex-maven-plugin</artifactId>
  <version>1.0.8</version>
  <executions>
    <execution>
      <id>make-index</id>
      <goals>
        <goal>jandex</goal>
      </goals>
      <phase>process-classes</phase>
    </execution>
  </executions>
</plugin>

Providing IDE Support

After you have annotated, instrumented, and registered the Portable Types as described in Class Versioning and Evolution, Instrumenting the Classes at Compile-Time, and Registration and Discovery, you can use them with Coherence just as easily as you would use plain Java Serializable classes, by configuring Coherence services to use pof serializer instead of the default java serializer.

However, there is still one problem. Serialization code is implemented by the pof-maven-plugin plug-in at compile-time, and only if you run the Maven build, which can make it a bit cumbersome to run unit and integration tests within the integrated development environment (IDE).

To solve this problem, Oracle has implemented IDE plug-ins for IntelliJ IDEA and Eclipse. These plug-ins can instrument your classes during incremental or full compilation performed by your IDE. This enables you to test both the serialization of your classes and the code that depends on it without having to run the Maven build or leave your IDE.

For detailed instructions to install and use the plug-in for your favorite IDE, see the documentation for these plug-ins:

Using the Advanced POF Serialization Options

Portable types are the recommended way to serialize POF objects. This section describes the various advanced serialization options related to the use of com.tangosol.io.pof.PofSerializer interface.

This section includes the following topics:

Implementing the PofSerializer Interface

The PofSerializer interface provides a way to externalize the serialization logic from the classes you want to serialize. This is particularly useful when you do not want to change the structure of your classes to work with POF and Coherence. The PofSerializer interface is also made up of two methods:

public Object deserialize(PofReader in)
public void serialize(PofWriter out, Object o)

As with the PortableObject interface, all elements written to or read from the POF stream must be uniquely indexed. Below is an example implementation of the PofSerializer interface:

public Object deserialize(PofReader in) 
   throws IOException 
   {
   Symbol symbol    = (Symbol)in.readObject(0);
   long   ldtPlaced = in.readLong(1);
   bool   fClosed   = in.readBoolean(2);
   
   // mark that reading the object is done
   in.readRemainder();
 
   return new Trade(symbol, ldtPlaced, fClosed);
   }
 
public void serialize(PofWriter out, Object o) 
   throws IOException 
   {
   Trade trade = (Trade) o;
   out.writeObject(0, trade.getSymbol());
   out.writeLong(1, trade.getTimePlaced());
   out.writeBoolean(2, trade.isClosed());
    
   // mark that writing the object is done
   out.writeRemainder(null);
   }

Implementing the Evolvable Interface

When implementing the Evolvable interface, care must be taken to handle adding new elements in new versions of classes. Newer classes should guard against reading newer elements to prevent mishandling data when reading older versions of the classes.

For example, when implementing PortableObject:

public void readExternal(PofReader in)
   throws IOException
   {
   m_symbol    = (Symbol) in.readObject(0);
   m_ldtPlaced = in.readLong(1);
   m_fClosed   = in.readBoolean(2);
  
   if (in.getVersionId() >= 2)
       {
       m_lShares = in.readInt(3);
       }
   }

Or, when implementing PofSerializer:

public Object deserialize(PofReader in)
   throws IOException
   {
   Symbol symbol    = (Symbol)in.readObject(0);
   long   ldtPlaced = in.readLong(1);
   bool   fClosed   = in.readBoolean(2);

   if (in.getVersionId() >= 2)
       {
       lShares = in.readInt(3);
       }
  
   // mark that reading the object is done
   in.readRemainder();
 
   if (in.getVersionId() >= 2)
       {
       return new Trade(symbol, ldtPlaced, fClosed, lShares);
       }
   else
       {
       return new Trade(symbol, ldtPlaced, fClosed);
       }
   }

Writing or serializing does not need to do these checks since the checking is done on reading.

Guidelines for Assigning POF Indexes

Note:

These guidelines are not relevant if you are using Portable Types because property index management is handled internally.

Use the following guidelines when assigning POF indexes to an object's attributes:

Order your reads and writes: start with the lowest index value in the serialization routine and finish with the highest. When deserializing a value, perform reads in the same order as writes.
Non-contiguous indexes are acceptable but must be read/written sequentially.
When Subclassing reserve index ranges: index's are cumulative across derived types. As such, each derived type must be aware of the POF index range reserved by its super class.
Do not re-purpose indexes: to support Evolvable, it's imperative that indexes of attributes are not re-purposed across class revisions.
Label indexes: indexes that are labeled with a public static final int, are much easier to work with, especially when using POF Extractors and POF Updaters. See Using POF Extractors and POF Updaters. Indexes that are labeled must still be read and written out in the same order as mentioned above.

Using POF Object References

This section includes the following topics:

Overview of Using POF Object References

POF supports the use of object identities and references for objects that occur more than once in a POF stream. Objects are labeled with an identity and subsequent instances of a labeled object within the same POF stream are referenced by its identity.

Using references avoids encoding the same object multiple times and helps reduce the data size. References are typically used when a large number of sizeable objects are created multiple times or when objects use nested or circular data structures. However, for applications that contain large amounts of data but only few repeats, the use of object references provides minimal benefits due to the overhead incurred in keeping track of object identities and references.

The use of object identity and references has the following limitations:

Object references are only supported for user defined object types.
Object references are not supported for Evolvable objects.
Object references are not supported for keys.
Objects that have been written out with a POF context that does not support references cannot be read by a POF context that supports references. The opposite is also true.
POF objects that use object identity and references cannot be queried using POF extractors. Instead, use the ValueExtractor API to query object values or disable object references.
The use of the PofNavigator and PofValue API has the following restrictions when using object references:
- Only read operations are allowed. Write operations result in an UnsupportedOperationException.
- User objects can be accessed in non-uniform collections but not in uniform collections.
- For read operations, if an object appears in the data stream multiple times, then the object must be read where it first appears before it can be read in the subsequent part of the data. Otherwise, an IOException: missing identity: <ID> may be thrown. For example, if there are 3 lists that all contain the same person object, p. The p object must be read in the first list before it can be read in the second or third list.

Enabling POF Object References

Object references are not enabled by default and must be enabled either within a pof-config.xml configuration file or programmatically when using the SimplePofContext class.

To enable object references in the POF configuration file, include the <enable-references> element, within the <pof-config> element, and set the value to true. For example:

<?xml version='1.0'?>

<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config
   coherence-pof-config.xsd">
   ...
   <enable-references>true</enable-references>
</pof-config>

To enable object references when using the SimplePofContext class, call the setReferenceEnabled method with a property set to true. For example:

SimplePofContext ctx = new SimplePofContext();
ctx.setReferenceEnabled(true);

Registering POF Object Identities for Circular and Nested Objects

Circular or nested objects must manually register an identity when creating the object. Otherwise, a child object that references the parent object will not find the identity of the parent in the reference map. Object identities can be registered from a serializer during the deserialization routine using the com.tangosol.io.pof.PofReader.registerIdentity method.

The following examples demonstrate two objects (Customer and Product) that contain a circular reference and a serializer implementation that registers an identity on the Customer object.

The Customer object is defined as follows:

public class Customer
   {
   private String m_sName;
   private Product m_product;
 
   public Customer(String sName)
      {
      m_sName = sName;
      }
 
   public Customer(String sName, Product product)
      {
      m_sName = sName;
      m_product = product;
      }

   public String getName()
      {
      return m_sName;
      }

   public Product getProduct()
      {
      return m_product;
      }
 
   public void setProduct(Product product)
      {
      m_product = product;
      }
   }

The Product object is defined as follows:

public class Product
   {
   private Customer m_customer;
 
   public Product(Customer customer)
      {
      m_customer = customer;
      }
 
   public Customer getCustomer()
      {
      return m_customer;
      }
   }

The serializer implementation registers an identity during deserialization and is defined as follows:

public class CustomerSerializer implements PofSerializer
   {
   @Override
   public void serialize(PofWriter pofWriter, Object o) throws IOException
      {
      Customer customer = (Customer) o;
      pofWriter.writeString(0, customer.getName());
      pofWriter.writeObject(1, customer.getProduct());
      pofWriter.writeRemainder(null);
      }
 
   @Override
   public Object deserialize(PofReader pofReader) throws IOException
      {
      String sName = pofReader.readString(0);
      Customer customer = new Customer(sName);

      pofReader.registerIdentity(customer);
      customer.setProduct((Product) pofReader.readObject(1));
      pofReader.readRemainder();
      return customer;
      }
   }

Registering POF Objects

Coherence provides the com.tangosol.io.pof.ConfigurablePofContext serializer class which is responsible for mapping a POF serialized object to an appropriate serialization routine (either a PofSerializer implementation or by calling through the PortableObject interface).

Once your classes have serialization routines, the classes are registered with the ConfigurablePofContext class using a pof-config.xml configuration file. The POF configuration file has a <user-type-list> element that contains a list of classes that implement PortableObject or have a PofSerializer associated with them. The <type-id> for each class must be unique, and must match across all cluster instances (including extend clients). See POF User Type Configuration Elements.

The following is an example of a POF configuration file:

<?xml version='1.0'?>

<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config
   coherence-pof-config.xsd">
   <user-type-list>
      <include>coherence-pof-config.xml</include>

      <!-- User types must be above 1000 -->
      <user-type>
         <type-id>1001</type-id>
         <class-name>com.examples.MyTrade</class-name>
         <serializer>
            <class-name>com.examples.MyTradeSerializer</class-name>
         </serializer>
      </user-type>
 
      <user-type>
        <type-id>1002</type-id>
        <class-name>com.examples.MyPortableTrade</class-name>
      </user-type>
   </user-type-list>
</pof-config>

Note:

Coherence reserves the first 1000 type-id's for internal use. As shown in the above example, the <user-type-list> includes the coherence-pof-config.xml file that is located in the root of the coherence.jar file. This is where Coherence specific user types are defined and should be included in all of your POF configuration files.

Configuring Coherence to Use the ConfigurablePofContext Class

This section includes the following topics:

Overview of Using the ConfigurablePofContext Class

Coherence can be configured to use the ConfigurablePofContext serializer class in three different ways based on the level of granularity that is required:

Per Service – Each service provides a full ConfigurablePofContext serializer class configuration or references a predefined configuration that is included in the operational configuration file.
All Services – All services use a global ConfigurablePofContext serializer class configuration. Services that provide their own configuration override the global configuration. The global configuration can also be a full configuration or reference a predefined configuration that is included in the operational configuration file.
JVM – The ConfigurablePofContext serializer class is enabled for the whole JVM.

Configuring the ConfigurablePofContext Class Per Service

To configure a service to use the ConfigurablePofContext class, add a <serializer> element to a cache scheme in a cache configuration file. See serializer.

The following example demonstrates a distributed cache that is configured to use the ConfigurablePofContext class and defines a custom POF configuration file:

 <distributed-scheme>
   <scheme-name>example-distributed</scheme-name>
   <service-name>DistributedCache</service-name>
   <serializer>
      <instance>
         <class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
         <init-params>
            <init-param>
               <param-type>String</param-type>
               <param-value>my-pof-config.xml</param-value>
            </init-param>
         </init-params>
      </instance>
   </serializer>
</distributed-scheme>

The following example references the default definition in the operational configuration file. See serializer.

 <distributed-scheme>
    <scheme-name>example-distributed</scheme-name>
    <service-name>DistributedCache</service-name>
    <serializer>pof</serializer>
 </distributed-scheme>

Configuring the ConfigurablePofContext Class for All Services

To globally configure the ConfigurablePofContext class for all services, add a <serializer> element within the <defaults> element in a cache configuration file. Both of the below examples globally configure a serializer for all cache scheme definitions and do not require any additional configuration within individual cache scheme definitions. See defaults.

The following example demonstrates a global configuration for the ConfigurablePofContext class and defines a custom POF configuration file:

<?xml version='1.0'?>

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config
   coherence-cache-config.xsd">
   <defaults>
      <serializer>
         <instance>
            <class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
            <init-params>
               <init-param>
                  <param-type>String</param-type>
                  <param-value>my-pof-config.xml</param-value>
               </init-param>
            </init-params>
         </instance>
      </serializer>
   </defaults>
   ...

The following example references the default definition in the operational configuration file. See serializer.

<?xml version='1.0'?>

<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config
   coherence-cache-config.xsd">
   <defaults>
      <serializer>pof</serializer>
   </defaults>
   ...

Configuring the ConfigurablePofContext Class for a JVM Instance

You can configure an entire JVM instance to use POF, using the following system properties:

coherence.pof.enabled=true - Enables POF for the entire JVM instance.
coherence.pof.config=CONFIG_FILE_PATH - The path to the POF configuration file you want to use. If the files is not in the classpath, then it must be presented as a file resource (for example, file:///opt/home/coherence/mycustom-pof-config.xml).

Making POF Configuration Files Discoverable at Runtime

In Java applications, where new modules may be added at deploy time or runtime, the full set of required POF configuration files may not be known ahead of time. Hence, it may not be possible to create a POF configuration file with all the correct <include> elements. To allow for this type of use case, it is possible to make POF configuration files discoverable at runtime by the ConfigurablePofContext class instead of needing to put them inside <include> elements.

To make a configuration file discoverable, create a class that implements com.tangosol.io.pof.PofConfigProvider. The PofConfigProvider has a single method that must be implemented to return the name of the configuration file. At runtime, the ConfigurablePofContext class uses the Java ServiceLoader to discover implementations of PofConfigProvider and load their provided POF configuration files, exactly as if they had been added in <include> elements.

For example, the RuntimeConfigProvider class below provides discovered-pof-config.xml as the configuration file name. At runtime, the ConfigurablePofContext class loads the discovered-pof-config.xml POF configuration file.

RuntimeConfigProvider.java

package com.oracle.coherence.examples;

import com.tangosol.io.pof.PofConfigProvider;

public class RuntimeConfigProvider
        implements PofConfigProvider
    {
    @Override
    public String getConfigURI()
        {
        return "discovered-pof-config.xml";
        }
    }

To make the RuntimeConfigProvider class discoverable:

Create a file META-INF/services/com.tangosol.io.pof.PofConfigProvider containing the following:
```
com.oracle.coherence.examples.RuntimeConfigProvider
```

If using Java Modules, add it to the module-info.java file:

module com.oracle.coherence.examples {
    provides com.tangosol.io.pof.PofConfigProvider
        with com.oracle.coherence.examples.RuntimeConfigProvider;
}

If required, a PofConfigProvider implementation may return multiple POF configuration files by overriding the PofConfigProvider.getConfigURIs() method. In this case, the singular getConfigURI() is not called.

In the example below, the RuntimeConfigProvider, the getConfigURIs() method returns the POF configuration file names discovered-pof-config.xml and additional-pof-config.xml, both of which are loaded by the ConfigurablePofContext at runtime.

package com.oracle.coherence.examples;

import com.tangosol.io.pof.PofConfigProvider;

import java.util.Set;

public class RuntimeConfigProvider
        implements PofConfigProvider
    {
    @Override
    public Set<String> getConfigURIs()
        {
        return Set.of("discovered-pof-config.xml",
                "additional-pof-config.xml");
        }

    @Override
    public String getConfigURI()
        {
        return null;
        }
    }

Using POF Extractors and POF Updaters

In Coherence, the ValueExtractor and ValueUpdater interfaces are used to extract and update values of objects that are stored in the cache.

The PofExtractor and PofUpdater interfaces take advantage of the POF indexed state to extract or update values without the requirement to go through the full serialization/deserialization routines.

PofExtractor and PofUpdater adds flexibility in working with non-primitive types in Coherence. For many extend client cases, a corresponding Java classes in the grid is no longer required. Because POF extractors and POF updaters can navigate the binary, the entire key and value does not have to be deserialized into object form. This implies that indexing can be achieved by simply using POF extractors to pull a value to index on. However, a corresponding Java class is still required when using a cache store. In this case, the deserialized version of the key and value is passed to the cache store to write to the back end.

This section includes the following topics:

Using POF Extractors

POF extractors are typically used when querying a cache and improves query performance. For example, using the class demonstrated above, to query the cache for all contacts with the last names Jones, the query is as follows:

ValueExtractor veName = new PofExtractor(String.class, Contact.LASTNAME);
Filter         filter = new EqualsFilter(veName, "Jones");
 
// find all entries that have a last name of Jones
Set setEntries = cache.entrySet(filter);

In the above case, PofExtractor has a convenience constructor that uses a SimplePofPath to retrieve a singular index, in our case the Contact.LASTNAME index. To find all contacts with the area code 01803, the query is as follows:

ValueExtractor veZip = new PofExtractor(
   String.class, new SimplePofPath(new int[] {Contact.WORK_ADDRESS, Address.ZIP}));
 
Filter filter = new EqualsFilter(veZip, "01803");
 
// find all entries that have a work address in the 01803 zip code
Set setEntries  = cache.entrySet(filter);

Notice that in the previous examples, the PofExtractor constructor has a first argument with the class of the extracted value or null. The reason for passing type information is that POF uses a compact form in the serialized value when possible. For example, some numeric values are represented as special POF intrinsic types in which the type implies the value. As a result, POF requires the receiver of a value to have implicit knowledge of the type. PofExtractor uses the class supplied in the constructor as the source of the type information. If the class is null, PofExtractor infers the type from the serialized state, but the extracted type may differ from the expected type. String types, in fact, can be correctly inferred from the POF stream, so null is sufficient in the previous examples. In general, however, null should not be used.

Using POF Updaters

POF updaters work in the same way as POF extractors except that they update the value of an object rather than extract it. To change all entries with the last name of Jones to Smith, use the UpdaterProcessor class as follows:

ValueExtractor veName  = new PofExtractor(String.class, Contact.LASTNAME);
Filter         filter  = new EqualsFilter(veName, "Jones");
ValueUpdater   updater = new PofUpdater(Contact.LASTNAME);

// find all Contacts with the last name Jones and change them to have the last
// name "Smith"

cache.invokeAll(filter, new UpdaterProcessor(updater, "Smith"));

Note:

while these examples operate on String based values, this functionality works on any POF encoded value.

Serializing Keys Using POF

Key objects, such as value objects, can be serialized using POF. However, there are a few points to consider when serializing keys:

POF defines a cross-platform object format, it cannot always provide a symmetrical conversion. That is, when a serialized object is deserialized, the object type may be different than the original type. This occurs because some data types in Java do not have equivalents in the .NET and C++ platforms. As a result, avoid using classes that potentially have an asymmetrical POF conversion as keys, or parts of keys, for caches and other Java collections.
Avoided using the java.util.Date type. POF is designed to serialize to java.sql.Timestamp (which extends java.util.Date). The wire formats for those two classes are identical, and a deserialization of that wire representation always results in a java.sql.Timestamp instance. Unfortunately, the equals method of those two base classes breaks the symmetry requirement for keys in Java collections. That is, if you have two objects: D (of java.util.Date) and T (of java.sql.Timestamp) that are equivalent from the POF wire format perspective, then D.equals(T) yields true, while T.equals(D) yields false. Therefore, the use of java.util.Date must be avoided. Use a Long representation of the date or the java.sql.Timestamp type to avoid breaking the key symmetry requirement.
Keys that are using POF object references cannot be serialized. In addition, POF object references support circular references. Therefore, you must ensure that your key class does not have circular references.