N Best Practices for Coherence Extend

This chapter describes best practices for configuring and running Coherence*Extend. The following sections are included in this chapter:

Run Proxy Servers with Local Storage Disabled

Each server in a partitioned cache, including the proxy server, can store a portion of the data. The proxy server has the responsibility of accepting POF formatted data from the client (either Java, C++, or .NET), deserializing POF data to get the Java objects, serializing the Java objects, then placing the resulting data in the cluster. These tasks can be expensive in terms of CPU and memory. You can preserve resources on the proxy server by disabling its local storage.

There are several ways in which you can disable storage:

Local storage for a proxy server can be enabled or disabled with the tangosol.coherence.distributed.localstorage Java property. For example:


You can also disable storage in the cache configuration file. See the description of the <local-storage> element in "distributed-scheme".

Storage can also be disabled for the proxy server by modifying the <local-storage> setting in its tangosol-coherence.xml (or tangosol-coherence-override.xml) file. Example N-1 illustrates setting <local-storage> to false in the tangosol-coherence-override.xml) file.

Example N-1 Disabling Storage in tangosol-coherence-override.xml

Example using tangosol-coherence-override.xml
      id value must match what's in tangosol-coherence.xml for DistributedCache service
      <service id="3">
          <init-param id="4">
            <param-value system-property="tangosol.coherence.distributed.localstorage">false</param-value>

Do Not Run a Near Cache on a Proxy Server

By definition, a near cache provides local cache access to recently- and/or often-used data. If a proxy server is configured with a near cache, it will locally cache data accessed by its remote clients. It is unlikely that these clients will be consistently accessing the same subset of data, thus resulting in a low hit ratio on the near cache. This will result in higher heap usage and more network traffic on the proxy nodes with little to no benefit. For these reasons, it is recommended that a near cache not be used on a proxy server. To ensure that the proxy server is not running a near cache, remove all near schemes from the cache configuration being used for the proxy. See "Near Cache" for more information.

Configure Heap NIO Space to be Equal to the Max Heap Size

NIO memory is used for the TCP connection into the proxy and for POF serialization and deserialization. Older Java installations tended to run out of heap memory because it was configured too low. Newer Java JDKs will configure off heap NIO space equal to the max heap space. On Sun JVMs, this can also be set manually with this value:


Set Worker Thread Pool Sizes According to the Needs of the Application

Client applications can be classified into two general categories: active and passive.

In active applications, the Coherence*Extend client sends many requests, such as put, get, and so on, to the proxy. These requests are serviced by the proxy service. The proxy will deserialize POF data put into the cache, and serialize data it returns to the client. For these tasks, configure a larger number of daemon (worker) threads for the proxy service.

In passive applications, the client waits on events (such as map listeners) based on some specified criteria. Events are serviced by the DistributedCache service. This service uses worker threads to push events to the client. For these tasks, the thread pool configuration for the DistributedCache service should include a sufficient number of worker threads.

Note that near caches on extend clients will use map listeners under the covers for the invalidation strategies of ALL, PRESENT, and AUTO. Applications that are write-heavy that use near caches will generate many map events.

Be Careful When Making InvocationService Calls

InvocationService allows a member of a service to invoke arbitrary code on any node in the cluster. On Coherence*Extend however, InvocationService calls are serviced by the proxy that the client is connected to by default. When sending the call through a proxy, you cannot choose the particular node on which the code will run.

Be Careful When Placing Collection Classes in the Cache

If a Coherence*Extend client puts a collection object, (such as an ArrayList, HashSet, HashMap, and so on) directly into the cache, it is deserialized as an immutable array. If you then extract it and cast it to its original type, then a ClassCastExceptions will be returned. As an alternative, uset a Java interface object (such as a List, Set, Map, and so on) or encapsulate the collection object in another object. Both of these techniques are illustrated in the following example:

Example N-2 Casting an ArrayList Object

public class ExtendExample 
    @SuppressWarnings({ "unchecked" })
    public static void main(String asArgs[])
        System.setProperty("tangosol.coherence.cacheconfig", "client-config.xml");
        NamedCache cache = CacheFactory.getCache("test");
        // Create a sample collection
        List list  = new ArrayList();
        for (int i = 0; i < 5; i++)
        cache.put("list", list);
        List listFromCache = (List) cache.get("list");
        System.out.println("Type of list put in cache: " + list.getClass());
        System.out.println("Type of list in cache: " + listFromCache.getClass());

        Map map = new TreeMap();
        for (Iterator i = list.iterator(); i.hasNext();)
            Object o = i.next();
            map.put(o, o);
        cache.put("map", map);
        Map mapFromCache = (Map) cache.get("map");
        System.out.println("Type of map put in cache: " + map.getClass());
        System.out.println("Type of map in cache: " + mapFromCache.getClass());

Run Multiple Proxies Instead of Increasing Thread Pool Size

The proxy performs POF/EL conversions in the service thread. A single proxy instance can easily bottleneck on a single core due to POF/EL conversions. Running multiple proxy instances on the same box (instead of increasing the thread pool size) helps spread the load across more cores.

Configure POF Serializers for Cache Servers

One of the tasks the proxy server performs is to deserialize POF data into Java objects. If you run C++ or .NET applications and store data to the cache, then the conversion to Java objects could be viewed as an unnecessary step. In the current release of Coherence, you have the option of configuring a POF serializer for cache servers. This will have the effect of storing POF format data directly in the cache.

This can have the following impact on your applications:

  • .NET or C++ clients that only perform puts or gets will not require a Java version of the object. Java versions will still be required if deserializing on the server side (for entry processors, cache stores, and so on).

  • POF serializers remove the requirement to serialize/deserialze on the proxy, thus reducing their memory and CPU requirements.

Example N-3 illustrates a fragment from example-pof-server.xml, which configures a POF serializer for the distributed cache. A full POF configuration file example,is attached to this topic.

Example N-3 Configuring a POFSerializer for a Distributed Cache





Use Node Locking Instead of Thread Locking

Coherence*Extend clients can send lock, put, and unlock requests to the cluster. The proxy holds the locks for the client. The requests for locking and unlocking can be issued at the thread level or the node level. In thread level locking, a particular thread instance belonging to the proxy (Thread 1, for example) issues the lock request. If any other threads (Thread 3, for example) issue an unlock request, they will be ignored. A successful unlock request can be issued only by the thread that issued the initial lock request. This can cause application errors since unlock requests will not succeed unless the original thread that issues the lock is also the one that receives the request to release the lock.

In node level locking, if a particular thread instance belonging to the proxy (Thread 1, for example) issues the lock request, then any other thread (Thread 3, for example) can successfully issue an unlock request.

As an alternative to using locks, Coherence recommends that you use the EntryProcessor API instead. EntryProcessors are described in “Chapter 2, "Implement Transactions, Locks, and Concurrency."