Skip Headers
Oracle® Coherence Developer's Guide
Release 3.7.1

Part Number E22837-01
Go to Documentation Home
Home
Go to Book List
Book List
Go to Table of Contents
Contents
Go to Feedback page
Contact Us

Go to previous page
Previous
Go to next page
Next
View PDF

13 Implementing Storage and Backing Maps

This chapter provides information on storage using backing maps. The following sections are included in this chapter:

13.1 Cache Layers

Partitioned (Distributed) cache service in Coherence has three distinct layers:

Coherence allows users to configure some out-of-the-box backing map implementations and custom ones. Basically, the only constraint that all these Map implementation have to be aware of is the understanding that the Storage Manager provides all keys and values in internal (Binary) format. To deal with conversions of that internal data to and from an Object format, the Storage Manager can supply Backing Map implementations with a BackingMapManagerContext reference.

Figure 13-1 shows a conceptual view of backing maps.

Figure 13-1 Backing Map Storage

storage architecture in Coherence

13.2 Local Storage

Local storage refers to the data structures that actually store or cache the data that is managed by Coherence. For an object to provide local storage, it must support the same standard collections interface, java.util.Map. When a local storage implementation is used by Coherence to store replicated or distributed data, it is called a backing map because Coherence is actually backed by that local storage implementation. The other common uses of local storage is in front of a distributed cache and as a backup behind the distributed cache.

Coherence supports the following local storage implementations:

13.3 Operations

There are number of operation types performed against the Backing Map:

13.4 Capacity Planning

Depending on the actual implementation, the Backing Map stores the cache data in the following ways:

Keeping data in memory naturally provides dramatically smaller access and update latencies and is most commonly used.

More often than not, applications must ensure that the total amount of data placed into the data grid does not exceed some predetermined amount of memory. It could be done either directly by the application tier logic or automatically using size- or expiry-based eviction. Quite naturally, the total amount of data held in a Coherence cache equals the sum of data volume in all corresponding backing maps (one per each cluster node that runs the corresponding partitioned cache service in a storage enabled mode).

Consider following cache configuration excerpts:

<backing-map-scheme>
  <local-scheme/>
</backing-map-scheme>

The backing map above is an instance of com.tangosol.net.cache.LocalCache and does not have any pre-determined size constraints and has to be controlled explicitly. Failure to do so could cause the JVM to go out-of-memory.

<backing-map-scheme>
  <local-scheme>
    <eviction-policy>LRU</eviction-policy>
    <high-units>100m</high-units>
    <unit-calculator>BINARY</unit-calculator>
  </local-scheme>
</backing-map-scheme>

This backing map above is also a com.tangosol.net.cache.LocalCache and has a capacity limit of 100MB. As the total amount of data held by this backing map exceeds that high watermark, some entries are removed from the backing map, bringing the volume down to the low watermark value (<low-units> configuration element, witch defaults to 75% of the <high-units>). The choice of the removed entries is based on the LRU (Least Recently Used) eviction policy. Other options are LFU (Least Frequently Used) and Hybrid (a combination of the LRU and LFU). The value of <high-units> is limited to 2GB. To overcome that limitation (but maintain backward compatibility) Coherence uses the <unit-factor> element. For example, the <high-units> value of 8192 with a <unit-factor> of 1048576 results in a high watermark value of 8GB.

<backing-map-scheme>
  <local-scheme>
    <expiry-delay>1h</expiry-delay>
  </local-scheme>
</backing-map-scheme>

The backing map above automatically evicts any entries that have not been updated for more than an hour. Note, that such an eviction is a "lazy" one and can happen any time after an hour since the last update happens; the only guarantee Coherence provides is that entries that exceed one hour are not returned to a caller.

The following backing map is an instance of com.tangosol.net.cache.SerializationCache which stores values in the extended (nio) memory and has a capacity limit of 100MB (100*1048576).

<backing-map-scheme>
  <external-scheme>
    <nio-memory-manager>
      <initial-size>1MB</initial-size>
      <maximum-size>100MB</maximum-size>
    </nio-memory-manager>
    <high-units>100</high-units>
    <unit-calculator>BINARY</unit-calculator>
    <unit-factor>1048576</unit-factor>
  </external-scheme>
</backing-map-scheme>

Configure a backup storage for this cache being off-heap (or file-mapped):

<backup-storage>
  <type>off-heap</type>
  <initial-size>1MB</initial-size>
  <maximum-size>100MB</maximum-size>
</backup-storage>

13.5 Using Partitioned Backing Maps

The conventional backing map implementation contained entries for all partitions owned by the corresponding node. (During partition transfer, it could also hold "in flight" entries that from the clients' perspective are temporarily not owned by anyone).

Figure 13-2 shows a conceptual view of the conventional backing map implementation.

Figure 13-2 Conventional Backing Map Implementation

conventional backing map implementation

A partitioned backing map is basically a multiplexer of actual Map implementations, each of which would contain only entries that belong to the same partition.

Figure 13-3 shows a conceptual view of the partitioned backing map implementation.

Figure 13-3 Partitioned Backing Map Implementation

partitioned backing map

To configure a partitioned backing map, add a <partitioned> element with a value of true. For example:

<backing-map-scheme>
  <partitioned>true</partitioned>
  <external-scheme>
    <nio-memory-manager>
      <initial-size>1MB</initial-size>
      <maximum-size>50MB</maximum-size>
    </nio-memory-manager>
    <high-units>8192</high-units>
    <unit-calculator>BINARY</unit-calculator>
    <unit-factor>1048576</unit-factor>
  </external-scheme>
</backing-map-scheme>

This backing map is an instance of com.tangosol.net.partition.PartitionSplittingBackingMap, with individual partition holding maps being instances of com.tangosol.net.cache.SerializationCache that each store values in the extended (nio) memory. The individual nio buffers have a limit of 50MB, while the backing map as whole has a capacity limit of 8GB (8192*1048576). Again, you must configure a backup storage for this cache being off-heap or file-mapped.

13.6 Using the Elastic Data Feature to Store Data

The Elastic Data feature is used to seamlessly store data across memory and disk-based devices. This feature is especially tuned to take advantage of fast disk-based devices such as Solid State Disks (SSD) and enables near memory speed while storing and reading data from SSDs. The Elastic Data feature uses a technique called journaling to optimize the storage across memory and disk.

Elastic data contains two distinct components: the RAM journal for storing data in-memory and the flash journal for storing data to disk-based devices. These can be combined in different combinations and are typically used for backing maps and backup storage but can also be used with composite caches (for example, a near cache). The RAM journal always works with the flash journal to enable seamless overflow to disk.

Caches that use RAM and flash journals are configured as part of a cache scheme definition within a cache configuration file. Journaling behavior is configured, as required, by using an operational override file to override the out-of-box configuration.

The following topics are included in this section:

13.6.1 Journaling Overview

Journaling refers to the technique of recording state changes in a sequence of modifications called a journal. As changes occur, the journal records each value for a specific key and a tree structure that is stored in memory keeps track of which journal entry contains the current value for a particular key. To find the value for an entry, you find the key in the tree which includes a pointer to the journal entry that contains the latest value.

As changes in the journal become obsolete due to new values being written for a key, stale values accumulate in the journal. At regular intervals, the stale values are evacuated making room for new values to be written in the journal.

The Elastic Data feature includes a RAM journal implementation and a Flash journal implementation that work seamlessly with each other. If for example the RAM Journal runs out of memory, the Flash Journal automatically accepts the overflow from the RAM Journal, allowing for caches to expand far beyond the size of RAM.

Note:

When journaling is enabled, additional capacity planning is required if you are performing data grid operations (such as queries and aggregations) on large result sets. See Oracle Coherence Administrator's Guide for details.

A resource manager controls journaling. The resource manager creates and utilizes a binary store to perform operations on the journal. The binary store is implemented by the JournalBinaryStore class. All reads and writes through the binary store are handled by the resource manager. There is a resource manager for RAM journals (RamJournalRM) and one for flash journals (FlashJournalRM). Lastly, journaling uses the SimpleSerializationMap class as the backing map implementation. Custom implementation of SimpleSerializationMap can be created as required. See Oracle Coherence Java API Reference for specific details on these APIs.

13.6.2 Defining Journal Schemes

The <ramjournal-scheme> and <flashjournal-scheme> elements are used to configure RAM and Flash journals (respectively) in a cache configuration file. See the "ramjournal-scheme" and the "flashjournal-scheme" for detailed configuration options for these scheme types.

The following topics are included in this section:

13.6.2.1 Configuring a RAM Journal Backing Map

To configure a RAM journal backing map, add the <ramjournal-scheme> element within the <backing-map-scheme> element of a cache definition. The following example creates a distributed cache that uses a RAM journal for the backing map. The RAM journal automatically delegates to a flash journal when the RAM journal exceeds the configured memory size. See "Changing Journaling Behavior" to change memory settings.

<distributed-scheme>
   <scheme-name>distributed-journal</scheme-name>
   <service-name>DistributedCacheRAMJournal</service-name>
   <backing-map-scheme>
      <ramjournal-scheme/>
   </backing-map-scheme>
   <autostart>true</autostart>
</distributed-scheme>

13.6.2.2 Configuring a Flash Journal Backing Map

To configure a flash journal backing map, add the <flashjournal-scheme> element within the <backing-map-scheme> element of a cache definition. The following example creates a distributed scheme that uses a flash journal for the backing map.

<distributed-scheme>
   <scheme-name>distributed-journal</scheme-name>
   <service-name>DistributedCacheFlashJournal</service-name>
   <backing-map-scheme>
      <flashjournal-scheme/>
   </backing-map-scheme>
   <autostart>true</autostart>
</distributed-scheme>

13.6.2.3 Referencing a Journal Scheme

The RAM and flash journal schemes both support the use of scheme references to reuse scheme definitions. The following example creates a distributed cache and configures a RAM journal backing map by referencing the RAM scheme definition called default-ram.

<caching-schemes>
   <distributed-scheme>
      <scheme-name>distributed-journal</scheme-name>
         <service-name>DistributedCacheJournal</service-name>
         <backing-map-scheme>
            <ramjournal-scheme>
               <scheme-ref>default-ram</scheme-ref>
            </ramjournal-scheme>
         </backing-map-scheme>
         <autostart>true</autostart>
   </distributed-scheme>

   <ramjournal-scheme>
      <scheme-name>default-ram</scheme-name>
   </ramjournal-scheme>
</caching-schemes>

13.6.2.4 Using a Journal Scheme for Backup Storage

Journal schemes are used for backup storage as well as for backing maps. By default, a distributed scheme that is configured to use a RAM journal as a backing map also uses a RAM journal for backup storage. Similarly, a distributed scheme that uses a flash journal for a backing map also uses a flash journal for backup storage. This default behavior can be modified by explicitly specifying the storage type within the <backup-storage> element. The following configuration uses a RAM journal for the backing map and explicitly configures a flash journal for backup storage:

<caching-schemes>
   <distributed-scheme>
      <scheme-name>default-distributed-journal</scheme-name>
         <service-name>DistributedCacheJournal</service-name>
         <backup-storage>
            <type>scheme</type>
            <scheme-name>example-flash</scheme-name>
         </backup-storage>
         <backing-map-scheme>
            <ramjournal-scheme/>
         </backing-map-scheme>
      <autostart>true</autostart>
   </distributed-scheme>

   <flashjournal-scheme>
      <scheme-name>example-flash</scheme-name>
   </flashjournal-scheme>
</caching-schemes>

13.6.2.5 Enabling a Custom Map Implementation for a Journal Scheme

Journal schemes can be configured to use a custom map as required. Custom map implementations must extend the SimpleSerializationMap class and declare the exact same set of public constructors. To enable, a custom implementation, add a <class-scheme> element whose value is the fully qualified name of the custom class. Any parameters that are required by the custom class can be defined using the <init-params> element. The following example enables a custom map implementation called MySimpleSerializationMap.

<flashjournal-scheme>
   <scheme-name>example-flash</scheme-name>
   <class-name>package.MySimpleSerializationMap</class-name>
</flashjournal-scheme>

13.6.3 Changing Journaling Behavior

A resource manager controls journaling behavior. There is a resource manager for RAM journals (RamJournalRM) and a resource manager for Flash journals (FlashJournalRM). The resource managers are configured for a cluster in the tangosol-coherence-override.xml operational override file. The resource managers' default out-of-box settings are used if no configuration overrides are set.

The following topics are included in this section:

13.6.3.1 Configuring the RAM Journal Resource Manager

The <ramjournal-manager> element is used to configure RAM journal behavior. The following lists provides a brief summary of the defaults that are set by the resource manager. See "ramjournal-manager" for details on all settings that are available and their defaults.

  • Binary values are limited by default to 64KB (and maximum 4MB)

  • An individual buffer (a journal file) is limited by default to 2MB (and maximum 2GB)

  • A journal is composed of up to 512 files

  • The total memory used by the journal is limited to 1GB by default (and maximum 64GB)

Note:

A flash journal is automatically used if the binary value setting, or memory setting, or both are exceeded.

To configure a RAM journal resource manager, add a <ramjournal-manager> element within a <journaling-config> element and define any subelements that are to be overridden. The following example demonstrates overriding each of the available subelements:

<?xml version='1.0'?>

<coherence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/
   coherence-operational-config coherence-operational-config.xsd">
   <cluster-config>
      <journaling-config>
         <ramjournal-manager>
            <maximum-value-size>64K</maximum-value-size>
            <maximum-size>2G</maximum-size>
         </ramjournal-manager>
      </journaling-config>
   </cluster-config>
</coherence>

13.6.3.2 Configuring the Flash Journal Resource Manager

The <flashjournal-manager> element is used to configure flash journal behavior. The following lists provides a brief summary of the defaults that are set by the resource manager. See "flashjournal-manager" for details on all settings that are available and their defaults.

  • Binary values are limited by default to 64MB

  • An individual buffer (a journal file) is limited by default to 2GB (and maximum 4GB)

  • A journal is composed of up to 512 files.

  • A journal is thus limited by default to 1TB, with a theoretical maximum of 2TB.

To configure a flash journal resource manager, add a <flashjournal-manager> element within a <journaling-config> element and define any subelements that are to be overridden. The following example demonstrates overriding each of the available subelements:

<?xml version='1.0'?>

<coherence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/
   coherence-operational-config coherence-operational-config.xsd">
   <cluster-config>
      <journaling-config>
         <flashjournal-manager>
            <maximum-value-size>64K</maximum-value-size>
            <maximum-file-size>8M</maximum-file-size>
            <block-size>512K</block-size>
            <maximum-pool-size>32M</maximum-pool-size>
            <directory>/coherence_storage</directory>
            <async-limit>32M</async-limit>
         </flashjournal-manager>
      </journaling-config>
   </cluster-config>
</coherence>

Note:

The directory specified for storing journal files must exist. If the directory does not exist, a warning is logged and the default temporary file directory, as designated by the JVM, is used.

13.7 Using Delta Backup

Delta backup is a technique that is used to apply changes to a backup binary entry rather than replacing the whole entry when the primary entry changes. Delta backup is ideal in situations where the entry being updated is large but only small changes are being made. In such cases, the cost for changing only a small portion of the entry is often less than the cost associated with rewriting the whole entry and results in better performance. Entries that change by more than 50% typically demonstrate little or no performance gain by using delta backup.

Delta backup uses a compressor that compares two in-memory buffers containing an old and a new value and produces a result (called a delta) that can be applied to the old value to create the new value. Coherence provides standard delta compressors for POF and non-POF formats. Custom compressors can also be created and configured as required.

13.7.1 Enabling Delta Backup

Delta backup is only available for distributed caches and is disabled by default. Delta backup is enabled either individually for each distributed cache or for all instances of the distributed cache service type.

To enable delta backup for a distributed cache, add a <compressor> element, within a <distributed-scheme> element, that is set to standard. For example:

<distributed-scheme>
   ...        
   <compressor>standard</compressor>
   ...
</distributed-scheme>

To enable delta backup for all instances of the distributed cache service type, override the partitioned cache service's compressor initialization parameter in an operational override file. For example:

<?xml version='1.0'?>

<coherence xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
   xmlns="http://xmlns.oracle.com/coherence/coherence-operational-config"
   xsi:schemaLocation="http://xmlns.oracle.com/coherence/
   coherence-operational-config coherence-operational-config.xsd">
   <cluster-config>
      <services>
         <service id="3">
            <init-params>
               <init-param id="22">
                  <param-name>compressor</param-name>
                  <param-value
                     system-property="tangosol.coherence.distributed.compressor">
                     standard</param-value>
               </init-param>
            </init-params>
         </service>
      </services>
   </cluster-config>
</coherence>

The tangosol.coherence.distributed.compressor system property is used to enable delta backup for all instances of the distributed cache service type instead of using the operational override file. For example:

-Dtangosol.coherence.distributed.compressor=standard

13.7.2 Enabling a Custom Delta Backup Compressor

To use a custom compressor for performing delta backup, include an <instance> subelement and provide a fully qualified class name that implements the DeltaCompressor interface. See "instance" for detailed instructions on using the <instance> element. The following example enables a custom compressor that is implemented in the MyDeltaCompressor class.

<distributed-scheme>
   ...        
   <compressor>
      <instance>
         <class-name>package.MyDeltaCompressor</class-name>
      </instance>
   </compressor>
   ...
</distributed-scheme>

As an alternative, the <instance> element supports the use of a <class-factory-name> element to use a factory class that is responsible for creating DeltaCompressor instances, and a <method-name> element to specify the static factory method on the factory class that performs object instantiation. The following example gets a custom compressor instance using the getCompressor method on the MyCompressorFactory class.

<distributed-scheme>
   ...        
   <compressor>
      <instance>
         <class-factory-name>package.MyCompressorFactory</class-factory-name>
         <method-name>getCompressor</method-name>
      </instance>
   </compressor>
   ...
</distributed-scheme>

Any initialization parameters that are required for an implementation can be specified using the <init-params> element. The following example sets the iMaxTime parameter to 2000.

<distributed-scheme>
   ...        
   <compressor>
      <instance>
         <class-name>package.MyDeltaCompressor</class-name>
         <init-params>
            <init-param>
               <param-name>iMaxTime</param-name>
               <param-value>2000</param-value>
            </init-param>
         </init-params>
      </instance>
   </compressor>
   ...
</distributed-scheme>