Home / Middleware / Oracle Coherence

5 Loading Data Into a Cache

In this exercise, you learn how to populate a Coherence cache with domain objects that are read from text files. It highlights the use of APIs from the Coherence aggregator, extractor, and filter packages.

This chapter contains the following sections:

• Introduction

• Populating a Cache with Domain Objects

• Querying and Aggregating Data in the Cache

5.1 Introduction

Until now, you put objects into the cache and retrieved them individually. Each call to the put method can result in increased network traffic, especially for partitioned and replicated caches. Additionally, each call to a put method returns the object it just replaced in the cache, which adds more unnecessary overhead. By using the putAll method, loading the cache can be made much more efficient.

To perform the tasks in this chapter, you must first complete the project described in Chapter 4, "Working with Complex Objects." You must also be familiar with using java.io.BufferedReader to read text files, java.lang.String.split method to parse text files, and java.text.SimpleDateFormat to parse dates.

5.2 Populating a Cache with Domain Objects

This exercise shows you how to create a console application that populates a Coherence cache with domain objects. The application will use the Coherence com.tangosol.io.pof.PortableObject implementation to serialize the objects into Portable Object format (POF).

In the exercise, you create a key that helps to get the Contact object, a generator to provide data for the cache, and a loader to load the cache.

1. Create a Class with the Key for the Domain Objects

2. Edit the POF Configuration File

3. Create the Data Generator

4. Create a Console Application to Load the Cache

5. Run the Cache Loading Example

5.2.1 Create a Class with the Key for the Domain Objects

To create a class that contains the key for a domain object:

1. Create a new Application Client Project called Loading. Select CoherenceConfig from the Configuration drop-down list. In the Application Client Module page of the New Application Client Project wizard, deselect the Create a default Main class checkbox.

   See "Creating and Caching Complex Objects" for information on creating a new project.

2. Add the classes and files related to the Address, PhoneNumber, and Contact classes that you created in an earlier exercise (Contacts). These files can be found in c:\home\oracle\workspace\Contacts\appClientModule and c:\home\oracle\workspace\Contacts\build\classes directories.

   Right click the Loading project in the Project Explorer and select Properties. In the Properties for Loading dialog box, select Java Build Path. In the Projects tab, click Add. Select the Contacts project from the Required Project Selection dialog box, as illustrated in Figure 5-1.

   Figure 5-1 Adding Folders to the Project Build Path

   
   Description of "Figure 5-1 Adding Folders to the Project Build Path"
   
   

   In the Order and Export tab, use the Up and Down buttons to move Contacts to the top of the list. The contents of the tab should look similar to Figure 5-2.

   Figure 5-2 Contents of the Order and Export Tab for the Loading Project

   
   Description of "Figure 5-2 Contents of the Order and Export Tab for the Loading Project "
   
   

3. Create a ContactId class that provides a key to the employee for whom information is tracked. See "Creating a Java Class" for detailed information on creating a Java class.

   Create the contact ID based on the employee's first name and last name. This object acts as the key to get the Contact object.

   Because this class uses POF serialization, it must implement the PortableObject interface, the writeExternal and readExternal PortableObject methods, and the equals, hashCode, and toString object methods.

   Note:

   Cache keys and values must be serializable (for example, java.io.Serializable). Cache keys must also provide an implementation of the hashCode and equals methods, and those methods must return consistent results across cluster nodes. This implies that the implementation of the hashCode and equals object methods must be based solely on the object's serializable state (that is, the object's non-transient fields). Most built-in Java types, such as String, Integer and Date, meet this requirement. Some cache implementations (specifically the partitioned cache) use the serialized form of the key objects for equality testing, which means that keys for which the equals method returns true must serialize identically; most built-in Java types meet this requirement.

   Example 5-1 illustrates a possible implementation of the ContactId class.

   Example 5-1 Simple Contact ID Class

   package com.oracle.handson;
   
   import com.tangosol.io.pof.PofReader;
   import com.tangosol.io.pof.PofWriter;
   import com.tangosol.io.pof.PortableObject;
    
   import com.tangosol.util.Base;
   import com.tangosol.util.HashHelper;
    
   import java.io.IOException;
    
   /**
   * ContactId is a key to the person for whom information is
   * tracked.
   */
   public class ContactId implements PortableObject
       {
       // ----- constructors ---------------------------------------------------
    
       /**
       * Default constructor (necessary for PortableObject implementation).
       */
       public ContactId() 
           {
           }
    
       /**
       * Construct a contact person.
       *
       */
       public ContactId(String FirstName, String LastName)
           {
           super();
           this.FirstName = FirstName;
           this.LastName  = LastName;
           }
    
       // ----- accessors ------------------------------------------------------
    
       /**
       * Return the first name.
       *
       */
       public String getFirstName()
           {
           return FirstName;
           }
    
       /**
       * Return the last name.
       *
       */
       public String getLastName()
           {
           return LastName;
           }
    
       // ----- PortableObject interface ---------------------------------------
    
       public void readExternal(PofReader reader)
               throws IOException
           {
           FirstName = reader.readString(0);
           LastName = reader.readString(1);
           }
    
       public void writeExternal(PofWriter writer)
               throws IOException
           {
           writer.writeString(0, FirstName);
           writer.writeString(1, LastName);
           }
    
       // ----- Object methods -------------------------------------------------
    
       public boolean equals(Object oThat)
           {
           if (this == oThat)
               {
               return true;
               }
           if (oThat == null)
               {
               return false;
               }
    
           ContactId that = (ContactId) oThat;
           return Base.equals(getFirstName(), that.getFirstName()) &&
                  Base.equals(getLastName(),  that.getLastName());
           }
    
       public int hashCode()
           {
           return HashHelper.hash(getFirstName(),
                  HashHelper.hash(getLastName(), 0));
           }
    
       public String toString()
           {
           return getFirstName() + " " + getLastName();
           }
    
       // ----- data members ---------------------------------------------------
    
       /**
       * First name.
       */
       private String FirstName;
    
       /**
       * Last name.
       */
       private String LastName;
       }
   

5.2.2 Edit the POF Configuration File

Edit the POF configuration file. Add a <user-type> entry for the ContactId class to the contacts-pof-config.xml file. The file looks similar to Example 5-2.

Example 5-2 POF Configuration File with the ContactId Entry

<?xml version="1.0"?>
 
<pof-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xmlns="http://xmlns.oracle.com/coherence/coherence-pof-config"
              xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-pof-config coherence-pof-config.xsd">
  <user-type-list>
 
    <!-- coherence POF user types -->
    <include>coherence-pof-config.xml</include>
 
    <!-- com.tangosol.examples package -->
    <user-type>
      <type-id>1001</type-id>
      <class-name>com.oracle.handson.Contact</class-name>
    </user-type>
    <user-type>
      <type-id>1002</type-id>
      <class-name>com.oracle.handson.Address</class-name>
    </user-type>
    <user-type>
        <type-id>1003</type-id>
      <class-name>com.oracle.handson.PhoneNumber</class-name>
    </user-type>
    <user-type> 
     <type-id>1004</type-id>
      <class-name>com.oracle.handson.ContactId</class-name>
    </user-type>  
   </user-type-list>
  <allow-interfaces>true</allow-interfaces>
  <allow-subclasses>true</allow-subclasses>
</pof-config>

5.2.3 Create the Data Generator

Create a Java class named DataGenerator to generate random employee contact names and addresses. See "Creating a Java Class" for detailed information.

Use the Address, PhoneNumber, and Contact classes that you created in an earlier exercise. Use java.util.Random to generate some random names, addresses, telephone numbers, and ages.

Example 5-3 illustrates a possible implementation of the data generator. This implementation creates a text file, contacts.cvs, that contains the employee contact information.

Example 5-3 Sample Data Generation Class

package com.oracle.handson;
 
import com.oracle.handson.Address;
import com.oracle.handson.Contact;
import com.oracle.handson.PhoneNumber;
import com.tangosol.util.Base;
 
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
 
import java.sql.Date;
 
import java.util.Collections;
import java.util.Random;
 
 
/**
* DataGenerator is a generator of sample contacts.
*/
public class DataGenerator
    {
    // ----- static methods -------------------------------------------------
 
    /**
    * Generate contacts.
    */
    public static void main(String[] asArg)
            throws IOException
        {
        String       sFile = asArg.length > 0 ? asArg[0] : FILENAME;
        int          cCon  = asArg.length > 1 ? Integer.parseInt(asArg[1]) : 1000;
        OutputStream out   = new FileOutputStream(sFile);
 
        generate(out, cCon);
        out.close();
        }
 
    /**
    * Generate the contacts and write them to a file.
    */
    public static void generate(OutputStream out, int cContacts)
            throws IOException
        {
        PrintWriter writer = new PrintWriter(new BufferedWriter(
                new OutputStreamWriter(out)));
 
        for (int i = 0; i < cContacts; ++i)
            {
            StringBuffer sb = new StringBuffer(256);
 
            //contact person
            sb.append("John,")
              .append(getRandomName())
              .append(',');
 
            // home and work addresses
            sb.append(Integer.toString(Base.getRandom().nextInt(999)))
              .append(" Beacon St.,,") /*street1,empty street2*/
              .append(getRandomName()) /*random city name*/
              .append(',')
              .append(getRandomState())
              .append(',')
              .append(getRandomZip())
              .append(",US,Yoyodyne Propulsion Systems,")
              .append("330 Lectroid Rd.,Grover's Mill,")
              .append(getRandomState())
              .append(',')
              .append(getRandomZip())
              .append(",US,");
 
            // home and work telephone numbers
            sb.append("home,")
              .append(Base.toDelimitedString(getRandomPhoneDigits(), ","))
              .append(",work,")
              .append(Base.toDelimitedString(getRandomPhoneDigits(), ","))
              .append(',');
 
            // random birth date in millis before or after the epoch
            sb.append(getRandomDateInMillis());
 
            writer.println(sb);
            }
        writer.flush();
        }
 
    /**
    * Return a random name.
    *
    */
    private static String getRandomName()
        {
        Random rand = Base.getRandom();
        int    cCh  = 4 + rand.nextInt(7);
        char[] ach  = new char[cCh];
 
        ach[0] = (char) ('A' + rand.nextInt(26));
        for (int of = 1; of < cCh; ++of)
            {
            ach[of] = (char) ('a' + rand.nextInt(26));
            }
        return new String(ach);
        }
 
    /**
    * Return a random phone muber.
    * The phone number includes access, country, area code, and local
    * number.
    *
    */
    private static int[] getRandomPhoneDigits()
        {
        Random rand = Base.getRandom();
        return new int[] 
            {
            11,                   // access code
            rand.nextInt(99),     // country code
            rand.nextInt(999),    // area code
            rand.nextInt(9999999) // local number
            };
        }
 
    /**
    * Return a random Phone.
    *
    */
    private static PhoneNumber getRandomPhone()
        {
        int[] anPhone = getRandomPhoneDigits();
 
        return new PhoneNumber((short)anPhone[0], (short)anPhone[1],
                (short)anPhone[2], anPhone[3]);
 
        }
    /**
    * Return a random Zip code.
    *
    */
    private static String getRandomZip()
        {
        return Base.toDecString(Base.getRandom().nextInt(99999), 5);
        }
 
    /**
    * Return a random state.
    *
    */
    private static String getRandomState()
        {
        return STATE_CODES[Base.getRandom().nextInt(STATE_CODES.length)];
        }
 
    /**
    * Return a random date in millis before or after the epoch.
    *
    */
    private static long getRandomDateInMillis()
        {
        return (Base.getRandom().nextInt(40) - 20) * Contact.MILLIS_IN_YEAR;
        }
 
    /**
    * Generate a Contact with random information.
    *
    */
    public static Contact getRandomContact()
        {
        return new Contact("John",
                getRandomName(),
                new Address("1500 Boylston St.", null, getRandomName(),
                    getRandomState(), getRandomZip(), "US"),
                new Address("8 Yawkey Way", null, getRandomName(),
                    getRandomState(), getRandomZip(), "US"),
                Collections.singletonMap("work", getRandomPhone()),
                new Date(getRandomDateInMillis()));
        }
 
    // ----- constants ------------------------------------------------------
 
    /**
    * US Postal Service two letter postal codes.
    */
    private static final String[] STATE_CODES = 
        {
            "AL", "AK", "AS", "AZ", "AR", "CA", "CO", "CT", "DE", "OF", "DC",
            "FM", "FL", "GA", "GU", "HI", "ID", "IL", "IN", "IA", "KS", "KY",
            "LA", "ME", "MH", "MD", "MA", "MI", "MN", "MS", "MO", "MT", "NE",
            "NV", "NH", "NJ", "NM", "NY", "NC", "ND", "MP", "OH", "OK", "OR",
            "PW", "PA", "PR", "RI", "SC", "SD", "TN", "TX", "UT", "VT", "VI",
            "VA", "WA", "WV", "WI", "WY"
        };
    /**
    * Default contacts file name.
    */
    public static final String FILENAME = "contacts.csv";
    }

The DataGenerator class generates, at random, information about an employee. The information includes the employee's name, home address, work address, home telephone number, work telephone number, and the employee's age. The information is stored in a CSV-formatted file named contacts.csv. This file follows the standard rules for a CSV-formatted file, where there is one record per line, and individual fields within the record are separated by commas.

Example 5-4 illustrates the first few entries of the contacts.csv file.

Example 5-4 Contents of the contacts.csv File

John,Dvcqbvcp,669 Beacon St.,,Tetuvusz,CA,68457,US,Yoyodyne Propulsion Systems,330 Lectroid Rd.,Grover's Mill,KS,30344,US,home,11,98,183,8707139,work,11,96,425,1175949,63072000000
John,Fxsr,549 Beacon St.,,Jutaswmaby,MI,16315,US,Yoyodyne Propulsion Systems,330 Lectroid Rd.,Grover's Mill,CT,60309,US,home,11,20,40,3662989,work,11,41,187,3148474,-189216000000
John,Gmyrolvfyd,73 Beacon St.,,Lpnztf,AR,82667,US,Yoyodyne Propulsion Systems,330 Lectroid Rd.,Grover's Mill,NY,42297,US,home,11,22,17,8579970,work,11,35,338,9286245,-567648000000
John,Efmpjlbj,85 Beacon St.,,Wyswpb,AK,29590,US,Yoyodyne Propulsion Systems,330 Lectroid Rd.,Grover's Mill,NJ,06219,US,home,11,20,744,4331451,work,11,39,911,9104590,-157680000000
...

5.2.4 Create a Console Application to Load the Cache

Create a Java class called LoaderExample. See "Creating a Java Class" for detailed information.

Implement the class to load the cache with employee data generated by the program described in "Create the Data Generator". Use input streams and buffered readers to load the employee information in the contacts.csv file into a single Coherence cache.

Add code to parse the employee information in the data file. After you have this information, create the individual contacts to put into the cache. To conserve processing effort and minimize network traffic, use the putAll method to load the cache.

Example 5-5 illustrates a possible implementation of the LoaderExample class.

Example 5-5 Sample Cache Loading Program

package com.oracle.handson;
 
import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;
 
import com.oracle.handson.ContactId;
import com.oracle.handson.Address;
import com.oracle.handson.PhoneNumber;
import com.oracle.handson.Contact;
 
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
 
import java.sql.Date;
 
import java.util.HashMap;
import java.util.Map;
 
 
/**
* LoaderExample loads contacts into the cache from a file.
*
*/
public class LoaderExample
    {
    // ----- static methods -------------------------------------------------
 
    /**
    * Load contacts.
    */
    public static void main (String[] asArg)
            throws IOException
        {
        String sFile  = asArg.length > 0 ? asArg[0] : DataGenerator.FILENAME;
        String sCache = asArg.length > 1 ? asArg[1] : CACHENAME;
 
        System.out.println("input file: " + sFile);
        System.out.println("cache name: " + sCache);
 
        new LoaderExample().load(CacheFactory.getCache(sCache),
                new FileInputStream(sFile));
        CacheFactory.shutdown();
        }
 
    /**
    * Load cache from stream.
    *
    */
    public void load(NamedCache cache, InputStream in)
            throws IOException
        {
        BufferedReader reader   = new BufferedReader(new InputStreamReader(in));
        Map            mapBatch = new HashMap(1024);
        String         sRecord;
        int            cRecord  = 0;
 
        while ((sRecord = reader.readLine()) != null)
            {
            // parse record
            String[]      asPart     = sRecord.split(",");
            int           ofPart     = 0;
            String        sFirstName =  asPart[ofPart++];
            String        sLastName  =  asPart[ofPart++];
            ContactId          id    = new ContactId(sFirstName, sLastName);
            Address       addrHome   = new Address(
                                         /*streetline1*/ asPart[ofPart++],
                                         /*streetline2*/ asPart[ofPart++],
                                         /*city*/        asPart[ofPart++],
                                         /*state*/       asPart[ofPart++],
                                         /*zip*/         asPart[ofPart++],
                                         /*country*/     asPart[ofPart++]);
            Address       addrWork   = new Address(
                                         /*streetline1*/ asPart[ofPart++],
                                         /*streetline2*/ asPart[ofPart++],
                                         /*city*/        asPart[ofPart++],
                                         /*state*/       asPart[ofPart++],
                                         /*zip*/         asPart[ofPart++],
                                         /*country*/     asPart[ofPart++]);
            Map           mapTelNum  = new HashMap();
 
            for (int c = asPart.length - 1; ofPart < c; )
                {
                mapTelNum.put(/*type*/ asPart[ofPart++], new PhoneNumber(
                    /*access code*/  Short.parseShort(asPart[ofPart++]),
                    /*country code*/ Short.parseShort(asPart[ofPart++]),
                    /*area code*/    Short.parseShort(asPart[ofPart++]),
                    /*local num*/    Integer.parseInt(asPart[ofPart++])));
                }
            Date dtBirth = new Date(Long.parseLong(asPart[ofPart]));
 
            // Construct Contact and add to batch
            Contact con1 = new Contact(sFirstName, sLastName, addrHome, 
                                       addrWork, mapTelNum, dtBirth);
            System.out.println(con1);
            mapBatch.put(id, con1);
            
            ++cRecord;
            if  (cRecord % 1024 == 0)
                {
                // load batch
                cache.putAll(mapBatch);
                mapBatch.clear();
                System.out.print('.');
                System.out.flush();
                }
            }
 
            if (!mapBatch.isEmpty())
               {
               // load final batch
               cache.putAll(mapBatch);
               }
 
        System.out.println("Added " + cRecord + " entries to cache");
        }
 
 
    // ----- constants ------------------------------------------------------
 
    /**
    * Default cache name.
    */
    public static final String CACHENAME = "ContactsCache";
 
}

5.2.5 Run the Cache Loading Example

To run the cache loading example:

1. Stop any running cache servers. See "Stopping Cache Servers" for detailed information.

2. Start a cache server with the ContactsCacheServer file.

   1. Right click the project and select Run As then Run Configurations to edit the ContactsCacheServer configuration. In the Main tab, click Browse and select the Loading project.

   2. In the Classpath tab, select User Entries and click Add Projects. In the Project Selection dialog box, select the Loading project. Click OK.

   3. In the Common tab, click Shared file and browse for the Loading project.

   4. Click Apply, then Run.

3. Create a run configuration for DataGenerator. Right click DataGenerator in the Project Explorer and select Run As. In the Run Configurations dialog box select Oracle Coherence and click the New Configuration icon.

   1. In the Name field, enter DataGenerator.

   2. In the Project field in the Main tab, enter Loading. In the Main class field, enter com.oracle.handson.DataGenerator.

   3. In the General tab of the Coherence tab, browse to the c:\home\oracle\workspace\Contacts\appClientModule\contacts-cache-config.xml file in the Cache configuration descriptor field. Select the Disabled (cache client) button. Enter 3155 in the Cluster port field. Click Apply.

      In the Other tab, scroll down to the tangosol.pof.config field. Enter the absolute path to the POF configuration file contacts-pof-config.xml. Click Apply.

   4. In the Common tab, select Shared file and browse for the Loading directory.

   5. Examine the contents of Loading in the Classpath tab. Contacts should appear as one of the entries under Loading, as in Figure 5-3.

      Figure 5-3 Classpath for the DataGenerator Program

      
      Description of "Figure 5-3 Classpath for the DataGenerator Program"
      
      

4. Create a run configuration for LoaderExample.

   • In the Name field, enter LoaderGenerator

   • In the Project field in the Main tab, enter Loading. In the Main class field, enter com.oracle.handson.LoaderExample.

   • In the General tab of the Coherence tab, browse to the c:\home\oracle\workspace\Contacts\appClientModule\contacts-cache-config.xml file in the Cache configuration descriptor field. Select the Disabled (cache client) button. Enter 3155 in the Cluster port field. Click Apply.

     In the Other tab, scroll down to the tangosol.pof.config field. Enter the absolute path to the POF configuration file contacts-pof-config.xml. Click Apply.

   • Examine the contents of the Classpath tab. Contacts should appear as one of the entries under Loading, as in Figure 5-4. If it does not appear, click User Entries, then add the project with the Add Project button.

     Figure 5-4 Classpath for the LoaderExample Program

     
     Description of "Figure 5-4 Classpath for the LoaderExample Program"
     
     

5. Run the DataGenerator configuration from the Run Configurations dialog box. Then run the LoaderExample configuration.

   The DataGenerator run configuration generates a data file; you should not see any output or response in the Eclipse console window. The LoaderGenerator run configuration sends a stream of employee contact information, similar to Example 5-6, to the Eclipse console window.

   Example 5-6 Output from the Sample Cache Loading Program

   ...
   2012-08-16  15:47:04.413/2.266 Oracle Coherence GE 12.1.2.0 <D5> (thread=Invocation:Management, member=2): Service Management joined the cluster with senior service member 1
   2012-08-16  15:47:04.413/2.266 Oracle Coherence GE 12.1.2.0 <Info> (thread=Cluster, member=2): Loaded POF configuration from "file:/C:/home/oracle/workspace/Contacts/appClientModule/contacts-pof-config.xml"
   2012-08-16  15:47:04.413/2.266 Oracle Coherence GE 12.1.2.0 <Info> (thread=Cluster, member=2): Loaded included POF configuration from "jar:file:/C:/coherence360/coherence/lib/coherence.jar!/coherence-pof-config.xml"
   2012-08-16  15:47:04.413/2.266 Oracle Coherence GE 12.1.2.0 <D5> (thread=DistributedCache:PartitionedPofCache, member=2): Service PartitionedPofCache joined the cluster with senior service member 1
   2012-08-16  15:47:04.413/2.266 Oracle Coherence GE 12.1.2.0 <D4> (thread=DistributedCache:PartitionedPofCache, member=2): Asking member 1 for 128 primary partitions
   John Hwolru
   Addresses
   Home: 742 Beacon St.
    
   Icymz, OH 74135
   US
   Work: Yoyodyne Propulsion Systems
   330 Lectroid Rd.
   Grover's Mill, WY 20222
   US
   Telephone Numbers
   work: [phone]
   home: [phone]
   Birth Date: 1981-12-28
   
   ...
   
   John Unglg
   Addresses
   Home: 973 Beacon St.
    
   Cgarhej, MI 10517
   US
   Work: Yoyodyne Propulsion Systems
   330 Lectroid Rd.
   Grover's Mill, GA 81835
   US
   Telephone Numbers
   work: [phone]
   home: [phone]
   Birth Date: 1965-01-01
   John Lkkwgw
   Addresses
   Home: 806 Beacon St.
    
   Zlpft, GU 55786
   US
   Work: Yoyodyne Propulsion Systems
   330 Lectroid Rd.
   Grover's Mill, WV 88125
   US
   Telephone Numbers
   work: [phone]
   home: [phone]
   Birth Date: 1987-12-27
   Added 1000 entries to cache
   2012-08-16 15:47:04.575/2.625 Oracle Coherence GE 12.1.2.0 <D5> (thread=Invocation:Management, member=n/a): Service Management left the cluster
   2012-08-16 15:47:04.591/2.641 Oracle Coherence GE 12.1.2.0 <D5> (thread=DistributedCache:PartitionedPofCache, member=n/a): Service PartitionedPofCache left the cluster
   2012-08-16 15:47:04.622/2.672 Oracle Coherence GE 12.1.2.0 <D5> (thread=Cluster, member=n/a): Service Cluster left the cluster
   

5.3 Querying and Aggregating Data in the Cache

This exercise introduces querying and aggregating data in a cache. This exercise shows you how to:

• Query the cache for specific data

• Aggregate information within the cache

After putting complex objects in the named caches, you can query and aggregate information within the grid. The com.tangosol.util.QueryMap interface provides methods for managing the values or keys within a cache. You can use filters to restrict your results. You can also define indexes to optimize your queries.

Because Coherence serializes information when storing it, you also have the overhead of deserializing when querying information. When indexes are added, the values kept in the index itself are deserialized and, therefore, are quicker to access. The objects that are indexed are always kept serialized.

Some of the often-used methods in the QueryMap interface are:

• Set entrySet(Filterfilter), which returns a set of entries that are contained in the map that satisfy the filter

• addIndex(ValueExtractorextractor,booleanfOrdered, Comparator comparator), which adds an index

• Set keySet(Filter filter), which is similar to entrySet, but returns keys, not values

It is important to note that filtering occurs at Cache Entry Owner level. In a partitioned topology, filtering can be performed in parallel because it is the primary partitions that do the filtering. The QueryMap interface uses the Filter classes. You can find more information on these classes in the API for the com.tangosol.util.filter package.

All Coherence NamedCache objects implement the com.tangosol.util.QueryMap interface. This enables NamedCache objects to support searching for keys or entries in a cache that satisfy a specified condition. The condition can be represented as an object that implements the com.tangosol.util.Filter interface.

The com.tangosol.util.filter package contains several predefined classes that provide implementations of standard query expressions. Examples of these classes include GreaterFilter, GreaterEquals, LikeFilter, NotEqualsFilter, InFilter, and so on. You can use these filters to construct object-based equivalents of most SQL WHERE clause expressions.

Note:

Coherence does not provide a SQLFilter because it is unlikely that the objects placed in a cache are modeled in a relational manner, that is, using rows and columns (as they are typically represented in a database). Additionally, it is common that objects placed in a cache are not easily modeled using relational models, for example, large BLOBS.

The Filter classes use standard Java method reflection to represent test conditions. For example, the following filter represents the condition where the value returned from the getHomeAddress.getState method on an object in a cache is for Massachusetts (MA):

(new EqualsFilter("getHomeAddress.getState", "MA");

If the object tested with this filter does not have a get method, then the test fails.

Here are additional examples of using the Filter classes:

• Return a set of people who live in a city whose name begins with the letter S:

  Set sPeople = cache.entrySet(new LikeFilter("getHomeAddress.getCity","S%");
  

• Return a set containing people over the age of 42:

  final int nAge = 42;// Find all contacts who are older than nAge
  Set sSeniors = cache.entrySet(new GreaterFilter("getAge", nAge));
  

In addition to the entrySet and keySet methods defined by the QueryMap interface, Coherence lets you define indexes to improve query performance by using the addIndex method. Unlike relational database systems, where indexes are defined according to well-known and strictly enforced collections of named columns (that is, a schema), Coherence does not have a schema. Because there is no schema, you define indexes differently from a traditional database.

To define the values that are to be indexed for each object placed in a cache, Coherence introduces the concept of a value extractor. The com.tangosol.util.ValueExtractor interface defines an extract method. If given an object parameter, a ValueExtractor implementation returns a value based on the parameter.

A simple example of a ValueExtractor implementation is the com.tangosol.util.extractor.ReflectionExtractor interface, which uses reflection to return the result of a method call on an object, for example:

new ReflectionExtractor("getCity")

You can use value extractors throughout the Coherence API. Typically, however, they are used to define indexes.

An especially useful type of extractor is the ChainedExtractor. This is a composite ValueExtractor implementation based on an array of extractors. Extractors in the array are applied sequentially, from left to right. The result of a previous extractor becomes the target object for a next extractor, for example:

new ChainedExtractor(new ReflectionExtractor("getHomeAddress"), new ReflectionExtractor("getState"))

This example assumes that the HomeAddress and State objects belong to a complex Contact object. ChainedExtractor first uses reflection to call the getHomeAddress method on each cached Contact object, and then uses reflection to call the getState method on the set of returned HomeAddress objects.

1. Create the Class to Query Cache Data

2. Run the Query Example

3. Edit the Query Example to Perform Aggregations

4. Run the Query and Aggregation Example

5.3.1 Create the Class to Query Cache Data

Create a new Java class called QueryExample to perform your queries. Ensure that the class has a main method. See "Creating a Java Class" for detailed information.

Use the entrySet method to get the employee contact information for:

• All employees who live in Massachusetts:

  cache.entrySet(new EqualsFilter("getHomeAddress.getState", "MA"));
  

• All employees who live in Massachusetts and work elsewhere:

  cache.entrySet(new AndFilter(
           new EqualsFilter("getHomeAddress.getState", "MA"),
           new NotEqualsFilter("getWorkAddress.getState", "MA")));
  

• All employees whose city name begins with an S:

  cache.entrySet(new LikeFilter("getHomeAddress.getCity", "S%"));
  

• All employees whose last name begins with an S and live in Massachusetts. Use both the key and value in the query. Note that the cache entries use ContactId objects as the keys. You can use the KeyExtractor to get these values. KeyExtractor is a specialized value extractor that indicates that a query be run against the key objects, rather than the values:

  cache.entrySet(new AndFilter(
       new LikeFilter(new KeyExtractor("getLastName"), "S%", 
           (char) 0, false),
       new EqualsFilter("getHomeAddress.getState", "MA")));
  

• All employees who are older than a specified age. Hint:

  final int nAge = 42;
      setResults = cache.entrySet(new GreaterFilter("getAge", nAge));
  

Use indexes to improve performance. Hint: Find the addIndex method in the Javadoc for the QueryMap interface.

Example 5-7 illustrates a possible implementation of the QueryExample class.

Example 5-7 Sample QueryExample Class

package com.oracle.handson;
 
import com.tangosol.net.CacheFactory;
import com.tangosol.net.NamedCache;
 
import com.tangosol.util.extractor.ChainedExtractor;
import com.tangosol.util.extractor.KeyExtractor;
import com.tangosol.util.extractor.ReflectionExtractor;
 
import com.tangosol.util.filter.AlwaysFilter;
import com.tangosol.util.filter.AndFilter;
import com.tangosol.util.filter.EqualsFilter;
import com.tangosol.util.filter.GreaterFilter;
import com.tangosol.util.filter.LikeFilter;
import com.tangosol.util.filter.NotEqualsFilter;
 
import java.util.Iterator;
import java.util.Set;
 
 
/**
* QueryExample runs sample queries for contacts.
*
*/
public class QueryExample{
  
    // ----- QueryExample methods ---------------------------------------
 
    public static void main(String[] args) {
      NamedCache cache = CacheFactory.getCache("ContactsCache");
       query(cache);
    }
    /**
    * Perform the example queries
    *
    */
     public static void query(NamedCache cache)
        {
         // Add indexes to make queries more efficient
        ReflectionExtractor reflectAddrHome =
                new ReflectionExtractor("getHomeAddress");

        // Add an index for the age
        cache.addIndex(new ReflectionExtractor("getAge"), true, null);

        // Add index for state within home address
       cache.addIndex(new ChainedExtractor(reflectAddrHome,
                new ReflectionExtractor("getState")), true, null);

        // Add index for state within work address
        cache.addIndex(new ChainedExtractor(
                new ReflectionExtractor("getWorkAddress"),
                new ReflectionExtractor("getState")), true, null);

        // Add index for city within home address
        cache.addIndex(new ChainedExtractor(reflectAddrHome,
                new ReflectionExtractor("getCity")), true, null);
 
        // Find all contacts who live in Massachusetts
        Set setResults = cache.entrySet(new EqualsFilter(
                "getHomeAddress.getState", "MA"));
        printResults("MA Residents", setResults);
 
        // Find all contacts who live in Massachusetts and work elsewhere
        setResults = cache.entrySet(new AndFilter(
                new EqualsFilter("getHomeAddress.getState", "MA"),
                new NotEqualsFilter("getWorkAddress.getState", "MA")));
        printResults("MA Residents, Work Elsewhere", setResults);
 
        // Find all contacts whose city name begins with 'S'
        setResults = cache.entrySet(new LikeFilter("getHomeAddress.getCity",
                "S%"));
        printResults("City Begins with S", setResults);
 
        final int nAge = 42;
        // Find all contacts who are older than nAge
        setResults = cache.entrySet(new GreaterFilter("getAge", nAge));
        printResults("Age > " + nAge, setResults);
 
        // Find all contacts with last name beginning with 'S' that live
        // in Massachusetts. Uses both key and value in the query.
        setResults = cache.entrySet(new AndFilter(
                new LikeFilter(new KeyExtractor("getLastName"), "S%",
                               (char) 0, false),
                new EqualsFilter("getHomeAddress.getState", "MA")));
        printResults("Last Name Begins with S and State Is MA", setResults);
 
        }
 
    /**
    * Print results of the query
    *
    * @param sTitle      the title that describes the results
    * @param setResults  a set of query results
    */
    private static void printResults(String sTitle, Set setResults)
        {
        System.out.println(sTitle);
        for (Iterator iter = setResults.iterator(); iter.hasNext(); )
            {
            System.out.println(iter.next());
            }
        }
    }

5.3.2 Run the Query Example

To run the query example:

1. Stop all running cache servers. See "Stopping Cache Servers" for more information.

2. Create a run configuration for the QueryExample class.

   1. Right click QueryExample in the Project Navigator and select Run As then Run Configurations. Select Oracle Coherence and click the New launch configuration icon.

   2. In the Name field, enter QueryExample.

   3. In the Project field in the Main tab, enter Loading. In the Main class field, enter com.oracle.handson.QueryExample.

   4. In the General tab of the Coherence tab, browse to the c:\home\oracle\workspace\Contacts\appClientModule\contacts-cache-config.xml file in the Cache configuration descriptor field. Select the Disabled (cache client) button. Enter 3155 in the Cluster port field. Click Apply.

      In the Other tab, scroll down to the tangosol.pof.config field. Enter the absolute path to the POF configuration file contacts-pof-config.xml. Click Apply.

   5. Examine the contents of the Classpath tab. The Contacts project should appear under the Loading project in User Entries.

3. Restart the ContactsCacheServer.

4. Run the DataGenerator class, the LoaderExample class, then the QueryExample class.

After printing all of the contact information in the cache, the QueryExample file displays the results of the queries. The results should look similar to Example 5-8.

Example 5-8 Results of the QueryExample Program

TcpRing{Connections=[1]}
IpMonitor{AddressListSize=0}
 
2012-08-16 16:15:01.778/2.094 Oracle Coherence GE 12.1.2.0 <D5> (thread=Invocation:Management, member=5): Service Management joined the cluster with senior service member 1
2012-08-16 16:15:01.825/2.141 Oracle Coherence GE 12.1.2.0 <Info> (thread=main, member=5): Loaded Reporter configuration from "jar:file:/C:/oracle/product/coherence/lib/coherence.jar!/reports/report-group.xml"
2012-08-16 16:15:02.137/2.453 Oracle Coherence GE 12.1.2.0 <Info> (thread=DistributedCache:PartitionedPofCache, member=5): Loaded POF configuration from "file:/C:/home/oracle/workspace/Contacts/build/classes/contacts-pof-config.xml"
2012-08-16 16:15:02.169/2.485 Oracle Coherence GE 12.1.2.0 <Info> (thread=DistributedCache:PartitionedPofCache, member=5): Loaded included POF configuration from "jar:file:/C:/oracle/product/coherence/lib/coherence.jar!/coherence-pof-config.xml"
2012-08-16 16:15:02.231/2.547 Oracle Coherence GE 12.1.2.0 <D5> (thread=DistributedCache:PartitionedPofCache, member=5): Service PartitionedPofCache joined the cluster with senior service member 1
MA Residents
ConverterEntry{Key="John Ueccc", Value="John Ueccc
Addresses
Home: 285 Beacon St.
 
Oxtqwisgti, MA 41063
US
Work: Yoyodyne Propulsion Systems
330 Lectroid Rd.
Grover's Mill, VA 28107
US
Telephone Numbers
work: [phone]
home: [phone]
Birth Date: 1972-12-30"}
...
MA Residents, Work Elsewhere
ConverterEntry{Key="John Ueccc", Value="John Ueccc
Addresses
Home: 285 Beacon St.
 
Oxtqwisgti, MA 41063
US
Work: Yoyodyne Propulsion Systems
330 Lectroid Rd.
Grover's Mill, VA 28107
US
Telephone Numbers
work: [phone]
home: [phone]
Birth Date: 1972-12-30"}
...
City Begins with S
ConverterEntry{Key="John Frepojf", Value="John Frepojf
Addresses
Home: 851 Beacon St.
 
Swnsfng, PR 00734
US
Work: Yoyodyne Propulsion Systems
330 Lectroid Rd.
Grover's Mill, AR 97794
US
Telephone Numbers
work: [phone]
home: [phone]
Birth Date: 1985-12-27"}
...
Age > 42
ConverterEntry{Key="John Qghbguyn", Value="John Qghbguyn
Addresses
Home: 49 Beacon St.
 
Dvftzpq, PR 34220
US
Work: Yoyodyne Propulsion Systems
330 Lectroid Rd.
Grover's Mill, MD 28247
US
Telephone Numbers
work: [phone]
home: [phone]
Birth Date: 1961-01-02"}
...
Last Name Begins with S and State Is MA
ConverterEntry{Key="John Smnfg", Value="John Smnfg
Addresses
Home: 178 Beacon St.
 
Hbpeak, MA 64081
US
Work: Yoyodyne Propulsion Systems
330 Lectroid Rd.
Grover's Mill, OK 92191
US
Telephone Numbers
work: [phone]
home: [phone]
Birth Date: 1978-12-29"}
...

5.3.3 Edit the Query Example to Perform Aggregations

Add code to the QueryExample.java file to perform aggregations on the cache data. An entry aggregator (com.tangosol.util.InvocableMap.EntryAggregator) enables you to perform operations on all or a specific set of objects and return an aggregation. EntryAggregator instances are agents that execute services in parallel against the data within the cluster. Aggregations are performed in parallel and can benefit from the addition of cluster members.

There are two ways of aggregating: aggregate over a collection of keys or by specifying a filter. Example 5-9 illustrates the EntryAggregator methods that perform these tasks.

Example 5-9 Methods to Aggregate Over Keys or by Specifying Filters

Object aggregate(Collection keys, InvocableMap.entryAggregator agg)

Object aggregate(Filter filter, InvocableMap.entryAggregator agg)

To add aggregations to return data for filtering:

1. Using aggregations, write code in the QueryExample class to calculate the following:

   • The number of employees that are older than a specified age. Use the GreaterFilter and the Count class:

     cache.aggregate(new GreaterFilter("getAge", nAge), new Count())
     

   • Lowest age in the set of employees. Use the AlwaysFilter and the LongMin class:

     cache.aggregate(AlwaysFilter.INSTANCE, new LongMin("getAge"))
     

   • Highest age in the set of employees. Use the AlwaysFilter and the LongMax class:

     cache.aggregate(AlwaysFilter.INSTANCE, new LongMax("getAge"))
     

   • Average age of employees. Use the AlwaysFilter and the DoubleAverage class:

     cache.aggregate(AlwaysFilter.INSTANCE, new DoubleAverage("getAge")
     

2. Import the Count, DoubleAverage, LongMax, and LongMin aggregator classes.

   import com.tangosol.util.aggregator.Count;
   import com.tangosol.util.aggregator.DoubleAverage;
   import com.tangosol.util.aggregator.LongMax;
   import com.tangosol.util.aggregator.LongMin;
   

   The QueryExample.java file looks similar to Example 5-10.

   Example 5-10 QueryExample with Aggregation

   package com.oracle.handson;
   
   import com.tangosol.net.CacheFactory;
   import com.tangosol.net.NamedCache;
   
   import com.tangosol.util.aggregator.Count;  
   import com.tangosol.util.aggregator.DoubleAverage;
   import com.tangosol.util.aggregator.LongMax;
   import com.tangosol.util.aggregator.LongMin;
   
   import com.tangosol.util.extractor.ChainedExtractor;
   import com.tangosol.util.extractor.KeyExtractor;
   import com.tangosol.util.extractor.ReflectionExtractor;
   
   import com.tangosol.util.filter.AlwaysFilter;
   import com.tangosol.util.filter.AndFilter;
   import com.tangosol.util.filter.EqualsFilter;
   import com.tangosol.util.filter.GreaterFilter;
   import com.tangosol.util.filter.LikeFilter;
   import com.tangosol.util.filter.NotEqualsFilter;
   
   import java.util.Iterator;
   import java.util.Set;
   
   /**
   * QueryExample runs sample queries for contacts.
   */
   public class QueryExample{
     
       // ----- QueryExample methods ---------------------------------------
   
       public static void main(String[] args) {
         NamedCache cache = CacheFactory.getCache("ContactsCache");
          query(cache);
       }
       /**
       * Perform the example queries
       *
       */
        public static void query(NamedCache cache)
           {
            // Add indexes to make queries more efficient
           ReflectionExtractor reflectAddrHome =
                   new ReflectionExtractor("getHomeAddress");
   
           cache.addIndex(new ReflectionExtractor("getAge"), true, null);
           cache.addIndex(new ChainedExtractor(reflectAddrHome,
                   new ReflectionExtractor("getState")), true, null);
           cache.addIndex(new ChainedExtractor(
                   new ReflectionExtractor("getWorkAddress"),
                   new ReflectionExtractor("getState")), true, null);
           cache.addIndex(new ChainedExtractor(reflectAddrHome,
                   new ReflectionExtractor("getCity")), true, null);
   
           // Find all contacts who live in Massachusetts
           Set setResults = cache.entrySet(new EqualsFilter(
                   "getHomeAddress.getState", "MA"));
           printResults("MA Residents", setResults);
   
           // Find all contacts who live in Massachusetts and work elsewhere
           setResults = cache.entrySet(new AndFilter(
                   new EqualsFilter("getHomeAddress.getState", "MA"),
                   new NotEqualsFilter("getWorkAddress.getState", "MA")));
           printResults("MA Residents, Work Elsewhere", setResults);
   
           // Find all contacts whose city name begins with 'S'
           setResults = cache.entrySet(new LikeFilter("getHomeAddress.getCity",
                   "S%"));
           printResults("City Begins with S", setResults);
   
           final int nAge = 42;
           // Find all contacts who are older than nAge
           setResults = cache.entrySet(new GreaterFilter("getAge", nAge));
           printResults("Age > " + nAge, setResults);
   
           // Find all contacts with last name beginning with 'S' that live
           // in Massachusetts. Uses both key and value in the query.
           setResults = cache.entrySet(new AndFilter(
                   new LikeFilter(new KeyExtractor("getLastName"), "S%",
                                  (char) 0, false),
                   new EqualsFilter("getHomeAddress.getState", "MA")));
           printResults("Last Name Begins with S and State Is MA", setResults);
    
          // Count contacts who are older than nAge
           System.out.println("count > " + nAge + ": "+ cache.aggregate(
                   new GreaterFilter("getAge", nAge), new Count()));
     
           // Find minimum age
           System.out.println("min age: " + cache.aggregate(AlwaysFilter.INSTANCE,
                   new LongMin("getAge")));
     
           // Calculate average age
           System.out.println("avg age: " + cache.aggregate(AlwaysFilter.INSTANCE,
                   new DoubleAverage("getAge")));
     
           // Find maximum age
           System.out.println("max age: " + cache.aggregate(AlwaysFilter.INSTANCE,
                   new LongMax("getAge")));
           }
     
       /**
       * Print results of the query
       *
       */
       private static void printResults(String sTitle, Set setResults)
           {
           System.out.println(sTitle);
           for (Iterator iter = setResults.iterator(); iter.hasNext(); )
               {
               System.out.println(iter.next());
               }
           }
       }
   

5.3.4 Run the Query and Aggregation Example

To query the cache and aggregate the results:

1. Stop the Contacts cache server, ContactsCacheServer. See "Stopping Cache Servers" for more information.

2. Restart ContactsCacheServer.

3. Run the DataGenerator, LoaderExample, and QueryExample applications.

   The output should look similar to Example 5-11 in the Eclipse Console.

   Example 5-11 Output from the Aggregators

   ...
   Last Name Begins with S and State Is MA
   ConverterEntry{Key="John Sqmyas", Value="John Sqmyas
   Addresses
   Home: 594 Beacon St.
   
   Jxaxt, MA 10081
   US
   Work: Yoyodyne Propulsion Systems
   330 Lectroid Rd.
   Grover's Mill, NJ 95236
   US
   Telephone Numbers
   work: [phone]
   home: [phone]
   Birth Date: 1977-12-29"}
   count > 42: 482
   min age: 22
   avg age: 41.627
   max age: 61