Secondary Database Example

Opening Secondary Databases with MyDbEnv
Using Secondary Databases with ExampleInventoryRead

In previous chapters in this book, we built applications that load and display several JE databases. In this example, we will extend those examples to use secondary databases. Specifically:

Before we can use a secondary database, we must implement a class to extract secondary keys for us. We use ItemNameKeyCreator for this purpose.

Example 10.1 ItemNameKeyCreator.java

This class assumes the primary database uses Inventory objects for the record data. The Inventory class is described in Inventory.java.

In our key creator class, we make use of a custom tuple binding called InventoryBinding. This class is described in InventoryBinding.java.

You can find the following class in:

JE_HOME/examples/je/gettingStarted/ItemNameKeyCreator.java

where JE_HOME is the location where you placed your JE distribution.

package je.gettingStarted;

import com.sleepycat.bind.tuple.TupleBinding;
import com.sleepycat.db.SecondaryKeyCreator;
import com.sleepycat.db.DatabaseEntry;
import com.sleepycat.db.DatabaseException;
import com.sleepycat.db.SecondaryDatabase;

public class ItemNameKeyCreator implements SecondaryKeyCreator {

    private TupleBinding theBinding;

    // Use the constructor to set the tuple binding
    ItemNameKeyCreator(TupleBinding binding) {
        theBinding = binding;
    }

    // Abstract method that we must implement
    public boolean createSecondaryKey(SecondaryDatabase secDb,
        DatabaseEntry keyEntry,    // From the primary
        DatabaseEntry dataEntry,   // From the primary
        DatabaseEntry resultEntry) // set the key data on this.
        throws DatabaseException {

        if (dataEntry != null) {
            // Convert dataEntry to an Inventory object
            Inventory inventoryItem =
                  (Inventory)theBinding.entryToObject(dataEntry);
            // Get the item name and use that as the key
            String theItem = inventoryItem.getItemName();
            resultEntry.setData(theItem.getBytes());
        }
        return true;
    }
} 

Now that we have a key creator, we can use it to generate keys for a secondary database. We will now extend MyDbEnv to manage a secondary database, and to use ItemNameKeyCreator to generate keys for that secondary database.

Opening Secondary Databases with MyDbEnv

In Stored Class Catalog Management with MyDbEnv we built MyDbEnv as an example of a class that encapsulates Environment and Database opens and closes. We will now extend that class to manage a SecondaryDatabase.

Example 10.2 SecondaryDatabase Management with MyDbEnv

We start by importing two additional classes needed to support secondary databases. We also add a global variable to use as a handle for our secondary database.

// File MyDbEnv.java

package je.gettingStarted;

import com.sleepycat.bind.tuple.TupleBinding;
import com.sleepycat.bind.serial.StoredClassCatalog;

import com.sleepycat.je.Database;
import com.sleepycat.je.DatabaseConfig;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.Environment;
import com.sleepycat.je.EnvironmentConfig;
import com.sleepycat.je.SecondaryConfig;
import com.sleepycat.je.SecondaryDatabase;

import java.io.File;

public class MyDbEnv {

    private Environment myEnv;

    // The databases that our application uses
    private Database vendorDb;
    private Database inventoryDb;
    private Database classCatalogDb;
    private SecondaryDatabase itemNameIndexDb;

    // Needed for object serialization
    private StoredClassCatalog classCatalog;

    // Our constructor does nothing
    public MyDbEnv() {}

Next we update the MyDbEnv.setup() method to open the secondary database. As a part of this, we have to pass an ItemNameKeyCreator object on the call to open the secondary database. Also, in order to instantiate ItemNameKeyCreator, we need an InventoryBinding object (we described this class in InventoryBinding.java). We do all this work together inside of MyDbEnv.setup().

    public void setup(File envHome, boolean readOnly)
        throws DatabaseException {

        EnvironmentConfig myEnvConfig = new EnvironmentConfig();
        DatabaseConfig myDbConfig = new DatabaseConfig();
        SecondaryConfig mySecConfig = new SecondaryConfig();

        // If the environment is read-only, then
        // make the databases read-only too.
        myEnvConfig.setReadOnly(readOnly);
        myDbConfig.setReadOnly(readOnly);
        mySecConfig.setReadOnly(readOnly);

        // If the environment is opened for write, then we want to be
        // able to create the environment and databases if
        // they do not exist.
        myEnvConfig.setAllowCreate(!readOnly);
        myDbConfig.setAllowCreate(!readOnly);
        mySecConfig.setAllowCreate(!readOnly);

        ...
        // Environment and database opens omitted for brevity
        ...

        // Open the secondary database. We use this to create a
        // secondary index for the inventory database

        // We want to maintain an index for the inventory entries based
        // on the item name. So, instantiate the appropriate key creator
        // and open a secondary database.
        ItemNameKeyCreator keyCreator =
            new ItemNameKeyCreator(new InventoryBinding());

        // Set up the secondary properties
        mySecConfig.setAllowPopulate(true); // Allow autopopulate
        mySecConfig.setKeyCreator(keyCreator);
        // Need to allow duplicates for our secondary database
        mySecConfig.setSortedDuplicates(true);

        // Now open it
        itemNameIndexDb =
            myEnv.openSecondaryDatabase(
                    null,     
                    "itemNameIndex", // Index name
                    inventoryDb,     // Primary database handle. This is
                                     // the db that we're indexing. 
                    mySecConfig);    // The secondary config
    } 

Next we need an additional getter method for returning the secondary database.

    public SecondaryDatabase getNameIndexDB() {
        return itemNameIndexDb;
    } 

Finally, we need to update the MyDbEnv.close() method to close the new secondary database. We want to make sure that the secondary is closed before the primaries. While this is not necessary for this example because our closes are single-threaded, it is still a good habit to adopt.

    public void close() {
        if (myEnv != null) {
            try {
                //Close the secondary before closing the primaries
                itemNameIndexDb.close();
                vendorDb.close();
                inventoryDb.close();
                classCatalogDb.close();

                // Finally, close the environment.
                myEnv.close();
            } catch(DatabaseException dbe) {
                System.err.println("Error closing MyDbEnv: " +
                                    dbe.toString());
                System.exit(-1);
            }
        }
    }
} 

That completes our update to MyDbEnv. You can find the complete class implementation in:

JE_HOME/examples/je/gettingStarted/MyDbEnv.java 

where JE_HOME is the location where you placed your JE distribution.


Because we performed all our secondary database configuration management in MyDbEnv, we do not need to modify ExampleDatabasePut at all in order to create our secondary indices. When ExampleDatabasePut calls MyDbEnv.setup(), all of the necessary work is performed for us.

However, we still need to take advantage of the new secondary indices. We do this by updating ExampleInventoryRead to allow us to query for an inventory record based on its name. Remember that the primary key for an inventory record is the item's SKU. The item's name is contained in the Inventory object that is stored as each record's data in the inventory database. But our new secondary index now allows us to easily query based on the item's name.

Using Secondary Databases with ExampleInventoryRead

In the previous section we changed MyDbEnv to cause a secondary database to be built using inventory item names as the secondary keys. In this section, we will update ExampleInventoryRead to allow us to query our inventory records based on the item name. To do this, we will modify ExampleInventoryRead to accept a new command line switch, -s, whose argument is the name of an inventory item. If the switch is present on the command line call to ExampleInventoryRead, then the application will use the secondary database to look up and display all the inventory records with that item name. Note that we use a SecondaryCursor to seek to the item name key and then display all matching records.

Remember that you can find the following class in:

JE_HOME/examples/je/gettingStarted/ExampleInventoryRead.java

where JE_HOME is the location where you placed your JE distribution.

Example 10.3 SecondaryDatabase usage with ExampleInventoryRead

First we need to import a few additional classes in order to use secondary databases and cursors, and then we add a single global variable:

package je.gettingStarted;

import com.sleepycat.je.Cursor;
import com.sleepycat.je.Database;
import com.sleepycat.je.DatabaseEntry;
import com.sleepycat.je.DatabaseException;
import com.sleepycat.je.LockMode;
import com.sleepycat.je.OperationStatus;
import com.sleepycat.je.SecondaryCursor;

import com.sleepycat.bind.EntryBinding;
import com.sleepycat.bind.serial.SerialBinding;
import com.sleepycat.bind.tuple.TupleBinding;

import java.io.File;
import java.io.IOException; 

public class ExampleInventoryRead {

    private static File myDbEnvPath =
        new File("/tmp/JEDB");

    // Encapsulates the database environment and databases.
    private static MyDbEnv myDbEnv = new MyDbEnv();

    private static TupleBinding inventoryBinding;
    private static EntryBinding vendorBinding;

    // The item to locate if the -s switch is used
    private static String locateItem; 

Next we update ExampleInventoryRead.run() to check to see if the locateItem global variable a value. If it does, then we show just those records related to the item name passed on the -s switch.

    private void run(String args[]) 
        throws DatabaseException {
            // Parse the arguments list
            parseArgs(args);
            myDbEnv.setup(myDbEnvPath, // path to the environment home
                          true);      // is this environment read-only?

            // Setup our bindings.
            inventoryBinding = new InventoryBinding();
            vendorBinding =
                 new SerialBinding(myDbEnv.getClassCatalog(),
                                   Vendor.class);

            if (locateItem != null) {
                showItem();
            } else {
                showAllInventory();
            }
    } 

Finally, we need to implement ExampleInventoryRead.showItem(). This is a fairly simple method that opens a secondary cursor, and then displays every primary record that is related to the secondary key identified by the locateItem global variable.

    private void showItem() throws DatabaseException {
            SecondaryCursor secCursor = null;
            try {
                // searchKey is the key that we want to find in the 
                // secondary db.
                DatabaseEntry searchKey = 
                    new DatabaseEntry(locateItem.getBytes("UTF-8"));

                // foundKey and foundData are populated from the primary
                // entry that is associated with the secondary db key.
                DatabaseEntry foundKey = new DatabaseEntry();
                DatabaseEntry foundData = new DatabaseEntry();

                // open a secondary cursor
                secCursor =
                  myDbEnv.getNameIndexDB().openSecondaryCursor(null, null);

                // Search for the secondary database entry.
                OperationStatus retVal =
                    secCursor.getSearchKey(searchKey, foundKey,
                        foundData, LockMode.DEFAULT);

                // Display the entry, if one is found. Repeat until no more
                // secondary duplicate entries are found
                while(retVal == OperationStatus.SUCCESS) {
                    Inventory theInventory =
                      (Inventory)inventoryBinding.entryToObject(foundData);
                    displayInventoryRecord(foundKey, theInventory);
                    retVal = secCursor.getNextDup(searchKey, foundKey,
                        foundData, LockMode.DEFAULT);
                }
            } catch (Exception e) {
                System.err.println("Error on inventory secondary cursor:");
                System.err.println(e.toString());
                e.printStackTrace();
            } finally {
                if (secCursor != null) {
                    secCursor.close();
                }
            }
        }

The only other thing left to do is to update ExampleInventoryRead.parseArgs() to support the -s command line switch. To see how this is done, see:

JE_HOME/examples/je/gettingStarted/ExampleInventoryRead.java

where JE_HOME is the location where you placed your JE distribution.