Developing Sun Master Indexes (Repository)

Sun Master Index Overview

The following topics provide information about Sun Master Index, how it is used to create a master index application, and the master index applications you create with Sun Master Index. It also includes a description of the files stored in the Java CAPS repository, the XML files that define the structure and configuration of the master index environment, and the runtime components.

About Sun Master Index

Sun Master Index provides a flexible framework to allow you to create matching and indexing applications called enterprise-wide master index applications. It is an application building tool to help you design, configure, and create a master index application that will uniquely identify and cross-reference the business objects stored in your system databases. Business objects can be any type of entity for which you store information, such as customers, patients, vendors, businesses, hardware parts, and so on. In Sun Master Index, you define the data structure of the business objects to be stored and cross-referenced as well as the logic that determines how data is updated, standardized, weighted, and matched in the master index application.

The structure and logic you define is located in a group of XML configuration files that you create using the Wizard Editor for Master Index. These files are created within the context of a Java CAPS project, and can be further customized using the XML editor provided in the NetBeans IDE.

Sun Master Index Features

Sun Master Index provides features and functions to allow you to create and configure an enterprise-wide master index application for any type of data. The primary function of Sun Master Index is to automate the creation of a highly configurable master index application . Sun Master Index provides a wizard to guide you through the initial setup steps, and various editors so you can further customize the configuration of the master index application. Sun Master Index automatically generates the components you need to implement a master index application.

Here are some of the features or Sun Master Index.

Master Index Repository Components

The components of Sun Master Index are designed to work within Java CAPS to create and configure the master index application and to define connectivity between external systems and the master index application. The primary components of Sun Master Index include the following.

Wizard Editor

The Wizard Editor takes you through each step of the master index application setup process, and creates the XML files that define the configuration of the application. The wizard allows you to define the name of the master index application, the objects to store, the fields in each object and their attributes, the Enterprise Data Manager (EDM) configuration, and the database and match engine platforms to use. The wizard generates a set of configuration files and database scripts based on the information you specify. You can further customize these files as needed.


Sun Master Index provides the following editors to help you customize the files generated in the Sun Master Index project.

Project Components

A master index application is implemented within a project in the Java CAPS repository. When you create an master index application, a set of configuration files and a set of database files are generated based on the information you specified in the wizard. When you generate the project, additional components are created, including a method OTD, an outbound OTD, Business Process methods, necessary .jar files, and a Custom Plug-in function that allows you to define additional custom processing for the index. To complete the project, you create a Connectivity Map and Deployment Profile.

Additional components can be added to the client projects that access the master index application, including Services, Collaborations, OTDs, Web Connectors, Adapters, JMS Queues, JMS Topics, Business Processes, and so on. You can use the standard Java CAPS editors, such as the OTD or Collaboration editors, to create these components.

Following is a list of Sun Master Index project components.

The following figure illustrates the project and Environment components of Sun Master Index.

Figure 1 Master Index Design-Time Components

Figure shows how the master index design-time components
relate to one another.

Configuration Files

Several XML files together determine certain characteristics of the master index application, such as how data is processed, queried, and matched. These files configure the runtime components of the master index application, which are listed in Master Index Runtime Components.

Database Scripts

Two database scripts are generated by the wizard to define external systems and code lists. Additional scripts to create or drop database tables are created when you generate the project (or by the wizard if you choose to generate all project files in the wizard).

You can also create custom scripts to store in the master index application project and run against the master index database.

Custom Plug-ins

Sun Master Index provides a method by which you can create custom processing logic for the master index application. To do this, you need to define and name a custom plug-in, which is a Java class that performs the required functions. Once you create a custom plug-in, you incorporate it into the index by adding it to the appropriate configuration file. You can create custom update procedures and field validations, as well as define custom master index components. Update procedures must be referenced in the update policies of the Best Record file; field validations must be referenced in the Field Validation file; and custom components must be referenced in the configuration file for that component. For example, if you create a custom Query Builder, it must be listed in the Candidate Select file to be accessible to the master index application.

Match Engine Configuration Files

Several configuration files for the Sun Match Engine are created in the master index project. The configuration files under the Match Engine node define certain weighting characteristics and constants for the match engine. The configuration files under the Standardization Engine node define how to standardize names, business names, and address fields. You can customize these files as needed.

Outbound Object Type Definition (OTD)

Sun Master Index generates an outbound OTD based on the object structure defined in the Object Definition file. This OTD is used for distributing information that has been added or updated in the master index application to external systems. It includes the objects and fields defined in the Object Definition file plus additional SBR information (such as the create date and create user) and additional system object information (such as the local ID and system code). If you plan to use this OTD to make the master index application data available to external systems, you must define a JMS Topic in the master index Connectivity Map to which the master index application can publish transactions.

Dynamic Java API

Due to the flexibility of the object structure, Sun Master Index generates several dynamic Java methods for use in Collaborations and in Business Processes. One set is provided in a method OTD for use in Collaborations and one set is provided for Business Processes. The names, parameter types, and return types of these methods vary based on the objects you defined in the object structure.

Generating the master index application creates a method OTD that includes Java functions you can use to define data processing rules in Collaborations. These functions allow you to define how messages received from external systems are processed by the Service. You can define rules for inserting new records, retrieving record information, updating existing records, performing match processing, and so on.

In addition to the method OTD, which can be used in Collaborations, Sun Master Index creates a set of Java methods that can be incorporated into Business Processes and into Web Services. These methods are a subset of those defined for the method OTD, providing the ability to view, retrieve, and match information in the master index application database.

Connectivity Components

The master index project Connectivity Map consists of two required components: the web application service and the application service. Two optional components are a JMS Topic for broadcasting messages and an Oracle or SQL Server Adapter for database connectivity. In client project Connectivity Maps you can use any of the standard project components to define connectivity and data flow to and from the master index application. Client projects include those created for the external systems sharing data with the index through a Collaboration or Business Process.

For client projects, you can use connectivity components from the master index server project and any standard Java CAPS connectivity components, such as OTDs, Services, Collaborations, JMS Queues, JMS Topics, and Adapters. Client project components transform and route incoming data into the master index database according to the rules contained in the Collaborations or Business Processes. They can also route the processed data back to the appropriate local systems through Adapters.

Deployment Profile

The Deployment Profile defines information about the production environment of the master index application. It contains information about the assignment of Services and message destinations to the application server and JMS IQ Managers within the master index system. Each master index project must have at least one Deployment Profile and can have several, depending on the project requirements and the number of Environments used. You must deploy the project before you can use the custom master index application you created using Sun Master Index.

Environment Components

The Sun Master Index Environments define the deployment environment of the master index application, including the domain, application server, external systems, and so on. If master index client projects use the same Environment, it might also include a JMS IQ Manager, constants, Web Connectors, and External Systems. Each Environment represents a unit of software that implements one or more master index applications. You must define and configure at least one Environment for the master index application before you can deploy the application. The application server hosting the master index application is configured within the Environment in NetBeans.

Learning about the Master Index Runtime Environment

In today’s business environment, important information about certain business objects in your organization might exist in many disparate information systems. It is vital that this information flow seamlessly and rapidly between departments and systems throughout the entire business network. As organizations grow, merge, and form affiliations, sharing data between different information systems becomes a complicated task. The master index applications you create from Sun Master Index can help you manage this data and ensure that the data you have is the most current and accurate information available.

Regardless of how you define the structure of the business object and configure the runtime environment for the master index application, the final product will include much of the same functions and features. The master index application provides a cross-reference of centralized information that is kept current by the logic you define for unique identification, matching, and update transactions.

Functions of the Runtime Environment

In the runtime environment, the master index application provides the following functions to help you monitor and maintain the data shared throughout the index system.

Features of the Runtime Environment

The runtime components of the master index application are designed to uniquely identify, match, and maintain information throughout a business enterprise. These components are highly configurable, allowing you to create a custom master index application suited to your specific data processing needs.

Primary features of the master index application include the following:

Master Index Runtime Components

The master index applications created by Sun Master Index are made up of several components that work together to form the complete indexing system. The primary components of the master index application include the following:

In addition, the master index application uses the connectivity components defined in the Sun Master Index server and client projects to route data between external systems and the master index application.

The Java CAPS repository stores information about the configuration and structure of the master index environment. Because the master index application is deployed within the repository, it can be implemented in a distributed environment. The master index system requires the Sun Java System Application Server.

The components of a master index application are illustrated in the following figure.

Figure 2 Master Index Runtime Components

Figure shows how the master index runtime components
relate to one another.

Matching Service

The Matching Service stores the logic for standardization (which includes data parsing and normalization), phonetic encoding, and matching. It includes the specified standardization and match engines, along with the configuration you defined for each. The Matching Service also contains the data standardization tables and configuration files for the match engine. The configuration of the Matching Service is defined in the Match Field file.

Manager Service

The Manager Service provides a session bean to all components of the master index application, such as the Enterprise Data Manager, Query Builder, and Update Manager. The service also manages connectivity to the master index database. The configuration of the Manager Service specifies the query to use for matching and defines system parameters that control EUID generation, matching thresholds, and update modes. The configuration of the Manager Service is defined in the Threshold file.

Query Builder

The Query Builder defines all queries available to the master index application, including the queries performed automatically by the master index application when searching for possible matches to an incoming record. It also includes the queries performed manually through the Enterprise Data Manager (EDM). The EDM queries can be either alphanumeric or phonetic and have the option of using wildcard characters. The configuration of the Query Builder is defined in the Candidate Select file.

Query Manager

The Query Manager is a service that performs queries against the master index database and returns a list of objects that match or closely match the query criteria. The Query Manager uses classes specified in the Match Field file to determine how to perform a query for match processing. All queries performed in the master index application are executed through the Query Manager.

Update Manager

The Update Manager controls how updates are made to an entity’s SBR by defining a survivor strategy for each field. The survivor calculator in the Update Manager uses these strategies to determine the relative reliability of the data from external systems and to determine which value for each field to populate into the SBR. The Update Manager also manages certain update policies, allowing you to define additional processing to be performed against incoming data. The configuration of the Update Manager is defined in the Best Record file.

Object Persistence Service (OPS)

OPS is a database service that translates high-level and descriptive object requests into actual JDBC calls. The service provides mapping from the Java object to the database and from the database to the Java object.


The master index application uses an Oracle or SQL Server database to store the information you specify for the business objects being cross-referenced. The database stores local system records, the single best record for each object record, and certain administrative information, such as drop-down menu lists, processing codes, and information about the systems from which data originates. The scripts that are generated to create the database tables are based on the information specified in the Object Definition file.

Enterprise Data Manager

The Enterprise Data Manager (EDM) is a web-based interface that allows you to monitor and maintain the data in your master index database. Most of the configurable attributes of the EDM are defined by information you specify in the wizard, but you can further configure the EDM in the Enterprise Data Manager file after you generate the Sun Master Index application. The EDM provides the ability to manually search for records; update, add, deactivate, and reactivate records; merge and unmerge records; view potential duplicates; and view comparisons of object records.

Enterprise Records

An enterprise record includes all components of a record that represents one entity. The master index application stores two different types of records in each enterprise record: system records and a single best record (SBR). A system record contains an enterprise record’s information as it appears in an incoming message from an external system. An enterprise record’s SBR stores data from a combination of external systems and it represents the most reliable and current information contained in all system records for an enterprise record. An enterprise record consists of both system records and the SBR.

System Records

The structure of a system record is different from the SBR in that each system record contains a system and local ID pair. The remaining information contained in the system records of an enterprise record is used to determine the best data for the SBR in that enterprise record. If an enterprise record only contains one system record, the SBR is identical to that system record (less the system and local ID information). However, if the enterprise record contains multiple system records, the SBR might be identical to one system record but will more likely include a combination of information from all system records.

The Single Best Record

The SBR for an object is created from the most reliable information contained in each system record representing that object. The information used from each external system to populate the SBR is determined by the survivor calculator, which is configured in the Best Record file. This data is determined to be the most reliable information from all system records in the enterprise record. The survivor calculator can consider factors such as the relative reliability of an external system, how recent the data is, and whether the SBR contains any “locked” field values. You define the rules that select a field value to be persisted in the SBR.

Objects in an Enterprise Record

In a master index application, each system record and SBR in an enterprise record typically contains a set of objects that store different types of information about the business object. A record usually contains a parent object and several child objects. A record can have only one parent object, but can have multiple child objects and multiple instances of each type of child object. For example, if the business object being indexed is a person, the record can only contain one primary name and social security number, which would be contained in the parent object (for example, a person object). However, the record could have multiple addresses, telephone numbers, and aliases, which would each be defined in different child objects (for example, in address, phone, and alias objects). Each address would be stored in a different instance of an address object.

Working with Project Components

Sun Master Index supports standard Java Composite Application Platform Suite functions for version control, XML validation, and copy, cut, and paste functions.

Version Control

Sun Master Index supports the version control functionality provided by Java CAPS. You can check files in and out, retrieve older versions to a workspace, view a version history, and so on. In addition, Sun Master Index supports recursive check-ins and check-outs. When you select Recurse Project, you can check in or out all components below the selected node or a subset of those components.

Copying, Cutting, and Pasting Files

You can use standard cut, copy, and paste commands to copy or move files between projects. Sun Master Index follows the standard functionality, with the exception that you can only copy or move a component from one project into the same node of another project. For example, you can only paste a copied configuration file into the Configuration node of another project. In addition, you cannot cut or delete components that are essential to a project, such as the configuration files, match and standardization files, and so on.