1. Implementation Guide
  2. Implement the HDR Platform
  3. Enterprise Terminology Services

Enterprise Terminology Services

Enterprise Terminology Services (ETS) is a core component of HDR that incorporates a range of terminology systems and provides extensive terminology services to HDR applications, including the following principal features:

  • Consistent and real-time access to all terminology content - whether standards-based or user-defined.
  • Support for standard-based terminologies:
    • SNOMED, LOINC, and FDB terminologies through a specialized model (referred as Core terminologies in ETS).
    • Other terminologies like ICD9, CPT4, and HCPCS through a generic model.
  • Support for user-defined terminologies.
  • High level of terminology integration - between different terminologies and between different versions of the same terminology.
  • Support for user-defined containers of terminology content such as Concept Lists and Classifications. These containers can be used for building application interfaces, constraining and validating attribute values, and generating context-sensitive reports.
  • Multi language support (MLS) on concept descriptions (except Classifications and editable terminologies).

ETS is based on the following core concepts:

Generic and Core Terminologies

ETS uses a generic terminology model that captures the essential features of disparate terminology systems. The generic terminology model provides:

  • Real-time access to terminology content.
  • A generic API that provides basic terminology services for all terminologies.
  • A data model for custom terminologies.

Terminologies are represented in ETS as Coding Schemes. A Coding Scheme is a generic structure that contains terminology meta-data, such as name, description, and active versions. Actual terminology content is loaded and stored in a Coding Scheme Version. Names of Coding Scheme Versions are decided by the user and are specified when the version is loaded. You can load new Coding Scheme Versions as required. For Coding Schemes that have multiple versions, exactly one version can be designated as default. Note that editable terminologies have only one version; Concepts in an editable terminology can be modified without loading a new version.

A Coding Scheme Version contains a definite set of Concepts, Descriptions, Attributes, and Relationships. A concept is the basic unit of information in a coding scheme version: It corresponds to a specific unit of meaning in the native terminology. Every concept has a Concept Code, which is the code given to it by the terminology. ETS identifies concepts and other ETS components (Descriptions and Relationships) using a system-generated identifier called ETS ID. Concepts (and other components) from different versions of a terminology have different ETS IDs, as the concept code may not correctly identify a concept in a different version.

A concept may have one or more textual descriptions. ETS supports multiple descriptions for a concept in the languages supported by that coding scheme version – whether defined by the terminology, or added later by the user. For concepts that have multiple descriptions, exactly one description must be designated as the preferred description for every language supported by the Coding Scheme Version. All other descriptions of a given language, associated with that concept are designated as synonyms. An application may use specific descriptions for designated contexts. This is done by defining Usage Contexts and associating local descriptions to those contexts.

A Relationship represents a directed relation between two concepts: from a source concept to a target concept. Relationships can be defined between concepts in the same Coding Scheme Version. These are usually provided as part of the terminology itself.

The generic terminology model serves as the base for a number of standard-based terminologies for which ETS provides special support. These terminologies are referred to as core terminologies in HDR. The following terminologies are referred to as core terminologies in ETS:

  • FDB4
  • HL7 v3 Code Systems (Seeded)
  • LOINC
  • IHTSDO

Special support for core terminologies is in the form of:

  • Terminology-specific APIs (in addition to the generic APIs).
  • Specific loaders (and associated integrity checking) for loading the terminologies into ETS.

As core terminologies are based on the generic terminology model, they support all features of generic terminologies, such as local descriptions, usage contexts, equivalence, attributes, and cross maps. However, core terminologies are not editable: new Coding Scheme Versions have to be loaded and activated to update the terminology content.

Equivalence

Equivalence is a symmetric, reflexive, and transitive relationship between two concepts. Two concepts—from the same or different terminologies—that have the same meaning, are considered equivalent in ETS. Equivalent concepts can be used interchangeably, without any loss of meaning. ETS provides APIs for identifying and retrieving equivalent concepts for a given concept.

Note:

ETS does not support authoring of equivalence information. Equivalence must be provided to ETS in one of the specified ETS file formats.

Equivalence may be Intra-terminology or Inter-terminology. Intra-terminology equivalence is defined between concepts from the same terminology. For example, the concept Cholera is represented in ICD-9-CM 2002 and in ICD-9-CM 2003 by the same concept code [001_CHOLERA]. Because both concepts have the same meaning, they can be treated as equivalents---ETS treats them as equivalent by default, as they belong to the same coding scheme and have the same concept code.

Intra-terminology equivalence information is provided to ETS as a “change file", when loading a new version of a terminology. Change files identify reuse of codes (codes that represent different meanings than the previous version), and reassignment of codes (meanings that are now represented with different codes).

Oracle provides Intra-terminology equivalence information only for core terminologies. You must author and load change files for new versions of custom terminologies not supported by HDR and generic terminologies supported by HDR.

Note:

ETS does not support the authoring of change files.

ETS allows concepts from two different terminologies to be defined as semantic equivalents. Equivalence between concepts from different terminologies, or Inter-terminology equivalence, is defined using cross maps. For example, the concept for the disease Cholera in the ICD-10 terminology (2016 version), and the concept representing the same disease in the IHTSDO terminology (2017 version), can be defined as equivalents using a cross map.

Interterminology Mapping

Inter-terminology Mapping provides support for any type of relationship, including equivalence, between two different terminologies. Relationships that can be defined include (but not restricted to) broader-than, narrower-than, and clinical-to-administrative relationships.

Inter-terminology Mapping is implemented using Cross Maps. A Cross Map defines the relationship between a source concept and a target concept. A number of Cross Maps are aggregated into a Map Set. ETS specifies file formats for Map Sets and Cross Maps. The HDR Loader job loads these files into ETS tables.

Concept Lists

Concept lists are arbitrary lists of ETS concepts that can be used for a variety of purposes, including validation of attribute values and populating user interface controls. Concept lists are used by all types of HDR solutions including HDR Messaging applications.

ETS Classifications

ETS Classifications are containers for grouping existing ETS concepts from different coding schemes and versions. Classifications are intended for large categorizations of concepts, while concept lists are intended for smaller sets for the purposes of validation or display. ETS Classifications provide the following features:

  • Classifications can be arranged in a hierarchy.
  • Tests for containment in a classification search down the levels of the hierarchy.
  • Classifications can be created and populated through an API or through creating and loading text files.
  • Classification contents incorporate equivalence.
  • Concepts added to an ETS Classification retain their equivalence information and characteristics.
  • ETS Classifications are themselves ETS Concepts--components of a special, predefined editable terminology called ETSClassifications. Accordingly the following applies:
    • A classification has a concept identifier.
    • A classification can have multiple descriptions, including local descriptions.
    • A classification's local descriptions can be associated with usage contexts.
  • ETS Classifications can be defined using the API or using the HDR Loader job and HDR Importer job.

Multi Language Support

ETS provides Multi Language Support (MLS) on:

  • Terminology-specified concept descriptions of non-editable terminologies.
  • Locally-specified concept descriptions of terminologies (editable and non-editable) and Classifications.

MLS in ETS lets you load Coding Scheme versions with Concept descriptions in multiple languages. Each Coding Scheme version can support Concept descriptions in multiple languages, but every Concept in a Coding Scheme version must be supplied with a terminology-preferred description in the languages supported by that version.

You can create local descriptions in multiple languages. Concept descriptions (both terminology-specified and local) based on a language can be obtained by calling methods that accept a language.

Note:

  • The terminology-specified Concept descriptions of editable terminologies and Classifications are created in the base language of the HDR installation.
  • ETS will not translate terminology content - whether seeded or loaded.
  • ETS does not perform any validation to ascertain whether a description is actually in the language that it claims to be in. You can load pseudo translated text as concept descriptions for a supported language along with the real description text. For example, you can load the data given in the following table as Concept descriptions into ETS without getting any errors:

Table 2-2 Sample Concept Descriptions

Version ID

Concept ID

Description Text

Language

v20070101

A25.66

Fracture of Femur

American English

v20070101

A25.66

Fracture of Femur

Korean

v20070101

A25.66

Fracture of Femur

Spanish

v20070101

A25.66

Fracture of Femur

Japanese

v20070101

A25.66

Fracture of Femur

German

See also:

Prerequisites

  • Implementing Security Services: User accounts must exist.

ETS Implementation Procedure

The following chart provides an overview of the implementation process:

Figure 2-2 Implementation Process for ETS

Implementation Process for ETS

To implement ETS, refer to the following procedure table:

All optional steps are done via the API. Loading and activating coding scheme versions is done using the API and scripts.

Creating New Generic Coding Schemes

ETS lets users define and implement custom terminologies for specific needs. Custom terminologies must be based on the generic ETS terminology model. Coding schemes that implement the generic terminology model are known as generic coding schemes.

Caution:

Do not attempt to create coding schemes for the following core and generic terminologies supported by ETS; they are created by default when ETS is installed:
  • CPT4
  • FDB4
  • HCPCS-2
  • HL7 v3 Code Systems (Seeded)
  • ICD-10
  • ICD-9-CM-DRG
  • ICD-9-CM-MDC
  • ICD-9-CM-V1
  • ICD-9-CM-V3
  • LOINC
  • IHTSDO

Create and implement new generic coding schemes:

  1. Create the terminology files based on the specified file structure. ETS expects the terminology content to be available as a control file and a set of three terminology files. For information about the file structure, refer the /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip folder.
  2. Create a coding scheme by using the HDR API.
  3. Load, import, and activate the coding scheme version.

The terminology content can now be loaded using the HDR Terminology Jobs.

See also:

  • Loading and Activating Coding Scheme Versions
  • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip for information about the formats required for loading ETS generic terminologies.

Implementing Editable Coding Schemes

Because user-defined terminologies frequently change, ETS lets you edit generic coding schemes in place, without loading a new version. You can define generic coding schemes as editable when they are being created.

Note:

The following terminologies supported by ETS cannot be designated as editable. They are created as non-editable by default when ETS is installed:
  • CPT4
  • FDB4
  • HCPCS-2
  • HL7 v3 Code Systems (Seeded)
  • ICD-10
  • ICD-9-CM-DRG
  • ICD-9-CM-MDC
  • ICD-9-CM-V1
  • ICD-9-CM-V3
  • LOINC
  • IHTSDO

Although editable coding schemes can be updated using ETS APIs, editable coding schemes cannot be updated by loading a new version, because they can have only one version.

Caution:

Do not attempt to load new versions for an editable coding scheme, other than the original version.

Creating an Editable Coding Scheme

Using the HDR API, you can create an editable coding scheme by creating a new generic coding scheme with the editable attribute set to true. After the coding scheme is created, you can edit the original version, but a new version is not permitted.

Loading an Editable Terminology

An initial version of the editable coding scheme must be created and loaded before it is used. Use the generic terminology loader to load an editable coding scheme version.

Editing an Editable Coding Scheme

Editing concepts, descriptions, and relationships is limited to addition and removal of attributes, changing status, and changing a description's preferred status. Other changes are made by retiring a component and adding a new component in its place. For example, a description's text cannot be changed, but the description can be retired and a new description added to replace it. The new description can optionally be designated preferred.

Equivalence

As an editable terminology has only a single version, equivalence in an editable terminology must be intraversion – in other words, equivalence may only be declared between concepts in the same version. In the initial version load, equivalence may be declared using a change file, just as for any terminology being loaded with intraversion equivalence information. After the initial load, reassignments (introduction of a new concept that has the same meaning as an existing concept but has a different concept code than the existing concept) may be declared when a new concept is added. The new concept's code may be declared, using the relevant API, to be a reassignment from an existing concept's code.

No reuse of codes (introduction of a concept whose concept code is the same as an existing code, but where the concept has a different meaning than the existing code represents) is permitted in an editable terminology.

See also:

  • Implementing Interterminology and Intraterminology Equivalence
  • Implementing Interterminology Mapping Using Cross Maps

Reference

Oracle Healthcare Data Repository API Documentation

The following table provided information about the ETSAuthoringService interface used to implement editable terminologies:

Oracle Healthcare Data Repository Javadoc

Table 2-3 Service and Methods: Editable Terminologies

Level

Detail

Package

oracle.hsgbu.ets.authoring

Interface

ETSAuthoringService

Methods

  • addAttributes

  • addConcepts

  • addConceptDescriptions

  • addRelationships

  • changeStatus

  • removeAttributes

Refer to the following sections to edit the components of an Editable Coding Scheme:

  • Adding Components

  • Changing Component Status

  • Adding and Removing Component Attributes

Note:

These changes are exclusive; any other changes to concepts, descriptions, and relationships can only be made by retiring the component and adding a new one.

Adding Components

Use the addConcepts, addConceptDescriptions, and addRelationships methods to add concepts, descriptions, relationships and attributes to an editable coding scheme. Candidate components are created first and passed to the add methods.

Changing Component Status

Components such as concepts, relationships and descriptions of an editable coding scheme cannot be edited or removed directly. To modify a component, it must be retired and replaced. A component can be retired or made active by changing the status flag associated with the component. Use the changeStatus method to change component status.

Adding and Removing Attributes

Use the addAttributes and removeAttributes methods to add and remove attributes, respectively.

Loading and Activating Coding Scheme Versions

Terminologies have to be loaded into ETS (as coding scheme versions), imported, and activated before they are used. This includes initial versions of core terminologies, which have to be loaded, imported and activated at implementation.

ETS provides terminology loader and importer jobs that load and import a terminology after performing required validations. ETS provides custom loader and importer jobs for core terminologies, and generic loader and importer jobs for generic and custom terminologies.

Note:

New versions of the following terminologies can be loaded if required; HDR does not seed versions of these terminologies (they are available from Apelon, Inc.):
  • IETF RFC 1766
  • ISO 3166-1 alpha-2
  • NUBC-UB92

Caution:

Do not load versions of terminologies that are seeded in HDR. These include:
  • HL7
  • HDR Supplemental

If you have already loaded such versions, mark them as retired and non-default.

The procedure for implementing a new coding scheme version is the same for both generic and core terminologies. To load and activate coding scheme versions, perform the following steps:

Steps

  1. Prepare the terminology files.
  2. Load the terminology into ETS as a coding scheme version.
  3. Publish the coding scheme version.
  4. Activate the coding scheme version.

You can use HDR Terminology Jobs to load and publish coding scheme versions.

See also:

  • Appendix D: Running HDR Terminology Jobs
  • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip for additional information about loading.

    The notes can be categorized into DBA, General, and Terminology related notes.

    • DBA Notes: Gives a brief description of some common database management issues related to ETS, such as sizing issues, rollback adjustments for loads and imports, intermedia text indexes, and load/import performance, as well as a general description of database access patterns of ETS.
    • General Notes: Gives the basic principals common to all terminology file formats. This section must be read before moving on to the details of specific loader file formats.
    • Terminology Notes: Gives additional information relating to the core terminologies supported in ETS. Each of the core terminologies have a separate notes file. These files are to be referenced for information relating to the respective loader file formats.

Preparing Terminology Content and Control Files

To create the terminology files and move them into the correct folder, perform the following steps:

Steps

  1. If the terminology is a generic terminology, create the terminology files in the format expected by the ETS generic loader. Otherwise, ensure that the files are in the format expected by the appropriate terminology loader. (If the terminology is supported by Apelon, this step is not required.)

    See also:

    • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip, for details regarding file formats required by ETS terminology loaders.

    • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme/Change_File_Formats_General.txt from hdr-1.0.0-8.0.0.zip, for details about change files.

    Note:

    • To load equivalence information for the terminology version being loaded, the change file must be specified while loading the version-ETS does not support retrospective loading of change information.
    • For equivalence processing to be performed correctly, versions must be loaded in order. Equivalence processing assumes that the codes referenced in the change file are from the version currently being loaded and its immediate predecessor.
  2. Move the terminology files to a directory in the same file system as the Applications instance-a directory that is accessible by the Oracle Database Scheduler (DBMS_SCHEDULER).
  3. Create a control file that reflects the locations of the terminology files and move it to a directory in the same file system as the Applications instance-a directory that is accessible by the Oracle Database Scheduler (DBMS_SCHEDULER).

Loading a Coding Scheme Version

To load a new coding scheme version, use the Oracle Database Scheduler (DBMS_SCHEDULER).

See also:

  • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip, for information about control files.

Using Oracle Database Scheduler (DBMS_SCHEDULER)

Select the HDR Loader Job, and enter values for the Control File (absolute path), Coding Scheme Name, and Coding Scheme Version Name parameters. For more information, refer to Appendix D: Running HDR Terminology Jobs.

Publishing a Coding Scheme Version

A loaded terminology is staged for importation into ETS. Use HDR Importer Job directly for publishing a coding scheme version. For more information refer Appendix D: Running HDR Terminology Jobs.

The published coding scheme version is in the quarantined state by default. The coding scheme version must be activated before it can be used.

Note:

In order to support concept equivalence, the HDR Importer job process does not publish a second quarantined version of a coding scheme if one already exists. This facilitates verification of equivalence between the quarantined version and the previous version of the terminology before the quarantined version is published.

Using Oracle Database Scheduler (DBMS_SCHEDULER)

For publishing a staged coding scheme version, select the HDR Importer job, and enter values for the Load Sequence Number and Dry Run Mode parameters. You can get the Load Sequence Number from the log file of the HDR Loader job that has successfully loaded the data (coding scheme versions, classifications, and cross maps).

See also:

Note:

In order to support concept equivalence, the HDR Importer job does not publish a second quarantined version of a terminology if one already exists. This facilitates verification of equivalence between the quarantined version and the previous version of the terminology before the quarantined version is published.
See also:
  • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme, for details regarding file formats for ETS terminologies and loaders.

Activating a New Terminology Version

A quarantined coding scheme version must be activated before it can be used. A quarantined version can be activated only ETS API.

Implementing Interterminology and Intraterminology Equivalence

Over a period of time, ETS can use different concepts to record the same meaning. This happens either because of changes to the terminology or because different terminologies are used to record the same meaning. The Concept Equivalence service lets HDR solutions find data recorded using different concepts.

With Concept Equivalence, concepts from the same or different terminologies—that have the same meaning— are considered equivalent. Concept Equivalence facilitates specification and query of concepts that are equivalent.

Concept equivalence is also used by HDR solutions that implement the HDR messaging services.

Concept equivalence services include:

  • Intraterminology Equivalence
  • Interterminology Equivalence

Intraterminology Equivalence

Intraterminology equivalence deals with identical concepts (those with the same meaning) within a single terminology. When a new version of a terminology is released, there may be several changes to the representation and meaning of concepts when compared to the previous version. Because there is no way for ETS to automatically determine these changes, by default it treats concepts from the previous and new versions as distinct and unrelated. However, it is possible to explicitly indicate the changes that have occurred between a prior version and a new version in a change file that is loaded with the new version. Using this information, Intraterminology equivalence services can determine whether two concepts from the previous version and the new version have the same meaning.

Given a concept from a version of a terminology, ETS can retrieve equivalent concepts from all contiguous versions that have change files loaded. Given two concepts from different versions of a terminology, ETS can verify if they are equivalent, provided that the more recent version and all the intermediate versions have change files loaded.

Interterminology Equivalence

Interterminology equivalence deals with identical concepts (those with the same meaning) from different terminologies. Concepts from two different terminologies can vary widely in their granularity and coverage of a domain. Because there is no way for ETS to automatically determine these differences, by default it treats concepts from the two terminologies as distinct and unrelated. However, it is possible to explicitly indicate equivalence between concepts from two versions of different terminologies in the form of an Interterminology Mapping. Using this information, Interterminology equivalence services can determine whether two concepts from different terminologies have the same meaning.

Combining Intraterminology and Interterminology Equivalence

Equivalence between concepts is a transitive relationship. In the following chart, if Concept A1 is equivalent to Concept A2, and Concept A2 is equivalent to Concept A3, it can be inferred that Concept A1 is equivalent to Concept A3. Consistent with this logic, Concept Equivalence services in ETS can determine if concepts from two terminologies are equivalent—provided that an inter terminology mapping exists between versions of the two terminologies, and, change files have been loaded for all versions.

Concept Equivalence Model

Concept Equivalence Model

In this chart, ETS transitively combines intraterminology equivalence and interterminology equivalence information to infer that Concept A1 is equivalent to Concept B1.

Prerequisites

None

Implementing Intraterminology Equivalence Using Change Files

Change files are used to document the differences between successive versions of a terminology that are loaded into ETS, in a format that is acceptable for loading purposes. Change files are loaded at the same time as the terminology version data using the same loader and importer. Change files contain the following types of information:

  • Reassignment: The meaning of one concept is occasionally reassigned to another concept represented by a different concept code. Following are some of the situations in which reassignment occurs:
    • Duplicate concepts are detected. One of them is elected to continue representing the meaning while the other is retired or deleted and reassigned to the retained concept.
    • A concept is detected to be erroneous. The erroneous concept is retired or deleted and reassigned to a correct concept.
    • The classification of a concept is changed. If the concept codes are hierarchical (as in ICD-9-CM), the change in classification translates to a change in concept code, necessitating reassignment.
    • A reassignment is indicated in a change file by a row containing an entry of type S (semantic reassignment), followed by a source concept code (the concept whose meaning is being reassigned to another code), and a target concept code (the concept whose code now captures the meaning).

    See also:

    • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme/Change_File_Formats_General.txt from hdr-1.0.0-8.0.08.0.0.zip for more information about the format of the change file.

      If both the source and target concepts of a reassignment are from the new version, the reassignment is said to be intra-version. For example, in SNOMED-CT, if a duplicate or erroneous concept is detected, the new version carries forward the duplicate or erroneous concept in an inactive status. The reassignment in this case is from the inactive concept in the new version to an active concept in the new version.

      If the source of a reassignment is from the previous version and the target is from the new version, the reassignment is said to be inter-version. For example, in ICD-9-CM, if an erroneous concept is detected, it is deleted and excluded from the new version. A correct concept is provided in the new version and a reassignment is created between the concepts in the previous version and the new version. To process an inter-version reassignment contained in a change file, the ETS importer looks for the source code in the non-quarantined version (of the terminology in question) that has the latest load date. The non-quarantined version can be either active or retired.

    • Reuse: Occasionally, a concept code used in the previous version is reused to represent a concept with a different meaning in the new version. Note that this is considered bad terminology practice and should only be used to account for inadvertent errors. Unless a reuse is explicitly called out in the change file, a concept in the previous version is always considered equivalent to a concept with the same concept code in the new version.

Change files are preceded for the following terminologies (no further implementation steps are required):

  • HL7
  • HDR Supplemental

For each of the following terminologies, new versions and their change files are available to customers, possessing licenses, from the vendor of the terminology:

  • IETF RFC 1766
  • ISO 3166-1 alpha2
  • NUBC-UB92
  • CPT-4
  • FDB
  • HCPCS-2
  • ICD-10
  • ICD-9-CM-DRG
  • ICD-9-CM-MDC
  • ICD-9-CM-V1
  • ICD-9-CM-V3
  • LOINC
  • SNOMED-CT

For all other terminologies that are loaded into ETS, change files must be created separately and loaded for each new version as described in the implementation steps.

Note that change files must be loaded concurrently with the new version of a terminology. It is not possible to load a change file for a version of a terminology after both versions have been loaded.

Note also that a change file can only equivalence concepts between two consecutively imported versions of a terminology. Hence the order in which versions are imported is significant if change files are being used. The following scenarios illustrate this constraint:

  • A concept (concept code X) exists in version 1 of a coding scheme. The concept neither appears in version 2 nor is reassigned to an equivalent version 2 concept. A concept with code X reappears in version 3 with the same meaning as in version 1. It is not possible to indicate to ETS that concept X from version 1 is equivalent to concept X from version 3-because it spans a version.
  • A concept (concept code X) exists in version 1 of a coding scheme. The concept neither appears in version 2 nor is reassigned to an equivalent version 2 concept. In version 3, another concept (concept code Y) is created that is equivalent to concept X from version 1. It is not possible to indicate to ETS that concept X from version 1 is equivalent to concept Y from version 3.

    Note:

    To support concept equivalence, the ETS importer does not import a second quarantined version of a terminology if one already exists. This facilitates verification of equivalence between the quarantined version and the previous version of the terminology before the quarantined version is published.

Steps

  1. Determine if the reassignment information for the terminology is inter-version or intra-version. Use the following rules to make this determination:
    • Intra-version reassignment is used by terminologies that carry forward the duplicate or erroneous concept into the new version, albeit with a retired status, and reassign it to an active concept with a different concept code in the new version.
    • Inter-version equivalence should be used for terminologies that do not carry over the concept to be reassigned into the new version. Such terminologies will instead reassign directly from the concept in the previous version to a concept in the new version.
  2. Create the change file with the appropriate Reassignment and Reuse entries. See /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme/Change_File_Formats_General.txt from hdr-1.0.0-8.0.0.zip for details about the format of the change file. Note that the S ENTRY_TYPE is called Reassignment in this guide.
  3. Set the HISTORY_TYPE property of the terminology loader control file to INTRAVERSION or INTERVERSION based on the determination made in Step 1. If the HISTORY_TYPE is set to INTRAVERSION, both the SOURCE_CONCEPT_CODE and TARGET_CONCEPT_CODE of a reassignment are assumed to be from the version being loaded. If the HISTORY_TYPE is INTERVERSION, the SOURCE_CONCEPT_CODE is assumed to be from the previous version, and the TARGET_CONCEPT_CODE is assumed to be from the new version.

    Note:

    If no HISTORY_TYPE is present, the value defaults to NONE and no equivalence information is processed. If the HISTORY_TYPE is INTERVERSION or INTRAVERSION, a change file must be specified using the CHANGE_FILE property. Even if no reassignment or reuse has occurred, an empty change file with the header line must be provided.
  4. Specify the location of the change file using the CHANGE_FILE property of the terminology loader control file.
  5. Perform the steps described in Loading and Activating New Terminology Versions to load and activate the terminology version along with the change file.

See also:

Implementing Interterminology Equivalence Using Interterminology Mapping

Interterminology equivalence is implemented by creating cross maps. Cross maps that implement equivalence constitute a special case of interterminology mapping. Note that interterminology mapping can also be used to map source and target concepts where the source is semantically broader than or narrower than the target, or where the source and target are related in some other manner.

Cross maps can be used to implement equivalence by specifying that the source and target are equivalent in the EQUIVALENCECONTEXT field of the Cross Maps file.

Implementing Interterminology Mapping Using Cross Maps

Interterminology mapping provide a mechanism by which concepts from a source version in one terminology can be mapped to concepts from a target version in another terminology. Mappings are typically tailored for a specific application. For example, a data-aggregating or reporting application may require a mapping between specialized SNOMED-CT codes and coarse ICD-9-CM codes. A data retrieval application may use mappings with the opposite semantics (from less granular classifier codes to more detailed codes). These examples serve to illustrate that mappings serve multiple purposes, and not all cross maps indicate equivalence. Those cross maps that truly do indicate equivalence must be explicitly flagged by the author of the cross maps. This section describes the steps you should follow to indicate equivalence between concepts from two different terminologies using interterminology mapping files.

Perform the following steps to define interterminology mapping using cross maps:

Steps

  1. Create Map Set Loader files.
  2. Perform the following steps on the file referenced by the CROSS_MAP_FILE property of the Map Set Loader Control File:
    • The eighth column of the cross maps file is called EQUIVALENCECONTEXT. The Map Set Loader inspects this column in each row to determine if the cross map in that row can be used for equivalence. If this column in a particular row is left empty or set to null, the cross map in that row will not be used by ETS Concept Equivalence Services.
    • If the EQUIVALENCECONTEXT column is populated, the cross map is interpreted as indicating equivalence between the concepts represented by the MAPCONCEPTID and the MAPTARGETID.
    • Because determining similarity of meaning between concepts from different terminologies is often subjective, it may not be appropriate to use the same set of cross maps for interterminology equivalence on all occasions. For example, the requirements of a reporting application may be satisfied by a looser definition of equivalence than a clinical order entry application. The EQUIVALENCECONTEXT parameter lets each cross map be associated with the context in which its use is appropriate. At runtime, the EQUIVALENCECONTEXT can be provided as a parameter to ETS Equivalence Services to selectively use only those cross maps that are associated with that context.
    • The default EQUIVALENCECONTEXT is SYSTEM. Cross maps that are flagged with this context will be used by IMP and OMP to determine equivalents in concept lists and the master catalog. If a context is not specified in an equivalence query at runtime, this context is assumed by default.
    • If the same cross map is deemed suitable for multiple contexts, it may be repeated several times in the cross maps file, each time with a different EQUIVALENCECONTEXT.

    See also:

    • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme/Terminology_File_Formats_MapSet.txt from hdr-1.0.0-8.0.0.zip for Map Set Loader file formats.
  3. Load the Map Set files.

Loading the Map Set Files

Perform the following steps to load the map set files:

  1. Create a cross map file and a control file in ETS format, and move them into a directory located in the same file system as the Applications instance-a directory that is accessible by the Oracle Database Scheduler (DBMS_SCHEDULER).

    See also:

    • Guidelines: Cross Maps
    • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip, for details about Cross Map file formats.
  2. Use Oracle Database Scheduler (DBMS_SCHEDULER) to load the map set files.

    Select the HDR Loader Job. Select the coding scheme name called Map Set Loader, and enter values for the control file and coding scheme version name.

    See also:

    • Oracle Applications System Administrator's Guide
  3. After completion of the load, use the HDR Importer Job to import the loaded terminology.

    Enter the load sequence number. You can get this value from the log file of the HDR Loader Job that has successfully loaded the data (coding scheme versions, classifications, and cross maps).

    Select Off as the value of dry run mode.

    See also:

Guidelines: Cross Maps

The ETS Cross mapping model is based on the SNOMED CT cross mapping model. Cross-mapping mechanisms provide support for the following:

  • Mapping a single concept to a target code (a one-to-one mapping).
  • Mapping to a set of Target codes (a one-to-many mapping).

The current structure does not support:

  • Mapping a set of Concepts to a Target.

The relationship between these tables is shown by the following chart:

Figure 2-3 ETS Cross Mapping Relationship

ETS Cross Mapping Relationship

A map set defines a mapping between two coding scheme versions, such as Terminology A version 2003 and Terminology B version 2.01. Each map set is composed of multiple cross maps. Each cross map consists of a source concept and one or more target concepts, such as a source concept from Terminology A and one or more target concepts from Terminology B—to which it maps.

Loading Cross Maps Provided by the College of American Pathologists

The principal difference between cross map files distributed by the College of American Pathologists (with SNOMED CT) and those expected by ETS loaders is that the SNOMED CT files could contain data regarding multiple map sets in a single file. The map set file may contain multiple rows, each pertaining to a different map set. The cross map file may contain cross maps relating to multiple map sets, and the map targets file may contain targets used by multiple map sets (targets related to multiple coding schemes).

To make the SNOMED CT files suitable for ETS loading, split the files into map sets. The map set file should contain only one row, representing one map set. The cross maps file should contain only rows containing the map set ID of the chosen map set. The map targets file should contain only targets related to the target coding scheme specific to the map set.

See also:

  • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme from hdr-1.0.0-8.0.0.zip for the cross-map table structures.

Creating Local Descriptions

You can specify local descriptions for any ETS concept. These descriptions can be used in place of terminology-specified descriptions for display purposes. You can assign a single usage context to each local description.

The usage context for each local description must be unique; no two local concept descriptions share the same usage context.

Note:

Local descriptions of a concept can be created for any language, not just for languages loaded with the coding scheme version.

Use ETS API to create local description. You can create local descriptions for retired or active concepts, but the typical procedure is to create a description for concepts in the active default version of the terminology.

Managing Usage Contexts

Usage Contexts let an HDR solution specify and determine which concept list or local description (of a specific ETS concept) should be used in a given application context. Usage contexts are an attribute of concept lists and local descriptions. A user may specify a usage context when a concept list or a local description is created or later.

When accessing a concept list or local description, HDR solutions may specify a usage context. Based on the specified usage context, HDR retrieves a matching concept list or local description. For example, a Utilization Review department may require diagnoses to be displayed as short names or abbreviations. For this to be implemented in HDR, first a usage context with a name such as Utilization Review must be created. Then, the required local descriptions (short names or abbreviations) with the Utilization Review usage context can be created for the appropriate concepts. Subsequently, applications developed for the department can use ETS APIs with the Utilization Review usage context to display the required local descriptions.

Each local description of a concept must have a single usage context that is unique for that concept. If a local description is assigned a usage context and a local description for that concept already exists with the same usage context, the operation succeeds—but the usage context is removed from the first local description.

A usage context can similarly be used by a concept list. associated with an organization can by an application. For example, a healthcare enterprise might have a concept list of medical services called ENT_MED_SERVICES. A hospital owned by the enterprise may require a specialization of that concept list that contains a subset of the original values. For this to be implemented in HDR, first create a usage context with a name such as Fair Oaks, and associate it with a hospital unit. Then, a specialization of the ENT_MED_SERVICES concept list can be created, with a name such as FAIR_OAKS_MED_SERVICES, and associated with the Fair Oaks usage context.

Use ETS API to manage usage context.

Associating a Usage Context with an Organization

Use the following method to associate a usage context with an organization:

associateUsageContextWithExternalOwner

See also:

  • Oracle Healthcare Data Repository Javadoc

Implementing Concept Lists

Concept lists group ETS concepts for a variety of purposes, such as displaying certain concepts in user interface drop-down lists and other controls, or constraining values of an attribute to certain concepts.

Concept list member concepts possess activation and retirement dates; and have active, retired, or pending status within a list. Each member concept in a concept list has a code by which it is known in the list (a membership code). The membership code is unique among active or pending members of the concept list, and it can be used by HDR solutions to identify the member concept.

HDR is shipped with a set of pre-defined concept lists. These concept lists are used within HDR to validate attributes that must have coded values, and to process inbound messages. These concept lists have predefined concept names that start with either CTB_ (prior releases) or CL_ (current release).

ETS treats concept lists as of three types based on whether the concept list can be extended or not:

  • SYSTEM: HDR is shipped with a set of pre-defined concept lists. These concept lists are used within HDR to validate attributes that must have coded values, and to process inbound and messages. These concept lists have predefined concept names that start with either CTB_ (prior releases) or CL_ (current release).
    • Do not use predefined concept lists for any other purpose (other than the defined purpose).
    • Do not use the CTB_ and CL_ prefix for concept lists you create.
  • EXTENSIBLE: Certain concept lists have been seeded empty-the concept lists are created in the system, but no concepts are added to them. They must be filled by concepts during implementation. Predefined lists that are empty are necessarily extensible. Other predefined lists that have concepts may be extensible also: content may be added to them as required.
  • USER:

You can create specializations of concept lists that are defined as EXTENSIBLE. Specializations are child concept lists that initially inherit parent concepts, but may be modified by adding or removing member concepts.

A Specialization can be associated with a usage context, which may in turn be associated with a particular HDR organization.

You can also subset seeded lists, if only a subset of the seeded values are applicable. Because HDR seeded lists are of the type SYSTEM or EXTENSIBLE, and only lists designated USER can be subsetted directly, you must employ indirect methods to subset seeded lists.

Creating and Updating a Concept List

Use ETS API to create a concept list. For more information on Concept Lists API, refer to the Oracle Healthcare Data Repository Javadoc.

Adding Concepts to a Concept List

Concepts can be added to extensible concept lists. Concepts to be added to a concept list must be contained in a coding scheme version that has already been loaded into ETS. Determine if the concept exists in ETS and the coding scheme version in which it is contained.

Perform the steps described in this section to add concepts to a concept list.

Caution:

Certain extensible concept lists are empty and must be populated with concepts before using their respective functionality.

Adding Concepts to a Concept List

Adding Concepts to a Concept List

A concept cannot be added under the following conditions: the concept is already active or pending in the selected list; the selected list is the parent of an additive child specialization and the concept is already active or pending on the child; the selected list is a SYSTEM list; or the selected list is a restricted child specialization and the concept is not active or pending on the parent lists.

  • If any (but not all) concepts chosen for addition are not addable, a warning diagnostic lists the concepts that will not be added. You can elect to continue or return to list selection.
  • If all concepts selected for addition are not addable, the List Selection window is reloaded with an information box listing these concepts. To continue, select another list.

If any of these conditions exist, select a new membership code or activation date as appropriate, and click Next. You can alternatively click Back to return to the List Selection page, or click Cancel to exit the addition process.

Note:

If the concept is being added to a restricted specialization, and the selected activation date would cause the concept's active period to exceed that of the corresponding concept in the parent list, an exception occurs; a dialog warns you that the concept has not been added.

See also:

  • Oracle Healthcare Data Repository Javadoc

Caution:

We strongly recommend that wherever possible, you only add concepts from the same terminology to a single concept list. Concept meanings can be sensitive to the context in which they are included in a terminology; mixing them with concepts from other terminologies may distort those meanings.

Specializing a Concept List

Concept lists can be specialized. A specialization of a concept list is a child concept list that initially inherits the active members of the parent list. It is a separate concept list, distinct from the parent list. Subsequent behavior of the specialization (a child concept list) with respect to the parent concept list depends upon the setting of its inheritance type:

  • Addition Inheritance: Any concept added to the parent list is added to the child list.
  • Deletion Inheritance: Any concept retired from the parent list is retired from the child list.
  • Restricted Inheritance: A child list cannot contain any concept not contained in its parent list; before a concept is added to the child list it must first either exist in the parent list or be added to the parent list. A concept in a restricted child list also inherits certain changes to the activation and retirement dates of the corresponding concept in the parent list.

The inheritance types are not mutually exclusive. A restrictive list must also exhibit deletion inheritance. You can update a list's addition inheritance and deletion inheritance by turning them on or off, but you cannot update its restricted inheritance.

A specialization can be associated with a usage context, as can a concept local description. A usage context can in turn be associated with an organization. Accordingly, a concept list can have multiple specializations, each associated with a particular organization.

A concept list specialization is created in the same manner as any other concept list (Steps).

Values in a concept list specialization can be added or retired as for any concept list.

Subsetting a Concept List

It may be desirable to subset a concept list—using only a subset of a concept list's members, for UI display purposes, or for validating data to be stored by an application. The following sections describe how to subset a concept list:

Subsetting a User Concept List

A concept list of extensibility type User can be subsetted by retiring unwanted members from the list. A member is retired by updating its retirement date and time.

Note:

This subsetting procedure does not apply to System or System extensible concept lists.

Subsetting a Concept List of any Extensibility Type

A concept list of any extensibility type (including user) can be subsetted using either of two additional procedures. These procedures are especially useful if the list to be subsetted is a System or System Extensible list, from which members cannot be retired (two methods):

Method 1: Using the Core Member Setting of List Members

The core set of members in the list can then be retrieved using the method:

getCoreSet

Checks of individual members of the list can be performed using the method:

isCoreMember

Method 2: Using a Specialization of the Concept List and Retiring Members

Reference:

Oracle Healthcare Data Repository Javadoc

Table 2-4 Service and Methods: Specializing Concept Lists

Level

Detail

Package

oracle.hsgbu.ets.base

Class

ConceptList

Methods

getChildConceptList

To subset a concept list of any extensibility type using Procedure-2, do the following:

Create a specialization of the concept list, and specify a usage context for the child list. You can then subset the child concept list, and you can use the subsetted list as required by your application.

Access the specialization using the method:

getChildconceptList

Implement Security

HDR EJB (Enterprise Java Beans) services require all callers of the service be authenticated by the application server.

WebLogic admin user can access the HDR services. But still it is recommended to create a HDR specific WebLogic user called hdradmin.

WebLogic users can be maintained in the default security provider or any WebLogic server supported providers such as LDAP, that can be configured and used with HDR.

HDR does not require user authorization to access the services. Any authenticated user in any role can access HDR services.

Reference

Implementing ETS Classifications

Large amounts of healthcare data are difficult to use unless they are well organized. Creating classifications is the most common means of organizing healthcare data. As institutions generally use a combination of (standard and local) terminologies, classifications need to incorporate concepts from different coding schemes and versions.

The following table summarizes the principal interfaces referenced by this section:

Table 2-5 Service and Methods: ETS Classifications

Level

Detail

Package

oracle.hsgbu.ets.base

Class

ETSAdministrationService

Methods

  • addConceptToConceptList

  • addDeclarationToClassification

  • createClassification

  • createCodingScheme

  • createConceptList

  • removeDeclarationFromClassification

Class

ETSService

Methods

  • findClassificationByCode

  • findcodingScheme

  • findCodingSchemeVersion

  • findConcept

  • findConcepts

  • isEquivalent

Class

Classification

Methods

  • contains

  • findChildClassifications

  • findConceptsInClassification

ETS Classifications provide a mechanism for grouping concepts from different coding schemes and versions, and arranging the groups in hierarchies navigable by the ETS API. ETS Classifications facilitate:

  • Viewing a large number of concepts
  • Selection of concepts
  • Class-based query of information

For example, a classification called cardiovascular diseases could be created and populated with concepts that represent different cardiovascular diseases from different terminologies. The following chart shows this classification:

Figure 2-4 Simple ETS Classification

Simple ETS Classification

ETS Classifications are internally represented as concepts in an editable generic coding scheme called ETSClassification. In this example, the classification, cardiovascular diseases is stored as a concept in the coding scheme ETSClassification.

But you could also create another classification such as heart diseases and make it a child classification of the cardiovascular diseases classification. Thereafter, any concept in the heart diseases classification, such as congestive heart failure, would be considered by the Classifications interface to be implicitly contained in the cardiovascular diseases classification.

Figure 2-5 Parent and Child ETS Classification

Parent and Child ETS Classification

ETS Classifications are basically containers of ETS concepts. In this chart, the concepts in each classification node could derive from different coding schemes.

ETS classification interfaces are oblivious to relationships between the classification concepts in their native terminologies. For example, in this chart, congestive heart failure and congenital cardiac failure are children of heart failure in their native terminology, but the heart disease ETS Classification views its contents as a flat list of concepts. There is no classification interface that is aware of relationships between the classification concepts in their native terminologies. However, those relationships can be navigated by querying the individual concepts, using the ETS generic terminology interface or a terminology-specific interface.

Classifications are themselves ETS Concepts – components of a special, pre-defined editable terminology called ETSClassifications. Creating a new classification actually is creating a new concept in the ETSClassifications terminology. Since classifications are concepts:

  • A classification may have multiple descriptions, one of which is designated preferred
  • Local descriptions may be associated with usage contexts
  • A classification has an ETS ID

Classifications can be Linked Hierarchically

Classifications may be linked in strict or multiple hierarchies (a classification may have multiple parent classifications) to form a network of linked classifications forming an acyclic graph.

Testing Containment

ETS classifications support testing of containment across levels of a hierarchy. For example, given the hierarchy in the chart Sample ETS Classification, if the concept congestive heart failure is in the heart disease classification, and the heart disease classification is a child of the cardiovascular diseases classification, you can test if congestive heart failure is a cardiovascular disease, and the answer will be Yes.

Equivalence is incorporated into Classifications.

The concepts contained in an ETS Classification retain their equivalence information. For example, what is actually considered as included in cardiovascular diseases are groups of concepts that are equivalent to heart failure, congenital cardiac failure, congestive heart failure, and arterial anuerysm. Consequently, if concept X is equivalent to congestive heart failure, and the question “Does ‘cardiovascular diseases' contain X?" is asked, the answer will be “yes".

Creating and Populating Classifications

Classifications are created by generating a hierarchical network of classification nodes and populating those nodes with individual ETS concepts. An individual classification node may contain concepts from multiple terminologies.

A single concept can be used in multiple classifications, including multiple sub-classifications of the same parent classification, but it cannot appear more than once in the same sub-classification. However, equivalent concepts can be inserted into the same sub-classification. As for all ETS concepts, concepts in a classification can have multiple text representations.

Classifications may be created and populated by two methods:

  • Specifying classification data in files, and loading the files using HDR Terminology Jobs
  • Using ETS APIs directly

A new classification will have a pending state when it is created and loaded. It will become effective only when the HDR Maintenance job is run. Until then, the new classification will be unusable. This enables new classifications to be created and readied for use without requiring a downtime.

Whenever changes are made that could affect classification contents (declarations are added or removed, a version of a terminology that contains classification concepts is loaded, or a mapping involving a version that contains classification concepts is loaded), the classification moves from the active state to a dirty state. In this state the classification in its former active state is still usable. The effects of the changes that placed the classification into the dirty state will not be usable until the classification is moved from the dirty state to the active state. Classifications are moved from the dirty state to the active state by running the HDR Maintenance Job.

Note:

Classification contents are defined declaratively. This enables a short-cut for adding concepts declared a terminology to be children of another concept. For example, the concepts heart failure, congenital cardiac failure, and congestive heart failure could have been added to heart diseases by a single statement (add heart failure and its descendents to heart diseases). This adds heart failure and its children as defined within its native terminology. A declaration can also add a concept and only those concepts deemed to be direct children in its native terminology, add a concept's descendents but not the concept itself, or add a concept's direct children but not the concept itself. The various insertion choices are called insert options.

See also:

  • Oracle Healthcare Data Repository Javadoc

    A declaration with an insert option of concept only adds a single concept ((the concept with none of its children). The contents of a classification amount to a series of declarations. Classification contents are augmented or reduced by adding or removing declarations.

    To retrieve a concept's children when implementing a declaration, the ETS classification build process must know which relationships in the concept's native terminology represent parent-child relationships. Accordingly, for each core terminology, HDR has identified certain relationships as defining parent-child associations. The ETS classification build process queries for such relationships when called upon to find a concept's children. For generic terminologies, the build process queries for relationships in which the relationship type concept is identified as type IS_PARENT_OF or IS_CHILD_OF.

See also:

  • Documentation at the Apelon, Inc. web site for descriptions of the treatment of specific terminologies.
  • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme /Terminology_File_Formats_Generic.txt from hdr-1.0.0-8.0.0.zip

Note:

Each ETS concept defined in a generic terminology must be identified as being valid or invalid for use as a relationship type, and is indicated by the RELATIONSHIPTYPEFLAG column in the Concepts fie. This flag can contain one of the following values (Table):

Table 2-6 Concept File: RELATIONSHIPTYPEFLAG Legal Values

Value

Description

N

The concept cannot be used as a relationship type.

Y

The concept can be used as a relationship type.

IS_PARENT_OF

The concept can be used as a relationship type, and the type indicates that the source concept is the parent of the target.

IS_CHILD_OF

The concept can be used as a relationship type, and the type indicates that the source concept is the child of the target.

Those concepts designated as valid relationship types in the concepts file can subsequently be used in the relationships file in the RELATIONSHIPTYPECONCEPTCODE column.

Use the following HDR interfaces to define and use ETS classifications:

  • ETSAdministrationService
  • ETSService
  • Classification

Creating a Classification

To create and populate a classification via the loader, perform the following steps:

Steps

  1. Create a Classifications file. This file lists the classifications to be created and their properties.
  2. Create a Classifications descriptions file: This file lists the descriptions to be associated with the classifications and their properties. Just as multiple descriptions for a concept can be listed in a terminology descriptions file, multiple descriptions can be created for a classification.

    See also:

  3. Create a Classifications declarations file. This file lists the declarations (each declaration consisting of a concept and an insert option) that will be added to classifications. The classifications referenced in this file can be new classifications listed in the classifications file, or pre-existing classifications.

    Note:

    Declarations must be removed through an API call; they cannot be removed through the loader.
  4. Create a control file specifying the Classifications, Classification Descriptions, and Classification Declarations files.

    See also:

    • /ets/hdr-ets-1.0.0-8.0.0/db/execute/readme/Terminology_File_Formats_Classifications.txt from hdr-1.0.0-8.0.0.zip for file formats.
  5. Load the classification. The procedure for loading a classification is the same as that for loading a Coding Scheme Version. Enter ETSClassifications in the Coding Scheme Name field on the Parameters page. Enter any text for the Coding Scheme Version Name (this field must contain text, but the actual text is ignored by the loader).

    See also:

  6. Import the classification. The procedure for importing a classification is the same that for importing a Coding Scheme Version.

    See also:

  7. Run the Healthcare HDR Maintenance Job to build the classification.

Building a Classification with the HDR Maintenance Job

To build a classification run the HDR Maintenance Job with CLASSIFICATIONS in run mode.

Note:

Run the HDR Maintenance job in the full mode whenever a significant amount of new classification data is created (including the first time classifications are created and populated in ETS.

Updating Published Coding Scheme Versions

After a coding scheme version is imported (published), you can update its properties (description, status, and default status) through the ETS API.

Running the HDR Maintenance Job

The HDR maintenance Job performs several database tasks. These tasks include:

  • Maintaining the ETS stage tables. For example, cleanup of incomplete or obsolete data in the staged tables.
  • Maintaining the ETS data in the active tables. For example, cleanup of failed imports in the active tables.
  • Building classifications by processing data for classifications in the pending or dirty state.
  • Building and synchronizing intermedia indexes.
  • Gathering statistics for the Cost-based Optimizer.
  • Ensuring that there is an entry in the language mapping table for each combination of coding scheme version and installed languages.

The maintenance job must be run in either FULL mode or CLASSIFICATIONS mode to move a classification from the 'dirty' or pending state to the active state. In general, it is a good idea to run the maintenance job periodically to keep ETS running optimally.

Note:

You should run the HDR Maintenance Job in the FULL mode:
  • Each time you apply a patch to ETS.
  • Whenever a significant amount of new classification data is created (including the first time classifications are created and populated in ETS).

Scheduling the Maintenance Job

Steps

Refer to Running HDR maintenance Job from Appendix D: Running HDR Terminology Jobs

Note:

In job arguments, select the desired Run Mode. The available choices are:
  • CLASSIFICATIONS: Builds classifications by matching definitions of classifications in the pending or dirty state. Classifications that have been created since the last time the maintenance job was run in CLASSIFICATIONS or FULL mode will be in the pending state. Classifications that have been modified since the last time the maintenance job was run in CLASSIFICATIONS or FULL mode will be in the dirty state. Successfully processed classifications obtain the active state.
  • CLEAN_ACTIVE: Performs maintenance on the ETS data in the active tables. Removes data from failed imports in the active tables, and rebuilds the intermedia indexes, if necessary. Ensures that there is an entry in the language mapping table for each combination of coding scheme version and installed languages.
  • CLEAN_STAGE: Performs maintenance on ETS stage tables. Removes incomplete or obsolete data from the staged tables, and rebuilds the intermedia indexes, if necessary.
  • DEFAULT: Performs maintenance on ETS stage, active, and language mapping tables, but does not build classifications. This mode is composed of CLEAN_ACTIVE and CLEAN_STAGE modes of the maintenance job.
  • FULL: Performs all operations, including maintenance of stage, active and language mapping tables, and building of classifications. This mode is composed of CLEAN_ACTIVE, CLEAN_STAGE, and CLASSIFICATION modes of the maintenance job.
  • TRUNCATE_STAGE: Removes all staged contents, regardless of their status. This mode is faster than CLEAN_STAGE for large datasets.

    Caution:

    The TRUNCATE_STAGE option can cause data that could have been used by the importer to be lost.

See also: