| Oracle® Enterprise Data Quality Release Notes Release 11g R1 (11.1.1.7) E49523-02 |
|
| PDF · Mobi · ePub |
| Oracle® Enterprise Data Quality Release Notes Release 11g R1 (11.1.1.7) E49523-02 |
|
| PDF · Mobi · ePub |
Release Notes
Release 11g R1 (11.1.1.7)
E49523-02
May 2014
This document contains the release information for Oracle Enterprise Data Quality (EDQ) Release 11g R1 (11.1.1.7) and includes the following:
Oracle recommends you review its contents before installing or working with the product.
The following sections describe the new features introduced in each release, as well as, improvements:
This section addresses Release 11g R1 (11.1.1.7.4).
If you use EDQ to provide stateless batch matching jobs and there is no need to keep any results of processing in the EDQ repository database, you must clean up all results related to a job once it completes. This is very important if each batch job runs with its own run label; meaning the results of previous jobs will not be overwritten when a new job runs. Currently, EDQ match processors write a lot of input data and cluster tables to the database and this is only cleared when a purge rule runs. Since the purge rules do not allow a '0 hours' configuration, this can only happen an hour after job completion, leading to undesirable database space usage.
Purge rules in the Server Console have been improved to include the option to purge immediately, making it possible to clear out results for jobs on completion where required.
Requested enhancement for the ability to write to a Reference Data set using a Writer in a process, in the same way as Staged Data. If the Reference Data set has a uniqueness constraint that the data in a process would break if written, the process will fail.
You can now configure a Writer processor in Director to write to a Reference Data set.
In some cases, it is desirable to publish processors that require the presence of a named reference data set, even when the option that uses the reference data set is not exposed in the published processor. This enables the publication of a series of processors that all use a common reference data set (for example, installed at the System level) for shared 'internal' functionality, such as punctuation normalization without passing on the ability for an end user to change to use a different set, and in a way that enables modification of the Reference Data set. This promotes improved reuse since a change to the reference data set will automatically be picked up by all published processors using it.
You can now choose whether or not to package Reference Data sets that are used in a created processor when publishing it to the Tool Palette.
A new requirement to support the graceful shutdown for EDQ when this procedure ('Shutdown when work completes') is invoked in WebLogic Server. This means that access to the EDQ server is immediately disallowed, but any currently running real-time jobs are shutdown gracefully, and any currently running batch jobs are completed before shutting the server down.
EDQ now respects the 'Shutdown when work completes' command in WebLogic Server. When an EDQ Server is shutdown in this way in WebLogic Server, it immediately transfers to a SUSPENDED state, shuts down all real-time jobs and UI sessions, and stops listening for new batch jobs (where a JMS queue is used to submit batch jobs to the server). The server does not shutdown until any batch jobs and system tasks that are running when the graceful shutdown procedure is invoked have completed.
This section addresses Release 11g R1 (11.1.1.7.3).
The key features of the new Reference Published processors are:
They are maintained centrally so any changes made to the original are automatically updated in any instances of the processor elsewhere in the system.
Users can view the configuration of these processors though cannot change them unless they have the required security permission to do so.
Locked Reference Published processors cannot be unlocked by any end user regardless of their permission level. Only the original can be edited.
The EDQ integration with Apache Subversion now supports release 1.7 in addition to the 1.6 release.
The EDQ integration with Subversion now provides the ability to compare your workspace with base and head revisions in tree to easily identify the differences.
You can now use the Oracle Universal Installer, an automated installation process for all supported operating systems, to install EDQ.
All EDQ graphical user interfaces (GUI) are now localized into Chinese, Japanese, Korean and Brazilian Portuguese. These languages are in addition to the previously supported languages, U.S. English, French, Italian, German, and Spanish.
All documentation, including the Online Help, is U.S. English only.
The Job Editor now supports the definition of 'blocking' triggers that will wait for the triggered action to complete before continuing.
The Server Administration Utility Application replaces Functional Pack selection, the location of the configuration folder, and database connection properties.
The Administration, Web Service Tester, and Dashboard user applications have been rewritten using Oracle Application Development Framework for improved security. You can access the EDQ Launchpad with the following URL:
http://server name:port number/edq
The default launchpad URL was changed from /dndirector. Existing connections are forwarded meaning that when your browse to http://servername/9002:dndirector you are redirected to the new launchpad URL. You may want to update any existing browser bookmarks.
Integration with the Oracle Fusion Middleware Audit Framework now allows you to audit modifications to user and security settings, and changes made to configuration using Director.
The In List comparison provides a way of making the application of a match rule conditional on one or both identifier values in a comparison being matched with a single value or list of values.
Where jobs are run using Run Labels, the results for a given set of staged data (either a snapshot or written staged data), a given process, a given job, or all results for a given project can now be purged on an automated basis.
The flexibility of workflows in Case Management Administration has been extended to allow the availability of transitions to users to be made dependent on the attributes of the case or alert, and new rules based on previous transitions that have been made on the Case or Alert by the same user. This enables the enforcement of '4-eye' workflow requirements.
Reports in Case Management now allow you to aggregate values into groups for reporting purposes. This makes it possible to report on more attributes, such as Dates.
The product has been fully integrated to use Oracle Platform Security Services for LDAP integrations and for Web Service security.
Data Interfaces now allow you to use array types to improve process reuse where arrays are used.
It is now possible to launch the Web Service Tester from within the Director GUI using the context menu of a connected server.
You can now externalize process task options and data interface mappings configured in a job, and so override the settings at runtime:
When a real-time matching record produced more than one cluster value, the cluster groups were processed on the same single thread as the message executor. Now a thread pool processes match cluster groups from any of the executors, which allows for faster processing of complicated match clusters (load permitting.)
You can now select multiple match groups then copy and paste these groups between match processors. Copying the match rule groups ensures rules that all rules are copied with them (similar to the rule copy and paste feature), and it copies the names of the comparisons.
Additionally, you can copy and paste multiple comparisons between match processors.
All five data type converters now support multiple attributes. Failure of conversion of one value (or for nulls if configured) results in a failure for that complete record.
You can now filter all of the rules in a match processor across all the match groups by its name. This filtering change allows case-insensitive filtering of rules based on the rule name.
You can now run a job in Director using a Run Profile (not a Run Label) and inspect all of the results that are produced.
Previously, the pattern generation and parse tokenization maps were limited to the latin range. This has been changed to extend them to the full Unicode range.
The Job Manager has been redesigned to implement numerous enhancements including:
Ability to add notes to the Job Canvas
Improved navigation of multiple phases
Job Canvas can be used in design mode
Job opening and progress feedback has been optimized
Job Tool Palette now supports multiple selections
The ability to filter Cases and Alerts by allowable transitions has been added.
The ability to use a Negate option when defining filters in Case Management was added.
Use of the Lucene indexing has been furthered to produce greatly improved performance in Case Management when performing reports and filters on large data volumes.
After running a report and drilling down on a given value, clicking the Back to Report button no longer reruns. This is a result of the indexing performance improvements.
Previously, the number of results for a filter exceeded 100 the total number of results was not reported. The was changed so that the title bar now reports the total number of results, as well as how many are displayed.
You can now automatically configure and start a job that uses a Run Profile with the Autorun functionality.
You can now search the Case Management history using filters to find Cases and Alerts that have ever been modified or commented on by a given user.
To avoid a Matching job running out of memory while trying to load review groups and match groups for output processing, reviewing and case generation limits have been added. By default, this limit is not enabled so that you can set the limit that works in your environment. When you enable the review group limit functionality, relationships are not be generated for any groups when the limit is reached.
Previously, the selection within the Canvas Overview pane could only be moved by dragging the currently selected area, which made it difficult to use with large processes. This has been improved so that clicking anywhere within the Canvas Overview pane shifts the Canvas to that location.
The Maximize Canvas button functionality has been improved so that the Tool Palette is no longer closed when you maximize the Canvas.
A duplicate option is now available on the Case Management Admin Case Sources list, which creates a new source based on the old one. It also copies any relevant display data though disposes of any historic display or metadata information for the source because it is now a new source.
The Attachments table has been added to the list of tables that can be queried in the System Information data store. For example, you could use this table selection to check which Cases have attachments or to report on which users are uploading attachments most frequently.
The following sections describe the bugs resolved in each release:
This section addresses Release 11g R1 (11.1.1.7.4).
When EDQ is integrated with Subversion, the functionality to compare a modified process in Director against a previous version never shows any changes.
The process comparison has been fixed. Changes in the process are now reported correctly when comparing against a different version.For example, the base or latest committed version.
Reports in Case Management that aggregated date values with an offset were not being calculated correctly, resulting in some reports that would 'hang' (never complete).
The offsets when reporting on aggregated Date values were fixed to work correctly.
When an attribute included in a Report view in Case Management contains a Null value, the cases or alerts are counted correctly in the report, but drilling down on the number does not take the user to the cases or alerts in question.
The drill down logic has been fixed to find cases or alerts with Null values when drilling down on a cell in a report.
In EDQ 9.0, a report that specified a Case Source and reported on workflow states would only show states valid for the workflow used by the case source. After upgrade, the report shows states in other workflows.
Reporting has been fixed to ensure that states for other workflows do not appear if a case source (and so a workflow) is selected.
EDQ is currently deployed in 'STAGE' mode which means that patches applied using OPatch do not take effect immediately.
EDQ is now deployed in 'NOSTAGE' mode.
An imported filter specifying the value of an Extended Attribute causes an error when searching for cases.
Filtering on Extended Attributes has been fixed.
The default deployment of EDQ on WebLogic Server currently sets the prepared statement cache for all its Data Sources (including the connections to the EDQ repository databases) to a default value of 10. This is undesirable for performance reasons.
The default deployment of EDQ on WebLogic Server has been altered to disable the statement cache by default (by setting the value to 0).
Where multiple user sessions are open in Dashboard Administration it is possible for a user to encounter an illegal argument exception when making a change. The exception encountered is:
java.lang.IllegalArgumentException: Comparison method violates its general contract!'
The sorting of tree nodes in Dashboard Administration has been fixed to ensure that the exception does not occur.
From Java version 7, update 51, Java WebStart applications without JAR Manifest file attributes cannot be launched using the default security settings.
Manifest attributes have been added to allow the applications to launch with the default Java security setting.
The Address Verification connector processor that ships with EDQ includes a check for a minimum version of the EDQ Address Verification API (Loqate) that failed to parse the version number correctly. Although no released versions of EDQ Address Verification include a version number that will wrongly trigger the failure of the version check, the check has been corrected to avoid any compatibility issues with future releases of EDQ Address Verification.
The version number check of the Address Verification processor has been modified to parse the Loqate version number correctly.
After upgrading to Java version 7 update 45 or later, processes with Parse processors with some types of Reclassification rules can trigger the following severe error:
runtime process terminated uncleanly: Comparison method violates its general contract!
The upgraded Java version performs tighter checks that comparators that are equal return an equal value. The Parse processor has been fixed such that this is always the case.
When changing an externalized option in a member processor of a compound processor to 'New', the following application error occurs:
IllegalArgumentException: argument type mismatch
The check performed when an option is changed has been fixed to ensure this does not occur.
The default deployment of EDQ sets up the Data Sources in WebLogic Server that connect to the EDQ Repository Databases to use global transactions (global-transactions-protocol is set to 1). This is undesirable for performance reasons.
The default deployment of EDQ now sets up Data Sources with global transactions turned off.
Searches in Case Management looking for recently modified cases and alerts within a date/time range are not returning the correct results, due to the search indexes not being updated correctly
The search index commit process has been modified to ensure the correct results are returned.
The EDQ WebLogic Configuration Wizard, used to configure a WebLogic domain for EDQ, shows invalid options such as 'non-Oracle' and 'Oracle XA' when setting up database connections.
The configuration templates for EDQ have been modified to include the correct database.xml file to ensure that only valid and correctly named database options appear.
The Job Factory and Results Set Export Job Factory mbeans do not have any descriptions, meaning they are effectively undocumented.
Descriptions have been added to the mbeans so that their use is clear in WebLogic Server or in other consoles connecting to the JMX interface.
Match processors with written output (such as Match Groups or Merged Output) disabled run correctly but results are not visible in the Director Results Browser (unless or until the output is enabled).
The results catalog has been fixed to ensure that match results are visible in Director.
In certain circumstances, an error can occur during the case generation phase caused by an environment issue such as lack of file space or network connectivity though not caught, leading to an incomplete publication of cases and alerts.
The error handling during case generation has been improved to ensure that such errors are caught and to prevent partial case and alert publication.
Where an aggressive results purge policy is configured, such as purging all results over an hour old, a busy server may generate huge numbers of warning messages where the purger is attempting to delete tables that are locked by running jobs.
Logging was changed not to log warning messages where the purger attempts to delete tables for running jobs.
SEVERE Error in LogAs the purger runs as a standalone task, it is possible for it to attempt to delete tables owned by configuration objects (such as, staged data sets) that have since been deleted, with the tables deleted during that process. Where this occurs, a SEVERE error is logged, but the error is redundant.
The purger has been changed to check that the tables it is tasked to delete still exist.
Purge rules are not deleting internal results tables (such as, those used by Match processors) for jobs that are not configured to write any staged data at all. This can lead to unwanted growth of the results database when running jobs with many run labels.
The purge logic has been fixed to ensure that it removes tables for in-scope jobs even if no staged data sets were written.
If an error occurs processing an AQ destination, a JMS exception occurs with important information displayed in linked exceptions. This is not being logged by the JMS trigger framework.
The JMS trigger framework logging output has been modified to include such exceptions.
EDQ on WebLogic Server can be configured to use the platform (JRF) web service stack. Where this is the case, the WSDL paths to the web services change to use a wss prefix rather than a webservices prefix. This change is not reflected on the Web Services page on the EDQ Launchpad.
The Web Services web page has been changed to determine the URLs correctly depending on the underlying stack.
A Null Pointer Exception can occur in the Job Manager when drag and drop is used to reorder phases, followed by adding a task.
The ability to reorder phases using drag and drop (rather than the buttons) has been removed in order to ensure this does not occur and also to avoid accidental phase reordering.
If the number of records in a cluster exactly matches an absolute cluster size limit, comparisons are not performed though the cluster may be considered as 'processed' not skipped.
The logic for processing clusters was fixed to ensure it can never skip a group 1 record early.
premigrate Check During UpgradeThe 11.1.1.7.3 premigrate command in the migration.jar file, run in order to check that the Case Management data can be upgraded without issues, returns an Illegal Argument Exception.
The premigrate command has been fixed to run without error.
The Provider and Consumer links under 'Available Operations' on the Web Services web page do not work on WebLogic Server.
The links have been fixed.
Clicking the Get Service button in the Web Service Tester with no web service selected (using Select in the drop-down list) causes a Null Pointer Exception.
The Get Service button is no longer enabled unless a web service is selected.
In EDQ 11.1.1.7.3, reports of cases and alerts in Case Management include numbers of cases and alerts that the logged in user does not have permission to view. The user cannot drill down and view the cases or alerts and they should also not appear in reports or lists.
Permission restrictions have been built in to the reports.
The display case of data types ('String'/'Number'/'Date') in snapshot XML files and other places in EDQ is driven from the Java locale. In a Turkish locale, the case conversion leads to a problem defining configuration and capturing data.
The Java locale is no longer used for comparing table and file names.
After jobs are invoked from Director, it is possible for users to encounter a Transporter Exception when logging into Server Console due to errors in the promulgation of progress messages.
The progress reporter was fixed to ensure the error does not occur.
In some circumstances, switching back to reviewing a group from reviewing its merged output causes a Null Pointer Exception in the client.
The Match Review UI was fixed to ensure there can be discrepancies between the expected and actual relationship table models.
An additional manual step (the addition of a file) is needed in order to install EDQ on an AIX/WebLogic server using Java 7 such that the EDQ Launchpad displays correctly.
The additional file is now installed by default meaning no additional steps are required.
An Illegal State Exception occurs when entering a space, a letter, then a backspace in the Job Manager Tool Palette search box.
The Job Manager Tool Palette search facility has been fixed.
Attempting to save a filter with either Source Id or the Derived State attribute set to 'None' causes a Null Pointer Exception.
Case Management has been fixed to work correctly in these cases.
The Refresh button in Case Management was removed in 11.1.1.7.3 because it did not work correctly in all instances.
The Refresh button has been reinstated and works correctly.
Starting an EDQ managed server without the node manager running does not work unless suitable memory and configuration settings are added to the environment configuration. In a default installation, the start Managed Server script should work with no need for additional steps.
The configuration template for EDQ was modified to include default settings in the setDomainEnv script that mean the start Managed Server script can be used. The Oracle Enterprise Data Quality Installation Guide Release 11g R1 (11.1.1.7) has been updated to include additional information about starting an EDQ managed server.
After upgrading an older version of EDQ to version 11, a match processor configured to publish to Case Management that is run in a process with intelligent execution enabled may trigger a Null Pointer Exception when run.
Error handling has been improved so that if a user attempts to run such a job with Intelligent Execution enabled, an error message is generated to suggest disabling it. The documentation for EDQ 11g already states that jobs need to be run with intelligent execution disabled at least once in order to generate new results tables.
This section addresses Release 11g R1 (11.1.1.7.3).
When running a job in Director and any of the required data sources do not exist, the respective snapshots error as expected. However, the resources that were being used by the job are left in a locked state forcing a restart of your application server to reset the lock. The has been corrected so that when a job fails no resources are left locked.
flags.xml Is Missing the EDQ Server Cannot StartWhen the flags.xml file is missing from the oedq.home/casemanagement directory, a NullPointerException error occurred during startup and the EDQ Server does not complete initialization. If this is the case, a message is now displayed to alert you that the file is missing and the error is written to the server log file.
When there are duplicate entries by lookup key used for value replacements, the replacement used could vary between process runs. The has been corrected with changes implemented in the Replace and Replace All widget logic.
The Case Management user interface did not correctly refresh the Source Attributes section. The only occurred when switching between two data sources where an attribute existed in multiple data items (for example, Id.) The new source attribute selection model has removed the chance for this problem to occur.
If an Oracle data source is created with a schema specified and an export using the primary key replace option is attempted on a table in the schema, the export fails even if the user has been granted DELETE privilege on the table. The has been corrected by adding privileges to the export.
The following sections describe known issues and any workarounds that exist in the Release 11g R1 (11.1.1.7).
As EDQ 11 has fully refactored results tables, all process tasks in jobs that have previous results (upgraded from an old version of EDQ) must be run with Intelligent Execution turned off to generate new results tables in the correct format. Process tasks that run with Intelligent Execution turned on will generate an error. This does not affect any processes that do not have results generated from previous versions.
A link is still displayed to the publishing summary of a rule even if the user does not have permission to view that summary. An error is displayed when the user clicks on the link.
If you are using the Dashboard and you attempt to return to the previous page using your browser Back button, the Dashboard web page may become unresponsive. To correct, you close and restart your browser then restart the EDQ Dashboard.
You may encounter an error attempting to insert multi-byte characters into the Reference Data editor in Director if you are running Java version 6 update 45. This bug has been resolved in later releases of Java.
Korean characters cannot be input or pasted into the Output Attribute Prefix text box on the Options tab of the Address Verification processor.
An error may occur in certain circumstances when you attempt to publish an option from a member processor of a 'compound' processor. This forces Director to close so you must restart it.
The User Sessions dialog gradually enlarges horizontally each time you select a record and may double in size if you select in excess of 100 records.
If you are running EDQ on a version of Java prior to 1.7 update 40, the Input Method Editors (IME) in Window was not always being disabled on the password field when logging into EDQ applications leading to login errors. This can be resolved by updating to Java 1.7 update 40.
Extended attributes in Case management are displaying a percent (%) character at the beginning of attribute names and not rendering the attribute correctly. You can correct this by copying the flags.xml and flags.properties files from your ...\oedq_home\casemanagement directory to your ...\oedq_local_home\casemanagement directory or by importing a case source.
If you are running an AIX and WebLogic environment using Java 7 (for example, an IBM JDK using Java 7 rather than Java 6), an additional post-installation step is needed after installing EDQ so that the product works correctly. For the specific post-installation step, see Oracle Enterprise Data Quality Installation Guide Release 11g R1 (11.1.1.7).
Earlier versions of EDQ can be upgraded to EDQ Release 11g R1 (11.1.1.7) using the instructions provided in the Oracle Enterprise Data Quality Installation Guide Release 11g R1 (11.1.1.7).
After upgrading to EDQ 11, make sure that all jobs that have existing results (in the upgraded EDQ results database) are run with Intelligent Execution turned off on all process tasks, in order to generate new results tables.
Process tasks that run with Intelligent Execution turned on will generate an error if results tables already exist.
Due to the fix for "BugDB # 14323577: Inconsistent Results are Produced with Replacements", EDQ Release 11g R1 (11.1.1.7) is less tolerant of Reference Data rows (or Staged or External Data rows used in a Lookup) that are duplicated by lookup key (on a case insensitive basis), and an error occurs when such rows are used in a Replace processor. The recommended solution to this error is to fix your data by deduplicating it. For example, you can run your data through a process in EDQ using the Group and Merge processor to remove any duplicates, and then repackage the Reference Data.
However, if you want to revert to the previous behavior in order to workaround the issue, it is possible to do so by adding the following line to director.properties:
replace.transform.allow.duplicate.key = true
For more information, see the following documents in the Oracle Enterprise Data Quality documentation set:
Oracle Enterprise Data Quality Installation Guide Release 11g R1 (11.1.1.7)
Oracle Enterprise Data Quality Architecture Guide Release 11g R1 (11.1.1.7)
See the latest version of this and all documents in the Oracle Enterprise Data Quality Documentation website at
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc.
Oracle customers have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired.
Oracle Enterprise Data Quality Release Notes, Release 11g R1 (11.1.1.7)
E49523-02
Copyright © 2013, 2014, Oracle and/or its affiliates. All rights reserved.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:
U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA 94065.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.
This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.
 Copyright © 2013, 2014, Oracle and/or its affiliates. All rights reserved. Legal Notices |
|