C H A P T E R 6 |
Troubleshooting the Database Migration |
This chapter provides troubleshooting information for the migration of the database from the Sun Java System Content Delivery Server, version 2004Q1PU1 to the Sun Java System Content Delivery Server, version 2005Q4.
This chapter covers the following topics:
Database migration and verification scripts provide several levels of logging and tracing to expedite the troubleshooting process. The following message levels are used:
All messages except those in the Debug category are printed to the console and logged into the $CDS_HOME/dist/cds/logs/migration.log file. All messages, including Debug messages are logged into the $CDS_HOME/dist/cds/logs/migration.trace.log file.
Typically, when a migration process fails, a few error messages and exception traces are printed to both the console and the log files. To understand the reason for the failure, and gather maximum information about the problem, it is important to scan the process output and log file from the beginning to end to identify the original error message.
Provide the screen output, the log file, and the log trace file when you contact support due to a failed migration attempt. Depending on the information in those files, the contents of your label directory or a copy of your database might be required to fully investigate and correct the problem.
The following sections describe Error and Warning messages in more detail.
Note - The %s symbol in the body of messages in the following section is a placeholder for data displayed in the actual message. |
Some error messages could indicate a problem with the configuration of the migration scripts or with the environment (for example, the Content Delivery Server or the Oracle server). This section provides a list of error messages with detailed descriptions and the most probable cause for the error. Error messages can chain as cascaded modules report exceptions on other modules that they were running. Pay attention to all error messages produced by a failed execution and to the messages associated with exceptions.
CONFIGURATION Exception!
Description: Occurs any time the configuration is invalid. This error can be caused by incorrect command line options or configuration files.
Action: Examine the entire migration process output to obtain more information about the error.
Exception while running %s
Exception from task %s
Thread %s failed
Delaying abnormal exit due to running threads
Exception while parsing arguments
Configuration exception
Verification problem
Initialization problem
DB failure, query : %s, params %s
Migration exception while executing SQL : %s
Exception in bootstrap()
Initialization failed
STEP %s FAILED
cmd process returned code %s import returned code %s
MR '%s' interrupts due to pending exceptions !
Exception while running shutdown task %s
Unexpected null %s (%s)
Thread stopped due to pending exception while processing edition %s
Execution failed on matrix %s, prefix %s, matrix %s exception while parsing arguments
Configuration error
Description: These messages are follow-up error message to another error.
Action: Examine the entire migration process output to identify the original error message.
VERIFICATION OF %s FAILED
VERIFICATION FAILED
Target query returned no rows
Source query returned no rows
Exceeded allowed number of failures (%s) for column %s
FAILED verification of %s
Counters don't match for prefix %s, table %s. Expected: %s, found: %s
Target query returned no row for %s,PK %s
Verification failed while comparing column %s, at primary key %s
Description: Verification failure.
Action: Examine the migration output to determine the actual causes for the verification failure and to identify the tables where the failure is found. Contact support to determine what action is required to correct the problem.
Error setting price argument %s
Failed to login to source database as catalog owner user
Failed to find vending user %s from %s
Server code must be specified
Operation can be either read OR write
Server type must be either catalog OR vending
operation can be 'read' or 'write' only, and must be specified
server type can be 'catalog' or 'vending' only, and must be specified
vending code must be specified for 'vending' server type
Error parsing configuration file %s
Expected root 'dbmigration' element
code attribute must be set and not empty for <server> tag
Found two or more <server> elements with code %s
Server code '%s' was not found in config file
Vending code '%s' not found in configuration for server '%s'
Catalog event source '%s' is not defined
Multiple <catalog> tags for server code '%s'
Multiple <label> tags for server code '%s'
Multiple <vending> tags with the same code '%s' found for server code '%s'
Source database config must be specified
Target database config must be specified
Can't read source database configuration file '%s'
Can't read target database configuration file '%s'
Can't find configuration file %s
Failed to parse recurring download properties. Check 'pricing.model.recurringDownload.*' properties in '%s' configuration file
Pricing option '%s' references non-existing content type %s
Pricing option '%s' has pricing model '%s' which is not supported by the preferred DRM '%s' of it's content type '%s'
Invalid value for the 'pk' attribute in the <catalog_events> tag
No deployment name specified for the <catalog> tag
Deployment '%s', directory '%s', can not be read or is not a directory
Cannot access deployment configuration file '%s'
Expected property file (%s) does not exist
DRMAgent property is not defined in %s
DRM '%s' is configured multiple times
Content type '%s' is configured multiple times
No DRM '%s' is found when configuring content type '%s'
Found configured content type '%s', that doesn't exist in the database
<content_type> tag must have a valid name attribute
Can not redefine system content type '%s'
Invalid free price code specified (%s) for content type '%s'
Can't request default pricing models and specify other pricing models
<drm> tag must have a valid name attribute
DRM name '%s' is reserved, and can not be defined by migration
<drm> tag must have a valid description attribute
<drm> tag must have a valid class attribute
Unknown delivery type capability %s, found in string %s
Invalid device id in the "by_id" attribute: %s
both by_id and by_name attributes specified for a device pattern
Neither by_id or by_name attributes specified for a device pattern
price argument error
invalid trial attempts: %s
Missing "name" or "value" attributes for tag <po_arg> for price option "%s"
Unknown price option argument '%s' specified in pricing option '%s'
Argument '%s' is not allowed for pricing model '%s', pricing option '%s'
Argument '%s' repeats multiple times for pricing option '%s'
Syntax error when trying to convert argument '%s' for pricing option '%s', value='%s'
Invalid regexp pattern specified in the by_name attribute : %s
<vending> tag must have 'code' attribute present and not empty
Non directory specified for search %s
<event_pk> tag should contain valid long value, got:%s
Valid pricing option name should be specified in <change_option_price> element
Valid new price should be specified for <change_option_price> element
<source_account> tag expected for <vending code="%s">
Can not reprice option '%s' - option is not defined
No location specified for the label
Failed to find deployment file %s
min attribute value is not a number
max attribute value is not a number
No value specified for coverage element
coverage value is not a double number : %s
coverage value must be in range of [0..100]
Unknown size character %s in <max_file_size> tag
bad file size specified in <max_file_size> : %s
bad integer specified in <max_index> : %s
Cannot find SOURCE vending schema %s
Cannot find SOURCE event schema %s
Failed to create directory %s
%s exists and not a directory
%s already populated. Verify you don't need existing data and remove this path
Label %s does not exist
Unknown pricing model %s specified
Label %s is not a directory
Path %s does not exist. Are you sure %s was created ?
%s is not a directory
Preparation for %s was not completed successfully
This migration step was already successfully completed. If you are sure you would like to retry it, remove the following file: %s
Directory is not writeable: %s
Cannot find target vending schema %s, in %
Cannot find SOURCE catalog schema %s
Cannot find SOURCE fuilfillment schema %s
Cannot find target catalog schema %s
No vending server %s found in the database
More than one vending server %s found in the database
DRM not configured: '%s'
Pricing option id=%s encountered, but not configured
FREE price selection for content type '%s' resutled into pricing model '%s', which is not support for this content type
Can't establish connection as a system user. Does DBConf have SystemUser and SystemPassword tags ?
Description: Error is caused by a misconfigured file. The migration configuration file or some of the deployment configuration files are invalid.
Action: Correct the configuration file and restart the migration.
Missing primary key %s in table %s
%s price is missing for content class %s
Can not proceed forward due to content pricing corruption in the database.
Pricing Models did not come in sets of 3. Data corrupt for class %s
Class %s, %s can only be assigned a FREE price. However such pricing model is not support by content type %s
Corruption detected in content pricing
Cannot retrieve valid price information for class %s, suspect corrupted pricing table
Description: Probable cause is data corruption.
Action: Contact support for help in identifying and fixing the corrupted database.
The catalog data has not yet been submitted to the database. You must complete catalog migration before doing this step
Description: The migration steps were executed in the wrong order.
Action: Execute migration steps in the order specified in this guide. See Chapter 5 for more information.
%s.selectFreePrice() returned unknown value (%s) for class %s
%s.selectFreePrice() returned pricing model (%s, translated to %s) for class %s which is not support for this content type
Price option sticker '%s' returned invalid value (%s)
Description: The supplied code violated the logical requirements as specified in Appendix A.
Action: Verify the code and make sure the implemented methods always return correct values.
DATA file doesn't exist for %s:%s
DUMP file doesn't exist for %s:%s(%s)
Expected to find constraints file %s
At least one record expected to be updated for %s
Description: Inconsistent label directory.
Action: Verify that no modifications were done to the label directory between different migration steps. If the problem persists, contact support.
migration is installed incorrectly
Unknown CDS DB version %s
4.0 PU1 Database version mismatch : found %s, expected %s
Description: Most likely caused by an incorrectly installed Content Delivery Server or migration patch.
Action: Verify that the installation is intact and that all migration patches were installed properly. If the problem persists, contact support.
export process failed on %s
Failed import for %s_%s
IMPORT process did not leave or left an empty log file '%s'
%s failed (sqlldr rc is %s)
%s sqlldr did not leave log file. This means id had not done anything
Description: An external Oracle process failed.
Action: Examine the trace file for messages left by the corresponding external process. Examine the external process log files available in the label directory. These errors often happen with the ORACLE_HOME variable is not set, or the PATH variable does not contain the path to the Oracle binary directory. Contact support if the root cause of the failure cannot be determined.
Other error messages usually indicate that a problem exists in the migration program code. You might need to contact customer support to resolve the problem. In most cases, the errors are due to an unexpected state of the database or to a customized server environment. Make sure you provide customer support with both the log and the trace files. It is also helpful if you provide the label directory contents along with the exported database on which the migration was performed.
Warning messages are indicators of irregularities in the database that were resolved during the migration process.
Interrupted while waiting for process output
Description: The thread that was watching for output generated by an external process received an unexpected interrupt. The external process might have unexpectedly died, or another migration thread requested all threads to be interrupted.
Process watcher caught exception
Description: The thread that was watching for output generated by an external process received an unexpected exception. The external process might have unexpectedly died or closed one of its output channels.
SQL warning: %s
Description: An SQL query submitted to the Oracle server returned a warning message.
Action: Examine the warning message to determine if any action is necessary.
Unknown tag %s found within tag %s
No tags are expected within tag '%s'
Description: The migration configuration file is structurally incorrect.
Action: Review and revise the migration configuration file.
Caught exception trying to retrieve environment
Description: An exception occurred during migration when trying to retrieve environment variables for debugging purposes.
Action: Examine the exception message and trace to determine if any action is necessary.
Terminating task due to pending exceptions
Terminating thread due to pending exceptions
Exiting due to external error conditions, destroying current process
Description: An executing thread detected an exception on another thread. This message is to inform you that the current task is not complete. The migration will fail due to the thread that received the original exception. An error message or other messages are also displayed.
Action: Follow the action for the error messages.
Missing entry with primary key %s in table %s
Description: The process that loads the BLOB entries into the database found a record that had BLOB in the source database but is missing in the target database.
Action: Contact support to verify if any action is necessary.
Class %s has editions that map into different %s price
Description: Multiple database entries denoting the specified type of price for the specified content item were found.
Action: Verify that after migration the price value is as expected for this class.
Class %s had both %s and %s prices set for %s price. %s price has prevailed
Description: The content item had multiple prices set. Migration was only capable of migrating one of the pricing models.
Action: Verify that the migrated price is the intended pricing model.
Class %s has % pricing model for % price, which can no longer be supported for content type %s.
For class %s, it is impossible to migrate %s price, as it's not supported by this content type. FREE price is assigned to the content.
For class %s, %s price, there are trials requested. They WILL NOT be applied, as the price is FREE.
Class %s has trials, and content type %s doesn't support trials. No trials will be available for this content
Description: Pricing collision has occurred as to which pricing model can be selected for which content type.
Action: Examine your configuration file to check if your pricing configuration is correct. If it is, verify that this content has the correct price after migration.
Null capability id 16 for one of the derived editions of edition %s.
Poorly formatted capability value for id 16, for one of the derived editions of edition %s : %s.
Edition %s has different capability value for capability %s. Old=%s, New=%s.
Description: The database contained an incorrect record or no record at all for one of the editions's Supported Devices capability. This indicates that the database is probably corrupted.
Action: Contact support to verify what, if any, action is necessary.
Inconsistency in LCM table, found a mix of temporary/final records
Description: The logic used to generate the new LARGE_CAPABILITY_MAP table has failed to produce the required set of records.
Action: Contact support to verify what, if any, action is necessary.
Model to instance mapping count doesn't match expectations. Expected : %s, found %s
Description: The resubmission and edition reverification process might have been compromised.
Action: Contact support to verify if any action is necessary.
Specified number of threads is not an integer
Description: The value of the <threads> tag in the migration configuration file is invalid.
Action: Review the migration configuration file and correct the error.
No verification coverage provided for %s, count=%s, defaulting to %s.
Description: The migration configuration file specified incomplete verification coverage.
Action: Review the migration configuration file and correct the problem.
Pricing model '%s' specified multiple times for DRM '%s'
Pricing model '%s' specified multiple times for content type '%s'
Pricing model '%s' is removed from content type '%s', because its DRM '%s' doesn't support it
Description: The migration configuration file has an inconsistency in its pricing configuration.
Action: Review the migration configuration file and correct the problem.
Source fulflilment_request table reports %s entries per this vending account. %s entries, however, were copied from the source database. Absolute delta : %s
%s entries must be deleted from SUBSCRIBER_GIFT table. Actually deleted %s entries from SUSBCRIBER_GIFT table
%s entries must be deleted from SUBSCRIBER_PURCHASE table. Actually deleted %s entries from SUBSCRIBER_PURCHASE table.
Description: During migration, inconsistencies were detected in the FULFILLMENT_REQUEST table. This is usually found when records in this table do not match all the records in the SUBSCRIBER_PURCHASE and SUBSCRIBER_GIFT tables.
Action: Contact support to verify if any action is necessary.
Category item %s has no instrumented editions
Description: When this edition was resubmitted, no instrumented editions were produced by the submission workflow. This error most likely occurred because this content stopped being eligible for any existing devices.
Action: Examine the corresponding content item and all of it's edition to determine why it's resubmission has failed.
Category item %s will be deleted as there are other category items for class %s
Category item %s is marked 'Unstocked' as there are no other category items for class %s
Description: During category item processing for the Vending Manager database, the class with these category items had editions that could no longer be stocked. Migration removes all such editions, until only one is left. This one is then marked as Unstocked, as at least one item must represent the content class in the Vending Database.
Target query returned no rows for '%s', PK %s
Description: Verification expects that all rows that existed in the source database are present in the target database, however the verifying matrix indicates that if they are not present, the condition is not fatal. This error usually indicates that some rows in this table were omitted during migration due to data corruption.
Action: Contact support to verify if any action is necessary
Deployment implements different migration version that does core migration code. Developer portal version: %s, core version : %s
Description: The migration file might be misconfigured, probably caused by an incorrectly applied migration patch. Check the instruction for all applied patches to verify they were properly followed.
Action: Contact support if the problem cannot be resolved by reapplying the migration patches.
Edition %s doesn't have content data
Description: Database corruption. During migration, an edition was detected that had no binaries associated with it.
Action: Contact support to verify if any action is necessary
Edition %s failed to verify
Description: When the specified edition was passed into the verification workflow, the verification rejected it. The database still stored the original edition, but no derived editions were created.
Action: Examine the corresponding content item and all of its edition to determine why its resubmission failed.
Edition %s doesn't have any devices matched to it.
Description: After verification, the edition was not determined to be capable with any of the currently known devices. The edition probably became incapable for any device if device profiles changed or if the edition changed during the reinstrumentation process.
Action: Examine the edition in question and check why did it stop matching any devices.
This section provides information to help you recover from problems you might encounter when migrating the Content Delivery Server database.
If any of the read steps fail during the migration, you need to perform the following actions:
1. Clear the condition that caused the error.
2. Remove the label contents associated with the read step.
The location for the label contents for the catalog sever is label_dir/catalog. The location for the vending server is label_dir/vending/target_vending_name.
If any of the write steps fail during the migration, the database could be left in a partially populated state. You need to reinitialize it before attempting to rerun the write step. To reinitialize the database, use the cdsi db commands described in Section 1.2, Preliminary Migration Steps in Chapter 1 with the modifications given here.
If the catalog write step fails, a reinitialization of the entire database must be performed. To reinitialize the entire database, run the following commands:
# cdsi db users
# cdsi db schemas_noc
Before running the cdsi db users command, make sure that there are no open connections made by any software to any of the database prefixes. You can specify a custom database configuration file as a third parameter to the cdsi command.
If a catalog write step fails, you only need to reinitialize the affected catalog database. Leave other vending and catalog databases intact. To reinitialize the catalog database, run the following command:
# cdsi db users -cs
If a vending write step fails, you only need to reinitialize the affected vending database. Leave other vending and catalog databases intact. To reinitialize the vending database, run the following command:
# cdsi db schemas_noc -vs name
name is the name of the Vending Manager server that you need to reinitialize (the one you are using for the Content Delivery Server version 2004Q1PU1).
Failure to clean the corresponding database after a failed write step can result in inconsistent behavior when the write step is repeated. The database migration might not detect such behavior. This behavior is evident in the exceptionally long processing time of the migration. The ultimate result is that the migration never completes.
To detect the problem, attach a tracer to the import process to see if it is executing properly. Proper execution consists of a series of read and write system calls. If no system calls are issued within one minute, there is a problem with the import process and the migration must be aborted.
A migration process can end in one of the following ways:
Be aware that when the process is interrupted, not all processes spawned by the migration process are terminated. The system administrator can identify such processes and terminate them manually. These processes can be Oracle import or export programs or Oracle SQL loader programs. Even when terminated manually, the processes could leave open connections to the Oracle database server. Having open connections prevents the corresponding database from being reset using the cdsi db commands. Refer to Oracle documentation for information on how to close open sessions.
Copyright © 2005, Sun Microsystems, Inc. All Rights Reserved.