| Migration Guide |    
|
| Migration Guide |
|
| C H A P T E R 5 |
|
Migrating the Database |
This chapter describes how to migrate the database from the Sun Java System Content Delivery Server, version 2004Q1PU1, Product Update 1 to the Sun Java System Content Delivery Server, version 2005Q4.
This chapter covers the following topics:
The database migration process consists of a series of steps where each step is considered a separate migration procedure. Steps can be classified as either a catalog read or write step or a vending read or write step. Catalog steps must be completed before beginning the vending steps. A read step must always precede a corresponding write step. The following list shows the sequence you must follow for the set of migration steps for a single server deployment:
When a catalog write step is run, the Content Delivery Server server code is run by the migration process to pass all content through the submission and verification workflows. This process can require a large amount of memory. To prevent OutOfMemory errors, increase the maximum heap size allowed for the Java virtual machine by setting the EXTRA_JAVA_ARGS environment variable before running the catalog write step. For example,
$ EXTRA_JAVA_ARGS="-Xmx1500M" cdsi db migrate db -s code -c -w
Do not change the minimum heap size. Setting the minimum heap size to a larger value results in the Java virtual machine requesting more memory as soon as it is started. When the Java virtual machine needs to fork Oracle processes, the operating system fails the execution requests. This is because every time a JVM machine needs to start a process, it must create a copy of itself first and there might be not enough available operating system memory to do so.
Database migration operates with labels for storing and loading transformed data. A label is a directory where migration data is stored between performing a read and a write step. The same label must be used during the migration of the catalog server and all its associated vending servers because catalog data is referenced during the vending step. The directory associated with the label has the following structure:
If your migration requires custom implementations as defined by the configuration file, you need to extend the Java class path so that it contains the path to your implementations. You can specify the path by setting the EXTRA_JAVA_CP environment variable before running the migration steps. For example:
$ EXTRA_JAVA_CP="/tmp/extra.jar" cdsi db migrate db -s code -c -r
The following conditions are required to run the database migration:
version of at least 1.4.2 must be installed.
For the rest of this guide, the term "source database" refers to the Sun Java System Content Delivery Server, version 2004Q1, Product Update 1 database. The term "target database" refers to the Sun Java System Content Delivery Server, version 2005Q4 database.
You must have the variables, CDS_HOME and ORACLE_HOME defined, and you must have the following directories in your PATH:
The Oracle home directory can refer to either the Oracle home directory created by the server installation or by the client installation. In each case, the Oracle database utilities must be fully installed.
Also, make sure the paths in your env_sh file are properly configured.
Only one configuration file is needed for database migration, DBMigration.xml located in the default directory, $CDS_HOME/cfg/migration/. This configuration file can be used for all migration steps. This section describes the rules for creating a configuration file and defines the tags used in the DBMigration.xml file.
The database migration configuration file you create can be named something other than DBMigration.xml, but it must be located in the $CDS_HOME/cfg/migration directory and must be specified when executing the cdsi migrate command.
Follow these general rules when creating a configuration file:
This section defines the <dbmigration> tag and tags that can be used within it. The <dbmigration> tag is required. It is a top-level tag and encloses all other migration configuration tags and must be the root tag in the XML file. The following tags are valid within <dbmigration>:
Specifies the number of parallel threads that are launched to run the migration. Takes an integer value of zero or higher. If zero is specified, the number of threads matches the number of CPUs found on the system. The default value is 0.
Required if the server's code is specified in the command line.
Encloses the server configuration entries. You must have one <server> tag for each Catalog Manager and all its associated Vending Managers. <server> takes the attribute, code, which distinguishes one server entry from another. The configuration file cannot contain more than one <server> tag with the same code value. The following tags are valid within <server>:
Encloses the catalog specific portion of the migration configuration. The following tags are valid within <catalog>:
Indicates that the catalog events are stored into this Catalog Manager's event table. Use this tag if the Catalog Manager is not deployed with a vending manager. The value indicates the vending code, which is used to extract existing catalog events. The optional pk attribute is used to restrict the events that are migrated. Refer to the <event_pk> tag description in this section for more information.
Specifies an additional classpath that must be added to the default classpath when the server is booted. This tag is usually used when the server submission code depends on additional libraries. If <classpath> is used, it must contain at least one <element> tag.
<element> represents a single classpath element. The following macros are recognized in the tag's value:
$cds.dist - Expands to the full path of the Content Delivery Server distribution directory.
$cds.home - Expands to the full path of the Content Delivery Server home directory.
Specifies the DRM and pricing model association for a content type. All content types must be specified in the source database before migration begins. Doing so enables the creation of all required MIME types and associated MIME types to the content types by using the Content Delivery Server user interface rather than configuring the migration. All of the new content types that are used with the new version must be present and fully configured in the source database. Use this tag only to specify additional configurations related to the new product version.
<content_type> contains the attributes name and drm. The name attribute is required and specifies the name of the content type. The drm attribute is optional and specifies the name of the DRM technology that is used when submitting content with this content type. <pricing_model> tags that are found within <content_type> tags restrict the pricing models associated with this content type. If no pricing models are specified, all pricing models associated with the specified DRM technology are used. If the <use_default_models> tag is specified, a standard pricing model association applies. See Chapter 4 for more information. The system content types, bundle and text, cannot be redefined using this tag. The following tags are valid within <content_type>:
Specifies how a free price is treated for this content type. In the new version of the Content Delivery Server, free price for a content can be specified in two ways: by assigning a Free pricing model or by assigning any other pricing model and having a price value of 0.0. The text value for this tag can be either 1, 2, 3, or 4. A value of 1 means that content with price of zero is assigned the Free pricing model. A value of 2 means that the content is assigned either a First Download or Recurring Downloads pricing model with a price value of 0. A value of 3 means that the Subscription pricing model with a price value of 0 is assigned. A value of 4 means that the Usage pricing model with a price value of 0 is assigned. If no value is specified, the Free pricing model is assigned. Note that this code is considered only when no pricing option sticker implementation is used.
Specifies allowed pricing models for a DRM type or for a content type. The name of the pricing model is specified as a text value. Only specified pricing models are allowed for the corresponding DRM or content type. The following names are the pricing model names that can be specified within this tag: First Download, Subscription, Usage, Limited Time, Free, Recurring Download, Period. This tag is valid within <drm> and <content_type>.
If used within <content_type>, it forces default pricing models to be associated with the specified content type. Free, Recurring Downloads, and First Download pricing models are used for all content types, and Subscription, Usage, Trial Usage, and Period pricing models are associated if the content type is midlet.
Specifies the deployment name that is used by the migration process during content resubmission.
Encloses device-related configurations. The following tags are valid within <devices>:
Specifies the default DeliveryType capability value. Valid values are OTA, NSM, and EMS. If multiple delivery types are supported, the values must be separated by commas (,).
Identifies a device list. <device_pattern> contains the attributes by_id and by_name. The list is controlled by setting one of the attributes. Only one of the attributes must be set. The by_id attribute accepts a list of primary keys (separated by commas) into the HANDSET table. The by_name attribute accepts a regular expression. A device matches a <device_pattern> tag if it's ID matches one of the IDs specified in the by_id tag or if it's name matches the regular expression specified in the by_name attribute.
The valid tag within <device_pattern> is <delivery_type>. It is not required. Use it to specify the DeliveryType capability to set for devices in a specific device pattern. Valid values are OTA, NSM, and EMS. If multiple delivery types are supported, the values must be separated by commas.
Indicates that an additional row must be added to the DRM table before migration begins. <drm> contains the required attributes name, description, and class, which provide the details about this entry. By default, the Content Delivery Server defines four DRM technologies, CDS DRM Agent for Java, CDS OMA DRM 1.0 Forward Lock, OMA DRM 1.0, and allows no DRM to be applied by specifying None. These names cannot be used when specifying additional DRM technologies, as they are reserved. <drm> can contain the valid tag, <pricing_model>. See the definition of <pricing_model> under the description of <content_type>.
Indicates whether OMA DRM 1.0 needs to be enabled during migration. The row corresponding to OMA DRM 1.0 is marked as enabled in the DRM table, and remains so after migration is complete.
Specifies an additional class that is used to change the value of the content type during migration. The tag provides the name of the Java class that must implement the interface: com.sun.content.server.migration.ContentTypeMapper. See Appendix A for more information.
Specifies the main Content Delivery Server configuration file that is used to start the deployment during content resubmission. The default value is CDS.properties.
Specifies the file system location where the data that was extracted from the source database is stored. The following tags are valid within <label>:
Specifies the path for the label directory that is used during write steps. If the steps are executed on different systems, the file system path can differ from it. <destination> has an optional style attribute. If the system where the write steps are executed uses a different path separator (for example, DOS instead of UNIX® standard), specify dos or unix as the value for the style attribute as appropriate.
Specifies the physical path for the label location.
Specifies a pricing option. <pricing_option> has the required attributes content_type, name, external_name, and model. It also the optional attributes, auto_assign and trials. The content_type attribute indicates which content type is used with this pricing option. The name attribute specifies the name of the pricing option. The value of name is used as a reference to this pricing option. The external_name attribute specifies the external name of the pricing option. The model attribute specifies the pricing model to assign to this option. See the description of <pricing_model> for the allowed values. The auto_assign attribute, when set to true, this pricing option is used during content migration. If content is found that has a price and price parameters that are identical to this pricing option, the content is associated with this pricing option rather than with a custom price. The trials attribute specifies that trials are allowed with this pricing option. The content type that is assigned with this pricing option must support the Trial Usage pricing model. The following tag is valid within <pricing_option>:
Required if <pricing_option> requires arguments.
Specifies the arguments for <pricing_option>. The values of the required attributes, name and value, are dependent on the pricing model specified for the pricing option. See Chapter 4 for information on pricing model arguments.
Specifies an additional class that is used to determine how the pricing options, custom prices, and free prices are selected for migrated content. The specified class must implement the interface: com.sun.content.server.migration.PricingOptionSticker. See Appendix A for more information.
Specifies the database configuration file, DBConf.xml, that is used to access the Catalog Manager and all Vending Manager databases that are migrated for that Catalog Manager. This file must be in the standard compliant with the database configuration file for Content Delivery Server, version 2004Q1 PU1. The only acceptable value is a file name. A search is done in the $CDS_HOME/cfg directory for the file.
The DBConf.xml file must include the <SystemUser> and <SystemPassword> tags, which must be located just after the <CDSDatabase> tag. The <SystemUser> and <SystemPassword> tags represent the system account login information to the source database server.
Specifies the database configuration file that is used to access the Catalog Manager and all Vending Manager databases that are migrated for that Catalog Manager. This file must be in the standard compliant with the database configuration file for Content Delivery Server, version 2005Q4. The only acceptable value is a file name. The file is searched for in the $CDS_HOME/cfg directory.
Required if vending code is used.
Encloses the entire configuration relative to a single Vending Manager. <vending> takes the attribute, CODE, which distinguishes one server entry from another. The configuration file cannot contain more than one <vending> tag with the same code value. The following tags are valid within <vending>:
Indicates that the retail price for the specified pricing option must be different for the current vending manager. The price is a floating point value specified as the text for this tag. <change_option_price> has a required name attribute that specifies a pricing option. That pricing option must be defined within the <server> tag. The pricing option must have a pricing model that allows a price value.
Instructs the migration process to delay the migration of the billing_info objects. By doing this, the objects are loaded into the database after the migration is completed and the server is started.
Specifies a primary key in the CDS_EVENT table. Including the primary key in the CDS_EVENT table limits the extraction of the data in the table to this primary key's value and to primary keys with larger values. The migration copies over only CDS_EVENT entries with primary keys that have values equal to or greater than the specified primary key.
The row with the specified primary key is included.
Specifies the vending account name at the source configuration. The vending account name is used to extract the correct database connection information from the database configuration files and is also used to extract data from the source database.
Specifies the vending account name at the target configuration. <target_account> is used to extract the correct database connection information from the database configuration files. If this tag is not used, the value of <source_account> is used instead.
Creates custom coverage definition for the verification process. The coverage specifies the percentage of records that must be verified for certain tables.
Each coverage tag can specify different ranges for either the amount of data in the table or the table name and the amount of data to be tested on the table. It has the following attributes:
Specify the table coverage by providing a minimum or maximum value or by providing a table name.
Default values for the attributes are applied in the following circumstances:
Just prior to verification, a table's name and number of rows are passed though the list of coverage tags. First, the name is checked, and if the name matches, the coverage is selected. Next, the number of row is matched. If the number matches, the coverage is selected. If there are no coverage tags, the following default coverage is used:
The default values were estimated as optimal for coverage and for the amount of time that the verification script takes to run. The coverage tags are processed in the order they are found in the configuration file. The scan stops at the first matching tag in cases where the minimum and maximum coverage overlaps. The tags specifying the table names are exempt and are always checked first.
Use a floating point value in the text form, specified as the text value for the <coverage> tag.
The following example shows how to select 95% coverage for the category_item table:
The following example shows how to select 3% coverage for all tables:
To migrate a single server deployment, you must define at least two catalog steps (one read and one write) and two vending steps (one read and one write). See the section, Section 5.12, Running the Migration to learn how these steps are called during the execution.
For examples of migration files, see Section 5.4, Configuration Examples.
This section provides three configuration examples that represent the most widely used deployment scenarios. For examples of setting pricing options, including custom pricing, see Section 4.1.4, Pricing and Pricing Options Configuration in Chapter 4.
This example shows a possible configuration file for single server deployment.
This example shows a possible configuration file for a deployment with one Catalog Manager and two Vending Manager servers.
During the catalog write step, the server code passes all content through the submission and the verification workflows. All the logging information from the server code is stored in the migration.server.log.* files in the $CDS_HOME/dist/cds/logs directory. To configure additional logging properties, such as the maximum size of log files, the maximum number of log files, and the logging thresholds, edit the file called logging.properties in the LABEL_DIRECTORY/catalog directory. This file is created during the catalog read step and is used only during a catalog write step.
The migration.log file contains migration messages, except for debugging messages, and can be found in the $CDS_HOME/dist/cds/logs directory. Examine the migration log files when resolving problems encountered in the migration. For more information on migration log files, see Section 6.1, Messages in Chapter 6.
Instrumentation is now handled through content resubmission. Modify the SubmissionVerifiedWorkflow.xml file as needed for your deployment of the Content Delivery Server. See the Sun Java System Content Delivery Server Installation Guide for information.
Use the <event_pk> tag to optimize the migration of the CDS_EVENT table. The event table is used primarily for reporting purposes. It provides information that can help you improve the execution time of migrating a single vending server. You can specify the first primary key that is migrated by using the <event_pk> tag. The rest of the event table can be migrated after the server is up and running. The userid argument must also be modified accordingly. For a description of the <event_pk> tag, see Section 5.3.2, Configuration File Tags.
For example, if the <event_pk> specified in the configuration file is 12345, then the following commands can be used to migrate the rest of the CDS_EVENT table:
If the catalog server is deployed standalone, that is, running on a server separate from the vending servers, modify the query argument to separate catalog and vending events. Use
'query="where server_instance_id = 0 and cds_event_id < 12345"'
to select catalog events and write them to the PS_OWNER schema. Use
'query="where server_instance_id != 0 and cds_event_id < 12345"'
to select vending events and write them to the VS_OWNER schema.
The userid argument must also be modified accordingly. The <connect> argument is the name that Oracle client tools can resolve into a server connection. If no naming services are set up on the system, you can use direct addressing of your database instances:
Refer to Oracle documentation for more information on Oracle naming schemas.
If a newly deployed Catalog Manager is running standalone, you must specify the <catalog_event> tag. When specified, the catalog-specific events in all vending event tables are migrated to the catalog event table. For a description of the <catalog_event> tag, see Section 5.3.2, Configuration File Tags.
BillingInfo objects store persistent billing information about purchase transactions and are used in the integration code. To optimize the migration of BillingInfo objects, objects are loaded into the database after the Vending Manager server is started.
The objects are loaded consecutively, one by one. However, objects relative to a specific purchase can be loaded immediately as needed. When the server retrieves a role and looks for the relevant object in the table, that object is immediately loaded into the table.
To turn the optimization process on, you must have a <delayed_billing_infos> tag in the vending section of the migration configuration file. Specify migration.load_billing_infos in the subscriber portal properties file. The properties must be in the form, filename:servername, where filename is the migration configuration file name and servername corresponds to the server code from the migration configuration file. See the description of the <server> tag in Section 5.3.2, Configuration File Tags for more information.
The performance of the subscriber portal that is running the delayed loading of the BillingInfo objects might be slower than expected. Note that performance improves as more objects are loaded into the database. The performance is impacted only when a requested BillingInfo object is not yet loaded. If testing shows that migration of a single vending server does not meet your downtime requirements, use the delayed_billinginfos in the migration to decrease the execution of the write step (approximately five times and increase the execution time of the read step.
The information in this section addresses cases where the Catalog Manager administrator has changed the supported devices for one or more content items.
Generally, any edition submitted into both version 2004Q1PU1 and version 2005Q4 of the Content Delivery Server database can have any number of derived content editions created by a DRM. Even when no DRM protection is chosen, a derived edition is still created. The Catalog Manager administrator can force the value of the supported devices property for any derived editions, thus modifying the input for the capability matching routine that determines which editions are allowed for download to which devices.
Since the content is being passed through the submission and verification workflows during migration, it is impossible to predict which derived editions created in the old database become which derived editions in the new database, especially since the number of editions in both databases might not be the same.
This factor prevents the migration process from matching the current selection of supported devices made for derived editions to newly created derived editions. Migration expands the supported devices property list to include devices selected for all derived editions of an originally submitted edition.
Migrating supported devices information for editions is handled in the following way: if the Catalog Manager administrator customized the supported devices property for any of the derived editions of an original edition, the values of the supported devices property for all of the derived editions of a single original edition are combined. This value is used as the supported devices property during resubmission of the original editions. Though the property is used during resubmission, it is inherited only by the derived editions. The property value originally defined for the original edition is not changed.
The information in this section addresses cases where multiple pricing options are selected for content in the source database or where trials are offered in a multivending deployment.
Version 2005Q4 of the Content Delivery Server allows only one pricing model to be associated with a content item. In the previous release, it was possible for a content item to have multiple pricing models, for example a content item of type midlet could have both a charge on first download and charge per subscription. To meet the requirement of one pricing model per content item, only one pricing model is chosen for all migrated content during migration. If a content item has a per usage model with a non-zero price value, the per use pricing model is selected. If a content item has a subscription pricing model with a non-zero price value, the subscription pricing model is selected. Otherwise, the content item is assigned the First Download Only pricing model, even if the price value is zero.
The previous release of the Content Delivery Server allowed the Vending Manager administrator to offer trials, that is the number of times the content could be used by a subscriber before payment is required. In version 2005Q4, the trial option is now specified by the Catalog Manager administrator. That means that the information associated with trials is now stored as a part of the catalog database rather than the vending database. Therefore, all trials set up by various Vending Manager administrators must be merged into a single trial option provided by the Catalog Manager. To do this, all vending databases are scanned for trial information. For all content items that have trials enabled, the minimum value of allowed trials is chosen. For instance, if the content item, GoldMiner, has three trials in Vending Manager1, five trials in Vending Manager2, and no trials in Vending Manager3, three trials are allowed for GoldMiner across all Vending Managers in the version 2005Q4 deployment. This minimum is selected only among those Vending Managers that have at least one trial enabled for GoldMiner.
Once the XML configuration file is in place, you are ready to execute the migration steps. To execute a single step, issue the following command:
Use this option if your configuration file name is other than DBMigration.xml.
For example to run a catalog read step, enter the following command:
To run a vending write step using an alternate configuration file, enter the following command:
When running a database migration command, you must specify -c or -v and -r or -w.
The default database migration file, DBMigration.xml, is located in the $CDS_HOME/cfg/migration/ directory.
|
Note - Once the migration process is started, do not modify the configuration file. Any change to it requires that the migration process be restarted from the beginning. |
To complete the migration, you must execute all the read and write steps for the Catalog Manager server and for all the Vending Manager servers being migrated. When migration begins, the Catalog Manager server must be placed in READ ONLY mode or the application server running it must be shut down. READ ONLY means that no modifications can be made to the Catalog Manager database, that is, developers cannot submit any content and administrators cannot make any modifications. Data integrity can be lost during the migration process if these requirements are not met.
To migrate a Vending Manager server, its application server must be shut down.
Keep the following points in mind regarding the deployment of the Content Delivery Server during the migration:
If you are running the Content Delivery Server on the same machine as both the source and target database servers, you can ignore this item. You do not have to start the Content Delivery Server on the database server.
For a single server deployment, follow these migration steps:
1. Restrict access to the Catalog Manager and Developer Portal.
2. Execute the catalog read step.
3. Execute the catalog write step on the target deployment.
4. (Optional) Verify that the migration of the Catalog Manager was successful.
You can use the migration verification script provided by the Content Delivery Server. See Section 5.13, Verifying Migration Completion for information on how to run the script.
5. Shut down the Content Delivery Server's single server completely.
6. Execute the vending read step.
7. Execute the vending write step.
8. (Optional) Verify that the migration of the Vending Manager was successful.
9. Start the new (target) server.
For deployments with more than one Vending Manager, follow these migration steps. This scenario assumes that at least one Vending Manager (denoted as Vending0)is running in single server mode with the Catalog Manager. Additional Vending Managers are denoted as VendingX.
1. Restrict access to the Catalog Manager and Developer Portal.
2. Execute the catalog read step.
3. Execute the catalog write step on the target deployment.
4. (Optional) Verify that the migration of the Catalog Manager was successful.
You can use the migration verification script provided by the Content Delivery Server. See Section 5.13, Verifying Migration Completion for information on how to run the script.
5. Shut down the old Catalog Manager and the Vending Manager (source) server.
6. Execute the vending read step for Vending0.
7. Execute the vending write step for Vending0.
8. (Optional) Verify that the migration of Vending0 was successful.
9. Start the new Catalog Manager and Vending0 (target) server.
10. Shut down the Vending Manager, VendingX.
11. Execute the vending read step for VendingX.
12. Execute the vending write step for VendingX.
13. (Optional) Verify that the migration of VendingX was successful.
14. Start VendingX on the target deployment.
By using this approach, the down time for subscribers is limited to the time spent during the vending read and write steps only.
Remember that when executing the catalog write step, you might run into problems if the paths to the Content Delivery Server home directory do not match for both the application server and the database server when network paths are used. To correct this problem, follow these steps:
1. Open the $CDS_HOME/deployment/deployment_name/conf/SubmissionVerifierWorkflows.xml file.
2. Save a backup copy of this file.
3. Replace all absolute paths to the JAR files with suitable paths on the database server.
4. Execute the catalog write step.
You can use the verification migration script to verify that the migration of the database is successful. To run the verification script, enter the following command at the prompt:
Use this option if your configuration file name is other than DBMigration.xml.
A -c or -v flag is required to identify a catalog or vending step.
For example, when verifying the migration of the Catalog Manager, enter the following command:
When verifying the migration of the Vending Manager, enter the following command:
After migration verification is completed, you might need to set some content types to be read only, you must run an SQL command to specify the content type to be read only before you deploy the latest version of the Content Delivery Server. See Step 12 in Chapter 1, Section 1.3, Migrating the Content Delivery Server for details.
| Migration Guide | 819-3217-10 |
|
Copyright © 2005, Sun Microsystems, Inc. All Rights Reserved.