24Overview of Performing a Siebel Database Upgrade

Overview of Performing a Siebel Database Upgrade

Roadmap for Siebel Database Environment Upgrade

The following roadmap provides an overview of tasks required to upgrade your Siebel environment. Use this roadmap to determine the applicability of each task to each environment. For example, if a task has No listed in the Prod column, then that task is not required for production test, or production environment upgrades. All tasks required for the production test upgrade are also required for the production environment upgrade. The development environment has more required tasks. Print and review this roadmap as you plan and work through your upgrade.

Note: Links to other topics in this guide are provided throughout this chapter. These links provide additional information that cannot be provided in these brief overviews. You must, however, perform the upgrade in the proper order. For information on general planning, see this chapter as well as Siebel Database Upgrade Planning and Application Planning for a Siebel Upgrade To begin the process of upgrading your Siebel database, see Preparing for Siebel Database Upgrade then refer to each subsequent chapter.

For a more detailed view of the upgrade steps for each Siebel database environment, see the following topics:

The following graphic displays the basic tasks for upgrading the Siebel database environments.

Upgrading the Siebel Database Environment

This roadmap includes the following topics:

    Evaluating the Environment

    Whether you are beginning your upgrade with the development environment or the production test environment, it is advisable to begin by evaluating the process to determine resource allocation and other concerns. The following table lists information relevant to this phase of the upgrade.

    Table Evaluating the Environment

    Upgrade Tasks Where to Find Documentation Dev? Prod?

    1. Check My Oracle Support for recently published articles regarding your upgrade.

    My Oracle Support ( http://support.oracle.com)

    Yes

    Yes

    2. Check Siebel Release Notes on My Oracle Support.

    My Oracle Support (http://support.oracle.com)

    See 557191.1 (Article ID).

    Yes

    Yes

    3. Review hardware and third-party software requirements.

    The Certifications tab on My Oracle Support.

    Yes

    Yes

    4. If you need help evaluating your environment, then contact your Oracle sales representative for Oracle Advanced Customer Services to request assistance from Oracle’s Application Expert Services.

    Contact your Oracle sales representative for Oracle Advanced Customer Services to request assistance from Oracle’s Application Expert Services

    Yes

    Yes

      Planning the Upgrade

      Review the Siebel Database Upgrade Guide to plan and further determine the scope of the upgrade process. The following table lists chapters in the Siebel Database Upgrade Guide to consider during the planning phase.

      Table Planning the Upgrade

      Upgrade Tasks Where to Find Documentation Dev? Prod?

      1. Review the Siebel Database Upgrade Guide for new features in the current release.

      What’s New in Siebel Database Upgrade Guide, Siebel CRM 18.12 Update

      Yes

      Yes

      2. Review how to perform the Siebel database upgrade and how the upgrade works step-by-step.

      Overview of Siebel Database Environments and Siebel Database Upgrade Planning

      Yes

      Yes

      3. Review Siebel database upgrade planning and about upgrade considerations for your Siebel user interface.

      Siebel Database Upgrade Planning

      Yes

      Yes

      4. Review planning for Siebel Business Applications prior to upgrade.

      Application Planning for a Siebel Upgrade

      Yes

      Yes

        Configuring the Environment

        This is the most time-consuming and resource-intensive phase of the Siebel database upgrade. Many tasks in this phase are performed only during a development environment upgrade. Perform these tasks only if you are upgrading a development environment. All other tasks are performed during all upgrades. The following table lists information relevant to this phase of the upgrade.

        Table Configuring the Environment

        Upgrade Tasks Where to Find Documentation Dev? Prod

        1. Copy the ancestor repositories.

        For details, see Copying Ancestor Repositories.

        Yes

        No

        2. Upgrade the servers.

        Verify that you have met all requirements for the upgrade. You might be required to install the Siebel Gateway, Siebel Servers, and Siebel Application Interface.

        For more information on server upgrade and installation, see the Siebel Installation Guide. See also the Certifications tab on My Oracle Support.

        Yes

        Yes

        3. Upgrade third-party software.

        You might need, for example, to upgrade operating system software.

        See the Certifications tab on My Oracle Support.

        Yes

        Yes

        4. Upgrade the RDBMS.

        See the Certifications tab on My Oracle Support.

        For details on upgrading the RDBMS, see About Upgrading Your RDBMS in the Siebel Environment.

        Yes

        Yes

        5. Perform any preupgrade tasks on your specific database.

        For details on IBM DB2.see Perform Preupgrade Tasks for IBM DB2.

        For details on Oracle Database, see Perform Preupgrade Tasks for Oracle Database.

        For details on Microsoft SQL Server, see Perform Preupgrade Tasks for Microsoft SQL Server.

        Yes

        Yes

        6. Preparing the Siebel database for an upgrade.

        Tasks include verifying that you have no pending workflows, stopping the Siebel Server, and so on.

        For an overview, see Perform Preupgrade Tasks for the Siebel Database.

        For details on basic database preparation, see Preparing for Siebel Database Upgrade.

        Yes

        Yes

        7. Preparing Siebel Business Applications data for an upgrade.

        See Preparing Siebel Application Data for Upgrade.

        Yes

        Yes

        8. Prepare your developers for the upgrade.

        Tasks include backing up development databases, ensuring that all developers’ projects are checked in and unlocked.

        See Prepare Developers for the Upgrade.

        Yes

        No

        9. Use the previous version of Siebel Tools to rename your Siebel Repository.

        For an overview, see Upgrade Siebel Database Schema (upgrep).

        For task details, see Renaming the Siebel Repository.

        Yes

        No

        10. Upgrade the Siebel database schema.

        Tasks include running the Database Configuration Wizard (upgrep) and the Upgrade Wizard.

        For an overview, see Upgrade Siebel Database Schema (upgrep).

        For specific task details, including running the Database Configuration Utilities, see Upgrading the Siebel Database.

        Yes

        Yes

        11. Prepare for the repository merge.

        For details, see Perform Repository Merge.

        Yes

        No

        12. Use the newly installed version of Siebel Tools to perform the repository merge.

        For details, see Perform Repository Merge.

        Yes

        No

        13. Upgrade custom database schema.

        Tasks include running the Database Configuration Wizard to upgrade previous database schema customizations, running the Upgrade Wizard, resolving errors with upgrade log files, and backing up the Siebel database.

        For an overview, see Upgrade Custom Database Schema (upgphys).

        For task details, see Upgrading the Siebel Database.

        Yes

        No

        14. Review the user interface.

        Tasks include reviewing potential object property conflicts, noting inherited behavior, and performing other tasks on the Siebel user interface.

        For an overview, see Review the User Interface.

        For task details, see Reviewing the Siebel User Interface.

        Yes

        Yes

        15. Create or update the Siebel Runtime Repository to assist in user interface testing.

        For details, see Using Siebel Tools.

        Yes

        Yes

        16. Perform postmerge development tasks.

        For an overview, see Perform Postmerge Development Tasks.

        For task details, see Siebel Postmerge Development Tasks.

        Yes

        No

        17. Perform postupgrade tasks on database and file system.

        These tasks are done following a completed upgrade.

        For an overview, see Perform Postupgrade Tasks for Database and File System.

        For task details, see Postupgrade Tasks for the Siebel Database.

        Yes

        Yes

        18. Perform postupgrade tasks for applications configuration.

        Tasks include installing activating and deploying any workflows, as well as performing other postupgrade configuration tasks.

        For an overview, see Perform Postupgrade Tasks for Applications Configuration.

        For task details, see Postupgrade Tasks for Siebel Business Applications.

        Yes

        Yes

          Testing the Environment

          In this phase, perform system tests and prepare for the transition to the next database environment. The following table lists information relevant to this phase of the upgrade.

          Table Testing the Environment

          Upgrade Tasks Where to Find Documentation Dev? Prod?

          1. Perform system tests.

          Use available test data to perform unit testing.

          For an overview, see Perform System Tests.

          Yes

          Yes

          2. Prepare for the transition to the next environment. If you upgraded a development environment, then you will move to the production test environment, if you upgraded the production test environment then you proceed to the production environment.

          Tasks include creating or updating a Siebel Runtime Repository, and regenerating Siebel repository definition files, if necessary.

          For an overview, see Prepare for Transition to Production Test Environment.

          For details on creating or updating a Siebel Runtime Repository, see Using Siebel Tools.

          For details on regenerating Siebel repository definition files, see Regenerating the Siebel Repository Definition Files.

          Yes

          Yes

            Implementing the Environment

            Implementing your environment occurs when you have fully upgraded and tested your production test environment and then move to your live, production environment. The following table lists information relevant to this phase of the upgrade.

            Note: Perform these tasks only if you are transitioning from a production test to a production environment.

            Table Implementing the Environment

            Upgrade Tasks Where to Find Documentation Dev? Prod?

            1. Apply the latest maintenance release (if any).

            Check with your Oracle representative for any information on Siebel CRM maintenance releases which might be applicable to your release.

            No

            Yes

            2. Perform an architecture and infrastructure review to ensure the production environment meets the requirements of the upgrade.

            For help with an architecture and infrastructure review, contact your Oracle sales representative for Oracle Advanced Customer Services to request assistance from Oracle’s Application Expert Services.

            No

            Yes

              Process of Upgrading a Siebel Development Environment

              Platforms: Windows and UNIX only. This topic does not apply to IBM z/OS.

              This topic lists the steps required to upgrade a Siebel development environment to the current release. Print this topic and use it as a checklist when doing the upgrade.

              Note: Links to other topics in this guide are provided throughout this chapter. These links are meant to provide additional information that cannot be provided in these brief overviews. You must, however, perform the upgrade in the proper order. For information on general planning, see this chapter as well as Siebel Database Upgrade Planning, and Application Planning for a Siebel Upgrade To begin the process of upgrading your Siebel database, see Preparing for Siebel Database Upgrade, then refer to each subsequent chapter.

              The topic is divided into sections, each containing a list of numbered steps. You complete each section in the order shown.

              Note: If you are performing a development environment upgrade on IBM z/OS, then see Siebel Database Upgrade Guide for DB2 for z/OS.

                Search for Articles on My Oracle Support

                  Upgrade the Servers

                  Verify that you have identified all of the Siebel releases that are required for the upgrade. Along with the current release, Siebel 2018, also install the latest Siebel update. For more information on server upgrade and installation, see Siebel Installation Guide. See also the Certifications tab on My Oracle Support.

                  Caution: Do not install a new Siebel database as part of upgrading the Siebel Enterprise.

                  To perform the following steps, see Siebel Installation Guide.

                  1. Upgrade the Siebel Gateway, Siebel Servers, and Siebel Application Interface.

                    For information on installing these Siebel Enterprise components, see Siebel Installation Guide. The modules named are for the current release.

                  2. Choose the option to install the Siebel Database Configuration Utilities along with the Siebel Gateway and Siebel Server.

                  3. Install language packs for your currently deployed languages and any new languages.

                  4. If you have customized the configuration of Enterprise components, such as Siebel Servers, then you can run a script to migrate configuration parameters to the upgraded Siebel Enterprise. (Do not perform this step for a migration installation, which is described in Siebel Installation Guide.)

                  5. If you are upgrading to Siebel CRM versions 8.1 or 8.2, then copy the ancestor repositories. In Siebel CRM versions 8.1 and 8.2, the ancestor repository files are located in an Ancestor Repositories media directory that you access from Oracle Software Delivery Cloud. For more information, see Preparing for the Repository Merge.

                    Upgrade Third-Party Software

                    • Upgrade third-party software as required due to dependencies on Oracle’s Siebel software or other installed software. For example, you might need to upgrade the following software:

                      • Operating system software. Some database upgrades require newer versions of operating systems, such as AIX or Windows.

                      Upgrade the RDBMS

                      • If required, then upgrade the RDBMS version. Refer to the vendor’s documentation to perform the upgrade:

                        Perform Preupgrade Tasks for the Siebel Database

                        These steps apply to all database types.

                        1. Review guidelines for configuring the RDBMS. See the Siebel Installation Guide.

                        2. Verify that the Workflow Monitor and Workflow action agents have processed all pending requests.

                        3. Stop Siebel Servers, but make sure the Siebel Gateway is running.

                          Note: The Siebel Gateway must be running for the Database Configuration Wizard to proceed.
                        4. Perform the tasks in Preparing for Siebel Database Upgrade

                          Perform Preupgrade Tasks for IBM DB2

                          1. Perform the tasks in Preparing an IBM DB2 Database for a Siebel Upgrade

                          2. If you are disabling archive logging, then allow time for a cold backup when reenabling archive logging.

                          3. Apply runstats to all tables, reorganize any that are out of cluster, or that are clustered less than 90%, then execute runstats on those tables again. This will improve performance and reduce the number of tasks that will need to be done after the upgrade is complete.

                            Perform Preupgrade Tasks for Oracle Database

                            1. Perform the tasks in Preparing an Oracle Database for a Siebel Upgrade

                            2. Run the Oracle Analyze command on the Siebel database. Highly fragmented indexes can cause the upgrade to fail.

                              Perform Preupgrade Tasks for Microsoft SQL Server

                              1. Perform the tasks in Preparing a Microsoft SQL Server Database for a Siebel Upgrade

                              2. Run Microsoft SQL Server statistics. This will improve upgrade performance.

                                Perform Preupgrade Tasks for Application Data

                                1. Perform the tasks in 477519.1 (Article ID) on My Oracle Support. This document was previously published as Siebel Technical Note 521. The following table shows the applicability of the tasks in the article.

                                  Item Environment

                                  Location and amendment of S_VIEW_WTMPL_IT.ITEM_NUM records

                                  Development, production test, production

                                  Update multi-value group (MVG) and association list applets

                                  Development only

                                  Update unnamed controls in applet Web template items

                                  Development only

                                2. Perform the tasks in the My Oracle Support article listed in the following table.

                                  Title Environment

                                  477254.1 (Article ID). This document was previously published as Siebel Technical Note 481.1

                                  Development only

                                3. Perform the tasks in Preparing Siebel Application Data for Upgrade

                                  Prepare Developers for the Upgrade

                                  To prepare developers for the upgrade, do the following.

                                  1. Back up the development database.

                                  2. Verify that all developers have checked in their projects and that all projects are unlocked.

                                  3. Verify that all developers have disconnected from the database. The only open account must be the account that is performing the upgrade.

                                  4. Install the new version of Siebel Tools on development workstations.

                                    Note: Keep at least one copy of the previous version of Siebel Tools. You will need it to perform repository operations before the repository merge.

                                    Upgrade Siebel Database Schema (upgrep)

                                    1. (Optional) Changing the Siebel Database Configuration Utilities Language.

                                    2. Run the Database Configuration Wizard:

                                    3. Edit generated SQL files as required by articles on My Oracle Support, and by the applicable version of Siebel Release Notes on My Oracle Support.

                                    4. Resume the upgrade: Starting the Siebel Upgrade Wizard.

                                    5. Review the upgrade logs and resolve errors:

                                    6. If the upgrade contains unacceptable errors, then do the following:

                                      1. Restore the backup of the database.

                                      2. Correct the errors.

                                      3. If errors occurred because you entered incorrect information in the Database Configuration Wizard, then see Regenerating SQL Files for a Siebel Upgrade.

                                      4. Rerun the Database Configuration Wizard.

                                    7. Back up the upgraded database.

                                      Prepare for Repository Merge

                                      1. Preparing for the Repository Merge.

                                        Note: If you are upgrading from Siebel CRM version 8.1.1.x (SIA repository), version 8.2.2.x, version 15.x, or version 16.x to version 17.0, then use Process of Meeting the Requirements for an Incremental Upgrade to Siebel 2018, Using the Incremental Repository Merge.
                                      2. Migrating Siebel Repository Objects to the Standard User Interface.

                                      3. Set the Upgrade Ancestor property for copied objects. See Configuring Siebel Repository Objects to Inherit Upgrade Behavior.

                                        Perform Repository Merge

                                        1. Performing a Siebel Repository Merge.

                                          Note: If you are upgrading from Siebel CRM version 8.1.1.x (SIA repository), version 8.2.2.x, version 15.x, or version 16.x to version 17.0, then use Process of Meeting the Requirements for an Incremental Upgrade to Siebel 2018, Using the Incremental Repository Merge.
                                        2. Reviewing the Siebel Repository Merge Log Files.

                                        3. If the repository merge contains unacceptable errors, then do the following:

                                          1. Restore the backup of the upgraded database.

                                          2. Correct the errors.

                                          3. Rerun the repository merge.

                                        4. Running the Siebel Postmerge Utilities.

                                        5. Generating Siebel Enterprise Integration Manager Temporary Columns.

                                        6. (Optional) Configure case insensitive queries in the New Customer Repository. See Running the Siebel Case Insensitivity Wizard.

                                        7. Back up the Siebel database.

                                          Upgrade Custom Database Schema (upgphys)

                                          1. Run the Database Configuration Wizard:

                                          2. Run the Upgrade Wizard: Starting the Siebel Upgrade Wizard.

                                          3. Review the upgrade log files and resolve errors: Reviewing Siebel Upgrade Log Files for Errors.

                                          4. If the upgrade contains unacceptable errors, then do the following:

                                            1. Restore the backup of the database you made after the repository merge.

                                            2. Correct the errors.

                                            3. Repeat the steps beginning after the backup.

                                          5. Manually Archiving Siebel Upgrade Log Files.

                                          6. Back up the upgraded database.

                                            Review the User Interface

                                            1. Reviewing Siebel Repository Object Property Conflicts.

                                            2. Review the following topics on user interface upgrade before proceeding:

                                            3. If you performed an ICL upgrade and chose Label on Top, then revise the cascading style sheet. See Deleting Unneeded Siebel Repository Files.

                                            4. Create or update the Siebel Runtime Repository to assist in user interface testing. For more information, see Using Siebel Tools.

                                            5. If you customized style sheet or web template files in the previous release, implement those customizations in the new release again and execute a Full Publish in Siebel Tools to compile the Siebel Runtime Repository data.

                                              Carefully review the user interface in the new release before reimplementing customizations to these files.

                                            6. Perform the tasks in Reviewing the Siebel User Interface

                                            7. The postmerge utilities do not convert certain types of flow-based applets to grid-based applets. For example, they do not convert custom form applets to grid-based applets. See the editing applet layout topic in Configuring Siebel Business Applications and convert the remaining flow-based applets as required.

                                              Perform Postmerge Development Tasks

                                              1. Perform the tasks in Siebel Postmerge Development Tasks

                                              2. Resolve any business component and join conflicts.

                                              3. Deleting Unneeded Siebel Repository Files.

                                                Perform Postupgrade Tasks for Database and File System

                                                1. Perform the tasks in Postupgrade Tasks for the Siebel Database

                                                2. Reset upgrade-specific database and database server parameters back to their recommended settings for production. See Siebel Installation Guide for recommended parameter settings.

                                                3. If you exported data from interface tables before the upgrade, then review the database and import the data as desired. See Siebel Enterprise Integration Manager Administration Guide.

                                                4. Generate a Siebel Remote database template file. See Siebel Remote and Replication Manager Administration Guide.

                                                5. Run database statistics.

                                                Note: The upgrade is complete. The remaining topics deal with configuration and validation tasks.

                                                  Perform Postupgrade Tasks for Applications Configuration

                                                  1. If applicable, then review the results of the Person and Organization merge. Make configuration changes as required.

                                                  2. Perform the tasks in Postupgrade Tasks for Siebel Business Applications

                                                  3. Verify the function of interfaces in integrated applications.

                                                  4. Activate and deploy workflows.

                                                  5. If you have set up integrations for transferring data to or from third-party applications using Siebel Enterprise Application Integration, then verify the integrations are configured correctly. For information on EAI, see Overview: Siebel Enterprise Application Integration.

                                                  6. If you have used Siebel Enterprise Integration Manager to set up batch processing jobs, then verify Siebel Enterprise Integration Manager is configured correctly. For information on Siebel Enterprise Integration Manager, see Siebel Enterprise Integration Manager Administration Guide.

                                                    Perform System Tests

                                                    • Use available test data to perform unit testing. Validate the operation of the application in the following areas:

                                                      • User interface

                                                      • Data interfaces

                                                      • Integrity of migrated data

                                                      • Workflow function

                                                      Prepare for Transition to Production Test Environment

                                                      1. Publish the Siebel Runtime Repository. For more information, see Using Siebel Tools.

                                                      2. If you revised the repository after running upgphys, then you must regenerate the repository definition files. See Regenerating the Siebel Repository Definition Files.

                                                        Process of Upgrading a Siebel Production Test Environment

                                                        Platforms: Windows and UNIX only. This topic does not apply to IBM z/OS.

                                                        This topic lists the steps required to upgrade a production test environment to the current release. Print this topic and use it as a checklist when doing the upgrade.

                                                        Note: Links to other topics in this guide are provided throughout this chapter. These links are meant to provide additional information that cannot be provided in these brief overviews. You must, however, perform the upgrade in the proper order. For information on general planning, see this chapter as well as Siebel Database Upgrade Planning and Application Planning for a Siebel Upgrade To begin the process of upgrading your Siebel database, see Preparing for Siebel Database Upgrade then refer to each subsequent chapter after that.

                                                        The topic is divided into sections, each containing a list of numbered steps. Complete each section in the order shown.

                                                        Note: If you are performing a Production Test environment upgrade on IBM z/OS, then see Siebel Database Upgrade Guide for DB2 for z/OS.

                                                          Search for Articles on My Oracle Support

                                                            Upgrade the Servers

                                                            Verify that you have identified all of the Siebel releases that are required for the upgrade. Along with the current release, Siebel 2018, also install the latest Siebel update. For more information on server upgrade and installation, see Siebel Installation Guide. See also the Certifications tab on My Oracle Support.

                                                            Caution: Do not install a new Siebel database as part of upgrading the Siebel Enterprise.

                                                            To perform the following steps, see Siebel Installation Guide:

                                                            1. Upgrade the Siebel Gateway, Siebel Servers, and Siebel Application Interface.

                                                              For information on upgrading these Siebel Enterprise components, see Siebel Installation Guide. The modules named are for the current release.

                                                            2. Install the Siebel Database Server files on the Siebel Server you will use to perform the upgrade. You only need to install the database server files for the database type that you are upgrading.

                                                              Note: You need only to install a new Siebel Server instance if you are using a different computer from that on which you performed the Production Test Upgrade. If you are using the same Siebel Server installation as for the Production Test upgrade, then the existing files can be leveraged instead.
                                                            3. Install language packs for your currently deployed languages and any new languages.

                                                            4. If you have customized the configuration of enterprise components, such as Siebel Servers, then you can run a script to migrate configuration parameters to the upgraded Siebel Enterprise. (Do not perform this step for a migration installation, which is described in Siebel Installation Guide.)

                                                              Upgrade Third-Party Software

                                                              • Upgrade third-party software as required due to dependencies on Oracle’s Siebel software or other installed software. For example, you might need to upgrade the following software:

                                                              Operating system software. Some database upgrades require newer versions of AIX or Windows.

                                                                Upgrade the RDBMS

                                                                • If required, upgrade the RDBMS version. Refer to the vendor’s documentation to perform the upgrade:

                                                                  Perform Preupgrade Tasks for the Siebel Database

                                                                  These steps apply to all database types.

                                                                  1. Review guidelines for configuring the RDBMS. See the Siebel Installation Guide.

                                                                  2. Verify that the Workflow Monitor and Workflow action agents have processed all pending requests.

                                                                  3. Perform the tasks in Preparing for Siebel Database Upgrade

                                                                    Perform Preupgrade Tasks for IBM DB2

                                                                    1. Perform the tasks in Preparing an IBM DB2 Database for a Siebel Upgrade

                                                                    2. Make sure that runstats is current in your environment and that tables with clustering indexes are not out of cluster. Reorganize any tables that are out of cluster or that are clustered less than 90% before the upgrade if possible.

                                                                      Table maintenance must be aggressively pursued prior to the upgrade to ensure that your system is in optimal condition. The same general consideration must also be kept in mind with regard to repository tables.

                                                                      Perform Preupgrade Tasks for Oracle Databases

                                                                      1. Perform the tasks in Preparing an Oracle Database for a Siebel Upgrade

                                                                      2. Run the Oracle Analyze command on the Siebel database. Highly fragmented indexes can cause the upgrade to fail.

                                                                        Perform Preupgrade Tasks for Microsoft SQL Server

                                                                        1. Perform the tasks in Preparing a Microsoft SQL Server Database for a Siebel Upgrade

                                                                        2. Run Microsoft SQL Server statistics. This will improve upgrade performance.

                                                                          Perform Preupgrade Tasks for Application Data

                                                                          1. Perform the tasks in 477519.1 (Article ID) on My Oracle Support. This document was previously published as Siebel Technical Note 521. The following table shows the applicability of the tasks in the article.

                                                                            Item Environment

                                                                            Location and amendment of S_VIEW_WTMPL_IT.ITEM_NUM records

                                                                            Development, production test, production

                                                                            Update multi-value group (MVG) and association list applets

                                                                            Development only

                                                                            Update unnamed controls in applet Web template items

                                                                            Development only

                                                                          2. Perform the tasks in Preparing Siebel Application Data for Upgrade

                                                                            Prepare the Siebel Database for Upgrade

                                                                            1. Refer to 477772.1 (Article ID) on My Oracle Support. This document was previously published as Siebel Technical Note 586, and includes instructions on how to set up the Siebel database and Siebel Servers in the production test environment.

                                                                            2. Stop the Siebel Servers and the Siebel Gateway.

                                                                            3. Close all database connections. The only database connection must be the account performing the upgrade.

                                                                            4. Copy application files to the environment, including:

                                                                              • Reports files.

                                                                            5. Copy the custrep.dat and schema.ddl files to the environment. See Moving the Siebel Repository Files.

                                                                              If you have made changes to the Siebel repository since performing the development upgphys, then you must regenerate the schema.ddl and custrep.dat files. See Regenerating the Siebel Repository Definition Files.

                                                                            6. Verify the production test database is either a copy of the current production database or has the same topology and amount of data.

                                                                              This is important for effective upgrade tuning before performing the production upgrade.

                                                                            7. Back up the production test environment database. (If you backed up the database as part of an RDBMS upgrade, then ignore this step.)

                                                                              To do upgrade tuning, you will restore this database and perform test-upgrades on it.

                                                                              Upgrade the Siebel Database (upgrep and upgphys)

                                                                              1. Run the Database Configuration Wizard:

                                                                              2. Refer to 475427.1 (Article ID) on My Oracle Support for any instructions specific to your upgrade scenario. This document was previously published as Siebel Alert 1322.

                                                                              3. Resume the upgrade. See Starting the Siebel Upgrade Wizard.

                                                                              4. Review the upgrade logs and resolve errors. See Reviewing Siebel Upgrade Log Files for Errors.

                                                                              5. If the upgrade contains unacceptable errors, then do the following:

                                                                                1. Restore the backup of the database.

                                                                                2. Correct the errors.

                                                                                3. If errors occurred because you entered incorrect information in the Database Configuration Wizard, then see Regenerating SQL Files for a Siebel Upgrade.

                                                                                4. Rerun the Database Configuration Wizard.

                                                                              6. See Manually Archiving Siebel Upgrade Log Files.

                                                                              7. Back up the upgraded database.

                                                                                Perform Postupgrade Tasks for Database and File System

                                                                                1. Perform the tasks in Postupgrade Tasks for the Siebel Database

                                                                                2. Reset upgrade-specific database and database server parameters back to their recommended settings for production. See Siebel Installation Guide for recommended parameter settings.

                                                                                3. If you exported data from interface tables before the upgrade, then review the database and import the data as desired.

                                                                                4. Generate a Siebel Remote database template file. See Siebel Remote and Replication Manager Administration Guide.

                                                                                5. Run database statistics.

                                                                                Note: The upgrade is complete. The remaining topics deal with configuration and validation tasks.

                                                                                  Perform Postupgrade Tasks for Applications Configuration

                                                                                  1. Perform the tasks in Postupgrade Tasks for Siebel Business Applications

                                                                                  2. Verify the function of interfaces in integrated applications.

                                                                                  3. Activate and deploy workflows.

                                                                                  4. If you have set up integrations for transferring data to or from third-party applications using Siebel Enterprise Application Integration, then verify that the integrations are configured correctly. For information on EAI, see Overview: Siebel Enterprise Application Integration.

                                                                                  5. If you have used Siebel Enterprise Integration Manager to set up batch processing jobs, then verify Siebel Enterprise Integration Manager is configured correctly. For information on Siebel Enterprise Integration Manager, see Siebel Enterprise Integration Manager Administration Guide.

                                                                                  6. If you customized style sheet or Web template files in the previous release, then reimplement those customizations in the new release as desired.

                                                                                    Carefully review the user interface in the new release before reimplementing customizations to these files.

                                                                                    Perform System Tests

                                                                                    • Use available test data to perform unit testing. Validate the operation of the application in the following areas:

                                                                                      • User interface

                                                                                      • Data interfaces

                                                                                      • Integrity of migrated data

                                                                                      • Workflow function

                                                                                      Process of Tuning Siebel Upgrade Performance

                                                                                      Platforms: Windows and UNIX only.

                                                                                      Note: This process is optional.

                                                                                      Use this process to run test upgrades in the production test environment so you can tune upgrade performance. Improving upgrade performance reduces downtime when you perform the production environment upgrade. The steps in this process cover standard performance tuning. To implement more advanced tuning, including high-availability tuning, contact your Oracle sales representative for Oracle Advanced Customer Services.

                                                                                      Also use this process to test the additive schema changes feature. This feature allows you to perform part of the upgrade on the production database without taking it offline. This reduces the downtime required to upgrade the production database. Test the additive schema changes feature to verify that it does not adversely affect the operation of the application.

                                                                                      Perform this process in the production test environment. Do not perform this process in the production environment.

                                                                                      Note: Links to other topics in this guide are provided throughout this chapter. These links are meant to provide additional information that cannot be provided in these brief overviews. You must, however, perform the upgrade in the proper order. For information on general planning, see this chapter as well as Siebel Database Upgrade Planning and Application Planning for a Siebel Upgrade To begin the process of upgrading your Siebel database, see Preparing for Siebel Database Upgrade then refer to each subsequent chapter.

                                                                                      Review the following upgrade planning and performance tuning resources before performing this process:

                                                                                      • About Siebel Additive Schema Changes Mode

                                                                                      • 478308.1 (Article ID) on My Oracle Support. This document was previously published as Siebel Technical Note 616. This article describes the strategies for minimizing production environment downtime during an upgrade. The steps that follow are intended primarily for use with the baseline guidelines described in 478308.1 (Article ID).

                                                                                      • Tuning the Siebel Upgrade Files This chapter provides detailed information on how to use Upgrade Tuner.

                                                                                      UNIX deployments. You must have a Siebel Server installed on a Windows host to run the Upgrade Tuner. To obtain a Windows version of Siebel Server, contact your account manager or Oracle Global Customer Support.

                                                                                      Caution: Do not use this topic to tune upgrade performance on IBM z/OS. Instead, refer to Siebel Database Upgrade Guide for DB2 for z/OS.

                                                                                        Set Up the Target Database

                                                                                        1. Back up and remove the upgraded production test database.

                                                                                          This preserves seed data and metadata changes you have made that must be migrated to the production environment. This database is called the production-test final database.

                                                                                        2. In the production test environment, install a recent backup of your production database.

                                                                                          This database has not been upgraded and is called the target database. You will use it to perform test upgrades as part of tuning upgrade performance.

                                                                                        3. Define an ODBC connection to the target database.

                                                                                        4. Verify that the target database and RDBMS server are configured for optimum upgrade performance:

                                                                                        5. Preparing Siebel Tables and Views for Upgrade.

                                                                                        6. Run statistics on the target database. This optimizes query performance.

                                                                                        7. UNIX. Securing AIX Memory Allocation Segment Space for the Siebel Database.

                                                                                        8. Perform the tasks in Preparing Siebel Application Data for Upgrade

                                                                                          Test Additive Schema Changes

                                                                                          If you do not plan to run additive schema changes on the production database in the production environment, then you do not need to perform the steps in this topic.

                                                                                          Perform these steps on the target database.

                                                                                          1. Review About Siebel Additive Schema Changes Mode.

                                                                                          2. (Optional) Changing the Siebel Database Configuration Utilities Language.

                                                                                          3. In the production test environment, run the Database Configuration Wizard:

                                                                                          4. Choose Apply Additive Schema Changes mode.

                                                                                            The Apply Additive Schema Changes mode identifies changes that can be made to the target database without taking it offline.

                                                                                            Windows. You will not be prompted whether you want to run the Upgrade Wizard. Instead, the Upgrade Wizard starts automatically, and creates the schema.additive.sql script. The Upgrade Wizard does not run the script against the target database.

                                                                                          5. UNIX. Run the Siebel Upgrade Wizard.

                                                                                            See Starting the Siebel Upgrade Wizard.

                                                                                            Specify master_additive_gen.ucf as the input file.

                                                                                            The Upgrade Wizard generates the schema.additive.sql script. It does not run the script against the target database.

                                                                                          6. Run the SQL script against the target database.

                                                                                            See Applying Siebel Additive Schema Changes.

                                                                                          7. Thoroughly review the operation of all applications. Verify that applying the additive schema changes has not adversely affected the operation of all applications.

                                                                                          8. If applying additive schema changes adversely affects the operation of the applications, then do the following:

                                                                                            1. For help diagnosing the cause of the problem and help in taking corrective action create a service request (SR) on My Oracle Support. Alternatively, you can phone Global Customer Support directly to create a service request or get a status update on your current SR. Support phone numbers remain the same and are listed on My Oracle Support.

                                                                                            2. Restore the target database from backup before proceeding with further upgrade tuning.

                                                                                            3. Do not use the additive schema changes feature during your production upgrade.

                                                                                          9. If applications function normally, then continue with upgrade tuning. Consider using the additive schema changes feature during your production upgrade.

                                                                                            Upgrade the Target Database (upgrep and upgphys)

                                                                                            1. (Optional) Changing the Siebel Database Configuration Utilities Language.

                                                                                            2. In the production test environment, run the Database Configuration Wizard:

                                                                                            3. In the production test environment, run Siebel Upgrade Wizard.

                                                                                              See Starting the Siebel Upgrade Wizard.

                                                                                              Note the time required to upgrade the database.

                                                                                            4. Review the upgrade log files for errors: Reviewing Siebel Upgrade Log Files for Errors.

                                                                                            5. If the upgrade contains errors that prevented completion or adversely affected performance, then correct the errors and rerun the upgrade.

                                                                                            6. Manually Archiving Siebel Upgrade Log Files.

                                                                                              Tune the Upgrade Files

                                                                                              1. Evaluate upgrade performance, particularly the time required to complete the upgrade.

                                                                                              2. Do one of the following:

                                                                                                • If the time required to complete the upgrade is acceptable, then no further tuning is needed. Perform the steps in Restore the Production-Test Final Database.

                                                                                                • If the time required to complete the upgrade is too long, then perform the remaining steps in this topic to continue tuning upgrade performance.

                                                                                                • If the time required to complete the upgrade is too long and you cannot tune further, then contact your Oracle sales representative for Oracle Advanced Customer Services to request assistance from Oracle’s Application Expert Services to apply advanced tuning.

                                                                                              3. Carefully review target database and RDBMS server configuration. Adjust as needed to further improve upgrade performance.

                                                                                              4. Run Upgrade Tuner to tune the upgrade files:

                                                                                                Restore the Target Database

                                                                                                Perform these steps if you have made changes to the upgrade environment or to the upgrade files and want to run the upgrade again to verify performance improvement.

                                                                                                1. In the production test environment, restore the target database from backup.

                                                                                                  This returns the target database to its non-upgraded state so that you can perform another test upgrade.

                                                                                                2. In the production test environment, apply additive schema changes again as desired.

                                                                                                3. Perform another test upgrade and evaluate upgrade performance.

                                                                                                4. Repeat the tuning process and perform test-upgrades until upgrade performance is acceptable.

                                                                                                  Restore the Production-Test Final Database

                                                                                                  Perform these steps only if you have completed tuning upgrade performance.

                                                                                                  1. In the production test environment, delete and remove the target database.

                                                                                                  2. In the production test environment, restore the production-test final database from backup.

                                                                                                    Process of Upgrading a Siebel Production Environment

                                                                                                    Platforms: Windows and UNIX only. This topic does not apply to IBM z/OS.

                                                                                                    To perform this process, you must be able to execute ODBC commands on the production database from within the production environment. For more information, see About the Siebel Database Configuration Wizard Utilities. For help executing ODBC commands within the production test environment, create a service request (SR) on My Oracle Support.

                                                                                                    Note: Links to other topics in this guide are provided throughout this chapter. These links are meant to provide additional information that cannot be provided in these brief overviews. You must, however, perform the upgrade in the proper order. For information on general planning, see this chapter as well as Siebel Database Upgrade Planning and Application Planning for a Siebel Upgrade To begin the process of upgrading your Siebel database, see Preparing for Siebel Database Upgrade then refer to each subsequent chapter.

                                                                                                    This topic lists the tasks required to transition your production test environment to production. Print this topic and use it as a checklist when doing the upgrade. Complete the steps in the order shown.

                                                                                                    Note: If you are performing a production environment upgrade on IBM z/OS, then see Siebel Database Upgrade Guide for DB2 for z/OS.

                                                                                                      Search for Articles on My Oracle Support

                                                                                                        Upgrade the Servers

                                                                                                        Verify that you have identified all of the Siebel CRM releases that are required for the upgrade. Along with the current release, Siebel 2018, also install the latest Siebel update. For more information on server upgrade and installation, see Siebel Installation Guide. See also the Certifications tab on My Oracle Support.

                                                                                                        Caution: Do not install a new Siebel database as part of upgrading the Siebel Enterprise.

                                                                                                        To perform the following steps, see the Siebel Installation Guide.

                                                                                                        1. Upgrade the Siebel Gateway, Siebel Servers, and Siebel Application Interface.

                                                                                                          For information on upgrading these Siebel Enterprise components, see Siebel Installation Guide. The modules named are for the current release.

                                                                                                        2. Install the Siebel Database Server files on the Siebel Server you will use to perform the upgrade. You only need to install the database server files for the database type that you are upgrading.

                                                                                                        3. Install the language packs for your currently deployed languages and any new languages.

                                                                                                        4. If you have customized the configuration of enterprise components, such as Siebel Servers, then you can run a script to migrate configuration parameters to the upgraded Siebel Enterprise. (Do not perform this step for a migration installation, which is described in the Siebel Installation Guide.)

                                                                                                          Upgrade Third-Party Software

                                                                                                          • Upgrade third-party software as required due to dependencies on Oracle’s Siebel software or other installed software. For example, you might need to upgrade the following software:

                                                                                                            • Operating system software. Some database upgrades require newer versions of AIX or Windows.

                                                                                                            Apply Additive Schema Changes

                                                                                                            These steps are optional. Perform these steps only after you have thoroughly tested the additive schema changes feature in the production test environment and have verified that it will not adversely affect your applications.

                                                                                                            Perform these steps on the production database. You do not need to take the production database offline. You can perform these steps anytime prior to running the production upgrep.

                                                                                                            After applying additive schema changes there is no need to run database statistics. The schema changes do not affect the performance of your release prior to the upgrade.

                                                                                                            1. Review About Siebel Additive Schema Changes Mode.

                                                                                                            2. Verify that you have a recent backup of the production database and that you are logging transactions.

                                                                                                              If you encounter problems applying additive schema changes, then you might need to restore the production database.

                                                                                                            3. (Optional) Changing the Siebel Database Configuration Utilities Language.

                                                                                                            4. In the production environment, start the Database Configuration Wizard:

                                                                                                            5. Choose Apply Additive Schema Changes mode.

                                                                                                              The Apply Additive Schema Changes mode identifies changes that can be made to the production database without taking it offline.

                                                                                                              Windows. You will not be prompted whether you want to run the Upgrade Wizard. Instead, the Upgrade Wizard starts automatically, and creates the schema.additive.sql script. The Upgrade Wizard does not run the script against the production database.

                                                                                                            6. UNIX. Run the Siebel Upgrade Wizard.

                                                                                                              See Starting the Siebel Upgrade Wizard.

                                                                                                              Specify master_additive_gen.ucf as the input file.

                                                                                                              The Upgrade Wizard generates the schema.additive.sqlscript. It does not run the script against the production database.

                                                                                                            7. Run the SQL script against the production database.

                                                                                                              See Applying Siebel Additive Schema Changes.

                                                                                                            8. Thoroughly review the operation of all applications. Verify that applying additive schema changes has not adversely affected applications function.

                                                                                                            9. If you are applying additive schema changes adversely affects the operation of the application, then do the following:

                                                                                                              1. Contact Oracle Global Support for help in diagnosing and correcting the problem.

                                                                                                              2. If required, restore the production database from backup.

                                                                                                              Upgrade the RDBMS

                                                                                                              • If required, upgrade the RDBMS version. Refer to the vendor’s documentation to perform the upgrade:

                                                                                                                Perform Preupgrade Tasks for the Siebel Database

                                                                                                                These steps apply to all database types.

                                                                                                                1. Review guidelines for configuring the RDBMS. See the Siebel Installation Guide.

                                                                                                                2. Verify that the Workflow Monitor and Workflow action agents have processed all pending requests.

                                                                                                                3. Stop the Siebel Servers.

                                                                                                                4. Verify that the Siebel Gateway is still running.

                                                                                                                5. Stop all other Siebel Servers.

                                                                                                                6. Close all database connections. The only open connection must be the account performing the upgrade.

                                                                                                                7. Perform the tasks in Preparing for Siebel Database Upgrade

                                                                                                                  Perform Preupgrade Tasks for IBM DB2

                                                                                                                  1. Perform the tasks in Preparing an IBM DB2 Database for a Siebel Upgrade

                                                                                                                  2. Make sure that all transactional tables with clustering indexes are at least 90% clustered and stats are current.

                                                                                                                    Note: You might be required to perform runstats on a small number of transactional tables and indexes following a successful upgrade. All Siebel Enterprise Integration Manager tables will have been replaced and might need statistics gathered.

                                                                                                                    Perform Preupgrade Tasks for Oracle Database

                                                                                                                    1. Perform the tasks in Preparing an Oracle Database for a Siebel Upgrade

                                                                                                                    2. Run the Oracle Analyze command on the Siebel database. Highly fragmented indexes can cause the upgrade to fail.

                                                                                                                      Perform Preupgrade Tasks for Microsoft SQL Server

                                                                                                                      1. Perform the tasks in Preparing a Microsoft SQL Server Database for a Siebel Upgrade

                                                                                                                      2. Run Microsoft SQL Server statistics. This will improve upgrade performance.

                                                                                                                        Perform Preupgrade Tasks for Application Data

                                                                                                                        1. Perform the tasks in 477519.1 (Article ID) on My Oracle Support. This document was previously published as Siebel Technical Note 521. The following table shows the applicability of the tasks in the article.

                                                                                                                          Item Environment

                                                                                                                          Location and amendment of S_VIEW_WTMPL_IT.ITEM_NUM records

                                                                                                                          Development, production test, production

                                                                                                                          Update multi-value group (MVG) and association list applets

                                                                                                                          Development only

                                                                                                                          Update unnamed controls in applet Web template items

                                                                                                                          Development only

                                                                                                                        2. Perform the tasks in Preparing Siebel Application Data for Upgrade

                                                                                                                          Perform Preupgrade Tasks for the User Interface

                                                                                                                          • Copy application files to the environment:

                                                                                                                            • Reports files.

                                                                                                                            Upgrade the Siebel Database (upgrep and upgphys)

                                                                                                                            You do not run the upgrep and upgphys portions of the upgrade in the production environment. Instead, the numbered steps in this topic implement the following process:

                                                                                                                            • Run the Database Configuration Wizard in the production environment.

                                                                                                                            • In the utilities, enter the information for the production environment instead of the production test environment. For example, you enter the ODBC connection for the production environment.

                                                                                                                              This information configures the driver file to run against the production database rather than the production test database. It also configures the driver file to use the upgrade SQL files you generated for the production test upgrade.

                                                                                                                            • Run the Upgrade Wizard. The Upgrade Wizard uses the SQL files in the production test environment to upgrade the database in the production environment.

                                                                                                                              This approach has several advantages:

                                                                                                                            • You do not have to generate upgrade SQL files in the production environment, and then manually transfer the customizations to them from the production test environment.

                                                                                                                            • You do not lose any changes to the SQL files that were made by Siebel Upgrade Tuner in the production test environment.

                                                                                                                            • You do not have to run the Database Configuration Wizard in Prepare for Production mode again.

                                                                                                                            • With some exceptions, you do not have to perform again the database-related configuration tasks required by Siebel Release Notes on My Oracle Support or Articles on My Oracle Support.

                                                                                                                            If your network configuration prevents creating an ODBC connection to your production database from inside your production test environment, then contact Oracle Global Customer Support for assistance in completing the production upgrade.

                                                                                                                            1. Verify that you have a current backup of the production environment database.

                                                                                                                            2. On the Siebel Server you used to upgrade the production test environment, create an ODBC to connect to the production environment database.

                                                                                                                            3. Navigate to DBSRVR_ROOT\common (UNIX: DBSRVR_ROOT/common) and verify that the file sqlgen.usg exists.

                                                                                                                              This file contains a record of when the SQL generator was run. When you run the Database Configuration Wizard, if this file exists, then no SQL commands are generated.

                                                                                                                              Caution: If this file does not exist, then do not run the Database Configuration Wizard. The Database Configuration Wizard will overwrite the SQL files used to upgrade your production test database. Contact Oracle Global Customer Support for guidance on proceeding with the upgrade.

                                                                                                                              You do not have to run the Database Configuration Wizard in Prepare for Production Mode.

                                                                                                                            4. (Optional) Changing the Siebel Database Configuration Utilities Language.

                                                                                                                            5. Run the Database Configuration Wizard:

                                                                                                                              1. Choose the following settings when you run the utility:
                                                                                                                                • Upgrade Options. Upgrade Siebel Database Schema (upgrep and upgphys)

                                                                                                                                • Environment Type. Production

                                                                                                                              2. Enter the information for the production environment instead of the production test environment.

                                                                                                                              3. Enter the name of the ODBC for connecting to the production database.

                                                                                                                              4. When prompted whether you want to run the Siebel Upgrade Wizard, answer No and exit.

                                                                                                                                This updates the master UCF file with the production environment configuration. When you run the Siebel Upgrade Wizard, the SQL commands will be executed on the production environment database.

                                                                                                                            6. In the production test environment, verify that the SQL scripts for performing the upgrade were not overwritten. You can do this by checking the modification times. If the scripts were overwritten, then do not continue. Instead, contact your Oracle sales representative for Oracle Advanced Customer Services to request assistance from Oracle’s Application Expert Services.

                                                                                                                            7. Perform the production database upgrade. See Starting the Siebel Upgrade Wizard.

                                                                                                                              The Siebel Upgrade Wizard uses the SQL commands generated for the production test environment to upgrade the production environment. If you used Upgrade Tuner to revise the SQL commands, then these changes are included.

                                                                                                                            8. Review the upgrade logs and resolve errors: Reviewing Siebel Upgrade Log Files for Errors.

                                                                                                                            9. If the upgrade contains unacceptable errors, then do the following:

                                                                                                                              1. Restore the backup of the database.

                                                                                                                              2. Correct the errors.

                                                                                                                              3. Rerun the Siebel Upgrade Wizard.

                                                                                                                            10. Manually Archiving Siebel Upgrade Log Files.

                                                                                                                            11. Use the Siebel Application Deployment Manager to migrate administrative data such as LOVs and responsibilities from production test to production.

                                                                                                                            12. Back up the upgraded production environment database.

                                                                                                                              Perform Postupgrade Tasks for Database and File System

                                                                                                                              1. Perform the tasks in Postupgrade Tasks for the Siebel Database

                                                                                                                              2. Reset upgrade-specific database and database server parameters back to their recommended settings for production. See Siebel Installation Guide for recommended parameter settings.

                                                                                                                              3. If you exported data from interface tables before the upgrade, then review the database and import the data as desired.

                                                                                                                              4. Generate a Siebel Remote database template file. See Siebel Remote and Replication Manager Administration Guide.

                                                                                                                              5. Run database statistics.

                                                                                                                              Note: The upgrade is complete. The remaining topics deal with configuration and validation tasks.

                                                                                                                                Perform Postupgrade Tasks for Applications Configuration

                                                                                                                                1. If applicable, review the results of the Person and Organization merge. Make configuration changes as required.

                                                                                                                                2. Perform the tasks in Postupgrade Tasks for Siebel Business Applications

                                                                                                                                3. Verify the function of interfaces in integrated applications.

                                                                                                                                4. Activate and deploy workflows.

                                                                                                                                5. If you have set up integrations for transferring data to or from third-party applications using Siebel Enterprise Application Integration, then verify that the integrations are configured correctly. For information on Siebel EAI, see Overview: Siebel Enterprise Application Integration.

                                                                                                                                6. If you have used Siebel Enterprise Integration Manager to set up batch processing jobs, then verify Siebel Enterprise Integration Manager is configured correctly. For information on Siebel Enterprise Integration Manager, see Siebel Enterprise Integration Manager Administration Guide.

                                                                                                                                7. If you customized the style sheet or Web template files in the previous release, then reimplement those customizations in the new release as desired.

                                                                                                                                  Perform System Tests

                                                                                                                                  • Use available test data to perform unit testing. Validate the operation of the application in the following areas:

                                                                                                                                    • User interface

                                                                                                                                    • Data interfaces

                                                                                                                                    • Integrity of migrated data

                                                                                                                                    • Workflow function