1. Migrate to Oracle Mobile Hub
  2. The Migration Tool

2 The Migration Tool

The migration tool is supported on Linux and Mac.

The migration tool is different for OMCe and for MCS. Download the appropriate one from the Mobile Downloads page athttps://www.oracle.com/downloads/cloud/mobile-cloud-service-downloads.html. Save the zip to a local location and unzip it.

The migration tool has four migration files:

  • config.json This is the file which you edit to specify details of the source instance, the storage account, and the target Mobile Hub.

  • Three files which you do not edit:

    • Dockerfile

    • functions.py

    • migrate.py

The information you use in the configuration file is used to:

  • Generate export bundles which are exported to the storage instance.
  • import the generated bundles from the storage instance to the Mobile Hub instance.

The OMCe migration tool also contains patch scripts that you use to patch the OMCe instance before performing the migration:

  • patch_OMCE-6385_OMCe18.2.1.sh patch script for OMCe 18.2.1

  • patch_OMCE-6385_OMCe18.2.2.sh patch script for OMCe 18.2.2

  • patch_OMCE-6530_OMCe18.2.3.sh patch script for OMCe 18.2.3

  • patch_OMCE-6529_OMCe18.2.5.sh patch script for OMCe 18.2.5

Patch OMCe

If you are migrating from OMCe to Mobile Hub you need to patch the OMCe instance using a patch script.

If you are migrating from MCS ignore this section and go to About the Configuration File.

Choose the appropriate patch script for the your OMCe instance:

  • Edit the script so that it can access the admin server using WLST
  • Save the patch script to the Core VM
  • Run the patch script

You can find the OMCe version by clicking the user icon, and then select About.

Edit the Patch Script

Edit the patch script to add the Core Instance WLS admin server credentials.

##### for APPLY operation
export WLS_ADMIN_USER="<OMCe admin user>"
export WLS_ADMIN_PASSWORD="<password for OMCe admin user>"
export WLS_ADMIN_SERVER_HOST="<WLS admin server host>"
export WLS_ADMIN_SERVER_PORT="7001"

Save the patch file with the original name.

The WLS credentials are Administrator Username and Administrator Password you used when you first provisioned OMCe. See Set Up the OMCe Service in Administering Oracle Mobile Cloud Enterprise in a Customer-Managed Environment. The default admin username is omceadmin.

Alternatively, you can try to retrieve the credentials.

  1. Find the Core VM’s IP address.

    1. From OMCe Stack Overview, click the resource with the name <stack>CORE.

    2. In the Service Overview for the Mobile Core POD Details, expand Resources and note the Public IP, for example 192.0.2.1.

  2. Connect to the Core VM using ssh and change to the oracle user. You can log in as the default user, opc. The opc user has sudo privileges.

    1. If you don't know the private key you can set a new one. See Add a New SSH Public Key in Administering Oracle Mobile Cloud Enterprise in a Customer-Managed Environment.

      To make the private key not user-readable, use:

      chmod 600 <private_key_file>

    2. Log in using:

      $ ssh -i <private_key_file> opc@<Core IP address>

      For example, ssh -i my_private_rsa_keyfile opc@192.0.2.1

    3. Switch to the oracle user using:

      $ sudo su - oracle

  3. Run the following command:

    [oracle@spomcebcore-wls-1 temp]$ cat /u01/data/domains/mobile/temp/WLSTLauncher.pod.postcreate.output.json | python -c 'import json,sys;obj = 
json.load(sys.stdin); print obj["SM_SENSITIVE_DATA"]["encodedString"]' | base64 -d | python -c 'import json,sys;obj =
json.load(sys.stdin); print json.dumps(obj["mobileWLSAdminCred_instance"])' | python -m json.tool

    Sample result:

    {
    "name": "omceadmin",
    "password": "Welc0me#",
    "type": "USERPASS"
}

Save the Patch Script to the Core VM

Connect to the Core VM using ssh.

  1. Find the Core VM’s IP address.

    1. From OMCe Stack Overview, click the resource with the name <stack>CORE.

    2. In the Service Overview for the Mobile Core POD Details, expand Resources and note the Public IP, for example 192.0.2.1.

  2. Connect to the Core VM using ssh and change to the oracle user. You can log in as the default user, opc. The opc user has sudo privileges.

    1. Use:

      $ ssh -i <private_key_file> opc@<Core IP address>

      For example, ssh -i my_private_rsa_keyfile opc@192.0.2.1

    2. Switch to the oracle user using:

      $ sudo su - oracle

  3. Give exec permissions for current user, and then you can run the patch script.

Alternatively, you can upload the script file to Core VM using the scp tool and using same <private_key_file> and<Core IP address> as for ssh. 

$ scp -i <private_key_file> <path to local script>
      opc@<Core IP address>:<path to upload script>

Run the Patch Script

The patch scripts support these operations:

  • FIND determines whether the patch has been applied or not
  • APPLY applies the patch
  • ROLLBACK rollbacks the applied patch

Note:

The examples use patch_OMCE-6529_OMCe18.2.5.sh. Make sure you use the correct patch script name when you run the script.

Running the Patch Script

  1. Verify that the patch hasn't already been applied. 
    ./patch_OMCE-6529_OMCe18.2.5.sh FIND

    Expected results:

    • FOUND the patch has been applied to the vm, executing APPLY would result in noop
    • NOT FOUND the patch not found on the vm, executing ROLLBACK (if defined) would be noop
    • HALF_FOUND executing both APPLY and ROLLBACK (if defined) would change the state of the vm
    • ERROR something has gone wrong. See Troubleshoot Patch Issues.

    FIND is default operation that will be started if you run patching script without specifying operation parameter.

  2. Apply the patch.
    ./patch_OMCE-6529_OMCe18.2.5.sh APPLY

    The operation performs these steps:

    • Find operation. Patch application stops if the patch has been already applied
    • Backup. Every APPLY operation creates a copy of artifacts to be patched in backup folder:
      • OMCe 18.2.1 /u01/app/oracle/crs/patch/OMCE-6385/data/backup
      • OMCe 18.2.2 /u01/app/oracle/crs/patch/OMCE-6385/data/backup
      • OMCe 18.2.3 /u01/app/oracle/crs/patch/OMCE-6530/data/backup
      • OMCe 18.2.5 /u01/app/oracle/crs/patch/OMCE-6529/data/backup

      If backup folder already exists, it is renamed to backup_<current_date_time>, for example backup_2019.08.02-16.46.55

      Expected results:

      • FOUND the patch has been already applied earlier
      • APPLIED the operation has applied the patch
      • ERROR something has gone wrong. See Troubleshoot Patch Issues.

Rolling Back the Patch Script

Rollback is invoked automatically if patch application fails. You can call it manually if the patch was applied successfully but something went wrong.

By default, rollback restores artifacts from the backup directory created by the last APPLY operation.

If you need to rollback the patch, use

./patch_OMCE-6529_OMCe18.2.5.sh ROLLBACK

If you have to restore from older backup, provide the backup folder name in patch script:

##### for ROLLBACK operation
# backup directory to restore patched artifacts from
# absolute path or directory name for auto-created backup (backups created in e.g. /u01/app/oracle/crs/patch/OMCE-6529/data/)
# defaultvalue will backup from the last created backup
export ROLLBACK_FROM_FOLDER=backup

Expected results:

  • NOT_FOUND the patch has not been applied, nothing to do
  • ROLLED_BACK the operation has rolled back the patch
  • ERROR something has gone wrong. See Troubleshoot Patch Issues.

Restart the Core VM

In the Stack Details page of the Stack Manager, locate the stack you want to start and select Start.

Troubleshoot Patch Issues

If one of the patch operations has returned an error, review the operation logs for operation detailed messages.

The operation logs are appended to the log file stored in the current patch data directory:

  • OMCe 18.2.1 /u01/app/oracle/crs/patch/OMCE-6385/log.txt
  • OMCe 18.2.2 /u01/app/oracle/crs/patch/OMCE-6385/log.txt
  • OMCe 18.2.3 /u01/app/oracle/crs/patch/OMCE-6530/log.txt
  • OMCe 18.2.5 /u01/app/oracle/crs/patch/OMCE-6529/log.txt

The log file contains everything output by the apply, rollback, and find operations, with a line between the output for each operation.

About the Configuration File

config.json is a configuration file that you update with details of:

  • Scope (Optional). Defines set of data to be migrated. If you are trying to identify problems with a test migration it can be useful to restrict the scope.

  • Source Instance Details of the MCS instance you are exporting the bundles from.

  • Target Instance Details of the Mobile Hub instance that you are importing the bundles to.

  • Storage Account Details of the object storage you use for migration.

  • Security (Optional) A security key to encrypt migrated security data.

This is a sample config.json for OMCe to Mobile Hub. There are some differences between the migration script for OMCe and for MCS so it's important to download and edit the correct version of the migration tool.

{
  "//": "defines set of data to be migrated. migrate: true/false value used to mark data types to be exported. true is default.",
  "scope": {
    "metadata": {
      "//": "migrate artifacts metadata",
      "migrate": true¸
      "//": "exclude clients by name or name+version",
      "exclude": ["<client1_name>", "<client2_name>(<client2_version>)"],
      "//": "migrate only default implementations. Default = false",
      "onlyDefaultImpl": false¸
      "//": "limit number of implementations stored per zip entry. Default = 0 means default limit",
      "implPageSize": 0
    },
    "policies": {
      "//": "migrate policies",
      "migrate": true
    },
    "collections": {
      "//": "migrate collection objects. Requires collection metadata to be also migrated",
      "migrate": true,
      "//": "do not migrate objects for collections with given names",
      "exclude": ["collectionName1", "collectionName2"],
      "//": "migrate only collection objects updated after specified date. Supported values `-5s`, `-5m`, `-5h` or date format `yyyy-MM-dd'T'HH:mm:ss.SSS'Z'`.",
      "fromTime": "2019-09-03T00:00:00.000-07:00",
      "//": "If collection table in target instance already exists, merge new objects into existing table, skipping duplicates (true) or report  conflict and stop (false).",
      "merge": true
    },
    "database": {
      "//": "migrate Database Management Service data: tables, indexes, constraints",
      "migrate": false,
      "//": "optional attribute to list tables to be migrated",
      "tables": ["tableName1", "tableName2"],
      "//": "optional attribute to list tables to be excluded from migration",
      "exclude": ["APEX$TEAM_DEV_FILES"]
    },
    "location": {
      "//": "migrate location data: Assets, Places, Devices",
      "migrate": true
    },
    "security": {
      "//": "migrate security data",
      "migrate": false,
      "//": "include Notification Profiles security data",
      "profiles": false
    },
    "analytics": {
      "//": "migrate analytics data. Disabled by default",
      "migrate": false
    }
 },
  "//": "OMCe instance to export data from",
  "source_instance": {
    "instance_url": "http://foo.oraclecloud.com:7777",
    "token_url": "https://foo.storage.oraclecloud.com/auth/v1/token",
    "client_id": "abcde12345abcde12345abcde12345ab",
    "client_secret": "abcde123-abcd-1234-1245-abcde12345ab",                 
    "username": "user1",
    "password": "password"
  },
  "//": "OMH instance to import data to",
  "target_instance": {
    "instance_url": "http://foo.oraclecloud.com:7777",
    "token_url": "https://foo.storage.oraclecloud.com/auth/v1/token",
    "client_id": "abcde12345abcde12345abcde12345ab",
    "client_secret": "abcde123-abcd-1234-1245-abcde12345ab",
    "username": "user1",
    "password": "password"
  },
  "//": "storage account, that is used during the migration",
  "storage_account": {
    "X-Storage-BaseUrl": "https://foo.storage.oraclecloud.com",
    "X-Storage-Account": "myservice",
    "X-Storage-Tenant": "myservice",
    "X-Storage-Container": "Migration",
    "X-Storage-User": "Storage user",
    "X-Storage-Password": "Storage password"
  },
  "//": "additional security configuration"
  "security": {
    "//": "Optional user-defined password to enable security data encryption",
    "X-Data-EncryptionKey": "UserPassword"
  }
}

Configure config.json for MCS

Before running the migration tool, you have to update the config.json with the appropriate information for your source MCS instance, the target Mobile Hub, and the storage instance used by the migration process.

Migration Scope

By default, migration is enabled for all data types except analytics data. You can define the set of data to be migrated by choosing to enable or disable each data type using true or false, as shown in this example. This can be useful when you are trying to isolate an issue that is causing a test migration to fail. 

{
  "//": "defines set of data to be migrated. migrate: true/false value used to mark data types to
        be exported. true is default.",
  "scope": {
    "metadata": {
      "//": "migrate artifacts metadata",
      "migrate": true
    },
    "policies": {
      "//": "migrate policies",
      "migrate": true
    },
    "collections": {
      "//": "migrate collection objects. Requires collection metadata to be also migrated",
      "migrate": true,
      "//": "Optional attribute. Do not migrate
        objects for collections with given names",
      "exclude": ["collectionName1", "collectionName2"]
    },
    "database": {
      "//": "migrate Database Management Service data: tables, indexes, constraints",
      "migrate": false,
      "//": "optional attribute to list tables to be migrated",
      "tables": ["tableName1", "tableName2"],
      "//": "optional attribute to list tables to be excluded from migration",
      "exclude": ["APEX$TEAM_DEV_FILES"]
    },
    "location": {
      "//": "migrate location data: Assets, Places, Devices",
      "migrate": true
    },
    "security": {
      "//": "migrate security data",
      "migrate": false,      "//":
        "include Notification Profiles security data",
      "profiles": false
    },
    "analytics": {
      "//": "migrate analytics data. Disabled by default",
      "migrate": false
    }
 },

Source Instance

The section of config.json where you enter details of the MCS instance. 

  "//": "MCS instance to export data from",
    "source_instance": {
    "instance_url": "<YOUR_BASE_URL>",
    "token_url": "<YOUR_OAUTH_TOKEN_ENDPOINT>",
    "client_id": "<YOUR_CLIENT_ID>",
    "client_secret": "<YOUR_CLIENT_SECRET>",
    "username": "<YOUR_MCS_USERNAME>",
    "password": "<YOUR_MCS_PASSWORD>",
    "identity_domain": "<YOUR_IDENTITY_DOMAIN>"
  }

  1. In MCS, open Applications > Backends, then open any Mobile Backend.

  2. Select Settings.

  3. Under Environment URLs, use the Base URL value for instance_url and the OAuth Token Endpoint value for token_url.

  4. Under Access Keys under OAuth Consumer, use the Client ID value for client_id and the Client Secret value for client_secret.

Target Instance

The section of config.json where you enter details of the Mobile Hub instance. 

  "//": "OMH instance to import data to",
    "target_instance": {
    "instance_url": "<YOUR_BASE_URL>",
    "token_url": "<YOUR_OAUTH_TOKEN_ENDPOINT>",
    "client_id": "<TEAM_MEMBER_APP_CLIENT_ID>",
    "client_secret": "<TEAM_MEMBER_APP_CLIENT_SECRET>",
    "username": "<YOUR_MOBILE_HUB_USERNAME>",
    "password": "<YOUR_MOBILE_HUB_PASSWORD>"
  }

  1. To find the Team Member App Client ID and Team Member App Client Secret , in Mobile Hub open Development and click on Instance Details.

  2. To find the instance_url and token_url:

    1. In Mobile Hub, open Development > Backend.

    2. Select Settings.

    3. Under Environment URLs find the Base URL value which you use for instance_url.

      Find the OAuth Token Endpoint value which you use for token_url.

Storage Account

The section of config.json where you enter details of the Object Storage or Storage Classic instance you are using. 

  "//": "storage account, that is used during the migration",
    "storage_account": {
    "X-Storage-BaseUrl": "<OBJECT_STORAGE_URL>",
    "X-Storage-Account": "<STORAGE_ACCOUNT>",
    "X-Storage-Tenant": "<TENANT_NAME>",
    "X-Storage-Container": "<CONTAINER_NAME>",
    "X-Storage-User": "<STORAGE_USERNAME>",
    "X-Storage-Password": "<STORAGE_PASSWORD>"
  }

Using Object Storage

Note:

The Storage Swift password is referenced in Oracle Cloud Infrastructure (OCI) as an access token. See Working with Auth Tokens.

If the Auth Token contains < you must generate a new token. Auth Tokens containing < are rejected by the security layer.

  1. Login to OCI Console.

  2. In the upper right choose the region you use for Object Storage, for example, us-ashburn-1.

  3. The value for X-Storage-BaseUrl is the hostname of the Swift API for the region you have chosen. Go to API Reference and Endpoints and expand Object Storage and Archive Storage then Swift API. Use the hostname for the region you have used, e.g. https://swiftobjectstorage.us-ashburn-1.oraclecloud.com.

  4. Open the User menu and choose Tenancy. From the Tenancy Information tab, set X-Storage-Tenant to be Name.

  5. Set X-Storage-Account to be Object Storage Namespace.

  6. From the Menu (upper left) choose Object Storage > Object Storage. Either use an existing bucket, or create a new bucket. Set X-Storage-Container to be Name for the bucket.

If login to OCI Object Storage using the Swift API fails with 401 then you may need to add policies so that the user accessing the bucket is part of the group which has a policy defined to access that bucket or all objects in the tenancy.

Using Storage Classic

  1. Go to My Services and then to Oracle Cloud Infrastructure Object Storage Classic.

    Go to Account and open the Account Information page.

  2. In Account Information find the Rest Endpoint. This has the format <BaseUrl>/v1/<account>.

    The X-Storage-BaseUrl is the <BaseUrl>.

    X-Account and X-Storage-Tenant are both the same and are <account>.

  3. X-Storage-Container is the Container value. The container will be created if it does not exist.

Security

Optionally use a password to encrypt security data during migration.

  "//": "additional security configuration"
  "security": {
    "//": "Optional user-defined password to enable security data encryption",
    "X-Data-EncryptionKey": "UserPassword",
}

Configure config.json for OMCe

Before running the migration tool, you have to update the JSON configuration file with the appropriate information for your source OMCe instance, the target Mobile Hub, and the storage instance used by the migration process.

Migration Scope

By default, migration is enabled for all data types except analytics data. You can define the set of data to be migrated by choosing to enable or disable each data type using true or false, as shown in this example. This can be useful when you are trying to isolate an issue that is causing a test migration to fail. 

{
  "//": "defines set of data to be migrated. migrate: true/false value used to mark data types to
        be exported. true is default.",
  "scope": {
    "metadata": {
      "//": "migrate artifacts metadata",
      "migrate": true
    },
    "policies": {
      "//": "migrate policies",
      "migrate": true
    },
    "collections": {
      "//": "migrate collection objects. Requires collection metadata to be also migrated",
      "migrate": true,
      "//": "Optional attribute. Do not migrate
        objects for collections with given names",
      "exclude": ["collectionName1", "collectionName2"]
    },
    "database": {
      "//": "migrate Database Management Service data: tables, indexes, constraints",
      "migrate": false,
      "//": "optional attribute to list tables to be migrated",
      "tables": ["tableName1", "tableName2"],
      "//": "optional attribute to list tables to be excluded from migration",
      "exclude": ["APEX$TEAM_DEV_FILES"]
    },
    "location": {
      "//": "migrate location data: Assets, Places, Devices",
      "migrate": true
    },
    "security": {
      "//": "migrate security data",
      "migrate": false,      "//":
        "include Notification Profiles security data",
      "profiles": false
    }
 },

Source Instance

The section of config.json where you enter details of the OMCe instance. 

  "//": "OMCe instance to export data from",
    "source_instance": {
    "instance_url": "<YOUR_BASE_URL>",
    "token_url": "<YOUR_OAUTH_TOKEN_ENDPOINT>",
    "client_id": "<TEAM_MEMBER_APP_CLIENT_ID>",
    "client_secret": "<TEAM_MEMBER_APP_CLIENT_SECRET>",
    "username": "<YOUR_OMCe_USERNAME>",
    "password": "<YOUR_OMCe_PASSWORD>"
  }

  1. To find the Team Member App Client ID and Team Member App Client Secret , in OMCe open Development and click on Instance Details.

  2. To find the instance_url and token_url:

    1. In Mobile Hub, open Development > Backend.

    2. Select Settings.

    3. Under Environment URLs find the Base URL value which you use for instance_url.

      Find the OAuth Token Endpoint value which you use for token_url.

Target Instance

The section of config.json where you enter details of the Mobile Hub instance. 

  "//": "OMH instance to import data to",
    "target_instance": {
    "instance_url": "<YOUR_BASE_URL>",
    "token_url": "<YOUR_OAUTH_TOKEN_ENDPOINT>",
    "client_id": "<TEAM_MEMBER_APP_CLIENT_ID>",
    "client_secret": "<TEAM_MEMBER_APP_CLIENT_SECRET>",
    "username": "<YOUR_MOBILE_HUB_USERNAME>",
    "password": "<YOUR_MOBILE_HUB_PASSWORD>"
  }

  1. To find the Team Member App Client ID and Team Member App Client Secret , in Mobile Hub open Development and click on Instance Details.

  2. To find the instance_url and token_url:

    1. In Mobile Hub, open Development > Backend.

    2. Select Settings.

    3. Under Environment URLs find the Base URL value which you use for instance_url.

      Find the OAuth Token Endpoint value which you use for token_url.

Storage Account

The section of config.json where you enter details of the Object Storage or Storage Classic instance you are using. 

  "//": "storage account, that is used during the migration",
    "storage_account": {
    "X-Storage-BaseUrl": "<OBJECT_STORAGE_URL>",
    "X-Storage-Account": "<STORAGE_ACCOUNT>",
    "X-Storage-Tenant": "<TENANT_NAME>",
    "X-Storage-Container": "<CONTAINER_NAME>",
    "X-Storage-User": "<STORAGE_USERNAME>",
    "X-Storage-Password": "<STORAGE_PASSWORD>"
  }

Using Object Storage

Note:

The Storage Swift password is referenced in Oracle Cloud Infrastructure (OCI) as an access token. See Working with Auth Tokens.

If the Auth Token contains < you must generate a new token. Auth Tokens containing < are rejected by the security layer.

  1. Login to Oracle Cloud Infrastructure (OCI) Console.

  2. In the upper right choose the region you use for Object Storage, for example, us-ashburn-1.

  3. The value for X-Storage-BaseUrl is the hostname of the Swift API for the region you have chosen. Go to API Reference and Endpoints and expand Object Storage and Archive Storage then Swift API. Use the hostname for the region you have used, e.g. https://swiftobjectstorage.us-ashburn-1.oraclecloud.com.

  4. Open the User menu and choose Tenancy. From the Tenancy Information tab, set X-Storage-Tenant to be Name.

  5. Set X-Storage-Account to be Object Storage Namespace.

  6. From the Menu (upper left) choose Object Storage > Object Storage. Either use an existing bucket, or create a new bucket. Set X-Storage-Container to be Name for the bucket.

Using Storage Classic

  1. Go to Infrastructure Classic Console and then to Oracle Cloud Infrastructure Object Storage Classic.

    Go to Account and open the Account Information page.

  2. In Account Information find the Rest Endpoint. This has the format <BaseUrl>/v1/<account>.

    The X-Storage-BaseUrl is the <BaseUrl>.

    X-Account and X-Storage-Tenant are both the same and are <account>.

  3. X-Storage-Container is the Container value. The container will be created if it does not exist.

Security

Optionally use a password to encrypt security data during migration.

  "//": "additional security configuration"
  "security": {
    "//": "Optional user-defined password to enable security data encryption",
    "X-Data-EncryptionKey": "UserPassword",
}