A.16 Migrating the Data from ElasticSearch to OpenSearch

Prerequisites:

• OpenSearch should be installed successfully and that service should be up and running.
• Wallet should be configured with Entity Resolution details.

  To configure OpenSearch, see Configure the OpenSearch Component section in the OFS Compliance Studio Installation Guide.

• Execute the following command for health check API of the OpenSearch:

  curl -X GET '<OPENSEARCH_CLUSTER_HOST>:<PORT_NUMBER>/_cat/health'

  or

  curl -X GET '<OPENSEARCH_CLUSTER_HOST>:<PORT_NUMBER>/_cat/health?v'

  Sample output:

  1675934006 09:13:26 <OPENSEARCH_CLUSTER_NAME> green 1 1 true 0 0 0 0 0 0
  - 100.0%

• To verify the health check API in the browser, navigate to the following URL:

  https://<OPENSEARCH_CLUSTER_HOST>:<PORT_NUMBER>/_cat/health?v

  Note:

  If https is not configured then use the following URL:

  http://<OPENSEARCH_CLUSTER_HOST>:<PORT_NUMBER>/_cat/
  health?v

To migrate data from ElasticSearch to OpenSearch, see OpenSearch documentation.

Migrating data for 'csa_stg_party_812' from ElasticSearch to OpenSearch, follow these steps:

1. Use the following curl command to load index 'csa_stg_party_812':

   Note:

   The following parameters to be configured as follows:

   • <SCHEMA-NAME> to be replaced with ER schema configured in the wallet.
   • <load_to_opensearch_service_port_number> to be replaced with default value 7053.
   • <FQDN_Compliance_Studio> to be replaced with fully qualified domain name of the Compliance Studio.

   curl -X POST 'http://
   <FQDN_Compliance_Studio>:<load_to_opensearch_service_port_number>/loadto-
   open-search/idx/createIndex' \
   -H 'Content-Type: application/json' \
   -d '{
   "schemaName": "<SCHEMA-NAME>",
   "tableName": "FCC_ER_FULL",
   "filterCondition": "1=1",
   "indexName": "stg_party_812",
   "indexAlias": "csa_812_alias",
   "indexLogicalName": "csa_stg_party_812",
   "indexBusinessName": "csa_stg_party_812",
   "indexKeyAttribute": "original_id",
   "loadType": "FullLoad",
   "shards": 1,
   "replicas": 3,
   "attributes": [
   {
   "name": "address",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "address",
   "fields": []
   },
   {
   "name": "business_domain",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "Organization",
   "fields": []
   },
   {
   "name": "city",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "address",
   "fields": []
   },
   {
   "name": "country",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "address",
   "fields": []
   },
   {
   "name": "given_name",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "namestop",
   "fields": []
   },
   {
   "name": "middle_name",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "namestop",
   "fields": []
   },
   {
   "name": "family_name",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "namestop",
   "fields": []
   },
   {
   "name": "concat_name",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "namestop",
   "fields": []
   },
   {
   "name": "alias",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "namestop",
   "fields": []
   },
   {
   "name": "state",
   "type": "text",
   "similarity": "boolean",
   "analyzerType": "address",
   "fields": []
   }
   ],
   "customAnalyzer": [],
   "customFilter": [],
   "customCharFilter": [],
   "customTokenizer": [],
   "others": [
   "original_id",
   "orgname",
   "dob",
   "source_name",
   "start_date",
   "jurisdiction",
   "industry",
   "naics_code",
   "tax_id",
   "doc_id",
   "email",
   "phone",
   "postal_code",
   "incorporation_date",
   "entity_type"
   ],
   "replaceCharFields": [
   {
   "name": "address",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "city",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "country",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "state",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "given_name",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "middle_name",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "family_name",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "concat_name",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   },
   {
   "name": "alias",
   "charArray": [";", "~"],
   "replaceWith": [",", ";"]
   }
   ],
   "replaceEmptyFields": [],
   "translateFields":
   ["middle_name","family_name","concat_name","alias","given_name",
   "address", "city", "country", "state"]
   }'

   After the successful execution, you will get the following response:

   {"STATUS":"SUCCESS","MESSAGE":"Index created and loaded
   successfully.","COUNT":<count of records loaded>}

2. Verify that the index is migrated from elastic search to OpenSearch by navigating the following URL:

   http://<OPENSEARCH_CLUSTER_HOST>:<PORT_NUMBER>/_cat/indices

   The sample output is as follows:

   open stg_party_812 E09Y31W_SBiZGlZjbX5zZA 1 3 346 4 521.4kb 521.4kb