| Oracle® Retail Predictive Application Server Cloud Edition Administration Guide Release 19.0 F25569-31 |
|
 Previous |
 Next |
| Oracle® Retail Predictive Application Server Cloud Edition Administration Guide Release 19.0 F25569-31 |
|
Previous |
Next |
This chapter describes the administration tasks for a Planning Data Store (PDS) domain.
A newly provisioned RPAS EE CS cloud environment is set up with PDS administration tasks that a customer can use to build a PDS or check the status of the PDS. Figure 17-1 shows a list of these administrative tasks, each of which is discussed in this chapter.
The complete list of OAT tasks to manage PDS will not be shown until the domain is registered to a new PDS domain or registered to an existing PDS domain that has already been built for the application.
EE customers who do not use BDI can create their own Integration Configuration using the Configuration Tools. EE customers who uses BDI can modify the GA Integration Configuration by adding more facts, new dimensions, or new leaf dimension levels. They can also change the application (domain) names in the configuration. The GA template integration configuration is available with the relevant application's starter kit.
The integration configuration must be named <config name>_config.tar.gz and uploaded to the config folder under the incoming FTP site. The GA integration configuration is named as RGBU_RDM.xml. EE customers can use the same name, RGBU_RDM.xml, for their integration configuration.
To start the PDS build, select the task Build Template Planning Data Store in the PDS Admin Tasks group.
Figure 17-2 Build the Planning Data Store
Enter the arguments for the build PDS task.
Figure 17-3 Planning Data Store Task Arguments
Config Name
The name of the configuration that was uploaded to the config folder under the incoming FTP site. If multiple configuration archives have the with _config.tar.gz extension, the drop-down list will have multiple entries.
Partition Dimension
The partition dimension is the dimension on which the domain will be partitioned. The drop-down list displays a list of the supported options.
|
Note: Be careful regarding the partition dimension when building PDS as it will be used across multiple application. The partition dimension must be below any of the participating domain partitions. |
Overwrite the Existing PDS
If the domain has already been built once, and the implementer must rebuild the domain from scratch, you must select this option. If it is not selected, then the domain build process will stop and report an error, rather than overwrite the existing domain.
Build the PDS Using RGBU Template
If this option is selected and a configuration name is not selected from the config drop-down list, the GA APPS integration configuration will be used to build the PDS.
|
Caution: The application / domain names from this GA configuration may not match the domain name that is currently present in your environment and can lead to PDS build failure. Please download any GA APPs starter kit and check the included integration configuration for more details on the contents of the PDS configuration used with this option. |
The Patch PDS task patches the existing PDS with an updated integration configuration. Similar to the build PDS task, the patch task expects the integration configuration to be named <config name>_config.tar.gz and uploaded to the config folder under the incoming FTP site.
Enter the argument for the patch PDS task.
Figure 17-4 Patch Planning Data Store Task Argument
Dry Run Only
If this option is selected, the patch task will run without actually making any changes to the PDS. Use this option to check and detect issues with the updated integration configuration and make sure there is no PDS corruption due to a failed patch.
This task can be used to download the PDS configuration that is currently being used in the PDS. This task does not take any arguments other than the task label.
Figure 17-5 Download Planning Data Store Configuration
This section describes the following tasks:
Check PDS
List PDS Status
Load Dimension Data
Load Fact Data
PDS Manager
Synchronize Dimension Data
Synchronize Fact Data
Use the Check PDS task to obtain diagnostic information from PDS and the domain when they are integrated. The user can select one of the following operations to perform.
List all non-shared dimensions of a shared hierarchy
With this option, the system logs all non-shared dimensions of a particular hierarchy. The user must select the hierarchy to be checked in Figure 17-7.
Figure 17-7 Arguments for Non-Shared Dimensions Operation.
List all sharing hierarchies between the domain and PDS
This option is the compliment of "List all non-shared dimensions of a shared hierarchy". It lists all the sharing hierarchies. The user can optionally enable the listing of the dimension levels in the log in Figure 17-8.
Figure 17-8 List All Sharing Hierarchies
List measures of the domain that are sharing with PDS
This option lists the measures, their corresponded fact names, and the groups they belong to int the log. No additional parameter are provided for this option.
Run validation checks on a specified hierarchy
The user must provide the hierarchy for the validation to be conducted. As shown in Figure 17-9, the user can optionally provide the dimension level to further limit the validation. If the value is left blank, the operation will perform the validation on all dimensions of the hierarchy.
Figure 17-9 Validation Checks on Specified Hierarchy
Validate the partitioning between the domain and PDS
This operation validates the PDS partition to domain local domain mapping.
Validate the PDS directory structure and version
This operation validates the PDS directory structure and version to make sure it is compatible with the current domain version. It must be run after the upgrade to ensure compatibility.
Print the dimension tree of a hierarchy in PDS
It is possible for PDS to have a different dimension structure for a hierarchy than in the domain. So this operation prints the dimension tree using the PDS hierarchy information. The user is required to select the hierarchy to be printed.
Print the dimension tree of a hierarchy in the domain
Similar to the previous operation, this operation prints the dimension tree of the hierarchy in the domain.
Check the rollups of all the positions in the domain
This operation is used to detect rollup corruption at dimension/level. The user must provide the hierarchy name and, as shown in Figure 17-10, select the dimension for the validation to begin with. If the Check All option is enable, the operation will perform the same rollup validation for all the dimensions that are rollups from the selected dimension. For example, if Day is selected and Check All enabled, then, in addition to the Day dimension, Month, Quarter, Half, and so on, will also be validated.
Figure 17-10 Check Rollups for All Positions
Check the rollups of all the positions in PDS
Similar to the operation for a domain, this operation performs the same validation against the levels in PDS. The user must select the level that the validation must be started with and, optionally, perform the same validation for all the rollup levels.
The List PDS Status task does not have any addition parameter. It provides current state of PDS regarding operation and domain consistency. It has following status codes:
1001 - Hierarchy load in one of the application is in progress.
1002 - PDS dimension loading is in progress.
1004 - PDS re-partition is in progress.
1005 - PDS fact data loading is in progress.
1006 - Data are being transfer from one of the domain to PDS.
1007 - Translation loading is in progress.
1008 - One of the domains registration is in progress.
1009 - One of the domains un-registration is in progress.
2001 - There are mismatch positions between one of the applications and the PDS.
2002 - Current domain partition change requires update to PDS partition.
After the PDS is build, user must load the dimension data and/or fact data to make PDS fully operational. Dimension data can be loaded from flat files, or from Bulk Data Integration (BDI) if BDI has been installed and set up for the PDS.
Prepare the flat files and place them in the SFTP upload location under the folder rdm_input/dimdata for hier files and rdm_input/factdata for data in the database.
Load Dimension Data is used to load hierarchy information into the PDS from one or more flat files.
Options
Language for the labels. Sets the language for the labels provided in the flat file. The default is English.
Enforce new hierarchy roll-up changes. When an incoming flat file contains only partial hierarchy data, it is possible that the data has different rollup positions than the records in PDS. Use this option to enforce the rollup from the flat file and apply it to all instance in PDS. If this option is not checked, the task will fail if it detects different rollups in the incoming file and the records in PDS.
Specify the purge age in days. Use to auto purge records that have not been seen from the load file by the number of days specified in this field. If a value is not specified, the purge age defined by the configuration will be used.
Operations. Choose between loading a particular dimension (the dimension file must exist) and loading all dimensions. This allows the system to load the dimensions if the corresponded data files have been uploaded.
Use this task to load measure data into the PDS from flat files. Once present in the PDS, this data is available to any domain that shares that measure throughout PDS.
If this task skips records for any reason, a warning will be printed out to the log file that can be obtained from the online administration tools dashboard. The maximum number of skipped records that will be output to the log is 1000. After that, any further reject records will be skipped silently.
Options
Run in Trickle Feed mode. When this mode is enabled, loadFactData will use a transactional type of query for inserting and committing. Data will either all loaded to the PDS or not loaded at all; no partial load is permitted. This is intended for small data feeds only.
Do not clean input files used after load. By default, data files are removed from the source location. Enable this option to have the file(s) remain at the original location after load.
Operations
Select the facts. Select the list of facts to load.
Specify the files to load. Select which file to load.
When Oracle Bulk Data Integration (BDI) is integrated with PDS, data (hierarchies and measures in PDS) can be loaded directly from the source (for example, RMS) using BDI without involving any flat files. Figure 17-16 provides a high level description of the integration topology. (The integration is actually two-way, though the diagram shows only one-way.) For details of Oracle Bulk Data Integration (BDI), see the end user documentation for BDI.
Figure 17-17 shows the User Interface of BDI RPAS Job Admin, which can be used to manage and view batch jobs for data integration to and from RPASCE PDS.
Use PDS Manager, shown in Figure 17-19 to perform various PDS operations.
Register an existing PDS instance.
Unregister from existing PDS. Unregister the current domain from PDS.
|
Caution: Take care when un-registering from an existing PDS. If unregistered, users may have to contact Oracle cloud team to re-register it. If the user chooses to rebuild, then they do not have to contact the cloud team. |
List the PDS properties. Print the list of properties for PDS.
List current domains and status. Obtain a list of domains that are registered with the current PDS and their statuses.
List the PDS schema and repository history. Print the history and the schema.
Physically purge all dimension data. Purge all deleted dimension data and associated fact data from the PDS.
When PDS is integrated in the RPASCE platform, it is essential to keep the data synchronized between the domains and PDS.
For any hierarchy configured in PDS, all positions in the domain must be loaded into PDS. For all shared measures, as soon as the domain is registered with PDS, data must be transferred to RDM; otherwise, any operation may result in data loss.
Two tasks are specifically designed for data synchronization: Synchronize Dimension Data and Synchronize Fact Data.
PDS is a common database platform shared by multiple applications. PDS is built with pre-created web services for measure metadata and measure data. The default web services are automatically available after PDS is built. Web services map to the PDS read-only schema in the database and uses Oracle Restful Services. (Web services are read-only.)
The types of users in Apex are:
Workspace Admin - Responsible for creating workspaces and additional users. This access is only provided to the Oracle/AMS team.
RPAS_READ_ONLY_WORKSPACE Provides viewer access to the workspaces. This user cannot create additional users. These users can access the default web services. If the customer requires a user to be a rest API developer who can log into and build restful services on the PDS read-only schema, the customer must send email to the Oracle/AMS team.
The Oracle/AMS team shares the base url for the web service during provisioning to the customer. This can be accessed by a user in the IDCS or OCI IAM with the following group association: RPAS_ORDS_GROUP (for example, https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords).
Any users who are not part of the above group will not be able to access the web service. For more fine grain control, it is possible to create multiple groups and map groups to specific rest urls.
|
Note: In order to control access to the restful services to specific groups in IDCS or OCI IAM, contact your Oracle/AMS Workspace Admin. |
The default web services includes the following data:
The metadata web service exposes the following metadata for all measures in PDS: domain, measure, (mapped) fact, base intersection, and description.
The web services for measure data provide one web service for each measure in PDS. The measure data is retrieved at the measure's base intersection with position names.
Figure 17-20 shows a high level architectural diagram of PDS with ORDS. Web services are accessible from any web service client (a web browser is an example of web client) through an ORDS Listener, which is deployed on a Tomcat server. Within the database, an ORDS enabled read-only schema provides access to the necessary database objects.
All web services are available under the base path: <ords_path>/<schema_alias>, where <ords_path> is determined by the ORDS installation/configuration. In production, this value is https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords and <schema_alias> is pds.
Table 17-1 Web Services URL
| Sl Number | Web Service | Example URL | Description |
|---|---|---|---|
|
1 |
Metadata |
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measures/all/ |
Web service exposes the following metadata for all measures in PDS: domain, measure, (mapped) fact, base intersection, and description. |
|
2 |
Measure |
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measuredata/<measure name>/ |
One web service for each measure in PDS. Measure data is retrieved at the measure's base intersection with position names. |
Example Output:
https://cust1-prod3-rpasee-wts.oracleindustry.com:443/ords/pds/measures/all/
https://cust1-prod3-rpasee-wts.oracleindustry.com:443/ords/pds/measuredata/drtynslsregr/
Figure 17-22 Example Output: Measure Data
Filtering in the web services is supported through ORDS. To add a filter to a web service, append the parameter q=FilterObject, where FilterObject is a JSON object that represents the custom selection and sorting.
For the FilterObject syntax in ORDS version 19.1, see:
Table 17-2 provides examples of the FilterObject.
Table 17-2 Example URLs
| Description | Web Service URL |
|---|---|
|
To select measure metadata for ascs or rdf domain only. |
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measures/all/?q={"domain":{"$or":[{"$eq":"ascs"},{"$eq":"rdf"}]}} |
|
To select metadata for domain name in ("rdf", "mfprcs") and measure name in ("drtyeop1r", "rsal" |
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measures/all/?q={ "domain": {"$or":[{"$eq": "rdf"},{"$eq": "mfprcs"}]}, "measure":{"$or":[{"$eq":"drtyeop1r"}, {"$eq":"rsal"}]} } |
|
To retrieve measure data for a given week and sku. |
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measuredata/<measure name>/?q={"week":"w01_2017","sku":"1000466"} |
Measure data is returned in pages, with page size 100. This means that each web service call returns 100 rows. To retrieve additional data, append ?offset parameter.
For example, the url for the first page (first 100 rows) of rsal data is
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measuredata/<measure name>
and the url for the second page is
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/measuredata/<measure name>/?offset=100
RPASCE also offers PUT (write) web service that allows customers to write/update source facts in PDS. Source facts are the facts that are usually loaded (not calculated). They are flagged in integration configuration.
See the last column on the right in Figure 17-23 for source facts.
Example URL for PUT service below.
https://<cust>-<env>-<app>-wts.oracleindustry.com:443/ords/pds/putfact/data/
The payload of the web service will be in a JSON format that can support input data for a fact at any intersection.
Here is a sample JSON file.
{
"fact_names": ["DRTYEOP1C"]
"intersection": ["week","stor","sku"]
"data_rows": [
["w01_2017","900002","1005",1],
["w01_2017","900002","1008",1]
]
}
This section provides details about using the REST API call to IDCS or OCI IAM to interact with APEX/ORDS to access planning data store web services. The example show how the REST API calls are typically executed in IDCS or OCI IAM.
The REST APIs provide a way to integrate IDCS or OCI IAM with custom applications and clients that support REST, such as Java, C#, Go, Python, or Ruby apps.
This tutorial uses Oracle Linux 7.2. It can be executed on other operating systems with a bash terminal (such as Red Hat, Ubuntu, or OSX), or in Windows (using a bash emulator such as git bash). The commands may have small variations on different operating systems. It is recommended that you are familiar with the REST architecture style.
Access to IDCS or OCI IAM with authorization to manage Applications (Identity Domain Administrator, Security Administrator, or Application Administrator).
A Linux machine with cURL utility available.
Credentials - (Contact an Oracle cloud administrators if you do not have access to the following credentials.)
IDCS-client-id
IDCS-client-secret
IDCS-tenant-id
IDCS_URL
Userid
Password
The access_token (Bearer) identifies your client access in IDCS or OCI IAM and will be used for subsequent REST API calls. This token is encoded following the JSON Web Token (JWT) standard. Replace the following credentials with the correct values for your environment.
curl -i -u "<IDCS-client-id>":"<IDCS-client-secret>" \
https://<idcs-tenant-id>.<IDCS_URL> \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Host: <idcs-tenant-id>.<IDCS_URL>' \
-d 'grant_type=client_credentials' -d 'scope=urn:opc:idm:__myscopes__'
Example Bearer Token Rest Call
curl -i -u "a3399f80531e4eff82637f0ba5c7d49d":"d3e376c2-d5fb-4d3b-b2d0-3bd52df6f5eb" \ https://idcs-148d8548c42947c983343db9b2e378ec.identity.c9dev2.oc9qadev.com/oauth2/v1/token \ -H 'Accept: application/json' \ -H 'Accept-Language: en' \ -H 'Accept-Version: 1.3.0' \ -H 'Content-Type: application/x-www-form-urlencoded' \ -H 'Host: idcs-148d8548c42947c983343db9b2e378ec.identity.c9dev2.oc9qadev.com' \ -d 'username=oci.admin&password=OracleRetail1&grant_type=password&scope=urn%3Aopc%3Aidm%3A__myscopes__%20offline_access' --tlsv1.2
You will see output that contains the following:
The Access Token request output in JSON format. The return contains the attributes access_token, token_type, and expires_in.
The access_token(Bearer) identifies your client access in IDCS or OCI IAM and will be used for subsequent REST API calls. This token is encoded following the JSON Web Token (JWT) standard.
To check the JWT token, you can copy the access_token and verify its value using: https://jwt.io/#debugger-io
The token_type identifies the Access Token as a Bearer token type. In future requests, you will use this token type to identify your token in the Authorization header of your request.
The expires_in identifies the validity period of the Access Token.
|
Caution: These bearer tokens may expire and may have to be regenerated in case a "401 Authorization Required" error occurs. |
Token
HTTP/1.1 200 OK
Cache-Control: no-store
Content-Length: 4667
Content-Type: application/json;charset=UTF-8
Date: Fri, 08 Jan 2021 20:46:40 GMT
Pragma: no-cache
Server: Oracle Identity Cloud Service
Strict-Transport-Security: max-age=315360000; includeSubDomains
Vary: Accept-Encoding
Via: 1.1 net-idcs-config
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-Oracle-Dms-Ecid: Roj^J0TA200000000
X-Oracle-Dms-Rid: 0:1
X-Xss-Protection: 1; mode=block
In this task, you perform your REST API call toIDCS or OCI IAM. The objective is to show you how the REST API calls are typically executed in IDCS or OCI IAM.To learn more about the application API, refer to the IDCS or OCI IAM REST API documentation.
In this example, it is assumed that a publicly accessible ORDS url has been provided to the customer.
Example: http://<hostname>:<port>/ords
RestAPI example url: http://<hostname>:<port>/ords/pds
Replace the bearer token with the value received from the section "Get Bearer Token"Note that these bearer tokens may expire and may have to be regenerated if a "401 Authorization Required" error occurs.
Output
All measures.
Here is a sample payload.json file with the configuration values.
Invoke the Web Service
Replace the bearer token with the value received in the section "Get Bearer Token"
Note that these bearer tokens may expire and may have to be regenerated if a "401 Authorization Required" error occurs.
Write PDS Web Service
Replace the bearer token with the value received in the section "Get Bearer Token"
Note that these bearer tokens may expire and may have to be regenerated if a "401 Authorization Required" error occurs.
Sample Output
