You can run the experiment in the following modes:
· Generating Experiment from Recommendation
You can compare two experiments as explained below:
This section provides a guide on creating and executing customized experiments to validate any hypotheses you may have. Conducting these experiments allows you to simulate the effects of changes made to your transaction monitoring system. In turn, this enables you to carry out thorough what-if analysis, evaluate the impact of various decisions and make the most informed decisions accordingly.
Topics:
· Copying or Modifying the Control Set
To generate a new experiment, follow these steps:
1. Click New Experiment on the dashboard to generate a new experiment. The New Experiment window is displayed.
Figure: New Experiment
2. Click Start to start configuring the new experiment. The Select Segment window is displayed.
NOTE:
If an experiment has been configured but has not been executed then dialog appears.
1. Click Continue to continue with the configured data for the experiment or click Discard to discard the configured data and the Select Segment window is displayed.
To select the segment, follow these steps:
Figure: Select Segment
1. Select the required segment from the Select Segment drop-down list.
2. Click Continue to navigate to the Select Agent step.
Or
Click Discard to discard the current activity and return to the New Experiment window and again click Start to start the New Experiment from the initial steps.
You can select from one of the available agents or create a new agent.
Figure: Select Agent
To select/create an agent, follow these steps:
1. Select the agent that is created during the initial setup on the Select Agent option. You can also select experiment for HT agent.
Or
2. If you want to create a new agent then click Create Agent. The following window is displayed.
For information about how to create an agent, see the Creating an Agent section. Once the agent is created, then select the created agent.
3. Click Continue to navigate to the Copy and modify control set step.
This section demonstrates how to specify the controls (scenario thresholds, account transaction product constraints) you want to evaluate in an experiment.
Figure: Copy / modify the Control Set
1. Enter/select the details in the following table:
| Field | Description |
Experiment account-transaction product constraint set name |
You need to select the drop-down list from the Account-transaction product constraint set to define the name. |
Experiment scenario threshold set name |
Enter the Threshold Set Name. The name must be unique to a particular threshold set. |
Experiment Description |
Enter the description of the Experiment. NOTE: This field is optional, but a good description can be useful for audit purposes. |
| Scenarios threshold set name | Using drop-down list, select the required scenario threshold
set you want to use or modify. The Add
Scenario button allows you to add new scenarios to your
control set.
Figure: Scenario 
You can perform the following: 1. Click Add scenario to add a new scenario for the respective threshold set. The following window is displayed. Figure: Add Scenario a. Search the required scenario in the Search box. b. Click on the Action option of the required scenario to be added to the threshold set. Figure: Selected Scenario c. Click
Navigation icon Figure: Threshold Details d. Double-click on the required Threshold Value field and then edit the threshold value. e. Click Add Scenario. The scenario is added to the respective threshold set. 2. Click
Delete icon 3. Click
Edit icon Figure: Edit a Scenario a. Double-click on the required Threshold Value field and then edit the threshold value.
b. Click Update Scenario to modify the information. 4. Click
Navigation icon Figure: Scenario Information |
| Account-transaction product constraints (per day) | When a new account or transaction
product is added to a segment, it is necessary to
Select the required option from the drop-down list. The account types will be displayed based on the selected Account-transaction product constraint set. Figure: Account-transaction product constraint set 
You can perform the following: 1. Search the account type in the Search box. 2. Click
Edit icon Figure: Edit an Account
a. Double-click on the Withdrawal Limit field and edit the required value. b. Click Update to update the modified value. 3. Click
Navigation icon Figure: Channel Details |
2. Click Continue on the Copy and modify control set window to navigate to the Review step.
Use this section to modify the threshold value for the newly added offerings (Account Type and transaction product).
For more information on how to add a new offering (Account Type or transaction product), see Modifying the System.
To monitor the newly added offerings, follow these steps:
1. On the Copy and Modify Control Set section, go to Scenarios Threshold Set Name drop-down list.
2. Select the required Scenarios threshold set name from the drop-down list. The scenarios are displayed based on the selected Scenarios threshold set name.
3. Click the
Edit
icon 
to modify the selected scenario threshold
values. The Threshold Details window is displayed.
4. Double-click on the required Threshold Value field and then edit the threshold value.
5. When a new offering (account type) is added, you can append the name of new account type to list in Mantas_Bus_Acct_Types_Lst.
6. When a new transaction product is added to system, you can append the name of that transaction product to the value of one of the following thresholds.
§ Incl_Cash_Trxn_Prdct_Type_Lst - If new Product is mapped to Cash transaction product.
§ Incl_MI_Trxn_Prdct_Type_Lst - If new Product is mapped to MI transaction product.
§ Incl_Wire_Trxn_Prdct_Type_Lst - If new Product is mapped to WIRE transaction product. For example, ZELLE is mapped to WIRE (Can be comma separated in case of multiple Products).
§ Incl_BO_Trxn_Prdct_Type_Lst - If new Product is mapped to BO transaction product.
c. Click Update Scenario to modify the information.
In this section, you can verify all the parameters before running the experiment.
Figure: Review Experiment
1. Click Run Experiment to generate the new experiment. Upon completion of the experiment, the status of the experiment can be viewed in the Experiments tab.
The results of the experiments will be available in the Transaction Monitoring Performance Dashboard or you can view results by clicking the View Results on the notification of the Ask Oracle window.
To view result of the generated experiment, follow these steps:
1. Click on the required Experiment Id in the Experiments tab to view the experiment results. The following window is displayed.
Figure: Segment Strength
Here, the expected change in performance of the system and the expected change in alert volume are displayed.
An experiment is successful, if there is an increase in system strength without a disproportionate increase in alert volume.
OFSCA offers recommendations to tackle the identified deficiencies. A deficiency can be a low scenario performance or a high account/ channel vulnerability. The recommendations aim to tune the scenario with the highest chances of addressing the selected deficiency. To generate recommended thresholds for the scenario, OFSCA evaluates the performance of simulated TMS against multiple sets of candidate thresholds within proximity of the production (currently applied) values. A set of candidates is evaluated by using a combined metric analyzing both the percentage of episodes getting alerted and the average number of distinct alerts per episodes. OFSCA recommends the set that has performed optimally as per the metric.
This section describes the process for generating experiments to assess the effectiveness of these recommendations.
To generate an experiment for the particular segments, follow these steps:
1. Click Open Ask Oracle 
to display the Ask Oracle window.
2. Click Transaction Monitoring Performance menu to display the Transaction Monitoring Performance dashboard. The following window is displayed.
Figure: Transaction Monitoring Performance Dashboard
3. Click the required segment (for example, APAC_MR) to view the individual segment dashboard. The following window is displayed.
Figure: Segment Dashboard
This dashboard presents the insights generated for the scenario performance, account vulnerability, and transaction product vulnerability.
For each Insight, we see OFSCA-identified scenarios that can be adapted to address the identified vulnerabilities for each section. Besides, OFSCA provides specific threshold recommendations for the scenarios identified.
You can generate an experiment to evaluate this specific recommendation by clicking Generate Experiment on the segment dashboard.
NOTE:
If the results of the generated experiments do not meet your expectation, then consider generating a new experiment based on other recommendations displayed in the Transaction Monitoring Performance dashboard.
To generate an experiment, follow these steps:
1. Click Generate Experiment on the segment dashboard. The Copy or Modify Control Set window is displayed. The recommended threshold values are auto populated in the listed Scenarios.
Figure: Copy or Modify Control Set
NOTE:
· The dialog box appears if the unfinished experiment is running on the existing segment. In that case, click OK on the dialog box to continue generating the experiment.
· The experiment description will be auto populated with the recommendation from the segment dashboard.
· Do not change the default value in the Scenarios Threshold Set Name drop-down list.
2. Enter the relevant name or description for threshold set in the Experiment Scenario Threshold Set Name field.
3. If you want to modify
the recommended thresholds, Click Edit
icon
against the existing scenarios. The Threshold Details window is displayed.
4. Modify the threshold value. Click Update Scenario. The Scenario List page is displayed.
5. Click Continue to navigate to the Review step. The following window is displayed.
Figure: Review Segment
6. Click Run Experiment to run the experiment. A notification will be delivered through the Ask Oracle window when the experiment is complete and you can view the generated experiment either through the Transaction Monitoring Performance dashboard or in the Experiments tab.
To view the identified vulnerability in detail, follow the step:
1. Click View Analysis on the segment dashboard in each of the tiles provides a more detailed analysis of the identified vulnerability.
For account vulnerability, it presents an analysis of the agent's transactional activity involving the most vulnerable account type.
It presents a breakdown of activity in the account type by transaction product as well as the range of activity observed for each of these transaction products for credits and debits.
Figure: For Account Type
For transaction product vulnerability, it presents an analysis of the agent's transactional activity involving the most vulnerable transaction product type.
It presents a breakdown of activity in the transaction product type by account as well as the range of activity observed for each of these account types for credits and debits.
Figure: For Channel
This can also inform hypotheses a user can test using custom experiments. For example, if the account vulnerability analysis indicates that wires are the most commonly used transaction product for a specific account, a user can try tuning thresholds of scenarios that monitor wires.
In this section, you can compare two experiments of the same segment.
To compare the two experiments, follow these steps:
1. Click Open Ask Oracle
to display the Ask Oracle window.
2. Click Experiments menu to display the Experiments window. The following window is displayed.
Figure: Experiments
3. Click Compare Experiment. The following window is displayed.
Figure: Compare Experiment
4. Select the required experiment that you want to compare with another experiment from the Experiment 1 drop-down list.
5. Select the required experiment that you want to compare with experiment 1 from the Experiment 2 drop-down list.
NOTE:
You can compare the experiments in the same segments only.
6. Click Compare to compare the selected experiments. The compared result is displayed.
Results from the two selected experiments and the currently accepted experiment for the segment are displayed as shown in the below figure. Any differences between the agents used, and the thresholds evaluated are also displayed.
Figure: Experiment Result
You can compare the results between two experiments and accept them based on the segment strength score.
7. Click Accept Results to accept the experiment for the particular segment.
NOTE:
You can not accept the result if the experiment has more than four accounts.
Use this section to verify the impact of adding a new offering such as account type and transaction product by comparing the experiments.
Topics:
· Risk of New Offering-Account Type
· Risk of New Offering-Transaction Product
Use this section to verify the impact of adding a new offering as an account type by comparing the experiments.
To compare the two experiments to assess the risk of the new offering as an account type, follow these steps:
1. Click Open Ask Oracle
to display the Ask Oracle window.
2. Click the Experiments menu. The Experiments window is displayed.
Figure: Experiments
3. Click Compare Experiment. The following window is displayed.
Figure: Compare Experiment
4. Select the experiment which has a newly added offering as an account type that you want to compare with another experiment from the Experiment 1 drop-down list.
5. Select another experiment that has a newly added offering as the account type with different set of constraints that you want to compare with Experiment 1 from the Experiment 2 drop-down list.
NOTE:
You can compare the experiments in the same segments only.
6. Click Compare to compare the selected experiments. The compared result is displayed.
Results from the two selected experiments and the currently accepted experiment for the segment are displayed as shown in the below figure. Any differences between the agents used, and the thresholds evaluated are also displayed.
Figure: Experiment Result
You can compare the results between the experiments to determine the incremental risk resulting from using different sets of controls to monitor the new account type. If the system strength has dropped relative to the accepted experiment, this indicates that the new account type is not monitored as effectively as existing accounts. This is acceptable if an institution knows the account is being offered to low risk customers.
If the system strength has not changed or increased relative to the accepted experiment, this indicates that the new account type is being more effectively monitored than existing account types. For accounts that are being offered to higher risk customers, institutions should look to devise controls that result in an increase in system strength.
7. Click Accept Results to accept the experiment for the particular segment.
NOTE:
You can add up to 5 account types in an Experiment but only Experiments with up to 4 account types can be accepted
Use this section to verify the impact of adding a new offering as a transaction product by comparing the experiments.
To compare the two experiments to assess the risk of the new offering as a Transaction Product, follow these steps:
1. Click Open Ask Oracle
to display the Ask Oracle window.
2. Click the Experiments menu. The Experiments window is displayed.
Figure: Experiments
3. Click Compare Experiment. The following window is displayed.
Figure: Compare Experiment
4. Select the experiment which has a newly added offering as a Transaction Product that you want to compare with another experiment from the Experiment 1 drop-down list.
5. Select another experiment that has a newly added offering as the Transaction Product with different set of constraints that you want to compare with Experiment 1 from the Experiment 2 drop-down list.
NOTE:
You can compare the experiments in the same segments only. You can add up to 5 account types in an Experiment.
6. Click Compare to compare the selected experiments. The compared result is displayed.
Results from the two selected experiments and the currently accepted experiment for the segment are displayed as shown in the below figure. Any differences between the agents used, and the thresholds evaluated are also displayed.
Figure: Experiment Result
You can compare the results between the experiments to determine the incremental risk resulting from using different sets of controls to monitor the new account type. If the system strength has dropped relative to the accepted experiment, this indicates that the new account type is not monitored as effectively as existing accounts. This is acceptable if an institution knows the account is being offered to low risk customers.
If the system strength has not changed or increased relative to the accepted experiment, this indicates that the new account type is being more effectively monitored than existing account types. For accounts that are being offered to higher risk customers, institutions should look to devise controls that result in an increase in system strength.
7. Click Accept Results to accept the experiment for the particular segment.
or click 
to delete the selected scenario.
