8.5 Account Vulnerability

The account vulnerability metric captures which account types are most liable to being abused by an intelligent agent to move money through your financial system. A high value for this metric indicates that this account type was the agent’s preferred account when moving money through your Institution.

OFSCA calculates the vulnerability of an account by the following:

1. Sampling episodes from the trained agent’s policy.
2. Estimating the funds that flowed through each account type. For example, if $100 was credited into an account and debited from the account, the funds that flowed through that account were $100. If only $50 was debited, only $50 flowed through that account.
3. Normalizing this across all account types.

An account type with a high value for this metric is preferred by the agent over an account type with a lower value of this metric. Enhancing controls that monitor a vulnerable account type can improve the performance of the TMS for the segment in question.

Limitations

1. Currently, any funds that flow through an account are attributed to that account even if those funds did not reach the destination account. This could lead to the vulnerability of an account type being inflated in a given episode. However, since the metric is computed by averaging across multiple episodes, this should not have a bearing on the final metric.
2. If two are more account types (e.g,. BRK and RBK) are highly vulnerable, then the agent will break ties randomly and will assign a high vulnerability score to one of these account types while assigning a lower vulnerability score to others. If the overall segment score does not improve significantly even after remediating the account type with the highest vulnerability score (e.g,. BRK), this could be because other account types continue to be vulnerable. Once an experiment to address the most vulnerable account type (BRK) has been run and accepted, the segment dashboard will update to now indicate that the second account type (RBK) is most vulnerable. You might have to run an experiment to address monitoring gaps for this second account type (RBK) before overall segment score improves.