Selection Criteria for Training ML Models

Working with the Machine Learning Workbench

As a rule of thumb, the criteria for training machine learning models as well as selecting models to train must be determined in consultation with project stakeholders in your organization.

Note: Oracle recommends that you initially train the seed models and check the results if these provide appropriate guidance. Otherwise, retrain and select custom models to retrain.

Assessing the performance of a machine learning model is an essential step in a predictive modeling pipeline. Once a model is ready, it has to be evaluated to establish its correctness. There are some widely used validation metrics that are used to assess a prediction model and we have used some of them:

• Accuracy: It is the ratio of correct predictions to the total number of predictions.

  For example, consider a prediction model predicting an activity is going to be delayed or not with an accuracy of 0.75. If the model predicts 100 times in total, then 75 times the model will predict it correctly.

• Recall: It answers how well the model can find all the positive results actually in the data. Of all the activities that are actually delayed, how many the model correctly identified.

  For example, consider a prediction model predicting an activity is going to be delayed or not. If the recall is 0.57, then it implies that for every 100 activities that are actually delayed, approximately 57 activities are correctly predicted to be delayed.

• Precision: It tells us, how often are we correct when we have a positive prediction, and how many are actually delayed out of all the activities that are predicted to be delayed.

  For example, consider a prediction model predicting an activity is going to be delayed or not. If the precision is 0.6, then for every 100 activities that are predicted to be delayed, 60 activities are actually delayed.

---

Last Published Friday, September 2, 2022