The dataset used with ML_SCORE should have the same feature columns as the dataset used to train the model, but the data sample should be different from the data used to train the model. For example, you might reserve 20 to 30 percent of a labeled dataset for scoring.

ML_SCORE returns a computed metric indicating the quality of the model. A value of None is reported if a score for the specified or default metric cannot be computed. If an invalid metric is specified, the following error message is reported: Invalid data for the metric. Score could not be computed.

Models with a low score can be expected to perform poorly, producing predictions and explanations that cannot be relied upon. A low score typically indicates that the provided feature columns are not a good predictor of the target values. In this case, consider adding more rows or more informative features to the training dataset.

You can also run ML_SCORE on the training dataset and a labeled test dataset and compare results to ensure that the test dataset is representative of the training dataset. A high score on a training dataset and low score on a test dataset indicates that the test data set is not representative of the training dataset. In this case, consider adding rows to the training dataset that better represent the test dataset.

AutoML supports a variety of scoring metrics to help you understand how your model performs across a series of benchmarks. The metric you select to score the model must be compatible with the task type and the target data. See Optimization and Scoring Metrics.

Before running ML_SCORE, you must train, and then load the trained model you want to use for scoring.

For more information about training and loading models, see Train a Model and Load a Model.

After training and loading the model, prepare a table of labeled data to score that has a different set of data from the trained model. This is considered the validation dataset. For parameter and option descriptions, see ML_SCORE.

To score a model, run the ML_SCORE routine.

mysql> CALL sys.ML_SCORE(table_name, target_column_name, model_handle, metric, score, [options]);

The following example uses the accuracy metric to compute model quality:

mysql> CALL sys.ML_SCORE('census_data.census_validate', 'revenue', @census_model, 'accuracy', @score, NULL);

Where:

To retrieve the computed score, query the @score session variable.

mysql> SELECT @score;
+--------------------+
| @score             |
+--------------------+
| 0.8888888955116272 |
+--------------------+
1 row in set (0.0409 sec)

Review the score value and determine if the trained model is reliable enough for generating predictions and explanations.