Before you Begin
This tutorial shows you how to compare machine learning predictive models in order to pick the most accurate model to use in Oracle Analytics.
Background
You can use Lift and Gain charts to assess the accuracy to predictive models in Oracle Analytics. The model lift and gain calculations work with machine learning classification models. Lift and gain charts show you visually how much the model is better than random guessing (the baseline). The greater the area between the lift and gain curves with the baseline, the better the model is at predicting the desired result.
In this tutorial, you use sample bank marketing campaign data as the dataset to predict which customers are likely to purchase additional services. After training the dataset using the predictive models, using the Naive Bayes and CART algorithms, you apply the models to the dataset, and then you create lift and gain charts to compare the resulting _LIFT datasets.
What Do You Need?
- Access to Oracle Analytics
- Download bmc_data.xlsx to your computer
Create a Dataset
In this section, you create a workbook to learn about the dataset.
- Sign in to Oracle Analytics.
- On the Home page, click Create, and then click Dataset.
- In Create Dataset, click Drop data file here or click to browse. In File Upload, click bmc_data.xlsx, and then click Open.
- In Create Dataset Table from bmc_data.xlsx, click OK.
- In New Dataset, double-click bmc_data in the Join Diagram.
Description of the illustration new_dataset_join_diagram.png - In the age column, click Measure
, and then click Attribute.
- Select to the balance column. In balance properties, click Measure in the Treat as row, and then click Attribute.
Description of the illustration transform_columns.png - Click Save. In Save Dataset as, enter
bmc_data
in Name, and then click OK. - In bmc_data, click Create Workbook.
- If open, close the Auto Insights panel.
Description of the illustration auto_insights_panel.png
Examine the Bank Marketing Campaign Data
In this section, you create a workbook to look at the results of the bank marketing campaign data.
- In the Data
pane, hold down the Ctrl key, select campaign and outcome, and then drag them to the canvas.
The no responses to the marketing campaign exceed the number of yes responses.
Description of the illustration campaign_outcome.png - Click Save. In Save Workbook, enter
bmc_initial_viz_results
, and then click Save. Click Go back.
Create a Naive Bayes Prediction Model
In this section, you create a Naive Bayes prediction model using the bmc_data dataset.
- On the Home page, click Create, and then click Data Flow.
- In Add Dataset, click bmc_data, and then click Add.
- In Data Flow Steps, double-click Train Binary Classifier.
- In Select Train Two-Classification Model Script, click Naive Bayes for Classification, and then click OK.
- In Train Binary Classifier next to Target, click Select a column and select outcome.
Yes is the default value required in the Positive Class in Target field.
Description of the illustration bmc_binary_nb_df.png - Click the Save Model node. In Save Model, enter
bmc_binary_nb_model
, and then click Save. In Save Data Flow As, enterbmc_binary_nb_df
, and then click OK. - Click Run Data Flow
.
- Click Go back
.
Create a CART Prediction Model
In this section, you create a CART predictive model using the bmc_data dataset.
- On the Home page, click Create, and then click Data Flow.
- In Add Dataset, click bmc_data, and then click Add.
- In Data Flow Steps, double-click Train Binary Classifier.
- In Select Train Two-Classification Model Script, click CART for model training, and then click OK.
- In Train Binary Classifier next to Target, click Select a column, and then select outcome.
Yes is the value you need in the Positive Class in Target field.
- Click Save Model node. In Save Model, enter
bmc_binary_cart_model
, and then click Save. In Save Data Flow As, enterbmc_binary_cart_df
, and then click OK. Click Run Data Flow.
- Click Go back
. In the Home page, click Machine Learning to see the Naive Bayes (bmc_binary_nb_model) and CART (bmc_binary_cart_model) models.
Description of the illustration predictive_models.png
Apply NB Predictive Model to Compute Lift and Gain
In this section, you apply the bmc_binary_nb_model to the bmc_data to create two datasets. One dataset represents the output from running the data flow, and the second dataset contains the lift and gain calculations. The second dataset appends LIFT to the dataset name.
- On the Home page, click Create, and then click Data Flow.
- In Add Dataset, click bmc_data, and then click Add.
- In Data Flow Steps, double-click Apply Model.
- In Select Model, click bmc_binary_nb_model, and then click Ok.
- Click Toggle Data Preview.
- In Apply Model under Parameters, select Yes in Compute lift and gain.
- In Target Column to compute lift, click Select a column, and then select outcome from Available data. In Positive class to compute lift, enter
Yes
. - On the Apply Model node, click Add a step
, and then select Save Data.
- In Save Model, enter
bmc_nb_lg
in Name. - Click Save As. In Save Data Flow as, enter
bmc_nb_lg_df
, and then click OK. - Click Run Data Flow
.
The message Data Flow "bmc_nb_lg_df" complete appears when the data flow run was successful.
- Click Go back
. On the Home page, click Data to view the datasets produced from running the data flow.
Description of the illustration nb_lift_gain_datasets.png
Apply CART Predictive Model to Compute Lift and Gain
In this section, you update and rename the existing bmc_nb_lg_df data flow to apply the bmc_binary_cart_model predictive model, specify the parameters, and rename the dataset and data flow to create the _LIFT dataset.
- In the bmc_nb_lg_df data flow, click Apply Model.
- In Apply Model, click bmc_nb_lg_model. In Select Model, click bmc_binary_cart_model, and then click OK.
- Click Toggle Data Preview.
- In Apply Model under Parameters, select Yes in Compute lift and gain.
- In Target Column to compute lift, click Select a column, and then select outcome from Available data. In Positive class to compute lift, enter
Yes
. - On the Apply Model node, click the Save Data node.
- In Save Dataset, enter
bmc_cart_lg
to replace bmc_nb_lg in Name. Click Save As. In Save Data Flow as, enterbmc_cart_lg_df
, and then click OK. - Click Run Data Flow
.
The message Data Flow "bmc_nb_lg_df" complete appears when the data flow run was successful.
- Click Go back
. On the Home page, click Data to view the results of the lift and gain data flows.
Description of the illustration cart_lift_gain_datasets.png
Visualize Gain
In this section, you create a workbook to visualize the output from the _LIFT datasets.
- On the Home page, click Create, and then click Workbook.
- In Add Dataset, enter
bmc
in Search. Selectbmc_nb_lg_LIFT
, and then click Add to Workbook. - In the Data pane, hold down the Ctrl key, select PopulationPercentile and CumulativeGain, right-click and select Pick Visualization, and then select Line
.
Description of the illustration gain_by_pop.png - Select GainChartBaseline and drag it to Values (Y-axis).
Description of the illustration gain_chart_baseline.png - In the Data pane, select OptimalGain and drag it to Values (Y-axis).
Description of the illustration optimal_gain.png - In the Data pane, click Add
, and then select Add Data. In Add Data, click bmc_cart_lg_LIFT, and then click Add to Workbook.
- In the Data pane, under the bmc_cart_lg_LIFT dataset, select CumulativeGain and drag it to Values (Y-axis).
Description of the illustration bn_cart_cumulgain_chart.png - In the Data pane, under the bmc_cart_lg_LIFT dataset, select OptimalGain and drag it to Values (Y-axis).
The optimal gain value for the bmc_cart_lg_LIFT dataset is higher than the bmc_nb_lg_LIFT dataset. The Binary Classification Naive Bayes machine learning model is better at predicting which users are likely to respond positively compared to Binary Classification CART model in this example.Description of the illustration optimal_gain_cart.png
Visualize Lift
In this section, you create visualizations of the lift results.
- In Data panel under the bmc_nb_lg_LIFT dataset, select LiftValue and drag it to Values (Y-Axis).
Description of the illustration lift_value_nb.png - Under the bmc_cart_lg_LIFT dataset, select LiftValue and drag it to Values (Y-Axis).
Description of the illustration cart_lift.png
Learn More
Evaluate Predictive Models with Lift and Gain Charts in Oracle Analytics
F51026-02
April 2025
Learn how to compare predictive models using lift and gain charts in Oracle Analytics Cloud.
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable:
U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs) and Oracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are "commercial computer software" or "commercial computer software documentation" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction, duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracle programs (including any operating system, integrated software, any programs embedded, installed or activated on delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) other Oracle data, is subject to the rights and limitations specified in the license contained in the applicable contract. The terms governing the U.S. Government's use of Oracle cloud services are defined by the applicable contract for such services. No other rights are granted to the U.S. Government.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc, and the AMD logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.
This software or hardware and documentation may provide access to or information about content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services unless otherwise set forth in an applicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.