Updating an Existing Model Deployment with Autoscaling

Learn how to enable autoscaling for an existing model deployment or update any existing autoscaling configuration.

For model deployments in an Active state, note that modifications to the Autoscaling Scaling Policy fields must occur independently, without simultaneous changes to other configurations. However, updates to fields such as Display name, Description, Tags, and other non-infrastructure related aspects are allowed.

In contrast, when the model deployment is Inactive, you have the flexibility to change all options simultaneously.

1. From the model deployments page, select the name of the model deployment to update. If you need help finding the list of model deployments, see Listing Model Deployments.Select Edit to update the configuration.
2. On the Edit model deployment page, select Enable autoscaling.
  Several lists and fields are displayed to let you configure the autoscaling.
3. (Optional) Update the values in each list or field as appropriate for the configuration.
4. (Optional) To autoscale the load balancer or change the bandwidth range, select Show advanced options.
  Change the value of the minimum bandwidth and maximum bandwidth, keeping the maximum to less than twice the minimum bandwidth value.
5. Select Submit.

Use the oci data-science model-deployment update command and required parameters to edit (update) a model deployment:

oci data-science model-deployment update --model-deployment-id <model-deployment-id>... [OPTIONS]

For example, update a deployment with:

oci data-science model-deployment update \
--model-deployment-id <MODEL_DEPLOYMENT_OCID>
--model-deployment-configuration-details file://<MODEL_DEPLOYMENT_CONFIGURATION_FILE>

Then use this model deployment JSON configuration file for update. Update the fields under the AUTOSCALING scaling policy as appropriate:

{
  "deploymentType": "SINGLE_MODEL",
  "modelConfigurationDetails": {
    "modelId": "ocid1.datasciencemodel....",
    "scalingPolicy": {
      "policyType": "AUTOSCALING",
      "coolDownInSeconds": 650,
      "isEnabled": true,
      "autoScalingPolicies": [
        {
          "autoScalingPolicyType": "THRESHOLD",
          "initialInstanceCount": 1,
          "maximumInstanceCount": 2,
          "minimumInstanceCount": 1,
          "rules": [
            {
              "metricExpressionRuleType": "PREDEFINED_EXPRESSION",
              "metricType": "CPU_UTILIZATION",
              "scaleInConfiguration": {
                "scalingConfigurationType": "THRESHOLD",
                "threshold": "10"
              },
              "scaleOutConfiguration": {
                "scalingConfigurationType": "THRESHOLD",
                "threshold": "65"
              }
            }
          ]
        }
      ]
    },
    "bandwidthMbps": 10,
    "maximumBandwidthMbps": 20
  }
}

For a complete list of parameters and values for CLI commands, see the CLI Command Reference.

Use the UpdateModelDeployment operation to edit (update) a model deployment.

Oracle Cloud Infrastructure Documentation

Updating an Existing Model Deployment with Autoscaling