Compute Targets

Compute targets support the model deployment workload. The workload must use the managed container provided by Data Science for deployment.

A compute target can host multiple workloads, whether similar or different. We recommend different compute targets for workloads that require stronger isolation.

Following are features supported by compute targets.

• Data Science-managed infrastructure: OCI Data Science manages operations, maintenance, and patching for the underlying infrastructure.
• Compute target autoscaling: Compute target instances can scale in or out based on workload requirements.
• Model deployment autoscaling: Model deployments hosted on a compute target can autoscale, including scaling to zero with request buffering.
• Custom and managed egress: For supported egress traffic, workloads can use custom egress in addition to OCI Data Science-managed network paths.
• Compute target metrics: Compute target metrics are available in OCI Monitoring.
• Model deployment metrics: Model deployment metrics are available in OCI Monitoring.

Before using Managed Compute Cluster compute targets, review these considerations.

• Managed Compute Cluster uses a homogeneous compute shape. If your workloads require different shapes, create a separate compute target for each shape.
• Use separate compute targets for workloads that require stronger isolation.
• Internet access from workload containers is restricted.
• Managed egress is available through supported OCI Data Science-managed network paths.
• A compute target is tenancy-specific and can't be shared across tenancies. Use separate compute targets for development, test, and production workloads.
• Plan sufficient spare capacity for patching and maintenance operations.
• Each compute target node reserves approximately 0.7 OCPU and 1 GiB of memory for Data Science service and platform components. Plan workload capacity using the remaining node capacity.

Managed Compute Cluster supports selected CPU and GPU shapes.

• When you create a Managed Compute Cluster compute target, you select the compute shape and the number of instances. Billing is based on the underlying Compute resources provisioned for the managed compute cluster compute target at standard OCI rates.
• If autoscaling is configured, billing reflects the active compute capacity. You're charged for additional instances when the compute target scales out, and you're no longer charged for instances that are removed when the compute target scales in.
• When you delete the compute target, billing stops for the infrastructure associated with that compute target.

Data Science manages patching for the underlying infrastructure used by compute targets. Patching can include updates to the managed infrastructure image and service bootstrap configuration used by the compute target. During patching, the compute target can enter an Updating state while patching is in progress and returns to Active when patching completes. Patching updates only Data Science-managed infrastructure. Patching doesn't update your model artifact, model deployment configuration, or the container image used by your model deployment.

Data Science applies safeguards during patching to keep model deployments available while the underlying compute target infrastructure is maintained. As part of patching, Data Science might replace or restart compute target instances, or move model deployment instances as needed. For production workloads, size the compute target with maintenance headroom and configure enough model deployment instances for your availability needs.

Patching is managed by Data Science and doesn't require customer action. If a compute target is in the Updating state, the lifecycle details can indicate whether Data Science-managed patching is in progress. When patching completes, the compute target returns to Active. If the compute target requires service attention, review the lifecycle details and any related work request messages. If the issue persists, contact Oracle Support.