Oracle Cloud Infrastructure Documentation

Switching Models on Dedicated AI Clusters

Switch a pretrained or imported model hosted on a dedicated AI cluster in OCI Generative AI. Reuse the cluster if the new model supports its full unit shape in the same region, or create a new cluster if a different shape is required.

You might want to switch models to add image support, improve reasoning or response quality for an application, or replace a model that's retiring. If you already host the current model on a dedicated AI cluster, compare the new model's supported unit shapes with the shape of the existing cluster before creating another cluster. This process also applies when you switch between imported models.

Deciding Whether to Reuse a Dedicated AI Cluster

You select one unit shape when you create a dedicated AI cluster. You can't change that shape after creation. To reuse the cluster, the new model must support the same unit shape in the cluster's region.

Compare complete unit shape names, such as Cohere_H100_X2 and Cohere_H100_X4, rather than only the hardware type, such as H100. Shapes can differ by model and region.

  • If the new model supports the existing unit shape, delete the current model's endpoints and create an endpoint for the new model on the same dedicated AI cluster. The cluster remains allocated to you.
  • If the new model requires a different unit shape, create a new dedicated AI cluster with that shape and an endpoint for the new model. You don't need to delete the original cluster to create the new one.

By default, a cluster can have up to 50 endpoints for the same hosted model. These endpoints share the cluster's model replicas, which serve the current model. To switch the cluster to another model, delete every endpoint for the current model before creating an endpoint for the replacement model. Requests to the deleted endpoints stop; after the new endpoint becomes active, update applications to use it.

Examples: Switching OCI Generative AI Pretrained Models in Ashburn

Suppose you host Cohere Command A on a dedicated AI cluster with the Cohere_H100_X2 shape in US East (Ashburn). The North America Baseline Shapes table shows these options for switching models in the same region.

Switching from Cohere Command A in Ashburn
Scenario Current model Current cluster shape New model Target cluster shape Action
Scenario 1 Cohere Command A Cohere_H100_X2 Cohere Command A Vision Cohere_H100_X2

Reuse the existing cluster. Both models support Cohere_H100_X2. Delete all Command A endpoints on the cluster, then create a Command A Vision endpoint on the same cluster.
Scenario 2 Cohere Command A Cohere_H100_X2 Cohere Command A Reasoning Cohere_H100_X4

Create a new cluster. Command A Reasoning requires Cohere_H100_X4, and you can't change the existing cluster's Cohere_H100_X2 shape. Create a new cluster with the required shape and an endpoint for Command A Reasoning.

Examples: Switching Imported Qwen Models

Suppose you host Qwen/Qwen3.6-35B-A3B on a dedicated AI cluster with the H100_X2 shape. In these examples, the target region has H100_X1, H100_X2, and H100_X4 available. The Compatible Alibaba Models page lists H100_X1 as the minimum H100 unit shape for Qwen/Qwen3.8-27B and H100_X4 for Qwen/Qwen3-Next-80B-A3B-Instruct. The Qwen/Qwen3.8-27B model also supports H100_X2 in these examples.

Switching from Qwen/Qwen3.6-35B-A3B
Scenario Current model Current cluster shape New model Target cluster shape Action
Scenario 1 Qwen/Qwen3.6-35B-A3B H100_X2 Qwen/Qwen3.8-27B H100_X2

Reuse the existing cluster. The new model supports H100_X2, even though its minimum H100 shape is H100_X1. Delete all endpoints for the current model, then create an endpoint for the new model on the same cluster.
Scenario 2 Qwen/Qwen3.6-35B-A3B H100_X2 Qwen/Qwen3.8-27B H100_X1

Create a new cluster to use a smaller shape. The new model can use the existing H100_X2 cluster. To use its minimum H100_X1 shape instead, create a new cluster because you can't change the existing cluster's shape. Create an endpoint on the new cluster and switch the application to it. Delete the old model's endpoints and cluster if you no longer need them.
Scenario 3 Qwen/Qwen3.6-35B-A3B H100_X2 Qwen/Qwen3-Next-80B-A3B-Instruct H100_X4

Create a new cluster. The new model requires at least H100_X4, so the existing H100_X2 cluster doesn't have enough hardware units. Create a new cluster with H100_X4 and an endpoint for the new model.

Before Deleting the Original Dedicated AI Cluster

Creating a new dedicated AI cluster is separate from deciding whether to delete the original one. The billing terms depend on the model type:

  • Hosting clusters for OCI Generative AI pretrained and custom models have a minimum commitment of 744 unit-hours. Deleting a cluster before that minimum is reached doesn't remove the minimum charge.
  • Hosting clusters for imported models don't require the 744-unit-hour commitment. Instead, they have a minimum billable duration of one hour. See Hardware Unit Shapes for Imported Models.

Before deciding whether or when to delete the original cluster, review its usage, any applicable minimum billing, and whether you still need it for other endpoints or a compatible model. Keeping a cluster allocated continues to accrue usage. Also account for the new cluster's billing terms. See Calculating Cost and the pricing page.

Switching Models

You have an endpoint for the current model on a hosting dedicated AI cluster and have selected the model you want to use instead.

  1. Review the existing dedicated AI cluster's details and note its region and full unit shape.
    For example, a Cohere Command A cluster in Ashburn uses Cohere_H100_X2. In the imported-model examples, a cluster hosting Qwen/Qwen3.6-35B-A3B uses H100_X2.
  2. Find the new model's supported unit shapes in the cluster's region and compare their full names with the existing cluster's shape.

    For OCI Generative AI pretrained models, use Baseline Hardware Unit Shapes for OCI Models. In Ashburn, Command A Vision uses Cohere_H100_X2, which matches the example cluster. Command A Reasoning uses Cohere_H100_X4, which requires a new cluster.

    For imported models, find a compatible shape on the new model's page in Compatible Models and check its regional availability in Hardware Unit Shapes for Imported Models. For example, a cluster with H100_X2 can be reused for another imported model that supports H100_X2 in the same region.

  3. If the new model supports the existing unit shape, reuse the cluster:
    1. Delete all endpoints for the current model on the cluster.
    2. Create a new endpoint for the new model using the same dedicated AI cluster.
    The dedicated AI cluster remains allocated to you. You don't need to delete it when the unit shape is unchanged.
  4. If the new model requires a different unit shape, create a new cluster and endpoint:
    1. Create a new hosting dedicated AI cluster with the shape required by the new model in your region.
    2. After the new cluster becomes active, create an endpoint for the new model on it.
  5. Update your application to use the new endpoint and verify its responses.
    The new endpoint has its own OCID. Update any application configuration that refers to the old endpoint.

If you created a new cluster, review Before Deleting the Original Dedicated AI Cluster before deciding whether or when to retire the old resources. If you decide to delete the original cluster, first delete its endpoints, then delete the cluster.