AI Compute

This chapter covers the use of specialized computing resources for powering AI agents in Oracle AI Data Platform.

An AI compute cluster lets you host agents in AI Data Platform workspaces. AI computes can be attached to agents to run the playground experience and host deployed agents for production workloads.

Where to Find AI Compute

You manage AI compute from the Compute page, accessible from the left navigation pane in your AI Data Platform. Click the AI compute tab to see the AI compute clusters available in your workspace.


The Compute page open with the AI Compute tab selected.

The AI compute tab is the main landing page for AI compute resources in the currently selected workspace. It provides table actions, filtering, sorting, and status information.

Column or Control Purpose
Filter Search the table for a resource by name or visible text.
Add (+) Start creation of a new AI compute.
Delete Remove selected AI computes.
AI compute name Name of the compute resource. Click on the AI compute name to open the resource details page.
State Current lifecycle state, such as Creating, Active, or Updating.
Number of replicas Number of compute replicas associated with the AI compute. If you select an auto-scaling policy for the compute, the minimum and maximum number of replicas are listed.
Number of agents Number of agents hosted in the AI compute. The count is shown as a link when agents are present. This includes playground hosting and deployed agent endpoints.
Configuration Summary of the compute shape of the replicas in the AI Compute, such as 1 OCPU, 16 GB or 2 OCPU, 32 GB.
Updated by / Updated on / Created on Audit information for the most recent update and creation time.
Action menu (...) Open resource-specific actions for the selected row.

Note:

A green check indicates an Active resource. A spinner indicates an in-progress operation such as Creating or Updating. Wait for a resource to be Active before relying on it for production work.

Clicking the name of an AI compute in the AI compute tab allows you to view detailed information on that AI compute. The AI compute view has four tabs: Details, Deployed Agents, Metrics, and Permissions.

Details Tab

The Details tab summarizes the configured shape and storage for each replica of the AI compute. AMD is used for all AI computes.


AI Compute page open with Details tab selected.

Details Tab Item Description
Driver shape The GPU shape selected for the AI Compute. AMD is the driver shape for all AI Compute.
OCPUs The selected number of OCPUs for the AI Compute.
Memory (GB) The AI compute memory in GB.
Block volume The block volume size of the AI compute in GB.

Deployed Agents Tab

The Deployed Agents tab lists agents that are hosted in the AI Compute resource. If an agent is not deployed, the AI compute is hosting the playground experience necessary for iterative development and testing. If the agent is deployed, the AI computed is hosting the agent deployment endpoint for production workloads.


AI Compute page open with the Agents tab selected.

Deployed Agents Tab Item Description
Filter by agent name Search the associated agent list.
Agent name Name of the agent associated with the compute resource.
Authoring mode How the agent was authored. Values of Visual or Code.
Deployment Deployment state. Agents hosted in the AI compute can either be in a Playground state or in a Deployed state.
Created on / Updated on / Updated by Creation and update metadata for the agent.

Metrics Tab

Use the Metrics tab to review CPU, memory, and network utilization over time. This is useful when deciding whether to scale replicas, OCPUs, or memory up or down. There is no downtime during a scale out/in AI compute operation.


AI Compute page open with Compute Utilization tab selected.

Note:

Each time series corresponds to a different replica. For example, if you create an AI compute with two replicas, two time series will be displayed just like the screenshot above shows.
Compute Utilization Tab Item Description
Applied Filters Choose the start and end date/time for the utilization view.
View Choose the metric. Options are:
  • CPU Utilization
  • Memory Utilization
  • Network Received Bytes
  • Network Transmitted Bytes
Time period Choose between a custom time period or the last 7 days.
Replica ID Filter metrics to a specific replica ID, or show All compute replicas.
Interval Choose the aggregation interval for the chart. Options are Auto, 1min, 5 min, 15min, 30min, and 60min.

Permissions Tab

The permission tab provides a table of all the principals that have permission to this AI compute. Use this tab to add principals, change the permission level, or otherwise modify permissions.


AI Compute page open with Permissions tab selected.

Permissions Tab Item Description
Principal name Name of the principal or role.
Principal type Principal type. USER or ROLE.
Permission Permission level.
Will be inherited If permissions granted for a parent object grant permissions to contained objects.

Create an AI Compute

You can create AI compute clusters to run AI agents in your Oracle AI Data Platform.

  1. On the Home page, navigate to your workspace.
  2. Click Create then click AI Compute.
  3. Provide a name and description for your AI compute.
    Create AI compute dialog showing default values before a user has completed it.

  4. Set the number of OCPUs in each replica. You can choose 1, 2, 4, 8, 16, 32, or 64 OCPUs.
  5. Set the memory (in GB) per replica. The allowed range of memory varies based on the number of OCPUs:
    OCPUs Memory
    1 OCPU 16 or 32 GB
    2 OCPUs 16, 32, or 64 GB
    4 OCPUs 32, 64, or 128 GB
    8 OCPUs 32. 64, 128, or 256 GB
    16 OCPUs 64, 128, 256, or 512 GB
    32 OCPUs 128, 256, or 512 GB
    64 OCPUs 256, 512, or 1024GB
  6. To enable horizontal autoscaling, turn on Auto-scaling, then configure the following fields:
    • Minimum replicas: Set the minimum number of replicas assigned to the AI compute cluster.
    • Maximum replicas: Set the maximum number of replicas assigned to the AI compute cluster. Scale out events occur until the number of replicas reaches the maximum replica count.
    • Scaling conditions: Set the thresholds for API requests per sec (the number of requests per seconds across all agents hosted on the AI compute) and Mean CPU utilization across replicas (the average CPU utilization across all replicas; the default is 80%).

      Scaling conditions use logic that initiates an action to add or remove a replica from the AI compute. When True, the scaling condition initiates a scale out action and one more compute replica is added. A scale out action is triggered when API requests per second exceed a threshold value or when the mean CPU utilization across all replicas exceeds a threshold value.

    • Cooldown period: Set the time period after an autoscaling action during which the autoscaling takes no further actions. This prevents excessive scaling actions in a short time frame. The minimum value is 5 minutes.
  7. If you left Auto-scaling disabled, set the number of Compute replicas. Replicas scale the compute pool. Each replica hosts a copy of all the agents hosted in the AI compute.

    Note:

    The maximum number of replicas is 10. Contact your Oracle representative if you need to increase your replica limit.
  8. Review the corresponding AIDP Units that the AI compute configuration costs per hour.
  9. Click Create. The new resource appears in the list in the Creating state. The state changes to Active after provisioning completes.

Create an AI Compute for an Agent

You can create a new AI compute cluster directly from the agent interface and attach it immediately.

For more information, see AI Compute.
  1. On the Home page, navigate to your agent.
  2. Click Compute then click Create a new AI compute.
  3. Provide a name and description for your AI compute.
  4. Select the number of OCPUs and memory size for your AI compute.
  5. Click Create.

Attach an Existing AI Compute to an Agent

You can attach an AI compute cluster you've previously created to an agent on which you have at least USE permissions.

  1. On the Home page, navigate to your agent.
  2. Click Compute then click Attach to AI Compute.
  3. Click on the AI compute you want to use from the list.
    Your agent shows AIcompute: (ClusterName) running when it has been successfully attached. This can take up to several minutes.

Edit an AI Compute

You can modify the configuration settings of an AI compute cluster through the Edit operation.

Changing the OCPU or memory for your AI compute can take several minutes to complete. You may also interrupt long running queries in your agent.
  1. On the Home page, navigate to your workspace.
  2. Click on Compute then click on the AI Compute tab.
  3. Next to the AI compute you want to modify, click Actions three dot icon Actions then click Edit. You can also click the name of the AI compute then click Edit in the top-right.

    Compute page open on the AI Compute tab. The actions menu for an AI compute is open and Edit is highlighted.


    AI Compute page open. The Edit button in the top right of the screen is highlighted.

  4. Modify the name, description, number of compute replicas, OCPU count per replica, and memory per replica. Changes to AI compute are completed with no downtime.
  5. Click Update. Your AI compute state displays as Updating while the update is in progress.

Detach an Agent from AI Compute

You can detach an agent from an AI compute. Detaching the AI compute removes the agent code running on the attached compute and prevents testing.

Note:

Detaching an agent from AI compute does not delete the agent. You can resume building and testing the agent by attaching it to the same or different AI compute.
  1. On the Home page, navigate to your agent.
  2. Click Compute then click Detach from AI Compute.

    Cropped image of the top of Agent page. Compute action menu is open with Detach from AI Compute highlighted

Delete an AI Compute

You can delete AI compute clusters that are unused or no longer needed.

  1. Navigate to your workspace and click Compute then click the AI Compute tab.
  2. Next to the AI compute you want to delete, click Actions three dot icon Actions and click Delete.
  3. Click Delete.

    AI compute page with Actions menu open and Delete highlighted

Stop an AI Compute

You can stop an AI compute cluster to stop all agents running on the AI compute. The compute is freed and the metering stops.

We recommended stopping an AI compute whenever the agents hosted on the compute are not in use.
  1. On the Home page, navigate to your workspace.
  2. Click on Compute then click on the AI Compute tab.
  3. Next to the AI compute you want to stop, click Actions three dot icon Actions and click Stop.

    AI compute page with Actions menu open ans Stop highlighted

Start an AI Compute

You can start an AI compute cluster that has been previously stopped.

  1. On the Home page, navigate to your workspace.
  2. Click on Compute then click on the AI Compute tab.
  3. Next to the AI compute you want to start, click Actions three dot icon Actions and click Start.

    AI compute page with Actions menu open and Start highlighted

Restart an AI Compute

You can restart an active AI compute cluster to pick up recent changes or updates.

Note:

Agents hosted on active compute will be interrupted when restarting.
  1. On the Home page, navigate to your workspace.
  2. Click on Compute then click on the AI Compute tab.
  3. Next to the AI compute you want to start, click Actions three dot icon Actions and click Restart.

    AI compute page with Actions menu open and Restart highlighted