23 AI Compute

This chapter covers the use of specialized computing resources for powering AI agents in Oracle AI Data Platform Workbench.

An AI compute cluster lets you host agents in AI Data Platform Workbench workspaces. AI computes can be attached to agents to run the playground experience and host deployed agents for production workloads.

Where to Find AI Compute

You manage AI compute from the Compute page, accessible from the left navigation pane in your AI Data Platform Workbench. Click the AI compute tab to see the AI compute clusters available in your workspace.

The AI compute tab is the main landing page for AI compute resources in the currently selected workspace. It provides table actions, filtering, sorting, and status information.

Column or Control	Purpose
Filter	Search the table for a resource by name or visible text.
Add (+)	Start creation of a new AI compute.
Delete	Remove selected AI computes.
AI compute name	Name of the compute resource. Click on the AI compute name to open the resource details page.
State	Current lifecycle state, such as Creating, Active, or Updating.
# of replicas	Number of compute replicas associated with the AI compute.
# of agents	Number of agents hosted in the AI compute. The count is shown as a link when agents are present.
Configuration	Summary of the compute shape of the replicas in the AI Compute, such as 1 OCPU, 16 GB or 2 OCPU, 32 GB.
Updated by / Updated on / Created on	Audit information for the most recent update and creation time.
Action menu (...)	Open resource-specific actions for the selected row.

Note:

A green check indicates an Active resource. A spinner indicates an in-progress operation such as Creating or Updating. Wait for a resource to be Active before relying on it for production work.

Clicking the name of an AI compute in the AI compute tab allows you to view detailed information on that AI compute. The AI compute view has four tabs: Agents, Details, Compute utilization, and Permissions.

Agents Tab

The Agents tab lists agents that are hosted in the AI Compute resource. If an agent is not deployed, the AI compute is hosting the playground experience necessary for iterative development and testing. If the agent is deployed, the AI computed is hosting the agent deployment endpoint for production workloads.

Agents Tab Item	Description
Filter by agent name	Search the associated agent list.
Agent name	Name of the agent associated with the compute resource.
Authoring mode	How the agent was authored. Values of Visual or Code.
Deployment	Deployment state. Agents hosted in the AI compute can either be not deployed (playground is hosted in AI compute) or deployed (when the compute is hosting the deployment endpoint of the agent)
URI / URI State	Endpoint URI information when an agent is deployed and has a URI.
AI Compute	The compute associated with the production endpoint of the agent.
Created on / Updated on / Updated by	Creation and update metadata for the agent.

Details Tab

The Details tab summarizes the configured shape and storage for each replica of the AI compute.

Details Tab Item	Description
Driver shape	The GPU shape selected for the AI Compute. AMD is the driver shape for all AI Compute.
OCPUs	The selected number of OCPUs for the AI Compute.
Memory (GB)	The AI compute memory in GB.
Block volume	The block volume size of the AI compute in GB.

Compute Utilization Tab

Use the Compute utilization tab to review CPU, memory, and network utilization over time. This is useful when deciding whether to scale replicas, OCPUs, or memory up or down. There is no downtime during a scale out/in AI compute operation.

Note:

Each time series corresponds to a different replica. For example, if you create an AI compute with two replicas, two time series will be displayed just like the screenshot above shows.

Compute Utilization Tab Item	Description
Applied Filters	Choose the start and end date/time for the utilization view.
View	Choose the metric. Options are: CPU Utilization Memory Utilization Network Received Bytes Network Transmitted Bytes
Time period	Choose between a custom time period or the last 7 days.
Replica ID	Filter metrics to a specific replica ID, or show All compute replicas.
Interval	Choose the aggregation interval for the chart. Options are Auto, 1min, 5 min, 15min, 30min, and 60min.

Permissions Tab

The permission tab provides a table of all the principals that have permission to this AI compute. Use this tab to add principals, change the permission level, or otherwise modify permissions.

Permissions Tab Item	Description
Principal name	Name of the principal or role.
Principal type	Principal type. USER or ROLE.
Permission	Permission level.
Will be inherited	If permissions granted for a parent object grant permissions to contained objects.

Create an AI Cluster

You can create AI compute clusters to run AI agents in your AI Data Platform.

1. On the Home page, navigate to your workspace.
2. Click Create then click AI Compute.
3. Provide a name and description for your AI compute cluster.
4. Set the number of compute replicas. Replicas scale the compute pool. Each replica hosts a copy of all the agents hosted in the AI compute.

   Note:

   The maximum number of replicas is 10. Contact your Oracle representative if you need to increase your replica limit.
5. Set the number of OCPUs in each replica. You can choose 1, 2, 4, 8, 16, 32, or 64 OCPUs.
6. Set the memory (in GB) per replica. The allowed range of memory varies based on the number of OCPUs:

   OCPUs	Memory
   1 OCPU	16 or 32 GB
   2 OCPUs	16, 32, or 64 GB
   4 OCPUs	32, 64, or 128 GB
   8 OCPUs	32. 64, 128, or 256 GB
   16 OCPUs	64, 128, 256, or 512 GB
   32 OCPUs	128, 256, or 512 GB
   64 OCPUs	256, 512, or 1024GB

7. Review the corresponding AIDP Units that such an AI compute configuration costs per hour.
8. Click Create. The new resource appears in the list in the Creating state. The state changes to Active after provisioning completes.

Edit an AI Cluster

You can modify the configuration settings of an AI compute through the Edit operation.

Changing the OCPU or memory for you AI compute cluster can take several minutes to complete. You may also interrupt long running queries in your agent.

1. On the Home page, navigate to your workspace.
2. Click on Compute then click on the AI Compute tab.
3. Next to the AI compute cluster you want to modify, click Actions then click Edit. You can also click the name of the AI compute then click Edit in the top-right.
   
   

   

   
   
   

   

   
   
4. Modify the name, description, number of compute replicas, OCPU count per replica, and memory per replica. Changes to AI compute are completed with no downtime.
5. Click Update. Your AI compute state displays as Updating while the update is in progress.

Delete an AI Cluster

You can delete AI compute clusters that are unused or no longer needed.

1. Navigate to your workspace and click Compute then click the AI Compute tab.
2. Next to the AI cluster you want to delete, click Actions and click Delete.
3. Click Delete.
   
   

   

   
   

Stop an AI Cluster

You can stop an AI compute to stop all agents running on the AI compute. The compute is freed and the metering stops.

We recommended stopping an AI compute whenever the agents hosted on the compute are not in use.

1. On the Home page, navigate to your workspace.
2. Click on Compute then click on the AI Compute tab.
3. Next to the AI cluster you want to stop, click Actions and click Stop.
   
   

   

   
   

Start an AI Cluster

You can start an AI compute that has been previously stopped.

1. On the Home page, navigate to your workspace.
2. Click on Compute then click on the AI Compute tab.
3. Next to the AI cluster you want to start, click Actions and click Start.
   
   

   

   
   

Restart an AI Cluster

You can restart an active AI compute to pick up recent changes or updates.

Note:

Agents hosted on active compute will be interrupted when restarting.

1. On the Home page, navigate to your workspace.
2. Click on Compute then click on the AI Compute tab.
3. Next to the AI cluster you want to start, click Actions and click Restart.