OCI GPU Quick Starts: AMD MI355X Instinct GPUs

This document provides hardware specifications, supported OS images, onboarding verification, sample benchmarks, and best-practices for OCI deployments using the AMD MI355X Pollara GPU bare-metal shape.

MI355X shapes represent a specialized class of heterogeneous accelerated compute in Oracle Cloud Infrastructure (OCI), built around the AMD Instinct MI355X GPU. These shapes provide 8 MI355X GPUs per node, each with 288 GiB of high-bandwidth HBM memory, targeting large-scale AI/ML workloads and HPC.

MI355X systems leverage AMD’s XGMI (Inter-GPU Express Global Memory Interconnect) for direct GPU-to-GPU connectivity within each node. On a typical MI355X system, all 8 GPUs are fully connected via XGMI, providing a single-hop, high-bandwidth, low-latency peer-to-peer path between any pair of GPUs. This topology is symmetrical, meaning all GPUs enjoy equal connectivity for workloads that depend on frequent collective or point-to-point communication.

Pollara

Pollara nodes are a variant of the MI355X bare metal GPU shape family in OCI that pair 8x AMD Instinct MI355X GPUs with the AMD Pensando™ Pollara 400 AI NIC for high-bandwidth scale-out training and HPC workloads over Ethernet (RoCE/RDMA). These systems are designed for multi-node collective performance and high-throughput east–west traffic.

Within each node, all 8 MI355X GPUs are connected via AMD XGMI, enabling a symmetrical, single-hop, high-bandwidth GPU-to-GPU topology suitable for communication-heavy workloads (e.g., model parallelism, pipeline parallelism, large batch training, and dense collectives).

At a Glance

• Shape family: MI355X-Pollara
• GPU configuration: 8 x AMD Instinct MI355X with Pollara networking
• Recommended OS baseline: Ubuntu Linux 22.04 or Ubuntu Linux 24.04
• Recommended software baseline: ROCm 7.0.2 or 7.2.0, RCCL 2.26.6, OpenMPI aligned with the approved OCI image
• Primary verification command: docker pull rocm/pytorch:latest plus GPU visibility checks
• Operational profile: scale-out AI and HPC over Ethernet with Pollara NICs

When To Use This Shape

Use this shape when you specifically need the MI355X Pollara networking path for scale-out Ethernet-based AI or HPC workloads.

It is best suited for customers validating Pollara-backed multi-node collective behavior, comparing Pollara against the standard MI355X variant, or standardizing on the OCI-approved Pollara image path.

First 15 Minutes

1. Launch the shape with the approved Pollara image from the Provided Images table.
2. Pull the ROCm container and verify GPU visibility in Hello World Verification.
3. Run rdma link and confirm the expected ionic* back-end interfaces are up.
4. Run the first single-node RCCL check before moving into the larger sweep or multi-node validation.

Table of Contents

• OCI GPU Quick Starts: AMD MI355X Instinct GPUs
  • Pollara
  • At a Glance
  • When To Use This Shape
  • First 15 Minutes
• Table of Contents
• Hardware Specifications
• Recommended Operating Systems
  • Recommended Software Version
  • Custom OS image Creation with Packer
  • Provided Images
• Hello World Verification
• Performance Benchmarks
  • RCCL \& Model Inference Performance
    • All Reduce - Single Node
    • All 2 All - Single Node
    • All Reduce - 2 Nodes
    • All 2 All - 2 Nodes
    • All Reduce - 4 Nodes
    • All 2 All - 4 Nodes
    • LLM Inference Performance Numbers
      • Example Results
  • Sweep Results
• OKE GPU Getting Started
• Troubleshooting
  • Confirm Hardware \& Drivers
  • RDMA Link
  • IB Write
• Further Reading \& Support
  • AMD Technical Documents
  • OCI AMD 355X Technical Docs \& Supported Solutions

Hardware Specifications

Note: Pollara nodes may use a different system image than ConnectX-based nodes. Align to the OCI-provided image for the rack.

See the OCI Compute Shapes Docs for up-to-date details.

Recommended Operating Systems

• Ubuntu Linux 22.04\ • Ubuntu Linux 24.04\ • Use the OCI-provided Pollara image for the rack whenever possible to keep kernel, driver, and networking components aligned

Recommended Software Version

• ROCm: 7.0.2 or 7.2.0\ • RCCL: 2.26.6\ • Oracle Cloud Agent: 1.55.0+\ • OFED: MLNX OFED 5.9 for Ubuntu 22.04 and 24.10 for Ubuntu 24.04 (for the 2 x CX7 front-end NICs)\ • OpenMPI: 4.1.6, 5.0.8\ • amd-anp (amd ainic network plugin designed to enhance performance for RCCL collective communications library)\ • amd-argus

Custom OS image Creation with Packer

To build your images using packer clone the OCI HPC Images repo and run the commands found there OCI HPC Images GitHub Repo.

Provided Images

It is recommended to use Oracle-provided or organizationally-approved images for security and supportability. See OS Images Table for region-specific OCID's and version info.

Note: The current Ubuntu 24.04 Pollara image tracks the ROCm 7.2 image line. Validate your target workload against the approved rack image before standardizing on it for production.

Hello World Verification

You will need docker installed on the OS image before you can run this command. More information on different functions that can be run is found here

docker pull rocm/pytorch:latest
docker run -it \
    --cap-add=SYS_PTRACE \
    --security-opt seccomp=unconfined \
    --device=/dev/kfd \
    --device=/dev/dri \
    --group-add video \
    --ipc=host \
    --shm-size 8G \
    rocm/pytorch:latest

Performance Benchmarks

The section below has the base line numbers achieved and how to reproduce this with MI355X. When you run the same there is typically a variance of 5%-8% which is acceptable.

For guidance on installing and running RCCL benchmarks, see the AMD RCCL documentation.

RCCL & Model Inference Performance

All Reduce - Single Node

#
#                                                              out-of-place                       in-place          
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
# <smaller message sizes truncated>
  8589934592    2147483648     float     sum      -1    36863  233.02  407.79      0    36835  233.20  408.10      0
 17179869184    4294967296     float     sum      -1    73544  233.60  408.80      0    73681  233.17  408.04      0
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 169.455 
#
# Collective test concluded: all_reduce_perf

All 2 All - Single Node

#                                                              out-of-place                       in-place          
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
# <smaller message sizes truncated>
  8589934592     268435456     float    none      -1    20839  412.20  360.67      0    20893  411.14  359.75    N/A
 17179869184     536870912     float    none      -1    41626  412.72  361.13      0    41546  413.52  361.83    N/A
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 133.643 

All Reduce - 2 Nodes

#                                                              out-of-place                       in-place          
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
# <smaller message sizes truncated>
  8589934592    2147483648     float     sum      -1    43674  196.68  368.78      0    43684  196.64  368.70      0
 17179869184    4294967296     float     sum      -1    86980  197.52  370.34      0    86978  197.52  370.35      0
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 135.701 
#
# Collective test concluded: all_reduce_perf

All 2 All - 2 Nodes

#                                                              out-of-place                       in-place          
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
# <smaller message sizes truncated>
  8589934592     134217728     float    none      -1    92988   92.38   86.60      0    92968   92.40   86.62    N/A
 17179869184     268435456     float    none      -1   185711   92.51   86.73      0   185928   92.40   86.63    N/A
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 35.7757 
#
# Collective test concluded: alltoall_perf

All Reduce - 4 Nodes

#                                                              out-of-place                       in-place          
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
# <smaller message sizes truncated>        
  8589934592    2147483648     float     sum      -1    44952  191.09  370.24      0    44947  191.11  370.28      0
 17179869184    4294967296     float     sum      -1    89839  191.23  370.51      0    89846  191.22  370.48      0
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 117.167 
#
# Collective test concluded: all_reduce_perf

All 2 All - 4 Nodes

#                                                              out-of-place                       in-place          
#       size         count      type   redop    root     time   algbw   busbw #wrong     time   algbw   busbw #wrong
#        (B)    (elements)                               (us)  (GB/s)  (GB/s)            (us)  (GB/s)  (GB/s)       
# <smaller message sizes truncated>
  8589934592      67108864     float    none      -1   165769   51.82   50.20      0   164522   52.21   50.58    N/A
 17179869184     134217728     float    none      -1   328749   52.26   50.63      0   327875   52.40   50.76    N/A
# Errors with asterisks indicate errors that have exceeded the maximum threshold.
# Out of bounds values : 0 OK
# Avg bus bandwidth    : 20.8042 
#
# Collective test concluded: alltoall_perf

LLM Inference Performance Numbers

The below output is for vLLM Throughput running the below command and the container release. For latest container version refer here.

Example output:

docker pull rocm/7.x-preview:rocm7.2_preview_ubuntu_22.04_vlm_0.10.1_instinct_20251029

While vLLM can download model weights at runtime, it’s recommended to download ahead of time. You will need:

A valid Hugging Face access token. Remember to set HF_TOKEN to your access token.

Access granted to the specific model from your Hugging Face account

model=openai/gpt-oss-120b

pip install huggingface_hub[cli] hf_transfer hf_xet
HF_HUB_ENABLE_HF_TRANSFER=1 \
HF_HOME=/data/huggingface-cache \
HF_TOKEN="<HF_TOKEN>" \ # Replace with your HF_TOKEN Hugging Face access token.
huggingface-cli download ${model} --exclude "original/*"

Start the container on MI355X. You will be inside the container once this command is successful.

docker run -it \
  --ipc=host \
  --network=host \
  --privileged \
  --cap-add=CAP_SYS_ADMIN \
  --device=/dev/kfd \
  --device=/dev/dri \
  --cap-add=SYS_PTRACE \
  --security-opt seccomp=unconfined \
  -v /data:/data \
  -e HF_HOME=/data/huggingface-cache \
  -e HF_HUB_OFFLINE=1 \
  --name vllm-server \
  rocm/7.x-preview:rocm7.2_preview_ubuntu_22.04_vlm_0.10.1_instinct_20251029

Start the vLLM server

model=openai/gpt-oss-120b
max_model_len=10368           # 1.125 x (input sequence length + output sequence length); e.g. 1.125 x (8192 + 1024) = 10368.
max_seq_len_to_capture=10368  # Beneficial to set this to max_model_len.
max_num_seqs=1024             # Set to max_concurrency of the client to get better throughput.
tensor_parallel_size=8

export VLLM_USE_AITER_UNIFIED_ATTENTION=1
export VLLM_ROCM_USE_AITER_MHA=0
export VLLM_ROCM_USE_AITER_FUSED_MOE_A16W4=1

vllm serve ${model} \
    --port 8000 \
    --swap-space 64 \
    --max-model-len ${max_model_len} \
    --tensor-parallel-size ${tensor_parallel_size} \
    --max-num-seqs ${max_num_seqs} \
    --gpu-memory-utilization 0.95 \
    --max-seq-len-to-capture ${max_seq_len_to_capture} \
    --compilation-config '{"compile_sizes":[1,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128,256,512,1024,2048,8192] , "cudagraph_capture_sizes":[1,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128,136,144,152,160,168,176,184,192,200,208,216,224,232,240,248,256,264,272,280,288,296,304,312,320,328,336,344,352,360,368,376,384,392,400,408,416,424,432,440,448,456,464,472,480,488,496,504,512,520,528,536,544,552,560,568,576,584,592,600,608,616,624,632,640,648,656,664,672,680,688,696,704,712,720,728,736,744,752,760,768,776,784,792,800,808,816,824,832,840,848,856,864,872,880,888,896,904,912,920,928,936,944,952,960,968,976,984,992,1000,1008,1016,1024,2048,4096,8192] , "cudagraph_mode": "FULL_AND_PIECEWISE"}' \
    --block-size=64 \
    --no-enable-prefix-caching \
    --async-scheduling

 # Wait for model to load and server is ready to accept requests.

Open another terminal on the same machine, connect to your running vllm-server container, and run the benchmark with the appropriate options. For example:

# Connect to server
docker exec -it vllm-server bash

Inside the container

# Run the client benchmark
model=openai/gpt-oss-120b
input_tokens=1024 # you can toggle the sizes here
output_tokens=1024 #you can toggle the sizes here
max_concurrency=4
num_prompts=32

python3 /app/vllm/benchmarks/benchmark_serving.py --host localhost --port 8000 \
    --model ${model} \
    --dataset-name random \
    --random-input-len ${input_tokens} \
    --random-output-len ${output_tokens} \
    --max-concurrency ${max_concurrency} \
    --num-prompts ${num_prompts} \
    --percentile-metrics ttft,tpot,itl,e2el \
    --ignore-eos

Example Results

============ Serving Benchmark Result ============
Successful requests:                     32        
Maximum request concurrency:             4         
Benchmark duration (s):                  29.20     
Total input tokens:                      32601     
Total generated tokens:                  32768     
Request throughput (req/s):              1.10      
Output token throughput (tok/s):         1122.33   
Total Token throughput (tok/s):          2238.93   
---------------Time to First Token----------------
Mean TTFT (ms):                          89.77     
Median TTFT (ms):                        51.25     
P99 TTFT (ms):                           522.23    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          3.48      
Median TPOT (ms):                        3.46      
P99 TPOT (ms):                           3.81      
---------------Inter-token Latency----------------
Mean ITL (ms):                           3.49      
Median ITL (ms):                         3.37      
P99 ITL (ms):                            8.25      
----------------End-to-end Latency----------------
Mean E2EL (ms):                          3648.97   
Median E2EL (ms):                        3591.23   
P99 E2EL (ms):                           4080.27   
==================================================

Sweep Results

---

OKE GPU Getting Started

1. Create OKE Cluster via OCI Console or CLI.
2. Add GPU Node Pool as self managed node: More info here. You can use the same OS Image listed above and bootstrap the kubernetes components and registration to Kubernetes control plane. More abut adding it as cloud init script here
3. Install AMD Device Plugin on OKE:

kubectl apply -f https://github.com/AMD/k8s-device-plugin/raw/main/AMD-device-plugin.yml

1. Run GPU-powered Kubernetes workloads: Example pod resource spec:

AMD RVS Test Yaml

1. Verify with Hello World ROCm container pod is deployable and you can see the AMD GPUs within it

kubectl create -f ./amd/MI355X/k8s/hello-world-rocm-oke.yaml

Output

kubectl logs amd-smi

============================================= ROCm System Management Interface =============================================                        
======================================================= Concise Info =======================================================                        
Device  Node  IDs              Temp        Power     Partitions          SCLK     MCLK     Fan  Perf  PwrCap   VRAM%  GPU%                      
              (DID,     GUID)  (Junction)  (Socket)  (Mem, Compute, ID)                                                                             
============================================================================================================================                        
0       3     0x75a3,   21010  54.0°C      1224.0W   NPS1, SPX, 0        2358Mhz  2000Mhz  0%   auto  1400.0W  98%    93%                       
1       5     0x75a3,   56525  54.0°C      1231.0W   NPS1, SPX, 0        2366Mhz  2000Mhz  0%   auto  1400.0W  98%    95%                       
2       4     0x75a3,   28206  55.0°C      1219.0W   NPS1, SPX, 0        2361Mhz  2000Mhz  0%   auto  1400.0W  98%    94%                       
3       2     0x75a3,   28720  56.0°C      1249.0W   NPS1, SPX, 0        2340Mhz  2000Mhz  0%   auto  1400.0W  98%    96%                       
4       7     0x75a3,   25266  52.0°C      1275.0W   NPS1, SPX, 0        2374Mhz  2000Mhz  0%   auto  1400.0W  98%    100%                      
5       9     0x75a3,   20206  56.0°C      1267.0W   NPS1, SPX, 0        2377Mhz  2000Mhz  0%   auto  1400.0W  98%    100%                      
6       8     0x75a3,   16143  54.0°C      1273.0W   NPS1, SPX, 0        2382Mhz  2000Mhz  0%   auto  1400.0W  98%    100%                      
7       6     0x75a3,   8465   54.0°C      1317.0W   NPS1, SPX, 0        2374Mhz  2000Mhz  0%   auto  1400.0W  98%    100%                      
============================================================================================================================                        
=================================================== End of ROCm SMI Log ====================================================    

AMDSMI Tool: 24.6.2+2b02a07 | AMDSMI Library version: 24.6.2.0 | ROCm version: 7.1.2

Troubleshooting

Here you can find suggested troubleshooting methods.

• Confirm Hardware & Drivers
• RDMA Link
• IB Write

Confirm Hardware & Drivers

Use the following commands to confirm that all eight GPUs are visible, the XGMI topology is healthy, and the host NUMA layout matches expectations:

amd-smi
amd-smi topology
numactl --hardware

RDMA Link

To verify Pollara back-end RDMA interfaces and the CX7 front-end interfaces, run:

rdma link

The front-end network should be visible on the mlx5_* interfaces and the Pollara back-end interfaces should appear as ionic*, with links reporting ACTIVE and LINK_UP.

IB Write

Use ib_write_bw on one of the ionic* interfaces for point-to-point network validation.

Server side:

numactl -N 0 -m 0 ib_write_bw -d ionic_7 -i 1 -q 4 -a --report_gbits --tx-depth 1024

Client side:

numactl -N 0 -m 0 ib_write_bw -d ionic_7 -i 1 -q 4 -a --report_gbits --tx-depth 1024 <svr_ip>

Expected performance is approximately 391 Gb/s on a healthy link.

Further Reading & Support

This section has additional reference material which you may find useful.

AMD Technical Documents

1. AMD ROCm Software Compatibility Matrix
2. AMD ROCm Software Examples
3. AMD Kubernetes GPU Operator

OCI AMD 355X Technical Docs & Supported Solutions

1. OCI HPC SLURM Deployed Terraform Stack
2. OCI HPC OKE Deployment Stack