OCI GPU Quick Start: NVIDIA B200

This document provides hardware specifications, supported OS images, onboarding verification, sample benchmarks, and best-practices for OCI deployments using the NVIDIA B200 GPU shape.

At a Glance

• Shape: BM.GPU.B200.8
• GPU configuration: 8 x NVIDIA B200
• Recommended OS baseline: Oracle Linux 9+ or Ubuntu Linux 22.04+
• Recommended software baseline: DOCA 3.0.0+, NVIDIA Driver 570+ (Open), CUDA 12.9+, NCCL 2.25.1+
• Primary verification command: nvidia-smi
• Operational profile: single-node and multi-node AI training workloads

When To Use This Shape

Use this shape when you need dense single-node GPU capacity with a straightforward path into multi-node NVIDIA training workloads.

It is best suited for customers who want large local GPU memory footprint, standard x86 operational patterns, and a validated image path before moving into larger Blackwell-scale deployments.

First 15 Minutes

1. Launch the shape with one of the approved images from the Provided Images table.
2. Run nvidia-smi and confirm that all eight B200 GPUs are visible.
3. Run the CUDA container smoke test in Hello World CUDA Container.
4. Review the OKE starter links or move directly into the first NCCL validation step.

Table of Contents

• OCI GPU Quick Start: NVIDIA B200
  • At a Glance
  • When To Use This Shape
  • First 15 Minutes
• Table of Contents
• Hardware Specifications
• Recommended Operating Systems
  • Recommended Software Version
  • Custom OS image Creation with Packer
  • Provided Images
• Hello World Verification
  • Hello World CUDA Container
• Performance Benchmarks
  • NCCL
  • Model Inference Performance
    • Training Numbers
• OKE GPU Getting Started
• Troubleshooting
  • Health Checks
  • Further Reading \& Support

Hardware Specifications

*See the OCI Compute Shapes Docs for up-to-date details.

Recommended Operating Systems

• Oracle Linux 9+\ • Ubuntu Linux 22.04+

Recommended Software Version

• DOCA 3.0.0+\ • NVIDIA Driver 570+ (Open)\ • CUDA 12.9+\ • NCCL 2.25.1+\ • HPCX 2.21+\ • Oracle Cloud Agent 1.51.0+

Custom OS image Creation with Packer

To build your images using packer clone the OCI HPC Images repo and run the commands found there OCI HPC Images GitHub Repo.

Provided Images

It is recommended to use Oracle provided or organizationally-approved images for security and supportability. See OS Images Table for region-specific OCIDs and version info.

Hello World Verification

Run nvidia-smi to verify that all eight GPUs are visible and healthy:

nvidia-smi

Hello World CUDA Container

docker run --rm --gpus all nvcr.io/nvidia/cuda:13.1.0-base-ubuntu24.04 nvidia-smi

Performance Benchmarks

NVIDIA publishes their NCCL (Nvidia Collective Communication Library) software as a toolkit for pre-defined  routines which are optimized for their hardware. This software is meant to accelerate Artificial Intelligence &  Machine Learning workloads running on NVIDIA GPU clusters. Detailed documentation can be found here. The NCCL operations which are important for this document are AllReduce and AlltoAll operations.

If the NCCL tests are not on your system, then you can build them using the commands below

git clone <https://github.com/NVIDIA/nccl-tests.git>
cd nccl-tests
make

All tests run as follows with appropriate --np and --hostfile values provided:

mpirun --bind-to numa
        --mca pml ucx
        --mca coll ^hcoll
        -x coll_hcoll_enable=0         
        -x NCCL_DEBUG=WARN
        -x NCCL_MNNVL_ENABLE=1
        -x NCCL_CUMEM_ENABLE=1
        -x UCX_NET_DEVICES=eth0
        -x NCCL_IB_HCA==mlx5_0,mlx5_1,mlx5_2,mlx5_3,mlx5_5,mlx5_6,mlx5_7,mlx5_8
        -x NCCL_NET_PLUGIN=/opt/hpcx-v2.24.1-gcc-doca_ofed-ubuntu24.04-cuda13-aarch64/nccl_rdma_sharp_plugin/lib/[libnccl-net.so](http://libnccl-net.so).0
        -x NCCL_NVLS_ENABLE=1
        -x NCCL_SOCKET_IFNAME=eth0
        -x NCCL_IB_GID_INDEX=3
        -x NCCL_IB_TC=41
        -x NCCL_IB_SL=0
        -x NCCL_IB_TIMEOUT=22
        -x RX_QUEUE_LEN=8192
        -x IB_RX_QUEUE_LEN=8192
        -x HCOLL_ENABLE_MCAST_ALL=0
        -x NCCL_BUFFSIZE=16777216
        -x NCCL_IB_QPS_PER_CONNECTION=4
        -x NCCL_IB_SPLIT_DATA_ON_QPS=0
        -x NCCL_NET_GDR_C2C=1
        -x NCCL_MNNVLS_ENABLE=1
        -x NCCL_DMABUF_ENABLE=1
        --np xxx --hostfile ./yyy all_reduce_perf -b 512K -e 16G -f 2 -g 1 -n 50

NCCL

For guidance on installing and running additional NCCL benchmarks on this family, see the NCCL user guide.

The source material for B200 includes both all-reduce and all-to-all coverage for single-node and multi-node deployments. When nccl-tests is installed, the baseline single-node all-reduce command is:

/opt/oci-hpc/nccl-tests/build/all_reduce_perf -b 512k -e 8G -f 2 -g 8

Model Inference Performance

15 billion parameter variant (FP8/BF16)

• At least 16 GPUs with at least 180GB memory each.

340 billion parameter variant (FP8/BF16)

• At least 128 GPUs with at least 180GB memory each.
• The B200 recipes listed below progressively increase GPU count, with configurations weak-scaled to match.

• Strong Scaling, where GBS remains constant across increasing GPU counts and reducing gradient accumulation.

Training Numbers

Example output:

Train Step Timing and TFLOPS Analysis (iterations 35-44)
================================================================================
Experiment                                                                                  Status Time Mean (s) Time Std (s) TFLOPS_per_GPU Mean TFLOPS_per_GPU Std
------------------------------------------------------------------------------------------ -------- ------------- ------------ ------------------- ------------------
pretrain_nemotron4_340b_fp8_gpus128_tp8_pp4_cp1_vp12_mbs1_gbs512_389757                     Success        21.542        0.010             1600.74               0.78

To obtain throughput as a tokens per second measurement, follow this formula:

(sequence length) * (global batch size) / (training_step_timing) = (throughput in tokens per second)

E.g. 4096 * 256 / 2.693 = 389371

To calculate time to train estimate:

(total tokens) / (throughput in tokens per second) / (number of seconds in a day) = (time to train in days) 

E.g. 1e12 / 389371 / 86400 = 29.7 days

To calculate the model flops utilization (MFU):

MFU = (global batch size) * (model flops) / (training step time) / (number of GPUs) / (peak GPU FLOPS)

The model flops for Nemotron4 15b for GBS=1 is 3.85e14. Calculation shown here.

E.g. NeMotron4 15b BF16 on 64x H100 GPUs (GBS=256)

peak FLOPS for H100 BF16 = 989 TFLOPS
training step time = 2.693 s
model flops = 3.85e14

MFU = 256 * 3.85e14 / 2.693 / 64 / 989e+12 = 57.82%

Peak theoretical throughput across GPUs and Data Types (in TFLOPS)

OKE GPU Getting Started

1. Create OKE Cluster via OCI Console or CLI.
2. Add GPU Node Pool: Choose B200 shape, use golden OS image.
3. Install NVIDIA Device Plugin:

kubectl apply -f https://github.com/NVIDIA/k8s-device-plugin/raw/main/nvidia-device-plugin.yml

1. Run GPU-powered Kubernetes workloads: Example pod resource spec:

resources:
  limits:
    nvidia.com/gpu: 1

1. Verify with Hello World CUDA container pod. See detailed instructions in OKE Onboarding Guide .

Useful B200-specific OKE starting points in oci-hpc-oke:

• B200 NCCL test manifest
• Running active health checks on OKE
• Running ib_write_bw on OKE

Troubleshooting

Here you can find suggested troubleshooting methods.

Health Checks

DR HPC Based Scrips & or OCI GPU Scanner Content

Further Reading & Support

1. Blog on B200 hardware & its multi node performance AI for Employees