Compatible NVIDIA Nemotron Model
You can import large language models from Hugging Face and OCI Object Storage buckets into OCI Generative AI, create endpoints for those models, and use them in the Generative AI service.
NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, suited for agentic workflows, long-context reasoning, high-volume workloads (for example, IT ticket automation), tool use, and RAG. For the compatible models on Hugging Face, see the links in the following table. Learn about NVIDIA Nemotron.
NVIDIA Nemotron
| Hugging Face Model ID | Model Capability | Minimum Dedicated AI Cluster Unit Shape |
|---|---|---|
| nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 | TEXT_TO_TEXT |
|
| nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 | TEXT_TO_TEXT |
|
| nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 | TEXT_TO_TEXT | B200_X1 |
| nvidia/Llama-3.1-Nemotron-Nano-8B-v1 | TEXT_TO_TEXT | A100_80G_X1 |
| nvidia/Llama-3.1-Nemotron-70B-Instruct-HF | TEXT_TO_TEXT | A100_80G_X4 |
| nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 | TEXT_TO_TEXT | B200_X4 |
| nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 | TEXT_TO_TEXT | H100_X8 |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 | TEXT_TO_TEXT | H100_X4 |
| nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 | TEXT_TO_TEXT | A100_80G_X1 |
- For imported models, you can use the native context length specified by the model provider. However, the effective maximum context length is limited by the underlying hardware setup that you select for the hosting dedicated AI clusters in OCI Generative AI. To take full advantage of a model's native context length, you might need to provision more hardware resources.
- Use the fine-tuned models only if they match the compatible base model's transformer version and have a parameter count within ±10% of the original.
- For available hardware shapes, see Hardware Unit Shapes for Imported Models.
- For steps on how to deploy the imported models, see Managing Imported Models.
Billing for Dedicated AI Clusters Hosting Imported Models
Dedicated AI clusters that host imported models don't require the 744-unit-hour minimum commitment that applies to dedicated AI clusters hosting pretrained models available in OCI Generative AI. Instead, an imported-model dedicated AI cluster has a minimum billable duration of one hour. For example, a cluster that uses one AI unit and runs for 30 minutes is billed for one AI unit-hour.
After the first hour, charges are based on the cluster's actual running time and are prorated to the millisecond. For example, a cluster that runs for one hour and one minute is billed for that duration, rather than for two full hours.
OCI sends usage records to Metering and Billing every five minutes while the cluster is running. If the cluster is deleted before the one-hour minimum is reached, OCI continues reporting billable usage until the minimum is met. If the cluster runs for more than one hour, OCI continues sending usage records every five minutes until the cluster is deleted.