Bring Your Own Models
If you have models you want to use instead of those curated by Data Science, you can bring them into AI Quick Actions from Object Storage by registering the model.
- Register a verified model.
- Register an unverified model.
The difference between a service curated model and a verified model is that, for a verified model, you must download the model artifact into Object Storage and register the model in AI Quick Actions before using it.
Service Managed Inference Containers
Two inference containers are available to use with Bring Your Own Model.
For curated and verified models, Data Science has tested which inference container works best with each model and so the inference container can't be chosen. For unverified models, you must decide which inference container is most suitable for each model. Two service managed inference containers are available, one for models compatible with inference engine vLLM 0.4.1 and another container for models compatible with TGI 2.0.1.
Dowload Model Artifacts
Before you can register a model in AI quick ACtions, you need to download the model artifacts, such as scoring logic, ML model, and dependent files, from the model source and upload them to a versioned OCI Object Storage bucket. If you don't have a versioned Object Storage bucket, follow these steps to create one. Then you can upload the model artifacts to Object Storage.
Register Verified Models
Data Science has models you can choose to use that have been tested, but you need to download the model artifacts to Object Storage.
Follow the steps in Prerequisites. Steps 2 and 3 show different ways of registering a verified model. For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI
This task can't be performed using the API.
Register Unverified Models
Follow these steps to use models that haven't been tested by Data Science.
Follow the steps in Prerequisites. For a complete list of parameters and values for AI Quick Actions CLI commands, see AI Quick Actions CLI
This task can't be performed using the API.