Implement Oracle Cloud Infrastructure Generative AI based on Cohere Large Language Model
Oracle Cloud Infrastructure Generative AI (OCI Generative AI) is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases for text generation. You can use the playground - an interface in the Console for exploring the hosted pretrained and custom models without writing a single line of code or create and host your own fine-tuned custom models based on your own data on dedicated AI clusters.
- Generation: Give instructions to generate text or extract information from your text.
- Summarization: Summarize text with your instructed format, length, and tone.
- Embedding: Convert text to vector embeddings to use in applications for semantic searches, text classification, or text clustering.
Architecture
You can use the OCI Generative AI playground to try pretrained models, run your prompts, adjust the parameters, update your prompts, and rerun the models until you are happy with the results. After this you can copy the code from the Console into your applications.
You can also create a copy of a pretrained model, add your own training dataset, and let the OCI Generative AI service fine-tune the model. OCI Generative AI uses powerful dedicated AI clusters specially sized for fine-tuning. These clusters belong only to your tenancy. After your model is fine-tuned, we create an endpoint for the custom model and host that model on a dedicated AI cluster designed for hosting.
Advantages of building a LLM on OCI
Generative AI Service: Oracle delivers native generative AI services in collaboration with Cohere, a leading enterprise AI platform equipped with advanced language comprehension for building the next generation of enterprise applications. OCI Generative AI is a fully managed service available via an API to seamlessly integrate these versatile language models into a wide range of use cases, including writing assistance, summarization, and chat.
Dedicated AI Clusters: Dedicated AI clusters are compute resources that you can use for fine-tuning custom models or for hosting endpoints for custom models. The clusters are dedicated to your models and not shared with users in other tenancies. Custom Model OCI Generative AI lets you refine the models using your own data.
Custom Model: OCI Generative AI lets you refine the models using your own data.
Note:
New AI vector similarity search feature is available in Oracle Database 23ai.- OCI Object Storage for data storage.
- Oracle Cloud Infrastructure Data Science Workspace for model building.
- OCI Generative AI with GPUs for compute.
- Oracle APEX Application Development tool for UI.
The following diagram illustrates this reference architecture.
oci-generative-ai-llm-arch-oracle.zip
The architecture has the following components:
- Region
An Oracle Cloud Infrastructure region is a localized geographic area that contains one or more data centers, called availability domains. Regions are independent of other regions, and vast distances can separate them (across countries or even continents).
- Virtual cloud network (VCN) and subnets
A VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you control over your network environment. A VCN can have multiple non-overlapping CIDR blocks that you can change after you create the VCN. You can segment a VCN into subnets, which can be scoped to a region or to an availability domain. Each subnet consists of a contiguous range of addresses that don't overlap with the other subnets in the VCN. You can change the size of a subnet after creation. A subnet can be public or private.
- Object storage
Object storage provides quick access to large amounts of structured and unstructured data of any content type, including database backups, analytic data, and rich content such as images and videos. You can safely and securely store and then retrieve data directly from the internet or from within the cloud platform. You can scale storage without experiencing any degradation in performance or service reliability. Use standard storage for "hot" storage that you need to access quickly, immediately, and frequently. Use archive storage for "cold" storage that you retain for long periods of time and seldom or rarely access.
- Data Science
Oracle Cloud Infrastructure Data Science is a fully managed, serverless platform that data science teams can use to build, train, and manage machine learning (ML) models on Oracle Cloud Infrastructure (OCI). It can easily integrate with other OCI services such as Oracle Autonomous Data Warehouse, Oracle Cloud Infrastructure Object Storage, and more. You can build and evaluate high-quality machine learning models that increase business flexibility by putting enterprise-trusted data to work quickly, and you can support data-driven business objectives with easier deployment of ML models.
- Oracle Database 23ai (AI Vector Search)
Oracle Database 23ai delivers the most complete and simple converged database for developers looking to build new microservice, graph, document, and relational applications.
Oracle has added semantic search capabilities using AI vectors to Oracle Database 23ai. The collection of features, called AI Vector Search, includes a new vector data type, vector indexes, and vector search SQL operators that enable the Oracle Database to store the semantic content of documents, images, and other unstructured data as vectors, and use these to run fast similarity queries. For more information, see the Blog link in the Explore More section.