49 Introduction to Coherence RAG

Retrieval Augmented Generation (RAG) is an approach developed to address the limitations of LLMs. RAG combines the strengths of pretrained language models with the ability to retrieve recent and accurate information from a dataset or database in real-time during the generation of responses.

Coherence RAG (Retrieval-Augmented Generation) is designed for massively scalable document ingestion and vector embeddings creation on both CPU and GPU-based hardware. It enables efficient and accurate retrieval-augmented responses by ingesting documents from any source, using both local and remote embedding models.

Additionally, Coherence RAG offers optional integration with external vector stores, providing users with flexibility in storage and retrieval solutions. This combination ensures high-performance document processing, retrieval, and augmentation for large-scale AI applications.

Security Prerequisites

Configure the RAG service’s authenticated identity, model locations and administrative policy before startup or ingestion. Model download and model/configuration administration are separate permissions from document search. Supply credentials through the supported provider configuration and make external downloads an explicit deployment decision.

For the complete configuration and policy, see Securing Coherence RAG, RAG Configuration and Model Download Policy in Securing Oracle Coherence.

• CPU and GPU Support
  
• Local and Remote Model Support
  
• Flexible Document Loaders
  
• Support for External Vector Stores
  

CPU and GPU Support

Coherence RAG supports deployment on both CPU and GPU infrastructures. While CPU-based deployment offers cost-effective scalability for moderate workloads, GPU acceleration significantly enhances performance, making it ideal for high-throughput and low-latency applications. Users can configure their environment based on hardware availability and performance requirements.

Local and Remote Model Support

Coherence RAG supports both local and remote models for embedding and scoring functionalities, allowing users to choose between on-premise execution and cloud-based inference.

Local models are automatically downloaded from Hugging Face and run on each cluster member using ONNX Runtime. These models provide optimized performance without requiring external API calls, ensuring cost efficiency, privacy, and independence from third-party services.

Remote models include OCI GenAI models, OpenAI, Cohere, Anthropic, AWS Bedrock, and any other models supported by LangChain4J. These models offer access to state-of-the-art AI capabilities via API-based cloud services, allowing for enhanced accuracy and broader model selection at the cost of external dependencies and API-related latency.

Flexible Document Loaders

Coherence RAG supports ingestion from a wide range of document sources, offering built-in support for file system and HTTP protocols, along with integrations for OCI Object Storage, AWS S3, Azure Blob Storage, and Google Cloud Storage.

Additionally, it provides a flexible framework that allows customers to extend support to proprietary document repositories such as OpenText, Documentum, SharePoint, Alfresco, and other enterprise content management systems.

Support for External Vector Stores

Coherence RAG enables seamless integration with external vector stores, allowing users to extend storage and retrieval capabilities beyond its built-in infrastructure. Supported external vector stores include Oracle DB 23ai, MySQL HeatWave, and OpenSearch.

Additionally, Coherence RAG provides a flexible framework for integrating with other vector stores, giving users the freedom to work with their preferred solutions for efficient vector search and retrieval, while still benefiting from unparalleled scalability of Coherence RAG document ingestion.)