Package com.oracle.coherence.rag.model.ollama

package com.oracle.coherence.rag.model.ollama
Ollama local model integration for the Oracle Coherence RAG framework.

This package provides integration with Ollama for running local open-source AI models. Ollama allows you to run large language models locally without requiring external API calls or internet connectivity.

Components include:

The Ollama integration provides:

  • Integration with locally running Ollama instances
  • Support for popular open-source models (Llama, Mistral, CodeLlama, etc.)
  • Local inference without external dependencies
  • Configurable connection parameters
  • Error handling and model availability checking

Popular models supported by Ollama:

  • Llama 2 (7B, 13B, 70B parameter variants)
  • Mistral (7B, 8x7B mixture of experts)
  • CodeLlama (specialized for code generation)
  • Alpaca (instruction-tuned models)
  • Vicuna (conversation-optimized models)

Local deployment advantages:

  • No external API dependencies
  • Data privacy and security
  • No usage costs or rate limits
  • Offline operation capability
  • Full control over model parameters

Example usage: 


 ModelProvider provider = new OllamaModelProvider();
 ChatModel chatModel = provider.getChatModel("llama2");
 
 ChatResponse response = chatModel.generate("Explain machine learning");

Configuration can be provided through:

  • System property: ollama.base.url (default: http://localhost:11434)
  • Environment variable: OLLAMA_BASE_URL
  • CDI configuration beans

The provider automatically handles model loading, parameter configuration, and error recovery for locally running Ollama instances. It requires Ollama to be installed and running on the local machine or accessible network endpoint.

Since:
25.09
Author:
Aleks Seovic 2025.07.04