Note:

Integrate LangChain, OCI Data Science Notebook, OCI with OpenSearch and OCI Generative AI to Accelerate LLM Development for RAG and Conversational Search

Introduction

In this tutorial, we will explore how LangChain, Oracle Cloud Infrastructure (OCI) Data Science Notebook, OCI with OpenSearch, OCI Data Science AI Quick Actions (AI Quick Actions), and OCI Generative AI service integrate seamlessly together to simplify large language models (LLM) application development. These tools accelerate workflows, improve productivity, and enable faster deployment of cutting-edge AI solutions tailored to specific business needs.

In the first tutorial: Tutorial 1: Integrate OCI Search with OpenSearch and LangChain within OCI Data Science Notebook, we discussed the transformative impact of integrating LangChain, OCI with OpenSearch and Hugging Face, demonstrating how these technologies can work with OCI Data Science Notebook to simplify the development and production of semantic search solutions for enterprise businesses. We detailed the end-to-end configuration steps for setting up OCI with OpenSearch, setting up OCI Data Science Notebook, configuring LangChain with notebook session, and deploying a complete use case for semantic search.

Building on that foundation, this tutorial focuses on how seamless integration across LangChain, OCI with OpenSearch, AI Quick Actions, and OCI Generative AI can speed up LLM application development for custom business use cases. We will particularly dive into solutions around conversational search, Retrieval-Augmented Generation (RAG), and question-answering chatbots, showcasing how these tools together empower developers to overcome challenges and deliver innovative, scalable applications with minimal coding.

Note: You can find the complete code samples for this tutorial in our public samples repo here: OCI OpenSearch Service Code Samples.

Benefits of the Integrated Approach

Building applications with LLMs involves multiple steps:

Each step can require specialized tools, intricate pipelines, and significant development effort. By integrating LangChain with OCI services, developers can address these challenges holistically, reducing complexity and enhancing productivity.

Objectives

Prerequisites

Task 1: Create an OCI Data Science Project and Configure Notebooks

OCI Data Science Notebook provides a flexible and interactive platform for developing, testing, and deploying AI applications. The integration of notebooks with OCI services creates a seamless development experience, reducing the need for external tools. With notebooks you can:

To create an OCI Data Science project and configure notebook, see Task 2: Launch a Jupyter Notebook in OCI Data Science.

Task 2: Install LangChain

LangChain is one of the most versatile frameworks for developing LLM-based applications. Its modular design and expansive library provide robust abstractions for:

LangChain’s ability to interface seamlessly with cloud services like OCI Generative AI, OCI with OpenSearch, and OCI Data Science enhances its utility for building enterprise grade applications. LangChain’s flexibility allows developers to rapidly prototype and iterate on applications, making it ideal for RAG and conversational search solutions. Its integrations with OCI services further streamline the development process, enabling developers to focus on delivering value. In the first tutorial, we talked about how to configure LangChain with notebook environment and we went over an end-to-end example of how to use LangChain to process unstructured data, ingest it into OCI with OpenSearch, and perform semantic search. 

Run the following command in your notebook to configure LangChain and other key libraries.

!pip install -U oci oracle_ads langchain langchain-community  langchain-huggingface  opensearch-py pypdf

Task 3: Use LangChain to Pre-process and Chunk your Data

Use the following Python code.

import os
from langchain.document_loaders import PyPDFLoader, CSVLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
 
# Step 1: Load PDF and CSV Documents
def load_documents():
    # Load PDF Document
    pdf_loader = PyPDFLoader("data/sample_document.pdf")
    pdf_documents = pdf_loader.load()
 
    # Load CSV Document
    csv_loader = CSVLoader(file_path="data/sample_data.csv")
    csv_documents = csv_loader.load()
 
    # Combine both sets of documents
    all_documents = pdf_documents + csv_documents
    return all_documents
 
# Step 2: Split the documents into smaller chunks for better processing
def split_documents(documents):
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=100
    )
    split_docs = text_splitter.split_documents(documents)
    return split_docs
 
 
# Load and process documents
documents = load_documents()
split_docs = split_documents(documents)

Task 4: Configure Embedding Model with OCI Generative AI Cohere

OCI Generative AI service provides a fully managed platform for accessing state-of-the-art LLMs, such as Cohere and Meta’s LLaMA-3. It supports the following models:

The major difference between AI Quick Actions and OCI Generative AI is that AI Quick Actions has a more extensive model catalog and offers a lot more flexibility with model training, evaluation, fine tuning and lifecycle management. The major advantage with OCI Generative AI is that you do not have to own the infra that hosts your model. You can use the generic endpoint for any of the available models and are only charged based on the number of requests and number of tokens. In contrast, with AI Quick Actions, while you have more autonomy over your LLM or any AI/ML workflow orchestration and lifecycle, you pay only for the cost of the infra and have unlimited requests. 

Moreover, OCI Generative AI service has ready to use enterprise embedding models such as Cohere Embed V3 model which is built for enterprise data and known to perform better than other pre-trained sentence transformer models from hugging face, particularly on custom data. You only need to invoke this model to generate embedding for your data. Therefore, we believe combining both OCI Generative AI and AI Quick Actions can give you the best value for you business application. You can invoke the already deployed OCI Generative AI Cohere embedding model endpoint to embed your data, and use AI Quick Actions to fine-tune, test, evaluate, and deploy your model. 

LangChain integrates seamlessly with OCI Generative AI and AI Quick Action, enabling developers to invoke models directly from their OCI environment. With minimal configuration, developers can start using pre-trained or fine-tuned models.

The following is an example on how you can use LangChain integration to invoke the Cohere embed V3 model, and use it to embed your documents during data ingestion.

from langchain_community.embeddings import OCIGenAIEmbeddings

oci_cohere_embedding_model = OCIGenAIEmbeddings(
    model_id="cohere.embed-english-v3.0",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="<>YOUR-COMPARTMENT-OCID",
    auth_type="RESOURCE_PRINCIPAL",
    model_kwargs={"input_type": "SEARCH_DOCUMENT"}
)

Task 5: Configure OCI with OpenSearch with LangChain to Enable RAG Workflows

RAG relies on fast and accurate retrieval systems. OCI with OpenSearch service provides the infrastructure to:

LangChain’s vector store functionality integrates with OCI with OpenSearch to support RAG workflows. Developers can use embeddings from OCI Generative AI to index and retrieve data efficiently from OCI with OpenSearch.

Use the following Python code.

from langchain.vectorstores import OpenSearchVectorSearch
import oci
 
# setup your script to use Resource principal for authentication
auth_provider = oci.auth.signers.get_resource_principals_signer()
auth = ("username", "password")
AUTH_TYPE="RESOURCE_PRINCIPAL"
opensearch_url="https://amaaaaaallb3......................opensearch.us-ashburn-1.oci.oraclecloud.com:9200"   // replace this whole value your opensearch url. Be sure sure to have the :9200 port in your url
 
# Initialize OpenSearch as the vector database
vector_db = OpenSearchVectorSearch(opensearch_url=opensearch_url,
                            index_name="<YOUR-INDEX-NAME>",
                            embedding_function=oci_cohere_embedding_model,
                            signer=auth_provider,
                            auth_type="RESOURCE_PRINCIPAL",
                            http_auth=auth)

Task 6: Ingest your Data

Use the following Python code to ingest your data. 

from langchain.vectorstores import OpenSearchVectorSearch
import oci
from tqdm import tqdm
 
batch_size=100
documents = load_documents() # function defined above feel free to or define a new one to process documents
document_chunks = split_documents(documents) // function defined above feel free to edit
index_name= <YOUR-INDEX-NAME>
 
# Ingest documents in batches
for i in tqdm(range(0, len(document_chunks), batch_size), desc="Ingesting batches"):
    batch = document_chunks[i:i + batch_size]
    vector_db.add_texts(texts=batch,
                     bulk_size=batch_size,
                     embedding=embedding_model,
                     opensearch_url=opensearch_url,
                     index_name= index_name,
                     signer=auth_provider,
                     auth_type=AUTH_TYPE,
                     http_auth=("username", "password"))
 
#refresh index
vector_db.client.indices.refresh(index=index_name)

Task 7: Configure LLM for RAG with LangChain

AI Quick Actions in OCI Data Science eliminates the complexity of deploying and managing LLMs. It provides a no-code interface for:

AI Quick Actions is easy to use and accessible through the OCI Data Science Notebook environment. It provides a great amount of flexibility. For example, you can use datasets stored in OCI Object Storage or your local notebook to train, fine-tune or evaluate your model. It is also a very cost-effective solution as there are no additional charges beyond the underlying compute and storage costs. Most importantly, you can access a catalog of vetted models available in AI Quick Actions, ready to deploy with just a few clicks. Moreover you have the flexibility to also fine-tune and evaluate any of these models on your custom dataset without writing any code. Not to mention the option to bring your own model artifacts and simply deploy with AI Quick Actions without having to worry about infra complexities. Lastly, you can use the built-in prompt interface to test and evaluate the performance of your deployed model.

AI Quick Action is available within your OCI Data Science Notebook session. To use AI Quick Actions make sure you set up the required policies in your tenancy. For more information, see AI Quick Actions Policies.

  1. Click AI quick actions to launch AI Quick Actions from your notebook session.

    image

  2. Access LLM Model Catalog.

    image

  3. Click Deployments and Create deployment.

    image

  4. In the Deploy model page, enter the required details to create a model deployment.

    image

  5. Once the model is created, you can view and test the deployed model.

    image

  6. Create evaluation for deployed model.

    image

Run the following code to use the LangChain integration to invoke a deployed LLM model for question answering. In this case, we will be using the LLM model deployed with AI Quick Actions, by calling its predict endpoint as follows:

import ads
from langchain_community.llms import OCIModelDeploymentLLM

ads.set_auth("resource_principal")
endpoint = "https://modeldeployment.us-ashburn-1.oci.customer-oci.com/ocid1.datasciencemodeldeployment.oc1.iad.am....................../predict"

oads_llm = OCIModelDeploymentLLM(
    endpoint=endpoint,
    model="odsc-llm",
    model_kwargs={"temperature": 0, "max_tokens": 500, 'top_p': 1.0, 'top_k': 1}
)

#test the invoke method to make sure model is deployed and active
oads_llm.invoke("who was the first president of the united states?")

Task 8: Perform Conversation Search and RAG 

Use the following Python code to perform conversation search and RAG.

# Create a retriever from the OpenSearch vector database
retriever = vector_db.as_retriever()

# Load a QA chain that combines the documents and generates an answer
combine_documents_chain = load_qa_chain(oads_llm, chain_type="stuff")

# Create the RetrievalQA chain
qa_chain = RetrievalQA(combine_documents_chain=combine_documents_chain, retriever=retriever)

# Example query
query="What was Notre Dame's first college? be explicit"
response = qa_chain.run(query)
print("Answer:", response)

The entire end-to-end code for this RAG application can be found here: OCI OpenSearch Service sample notebook. The entire GitHub repo for this tutorial can be found here: opensearch-integration-with-langchain.

Next Steps

The integration of LangChain, OCI with OpenSearch, AI Quick Actions, OCI Generative AI, and notebooks offers a comprehensive solution for building LLM applications. Whether you are developing RAG systems or conversational search tools, these technologies work together to deliver powerful, scalable, and efficient solutions.

By adopting this integrated approach, developers can unlock the full potential of generative AI and deliver transformative applications that drive business success. Start exploring these tools today and take the first step toward building the future of AI-powered applications. 

Acknowledgments

More Learning Resources

Explore other labs on docs.oracle.com/learn or access more free learning content on the Oracle Learning YouTube channel. Additionally, visit education.oracle.com/learning-explorer to become an Oracle Learning Explorer.

For product documentation, visit Oracle Help Center.