Set Up LangChain for Generative AI

Set up LangChain packages including packages for integration with OCI Generative AI. Then, test the installation by chatting with a model hosted on OCI Generative AI.

Overview

Key tasks:

Install the LangChain and LangChain OCI packages.
Run the Python code and receive a response from the model.

For more information, see the following resources:

Before You Begin

To successfully perform this tutorial, you must have the following configuration in place.

A MacOS, Linux, or a Windows environment with Windows Subsystem for Linux (WSL).
Python 3 installed. For the supported Python versions, see the Python SDK documentation.
Have performed the following tutorials:
- Create a Sandbox User for Tutorials
- Set Up API Authentication for OCI

1. Install LangChain

Install the langchain package using pip:
```
pip install -U langchain
```
This command installs or upgrades the langchain package to the latest version available. If you already have an older version installed, pip replaces the package with the newest one.
Verify that the langchain package is installed. Enter the following command.
```
pip show langchain
```
The command displays details for the langchain package including required packages for working with this package. For example,
```
Requires: langchain-core, langchain-text-splitters, 
langsmith, pydantic, PyYAML, requests, SQLAlchemy
```
Verify that the required packages are installed. Enter the following command:
```
pip list
```
Install any missing packages.
Verify that the installation works.
```
python3 -c "import langchain; 
print(langchain.__version__)"
```
This command prints the langchain package version.

2. Install the LangChain OCI Package

Install the langchain-oci package using pip:
```
pip install -U langchain-oci
```
Important

The OCI integration features available in the langchain-community package are now deprecated. Ensure that you install the langchain-oci package using the preceding command.
Verify that the langchain-oci package is installed. Enter the following command.
```
pip show langchain-oci
```
Find the required packages for working with the langchain-oci package. For example,
```
Requires: aiohttp, langchain, langchain-core, oci, pydantic
```
Verify that the required packages are installed. Enter the following command:
```
pip list
```
Install any missing packages.

3. Gather Required Information

Collect all the information that you need to complete this tutorial. Copy the required information into a secure text file.

This tutorial sends one chat message to the Meta Llama 4 Scout model hosted on OCI Generative AI and receives a response. The cost for one on-demand chat message is close to zero dollars, but it's not free. With this section, we want you to learn how to calculate cost and decide which model to use when you have thousands of transactions.

Navigate to the Pricing Page and select a model based on its price for 10,000 transactions (10,000 characters). Instead of 10,000 transactions, some prices are listed for 1,000,000 tokens. Estimate 3 to 4 characters per token.

This tutorial uses about 200 characters or 53 tokens for input and about the same amount for its output which adds up to about 400 characters or 106 tokens, so let's round it up to about 500 characters.
If your organization approves the cost of Oracle Cloud Infrastructure Generative AI - Meta Llama 4 Scout for 500 characters, then use the information provided in this tutorial. Otherwise, in the tutorial steps replace the Meta Llama 4 Scout model with an approved model.
For larger transactions, you can also go to the cost estimator tool by selecting the AI and Machine Learning category and loading the cost estimator for OCI Generative AI. Here are example values for this tutorial:
- Service metric: on-demand
- Model provider: Meta
- Model: Llama 4 Scout
- Expected number of requests per month: 5 (Suppose you run the Python file 5 times)
- Prompt length (in characters): 200 (for this tutorial)
- Response length (in characters): 200 (for this tutorial)

3.1 Get Region and Model ID

Navigate to Pretrained Foundational Models in Generative AI and select Meta Llama 4 Scout (New).

From the Available in These Regions section, copy a region not listed as (dedicated AI cluster only): US Midwest (Chicago)
In the On-Demand Mode section, copy the OCI Model Name from the table: meta.llama-4-scout-17b-16e-instruct

3.2 Get Compartment Information

To get the compartment OCID of a compartment:

Open the navigation menu and select Identity & Security. Under Identity, select Compartments.
Select the compartment that you created for the Create a Sandbox User for Tutorials tutorial.
Select the Copy link for the OCID field.

Save the compartment OCID.

Example: ocid1.compartment.oc1..xxx

3.3 Get Path to Config File

From the Set Up API Authentication for OCI tutorial, copy the following information:

Path to the config file such as <your-home-directory>/.oci/config
Authentication profile name to use in the config file. For example, Default.

3.4 Get Inference API Endpoint

Go to Generative AI Inference API.
In the listed API endpoints, copy the endpoint for the Chicago region:
```
https://inference.generativeai.us-chicago-1.oci.oraclecloud.com
```

3.5 Collected Information

Ensure you have the following information written down for the tutorial.

Compartment ID: <sandbox-compartment>
Example: ocid1.compartment.oc1.aaaaaaa...
Model ID: meta.llama-4-scout-17b-16e-instruct
API Endpoint: https://inference.generativeai.us-chicago-1.oci.oraclecloud.com
Config File Path: <path-to-config-file
Example: <your-home-directory>/.oci/config
Authentication Profile Name in config File: <auth-profile-name-in-config-file>
Example: Default

4. Chat

Chat using a model hosted in OCI Generative AI. Reach this model with the langchain-oci package.

Important

Ensure to perform the steps within your <sandbox-compartment>. You might not have permission to view or create resources in the tenancy or in other compartments.

Create a file called langchain-1-translate.py.

Add the following code to langchain-1-translate.py.

from langchain.prompts import ChatPromptTemplate
from langchain_oci.chat_models import ChatOCIGenAI
from langchain.schema.output_parser import StrOutputParser

prompt = ChatPromptTemplate.from_template("""Translate the following text into French. 
If the text is a question, first translate the question 
and then answer the question in French.: {text}""")

llm = ChatOCIGenAI(
    model_id="meta.llama-4-scout-17b-16e-instruct",
    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
    compartment_id="<sandbox-compartment>",
    model_kwargs={"temperature": 0, "max_tokens": 500},
    auth_profile="<auth-profile-name-in-config-file>",  
    auth_file_location="<path-to-config-file>",
    )

output_parser = StrOutputParser()

chain = prompt | llm | output_parser
input_data = {"text": "What are the four seasons in a year?"}

result = chain.invoke(input_data)
print(result)

Run the langchain-1-translate.py Python file with the following command.

python3 langchain-1-translate.py

Example Answer:

The translation of the question is:

Quelles sont les quatre saisons dans une année ?

And the answer is:

Les quatre saisons dans une année sont : le printemps, l'été, l'automne et l'hiver.

Oracle Cloud Infrastructure Documentation