Class LocalOnnxEmbeddingModel

java.lang.Object
dev.langchain4j.model.embedding.DimensionAwareEmbeddingModel
com.oracle.coherence.rag.model.LocalOnnxEmbeddingModel
All Implemented Interfaces:
dev.langchain4j.model.embedding.EmbeddingModel, AutoCloseable
public class LocalOnnxEmbeddingModel extends dev.langchain4j.model.embedding.DimensionAwareEmbeddingModel implements AutoCloseable
Local ONNX embedding model implementation that integrates with LangChain4J.

This class provides a local embedding model that can run inference using ONNX Runtime, with support for both CPU and GPU acceleration via CUDA. It automatically downloads model files from HuggingFace when needed and supports efficient batch processing.

The model extends DimensionAwareEmbeddingModel to provide seamless integration with LangChain4J's ecosystem. It supports automatic model downloading, CUDA acceleration, and efficient resource management.

Example usage: 

 // Create default model from classpath
 LocalOnnxEmbeddingModel model = LocalOnnxEmbeddingModel.createDefault(ModelName.ALL_MINILM_L6_V2);
 
 // Create model with automatic download
 LocalOnnxEmbeddingModel model = LocalOnnxEmbeddingModel.create(ModelName.ALL_MPNET_BASE_V2);
 
 // Generate embeddings
 Response<Embedding> response = model.embed("Hello world");
 Embedding embedding = response.content();
 
 // Batch processing
 List<TextSegment> segments = Arrays.asList(
     TextSegment.from("First text"), 
     TextSegment.from("Second text")
 );
 Response<List<Embedding>> batchResponse = model.embedAll(segments);

Since:
25.09
Author:
Aleks Seovic 2025.07.04

  • Field Summary

    Fields inherited from class dev.langchain4j.model.embedding.DimensionAwareEmbeddingModel

    dimension

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    LocalOnnxEmbeddingModel(ModelName name, InputStream inModel, InputStream inTokenizer, PoolingConfig config)
    Constructs a local ONNX embedding model with the specified components.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
    Close the native ONNX model and release associated resources.
    static LocalOnnxEmbeddingModel
    create(ModelName name)
    Creates an embedding model with automatic download if needed.
    static LocalOnnxEmbeddingModel
    createDefault(ModelName name)
    Creates a default embedding model from the classpath.
    dev.langchain4j.model.output.Response<dev.langchain4j.data.embedding.Embedding>
    embed(dev.langchain4j.data.segment.TextSegment textSegment)
    Embeds a single text segment.
    dev.langchain4j.model.output.Response<dev.langchain4j.data.embedding.Embedding>
    embed(String text)
    Embeds a single text string.
    dev.langchain4j.model.output.Response<List<dev.langchain4j.data.embedding.Embedding>>
    embedAll(List<dev.langchain4j.data.segment.TextSegment> segments)
    Embeds multiple text segments efficiently as a batch.
    protected OnnxEmbeddingModel
    model()
    Returns the underlying ONNX embedding model.
    ModelName
    name()
    Returns the model name.
    String
    toString()
    Returns a string representation of this model.

    Methods inherited from class dev.langchain4j.model.embedding.DimensionAwareEmbeddingModel

    dimension, knownDimension

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • LocalOnnxEmbeddingModel

      protected LocalOnnxEmbeddingModel(ModelName name, InputStream inModel, InputStream inTokenizer, PoolingConfig config)
      Constructs a local ONNX embedding model with the specified components.
      Parameters:
      name - the model name
      inModel - the input stream for the model
      inTokenizer - the input stream for the tokenizer
      config - the pooling configuration

  • Method Details

    • createDefault

      public static LocalOnnxEmbeddingModel createDefault(ModelName name)
      Creates a default embedding model from the classpath.
      Parameters:
      name - the model name to create
      Returns:
      the configured embedding model

    • create

      public static LocalOnnxEmbeddingModel create(ModelName name)
      Creates an embedding model with automatic download if needed.
      Parameters:
      name - the model name to create
      Returns:
      the configured embedding model

    • name

      public ModelName name()
      Returns the model name.
      Returns:
      the model name

    • model

      protected OnnxEmbeddingModel model()
      Returns the underlying ONNX embedding model.
      Returns:
      the ONNX embedding model

    • embed

      public dev.langchain4j.model.output.Response<dev.langchain4j.data.embedding.Embedding> embed(String text)
      Embeds a single text string.
      Specified by:
      embed in interface dev.langchain4j.model.embedding.EmbeddingModel
      Parameters:
      text - the text to embed
      Returns:
      the response containing the embedding

    • embed

      public dev.langchain4j.model.output.Response<dev.langchain4j.data.embedding.Embedding> embed(dev.langchain4j.data.segment.TextSegment textSegment)
      Embeds a single text segment.
      Specified by:
      embed in interface dev.langchain4j.model.embedding.EmbeddingModel
      Parameters:
      textSegment - the text segment to embed
      Returns:
      the response containing the embedding

    • embedAll

      public dev.langchain4j.model.output.Response<List<dev.langchain4j.data.embedding.Embedding>> embedAll(List<dev.langchain4j.data.segment.TextSegment> segments)
      Embeds multiple text segments efficiently as a batch.
      Specified by:
      embedAll in interface dev.langchain4j.model.embedding.EmbeddingModel
      Parameters:
      segments - the list of text segments to embed
      Returns:
      the response containing the list of embeddings

    • toString

      public String toString()
      Returns a string representation of this model.
      Overrides:
      toString in class Object
      Returns:
      the string representation

    • close

      public void close() throws Exception
      Close the native ONNX model and release associated resources.
      Specified by:
      close in interface AutoCloseable
      Throws:
      Exception - if an error occurs