52 Ingestion API

The Ingestion API facilitates document retrieval, chunking and creation of vector embeddings. It is one of the key differentiators between Coherence RAG and its competitors, as it uses full processing power of the Coherence RAG cluster to parallelize all parts of ingestion: documents are loaded and split in parallel, and vector embeddings are then created in parallel.

To kick off document ingestion, the user can submit a list of document URIs to process to the ingestion endpoint:

 POST /api/kb/<storeName>/docs
 [
  "https://docs.oracle.com/en/database/oracle/oracle-database/23/vecse/ai-vector-search-users-guide.pdf",
  "https://docs.oracle.com/en/middleware/standalone/coherence/26.1.0/manage/managing-oracle-coherence.pdf",
  "https://docs.oracle.com/en/middleware/standalone/coherence/26.1.0/secure/securing-oracle-coherence.pdf",
  "oci.os://example-namespace/example-documents/coherence/administering-http-session-management-oracle-coherenceweb.pdf"
 ]

The protocol component of the URI determines which document loader should be used to retrieve the document, as described in Document Loaders.

Security Prerequisites

A document loader can cross filesystem, network and parser trust boundaries. Restrict import locations and URI schemes, validate size and content expectations, and deploy custom loader classes and application metadata deliberately. Use the configured import policy for local files and remote documents before invoking ingestion.

For the complete configuration and policy, see Restricting RAG Document Imports in Securing Oracle Coherence.

Authenticated Import Setup

The request examples in this chapter show endpoint paths and payloads. Send them to the configured RAG base URL over the approved protected transport, with the identity established by the hosting authentication provider. Document imports require the configured admin role. Configure the permitted local paths and URI schemes before ingestion, and provision the example object-store namespace, bucket and document with the appropriate provider credentials.

Use only documents the caller is entitled to ingest and retrieve. A URL’s presence in the sample payload is not an import allowlist entry. The secured server can reject a source before the loader or parser runs; diagnose that rejection as policy rather than retrying with a different URL syntax.