Hierarchical Navigable Small World Index Syntax and Parameters

Syntax and examples for creating Hierarchical Navigable Small World vector indexes.

Syntax

CREATE VECTOR INDEX vector_index_name
ON table_name (vector_column)
[GLOBAL] ORGANIZATION INMEMORY [NEIGHBOR] GRAPH
[WITH] [DISTANCE [CUSTOM [<schema>.][<package>.]] metric name]
[WITH TARGET ACCURACY percentage_value]
[QUANTIZATION SCALAR COMPRESSION RATIO {2|4|8}]
[PARAMETERS (TYPE HNSW ,
             {NEIGHBORS max_closest_vectors_connected | M max_closest_vectors_connected},
             EFCONSTRUCTION max_candidates_to_consider,
             [OFFLOAD_CREDENTIAL_NAME credential_name_containing_gpu_access_token,
              OFFLOAD_URL https_endpoint_for_gpu_index_construction]
             RESCORE FACTOR rescore_factor,
             ALGORITHM UNIFORM_QUANTIZATION
            )]
[DUPLICATE ALL | DISTRIBUTE [AUTO | BY ROWID RANGE | BY PARTITION | BY SUBPARTITION  ]]
[PARALLEL degree_of_parallelism]

HNSW Parameters

NEIGHBORS and M are equivalent and represent the maximum number of neighbors a vector can have on any layer, except the last layer, where a vector can have up to 2M neighbors.

EFCONSTRUCTION represents the maximum number of closest vector candidates considered at each step of the search during insertion.

OFFLOAD_CREDENTIAL_NAME is the identifier of the credential that should be used when authenticating with the Private AI Services Container GPU offloading service. The referenced credential name should be provided as a JSON object with an access_token field containing the API key to access the container. If OFFLOAD_CREDENTIAL_NAME is provided, OFFLOAD_URL must also be provided (and vice versa).

OFFLOAD_URL is the Private AI Services Container’s service endpoint for index creation. It should be provided as an https:// URL with the hostname or IP address of the remote server and the /v1/index path. For example,

https://<my_privateai_service_hostname>/v1/index

where <my_privateai_service_hostname> is the URL that the Oracle AI Database can use to connect to the Private AI Services Container. For more information about the Private AI Services Container, see Oracle Private AI Services Container User’s Guide

RESCORE FACTOR is an optional integer parameter that defines the rescoring multiplier for semantic search queries. It has a default value of 1 and is applied only when the index is quantized (when QUANTIZATION is specified).

The valid range for HNSW vector index parameters are:

Parameter Valid Values
ACCURACY > 0 and <= 100
DISTANCE COSINE (default)

EUCLIDEAN

L2_SQUARED (as in EUCLIDEAN_SQUARED)

DOT

MANHATTAN

HAMMING

JACCARD

CUSTOM <custom metric name>
QUANTIZATION SCALAR
TYPE HNSW
NEIGHBORS >= 2 and <= 2048
EFCONSTRUCTION > 0 and <= 65535
RESCORE FACTOR >= 1 and <= 100 (default 1)
ALGORITHM UNIFORM_QUANTIZATION

Usage Notes

You can specify either DUPLICATE ALL or DISTRIBUTE, but not both. These clauses are optional. If they are omitted, the HNSW vector index is created using the default behavior, consistent with the default behavior of a standard CREATE INDEX DDL execution.

You can optionally use the QUANTIZATION keyword together with the COMPRESSION RATIO parameter in order to apply scalar quantization to a new vector index.

The COMPRESSION RATIO specifies the level of compression for a vector column when creating a scalar-quantized index. Higher compression ratios yield smaller storage requirements but increased quantization. The possible values for COMPRESSION RATIO are 2, 4, and 8.

The following table shows the resulting data type stored in the index for each compression ratio, depending on the original vector column format.

Compression Ratio FLOAT64 Input FLOAT32 Input FLOAT16 Input
2 Float32 Float16 Int8
4 Float16 Int8 Not Supported
6 Int8 Not Supported Not Supported

Note that “not supported” means that scalar quantization is not available for the given combination of compression ratio and vector column format.

Note: Any parallel DML executed against a table with an HNSW index is automatically converted to a serial DML. HNSW indexes are not currently supported with parallel DML operations.

Examples

CREATE VECTOR INDEX galaxies_hnsw_idx ON galaxies (embedding)
  ORGANIZATION INMEMORY NEIGHBOR GRAPH
  DISTANCE COSINE
  WITH TARGET ACCURACY 95;

CREATE VECTOR INDEX galaxies_hnsw_idx ON galaxies (embedding)
  ORGANIZATION INMEMORY NEIGHBOR GRAPH
  DISTANCE COSINE
  WITH TARGET ACCURACY 90
  PARAMETERS (
    TYPE             HNSW,
    NEIGHBORS        40,
    EFCONSTRUCTION   500
  ) PARALLEL 8;

CREATE VECTOR INDEX galaxies_quantized_idx ON galaxies (embedding)
  ORGANIZATION INMEMORY NEIGHBOR GRAPH
  DISTANCE EUCLIDEAN
  WITH TARGET ACCURACY 90
  QUANTIZATION SCALAR COMPRESSION RATIO 2
  PARAMETERS (
    TYPE             HNSW,
    NEIGHBORS        32,
    EFCONSTRUCTION   200,
    RESCORE FACTOR   2,
    ALGORITHM        UNIFORM_QUANTIZATION
  );

CREATE VECTOR INDEX gist_idx ON gist(embedding)
  ORGANIZATION INMEMORY NEIGHBOR GRAPH
  DISTANCE EUCLIDEAN
  PARAMETERS (
    TYPE                       HNSW,
    NEIGHBORS                  32,
    EFCONSTRUCTION             200,
    OFFLOAD_CREDENTIAL_NAME    mycredname,
    OFFLOAD_URL                'https://<container_url>:<port>/v1/index'
  ) PARALLEL 4;

For detailed information, see CREATE VECTOR INDEX in Oracle AI Database SQL Language Reference.

Online Hierarchical Navigable Small World Index

In situations where a base table needs to be available for ongoing updates and cannot be locked during index creation, you can create your HNSW (Hierarchical Navigable Small World) index online with the ONLINE clause of CREATE VECTOR INDEX. This way an application with heavy DML need not stop updating the base table for indexing.

Syntax

CREATE VECTOR INDEX index_name ON table_name(vector_column)
  ORGANIZATION INMEMORY NEIGHBOR GRAPH
  [WITH TARGET ACCURACY 95]
  [DISTANCE EUCLIDEAN]
  PARAMETERS (type HNSW, neighbors 32, efConstruction 500)
  ONLINE;

Usage Notes

Example

CREATE VECTOR INDEX galaxies_hnsw_idx ON galaxies(embedding)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE EUCLIDEAN
WITH TARGET ACCURACY 95
PARAMETERS (type HNSW, neighbors 32, efConstruction 500)
ONLINE;

Function-Based Hierarchical Navigable Small World Index

Oracle AI Database has long supported function-based indexes, allowing indexing expressions and computed values. The function-based HNSW index is an extension of this capability which applies the function-based indexing paradigm to the vector data type.

This index is created on an expression instead of a vector column. The index expression can be an arithmetic, conditional expression or any other expression that produces a vector. Oracle AI Database computes the value of the expression and stores the resulting vectors in the index. The base table does not get an extra vector column; the index holds the vector values resulting from the expression.

The index is automatically maintained and remains consistent whenever a DML operation is performed.

Function-based HNSW indexes are especially useful to extract vector values from JSON documents for indexing. When creating the index, you provide an expression to extract vectors from the JSON document. At the time of indexing, the vector data is extracted from JSON documents by evaluating the specified expression.

When a query includes an expression that matches a function-based vector index expression, the optimizer automatically rewrites and routes queries to leverage the function-based index. This boosts query performance for similarity searches.

Usage Notes

Syntax

CREATE VECTOR INDEX index_name ON table_name (expression)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
ONLINE;

Examples