Oracle Cloud Infrastructure Documentation

Connecting to an OpenSearch Cluster

This topic covers the prerequisites and connection details required to connect to your OpenSearch cluster for OCI Search with OpenSearch. It also includes a walkthrough that covers the steps for some basic connection scenarios using a VM instance in the same VCN and subnet as the cluster.

Use the information provided here to enable your OpenSearch clients and custom search solutions to connect and ingest data from your cluster.

Note

While Search with OpenSearch is applying patches to a cluster's instances or resizing the cluster, the cluster's search functionality is in read-only mode, write mode is disabled for a maintenance window. See Cluster Patching and Resize Operations for more information.

Private Endpoints

Search with OpenSearch provides access to your cluster and the OpenSearch Dashboards for your cluster over private endpoints. The service provides two ways to connect to the private endpoints for your cluster:

  • Private IP: The private IP address for the private endpoint. Use the private IP if you're connecting from a VM instance within the same VCN and subnet as your cluster. You can also use the private IP when you connect from your local machine with port forwarding configured to the VM instance.
  • API endpoint: The a Fully Qualified Domain Name (FQDN) provided by the service to access cluster's private endpoints. Use the API endpoint when connecting from your OpenSearch client, such as LogStash or Beats.

See Prerequisites for specific details.

With private endpoints, traffic doesn't go over the internet. The endpoints are only accessible from within your cluster's VCN. To make the API endpoint access to your OpenSearch clients, more configuration is required to make the endpoint accessible to external clients. For example, you could use API Gateway to enable access with a public IP address.

Prerequisites

Following are the list of prerequisites required to connect to your cluster

  1. Determine the cluster's connection information.

  2. Create the required security rules.

  3. (Optional) Create and connect to a VM instance.

Cluster Connection Information

The cluster's connection information includes the API endpoints and IP addresses that you use to connect to and work with your cluster and the cluster's OpenSearch Dashboards. For more information, see Private Endpoints.

You can find all the connection information on the Cluster Details page in the Console.

Note

Connections to clusters created before September 15th, 2022 require that you download a certificate. The API endpoint's domain for these clusters is oci.oracleiaas.com.Connections to clusters created September 15th, 2022 or later don't require the extra step of downloading a certificate. The API endpoint's domain for these clusters is oci.oraclecloud.com.

Viewing Cluster Details

  1. On the Clusters list page, find the cluster that you want to work with. If you need help finding the list page or the cluster, see Listing OpenSearch Clusters.
  2. In the Clusters list, select the name of the cluster you want to view the details for.
  3. On the Cluster Details page, make note of the following:
    • API endpoint: The FQDN for the cluster's private endpoint.
    • Private IP: The private IP address for the cluster's private endpoint.
    • OpenSearch Dashboards API endpoint: The FQDN for the cluster's OpenSearch Dashboards..
    • OpenSearch Dashboards Private IP: The IP address for the private endpoint you can use to connect to the cluster's OpenSearch Dashboards.

Create Security Rules

You need to create security rules for the cluster's VCN to allow connections to the cluster and the cluster's OpenSearch Dashboards.

To make the data accessible from your cluster, you need to add stateful ingress security rules for TCP traffic for the following:

  • OpenSearch on destination port 9200 from source 0.0.0.0/0 and any source port.
  • OpenSearch Dashboards on destination port 5601 from source 0.0.0.0/0 and any source port.

You can create a security list with these rules, or you can add them to the default security list for the cluster's VCN.

Option 1: Create a Security List

  1. On the Clusters list page, find the cluster that you want to work with. If you need help finding the list page or the cluster, see Listing OpenSearch Clusters.
  2. Select the cluster that you're interested in.
  3. Under Cluster details, for Virtual cloud network, click the name of the cloud network. The Virtual Cloud Network Details page opens.
  4. Under Resources, click Security Lists.
  5. Select Create Security List.
  6. Enter the following:
    1. Name: A descriptive name for the security list. The name doesn't have to be unique, and it can't be changed later in the Console (but you can change it with the API). Avoid entering confidential information.
    2. Create in Compartment: The compartment where you want to create the security list, if different from the compartment you're working in.
  7. Click Another Ingress Rule.
  8. Enter the following values for the rule:
    • Stateless: Leave the check box cleared.
    • Source Type: CIDR
    • Source CIDR: 0.0.0.0/0
    • IP Protocol: TCP
    • Source Port Range: All
    • Destination Port Range: 9200
    • Description: An optional description of the rule.
  9. Select Another Ingress Rule.
  10. Enter the following values for the rule:
    • Stateless: Leave the check box cleared.
    • Source Type: CIDR
    • Source CIDR: 0.0.0.0/0
    • IP Protocol: TCP
    • Source Port Range: All
    • Destination Port Range: 5601
    • Description: An optional description of the rule.
  11. Select Create Security List.

Option 2: Add rules to the default security list

  1. On the Clusters list page, find the cluster that you want to work with. If you need help finding the list page or the cluster, see Listing OpenSearch Clusters.
  2. Under Cluster details, for Virtual cloud network, click the name of the cloud network. The Virtual Cloud Network Details page opens.
  3. Under Resources, click Security Lists.
  4. Select the security list named Default Security List for VCN_name.
  5. Select Add Ingress Rules.
  6. Enter the following values for the rule:
    • Stateless: Leave the check box cleared.
    • Source Type: CIDR
    • Source CIDR: 0.0.0.0/0
    • IP Protocol: TCP
    • Source Port Range: All
    • Destination Port Range: 9200
    • Description: An optional description of the rule.
  7. Select Another Ingress Rule.
  8. Enter the following values for the rule:
    • Stateless: Leave the check box cleared.
    • Source Type: CIDR
    • Source CIDR: 0.0.0.0/0
    • IP Protocol: TCP
    • Source Port Range: All
    • Destination Port Range: 5601
    • Description: An optional description of the rule.
  9. Select Add Ingress Rules.

Create a VM Instance (Optional)

After completing the connection prerequisites, optionally you can create a VM instance to run some basic connection tests to verify that the prerequisites have been configured correctly and that you have the correct connection information. You create the VM instance in the same subnet for the VCN you created the cluster in. See one of the following topics for how to create an instance:

After the instance is created, connect to the instance from your local machine. For more information, see the following:

Test the Connection to Your Cluster

After you're connected to the VM instance from your local machine, run one of the following commands to test the connection.

For clusters where the API endpoint domain is oci.oraclecloud.com (created on or after September 15, 2022):

curl https://<cluster_API_endpoint>:9200

The URL passed in this command is the value specified in the API endpoint field on the Cluster Details page. See Prerequisites.

If the connection to the endpoint is successful, a response like the following is returned:

{
  "name" : "opensearch-master-0",
  "cluster_name" : "<cluster_name>",
  "cluster_uuid" : "<cluster_UUID>",
  "version" : {
    "distribution" : "opensearch",
    "number" : "1.2.4-SNAPSHOT",
    "build_type" : "tar",
    "build_hash" : "<build_hash>",
    "build_date" : "2022-02-08T16:44:39.596468Z",
    "build_snapshot" : true,
    "lucene_version" : "8.10.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "The OpenSearch Project: https://opensearch.org/"
}

For other commands you can test, including pushing data to the cluster, and submitting search queries to the cluster, see the Search and visualize data using OCI Search Service with OpenSearch tutorial.

Test the Connection to the Cluster's OpenSearch Dashboards

One of the ways that you can connect to your cluster's OpenSearch Dashboards is by configuring port forwarding from your local machine to the VM instance you created in the same VCN and subnet as the cluster.

To configure port forwarding to connect to OpenSearch Dashboards, follow these steps:

  1. Create a VM instance.
  2. Run the following command from your local machine: 
    ssh -C -v -t -L 127.0.0.1:5601:<OpenSearch_Dashboards_private_IP>:5601 -L 127.0.0.1:9200:<OpenSearch_cluster_private_IP>:9200 opc@<VM_instance_public_IP> -i <private_key_file>

    Replace <OpenSearch_Dashboards_private_IP> with the private IP for your cluster's OpenSearch Dashboards. Replace <OpenSearch_cluster_private_IP> with your cluster's private IP. To locate the private IPs, see Prerequisites.

    Replace <VM_instance_public_IP> with the public IP address for the VM instance. Replace <private_key_file> with the path to the private key you use for connecting to the instance. For more information about these values and how to locate them, see Connecting to an Instance.

  3. From a browser on your local machine, open https://localhost:5601 to access the OpenSearch Dashboards.