Elasticsearch and Secure Oracle NoSQL Database

Up to this point, all information and examples presented in the previous sections discussed how data stored in an Oracle NoSQL Database table is indexed in Elasticsearch when the communication between Oracle NoSQL Database and Elasticsearch is not secure. This section discusses how that data can be sent to the Elasticsearch cluster over a secure communication channel.

As described previously, data sent to Elasticsearch for indexing is sent by a process running on the master replication node of the Oracle NoSQL store's replication group (or shard). When the system is not configured for security, the replication node communicates with Elasticsearch over HTTP. For the replication node to send the data to Elasticsearch over a secure communication channel, the NoSQL store must be configured to run securely. See Introducing Oracle NoSQL Database Security in the Security Guide. When configured for secure communication, the replication node will send the data to Elasticsearch, in encrypted form, over HTTPS. This means that Elastisticsearch must be configured to perform the necessary authentication and decryption before indexing the data received from a secure Oracle NoSQL store.

Elasticsearch version 2 does not provide a fully integrated, out-of-the-box option for communicating with clients over a secure channel in the manner just described. For secure communication with Elasticsearch, some users choose to run their Elasticsearch deployment "behind" (or "wrapped" within) a secure web server. Others choose to employ one of the commercially available plugins that support TLS (SSL) for this purpose. Oracle NoSQL Database has chosen to support the latter model.

In order to communicate securely with the Elasticsearch cluster, Oracle NoSQL Database recommends that the Shield proprietary plugin be used to provide a port to which clients of the Elasticsearch cluster can connect and communicate securely over HTTPS.

Note:

Although the Shield plugin has been used when testing secure communication between the current Oracle NoSQL Database implementation and Elasticsearch version 2, there is nothing in the NoSQL implementation that should prevent the use of other such Elasticsearch security plugins; as long as the plugin supports HTTPS, and can be configured to support the Oracle NoSQL Database authentication scheme.

Compared to the non-secure case presented previously, there are additional steps you must take when working with the secure case. For the secure case, the Oracle NoSQL store will be populated using the secure mode of the same example program, and the indexed data will be queried using similar queries, as that presented for the non-secure case. The only difference is that the Oracle NoSQL store and the Elasticsearch cluster will each be deployed to communicate securely, and the queries will specify the necessary keys and certificates required by the Elasticsearch cluster.

Deploying a secure Oracle NoSQL store and Elasticsearch cluster and configuring them to communicate securely with each other requires many more steps than the non-secure case. Appendices Secure Elasticsearch using Sheild, Deploying and Configuring a Secure Oracle NoSQL Store, and Install the Full Text Search Public Certificate in Elasticsearch provide detailed descriptions of all the steps necessary to deploy such a system. And once you have successfully deployed a secure Oracle NoSQL store and a secure Elasticsearch cluster, and you have installed the necessary artifacts (certificates) for the store and cluster to communicate, there are only minor differences between the commands and queries presented previously for the non-secure case and their counterparts in the secure case.

One of the first differences to note is that when executing the LoadJsonExample program to populate the NoSQL store with data to index in Elasticsearch, you must specify the security parameter with the absolute path to the file containing the login properties required by Oracle NoSQL Database Security (see Deploying and Configuring a Secure Oracle NoSQL Store for details). For example,

java –classpath /opt/ondb/kv/lib/kvstore.jar:src es.table.LoadJsonExample 
        –store kvstore 
        –host localhost 
        –port 5000 
        –file ~/examples/es/docs/senator-info.json 
        -table exampleJsonTable 
        -security /tmp/FTS-client.login

Next, when executing the register-es command to register the NoSQL store with the secure Elasticsearch cluster, you must specify true for that command's secure parameter. For example,

kv-> plan register-es 
        –clustername escluster 
        –host eshost1 
        –port 29100 
        –secure true 
        -wait

Finally, when querying the data indexed by the secure Elasticsearch cluster, the curl command must include the OpenSSL public certificate and private key required by the cluster for authentication of the request. See Secure Elasticsearch using Sheild. For example,

curl –k -E /tmp/elasticsearch-eshost1.pem 
        --key /tmp/elasticsearch-eshost1.pkey 
        –X GET 'http://eshost1:29100/ondb.kvstore.jsontable.jsontxtindex/_search?pretty' 
        '-d {query":{"bool":{
                "must":{"match":{"jsonField.party":"Democrat"}},
                "must":{"match":"jsonField.current":"true"}},
                "must":{"range":{"jsonField.contrib":{"gte":"1000000.00","lte":20000000.00"}}}, 
                "must":"match":{"jsonField.duties.committe":"Judiciary Apropriations"}}}}}' 

With respect to secure Full Text Search and the example commands presented in this document, it is assumed you have followed the directions presented in Secure Elasticsearch using Sheild, Deploying and Configuring a Secure Oracle NoSQL Store, and Install the Full Text Search Public Certificate in Elasticsearch appendices; which, for clarity and convenience, organize the steps to configure and deploy a secure Elasticsearch and Oracle NoSQL system into separate, self-contained sections.

Secure Elasticsearch using Sheild presents the steps required to configure Elasticsearch for security. These steps must be taken whether the Elasticsearch cluster will be communicating with a secure Oracle NoSQL store or some other service or client unrelated to Oracle NoSQL.

Deploying and Configuring a Secure Oracle NoSQL Store describes how to deploy a secure Oracle NoSQL store and then configure it to communicate securely with the Elasticsearch cluster described in Secure Elasticsearch using Sheild.

The final steps required to complete the deployment of the secure Oracle NoSQL and Elasticsearch system are presented in Install the Full Text Search Public Certificate in Elasticsearch. Those steps will complete the security configuration of the Elasticsearch cluster from Secure Elasticsearch using Sheild, and are required for the nodes of the cluster to communicate with the secure Oracle NoSQL store from Deploying and Configuring a Secure Oracle NoSQL Store. The steps presented in Install the Full Text Search Public Certificate in Elasticsearch should be executed only after executing the steps in Secure Elasticsearch using Sheild and Deploying and Configuring a Secure Oracle NoSQL Store.

After completing the steps presented in Secure Elasticsearch using Sheild, Deploying and Configuring a Secure Oracle NoSQL Store and Install the Full Text Search Public Certificate in Elasticsearch appendices, you should then be able to run the example program LoadJsonExample to populate a table in the secure Oracle NoSQL store deployed in Deploying and Configuring a Secure Oracle NoSQL Store, index data from that table in the secure Elasticsearch cluster from Secure Elasticsearch using Sheild and Install the Full Text Search Public Certificate in Elasticsearch, and finally run secure queries against the indexed data. For convenience, the secure versions of example commands you can execute are presented in Running the Examples in Secure Mode.

Note:

Unlike the non-secure example presented previously, instead of using KVLite to deploy an Oracle NoSQL store on a single node, Secure Elasticsearch using Sheild, Deploying and Configuring a Secure Oracle NoSQL Store, Install the Full Text Search Public Certificate in Elasticsearch, and Running the Examples in Secure Mode appendices show how to work with a secure Oracle NoSQL store and Elasticsearch cluster where both consist of three nodes rather than a single node. This is done to present a more realistic example, to demonstrate what one might typically encounter in production.