1. Integrations Guide
  2. Integration with Elastic Search for Full Text Search
  3. Appendix
  4. Deploying and Configuring a Secure Oracle NoSQL Store

Deploying and Configuring a Secure Oracle NoSQL Store

There are a number of different methods to deploy and configure an Oracle NoSQL store for secure access. This section presents one particular set of steps you can take to deploy and configure such a store. For other methods, see Security Configuration in the Security Guide.

Additionally, since the store that is deployed must communicate with the secure Elasticsearch cluster from Secure Elasticsearch using Sheild, this section also shows how to generate and install the private keys and public certificates needed by the store and cluster for secure communication.

Whether you prefer the method presented here or one of the other methods presented in the Security Guide, the following assumptions and requirements apply when configuring an Oracle NoSQL store for secure deployment and communication with a secure Elasticsearch cluster:

Assumptions about the Secure Oracle NoSQL Store

  • The Oracle NoSQL Database distribution is installed under the directory /opt/ondb/kv.
  • There are three nodes hosting the store, named kvhost1, kvhost2, and kvhost3 respectively.
  • The store is deployed with a replication factor (rf) of 3, and is named mystore.
  • An admin service, listening on port 5000, is deployed on each of the store's nodes.
  • The range of ports used to support high availability (harange) consists of port 5002 through 5007.
  • One storage node (SN) per store host will be deployed (capacity 1), with default values for the number of cpu's and memory (num_cpus 0 and memory_mb 0).
  • The contents of the shards (replication groups) managed by the store are located under the storage directory /disk1/shard on each node of the store; where the size specified for each storage directory is 1GB (1,000,000,000 bytes).
  • For convenience, the password manager the store uses to store and retrieve passwords for access to the store's keystore and truststore is a password file (available in all editions of Oracle NoSQL Database), rather than the Oracle Wallet (available in only the Enterprise Edition).
  • For simplicity, all passwords are set to No_Sql_00.
  • The name of the alias used in the public/private keypair generated by the store and provided to Elasticsearch for secure communication with the store, is FTS. Note that this is a requirement, as communication with a secure store will fail if Elasticsearch responds to a request from the store by presenting a certificate with an alias different than FTS.
  • A user with administrative privileges is provisioned in the store's access control list. The name given to this user is FTS; the same as the alias of the keypair the store generates for Elasticsearch. Although the user name is not required to be the same as the alias, it is given that value for consistency, and to avoid confusion.

Provision the Store Boot Node for Secure Deployment and Elasticsearch Communication

All of the commands presented in this sub-section are executed on only the first (boot) node of the store (example: kvhost1). Using the assumptions previously listed, when provisioning the boot node of a store that will be deployed with security, the first command to execute is:

On kvhost1

export JAVA_HOME=/opt/java/java8  [if necessary]

java -jar /opt/ondb/kv/lib/kvstore.jar makebootconfig 
  -root /opt/ondb/kvroot 
  -config config.xml 
  -port 5000 
  -host kvhost1 
  -harange 5002,5007 
  -capacity 1 
  -num_cpus 0 
  -memory_mb 0 
  -storagedir /disk1/shard 
  -storagedirsize 1000000000 
  -store-security configure 
  -pwdmgr pwdfile 
  -kspwd No_Sql_00

The command above creates the security directory /opt/ondb/kvroot/security on the store's boot node kvhost1, and populates it with security artifacts such as the store's keystore (store.keys) and trustore (store.trust). For convenience, it also creates artifacts that can be distributed to clients for secure access to the store (client.trust and client.security). After executing the command above, you should see the following files in the security directory: 

ls /opt/ondb/kvroot/security 
  store.trust 
  client.trust 
  client.security 
  security.xml 
  store.keys 
  store.passwd

Athough the command above is necessary to deploy a secure store, it is not sufficient for secure communication with the Elasticsearch cluster from Secure Elasticsearch using Sheild. To facilitate secure communication with Elasticsearch, a public/private keypair with the alias FTS must be generated and installed in the store's keystore. For example,

On kvhost1

keytool -genkeypair 
  -alias FTS 
  -keystore /opt/ondb/kvroot/security/store.keys 
  -keyalg RSA 
  -keysize 2048 
  -validity 1712 
  -storepass No_Sql_00 
  -keypass No_Sql_00 
  -dname CN=FTS,OU=nosql.org,L=nosql.city,S=nosql.state,C=US 
  -ext san=dns:localhost,dns:eshost1,dns:eshost2,dns:eshost3,
          dns:kvhost1,dns:kvhost2,dns:kvhost3

After generating the keypair above, the public certificate from that keypair must be exported from the keystore. In order for any node of the Elasticsearch cluster to securely communicate with the NoSQL store, the Elasticsearch node must send this certificate to the store. Thus, the certificate produced by the following export command will ultimately be installed on each node of the Elasticsearch cluster. See Install the Full Text Search Public Certificate in Elasticsearch. On kvhost1, 

keytool -export 
  -alias FTS 
  -keystore /opt/ondb/kvroot/security/store.keys 
  -storepass No_Sql_00 
  -file /opt/ondb/kvroot/security/FTS.crt

Whereas the FTS public certificate created by this command must be presented to the Oracle NoSQL store by each Elasticsearch node when the node attempts to communicate with the store, the store must also present the Elasticsearch node's public certificate. This is because the model for secure communication between Elasticsearch and Oracle NoSQL requires mutual authentication. As a result, the public certificates created on each of the Elasticsearch nodes in Secure Elasticsearch using Sheild must be retrieved and installed in the store's truststore. For example, 

scp <username>@eshost1:/opt/es/install-xfer/certs/elasticsearch-eshost1.crt /opt/ondb/kvroot/security
scp <username>@eshost2:/opt/es/install-xfer/certs/elasticsearch-eshost2.crt /opt/ondb/kvroot/security
scp <username>@eshost3:/opt/es/install-xfer/certs/elasticsearch-eshost3.crt /opt/ondb/kvroot/security

keytool -importcert 
  -alias elasticsearch-eshost1 
  -file /opt/ondb/kvroot/security/elasticsearch-eshost1.crt 
  -keystore /opt/ondb/kvroot/security/store.trust 
  -storepass No_Sql_00 
  -keypass No_Sql_00 
  -noprompt

keytool -importcert 
  -alias elasticsearch-eshost2 
  -file /opt/ondb/kvroot/security/elasticsearch-eshost2.crt 
  -keystore /opt/ondb/kvroot/security/store.trust 
  -storepass No_Sql_00 
  -keypass No_Sql_00 
  -noprompt

keytool -importcert 
  -alias elasticsearch-eshost3 
  -file /opt/ondb/kvroot/security/elasticsearch-eshost3.crt 
  -keystore /opt/ondb/kvroot/security/store.trust 
  -storepass No_Sql_00 
  -keypass No_Sql_00 
  -noprompt

At this point, the store's boot node is configured for secure deployment, and its security directory has been provisioned with the necessary security artifacts for communication with the Elasticsearch cluster from Secure Elasticsearch using Sheild.

The final step in the provisioning process is to install the same security artifacts created on the boot node in each of the remaining nodes of the store. This is accomplished by simply copying the boot node's security directory to each of those other nodes. For example, if the boot node is kvhost1, then you would do something like the following from that node: 

scp -r /opt/ondb/kvroot/security <username>@kvhost2:/opt/ondb/kvroot
scp -r /opt/ondb/kvroot/security <username>@kvhost3:/opt/ondb/kvroot

Configure the Store's Remaining non-Boot Nodes for Security

Once the store's boot node is configured for security and the security directory of all of the nodes in the store have been fully provisioned as described in the previous sub-section, the remaining (non-boot) nodes of the store must also be configured for security. This is accomplished by using Java 8 or greater to execute, respectively, the following commands on each of the remaining nodes.

On kvhost2

java -jar /opt/ondb/kv/lib/kvstore.jar makebootconfig 
  -root /opt/ondb/kvroot 
  -config config.xml 
  -port 5000 
  -host kvhost2 
  -harange 5002,5007 
  -capacity 1 
  -num_cpus 0 
  -memory_mb 0 
  -storagedir /disk1/shard 
  -storagedirsize 1000000000 
  -store-security enable 
  -pwdmgr pwdfile

On kvhost3

java -jar /opt/ondb/kv/lib/kvstore.jar makebootconfig 
  -root /opt/ondb/kvroot 
  -config config.xml 
  -port 5000 
  -host kvhost3 
  -harange 5002,5007 
  -capacity 1 
  -num_cpus 0 
  -memory_mb 0 
  -storagedir /disk1/shard 
  -storagedirsize 1000000000 
  -store-security enable 
  -pwdmgr pwdfile

At this point, the store is configured and fully provisioned for secure deployment. The following sub-sections describe how this is accomplished.

Start Each Node of the NoSQL Store

Using Java 8 or greater, execute the following command on each node of the store.

On kvhost1, kvhost2, and kvhost3

java -jar /opt/ondb/kv/lib/kvstore.jar start 
  -root /opt/ondb/kvroot 
  -config config.xml

Deploy the Secure NoSQL Store

To deploy an Oracle NoSQL store based on the assumptions listed previously, first create a text file containing the following Oracle NoSQL administrative commands that can be executed as a script from the Oracle NoSQL Admin CLI.

configure -name mystore
plan deploy-zone -name zn1 -rf 3 -wait

plan deploy-sn -znname zn1 -host kvhost1 -port 5000 -wait
plan deploy-admin -sn 1 -wait
pool create -name snpool
pool join -name snpool -sn sn1

plan deploy-sn -znname zn1 -host kvhost2 -port 5000 -wait
plan deploy-admin -sn 2 -wait
pool join -name snpool -sn sn2

plan deploy-sn -znname zn1 -host kvhost3 -port 5000 -wait
plan deploy-admin -sn 3 -wait
pool join -name snpool -sn sn3

change-policy -params "loggingConfigProps=oracle.kv.level=INFO;"

topology create -name snlayout -pool snpool -partitions 300
plan deploy-topology -name snlayout -plan sndeploy -wait

execute "CREATE USER root IDENTIFIED BY 'No_Sql_00' ADMIN";

Note that a user named root with ADMIN privileges will be created when the store is deployed. That user will be used to add other users to the store's access control list (ACL); for example, the user named FTS described previously.

Once you have created the command file above, start the Admin CLI and deploy the store by loading that file. For example, suppose the commands are stored in the file, /tmp/deploy-secure-store.cmds. You would then deploy the store by doing the following:

On kvhost1

java -jar /opt/ondb/kv/lib/kvstore.jar runadmin 
  -host kvhost1 
  -port 5000 
  -security /opt/ondb/kvroot/security/client.security

  Logged in admin as anonymous
  Connected to Admin in read-only mode

kv-> load -file /tmp/deploy-secure-store.cmds

  Connected to Admin in read-only mode
  Store configured: mystore
  ....
  Created: snlayout
  Executed plan 13, waiting for completion...
  Plan 13 ended successfully
  Statement completed successfully

kv-> exit

Note:

The clocks on kvhost1, kvhost2, and kvhost3 must by synchronized (by default, within a 2 second delta), otherwise store deployment will fail. To determine whether a failed deployment was caused by unsynchronized clocks, check the admin logs on the affected node /opt/ondb/kvroot/mystore/log/adminN_0.log.

Provision the root User

To provison the user named root that was created during store deployment, do the following:

On kvhost1

java -jar /opt/ondb/kv/lib/kvstore.jar securityconfig pwdfile create 
  -file /opt/ondb/kvroot/security/root.passwd

java -jar /opt/ondb/kv/lib/kvstore.jar securityconfig pwdfile secret 
  -file /opt/ondb/kvroot/security/root.passwd -set -alias root

  Enter the secret value to store: No_Sql_00
  Re-enter the secret value for verification: No_Sql_00

Create a properties file that you can use to access (login to) the Admin CLI as the root user just created. This file should contain the same entries as the default client.security file generated when the store was initially provisioned for security, along with entries that specify the username and password file specific to the root user. For example,

cp /opt/ondb/kvroot/security/client.security /opt/ondb/kvroot/security/root.login

echo oracle.kv.auth.username=root >> /opt/ondb/kvroot/security/root.login
echo oracle.kv.auth.pwdfile.file= 
/opt/ondb/kvroot/security/root.passwd >> /opt/ondb/kvroot/security/root.login

Create and Provision the FTS User For Indexing Data in Secure Elasticsearch

In a production system, you would not typically use the root user to create and populate tables and Secondary Indexes in the Oracle NoSQL store or Text Indexes in the Elasticsearch cluster. Instead, you would generally use the root user to create other client users of the store whose roles are specific to a particular task; for example, indexing data in Elasticsearch.

For this example, a user named FTS is created and granted the privileges needed to create and populate a table, as well as index the table's data in Elasticsearch. To do this, you need to first create an Admin CLI command file that contains entries such as: 

execute 'CREATE ROLE ftsadmin'
execute 'GRANT SYSDBA TO ftsadmin'
execute 'GRANT READ_ANY TO ftsadmin'
execute 'GRANT WRITE_ANY TO ftsadmin'
execute 'CREATE USER FTS IDENTIFIED BY "No_Sql_00"'
execute 'GRANT ftsadmin TO USER FTS'
execute 'GRANT SYSADMIN TO USER FTS'

Then, assuming /tmp/create-user-FTS.cmds is the path to that command file, you create the user by logging into the Admin CLI as the root user and then loading the command file. For example,

On kvhost1

java -jar /opt/ondb/kv/lib/kvcli.jar runadmin 
  -host kvhost1 
  -port 5000 
  -store mystore 
  -security /opt/ondb/kvroot/security/root.login

  Logged in admin as root

kv-> load -file /tmp/create-user-FTS.cmds

  Statement completed successfully
  Statement completed successfully
  ....

kv-> exit

To complete the provisioning of the FTS user just created, you should create a password file for that user and install it in a directory (for example, /tmp) on the client node you will be using to load and index data. For completeness (and convenience), in a fashion similar to what was done for the root user, you should also create a properties file that can be used to login to the Admin CLI as the user FTS. For example,

On kvhost1

java -jar /opt/ondb/kv/lib/kvstore.jar 
  securityconfig pwdfile create 
  -file /tmp/FTS.passwd

java -jar /opt/ondb/kv/lib/kvstore.jar 
  securityconfig pwdfile secret 
  -file /tmp/FTS.passwd 
  -set 
  -alias FTS

  Enter the secret value to store: No_Sql_00
  Re-enter the secret value for verification: No_Sql_00

cp /opt/ondb/kvroot/security/client.security /tmp/FTS-client.login

echo oracle.kv.auth.username=FTS >> /opt/ondb/kvroot/security/FTS-client.login
echo oracle.kv.auth.pwdfile.file=/tmp/FTS.passwd >> /tmp/FTS-client.login

cp /opt/ondb/kvroot/security/client.trust /tmp

Once the three artifacts above (FTS-client.login, FTS.passwd, and client.trust) have been created and installed in the /tmp directory on kvhost1, you can install them on any client. For example, 

scp /tmp/FTS-client.login <username>@clhost1:/tmp
scp /tmp/FTS.passwd <username>@clhost1:/tmp
scp /tmp/client.trust <username>@clhost1:/tmp

At this point the store is fully deployed and ready to interact with the Elasticsearch cluster.

The only thing left to do before running the example is to install the Oracle NoSQL store's public certificate (alias=FTS) in the truststore on each Elasticsearch node. See Install the Full Text Search Public Certificate in Elasticsearch.