Full Text Search of Indexed JSON Documents

This section presents the steps to execute a simple but complete example, without security. Although in a production setting, both the Oracle NoSQL Database and the Elasticsearch cluster should generally be run on separate nodes, for simplicity, these steps are executed on a single node. Additionally, if you already have an Elasticsearch version 2 cluster running in your environment, then feel free to use that cluster in place of the Elasticsearch single-node cluster used below. Note finally, that you may have to change some of the tokens (directory locations, version numbers, etc.) to suit your particular environment.

  1. Download, install, and run Elasticsearch, version 2.

    Download the tar file https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.6/elasticsearch-2.4.6.tar.gz and place it under the directory /opt/es.

    cd /opt/es
    tar xzvf elasticsearch-2.4.6.tar.gz
    ln –s elasticsearch-2.4.6 elasticsearch
    export JAVA_HOME=/usr/lib/jvm/java8 
    /opt/es/elasticsearch/bin/elasticsearch 
            -Dnetwork.host=localhost
            --cluster.name kv-es-cluster 
            --node.name localhost

    Note:

    Elasticsearch version 2 requires Java 8. Thus, you should install Java 8 and set the JAVA_HOME environment to point to the Java 8's home directory.
  2. Use KVLite to deploy an Oracle NoSQL Database store named kvstore.

    Assuming that you have installed Oracle NoSQL Database under the directory /opt/ondb, and that you have write permission for your system's /tmp directory, execute the following command from a command line:

    java –jar /opt/ondb/kv/lib/kvstore.jar kvlite 
      –root /tmp/kvroot 
      –host localhost 
      –port 5000 
      –store kvstore 
      –secure-config disable
  3. Start the Oracle NoSQL Database Admin CLI.

    From a separate command window, execute the command:

    java –jar /opt/ondb/kv/lib/kvstore.jar runadmin 
      –host localhost 
      –port 5000 
      –store kvstore
  4. Install a file containing the JSON documents to load.

    Under a directory such as ~/examples/es/docs, create a file named senator-info.json and populate it with one or more JSON documents like those shown in the example file presented in Sample: Array of JSON Documents. Be sure to format the file you create with the same format shown in Sample: Array of JSON Documents.

  5. Compile and execute the LoadJsonExample program (or similar).

    Under a directory such as ~/examples/es/src, create the sub-directory es/table, and then create a file named LoadJsonExample.java under the directory ~/examples/es/src/es/table. After creating the file ~/examples/es/src/es/table/LoadJsonExample.java, add the source code presented in The LoadJsonExample Program Source (or source with similar functionality).

    Once the LoadJsonExample.java program is created, execute the following from a separate command window:

    cd ~/examples/es/src
    
    javac –classpath /opt/ondb/kv/lib/kvstore.jar:src 
      examples/es/table/LoadJsonExample.java 
    
    java –classpath /opt/ondb/kv/lib/kvstore.jar:src 
      es.table.LoadJsonExample 
      –store kvstore 
      –host localhost 
      –port 5000 
      –file ~/examples/es/docs/senator-info.json 
      -table exampleJsonTable

    Note:

    The source code for the LoadJsonExample program that is presented in The LoadJsonExample Program Source is only intended to provide a convenient mechanism for loading non-trivial JSON content into an Oracle NoSQL table. You should feel free to write your own program to provide similar functionality.
  6. Create a Text Index on the JSON data loaded into the NoSQL table.

    After verifying that the table has been successfully created and populated with the desired table data, execute the following from the Admin CLI:

    kv-> plan register-es 
      –clustername kv-es-cluster 
      –host localhost 
      –port 9200 
      –secure false 
      -wait
    
    kv-> execute 'CREATE FULLTEXT INDEX jsonTxtIndex ON exampleJsonTable ( 
      jsonField.current{"type":"boolean"}, 
      jsonField.party{"type":"string","analyzer":"standard"}, 
      jsonField.duties.committe{"type":"string"}, 
      jsonField.contrib{"type":"double"})'; 
  7. Execute Full Text Search queries against data indexed in Elasticsearch.

    To first verify that the desired index has been created in Elasticsearch as expected, execute the following from a command line:

    curl –X GET 'http://localhost:9200/_cat/indices'
    
    yellow open ondb.kvstore._checkpoint ...
    yellow open ondb.kvstore.examplejsontable.jsontxtindex ...

    Note that Elasticsearch reports the status of each index is yellow. This occurs here because the Elasticsearch cluster was deployed as a single-node cluster.

    To examine the mapping that Oracle NoSQL constructs for Elasticsearch, execute:

    curl –X GET 'http://localhost:9200/ondb.kvstore.examplejsontable.jsontxtindex/_mapping?pretty'

    To display all documents from the exampleJsonTable that were indexed in Elasticsearch, execute:

    curl –X GET 'http://localhost:9200/ondb.kvstore.examplejsontable.jsontxtindex/_search?pretty'

    Finally, to find all current democratic senators with contributions totaling between 5 million and 15 million dollars, who are members of either the "Progressive" caucus or the "Human Rights" caucus, execute the following command:

    curl –X GET 
      'http://localhost:9200/ondb.kvstore.examplejsontable.jsontxtindex/_search?pretty' 
      '-d {query":{"bool":{
        "must":{"match":{"jsonField.party":"Democrat"}},
        "must":{"match":"jsonField.current":"true"}},
        "must":{"range":{"jsonField.contrib":{"gte":"5000000.00","lte":15000000.00"}}},
        "must":"match":{"jsonField.duties.caucus":"Progressive Human Rights"}}}}}'