Records, assignments, and data types

A Data.Record is the fundamental data type for loading data into an Endeca data domain.

Records and assignments

Client programs using the Bulk Load API to load data into an Endeca data domain must convert the source data records from their native format into a Record object. Conceptually, a Record is a named set of key-value pairs. A key-value pair is a Data.Assignment. The key is a String, and the value can be any of the supported data types listed below.

The name of the Record is also an Assignment, and can therefore also be any of these types; for this reason it is known as the Spec. The Spec serves as the primary key of the data record, and each Record must have one.

The Spec is set on the Record with the setSpec() method of the Data.Record.Builder class, as shown in this snippet from the sample program: 
Data.Record.Builder recordBuilder = Data.Record.newBuilder();
Data.Assignment.Builder assignmentBuilder = Data.Assignment.newBuilder();

assignmentBuilder.clear();
assignmentBuilder.setName("Value").setDataType(Data.Assignment.DataType.INT).setInt32Value(val);
recordBuilder.setSpec(assignmentBuilder.build());

In this example, the name of the Spec is "Value" and it has an integer value ("val") that is input to the program.

Non-Spec key/value assignments are set with the addAssignments() method of the same Data.Record.Builder class: 
assignmentBuilder.clear();
assignmentBuilder.setName("Words").setDataType(Data.Assignment.DataType.STRING).setStringValue(words);
recordBuilder.addAssignments(assignmentBuilder.build());

The sample program creates one record with three non-Spec attributes (the String "Words" and the two Boolean "Square" and "Prime").

Builders

Data and Record objects are created via an associated Builder type:
  • The Data.Record.Builder has setSpec() methods for the Spec and setAssignments() methods for the Assignment objects.
  • The Data.Assignment.Builder has the setName() method for its key and various setters for its supported data types.
All of the setter functions in the Builder classes return the object they were invoked on, so they can be strung together. This makes it possible, if desired, to build the entire thing on one line: 
Data.Assignment.newBuilder().setName("SKU").setString("AB1234").setDataType(Data.Assignment.DataType.STRING).build();

Data types

The Data.Assignment.DataType enum contains entries for each of the data types supported by the Dgraph process. These correspond to the Data.Assignment.Builder class's setter methods as follows:
Enum DataType Data.Assignment.Builder setter Dgraph data type
BOOLEAN setBoolValue() mdex:boolean
DOUBLE setDoubleValue() mdex:double
DATETIME setStringValue() mdex:dateTime
DURATION setStringValue() mdex:duration
GEOCODE setStringValue() mdex:geocode
INT setInt32Value() mdex:int
INT64 setInt64Value() mdex:long
STRING setStringValue() mdex:string
TIME setStringValue() mdex:time

The setDataType() method tells the Dgraph what the Assignment data type is. For example, if you call setStringValue("foo") and setDataType(Data.Assignment.DataType.DOUBLE), and include the resulting Assignment in a Record that you send to the server, the server will attempt to extract the DOUBLE value and get back nothing. If you do not specify a data type, the Assignment is invalid.

Examples of setting the data types are as follows: 
// BOOLEAN example:
assignmentBuilder.setBoolValue(false);
assignmentBuilder.setDataType(Assignment.DataType.BOOLEAN);

// DOUBLE example:
assignmentBuilder.setDoubleValue(99.95);
assignmentBuilder.setDataType(Assignment.DataType.DOUBLE);

// DATETIME example:
assignmentBuilder.setStringValue("2010-11-18T17:00:00Z");
assignmentBuilder.setDataType(Assignment.DataType.DATETIME);

// DURATION example:
assignmentBuilder.setStringValue("P429DT1H2M3S");
assignmentBuilder.setDataType(Assignment.DataType.DURATION);

// GEOCODE example:
assignmentBuilder.setStringValue("42.365615 -71.075647");
assignmentBuilder.setDataType(Assignment.DataType.GEOCODE);

// INT example:
assignmentBuilder.setInt32Value(128);
assignmentBuilder.setDataType(Assignment.DataType.INT);

// INT64 example:
assignmentBuilder.setInt64Value(92233720);
assignmentBuilder.setDataType(Assignment.DataType.INT64);

// STRING example:
assignmentBuilder.setStringValue("Road Bikes");
assignmentBuilder.setDataType(Assignment.DataType.STRING);

// TIME example:
assignmentBuilder.setStringValue("09:14:52");
assignmentBuilder.setDataType(Assignment.DataType.TIME);