Records, assignments, and data types

Records and assignments

Client programs using the Bulk Load API to load data into an Endeca data domain must convert the source data records from their native format into a Record object. Conceptually, a Record is a named set of key-value pairs. A key-value pair is a Data.Assignment. The key is a String, and the value can be any of the supported data types listed below.

The name of the Record is also an Assignment, and can therefore also be any of these types; for this reason it is known as the Spec. The Spec serves as the primary key of the data record, and each Record must have one.

The Spec is set on the Record with the setSpec() method of the Data.Record.Builder class, as shown in this snippet from the sample program:

Record.Builder recBuilder = Record.newBuilder(); 

   Assignment.Builder assgtBuilder = Assignment.newBuilder();
   assgtBuilder.setName("Name");
   assgtBuilder.setStringValue(name);
   assgtBuilder.setDataType(Assignment.DataType.STRING);
   Assignment nameAssgt = assgtBuilder.build();
   recBuilder.setSpec(nameAssgt);

Non-Spec key/value assignments are set with the addAssignments() method of the same Data.Record.Builder class:

assgtBuilder.setName("Price");
assgtBuilder.setDoubleValue(price);
assgtBuilder.setDataType(Assignment.DataType.DOUBLE);
Assignment priceAssgt = assgtBuilder.build();
recBuilder.addAssignments(priceAssgt);

Builders

Data and Record objects are created via an associated Builder type:

The Data.Record.Builder has set() and clear() methods for the spec and for the Assignment objects.
The Data.Assignment.Builder has the same methods for its key and all of the supported data types.

All of the setter functions in the Builder classes return the object they were invoked on, so they can be strung together. This makes it possible, if desired, to build the entire thing on one line:

Data.Assignment.newBuilder().setName("SKU").setString("AB1234").setDataType(Data.Assignment.DataType.STRING).build();

Data types

The Data.Assignment.DataType enum contains entries for each of the data types supported by the Dgraph process. These correspond to the Assignment.Builder class's setter methods as follows:

Enum DataType	Assignment.Builder setter	Dgraph data type
BOOLEAN	setBoolValue()	mdex:boolean
DOUBLE	setDoubleValue()	mdex:double
DATETIME	setStringValue()	mdex:dateTime
DURATION	setStringValue()	mdex:duration
GEOCODE	setStringValue()	mdex:geocode
INT	setInt32Value()	mdex:int
INT64	setInt64Value()	mdex:long
STRING	setStringValue()	mdex:string
TIME	setStringValue()	mdex:time

The setDataType() method tells the Endeca server what the Assignment data type is. For example, if you call setStringValue("foo") and setDataType(Data.Assignment.DataType.DOUBLE), and include the resulting Assignment in a Record that you send to the server, the server will attempt to extract the DOUBLE value and get back nothing. If you do not specify a data type, the Assignment is invalid.

Examples of setting the data types are as follows:

// BOOLEAN example:
assgtBuilder.setBoolValue(false);
assgtBuilder.setDataType(Assignment.DataType.BOOLEAN);

// DOUBLE example:
assgtBuilder.setDoubleValue(99.95);
assgtBuilder.setDataType(Assignment.DataType.DOUBLE);

// DATETIME example:
assgtBuilder.setStringValue("2010-11-18T17:00:00Z");
assgtBuilder.setDataType(Assignment.DataType.DATETIME);

// DURATION example:
assgtBuilder.setStringValue("P429DT1H2M3S");
assgtBuilder.setDataType(Assignment.DataType.DURATION);

// GEOCODE example:
assgtBuilder.setStringValue("42.365615 -71.075647");
assgtBuilder.setDataType(Assignment.DataType.GEOCODE);

// INT example:
assgtBuilder.setInt32Value(128);
assgtBuilder.setDataType(Assignment.DataType.INT);

// INT64 example:
assgtBuilder.setInt64Value(92233720);
assgtBuilder.setDataType(Assignment.DataType.INT64);

// STRING example:
assgtBuilder.setStringValue("Road Bikes");
assgtBuilder.setDataType(Assignment.DataType.STRING);

// TIME example:
assgtBuilder.setStringValue("09:14:52");
assgtBuilder.setDataType(Assignment.DataType.TIME);