Creating mdexType Custom properties

Integrator allows you to create an mdexType Custom property that you can use to explicitly specify the MDEX type to which a particular Endeca standard attribute should map.

The Custom property feature can be used to specify MDEX types (such as mdex:duration, mdex:time, and mdex:geocode) that are not natively supported in Integrator. In this case, the ETL developer has to send a string through Integrator, making sure that the string value is formatted in the way that the Dgraph expects. The new mdexType Custom property, in other words, overrides the Integrator native property type when the records are sent to the Dgraph.

This functionality is particularly useful for non-String multi-assign properties, because Integrator natively has to treat the property as a string, because it has to include a delimiter. Thus, you can include delimiters in the multi-assign property (as though it were a String) but send the property to the Dgraph with mdex:int (for example) as the MDEX property type.

Important: Although the property will be designated as Integrator type String, you must make sure that the string value is formatted according to the rules of the MDEX property type to which it will be mapped. For example, if it will be created as an mdex:duration attribute in the Dgraph, then the String value must use the mdex:duration format.

You add Custom properties by invoking the Custom property editor from the Fields pane in the Metadata Editor:

Custom Property dialog

The Name field must be mdexType, and the Value field must be one of the MDEX property types (such as mdex:duration). The Name and Value are used by the Information Discovery component to specify (to the Dgraph) what MDEX property type should be used for when creating the standard attribute.

The source input file used as an example is a simple one: 
ProductKey|ProductName|Duration|Location
95000|HL Mountain Rim|P429DT2M3.25S|42.365615 -71.075647
It creates only one record with four standard attributes:
  • The ProductKey attribute is the primary key and is an Integer. Its value is 9500.
  • The ProductName attribute is a String type with a value of "HL Mountain Rim".
  • The Duration attribute will be a String property in the Designer metadata, but will use a Custom property of mdex:duration in order to create a Duration standard attribute. Its value is "P429DT2M3.25S" (which specifies a duration of 429 days, 2 minutes, and 3.25 seconds).
  • The Location attribute will be a String property in the Integrator metadata, but will use a Custom property of mdex:geocode in order to create a Geocode standard attribute. Its value is "42.365615 -71.075647" (which specifies a location at 42.365615 north latitude, 71.075647 west longitude).

To create a Custom property:

  1. Create a graph with at least one reader, an Information Discovery component (such as the Add/Update Records component), and an Edge component.
  2. Right-click on the Edge and select New metadata > Extract from flat file.
  3. In the Flat File dialog, select the input file and then click Next to display the Metadata editor.
  4. In the middle pane of the Metadata editor:
    1. Check the Extract names box.
    2. Click Reparse.
    3. Click Yes in the Warning message.

    At this point, the Record pane of the Metadata editor should look like this:

    Record pane with initial data

  5. In the Record pane of the Metadata editor, make these changes:
    1. Click the Record:recordName1 Name field and change the recordName1 default value to a more descriptive name.
    2. Change the ProductKey Type to integer.
    3. Leave the ProductName Type as string.
  6. To create a Custom property type for the Duration property:
    1. In the Record pane, click the Duration property to high-light it.

      The Duration property is displayed in the Field pane on the right, as in this example:

      Configuration for the Duration property

    2. In the Field pane, click the green + icon to bring up the Custom property editor.
    3. Enter mdexType in the Name field and mdex:duration in the Value field.

      The Custom property editor should look like this:

      Custom property configuration

    4. Click OK in the Custom property editor.

      As a result, a Custom section (with the new mdexType property) is added to the Duration property in the Field pane:

      Duration property with Custom section

  7. Repeat Step 6 if you want to create another mdexType Custom property type for another of your source properties.

    For example, for the Location attribute, you would create an mdexType Custom property with mdex:geocode in the Value field.

  8. Click OK to apply your changes and close the Metadata editor.

As mentioned above, when the graph is run to add records, the Dgraph will use the mdexType Custom properties to create the standard attributes.

Keep in mind that you can create mdexType Custom properties for any of the MDEX property types, by setting the Value field to:
  • mdex:boolean for Booleans
  • mdex:dateTime to represent the date and time to a resolution of milliseconds
  • mdex:double for floating-point values
  • mdex:duration to represent a length of time with a resolution of milliseconds
  • mdex:geocode to represent latitude and longitude pairs
  • mdex:int for 32-bit signed integers
  • mdex:long for 64-bit signed integers
  • mdex:string for XML-valid character strings
  • mdex:time for time-of-day values to a resolution of milliseconds