Exception handling and troubleshooting your scripts

Transform uses a static parser to override some of Groovy's dynamic typing behavior and detect parsing errors, such as undefined variables, when you preview or save your transformations.

Important: Because the static parser forces Groovy to behave like a statically-typed language, you cannot use Groovy's dynamic typing features in your transformations. For example, while undeclared variables are normally allowed in Groovy, they produce parsing errors in Transform.

The static parser also verifies that the attributes referenced directly in your script match those defined in your data set's schema. Any attributes that don't match (for example, ones that are misspelled) produce an error.

Important: The static parser does not verify that parameters included in your transformation match their syntax as referenced in the row map of some custom functions, such as enrichment functions. If you incorrectly reference a parameter from a function, your transformation script will not validate, but the parser will not specify an error. Therefore, check the Transform API Reference (either in this document or in the Groovydoc), to verify that you correctly reference function parameters in the row map.

If you include attributes as variables in your transformation scripts, the format you use for an attribute affects how it is handled by the static parser. For information about attribute formats, see Formats for variables.

If your transformation contains any parsing errors, Transform displays the resulting messages in the Transformation Error dialog box when you preview or save the transformation. Additionally, the Transformation Editor displays a red X icon next to each line that contains an error. You can hover over these icons to view more information about the error.

You should close the dialog box, fix the errors, then preview your transformation again to verify that all errors have been fixed. You cannot save your transformation to your script until it is free of errors.

Troubleshooting exceptions for set functions

You can run the following set functions from the Transform API only on multi-assign attributes (these attributes are known as multi-value attributes in Studio):
  • cardinality()
  • isSet()
  • isEmpty()
  • isMemberOf()
  • toSet()
  • toSingle()

These functions belong to the in-line transformations you can do in Transform. These set functions are applicable to sets of values on attributes that are multi-assign.

If you run any of these functions from Transform in Studio, and the attribute on which you attempt to run them is a single-assign (single-value) attribute, the Transform API may throw NULL or an exception, depending on the Dgraph type of the attribute.

Note: You can check if an attribute is multi-value by looking at a data set in Explore, and selecting a table view. A column that will have more than a single value in a cell indicates that this column represents a multi-value attribute. You can also check the value of the Multi-Value column for your data set in the Data Views.page (under Project Settings).

To summarize, if you receive an exception when attempting to run a transformation, check if the attribute you run the transformation on is a single-value. In this case, set transformation functions do not apply.

Security exceptions

If your transformation script contains any of the Groovy language features that are not supported, the parser throws a security exception, which displays in the Transformation Error dialog box. Remove the code that caused the error.

For more information on the Groovy language features that can cause security exceptions, see Groovy reserved keywords and unsupported functions.

Troubleshooting runtime exceptions

The static parser can't detect all errors, particularly runtime exceptions caused by anomalies in your data. Transform typically handles these errors by returning null values for data it can't process.

If you want to know more about why your transformation script is producing null values, you can wrap your code in a try block and set its output to a new temporary attribute of type String (it will show up in your project's data set table as a new column for an attribute of type String): 
try {
    <transformation script>  // replace this with your transformation script code
    'OK'
  } catch (Exception ex) {
    ex.getMessage()
  }

When you preview the transformation script, any error messages it produces will be output to the temporary column of type String. Once you have debugged it, you can delete the try block and remove the temporary attribute.

Troubleshooting dateTime conversions

If you implicitly convert a dateTime object (that is, an mdex:dateTime attribute) to a String object, you might get a different result depending on where the cluster Spark job ran. In particular, the resulting time zone may not be what you expected.

For example, assume the following statements: 
// mydatetime is an mdex:dateTime object
mydatetime = 1/1/1970 03:00:00 PM UTC
// convert to mdex:string-set object
new_set= toSet(mydatetime)

If you have a wide-spread cluster, you may not know where the job will be run. If the job is run on a machine configured for the New York, NY time zone, the result will be "Jan 1, 1970 10:00:00 AM" because EST is 5 hours ahead of UTC. However, if the job is run on a machine configured for the San Jose, CA time zone, the result will be "Jan 1, 1970 07:00:00 AM".

If your intention is to use mydatetime as a string, it is best to explicitly convert it. For example: 
new_set = toSet(toString(mydatetime,"MMM d, yyyy HH:mm:ss a","UTC"))
In this case, the time zone is specified by the user, instead of by the Spark node.