18 Overview of Transforming Project Data

This section provides conceptual information about transforming project data sets using both default Studio transformations and the custom transform functions (Groovy code).

Use this section together with the Transform API Reference (generated Groovydoc) that is packaged with Big Data Discovery and also available on the Oracle Technology Network.

• About transformations and transformation scripts
  Transformations are changes you can make to your project data set, after the source data has been processed and loaded into Studio. Transformations are effectively an ETL process to clean your data. Transformations can overwrite an existing attribute, modify attributes, or create new attributes. Any number of transformations can be combined into a transformation script. You run the script against the project data set. And if desired, you can create a new data set in the Catalog based on the transformed version.
• About the Transform page user interface
  Provides basic information about the header and menu features.
• Transformation script workflow
  At a high level, creating a transformation script and running it against your project data set involves the following steps:
• Transformations workflow diagram
  When you run a transformation script against a project data set and commit it, Studio modifies the project data set (this is the private copy of the data set), but does not modify the underlying data set as it appears in the Catalog to other Studio users (the public copy of the data set).
• About Groovy
  Groovy is a dynamically-typed scripting language. Code written in the Java language is valid in Groovy, so users not familiar with Groovy may resort to the Java syntax. All transform functions available for you in Studio are written in Groovy.
• About transform functions
  Transform functions are customized Groovy functions available in Studio that you can include in your transformation scripts. Each transform function performs a specific operation on your data, from simple ones, such as converting an attribute to a different data type, to more complex ones, such as determining the overall sentiment of a document or a String of text.
• Transform locking
  The Transform page in Studio has a locking mechanism to ensure that multiple users can't transform the same project data set at the same time.
• Preview mode
  You can preview a transformation at any time by clicking Preview, to see the effect it will have on your project data set. Preview mode is also a useful debugging tool, as it detects any runtime errors or corner case exceptions your transformation contains.