1. Release Notes for Oracle Data Transforms
  2. What’s New in Oracle Data Transforms

1 What’s New in Oracle Data Transforms

This chapter summarizes the new features and significant product changes for Oracle Data Transforms.

This chapter includes the following sections:

1.1 Oracle Data Transforms 2023.12.11.00

This release introduces the following new functionality:

  • Enhancements to Generic Rest Support - You can now choose from two options to create a generic REST connector. The Application tab on the Create Connection page includes the following two connection types:
    • Generic Rest - Use this to connect to any REST service endpoint to import data entities into Data Transforms. This option is available for Data Transforms that is available as a separate listing on Marketplace, Data Transforms built into Autonomous Database, and Data Transforms built into OCI GoldenGate.
    • Generic Rest Config - Use this to use a config file that contains information such as the authentication methods, endpoints, and tables that you want to import data entities from. This option is available only for Data Transforms that is available as a separate listing on Marketplace.

    See Create a REST Server Connection for detailed information.

  • Supported Connection Type - This release includes support for the SAP HANA application connection type.

    For the complete list of supported connection types, see Supported Connection Types.

1.2 Oracle Data Transforms 2023.08.21.00

This release introduces the following new functionality:

  • OCI GoldenGate Data Transforms - You can now deploy and access Data Transforms from the OCI GoldenGate Deployments page. Note that Data Transforms in OCI GoldenGate Data Transforms is currently in Limited Availability and only available in specific regions.
  • Enhancements to Generic Rest Support - To create a generic REST connector, you can now create and upload a config file that contains information such as the authentication methods, endpoints, and tables that you want to import data entities from. See Create a REST Server Connection for detailed information.
  • Support for Geocode Cloud Service - The Data Flow Editor database function palette now includes a function that utilizes the Geocode Cloud Service. You can use this function as part of your flows. You can find this component as part of the Geocode tools within the Oracle Spatial and Graph database function. Note that this Geocode tool works only in an Autonomous Database environment. For the complete list of supported database functions, see Supported Database Functions.

1.3 Oracle Data Transforms 2023.06.28.00

This release introduces the following new functionality:

  • Data Load support for Databricks Delta Share - When you create a Data Load, you now have the option to connect to a Delta Share server, select a share, and cache the tables from the share in an Oracle Autonomous Database. The dataload supports incremental loading and can be scheduled to run regularly to keep the cache up to date. To use a Delta Share sharing server to load data from Delta Lake servers to Oracle Autonomous Database see Create a Delta Share Connection for detailed instructions.
  • Delta Share compatibility with Oracle Data Share - You can use the Delta Share connection to cache tables from Oracle Data Share to Oracle Autonomous Database. See Create a Delta Share Connection for more information.
  • Reserved Words support for Data Loads - The Data Load Detail page now includes options that you can use to specify how you want to name the columns in the target tables. You can choose to retain the columns names as is from the source tables, convert column names to uppercase and replace spaces and special characters with underscores, or add a prefix to column names that are reserved words. See Run a Data Load for more details.
  • Multiple Data Flows not supported - From this release onwards a Data Flow can only have one execution flow. You cannot put multiple flows on a Data Flow and a flow cannot diverge into multiple flows. As a consequence of this, the Split component is no longer available for use.

1.4 Oracle Data Transforms 2023.04.21.00

This release introduces the following new functionality:

  • Provisioning Wizard Enhancements to Allow Searching using OCID - You can now search for the Autonomous Database instance where the repository will be created by using the OCID of the compartment that contains the database instance. See Provision the Oracle Data Transforms Metadata Repository for more details.
  • Supported Connection Type - This release includes support for the ServiceNow service connection type.

    For the complete list of supported connection types, see Supported Connection Types.

  • AutoREST Connector Configuration – For connectors that use an autoREST driver that provides the model files along with the driver, Data Transforms samples all the API endpoints listed in the model file when it creates the connection to these datasources. See Work with Connections. When you import data entities from such source connections, Data Transforms retrieves the data from all API endpoints listed in the model file and lists them as separate entities in the Data Entities page.

1.5 Oracle Data Transforms 2023.03.20.00

This release introduces the following new functionality:

  • Redwood UI for Data Transform - Oracle Data Transforms now supports the Oracle Redwood theme to make for a standardized user experience while using the application.
  • Supported Connection Type - This release includes support for the Shopify application connection type.

    For the complete list of supported connection types, see Supported Connection Types.

1.6 Oracle Data Transforms 2023.02.22.00

This release introduces the following new functionality:

  • SQL and PLSQL step in Workflows - The Workflow Editor now includes the SQL icon that allows you to add a SQL or PL/SQL step in the workflow. You can choose an Oracle connection in the step properties page and add the query that you want to run as part of the workflow. See Create a New Workflow for detailed instructions.

1.7 Oracle Data Transforms 2023.01.08.01

This release introduces the following new functionality:

  • Additional Connection Types - The supported application connection types in this release are:
    • Google Ads
    • Magento
    • WooCommerce
    • Workday

    For the complete list of supported connection types, see Supported Connection Types.

1.8 Oracle Data Transforms 2022.11.16.00

This release introduces the following new functionality:

  • Additional Connection Types - The supported connection types in this release are:
    • Ahrefs
    • Amazon Aurora
    • Amazon EMR Hive
    • Apache Hive
    • AWS S3
    • Azure Billing
    • Azure Compute
    • Azure Reserved VM Instances
    • Azure Resource Health
    • Azure SQL Database
    • Azure Synapse Analytics
    • Cloudera CDH Hive
    • Confluence Cloud
    • DataStax
    • DocuSign
    • EnterpriseDB
    • FinancialForce
    • FourSquare
    • Google AdSense
    • Google Calendar
    • Google Campaign Manager
    • Google Contacts
    • Google Drive
    • Google Search Ads 360
    • Hortonworks Hive
    • IBM BigInsights
    • IBM DB2 Hosted
    • IBM DB2 Warehouse
    • Klaviyo
    • Mailchimp
    • MapR Hive
    • Marketo
    • PayPal
    • Pivotal HD
    • Pivotal HDB
    • Qmetry
    • QuickBooks Online
    • QuickBooks Payments
    • Quora Ads
    • Sage
    • Salesforce Chatter
    • SAP BW/4HANA
    • SAP NetWeaver
    • Semrush
    • Snowflake
    • Square
    • Stripe
    • Tumblr
    • Twitter
    • Veeva CRM
    • Volusion
    • Wistia
    • WordPress
    • Yelp
    • Zoho CRM
    • Zoom

    For the complete list of supported connection types, see Supported Connection Types.

  • Incremental Merge and Incremental Append for Data Loads – The Data Load Detail page now includes two options - Incremental Merge and Incremental Append - that you can use to load tables to the target schema. For more information, see Run a Data Load.
  • Inline View for Creating Data Entities - The Create Data Entity page includes the Inline View data entity type option that you can use to run SELECT statement queries to create data entities. The page includes tabs where you can to add the query, view a read-only list of the columns that the query returns, and preview the column data. See Create Data Entities for more information.
  • Multiple Schema selection for Data Flows - You can now add multiple schemas to import data entities that you want to use in a data flow.
  • Run Commands from OCI Console - When the Oracle Data Transforms repository provisioning process completes, the Run command feature of the OCI Console includes commands that you can run without having to login into the instance. See Use the OCI Console to run commands in the Data Transforms instance for more details.
  • Resetting the Schema Password - The schema password expires every 180 days from the repository creation date or from the last password reset date. You need to reset the password in the Autonomous Database as well the local wallet. See Reset the Schema User Password for detailed instructions.

1.9 Oracle Data Transforms 2022.08.23.00

This release introduces the following new functionality:

  • Additional Connection Types - The supported connection types in this release are:
    • 4HANA Cloud
    • Cloudera Impala
    • Microsoft SharePoint
    • SAP S

    For the complete list of supported connection types, see Supported Connection Types.

  • Remote Data Loads - You can now connect to any Data Transforms instance and run data load jobs remotely. The left panel of the Workflow Details page now also includes two folders named Data Loads and Remote Data Loads. The Data Loads folder lists all the data loads that you have created in your local Data Transforms instance. The Remote Data folder lists all the data loads that you have created in the remote Data Transforms instance that you are connected to. See Create a Data Transforms Connection for Remote Data Load and Run a Data Load for more information.
  • Pre-built Workflows to Build Oracle Netsuite Data Warehouse - For Oracle NetSuite connections, Data Transforms now includes the Build Data Warehouse wizard that you can use to choose the business areas that you want to include in the data warehouse you want to create in a target server. When the wizard finishes processing completely it installs pre-built dataflows and workflows that you can run to transfer data from the NetSuite connection to the target schema. See Create an Oracle Netsuite Connection for detailed information.
  • Generic Rest Support - You can now connect to any REST service endpoint to import data entities into Data Transforms and use them in data flows. See Create a REST Server Connection for instructions on how to create a Rest Server connection.
  • DBMS Cloud Authentication Support for Oracle Object Storage Connections - You can now use DBMS Cloud authentication to connect to your Oracle Object Storage connections. To use this connection mode you need to create the credentials in the repository as well as in the Autonomous Database that you created during instance creation. See Create an Oracle Object Storage Connection for more details.

1.10 Oracle Data Transforms 2022.07.20.00

This release introduces the following new functionality:

  • Additional Connection Types - The supported connection types in this release are:
    • Aha!
    • GitHub
    • Jira
    • TeamCity

    For the complete list of supported connection types, see Supported Connection Types.

  • Provisioning Wizard Enhancements to Allow Custom Wallet Password - When you select the Autonomous Database instance where the repository will be created, Data Transforms generates a one-time password required to download the wallet file. If you want to specify a custom wallet password that matches specific validation criteria you can now do that in the provisioning wizard.

1.11 Oracle Data Transforms 2022.06.22.00

This release introduces the following new functionality:

  • Additional Connection Types - The supported connection types in this release are:
    • Apache Spark SQL
    • Azure Data Lake Storage
    • Greenplum
    • MySQL Heatwave

    For the complete list of supported connection types, see Supported Connection Types.

  • Compression option for incremental loads on Oracle - The Incremental Update data integration mode now includes an Auto compression option. When this option is set to True, data flow jobs that use the Incremental Update mode to load data onto a compressed Oracle target partition will recompress the modified target partitions after the load completes successfully. Note that for table partitions that are not originally compressed, the compression is skipped irrespective of whether Auto compression is set to true.
  • Search and Filter Enhancements in the Data Load User Interface - You can now filter the list of data entities displayed in the Data Load Detail page by specifying regular expression characters in the search keyword to refine the results. You can then select items from the filtered list and load them to the target schema. You can also sort the displayed list to view the selected and unselected items. See Run a Data Load.

1.12 Oracle Data Transforms 2022.05.19.00

This release introduces the following new functionality:

  • Support for Oracle Financials Cloud – Data Transforms now includes support for fetching real time transactional data from Oracle Financials Cloud REST endpoints. You can import the resources that are associated with the specified REST endpoint and load it on to ADW. See Create an Oracle Financials Cloud Connection for more information.
  • Integration with Oracle Object Storage – You can now connect to Oracle Object Storage from Data Transforms for uploading data onto Oracle Object Store and downloading data to ADW from Oracle Object Store. For more information about how to create an Oracle Object Store connection, see Create an Oracle Object Storage Connection.
  • Creation of Custom Connectors – Apart from the built-in connectors that Data Transforms includes, this release onward you can create custom connectors that you can use to connect Data Transforms to any JDBC supported datasources. For more information see, Create Custom Connectors.
  • Supported Connection Type – This release supports the Google Big Query database connection type.

    For the latest list of supported connection types, see Supported Connection Types.

1.13 Oracle Data Transforms 2022.04.18.01

This release introduces the following new functionality:

  • Simulate Code for Data Flows – The Data Flow Editor now includes the capability to display the simulated code of a data flow job execution. The displayed details are color coded in green and red so that you can easily differentiate between the code related to the source and target schema respectively. The feature is useful if you want to check the mappings in the data flow and the code that will run in the background to complete the data flow job.

1.14 Oracle Data Transforms 2022.03.03.00

This release introduces the following new functionality and enhancements:

  • Workflows Within Workflows – You can now add other workflows to the design in the Workflow Editor.
  • Data Entities can contain Tags – When creating or editing Data Entities, you can add Tags. Tags can be used to filter the Data Entities shown in the Data Entity Page.
  • Feature Group – Data Entities can be marked as a "Feature Group" entity type. You can do this in the Advanced Options of the Data Entity creation or edit page.

1.15 Oracle Data Transforms 2022.01.30.00

This release introduces the following new functionality and enhancements:

  • Instance Provisioning Improvements – Provisioning Data Transforms instance is now a two-step process. When you provision the instance from OCI Marketplace, you only need to provide details about the network and compute instance configuration. When you connect to the instance for the first time, you are prompted to provide additional information such as whether to create a new repository or use an existing one. This allows for more accurate validation on all user inputs. See Provision the Oracle Data Transforms Metadata Repository for more details.
  • Data Load now supports Oracle GoldenGate – When you create a Data Load, you now have the option to choose an OCI GoldenGate service to load the data. In the Services section of the Connections page, you can configure the connection to OCI GoldenGate, which then shows up as an option in the Data Load page.
  • Data Flow Editor Enhancements – The Data Entities panel in the Data Flow Editor is now enhanced to list only those schemas that are associated with the data flow.
  • User Management Enhancements – This release onwards you can change the password of the SUPERVISOR user. For more information, see Change user password.
  • Unregister from ADP Utility – This release includes the UnregisterODI.sh script that allows you unregister your Oracle Data Transforms instance from Autonomous Database. See Unregister the ODI Instance from Autonomous Database for more information.

1.16 Oracle Data Transforms 2021.12.24.00

This release includes bug fixes.

1.17 Oracle Data Transforms 2021.11.29.00

This release introduces the following new functionality and enhancements:

  • Automated Repository Upgrade – If you have been using an earlier version of Data Integrator: Web Edition and are connecting to the latest Oracle Data Transforms instance, the repository upgrade happens automatically.
  • User Management Enhancements – This release includes the addition of a new role called Oracle Data Integrator Admin (odiadmin) that allows the assigned user to change user passwords, delete users, and assign the role to other users. The SUPERVISOR user is assigned the role by default. For more information, see Users
  • Sleep Operation in Workflows – The Workflow Details page now includes a sleep operator that allows you to define the duration of delay between the steps in the workflow.

1.18 Oracle Data Transforms 2021.10.08.01

This release includes bug fixes.

1.19 Oracle Data Transforms 2021.10.08.00

This release includes the following new functionality and enhancements:

  • User Management – This release introduces the User Management page in the Administration tab of Oracle Data Transforms that allows you to add users and change user passwords. For more information, see Users.
  • Default Project Names – This release includes an enhancement wherein a default project name is auto-populated in the Projects page based on the logged-in user. If the user is logged in as SUPERVISOR, the default project name is Home. For other users, the default project name is in the format <username>_Home. You can edit the default value.
  • Enhancements to the Schema selection feature – The schema list would fail to populate for connections that have missing information such as user name or password not specified, wallet missing, and so on. This release includes an Edit option that opens the Update Connection page where you can fill in the missing details.
  • Embedded Job Status – When you run a data load, data flow, or workflow Oracle Data Transforms runs jobs in the background to complete the request. You can see the status of the last job or the job that is currently running in the Status Panel of the Data Load Details, the Data Flow Editor, and the Workflow Editor page. For more information about the Status panel, see Monitor Status of Data Loads, Data Flows, and Workflows.
  • Supported Connection Types - The supported application connection types in this release are:
    • BigCommerce (requires driver installation)
    • eBay (requires driver installation)
    • Google Ads (requires driver installation)
    • Google Analytics
    • HubSpot
    • Magento (requires driver installation)
    • Shopify (requires driver installation)
    • WooCommerce (requires driver installation)
    • Workday (requires driver installation)
    • Xero (requires driver installation)
    • Zendesk (requires driver installation)

    The supported database connection types in this release are:

    • Netezza (requires driver installation)
    • PostgreSQL
    • Teradata (requires driver installation)

    For the latest list of supported connection types, see Supported Connection Types.

1.20 Oracle Data Transforms 2021.09.01.01

This release includes the following bug fix:

  • Authorizing Logins from the Database Actions page - If you choose the Register Oracle Data Transforms with The Autonomous Database Actions Page option when creating a Data Transforms instance, you see the Data Transforms card on your Database Actions page. Login redirections to the Data Transforms instance from the Data Transforms card could result in test connection or import data entity failures. This has been fixed.

1.21 Oracle Data Transforms 2021.09.01.00 – Initial Release

This release includes the following new functionality and enhancements:

  • Simplified User Interface – The Oracle Data Transforms Home page provides easy wizards to get you started. This is very handy for first time users who may not be very familiar with Oracle Data Integrator. The Load Data and Transform Data wizards take users through the prerequisites needed to create a data flow such as creating a connection, importing data entities, and creating and running a data load.
  • Data Loads – A data load allows you to load multiple data entities from a source connection to a target connection. You can view details of the source schema, the data entities that are loaded from the source schema, and the details of the target schema. You can then choose the action that you want to apply on each data entity – recreate, truncate, append - and load tables in bulk to the target schema. For more information, see Create a Data Load.
  • Statistics of Data Entities – You can preview detailed statistics of a data entity either from the list of data entities or from any data flow. The information you can see includes the total number of rows and columns in the database table, and two types of thumbnail representations – bar chart and table. You can click the thumbnail to drill down to view detailed statistics of each column. To know more, see View Statistics of Data Entities.
  • Data Flow Attributes – The data flow diagram now has better grouping of attributes.
  • Supported Connection Types - The connection types that Data Transforms supports are categorized as Applications and Databases.

    The supported application connection types in this release are:

    • Amazon Redshift
    • Cassandra
    • Hypersonic SQL
    • IBM DB2 UDB
    • IBM DB2/400
    • Informix
    • Microsoft SQL Server
    • Mongo DB
    • Oracle
    • MySQL
    • Sybase As Anywhere
    • Sybase as Enterprise
    • Sybase AS IQ

    The supported database connection types in this release are:

    • Oracle Analytics Cloud
    • Oracle BI Cloud Connector
    • Oracle EBS
    • Oracle ERP Cloud
    • Oracle Fusion ERP
    • Oracle Fusion Sales
    • Oracle Fusion Service
    • Oracle Marketing Cloud
    • Oracle NetSuite
    • Oracle People Soft
    • Oracle Sales Cloud
    • Oracle Service Cloud
    • Oracle SIEBEL
    • Salesforce.com

    For the latest list of supported connection types, see Supported Connection Types.

  • Database Functions – The Data Flow Editor supports the use of the following database functions:
    • Data Transform: Aggregate, Expression, Filter, Join, Distinct, Lookup, Set, Sort, Split, Subquery Filter, and Table Function.
    • Data Preparation: Data Cleanse (with Imputation), Substitution, Equi-Width Binning, Quantile Binning, Lead, Lag, and Replace.
    • Machine Learning: Prediction and Outlier Detection.
    • Text: Regexp Count, Regexp Instr, Regexp Substr, Regexp Replace, Edit Distance Similarity, and Contains.
    • Oracle Spatial and Graph: Buffer Dim, Bunner Tol, Distance Dim, Distance Tol, Nearest, Simplify, Point, Geocode As Geometry, Geocode, Geocode Add, Geocode All, Geocode Addr All, Reverse Geocode, and Spatial Join.
  • Enhanced Oracle Data Integrator Knowledge Modules – Oracle Data Integrator Knowledge Modules are optimized so that users can use MEDIUM and HIGH connections to Autonomous Database. This enhancement improves the performance of E-LT-Style data integration executed in Autonomous Database.