Register and Use Oracle Cloud Infrastructure Functions in Data Flows

Register Oracle Cloud Infrastructure (OCI) functions in Oracle Analytics so that you can use them in data flows to transform data.

About Using OCI Functions in Oracle Analytics

You can transform data in Oracle Analytics using functions created in OCI. For example, you might use a language conversion function to convert English text into Spanish or German.

You register OCI functions in Oracle Analytics first, and then any Oracle Analytics user with BI Service Administrator or DV Content Author privileges can use them in data flows.

About Configuring OCI Functions to use in Oracle Analytics

Create functions in OCI Console so that you can use them to transform data in Oracle Analytics data flows.

About Registering OCI Functions in Oracle Analytics

When you register an OCI function in Oracle Analytics, if it's grayed out in the Select a Function dialog, check that it's configured with the oac-compatible tag and invoked with the correct funcMode value.
Description of oci-functions-select-function-register.png follows
Description of the illustration oci-functions-select-function-register.png

About Configuring OCI Functions to use in Oracle Analytics

OCI functions that you want to use in Oracle Analytics must have an oac-compatible tag and the function code must include the funcMode variable. Configure these settings in OCI Console:

  • oac-compatible - Add a free form tag to the function with the name oac-compatible and set the value to true.Description of oci-function-tags.png follows
    Description of the illustration oci-function-tags.png
  • funcMode - Include the funcMode variable in your function code. Oracle Analytics sends a request to register an OCI function with funcMode=describeFunction and a request to invoke an OCI function with funcMode=executeFunction. The handler function in func.py should operate in describeFunction mode or executeFunction mode. This can be done based on the value of the input variable funcMode from the request as shown below.

    Description of oci-function-mode.png follows
    Description of the illustration oci-function-mode.png

    Here's the request and response format of OCI Functions for Word Count example in python.

    funcMode = describeFunction

    { "funcMode": "describeFunction"}

    When funcMode in the request is 'describeFunction', the function should return the function definition with status (returnCode & errorMessage), outputs(name, dataType of output column), parameters (name, description, type and so on of the input parameter), bucketName and so on as a JSON object in the following format (example of word count):

    funcDefinition = {
        "status": {
            "returnCode": 0,
            "errorMessage": ""
        },
        "funcDescription": {
            "outputs": [
                {"name": "word_count", "dataType": "integer"}
            ],
            "parameters": [
                {"name": "textColumn", "displayName": "Text Column",
                 "description": "Choose column to count words", "required": True,
                 "value": {"type": "column"}}
            ],
            "bucketName": "bucket-OCI-FAAS",
            "isOutputJoinableWithInput": True
        }
    }

    Add the following code in func.py to return funcDefinition when funcMode in the request is 'describeFunction'.

    def handler(ctx, data: io.BytesIO = None):
        response_data = ""
        try:
            body = json.loads(data.getvalue())
            funcMode = body.get("funcMode")
            if funcMode == 'describeFunction':
               response_data = json.dumps(funcDefinition)
        except (Exception, ValueError) as ex:
            response_data = json.dumps(
                {"error": "{0}".format(str(ex))})
        return response.Response(
            ctx, response_data,
            headers={"Content-Type": "application/json"}
        )

    funcMode = executeFunction - When funcMode is 'executeFunction', the actual function logic should be executed and the response should be sent back to Oracle Analytics with the output. When the registered function is invoked from dataflow in Oracle Analytics, the request object will be in the following format with the column name in args, input(bucketName, fileName, fileExtension, fileName, method & rowID) and output(bucketName, fileName & fileExtension).

    {
        "args":
        {
            "textColumn": "REVIEW"
        },
        "funcMode": "executeFunction",
        "input":
        {
            "bucketName": "bucket-OCI-FAAS",
            "fileExtension": ".csv",
            "fileName": "oac-fn-e99cd4fddb3844be89c7af6ea4bbeb76-input",
            "method": "csv",
            "rowID": "row_id"
        },
        "output":
        {
            "bucketName": "bucket-OCI-FAAS",
            "fileExtension": ".csv",
            "fileName": "oac-fn-e99cd4fddb3844be89c7af6ea4bbeb76-output"
        }
    }

About Creating Functions in OCI

You can create functions using the OCI Console, command line (Fn Project CLI), or API. For example, in OCI Console, click Developer Services, then Functions and follow the on-screen instructions to create applications and one or more functions. For details, see Creating Functions in OCI Documentation. If you're creating functions for the first time, follow the steps in the end to end example Creating, Deploying, and Invoking a Helloworld Function. Any functions that you want to use in Oracle Analytics must include the free form tag oac-compatible=true (see About Configuring OCI Functions to use in Oracle Analytics above).

Tips on Creating Functions in OCI Console

  • Enable Logs - By default, logs are disabled for functions. To enable logs for functions, on the Applications page, click Logs, and enable the log provided for the Log Name. Under Resources, you can now select Explore Log, and drill into log entries to debug and diagnose function issues.
  • Include Dependencies - Include dependent packages in the requirements.txt file for the function.
  • Follow Indentation Rules- Follow the Python indentation rules while coding Python functions.
  • Use Function Response Errors - Use the error details listed on the Dataflow page to diagnose issues.
  • Test Functions Locally First - Before creating a function in OCI, test the function locally in your development environment to ensure that the function is syntactically and logically correct.
  • Test functions in OCI - Before integrating functions with Oracle Analytics, make sure that you can deploy and invoke them successfully as standalone functions in OCI.
  • Increase the Timeout- By default, functions have a timeout of 30 seconds. If necessary, increase the timeout setting (for example, change it to 300 seconds) in the Edit Function page in OCI Console.
  • Increase the Memory- By default, functions have a memory limit of 1024 MB. If necessary, increase the memory limit in the Edit Function page in OCI Console.
  • Optimize Space - Delete unused files immediately after use.
  • Beware of Network Latency - Bear in mind that network latency might cause a slight delay in overall processing.

Typical Workflow for Transforming Data Using OCI Functions

Follow these high-level tasks in the order listed below to transform data using OCI functions.

Task Description More Information

Create functions in OCI (requires functions developer privileges)

In your OCI tenancy, create your functions, and make sure that they comply with the prerequisites specified for use with Oracle Analytics.

About Configuring OCI Functions to use in Oracle Analytics

Connect Oracle Analytics to your OCI tenancy (requires admin or DV author privileges) In Oracle Analytics, create a connection to your OCI tenancy.

Create a Connection to Your OCI Tenancy

Register OCI functions in Oracle Analytics (requires admin or DV author privileges) In Oracle Analytics, register your OCI functions so that you can call them from data flows.

Register OCI Functions in Oracle Analytics Cloud

Transform your data using the OCI functions (requires admin or DV author privileges) Create a data flow, and use the Apply Custom Script step to invoke an OCI function.

Transform Data Using OCI Functions

Create a Connection to Your OCI Tenancy

Create a connection to your OCI Tenancy so that you can register OCI functions in Oracle Analytics. For example, you might register a language conversion function so that you can convert English text into Spanish or German.

  1. In your OCI Tenancy, copy the Tenancy OCID and User OCID from Profile page in OCI Settings.
  2. In Oracle Analytics, on the Home page, click Create, then click Connection.
  3. At the Select Connection Type dialog, click Oracle OCI Resource.
  4. At the Create Connection dialog, paste in the Tenancy OCID and User OCID from OCI Settings.
    • For Region Identifier, select the region in which your OCI Tenancy is located.
    • For Tenancy OCID, paste in the Tenancy OCID that you copied in step 1.
    • For User OCID, paste in the User OCID that you copied in step 1.
  5. Click Generate next to API Key, then click Copy.
  6. In your OCI Tenancy, paste the API key into the API Keys page under User Settings and click Add.
  7. In Oracle Analytics, in the Create Connection dialog, click Save.

Register OCI Functions in Oracle Analytics Cloud

Register OCI functions in Oracle Analytics so that you can use them in data flows to transform data. For example, you might register a language conversion function so that data analysts can convert English text into Spanish or German.

  1. In Oracle Analytics, on the home page, click Page Menu, then Register Model/Function, then OCI Functions.
  2. At the Register a Custom Function dialog, select a connection to the OCI tenancy where your functions are located.
  3. At the Select Application dialog, select the application that contains your OCI functions.
    If you're not sure, ask the person who created the functions in OCI.
  4. At the Select a Function dialog, select a function, then click Register.
    If the function you want to use is grayed out, ask the administrator to make sure that it's configured for Oracle Analytics. See About Configuring OCI Functions to use in Oracle Analytics.
You can use registered OCI functions in data flows to transform data. To verify which functions are registered, you can view registered functions on the Scripts tab of the Machine Learning page (from the Home page, click Navigator, then Machine Learning, then Scripts).