Analyze and Extract Information from Images with a Vision Action

You can use AI to perform deep-learning–based image analysis with a vision action. Oracle Cloud Infrastructure Vision enables you to automatically extract textual or visual information from images.

Capabilities

OCI Vision is an AI service that enables you to detect objects and text in images, classify images using labels, and detect faces in images. You can use OCI Vision to detect visual anomalies (for example, anomalies in manufacturing), identify objects in images to automate counting of common items (such as packages, products, shipments, vehicles, and so on), identify items that need attention (such as overgrowth of vegetation near an electric power line), classify items (for example, digital media assets) to organize them, identify textual/visual data in images to power analytic applications, and much more. Oracle Integration supports using OCI Vision in an integration with the vision action.

See AI Vision.

Prerequisites

See Prerequisites for information on the prerequisites you must satisfy in the Oracle Cloud Infrastructure Console.

Invoke Oracle Cloud Infrastructure Vision from an Integration

  1. Add a Vision action to an integration in either of the following ways:
    • On the side of the canvas, click Actions Integration actions icon and drag the OCI Vision action to the appropriate location.
    • Click Add icon at the location where you want to add the OCI Vision action, then select OCI Vision.
  2. Enter a name and optional description, then click Continue.
  3. Select the following information, then click Continue.
    Element Description
    Categories
    Select one of the following categories to use:
    • Pre-trained models: Pre-trained models let you perform image analysis without the need to create your own model. Pre-trained models enable you to use AI with no data science experience. You can provide an image to the OCI Vision action service and get information about the objects, text, and faces in the image. You can also classify the objects in an image using labels.
    • Image Jobs: Image jobs let you perform analysis on a set of images by creating batch jobs. It also enables you to get information about a batch image job and cancel a batch image job.
    Action

    If you selected the Pre-trained models category, select the Analyze Image action to perform image analysis.

    If you selected the Image Jobs category, select one of the following jobs:
    • CancelImageJob: Cancels the image job using the image job ID you specify.
    • CreateImageJob: Creates an image job. This action reads the images from object storage and returns the results of the analysis along with the image job ID.
    • GetImageJob: Gets the status of the image job using the image job ID you specify. This action returns the status of the image job.
  4. You can perform the following image analysis tasks using the OCI Vision action:
    • Image Classification: Classifies objects in an image using labels. For example, if an image is a surveillance image of a street that has a vehicle, an electric power line, and a tree, OCI Vision can identify these objects, classify them using labels, and return this information. This might be helpful to identify items that need attention such as overgrowth of vegetation near an electric power line.
    • Object Detection: Identifies objects in an image. For example, if an image includes vehicles such as a car, a bus, or a truck, OCI Vision can identify these objects and return this information.
    • Text Detection: Identifies words and lines of text in an image. For example, if an image is a visa application form and it contains text such as name, city, citizenship, country and other text content, OCI Vision can recognize the text and return this information.
    • Face Detection: Identifies faces and facial features of people in an image. For example, if an image is a photograph of a group of people, OCI Vision can identify the faces and facial features of the people in the image and return this information.

    To select an image analysis task, perform the following.

    1. If you selected the Pre-trained models category in Step 3, then select the following information, and click Continue.
      Element Description
      Compartment

      Select the Oracle Cloud Infrastructure compartment in which your Oracle Integration is installed.

      Task
      Select the task to perform:
      • Image Classification
      • Object Detection
      • Text Detection
      • Face Detection
      Input source Select an input source where the image to analyze will be available:
      • Inline: If you select this option, you need to provide the image in base-64 format using the mapper.
      • Object storage: If you select this option, you need to make the image available in an object storage bucket. You can select the object storage bucket that has the image in the Input storage bucket field.
      Input storage bucket

      This field is available only when you select Object storage as the input source.

      Select the input storage bucket that contains the image to analyze from the list of buckets.
      Namespace

      This field is available only when you select Object storage as the input source.

      Select a namespace. To understand namespaces, see Understanding Object Storage Namespaces.

      Maximum no. of results to be returned Select the maximum number of results to be returned after the image has been analyzed using the selected task.
    2. If you selected the Image Jobs category in Step 3, then select the following information, and click Continue.
      Element Description
      Compartment

      Select the Oracle Cloud Infrastructure compartment in which your Oracle Integration is installed.

      Task
      Select the task to perform:
      • Image Classification
      • Object Detection
      • Text Detection
      • Face Detection
      Output storage bucket

      Select the output storage bucket where you want to store the results of the image analysis task.
      Namespace

      Select a namespace. To understand namespaces, see Understanding Object Storage Namespaces.

      Maximum no. of results to be returned Select the maximum number of results to be returned after the images have been analyzed using the selected task.
  5. On the Summary page, click Finish.
Upon completing the configuration in the wizard, you must perform specific tasks in the mapper based on the category and input source you configured.

Configure the Mapper to Read the Inline Image for the Pre-trained Models Category

You must send the data in base-64 format. Open the mapper and define the mappings between the source and target elements as needed.

  1. Drag Stream Reference from the Sources section into Data in the Target section.
  2. Click Functions Mapper functions icon.
  3. Click Design View Switch view icon in the Expression Builder.
  4. In the Functions section, expand Advanced, and drag encodeReferenceToBase64 into the Expression Builder for the Data target element.
    oraext:encodeReferenceToBase64 (/nssrcmpr:execute/ns23:streamReference)

    Note:

    You can optionally specify Compartment Id in the mapper to override the value you selected initially for Compartment.
  5. Exit the mapper.

    The vision action is now configured.

Configure the Mapper to Read the Image from a Storage Bucket for the Pre-trained Models Category

You must select the object name (of the image) to read from the object storage bucket in the mapper. Open the mapper and define the mappings between the source and target elements as needed.

  1. Map the object name from the Sources section to the target Object Name in the Target section.

    Note:

    • Object name is the unique identifier you defined when uploading the image (object) to the object storage bucket. For more information about object names, see Object Names.
    • You can optionally specify Compartment Id, Namespace Name, and Bucket Name in the mapper to override the value you selected initially for these elements in Step 4.
  2. Exit the mapper.

    The vision action is now configured.

Configure the Mapper for the CancelImageJob Action

This action cancels the image job using the image job ID you specify in the request payload. You can get the image job ID from the response of the CreateImageJob action. You can only cancel an image job that is in progress.

  1. Map the job ID from the Sources section to the target Image Job Id in the Target section.
  2. Exit the mapper.

    The vision action is now configured.

Configure the Mapper for the CreateImageJob Action

This action creates an image job to read a set of images from the object storage bucket and perform the analysis tasks specified. The input from the storage bucket is sent through the request mapper. The output is sent to the object storage bucket. The output returned also includes the image job ID. Open the mapper and define the mappings between the source and target elements as needed.

  1. Map the appropriate source elements to the following target elements:
    • Features
    • Feature Type
    • Source Type under Input Location
    • Object Locations under Input Location
    • Bucket Name under Object Locations
    • Namespace Name under Object Locations
    • Object Name under Object Locations
    • Prefix under Output Location
    • Display Name under Output Location
    • Is Zip Output Enabled under Output Location

    The values such as Display Name and Is Zip Output Enabled are optional. For information about the above parameters, see ImageJob Reference.

    Note:

    • Feature type is the task you selected in Step 4b.
    • Prefix is used in the file name of the output files.
  2. Exit the mapper.

    The vision action is now configured.

Configure the Mapper for the GetImageJob Action

This action returns the status of the image job using the image job ID you specify in the request payload. You can get the image job ID from the response of the CreateImageJob action.

  1. Map the job ID from the Sources section to the target Image Job Id in the Target section.
  2. Exit the mapper.

    The vision action is now configured.