Use OCI Vision to automate inventory management

Manual inventory management can be a time consuming and labour intensive process. Manufacturers often struggle with keeping an updated stock of inventory and track of their location leading to lost revenue and under-utilised capital. The inventory counting process on-site for large remote factories or warehouse facilities involves time and resource intensive steps, constraints and is prone to errors.

OCI Vision can be used to supplement manual inventory management efforts by using drones (or other robots) to observe, count and report information reliably and faster.

OCI Vision is one of several AI services available on Oracle Cloud Infrastructure. OCI Vision gives you the power to apply machine learning and artificial intelligence without needing data science expertise. OCI Vision includes pre-built models available out-of-the-box, which allow image recognition and text recognition to be built into your applications, without prior machine learning (ML) expertise. These models can also be used to detect visual anomalies in manufacturing, extract text from documents to automate business workflows.

This reference architecture describes the recently developed proof-of-concept for Oracle Industry Lab, where the goal was to use stacked pipes as an example of an automated inventory counting scenario, using a programmable drone and a trained, custom OCI Vision model.

Architecture

The following diagram illustrates the logical flow for this reference architecture.


Description of oci-vision-inventory-flow.png follows
Description of the illustration oci-vision-inventory-flow.png

The first step of the development process was labeling a data set of pictures of stacked pipes, using the OCI Data Labeling service. After the labeling was completed, an OCI Vision project and an OCI Vision custom model were created. The OCI Vision custom model was trained using the labeled data set of stacked pipe images.

Once the OCI Vision model was ready to use, Python code was deployed for handling the following operations to OCI Functions:
  • Download the drone pictures on docking via the Skydio Cloud API
  • Upload the images to OCI Object Storage
  • Count the pipes
  • Update the pipe count in the Oracle Cloud Enterprise Resource Planning system using the Oracle Fusion Cloud Inventory Management API

The created OCI Vision model is natively able to provide as output, the metadata of each recognized object (in this reference architecture: pipe). The metadata for each object contains the coordinates of the bounding box that encloses the object, and a confidence value. The total count is not present in the native response, hence the need to include it in the code.

OCI Events was used to automate the inventory management processes. When images are uploaded to Object Storage, the associated event triggers OCI Functions. The function then directly calls the Oracle Fusion Cloud Inventory Management API to update the item count.

The following diagram shows the example reference architecture which can be mapped to the logical flow diagram above.


Description of oci-vision-inventory-arch.png follows
Description of the illustration oci-vision-inventory-arch.png

The architecture has the following components:

  • OCI Vision

    OCI Vision is an AI service for performing deep-learning–based image analysis at scale. With prebuilt models available out-of-the-box, developers can easily build image recognition and text recognition into their applications without machine learning (ML) expertise. For industry-specific use cases, developers can automatically train custom Vision models with their own data. These models can be used to detect visual anomalies in manufacturing, extract text from documents to automate business workflows, and tag items in images to count products or shipments.

    In addition to gaining access to pre-trained models, developers can create custom models without data science expertise or managing custom model infrastructure.

  • OCI Data Labeling

    The OCI Data Labeling service is an OCI native service that allows customers to create and browse datasets, view data records (text, images) and apply labels for the purposes of building AI/ML models. The service also provides interactive user interfaces designed to aide in the labeling process. Developers can upload raw images, add labels, and highlight areas of images. By adding these labels to images, the resulting datasets can be used to train custom image classification and object-detection models.

    Once records are labeled, the dataset can be exported as line-delimited JSON for use in AI/ML model development. OCI Data Labeling can also be accessed directly from other services such as OCI Vision and OCI Language as part of a custom model-training workflow. Data scientists who prefer to build and train their own deep learning or natural language processing models can consume the labeled dataset through OCI Data Science.

  • Object storage

    Object storage provides quick access to large amounts of structured and unstructured data of any content type, including database backups, analytic data, and rich content such as images and videos. You can safely and securely store and then retrieve data directly from the internet or from within the cloud platform. You can seamlessly scale storage without experiencing any degradation in performance or service reliability. Use standard storage for "hot" storage that you need to access quickly, immediately, and frequently. Use archive storage for "cold" storage that you retain for long periods of time and seldom or rarely access.

  • Functions

    Oracle Functions is a fully managed, multitenant, highly scalable, on-demand, Functions-as-a-Service (FaaS) platform. It is powered by the Fn Project open source engine. Functions enable you to deploy your code, and either call it directly or trigger it in response to events. Oracle Functions uses Docker containers hosted in Oracle Cloud Infrastructure Registry.

  • Events

    Oracle Cloud Infrastructure services emit events, which are structured messages that describe the changes in resources. Events are emitted for create, read, update, or delete (CRUD) operations, resource lifecycle state changes, and system events that affect cloud resources.

  • Internet gateway

    The internet gateway allows traffic between the public subnets in a VCN and the public internet.

  • Network address translation (NAT) gateway

    A NAT gateway enables private resources in a VCN to access hosts on the internet, without exposing those resources to incoming internet connections.

Recommendations

Use the following recommendations as a starting point. Your requirements might differ from the architecture described here.
  • Virtual cloud network (VCN)

    When you create a VCN, determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.

    When you create a VCN, determine the number of CIDR blocks required and the size of each block based on the number of resources that you plan to attach to subnets in the VCN. Use CIDR blocks that are within the standard private IP address space.

    Select CIDR blocks that don't overlap with any other network (in Oracle Cloud Infrastructure, your on-premises data center, or another cloud provider) to which you intend to set up private connections.

    After you create a VCN, you can change, add, and remove its CIDR blocks.

    When you design the subnets, consider your traffic flow and security requirements. Attach all the resources within a specific tier or role to the same subnet, which can serve as a security boundary.

    Use regional subnets.

  • Functions

    This architecture uses a function to call the OCI Vision REST API with a specific image and then store the metadata that is returned by OCI Vision. The function can be built using the Java or Python SDK.

  • Events

    In this architecture, the OCI Events service is configured to listen to changes in Object Storage creation. The service is invoked after the object is uploaded to Object Storage and calls the function for processing.

  • Object Storage

    When images are uploaded to Object Storage, the associated event can serve as a trigger to the item counting OCI Functions. The function can then directly call the Oracle Fusion Cloud Inventory Management API to update the item count. In this case, the implementation can be done in a way that the detailed results (including images with bounding boxes added) are stored in Object Storage, for auditing. Archive storage can be used in this scenario, as a more cost-effective option.

Considerations

Consider the following points when deploying this architecture.

  • User interface

    In the Oracle Industry Lab proof-of-concept, for demo purposes, a simple web application was developed to display bounding boxes on top of the input pictures, as a step of the process. If desired, such web application can be built and used so an operator manually validates the object count by looking at the bounding boxes.

  • Access

    OCI Vision supports access through the OCI Console, Java and Python SDK client, and OCI CLI. When testing it is recommended to use the CLI tool or Console.

Acknowledgments

  • Author: Nuno Goncalves
  • Contributors: Mihai Dragnea, Jason Monden, Hassan Ajan, Surya Kommareddy, Nithin TS