Getting Started with Data Ingestion

Overview of Data Ingestion

To analyze shop floor data, you must first acquire the data from sources such as ERP applications, Manufacturing Execution Systems (MES), and Quality/Laboratory Information Management Systems (LIMS) as well as from shop floor sensor devices mounted on machines. Oracle Adaptive Intelligent Apps for Manufacturing (AIAMFG) provides CSV file-based upload tools to collect data. Users can extract structured data from external source systems and semi structured data from machines and equipment sensor devices and load them into the data lake in Oracle Cloud. Equipment and sensor device data is then contextualized with equipment and work order information and summarized for analysis.

You can use spreadsheet templates and REST web services in AIAMFG to load data from various information technology and operational technology systems. Upload this data for any historical period and for one or more products using periodic and incremental batch uploads. The application also provides out-of-the-box integrations between AIAMFG and Oracle E-Business Suite (EBS), which enable the collection of data from EBS applications.

Structured Data

Import enterprise or business data into AIAMFG using CSV file templates. Two types of data uploads are supported, one for upload of prepared data for quick analysis and the other a detailed upload of entities to take full advantage of all AIAMFG features. Users may choose the upload method that meets their requirement. Oracle partners can help users extract data from external source systems and load the data into the AIAMFG data lake in Oracle Cloud.

Case Record Data

Upload case record data to obtain insights and predictions on historical data and in-progress work orders. Case Record Data CSV files captures data like item number, reference information like work order number, actual start date, actual end date, operation code, and attributes which can be targets or input features. Use this data, captured in a flattened file format, for data preparation, and then model building. Upload case record data in order to take advantage of AIAMFG features like Insights and Predictions.

Business Entity Data

Upload business entity data to capture the key entities from external source systems, entity by entity. Use this data in AIAMFG for data preparation and subsequent model building, as well as trace analysis. The CSV data files capture data for individual business entities such as items, lots, departments, persons, machines, receiving, work orders, quality, and so on. AIAMFG stitches all of the uploaded data together for analysis by relating the underlying data structures. Upload business entity data in order to take advantage of all AIAMFG features, including Insights, Predictions, Genealogy and Trace, and Factory Command Center.

Semi-structured Data

Import sensor stream and alert data from shop floor sensor devices into AIAMFG using CSV files. The sensor data is contextualized with the business entity data and summarized for analysis. Separate templates are available for sensor stream and alert data.

Sensor Data

Oracle partners can help users to configure machine data acquisition systems, such as Supervisory Control and Data Acquisition (SCADA), Distributed Control Systems (DCS) and other gateway device systems, to extract machine sensor data into the CSV file format. Users can then upload these sensor data files into AIAMFG in batch mode. The application processes the sensor stream data (for example, temperature) and alert data (for example, idle, paused), contextualizes it with equipment and work order information, and then summarizes the contextualized data for analysis.

Understanding Data Ingestion Methods

Users can ingest data into AIAMFG using either CSV files or out-of -the-box integration tools such as Oracle Data Pump and Oracle GoldenGate.

Uploading CSV Files

Users can import both structured and semi-structured data into AIAMFG using CSV files. Oracle partners can help users to configure and map data from external data sources into seeded templates for upload into AIAMFG. The process of extracting, configuring, and mapping the data from an external data source to a CSV file is the same irrespective of the data ingestion method used.

User Interfaces

AIAMFG provides three distinct user interfaces to import case record, business entity, and sensor data. Users can extract the data from external systems, enter the data into seeded templates, and then import the data into the application using these user interfaces.

REST Services

AIAMFG provides three REST services to import Business Entity data into AIAMFG using CSV files. Download a seeded template, enter data as suggested in the template guidelines, and then save the template as a CSV file. You can then import/upload the data file by calling a REST service. See: REST Web Services, Oracle Adaptive Intelligent Apps for Manufacturing User's Guide.

Using Oracle Data Pump and Oracle GoldenGate

AIAMFG provides out-of-the-box integration with Oracle E-Business Suite applications using Oracle Data Pump and Oracle GoldenGate. Oracle Data Pump enables high-speed transfer of data and metadata from a source database to the target database. Oracle GoldenGate provides near real time replication of data and real time capture, routing, and delivery of data across databases.

Comparing Data Types and Ingestion Methods

The following table compares the types of data available for ingestion with the various ingestion methods and features you can choose from.

Additional Information: By importing Business Entity work order CSV files, you are able to import work orders that have not started or have partially progressed. By creating appropriate models with these work orders, you can make predictions for yield and quality attributes.

Data Ingestion: A Comparison

the picture is described in the document text

Understanding the Data Ingestion Process

The Oracle Adaptive Intelligent Apps for Manufacturing Data Ingestion process consists of the following steps:

  1. Copying a template to use as the basis for a CSV file, which matches the requirements of the target application table. Oracle and its partners can help users to configure and map the data.

  2. Following an upload process to load the data files from the source system or local machine to storage cloud services.

  3. Using data import programs to import data from interface tables in Oracle Storage Cloud Service to the AIAMFG tables in Oracle Database Cloud Service.

Storage Cloud Services

Oracle Storage Cloud Service serves as a storage area for uploaded CSV files and contextualized machine sensor data.. It also stores enterprise data from external ERP systems such as JDE, SAP, SCM Cloud, EBS, or shop floor systems such as MES and LIMS. Both sensor and enterprise data are captured in a CSV file format and stored in the Storage Cloud.

Database Cloud Services

Oracle Database Cloud Service serves as the data lake for AIAMFG by storing data for analysis. The types of data stored in this database include Enterprise Resource Planning (ERP) application data, Manufacturing Execution System (MES) data, Quality/LIMS data, sensor device mapping definitions, and summarized machine sensor data. This data is used for model building and analysis, lot genealogy, presenting the manufacturing time line, and providing a real-time overview of factory events in the Factory Command Center.

Following Template Guidelines for Data Ingestion

When uploading data files using Data Ingestion templates, follow these guidelines:

When uploading data files directly into the Storage Cloud, all file names within the same folder must be unique.

Setting Up Data Ingestion

Perform the following setup steps in order to ingest data:

  1. Define organizations. See: Defining Organizations.

  2. Define users, which includes the following tasks. See: Creating and Managing Users, Oracle Adaptive Intelligent Apps for Manufacturing User's Guide.

    1. Add a new user.

    2. Assign the user to an application and a role.

    3. Assign organization access to a user.

  3. Define user preferences. See: Defining User Preferences, Oracle Adaptive Intelligent Apps for Manufacturing User's Guide.

    Important: In order to upload business entity CSV files, a user must set their user preference to an organization that allows business entity data ingestion only. Similarly, in order to upload case record CSV files, set your user preferences to an organization that allows case record data ingestion only.

Defining Organizations

When defining organizations, consider the following information:

The following chart summarizes which type of organization to use with each type of data ingestion method.

the picture is described in the document text

To create a Case Record or Business Entity data ingestion organization:

AIAMFG displays analysis data by organization, so data collected from various data sources using different ingestion methods belongs to a unique organization code. Each organization lists the ingestion method under the organization name:

  1. Navigate to the Create Organization page.

    From the Setup page, click Organization Access, then Create Organization.

  2. In the Name field, enter a unique name for the organization.

  3. In the Code field, enter a unique organization code.

  4. Select one of the following organization Types:

    • Discrete Manufacturing

    • Process Manufacturing

  5. Select the Data Ingestion organization type:

    • Allow case record data ingestion only

    • Allow business entity data ingestion only

  6. Click Create.

    the picture is described in the document text

Supported Browsers

AIAMFG is supported on multiple web browser platforms. Support is provided by Oracle on all platforms for which the browser vendor provides support. For mobile device operating systems, Oracle provides support for the most recent browser delivered by the device operating system only.

Platform Google Chrome Mozilla Firefox Microsoft Browsers Apple Safari
Android Not Supported Not Supported N/A N/A
iOS Not Supported Not Supported N/A Supported
Mac OS X Supported Supported N/A Supported
Windows Supported Supported Supported* N/A

* Internet Explorer 11+ and Microsoft Edge, although the Network Viewer in the Genealogy and Trace page does not display when using Internet Explorer. Support for Microsoft Browsers will follow the same N-1 (the most recent version plus one previous release) support policy that iOS provides.