About System Architecture

System Architecture Overview

The system architecture uses a modular approach, with distinct stages for data storage, data fetching, model training, model deployment and management, prediction services, and integration with client applications or AI agents. Each stage consists of independently deployable microservices, which operate as pods in a Kubernetes environment.

The key components of the architecture include:

Data layer: The source layer for preparing, fragmenting, and supplying data through a configurable data service mesh.

For more information, see "About the Data Layer".
Persistence layer: The storage repository for both raw and processed data.

For more information, see "About the Persistence Layer".
Training services: A framework that supports model training using curated data.

For more information, see "About Training Services".
ML Ops – model management: A repository for managing, versioning, and monitoring trained models.

For more information, see "About ML Ops – Model Management".
Prediction services: Inference services that use deployed ML models.

For more information, see "About Prediction Services".
Client applications or AI agents: End-user applications or automated agents that consume AI-driven services.

For more information, see "About Client Applications and AI Agents".

Figure 2-1 shows the high-level system architecture and its major components.

Figure 2-1 System Architecture Overview

Description of "Figure 2-1 System Architecture Overview"

In Figure 2-1, the data fetch service layer ingests raw data from source systems, transforms and enriches it, and caches it as domain-specific data fragments. This data is then stored in the persistence layer, which securely stores both raw and processed datasets for downstream use. When you develop a model, training services access high-volume, engineered data from the persistence layer for model training.

After training, the model is published to the ML Ops (model management) layer for versioning, tracking, and lifecycle governance. The prediction services use the published model to retrieve relevant cached input data from the data layer. They then use this data to respond to real-time prediction requests. These predictions are then consumed by the client applications or AI agents to get insights for decision making.

About the Data Layer

The data layer is the primary source that prepares, fragments, and supplies the required data for downstream ML tasks. This layer includes two data zones:

Bronze zone: Contains raw, unprocessed data ingested from primary sources, such as Oracle DB or Prometheus.
Silver zone: Holds cleaned, enriched, and aggregated data optimized for modeling.

The silver zone implements a data mesh, in which each service owns and caches a specific data fragment (for example, a service for subscriber usage data in Billing and Revenue Management).

Data fragments in the silver zone are cached and validated for quick access. Each service includes metadata describing its data. You configure data services in the silver zone to regularly fetch updates from the bronze zone and update cached fragments and the persistence layer as needed.

The data service can collect data from multiple sources, such as Siebel, BRM, and Prometheus. For example, it can collect usage, account, product, and related data from BRM and collect customer, purchase, campaign, offer, and label data from Siebel.

The data service processes each source and prepares refined data fragments for downstream services. Depending on the use case, BRM and Siebel data can remain as separate fragments or can be combined during preprocessing.

When configuring multiple sources in one request, you need to specify the source type, query configuration, and connection details in the same order. This ensures that the data service uses the correct connection details for each source.

Data services can be deployed on OCI or non-OCI environments. For OCI deployments, data engineering either uses OCI Data Flow directly or calls OCI Data Flow using OCI Data Science, and data is stored in OCI Object Storage. For non-OCI environments, data engineering is handled by PySpark, and engineered data is persisted to either PVC, OCI Object Storage, or Oracle DB, based on the configuration.

Figure 2-2 shows the high-level architecture of the Data Service layer.

Figure 2-2 Data Service Layer Architecture

Description of "Figure 2-2 Data Service Layer Architecture"

About the Persistence Layer

The persistence layer acts as the centralized, long-term storage repository for all data. After data services process raw data from the bronze zone and cache it for immediate use, they persist this data for durability and downstream analytics, training, and prediction.

Data services maintain and secure the persistent data, ensuring it is current and available. They also use the persistence layer to identify and fetch new or changed data from the raw data source, enabling efficient incremental updates.

The data can be persisted in one of the following:

Persistent Volume Claims (PVC) or Oracle DB for containerized environments
OCI Object Store for scalable, object-based storage

About Training Services

Training Services are primarily responsible for orchestrating machine learning model development. The microservices begin with fetching the input data in bulk directly from the persistence layer as required for the high-volume training workload. Using the persistence layer also reduces the time for data fetching.

The microservices then run the training jobs to train the model based on the input data. When the training is complete, the model is pushed to the model management layer along with the relevant metadata or metrics.

Training service supports unsupervised algorithms like KNN and Cosine Similarity, supervised algorithms like DNN (for single classification) and multi-head DNN (for multiple classification), and XGBoost (for time series based prediction).

For example, training services support multi-head DNN models for use cases that require multiple predictions from the same customer data. A multi-head model uses one shared set of input features and creates a separate output head for each target label.

In this case, the training data must contain two or more label columns that use the LABEL_ prefix, such as LABEL_1 and LABEL_2. Each label column is encoded separately and is associated with a separate model output.

Note:

If you are using models like DNN and XGBoost, there should be just one column named LABEL.

The training service preprocesses the data, trains the model, evaluates each output head, and stores the model with its metadata. The metadata can include feature definitions, label mappings, and model metrics. The prediction service uses this information to prepare input data and interpret prediction results.

You can deploy training services on an OCI environment using OCI Data Science or on a non-OCI environment like Kubernetes.

Figure 2-3 shows the high-level architecture of the Training Service layer.

Figure 2-3 Training Service Layer Architecture

Description of "Figure 2-3 Training Service Layer Architecture"

About ML Ops – Model Management

The ML Ops (model management) layer functions as the central repository and governance platform for trained models. Once a model is trained by the training services, it is published here for versioning, tracking, and lifecycle management. The model management system stores metadata for each model, including version history and associated metrics.

The training service stores the model in ONNX format. Associated model files can include label-encoder metadata, feature-column metadata, training metrics, and evaluation metrics for each output model.

Before using a model, the user must manually specify the model version that the prediction service must load. The prediction service loads the specified model version and uses the associated metadata to prepare input data and interpret output values.

This layer is certified with OCI Data Science offering ML Ops capabilities and with the open-source MLFlow technology.

About Prediction Services

Prediction services are specialized microservices that perform inference operations within the architecture. The microservices retrieve the required input data from the data mesh in the data service layer. The data mesh is a collection of services responsible for owning and caching specific data fragments for immediate use. This design ensures low latency and high throughput, optimizing overall performance.

When the prediction service starts, it loads the specified trained model version from the model management layer. For each request, the service uses the relevant input data to generate predictions. Client applications or AI agents then consume and act upon these predictions.

For offer recommendation use cases, the prediction service retrieves cached customer data from data service and uses an approved multi-head model to generate recommendations. A prediction can return more than one result type. For example, it can return a recommended charge or product offer and a recommended discount offer for the same customer.

The prediction service returns structured results to the calling integration layer. The result include expected output and configured details, such as confidence score, prediction, and the used algorithm.

For time-series anomaly detection, the prediction service retrieves current metrics directly from Prometheus. The service prepares the metrics using the deployed model metadata and returns the classification for the latest timestamp in the requested time window.

You can use the prediction services by connecting them to a model on OCI Data Science or implement them using Python Flask sample frameworks.

Figure 2-4 shows the high-level architecture of the Prediction Service layer.

Figure 2-4 Prediction Service Layer Architecture

Description of "Figure 2-4 Prediction Service Layer Architecture"

About Client Applications and AI Agents

You can use client applications or AI agents as the interface between end users and AI-ML models. These may include Cloud Scale Monetization applications or AI-driven agents that invoke prediction services for real-time or batch inference.

For client applications, you can invoke prediction services through REST APIs. For AI agents, you can request recommendations through a MCP server.

The MCP server provides a controlled interface between the client and the prediction service. The client sends the required request context to the MCP server. The MCP server validates the request, calls the prediction service, and returns the result as a structured MCP tool response.

The MCP server enables clients to use prediction capabilities without requiring direct access to data sources, model artifacts, or model-management systems. Access to these resources is controlled by the configured monetization services and deployment security policies.