Prepare to Deploy On-Demand HPC and AI Platforms on OCI

Use this preparation guidance to design access, compute, storage, identity, and operations foundations before deploying the platform.

Review the preparation domains for an on-demand HPC and AI platform on OCI.

User Access and OOD Services

Prepare the user access path and OOD service design so browser-based workflows remain secure, consistent, and supportable.

Users access the platform over HTTPS through OOD, while approved administrators use controlled SSH paths for support and break-glass operations.

  • Use OCI Load Balancer for HTTPS entry, with optional OCI Web Application Firewall, network security groups (NSGs), or IP allowlists.
  • Use Microsoft Entra ID and OIDC for SSO to the OOD portal.
  • Treat OOD as an application platform, not only a package install, and version-control customizations.
  • Validate Apache routing, OOD applications, and /node and /rnode proxy behavior as part of readiness checks.
  • Confirm interactive application launch paths for Remote Desktop, VS Code, JupyterLab, RStudio, and MATLAB.

Data Movement, Provisioning, Monitoring, and Automation

Prepare shared data handling, project provisioning, observability, and automation so scaling events remain reliable for user-facing workflows.

Design the operational platform so data movement, identity provisioning, monitoring, and recurring maintenance workflows are consistent across static and autoscaled nodes.

  • Define data mover paths between OCI Object Storage, shared file systems, project workspaces, scratch paths, and Lustre.
  • Define just-in-time (JIT) provisioning for home directories, project directories, permissions, and Slurm account alignment.
  • Use Prometheus and Grafana to monitor Slurm services, node readiness, storage mounts, GPU runtime, and OOD application health.
  • Run idempotent cron and automation jobs for host synchronization, cleanup workflows, quota checks, bootstrap repair, and post-upgrade OOD patching.

HPC Cluster Topology and Slurm Partitions

Prepare Slurm control-plane topology and partition strategy so CPU and GPU capacity scales predictably across workload types.

Use a cluster design that separates controller, login, permanent compute, autoscaling CPU, and autoscaling GPU roles for operational clarity.

  • Design Slurm controller services for scheduling, accounting, node state management, and autoscale integration.
  • Keep login nodes focused on submission and diagnostics, not heavy workload execution.
  • Use permanent compute partitions for low-latency workloads and baseline troubleshooting.
  • Use autoscaling CPU partitions for bursty interactive and batch workloads.
  • Separate autoscaling GPU partitions by workload type and image lane to handle runtime, driver, and storage dependencies.
  • Validate scheduler readiness with MUNGE, slurmd, node identity, generic resources (GRES), and storage mounts before marking nodes ready.

Multi-Tier Storage Design

Prepare storage tiers by workload behavior so user state, shared datasets, and high-throughput I/O use appropriate services and mount paths.

Use a tiered storage design to separate persistent user state, shared project datasets, and high-performance workload I/O paths.

  • Use OCI File Storage (FSS) for shared user and runtime paths such as /home , /scratch , and /apps .
  • Use OCI Object Storage with JuiceFS for durable shared datasets and project data, commonly mounted at /odata.
  • Use File Storage with Lustre for high-throughput AI training, checkpointing, and simulation I/O, commonly mounted at /lustre.
  • Validate each mount independently on every node class used for batch and interactive workflows.

Identity Management

Prepare centralized Linux identity and project authorization so access, quotas, and scheduling controls remain consistent across the platform.

Use centralized identity management to keep Linux accounts, groups, host access controls, and scheduler authorization aligned for multi-tenant workloads.

  • Use FreeIPA for LDAP users, POSIX groups, Kerberos authentication, and host-based access control.
  • Map project and lab groups to Slurm accounts to enforce isolation, quota ownership, and chargeback or showback reporting.
  • Validate identity service restart behavior so SSH, home-directory ownership, and Slurm startup remain stable after maintenance events.
  • Plan integration between Microsoft Entra ID group sources and Linux identity governance workflows where project authorization is required.