Use an existing job as a pipeline step in a new Data Science pipeline.
Important Review prerequisites at
Before You Begin on the pipeline creation page.
This task documents the steps for adding a pipeline step from an existing job when creating a pipeline by using the Oracle Cloud Console.
When creating and managing pipelines, we recommend that you use the ADS SDK. Alternatives to the Console or ADS include API and CLI. To use the API to create a pipeline, see CreatePipeline in the API Reference. To use the CLI to create a pipeline, see oci data-science pipeline create in the OCI CLI Command Reference.
-
Follow the steps at Creating a Pipeline to open the Create pipeline page.
-
On the Create pipeline page, select Add pipeline steps.
-
Select From jobs and enter the following information.
Note
Optionally create a default pipeline configuration that's used when the pipeline is run by entering environment variable, command line arguments, and maximum runtime options.
- Step name: Enter a unique name for the step. You can't repeat a step name in a pipeline.
- Step description (Optional): Enter a step description, which can help you find step dependencies.
- Step run name
- Depends on (Optional): If this step depends on another step, select one or more steps to run before this step.
- Select a job compartment: Select the compartment containing the job that you want to use as a pipeline step.
- Select a job: Select the job that you want to use as a pipeline step.
- Default configuration (Optional):
Note
The step needs to ensure that the specified file (for example,
/home/datascience/output.json) is populated with a valid JSON defining the specified variables. For example:
{ "message":"Hello John!", "ocpu": 2, "memory": 10 }
- Custom environment variable key (Optional): The environment variables for this pipeline step.
- Value (Optional): The key's value.
- Command line arguments (Optional): Enter the command line arguments that you want to use for running the pipeline step.
- Maximum runtime (in minutes) (Optional): The maximum number of minutes that the pipeline step is allowed to run. The service cancels a pipeline run if its runtime exceeds the specified value. The maximum runtime is 30 days (43,200 minutes). We recommend that you configure a maximum runtime on all pipeline runs to prevent runaway pipeline runs.
- Output parameters (Optional):
- Output parameter type: Select
JSON.
- Parameter name: Enter a parameter name.
- Output file name: Select the output file name in which the step stores the output parameters. For example:
/home/datascience/output.json.
-
Select Save.
The Create pipeline page reopens with the step added.
Continue following steps at Creating a Pipeline to define and create the new pipeline.