Pipelines

This article explains how to create, upload, and run a simple MLOps Studio Pipeline by using the MLOps Studio Pipelines 2.x SDK from a local terminal with Python 3.

Overview

This procedure is intended as a basic validation of the MLOps Studio Pipelines environment. It confirms that:

the MLOps Studio Pipelines SDK is installed correctly,
a pipeline definition can be compiled into a YAML file,
the YAML file can be uploaded through the MLOps Studio UI,
a pipeline run can be started successfully,
the run finishes with a Succeeded status,
the expected log output is visible in the step details.

Prerequisites

Before starting, make sure that:

You have access to a cluster with MLOps Studio Pipelines 2.x.
You can use a local terminal with Python 3.
You can sign in to the MLOps Studio UI.
You have permission to upload and run pipelines.

Procedure

On your local machine, navigate to your working directory, for example:
```
cd ~/Desktop/H
```
Create a virtual environment:
```
python3 -m venv .venv
```
Activate the environment:
```
source .venv/bin/activate
```
Install the MLOps Studio Pipelines SDK:
```
pip install kfp
```

Create the pipe.py file with the following content:

from kfp import dsl, compiler


@dsl.component(
    base_image="python:3.11-slim"
)
def print_hello():
    import datetime
    now = datetime.datetime.now().isoformat()
    print("Hello from MLOps Studio Pipelines!")
    print(f"Current time: {now}")


@dsl.pipeline(
    name="hello-world-pipeline",
    description="Simple KFP 2.x pipeline for installation testing."
)
def hello_world_pipeline():
    print_hello()


if __name__ == "__main__":
    compiler.Compiler().compile(
        pipeline_func=hello_world_pipeline,
        package_path="hello-world-pipeline.yaml"
    )

Generate the pipeline file:
```
python pipe.py
```
Go to MLOps Studio UI → Pipelines.
Click + Upload pipeline / Create a new pipeline.
Set the pipeline name, for example:
```
Hello-abc
```
Select Upload a file and choose the generated hello-world-pipeline.yaml file.
Click Create.
Once the pipeline is created, click Create run.
Provide a run name and select or create an Experiment, then click Start.
Open the run details and go to the print-hello step.
Go to Logs for the print-hello step.

Results

The test is successful if all of the following are true:

the pipeline appears on the Pipelines list in the MLOps Studio UI,
the pipeline run finishes with a Succeeded status,
in the logs of the print-hello step, the following text, among others, is visible:

Hello from MLOps Studio Pipelines!

Scenario - Recurring Run in MLOps Studio Pipelines

Objective

Verify through the MLOps Studio UI that a simple pipeline can be executed automatically every minute and that five consecutive runs complete successfully.

This scenario uses a Kubeflow Pipelines KFP v2 pipeline, not an Apache Airflow pipeline.

Test Pipeline File

Use the following file:

hello_recurring_pipeline.yaml

components:
  comp-say-hello:
    executorLabel: exec-say-hello

deploymentSpec:
  executors:
    exec-say-hello:
      container:
        args:
          - |
            echo "Hello from recurring run with CA certificates"
            date -u
            test -f /etc/ssl/certs/ca-certificates.crt
            echo "CA bundle exists"
        command:
          - sh
          - -c
        image: curlimages/curl:8.12.1

pipelineInfo:
  description: Test KFP 2.5.0 with an image containing system CA certificates.
  name: hello-recurring-pipeline-with-ca

root:
  dag:
    tasks:
      say-hello:
        cachingOptions:
          enableCache: false
        componentRef:
          name: comp-say-hello
        taskInfo:
          name: say-hello

schemaVersion: 2.1.0
sdkVersion: kfp-2.5.0

The pipeline contains one step that:

starts a container based on curlimages/curl:8.12.1,
prints Hello from recurring run with CA certificates,
prints the current UTC date and time,
verifies that the system CA bundle exists,
completes with the Succeeded status.

Caching is disabled, so the step should execute during every run.

Prerequisites

The user has access to MLOps Studio.
The user is working in the correct namespace.
The Pipelines component is available.
The user can upload pipelines and create recurring runs.
The cluster can pull the curlimages/curl:8.12.1 image.

Test Steps

1. Upload the Pipeline

Open MLOps Studio.
Select the correct namespace.
From the navigation menu, open Pipelines.
Click Upload pipeline.
Select the option to upload a local file.
Upload hello_recurring_pipeline.yaml.
Set the pipeline name, for example:

hello-recurring-pipeline-with-ca
Confirm the upload.

Expected Result

The pipeline appears in the pipeline list.
The pipeline contains one operation named say-hello.
No YAML validation error is displayed.

2. Create a Recurring Run

From the navigation menu, open Recurring Runs.
Click Create recurring run.
Set the recurring run name:

hello-every-minute-5-times
Select the previously uploaded pipeline:

hello-recurring-pipeline-with-ca
Select its latest version.
Select an existing experiment or create a new one:

recurring-run-ui-test
Select Periodic as the schedule type.
Configure the interval:
- value: 1
- unit: minute
Set Maximum concurrent runs to 1.
Configure the schedule to start immediately.
If the UI provides a Catchup option, leave it disabled.
Click Create.

Expected Result

The recurring run is created.
The recurring run has the Enabled status.
The schedule shows execution every 1 minute.
The correct pipeline and experiment are selected.

3. Verify Five Runs

Open the recurring run details.
Observe the list of created runs.
Wait until exactly five runs have been created.
After the fifth run is created, disable the recurring run through the UI:
- use the Enabled/Disabled switch, or
- select Disable from the actions menu.
Open Runs or the recurring-run-ui-test experiment.
Verify the status of all five runs.
Open at least one run.
Open the say-hello step and inspect its logs.

Expected Result

Exactly 5 runs are created.
Consecutive runs are created approximately one minute apart.
Every run completes with the Succeeded status.
The say-hello step also completes with the Succeeded status.

The logs contain:

Hello from recurring run with CA certificates

The logs contain the current UTC date and time.
The logs contain:
```
CA bundle exists
```
The logs show successful publication of execution logs, for example:
```
publish success
```
No sixth run is created after the recurring run is disabled.

Maximum concurrent runs limits how many runs can execute at the same time. It does not limit the total number of scheduled executions.

Pass Criteria

The test passes when:

the pipeline is uploaded through the UI,
the recurring run starts the pipeline every minute,
exactly 5 runs are created,
all 5 runs complete with the Succeeded status,
the say-hello step completes successfully,
the execution logs are successfully published,
no additional run is created after the recurring run is disabled.

Cleanup

Confirm that the recurring run has the Disabled status.
Optionally delete or archive:
- the recurring run,
- the five created runs,
- the recurring-run-ui-test experiment,
- the hello-recurring-pipeline-with-ca pipeline.

What to do next

After confirming that this basic pipeline works, you can continue with more advanced MLOps Studio Pipelines tasks, such as:

creating multi-step pipelines,
passing parameters between components,
storing outputs in object storage,
running pipelines from notebooks through the kfp SDK,
validating pipeline results in Runs and Experiments.