Access MLOps Studio Pipelines API from outside the cluster
This guide explains how to connect to the MLOps Studio Pipelines API from outside the cluster by using a local Python script.
MLOps Studio Pipelines let you turn machine learning work into repeatable workflows. Instead of running every step manually, you define a sequence of steps such as data preparation, training, evaluation, and publishing results. The pipeline can then be run again with the same structure, which makes experiments easier to repeat, compare, and automate.
Accessing the Pipelines API from outside the cluster is useful when you want to trigger runs from your local machine, integrate pipeline execution with another tool, list experiments, check run status, or build your own automation around MLOps Studio. Because MLOps Studio is protected by authentication, the local script must first obtain a valid authenticated session cookie. After that, it can create a Pipelines client and use the API.
This article covers three external-access workflows in the following order:
working with an MLOps Studio notebook from a local IDE,
emulating a browser login and capturing the authenticated session cookie for the Pipelines API,
using Keycloak credentials loaded from a local environment file for advanced, non-interactive access.
The IDE workflow connects to a notebook environment and does not authenticate a local script to the Pipelines API. For API access, browser-based login avoids storing a password locally but requires interactive login when the session expires. The advanced Keycloak method is suitable for repeatable automation only when the local environment can protect the credential file.
Prerequisites
No. 1 Access to the MLOps Studio workspace
You must be able to sign in to MLOps Studio and open your workspace. Follow Access the MLOps Studio workspace, then keep the workspace URL available for the local script.
No. 2 Existing namespace and namespace access
Use an existing namespace that your account can access. If needed, follow Create and manage a MLOps Studio namespace, then have the exact namespace name ready for the Pipelines client.
No. 3 MLOps Studio Pipelines access
Confirm that your namespace and user session can open MLOps Studio Pipelines. The same account and namespace are used when the local client lists or manages pipeline experiments.
No. 4 Local Python environment
Install Python 3.10 or newer on your local machine. This version is required by the type-annotation syntax used in the browser-login example.
No. 5 MLOps Studio URL and Pipelines API path
Have the URL of your MLOps Studio environment ready for the KF_URL setting, and confirm the Pipelines API path for the KFP_API_PREFIX setting. Both authentication examples use these values.
No. 6 Credential storage for advanced login
The advanced Keycloak method requires a local .env file that is excluded from version control. Use browser-based login instead if you do not want to store a password locally.
Security considerations
Accessing MLOps Studio from outside the cluster requires an authenticated session. The examples in this article obtain an authentication cookie and pass it to the MLOps Studio Pipelines client.
Treat authentication cookies as sensitive data. A valid cookie may allow access to MLOps Studio resources without entering the password again.
The credential-based method reads the username and password from a local .env file. Do not place real usernames, passwords, generated cookies, or the .env file in shared repositories, tickets, documentation screenshots, or other publicly accessible locations. Add .env to the project’s .gitignore file before creating it.
If you do not need unattended execution, use the browser-based login method instead of storing credentials locally.
Using MLOps Studio notebooks from your IDE
If you want to work with MLOps Studio notebooks from a local IDE, follow Access a MLOps Studio notebook environment with Visual Studio Code Remote Tunnels. The tutorial explains how to connect to the notebook environment while keeping the runtime, files, and terminal inside MLOps Studio.
Emulating a browser login
If you do not want to store credentials locally, but you are fine with opening a browser window and logging in manually, use the browser-based version.
This approach requires only the MLOps Studio URL. After you install the required libraries and run the script, it opens a browser window, waits for you to complete login, captures the authenticated cookie, and then tests the MLOps Studio Pipelines client.
Install the required libraries for browser login
Create a virtual environment and install the required libraries before running the browser-login script:
python3 -m venv .venv
source .venv/bin/activate
pip install kfp requests playwright
playwright install chromium
Create the browser-login script
Create a file named kfp-browser-login.py:
import kfp
import time
import importlib
import requests
from urllib.parse import urlparse, urljoin
# --- CONFIGURATION ---
KF_URL = "https://tenant-kmlb4o52m4-cluster-csc4pjyla1.mkf.leonardo.data.destination-earth.eu"
NAMESPACE = "yourNamespace" # <--- UPDATE THIS
KFP_API_PREFIX = "/pipeline" # <--- UPDATE THIS
VERIFY_SSL = True # Keep False if using self-signed certs
LOGIN_TIMEOUT_MINUTES = 10
class BrowserAuthManager:
def __init__(self):
parsed = urlparse(KF_URL)
self.target_host = parsed.hostname
@staticmethod
def _is_auth_cookie(name: str) -> bool:
return name.startswith("_oauth2_proxy") or name == "authservice_session"
@staticmethod
def _is_usable_auth_cookie(name: str) -> bool:
lowered = name.lower()
if "csrf" in lowered:
return False
return name.startswith("_oauth2_proxy") or name == "authservice_session"
def _is_cookie_for_target(self, domain: str) -> bool:
if not self.target_host:
return True
normalized_domain = (domain or "").lstrip(".").lower()
target = self.target_host.lower()
return target == normalized_domain or target.endswith(f".{normalized_domain}")
def _build_cookie_string(self, cookies) -> str | None:
usable = [
cookie
for cookie in cookies
if self._is_usable_auth_cookie(cookie.get("name", ""))
and self._is_cookie_for_target(cookie.get("domain", ""))
]
if not usable:
return None
return "; ".join(f"{cookie['name']}={cookie['value']}" for cookie in usable)
@staticmethod
def _cookie_auth_valid(cookie_str: str) -> bool:
healthz_url = f"{KF_URL.rstrip('/')}{KFP_API_PREFIX.rstrip('/')}/apis/v2beta1/healthz"
try:
response = requests.get(
healthz_url,
headers={"Cookie": cookie_str},
verify=VERIFY_SSL,
timeout=10,
allow_redirects=False,
)
return response.status_code == 200
except requests.RequestException:
return False
@staticmethod
def _is_keycloak_page(url: str) -> bool:
lowered = url.lower()
return "realms" in lowered or "keycloak" in lowered
@staticmethod
def _try_click_keycloak_entry(page) -> bool:
selectors = [
"a:has-text('Sign in with Keycloak')",
"button:has-text('Sign in with Keycloak')",
"a[href*='oauth2/start']",
"form[action*='oauth2/start'] button",
"form[action*='oauth2/start'] input[type='submit']",
]
for selector in selectors:
try:
locator = page.locator(selector).first
if locator.count() > 0:
locator.click(timeout=1000)
return True
except Exception:
continue
return False
def login(self):
print(f"1. Opening browser for login at: {KF_URL}")
print(" -> Complete login in the opened browser window.")
try:
playwright_sync = importlib.import_module("playwright.sync_api")
sync_playwright = getattr(playwright_sync, "sync_playwright")
except ModuleNotFoundError:
raise RuntimeError(
"Playwright is not installed. Install it with: pip install playwright && playwright install chromium"
)
with sync_playwright() as p:
browser = p.chromium.launch(headless=False)
context = browser.new_context(ignore_https_errors=not VERIFY_SSL)
page = context.new_page()
page.goto(KF_URL, wait_until="domcontentloaded")
print("2. Handling two-step auth entry (Sign in with Keycloak)...")
timeout_seconds = LOGIN_TIMEOUT_MINUTES * 60
start_time = time.time()
deadline = start_time + timeout_seconds
forced_oidc_start = False
clicked_entry = False
keycloak_prompt_printed = False
rejected_once = False
while time.time() < deadline:
if not self._is_keycloak_page(page.url):
clicked_now = self._try_click_keycloak_entry(page)
if clicked_now and not clicked_entry:
print(" -> Clicked 'Sign in with Keycloak'.")
clicked_entry = True
if (
not clicked_now
and not forced_oidc_start
and (time.time() - start_time) > 8
):
fallback_url = urljoin(KF_URL, "/oauth2/start?rd=/")
print(f" -> Fallback to OIDC start URL: {fallback_url}")
page.goto(fallback_url, wait_until="domcontentloaded")
forced_oidc_start = True
cookies = context.cookies()
cookie_str = self._build_cookie_string(cookies)
if cookie_str:
if self._cookie_auth_valid(cookie_str):
print("SUCCESS: Valid authenticated cookie captured from browser session.")
browser.close()
return cookie_str
if not rejected_once:
print(
" -> Found cookie candidate, but session is not authorized yet. "
"Continue login in the browser window..."
)
rejected_once = True
if self._is_keycloak_page(page.url) and not keycloak_prompt_printed:
print("3. Keycloak login page detected. Complete login in browser window...")
keycloak_prompt_printed = True
page.wait_for_timeout(1500)
all_cookie_names = [cookie["name"] for cookie in context.cookies()]
print(
"FAILURE: Timed out waiting for auth cookie. "
f"Cookies found: {all_cookie_names}"
)
browser.close()
return None
if __name__ == "__main__":
auth = BrowserAuthManager()
cookie_str = auth.login()
if cookie_str:
print("\n--- Testing KFP Client ---")
host = f"{KF_URL.rstrip('/')}{KFP_API_PREFIX.rstrip('/')}"
try:
client = kfp.Client(host=host, cookies=cookie_str, verify_ssl=VERIFY_SSL)
print("Listing experiments...")
# We use page_size=1 to keep output clean
print(client.list_experiments(page_size=1, namespace=NAMESPACE))
except Exception as e:
print(f"KFP Error: {e}")
Run the script:
python3 ./kfp-browser-login.py
A browser window opens. Complete the login in that browser window.
A successful run may look like this:
1. Opening browser for login at: https://kubeflow.domain.tld/
-> Complete login in the opened browser window.
2. Handling two-step auth entry (Sign in with Keycloak)...
3. Keycloak login page detected. Complete login in browser window...
SUCCESS: Valid authenticated cookie captured from browser session.
--- Testing KFP Client ---
Listing experiments...
{'experiments': None, 'next_page_token': None, 'total_size': None}
Using Keycloak credentials - advanced
If you need repeatable, non-interactive access to MLOps Studio from outside the cluster, you can use a Python script that authenticates with Keycloak and returns the cookie needed for further interaction. The script loads the credentials from a local .env file; it does not contain the credentials themselves.
Important
This method is only applicable to accounts with credentials stored directly in Keycloak (local Keycloak accounts). Users normally authenticate through DESP OpenID and should use the browser-based login method above. Local Keycloak accounts are an exceptional use case and are not the standard authentication method for MLOps Studio users.
Create the credential file
Before creating the credential file, add .env to .gitignore. Then create .env in the same directory as the script and replace the example values with your own values:
USERNAME="yourUsername"
PASSWORD="yourPassword"
NAMESPACE="yourNamespace"
KF_URL="https://tenant-kmlb4o52m4-cluster-csc4pjyla1.mkf.leonardo.data.destination-earth.eu"
KFP_API_PREFIX="/pipeline"
VERIFY_SSL=True
On Linux, run chmod 600 .env to restrict read and write access to your user account. Before committing or sharing the directory, use git status --short to confirm that .env is not listed.
The NAMESPACE value is used when the script lists pipeline experiments.
Install the required libraries for credential-based login
To run the credential-based script locally, create a virtual environment and install the required libraries:
python3 -m venv .venv
source .venv/bin/activate
pip install kfp requests beautifulsoup4 python-dotenv
Create the credential-based script
Create a file named kfp-test.py. The configuration values are loaded from .env:
import kfp
import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin, urlparse
import os
from pathlib import Path
from dotenv import load_dotenv
ENV_PATH = Path(__file__).resolve().parent / ".env"
load_dotenv(ENV_PATH)
KF_URL = os.environ.get("KF_URL", "").rstrip("/")
KFP_API_PREFIX = os.environ.get("KFP_API_PREFIX", "/pipeline")
USERNAME = os.environ.get("USERNAME", "")
PASSWORD = os.environ.get("PASSWORD", "")
NAMESPACE = os.environ.get("NAMESPACE", "")
VERIFY_SSL = os.environ.get("VERIFY_SSL", "True").lower() in ("true", "1", "yes")
class TwoStageAuthManager:
def __init__(self):
self.session = requests.Session()
self.session.headers.update({
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
})
if not VERIFY_SSL:
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
def login(self):
print(f"1. hitting initial URL: {KF_URL}")
r = self.session.get(KF_URL, verify=VERIFY_SSL, allow_redirects=True, timeout=10)
# STAGE 1: Check if we need to click "Start Login" (if we got a 403 or are not on Keycloak yet)
if "realms" not in r.url and (r.status_code == 403 or "/oauth2/start" in r.text):
print(" -> Landed on Access Denied/Start page. Looking for login start link...")
soup = BeautifulSoup(r.text, 'html.parser')
# Find the start action (usually in a generic form or link)
# We look for the form that points to /oauth2/start
start_url = None
for form in soup.find_all('form'):
action = form.get('action', '')
if 'oauth2/start' in action:
start_url = urljoin(r.url, action)
break
if not start_url:
# Fallback: Just try forcing the oauth2 start URL
start_url = urljoin(KF_URL, "/oauth2/start?rd=/")
print(f" -> Triggering OIDC flow via: {start_url}")
r = self.session.get(start_url, verify=VERIFY_SSL, allow_redirects=True, timeout=10)
# STAGE 2: We should now be on the Keycloak Login Page
if "realms" not in r.url and "Sign in" not in r.text:
print(f"CRITICAL: Could not reach Keycloak page. Current URL: {r.url}")
return None
print(f"2. Landed on Keycloak Login Page: {r.url.split('?')[0]}...")
soup = BeautifulSoup(r.text, 'html.parser')
login_form = soup.find('form')
if not login_form:
raise Exception("Could not find login form on Keycloak page.")
# Prepare Payload
action_url = login_form.get('action')
if not action_url.startswith('http'):
action_url = urljoin(r.url, action_url)
payload = {}
for input_tag in login_form.find_all('input'):
name = input_tag.get('name')
value = input_tag.get('value', '')
if name:
payload[name] = value
payload['username'] = USERNAME
payload['password'] = PASSWORD
# Submit Credentials
print("3. Submitting credentials to Keycloak...")
post_r = self.session.post(action_url, data=payload, verify=VERIFY_SSL, allow_redirects=True, timeout=10)
# STAGE 3: Finalize (The Callback)
# Sometimes Keycloak redirects to a "Callback" URL which then sets the cookie.
# We ensure we are back on the original domain.
if "tenant" not in post_r.url:
print(" -> Still on Keycloak? Attempting to follow redirect manually if needed...")
# One last check to the home page to ensure cookies are set
final_check = self.session.get(KF_URL, verify=VERIFY_SSL, timeout=10)
cookies = self.session.cookies.get_dict()
if any(k.startswith('_oauth2_proxy') or k == 'authservice_session' for k in cookies):
print("SUCCESS: Authenticated and Proxy Cookie obtained.")
return "; ".join([f"{k}={v}" for k, v in cookies.items()])
else:
print("FAILURE: Login flow finished but Proxy Cookie missing.")
print(f"Cookies found: {list(cookies.keys())}")
return None
if __name__ == "__main__":
auth = TwoStageAuthManager()
cookie_str = auth.login()
if cookie_str:
print("\n--- Testing KFP Client ---")
host = urljoin(KF_URL, KFP_API_PREFIX)
try:
client = kfp.Client(host=host, cookies=cookie_str, verify_ssl=VERIFY_SSL)
print("Listing experiments...")
# We use page_size=1 to keep output clean
print(client.list_experiments(page_size=1, namespace=NAMESPACE))
except Exception as e:
print(f"KFP Error: {e}")
Run the script:
python3 kfp-test.py
A successful run may produce output similar to this:
1. hitting initial URL: https://REDACTED.com
-> Landed on Access Denied/Start page. Looking for login start link...
-> Triggering OIDC flow via: https://REDACTED.com/oauth2/start
2. Landed on Keycloak Login Page: https://REDACTED/realms/qa-kubeflow/protocol/openid-connect/auth...
3. Submitting credentials to Keycloak...
SUCCESS: Authenticated and Proxy Cookie obtained.
--- Testing KFP Client ---
Listing experiments...
{'experiments': None, 'next_page_token': None, 'total_size': None}
The exact output depends on the namespace and on whether pipeline experiments already exist.
Troubleshooting
If the script cannot reach the Keycloak login page, verify the MLOps Studio URL and confirm that you can open the same URL in a browser.
If the script finishes login but cannot obtain the authentication cookie, check whether the authentication flow has changed or whether your user has the required access.
If the MLOps Studio Pipelines client returns an authorization error, confirm that:
your user can access MLOps Studio in the browser,
your namespace name is correct,
your user has access to that namespace,
MLOps Studio Pipelines access is enabled for your namespace.
If the browser-based script times out, increase the value of LOGIN_TIMEOUT_MINUTES and run it again.
What to do next
After confirming that the MLOps Studio Pipelines client can connect from outside the cluster, continue with Pipelines to create, upload, and run a basic pipeline.
If pipeline components need to communicate with Model Registry, follow MLOps Studio Pipelines: Accessing Model Registry via Istio.
Before relying on the workflow in external automation, review Known limitations in MLOps Studio.