examples/tutorials/run-management/cloud-run-management.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Flow run management in Azure"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "**Requirements** - In order to benefit from this tutorial, you will need:\n",
    "- An Azure account with an active subscription - [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- An Azure ML workspace - [Configure workspace](https://github.com/microsoft/promptflow/blob/main/examples/configuration.ipynb)\n",
    "- A python environment\n",
    "- Installed prompt flow SDK\n",
    "\n",
    "\n",
    "**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
    "- create run with remote data\n",
    "- create run which references another runs inputs\n",
    "- manage runs via run.yaml\n",
    "- create run with connection override\n",
    "\n",
    "**Motivations** - This guide will walk you through cloud run management abilities."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Install dependent packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -r ../../requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Connect to Azure Machine Learning Workspace\n",
    "\n",
    "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the job will be run.\n",
    "\n",
    "## 1.1 Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n",
    "from azure.ai.ml.entities import Data\n",
    "from azure.core.exceptions import ResourceNotFoundError\n",
    "\n",
    "from promptflow.azure import PFClient\n",
    "from promptflow.entities import Run"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Configure credential\n",
    "\n",
    "We are using `DefaultAzureCredential` to get access to workspace. \n",
    "`DefaultAzureCredential` should be capable of handling most Azure SDK authentication scenarios. \n",
    "\n",
    "Reference for more available credentials if it does not work for you: [configure credential example](https://github.com/microsoft/promptflow/blob/main/examples/configuration.ipynb), [azure-identity reference doc](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity?view=azure-python)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    credential = DefaultAzureCredential()\n",
    "    # Check if given credential can get token successfully.\n",
    "    credential.get_token(\"https://management.azure.com/.default\")\n",
    "except Exception as ex:\n",
    "    # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work\n",
    "    credential = InteractiveBrowserCredential()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.3 Get a handle to the workspace\n",
    "\n",
    "We use config file to connect to a workspace. The Azure ML workspace should be configured with computer cluster. [Check this notebook for configure a workspace](https://github.com/microsoft/promptflow/blob/main/examples/configuration.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get a handle to workspace\n",
    "pf = PFClient.from_config(credential=credential)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.4 Create necessary connections\n",
    "Connection helps securely store and manage secret keys or other sensitive credentials required for interacting with LLM and other external tools for example Azure Content Safety.\n",
    "\n",
    "In this notebook, we will use flow `web-classification` which uses connection `open_ai_connection` inside, we need to set up the connection if we haven't added it before.\n",
    "\n",
    "Prepare your Azure OpenAI resource follow this [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal) and get your `api_key` if you don't have one.\n",
    "\n",
    "Please go to [workspace portal](https://ml.azure.com/), click `Prompt flow` -> `Connections` -> `Create`, then follow the instruction to create your own connections. \n",
    "Learn more on [connections](https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/concept-connections?view=azureml-api-2)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Create run with remote data\n",
    "\n",
    "Instead of relying on local files, there may be situations where you want to reuse data that's already available in your workspace when submitting a flow.\n",
    "The following code cells show how to create flow run with remote data.\n",
    "\n",
    "### 2.1 Create or update remote data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_name, data_version = \"flow_run_test_data\", \"1\"\n",
    "\n",
    "try:\n",
    "    data = pf.ml_client.data.get(name=data_name, version=data_version)\n",
    "except ResourceNotFoundError:\n",
    "    data = Data(\n",
    "        name=data_name,\n",
    "        version=data_version,\n",
    "        path=f\"../../flows/standard/web-classification/data.jsonl\",\n",
    "        type=\"uri_file\",\n",
    "    )\n",
    "    data = pf.ml_client.data.create_or_update(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Prepare remote data id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_id = f\"azureml:{data.name}:{data.version}\"\n",
    "print(data_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Create a flow run with remote data\n",
    "\n",
    "You can change instance type or idle time of the runtime or reset it to clean state. The following code cell shows how to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create run\n",
    "run = Run(\n",
    "    # local flow file\n",
    "    flow=\"../../flows/standard/web-classification\",\n",
    "    # remote data\n",
    "    data=data_id,\n",
    "    # to customize runtime instance type and compute instance, you can provide them in resources\n",
    "    # resources={\n",
    "    #     \"instance_type\": \"STANDARD_DS11_V2\",\n",
    "    #     \"compute\": \"my_compute_instance\"\n",
    "    # }\n",
    "    # to customize identity, you can provide them in identity\n",
    "    # identity={\n",
    "    #     \"type\": \"managed\",\n",
    "    # }\n",
    ")\n",
    "\n",
    "base_run = pf.runs.create_or_update(run=run)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Stream the flow run to make sure it runs successfully"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pf.runs.stream(base_run)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 Create a flow run which uses an existing run's inputs\n",
    "\n",
    "When running a flow with an existing run, you can reference either it's inputs or outputs in column mapping.\n",
    "The following code cell show how to reference a run's inputs in column mapping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run = Run(\n",
    "    # local flow file\n",
    "    flow=\"../../flows/standard/web-classification\",\n",
    "    # run name\n",
    "    run=run,\n",
    "    column_mapping={\n",
    "        # reference another run's input data columns\n",
    "        \"url\": \"${run.inputs.url}\",\n",
    "        \"answer\": \"${run.inputs.answer}\",\n",
    "        \"evidence\": \"${run.inputs.evidence}\",\n",
    "    },\n",
    ")\n",
    "\n",
    "base_run = pf.runs.create_or_update(\n",
    "    run=run,\n",
    ")\n",
    "\n",
    "pf.runs.stream(base_run)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Create a flow run with connection override\n",
    "\n",
    "Sometime you want to switch connection or deployment name inside a flow when submitting it.\n",
    "Connection override provided an easy way to do it without changing original `flow.dag.yaml`.\n",
    "In the following code cell, we will submit flow `web-classification` and override it's connection `open_ai_connection` to `azure_open_ai_connection`. \n",
    "Please make sure the connection `azure_open_ai_connection` exists in your workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run = Run(\n",
    "    # local flow file\n",
    "    flow=\"../../flows/standard/web-classification\",\n",
    "    data=\"../../flows/standard/web-classification/data.jsonl\",\n",
    "    # override connection for node classify_with_llm & summarize_text_content\n",
    "    connections={\n",
    "        \"classify_with_llm\": {\"connection\": \"azure_open_ai_connection\"},\n",
    "        \"summarize_text_content\": {\"connection\": \"azure_open_ai_connection\"},\n",
    "    },\n",
    ")\n",
    "\n",
    "base_run = pf.runs.create_or_update(\n",
    "    run=run,\n",
    ")\n",
    "\n",
    "pf.runs.stream(base_run)"
   ]
  }
 ],
 "metadata": {
  "build_doc": {
   "author": [
    "0mza987@github.com",
    "wangchao1230@github.com"
   ],
   "category": "azure",
   "section": "Flow",
   "weight": 50
  },
  "description": "Flow run management in Azure AI",
  "kernelspec": {
   "display_name": "github_v2",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  },
  "resources": "examples/requirements.txt, examples/flows/standard/web-classification"
 },
 "nbformat": 4,
 "nbformat_minor": 2
}