Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add subprocess pip install example in lightweight component example notebook #1817

Merged
merged 1 commit into from
Aug 15, 2019
Merged

Add subprocess pip install example in lightweight component example notebook #1817

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions samples/core/lightweight_component/Lightweight Python components - basics.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"\n",
"There are several requirements for the function:\n",
"* The function should be stand-alone. It should not use any code declared outside of the function definition. Any imports should be added inside the main function. Any helper functions should also be defined inside the main function.\n",
"* The function can only import packages that are available in the base image. If you need to import a package that's not available you can try to find a container image that already includes the required packages. (As a workaround you can use the module subprocess to run pip install for the required package.)\n",
"* The function can only import packages that are available in the base image. If you need to import a package that's not available you can try to find a container image that already includes the required packages. (As a workaround you can use the module subprocess to run pip install for the required package. There is an example below in my_divmod function.)\n",
"* If the function operates on numbers, the parameters need to have type hints. Supported types are ```[int, float, bool]```. Everything else is passed as string.\n",
"* To build a component with multiple output values, use the typing.NamedTuple type hint syntax: ```NamedTuple('MyFunctionOutputs', [('output_name_1', type), ('output_name_2', float)])```"
]
Expand All @@ -38,9 +38,8 @@
"metadata": {},
"outputs": [],
"source": [
"# Install the SDK and tensorflow\n",
"!pip3 install kfp --upgrade\n",
"!pip3 install tensorflow==1.8.0"
"# Install the SDK\n",
"!pip3 install kfp --upgrade"
]
},
{
Expand Down Expand Up @@ -105,6 +104,12 @@
"from typing import NamedTuple\n",
"def my_divmod(dividend: float, divisor:float, output_dir:str = './') -> NamedTuple('MyDivmodOutput', [('quotient', float), ('remainder', float)]):\n",
" '''Divides two numbers and calculate the quotient and remainder'''\n",
" #Pip installs inside a component function.\n",
" #NOTE: installs should be placed right at the beginning to avoid upgrading a package\n",
" # after it has already been imported and cached by python\n",
" import sys, subprocess;\n",
" subprocess.run([sys.executable, '-m', 'pip', 'install', 'tensorflow==1.8.0'])\n",
" \n",
" #Imports inside a component function:\n",
" import numpy as np\n",
"\n",
Expand Down Expand Up @@ -154,7 +159,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 5,
"metadata": {},
"outputs": [
{
Expand All @@ -163,7 +168,7 @@
"MyDivmodOutput(quotient=14, remainder=2)"
]
},
"execution_count": 9,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -289,7 +294,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.5"
"version": "3.6.7"
}
},
"nbformat": 4,
Expand Down