Skip to content
This repository has been archived by the owner on Oct 28, 2021. It is now read-only.

Workflow Parameters #11

Open
sphadley opened this issue Sep 15, 2020 · 11 comments
Open

Workflow Parameters #11

sphadley opened this issue Sep 15, 2020 · 11 comments
Assignees
@binarycrayon

Comments

@sphadley
Copy link

How do you pass workflow parameters using create_namespaced_workflow. I don't see anything about in the documentation, and I don't see anything specifically handling in the kwargs parsing. Am I missing something? Is it supported?
@abaiju15
Copy link

abaiju15 commented Sep 17, 2020 •
edited
Loading

@sphadley I'm actually wondering the same thing with the args. Were you at least able to run the example locally? When I try to submit a workflow, the workflow does get created but I'm also getting this:

Traceback (most recent call last):
  File "test.py", line 67, in <module>
    workflow = v1alpha1.create_namespaced_workflow(namespace, manifest)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api/v1alpha1_api.py", line 311, in create_namespaced_workflow
    return self.create_namespaced_workflow_with_http_info(namespace, body, **kwargs)  # noqa: E501
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api/v1alpha1_api.py", line 408, in create_namespaced_workflow_with_http_info
    collection_formats=collection_formats)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 353, in call_api
    _preload_content, _request_timeout, _host)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 192, in __call_api
    return_data = self.deserialize(response_data, response_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 264, in deserialize
    return self.__deserialize(data, response_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 639, in __deserialize_model
    kwargs[attr] = self.__deserialize(value, attr_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 639, in __deserialize_model
    kwargs[attr] = self.__deserialize(value, attr_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 281, in __deserialize
    for sub_data in data]
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 281, in <listcomp>
    for sub_data in data]
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 639, in __deserialize_model
    kwargs[attr] = self.__deserialize(value, attr_type)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 303, in __deserialize
    return self.__deserialize_model(data, klass)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/argo/workflows/client/api_client.py", line 641, in __deserialize_model
    instance = klass(**kwargs)
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/kubernetes/client/models/v1_container.py", line 123, in __init__
    self.name = name
  File "/Users/ashishbaiju/.pyenv/versions/3.7.8/lib/python3.7/site-packages/kubernetes/client/models/v1_container.py", line 350, in name
    raise ValueError("Invalid value for `name`, must not be `None`")  # noqa: E501
ValueError: Invalid value for `name`, must not be `None`

Did you happen to run into this?

@binarycrayon
Copy link
Member

@abaiju15 Long story short, the repo hasn't been updated for a while and I just completed an update in branch 2.10.2 https://github.com/argoproj-labs/argo-client-python/tree/argo/v2.10.2

I have not had a chance to release it to pypi yet, if you don't want to wait, you can try it by:

pip install -e git+https://github.com/argoproj-labs/argo-client-python.git@argo/v2.10.2#egg=argo-workflows

The new release has multiple integration tests to submit most of the example yaml via the sdk so you shouldn't run into above ValueError.

Please make sure you are running argo 2.10.x

@binarycrayon
Copy link
Member

@sphadley would you mind be more specific about your use case? Are you talking about overriding a parameter upon submitting a workflow? How is your workflow constructed?

@abaiju15
Copy link

Thanks @binarycrayon - that worked for the example! In terms of my use case of being able to submit parameters with the workflow, I'm trying to basically add an arbitrary number of input parameters to be used within the workflow. Let's say I have a template with these arguments:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: service-
spec:
  entrypoint: workflow-start
  arguments:
    parameters:
    - name: tag
      value: ""
    - name: registry
      value: ""

and I want to be able to add in more parameters at runtime, so it would look something like this:

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: service-
spec:
  entrypoint: workflow-start
  arguments:
    parameters:
    - name: tag
      value: ""
    - name: registry
      value: ""
    - name: newParam1
      value: "newValue1"
    - name: newParam2
      value: "newValue2"

How would this be possible without having to modify the workflow.yaml file ahead of time?

@sphadley
Copy link
Author

sphadley commented Sep 18, 2020 •
edited
Loading

@binarycrayon,
@abaiju15 's use case matches mine almost exactly.

Thanks,
Steven

@samuelamar15
Copy link

samuelamar15 commented Sep 28, 2020 •
edited
Loading

Hey @abaiju15!

I am getting the same issue as you except the workflow doesn't even get created.
@binarycrayon i couldn't find the branch you suggested so i tried both the argo/v2.10.0 and latest release-3.5 with argo version 2.10.2 but i am still getting the following trace:

Traceback (most recent call last):
  File "C:/Users/Samuel/AppData/Roaming/JetBrains/PyCharmCE2020.2/scratches/scratch.py", line 35, in <module>
    res = thread.get()
  File "C:\Users\Samuel\AppData\Local\Programs\Python\Python37\Lib\multiprocessing\pool.py", line 683, in get
    raise self._value
  File "C:\Users\Samuel\AppData\Local\Programs\Python\Python37\Lib\multiprocessing\pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 208, in __call_api
    return_data = self.deserialize(response_data, response_type)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 280, in deserialize
    return self.__deserialize(data, response_type)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 319, in __deserialize
    return self.__deserialize_model(data, klass)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\api_client.py", line 660, in __deserialize_model
    instance = klass(**kwargs)
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\models\v1alpha1_workflow.py", line 68, in __init__
    self.metadata = metadata
  File "c:\users\samuel\desktop\src\argo-workflows\argo\workflows\client\models\v1alpha1_workflow.py", line 138, in metadata
    raise ValueError("Invalid value for `metadata`, must not be `None`")  # noqa: E501
ValueError: Invalid value for `metadata`, must not be `None`

Process finished with exit code 1

What exactly is the versions of argo, kubenetes and argo-python-client i shoud use in order to use the api with this python client?

I also noticed argo-server is using v1 api but this client uses v1alpha1, maybe this is the problem?

Thanks!

@binarycrayon
Copy link
Member

@samuelamar15 @abaiju15 @sphadley Could you try argo-workflows==4.0.1 release?
Thanks!

@binarycrayon binarycrayon self-assigned this Dec 1, 2020
@samuelamar15
Copy link

Unfortunatly I still cannot use it because the 4.0.1 release is not compatible with the 0.3.0 of argo-dsl.

@binarycrayon
Copy link
Member

binarycrayon commented Dec 5, 2020 via email

@binarycrayon
Copy link
Member

@samuelamar15 I just released argo-python-dsl 0.4.0 -- also included an example to submit with dsl https://github.com/argoproj-labs/argo-python-dsl#submitting-with-dsl

Would love to know if this works for you

@samuelamar15
Copy link

@binarycrayon seems like all is working well now 👍

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@binarycrayon binarycrayon

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@binarycrayon @sphadley @abaiju15 @samuelamar15