How to set environment variables or pass structure data when submitting job?

[0xshawn]

Organization Name: Private Test
Short summary about the issue/question:
How to set environment variables or pass structure data when submitting job?

I used to use jobEnv to set structure like:

  "jobEnvs": {
    "TASK_NAME": "the_task",
    "USER_NAME": "someone",
    "JOB_TYPE": [
      "train",
      "test"
    ],
    "trainSets": [
      "xxx", "yyy"
    ],
    "extra_config": {
      "ROOT_DIR": "/opt/outputs",
      "TRAIN": {
        "MAX_ITER": 1
      }
    }
  }

But I can not find any doc or code to submit with jobEnv in v0.14.0.

OpenPAI Environment:

• OpenPAI version: v0.14.0

Thanks!

[abuccts]

if you want to use v2 api, pls refer to parameters in protocol spec

[0xshawn]

@abuccts Thanks for the help!

It seems that parameters can be accessed in YAML value when submitting job.

But the trouble I met is that I wanna access the value inside the running job or container.

[abuccts]

You can either add arguments for your code to accept those parameters (recommended),

main.py --root-dir <% $parameters.root_dir %> --max-iter <% $parameters.max_iter %>

or use a JSON string (e.g. json.dumps in py) as a parameter and export to env/write to file by yourself.

parameters:
  env: '{"jobEnvs": {"TASK_NAME": "the_task", "USER_NAME": "someone", "JOB_TYPE": ["train", "test"], "trainSets": ["xxx", "yyy"], "extra_config": {"ROOT_DIR": "/opt/outputs", "TRAIN": {"MAX_ITER": 1}}}}'
...
commands:
- export CONFIG=<% $parameters.env %>

[0xshawn]

@abuccts , So much for the help, and it helps a lot.

And could you please add a new feature when submitting new job(v2), which may directly set the environment. I love jobEnvs in v1.

Thanks again!

[mydmdm]

we may do it via a kube runtime plugin, which could export such parameters to env