Using variables in recource paths in config

[mengerj]

Hello viash people,

I am just starting to work with viash and i am trying to figure out if i can use a variable inside a path definition for resources in the config file.
I have included the config file. I would like to have something like "path: '/data/cellxgene/{data_id}/obs_filters.json'" and use the component on different data_ids. If this kind of thing is not possible, I would greatly appreciate an idea on how to get the correct file into the recource directory so I can access it in the component.

Thanks a lot!
Jonatan

name: load_census_adata
description: uses a premade config file to load a census dataset
arguments:
  - name: "--data_id"
    type: string
    required: true
resources:
  - type: python_script
    path: script.py
  - type: file
    path: /configs/census_config.yaml
  - type: python_script
    path: /py/data_helpers.py
  - type: file
    path: '/data/cellxgene/{data_id}/obs_filters.json'
engines:
  - type: native
  - type: docker
    image: "python:3.10-slim"
    setup:
      - type: python
        pip: ["pyyaml","anndata","requests","cellxgene-census"]

runners:
  - type: executable
  - type: nextflow

[rcannood]

Hi @mengerj !

It looks like you're trying to use a runtime parameter to configure which resources are used during build time, which isn't feasible. The data_id parameter can be used during runtime for accessing different resources, but it can't change the build-time configuration paths.

Possible solutions:

• Add obs_filters.json as an argument to the config:

  arguments:
    - name: --obs_filters
      type: file
      required: true
      example: path/to/obs_filters.json

• Add the whole /data/cellxgene as a resource and use --data_id to select which obs_filters to use. Note that this is very inefficient if cellxgene is large.

  arguments:
    - name: --data_id
      type: file
  resources:
    - type: python_script
      path: script.py
    - path: /data/cellxgene

  This would allow you to access obs_filters.json in your script.py using:

  obs_filters_file = `f"{meta['resources_dir']}/cellxgene/{par['data_id']/obs_filters.json"`

Hope this helps!