nf-core modules lint breaks when using stageAs: or folder in the input line

[mahesh-panchal]

Description of the bug

nf-core modules lint breaks when using stageAs: or a folder name in the input line
https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true

Command used and terminal output

nf-core modules lint busco

Can reproduce on this pull request: https://gitpod.io/#https://github.com/nf-core/modules/pull/1239

System information

No response

[sateeshperi]

related slack discussion

[mahesh-panchal]

Closing as the error was elsewhere: See: nf-core/modules#1239 (comment)

[ewels]

I don't quite see why this should be closed based on that comment? Tools should never trigger a big unhandled crash like that.

Please paste the error log in the future, I can't find the original CI job that triggered this. But the link from slack points to this traceback:

,--./,-.
          ___     __   __   __   ___     /,-._.--~\
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

    nf-core/tools version 2.4.dev0 - https://nf-co.re


INFO     Linting modules repo: '.'                               __init__.py:15[9](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:9)
INFO     Linting module: 'busco'                                 __init__.py:163

╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /opt/hostedtoolcache/Python/3.[10](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:10).4/x64/bin/nf-core:8 in <module>             │
│                                                                              │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])     │
│ ❱ 8 │   sys.exit(run_nf_core())                                              │
│   9                                                                          │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ __main__.py:89 in run_nf_core                                                │
│                                                                              │
│    88 │   # Lanch the click cli                                              │
│ ❱  89 │   nf_core_cli()                                                      │
│    90                                                                        │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/co │
│ re.py:[11](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:11)30 in __call__                                                       │
│                                                                              │
│   1[12](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:12)9 │   │   """Alias for :meth:`main`."""                                 │
│ ❱ 1[13](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:13)0 │   │   return self.main(*args, **kwargs)                             │
│   1131                                                                       │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/rich_cli │
│ ck/rich_group.py:21 in main                                                  │
│                                                                              │
│   20 │   │   try:                                                            │
│ ❱ 21 │   │   │   return super().main(*args, standalone_mode=False, **kwargs) │
│   22 │   │   except click.ClickException as e:                               │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/co │
│ re.py:1055 in main                                                           │
│                                                                              │
│   1054 │   │   │   │   with self.make_context(prog_name, args, **extra) as c │
│ ❱ 1055 │   │   │   │   │   rv = self.invoke(ctx)                             │
│   1056 │   │   │   │   │   if not standalone_mode:                           │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/co │
│ re.py:1657 in invoke                                                         │
│                                                                              │
│   1656 │   │   │   │   with sub_ctx:                                         │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub │
│   1658                                                                       │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/co │
│ re.py:1657 in invoke                                                         │
│                                                                              │
│   1656 │   │   │   │   with sub_ctx:                                         │
│ ❱ 1657 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub │
│   1658                                                                       │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/co │
│ re.py:[14](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:14)04 in invoke                                                         │
│                                                                              │
│   1403 │   │   if self.callback is not None:                                 │
│ ❱ 1404 │   │   │   return ctx.invoke(self.callback, **ctx.params)            │
│   1405                                                                       │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/co │
│ re.py:760 in invoke                                                          │
│                                                                              │
│    759 │   │   │   with ctx:                                                 │
│ ❱  760 │   │   │   │   return __callback(*args, **kwargs)                    │
│    761                                                                       │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/click/de │
│ corators.py:26 in new_func                                                   │
│                                                                              │
│    25 │   def new_func(*args, **kwargs):  # type: ignore                     │
│ ❱  26 │   │   return f(get_current_context(), *args, **kwargs)               │
│    27                                                                        │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ __main__.py:610 in lint                                                      │
│                                                                              │
│   609 │   │   module_lint.modules_repo = ctx.obj["modules_repo_obj"]         │
│ ❱ 610 │   │   module_lint.lint(module=tool, key=key, all_modules=all, print_ │
│       local=local, show_passed=passed)                                       │
│   611 │   │   if len(module_lint.failed) > 0:                                │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ modules/lint/__init__.py:180 in lint                                         │
│                                                                              │
│   [17](https://github.com/nf-core/modules/runs/6308115663?check_suite_focus=true#step:9:17)9 │   │   if len(nfcore_modules) > 0:                                    │
│ ❱ 180 │   │   │   self.lint_modules(nfcore_modules, local=False)             │
│   181                                                                        │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ modules/lint/__init__.py:306 in lint_modules                                 │
│                                                                              │
│   305 │   │   │   │   progress_bar.update(lint_progress, advance=1, test_nam │
│ ❱ 306 │   │   │   │   self.lint_module(mod, local=local)                     │
│   307                                                                        │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ modules/lint/__init__.py:334 in lint_module                                  │
│                                                                              │
│   333 │   │   │   for test_name in self.lint_tests:                          │
│ ❱ 334 │   │   │   │   getattr(self, test_name)(mod)                          │
│   335                                                                        │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ modules/lint/main_nf.py:80 in main_nf                                        │
│                                                                              │
│    79 │   │   if state == "input" and not _is_empty(module, l):              │
│ ❱  80 │   │   │   inputs += _parse_input(module, l)                          │
│    81 │   │   if state == "output" and not _is_empty(module, l):             │
│                                                                              │
│ /opt/hostedtoolcache/Python/3.10.4/x64/lib/python3.10/site-packages/nf_core/ │
│ modules/lint/main_nf.py:277 in _parse_input                                  │
│                                                                              │
│   276 │   │   │   if elem:                                                   │
│ ❱ 277 │   │   │   │   elem = elem.split("(")[1]                              │
│   278 │   │   │   │   elem = elem.split(",")[0].strip()                      │
╰──────────────────────────────────────────────────────────────────────────────╯
IndexError: list index out of range
Error: Process completed with exit code 1.

[ewels]

Mostly reopening as I want to refactor the underlying code that this error came from, so this is a good reminder for that.

[ewels]

x-ref comment here: #1374 (comment)

[mirpedrol]

Hi @mahesh-panchal @ewels, I was looking at this bug but it seems it's already solved. The only missing task would be to get singularity and docker tags by regex in nf_core/modules/lint/main_nf.py? Are you working on it @ewels or should I address it?

[mahesh-panchal]

Sorry, the issue is something else and I haven't had time today to generating what the actual issue is. I was misinformed when I wrote this, and then Harshil showed the issue was to do with the comment instead that was breaking linting.

[mahesh-panchal]

The original line that was causing the error was:
tuple val(meta), path('tmp_input/*') // Required: meta map, and fasta sequence files
and removing the comment stops linting breaking.

[mahesh-panchal]

There's also another one to do with using single or double quotes in the comment, but I haven't tried to replicate yet.
Single quoting breaks linting, but double quoting works ok. I'll try and replicate later though.

[ewels]

Are you working on it @ewels or should I address it?

Please go ahead 👍🏻 Basically wherever there is complex split commands relying on list indices it would probably be more robust to use a regex..

[ewels]

@mahesh-panchal are you able to try to replicate again? Just merged a refactor of some of this code in #1552 but I'm not 100% clear on what the original error in this issue was, so not sure if we can close or not..

[ewels]

Ok, after a bit of digging it seems that this issue is not resolved yet.

If I dig back in the git history of nf-core/modules to this commit of the busco module: https://github.com/priyanka-surana/modules/blob/012f08537880805ad70a006f0076e6f8933b860f/modules/busco/main.nf

I can still trigger an unhandled traceback from nf-core modules linting. Working on this now.

[mahesh-panchal]

Yes. Still errors for me too.

tuple val(meta), path('tmp_input/*') // Required: meta map, and fasta sequence files

That was the line causing issues in the busco module

[ewels]

@mahesh-panchal should be fixed in #1571 👍🏻