[roslaunch] roslaunch-check only considers "default" path

[miguelprada]

When conditionally including launch files based on input arguments, only the includes that are pulled in for the default values of the input arguments are considered by roslaunch-check.

The issue can be reproduced with the following launchfile which should, in my opinion, fail:

<launch>
  <arg name="fail" default="false"/>
  <include if="$(arg fail)" file="does/not/exist.launch"/>
</launch>

I know this directly contradicts #993/#998. I agree that in the simple example given in #993, it is not reasonable for roslaunch-check to fail, but in an example such as the one above, I would expect all possible paths to be tested.

I am aware that the possible paths is not something which can be dynamically computed, particularly since one can have arbitrary $(eval) substitution args which depend on arbitrary input arguments. Therefore I can't think of any way to properly cover the possible cases other than evaluating everything irrespectively of the conditional.

One alternative could be that we add test-only launch files to our packages, which do exercise all possible variants of the input arguments. However, the number of required test launch files [1] grows exponentially with the number of input arguments of the original launch file, and quickly becomes unmanageable.

[1] Instead of using multiple test launch files, one could think of using a single test launch file, which would include multiple copies of the launch file to be tested with different sets of input arguments but I found a separate issue about this which I'm about to post.

[mikepurvis]

One alternative could be that we add test-only launch files to our packages, which do exercise all possible variants of the input arguments. However, the number of required test launch files [1] grows exponentially with the number of input arguments of the original launch file, and quickly becomes unmanageable.

IMO this is a reasonable approach to testing multiple possible configurations of a given launch file.

[miguelprada]

One alternative could be that we add test-only launch files to our packages, which do exercise all possible variants of the input arguments. However, the number of required test launch files [1] grows exponentially with the number of input arguments of the original launch file, and quickly becomes unmanageable.

IMO this is a reasonable approach to testing multiple possible configurations of a given launch file.

You may well be right, but I was certainly surprised when I found an example like the one above not failing the tests (it took a different package including that launch file with fail set to true to trigger a test failure).

What would you think of adding some --ignore-conditionals-like input flag to roslaunch-check (and exposing it somehow in roslaunch_add_file_check)?

[130s]

What would you think of adding some --ignore-conditionals-like input flag to roslaunch-check (and exposing it somehow in roslaunch_add_file_check)?

That might be reasonably practical given the complication discussed above.

@collin-scribner just hit similar/the same issue while tackling #953