Fix runfiles_path in Linux for Loop Opt Service

[mostafaelhoushi]

Possible Solution:

Another problem may be that the file does not get where it should be. This means that something is wrong with the dependencies and the target that makes the files is not triggered. There may be a bug in cg_add_data_dependencies(...).
Another problem may come from cg_genrule resolving to the wrong command, but this should generate an error when executed.
@mostafaelhoushi, you can run the CMake configuration with --trace-expand flag and see what happens line-by-line.
You can also use CMake's --graphviz option to visualize the dependency graph.

Originally posted by @sogartar in #570 (comment)

[ChrisCummins]

I can try and take a look at this if you want? How do I repro?

[mostafaelhoushi]

Thanks @ChrisCummins !
To reproduce , in
https://github.com/facebookresearch/CompilerGym/blob/development/examples/loop_optimizations_service/service_py/loops_opt_service.py#L217 simply replace:

os.path.join(os.path.dirname(__file__), "../opt_loops/opt_loops")

with:

runfiles_path("examples/loop_optimizations_service/opt_loops/opt_loops")

[mostafaelhoushi]

Please note that in this PR #608 to fix other errors in Linux CMake, I had to change 2 further runfiles_path(...) to os.path.join(os.path.direname(__file__), ...)

[mostafaelhoushi]

I would like to clarify that CMake does actually build opt_loops and you will find it at <build directory>/examples/loop_optimizations_service/opt_loops/opt_loops. And CMake also build loops_opt_service.py as an executable file at <build directory>/examples/loop_optimizations_service/service_py.

However, it seems that runfiles_path doesn't look at the build directory. It instead looks at the installation directory of the compiler-gym Python package (i.e. that is created after we call pip install the .whl file). That Python package installation directory contains neither the opt_loops binary nor the loops_opt_service.py Python.

The cause could be that we need to ensure that we add the example.py script as py_binary target in CMakeLists.txt, which would ensure that both loops_opt_service.py and opt_loops are added to the .whl file output?

Cc @sogartar

[ChrisCummins]

Ahhh, I see now, sorry this took me so long to figure out! 🙂

So, the problem is that runfiles_path() will only work for files that installed as part of the core compiler_gym package. Since the examples/ directory is not installed anywhere, runfiles_path() can't be used.

This is working "as expected", but it means that you need an alternate way to locate runtime dependencies for example code. This is entirely why "demo_without_bazel.py" only works for the python backend and not for the compiler C++ backend.

Once we complete the cmake migration we can replace the complicated runfiles_path implementation with importlib.resources.path. Filed #616 to track.

Cheers,
Chris