Find python path properly for installed underlays.

[stonier]

This is the only change I need to work with underlays that are completely defined by CMAKE_PREFIX_PATH in an initial cache rather than environment variables persistently set by sourcing underlay setup.xxx scripts every time you cmake/make.

See parallel email thread in ros-sig-buildsystem. I think it would be very convenient (if for us, then I'm sure others as well) to support this and it doesn't break support of the former manual sourcing style.

[dirk-thomas]

@wjwwood Can you think about side effects of such a change?

[wjwwood]

As long as he iterates in the correct order, I see no problem with adding these to the Python path, either:

• The dir doesn't exist
• The dir doesn't contain anything being imported
• The dir contains something being imported
• The dir contains something being imported, which also exists in another workspace

The last point is the only one of concern, so as long as the workspace order is correct then it should be fine.

How does this method handle the environment hooks? Are they needed for build time?

[stonier]

I haven't had any need for the environment hooks with our build scripts yet. I suppose there could be a need sometime in the future, but maybe just worry about that if at some point it's needed.

[wjwwood]

@stonier the environment hooks sets the ROS_PACKAGE_PATH, ROS_ROOT, and ROS_DISTRO environment variables (among other things). Those are important if you are using the ROS tools at a later point.

[stonier]

Good point. I'll check out what's going on in there.

[stonier]

Some results from testing:

All generate_cached_setup.py does is 1) find catkin's python module, and then compare the difference between current environment (in my case, almost blank) and the result of devel/env.sh. It doesn't actually have to go looking for environment hooks and variables and stuff - they're already setup to go in devel/env.sh. All I need to do here is help the script find catkin's python module.

generate_setup_cached.py

from __future__ import print_function
import argparse
import os
import stat
import sys

# find the import relatively if available to work before installing catkin or overlaying installed version
if os.path.exists(os.path.join('/opt/ros/groovy/share/catkin/cmake', 'catkinConfig.cmake.in')):
    sys.path.insert(0, os.path.join('/opt/ros/groovy/share/catkin/cmake', '..', 'python'))
########################### New Code ######################################
for workspace in reversed("/opt/rocon/devel;/opt/ros/groovy".split(';')):
    sys.path.insert(0, os.path.join(workspace, 'lib/python2.7/dist-packages'))
##########################################################################
from catkin.environment_cache import generate_environment_script

code = generate_environment_script('/opt/rocon/devel/env.sh')
output_filename = '/opt/rocon/build/catkin_generated/setup_cached.sh'
with open(output_filename, 'w') as f:
    #print('Generate script for cached setup "%s"' % output_filename)
    f.write('\n'.join(code))

mode = os.stat(output_filename).st_mode
os.chmod(output_filename, mode | stat.S_IXUSR)

setup_cached.sh

#!/usr/bin/env sh
# generated from catkin/python/catkin/environment_cache.py

# based on a snapshot of the environment before and after calling the setup script
# it emulates the modifications of the setup script without recurring computations

# new environment variables
export CATKIN_TEST_RESULTS_DIR="/opt/rocon/build/test_results"
export CMAKE_PREFIX_PATH="/opt/rocon/devel:/opt/ros/groovy"
export CPATH="/opt/rocon/devel/include:/opt/ros/groovy/include"
export LD_LIBRARY_PATH="/opt/rocon/devel/lib:/opt/ros/groovy/lib"
export PKG_CONFIG_PATH="/opt/rocon/devel/lib/pkgconfig:/opt/ros/groovy/lib/pkgconfig"
export ROS_DISTRO="groovy"
export ROS_ETC_DIR="/opt/ros/groovy/etc/ros"
export ROS_MASTER_URI="http://localhost:11311"
export ROS_PACKAGE_PATH="/opt/rocon/src:/opt/ros/groovy/share:/opt/ros/groovy/stacks"
export ROS_ROOT="/opt/ros/groovy/share/ros"
export ROS_TEST_RESULTS_DIR="/opt/rocon/build/test_results"

# modified environment variables
export PATH="/opt/rocon/devel/bin:/opt/ros/groovy/bin:$PATH"
export PWD="/opt/rocon/build"
export PYTHONPATH="/opt/rocon/devel/lib/python2.7/dist-packages:$PYTHONPATH"

Compare that with a setup_cached.sh generated the usual way from sourcing /opt/ros/groovy/setup.sh:

#!/usr/bin/env sh
# generated from catkin/python/catkin/environment_cache.py

# based on a snapshot of the environment before and after calling the setup script
# it emulates the modifications of the setup script without recurring computations

# new environment variables
export CATKIN_TEST_RESULTS_DIR="/home/snorri/rocon/build/test_results"
export ROS_TEST_RESULTS_DIR="/home/snorri/rocon/build/test_results"

# modified environment variables
export CMAKE_PREFIX_PATH="/home/snorri/rocon/devel:$CMAKE_PREFIX_PATH"
export CPATH="/home/snorri/rocon/devel/include:$CPATH"
export LD_LIBRARY_PATH="/home/snorri/rocon/devel/lib:$LD_LIBRARY_PATH"
export PATH="/home/snorri/rocon/devel/bin:$PATH"
export PKG_CONFIG_PATH="/home/snorri/rocon/devel/lib/pkgconfig:$PKG_CONFIG_PATH"
export PWD="/home/snorri/rocon/build"
export PYTHONPATH="/home/snorri/rocon/devel/lib/python2.7/dist-packages:$PYTHONPATH"
export ROS_PACKAGE_PATH="/home/snorri/rocon/src:$ROS_PACKAGE_PATH"

The only important difference is that the former creates alot of new environment variables (naturally, since the environment was almost blank) and the latter just modifies variables that were sourced.

[stonier]

Closing this and following up in #406