Unexpected installation dependency on source location

[jensenb]

While debugging the issue discussed here http://answers.ros.org/question/105771/rqt_reconfigure-runtime-error-osx/ with @MrTatsch we discovered that the installed package has an unexpected dependency on the source directory. It seems that during the python package installation, $INSTALL_DIR/dynamic_reconfigure/__init__.py gets generated with a reference to the current source directory, so that when the installed package is used, the source directory is read.

This is very counter intuitive behaviour. If the source directory is deleted or renamed after installation, as was the case in the ROS Answers problem, then the installed versions of dynamic reconfigure ceases to function.

[jensenb]

So i did some more digging, and it turns out that the __init__.py.in template file is copied from workspace's devel_isolated folder during package installation, instead of dynamic_reconfigure's own package __init__.py file. No idea why this is happening.

[dirk-thomas]

The official Debian package contains the correct __init__.py file (https://github.com/ros/dynamic_reconfigure/blob/master/src/dynamic_reconfigure/__init__.py). So does any local build I tried.

How did you install the package?

[jensenb]

I am installing hydro from source on OS X. I also noticed that this was not the only package with this problem, but most packages with an existing python package did not exhibit this problem.

To make things more interesting I noticed the build_isolated folder contained correct `__init__py, the devel_isolated contained this expanded template, and install_isolated somehow needed up with the expanded template and not the package's owninit.py``. I have not yet understand how catkin processes and installs python packages.

[dirk-thomas]

I haven't seen the issue you describe before and can't imagine why this is happening for you. Please describe the exact setup what you have checked out and all steps to reproduce your scenario. Copying @wjwwood on that since he might reproduce your setup on OS X.

[wjwwood]

This is also the case for me:

% cat hydro/install_isolated/lib/python2.7/site-packages/dynamic_reconfigure/__init__.py
# generated from catkin/cmake/template/__init__.py.in
# keep symbol table as clean as possible by deleting all unnecessary symbols

from os import path as os_path
from sys import path as sys_path

from pkgutil import extend_path

__extended_path = "/Users/william/hydro/src/dynamic_reconfigure/src".split(";")
for p in reversed(__extended_path):
    sys_path.insert(0, p)
    del p
del sys_path

__path__ = extend_path(__path__, __name__)
del extend_path

__execfiles = []
for p in __extended_path:
    src_init_file = os_path.join(p, __name__ + '.py')
    if os_path.isfile(src_init_file):
        __execfiles.append(src_init_file)
    else:
        src_init_file = os_path.join(p, __name__, '__init__.py')
        if os_path.isfile(src_init_file):
            __execfiles.append(src_init_file)
    del src_init_file
    del p
del os_path
del __extended_path

for __execfile in __execfiles:
    with open(__execfile, 'r') as __fh:
        exec(__fh.read())
    del __fh
    del __execfile
del __execfiles

[wjwwood]

This is also the case for a few other packages:

% grep -R __extend hydro/install_isolated
hydro/install_isolated/lib/python2.7/site-packages/actionlib/__init__.py:__extended_path = "/Users/william/hydro/src/actionlib/src".split(";")
hydro/install_isolated/lib/python2.7/site-packages/actionlib/__init__.py:for p in reversed(__extended_path):
hydro/install_isolated/lib/python2.7/site-packages/actionlib/__init__.py:for p in __extended_path:
hydro/install_isolated/lib/python2.7/site-packages/actionlib/__init__.py:del __extended_path
Binary file hydro/install_isolated/lib/python2.7/site-packages/actionlib/__init__.pyc matches
hydro/install_isolated/lib/python2.7/site-packages/dynamic_reconfigure/__init__.py:__extended_path = "/Users/william/hydro/src/dynamic_reconfigure/src".split(";")
hydro/install_isolated/lib/python2.7/site-packages/dynamic_reconfigure/__init__.py:for p in reversed(__extended_path):
hydro/install_isolated/lib/python2.7/site-packages/dynamic_reconfigure/__init__.py:for p in __extended_path:
hydro/install_isolated/lib/python2.7/site-packages/dynamic_reconfigure/__init__.py:del __extended_path
Binary file hydro/install_isolated/lib/python2.7/site-packages/dynamic_reconfigure/__init__.pyc matches
hydro/install_isolated/lib/python2.7/site-packages/sensor_msgs/__init__.py:__extended_path = "/Users/william/hydro/src/sensor_msgs/src".split(";")
hydro/install_isolated/lib/python2.7/site-packages/sensor_msgs/__init__.py:for p in reversed(__extended_path):
hydro/install_isolated/lib/python2.7/site-packages/sensor_msgs/__init__.py:for p in __extended_path:
hydro/install_isolated/lib/python2.7/site-packages/sensor_msgs/__init__.py:del __extended_path
Binary file hydro/install_isolated/lib/python2.7/site-packages/sensor_msgs/__init__.pyc matches
hydro/install_isolated/lib/python2.7/site-packages/tf/__init__.py:__extended_path = "/Users/william/hydro/src/tf/src".split(";")
hydro/install_isolated/lib/python2.7/site-packages/tf/__init__.py:for p in reversed(__extended_path):
hydro/install_isolated/lib/python2.7/site-packages/tf/__init__.py:for p in __extended_path:
hydro/install_isolated/lib/python2.7/site-packages/tf/__init__.py:del __extended_path
Binary file hydro/install_isolated/lib/python2.7/site-packages/tf/__init__.pyc matches
hydro/install_isolated/share/catkin/cmake/templates/__init__.py.in:__extended_path = "@PACKAGE_PYTHONPATH@".split(";")
hydro/install_isolated/share/catkin/cmake/templates/__init__.py.in:for p in reversed(__extended_path):
hydro/install_isolated/share/catkin/cmake/templates/__init__.py.in:for p in __extended_path:
hydro/install_isolated/share/catkin/cmake/templates/__init__.py.in:del __extended_path

But definitely not all of them.

[wjwwood]

% cmake --version
cmake version 2.8.11.2

[jensenb]

yeah its strange, I have not understood why some packages work fine and others do not, even though they appear to have near identical CMake and distutils settings. Actually I never would have noticed because everything still works seamlessly, as long as the originating source folder remains intact. While experimenting @MrTatsch changed his source folder, and noticed that dynamic reconfigure would not start. Only after we investigated for while did we discover what the cause was.

For what its worth im using cake 2.8.12.1 on 10.9.

[dirk-thomas]

I am able to reproduce this on Ubuntu with catkin_make_isolated --install. I will look into it and then likely move the ticket to a different repo.

[dirk-thomas]

Closing this in favor of new ticket against genmsg.