catkin_make barfs on non-ascii characters

[jonbinney]

When running cmake causes a non-ascii character to be output to stdout, catkin_make crashes.

This problem seems to have been reintroduced with this commit:
22ce2ba

On line 171: 22ce2ba#L1L171 the default unicode encoding seems to be 'ascii', which means it will barf if cmake outputs any non-ascii characters. The resulting exception isn't caught and crashes catkin_make with an exception:

'ascii' codec can't decode byte 0xe2 in position 33: ordinal not in range(128)

The fix would be to revert that line to:

proc.stdout.readline().decode('utf8', 'replace')

so that it can handle non-ascii characters.

[jonbinney]

To reproduce, create a basic catkin package in a new workspace:

catkin_create_pkg foo --rosdistro hydro

add the following line to the cmakelists:

execute_process(COMMAND printf '\\xe2')

then run catkin_make

[jonbinney]

Interestingly, it works fine if you use the --no-color option. Also works fine on groovy, which is confusing since it appears that the same version of catkin is released on both groovy and hydro.

[jonbinney]

To be clear, I'm only seeing this issue on hydro

[dirk-thomas]

Which versions are you using on Groovy and Hydro?

[jonbinney]

ii  python-catkin-pkg                           0.1.18-1                                            catkin package library
ii  ros-groovy-catkin                           0.5.70-0precise-20130714-0827-+0000                 Low-level build system macros and infrastructure for ROS.
ii  ros-hydro-catkin                            0.5.70-0precise-20130714-0827-+0000                 Low-level build system macros and infrastructure for ROS.

[dirk-thomas]

Do you have a custom catkin version in one of your workspace (groovy or hydro)?

[jonbinney]

Nope. I made an empty workspace (following the instructions i posted above) and saw the same error. The only non-debian package in my workspace was foo, the one i created. In that same workspace, i've double checked that it is using the right catkin python code:

In [1]: from catkin import builder

In [2]: builder.__file__
Out[2]: '/opt/ros/hydro/lib/python2.7/dist-packages/catkin/builder.pyc'

In [3]: !dpkg -S /opt/ros/hydro/lib/python2.7/dist-packages/catkin/builder.pyc
ros-hydro-catkin: /opt/ros/hydro/lib/python2.7/dist-packages/catkin/builder.pyc

[dirk-thomas]

Well, but something must be different. Is the package which is triggering the exception in both workspace and also equal or is the Groovy and Hydro version of the package different?

[tfoote]

@wjwwood

[wjwwood]

@wjwwood

[jonbinney]

Its the package which i describe above (in second comment on this issue). It's a completely new catkin package, which i've added exactly one line to the CMakeLists.txt

[dirk-thomas]

I will commit a fix later - the bug in the code is obvious.

But I would still like to know who you get different behavior while running the same version. If you do a pure cmake invocation on both workspaces do you see the special character in both cases in the console output?

[jonbinney]

OK, actually i realized that i do see the same issue on both groovy and hydro. The reason i didn't see it on groovy before is that i forgot to remove the build and devel directories, and so it wasn't actually re-running cmake; just make.

[dirk-thomas]

Thanks for clarifying.

The fix is checked in but don't expect a release very soon since catkin has just been released and the effort of rebuilding stuff is pretty high. Until then you can just checkout catkin into your workspace which has the issue.

[jonbinney]

Thanks!