homebrew: rosdep assumes location of desired packages

[ahundt]

I'm dealing with breakage on OS X and homebrew administrative issues. To work around them I need to use my own fork/tap of homebrew components, however it seems rosdep assumes specific versions and can't deal when more than one version of a specific package is available.

The original problem with a change to vtk:
mikepurvis/ros-install-osx#32

The recommendation that I maintian my own tap:
Homebrew/brew#62

Would it make sense for rosdep to include some kind of support for this situation? I believe there needs to be something so that ROS on OS X can be as stable as possible. Thanks!

[wjwwood]

Yeah, possibly. Unfortunately, it looks like many of the rosdep rules encapsulate the tap, i.e. homebrew/science/vtk rather than just vtk. I think this was recommended to use at some point by the Homebrew people. You could create custom rosdep rules locally which override our rosdep rules and point to your fork instead.

@NikolausDemmel do you have any ideas for how to make this easier?

[tfoote]

I think that the fully qualified names are correct as the default as you don't want to accidentally have different systems depending on potentially different versions of the package which could lead to potentially unreasonable dependency conflicts that are not detectable apriori.

The custom rosdep rules are a way to override the specific rosdep rule for the case that you want to use your fork instead of the default one is a reasonable solution.

[ahundt]

Well the problem is that currently the installation process fails because of dependency conflicts in the parent homebrew! I've manually gotten things to work now for myself, but the maintainer of homebrew-science isn't yet willing to support older versions of vtk for pcl... so the situation is likely dire for anyone trying to install ROS on OS X without extensive package maintenance / homebrew / general fixing skills...

[wjwwood]

I agree it's not a good user experience, but I'm not sure how to improve the situation. It seems to me it would be rosdep providing functionality to workaround temporary issue in the upstream packaging.

I'm not opposed to adding some functionality to help ease the issue in the meantime, but it's not clear to me how it should change or what the tool would do. I don't think it's the right thing to remove the explicit user/tap_name/dep name in favor of just dep, but I might be wrong about that. I seem to remember some benefits to using the fully qualified name, like brew automatically tapping when doing dependency resolution (but that may have changed, as Homebrew tends to change quickly).

Rather than saying rosdep keys (or rosdep itself) is assuming something about upstream dependencies by using the fully qualified name, I would say that it is explicit on purpose. Now, it maybe that this strategy should be relaxed based on the use case or new recommendations from the Homebrew community, but neither is clear to me at the moment. I could be convinced otherwise, but I have seen any concrete suggest or any argue for what should change and why.

Can you suggest what a better workflow would be? (in terms of new options or behaviors from rosdep)

@mikepurvis do you have any input on this? Should we just change the rosdep keys to point to a patched version of things? We've done that in the past: https://github.com/ros/homebrew-deps But the risk is always that those formula get out of date...

[mikepurvis]

I'm curious to know how other rolling release systems like Arch deal with this kind of thing. Homebrew is always on the bleeding edge of everything, which basically guarantees some level of brokenness, at least as far as new major versions with breaking changes. That said, a disproportionate number of issues seem to come up around bottled versions of things with baked-in paths to their dependencies, that then go stale when those depended-upon packages move on— we've seen this with Python, VTK, PCL, CMake, and probably others. It feels like there's a fundamental issue there with Homebrew, that it should have a dependency type for "rebottle when this dependency updates".

In any case, if upstream Homebrew can't commit to a satisfactory level of stability across the rather large base of packages upon which we depend, then I don't see much choice other than to maintain at least some of the system in taps. Random assorted thoughts on this direction:

• Unless we are also committed to bottling stuff, the cost to users of building big dependencies like VTK and PCL locally is pretty big— my understanding is that bottling is not hard; you just have to set it up.
• Certain dependencies (CMake, Python) are not realistic to maintain in taps due to the weirdness of a system with multiple versions of these things.
• It may be possible to build some light Travis-based tooling to provide notification of upstream formulae changes, so we can hopefully mitigate a bit the issue of stuff falling behind.
• Maintaining any of this will be nightmare without regular CI builds across the supported OS X versions. I know OSRF had expressed interest last year in doing some of this around ros-install-osx, but it didn't seem to go anywhere.

[ahundt]

I've started the ball rolling and the homebrew people are at least thinking about some of these issues, and they are planning to address at least some aspects. To help get a good result for our use cases I suggest sending them feedback while the getting's hot!

Homebrew/brew#60

[wjwwood]

It seems there has been some movement upstream (thanks to @ahundt for getting things going over there). I don't think there is anything left for rosdep to do though, so I'm going to close this.

I can reopen if needed.