Consider Linux packages / binaries.

[coryan]

Consider creating binary packages for Linux. We can host them in GCS or bintray. We can have CMake create them, but it might be better to manually create them to spell out the build dependencies and install dependencies.

At least the following should be considered, and if rejected say why:

• Docker image with all dependencies pre-installed
• tar.gz: for Ubuntu, Fedora, and CentOS
• Linuxbrew: https://linuxbrew.sh
• Conan: https://conan.io
• Hunter: https://github.com/ruslo/hunter
• cget: http://cget.readthedocs.io/en/latest/
• .rpm: http://rpm.org/
• .deb: https://wiki.debian.org/Packaging/Intro?action=show&redirect=IntroDebianPackaging
• Spack: https://www.spack.io/
• Buckaroo: https://buckaroo.pm/

[ras0219-msft]

In case you haven't seen our announcement[1], Vcpkg now supports Linux and OSX and so may be worthy of inclusion in this list :)

We also confirmed that our google-cloud-cpp package already builds for both of these new platforms.

[1] https://blogs.msdn.microsoft.com/vcblog/2018/04/24/announcing-a-single-c-library-manager-for-linux-macos-and-windows-vcpkg/

[coryan]

@sduskis I am moving this out of the current milestones per the meeting on Friday.

[coryan]

For the record. We are looking for customer feedback here, if anybody has suggestions or opinions on what packaging solutions we should adopt please let us know.

[ghost]

I've started work on some RPMs for crc32c and google-cloud-cpp. You can download them here:

https://copr.fedorainfracloud.org/coprs/remyabel/google-cloud-cpp/

They have built successfully so please feel free to test them out.

For tarballs, we can follow a similar approach to Bazel. They use github-release which is a CLI tool for uploading artifacts to the Github API. We may also need to use git-archive-all to solve the issue mentioned in #2246.

[ghost]

The RPM's are going to be put on hold for now. It fails due to this:

CMake Error at cmake/IncludeProtobuf.cmake:51 (find_package):
  By not providing "Findprotobuf.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "protobuf",
  but CMake did not find one.

  Could not find a package configuration file provided by "protobuf" with any
  of the following names:

    protobufConfig.cmake
    protobuf-config.cmake

  Add the installation prefix of "protobuf" to CMAKE_PREFIX_PATH or set
  "protobuf_DIR" to a directory containing one of the above files.  If
  "protobuf" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  cmake/IncludeGrpc.cmake:53 (include)
  CMakeLists.txt:151 (include)

When using -DGOOGLE_CLOUD_CPP_PROTOBUF_PROVIDER=package it fails at RPM build time.

[coryan]

Doh! That is because the module is FindProtobuf, but protobuf installs protobufConfig. I think we should use the module. Note that I am already working on changes like this for #2234 and company.

[ghost]

What do you prefer: keeping the spec in a separate repo or keeping it part of the repo and using something like https://github.com/dgoodwin/tito? The workflow is:

• tito init once
• For a new release, tito tag or tito tag --keep-version if the version in the spec is manually updated. This updates the spec's changelog and creates a new git tag
• Automatically build the RPM on git push (more realistically from the master branch or version tags), or manually with tito build --srpm and upload the SRPM to be built on whatever platform (such as COPR)

[coryan]

What do you prefer: keeping the spec in a separate repo or keeping it part of the repo and using something like https://github.com/dgoodwin/tito?

I do not think we have made up our minds about how we want to support packaging (that is why this bug says "consider ..."). I think we generally have agreement on this (@devjgm to correct me if this is not the case):

• We would not be distributing binary packages ourselves.
• We want to make it possible (I would say easy) for package maintainers to create packages so they can distribute binary packages.
• If somebody contributes packaging files for package system X we would probably be happy to add them to the repository.
• We will not be creating packaging files for package system X, or Y, or Z.

So in that context, I think contributing the files into the repository is fine, but we cannot make the builds depend on them.

On tito: we do not want to make the .rpm packages "privileged" over vcpkg, or .deb, or the other dozen systems. Happy to get the .spec files, but I think we should not make one package manager much easier than the others.

[ghost]

Generally I agree with everything you said. Package maintainers should be able to create packages independent of upstream cooperation (sans patches or blockers that cannot be resolved downstream). I suggested tito because it (and rpkg) have the strange property that they need to be run from the git sources repository. tito takes it a step further and requires committing a .tito folder and running tito tag to generate a new release.

Traditionally creating an RPM, i.e using rpm-build is done from tarball sources. Since the submodule problem is fixed, keeping the spec file in a separate repository and hosting the binary RPM in COPR is probably the idiomatic way of doing things.

[devjgm]

I probably have more questions on this topic than answers. I basically agree with what @coryan said 4 hours ago, with one comment: the third bullet is a little surprising to me. I mean, we will not be distributing or maintaining binary packages ourselves (agreed). It's fine if other binary package maintainers do this for our project though. Indeed, we'll even do our best to make our repo amenable to being easily packaged in a wide variety of ways for those who do want to make binary packages. However, I was not expecting us to endorse or even accept packaging files for any binary packaging system. I agree that we don't want our builds to depend on any particular binary packaging system, and therefore, I'm not sure if we want our repo to host files that we don't really own/maintain/test. Can all the binary packaging files live in some other repo instead?

[ghost]

Can all the binary packaging files live in some other repo instead?

Well, that was my fault. I was enthusiastically exploring various options for packaging and wanted to contribute. This led to the following chain of events:

• can't build tarball because the submodule contents aren't included
• can't build RPM with COPR's preferred tools because they don't handle submodules
• I throw the idea of using tito at the wall for reasons stated above

But all of these issues have been resolved and I agree it's best practice for the files to exist elsewhere (and I actually find it a pet peeve when upstream developers try to do packaging, because it makes downstream maintainer's lives harder.)

[coryan]

Good points @devjgm and @remyabel. You both sold me on not keeping packaging files in this repository. @devjgm do you want to take a stab at writing an ADR?

[devjgm]

Yep. I'll do that.