Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] switch to manylinux_2_28 for Linux artifacts (fixes #5514, fixes #5589) #5580

Merged
merged 21 commits into from
Nov 21, 2022

Conversation

jameslamb
Copy link
Collaborator

@jameslamb jameslamb commented Nov 9, 2022

Fixes #5514.
Fixes #5589.
Contributes to #5390.
Contributes to #5391 (via #5396 (comment)).

Proposes switching from an Ubuntu 14.04 image to one based on pypa/manylinux_2_28 for building artifacts on Linux including:

  • Python source and built distributions (e.g. wheels)
  • lightgbmlib.jar (Java interface to LightGBM)
  • lib_lightgbm.so (C++ shared library)

And increases the floors on symbol versions for key linked libraries on Linux.

Notes for Reviewers

This PR uses images build from guolinke/lightgbm-ci-docker#27. That lightgbm-ci-docker PR should be reviewed and merged first.

Why all these if-else forks in the CI scripts?

pypa/manylinux_2_28 is based on Alma Linux, so it used dnf and yum as package managers (where Ubuntu uses apt).

@jameslamb jameslamb changed the title WIP: [ci] switch to manylinux_2_28 for Linux artifacts (fixes #5514) [ci] switch to manylinux_2_28 for Linux artifacts (fixes #5514) Nov 19, 2022
@jameslamb jameslamb marked this pull request as ready for review November 19, 2022 05:26
@jameslamb
Copy link
Collaborator Author

@guolinke @shiyu1994 @jmoralez @StrikerRUS I have been struggling to fix the issues showing up on the Ubuntu 14.04 image (#5588 (comment)).

Instead of spending more time on that, given that we want to move to a new image for building wheels anyway (#5514), can we please just focus on making that switch?

What I'm asking for from you:

@@ -7,6 +7,7 @@ if [[ $OS_NAME == "macos" ]]; then
sudo xcode-select -s /Applications/Xcode_10.3.app/Contents/Developer || exit -1
fi
else # gcc
sudo xcode-select -s /Applications/Xcode_14.1.app/Contents/Developer || exit -1
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this fixes #5589

@jameslamb
Copy link
Collaborator Author

✅ building successfully with COMPILER=clang (commit - build)

✅ building successfully with COMPILER=gcc (commit - build)

@jameslamb
Copy link
Collaborator Author

@guolinke whenever you have time, could you review this PR next?

This will unblock LightGBM's CI and allow us to continue making progress on the others things needed to stabilize the CI and improve the process of building Python wheels.

@jameslamb jameslamb changed the title [ci] switch to manylinux_2_28 for Linux artifacts (fixes #5514) [ci] switch to manylinux_2_28 for Linux artifacts (fixes #5514, fixes #5589) Nov 21, 2022
@@ -26,7 +26,7 @@ Compiled library that is included in the wheel file supports both **GPU** and **

For **Windows** users, `VC runtime <https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads>`_ is needed if **Visual Studio** (2015 or newer) is not installed.

For **Linux** users, **glibc** >= 2.14 is required. Also, in some rare cases, when you hit ``OSError: libgomp.so.1: cannot open shared object file: No such file or directory`` error during importing LightGBM, you need to install OpenMP runtime library separately (use your package manager and search for ``lib[g|i]omp`` for doing this).
For **Linux** users, **glibc** >= 2.28 is required. Also, in some rare cases, when you hit ``OSError: libgomp.so.1: cannot open shared object file: No such file or directory`` error during importing LightGBM, you need to install OpenMP runtime library separately (use your package manager and search for ``lib[g|i]omp`` for doing this).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is for version > 3.3.3, should we add a note about the minimal version of glibc for version <= 3.3.3 ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

I added this note inline in 8def665.

The next time we change this floor, it should probably be replaced with a table similar to those in the description of #5514.

@guolinke
Copy link
Collaborator

A minor comment about the documentation.

@jameslamb
Copy link
Collaborator Author

Thanks very much @guolinke ! I'll merge this without waiting on any other reviews so we can keep making progress on stabilizing CI and preparing for the v4.0.0 release.

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
2 participants