Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to Fix Linux ARM32/ARM64 CI #18575

Merged
merged 9 commits into from
Nov 23, 2023

Conversation

theofficialgman
Copy link
Contributor

@theofficialgman theofficialgman commented Jul 13, 2023

This resolves various issues with the ARM32/ARM64 CI. It switches the CI to building on Focal (like is done for x86_64 linux in the CI). Building on bionic is not possible currently due to this error -> #18516

fixes: #18353

  • I signed the CLA
  • The title of the PR describes the problem it addresses
  • Each commit's message describes its purpose and effects, and references the issue it resolves
  • If changes are extensive, there is a sequence of easily reviewable commits
  • The code in the PR follows the coding rules
  • There are no unnecessary changes
  • The code compiles and runs on my machine, preferably after each commit individually
  • I created a unit test or vtest to verify the changes I made (if applicable)

@theofficialgman
Copy link
Contributor Author

looking into using clang-10 which might have even better compatability. don't merge yet till I check later today (few hours)

@theofficialgman
Copy link
Contributor Author

theofficialgman commented Jul 14, 2023

this codebase is easily the most unstable thing I have ever worked on. when I crush one bug, 3 more pop up in its place.
It is a minor miracle it can be built at all on anything.
https://github.com/theofficialgman/MuseScore/actions/runs/5547973233/jobs/10130347313#step:7:10320
https://github.com/theofficialgman/MuseScore/actions/runs/5548449195/jobs/10131437814#step:7:5937

@cbjeukendrup
Copy link
Contributor

Interesting. We actually never have any issues. Remarkably, both those errors are in third party code. Apparently those projects are not so well maintained for ARM.

@theofficialgman
Copy link
Contributor Author

theofficialgman commented Jul 14, 2023

the two that I already mentioned here aren't thirdparty #18575 (comment)

also https://github.com/theofficialgman/MuseScore/actions/runs/5547973233/jobs/10130347313#step:7:10320 was fixed upstream at crashpad years ago, you just never pulled it
https://github.com/theofficialgman/MuseScore/actions/runs/5548449195/jobs/10131437814#step:7:5937 was fixed upstream at mksqaushfs years ago but libappimage includes an older version plougher/squashfs-tools@fe2f5da

@theofficialgman
Copy link
Contributor Author

theofficialgman commented Jul 14, 2023

@theofficialgman theofficialgman marked this pull request as draft July 14, 2023 13:19
@theofficialgman theofficialgman changed the title change to GCC/G++-8 in ARM Linux CI change to Clang-10 in ARM Linux CI Jul 14, 2023
@cbjeukendrup
Copy link
Contributor

That mscore4portablenightly.svg error can be ignored. The reason that your build fails is that bash ./build/ci/linux/dumpsyms.sh fails. It looks like you've already found that the error is caused by lld, which is called from generate_debug_symbols.py, which originates from https://chromium.googlesource.com/chromium/src/+/master/components/crash/content/tools/generate_breakpad_symbols.py (and it not outdated afaics). It's unclear to me why lld fails.

@theofficialgman
Copy link
Contributor Author

theofficialgman commented Jul 15, 2023

Its a reported bug with QEMU. I was hoping that bumping from QEMU 7.0.0 to 7.2.0 would fix it but no dice. Only affects arm64 and does not affect the old 4.0.2 build that was able to be built with GCC-7.
Unfortunatly Musescore implemented changes that make it not possible to be built with GCC-7 anymore (as you saw in your initial CI builds)

This is the reported QEMU bug: multiarch/qemu-user-static#172 but thats not the QEMU repo. It should probably be reported upstream to have any chance it being fixed

edit: I went ahead and created a bug upstream https://gitlab.com/qemu-project/qemu/-/issues/1763
If they fix we can build QEMU and use it

armhf is not affected and already passes

@theofficialgman
Copy link
Contributor Author

@igorkorsukov I believe I have found the original source of the issue with building on the GCC-7 by digging into the log from the CI
f8eeffb
https://github.com/musescore/MuseScore/actions/runs/5530887371/jobs/10090856117
Can you please rework that to make it C+11/14 compatible. This was not patched in GCC until 10 (which can not be used due to not being available on bionic, and no ppas are not an option since they bump the libstdc++6 requirements and make binaries not compabile with the host) https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80265

@igorkorsukov
Copy link
Contributor

@theofficialgman To support cpp11/14, we need to abandon std::string_view, I don’t want to do this ... But as I understand it, the problem is not in std::string_view, but in the fact that its methods are not constexpr for gcc7. I made a version without constexpr, try it.
#19268

@theofficialgman
Copy link
Contributor Author

Sorry this is taking so long... I was running into issues on armhf and could not diagnose them. I finally determined it is a bug in wget of all things on armhf that is currently blocking me. In 2023, we can't even have functional wget, incredible

https://archlinuxarm.org/forum/viewtopic.php?t=16337

https://github.com/theofficialgman/MuseScore/actions/runs/6886325435/job/18731845220#step:7:155

What a pain... guess we will be building wget from source on armhf unless Ubuntu fixes the bug promptly which I have just filed https://bugs.launchpad.net/ubuntu/+source/wget/+bug/2043636

@theofficialgman theofficialgman changed the title change to Clang-10 in ARM Linux CI Updates to Fix Linux ARM32/ARM64 CI Nov 17, 2023
@theofficialgman theofficialgman marked this pull request as ready for review November 18, 2023 05:13
@theofficialgman
Copy link
Contributor Author

theofficialgman commented Nov 18, 2023

@igorkorsukov Sorry this took so long. This PR is ready for merge. Refer to the top initial comment for details.
You will see this CI succeed once it finishes in a few hours https://github.com/theofficialgman/MuseScore/actions/runs/6912018032

edit:
CC @MarcSabatella I have also gone ahead and picked these changes directly ontop of the current 4.1.1 tag and put them on their own branch. the CI for that is here and will also be done in a few hours https://github.com/theofficialgman/MuseScore/actions/runs/6912077699

@Jojo-Schmitz
Copy link
Contributor

I wonder whether it'd make sense to also build for arm on PRs (like this)

@igorkorsukov igorkorsukov merged commit 78511d6 into musescore:master Nov 23, 2023
11 checks passed
@cbjeukendrup
Copy link
Contributor

Thanks so much for your perseverance with this one, @theofficialgman !

@Jojo-Schmitz I don't think adding this to PR builds is a viable option since these builds take more than 3 hours to complete (because everything needs to be emulated because GitHub has no ARM Linux CI). But it will give nightly builds, if I'm not mistaken.

@MarcSabatella
Copy link
Contributor

Excellent news, this should be great for the Chromebook world! Thanks, @theofficialgman and all who helped make this possible. Now, if someone hadn't stolen the tablet I had been using, I'd be able to help test...

@cbjeukendrup
Copy link
Contributor

And it seems to work flawlessly in my virtual arm64 Ubuntu on M1 Mac! That's also very nice!

@theofficialgman
Copy link
Contributor Author

@RomanPudashkin any chance you could add the releases from the actions https://github.com/musescore/MuseScore/actions/runs/6980300818 to the https://github.com/musescore/MuseScore/releases/tag/v4.2.0-beta release page (as well as future releases)? I saw you ran both the linux actions and the linux arm actions but the linux arm appimages were not published

I think putting it in the official releases will bring a larger audience

@Jojo-Schmitz
Copy link
Contributor

Jojo-Schmitz commented Nov 27, 2023

Same request for the Windows PortableApps, see #15876

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fail to build on aarch64 Linux
5 participants