Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Auto update failing for rust #20970

Closed
TomJo2000 opened this issue Jul 28, 2024 · 6 comments · Fixed by #20888
Closed

Auto update failing for rust #20970

TomJo2000 opened this issue Jul 28, 2024 · 6 comments · Fixed by #20888
Labels
auto update failing Issues with packages auto update bot Created by Termux bot

Comments

@TomJo2000
Copy link
Member

Stop rust from getting auto updated for the time being,
until we can figure out why it's running out of space on the CI.

@TomJo2000 TomJo2000 added auto update failing Issues with packages auto update bot Created by Termux bot labels Jul 28, 2024
@TomJo2000
Copy link
Member Author

This issue should stop it from running rust in the next auto update cycle.

done <<<"$(
gh issue list \
--limit 10000 \
--label "auto update failing" --label "bot" \
--state open \
--search "Auto update failing for in:title type:issue" \
--json title | jq -r '.[] | .title' | sort | uniq
)"

@twaik
Copy link
Member

twaik commented Jul 28, 2024

Probably we should setup multiple building stages like it is done for chromium. It can let us avoid both storage and execution time limitations if we do that right.
I mean in the case if some building targets (static/shared libraries? executables? archives?) are already processed CI will not build them again, right? So it will not spend time and storage for them.
I am not sure how exactly chromium is built in termux, I should read the build script again.

@TomJo2000
Copy link
Member Author

Porting the chromium builder infrastructure from the TUR is definitely a good idea.
That would also allow us to move chromium, electron, etc. out of the TUR.

I already got my hands full though and GH Actions is not really my strong suit.

@twaik
Copy link
Member

twaik commented Jul 28, 2024

I am not sure how exactly it is split for multiple CI runs.
https://github.com/termux-user-repository/chromium-builder/blob/main/tur-chromium/chromium-stable/build.sh
Probably @licy183 can explain it.

@licy183
Copy link
Member

licy183 commented Jul 31, 2024

The main logic of tur-continuous is in https://github.com/termux-user-repository/tur/blob/master/.github/workflows/package_continuous.yml and https://github.com/termux-user-repository/tur/blob/master/continuous-build-wrapper.sh.

The package that needs long-time compilation is stored in a different folder, so that CI will trigger package_continuous.yml.

In package_continuous.yml, there is a marker file to indicate whether the build processis completed. If the build is not completed, run continuous-build-wrapper.sh to continue the build. If the build is completed, skip it.

In continuous-build-wrapper.sh, zstd is first installed to package the build status into a file, and then, run timeout ./scripts/run-docker.sh. EXIT_CODE will be used to determine whether the build process is completed. If it is not, the build status (~/.termux-build/package-name/build) will be packaged into a file using zstd; if it is, delete the build status file and create the build flag file.

@TomJo2000
Copy link
Member Author

The GitHub actions part of that is still dark magic to me.
But I get the main idea.

If we're about to run out of time, save the state as an artifact and continue it on a fresh runner.
With all the associated plumbing to make that happen.

Separate directory seems fine to me, but an approach similar to scripts/big-pkgs.list might be preferable to not separate packages that require the continuous runner from regular packages.
That separation may very well be desirable. My main reason for thinking it wouldn't be is that it would separate them from the normal main, x11 and root channels.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto update failing Issues with packages auto update bot Created by Termux bot
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants