Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove local gawk and make builds, add xx30 top and bottom hashes under hashes.txt #1178

Merged
merged 2 commits into from
Jun 28, 2022
Merged

Remove local gawk and make builds, add xx30 top and bottom hashes under hashes.txt #1178

merged 2 commits into from
Jun 28, 2022

Conversation

tlaurion
Copy link
Collaborator

@tlaurion tlaurion commented Jun 21, 2022 •
edited
Loading

  • Makefile: remove local gawk and make references to see if everything builds
  • xx30 boards: add proper statements to have roms output hashes under hashes.txt
  • .circleci/config.yml: remove bootstrap
  • patches/make-4.2.1.patch removal

Since gawk was outputed into hashes.txt in a fashion I was not able to fix, this is the solution on top of debian-11 based distributions (in terms of gawk and make being > 4.2.1 which was problematic before).

Replaces #1128.

TODO: If works, remove heads-wiki need to build bootstrap prior of boards

@tlaurion tlaurion marked this pull request as draft June 21, 2022 20:25
@tlaurion tlaurion force-pushed the remove_gawk_make_local_build-fix_xx30_maximized-boards_produce_top_bottom_hashes branch from b3f163d to 1087912 Compare June 21, 2022 20:27
@tlaurion
Copy link
Collaborator Author

tlaurion commented Jun 22, 2022 •
edited
Loading

Some historical analysis:

There seems to be no reason to keep those from now on.
On a reproducibility perspective (reproducible builds broken as of now anyway), building a specific commit will always follow CircleCI steps, which defines also the docker image and OS used and dependencies installed. This comes with problems with apt installed depenencies being a moving target, but this is what is supported now.

Until we move to another building system or integrate Nixos toolstack layer for poked binaries reproducibility, having gawk and make being locally built right now is not bringing any value and causes issues (hashes containing gawk per $@ call instead of roms hashes since gawk being locally built being the unseen regression for a while now).

@tlaurion tlaurion mentioned this pull request Jun 22, 2022
Merged
@tlaurion tlaurion force-pushed the remove_gawk_make_local_build-fix_xx30_maximized-boards_produce_top_bottom_hashes branch 3 times, most recently from 27c20eb to 27aa407 Compare June 22, 2022 19:18
@tlaurion
Copy link
Collaborator Author

Coreboot 4.11 based boards are causing a lot of race conditions under CircleCI since a long while.

@tlaurion tlaurion mentioned this pull request Jun 22, 2022
Closed
@tlaurion tlaurion force-pushed the remove_gawk_make_local_build-fix_xx30_maximized-boards_produce_top_bottom_hashes branch from 27aa407 to cab2089 Compare June 22, 2022 20:36
@tlaurion
Copy link
Collaborator Author

Based on top of #1182
@MrChromebox : please review

MrChromebox
MrChromebox suggested changes Jun 22, 2022
View reviewed changes
Copy link
Contributor

@MrChromebox MrChromebox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should really be 2 separate commits, unless there's a dependency I'm not seeing

@tlaurion
Copy link
Collaborator Author

It will be the rebased once #1182 is merged.

This was just a test to see and confirm that builds stop failing with #1182 merged. And that test succeeded.

@tlaurion
Copy link
Collaborator Author

@MrChromebox unless you mean removing gawk and make modules and Makefile references in one commit

And another commit adding proper statements to get back rom hashes under hashes.txt into a separate commit?

And then another commit to remove CircleCI bootstrap step?

(The goal of this commit is to bring back ROM hashes, which those 3 actions depend to test this commit...)

@tlaurion
Copy link
Collaborator Author

tlaurion commented Jun 23, 2022 •
edited
Loading

Forgot to remove make patch.

@MrChromebox you want 4 separate commits?

@tlaurion tlaurion force-pushed the remove_gawk_make_local_build-fix_xx30_maximized-boards_produce_top_bottom_hashes branch from cab2089 to 13a38cb Compare June 23, 2022 13:37
@tlaurion
Remove local build of gawk make
bf415a8 
-Makefile: remove local gawk and make version compare and local build
-modules: remove gawk and make
-patches: remove make

local make was added to build 4.2.1 on OSes that were having older version. It was then patched to be built on OSes having newer buildstack.
local gawk was added when GPG toolstack was older then libgpg-error 1.37. GPG toolstack was then upgraded, but local gawk stayed.

Removing those permits better parallelization and of builds and reduces CircleCI (and higher cores systems) to have race conditions and stalled builds
@tlaurion
xx30 boards: add top bottom roms statements to get hashes under hashe…
ba9235a 
…s.txt
@tlaurion tlaurion force-pushed the remove_gawk_make_local_build-fix_xx30_maximized-boards_produce_top_bottom_hashes branch from 13a38cb to ba9235a Compare June 23, 2022 15:08
@tlaurion
Copy link
Collaborator Author

@MrChromebox seperated in two commits.

@tlaurion tlaurion marked this pull request as ready for review June 23, 2022 15:09
@tlaurion tlaurion changed the title WiP: Remove local gawk and make, add xx30 top and bottom hashes under hashes.txt Remove local gawk and make builds, add xx30 top and bottom hashes under hashes.txt Jun 23, 2022
This was referenced Jun 23, 2022
Open
Closed
@MrChromebox
Copy link
Contributor

@MrChromebox seperated in two commits.

perfect, thx. I'll test then approve

@tlaurion
Copy link
Collaborator Author

@MrChromebox Can't wait for merge. Sick of CircleCI builds failing for race conditions linked to old make

This was referenced Jun 23, 2022
Closed
Closed
@tlaurion
Copy link
Collaborator Author

@MrChromebox : conclusion?

MrChromebox
MrChromebox approved these changes Jun 28, 2022
View reviewed changes
Copy link
Contributor

@MrChromebox MrChromebox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested successfully under Debian 11.3

@tlaurion tlaurion merged commit 2cfa940 into linuxboot:master Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MrChromebox MrChromebox MrChromebox approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tlaurion @MrChromebox