Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T6674: Add build-scrips for packages without Jenkins #745

Merged
merged 1 commit into from
Sep 2, 2024

Conversation

sever-sever
Copy link
Member

Change Summary

Add build scripts for .deb packages without Jenkins. To exclude Jenkins we need some place where we can put new builds-scripts to run in parallel (old/new) during meantime.
We will deprecate old Jenkins package builds in the future.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes)
  • Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
  • Other (please describe):

Related Task(s)

Component(s) name

package-build

Proposed changes

How to test

Checklist:

  • I have read the CONTRIBUTING document
  • I have linked this PR to one or more Phabricator Task(s)
  • My commit headlines contain a valid Task id
  • My change requires a change to the documentation
  • I have updated the documentation accordingly

Copy link

github-actions bot commented Aug 26, 2024

👍
No issues in PR Title / Commit Title

@c-po
Copy link
Member

c-po commented Aug 26, 2024

There are 26 instances of build.py

(20:37) cpo helix:~/vyos-build [T6674] # find scripts/package-build -name build.py | xargs md5sum | wc -l
26

With 8 different versions

(20:37) cpo helix:~/vyos-build [T6674] # find scripts/package-build -name build.py | xargs md5sum | awk $'{print $1}' | sort -u | wc -l
8

I wonder why there can not be one single build.py script that's referenced (e.g. symlink) by all the packages to have a single source. Some package also have build.sh and build.py

In an ideal world there would be one single build.py script that's fed by package specific toml files telling the build script if any specialties need to be done while building the package, or the default is sufficient. That's what the old Jenkins Pipeline did, if there is no custom buildCmd available per package, use a sane default https://github.com/vyos/vyos-build/blob/current/vars/cloneAndBuild.groovy#L58-L72

@sever-sever
Copy link
Member Author

It will take +1 month (or several weeks) to rewrite and test all packages again.
Some of the packets use the original .sh scripts because I don't want to break anything. this way build.py just calls .sh
Where it is possible it uses just build.py, but some of the sources do not have /debian dir, this way build.py is uniq for some packets.
Some original .sh scripts use fpm --input-type dir and this is not used for others, build.py do not use it at all (fpm).

Also the path for patches are different, for example for the kernel patches/kernel
For stronsgswan and other just patches

Some packages like frr and linux-kernel requires subpackages to build them and other packages do not require it.

If build_cmd is not provided it uses the similar logic as Jenkins https://github.com/vyos/vyos-build/pull/745/files#diff-8b31a337cc794c445366ff42385a09399f406f28315b2f972e02830b5bff94f2R108-R115

The frr and linux kernel uses package.toml with list of packages https://github.com/vyos/vyos-build/pull/745/files#diff-6a29cb4e9e70a3de99e08b7adddf1d106cf5af2e8f85881fd0a974cd6b8a3e52R1
, but other use dictionary https://github.com/vyos/vyos-build/pull/745/files#diff-afb27e2f01f15b032b9b15d98abda71ce40ec8ca27ab3b9574a4310e25aabdf3R1
Should we use LIST of packages for all packages (for package.toml)?

The good idea if someone with good python3 and build-src skills will provide example of "good" build.py that we can use for all other packages.

I do not see the same and only build.py for all packages, or just do not have enough skills to do it.
I know that all packages in the current state works/builds fine.
Needs to recheck all packages per each change of build.py if we want to have the same file for all builds. Is it a requirement?

Can we merge it as it is and then we can spend several month to get the ideal build script?

@@ -0,0 +1,98 @@
#!/usr/bin/env python3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it's possible to call the script from a reusable location? I really would like to avoid duplicating it, if that's at all possible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try it and then see what it will be.
At least some of them could be reusable.

Copy link
Member Author

@sever-sever sever-sever Aug 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmbaturin @c-po PR ready for review
Most of the packages a built by symlinked to general build.py
Only these packages use their own build.py

linux-kernel
netfilter

Add build scripts for .deb packages without Jenkins.
To exclude Jenkins we need some place where we can put new builds-scripts
to run in parallel (old/new) during meantime
We will deprecate old Jenkins package builds in the future.
@dmbaturin dmbaturin merged commit 9833897 into vyos:current Sep 2, 2024
9 checks passed
@sever-sever sever-sever deleted the T6674 branch September 6, 2024 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

3 participants