Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install native packages #4693

Merged
merged 8 commits into from
Jun 30, 2021
Merged

Conversation

hgy59
Copy link
Contributor

@hgy59 hgy59 commented Jun 26, 2021

Motivation: Omit compilation of native packages that support direct installation for linux amd64/x86_64
Linked issues: #4674

Checklist

  • Build rule all-supported completed successfully
  • Package upgrade completed successfully
  • New installation of package completed successfully

Remarks

We introduced native packages as we do not want to blow up the spksrc docker image with all the tools needed to build different packages.
Until now most of the native packages are compiled for the target linux system (debian, in the spksrc docker image) on demand, with the exception for the dotnet sdk packages.
With the introduction of native/cmake (#4674) I noticed that the build of cmake takes a lot of time (and resources). The (powerful) github build action environment takes about 7 minutes to build cmake form sources (as of June 2021), and my personal build system more than 20 min.
The download and installation (i.e. extraction) takes only a tiny little bit of resources.
This was the kick off for this PR.

hgy59 added 2 commits June 26, 2021 14:49
- similar as spksrc.install-resources.mk but for native packages
- update installation of native dotnet packages
- typos
- avoid compilation of native go and bootstrap with go 1.4
@hgy59 hgy59 force-pushed the install_native_packages branch from bb4ca34 to d098cd1 Compare June 26, 2021 13:28
@th0ma7
Copy link
Contributor

th0ma7 commented Jun 26, 2021

Just a thought, what if instead we add deb build dependencies, e.g. force apt install of native dependencies as we see fit? and leave the native/* only for version specific requirements we may have?

Something like: DEPENDS = deb/cmake where a spksrc.deb.mk would invoke an install of packages within the debian container.

@hgy59
Copy link
Contributor Author

hgy59 commented Jun 26, 2021

Just a thought, what if instead we add deb build dependencies, e.g. force apt install of native dependencies as we see fit? and leave the native/* only for version specific requirements we may have?

Something like: DEPENDS = deb/cmake where a spksrc.deb.mk would invoke an install of packages within the debian container.

IMHO we have specific versions in native packages only. As the debian cmake package is outdated (we already have cmake in the docker image) we added newer cmake to native.

I do not like to use apt-get within the container

  • the package cache is cleaned when building the docker image, so we would have to call apt-get update each time before installing such a package
  • as I avoid to use root but a sepecific user without root access in the container this is a no go too.

@th0ma7
Copy link
Contributor

th0ma7 commented Jun 26, 2021

IMHO we have specific versions in native packages only. As the debian cmake package is outdated (we already have cmake in the docker image) we added newer cmake to native.

We could probably use backports for that... And to that effect allow adding ppa or other channels as needed.

I do not like to use apt-get within the container

the package cache is cleaned when building the docker image, so we would have to call apt-get update each time before installing such a package

Unless the cache is entirely empty (and even then) we can specify the exact version to use, thus, avoiding the update. Its no different than maintaining version in our makefiles. And even if update was needed, it is faster than rebuilding a native/* and probably not that much longer than download an external .tar file.

as I avoid to use root but a sepecific user without root access in the container this is a no go too.

A simple sudo can do the trick at docker image creation while making sure our user is part of the sudo group:

$ sudo perl -i.bak -pe 's/=.*$/=NOPASSWD: ALL/ if /^.*sudo.*ALL=/' /etc/sudoers

Just saying it may be best relying more on Debian packages as much as possible rather than downloading pre-built tar-balls when we can avoid it.

hgy59 added 6 commits June 26, 2021 16:02
- update native/cmake to v3.20.5
- use installer script to avoid building from sources
- update dotnet-sdk-5.0 to v5.0.7, SDK 5.0.301
- udpate dotnet-sdk-3.1 to v3.1.16, SDK 3.1.410
- only protoc (the protobuf compiler) is required as native to build cross/protobuf
- avoid compilation by installation of protoc binary in native/protoc
- update native/nodejs and use spksrc.native-install.mk
- use linux-x86_64 package to avoid building from source
@hgy59
Copy link
Contributor Author

hgy59 commented Jun 30, 2021

Now all native packages with downloadable binaries are installed instead of compiled from sources.
The remaining native packages could be installed from deb packages later.
native/squid is going to be obsolete by #4695

@hgy59 hgy59 merged commit 4b69f37 into SynoCommunity:master Jun 30, 2021
@hgy59 hgy59 deleted the install_native_packages branch June 30, 2021 19:23
@hgy59 hgy59 mentioned this pull request Aug 15, 2021
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants