Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build] tune wget settings for debootstrap #12803

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kv-y
Copy link
Contributor

@kv-y kv-y commented Nov 22, 2022

Why I did it

wget is used by debootstrap to download deb packages for base system.
Sometime it can fail to download one or more packages and we get the error:

E: Couldn't download packages: libffi7
chroot: failed to run command '/debootstrap/debootstrap': No such file or directory

How I did it

  1. Consider "connection refused" a transient error and try again.
  2. Set number of tries to 3 (default is 20)
    If 3 is not enough then we probably need to fix deb server or our network connection.
    Also it takes too much time if we wait all 20 attempts because:
    default connection timeout in Linux: 127 sec
    default read/write timeout in wget: 900 sec

Signed-off-by: Konstantin Vasin k.vasin@yadro.com

@kv-y
Copy link
Contributor Author

kv-y commented Nov 22, 2022

@xumia
Could you please review?

Debootstrap fails was very annoying to me. Because it's a final part of build.
These errors were not rare when we download packages for debootstrap from deb.debian.org or another public debian mirrors. trafficmanager.net is more stable but it's not used for arm (see scripts/build_debian_base_system.sh)

Signed-off-by: Konstantin Vasin <k.vasin@yadro.com>
@kv-y
Copy link
Contributor Author

kv-y commented Nov 24, 2022

wget -t 3 --retry-connrefused passed a test successfully.
Run 200 times debootstrap --download-only --variant=minbase --arch arm64 bullseye $1 http://deb.debian.org/debian
There were 6 retries due to connection timeout. Without fix it would lead to 6 failed builds.
With this fix deb packages were successfully downloaded after 1 retry:

Resolving deb.debian.org (deb.debian.org)... 199.232.138.132, 2a04:4e42:3a::644
Connecting to deb.debian.org (deb.debian.org)|199.232.138.132|:80... failed: Connection timed out.
Connecting to deb.debian.org (deb.debian.org)|2a04:4e42:3a::644|:80... failed: Network is unreachable.
Retrying.

(try: 2)  http://deb.debian.org/debian/pool/main/u/util-linux/bsdutils_2.36.1-8+deb11u1_arm64.deb
Connecting to deb.debian.org (deb.debian.org)|199.232.138.132|:80... connected.
HTTP request sent, awaiting response... 200 OK 
Length: 145188 (142K) [application/vnd.debian.binary-package]

Yes, we have 127 sec timeout here between 1st and 2nd try, but it's not critical due to low rate of these errors.

@kv-y
Copy link
Contributor Author

kv-y commented Dec 6, 2022

@xumia
Could you please review updated version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants