Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[build] Migrate libyang2 sources download from wget to dget #13394

Merged
merged 1 commit into from
Jan 19, 2023

Conversation

guillaumelambert
Copy link
Contributor

According to its manual page,
"[dget in its] first form, [..] fetches the requested URLs. If this is a .dsc or .changes file, then dget acts as a source-package
aware form of wget: it also fetches any files referenced in the
.dsc/.changes file.
The downloaded source is then checked with dscverify and, if successful, unpacked by dpkg-source."

Thus, when possible, dget use is preferable to wget so that sources authenticity can be performed automatically by dscverify"

Which release branch to backport (provide reason below if selected)

all branches using libyang2 sources

Description for the changelog

[build] Migrate libyang2 sources download from wget to dget

A picture of a cute animal (not mandatory but encouraged)

                  .".
                 /  |
                /  /
               / ,"
   .-------.--- /
  "._ __.-/ o. o\  
     "   (    Y  )
          )     /
         /     (
        /       Y
    .-"         |
   /  _     \    \ 
  /    `. ". ) /' )
 Y       )( / /(,/
,|      /     )
( |     /     /
 " \_  (__   (__        [nabis]
      "-._,)--._,)

According to its manual page,
"[dget in its] first form, [..] fetches the requested URLs.
If this is a .dsc or .changes file, then dget acts as a source-package
 aware form of wget: it also fetches any files referenced in the
.dsc/.changes file.
The downloaded source is then checked with dscverify and,
if successful, unpacked by dpkg-source."

Thus, when possible, dget use is preferable to wget so that sources
authenticity can be performed automatically by dscverify"

Signed-off-by: Guillaume Lambert <guillaume.lambert@orange.com>
@guillaumelambert guillaumelambert marked this pull request as ready for review January 17, 2023 20:08
@lguohan lguohan merged commit 9a0bf56 into sonic-net:master Jan 19, 2023
@dgsudharsan
Copy link
Collaborator

@guillaumelambert @lguohan I am getting the following failure when trying to build.

[2023-01-25T03:16:57.941Z] # Move the newly-built .deb packages to the destination directory
[2023-01-25T03:16:57.941Z] mv libyang2_2.0.112-6_amd64.deb libyang2-dev_2.0.112-6_amd64.deb libyang2-dbgsym_2.0.112-6_amd64.deb libyang2-tools_2.0.112-6_amd64.deb libyang2-tools-dbgsym_2.0.112-6_amd64.deb /sonic/target/debs/buster/
[2023-01-25T03:16:57.942Z] dget: retrieving https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc
[2023-01-25T03:16:57.942Z] chown: changing ownership of '/vcache': Operation not permitted
[2023-01-25T03:16:57.942Z] get_url_version https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc failed
[2023-01-25T03:16:57.942Z] dget: curl libyang2_2.0.112-6.dsc https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc failed
[2023-01-25T03:16:57.942Z] make[1]: *** [Makefile:10: /sonic/target/debs/buster/libyang2_2.0.112-6_amd64.deb] Error 1
[2023-01-25T03:16:57.942Z] make[1]: Leaving directory '/sonic/src/libyang2'
[2023-01-25T03:16:57.942Z] [ FAIL LOG END ] [ target/debs/buster/libyang2_2.0.112-6_amd64.deb ]
[2023-01-25T03:16:57.942Z] WARNING: No swap limit support

Are these due to these changes?

@lguohan
Copy link
Collaborator

lguohan commented Jan 25, 2023

@guillaumelambert , i am reverting this pr to avoid build break, can you take a look at this issue?

@guillaumelambert
Copy link
Contributor Author

@guillaumelambert , i am reverting this pr to avoid build break, can you take a look at this issue?

OK but I doubt this is due to PR itself.
I can see that there is a permission problem on '/vcache'
=> chown: changing ownership of '/vcache': Operation not permitted
Might this be due to a concurrency problem with another thread that modified the /vcache folder ownership ?

lguohan added a commit that referenced this pull request Jan 25, 2023
@dgsudharsan
Copy link
Collaborator

@guillaumelambert , i am reverting this pr to avoid build break, can you take a look at this issue?

OK but I doubt this is due to PR itself. I can see that there is a permission problem on '/vcache' => chown: changing ownership of '/vcache': Operation not permitted Might this be due to a concurrency problem with another thread that modified the /vcache folder ownership ?

I am not sure if vcache is related but when I did dget from my server I still get the error
dget https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc
dget: retrieving https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404
dget: curl libyang2_2.0.112-6.dsc https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc failed

@guillaumelambert
Copy link
Contributor Author

@guillaumelambert , i am reverting this pr to avoid build break, can you take a look at this issue?

OK but I doubt this is due to PR itself. I can see that there is a permission problem on '/vcache' => chown: changing ownership of '/vcache': Operation not permitted Might this be due to a concurrency problem with another thread that modified the /vcache folder ownership ?

I am not sure if vcache is related but when I did dget from my server I still get the error dget https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc dget: retrieving https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (22) The requested URL returned error: 404 dget: curl libyang2_2.0.112-6.dsc https://deb.debian.org/debian/pool/main/liby/libyang2/libyang2_2.0.112-6.dsc failed

OK thanks for the pointer.
This package was in debian testing and after having taken a look at the repo content
at https://deb.debian.org/debian/pool/main/liby/libyang2/
it appears it was removed from the official distribution.
Can you confirm it works better with the previous URL pointing to the mirror in sonicstorage ?
dget https://sonicstorage.blob.core.windows.net/debian/pool/main/liby/libyang/libyang2_2.0.112-6.dsc

@dgsudharsan
Copy link
Collaborator

I get the following error
dget https://sonicstorage.blob.core.windows.net/debian/pool/main/liby/libyang/libyang2_2.0.112-6.dsc
dget: retrieving https://sonicstorage.blob.core.windows.net/debian/pool/main/liby/libyang/libyang2_2.0.112-6.dsc
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2267 100 2267 0 0 2341 0 --:--:-- --:--:-- --:--:-- 2339
dget: retrieving https://sonicstorage.blob.core.windows.net/debian/pool/main/liby/libyang/libyang2_2.0.112.orig.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1050k 100 1050k 0 0 193k 0 0:00:05 0:00:05 --:--:-- 248k
dget: retrieving https://sonicstorage.blob.core.windows.net/debian/pool/main/liby/libyang/libyang2_2.0.112-6.debian.tar.xz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11468 100 11468 0 0 17835 0 --:--:-- --:--:-- --:--:-- 17835
libyang2_2.0.112-6.dsc:
dscverify: libyang2_2.0.112-6.dsc failed signature check:
gpg: WARNING: no command supplied. Trying to guess what you mean ...
gpg: Signature made Thu 25 Nov 2021 11:08:55 AM IST
gpg: using RSA key C361B1E3029543EBC625E97D8372A477EFAE59C2
gpg: Can't check signature: No public key
gpg: can't create `/root/.gnupg/random_seed': No such file or directory
Validation FAILED!!

@guillaumelambert
Copy link
Contributor Author

Thanks for the prompt feedback.
The following error is normal if you have not installed the debian-keyring.
$ sudo apt-get install debian-keyring
It does not come by default on ubuntu.
The download passed but it fails at the dscverify step to check the signature that authentifies the sources..

guillaumelambert added a commit to Orange-OpenSource/sonic-buildimage that referenced this pull request Jan 25, 2023
…t#13394)

According to its manual page,
"[dget in its] first form, [..] fetches the requested URLs.
If this is a .dsc or .changes file, then dget acts as a source-package
 aware form of wget: it also fetches any files referenced in the
.dsc/.changes file.
The downloaded source is then checked with dscverify and,
if successful, unpacked by dpkg-source."

Thus, when possible, dget use is preferable to wget so that sources
authenticity can be performed automatically by dscverify"

Signed-off-by: Guillaume Lambert <guillaume.lambert@orange.com>
@guillaumelambert
Copy link
Contributor Author

I propose a new PR at #13517
since this one is closed

EdenGri pushed a commit to EdenGri/sonic-buildimage that referenced this pull request Jan 26, 2023
mlok-nokia pushed a commit to mlok-nokia/sonic-buildimage that referenced this pull request Jan 26, 2023
…t#13394)

According to its manual page,
"[dget in its] first form, [..] fetches the requested URLs.
If this is a .dsc or .changes file, then dget acts as a source-package
 aware form of wget: it also fetches any files referenced in the
.dsc/.changes file.
The downloaded source is then checked with dscverify and,
if successful, unpacked by dpkg-source."

Thus, when possible, dget use is preferable to wget so that sources
authenticity can be performed automatically by dscverify"

Signed-off-by: Guillaume Lambert <guillaume.lambert@orange.com>
mlok-nokia pushed a commit to mlok-nokia/sonic-buildimage that referenced this pull request Jan 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants