Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix importlib-metadata dependency #1748

Merged

Conversation

mgmgwi
Copy link
Contributor

@mgmgwi mgmgwi commented May 6, 2022

This should fix the wrong dependency in the indy-node .deb package.

The following packages have unmet dependencies:
indy-node : Depends: importlib-metadata (< 3.0) but it is not installable

Signed-off-by: mgmgwi guido.wischrop@mgm-tp.com

@mgmgwi mgmgwi requested a review from a team as a code owner May 6, 2022 12:52
@sovbot
Copy link

sovbot commented May 6, 2022

Can one of the admins verify this patch?

@mgmgwi mgmgwi added the Ubuntu 20.04 Ubuntu 20.04 related activity. label May 6, 2022
@WadeBarnes
Copy link
Member

Thanks @mgmgwi. @udosson, @pSchlarb, this PR is relying on the fact that indy-plenum is building and publishing this particular 3rd party package; https://github.com/hyperledger/indy-plenum/blob/ubuntu-20.04-upgrade/build-scripts/ubuntu-2004/build-3rd-parties.sh#L106. How do we feel about this implicit dependency? Perhaps a comment in the prepare package script would be enough.

@pSchlarb
Copy link
Contributor

pSchlarb commented May 9, 2022

Hm since version setting is also commented in plenum, i see no problem with that so far.
IMHO a comment should be sufficient and wouldn't hurt.

Copy link
Contributor

@udosson udosson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mgmgwi thanks for the PR and spotting the issue.
Could you please make the small change?

@@ -33,6 +33,7 @@ if [ "$distro_packages" = "debian-packages" ]; then
echo -e "\nAdapt the dependencies for the Canonical archive"
sed -i "s~timeout-decorator~python3-timeout-decorator~" setup.py
sed -i "s~distro~python3-distro~" setup.py
sed -i "s~importlib-metadata~python3-importlib-metadata~" setup.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mgmgwi could you remove this line?
We don't use python3-importlib-metadata from the Canonica archive because python3-importlib-metadata is still at version 1.5.0-1 there (https://packages.ubuntu.com/focal/python3-importlib-metadata)

We build the Debian package based on the PyPi packages of python3-importlib-metadata, publish it to Hyperledger's Artifactory and pull the artifact from there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the problem is, that the .deb package contains a reference to importlib-metadata which we have to replace or remove. If not here, where then?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependencies are taken from setup.py. importlib-metadata==3.10.1 is referenced there already.
When installing the Debian package of Indy-Node dpgk looks for python3-importlib-metadata==3.10.1 which will be pulled from here https://hyperledger.jfrog.io/ui/native/indy/pool/focal/dev/i/importlib-metadata/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 are you sure? I think it will look for importlib-metadata=3.10.1 (missing the python3- prefix) which still does not exists. I tried manually patching the control file in the .deb package like this:

Package: indy-node
Version: 1.13.0~dev24
License: Apache 2.0
Vendor: none
Architecture: amd64
Maintainer: Hyperledger <hyperledger-indy@lists.hyperledger.org>
Installed-Size: 2807
Depends: indy-plenum (= 1.13.0~dev14), importlib-metadata (= 3.10.1), python3-timeout-decorator (>= 0.5.0), python3-distro (>= 1.5.0), at, iptables, libsodium23
Section: default
Priority: extra
Homepage: https://github.com/hyperledger/indy-node
Description: Indy node

Please note the Depends line. Then I repackaged it, but when I install it with apt install /tmp/indy-node_1.13.0_dev24-patched2_amd64.deb it still complains:

The following packages have unmet dependencies:
 indy-node : Depends: importlib-metadata (= 3.10.1) but it is not installable

Copy link
Contributor

@udosson udosson May 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm quite sure but I cannot verify before Tuesday.
We use fpm to build the Debian artifacts with the following flag --python-package-name-prefix "python3"
All install_requires (https://github.com/hyperledger/indy-node/blob/ubuntu-20.04-upgrade/setup.py#L58) packages will be converted to python3-package_name. In our case python3-importlib-metadata

Therefore we publish the third-party dependencies with the python3 prefix. And in case we use the artifacts from the canonical archive we adjust them to the correct build version. An example for Indy Plenum can be found here for Ubuntu 16.04.
We removed the override with sed for Ubuntu 20.04 because we try to not pin versions anymore.
@WadeBarnes @pSchlarb Do you think we can remove this? https://github.com/hyperledger/indy-node/blob/ubuntu-20.04-upgrade/build-scripts/ubuntu-2004/prepare-package.sh#L34

@udosson
Copy link
Contributor

udosson commented May 9, 2022

I'd rather add the comment to setup.py because we don't need to change the prepare-package.sh script

setup.py Show resolved Hide resolved
@mgmgwi mgmgwi force-pushed the ubuntu-20.04-upgrade branch 2 times, most recently from c7fa709 to fa686a7 Compare May 13, 2022 06:26
@mgmgwi
Copy link
Contributor Author

mgmgwi commented May 13, 2022

I removed the sed replacement in build-scripts/ubuntu-2004/prepare-package.sh#L34 as requested by @udosson in the comment above.

@mgmgwi mgmgwi force-pushed the ubuntu-20.04-upgrade branch from fa686a7 to c7ac8e0 Compare May 13, 2022 06:32
@udosson
Copy link
Contributor

udosson commented May 13, 2022

@mgmgwi - thank you for the changes. I'll test the build and install process on Tuesday and will merge after the successful testing.

@mgmgwi mgmgwi force-pushed the ubuntu-20.04-upgrade branch from c7ac8e0 to af29e5e Compare May 20, 2022 08:22
@mgmgwi
Copy link
Contributor Author

mgmgwi commented May 20, 2022

I talked with @udosson and for now, we have to add the python3 prefix in prepare-package.sh. I updated the file accordingly.

- use sed to replace package name prefix
- pinned importlib-metadata to 3.10.1 in setup.py

Signed-off-by: mgmgwi <guido.wischrop@mgm-tp.com>
@mgmgwi mgmgwi force-pushed the ubuntu-20.04-upgrade branch from af29e5e to 9244e1f Compare May 20, 2022 08:28
@udosson
Copy link
Contributor

udosson commented May 20, 2022

Thanks for adjusting the PR according to our discussion, @mgmgwi

We need to at the python3- prefix in prepare-package.sh. However, I cannot say at the moment why we don't need this in Indy Plenum. I'll do further investigations and please let me know your thoughts @WadeBarnes and @pSchlarb if you have an idea.

Copy link
Contributor

@udosson udosson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGMT. Thanks, @mgmgwi

@mgmgwi mgmgwi merged commit 28c5882 into hyperledger:ubuntu-20.04-upgrade May 20, 2022
@WadeBarnes
Copy link
Member

Thanks for adjusting the PR according to our discussion, @mgmgwi

We need to at the python3- prefix in prepare-package.sh. However, I cannot say at the moment why we don't need this in Indy Plenum. I'll do further investigations and please let me know your thoughts @WadeBarnes and @pSchlarb if you have an idea.

Perhaps an oversight.

@mgmgwi mgmgwi deleted the ubuntu-20.04-upgrade branch May 23, 2022 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ubuntu 20.04 Ubuntu 20.04 related activity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants