Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #29075 - add pulp_rpm plugin #89

Merged
merged 1 commit into from
Jun 4, 2020
Merged

Conversation

wbclark
Copy link
Collaborator

@wbclark wbclark commented Apr 7, 2020

No description provided.

@wbclark
Copy link
Collaborator Author

wbclark commented Apr 7, 2020

As expected at this time, it's failing to find a package to install:

Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install python3-pulp-pulp_rpm' returned 1: Error: Nothing to do

  Error: /Stage[main]/Pulpcore::Plugin::Rpm/Pulpcore::Plugin[pulp_rpm]/Package[python3-pulp-pulp_rpm]/ensure: change from 'purged' to 'present' failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install python3-pulp-pulp_rpm' returned 1: Error: Nothing to do

Although the package name is probably not correct, but I don't find under any other name in http://koji.katello.org/releases/yum/katello-nightly/pulpcore/el7/x86_64/ either.

@evgeni could you direct me where to find it, if it exists at this point? I did see theforeman/foreman-packaging#4191 was merged, hence my confusion on the topic.

Thanks!

@evgeni
Copy link
Member

evgeni commented Apr 7, 2020

The python plugin was not packaged yet

@wbclark wbclark force-pushed the rpm_plugin branch 2 times, most recently from 88bd025 to 2d0e0aa Compare May 8, 2020 18:41
manifests/plugin/rpm.pp Outdated Show resolved Hide resolved
@wbclark
Copy link
Collaborator Author

wbclark commented May 8, 2020

I updated to have the correct plugin/package name but I'm still anticipating a failure based on what I observed when I tried manually installing it.

Gemfile Outdated Show resolved Hide resolved
Copy link
Member

@ekohl ekohl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 assuming test are green

@wbclark wbclark force-pushed the rpm_plugin branch 2 times, most recently from 546bb7a to fc9779e Compare May 11, 2020 16:09
@ehelms
Copy link
Member

ehelms commented May 11, 2020

Looks like now it's having issues on EL7 and EL8 with finding solv even though that is packaged and in the repository:

  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns: pkg_resources.DistributionNotFound: The 'solv' distribution was not found and is required by the application
  Info: Pulpcore::Admin[reset-admin-password --random]: Unscheduling all events on Pulpcore::Admin[reset-admin-password --random]
  Error: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]: Failed to call refresh: 'python3-django-admin migrate --noinput' returned 1 instead of one of [0]
  Error: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]: 'python3-django-admin migrate --noinput' returned 1 instead of one of [0]
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[reset-admin-password --random]/Exec[python3-django-admin reset-admin-password --random]: Dependency Exec[python3-django-admin migrate --noinput] has failures: true
  Warning: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[reset-admin-password --random]/Exec[python3-django-admin reset-admin-password --random]: Skipping because of failed dependencies

I'll try to debug it.

@evgeni
Copy link
Member

evgeni commented May 11, 2020

When you start a python3 shell, can you run the following:

import pkg_resources
import solv

pkg_resources.get_distribution('solv')

And post the output?

@ehelms
Copy link
Member

ehelms commented May 11, 2020

Looks similar:

[root@centos8 vagrant]# python3
Python 3.6.8 (default, May 21 2019, 23:51:36) 
[GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> import solv
>>> 
>>> pkg_resources.get_distribution('solv')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 468, in get_distribution
    dist = get_provider(dist)
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 344, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 888, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 774, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'solv' distribution was not found and is required by the application

@ehelms
Copy link
Member

ehelms commented May 11, 2020

If it helps:

[root@centos8 vagrant]# rpm -q python3-solv --list
/usr/lib/.build-id
/usr/lib/.build-id/c8
/usr/lib/.build-id/c8/f91b58a1c9bd8367b3ba20a7f5df00fc90c144
/usr/lib64/python3.6/site-packages/__pycache__/solv.cpython-36.opt-1.pyc
/usr/lib64/python3.6/site-packages/__pycache__/solv.cpython-36.pyc
/usr/lib64/python3.6/site-packages/_solv.so
/usr/lib64/python3.6/site-packages/solv.py

@evgeni
Copy link
Member

evgeni commented May 11, 2020

If it helps:

[root@centos8 vagrant]# rpm -q python3-solv --list
/usr/lib/.build-id
/usr/lib/.build-id/c8
/usr/lib/.build-id/c8/f91b58a1c9bd8367b3ba20a7f5df00fc90c144
/usr/lib64/python3.6/site-packages/__pycache__/solv.cpython-36.opt-1.pyc
/usr/lib64/python3.6/site-packages/__pycache__/solv.cpython-36.pyc
/usr/lib64/python3.6/site-packages/_solv.so
/usr/lib64/python3.6/site-packages/solv.py

Indeed. It's missing the egg files, which seem not to be built by cmake for some reason.

@ehelms
Copy link
Member

ehelms commented May 26, 2020

@evgeni another related issue to how the python package was created?

  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns: Traceback (most recent call last):
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/bin/python3-django-admin", line 11, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     load_entry_point('Django==2.2.12', 'console_scripts', 'django-admin')()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     utility.execute()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 357, in execute
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     django.setup()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/__init__.py", line 24, in setup
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     apps.populate(settings.INSTALLED_APPS)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/apps/registry.py", line 122, in populate
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     app_config.ready()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulpcore/app/apps.py", line 81, in ready
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     self.import_viewsets()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulpcore/app/apps.py", line 115, in import_viewsets
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     self.viewsets_module = import_module(viewsets_module_name)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib64/python3.6/importlib/__init__.py", line 126, in import_module
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     return _bootstrap._gcd_import(name[level:], package, level)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/viewsets.py", line 22, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     from .serializers import (
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/serializers.py", line 10, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     from pulp_2to3_migration.app.plugin import PLUGIN_MIGRATORS
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/__init__.py", line 16, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     PLUGIN_MIGRATORS[entry_point.name] = entry_point.load()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2318, in load
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     return self.resolve()
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in resolve
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     module = __import__(self.module_name, fromlist=['__name__'], level=0)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 21, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     from pulp_rpm.app.tasks.synchronizing import RpmContentSaver
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/__init__.py", line 2, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     from .synchronizing import synchronize  # noqa
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 68, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     from pulp_rpm.app.modulemd import (
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulp_rpm/app/modulemd.py", line 9, in <module>
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     gi.require_version('Modulemd', '2.0')
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:   File "/usr/lib64/python3.6/site-packages/gi/__init__.py", line 99, in require_version
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns:     raise ValueError('Namespace %s not available' % namespace)
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[python3-django-admin migrate --noinput]/returns: ValueError: Namespace Modulemd not available
  Notice: /Stage[main]/Pulpcore::Database/Pulpcore::Admin[reset-admin-password --random]/Exec[python3-django-admin reset-admin-password --random]: Dependency Exec[python3-django-admin migrate --noinput] has failures: true

@evgeni
Copy link
Member

evgeni commented May 27, 2020

@ehelms we're probably missing a Requires: libmodulemd or so, will look into it.

@evgeni
Copy link
Member

evgeni commented May 27, 2020

Oh, no, our libmodulemd is too old. Fuck.

@jlsherrill
Copy link
Contributor

The libmodulemd issue is now resolved, right @evgeni ? theforeman/foreman-packaging@764bec8

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

The libmodulemd issue is now resolved, right @evgeni ? theforeman/foreman-packaging@764bec8

no, that's the "solv" fix, not the "modulemd" fix. you can watch me throwing tables on twitch if you like :)

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

modulemd might be fixed by theforeman/foreman-packaging#5355

@ehelms can I easily test that with a scratch build?

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

alternative to test this, w/o the fixed build: install libmodulemd (on el8) or libmodulemd2 (on el7) manually/via puppet

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

btw, I don't understand the EL8 failure:

pkg_resources.DistributionNotFound: The 'libcomps~=0.1.11' distribution was not found and is required by the application

but

>>> import pkg_resources
>>> pkg_resources.get_distribution('libcomps~=0.1.11')
libcomps 0.1.15 (/usr/lib64/python3.6/site-packages)

@evgeni evgeni closed this Jun 3, 2020
@evgeni evgeni reopened this Jun 3, 2020
@ehelms
Copy link
Member

ehelms commented Jun 3, 2020

El7 is at least happy. I do not understand that EL8 failure myself either.

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

It tells us that python3-libcomps is not installed, which is not possible?

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

Hah, the python3-libcomps from centos does not have am egg info. This would explain the error. But we ship this package ourself on el8...

But if it's already installed, we wouldn't update it (in yum/puppet), as it's new enough.

yeah, that must be it.

@evgeni
Copy link
Member

evgeni commented Jun 3, 2020

Yepp, only 0.1.12 got an egg: rpm-software-management/libcomps@972ba0a

We could artificially bump the dep to .12 or do the same trick as we did for solv.

@ehelms preferences?

@ekohl
Copy link
Member

ekohl commented Jun 3, 2020

Do we ship a new enough version? If so, I think I'd prefer to add a version requirement that's >= 0.1.12.

@ehelms
Copy link
Member

ehelms commented Jun 3, 2020 via email

@evgeni
Copy link
Member

evgeni commented Jun 4, 2020

@ehelms there is no issue with libcomps, as their newer versions provide the egg, just not the one in EL8 proper (but the one we ship).
I've opted in to bumping the dep (see theforeman/foreman-packaging@89caae6) as that felt less jacky than deleting dependencies ;)

@evgeni evgeni closed this Jun 4, 2020
@evgeni evgeni reopened this Jun 4, 2020
@evgeni evgeni closed this Jun 4, 2020
@evgeni evgeni reopened this Jun 4, 2020
@evgeni
Copy link
Member

evgeni commented Jun 4, 2020

And now it's modulemd on EL8, ffs.

@evgeni
Copy link
Member

evgeni commented Jun 4, 2020

because, you know, it works in my container:

[root@315aeea31b8a /]# python3
Python 3.6.8 (default, May 21 2019, 23:51:36) 
[GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>> gi.require_version('Modulemd', '2.0')
>>> 

@ehelms
Copy link
Member

ehelms commented Jun 4, 2020

GREEEN!

@ehelms ehelms merged commit e736afb into theforeman:master Jun 4, 2020
@ekohl ekohl added the Enhancement New feature or request label Jun 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Needs testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants