manage.py import error

[cjryan]

When running

oc new-app --file https://raw.githubusercontent.com/openshift/django-ex/master/openshift/templates/django.json

on the latest RHEL or CentOS devenv, I'm noticing the following error in the build:

I0302 20:15:44.120427       1 common.go:134] Running post commit hook with image hwlzr/django-example-1:a2264c0c ...
Traceback (most recent call last):
  File "./manage.py", line 8, in <module>
    from django.core.management import execute_from_command_line
ImportError: No module named django.core.management
F0302 20:15:45.168126 1 builder.go:204] Error: build error: container "openshift_s2i-build_django-example-1_hwlzr_post-commit" returned non-zero exit code: 1

The client machine is Fedora 23 x64, with
oc v1.1.3-232-gf39a5a8
kubernetes v1.2.0-alpha.7-703-gbc4550d
etcd 2.2.5

I will try this evening to test against the lastest Fedora 'origin', @csrwng indicated some success with that.

[cjryan]

Seeing similar behavior when running the latest Vagrant install of origin, as well.

[csrwng]

@cjryan I can reproduce this on an aws RHEL machine with the latest origin ami

[rhcarvalho]

@cjryan the problem is origin v1.1.3 + template@latest-master.

See openshift/origin#7379 (comment)

Unfortunately, unlike in origin's sample-app, this repo is not being tagged with every origin release.
Please use an older version of the template for now:

$ oc new-app https://raw.githubusercontent.com/openshift/django-ex/4706c39a7035/openshift/templates/django-postgresql.json

And for the next releases we'll have vendored copies of the templates in origin:
https://github.com/openshift/origin/tree/master/examples/quickstarts

Other than that we need to discuss if we want to make any changes here.
@bparees any opinion on the matter?

[cjryan]

@rhcarvalho thanks for taking a look at this. Unfortunately, that version of the the older template won't work, as we are explicitly testing the

"postCommit": {
     "script": "./manage.py test"
}

functionality. It looks like we'll have to hang tight until the vendored copies of the templates are available.

[rhcarvalho]

@cjryan aha got it. You can use the command and/or args form:

"postCommit": {
     "args": ["./manage.py", "test"]
}

or

"postCommit": {
     "command": ["/bin/bash", "-c", "./manage.py", "test"]
}

or

"postCommit": {
     "command": ["/bin/bash", "-c"],
     "args": ["./manage.py", "test"]
}

and some other combinations of those ;)

The underlying problem here is that v1.1.3 runs the script form of the hook using /bin/sh -c, and that doesn't work with our automation to enable scl collections. After the release we changed it to /bin/sh -ic which will work with scl out-of-the-box.

If you still want to test the script form in v1.1.3, you will need to enable the Python collection explicitly in the script:

"postCommit": {
     "script": "scl_enable python[VERSION] ./manage.py test"
}

[rhcarvalho]

FWIW, the error message ImportError: No module named django.core.management is coming from the system Python, not from the SCL Python version where django and all other dependencies were installed.

[GrahamDumpleton]

This problem would be avoided if a Python virtual environment was being used in the image and the bin directory for that was in the PATH. SCL Python only then needs to be enabled during the build phase for the image. See sclorg/s2i-python-container#90 (comment)

[rhcarvalho]

@GrahamDumpleton I'm not entirely sure, at least not for all apps.
If you run something that depends on a shared library, scl_enable is also setting LD_LIBRARY_PATH, among other things, that would not be correctly set in the virtualenv. Does that make sense? Or is that important only on build time?

I'm all in favor of having the Python image use virtualenv.

While it could alleviate the problem for Python, it's a non-solution for other languages. We still need to deal with enabling collections...

[GrahamDumpleton]

Especially if the install location of SCL packages is fixed as being /opt/rh they should be compiling the Python binaries with a LD_RUN_PATH compiled into them of /opt/rh/python27/root/usr/lib64. That is what one would normally do if installing Python into a non standard location. That way LD_LIBRARY_PATH isn't needed at run time. Either way, simply set LD_LIBRARY_PATH in the Docker image.

[rhcarvalho]

@GrahamDumpleton thanks for the explanation! I don't know how the SCL Python binaries are compiled. The LD_LIBRARY_PATH was one that came to mind because it tripped me up in the past ;)

Setting envs in the Dockerfile is something we have discussed though, but yet haven't gone that path because AFAIK we don't have a clean way to get what are the env changes after a "scl_enable some-collection" run.

If you have any trick on how to do that, I'd be interested to hear/read ;)
I believe if we had a script that we run and it spits out Dockerfile ENV instructions that are equivalent to enabling a collection, that would make our lives happier.

💡 Perhaps a Python script that captures the current environment, run scl_enable and then compare the new with the old environment interpreting diffs in a smart way (like additions to PATH instead of replacement).

[rhcarvalho]

The problem has been explained and solutions listed in #34 (comment)

Closing this, @cjryan please let me know if you still have trouble and I can reopen.

[cjryan]

@rhcarvalho @GrahamDumpleton Apologies for the delayed response, thank you for the solutions to the issue at hand.