Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken (community) image due to today's release of CentOS 7.6 #206

Closed
vorburger opened this issue Dec 6, 2018 · 12 comments
Closed

Broken (community) image due to today's release of CentOS 7.6 #206

vorburger opened this issue Dec 6, 2018 · 12 comments

Comments

@vorburger
Copy link
Collaborator

Unless I'm stupid, I think the new CentOS just released to Docker Hub, hours ago, broke ./test.sh for the existing Java 8 image (forget about Java 11 for #193) ... this seems weird, and worth investigating and possibly escalating to the CentOS community, if confirmed.

@vorburger
Copy link
Collaborator Author

Yup, it would seem to me that despite using the completely fixed versions (which is defined here) using the Base Container FROM "centos:7.5.1804" and RPM yum install "java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5", the release of the new CentOS 7.6 container image just broke the old one 7.5.1804, check this out, which worked before:

Step 7/37 : RUN yum install -y        unzip        java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5        java-1.8.0-openjdk-devel-1.8.0.181-3.b13.el7_5   && curl https://archive.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz |     tar -xzf - -C /opt   && ln -s /opt/apache-maven-3.5.4 /opt/maven   && ln -s /opt/maven/bin/mvn /usr/bin/mvn   && groupadd -r jboss -g 1000   && useradd -u 1000 -r -g jboss -m -d /opt/jboss -s /sbin/nologin -c "JBoss user" jboss   && chmod 755 /opt/jboss
 ---> Using cache
 ---> 093f954f1101
Step 8/37 : ENV JAVA_HOME /etc/alternatives/jre
 ---> Using cache
 ---> 4720429c8d1d
Step 9/37 : RUN echo securerandom.source=file:/dev/urandom >> /usr/lib/jvm/java/jre/lib/security/java.security  && usermod -g root -G jboss jboss
 ---> Running in fc3c875102bb
/bin/sh: /usr/lib/jvm/java/jre/lib/security/java.security: No such file or directory
The command '/bin/sh -c echo securerandom.source=file:/dev/urandom >> /usr/lib/jvm/java/jre/lib/security/java.security  && usermod -g root -G jboss jboss' returned a non-zero code: 1

What's curious is that centos:7.5.1804 on https://hub.docker.com/r/library/centos/tags/ says "2 months ago" ... unless I'm not getting it from there? This is weird... @ctron @rhuss please jump in and shout if you get what' going on here...

@vorburger
Copy link
Collaborator Author

Also seen here https://circleci.com/gh/fabric8io-images/s2i/498 following merge of #205 - not caused by it.

@ctron
Copy link
Contributor

ctron commented Dec 6, 2018

Looks like the parent directory is missing

@vorburger
Copy link
Collaborator Author

Looks like the parent directory is missing

sure but how did it suddenly disappear given a fixed version of base image and OpenJDK?

This is really weird, the CentOS (7.5.1804) image hasn't changed in 4 months even locally:

docker.io/tripleomaster/centos-binary-opendaylight   current-tripleo-rdo   78eaf446417a        11 days ago         977 MB
docker.io/centos                                     7.5.1804              fdf13fa91c6e        4 months ago        200 MB
docker.io/centos                                     7                     e934aafc2206        8 months ago        199 MB

I'm going to look more at the OpenJDK (RPM) installation side of things...

@vorburger
Copy link
Collaborator Author

vorburger commented Dec 6, 2018

Commenting out that echo securerandom.source=file:/dev/urandom >> /usr/lib/jvm/java/jre/lib/security/java.security and building it without that to have a look inside, we see that somehow the Java installation is completely broken:

docker build java/images/centos/ -t test
docker run --rm -it test bash
[jboss@e4f73e0a4975 ~]$ java -version
bash: java: command not found

Did I just completely break this with #205 ? I had tested it, and ./test.sh had passed, that seems unlikely.

@vorburger
Copy link
Collaborator Author

Did I just completely break this with #205 ? I had tested it, and ./test.sh had passed

Looks like it.. does the exact position of the EXPOSE matter? Top NOK needs to be last for OK? Trying..

@vorburger vorburger changed the title CentOS upgrade broke Java 8 (not Java 11) image Broken image due to #205 (#115) ARG & EXPOSE Dec 6, 2018
@vorburger
Copy link
Collaborator Author

Ignore earlier comments above re. CentOS, this was a misinterpretation; it's got absolutely nothing to do with that, but it's the ARG EXPOSE_PORT=8080 (not the EXPOSE ${EXPOSE_PORT}) which somehow, weirdly, seemed to cause this. And only for the CentOS 7.5 Java 8 image, whereas for the Fedora 29 one wasn't affected by this; makes no sense at all, to me. I'm just removing that ARG again and hard-coding 8080, for now - that fixes it.

@vorburger vorburger mentioned this issue Dec 6, 2018
@vorburger vorburger reopened this Dec 6, 2018
@vorburger
Copy link
Collaborator Author

It still failed on https://circleci.com/gh/fabric8io-images/s2i/500, and now that I'm looking more closely I am seeing what already on https://circleci.com/gh/fabric8io-images/s2i/498 4 hours ago but I initially missed:

No package java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5 available.
No package java-1.8.0-openjdk-devel-1.8.0.181-3.b13.el7_5 available.

@vorburger
Copy link
Collaborator Author

It likely had nothing at all to do with #115, so reverting a9bd456 in c23d43f.

80d07d6 ensures future non-regression and helps to sure that we get a much clearer error now:

Step 5/36 : RUN yum install -y --setopt=skip_missing_names_on_install=False        unzip        java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5        java-1.8.0-openjdk-devel-1.8.0.181-3.b13.el7_5   && curl https://archive.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz |     tar -xzf - -C /opt   && ln -s /opt/apache-maven-3.5.4 /opt/maven   && ln -s /opt/maven/bin/mvn /usr/bin/mvn   && groupadd -r jboss -g 1000   && useradd -u 1000 -r -g jboss -m -d /opt/jboss -s /sbin/nologin -c "JBoss user" jboss   && chmod 755 /opt/jboss
 ---> Running in 8baf5142e639
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
 * base: mirror.init7.net
 * extras: mirror.init7.net
 * updates: mirror.init7.net
No package java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5 available.
Error: Not tolerating missing names on install, stopping.
The command '/bin/sh -c yum install -y --setopt=skip_missing_names_on_install=False        unzip        java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5        java-1.8.0-openjdk-devel-1.8.0.181-3.b13.el7_5   && curl https://archive.apache.org/dist/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz |     tar -xzf - -C /opt   && ln -s /opt/apache-maven-3.5.4 /opt/maven   && ln -s /opt/maven/bin/mvn /usr/bin/mvn   && groupadd -r jboss -g 1000   && useradd -u 1000 -r -g jboss -m -d /opt/jboss -s /sbin/nologin -c "JBoss user" jboss   && chmod 755 /opt/jboss' returned a non-zero code: 1

The observed very confusing behaviour which lead me down a wrong interpretation re. ARG, EXPOSE and CentOS 7.5 Java 8 vs Fedora 29 Java 11 was perhaps somehow related to (a bug with not rebuilding when it should have?) local Docker Image layer caching?

Now we're back to square one, and I do suspect something is really wrong, in Docker itself, or on Docker Hub, or with the CentOS image update, because how on earth is this possible:

docker run --rm -it centos:7.5.1804 yum list "java-*"
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
 * base: pkg.adfinis-sygroup.ch
 * extras: mirror.infonline.de
 * updates: mirror.init7.net
base                                                                                                                       | 3.6 kB  00:00:00     
extras                                                                                                                     | 3.4 kB  00:00:00     
updates                                                                                                                    | 3.4 kB  00:00:00     
(1/4): base/7/x86_64/group_gz                                                                                              | 166 kB  00:00:00     
(2/4): extras/7/x86_64/primary_db                                                                                          | 154 kB  00:00:00     
(3/4): updates/7/x86_64/primary_db                                                                                         | 1.3 MB  00:00:01     
(4/4): base/7/x86_64/primary_db                                                                                            | 6.0 MB  00:00:02     
Available Packages
java-1.6.0-openjdk.x86_64                                                      1:1.6.0.41-1.13.13.1.el7_3                                  base   
java-1.6.0-openjdk-demo.x86_64                                                 1:1.6.0.41-1.13.13.1.el7_3                                  base   
java-1.6.0-openjdk-devel.x86_64                                                1:1.6.0.41-1.13.13.1.el7_3                                  base   
java-1.6.0-openjdk-javadoc.x86_64                                              1:1.6.0.41-1.13.13.1.el7_3                                  base   
java-1.6.0-openjdk-src.x86_64                                                  1:1.6.0.41-1.13.13.1.el7_3                                  base   
java-1.7.0-openjdk.x86_64                                                      1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.7.0-openjdk-accessibility.x86_64                                        1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.7.0-openjdk-demo.x86_64                                                 1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.7.0-openjdk-devel.x86_64                                                1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.7.0-openjdk-headless.x86_64                                             1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.7.0-openjdk-javadoc.noarch                                              1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.7.0-openjdk-src.x86_64                                                  1:1.7.0.201-2.6.16.1.el7_6                                  updates
java-1.8.0-openjdk.i686                                                        1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk.x86_64                                                      1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-accessibility.i686                                          1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-accessibility.x86_64                                        1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-accessibility-debug.i686                                    1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-accessibility-debug.x86_64                                  1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-debug.i686                                                  1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-debug.x86_64                                                1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-demo.i686                                                   1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-demo.x86_64                                                 1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-demo-debug.i686                                             1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-demo-debug.x86_64                                           1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-devel.i686                                                  1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-devel.x86_64                                                1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-devel-debug.i686                                            1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-devel-debug.x86_64                                          1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-headless.i686                                               1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-headless.x86_64                                             1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-headless-debug.i686                                         1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-headless-debug.x86_64                                       1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-javadoc.noarch                                              1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-javadoc-debug.noarch                                        1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-javadoc-zip.noarch                                          1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-javadoc-zip-debug.noarch                                    1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-src.i686                                                    1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-src.x86_64                                                  1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-src-debug.i686                                              1:1.8.0.191.b12-1.el7_6                                     updates
java-1.8.0-openjdk-src-debug.x86_64                                            1:1.8.0.191.b12-1.el7_6                                     updates
java-11-openjdk.i686                                                           1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk.x86_64                                                         1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-debug.i686                                                     1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-debug.x86_64                                                   1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-demo.i686                                                      1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-demo.x86_64                                                    1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-demo-debug.i686                                                1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-demo-debug.x86_64                                              1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-devel.i686                                                     1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-devel.x86_64                                                   1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-devel-debug.i686                                               1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-devel-debug.x86_64                                             1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-headless.i686                                                  1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-headless.x86_64                                                1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-headless-debug.i686                                            1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-headless-debug.x86_64                                          1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc.i686                                                   1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc.x86_64                                                 1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc-debug.i686                                             1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc-debug.x86_64                                           1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc-zip.i686                                               1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc-zip.x86_64                                             1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc-zip-debug.i686                                         1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-javadoc-zip-debug.x86_64                                       1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-jmods.i686                                                     1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-jmods.x86_64                                                   1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-jmods-debug.i686                                               1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-jmods-debug.x86_64                                             1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-src.i686                                                       1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-src.x86_64                                                     1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-src-debug.i686                                                 1:11.0.1.13-3.el7_6                                         updates
java-11-openjdk-src-debug.x86_64                                               1:11.0.1.13-3.el7_6                                         updates
java-atk-wrapper.i686                                                          0.30.4-5.el7                                                base   
java-atk-wrapper.x86_64                                                        0.30.4-5.el7                                                base  

How can the (old) centos:7.5.1804 which according to https://hub.docker.com/r/library/centos/tags/ has not changed since 2 months ago not contain java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5 anymore, which used to still work 1 week ago, as proven here https://circleci.com/gh/fabric8io-images/s2i/496, and only contain 1.8.0.191.b12-1.el7_6, and more fun also have 11.0.1.13-3.el7_6 which should be only in 7.6?

Something is really off here.

@vorburger vorburger reopened this Dec 6, 2018
@vorburger
Copy link
Collaborator Author

Just to make sure this isn't some local screw-up, even though it clearly does happen on Circle CI as well, I've just done a docker rm $(docker ps -a -q) ; docker rmi $(docker images -q) but the result is the same as above.

@TrevorPH
Copy link

TrevorPH commented Dec 6, 2018

It's BECAUSE you are using fixed versions that you have the error. Don't. Install just the package name and you get the latest.

@vorburger
Copy link
Collaborator Author

b57b0df ditches Java 8 package minor version number which fixes this; https://circleci.com/gh/fabric8io-images/s2i/504 passed again.

8bad895 does the same for the karaf/ image (b57b0df just took care of java/ to unblock); https://circleci.com/gh/fabric8io-images/s2i/506 passed again.

#172 has something somewhat related to this problem; basically this project may have misunderstood CentOS versions - for both packages and the images themselves: Full transcript of what I learnt today in this regard preserved on https://gist.github.com/vorburger/ea367a76f69aafb8260c62f6bedf2788, but the gist of it is basically that there is FROM centos:7.5.1804 since today's release "becomes" a 7.6 as soon as you do any dnf install (not just dnf update, but we should probably also do that) and specific minor package versions like java-1.8.0-openjdk-1.8.0.181-3.b13.el7_5 are not available on the repos anymore. Someone commented on private IRC pointing out that on http://vault.centos.org old package versions remain available, if "you modify the mirror paths in /etc/yum.repos.d/CentOS-Base.repo e.g. baseurl=http://vault.centos.org/7.5.1804/os/x86_64/ but I doubt we want to use that here in this project (I personally certainly don't).

@rhuss @ctron FYI

@vorburger vorburger changed the title Broken image due to #205 (#115) ARG & EXPOSE Broken (community) image due to today's release of CentOS 7.6 Dec 6, 2018
vorburger added a commit that referenced this issue Dec 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants