ocp4 admin credentials #62

akostadinov · 2019-01-04T20:33:31Z

https://projects.engineering.redhat.com/browse/OPENSHIFTQ-12575

openshift-ci-robot · 2019-01-04T20:33:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akostadinov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [akostadinov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

pruan-rht · 2019-01-04T20:36:18Z

LGTM

akostadinov · 2019-01-04T20:39:54Z

@pruan-rht , now you can copy content of host.spec artifact from jenkins and it should work OOB for a 4.0 cluster. Also you can use ocp4 env. Still SSH to nodes not working because they don't have public IPs. I'll be adding SSH bastion support for the purpose next week.
And after that would be some more mundane stuff like supporting router and registry new object type, etc.

xingxingxia · 2019-01-07T07:46:19Z

config/config.yaml

+    cli: SharedLocalCliExecutor
+    admin_creds: MasterOsAdminCredentials
+    api_port: 6443
+    version: 4.0.0.0 # version not supported on cluster (yet)


Hardcode version: 4.0.0.0 LG currently for a quite period before 4.1 comes :)
I suggest we implement getting version by oc get clusterversion as https://bugzilla.redhat.com/show_bug.cgi?id=1658957#c3 hints :)

@xingxingxia , not having access to version anonymously will require some changes. I'd like to make sure we are not getting what we need before implementing something that may not be as nice as present implementation.

xingxingxia · 2019-01-07T07:52:20Z

lib/admin_credentials.rb

+      res = master_host.exec_admin("cat /root/.kube/config", quiet: true)
+      if !res[:success] && res[:response].include?("No such file or directory")
+        # try to find kubeconfig in other locations
+        locations = [["kubeconfig", "/etc/kubernetes/static-pod-resources"]]


Heard from @jianlinliu that Dev is in-flight making cluster not ssh-able. So I think we still need URL support for kubeconfig.

This is unbelievable, how is customer supposed to access cluster if it is not SSH-able?

I don't read the linked to issue that SSH will be disabled. It makes not sense IMO to disable SSH.

xingxingxia · 2019-01-07T07:57:15Z

config/config.yaml

@@ -84,6 +84,20 @@ environments:
    admin_creds: MasterOsAdminCredentials
    api_port: 8443
    # version: 3.6.1.44.16
+  ocp4:


@openshift/team-qe fyi, now using ocp4 as BUSHSLICER_DEFAULT_ENVIRONMENT would let us need not add air_port and version for nextgen auto test

xingxingxia · 2019-01-08T06:48:43Z

lib/admin_credentials.rb

@@ -19,7 +19,21 @@ def initialize(env, **opts)

    # @return [APIAccessor]
    def get
-      res = master_host.exec_as(:admin, "cat /root/.kube/config", quiet: true)
+      res = master_host.exec_admin("cat /root/.kube/config", quiet: true)


Hi @akostadinov , tried ocp4 in jobs 37492/consoleFull, 37493/console , both failed with error:

When I run the :annotate admin command with: # features/step_definitions/cli.rb:31 ... Inappropriate ioctl for device (Errno::ENOTTY) /home/jenkins/workspace/Runner-v3/lib/ssh.rb:244:in `initialize' /home/jenkins/workspace/Runner-v3/lib/host.rb:744:in `new' ... /home/jenkins/workspace/Runner-v3/lib/admin_credentials.rb:22:in `get'

Env is good because then I turned back to my former workaround as below and that still passes (37495/console):

[root@ip-10-0-11-47 ~]# mkdir -p ~/.ssh; chmod 700 ~/.ssh; cat /home/core/.ssh/authorized_keys >> ~/.ssh/authorized_keys; mkdir ~/.kube; mv `find /etc/kubernetes/static-pod-resources/ -name "kubeconfig" | head -n 1` ~/.kube/

In console log I see core@ec2-18-191-132-235.us-east-2.compute.amazonaws.com's password: # Update with cluster-admin (*admin* command). This looks like SSH login to machine failed. Which is unrelated to this PR.

You sure installer created the machine with the correct key?

Sure correct key, compare below:

[tester@fedora28 ~]$ rm -rf ~/.ssh [tester@fedora28 ~]$ ssh core@ec2-...-235.us-east-2.compute.amazonaws.com Are you sure you want to continue connecting (yes/no)? yes core@ec2-...-235.us-east-2.compute.amazonaws.com's password: [tester@fedora28 ~]$ ssh -i verification-tests/features/tierN/private/config/keys/....pem core@ec2-...-235.us-east-2.compute.amazonaws.com Last login: Tue Jan 8 08:25:03 2019 from ... ... [core@ip-10-0-11-47 ~]$

I asked other colleague @yanpzhan trying their subteam env and met same error 37496/console

In my cluster it has been working fine. I've never did your workaround - neither with root nor kubeconfig location. Tomorrow I'll create a new cluster and see if something changed.

I managed to reproduce.

Thanks @wjiangjay Adding these fix the problem, but I didn't have those settings for 3.x auto tests and they work. Why do we need add them in 4.0?

I guess we change some logic which handle the stderr and stdout
@akostadinov

Thank you all. I added ocp4 explicitly in Runner-v3 param description for ..._DEFAULT_ENVIRONMENT.

@wjiangjay , we can change conf = YAML.load(res[:response]) to conf = YAML.load(res[:stdout]) in lib/admin_credentials.rb:46 so that error messages are ignored.

make sense!

xingxingxia · 2019-01-24T05:47:01Z

lib/admin_credentials.rb

@@ -19,7 +19,21 @@ def initialize(env, **opts)

    # @return [APIAccessor]
    def get
-      res = master_host.exec_as(:admin, "cat /root/.kube/config", quiet: true)
+      res = master_host.exec_admin("cat /root/.kube/config", quiet: true)
+      if !res[:success] && res[:response].include?("No such file or directory")


@akostadinov Hmm, today encountered another failure due to non-robust code here:

undefined method `unpack' for nil:NilClass NoMethodError /opt/rh/rh-ruby24/root/usr/share/ruby/base64.rb:59:in `decode64' /home/jenkins/workspace/Runner-v3/lib/admin_credentials.rb:54:in `get' /home/jenkins/workspace/Runner-v3/lib/environment.rb:89:in `admin' /home/jenkins/workspace/Runner-v3/lib/environment.rb:333:in `nodes' /home/jenkins/workspace/Runner-v3/features/step_definitions/node.rb:11:in `/^I select a random node's host$/' features/tierN/cli/service.feature:125:in `And I select a random node's host'

I debugged on the master: http://pastebin.test.redhat.com/701353 , its /root/.kube/config is just the one shown in the failure log's output, which does not include admin cert/key. The failure cause is somebody did oc login on root@master with normal OCP user, which will create new /root/.kube/config, having no admin.
Could you help fix it to work more robust even when somebody did normal user oc login on master under root user?
BTW, I'll also suggest team try to not do normal user oc login on root@master. And I removed that /root/.kube/config to make auto test continue.

We can make it work with the token from the file but token is temporary and next day it will not work. Not sure it is a good idea.

ocp4 admin credentials

e997c86

openshift-ci-robot requested a review from cjryan January 4, 2019 20:33

openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 4, 2019

akostadinov merged commit 1f28917 into openshift:master Jan 4, 2019

xingxingxia reviewed Jan 7, 2019

View reviewed changes

xingxingxia reviewed Jan 8, 2019

View reviewed changes

yanpzhan mentioned this pull request Jan 10, 2019

Adjust admin console login #57

Merged

xingxingxia reviewed Jan 24, 2019

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ocp4 admin credentials #62

ocp4 admin credentials #62

akostadinov commented Jan 4, 2019

openshift-ci-robot commented Jan 4, 2019

pruan-rht commented Jan 4, 2019

akostadinov commented Jan 4, 2019

xingxingxia Jan 7, 2019 •

edited

Loading

akostadinov Jan 7, 2019

xingxingxia Jan 7, 2019

akostadinov Jan 7, 2019

akostadinov Jan 7, 2019

xingxingxia Jan 7, 2019

xingxingxia Jan 8, 2019

akostadinov Jan 8, 2019

xingxingxia Jan 8, 2019 •

edited

Loading

akostadinov Jan 8, 2019

akostadinov Jan 9, 2019

yapei Jan 10, 2019 •

edited

Loading

ghost Jan 10, 2019 •

edited by ghost

Loading

xingxingxia Jan 10, 2019

akostadinov Jan 16, 2019

ghost Jan 20, 2019

xingxingxia Jan 24, 2019 •

edited

Loading

akostadinov Jan 24, 2019

ocp4 admin credentials #62

ocp4 admin credentials #62

Conversation

akostadinov commented Jan 4, 2019

openshift-ci-robot commented Jan 4, 2019

pruan-rht commented Jan 4, 2019

akostadinov commented Jan 4, 2019

xingxingxia Jan 7, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xingxingxia Jan 8, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yapei Jan 10, 2019 • edited Loading

Choose a reason for hiding this comment

ghost Jan 10, 2019 • edited by ghost Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xingxingxia Jan 24, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xingxingxia Jan 7, 2019 •

edited

Loading

xingxingxia Jan 8, 2019 •

edited

Loading

yapei Jan 10, 2019 •

edited

Loading

ghost Jan 10, 2019 •

edited by ghost

Loading

xingxingxia Jan 24, 2019 •

edited

Loading