Support keystone with certificate #224

hidekazuna · 2019-02-12T02:05:35Z

What this PR does / why we need it:

This PR supports keystone endpoint with certificate by the following clouds.yaml with cacert key.

clouds:
  openstack:
    auth:
      auth_url: https://yourauthurl:5000/v3
      username: foo
      password: bar
      project_id: foobar123
      project_name: foobar
      user_domain_name: "Default"
    cacert: /path/to/cacertfile
    region_name: "Region_1"
    interface: "public"
    identity_api_version: 3

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #209

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

Release note:

k8s-ci-robot · 2019-02-12T02:05:48Z

Hi @hidekazuna. Thanks for your PR.

I'm waiting for a kubernetes-sigs or kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jichenjc · 2019-02-12T06:33:52Z

/ok-to-test

pkg/cloud/openstack/clients/machineservice.go

jichenjc · 2019-02-12T06:39:12Z

pkg/cloud/openstack/clients/machineservice.go

+	transport := &http.Transport{TLSClientConfig: tlsConfig}
+	client.HTTPClient = http.Client{Transport: transport}
+
+	err = openstack.Authenticate(client, *ao)


forgive me if I misunderstand something here.. I believe https need ca and http doesn't need it
from context we are requesting that to be mandatory https ,maybe I missed something?

Yes, you are right. I should have update code https is not mandatory. I will fix.

Fixed in the following commit: 192e70c

cmd/clusterctl/examples/openstack/ubuntu/provider-components.yaml.template

jichenjc · 2019-03-13T05:46:37Z

cmd/clusterctl/examples/openstack/generate-yaml.sh

@@ -143,7 +144,7 @@ PASSWORD=$(echo "$OPENSTACK_CLOUD_CONFIG_PLAIN" | yq r - clouds.$CLOUD.auth.pass
 REGION=$(echo "$OPENSTACK_CLOUD_CONFIG_PLAIN" | yq r - clouds.$CLOUD.region_name)
 PROJECT_ID=$(echo "$OPENSTACK_CLOUD_CONFIG_PLAIN" | yq r - clouds.$CLOUD.auth.project_id)
 DOMAIN_NAME=$(echo "$OPENSTACK_CLOUD_CONFIG_PLAIN" | yq r - clouds.$CLOUD.auth.user_domain_name)
-
+CACERT_ORIGINAL=$(echo "$OPENSTACK_CLOUD_CONFIG_PLAIN" | yq r - clouds.$CLOUD.cacert)


this should be problem if cacert doesn't exist...

Fixed in the following commit: 192e70c

chrigl · 2019-03-13T09:22:00Z

pkg/cloud/openstack/clients/machineservice.go

+
+	cloudFromYaml, err := clientconfig.GetCloudFromYAML(clientOpts)
+	if cloudFromYaml.CACertFile != "" {
+		roots, err := certutil.NewPool(caFile)


I am thinking about the mounted caFile here. In theory, the entire code allows us to use a separate clouds.yaml per machine. But it is only possible to pass in a caFile for all machines.

How about extending the cloud-config secret with another field for the contents of the caFile.... and create a separate x509.CertPool from this secret?

I think it would be ok to extend the cloud-config secret, because it relates directly to the clouds.yaml, rather than having a separate secret.

@hidekazuna @gyliu513 provided this to me
master...cdc1807:cacert-auth

I think we can try to refer to this one, at least it works for me now on a https openstack and I think we can use secret to replace the CACERT in the commit diff

@jichenjc Thanks advice.
I updated like this: 672c5af#diff-6f1595bb05179a80513b3baba49fbaf1R58
I noticed I need to update getNetworkClient function in pkg/cloud/openstack/cluster/actuator.go.
But I wonder how to fix this function as well. Could you advice?

I might need more time to figure out but FYI https://github.com/kubernetes/cloud-provider-openstack has a set of https handling including network, maybe we can refer to their implementation @hidekazuna

@chrigl @jichenjc getNetworkClient is the method of Actuator in cluster package and it's augment is *clusterv1.Cluster. We can not secrets from Actuator nor clusterv1.Cluster. I thinks this is from design issue.
Since we have to create a CA file for using OpenStack Cloud provider anyway, can't we make a compromise to use CA file?

I am not aware clusterv1 can't use secrets from Actuator, ... but conceptually yes, if you want to sync cluster to create network, apparently we need CA in order to connect to openstack, so I think it's mandatory to use CA in actuator, can we define in openstackclusterspec for this ? @hidekazuna

@jichenjc Thanks, now I am trying to update.

@jichenjc I updated by 0bf8a1b

pkg/cloud/openstack/clients/machineservice.go

morvencao · 2019-03-25T14:55:10Z

/cc

k8s-ci-robot · 2019-03-25T14:55:18Z

@morvencao: GitHub didn't allow me to request PR reviews from the following users: morvencao.

Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hchenxa · 2019-04-25T07:07:22Z

@gyliu513 @jichenjc , I create create the VM with cacert now.

gyliu513 · 2019-04-25T07:11:18Z

Thanks @hchenxa

/lgtm

/cc @jichenjc @chrigl

jichenjc · 2019-04-25T08:45:42Z

...sterctl/examples/openstack/provider-component/user-data/ubuntu/templates/worker-user-data.sh

@@ -79,6 +79,8 @@ CLUSTER_DNS_SERVER=$(prips ${SERVICE_CIDR} | head -n 11 | tail -n 1)

 # Write the cloud.conf so that the kubelet can use it.
 echo $OPENSTACK_CLOUD_PROVIDER_CONF | base64 -d > /etc/kubernetes/cloud.conf
+mkdir /etc/certs
+echo $OPENSTACK_CLOUD_CACERT_CONFIG | base64 -d > /etc/certs/cacert


Just curious
this is only for ubuntu, do we have a centos version? I didn't see it ...

Thanks added this for centos.

jichenjc · 2019-04-25T08:47:09Z

cmd/clusterctl/examples/openstack/generate-yaml.sh

@@ -186,14 +199,17 @@ if [[ "$PROVIDER_OS" == "coreos" ]]; then
 else


you updated centos and ubuntu
but how about coreos ? I didn't see the change ,maybe a follow up??

I Updated files for coreos.

jichenjc · 2019-04-25T08:48:35Z

thanks @hchenxa and @hidekazuna
I think we may leave this for a few days to get some opinion then we can merge it

in addition, I had some comments above, can you help to take a look @hidekazuna ? thank you

gyliu513 · 2019-04-26T08:46:11Z

@hidekazuna make generate_yaml_test failed, can you help check?

mv kustomize_1.0.11_linux_amd64 /usr/local/bin/kustomize
chmod +x /usr/local/bin/kustomize
# Create a dummy file for test only
echo 'clouds' > dummy-clouds-test.yaml
cmd/clusterctl/examples/openstack/generate-yaml.sh -f dummy-clouds-test.yaml openstack ubuntu dummy-make-auto-test
Generating SSH key files for machine controller.
Generating public/private rsa key pair.
Created directory '/root/.ssh'.
Your identification has been saved in /root/.ssh/openstack_tmp.
Your public key has been saved in /root/.ssh/openstack_tmp.pub.
The key fingerprint is:
SHA256:L90G006kSqwbMxM0lT0drVo+4bXNMIGj7bsPZaG91j8 root@5807cc27-67fb-11e9-be50-0a580a6c05bf
The key's randomart image is:
+---[RSA 2048]----+
|        .o ..+   |
|       .. o + o  |
|      o    +.o.. |
|     . o  .+=o+. |
|      . S +*+o+* |
|       + +.*=o.oo|
|      * o o =oo .|
|       * . ..o E.|
|      .     .o. o|
+----[SHA256]-----+
cat: null: No such file or directory
Makefile:78: recipe for target 'generate_yaml_test' failed
make: *** [generate_yaml_test] Error 1
+ EXIT_VALUE=2
+ set +o xtrace

hidekazuna · 2019-04-26T09:36:57Z

/test pull-cluster-api-provider-openstack-test

jichenjc · 2019-04-26T09:38:19Z

copy the link I added in the slack channel :)

@hidekazuna the #224 failed with a new test I added recently #322, at least it works when I submit the PR with my local stuffs and gate, please let me know whether some edge case triggered by your PR and any info needed by me ,thanks

jichenjc · 2019-04-27T10:46:44Z

/lgtm

jichenjc · 2019-04-27T10:47:19Z

I think we are ready for this one, as the @hchenxa already tested it

gyliu513 · 2019-04-27T13:14:48Z

LGTM

Leave this to @chrigl for approve.

gyliu513 · 2019-04-29T02:03:30Z

/approve

k8s-ci-robot · 2019-04-29T02:03:35Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gyliu513, hidekazuna

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [gyliu513]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Feb 12, 2019

k8s-ci-robot requested review from scruplelesswizard and chrigl February 12, 2019 02:05

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Feb 12, 2019

hidekazuna mentioned this pull request Feb 12, 2019

Add ability to authenticate with CA certificate #223

Closed

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Feb 12, 2019

jichenjc reviewed Feb 12, 2019

View reviewed changes

pkg/cloud/openstack/clients/machineservice.go Outdated Show resolved Hide resolved

jichenjc reviewed Feb 12, 2019

View reviewed changes

pkg/cloud/openstack/clients/machineservice.go Outdated Show resolved Hide resolved

jichenjc reviewed Feb 12, 2019

View reviewed changes

hidekazuna force-pushed the keystone_cacert branch 3 times, most recently from f172002 to 1a44f4e Compare February 14, 2019 01:45

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 17, 2019

hidekazuna force-pushed the keystone_cacert branch from 1a44f4e to 4ce9537 Compare February 18, 2019 07:21

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 18, 2019

hidekazuna force-pushed the keystone_cacert branch 3 times, most recently from f4901b3 to 41b7b6c Compare February 19, 2019 07:12

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 28, 2019

hidekazuna force-pushed the keystone_cacert branch from 41b7b6c to 3b2ecfe Compare March 6, 2019 01:56

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 6, 2019

jichenjc requested changes Mar 13, 2019

View reviewed changes

chrigl reviewed Mar 13, 2019

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 31, 2019

k8s-ci-robot assigned gyliu513 Apr 25, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 25, 2019

jichenjc reviewed Apr 25, 2019

View reviewed changes

hidekazuna force-pushed the keystone_cacert branch from 0bf8a1b to b06fd90 Compare April 26, 2019 08:14

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 26, 2019

gyliu513 mentioned this pull request Apr 26, 2019

'unknown authority' during creating providerClient #264

Closed

hidekazuna force-pushed the keystone_cacert branch from b06fd90 to 7cff3fa Compare April 26, 2019 09:18

hidekazuna closed this Apr 26, 2019

hidekazuna reopened this Apr 26, 2019

Support keystone with cacert

ab62a6c

hidekazuna force-pushed the keystone_cacert branch from 7cff3fa to ab62a6c Compare April 27, 2019 02:33

k8s-ci-robot assigned jichenjc Apr 27, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 27, 2019

hchenxa mentioned this pull request Apr 28, 2019

REQUEST: New membership for hchenxa kubernetes/org#774

Closed

6 tasks

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 29, 2019

k8s-ci-robot merged commit dc0834e into kubernetes-sigs:master Apr 29, 2019

gyliu513 mentioned this pull request Apr 29, 2019

Adding cacert to clouds.yaml. #336

Merged

hidekazuna deleted the keystone_cacert branch May 9, 2019 04:24

iamemilio pushed a commit to iamemilio/cluster-api-provider-openstack that referenced this pull request Jun 14, 2019

Support keystone with cacert (kubernetes-sigs#224)

cefc626

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support keystone with certificate #224

Support keystone with certificate #224

hidekazuna commented Feb 12, 2019

k8s-ci-robot commented Feb 12, 2019

jichenjc commented Feb 12, 2019

jichenjc Feb 12, 2019

hidekazuna Feb 13, 2019

hidekazuna Apr 3, 2019

jichenjc Mar 13, 2019

hidekazuna Apr 3, 2019

chrigl Mar 13, 2019

jichenjc Apr 2, 2019

hidekazuna Apr 3, 2019

jichenjc Apr 11, 2019

hidekazuna Apr 17, 2019

jichenjc Apr 17, 2019

hidekazuna Apr 23, 2019

hidekazuna Apr 24, 2019

morvencao commented Mar 25, 2019

k8s-ci-robot commented Mar 25, 2019

hchenxa commented Apr 25, 2019

gyliu513 commented Apr 25, 2019

jichenjc Apr 25, 2019

hidekazuna Apr 27, 2019

jichenjc Apr 25, 2019

hidekazuna Apr 27, 2019

jichenjc commented Apr 25, 2019

gyliu513 commented Apr 26, 2019

hidekazuna commented Apr 26, 2019

jichenjc commented Apr 26, 2019

jichenjc commented Apr 27, 2019

jichenjc commented Apr 27, 2019

gyliu513 commented Apr 27, 2019

gyliu513 commented Apr 29, 2019

k8s-ci-robot commented Apr 29, 2019

		@@ -186,14 +199,17 @@ if [[ "$PROVIDER_OS" == "coreos" ]]; then
		else

Support keystone with certificate #224

Support keystone with certificate #224

Conversation

hidekazuna commented Feb 12, 2019

k8s-ci-robot commented Feb 12, 2019

jichenjc commented Feb 12, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

morvencao commented Mar 25, 2019

k8s-ci-robot commented Mar 25, 2019

hchenxa commented Apr 25, 2019

gyliu513 commented Apr 25, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jichenjc commented Apr 25, 2019

gyliu513 commented Apr 26, 2019

hidekazuna commented Apr 26, 2019

jichenjc commented Apr 26, 2019

jichenjc commented Apr 27, 2019

jichenjc commented Apr 27, 2019

gyliu513 commented Apr 27, 2019

gyliu513 commented Apr 29, 2019

k8s-ci-robot commented Apr 29, 2019